2024-02-16T21:59:14.934 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-16T21:59:14.935 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_pdonnell@teuthology --description fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 0a 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --name pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi --archive /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562722 -- /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562722/orig.config.yaml 2024-02-16T21:59:14.983 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T21:59:15.066 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562722 branch: wip-batrick-testing-20240215.160715 description: fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 0a 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: pdonnell@redhat.com first_in_suite: false job_id: '7562722' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/v18.2.1 ' name: pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20240215.160715 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: f78a58c0ffd401d1493058a1022c35f011d65275 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: f78a58c0ffd401d1493058a1022c35f011d65275 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: i64440 sha1: ed7ab725e1a43615009dfb74a42bf33d80081bc8 owner: scheduled_pdonnell@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 2634 sha1: f78a58c0ffd401d1493058a1022c35f011d65275 sleep_before_teardown: 0 suite: fs:upgrade:mds_upgrade_sequence suite_branch: i64440 suite_path: /home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa suite_relpath: qa suite_repo: https://github.com/batrick/ceph.git suite_sha1: ed7ab725e1a43615009dfb74a42bf33d80081bc8 targets: smithi114.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHqRF3RrMrY7fWTDdHioCer4QZhuKiE5kYZ5ZNnjJNRWkcUMwib+25JATnUZNcGvz20swiBnd14cmpRtX6DSU/Y= smithi123.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM8Rod5KZ0M6l2+lTm+twAtgRW0T8eRdZlk8lIKyLf/yy4bdI9KiEK11c6CIQU/CcEQa0zy70PlIuE0UxgEu3kw= tasks: - install: exclude_packages: - ceph-volume tag: v18.2.1 - print: '**** done install task...' - cephadm: compiled_cephadm_branch: reef conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.io/ceph/ceph:v18.2.1 roleless: true - print: '**** done end installing v18.2.1 cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - full_sequential_finally: - duration: 3600 sleep: null - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 1 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data true --yes-i-really-really-mean-it - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: - /cephfs/home/pdonnell/src/github.com_batrick_ceph_i64440/qa/suites/fs/upgrade/mds_upgrade_sequence/tasks/3-upgrade-mgr-staggered.yaml meta: {} postmerge: [] variables: fail_fs: false teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-02-16_18:26:20 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs false || true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-02-16T21:59:15.066 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa; will attempt to use it 2024-02-16T21:59:15.066 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks 2024-02-16T21:59:15.067 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-16T21:59:15.068 INFO:teuthology.task.internal:Checking packages... 2024-02-16T21:59:15.088 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'f78a58c0ffd401d1493058a1022c35f011d65275' 2024-02-16T21:59:15.088 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-16T21:59:15.088 INFO:teuthology.packaging:ref: None 2024-02-16T21:59:15.088 INFO:teuthology.packaging:tag: None 2024-02-16T21:59:15.088 INFO:teuthology.packaging:branch: wip-batrick-testing-20240215.160715 2024-02-16T21:59:15.089 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T21:59:15.089 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20240215.160715 2024-02-16T21:59:15.402 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1450.gf78a58c0 2024-02-16T21:59:15.404 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-16T21:59:15.410 INFO:teuthology.task.internal:no buildpackages task found 2024-02-16T21:59:15.410 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-16T21:59:15.416 INFO:teuthology.task.internal:Saving configuration 2024-02-16T21:59:15.434 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-16T21:59:15.440 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-16T21:59:15.466 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi114.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562722', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-16 21:54:15.620050', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHqRF3RrMrY7fWTDdHioCer4QZhuKiE5kYZ5ZNnjJNRWkcUMwib+25JATnUZNcGvz20swiBnd14cmpRtX6DSU/Y='} 2024-02-16T21:59:15.485 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562722', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-16 21:54:15.618599', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM8Rod5KZ0M6l2+lTm+twAtgRW0T8eRdZlk8lIKyLf/yy4bdI9KiEK11c6CIQU/CcEQa0zy70PlIuE0UxgEu3kw='} 2024-02-16T21:59:15.485 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-16T21:59:15.542 INFO:teuthology.task.internal:roles: ubuntu@smithi114.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-02-16T21:59:15.542 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-02-16T21:59:15.542 INFO:teuthology.run_tasks:Running task console_log... 2024-02-16T21:59:15.684 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3def0ae0d0>, signals=[15]) 2024-02-16T21:59:15.685 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-16T21:59:15.691 INFO:teuthology.task.internal:Opening connections... 2024-02-16T21:59:15.691 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi114.front.sepia.ceph.com 2024-02-16T21:59:15.693 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T21:59:15.788 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi123.front.sepia.ceph.com 2024-02-16T21:59:15.789 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T21:59:15.875 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-16T21:59:15.882 DEBUG:teuthology.orchestra.run.smithi114:> uname -m 2024-02-16T21:59:15.899 INFO:teuthology.orchestra.run.smithi114.stdout:x86_64 2024-02-16T21:59:15.900 DEBUG:teuthology.orchestra.run.smithi114:> cat /etc/os-release 2024-02-16T21:59:15.956 INFO:teuthology.orchestra.run.smithi114.stdout:NAME="CentOS Stream" 2024-02-16T21:59:15.957 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION="9" 2024-02-16T21:59:15.957 INFO:teuthology.orchestra.run.smithi114.stdout:ID="centos" 2024-02-16T21:59:15.957 INFO:teuthology.orchestra.run.smithi114.stdout:ID_LIKE="rhel fedora" 2024-02-16T21:59:15.957 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_ID="9" 2024-02-16T21:59:15.957 INFO:teuthology.orchestra.run.smithi114.stdout:PLATFORM_ID="platform:el9" 2024-02-16T21:59:15.957 INFO:teuthology.orchestra.run.smithi114.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-16T21:59:15.957 INFO:teuthology.orchestra.run.smithi114.stdout:ANSI_COLOR="0;31" 2024-02-16T21:59:15.957 INFO:teuthology.orchestra.run.smithi114.stdout:LOGO="fedora-logo-icon" 2024-02-16T21:59:15.957 INFO:teuthology.orchestra.run.smithi114.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-16T21:59:15.957 INFO:teuthology.orchestra.run.smithi114.stdout:HOME_URL="https://centos.org/" 2024-02-16T21:59:15.957 INFO:teuthology.orchestra.run.smithi114.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-16T21:59:15.957 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-16T21:59:15.957 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-16T21:59:15.958 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server 2024-02-16T21:59:15.988 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2024-02-16T21:59:16.005 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2024-02-16T21:59:16.005 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2024-02-16T21:59:16.065 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="CentOS Stream" 2024-02-16T21:59:16.065 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="9" 2024-02-16T21:59:16.065 INFO:teuthology.orchestra.run.smithi123.stdout:ID="centos" 2024-02-16T21:59:16.065 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="rhel fedora" 2024-02-16T21:59:16.065 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="9" 2024-02-16T21:59:16.065 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el9" 2024-02-16T21:59:16.065 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-16T21:59:16.065 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2024-02-16T21:59:16.065 INFO:teuthology.orchestra.run.smithi123.stdout:LOGO="fedora-logo-icon" 2024-02-16T21:59:16.065 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-16T21:59:16.065 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://centos.org/" 2024-02-16T21:59:16.065 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-16T21:59:16.066 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-16T21:59:16.066 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-16T21:59:16.066 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2024-02-16T21:59:16.088 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-16T21:59:16.097 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-16T21:59:16.154 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-16T21:59:16.155 DEBUG:teuthology.orchestra.run.smithi114:> test '!' -e /home/ubuntu/cephtest 2024-02-16T21:59:16.157 DEBUG:teuthology.orchestra.run.smithi123:> test '!' -e /home/ubuntu/cephtest 2024-02-16T21:59:16.173 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-16T21:59:16.223 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-16T21:59:16.223 DEBUG:teuthology.orchestra.run.smithi114:> test -z $(ls -A /var/lib/ceph) 2024-02-16T21:59:16.225 DEBUG:teuthology.orchestra.run.smithi123:> test -z $(ls -A /var/lib/ceph) 2024-02-16T21:59:16.322 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-16T21:59:16.377 INFO:teuthology.run_tasks:Running task kernel... 2024-02-16T21:59:16.457 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-16T21:59:16.457 INFO:teuthology.task.kernel:config {'host.a': {'kdb': True, 'sha1': 'distro'}, 'host.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-16T21:59:16.457 DEBUG:teuthology.orchestra.run.smithi114:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-16T21:59:16.458 DEBUG:teuthology.orchestra.run.smithi123:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-16T21:59:16.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T21:59:16.474 DEBUG:teuthology.orchestra.run.smithi123:> uname -r 2024-02-16T21:59:16.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T21:59:16.475 DEBUG:teuthology.orchestra.run.smithi114:> uname -r 2024-02-16T21:59:16.529 INFO:teuthology.orchestra.run.smithi114.stdout:5.14.0-361.el9.x86_64 2024-02-16T21:59:16.529 INFO:teuthology.task.kernel:Running kernel on smithi114: 5.14.0-361.el9.x86_64 2024-02-16T21:59:16.530 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum install -y kernel 2024-02-16T21:59:16.535 INFO:teuthology.orchestra.run.smithi123.stdout:5.14.0-361.el9.x86_64 2024-02-16T21:59:16.535 INFO:teuthology.task.kernel:Running kernel on smithi123: 5.14.0-361.el9.x86_64 2024-02-16T21:59:16.535 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum install -y kernel 2024-02-16T21:59:18.981 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS-9 - AppStream 8.8 MB/s | 6.7 MB 00:00 2024-02-16T21:59:19.024 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS-9 - AppStream 8.1 MB/s | 6.7 MB 00:00 2024-02-16T21:59:20.749 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS-9 - BaseOS 3.3 MB/s | 2.0 MB 00:00 2024-02-16T21:59:20.756 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS-9 - BaseOS 3.0 MB/s | 2.0 MB 00:00 2024-02-16T21:59:22.130 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - BaseOS 7.8 MB/s | 8.0 MB 00:01 2024-02-16T21:59:22.975 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.0 MB 00:01 2024-02-16T21:59:28.333 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - AppStream 4.4 MB/s | 19 MB 00:04 2024-02-16T21:59:33.047 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - AppStream 1.9 MB/s | 19 MB 00:09 2024-02-16T21:59:35.133 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - CRB 2.1 MB/s | 6.1 MB 00:02 2024-02-16T21:59:36.792 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 15 kB 00:00 2024-02-16T21:59:37.847 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.1 MB 00:01 2024-02-16T21:59:39.489 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 15 kB 00:00 2024-02-16T21:59:40.432 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 25 MB/s | 20 MB 00:00 2024-02-16T21:59:45.257 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-02-16T21:59:47.530 INFO:teuthology.orchestra.run.smithi123.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-16T21:59:47.531 INFO:teuthology.orchestra.run.smithi123.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-16T21:59:47.624 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-02-16T21:59:47.632 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T21:59:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-02-16T21:59:47.632 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T21:59:47.632 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-02-16T21:59:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: kernel x86_64 5.14.0-421.el9 CentOS-BaseOS 6.1 M 2024-02-16T21:59:47.632 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-02-16T21:59:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-core x86_64 5.14.0-421.el9 CentOS-BaseOS 21 M 2024-02-16T21:59:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-modules x86_64 5.14.0-421.el9 CentOS-BaseOS 39 M 2024-02-16T21:59:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-modules-core x86_64 5.14.0-421.el9 CentOS-BaseOS 33 M 2024-02-16T21:59:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T21:59:47.633 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-02-16T21:59:47.633 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T21:59:47.633 INFO:teuthology.orchestra.run.smithi123.stdout:Install 4 Packages 2024-02-16T21:59:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T21:59:47.634 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 100 M 2024-02-16T21:59:47.634 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 123 M 2024-02-16T21:59:47.634 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-02-16T21:59:48.249 INFO:teuthology.orchestra.run.smithi123.stdout:(1/4): kernel-5.14.0-421.el9.x86_64.rpm 9.8 MB/s | 6.1 MB 00:00 2024-02-16T21:59:49.117 INFO:teuthology.orchestra.run.smithi123.stdout:(2/4): kernel-core-5.14.0-421.el9.x86_64.rpm 14 MB/s | 21 MB 00:01 2024-02-16T21:59:49.917 INFO:teuthology.orchestra.run.smithi123.stdout:(3/4): kernel-modules-core-5.14.0-421.el9.x86_6 20 MB/s | 33 MB 00:01 2024-02-16T21:59:50.342 INFO:teuthology.orchestra.run.smithi123.stdout:(4/4): kernel-modules-5.14.0-421.el9.x86_64.rpm 14 MB/s | 39 MB 00:02 2024-02-16T21:59:50.345 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-02-16T21:59:50.345 INFO:teuthology.orchestra.run.smithi123.stdout:Total 37 MB/s | 100 MB 00:02 2024-02-16T21:59:50.346 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-02-16T21:59:50.709 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-02-16T21:59:50.709 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-02-16T21:59:51.730 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-02-16T21:59:51.731 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-02-16T21:59:54.146 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-02-16T21:59:54.917 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : kernel-modules-core-5.14.0-421.el9.x86_64 1/4 2024-02-16T21:59:55.042 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-16T21:59:57.815 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-16T21:59:57.987 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-16T22:00:02.664 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-16T22:00:02.857 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : kernel-5.14.0-421.el9.x86_64 4/4 2024-02-16T22:00:07.293 INFO:teuthology.orchestra.run.smithi114.stdout:Extra Packages for Enterprise Linux 686 kB/s | 20 MB 00:30 2024-02-16T22:00:07.557 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-16T22:00:12.315 INFO:teuthology.orchestra.run.smithi114.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-02-16T22:00:14.597 INFO:teuthology.orchestra.run.smithi114.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-16T22:00:14.597 INFO:teuthology.orchestra.run.smithi114.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-16T22:00:14.711 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-02-16T22:00:14.721 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:00:14.721 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-02-16T22:00:14.721 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:00:14.721 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-02-16T22:00:14.721 INFO:teuthology.orchestra.run.smithi114.stdout: kernel x86_64 5.14.0-421.el9 CentOS-BaseOS 6.1 M 2024-02-16T22:00:14.722 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-02-16T22:00:14.722 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-core x86_64 5.14.0-421.el9 CentOS-BaseOS 21 M 2024-02-16T22:00:14.722 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-modules x86_64 5.14.0-421.el9 CentOS-BaseOS 39 M 2024-02-16T22:00:14.722 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-modules-core x86_64 5.14.0-421.el9 CentOS-BaseOS 33 M 2024-02-16T22:00:14.722 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:00:14.722 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-02-16T22:00:14.722 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:00:14.722 INFO:teuthology.orchestra.run.smithi114.stdout:Install 4 Packages 2024-02-16T22:00:14.722 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:00:14.723 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 100 M 2024-02-16T22:00:14.723 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 123 M 2024-02-16T22:00:14.723 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-02-16T22:00:15.149 INFO:teuthology.orchestra.run.smithi114.stdout:(1/4): kernel-5.14.0-421.el9.x86_64.rpm 14 MB/s | 6.1 MB 00:00 2024-02-16T22:00:15.750 INFO:teuthology.orchestra.run.smithi114.stdout:(2/4): kernel-core-5.14.0-421.el9.x86_64.rpm 20 MB/s | 21 MB 00:01 2024-02-16T22:00:16.591 INFO:teuthology.orchestra.run.smithi114.stdout:(3/4): kernel-modules-5.14.0-421.el9.x86_64.rpm 21 MB/s | 39 MB 00:01 2024-02-16T22:00:16.892 INFO:teuthology.orchestra.run.smithi114.stdout:(4/4): kernel-modules-core-5.14.0-421.el9.x86_6 19 MB/s | 33 MB 00:01 2024-02-16T22:00:16.895 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:00:16.895 INFO:teuthology.orchestra.run.smithi114.stdout:Total 46 MB/s | 100 MB 00:02 2024-02-16T22:00:16.895 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-02-16T22:00:17.267 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-02-16T22:00:17.267 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-02-16T22:00:18.280 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-02-16T22:00:18.281 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-02-16T22:00:20.672 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-02-16T22:00:21.580 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : kernel-modules-core-5.14.0-421.el9.x86_64 1/4 2024-02-16T22:00:21.708 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-16T22:00:24.430 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-16T22:00:24.726 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-16T22:00:29.437 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-16T22:00:29.638 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : kernel-5.14.0-421.el9.x86_64 4/4 2024-02-16T22:00:34.353 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-16T22:00:36.692 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 4/4 2024-02-16T22:00:36.692 INFO:teuthology.orchestra.run.smithi123.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-16T22:00:36.692 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:00:36.762 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 4/4 2024-02-16T22:00:37.134 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-5.14.0-421.el9.x86_64 4/4 2024-02-16T22:00:37.134 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : kernel-5.14.0-421.el9.x86_64 1/4 2024-02-16T22:00:37.134 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-16T22:00:37.134 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-16T22:00:37.551 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-16T22:00:37.551 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:00:37.551 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-02-16T22:00:37.551 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-5.14.0-421.el9.x86_64 2024-02-16T22:00:37.551 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-core-5.14.0-421.el9.x86_64 2024-02-16T22:00:37.551 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-modules-5.14.0-421.el9.x86_64 2024-02-16T22:00:37.551 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-modules-core-5.14.0-421.el9.x86_64 2024-02-16T22:00:37.551 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:00:37.552 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-02-16T22:00:38.080 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q kernel | sort -rV | head -n 1 2024-02-16T22:00:38.260 INFO:teuthology.orchestra.run.smithi123.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-16T22:00:38.261 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-16T22:00:38.261 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-421.el9.x86_64 2024-02-16T22:00:38.261 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-16T22:00:38.261 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-16T22:00:38.261 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-02-16T22:00:38.261 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi123.front.sepia.ceph.com, path=None, version=distro) 2024-02-16T22:00:38.261 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q kernel | sort -rV | head -n 1 2024-02-16T22:00:38.368 INFO:teuthology.orchestra.run.smithi123.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-16T22:00:38.368 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-16T22:00:38.368 DEBUG:teuthology.orchestra.run.smithi123:> sudo rpm -qi grub2-tools 2024-02-16T22:00:38.402 INFO:teuthology.orchestra.run.smithi123.stdout:Name : grub2-tools 2024-02-16T22:00:38.402 INFO:teuthology.orchestra.run.smithi123.stdout:Epoch : 1 2024-02-16T22:00:38.402 INFO:teuthology.orchestra.run.smithi123.stdout:Version : 2.06 2024-02-16T22:00:38.402 INFO:teuthology.orchestra.run.smithi123.stdout:Release : 68.el9 2024-02-16T22:00:38.402 INFO:teuthology.orchestra.run.smithi123.stdout:Architecture: x86_64 2024-02-16T22:00:38.402 INFO:teuthology.orchestra.run.smithi123.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-16T22:00:38.402 INFO:teuthology.orchestra.run.smithi123.stdout:Group : Unspecified 2024-02-16T22:00:38.402 INFO:teuthology.orchestra.run.smithi123.stdout:Size : 8264254 2024-02-16T22:00:38.402 INFO:teuthology.orchestra.run.smithi123.stdout:License : GPLv3+ 2024-02-16T22:00:38.402 INFO:teuthology.orchestra.run.smithi123.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-16T22:00:38.403 INFO:teuthology.orchestra.run.smithi123.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-16T22:00:38.403 INFO:teuthology.orchestra.run.smithi123.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-16T22:00:38.403 INFO:teuthology.orchestra.run.smithi123.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-16T22:00:38.403 INFO:teuthology.orchestra.run.smithi123.stdout:Packager : builder@centos.org 2024-02-16T22:00:38.403 INFO:teuthology.orchestra.run.smithi123.stdout:Vendor : CentOS 2024-02-16T22:00:38.403 INFO:teuthology.orchestra.run.smithi123.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-16T22:00:38.403 INFO:teuthology.orchestra.run.smithi123.stdout:Summary : Support tools for GRUB. 2024-02-16T22:00:38.403 INFO:teuthology.orchestra.run.smithi123.stdout:Description : 2024-02-16T22:00:38.403 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:00:38.403 INFO:teuthology.orchestra.run.smithi123.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-16T22:00:38.403 INFO:teuthology.orchestra.run.smithi123.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-16T22:00:38.403 INFO:teuthology.orchestra.run.smithi123.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-16T22:00:38.403 INFO:teuthology.orchestra.run.smithi123.stdout:hardware devices. 2024-02-16T22:00:38.403 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:00:38.404 INFO:teuthology.orchestra.run.smithi123.stdout:This subpackage provides tools for support of all platforms. 2024-02-16T22:00:38.404 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-16T22:00:38.404 INFO:teuthology.task.kernel:Updating grub on smithi123 to boot 5.14.0-421.el9.x86_64 2024-02-16T22:00:38.405 DEBUG:teuthology.orchestra.run.smithi123:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-16T22:00:39.043 INFO:teuthology.orchestra.run.smithi123.stderr:Generating grub configuration file ... 2024-02-16T22:00:40.411 INFO:teuthology.orchestra.run.smithi123.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-16T22:00:40.440 INFO:teuthology.orchestra.run.smithi123.stderr:done 2024-02-16T22:00:40.442 DEBUG:teuthology.orchestra.run.smithi123:> mktemp 2024-02-16T22:00:40.457 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.UjIN75xiYU 2024-02-16T22:00:40.457 DEBUG:teuthology.orchestra.run.smithi123:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.UjIN75xiYU 2024-02-16T22:00:40.526 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0666 /tmp/tmp.UjIN75xiYU 2024-02-16T22:00:40.664 DEBUG:teuthology.orchestra.remote:smithi123:/tmp/tmp.UjIN75xiYU is 6KB 2024-02-16T22:00:40.712 DEBUG:teuthology.orchestra.run.smithi123:> rm -fr /tmp/tmp.UjIN75xiYU 2024-02-16T22:00:40.726 DEBUG:teuthology.orchestra.run.smithi123:> sudo /bin/ls /boot/loader/entries || true 2024-02-16T22:00:40.790 INFO:teuthology.orchestra.run.smithi123.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-16T22:00:40.791 INFO:teuthology.orchestra.run.smithi123.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-16T22:00:40.791 INFO:teuthology.orchestra.run.smithi123.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-16T22:00:40.791 INFO:teuthology.orchestra.run.smithi123.stdout:15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64.conf 2024-02-16T22:00:40.792 DEBUG:teuthology.orchestra.run.smithi123:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64 2024-02-16T22:00:40.966 DEBUG:teuthology.orchestra.run.smithi123:> sudo shutdown -r now 2024-02-16T22:01:03.717 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 4/4 2024-02-16T22:01:03.717 INFO:teuthology.orchestra.run.smithi114.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-16T22:01:03.717 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:01:03.787 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 4/4 2024-02-16T22:01:04.198 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-5.14.0-421.el9.x86_64 4/4 2024-02-16T22:01:04.198 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : kernel-5.14.0-421.el9.x86_64 1/4 2024-02-16T22:01:04.198 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-16T22:01:04.198 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-16T22:01:04.569 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-16T22:01:04.569 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:01:04.569 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-02-16T22:01:04.569 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-5.14.0-421.el9.x86_64 2024-02-16T22:01:04.569 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-core-5.14.0-421.el9.x86_64 2024-02-16T22:01:04.570 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-modules-5.14.0-421.el9.x86_64 2024-02-16T22:01:04.570 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-modules-core-5.14.0-421.el9.x86_64 2024-02-16T22:01:04.570 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:01:04.570 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-02-16T22:01:05.067 DEBUG:teuthology.orchestra.run.smithi114:> rpm -q kernel | sort -rV | head -n 1 2024-02-16T22:01:05.198 INFO:teuthology.orchestra.run.smithi114.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-16T22:01:05.198 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-16T22:01:05.198 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-421.el9.x86_64 2024-02-16T22:01:05.198 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-16T22:01:05.198 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-16T22:01:05.198 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-02-16T22:01:05.199 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi114.front.sepia.ceph.com, path=None, version=distro) 2024-02-16T22:01:05.199 DEBUG:teuthology.orchestra.run.smithi114:> rpm -q kernel | sort -rV | head -n 1 2024-02-16T22:01:05.306 INFO:teuthology.orchestra.run.smithi114.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-16T22:01:05.306 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-16T22:01:05.306 DEBUG:teuthology.orchestra.run.smithi114:> sudo rpm -qi grub2-tools 2024-02-16T22:01:05.346 INFO:teuthology.orchestra.run.smithi114.stdout:Name : grub2-tools 2024-02-16T22:01:05.347 INFO:teuthology.orchestra.run.smithi114.stdout:Epoch : 1 2024-02-16T22:01:05.347 INFO:teuthology.orchestra.run.smithi114.stdout:Version : 2.06 2024-02-16T22:01:05.347 INFO:teuthology.orchestra.run.smithi114.stdout:Release : 68.el9 2024-02-16T22:01:05.347 INFO:teuthology.orchestra.run.smithi114.stdout:Architecture: x86_64 2024-02-16T22:01:05.347 INFO:teuthology.orchestra.run.smithi114.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-16T22:01:05.347 INFO:teuthology.orchestra.run.smithi114.stdout:Group : Unspecified 2024-02-16T22:01:05.347 INFO:teuthology.orchestra.run.smithi114.stdout:Size : 8264254 2024-02-16T22:01:05.347 INFO:teuthology.orchestra.run.smithi114.stdout:License : GPLv3+ 2024-02-16T22:01:05.347 INFO:teuthology.orchestra.run.smithi114.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-16T22:01:05.347 INFO:teuthology.orchestra.run.smithi114.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-16T22:01:05.347 INFO:teuthology.orchestra.run.smithi114.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-16T22:01:05.347 INFO:teuthology.orchestra.run.smithi114.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-16T22:01:05.347 INFO:teuthology.orchestra.run.smithi114.stdout:Packager : builder@centos.org 2024-02-16T22:01:05.348 INFO:teuthology.orchestra.run.smithi114.stdout:Vendor : CentOS 2024-02-16T22:01:05.348 INFO:teuthology.orchestra.run.smithi114.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-16T22:01:05.348 INFO:teuthology.orchestra.run.smithi114.stdout:Summary : Support tools for GRUB. 2024-02-16T22:01:05.348 INFO:teuthology.orchestra.run.smithi114.stdout:Description : 2024-02-16T22:01:05.348 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:01:05.348 INFO:teuthology.orchestra.run.smithi114.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-16T22:01:05.348 INFO:teuthology.orchestra.run.smithi114.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-16T22:01:05.348 INFO:teuthology.orchestra.run.smithi114.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-16T22:01:05.348 INFO:teuthology.orchestra.run.smithi114.stdout:hardware devices. 2024-02-16T22:01:05.348 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:01:05.348 INFO:teuthology.orchestra.run.smithi114.stdout:This subpackage provides tools for support of all platforms. 2024-02-16T22:01:05.349 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-16T22:01:05.349 INFO:teuthology.task.kernel:Updating grub on smithi114 to boot 5.14.0-421.el9.x86_64 2024-02-16T22:01:05.350 DEBUG:teuthology.orchestra.run.smithi114:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-16T22:01:05.970 INFO:teuthology.orchestra.run.smithi114.stderr:Generating grub configuration file ... 2024-02-16T22:01:07.316 INFO:teuthology.orchestra.run.smithi114.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-16T22:01:07.352 INFO:teuthology.orchestra.run.smithi114.stderr:done 2024-02-16T22:01:07.354 DEBUG:teuthology.orchestra.run.smithi114:> mktemp 2024-02-16T22:01:07.369 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/tmp.gkOvZDLzkn 2024-02-16T22:01:07.370 DEBUG:teuthology.orchestra.run.smithi114:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.gkOvZDLzkn 2024-02-16T22:01:07.438 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0666 /tmp/tmp.gkOvZDLzkn 2024-02-16T22:01:07.577 DEBUG:teuthology.orchestra.remote:smithi114:/tmp/tmp.gkOvZDLzkn is 6KB 2024-02-16T22:01:07.624 DEBUG:teuthology.orchestra.run.smithi114:> rm -fr /tmp/tmp.gkOvZDLzkn 2024-02-16T22:01:07.638 DEBUG:teuthology.orchestra.run.smithi114:> sudo /bin/ls /boot/loader/entries || true 2024-02-16T22:01:07.704 INFO:teuthology.orchestra.run.smithi114.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-16T22:01:07.705 INFO:teuthology.orchestra.run.smithi114.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-16T22:01:07.705 INFO:teuthology.orchestra.run.smithi114.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-16T22:01:07.705 INFO:teuthology.orchestra.run.smithi114.stdout:15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64.conf 2024-02-16T22:01:07.706 DEBUG:teuthology.orchestra.run.smithi114:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64 2024-02-16T22:01:07.849 DEBUG:teuthology.orchestra.run.smithi114:> sudo shutdown -r now 2024-02-16T22:01:10.972 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-02-16T22:01:10.972 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-02-16T22:01:10.973 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T22:01:29.408 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.123 2024-02-16T22:01:37.859 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-02-16T22:01:37.860 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-02-16T22:01:37.861 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T22:01:38.410 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-02-16T22:01:38.411 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T22:01:41.468 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.123 2024-02-16T22:01:53.476 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-02-16T22:01:53.477 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T22:01:56.284 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2024-02-16T22:02:05.293 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-02-16T22:02:05.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T22:02:08.348 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2024-02-16T22:02:20.360 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-02-16T22:02:20.361 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T22:02:25.285 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-02-16T22:02:25.811 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-02-16T22:02:25.811 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-421.el9.x86_64"... 2024-02-16T22:02:25.812 DEBUG:teuthology.orchestra.run.smithi123:> uname -r 2024-02-16T22:02:25.826 INFO:teuthology.orchestra.run.smithi123.stdout:5.14.0-421.el9.x86_64 2024-02-16T22:02:25.826 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-421.el9.x86_64 vs 5.14.0-421.el9.x86_64 2024-02-16T22:02:25.826 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-16T22:02:25.826 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-16T22:02:26.827 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-02-16T22:02:26.827 DEBUG:teuthology.orchestra.run.smithi123:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-16T22:02:26.909 INFO:teuthology.orchestra.run.smithi123.stdout:ttyS1 2024-02-16T22:02:26.964 DEBUG:teuthology.parallel:result is None 2024-02-16T22:02:38.780 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2024-02-16T22:02:53.782 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-02-16T22:02:53.783 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T22:02:54.214 DEBUG:teuthology.orchestra.run.smithi114:> true 2024-02-16T22:02:54.637 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-02-16T22:02:54.637 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-421.el9.x86_64"... 2024-02-16T22:02:54.638 DEBUG:teuthology.orchestra.run.smithi114:> uname -r 2024-02-16T22:02:54.652 INFO:teuthology.orchestra.run.smithi114.stdout:5.14.0-421.el9.x86_64 2024-02-16T22:02:54.652 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-421.el9.x86_64 vs 5.14.0-421.el9.x86_64 2024-02-16T22:02:54.653 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-16T22:02:54.653 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-16T22:02:55.653 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-02-16T22:02:55.654 DEBUG:teuthology.orchestra.run.smithi114:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-16T22:02:55.729 INFO:teuthology.orchestra.run.smithi114.stdout:ttyS1 2024-02-16T22:02:55.780 DEBUG:teuthology.parallel:result is None 2024-02-16T22:02:55.781 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-16T22:02:55.789 INFO:teuthology.task.internal:Creating test directory... 2024-02-16T22:02:55.789 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-16T22:02:55.791 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-16T22:02:55.810 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-16T22:02:55.816 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-16T22:02:55.823 INFO:teuthology.task.internal:Creating archive directory... 2024-02-16T22:02:55.823 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-16T22:02:55.842 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-16T22:02:55.943 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-16T22:02:55.950 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-16T22:02:55.950 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-16T22:02:55.952 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-16T22:02:55.976 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-16T22:02:55.989 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-16T22:02:55.993 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-16T22:02:56.007 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-16T22:02:56.008 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-16T22:02:56.016 INFO:teuthology.task.internal:Configuring sudo... 2024-02-16T22:02:56.016 DEBUG:teuthology.orchestra.run.smithi114:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-16T22:02:56.030 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-16T22:02:56.078 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-16T22:02:56.087 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-16T22:02:56.088 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-16T22:02:56.097 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-16T22:02:56.134 DEBUG:teuthology.orchestra.run.smithi114:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-16T22:02:56.193 DEBUG:teuthology.orchestra.run.smithi114:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-16T22:02:56.241 DEBUG:teuthology.orchestra.run.smithi114:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-16T22:02:56.319 DEBUG:teuthology.orchestra.run.smithi114:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-16T22:02:56.357 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-02-16T22:02:56.357 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-16T22:02:56.420 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-16T22:02:56.463 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-16T22:02:56.522 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-16T22:02:56.606 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-16T22:02:56.651 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-02-16T22:02:56.651 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-16T22:02:56.720 DEBUG:teuthology.orchestra.run.smithi114:> sudo service rsyslog restart 2024-02-16T22:02:56.722 DEBUG:teuthology.orchestra.run.smithi123:> sudo service rsyslog restart 2024-02-16T22:02:56.809 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-16T22:02:56.841 INFO:teuthology.orchestra.run.smithi114.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-16T22:02:57.109 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-16T22:02:57.117 INFO:teuthology.task.internal:Starting timer... 2024-02-16T22:02:57.117 INFO:teuthology.run_tasks:Running task pcp... 2024-02-16T22:02:57.139 INFO:teuthology.run_tasks:Running task selinux... 2024-02-16T22:02:57.148 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-16T22:02:57.149 DEBUG:teuthology.orchestra.run.smithi114:> sudo service auditd rotate 2024-02-16T22:02:57.231 INFO:teuthology.orchestra.run.smithi114.stdout:Rotating logs: 2024-02-16T22:02:57.238 DEBUG:teuthology.orchestra.run.smithi123:> sudo service auditd rotate 2024-02-16T22:02:57.308 INFO:teuthology.orchestra.run.smithi123.stdout:Rotating logs: 2024-02-16T22:02:57.310 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-16T22:02:57.310 DEBUG:teuthology.orchestra.run.smithi114:> /usr/sbin/getenforce 2024-02-16T22:02:57.350 INFO:teuthology.orchestra.run.smithi114.stdout:Permissive 2024-02-16T22:02:57.351 DEBUG:teuthology.orchestra.run.smithi123:> /usr/sbin/getenforce 2024-02-16T22:02:57.374 INFO:teuthology.orchestra.run.smithi123.stdout:Permissive 2024-02-16T22:02:57.375 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi114.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi123.front.sepia.ceph.com': 'permissive'} 2024-02-16T22:02:57.375 DEBUG:teuthology.orchestra.run.smithi114:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-16T22:02:57.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T22:02:57.437 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-16T22:02:57.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T22:02:57.466 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-16T22:02:57.467 DEBUG:teuthology.orchestra.run.smithi114:> sudo /usr/sbin/setenforce permissive 2024-02-16T22:02:57.520 DEBUG:teuthology.orchestra.run.smithi123:> sudo /usr/sbin/setenforce permissive 2024-02-16T22:02:57.551 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-16T22:02:57.561 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-16T22:02:57.582 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-16T22:02:57.609 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi114.front.sepia.ceph.com,smithi123.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-16T22:07:09.979 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi114.front.sepia.ceph.com'), Remote(name='ubuntu@smithi123.front.sepia.ceph.com')] 2024-02-16T22:07:09.981 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-02-16T22:07:09.982 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T22:07:10.076 DEBUG:teuthology.orchestra.run.smithi114:> true 2024-02-16T22:07:10.155 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-02-16T22:07:10.155 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-02-16T22:07:10.156 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T22:07:10.241 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-02-16T22:07:10.315 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-02-16T22:07:10.316 INFO:teuthology.run_tasks:Running task clock... 2024-02-16T22:07:10.327 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-16T22:07:10.327 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-16T22:07:10.328 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-16T22:07:10.330 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-16T22:07:10.330 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-16T22:07:10.361 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-16T22:07:10.379 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-16T22:07:10.385 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-16T22:07:10.401 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-16T22:07:10.409 INFO:teuthology.orchestra.run.smithi114.stderr:sudo: ntpd: command not found 2024-02-16T22:07:10.423 INFO:teuthology.orchestra.run.smithi114.stdout:506 Cannot talk to daemon 2024-02-16T22:07:10.430 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: ntpd: command not found 2024-02-16T22:07:10.439 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-16T22:07:10.443 INFO:teuthology.orchestra.run.smithi123.stdout:506 Cannot talk to daemon 2024-02-16T22:07:10.456 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-16T22:07:10.460 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-16T22:07:10.476 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-16T22:07:10.515 INFO:teuthology.orchestra.run.smithi114.stderr:bash: line 1: ntpq: command not found 2024-02-16T22:07:10.520 INFO:teuthology.orchestra.run.smithi114.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-16T22:07:10.520 INFO:teuthology.orchestra.run.smithi114.stdout:=============================================================================== 2024-02-16T22:07:10.520 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T22:07:10.520 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T22:07:10.520 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T22:07:10.520 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T22:07:10.526 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-02-16T22:07:10.529 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-16T22:07:10.530 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-02-16T22:07:10.530 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T22:07:10.530 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T22:07:10.530 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T22:07:10.530 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T22:07:10.530 INFO:teuthology.run_tasks:Running task install... 2024-02-16T22:07:10.538 DEBUG:teuthology.task.install:project ceph 2024-02-16T22:07:10.538 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275'}} 2024-02-16T22:07:10.539 DEBUG:teuthology.task.install:config {'exclude_packages': ['ceph-volume'], 'tag': 'v18.2.1', 'flavor': 'default', 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275'} 2024-02-16T22:07:10.539 INFO:teuthology.task.install:Using flavor: default 2024-02-16T22:07:10.544 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-16T22:07:10.544 INFO:teuthology.task.install:extra packages: [] 2024-02-16T22:07:10.545 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275', 'tag': 'v18.2.1', 'wait_for_package': False} 2024-02-16T22:07:10.545 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-16T22:07:10.545 INFO:teuthology.packaging:ref: None 2024-02-16T22:07:10.545 INFO:teuthology.packaging:tag: v18.2.1 2024-02-16T22:07:10.545 INFO:teuthology.packaging:branch: None 2024-02-16T22:07:10.545 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T22:07:10.872 DEBUG:teuthology.repo_utils:git ls-remote https://git.ceph.com/ceph-ci v18.2.1^{} -> None 2024-02-16T22:07:10.873 INFO:teuthology.packaging:Tag 'v18.2.1' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-02-16T22:07:12.430 DEBUG:teuthology.repo_utils:git ls-remote https://git.ceph.com/ceph v18.2.1^{} -> 7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-02-16T22:07:12.430 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-02-16T22:07:12.433 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275', 'tag': 'v18.2.1', 'wait_for_package': False} 2024-02-16T22:07:12.433 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-16T22:07:12.433 INFO:teuthology.packaging:ref: None 2024-02-16T22:07:12.433 INFO:teuthology.packaging:tag: v18.2.1 2024-02-16T22:07:12.433 INFO:teuthology.packaging:branch: None 2024-02-16T22:07:12.433 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T22:07:12.434 INFO:teuthology.packaging:Tag 'v18.2.1' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-02-16T22:07:12.434 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-02-16T22:07:12.701 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/ 2024-02-16T22:07:12.701 INFO:teuthology.task.install.rpm:Package version is 18.2.1-0 2024-02-16T22:07:12.707 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/ 2024-02-16T22:07:12.707 INFO:teuthology.task.install.rpm:Package version is 18.2.1-0 2024-02-16T22:07:12.751 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-16T22:07:12.752 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-02-16T22:07:12.752 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-16T22:07:12.754 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-16T22:07:12.754 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-02-16T22:07:12.754 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-16T22:07:12.786 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-16T22:07:12.786 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-16T22:07:12.787 INFO:teuthology.packaging:ref: None 2024-02-16T22:07:12.787 INFO:teuthology.packaging:tag: v18.2.1 2024-02-16T22:07:12.787 INFO:teuthology.packaging:branch: None 2024-02-16T22:07:12.787 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T22:07:12.787 DEBUG:teuthology.orchestra.run.smithi123:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-16T22:07:12.788 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-16T22:07:12.788 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-16T22:07:12.788 INFO:teuthology.packaging:ref: None 2024-02-16T22:07:12.788 INFO:teuthology.packaging:tag: v18.2.1 2024-02-16T22:07:12.788 INFO:teuthology.packaging:branch: None 2024-02-16T22:07:12.788 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T22:07:12.788 DEBUG:teuthology.orchestra.run.smithi114:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-16T22:07:12.858 DEBUG:teuthology.orchestra.run.smithi123:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-16T22:07:12.859 DEBUG:teuthology.orchestra.run.smithi114:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-16T22:07:12.944 DEBUG:teuthology.orchestra.run.smithi114:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-16T22:07:12.945 DEBUG:teuthology.orchestra.run.smithi123:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-16T22:07:12.973 INFO:teuthology.orchestra.run.smithi114.stdout:check_obsoletes = 1 2024-02-16T22:07:12.974 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum clean all 2024-02-16T22:07:12.976 INFO:teuthology.orchestra.run.smithi123.stdout:check_obsoletes = 1 2024-02-16T22:07:12.978 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum clean all 2024-02-16T22:07:13.217 INFO:teuthology.orchestra.run.smithi114.stdout:74 files removed 2024-02-16T22:07:13.233 INFO:teuthology.orchestra.run.smithi123.stdout:74 files removed 2024-02-16T22:07:13.244 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-radosgw 2024-02-16T22:07:13.260 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-radosgw 2024-02-16T22:07:13.693 INFO:teuthology.orchestra.run.smithi114.stdout:ceph packages for x86_64 574 kB/s | 76 kB 00:00 2024-02-16T22:07:13.725 INFO:teuthology.orchestra.run.smithi123.stdout:ceph packages for x86_64 513 kB/s | 76 kB 00:00 2024-02-16T22:07:13.869 INFO:teuthology.orchestra.run.smithi114.stdout:ceph noarch packages 73 kB/s | 9.4 kB 00:00 2024-02-16T22:07:13.909 INFO:teuthology.orchestra.run.smithi123.stdout:ceph noarch packages 69 kB/s | 9.4 kB 00:00 2024-02-16T22:07:14.036 INFO:teuthology.orchestra.run.smithi114.stdout:ceph source packages 17 kB/s | 2.2 kB 00:00 2024-02-16T22:07:14.068 INFO:teuthology.orchestra.run.smithi123.stdout:ceph source packages 18 kB/s | 2.2 kB 00:00 2024-02-16T22:07:15.062 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS-9 - AppStream 7.0 MB/s | 6.7 MB 00:00 2024-02-16T22:07:15.155 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS-9 - AppStream 6.2 MB/s | 6.7 MB 00:01 2024-02-16T22:07:16.704 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS-9 - BaseOS 3.8 MB/s | 2.0 MB 00:00 2024-02-16T22:07:16.805 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS-9 - BaseOS 3.7 MB/s | 2.0 MB 00:00 2024-02-16T22:07:17.586 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-02-16T22:07:23.353 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - AppStream 4.1 MB/s | 19 MB 00:04 2024-02-16T22:07:27.487 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - BaseOS 789 kB/s | 8.0 MB 00:10 2024-02-16T22:07:28.979 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 6.1 MB 00:02 2024-02-16T22:07:30.687 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - AppStream 9.0 MB/s | 19 MB 00:02 2024-02-16T22:07:30.821 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 15 kB 00:00 2024-02-16T22:07:32.956 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 10 MB/s | 20 MB 00:02 2024-02-16T22:07:37.848 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-02-16T22:07:40.451 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-02-16T22:07:40.465 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:07:40.465 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-02-16T22:07:40.465 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:07:40.465 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-02-16T22:07:40.465 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw x86_64 2:18.2.1-0.el9 ceph 7.7 M 2024-02-16T22:07:40.465 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrading: 2024-02-16T22:07:40.466 INFO:teuthology.orchestra.run.smithi123.stdout: librados2 x86_64 2:18.2.1-0.el9 ceph 3.3 M 2024-02-16T22:07:40.466 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1 x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-02-16T22:07:40.466 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-02-16T22:07:40.466 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-16T22:07:40.466 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base x86_64 2:18.2.1-0.el9 ceph 5.2 M 2024-02-16T22:07:40.466 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common x86_64 2:18.2.1-0.el9 ceph 18 M 2024-02-16T22:07:40.466 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux x86_64 2:18.2.1-0.el9 ceph 24 k 2024-02-16T22:07:40.466 INFO:teuthology.orchestra.run.smithi123.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-02-16T22:07:40.466 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-16T22:07:40.466 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-16T22:07:40.466 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-16T22:07:40.466 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2 x86_64 2:18.2.1-0.el9 ceph 658 k 2024-02-16T22:07:40.466 INFO:teuthology.orchestra.run.smithi123.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-16T22:07:40.466 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-16T22:07:40.466 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-16T22:07:40.467 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1 x86_64 2:18.2.1-0.el9 ceph 474 k 2024-02-16T22:07:40.467 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-16T22:07:40.467 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2 x86_64 2:18.2.1-0.el9 ceph 4.5 M 2024-02-16T22:07:40.467 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-16T22:07:40.467 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-16T22:07:40.467 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-16T22:07:40.467 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse x86_64 2:18.2.1-0.el9 ceph 45 k 2024-02-16T22:07:40.467 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common x86_64 2:18.2.1-0.el9 ceph 124 k 2024-02-16T22:07:40.467 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs x86_64 2:18.2.1-0.el9 ceph 161 k 2024-02-16T22:07:40.467 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-16T22:07:40.467 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados x86_64 2:18.2.1-0.el9 ceph 321 k 2024-02-16T22:07:40.467 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd x86_64 2:18.2.1-0.el9 ceph 297 k 2024-02-16T22:07:40.467 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw x86_64 2:18.2.1-0.el9 ceph 99 k 2024-02-16T22:07:40.467 INFO:teuthology.orchestra.run.smithi123.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-16T22:07:40.467 INFO:teuthology.orchestra.run.smithi123.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-16T22:07:40.468 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:07:40.468 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-02-16T22:07:40.468 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:07:40.468 INFO:teuthology.orchestra.run.smithi123.stdout:Install 28 Packages 2024-02-16T22:07:40.468 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrade 2 Packages 2024-02-16T22:07:40.468 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:07:40.468 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 53 M 2024-02-16T22:07:40.469 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-02-16T22:07:40.724 INFO:teuthology.orchestra.run.smithi123.stdout:(1/30): ceph-base-18.2.1-0.el9.x86_64.rpm 29 MB/s | 5.2 MB 00:00 2024-02-16T22:07:40.740 INFO:teuthology.orchestra.run.smithi123.stdout:(2/30): ceph-selinux-18.2.1-0.el9.x86_64.rpm 1.4 MB/s | 24 kB 00:00 2024-02-16T22:07:40.774 INFO:teuthology.orchestra.run.smithi123.stdout:(3/30): libcephfs2-18.2.1-0.el9.x86_64.rpm 19 MB/s | 658 kB 00:00 2024-02-16T22:07:40.816 INFO:teuthology.orchestra.run.smithi123.stdout:(4/30): libradosstriper1-18.2.1-0.el9.x86_64.rp 11 MB/s | 474 kB 00:00 2024-02-16T22:07:40.941 INFO:teuthology.orchestra.run.smithi123.stdout:(5/30): ceph-radosgw-18.2.1-0.el9.x86_64.rpm 19 MB/s | 7.7 MB 00:00 2024-02-16T22:07:40.958 INFO:teuthology.orchestra.run.smithi123.stdout:(6/30): python3-ceph-argparse-18.2.1-0.el9.x86_ 2.7 MB/s | 45 kB 00:00 2024-02-16T22:07:40.983 INFO:teuthology.orchestra.run.smithi123.stdout:(7/30): python3-ceph-common-18.2.1-0.el9.x86_64 4.9 MB/s | 124 kB 00:00 2024-02-16T22:07:41.042 INFO:teuthology.orchestra.run.smithi123.stdout:(8/30): librgw2-18.2.1-0.el9.x86_64.rpm 20 MB/s | 4.5 MB 00:00 2024-02-16T22:07:41.067 INFO:teuthology.orchestra.run.smithi123.stdout:(9/30): python3-cephfs-18.2.1-0.el9.x86_64.rpm 1.9 MB/s | 161 kB 00:00 2024-02-16T22:07:41.092 INFO:teuthology.orchestra.run.smithi123.stdout:(10/30): python3-rados-18.2.1-0.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2024-02-16T22:07:41.118 INFO:teuthology.orchestra.run.smithi123.stdout:(11/30): python3-rbd-18.2.1-0.el9.x86_64.rpm 5.8 MB/s | 297 kB 00:00 2024-02-16T22:07:41.143 INFO:teuthology.orchestra.run.smithi123.stdout:(12/30): python3-rgw-18.2.1-0.el9.x86_64.rpm 1.9 MB/s | 99 kB 00:00 2024-02-16T22:07:41.393 INFO:teuthology.orchestra.run.smithi123.stdout:(13/30): ceph-common-18.2.1-0.el9.x86_64.rpm 22 MB/s | 18 MB 00:00 2024-02-16T22:07:41.419 INFO:teuthology.orchestra.run.smithi123.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 352 kB/s | 106 kB 00:00 2024-02-16T22:07:41.444 INFO:teuthology.orchestra.run.smithi123.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 892 kB/s | 44 kB 00:00 2024-02-16T22:07:41.469 INFO:teuthology.orchestra.run.smithi123.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 489 kB/s | 159 kB 00:00 2024-02-16T22:07:41.503 INFO:teuthology.orchestra.run.smithi123.stdout:(17/30): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 661 kB 00:00 2024-02-16T22:07:41.528 INFO:teuthology.orchestra.run.smithi123.stdout:(18/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.3 MB/s | 282 kB 00:00 2024-02-16T22:07:41.553 INFO:teuthology.orchestra.run.smithi123.stdout:(19/30): python3-prettytable-0.7.2-27.el9.noarc 501 kB/s | 41 kB 00:00 2024-02-16T22:07:41.745 INFO:teuthology.orchestra.run.smithi123.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 458 kB/s | 111 kB 00:00 2024-02-16T22:07:41.787 INFO:teuthology.orchestra.run.smithi123.stdout:(21/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 624 kB/s | 26 kB 00:00 2024-02-16T22:07:41.815 INFO:teuthology.orchestra.run.smithi123.stdout:(22/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.2 MB/s | 309 kB 00:00 2024-02-16T22:07:41.829 INFO:teuthology.orchestra.run.smithi123.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-02-16T22:07:41.855 INFO:teuthology.orchestra.run.smithi123.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-02-16T22:07:41.905 INFO:teuthology.orchestra.run.smithi123.stdout:(25/30): re2-20211101-3.el9.x86_64.rpm 3.8 MB/s | 192 kB 00:00 2024-02-16T22:07:41.938 INFO:teuthology.orchestra.run.smithi123.stdout:(26/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.5 MB/s | 838 kB 00:00 2024-02-16T22:07:42.022 INFO:teuthology.orchestra.run.smithi123.stdout:(27/30): librados2-18.2.1-0.el9.x86_64.rpm 39 MB/s | 3.3 MB 00:00 2024-02-16T22:07:42.089 INFO:teuthology.orchestra.run.smithi123.stdout:(28/30): libarrow-9.0.0-10.el9.x86_64.rpm 8.2 MB/s | 4.4 MB 00:00 2024-02-16T22:07:42.139 INFO:teuthology.orchestra.run.smithi123.stdout:(29/30): thrift-0.15.0-2.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-02-16T22:07:42.190 INFO:teuthology.orchestra.run.smithi123.stdout:(30/30): librbd1-18.2.1-0.el9.x86_64.rpm 18 MB/s | 3.0 MB 00:00 2024-02-16T22:07:42.195 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:07:42.195 INFO:teuthology.orchestra.run.smithi123.stdout:Total 31 MB/s | 53 MB 00:01 2024-02-16T22:07:42.196 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-02-16T22:07:42.227 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-02-16T22:07:42.227 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-02-16T22:07:42.674 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-02-16T22:07:42.674 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-02-16T22:07:44.051 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-02-16T22:07:44.133 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-02-16T22:07:44.169 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-02-16T22:07:44.221 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-02-16T22:07:44.537 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-02-16T22:07:44.585 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librados2-2:18.2.1-0.el9.x86_64 5/32 2024-02-16T22:07:44.679 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:18.2.1-0.el9.x86_64 5/32 2024-02-16T22:07:44.758 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rados-2:18.2.1-0.el9.x86_64 6/32 2024-02-16T22:07:44.794 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-02-16T22:07:44.857 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-02-16T22:07:44.886 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-02-16T22:07:44.929 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 9/32 2024-02-16T22:07:45.009 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-02-16T22:07:45.039 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-02-16T22:07:45.087 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-02-16T22:07:45.195 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-02-16T22:07:45.245 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-02-16T22:07:45.287 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-02-16T22:07:45.327 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-02-16T22:07:45.354 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-02-16T22:07:45.390 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-02-16T22:07:45.423 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-02-16T22:07:45.722 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-02-16T22:07:45.744 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-02-16T22:07:45.833 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-02-16T22:07:45.871 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rbd-2:18.2.1-0.el9.x86_64 21/32 2024-02-16T22:07:45.919 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-02-16T22:07:46.313 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-02-16T22:07:46.347 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-02-16T22:07:46.410 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-02-16T22:07:46.477 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rgw-2:18.2.1-0.el9.x86_64 25/32 2024-02-16T22:07:46.507 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-common-2:18.2.1-0.el9.x86_64 26/32 2024-02-16T22:07:48.578 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-02-16T22:07:48.621 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-02-16T22:07:49.165 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-02-16T22:07:49.199 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-02-16T22:07:49.290 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-02-16T22:07:49.290 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-16T22:07:49.290 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-16T22:07:49.290 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:07:49.333 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-02-16T22:07:53.270 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - CRB 326 kB/s | 6.1 MB 00:19 2024-02-16T22:07:54.671 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - Extras packages 64 kB/s | 15 kB 00:00 2024-02-16T22:07:58.712 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-02-16T22:07:58.712 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /sys 2024-02-16T22:07:58.712 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /proc 2024-02-16T22:07:58.712 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /mnt 2024-02-16T22:07:58.712 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /var/tmp 2024-02-16T22:07:58.713 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /home 2024-02-16T22:07:58.713 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /root 2024-02-16T22:07:58.713 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /tmp 2024-02-16T22:07:58.713 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:07:59.468 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-02-16T22:07:59.493 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-02-16T22:07:59.493 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T22:07:59.493 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-16T22:07:59.493 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-16T22:07:59.493 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-16T22:07:59.493 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:07:59.495 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-02-16T22:07:59.562 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-02-16T22:07:59.563 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-02-16T22:08:00.640 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-02-16T22:08:00.640 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-base-2:18.2.1-0.el9.x86_64 1/32 2024-02-16T22:08:00.643 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-common-2:18.2.1-0.el9.x86_64 2/32 2024-02-16T22:08:00.643 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-radosgw-2:18.2.1-0.el9.x86_64 3/32 2024-02-16T22:08:00.643 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-selinux-2:18.2.1-0.el9.x86_64 4/32 2024-02-16T22:08:00.643 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs2-2:18.2.1-0.el9.x86_64 5/32 2024-02-16T22:08:00.643 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libradosstriper1-2:18.2.1-0.el9.x86_64 6/32 2024-02-16T22:08:00.644 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librgw2-2:18.2.1-0.el9.x86_64 7/32 2024-02-16T22:08:00.644 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 8/32 2024-02-16T22:08:00.644 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-common-2:18.2.1-0.el9.x86_64 9/32 2024-02-16T22:08:00.644 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-02-16T22:08:00.644 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rados-2:18.2.1-0.el9.x86_64 11/32 2024-02-16T22:08:00.644 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rbd-2:18.2.1-0.el9.x86_64 12/32 2024-02-16T22:08:00.644 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rgw-2:18.2.1-0.el9.x86_64 13/32 2024-02-16T22:08:00.644 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-02-16T22:08:00.644 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-02-16T22:08:00.644 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-02-16T22:08:00.644 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-02-16T22:08:00.644 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-02-16T22:08:00.644 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-02-16T22:08:00.644 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-02-16T22:08:00.645 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-02-16T22:08:00.645 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-02-16T22:08:00.645 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-02-16T22:08:00.645 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-02-16T22:08:00.645 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-02-16T22:08:00.645 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-02-16T22:08:00.645 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-02-16T22:08:00.645 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-02-16T22:08:00.645 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:18.2.1-0.el9.x86_64 29/32 2024-02-16T22:08:00.645 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-02-16T22:08:00.645 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:18.2.1-0.el9.x86_64 31/32 2024-02-16T22:08:01.753 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-02-16T22:08:01.753 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:01.753 INFO:teuthology.orchestra.run.smithi123.stdout:Upgraded: 2024-02-16T22:08:01.753 INFO:teuthology.orchestra.run.smithi123.stdout: librados2-2:18.2.1-0.el9.x86_64 librbd1-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:01.753 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-02-16T22:08:01.753 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-16T22:08:01.753 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:01.753 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:01.753 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:01.754 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:01.754 INFO:teuthology.orchestra.run.smithi123.stdout: fmt-8.1.1-5.el9.x86_64 2024-02-16T22:08:01.754 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-16T22:08:01.754 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-16T22:08:01.754 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-16T22:08:01.754 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:01.754 INFO:teuthology.orchestra.run.smithi123.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-16T22:08:01.754 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-16T22:08:01.754 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-16T22:08:01.754 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:01.754 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-16T22:08:01.754 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:01.754 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-16T22:08:01.754 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-16T22:08:01.755 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-16T22:08:01.755 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:01.755 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:01.755 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:01.755 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-16T22:08:01.755 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:01.755 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:01.755 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:01.755 INFO:teuthology.orchestra.run.smithi123.stdout: re2-1:20211101-3.el9.x86_64 2024-02-16T22:08:01.755 INFO:teuthology.orchestra.run.smithi123.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-16T22:08:01.755 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:01.755 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-02-16T22:08:02.099 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-test 2024-02-16T22:08:02.673 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:00:25 ago on Fri 16 Feb 2024 10:07:37 PM UTC. 2024-02-16T22:08:02.836 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-02-16T22:08:02.837 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:02.837 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-02-16T22:08:02.837 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:02.837 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-02-16T22:08:02.837 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test x86_64 2:18.2.1-0.el9 ceph 40 M 2024-02-16T22:08:02.838 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-02-16T22:08:02.838 INFO:teuthology.orchestra.run.smithi123.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-16T22:08:02.838 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite x86_64 2:18.2.1-0.el9 ceph 165 k 2024-02-16T22:08:02.838 INFO:teuthology.orchestra.run.smithi123.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-16T22:08:02.838 INFO:teuthology.orchestra.run.smithi123.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-16T22:08:02.838 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-16T22:08:02.838 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:02.838 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-02-16T22:08:02.838 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:02.838 INFO:teuthology.orchestra.run.smithi123.stdout:Install 6 Packages 2024-02-16T22:08:02.838 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:02.839 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 41 M 2024-02-16T22:08:02.839 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 167 M 2024-02-16T22:08:02.839 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-02-16T22:08:02.951 INFO:teuthology.orchestra.run.smithi123.stdout:(1/6): libcephsqlite-18.2.1-0.el9.x86_64.rpm 1.5 MB/s | 165 kB 00:00 2024-02-16T22:08:03.009 INFO:teuthology.orchestra.run.smithi123.stdout:(2/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 63 kB 00:00 2024-02-16T22:08:03.043 INFO:teuthology.orchestra.run.smithi123.stdout:(3/6): socat-1.7.4.1-5.el9.x86_64.rpm 1.4 MB/s | 300 kB 00:00 2024-02-16T22:08:03.068 INFO:teuthology.orchestra.run.smithi123.stdout:(4/6): jq-1.6-16.el9.x86_64.rpm 3.1 MB/s | 186 kB 00:00 2024-02-16T22:08:03.095 INFO:teuthology.orchestra.run.smithi123.stdout:(5/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-02-16T22:08:03.518 INFO:teuthology.orchestra.run.smithi123.stdout:(6/6): ceph-test-18.2.1-0.el9.x86_64.rpm 58 MB/s | 40 MB 00:00 2024-02-16T22:08:03.521 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:08:03.522 INFO:teuthology.orchestra.run.smithi123.stdout:Total 59 MB/s | 41 MB 00:00 2024-02-16T22:08:03.522 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-02-16T22:08:03.540 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-02-16T22:08:03.541 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-02-16T22:08:03.895 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-02-16T22:08:03.896 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-02-16T22:08:04.432 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-02-16T22:08:04.500 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 1/6 2024-02-16T22:08:04.533 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : jq-1.6-16.el9.x86_64 2/6 2024-02-16T22:08:04.586 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/6 2024-02-16T22:08:04.636 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 4/6 2024-02-16T22:08:04.663 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephsqlite-2:18.2.1-0.el9.x86_64 5/6 2024-02-16T22:08:05.688 INFO:teuthology.orchestra.run.smithi114.stdout:Extra Packages for Enterprise Linux 1.9 MB/s | 20 MB 00:10 2024-02-16T22:08:08.402 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephsqlite-2:18.2.1-0.el9.x86_64 5/6 2024-02-16T22:08:08.488 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-test-2:18.2.1-0.el9.x86_64 6/6 2024-02-16T22:08:10.603 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-test-2:18.2.1-0.el9.x86_64 6/6 2024-02-16T22:08:10.603 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-test-2:18.2.1-0.el9.x86_64 1/6 2024-02-16T22:08:10.603 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephsqlite-2:18.2.1-0.el9.x86_64 2/6 2024-02-16T22:08:10.603 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/6 2024-02-16T22:08:10.603 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/6 2024-02-16T22:08:10.604 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : jq-1.6-16.el9.x86_64 5/6 2024-02-16T22:08:10.687 INFO:teuthology.orchestra.run.smithi114.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-02-16T22:08:10.967 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 6/6 2024-02-16T22:08:10.967 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:10.967 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-02-16T22:08:10.967 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test-2:18.2.1-0.el9.x86_64 jq-1.6-16.el9.x86_64 2024-02-16T22:08:10.967 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite-2:18.2.1-0.el9.x86_64 oniguruma-6.9.6-1.el9.5.x86_64 2024-02-16T22:08:10.967 INFO:teuthology.orchestra.run.smithi123.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-02-16T22:08:10.968 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:10.968 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-02-16T22:08:11.267 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph 2024-02-16T22:08:11.839 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:00:34 ago on Fri 16 Feb 2024 10:07:37 PM UTC. 2024-02-16T22:08:12.011 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-02-16T22:08:12.014 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:12.014 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-02-16T22:08:12.014 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:12.014 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-02-16T22:08:12.014 INFO:teuthology.orchestra.run.smithi123.stdout: ceph x86_64 2:18.2.1-0.el9 ceph 6.4 k 2024-02-16T22:08:12.014 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-02-16T22:08:12.014 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds x86_64 2:18.2.1-0.el9 ceph 2.1 M 2024-02-16T22:08:12.014 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr x86_64 2:18.2.1-0.el9 ceph 1.4 M 2024-02-16T22:08:12.014 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core noarch 2:18.2.1-0.el9 ceph-noarch 242 k 2024-02-16T22:08:12.014 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon x86_64 2:18.2.1-0.el9 ceph 4.4 M 2024-02-16T22:08:12.015 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd x86_64 2:18.2.1-0.el9 ceph 18 M 2024-02-16T22:08:12.015 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-16T22:08:12.015 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-16T22:08:12.015 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-16T22:08:12.015 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-16T22:08:12.015 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-16T22:08:12.015 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-16T22:08:12.015 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-16T22:08:12.015 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-16T22:08:12.015 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-16T22:08:12.015 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-16T22:08:12.015 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-16T22:08:12.015 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-16T22:08:12.016 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-16T22:08:12.016 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-16T22:08:12.016 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-16T22:08:12.016 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-16T22:08:12.016 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-16T22:08:12.016 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-16T22:08:12.016 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-16T22:08:12.016 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-16T22:08:12.016 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-16T22:08:12.016 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-16T22:08:12.016 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-16T22:08:12.016 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-16T22:08:12.016 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-16T22:08:12.016 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-16T22:08:12.017 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-16T22:08:12.017 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:12.017 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-02-16T22:08:12.017 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:12.017 INFO:teuthology.orchestra.run.smithi123.stdout:Install 33 Packages 2024-02-16T22:08:12.017 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:12.017 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 30 M 2024-02-16T22:08:12.018 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 110 M 2024-02-16T22:08:12.018 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-02-16T22:08:12.307 INFO:teuthology.orchestra.run.smithi123.stdout:(1/33): ceph-18.2.1-0.el9.x86_64.rpm 61 kB/s | 6.4 kB 00:00 2024-02-16T22:08:12.356 INFO:teuthology.orchestra.run.smithi123.stdout:(2/33): ceph-mgr-18.2.1-0.el9.x86_64.rpm 9.4 MB/s | 1.4 MB 00:00 2024-02-16T22:08:12.398 INFO:teuthology.orchestra.run.smithi123.stdout:(3/33): ceph-mds-18.2.1-0.el9.x86_64.rpm 11 MB/s | 2.1 MB 00:00 2024-02-16T22:08:12.431 INFO:teuthology.orchestra.run.smithi123.stdout:(4/33): ceph-mgr-modules-core-18.2.1-0.el9.noar 7.1 MB/s | 242 kB 00:00 2024-02-16T22:08:12.507 INFO:teuthology.orchestra.run.smithi123.stdout:(5/33): ceph-mon-18.2.1-0.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-02-16T22:08:12.774 INFO:teuthology.orchestra.run.smithi123.stdout:(6/33): ceph-osd-18.2.1-0.el9.x86_64.rpm 42 MB/s | 18 MB 00:00 2024-02-16T22:08:12.857 INFO:teuthology.orchestra.run.smithi123.stdout:(7/33): python3-markupsafe-1.1.1-12.el9.x86_64. 97 kB/s | 32 kB 00:00 2024-02-16T22:08:12.907 INFO:teuthology.orchestra.run.smithi123.stdout:(8/33): python3-mako-1.1.4-6.el9.noarch.rpm 303 kB/s | 144 kB 00:00 2024-02-16T22:08:12.941 INFO:teuthology.orchestra.run.smithi123.stdout:(9/33): python3-pytz-2021.1-5.el9.noarch.rpm 286 kB/s | 47 kB 00:00 2024-02-16T22:08:12.983 INFO:teuthology.orchestra.run.smithi123.stdout:(10/33): python3-toml-0.10.2-6.el9.noarch.rpm 308 kB/s | 39 kB 00:00 2024-02-16T22:08:13.208 INFO:teuthology.orchestra.run.smithi123.stdout:(11/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 801 kB/s | 241 kB 00:00 2024-02-16T22:08:13.347 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-02-16T22:08:13.356 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:08:13.356 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-02-16T22:08:13.356 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:08:13.356 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-02-16T22:08:13.356 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-radosgw x86_64 2:18.2.1-0.el9 ceph 7.7 M 2024-02-16T22:08:13.356 INFO:teuthology.orchestra.run.smithi114.stdout:Upgrading: 2024-02-16T22:08:13.356 INFO:teuthology.orchestra.run.smithi114.stdout: librados2 x86_64 2:18.2.1-0.el9 ceph 3.3 M 2024-02-16T22:08:13.356 INFO:teuthology.orchestra.run.smithi114.stdout: librbd1 x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-02-16T22:08:13.356 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-02-16T22:08:13.357 INFO:teuthology.orchestra.run.smithi114.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-16T22:08:13.357 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-base x86_64 2:18.2.1-0.el9 ceph 5.2 M 2024-02-16T22:08:13.357 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-common x86_64 2:18.2.1-0.el9 ceph 18 M 2024-02-16T22:08:13.357 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-selinux x86_64 2:18.2.1-0.el9 ceph 24 k 2024-02-16T22:08:13.357 INFO:teuthology.orchestra.run.smithi114.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-02-16T22:08:13.357 INFO:teuthology.orchestra.run.smithi114.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-16T22:08:13.357 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-16T22:08:13.357 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-16T22:08:13.357 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs2 x86_64 2:18.2.1-0.el9 ceph 658 k 2024-02-16T22:08:13.357 INFO:teuthology.orchestra.run.smithi114.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-16T22:08:13.357 INFO:teuthology.orchestra.run.smithi114.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-16T22:08:13.357 INFO:teuthology.orchestra.run.smithi114.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-16T22:08:13.357 INFO:teuthology.orchestra.run.smithi114.stdout: libradosstriper1 x86_64 2:18.2.1-0.el9 ceph 474 k 2024-02-16T22:08:13.357 INFO:teuthology.orchestra.run.smithi114.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-16T22:08:13.358 INFO:teuthology.orchestra.run.smithi114.stdout: librgw2 x86_64 2:18.2.1-0.el9 ceph 4.5 M 2024-02-16T22:08:13.358 INFO:teuthology.orchestra.run.smithi114.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-16T22:08:13.358 INFO:teuthology.orchestra.run.smithi114.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-16T22:08:13.358 INFO:teuthology.orchestra.run.smithi114.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-16T22:08:13.358 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-argparse x86_64 2:18.2.1-0.el9 ceph 45 k 2024-02-16T22:08:13.358 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-common x86_64 2:18.2.1-0.el9 ceph 124 k 2024-02-16T22:08:13.358 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cephfs x86_64 2:18.2.1-0.el9 ceph 161 k 2024-02-16T22:08:13.358 INFO:teuthology.orchestra.run.smithi114.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-16T22:08:13.358 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rados x86_64 2:18.2.1-0.el9 ceph 321 k 2024-02-16T22:08:13.358 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rbd x86_64 2:18.2.1-0.el9 ceph 297 k 2024-02-16T22:08:13.358 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rgw x86_64 2:18.2.1-0.el9 ceph 99 k 2024-02-16T22:08:13.358 INFO:teuthology.orchestra.run.smithi114.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-16T22:08:13.358 INFO:teuthology.orchestra.run.smithi114.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-16T22:08:13.358 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:08:13.358 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-02-16T22:08:13.359 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:08:13.359 INFO:teuthology.orchestra.run.smithi114.stdout:Install 28 Packages 2024-02-16T22:08:13.359 INFO:teuthology.orchestra.run.smithi114.stdout:Upgrade 2 Packages 2024-02-16T22:08:13.359 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:08:13.359 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 53 M 2024-02-16T22:08:13.359 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-02-16T22:08:13.634 INFO:teuthology.orchestra.run.smithi123.stdout:(12/33): python3-ply-3.11-14.el9.noarch.rpm 158 kB/s | 103 kB 00:00 2024-02-16T22:08:13.659 INFO:teuthology.orchestra.run.smithi123.stdout:(13/33): python3-pycparser-2.20-6.el9.noarch.rp 275 kB/s | 124 kB 00:00 2024-02-16T22:08:13.692 INFO:teuthology.orchestra.run.smithi123.stdout:(14/33): python3-cryptography-36.0.1-4.el9.x86_ 1.5 MB/s | 1.1 MB 00:00 2024-02-16T22:08:13.718 INFO:teuthology.orchestra.run.smithi123.stdout:(15/33): python3-requests-2.25.1-8.el9.noarch.r 1.3 MB/s | 113 kB 00:00 2024-02-16T22:08:13.743 INFO:teuthology.orchestra.run.smithi123.stdout:(16/33): python3-urllib3-1.26.5-5.el9.noarch.rp 2.2 MB/s | 187 kB 00:00 2024-02-16T22:08:13.872 INFO:teuthology.orchestra.run.smithi114.stdout:(1/30): ceph-base-18.2.1-0.el9.x86_64.rpm 30 MB/s | 5.2 MB 00:00 2024-02-16T22:08:13.897 INFO:teuthology.orchestra.run.smithi114.stdout:(2/30): ceph-selinux-18.2.1-0.el9.x86_64.rpm 966 kB/s | 24 kB 00:00 2024-02-16T22:08:13.931 INFO:teuthology.orchestra.run.smithi114.stdout:(3/30): libcephfs2-18.2.1-0.el9.x86_64.rpm 19 MB/s | 658 kB 00:00 2024-02-16T22:08:13.935 INFO:teuthology.orchestra.run.smithi123.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 201 kB/s | 43 kB 00:00 2024-02-16T22:08:13.968 INFO:teuthology.orchestra.run.smithi123.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 688 kB/s | 172 kB 00:00 2024-02-16T22:08:13.973 INFO:teuthology.orchestra.run.smithi114.stdout:(4/30): libradosstriper1-18.2.1-0.el9.x86_64.rp 11 MB/s | 474 kB 00:00 2024-02-16T22:08:13.985 INFO:teuthology.orchestra.run.smithi123.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 213 kB/s | 11 kB 00:00 2024-02-16T22:08:14.002 INFO:teuthology.orchestra.run.smithi123.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 527 kB/s | 18 kB 00:00 2024-02-16T22:08:14.027 INFO:teuthology.orchestra.run.smithi123.stdout:(21/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.2 MB/s | 358 kB 00:00 2024-02-16T22:08:14.044 INFO:teuthology.orchestra.run.smithi123.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 393 kB/s | 23 kB 00:00 2024-02-16T22:08:14.061 INFO:teuthology.orchestra.run.smithi123.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 331 kB/s | 19 kB 00:00 2024-02-16T22:08:14.078 INFO:teuthology.orchestra.run.smithi123.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 390 kB/s | 20 kB 00:00 2024-02-16T22:08:14.090 INFO:teuthology.orchestra.run.smithi114.stdout:(5/30): ceph-radosgw-18.2.1-0.el9.x86_64.rpm 20 MB/s | 7.7 MB 00:00 2024-02-16T22:08:14.095 INFO:teuthology.orchestra.run.smithi123.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 1.4 MB/s | 46 kB 00:00 2024-02-16T22:08:14.115 INFO:teuthology.orchestra.run.smithi114.stdout:(6/30): python3-ceph-argparse-18.2.1-0.el9.x86_ 1.8 MB/s | 45 kB 00:00 2024-02-16T22:08:14.120 INFO:teuthology.orchestra.run.smithi123.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 1.3 MB/s | 79 kB 00:00 2024-02-16T22:08:14.137 INFO:teuthology.orchestra.run.smithi123.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 393 kB/s | 16 kB 00:00 2024-02-16T22:08:14.140 INFO:teuthology.orchestra.run.smithi114.stdout:(7/30): python3-ceph-common-18.2.1-0.el9.x86_64 4.8 MB/s | 124 kB 00:00 2024-02-16T22:08:14.162 INFO:teuthology.orchestra.run.smithi123.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 2.1 MB/s | 90 kB 00:00 2024-02-16T22:08:14.166 INFO:teuthology.orchestra.run.smithi114.stdout:(8/30): python3-cephfs-18.2.1-0.el9.x86_64.rpm 6.3 MB/s | 161 kB 00:00 2024-02-16T22:08:14.179 INFO:teuthology.orchestra.run.smithi123.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 854 kB/s | 36 kB 00:00 2024-02-16T22:08:14.223 INFO:teuthology.orchestra.run.smithi123.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-02-16T22:08:14.224 INFO:teuthology.orchestra.run.smithi114.stdout:(9/30): librgw2-18.2.1-0.el9.x86_64.rpm 18 MB/s | 4.5 MB 00:00 2024-02-16T22:08:14.250 INFO:teuthology.orchestra.run.smithi114.stdout:(10/30): python3-rados-18.2.1-0.el9.x86_64.rpm 3.8 MB/s | 321 kB 00:00 2024-02-16T22:08:14.263 INFO:teuthology.orchestra.run.smithi123.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-02-16T22:08:14.275 INFO:teuthology.orchestra.run.smithi114.stdout:(11/30): python3-rbd-18.2.1-0.el9.x86_64.rpm 5.8 MB/s | 297 kB 00:00 2024-02-16T22:08:14.288 INFO:teuthology.orchestra.run.smithi123.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 310 kB/s | 20 kB 00:00 2024-02-16T22:08:14.300 INFO:teuthology.orchestra.run.smithi114.stdout:(12/30): python3-rgw-18.2.1-0.el9.x86_64.rpm 1.9 MB/s | 99 kB 00:00 2024-02-16T22:08:14.330 INFO:teuthology.orchestra.run.smithi123.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.8 MB/s | 427 kB 00:00 2024-02-16T22:08:14.337 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:08:14.337 INFO:teuthology.orchestra.run.smithi123.stdout:Total 13 MB/s | 30 MB 00:02 2024-02-16T22:08:14.338 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-02-16T22:08:14.375 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-02-16T22:08:14.375 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-02-16T22:08:14.550 INFO:teuthology.orchestra.run.smithi114.stdout:(13/30): ceph-common-18.2.1-0.el9.x86_64.rpm 22 MB/s | 18 MB 00:00 2024-02-16T22:08:14.609 INFO:teuthology.orchestra.run.smithi114.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 317 kB/s | 106 kB 00:00 2024-02-16T22:08:14.676 INFO:teuthology.orchestra.run.smithi114.stdout:(15/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 453 kB/s | 159 kB 00:00 2024-02-16T22:08:14.701 INFO:teuthology.orchestra.run.smithi114.stdout:(16/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 294 kB/s | 44 kB 00:00 2024-02-16T22:08:14.731 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-02-16T22:08:14.732 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-02-16T22:08:14.843 INFO:teuthology.orchestra.run.smithi114.stdout:(17/30): librdkafka-1.6.1-102.el9.x86_64.rpm 2.8 MB/s | 661 kB 00:00 2024-02-16T22:08:14.868 INFO:teuthology.orchestra.run.smithi114.stdout:(18/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.4 MB/s | 282 kB 00:00 2024-02-16T22:08:14.894 INFO:teuthology.orchestra.run.smithi114.stdout:(19/30): python3-prettytable-0.7.2-27.el9.noarc 214 kB/s | 41 kB 00:00 2024-02-16T22:08:15.094 INFO:teuthology.orchestra.run.smithi114.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 443 kB/s | 111 kB 00:00 2024-02-16T22:08:15.119 INFO:teuthology.orchestra.run.smithi114.stdout:(21/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.3 MB/s | 309 kB 00:00 2024-02-16T22:08:15.144 INFO:teuthology.orchestra.run.smithi114.stdout:(22/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-02-16T22:08:15.171 INFO:teuthology.orchestra.run.smithi114.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 969 kB/s | 49 kB 00:00 2024-02-16T22:08:15.195 INFO:teuthology.orchestra.run.smithi114.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-02-16T22:08:15.279 INFO:teuthology.orchestra.run.smithi114.stdout:(25/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.6 MB/s | 838 kB 00:00 2024-02-16T22:08:15.345 INFO:teuthology.orchestra.run.smithi114.stdout:(26/30): libarrow-9.0.0-10.el9.x86_64.rpm 9.8 MB/s | 4.4 MB 00:00 2024-02-16T22:08:15.371 INFO:teuthology.orchestra.run.smithi114.stdout:(27/30): re2-20211101-3.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-02-16T22:08:15.421 INFO:teuthology.orchestra.run.smithi114.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-02-16T22:08:15.479 INFO:teuthology.orchestra.run.smithi114.stdout:(29/30): librados2-18.2.1-0.el9.x86_64.rpm 31 MB/s | 3.3 MB 00:00 2024-02-16T22:08:15.529 INFO:teuthology.orchestra.run.smithi114.stdout:(30/30): librbd1-18.2.1-0.el9.x86_64.rpm 19 MB/s | 3.0 MB 00:00 2024-02-16T22:08:15.536 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:08:15.536 INFO:teuthology.orchestra.run.smithi114.stdout:Total 24 MB/s | 53 MB 00:02 2024-02-16T22:08:15.536 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-02-16T22:08:15.566 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-02-16T22:08:15.567 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-02-16T22:08:15.949 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-02-16T22:08:16.016 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-02-16T22:08:16.020 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-02-16T22:08:16.020 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-02-16T22:08:16.040 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-02-16T22:08:16.064 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-02-16T22:08:16.155 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-02-16T22:08:16.233 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-02-16T22:08:16.275 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-02-16T22:08:16.346 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-02-16T22:08:16.400 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-02-16T22:08:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-02-16T22:08:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-02-16T22:08:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-02-16T22:08:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-02-16T22:08:16.776 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-02-16T22:08:16.812 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-02-16T22:08:16.849 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-02-16T22:08:16.879 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-02-16T22:08:16.955 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-02-16T22:08:16.998 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-02-16T22:08:17.039 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-02-16T22:08:17.080 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-02-16T22:08:17.106 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-02-16T22:08:17.141 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-02-16T22:08:17.229 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-02-16T22:08:17.263 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-02-16T22:08:17.351 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-02-16T22:08:17.394 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-02-16T22:08:17.472 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-02-16T22:08:17.488 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-02-16T22:08:17.503 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-02-16T22:08:17.553 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-02-16T22:08:17.604 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-02-16T22:08:17.760 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 28/33 2024-02-16T22:08:17.785 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-02-16T22:08:17.810 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-02-16T22:08:17.810 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T22:08:17.810 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-16T22:08:17.810 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-16T22:08:17.810 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-16T22:08:17.810 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:17.861 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-02-16T22:08:17.919 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : librados2-2:18.2.1-0.el9.x86_64 5/32 2024-02-16T22:08:18.010 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librados2-2:18.2.1-0.el9.x86_64 5/32 2024-02-16T22:08:18.090 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rados-2:18.2.1-0.el9.x86_64 6/32 2024-02-16T22:08:18.120 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-02-16T22:08:18.200 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-02-16T22:08:18.220 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-02-16T22:08:18.255 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 9/32 2024-02-16T22:08:18.321 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-02-16T22:08:18.348 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-02-16T22:08:18.413 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-02-16T22:08:18.519 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-02-16T22:08:18.550 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-02-16T22:08:18.587 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-02-16T22:08:18.620 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-02-16T22:08:18.647 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-02-16T22:08:18.683 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-02-16T22:08:18.715 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-02-16T22:08:19.014 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-02-16T22:08:19.045 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-02-16T22:08:19.118 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-02-16T22:08:19.150 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rbd-2:18.2.1-0.el9.x86_64 21/32 2024-02-16T22:08:19.190 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-02-16T22:08:19.353 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-02-16T22:08:19.381 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-02-16T22:08:19.381 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T22:08:19.381 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-16T22:08:19.381 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-16T22:08:19.381 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-16T22:08:19.381 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:19.589 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-02-16T22:08:19.614 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-02-16T22:08:19.710 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-02-16T22:08:19.786 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rgw-2:18.2.1-0.el9.x86_64 25/32 2024-02-16T22:08:19.817 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ceph-common-2:18.2.1-0.el9.x86_64 26/32 2024-02-16T22:08:19.837 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-02-16T22:08:19.861 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-02-16T22:08:19.861 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T22:08:19.861 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-16T22:08:19.861 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-16T22:08:19.861 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-16T22:08:19.862 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:20.063 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-02-16T22:08:20.086 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-02-16T22:08:20.086 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T22:08:20.086 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-16T22:08:20.086 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-16T22:08:20.086 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-16T22:08:20.086 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:20.152 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-2:18.2.1-0.el9.x86_64 33/33 2024-02-16T22:08:21.861 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-02-16T22:08:21.897 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-02-16T22:08:21.998 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-2:18.2.1-0.el9.x86_64 33/33 2024-02-16T22:08:21.998 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-2:18.2.1-0.el9.x86_64 1/33 2024-02-16T22:08:21.998 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mds-2:18.2.1-0.el9.x86_64 2/33 2024-02-16T22:08:21.998 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-2:18.2.1-0.el9.x86_64 3/33 2024-02-16T22:08:21.998 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mon-2:18.2.1-0.el9.x86_64 4/33 2024-02-16T22:08:21.998 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-osd-2:18.2.1-0.el9.x86_64 5/33 2024-02-16T22:08:21.998 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 6/33 2024-02-16T22:08:21.998 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-02-16T22:08:21.998 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-02-16T22:08:21.999 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-02-16T22:08:21.999 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-02-16T22:08:21.999 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-02-16T22:08:21.999 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-02-16T22:08:21.999 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-02-16T22:08:21.999 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-02-16T22:08:21.999 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 15/33 2024-02-16T22:08:21.999 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 16/33 2024-02-16T22:08:21.999 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-02-16T22:08:21.999 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-02-16T22:08:21.999 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-02-16T22:08:21.999 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-02-16T22:08:21.999 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-02-16T22:08:21.999 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-02-16T22:08:22.000 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-02-16T22:08:22.000 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-02-16T22:08:22.000 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-02-16T22:08:22.000 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-02-16T22:08:22.000 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-02-16T22:08:22.000 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-02-16T22:08:22.000 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-02-16T22:08:22.000 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-02-16T22:08:22.000 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-02-16T22:08:22.000 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-02-16T22:08:22.444 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-02-16T22:08:22.475 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-02-16T22:08:22.559 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-02-16T22:08:22.559 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-16T22:08:22.559 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-16T22:08:22.559 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:08:22.593 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-02-16T22:08:22.940 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-02-16T22:08:22.941 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:22.941 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-02-16T22:08:22.941 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:22.941 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:22.941 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:22.941 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 2024-02-16T22:08:22.941 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:22.941 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:22.941 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-16T22:08:22.941 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-16T22:08:22.941 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-16T22:08:22.942 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-16T22:08:22.942 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-16T22:08:22.942 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-16T22:08:22.942 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-16T22:08:22.942 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-16T22:08:22.942 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-16T22:08:22.942 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-16T22:08:22.942 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-16T22:08:22.942 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-16T22:08:22.942 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-16T22:08:22.942 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-16T22:08:22.942 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-16T22:08:22.942 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply-3.11-14.el9.noarch 2024-02-16T22:08:22.943 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-16T22:08:22.943 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-16T22:08:22.943 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-16T22:08:22.943 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-16T22:08:22.943 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-16T22:08:22.943 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-16T22:08:22.943 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-16T22:08:22.943 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-16T22:08:22.943 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-16T22:08:22.943 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-16T22:08:22.943 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-16T22:08:22.943 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:22.943 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-02-16T22:08:23.369 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-base 2024-02-16T22:08:23.955 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:00:46 ago on Fri 16 Feb 2024 10:07:37 PM UTC. 2024-02-16T22:08:24.068 INFO:teuthology.orchestra.run.smithi123.stdout:Package ceph-base-2:18.2.1-0.el9.x86_64 is already installed. 2024-02-16T22:08:24.119 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-02-16T22:08:24.120 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-02-16T22:08:24.120 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-02-16T22:08:24.181 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install cephadm 2024-02-16T22:08:24.756 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:00:47 ago on Fri 16 Feb 2024 10:07:37 PM UTC. 2024-02-16T22:08:24.919 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-02-16T22:08:24.920 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:24.920 INFO:teuthology.orchestra.run.smithi123.stdout: Package Architecture Version Repository Size 2024-02-16T22:08:24.920 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:24.920 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-02-16T22:08:24.921 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm noarch 2:18.2.1-0.el9 ceph-noarch 221 k 2024-02-16T22:08:24.921 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:24.921 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-02-16T22:08:24.921 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:24.921 INFO:teuthology.orchestra.run.smithi123.stdout:Install 1 Package 2024-02-16T22:08:24.921 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:24.921 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 221 k 2024-02-16T22:08:24.921 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 213 k 2024-02-16T22:08:24.921 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-02-16T22:08:24.994 INFO:teuthology.orchestra.run.smithi123.stdout:cephadm-18.2.1-0.el9.noarch.rpm 3.0 MB/s | 221 kB 00:00 2024-02-16T22:08:24.995 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:08:24.995 INFO:teuthology.orchestra.run.smithi123.stdout:Total 2.9 MB/s | 221 kB 00:00 2024-02-16T22:08:24.995 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-02-16T22:08:25.000 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-02-16T22:08:25.000 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-02-16T22:08:25.003 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-02-16T22:08:25.003 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-02-16T22:08:25.071 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-02-16T22:08:25.513 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-02-16T22:08:25.623 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-02-16T22:08:26.017 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-02-16T22:08:26.330 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-02-16T22:08:26.330 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:26.331 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-02-16T22:08:26.331 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm-2:18.2.1-0.el9.noarch 2024-02-16T22:08:26.331 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:26.331 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-02-16T22:08:26.522 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-immutable-object-cache 2024-02-16T22:08:27.103 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:00:50 ago on Fri 16 Feb 2024 10:07:37 PM UTC. 2024-02-16T22:08:27.268 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-02-16T22:08:27.269 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:27.269 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repo Size 2024-02-16T22:08:27.269 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:27.269 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-02-16T22:08:27.269 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-0.el9 ceph 142 k 2024-02-16T22:08:27.269 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:27.269 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-02-16T22:08:27.269 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:27.269 INFO:teuthology.orchestra.run.smithi123.stdout:Install 1 Package 2024-02-16T22:08:27.269 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:27.270 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 142 k 2024-02-16T22:08:27.270 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 395 k 2024-02-16T22:08:27.270 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-02-16T22:08:27.339 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-immutable-object-cache-18.2.1-0.el9.x86_64 2.0 MB/s | 142 kB 00:00 2024-02-16T22:08:27.340 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:08:27.340 INFO:teuthology.orchestra.run.smithi123.stdout:Total 2.0 MB/s | 142 kB 00:00 2024-02-16T22:08:27.340 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-02-16T22:08:27.348 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-02-16T22:08:27.348 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-02-16T22:08:27.399 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-02-16T22:08:27.399 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-02-16T22:08:27.539 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-02-16T22:08:27.599 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:08:27.623 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:08:27.623 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T22:08:27.623 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-16T22:08:27.623 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:28.611 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:08:28.611 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:28.611 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-02-16T22:08:28.611 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:28.611 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:28.611 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-02-16T22:08:28.844 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-mgr 2024-02-16T22:08:29.443 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:00:52 ago on Fri 16 Feb 2024 10:07:37 PM UTC. 2024-02-16T22:08:29.582 INFO:teuthology.orchestra.run.smithi123.stdout:Package ceph-mgr-2:18.2.1-0.el9.x86_64 is already installed. 2024-02-16T22:08:29.633 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-02-16T22:08:29.634 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-02-16T22:08:29.634 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-02-16T22:08:29.702 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-mgr-dashboard 2024-02-16T22:08:30.267 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:00:53 ago on Fri 16 Feb 2024 10:07:37 PM UTC. 2024-02-16T22:08:30.432 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-02-16T22:08:30.433 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:30.433 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-02-16T22:08:30.433 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:30.433 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-02-16T22:08:30.433 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard noarch 2:18.2.1-0.el9 ceph-noarch 1.8 M 2024-02-16T22:08:30.433 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-02-16T22:08:30.433 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards noarch 2:18.2.1-0.el9 ceph-noarch 23 k 2024-02-16T22:08:30.433 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts noarch 2:18.2.1-0.el9 ceph-noarch 15 k 2024-02-16T22:08:30.433 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-02-16T22:08:30.434 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-16T22:08:30.434 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-16T22:08:30.434 INFO:teuthology.orchestra.run.smithi123.stdout:Installing weak dependencies: 2024-02-16T22:08:30.434 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-02-16T22:08:30.434 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:30.434 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-02-16T22:08:30.434 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:30.434 INFO:teuthology.orchestra.run.smithi123.stdout:Install 7 Packages 2024-02-16T22:08:30.434 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:30.435 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 2.1 M 2024-02-16T22:08:30.435 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 8.5 M 2024-02-16T22:08:30.435 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-02-16T22:08:30.760 INFO:teuthology.orchestra.run.smithi123.stdout:(1/7): ceph-grafana-dashboards-18.2.1-0.el9.noa 181 kB/s | 23 kB 00:00 2024-02-16T22:08:30.785 INFO:teuthology.orchestra.run.smithi123.stdout:(2/7): ceph-prometheus-alerts-18.2.1-0.el9.noar 95 kB/s | 15 kB 00:00 2024-02-16T22:08:30.844 INFO:teuthology.orchestra.run.smithi123.stdout:(3/7): ceph-mgr-dashboard-18.2.1-0.el9.noarch.r 8.3 MB/s | 1.8 MB 00:00 2024-02-16T22:08:31.011 INFO:teuthology.orchestra.run.smithi123.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 36 kB/s | 9.0 kB 00:00 2024-02-16T22:08:31.194 INFO:teuthology.orchestra.run.smithi123.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 100 kB/s | 41 kB 00:00 2024-02-16T22:08:31.244 INFO:teuthology.orchestra.run.smithi123.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 77 kB/s | 31 kB 00:00 2024-02-16T22:08:31.886 INFO:teuthology.orchestra.run.smithi123.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 215 kB/s | 188 kB 00:00 2024-02-16T22:08:31.889 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:08:31.889 INFO:teuthology.orchestra.run.smithi123.stdout:Total 1.4 MB/s | 2.1 MB 00:01 2024-02-16T22:08:31.889 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-02-16T22:08:31.898 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-02-16T22:08:31.898 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-02-16T22:08:31.969 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-02-16T22:08:31.970 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-02-16T22:08:32.022 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-02-16T22:08:32.022 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /sys 2024-02-16T22:08:32.022 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /proc 2024-02-16T22:08:32.022 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /mnt 2024-02-16T22:08:32.022 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /var/tmp 2024-02-16T22:08:32.022 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /home 2024-02-16T22:08:32.022 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /root 2024-02-16T22:08:32.022 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /tmp 2024-02-16T22:08:32.022 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:08:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-02-16T22:08:32.337 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-02-16T22:08:32.371 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-02-16T22:08:32.426 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-02-16T22:08:32.458 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-02-16T22:08:32.486 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 5/7 2024-02-16T22:08:32.721 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 6/7 2024-02-16T22:08:32.748 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-02-16T22:08:32.774 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-02-16T22:08:32.799 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-02-16T22:08:32.799 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T22:08:32.799 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-16T22:08:32.799 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-16T22:08:32.799 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-16T22:08:32.799 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:08:32.801 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-02-16T22:08:32.885 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-02-16T22:08:32.885 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-02-16T22:08:33.289 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-02-16T22:08:33.289 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 1/7 2024-02-16T22:08:33.289 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2/7 2024-02-16T22:08:33.289 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 3/7 2024-02-16T22:08:33.290 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-02-16T22:08:33.290 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-02-16T22:08:33.290 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-02-16T22:08:33.594 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-02-16T22:08:33.594 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:33.594 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-02-16T22:08:33.594 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 2024-02-16T22:08:33.594 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2024-02-16T22:08:33.594 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 2024-02-16T22:08:33.594 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-02-16T22:08:33.594 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-02-16T22:08:33.594 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-16T22:08:33.594 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-16T22:08:33.594 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:33.595 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-02-16T22:08:33.810 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-16T22:08:34.018 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-02-16T22:08:34.018 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-base-2:18.2.1-0.el9.x86_64 1/32 2024-02-16T22:08:34.018 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-common-2:18.2.1-0.el9.x86_64 2/32 2024-02-16T22:08:34.019 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-radosgw-2:18.2.1-0.el9.x86_64 3/32 2024-02-16T22:08:34.019 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-selinux-2:18.2.1-0.el9.x86_64 4/32 2024-02-16T22:08:34.019 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephfs2-2:18.2.1-0.el9.x86_64 5/32 2024-02-16T22:08:34.019 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libradosstriper1-2:18.2.1-0.el9.x86_64 6/32 2024-02-16T22:08:34.019 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librgw2-2:18.2.1-0.el9.x86_64 7/32 2024-02-16T22:08:34.019 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 8/32 2024-02-16T22:08:34.019 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ceph-common-2:18.2.1-0.el9.x86_64 9/32 2024-02-16T22:08:34.019 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-02-16T22:08:34.019 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rados-2:18.2.1-0.el9.x86_64 11/32 2024-02-16T22:08:34.019 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rbd-2:18.2.1-0.el9.x86_64 12/32 2024-02-16T22:08:34.019 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rgw-2:18.2.1-0.el9.x86_64 13/32 2024-02-16T22:08:34.019 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-02-16T22:08:34.019 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-02-16T22:08:34.020 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-02-16T22:08:34.020 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-02-16T22:08:34.020 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-02-16T22:08:34.020 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-02-16T22:08:34.020 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-02-16T22:08:34.020 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-02-16T22:08:34.020 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-02-16T22:08:34.020 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-02-16T22:08:34.020 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-02-16T22:08:34.020 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-02-16T22:08:34.020 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-02-16T22:08:34.021 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-02-16T22:08:34.021 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-02-16T22:08:34.021 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados2-2:18.2.1-0.el9.x86_64 29/32 2024-02-16T22:08:34.021 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-02-16T22:08:34.021 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librbd1-2:18.2.1-0.el9.x86_64 31/32 2024-02-16T22:08:34.389 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:00:57 ago on Fri 16 Feb 2024 10:07:37 PM UTC. 2024-02-16T22:08:34.556 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-02-16T22:08:34.557 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:34.557 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-02-16T22:08:34.557 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:34.557 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-02-16T22:08:34.558 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-0.el9 ceph-noarch 7.4 M 2024-02-16T22:08:34.558 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-02-16T22:08:34.558 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-16T22:08:34.558 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-16T22:08:34.558 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-16T22:08:34.558 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-16T22:08:34.558 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-16T22:08:34.558 INFO:teuthology.orchestra.run.smithi123.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-16T22:08:34.558 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-16T22:08:34.558 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-16T22:08:34.558 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-16T22:08:34.558 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-16T22:08:34.558 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-16T22:08:34.559 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:34.559 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-02-16T22:08:34.559 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:34.559 INFO:teuthology.orchestra.run.smithi123.stdout:Install 12 Packages 2024-02-16T22:08:34.559 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:34.559 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 37 M 2024-02-16T22:08:34.559 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 209 M 2024-02-16T22:08:34.560 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-02-16T22:08:34.843 INFO:teuthology.orchestra.run.smithi123.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 220 kB/s | 32 kB 00:00 2024-02-16T22:08:34.943 INFO:teuthology.orchestra.run.smithi123.stdout:(2/12): ceph-mgr-diskprediction-local-18.2.1-0. 30 MB/s | 7.4 MB 00:00 2024-02-16T22:08:34.960 INFO:teuthology.orchestra.run.smithi123.stdout:(3/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 147 kB/s | 17 kB 00:00 2024-02-16T22:08:34.976 INFO:teuthology.orchestra.run.smithi123.stdout:(4/12): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 34 kB 00:00 2024-02-16T22:08:35.010 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-02-16T22:08:35.010 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:08:35.010 INFO:teuthology.orchestra.run.smithi114.stdout:Upgraded: 2024-02-16T22:08:35.010 INFO:teuthology.orchestra.run.smithi114.stdout: librados2-2:18.2.1-0.el9.x86_64 librbd1-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:35.010 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-02-16T22:08:35.010 INFO:teuthology.orchestra.run.smithi114.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-16T22:08:35.010 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-base-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:35.010 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-common-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:35.010 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-radosgw-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:35.010 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-selinux-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:35.010 INFO:teuthology.orchestra.run.smithi114.stdout: fmt-8.1.1-5.el9.x86_64 2024-02-16T22:08:35.010 INFO:teuthology.orchestra.run.smithi114.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-16T22:08:35.011 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-16T22:08:35.011 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-16T22:08:35.011 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs2-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:35.011 INFO:teuthology.orchestra.run.smithi114.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-16T22:08:35.011 INFO:teuthology.orchestra.run.smithi114.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-16T22:08:35.011 INFO:teuthology.orchestra.run.smithi114.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-16T22:08:35.011 INFO:teuthology.orchestra.run.smithi114.stdout: libradosstriper1-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:35.011 INFO:teuthology.orchestra.run.smithi114.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-16T22:08:35.011 INFO:teuthology.orchestra.run.smithi114.stdout: librgw2-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:35.011 INFO:teuthology.orchestra.run.smithi114.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-16T22:08:35.011 INFO:teuthology.orchestra.run.smithi114.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-16T22:08:35.011 INFO:teuthology.orchestra.run.smithi114.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-16T22:08:35.011 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-argparse-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:35.011 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-common-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:35.011 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cephfs-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:35.012 INFO:teuthology.orchestra.run.smithi114.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-16T22:08:35.012 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rados-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:35.012 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rbd-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:35.012 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rgw-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:35.012 INFO:teuthology.orchestra.run.smithi114.stdout: re2-1:20211101-3.el9.x86_64 2024-02-16T22:08:35.012 INFO:teuthology.orchestra.run.smithi114.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-16T22:08:35.012 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:08:35.012 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-02-16T22:08:35.168 INFO:teuthology.orchestra.run.smithi123.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 23 MB/s | 4.7 MB 00:00 2024-02-16T22:08:35.243 INFO:teuthology.orchestra.run.smithi123.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 18 MB/s | 4.8 MB 00:00 2024-02-16T22:08:35.265 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-test 2024-02-16T22:08:35.269 INFO:teuthology.orchestra.run.smithi123.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.2 MB/s | 217 kB 00:00 2024-02-16T22:08:35.469 INFO:teuthology.orchestra.run.smithi123.stdout:(8/12): libgfortran-11.4.1-3.el9.x86_64.rpm 4.0 MB/s | 802 kB 00:00 2024-02-16T22:08:35.544 INFO:teuthology.orchestra.run.smithi123.stdout:(9/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.5 MB/s | 3.0 MB 00:00 2024-02-16T22:08:35.703 INFO:teuthology.orchestra.run.smithi123.stdout:(10/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-02-16T22:08:35.728 INFO:teuthology.orchestra.run.smithi123.stdout:(11/12): libquadmath-11.4.1-3.el9.x86_64.rpm 759 kB/s | 194 kB 00:00 2024-02-16T22:08:35.836 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:25 ago on Fri 16 Feb 2024 10:08:10 PM UTC. 2024-02-16T22:08:35.903 INFO:teuthology.orchestra.run.smithi123.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 686 kB/s | 245 kB 00:00 2024-02-16T22:08:35.906 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:08:35.906 INFO:teuthology.orchestra.run.smithi123.stdout:Total 28 MB/s | 37 MB 00:01 2024-02-16T22:08:35.928 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-02-16T22:08:35.952 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-02-16T22:08:35.952 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-02-16T22:08:36.006 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-02-16T22:08:36.007 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:08:36.007 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-02-16T22:08:36.007 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:08:36.008 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-02-16T22:08:36.008 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-test x86_64 2:18.2.1-0.el9 ceph 40 M 2024-02-16T22:08:36.008 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-02-16T22:08:36.008 INFO:teuthology.orchestra.run.smithi114.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-16T22:08:36.008 INFO:teuthology.orchestra.run.smithi114.stdout: libcephsqlite x86_64 2:18.2.1-0.el9 ceph 165 k 2024-02-16T22:08:36.008 INFO:teuthology.orchestra.run.smithi114.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-16T22:08:36.008 INFO:teuthology.orchestra.run.smithi114.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-16T22:08:36.008 INFO:teuthology.orchestra.run.smithi114.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-16T22:08:36.008 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:08:36.008 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-02-16T22:08:36.008 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:08:36.008 INFO:teuthology.orchestra.run.smithi114.stdout:Install 6 Packages 2024-02-16T22:08:36.009 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:08:36.009 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 41 M 2024-02-16T22:08:36.009 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 167 M 2024-02-16T22:08:36.009 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-02-16T22:08:36.117 INFO:teuthology.orchestra.run.smithi114.stdout:(1/6): libcephsqlite-18.2.1-0.el9.x86_64.rpm 1.5 MB/s | 165 kB 00:00 2024-02-16T22:08:36.209 INFO:teuthology.orchestra.run.smithi114.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 1.5 MB/s | 300 kB 00:00 2024-02-16T22:08:36.234 INFO:teuthology.orchestra.run.smithi114.stdout:(3/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 537 kB/s | 63 kB 00:00 2024-02-16T22:08:36.276 INFO:teuthology.orchestra.run.smithi114.stdout:(4/6): jq-1.6-16.el9.x86_64.rpm 2.7 MB/s | 186 kB 00:00 2024-02-16T22:08:36.301 INFO:teuthology.orchestra.run.smithi114.stdout:(5/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.2 MB/s | 217 kB 00:00 2024-02-16T22:08:36.400 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-02-16T22:08:36.401 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-02-16T22:08:36.710 INFO:teuthology.orchestra.run.smithi114.stdout:(6/6): ceph-test-18.2.1-0.el9.x86_64.rpm 57 MB/s | 40 MB 00:00 2024-02-16T22:08:36.713 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:08:36.714 INFO:teuthology.orchestra.run.smithi114.stdout:Total 57 MB/s | 41 MB 00:00 2024-02-16T22:08:36.714 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-02-16T22:08:36.733 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-02-16T22:08:36.734 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-02-16T22:08:37.097 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-02-16T22:08:37.098 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-02-16T22:08:37.114 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-02-16T22:08:37.185 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-02-16T22:08:37.216 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-02-16T22:08:37.309 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-02-16T22:08:37.344 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-02-16T22:08:37.523 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-16T22:08:37.551 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-16T22:08:37.629 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-02-16T22:08:37.637 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-02-16T22:08:37.700 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 1/6 2024-02-16T22:08:37.734 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : jq-1.6-16.el9.x86_64 2/6 2024-02-16T22:08:37.781 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/6 2024-02-16T22:08:37.827 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 4/6 2024-02-16T22:08:37.856 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephsqlite-2:18.2.1-0.el9.x86_64 5/6 2024-02-16T22:08:38.310 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-02-16T22:08:38.423 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-02-16T22:08:39.659 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-02-16T22:08:40.690 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-02-16T22:08:40.733 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-02-16T22:08:41.600 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libcephsqlite-2:18.2.1-0.el9.x86_64 5/6 2024-02-16T22:08:41.687 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-test-2:18.2.1-0.el9.x86_64 6/6 2024-02-16T22:08:43.305 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-02-16T22:08:43.306 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 1/12 2024-02-16T22:08:43.306 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-02-16T22:08:43.306 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-02-16T22:08:43.306 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-02-16T22:08:43.306 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-16T22:08:43.306 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-16T22:08:43.306 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-02-16T22:08:43.306 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-02-16T22:08:43.306 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-02-16T22:08:43.306 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-02-16T22:08:43.306 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-02-16T22:08:43.741 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-02-16T22:08:43.741 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:43.741 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-02-16T22:08:43.741 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarch 2024-02-16T22:08:43.741 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-16T22:08:43.741 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-16T22:08:43.741 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-16T22:08:43.741 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-16T22:08:43.741 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-16T22:08:43.741 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-16T22:08:43.742 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-16T22:08:43.742 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-16T22:08:43.742 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-16T22:08:43.742 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-16T22:08:43.742 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-16T22:08:43.742 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:43.742 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-02-16T22:08:43.853 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-test-2:18.2.1-0.el9.x86_64 6/6 2024-02-16T22:08:43.854 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-test-2:18.2.1-0.el9.x86_64 1/6 2024-02-16T22:08:43.854 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephsqlite-2:18.2.1-0.el9.x86_64 2/6 2024-02-16T22:08:43.854 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/6 2024-02-16T22:08:43.854 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/6 2024-02-16T22:08:43.854 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : jq-1.6-16.el9.x86_64 5/6 2024-02-16T22:08:44.148 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-mgr-rook 2024-02-16T22:08:44.234 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 6/6 2024-02-16T22:08:44.234 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:08:44.234 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-02-16T22:08:44.234 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-test-2:18.2.1-0.el9.x86_64 jq-1.6-16.el9.x86_64 2024-02-16T22:08:44.235 INFO:teuthology.orchestra.run.smithi114.stdout: libcephsqlite-2:18.2.1-0.el9.x86_64 oniguruma-6.9.6-1.el9.5.x86_64 2024-02-16T22:08:44.235 INFO:teuthology.orchestra.run.smithi114.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-02-16T22:08:44.235 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:08:44.235 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-02-16T22:08:44.534 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph 2024-02-16T22:08:44.735 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:07 ago on Fri 16 Feb 2024 10:07:37 PM UTC. 2024-02-16T22:08:44.900 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-02-16T22:08:44.901 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:44.901 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-02-16T22:08:44.901 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:44.901 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-02-16T22:08:44.901 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook noarch 2:18.2.1-0.el9 ceph-noarch 50 k 2024-02-16T22:08:44.901 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-02-16T22:08:44.901 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-16T22:08:44.901 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-16T22:08:44.901 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-16T22:08:44.902 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-16T22:08:44.902 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-16T22:08:44.902 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-16T22:08:44.902 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-16T22:08:44.902 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-16T22:08:44.902 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-16T22:08:44.902 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-16T22:08:44.902 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-16T22:08:44.902 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-16T22:08:44.902 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:44.902 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-02-16T22:08:44.902 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:44.902 INFO:teuthology.orchestra.run.smithi123.stdout:Install 13 Packages 2024-02-16T22:08:44.902 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:44.903 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 2.0 M 2024-02-16T22:08:44.903 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 26 M 2024-02-16T22:08:44.903 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-02-16T22:08:45.108 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:35 ago on Fri 16 Feb 2024 10:08:10 PM UTC. 2024-02-16T22:08:45.149 INFO:teuthology.orchestra.run.smithi123.stdout:(1/13): ceph-mgr-rook-18.2.1-0.el9.noarch.rpm 503 kB/s | 50 kB 00:00 2024-02-16T22:08:45.174 INFO:teuthology.orchestra.run.smithi123.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 196 kB/s | 24 kB 00:00 2024-02-16T22:08:45.199 INFO:teuthology.orchestra.run.smithi123.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 120 kB/s | 18 kB 00:00 2024-02-16T22:08:45.225 INFO:teuthology.orchestra.run.smithi123.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.3 MB/s | 174 kB 00:00 2024-02-16T22:08:45.250 INFO:teuthology.orchestra.run.smithi123.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-02-16T22:08:45.275 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-02-16T22:08:45.277 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:08:45.278 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-02-16T22:08:45.278 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:08:45.278 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-02-16T22:08:45.278 INFO:teuthology.orchestra.run.smithi114.stdout: ceph x86_64 2:18.2.1-0.el9 ceph 6.4 k 2024-02-16T22:08:45.278 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-02-16T22:08:45.278 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mds x86_64 2:18.2.1-0.el9 ceph 2.1 M 2024-02-16T22:08:45.278 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr x86_64 2:18.2.1-0.el9 ceph 1.4 M 2024-02-16T22:08:45.278 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-modules-core noarch 2:18.2.1-0.el9 ceph-noarch 242 k 2024-02-16T22:08:45.278 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mon x86_64 2:18.2.1-0.el9 ceph 4.4 M 2024-02-16T22:08:45.278 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-osd x86_64 2:18.2.1-0.el9 ceph 18 M 2024-02-16T22:08:45.278 INFO:teuthology.orchestra.run.smithi114.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-16T22:08:45.278 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-16T22:08:45.279 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-16T22:08:45.279 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-16T22:08:45.279 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-16T22:08:45.279 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-16T22:08:45.279 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-16T22:08:45.279 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-16T22:08:45.279 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-16T22:08:45.279 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-16T22:08:45.279 INFO:teuthology.orchestra.run.smithi114.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-16T22:08:45.279 INFO:teuthology.orchestra.run.smithi114.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-16T22:08:45.279 INFO:teuthology.orchestra.run.smithi114.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-16T22:08:45.279 INFO:teuthology.orchestra.run.smithi114.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-16T22:08:45.280 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-16T22:08:45.280 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-16T22:08:45.280 INFO:teuthology.orchestra.run.smithi114.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-16T22:08:45.280 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-16T22:08:45.280 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-16T22:08:45.280 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-16T22:08:45.280 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-16T22:08:45.280 INFO:teuthology.orchestra.run.smithi114.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-16T22:08:45.280 INFO:teuthology.orchestra.run.smithi114.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-16T22:08:45.280 INFO:teuthology.orchestra.run.smithi114.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-16T22:08:45.280 INFO:teuthology.orchestra.run.smithi114.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-16T22:08:45.280 INFO:teuthology.orchestra.run.smithi114.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-16T22:08:45.280 INFO:teuthology.orchestra.run.smithi114.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-16T22:08:45.280 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:08:45.281 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-02-16T22:08:45.281 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:08:45.281 INFO:teuthology.orchestra.run.smithi114.stdout:Install 33 Packages 2024-02-16T22:08:45.281 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:08:45.281 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 30 M 2024-02-16T22:08:45.281 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 110 M 2024-02-16T22:08:45.281 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-02-16T22:08:45.284 INFO:teuthology.orchestra.run.smithi123.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.5 MB/s | 211 kB 00:00 2024-02-16T22:08:45.309 INFO:teuthology.orchestra.run.smithi123.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 512 kB/s | 43 kB 00:00 2024-02-16T22:08:45.392 INFO:teuthology.orchestra.run.smithi123.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 180 kB/s | 15 kB 00:00 2024-02-16T22:08:45.418 INFO:teuthology.orchestra.run.smithi123.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 192 kB/s | 32 kB 00:00 2024-02-16T22:08:45.485 INFO:teuthology.orchestra.run.smithi123.stdout:(10/13): python3-google-auth-2.27.0-1.el9.noarc 1.1 MB/s | 203 kB 00:00 2024-02-16T22:08:45.510 INFO:teuthology.orchestra.run.smithi123.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 645 kB/s | 59 kB 00:00 2024-02-16T22:08:45.552 INFO:teuthology.orchestra.run.smithi123.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.3 MB/s | 90 kB 00:00 2024-02-16T22:08:45.652 INFO:teuthology.orchestra.run.smithi123.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.9 MB/s | 1.0 MB 00:00 2024-02-16T22:08:45.654 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:08:45.654 INFO:teuthology.orchestra.run.smithi123.stdout:Total 2.7 MB/s | 2.0 MB 00:00 2024-02-16T22:08:45.654 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-02-16T22:08:45.667 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-02-16T22:08:45.667 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-02-16T22:08:45.763 INFO:teuthology.orchestra.run.smithi114.stdout:(1/33): ceph-18.2.1-0.el9.x86_64.rpm 58 kB/s | 6.4 kB 00:00 2024-02-16T22:08:45.765 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-02-16T22:08:45.765 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-02-16T22:08:45.813 INFO:teuthology.orchestra.run.smithi114.stdout:(2/33): ceph-mgr-18.2.1-0.el9.x86_64.rpm 9.0 MB/s | 1.4 MB 00:00 2024-02-16T22:08:45.863 INFO:teuthology.orchestra.run.smithi114.stdout:(3/33): ceph-mds-18.2.1-0.el9.x86_64.rpm 10 MB/s | 2.1 MB 00:00 2024-02-16T22:08:45.890 INFO:teuthology.orchestra.run.smithi114.stdout:(4/33): ceph-mgr-modules-core-18.2.1-0.el9.noar 9.5 MB/s | 242 kB 00:00 2024-02-16T22:08:45.981 INFO:teuthology.orchestra.run.smithi114.stdout:(5/33): ceph-mon-18.2.1-0.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-02-16T22:08:46.039 INFO:teuthology.orchestra.run.smithi114.stdout:(6/33): python3-mako-1.1.4-6.el9.noarch.rpm 959 kB/s | 144 kB 00:00 2024-02-16T22:08:46.064 INFO:teuthology.orchestra.run.smithi114.stdout:(7/33): python3-markupsafe-1.1.1-12.el9.x86_64. 383 kB/s | 32 kB 00:00 2024-02-16T22:08:46.089 INFO:teuthology.orchestra.run.smithi114.stdout:(8/33): python3-pytz-2021.1-5.el9.noarch.rpm 947 kB/s | 47 kB 00:00 2024-02-16T22:08:46.115 INFO:teuthology.orchestra.run.smithi114.stdout:(9/33): python3-toml-0.10.2-6.el9.noarch.rpm 773 kB/s | 39 kB 00:00 2024-02-16T22:08:46.140 INFO:teuthology.orchestra.run.smithi114.stdout:(10/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-02-16T22:08:46.191 INFO:teuthology.orchestra.run.smithi114.stdout:(11/33): python3-cryptography-36.0.1-4.el9.x86_ 15 MB/s | 1.1 MB 00:00 2024-02-16T22:08:46.280 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-02-16T22:08:46.366 INFO:teuthology.orchestra.run.smithi114.stdout:(12/33): ceph-osd-18.2.1-0.el9.x86_64.rpm 32 MB/s | 18 MB 00:00 2024-02-16T22:08:46.391 INFO:teuthology.orchestra.run.smithi114.stdout:(13/33): python3-ply-3.11-14.el9.noarch.rpm 411 kB/s | 103 kB 00:00 2024-02-16T22:08:46.416 INFO:teuthology.orchestra.run.smithi114.stdout:(14/33): python3-pycparser-2.20-6.el9.noarch.rp 550 kB/s | 124 kB 00:00 2024-02-16T22:08:46.431 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-02-16T22:08:46.450 INFO:teuthology.orchestra.run.smithi114.stdout:(15/33): python3-requests-2.25.1-8.el9.noarch.r 1.9 MB/s | 113 kB 00:00 2024-02-16T22:08:46.475 INFO:teuthology.orchestra.run.smithi114.stdout:(16/33): python3-urllib3-1.26.5-5.el9.noarch.rp 2.2 MB/s | 187 kB 00:00 2024-02-16T22:08:46.477 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-02-16T22:08:46.533 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-02-16T22:08:46.562 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-02-16T22:08:46.603 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-02-16T22:08:46.660 INFO:teuthology.orchestra.run.smithi114.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 178 kB/s | 43 kB 00:00 2024-02-16T22:08:46.697 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-02-16T22:08:46.734 INFO:teuthology.orchestra.run.smithi114.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 146 kB/s | 11 kB 00:00 2024-02-16T22:08:46.784 INFO:teuthology.orchestra.run.smithi114.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 514 kB/s | 172 kB 00:00 2024-02-16T22:08:46.797 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 7/13 2024-02-16T22:08:46.801 INFO:teuthology.orchestra.run.smithi114.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 264 kB/s | 18 kB 00:00 2024-02-16T22:08:46.843 INFO:teuthology.orchestra.run.smithi114.stdout:(21/33): python3-cherrypy-18.6.1-2.el9.noarch.r 974 kB/s | 358 kB 00:00 2024-02-16T22:08:46.846 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-02-16T22:08:46.868 INFO:teuthology.orchestra.run.smithi114.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 276 kB/s | 23 kB 00:00 2024-02-16T22:08:46.894 INFO:teuthology.orchestra.run.smithi114.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 211 kB/s | 19 kB 00:00 2024-02-16T22:08:46.919 INFO:teuthology.orchestra.run.smithi114.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 261 kB/s | 20 kB 00:00 2024-02-16T22:08:46.978 INFO:teuthology.orchestra.run.smithi114.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 944 kB/s | 79 kB 00:00 2024-02-16T22:08:47.002 INFO:teuthology.orchestra.run.smithi114.stdout:(26/33): python3-logutils-0.3.5-21.el9.noarch.r 426 kB/s | 46 kB 00:00 2024-02-16T22:08:47.028 INFO:teuthology.orchestra.run.smithi114.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-02-16T22:08:47.057 INFO:teuthology.orchestra.run.smithi114.stdout:(28/33): python3-portend-3.1.0-2.el9.noarch.rpm 207 kB/s | 16 kB 00:00 2024-02-16T22:08:47.079 INFO:teuthology.orchestra.run.smithi114.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-02-16T22:08:47.104 INFO:teuthology.orchestra.run.smithi114.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 470 kB/s | 36 kB 00:00 2024-02-16T22:08:47.163 INFO:teuthology.orchestra.run.smithi114.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 343 kB/s | 20 kB 00:00 2024-02-16T22:08:47.188 INFO:teuthology.orchestra.run.smithi114.stdout:(32/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-02-16T22:08:47.222 INFO:teuthology.orchestra.run.smithi114.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.9 MB/s | 427 kB 00:00 2024-02-16T22:08:47.225 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:08:47.225 INFO:teuthology.orchestra.run.smithi114.stdout:Total 15 MB/s | 30 MB 00:01 2024-02-16T22:08:47.225 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-02-16T22:08:47.264 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-02-16T22:08:47.264 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-02-16T22:08:47.556 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-02-16T22:08:47.602 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-02-16T22:08:47.619 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-02-16T22:08:47.619 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-02-16T22:08:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-02-16T22:08:47.683 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-02-16T22:08:47.705 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-02-16T22:08:48.550 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-02-16T22:08:48.551 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-rook-2:18.2.1-0.el9.noarch 1/13 2024-02-16T22:08:48.551 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-02-16T22:08:48.551 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-02-16T22:08:48.551 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-02-16T22:08:48.551 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-02-16T22:08:48.551 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-02-16T22:08:48.552 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-02-16T22:08:48.552 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-02-16T22:08:48.552 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-02-16T22:08:48.552 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 10/13 2024-02-16T22:08:48.552 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-02-16T22:08:48.552 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-02-16T22:08:48.846 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-02-16T22:08:48.916 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-02-16T22:08:48.919 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-02-16T22:08:48.919 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:48.919 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-02-16T22:08:48.919 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook-2:18.2.1-0.el9.noarch 2024-02-16T22:08:48.919 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-16T22:08:48.919 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-16T22:08:48.919 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-16T22:08:48.919 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-16T22:08:48.919 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-16T22:08:48.919 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-16T22:08:48.919 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-16T22:08:48.919 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-16T22:08:48.920 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-16T22:08:48.920 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-16T22:08:48.920 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-16T22:08:48.920 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-16T22:08:48.920 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:48.920 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-02-16T22:08:48.956 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-02-16T22:08:48.989 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-02-16T22:08:49.080 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-02-16T22:08:49.166 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-02-16T22:08:49.183 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-mgr-cephadm 2024-02-16T22:08:49.208 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-02-16T22:08:49.279 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-02-16T22:08:49.342 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-02-16T22:08:49.380 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-02-16T22:08:49.429 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-02-16T22:08:49.492 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-02-16T22:08:49.684 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-02-16T22:08:49.743 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-02-16T22:08:49.765 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:12 ago on Fri 16 Feb 2024 10:07:37 PM UTC. 2024-02-16T22:08:49.783 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-02-16T22:08:49.825 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-02-16T22:08:49.855 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-02-16T22:08:49.931 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-02-16T22:08:49.932 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-02-16T22:08:49.933 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:49.933 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-02-16T22:08:49.933 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:49.933 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-02-16T22:08:49.933 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm noarch 2:18.2.1-0.el9 ceph-noarch 132 k 2024-02-16T22:08:49.933 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-02-16T22:08:49.933 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-02-16T22:08:49.933 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-16T22:08:49.933 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-16T22:08:49.934 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-16T22:08:49.934 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-16T22:08:49.934 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:49.934 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-02-16T22:08:49.934 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:49.934 INFO:teuthology.orchestra.run.smithi123.stdout:Install 6 Packages 2024-02-16T22:08:49.934 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:49.934 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 6.8 M 2024-02-16T22:08:49.935 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 33 M 2024-02-16T22:08:49.935 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-02-16T22:08:49.965 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-02-16T22:08:49.999 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-02-16T22:08:50.039 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-02-16T22:08:50.074 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-02-16T22:08:50.087 INFO:teuthology.orchestra.run.smithi123.stdout:(1/6): ceph-mgr-cephadm-18.2.1-0.el9.noarch.rpm 1.2 MB/s | 132 kB 00:00 2024-02-16T22:08:50.109 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-02-16T22:08:50.187 INFO:teuthology.orchestra.run.smithi123.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.1 MB/s | 227 kB 00:00 2024-02-16T22:08:50.205 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-02-16T22:08:50.248 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-02-16T22:08:50.319 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-02-16T22:08:50.445 INFO:teuthology.orchestra.run.smithi123.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 223 kB/s | 58 kB 00:00 2024-02-16T22:08:50.456 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-02-16T22:08:50.471 INFO:teuthology.orchestra.run.smithi123.stdout:(4/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 1.4 MB/s | 548 kB 00:00 2024-02-16T22:08:50.512 INFO:teuthology.orchestra.run.smithi123.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 772 kB/s | 51 kB 00:00 2024-02-16T22:08:50.571 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-02-16T22:08:50.696 INFO:teuthology.orchestra.run.smithi123.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 8.1 MB/s | 5.8 MB 00:00 2024-02-16T22:08:50.700 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:08:50.700 INFO:teuthology.orchestra.run.smithi123.stdout:Total 8.8 MB/s | 6.8 MB 00:00 2024-02-16T22:08:50.700 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-02-16T22:08:50.709 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-02-16T22:08:50.709 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-02-16T22:08:50.727 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 28/33 2024-02-16T22:08:50.753 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-02-16T22:08:50.778 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-02-16T22:08:50.778 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T22:08:50.778 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-16T22:08:50.778 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-16T22:08:50.778 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-16T22:08:50.778 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:08:50.799 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-02-16T22:08:50.800 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-02-16T22:08:51.106 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-02-16T22:08:51.277 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-02-16T22:08:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2024-02-16T22:08:51.841 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-02-16T22:08:51.933 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-02-16T22:08:51.994 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-02-16T22:08:52.018 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-02-16T22:08:52.281 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-02-16T22:08:52.308 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-02-16T22:08:52.308 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T22:08:52.308 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-16T22:08:52.308 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-16T22:08:52.308 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-16T22:08:52.309 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:08:52.757 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-02-16T22:08:52.780 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-02-16T22:08:52.780 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T22:08:52.780 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-16T22:08:52.780 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-16T22:08:52.780 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-16T22:08:52.780 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:08:52.821 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-02-16T22:08:52.821 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 1/6 2024-02-16T22:08:52.821 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-02-16T22:08:52.821 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-02-16T22:08:52.821 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2024-02-16T22:08:52.821 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-02-16T22:08:52.982 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-02-16T22:08:53.005 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-02-16T22:08:53.006 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T22:08:53.006 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-16T22:08:53.006 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-16T22:08:53.006 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-16T22:08:53.006 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:08:53.063 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-2:18.2.1-0.el9.x86_64 33/33 2024-02-16T22:08:53.134 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-02-16T22:08:53.134 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:53.134 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-02-16T22:08:53.134 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 2024-02-16T22:08:53.134 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-02-16T22:08:53.134 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-16T22:08:53.134 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-16T22:08:53.134 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-16T22:08:53.134 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-16T22:08:53.134 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:53.134 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-02-16T22:08:53.401 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-fuse 2024-02-16T22:08:53.984 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:16 ago on Fri 16 Feb 2024 10:07:37 PM UTC. 2024-02-16T22:08:54.150 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-02-16T22:08:54.150 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:54.150 INFO:teuthology.orchestra.run.smithi123.stdout: Package Architecture Version Repository Size 2024-02-16T22:08:54.150 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:54.150 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-02-16T22:08:54.150 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse x86_64 2:18.2.1-0.el9 ceph 839 k 2024-02-16T22:08:54.151 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-02-16T22:08:54.151 INFO:teuthology.orchestra.run.smithi123.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-16T22:08:54.151 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:54.151 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-02-16T22:08:54.151 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:54.151 INFO:teuthology.orchestra.run.smithi123.stdout:Install 2 Packages 2024-02-16T22:08:54.151 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:54.151 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 918 k 2024-02-16T22:08:54.151 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 2.7 M 2024-02-16T22:08:54.152 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-02-16T22:08:54.256 INFO:teuthology.orchestra.run.smithi123.stdout:(1/2): ceph-fuse-18.2.1-0.el9.x86_64.rpm 7.9 MB/s | 839 kB 00:00 2024-02-16T22:08:54.314 INFO:teuthology.orchestra.run.smithi123.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 485 kB/s | 79 kB 00:00 2024-02-16T22:08:54.315 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:08:54.316 INFO:teuthology.orchestra.run.smithi123.stdout:Total 5.4 MB/s | 918 kB 00:00 2024-02-16T22:08:54.316 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-02-16T22:08:54.337 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-02-16T22:08:54.337 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-02-16T22:08:54.414 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-02-16T22:08:54.414 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-02-16T22:08:54.603 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-02-16T22:08:54.728 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-02-16T22:08:54.797 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-02-16T22:08:54.803 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-2:18.2.1-0.el9.x86_64 33/33 2024-02-16T22:08:54.803 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-2:18.2.1-0.el9.x86_64 1/33 2024-02-16T22:08:54.803 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mds-2:18.2.1-0.el9.x86_64 2/33 2024-02-16T22:08:54.803 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-2:18.2.1-0.el9.x86_64 3/33 2024-02-16T22:08:54.803 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mon-2:18.2.1-0.el9.x86_64 4/33 2024-02-16T22:08:54.803 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-osd-2:18.2.1-0.el9.x86_64 5/33 2024-02-16T22:08:54.803 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 6/33 2024-02-16T22:08:54.803 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-02-16T22:08:54.803 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-02-16T22:08:54.803 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-02-16T22:08:54.803 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-02-16T22:08:54.803 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-02-16T22:08:54.803 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-02-16T22:08:54.804 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-02-16T22:08:54.804 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-02-16T22:08:54.804 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 15/33 2024-02-16T22:08:54.804 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 16/33 2024-02-16T22:08:54.804 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-02-16T22:08:54.804 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-02-16T22:08:54.804 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-02-16T22:08:54.804 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-02-16T22:08:54.804 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-02-16T22:08:54.804 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-02-16T22:08:54.804 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-02-16T22:08:54.804 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-02-16T22:08:54.804 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-02-16T22:08:54.805 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-02-16T22:08:54.807 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-02-16T22:08:54.807 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-02-16T22:08:54.807 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-02-16T22:08:54.807 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-02-16T22:08:54.807 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-02-16T22:08:54.807 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-02-16T22:08:55.469 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-02-16T22:08:55.470 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-fuse-2:18.2.1-0.el9.x86_64 1/2 2024-02-16T22:08:55.718 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-02-16T22:08:55.718 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:08:55.718 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-02-16T22:08:55.718 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:55.719 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mds-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:55.719 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:55.719 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 2024-02-16T22:08:55.719 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mon-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:55.719 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-osd-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:55.719 INFO:teuthology.orchestra.run.smithi114.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-16T22:08:55.719 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-16T22:08:55.719 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-16T22:08:55.719 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-16T22:08:55.719 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-16T22:08:55.719 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-16T22:08:55.719 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-16T22:08:55.719 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-16T22:08:55.719 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-16T22:08:55.719 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-16T22:08:55.720 INFO:teuthology.orchestra.run.smithi114.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-16T22:08:55.720 INFO:teuthology.orchestra.run.smithi114.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-16T22:08:55.720 INFO:teuthology.orchestra.run.smithi114.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-16T22:08:55.720 INFO:teuthology.orchestra.run.smithi114.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-16T22:08:55.720 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-16T22:08:55.720 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ply-3.11-14.el9.noarch 2024-02-16T22:08:55.720 INFO:teuthology.orchestra.run.smithi114.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-16T22:08:55.720 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-16T22:08:55.720 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-16T22:08:55.720 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-16T22:08:55.720 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-16T22:08:55.720 INFO:teuthology.orchestra.run.smithi114.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-16T22:08:55.720 INFO:teuthology.orchestra.run.smithi114.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-16T22:08:55.720 INFO:teuthology.orchestra.run.smithi114.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-16T22:08:55.720 INFO:teuthology.orchestra.run.smithi114.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-16T22:08:55.721 INFO:teuthology.orchestra.run.smithi114.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-16T22:08:55.721 INFO:teuthology.orchestra.run.smithi114.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-16T22:08:55.721 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:08:55.721 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-02-16T22:08:55.774 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-02-16T22:08:55.775 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:55.775 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-02-16T22:08:55.775 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse-2:18.2.1-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-02-16T22:08:55.775 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:55.775 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-02-16T22:08:56.051 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-base 2024-02-16T22:08:56.071 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install librados-devel 2024-02-16T22:08:56.643 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:46 ago on Fri 16 Feb 2024 10:08:10 PM UTC. 2024-02-16T22:08:56.651 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:19 ago on Fri 16 Feb 2024 10:07:37 PM UTC. 2024-02-16T22:08:56.757 INFO:teuthology.orchestra.run.smithi114.stdout:Package ceph-base-2:18.2.1-0.el9.x86_64 is already installed. 2024-02-16T22:08:56.808 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-02-16T22:08:56.809 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-02-16T22:08:56.809 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-02-16T22:08:56.818 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-02-16T22:08:56.819 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:56.819 INFO:teuthology.orchestra.run.smithi123.stdout: Package Architecture Version Repository Size 2024-02-16T22:08:56.819 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:56.819 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-02-16T22:08:56.819 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel x86_64 2:18.2.1-0.el9 ceph 127 k 2024-02-16T22:08:56.819 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:56.819 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-02-16T22:08:56.819 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:56.819 INFO:teuthology.orchestra.run.smithi123.stdout:Install 1 Package 2024-02-16T22:08:56.820 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:56.820 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 127 k 2024-02-16T22:08:56.820 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 456 k 2024-02-16T22:08:56.820 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-02-16T22:08:56.870 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install cephadm 2024-02-16T22:08:56.876 INFO:teuthology.orchestra.run.smithi123.stdout:librados-devel-18.2.1-0.el9.x86_64.rpm 2.2 MB/s | 127 kB 00:00 2024-02-16T22:08:56.877 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:08:56.877 INFO:teuthology.orchestra.run.smithi123.stdout:Total 2.2 MB/s | 127 kB 00:00 2024-02-16T22:08:56.877 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-02-16T22:08:56.883 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-02-16T22:08:56.883 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-02-16T22:08:56.937 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-02-16T22:08:56.937 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-02-16T22:08:57.079 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-02-16T22:08:57.173 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:08:57.438 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:47 ago on Fri 16 Feb 2024 10:08:10 PM UTC. 2024-02-16T22:08:57.587 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:08:57.601 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-02-16T22:08:57.602 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:08:57.602 INFO:teuthology.orchestra.run.smithi114.stdout: Package Architecture Version Repository Size 2024-02-16T22:08:57.602 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:08:57.602 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-02-16T22:08:57.602 INFO:teuthology.orchestra.run.smithi114.stdout: cephadm noarch 2:18.2.1-0.el9 ceph-noarch 221 k 2024-02-16T22:08:57.602 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:08:57.602 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-02-16T22:08:57.602 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:08:57.602 INFO:teuthology.orchestra.run.smithi114.stdout:Install 1 Package 2024-02-16T22:08:57.603 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:08:57.603 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 221 k 2024-02-16T22:08:57.603 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 213 k 2024-02-16T22:08:57.603 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-02-16T22:08:57.672 INFO:teuthology.orchestra.run.smithi114.stdout:cephadm-18.2.1-0.el9.noarch.rpm 3.1 MB/s | 221 kB 00:00 2024-02-16T22:08:57.673 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:08:57.673 INFO:teuthology.orchestra.run.smithi114.stdout:Total 3.0 MB/s | 221 kB 00:00 2024-02-16T22:08:57.673 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-02-16T22:08:57.678 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-02-16T22:08:57.678 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-02-16T22:08:57.681 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-02-16T22:08:57.681 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-02-16T22:08:57.752 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-02-16T22:08:57.874 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:08:57.874 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:57.874 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-02-16T22:08:57.874 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel-2:18.2.1-0.el9.x86_64 2024-02-16T22:08:57.874 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:57.874 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-02-16T22:08:58.099 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-02-16T22:08:58.154 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install libcephfs2 2024-02-16T22:08:58.176 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-02-16T22:08:58.556 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-02-16T22:08:58.739 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:21 ago on Fri 16 Feb 2024 10:07:37 PM UTC. 2024-02-16T22:08:58.863 INFO:teuthology.orchestra.run.smithi123.stdout:Package libcephfs2-2:18.2.1-0.el9.x86_64 is already installed. 2024-02-16T22:08:58.877 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-02-16T22:08:58.877 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:08:58.877 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-02-16T22:08:58.877 INFO:teuthology.orchestra.run.smithi114.stdout: cephadm-2:18.2.1-0.el9.noarch 2024-02-16T22:08:58.877 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:08:58.877 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-02-16T22:08:58.914 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-02-16T22:08:58.916 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-02-16T22:08:58.916 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-02-16T22:08:58.990 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install libcephfs-devel 2024-02-16T22:08:59.087 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-immutable-object-cache 2024-02-16T22:08:59.578 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:22 ago on Fri 16 Feb 2024 10:07:37 PM UTC. 2024-02-16T22:08:59.664 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:49 ago on Fri 16 Feb 2024 10:08:10 PM UTC. 2024-02-16T22:08:59.743 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-02-16T22:08:59.743 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:59.744 INFO:teuthology.orchestra.run.smithi123.stdout: Package Architecture Version Repository Size 2024-02-16T22:08:59.744 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:59.744 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-02-16T22:08:59.744 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel x86_64 2:18.2.1-0.el9 ceph 31 k 2024-02-16T22:08:59.744 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:59.744 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-02-16T22:08:59.744 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:08:59.744 INFO:teuthology.orchestra.run.smithi123.stdout:Install 1 Package 2024-02-16T22:08:59.744 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:08:59.744 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 31 k 2024-02-16T22:08:59.744 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 139 k 2024-02-16T22:08:59.744 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-02-16T22:08:59.813 INFO:teuthology.orchestra.run.smithi123.stdout:libcephfs-devel-18.2.1-0.el9.x86_64.rpm 451 kB/s | 31 kB 00:00 2024-02-16T22:08:59.813 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:08:59.814 INFO:teuthology.orchestra.run.smithi123.stdout:Total 443 kB/s | 31 kB 00:00 2024-02-16T22:08:59.814 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-02-16T22:08:59.817 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-02-16T22:08:59.817 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-02-16T22:08:59.833 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-02-16T22:08:59.833 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:08:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repo Size 2024-02-16T22:08:59.834 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:08:59.834 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-02-16T22:08:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-0.el9 ceph 142 k 2024-02-16T22:08:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:08:59.834 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-02-16T22:08:59.834 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:08:59.834 INFO:teuthology.orchestra.run.smithi114.stdout:Install 1 Package 2024-02-16T22:08:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:08:59.835 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 142 k 2024-02-16T22:08:59.835 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 395 k 2024-02-16T22:08:59.835 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-02-16T22:08:59.846 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-02-16T22:08:59.846 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-02-16T22:08:59.911 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-immutable-object-cache-18.2.1-0.el9.x86_64 1.8 MB/s | 142 kB 00:00 2024-02-16T22:08:59.912 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:08:59.912 INFO:teuthology.orchestra.run.smithi114.stdout:Total 1.8 MB/s | 142 kB 00:00 2024-02-16T22:08:59.912 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-02-16T22:08:59.920 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-02-16T22:08:59.920 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-02-16T22:08:59.939 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-02-16T22:08:59.971 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-02-16T22:08:59.971 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-02-16T22:09:00.035 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:09:00.110 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-02-16T22:09:00.179 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:09:00.204 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:09:00.204 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T22:09:00.204 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-16T22:09:00.204 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:00.429 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:09:00.708 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:09:00.708 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:09:00.708 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-02-16T22:09:00.708 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel-2:18.2.1-0.el9.x86_64 2024-02-16T22:09:00.708 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:09:00.708 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-02-16T22:09:00.889 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install librados2 2024-02-16T22:09:01.166 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:09:01.167 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:01.167 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-02-16T22:09:01.167 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 2024-02-16T22:09:01.167 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:01.167 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-02-16T22:09:01.415 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-mgr 2024-02-16T22:09:01.469 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:24 ago on Fri 16 Feb 2024 10:07:37 PM UTC. 2024-02-16T22:09:01.583 INFO:teuthology.orchestra.run.smithi123.stdout:Package librados2-2:18.2.1-0.el9.x86_64 is already installed. 2024-02-16T22:09:01.635 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-02-16T22:09:01.636 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-02-16T22:09:01.636 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-02-16T22:09:01.701 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install librbd1 2024-02-16T22:09:02.005 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:52 ago on Fri 16 Feb 2024 10:08:10 PM UTC. 2024-02-16T22:09:02.119 INFO:teuthology.orchestra.run.smithi114.stdout:Package ceph-mgr-2:18.2.1-0.el9.x86_64 is already installed. 2024-02-16T22:09:02.171 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-02-16T22:09:02.172 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-02-16T22:09:02.172 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-02-16T22:09:02.228 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-mgr-dashboard 2024-02-16T22:09:02.276 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:25 ago on Fri 16 Feb 2024 10:07:37 PM UTC. 2024-02-16T22:09:02.391 INFO:teuthology.orchestra.run.smithi123.stdout:Package librbd1-2:18.2.1-0.el9.x86_64 is already installed. 2024-02-16T22:09:02.443 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-02-16T22:09:02.444 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-02-16T22:09:02.444 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-02-16T22:09:02.511 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install python3-rados 2024-02-16T22:09:02.799 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:52 ago on Fri 16 Feb 2024 10:08:10 PM UTC. 2024-02-16T22:09:02.964 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-02-16T22:09:02.964 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:09:02.964 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-02-16T22:09:02.965 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:09:02.965 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-02-16T22:09:02.965 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-dashboard noarch 2:18.2.1-0.el9 ceph-noarch 1.8 M 2024-02-16T22:09:02.965 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-02-16T22:09:02.965 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-grafana-dashboards noarch 2:18.2.1-0.el9 ceph-noarch 23 k 2024-02-16T22:09:02.965 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-prometheus-alerts noarch 2:18.2.1-0.el9 ceph-noarch 15 k 2024-02-16T22:09:02.965 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-02-16T22:09:02.965 INFO:teuthology.orchestra.run.smithi114.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-16T22:09:02.965 INFO:teuthology.orchestra.run.smithi114.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-16T22:09:02.965 INFO:teuthology.orchestra.run.smithi114.stdout:Installing weak dependencies: 2024-02-16T22:09:02.965 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-02-16T22:09:02.965 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:02.966 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-02-16T22:09:02.966 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:09:02.966 INFO:teuthology.orchestra.run.smithi114.stdout:Install 7 Packages 2024-02-16T22:09:02.966 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:02.966 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 2.1 M 2024-02-16T22:09:02.966 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 8.5 M 2024-02-16T22:09:02.966 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-02-16T22:09:03.079 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:26 ago on Fri 16 Feb 2024 10:07:37 PM UTC. 2024-02-16T22:09:03.173 INFO:teuthology.orchestra.run.smithi114.stdout:(1/7): ceph-grafana-dashboards-18.2.1-0.el9.noa 199 kB/s | 23 kB 00:00 2024-02-16T22:09:03.190 INFO:teuthology.orchestra.run.smithi114.stdout:(2/7): ceph-prometheus-alerts-18.2.1-0.el9.noar 110 kB/s | 15 kB 00:00 2024-02-16T22:09:03.191 INFO:teuthology.orchestra.run.smithi123.stdout:Package python3-rados-2:18.2.1-0.el9.x86_64 is already installed. 2024-02-16T22:09:03.242 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-02-16T22:09:03.243 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-02-16T22:09:03.243 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-02-16T22:09:03.249 INFO:teuthology.orchestra.run.smithi114.stdout:(3/7): ceph-mgr-dashboard-18.2.1-0.el9.noarch.r 9.2 MB/s | 1.8 MB 00:00 2024-02-16T22:09:03.313 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install python3-rgw 2024-02-16T22:09:03.316 INFO:teuthology.orchestra.run.smithi114.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 64 kB/s | 9.0 kB 00:00 2024-02-16T22:09:03.383 INFO:teuthology.orchestra.run.smithi114.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 212 kB/s | 41 kB 00:00 2024-02-16T22:09:03.408 INFO:teuthology.orchestra.run.smithi114.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 193 kB/s | 31 kB 00:00 2024-02-16T22:09:03.558 INFO:teuthology.orchestra.run.smithi114.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 779 kB/s | 188 kB 00:00 2024-02-16T22:09:03.560 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:09:03.560 INFO:teuthology.orchestra.run.smithi114.stdout:Total 3.5 MB/s | 2.1 MB 00:00 2024-02-16T22:09:03.560 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-02-16T22:09:03.568 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-02-16T22:09:03.568 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-02-16T22:09:03.632 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-02-16T22:09:03.633 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-02-16T22:09:03.887 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:26 ago on Fri 16 Feb 2024 10:07:37 PM UTC. 2024-02-16T22:09:03.924 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-02-16T22:09:03.976 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-02-16T22:09:04.000 INFO:teuthology.orchestra.run.smithi123.stdout:Package python3-rgw-2:18.2.1-0.el9.x86_64 is already installed. 2024-02-16T22:09:04.010 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-02-16T22:09:04.051 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-02-16T22:09:04.052 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-02-16T22:09:04.052 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-02-16T22:09:04.056 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-02-16T22:09:04.088 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-02-16T22:09:04.109 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 5/7 2024-02-16T22:09:04.118 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install python3-cephfs 2024-02-16T22:09:04.335 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 6/7 2024-02-16T22:09:04.391 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-02-16T22:09:04.697 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:27 ago on Fri 16 Feb 2024 10:07:37 PM UTC. 2024-02-16T22:09:04.812 INFO:teuthology.orchestra.run.smithi123.stdout:Package python3-cephfs-2:18.2.1-0.el9.x86_64 is already installed. 2024-02-16T22:09:04.864 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-02-16T22:09:04.865 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-02-16T22:09:04.865 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-02-16T22:09:04.926 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-02-16T22:09:04.926 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 1/7 2024-02-16T22:09:04.926 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2/7 2024-02-16T22:09:04.927 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 3/7 2024-02-16T22:09:04.927 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-02-16T22:09:04.927 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-02-16T22:09:04.927 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-02-16T22:09:04.932 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install python3-rbd 2024-02-16T22:09:05.245 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-02-16T22:09:05.245 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:05.245 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-02-16T22:09:05.245 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 2024-02-16T22:09:05.245 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2024-02-16T22:09:05.245 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 2024-02-16T22:09:05.245 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-02-16T22:09:05.245 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-02-16T22:09:05.245 INFO:teuthology.orchestra.run.smithi114.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-16T22:09:05.246 INFO:teuthology.orchestra.run.smithi114.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-16T22:09:05.246 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:05.246 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-02-16T22:09:05.463 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-16T22:09:05.502 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:28 ago on Fri 16 Feb 2024 10:07:37 PM UTC. 2024-02-16T22:09:05.614 INFO:teuthology.orchestra.run.smithi123.stdout:Package python3-rbd-2:18.2.1-0.el9.x86_64 is already installed. 2024-02-16T22:09:05.665 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-02-16T22:09:05.666 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-02-16T22:09:05.666 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-02-16T22:09:05.731 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install rbd-fuse 2024-02-16T22:09:06.031 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:56 ago on Fri 16 Feb 2024 10:08:10 PM UTC. 2024-02-16T22:09:06.195 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-02-16T22:09:06.196 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:09:06.196 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-02-16T22:09:06.197 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:09:06.197 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-02-16T22:09:06.197 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-0.el9 ceph-noarch 7.4 M 2024-02-16T22:09:06.197 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-02-16T22:09:06.197 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-16T22:09:06.197 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-16T22:09:06.197 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-16T22:09:06.197 INFO:teuthology.orchestra.run.smithi114.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-16T22:09:06.197 INFO:teuthology.orchestra.run.smithi114.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-16T22:09:06.197 INFO:teuthology.orchestra.run.smithi114.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-16T22:09:06.197 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-16T22:09:06.197 INFO:teuthology.orchestra.run.smithi114.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-16T22:09:06.197 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-16T22:09:06.197 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-16T22:09:06.197 INFO:teuthology.orchestra.run.smithi114.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-16T22:09:06.198 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:06.198 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-02-16T22:09:06.198 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:09:06.198 INFO:teuthology.orchestra.run.smithi114.stdout:Install 12 Packages 2024-02-16T22:09:06.198 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:06.198 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 37 M 2024-02-16T22:09:06.198 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 209 M 2024-02-16T22:09:06.199 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-02-16T22:09:06.305 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:29 ago on Fri 16 Feb 2024 10:07:37 PM UTC. 2024-02-16T22:09:06.472 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-02-16T22:09:06.473 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:09:06.473 INFO:teuthology.orchestra.run.smithi123.stdout: Package Architecture Version Repository Size 2024-02-16T22:09:06.473 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:09:06.473 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-02-16T22:09:06.473 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse x86_64 2:18.2.1-0.el9 ceph 86 k 2024-02-16T22:09:06.473 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:09:06.473 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-02-16T22:09:06.474 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:09:06.474 INFO:teuthology.orchestra.run.smithi123.stdout:Install 1 Package 2024-02-16T22:09:06.474 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:09:06.474 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 86 k 2024-02-16T22:09:06.474 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 226 k 2024-02-16T22:09:06.474 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-02-16T22:09:06.489 INFO:teuthology.orchestra.run.smithi114.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 242 kB/s | 32 kB 00:00 2024-02-16T22:09:06.543 INFO:teuthology.orchestra.run.smithi123.stdout:rbd-fuse-18.2.1-0.el9.x86_64.rpm 1.2 MB/s | 86 kB 00:00 2024-02-16T22:09:06.544 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:09:06.544 INFO:teuthology.orchestra.run.smithi123.stdout:Total 1.2 MB/s | 86 kB 00:00 2024-02-16T22:09:06.544 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-02-16T22:09:06.551 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-02-16T22:09:06.551 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-02-16T22:09:06.581 INFO:teuthology.orchestra.run.smithi114.stdout:(2/12): ceph-mgr-diskprediction-local-18.2.1-0. 33 MB/s | 7.4 MB 00:00 2024-02-16T22:09:06.607 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-02-16T22:09:06.607 INFO:teuthology.orchestra.run.smithi114.stdout:(3/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 147 kB/s | 17 kB 00:00 2024-02-16T22:09:06.607 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-02-16T22:09:06.632 INFO:teuthology.orchestra.run.smithi114.stdout:(4/12): openblas-0.3.21-2.el9.x86_64.rpm 686 kB/s | 34 kB 00:00 2024-02-16T22:09:06.740 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-02-16T22:09:06.808 INFO:teuthology.orchestra.run.smithi114.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 24 MB/s | 4.7 MB 00:00 2024-02-16T22:09:06.832 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:09:06.875 INFO:teuthology.orchestra.run.smithi114.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 4.8 MB 00:00 2024-02-16T22:09:06.899 INFO:teuthology.orchestra.run.smithi114.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.4 MB/s | 217 kB 00:00 2024-02-16T22:09:07.108 INFO:teuthology.orchestra.run.smithi114.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 4.0 MB/s | 3.0 MB 00:00 2024-02-16T22:09:07.209 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:09:07.267 INFO:teuthology.orchestra.run.smithi114.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 40 MB/s | 16 MB 00:00 2024-02-16T22:09:07.292 INFO:teuthology.orchestra.run.smithi114.stdout:(10/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.0 MB/s | 802 kB 00:00 2024-02-16T22:09:07.317 INFO:teuthology.orchestra.run.smithi114.stdout:(11/12): libquadmath-11.4.1-3.el9.x86_64.rpm 939 kB/s | 194 kB 00:00 2024-02-16T22:09:07.506 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:09:07.506 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:09:07.506 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-02-16T22:09:07.506 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse-2:18.2.1-0.el9.x86_64 2024-02-16T22:09:07.506 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:09:07.506 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-02-16T22:09:07.534 INFO:teuthology.orchestra.run.smithi114.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 922 kB/s | 245 kB 00:00 2024-02-16T22:09:07.536 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:09:07.537 INFO:teuthology.orchestra.run.smithi114.stdout:Total 28 MB/s | 37 MB 00:01 2024-02-16T22:09:07.557 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-02-16T22:09:07.580 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-02-16T22:09:07.580 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-02-16T22:09:07.781 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install rbd-mirror 2024-02-16T22:09:08.006 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-02-16T22:09:08.006 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-02-16T22:09:08.359 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:31 ago on Fri 16 Feb 2024 10:07:37 PM UTC. 2024-02-16T22:09:08.529 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-02-16T22:09:08.530 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:09:08.530 INFO:teuthology.orchestra.run.smithi123.stdout: Package Architecture Version Repository Size 2024-02-16T22:09:08.530 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:09:08.530 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-02-16T22:09:08.530 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-02-16T22:09:08.530 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:09:08.530 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-02-16T22:09:08.530 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:09:08.530 INFO:teuthology.orchestra.run.smithi123.stdout:Install 1 Package 2024-02-16T22:09:08.530 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:09:08.531 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 3.0 M 2024-02-16T22:09:08.531 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 12 M 2024-02-16T22:09:08.531 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-02-16T22:09:08.637 INFO:teuthology.orchestra.run.smithi123.stdout:rbd-mirror-18.2.1-0.el9.x86_64.rpm 28 MB/s | 3.0 MB 00:00 2024-02-16T22:09:08.637 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:09:08.638 INFO:teuthology.orchestra.run.smithi123.stdout:Total 28 MB/s | 3.0 MB 00:00 2024-02-16T22:09:08.638 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-02-16T22:09:08.647 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-02-16T22:09:08.647 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-02-16T22:09:08.704 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-02-16T22:09:08.717 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-02-16T22:09:08.718 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-02-16T22:09:08.773 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-02-16T22:09:08.805 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-02-16T22:09:08.896 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-02-16T22:09:08.933 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-02-16T22:09:09.110 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-16T22:09:09.135 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-02-16T22:09:09.140 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-16T22:09:09.196 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:09:09.220 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:09:09.220 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T22:09:09.220 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-16T22:09:09.221 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-16T22:09:09.221 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-16T22:09:09.221 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:09:09.224 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-02-16T22:09:09.871 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-02-16T22:09:09.985 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-02-16T22:09:10.401 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:09:10.401 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:09:10.401 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-02-16T22:09:10.401 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror-2:18.2.1-0.el9.x86_64 2024-02-16T22:09:10.401 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:09:10.401 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-02-16T22:09:10.656 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install rbd-nbd 2024-02-16T22:09:11.203 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-02-16T22:09:11.248 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:34 ago on Fri 16 Feb 2024 10:07:37 PM UTC. 2024-02-16T22:09:11.416 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-02-16T22:09:11.417 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:09:11.417 INFO:teuthology.orchestra.run.smithi123.stdout: Package Architecture Version Repository Size 2024-02-16T22:09:11.417 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:09:11.417 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-02-16T22:09:11.418 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd x86_64 2:18.2.1-0.el9 ceph 171 k 2024-02-16T22:09:11.418 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:09:11.418 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-02-16T22:09:11.418 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-02-16T22:09:11.418 INFO:teuthology.orchestra.run.smithi123.stdout:Install 1 Package 2024-02-16T22:09:11.418 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:09:11.418 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 171 k 2024-02-16T22:09:11.418 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 494 k 2024-02-16T22:09:11.418 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-02-16T22:09:11.481 INFO:teuthology.orchestra.run.smithi123.stdout:rbd-nbd-18.2.1-0.el9.x86_64.rpm 2.6 MB/s | 171 kB 00:00 2024-02-16T22:09:11.482 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:09:11.482 INFO:teuthology.orchestra.run.smithi123.stdout:Total 2.6 MB/s | 171 kB 00:00 2024-02-16T22:09:11.482 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-02-16T22:09:11.490 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-02-16T22:09:11.490 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-02-16T22:09:11.545 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-02-16T22:09:11.545 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-02-16T22:09:11.691 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-02-16T22:09:11.779 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:09:12.225 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:09:12.228 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-02-16T22:09:12.256 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-02-16T22:09:12.506 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:09:12.506 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:09:12.506 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-02-16T22:09:12.506 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd-2:18.2.1-0.el9.x86_64 2024-02-16T22:09:12.506 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:09:12.506 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-02-16T22:09:12.761 DEBUG:teuthology.parallel:result is None 2024-02-16T22:09:15.433 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-02-16T22:09:15.433 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 1/12 2024-02-16T22:09:15.433 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-02-16T22:09:15.433 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-02-16T22:09:15.433 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-02-16T22:09:15.433 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-16T22:09:15.434 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-16T22:09:15.434 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-02-16T22:09:15.434 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-02-16T22:09:15.434 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-02-16T22:09:15.434 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-02-16T22:09:15.434 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-02-16T22:09:15.896 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-02-16T22:09:15.896 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:15.896 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-02-16T22:09:15.896 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarch 2024-02-16T22:09:15.896 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-16T22:09:15.896 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-16T22:09:15.896 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-16T22:09:15.896 INFO:teuthology.orchestra.run.smithi114.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-16T22:09:15.897 INFO:teuthology.orchestra.run.smithi114.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-16T22:09:15.897 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-16T22:09:15.897 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-16T22:09:15.897 INFO:teuthology.orchestra.run.smithi114.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-16T22:09:15.897 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-16T22:09:15.897 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-16T22:09:15.897 INFO:teuthology.orchestra.run.smithi114.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-16T22:09:15.897 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:15.897 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-02-16T22:09:16.271 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-mgr-rook 2024-02-16T22:09:16.854 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:06 ago on Fri 16 Feb 2024 10:08:10 PM UTC. 2024-02-16T22:09:17.019 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-02-16T22:09:17.020 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:09:17.020 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-02-16T22:09:17.020 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:09:17.020 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-02-16T22:09:17.020 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-rook noarch 2:18.2.1-0.el9 ceph-noarch 50 k 2024-02-16T22:09:17.020 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-02-16T22:09:17.021 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-16T22:09:17.021 INFO:teuthology.orchestra.run.smithi114.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-16T22:09:17.021 INFO:teuthology.orchestra.run.smithi114.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-16T22:09:17.021 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-16T22:09:17.021 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-16T22:09:17.021 INFO:teuthology.orchestra.run.smithi114.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-16T22:09:17.021 INFO:teuthology.orchestra.run.smithi114.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-16T22:09:17.021 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-16T22:09:17.021 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-16T22:09:17.021 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-16T22:09:17.021 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-16T22:09:17.021 INFO:teuthology.orchestra.run.smithi114.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-16T22:09:17.021 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:17.022 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-02-16T22:09:17.022 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:09:17.022 INFO:teuthology.orchestra.run.smithi114.stdout:Install 13 Packages 2024-02-16T22:09:17.022 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:17.022 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 2.0 M 2024-02-16T22:09:17.022 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 26 M 2024-02-16T22:09:17.022 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-02-16T22:09:17.467 INFO:teuthology.orchestra.run.smithi114.stdout:(1/13): ceph-mgr-rook-18.2.1-0.el9.noarch.rpm 471 kB/s | 50 kB 00:00 2024-02-16T22:09:17.492 INFO:teuthology.orchestra.run.smithi114.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 185 kB/s | 24 kB 00:00 2024-02-16T22:09:17.534 INFO:teuthology.orchestra.run.smithi114.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 104 kB/s | 18 kB 00:00 2024-02-16T22:09:17.551 INFO:teuthology.orchestra.run.smithi114.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 174 kB 00:00 2024-02-16T22:09:17.567 INFO:teuthology.orchestra.run.smithi114.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-02-16T22:09:17.593 INFO:teuthology.orchestra.run.smithi114.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.5 MB/s | 211 kB 00:00 2024-02-16T22:09:17.618 INFO:teuthology.orchestra.run.smithi114.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 855 kB/s | 43 kB 00:00 2024-02-16T22:09:17.710 INFO:teuthology.orchestra.run.smithi114.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 128 kB/s | 15 kB 00:00 2024-02-16T22:09:17.877 INFO:teuthology.orchestra.run.smithi114.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 104 kB/s | 32 kB 00:00 2024-02-16T22:09:18.244 INFO:teuthology.orchestra.run.smithi114.stdout:(10/13): python3-google-auth-2.27.0-1.el9.noarc 324 kB/s | 203 kB 00:00 2024-02-16T22:09:18.394 INFO:teuthology.orchestra.run.smithi114.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 114 kB/s | 59 kB 00:00 2024-02-16T22:09:18.528 INFO:teuthology.orchestra.run.smithi114.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 316 kB/s | 90 kB 00:00 2024-02-16T22:09:25.220 INFO:teuthology.orchestra.run.smithi114.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 139 kB/s | 1.0 MB 00:07 2024-02-16T22:09:25.222 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:09:25.222 INFO:teuthology.orchestra.run.smithi114.stdout:Total 256 kB/s | 2.0 MB 00:08 2024-02-16T22:09:25.222 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-02-16T22:09:25.237 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-02-16T22:09:25.237 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-02-16T22:09:25.344 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-02-16T22:09:25.345 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-02-16T22:09:25.905 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-02-16T22:09:26.075 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-02-16T22:09:26.128 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-02-16T22:09:26.183 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-02-16T22:09:26.221 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-02-16T22:09:26.262 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-02-16T22:09:26.356 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-02-16T22:09:26.463 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 7/13 2024-02-16T22:09:26.513 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-02-16T22:09:27.228 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-02-16T22:09:27.277 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-02-16T22:09:27.319 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-02-16T22:09:27.366 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-02-16T22:09:27.389 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-02-16T22:09:28.206 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-02-16T22:09:28.206 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-rook-2:18.2.1-0.el9.noarch 1/13 2024-02-16T22:09:28.206 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-02-16T22:09:28.206 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-02-16T22:09:28.207 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-02-16T22:09:28.207 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-02-16T22:09:28.207 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-02-16T22:09:28.207 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-02-16T22:09:28.207 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-02-16T22:09:28.207 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-02-16T22:09:28.207 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 10/13 2024-02-16T22:09:28.207 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-02-16T22:09:28.207 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-02-16T22:09:28.567 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-02-16T22:09:28.567 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:28.567 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-02-16T22:09:28.567 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-rook-2:18.2.1-0.el9.noarch 2024-02-16T22:09:28.567 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-16T22:09:28.567 INFO:teuthology.orchestra.run.smithi114.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-16T22:09:28.567 INFO:teuthology.orchestra.run.smithi114.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-16T22:09:28.567 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-16T22:09:28.568 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-16T22:09:28.568 INFO:teuthology.orchestra.run.smithi114.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-16T22:09:28.568 INFO:teuthology.orchestra.run.smithi114.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-16T22:09:28.568 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-16T22:09:28.568 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-16T22:09:28.568 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-16T22:09:28.568 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-16T22:09:28.568 INFO:teuthology.orchestra.run.smithi114.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-16T22:09:28.568 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:28.568 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-02-16T22:09:28.837 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-mgr-cephadm 2024-02-16T22:09:29.419 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:19 ago on Fri 16 Feb 2024 10:08:10 PM UTC. 2024-02-16T22:09:29.584 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-02-16T22:09:29.585 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:09:29.585 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-02-16T22:09:29.585 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:09:29.585 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-02-16T22:09:29.585 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-cephadm noarch 2:18.2.1-0.el9 ceph-noarch 132 k 2024-02-16T22:09:29.585 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-02-16T22:09:29.585 INFO:teuthology.orchestra.run.smithi114.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-02-16T22:09:29.586 INFO:teuthology.orchestra.run.smithi114.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-16T22:09:29.586 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-16T22:09:29.586 INFO:teuthology.orchestra.run.smithi114.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-16T22:09:29.586 INFO:teuthology.orchestra.run.smithi114.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-16T22:09:29.586 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:29.586 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-02-16T22:09:29.586 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:09:29.586 INFO:teuthology.orchestra.run.smithi114.stdout:Install 6 Packages 2024-02-16T22:09:29.586 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:29.587 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 6.8 M 2024-02-16T22:09:29.587 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 33 M 2024-02-16T22:09:29.587 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-02-16T22:09:29.862 INFO:teuthology.orchestra.run.smithi114.stdout:(1/6): ceph-mgr-cephadm-18.2.1-0.el9.noarch.rpm 1.1 MB/s | 132 kB 00:00 2024-02-16T22:09:29.929 INFO:teuthology.orchestra.run.smithi114.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.2 MB/s | 227 kB 00:00 2024-02-16T22:09:30.171 INFO:teuthology.orchestra.run.smithi114.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 5.8 MB 00:00 2024-02-16T22:09:31.162 INFO:teuthology.orchestra.run.smithi114.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 47 kB/s | 58 kB 00:01 2024-02-16T22:09:31.329 INFO:teuthology.orchestra.run.smithi114.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 46 kB/s | 51 kB 00:01 2024-02-16T22:09:31.838 INFO:teuthology.orchestra.run.smithi114.stdout:(6/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 277 kB/s | 548 kB 00:01 2024-02-16T22:09:31.843 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:09:31.843 INFO:teuthology.orchestra.run.smithi114.stdout:Total 3.0 MB/s | 6.8 MB 00:02 2024-02-16T22:09:31.843 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-02-16T22:09:31.853 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-02-16T22:09:31.854 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-02-16T22:09:31.947 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-02-16T22:09:31.947 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-02-16T22:09:32.250 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-02-16T22:09:32.411 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-02-16T22:09:32.456 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2024-02-16T22:09:32.971 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-02-16T22:09:33.057 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-02-16T22:09:33.118 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-02-16T22:09:33.143 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-02-16T22:09:33.955 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-02-16T22:09:33.955 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 1/6 2024-02-16T22:09:33.955 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-02-16T22:09:33.955 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-02-16T22:09:33.955 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2024-02-16T22:09:33.955 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-02-16T22:09:34.256 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-02-16T22:09:34.256 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:34.256 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-02-16T22:09:34.256 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 2024-02-16T22:09:34.256 INFO:teuthology.orchestra.run.smithi114.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-02-16T22:09:34.256 INFO:teuthology.orchestra.run.smithi114.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-16T22:09:34.257 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-16T22:09:34.257 INFO:teuthology.orchestra.run.smithi114.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-16T22:09:34.257 INFO:teuthology.orchestra.run.smithi114.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-16T22:09:34.257 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:34.257 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-02-16T22:09:34.518 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-fuse 2024-02-16T22:09:35.096 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:25 ago on Fri 16 Feb 2024 10:08:10 PM UTC. 2024-02-16T22:09:35.260 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-02-16T22:09:35.261 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:09:35.261 INFO:teuthology.orchestra.run.smithi114.stdout: Package Architecture Version Repository Size 2024-02-16T22:09:35.261 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:09:35.261 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-02-16T22:09:35.261 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-fuse x86_64 2:18.2.1-0.el9 ceph 839 k 2024-02-16T22:09:35.262 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-02-16T22:09:35.262 INFO:teuthology.orchestra.run.smithi114.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-16T22:09:35.262 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:35.262 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-02-16T22:09:35.262 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:09:35.262 INFO:teuthology.orchestra.run.smithi114.stdout:Install 2 Packages 2024-02-16T22:09:35.262 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:35.262 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 918 k 2024-02-16T22:09:35.262 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 2.7 M 2024-02-16T22:09:35.262 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-02-16T22:09:35.365 INFO:teuthology.orchestra.run.smithi114.stdout:(1/2): ceph-fuse-18.2.1-0.el9.x86_64.rpm 8.0 MB/s | 839 kB 00:00 2024-02-16T22:09:35.490 INFO:teuthology.orchestra.run.smithi114.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 345 kB/s | 79 kB 00:00 2024-02-16T22:09:35.491 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:09:35.492 INFO:teuthology.orchestra.run.smithi114.stdout:Total 3.9 MB/s | 918 kB 00:00 2024-02-16T22:09:35.492 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-02-16T22:09:35.511 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-02-16T22:09:35.511 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-02-16T22:09:35.587 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-02-16T22:09:35.588 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-02-16T22:09:35.773 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-02-16T22:09:35.895 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-02-16T22:09:35.965 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-02-16T22:09:36.701 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-02-16T22:09:36.702 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-fuse-2:18.2.1-0.el9.x86_64 1/2 2024-02-16T22:09:37.014 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-02-16T22:09:37.014 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:37.014 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-02-16T22:09:37.015 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-fuse-2:18.2.1-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-02-16T22:09:37.015 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:37.015 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-02-16T22:09:37.277 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install librados-devel 2024-02-16T22:09:37.872 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:27 ago on Fri 16 Feb 2024 10:08:10 PM UTC. 2024-02-16T22:09:38.041 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-02-16T22:09:38.042 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:09:38.042 INFO:teuthology.orchestra.run.smithi114.stdout: Package Architecture Version Repository Size 2024-02-16T22:09:38.042 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:09:38.042 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-02-16T22:09:38.042 INFO:teuthology.orchestra.run.smithi114.stdout: librados-devel x86_64 2:18.2.1-0.el9 ceph 127 k 2024-02-16T22:09:38.042 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:38.043 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-02-16T22:09:38.043 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:09:38.043 INFO:teuthology.orchestra.run.smithi114.stdout:Install 1 Package 2024-02-16T22:09:38.043 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:38.043 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 127 k 2024-02-16T22:09:38.043 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 456 k 2024-02-16T22:09:38.043 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-02-16T22:09:38.103 INFO:teuthology.orchestra.run.smithi114.stdout:librados-devel-18.2.1-0.el9.x86_64.rpm 2.1 MB/s | 127 kB 00:00 2024-02-16T22:09:38.103 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:09:38.104 INFO:teuthology.orchestra.run.smithi114.stdout:Total 2.0 MB/s | 127 kB 00:00 2024-02-16T22:09:38.104 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-02-16T22:09:38.109 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-02-16T22:09:38.110 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-02-16T22:09:38.165 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-02-16T22:09:38.165 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-02-16T22:09:38.312 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-02-16T22:09:38.417 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:09:38.893 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:09:39.230 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:09:39.231 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:39.231 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-02-16T22:09:39.231 INFO:teuthology.orchestra.run.smithi114.stdout: librados-devel-2:18.2.1-0.el9.x86_64 2024-02-16T22:09:39.231 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:39.231 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-02-16T22:09:39.500 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install libcephfs2 2024-02-16T22:09:40.074 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:30 ago on Fri 16 Feb 2024 10:08:10 PM UTC. 2024-02-16T22:09:40.192 INFO:teuthology.orchestra.run.smithi114.stdout:Package libcephfs2-2:18.2.1-0.el9.x86_64 is already installed. 2024-02-16T22:09:40.244 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-02-16T22:09:40.245 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-02-16T22:09:40.245 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-02-16T22:09:40.322 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install libcephfs-devel 2024-02-16T22:09:40.900 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:30 ago on Fri 16 Feb 2024 10:08:10 PM UTC. 2024-02-16T22:09:41.069 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-02-16T22:09:41.070 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:09:41.070 INFO:teuthology.orchestra.run.smithi114.stdout: Package Architecture Version Repository Size 2024-02-16T22:09:41.070 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:09:41.070 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-02-16T22:09:41.070 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs-devel x86_64 2:18.2.1-0.el9 ceph 31 k 2024-02-16T22:09:41.071 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:41.071 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-02-16T22:09:41.071 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:09:41.071 INFO:teuthology.orchestra.run.smithi114.stdout:Install 1 Package 2024-02-16T22:09:41.071 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:41.071 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 31 k 2024-02-16T22:09:41.071 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 139 k 2024-02-16T22:09:41.071 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-02-16T22:09:41.148 INFO:teuthology.orchestra.run.smithi114.stdout:libcephfs-devel-18.2.1-0.el9.x86_64.rpm 403 kB/s | 31 kB 00:00 2024-02-16T22:09:41.149 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:09:41.149 INFO:teuthology.orchestra.run.smithi114.stdout:Total 395 kB/s | 31 kB 00:00 2024-02-16T22:09:41.149 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-02-16T22:09:41.152 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-02-16T22:09:41.152 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-02-16T22:09:41.184 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-02-16T22:09:41.184 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-02-16T22:09:41.269 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-02-16T22:09:41.388 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:09:41.844 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:09:42.106 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:09:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:42.107 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-02-16T22:09:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs-devel-2:18.2.1-0.el9.x86_64 2024-02-16T22:09:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:42.107 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-02-16T22:09:42.287 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install librados2 2024-02-16T22:09:42.868 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:32 ago on Fri 16 Feb 2024 10:08:10 PM UTC. 2024-02-16T22:09:42.994 INFO:teuthology.orchestra.run.smithi114.stdout:Package librados2-2:18.2.1-0.el9.x86_64 is already installed. 2024-02-16T22:09:43.049 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-02-16T22:09:43.050 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-02-16T22:09:43.050 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-02-16T22:09:43.117 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install librbd1 2024-02-16T22:09:43.712 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:33 ago on Fri 16 Feb 2024 10:08:10 PM UTC. 2024-02-16T22:09:43.827 INFO:teuthology.orchestra.run.smithi114.stdout:Package librbd1-2:18.2.1-0.el9.x86_64 is already installed. 2024-02-16T22:09:43.879 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-02-16T22:09:43.880 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-02-16T22:09:43.881 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-02-16T22:09:43.952 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install python3-rados 2024-02-16T22:09:44.525 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:34 ago on Fri 16 Feb 2024 10:08:10 PM UTC. 2024-02-16T22:09:44.637 INFO:teuthology.orchestra.run.smithi114.stdout:Package python3-rados-2:18.2.1-0.el9.x86_64 is already installed. 2024-02-16T22:09:44.689 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-02-16T22:09:44.690 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-02-16T22:09:44.690 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-02-16T22:09:44.751 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install python3-rgw 2024-02-16T22:09:45.319 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:35 ago on Fri 16 Feb 2024 10:08:10 PM UTC. 2024-02-16T22:09:45.432 INFO:teuthology.orchestra.run.smithi114.stdout:Package python3-rgw-2:18.2.1-0.el9.x86_64 is already installed. 2024-02-16T22:09:45.483 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-02-16T22:09:45.484 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-02-16T22:09:45.484 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-02-16T22:09:45.546 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install python3-cephfs 2024-02-16T22:09:46.118 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:36 ago on Fri 16 Feb 2024 10:08:10 PM UTC. 2024-02-16T22:09:46.231 INFO:teuthology.orchestra.run.smithi114.stdout:Package python3-cephfs-2:18.2.1-0.el9.x86_64 is already installed. 2024-02-16T22:09:46.282 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-02-16T22:09:46.283 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-02-16T22:09:46.284 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-02-16T22:09:46.349 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install python3-rbd 2024-02-16T22:09:46.915 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:36 ago on Fri 16 Feb 2024 10:08:10 PM UTC. 2024-02-16T22:09:47.027 INFO:teuthology.orchestra.run.smithi114.stdout:Package python3-rbd-2:18.2.1-0.el9.x86_64 is already installed. 2024-02-16T22:09:47.079 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-02-16T22:09:47.080 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-02-16T22:09:47.080 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-02-16T22:09:47.138 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install rbd-fuse 2024-02-16T22:09:47.711 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:37 ago on Fri 16 Feb 2024 10:08:10 PM UTC. 2024-02-16T22:09:47.876 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-02-16T22:09:47.877 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:09:47.877 INFO:teuthology.orchestra.run.smithi114.stdout: Package Architecture Version Repository Size 2024-02-16T22:09:47.877 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:09:47.877 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-02-16T22:09:47.878 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-fuse x86_64 2:18.2.1-0.el9 ceph 86 k 2024-02-16T22:09:47.878 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:47.878 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-02-16T22:09:47.878 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:09:47.878 INFO:teuthology.orchestra.run.smithi114.stdout:Install 1 Package 2024-02-16T22:09:47.878 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:47.878 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 86 k 2024-02-16T22:09:47.878 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 226 k 2024-02-16T22:09:47.878 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-02-16T22:09:47.951 INFO:teuthology.orchestra.run.smithi114.stdout:rbd-fuse-18.2.1-0.el9.x86_64.rpm 1.2 MB/s | 86 kB 00:00 2024-02-16T22:09:47.952 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:09:47.952 INFO:teuthology.orchestra.run.smithi114.stdout:Total 1.1 MB/s | 86 kB 00:00 2024-02-16T22:09:47.952 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-02-16T22:09:47.959 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-02-16T22:09:47.959 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-02-16T22:09:48.013 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-02-16T22:09:48.013 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-02-16T22:09:48.152 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-02-16T22:09:48.249 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:09:48.681 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:09:48.971 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:09:48.972 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:48.972 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-02-16T22:09:48.972 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-fuse-2:18.2.1-0.el9.x86_64 2024-02-16T22:09:48.972 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:48.972 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-02-16T22:09:49.223 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install rbd-mirror 2024-02-16T22:09:49.805 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:39 ago on Fri 16 Feb 2024 10:08:10 PM UTC. 2024-02-16T22:09:49.975 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-02-16T22:09:49.976 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:09:49.976 INFO:teuthology.orchestra.run.smithi114.stdout: Package Architecture Version Repository Size 2024-02-16T22:09:49.976 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:09:49.976 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-02-16T22:09:49.976 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-mirror x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-02-16T22:09:49.976 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:49.976 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-02-16T22:09:49.976 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:09:49.976 INFO:teuthology.orchestra.run.smithi114.stdout:Install 1 Package 2024-02-16T22:09:49.976 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:49.977 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 3.0 M 2024-02-16T22:09:49.977 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 12 M 2024-02-16T22:09:49.977 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-02-16T22:09:50.071 INFO:teuthology.orchestra.run.smithi114.stdout:rbd-mirror-18.2.1-0.el9.x86_64.rpm 32 MB/s | 3.0 MB 00:00 2024-02-16T22:09:50.072 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:09:50.072 INFO:teuthology.orchestra.run.smithi114.stdout:Total 31 MB/s | 3.0 MB 00:00 2024-02-16T22:09:50.072 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-02-16T22:09:50.081 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-02-16T22:09:50.081 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-02-16T22:09:50.154 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-02-16T22:09:50.155 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-02-16T22:09:50.560 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-02-16T22:09:50.631 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:09:50.656 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:09:50.656 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T22:09:50.656 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-16T22:09:50.656 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-16T22:09:50.656 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-16T22:09:50.657 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:51.803 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:09:51.804 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:51.804 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-02-16T22:09:51.804 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-mirror-2:18.2.1-0.el9.x86_64 2024-02-16T22:09:51.804 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:51.804 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-02-16T22:09:52.050 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install rbd-nbd 2024-02-16T22:09:52.641 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:42 ago on Fri 16 Feb 2024 10:08:10 PM UTC. 2024-02-16T22:09:52.807 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-02-16T22:09:52.808 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:09:52.808 INFO:teuthology.orchestra.run.smithi114.stdout: Package Architecture Version Repository Size 2024-02-16T22:09:52.808 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:09:52.808 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-02-16T22:09:52.808 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-nbd x86_64 2:18.2.1-0.el9 ceph 171 k 2024-02-16T22:09:52.808 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:52.808 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-02-16T22:09:52.808 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-02-16T22:09:52.808 INFO:teuthology.orchestra.run.smithi114.stdout:Install 1 Package 2024-02-16T22:09:52.808 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:52.809 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 171 k 2024-02-16T22:09:52.809 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 494 k 2024-02-16T22:09:52.809 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-02-16T22:09:52.875 INFO:teuthology.orchestra.run.smithi114.stdout:rbd-nbd-18.2.1-0.el9.x86_64.rpm 2.5 MB/s | 171 kB 00:00 2024-02-16T22:09:52.875 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:09:52.876 INFO:teuthology.orchestra.run.smithi114.stdout:Total 2.5 MB/s | 171 kB 00:00 2024-02-16T22:09:52.876 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-02-16T22:09:52.883 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-02-16T22:09:52.884 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-02-16T22:09:52.939 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-02-16T22:09:52.939 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-02-16T22:09:53.079 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-02-16T22:09:53.173 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:09:53.649 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:09:53.966 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-02-16T22:09:53.966 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:53.967 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-02-16T22:09:53.967 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-nbd-2:18.2.1-0.el9.x86_64 2024-02-16T22:09:53.967 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:09:53.967 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-02-16T22:09:54.189 DEBUG:teuthology.parallel:result is None 2024-02-16T22:09:54.190 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-16T22:09:54.190 INFO:teuthology.packaging:ref: None 2024-02-16T22:09:54.190 INFO:teuthology.packaging:tag: v18.2.1 2024-02-16T22:09:54.190 INFO:teuthology.packaging:branch: None 2024-02-16T22:09:54.190 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T22:09:54.190 INFO:teuthology.packaging:Tag 'v18.2.1' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-02-16T22:09:54.191 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-02-16T22:09:54.569 DEBUG:teuthology.orchestra.run.smithi114:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-16T22:09:54.590 INFO:teuthology.orchestra.run.smithi114.stdout:18.2.1-0.el9 2024-02-16T22:09:54.591 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-0.el9 2024-02-16T22:09:54.591 INFO:teuthology.task.install:The correct ceph version 18.2.1-0 is installed. 2024-02-16T22:09:54.592 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-16T22:09:54.593 INFO:teuthology.packaging:ref: None 2024-02-16T22:09:54.593 INFO:teuthology.packaging:tag: v18.2.1 2024-02-16T22:09:54.593 INFO:teuthology.packaging:branch: None 2024-02-16T22:09:54.593 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T22:09:54.593 INFO:teuthology.packaging:Tag 'v18.2.1' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-02-16T22:09:54.593 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-02-16T22:09:54.999 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-16T22:09:55.024 INFO:teuthology.orchestra.run.smithi123.stdout:18.2.1-0.el9 2024-02-16T22:09:55.024 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-0.el9 2024-02-16T22:09:55.024 INFO:teuthology.task.install:The correct ceph version 18.2.1-0 is installed. 2024-02-16T22:09:55.026 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-16T22:09:55.026 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-02-16T22:09:55.026 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-16T22:09:55.054 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-02-16T22:09:55.054 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-16T22:09:55.096 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-16T22:09:55.096 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-02-16T22:09:55.097 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/daemon-helper 2024-02-16T22:09:55.124 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-16T22:09:55.190 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-02-16T22:09:55.190 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/daemon-helper 2024-02-16T22:09:55.221 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-16T22:09:55.292 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-16T22:09:55.292 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-02-16T22:09:55.292 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-16T22:09:55.319 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-16T22:09:55.386 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-02-16T22:09:55.386 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-16T22:09:55.413 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-16T22:09:55.480 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-16T22:09:55.480 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-02-16T22:09:55.480 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/stdin-killer 2024-02-16T22:09:55.506 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-16T22:09:55.573 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-02-16T22:09:55.574 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/stdin-killer 2024-02-16T22:09:55.601 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-16T22:09:55.669 INFO:teuthology.run_tasks:Running task print... 2024-02-16T22:09:55.678 INFO:teuthology.task.print:**** done install task... 2024-02-16T22:09:55.678 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-16T22:09:55.785 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}}, 'image': 'quay.io/ceph/ceph:v18.2.1', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275'} 2024-02-16T22:09:55.785 INFO:tasks.cephadm:Cluster image is quay.io/ceph/ceph:v18.2.1 2024-02-16T22:09:55.786 INFO:tasks.cephadm:Cluster fsid is 1dcbaabe-cd18-11ee-95ba-87774f69a715 2024-02-16T22:09:55.786 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-16T22:09:55.786 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-02-16T22:09:55.786 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi114': '172.21.15.114', 'mon.smithi123': '172.21.15.123'} 2024-02-16T22:09:55.786 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-16T22:09:55.786 DEBUG:teuthology.orchestra.run.smithi114:> sudo hostname $(hostname -s) 2024-02-16T22:09:55.813 DEBUG:teuthology.orchestra.run.smithi123:> sudo hostname $(hostname -s) 2024-02-16T22:09:55.841 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for reef 2024-02-16T22:09:55.841 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T22:09:56.144 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/76790/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1450-gf78a58c0', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1450.gf78a58c0'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240215.160715/f78a58c0ffd401d1493058a1022c35f011d65275/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-02-15 19:25:41.455235', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20240215.160715', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-batrick-testing-20240215.160715/f78a58c0ffd401d1493058a1022c35f011d65275/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/76792/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1450-gf78a58c0', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1450.gf78a58c0'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240215.160715/f78a58c0ffd401d1493058a1022c35f011d65275/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-02-15 19:17:06.436952', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20240215.160715', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-batrick-testing-20240215.160715/f78a58c0ffd401d1493058a1022c35f011d65275/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-02-16T22:09:56.612 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref reef, sha1 744c573dfc29e50959567861c524f9e6c038171f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&ref=reef 2024-02-16T22:09:56.614 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/reef/744c573dfc29e50959567861c524f9e6c038171f/centos/9/x86_64/flavors/default/cephadm 2024-02-16T22:09:56.614 DEBUG:teuthology.orchestra.run.smithi114:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/reef/744c573dfc29e50959567861c524f9e6c038171f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-16T22:09:57.020 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-r--r--. 1 ubuntu ubuntu 214739 Feb 16 22:09 /home/ubuntu/cephtest/cephadm 2024-02-16T22:09:57.021 DEBUG:teuthology.orchestra.run.smithi123:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/reef/744c573dfc29e50959567861c524f9e6c038171f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-16T22:09:57.848 INFO:teuthology.orchestra.run.smithi123.stdout:-rw-r--r--. 1 ubuntu ubuntu 214739 Feb 16 22:09 /home/ubuntu/cephtest/cephadm 2024-02-16T22:09:57.848 DEBUG:teuthology.orchestra.run.smithi114:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-16T22:09:57.867 DEBUG:teuthology.orchestra.run.smithi123:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-16T22:09:57.898 INFO:tasks.cephadm:Pulling image quay.io/ceph/ceph:v18.2.1 on all hosts... 2024-02-16T22:09:57.898 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 pull 2024-02-16T22:09:57.910 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 pull 2024-02-16T22:09:58.659 INFO:teuthology.orchestra.run.smithi123.stderr:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-02-16T22:09:58.744 INFO:teuthology.orchestra.run.smithi114.stderr:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-02-16T22:10:21.146 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:10:21.147 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-02-16T22:10:21.147 INFO:teuthology.orchestra.run.smithi114.stdout: "image_id": "94eadd990e0a5619d90a7220f97c70d1fe3558d9b0fb9e0c1935661dd49a6d5f", 2024-02-16T22:10:21.147 INFO:teuthology.orchestra.run.smithi114.stdout: "repo_digests": [ 2024-02-16T22:10:21.147 INFO:teuthology.orchestra.run.smithi114.stdout: "quay.io/ceph/ceph@sha256:6176526d5ef4047f148d47c951653dcc6b0de49091fda006d3a088e7a1f404d0", 2024-02-16T22:10:21.147 INFO:teuthology.orchestra.run.smithi114.stdout: "quay.io/ceph/ceph@sha256:e540726c37a4335a713579a0c9bf4413cb31ad156d22ea1ec00ed9bbc4317f66" 2024-02-16T22:10:21.147 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-02-16T22:10:21.147 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:10:40.943 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-02-16T22:10:40.944 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-02-16T22:10:40.944 INFO:teuthology.orchestra.run.smithi123.stdout: "image_id": "94eadd990e0a5619d90a7220f97c70d1fe3558d9b0fb9e0c1935661dd49a6d5f", 2024-02-16T22:10:40.944 INFO:teuthology.orchestra.run.smithi123.stdout: "repo_digests": [ 2024-02-16T22:10:40.944 INFO:teuthology.orchestra.run.smithi123.stdout: "quay.io/ceph/ceph@sha256:6176526d5ef4047f148d47c951653dcc6b0de49091fda006d3a088e7a1f404d0", 2024-02-16T22:10:40.944 INFO:teuthology.orchestra.run.smithi123.stdout: "quay.io/ceph/ceph@sha256:e540726c37a4335a713579a0c9bf4413cb31ad156d22ea1ec00ed9bbc4317f66" 2024-02-16T22:10:40.944 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-02-16T22:10:40.944 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-02-16T22:10:40.961 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /etc/ceph 2024-02-16T22:10:40.996 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph 2024-02-16T22:10:41.025 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 777 /etc/ceph 2024-02-16T22:10:41.066 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 777 /etc/ceph 2024-02-16T22:10:41.092 INFO:tasks.cephadm:Writing seed config... 2024-02-16T22:10:41.093 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-02-16T22:10:41.093 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-02-16T22:10:41.094 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-02-16T22:10:41.094 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-02-16T22:10:41.094 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-02-16T22:10:41.094 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-02-16T22:10:41.094 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-02-16T22:10:41.094 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-02-16T22:10:41.094 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-02-16T22:10:41.094 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-16T22:10:41.094 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-16T22:10:41.094 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-02-16T22:10:41.094 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-02-16T22:10:41.094 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-02-16T22:10:41.094 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-02-16T22:10:41.094 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-02-16T22:10:41.095 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-02-16T22:10:41.095 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-02-16T22:10:41.095 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-02-16T22:10:41.095 INFO:tasks.cephadm: override: [client] debug client = 20 2024-02-16T22:10:41.095 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-16T22:10:41.095 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-02-16T22:10:41.095 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-02-16T22:10:41.095 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-02-16T22:10:41.095 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-02-16T22:10:41.095 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-02-16T22:10:41.095 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-02-16T22:10:41.095 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-02-16T22:10:41.095 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-02-16T22:10:41.095 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-02-16T22:10:41.095 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-02-16T22:10:41.095 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-02-16T22:10:41.095 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-02-16T22:10:41.096 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-02-16T22:10:41.096 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-16T22:10:41.096 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-16T22:10:41.096 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-16T22:10:41.096 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-16T22:10:41.096 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-16T22:10:41.096 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-02-16T22:10:41.097 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-02-16T22:10:41.097 DEBUG:teuthology.orchestra.run.smithi114:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-16T22:10:41.123 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 1dcbaabe-cd18-11ee-95ba-87774f69a715 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-02-16T22:10:41.123 DEBUG:teuthology.orchestra.run.smithi114:mon.smithi114> sudo journalctl -f -n 0 -u ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@mon.smithi114.service 2024-02-16T22:10:41.165 INFO:tasks.cephadm:Bootstrapping... 2024-02-16T22:10:41.165 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 -v bootstrap --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.114 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-16T22:10:41.323 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-02-16T22:10:41.323 INFO:teuthology.orchestra.run.smithi114.stdout:cephadm ['--image', 'quay.io/ceph/ceph:v18.2.1', '-v', 'bootstrap', '--fsid', '1dcbaabe-cd18-11ee-95ba-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.114', '--skip-admin-label'] 2024-02-16T22:10:41.347 INFO:teuthology.orchestra.run.smithi114.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-16T22:10:41.347 INFO:teuthology.orchestra.run.smithi114.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-16T22:10:41.348 INFO:teuthology.orchestra.run.smithi114.stdout:Verifying podman|docker is present... 2024-02-16T22:10:41.371 INFO:teuthology.orchestra.run.smithi114.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-16T22:10:41.371 INFO:teuthology.orchestra.run.smithi114.stdout:Verifying lvm2 is present... 2024-02-16T22:10:41.371 INFO:teuthology.orchestra.run.smithi114.stdout:Verifying time synchronization is in place... 2024-02-16T22:10:41.378 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-16T22:10:41.378 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-16T22:10:41.384 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-16T22:10:41.384 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout inactive 2024-02-16T22:10:41.390 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout enabled 2024-02-16T22:10:41.396 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout active 2024-02-16T22:10:41.396 INFO:teuthology.orchestra.run.smithi114.stdout:Unit chronyd.service is enabled and running 2024-02-16T22:10:41.396 INFO:teuthology.orchestra.run.smithi114.stdout:Repeating the final host check... 2024-02-16T22:10:41.419 INFO:teuthology.orchestra.run.smithi114.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-16T22:10:41.419 INFO:teuthology.orchestra.run.smithi114.stdout:podman (/bin/podman) version 4.9.3 is present 2024-02-16T22:10:41.419 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl is present 2024-02-16T22:10:41.419 INFO:teuthology.orchestra.run.smithi114.stdout:lvcreate is present 2024-02-16T22:10:41.425 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-16T22:10:41.425 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-16T22:10:41.431 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-16T22:10:41.431 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout inactive 2024-02-16T22:10:41.437 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout enabled 2024-02-16T22:10:41.443 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout active 2024-02-16T22:10:41.443 INFO:teuthology.orchestra.run.smithi114.stdout:Unit chronyd.service is enabled and running 2024-02-16T22:10:41.443 INFO:teuthology.orchestra.run.smithi114.stdout:Host looks OK 2024-02-16T22:10:41.443 INFO:teuthology.orchestra.run.smithi114.stdout:Cluster fsid: 1dcbaabe-cd18-11ee-95ba-87774f69a715 2024-02-16T22:10:41.443 INFO:teuthology.orchestra.run.smithi114.stdout:Acquiring lock 140094810233056 on /run/cephadm/1dcbaabe-cd18-11ee-95ba-87774f69a715.lock 2024-02-16T22:10:41.443 INFO:teuthology.orchestra.run.smithi114.stdout:Lock 140094810233056 acquired on /run/cephadm/1dcbaabe-cd18-11ee-95ba-87774f69a715.lock 2024-02-16T22:10:41.443 INFO:teuthology.orchestra.run.smithi114.stdout:Verifying IP 172.21.15.114 port 3300 ... 2024-02-16T22:10:41.444 INFO:teuthology.orchestra.run.smithi114.stdout:Verifying IP 172.21.15.114 port 6789 ... 2024-02-16T22:10:41.444 INFO:teuthology.orchestra.run.smithi114.stdout:Base mon IP(s) is [172.21.15.114:3300, 172.21.15.114:6789], mon addrv is [v2:172.21.15.114:3300,v1:172.21.15.114:6789] 2024-02-16T22:10:41.447 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.114 metric 100 2024-02-16T22:10:41.447 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.114 metric 100 2024-02-16T22:10:41.449 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-16T22:10:41.449 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-02-16T22:10:41.450 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1788sec hoplimit 64 pref medium 2024-02-16T22:10:41.452 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-16T22:10:41.452 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-16T22:10:41.452 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-16T22:10:41.452 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-02-16T22:10:41.452 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:cb63/64 scope link 2024-02-16T22:10:41.452 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-16T22:10:41.453 INFO:teuthology.orchestra.run.smithi114.stdout:Mon IP `172.21.15.114` is in CIDR network `172.21.0.0/20` 2024-02-16T22:10:41.453 INFO:teuthology.orchestra.run.smithi114.stdout:Mon IP `172.21.15.114` is in CIDR network `172.21.0.0/20` 2024-02-16T22:10:41.453 INFO:teuthology.orchestra.run.smithi114.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-16T22:10:41.453 INFO:teuthology.orchestra.run.smithi114.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-16T22:10:41.454 INFO:teuthology.orchestra.run.smithi114.stdout:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-02-16T22:10:42.339 INFO:teuthology.orchestra.run.smithi114.stdout:/bin/podman: stdout 94eadd990e0a5619d90a7220f97c70d1fe3558d9b0fb9e0c1935661dd49a6d5f 2024-02-16T22:10:42.339 INFO:teuthology.orchestra.run.smithi114.stdout:/bin/podman: stderr Trying to pull quay.io/ceph/ceph:v18.2.1... 2024-02-16T22:10:42.339 INFO:teuthology.orchestra.run.smithi114.stdout:/bin/podman: stderr Getting image source signatures 2024-02-16T22:10:42.339 INFO:teuthology.orchestra.run.smithi114.stdout:/bin/podman: stderr Copying blob sha256:6345d49baed062606681bf114c10cebf2c69e90ec4d758270fae85ad0cc5c86b 2024-02-16T22:10:42.339 INFO:teuthology.orchestra.run.smithi114.stdout:/bin/podman: stderr Copying blob sha256:0f642e61a11afb42c0ccb5b05f5809250ad643ed143780d6ff49a5a4c492fc38 2024-02-16T22:10:42.339 INFO:teuthology.orchestra.run.smithi114.stdout:/bin/podman: stderr Copying config sha256:94eadd990e0a5619d90a7220f97c70d1fe3558d9b0fb9e0c1935661dd49a6d5f 2024-02-16T22:10:42.339 INFO:teuthology.orchestra.run.smithi114.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-16T22:10:47.856 INFO:teuthology.orchestra.run.smithi114.stdout:ceph: stdout ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable) 2024-02-16T22:10:47.856 INFO:teuthology.orchestra.run.smithi114.stdout:Ceph version: ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable) 2024-02-16T22:10:47.856 INFO:teuthology.orchestra.run.smithi114.stdout:Extracting ceph user uid/gid from container image... 2024-02-16T22:10:56.352 INFO:teuthology.orchestra.run.smithi114.stdout:stat: stdout 167 167 2024-02-16T22:10:56.352 INFO:teuthology.orchestra.run.smithi114.stdout:Creating initial keys... 2024-02-16T22:10:57.188 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph-authtool: stdout AQBw3c9lpsE0JhAA55uKpr0bDC7G6NLpKZssag== 2024-02-16T22:10:58.052 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph-authtool: stdout AQBx3c9lG78NHxAAJdOyhl28B8O6DWgbTTovIA== 2024-02-16T22:10:58.967 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph-authtool: stdout AQBy3c9lT8EBFRAAV537jComnGs9zeDudYAdig== 2024-02-16T22:10:58.968 INFO:teuthology.orchestra.run.smithi114.stdout:Creating initial monmap... 2024-02-16T22:10:59.856 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-16T22:10:59.856 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-02-16T22:10:59.856 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 1dcbaabe-cd18-11ee-95ba-87774f69a715 2024-02-16T22:10:59.856 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-16T22:10:59.856 INFO:teuthology.orchestra.run.smithi114.stdout:monmaptool for smithi114 [v2:172.21.15.114:3300,v1:172.21.15.114:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-16T22:10:59.857 INFO:teuthology.orchestra.run.smithi114.stdout:setting min_mon_release = pacific 2024-02-16T22:10:59.857 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/monmaptool: set fsid to 1dcbaabe-cd18-11ee-95ba-87774f69a715 2024-02-16T22:10:59.857 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-16T22:10:59.857 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:10:59.857 INFO:teuthology.orchestra.run.smithi114.stdout:Creating mon... 2024-02-16T22:11:00.763 INFO:teuthology.orchestra.run.smithi114.stdout:create mon.smithi114 on 2024-02-16T22:11:00.981 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-02-16T22:11:01.125 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-16T22:11:01.309 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715.target → /etc/systemd/system/ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715.target. 2024-02-16T22:11:01.310 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715.target → /etc/systemd/system/ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715.target. 2024-02-16T22:11:01.529 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@mon.smithi114 2024-02-16T22:11:01.529 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@mon.smithi114.service: Unit ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@mon.smithi114.service not loaded. 2024-02-16T22:11:01.690 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715.target.wants/ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@mon.smithi114.service → /etc/systemd/system/ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@.service. 2024-02-16T22:11:02.190 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:02 smithi114 podman[28839]: 2024-02-16 22:11:02.124031507 +0000 UTC m=+0.307779079 container start e494c505b2a163fe372427f2d1c651872b96143c6f4a472efb0afb9c629fc7e3 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi114, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, RELEASE=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T22:11:02.210 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-16T22:11:02.210 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout disabled 2024-02-16T22:11:02.217 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-16T22:11:02.218 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout inactive 2024-02-16T22:11:02.218 INFO:teuthology.orchestra.run.smithi114.stdout:firewalld.service is not enabled 2024-02-16T22:11:02.218 INFO:teuthology.orchestra.run.smithi114.stdout:Not possible to enable service . firewalld.service is not available 2024-02-16T22:11:02.218 INFO:teuthology.orchestra.run.smithi114.stdout:Waiting for mon to start... 2024-02-16T22:11:02.218 INFO:teuthology.orchestra.run.smithi114.stdout:Waiting for mon... 2024-02-16T22:11:02.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:02 smithi114 ceph-mon[28858]: mkfs 1dcbaabe-cd18-11ee-95ba-87774f69a715 2024-02-16T22:11:02.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:02 smithi114 ceph-mon[28858]: mon.smithi114 is new leader, mons smithi114 in quorum (ranks 0) 2024-02-16T22:11:02.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:02 smithi114 bash[28839]: e494c505b2a163fe372427f2d1c651872b96143c6f4a472efb0afb9c629fc7e3 2024-02-16T22:11:02.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:02 smithi114 systemd[1]: Started Ceph mon.smithi114 for 1dcbaabe-cd18-11ee-95ba-87774f69a715. 2024-02-16T22:11:03.279 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout cluster: 2024-02-16T22:11:03.279 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout id: 1dcbaabe-cd18-11ee-95ba-87774f69a715 2024-02-16T22:11:03.279 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-16T22:11:03.279 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-02-16T22:11:03.279 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout services: 2024-02-16T22:11:03.279 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi114 (age 0.569981s) 2024-02-16T22:11:03.279 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-16T22:11:03.279 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-16T22:11:03.279 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-02-16T22:11:03.280 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout data: 2024-02-16T22:11:03.280 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-16T22:11:03.280 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-16T22:11:03.280 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-16T22:11:03.280 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout pgs: 2024-02-16T22:11:03.280 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-02-16T22:11:03.280 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.674+0000 7fbab0908700 1 Processor -- start 2024-02-16T22:11:03.280 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.674+0000 7fbab0908700 1 -- start start 2024-02-16T22:11:03.280 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.674+0000 7fbab0908700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbaac104730 0x7fbaac104b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:03.280 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.674+0000 7fbab0908700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaac105120 con 0x7fbaac104730 2024-02-16T22:11:03.280 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.674+0000 7fbaaa59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbaac104730 0x7fbaac104b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:03.280 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.675+0000 7fbaaa59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbaac104730 0x7fbaac104b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53004/0 (socket says 172.21.15.114:53004) 2024-02-16T22:11:03.281 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.675+0000 7fbaaa59c700 1 -- 172.21.15.114:0/1631556948 learned_addr learned my addr 172.21.15.114:0/1631556948 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:03.281 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.675+0000 7fbaaa59c700 1 -- 172.21.15.114:0/1631556948 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaac1076c0 con 0x7fbaac104730 2024-02-16T22:11:03.281 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.675+0000 7fbaaa59c700 1 --2- 172.21.15.114:0/1631556948 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbaac104730 0x7fbaac104b50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fba9c0098a0 tx=0x7fba9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44381ae66dfcc50e server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:03.281 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.675+0000 7fbaa959a700 1 -- 172.21.15.114:0/1631556948 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fba9c004180 con 0x7fbaac104730 2024-02-16T22:11:03.281 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.675+0000 7fbaa959a700 1 -- 172.21.15.114:0/1631556948 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba9c0042e0 con 0x7fbaac104730 2024-02-16T22:11:03.281 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.675+0000 7fbaa959a700 1 -- 172.21.15.114:0/1631556948 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fba9c0036a0 con 0x7fbaac104730 2024-02-16T22:11:03.281 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.676+0000 7fbab0908700 1 -- 172.21.15.114:0/1631556948 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbaac104730 msgr2=0x7fbaac104b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:03.281 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.676+0000 7fbab0908700 1 --2- 172.21.15.114:0/1631556948 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbaac104730 0x7fbaac104b50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fba9c0098a0 tx=0x7fba9c009bb0 comp rx=0 tx=0).stop 2024-02-16T22:11:03.281 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.676+0000 7fbab0908700 1 -- 172.21.15.114:0/1631556948 shutdown_connections 2024-02-16T22:11:03.281 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.676+0000 7fbab0908700 1 --2- 172.21.15.114:0/1631556948 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbaac104730 0x7fbaac104b50 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:03.281 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.676+0000 7fbab0908700 1 -- 172.21.15.114:0/1631556948 >> 172.21.15.114:0/1631556948 conn(0x7fbaac100040 msgr2=0x7fbaac1024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:03.281 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.676+0000 7fbab0908700 1 -- 172.21.15.114:0/1631556948 shutdown_connections 2024-02-16T22:11:03.282 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.676+0000 7fbab0908700 1 -- 172.21.15.114:0/1631556948 wait complete. 2024-02-16T22:11:03.282 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.677+0000 7fbab0908700 1 Processor -- start 2024-02-16T22:11:03.282 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.677+0000 7fbab0908700 1 -- start start 2024-02-16T22:11:03.282 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.677+0000 7fbab0908700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbaac104730 0x7fbaac107020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:03.282 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.677+0000 7fbab0908700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaac107560 con 0x7fbaac104730 2024-02-16T22:11:03.282 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.677+0000 7fbaaa59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbaac104730 0x7fbaac107020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:03.282 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.677+0000 7fbaaa59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbaac104730 0x7fbaac107020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53020/0 (socket says 172.21.15.114:53020) 2024-02-16T22:11:03.282 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.677+0000 7fbaaa59c700 1 -- 172.21.15.114:0/1802711900 learned_addr learned my addr 172.21.15.114:0/1802711900 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:03.282 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.677+0000 7fbaaa59c700 1 -- 172.21.15.114:0/1802711900 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba9c009550 con 0x7fbaac104730 2024-02-16T22:11:03.282 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.677+0000 7fbaaa59c700 1 --2- 172.21.15.114:0/1802711900 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbaac104730 0x7fbaac107020 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fba9c004000 tx=0x7fba9c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:03.282 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.678+0000 7fba9b7fe700 1 -- 172.21.15.114:0/1802711900 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fba9c003ce0 con 0x7fbaac104730 2024-02-16T22:11:03.283 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.678+0000 7fba9b7fe700 1 -- 172.21.15.114:0/1802711900 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba9c020070 con 0x7fbaac104730 2024-02-16T22:11:03.283 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.678+0000 7fbab0908700 1 -- 172.21.15.114:0/1802711900 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaac105670 con 0x7fbaac104730 2024-02-16T22:11:03.283 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.678+0000 7fbab0908700 1 -- 172.21.15.114:0/1802711900 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaac105b10 con 0x7fbaac104730 2024-02-16T22:11:03.283 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.678+0000 7fba9b7fe700 1 -- 172.21.15.114:0/1802711900 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fba9c011630 con 0x7fbaac104730 2024-02-16T22:11:03.283 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.678+0000 7fba9b7fe700 1 -- 172.21.15.114:0/1802711900 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fba9c022030 con 0x7fbaac104730 2024-02-16T22:11:03.283 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.678+0000 7fba9b7fe700 1 -- 172.21.15.114:0/1802711900 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fba9c025d60 con 0x7fbaac104730 2024-02-16T22:11:03.283 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.679+0000 7fbab0908700 1 -- 172.21.15.114:0/1802711900 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba8c005320 con 0x7fbaac104730 2024-02-16T22:11:03.283 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.683+0000 7fba9b7fe700 1 -- 172.21.15.114:0/1802711900 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fba9c025480 con 0x7fbaac104730 2024-02-16T22:11:03.283 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.754+0000 7fbab0908700 1 -- 172.21.15.114:0/1802711900 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fba8c005cc0 con 0x7fbaac104730 2024-02-16T22:11:03.283 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.754+0000 7fba9b7fe700 1 -- 172.21.15.114:0/1802711900 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7fba9c011c60 con 0x7fbaac104730 2024-02-16T22:11:03.283 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.755+0000 7fbab0908700 1 -- 172.21.15.114:0/1802711900 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbaac104730 msgr2=0x7fbaac107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:03.284 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.755+0000 7fbab0908700 1 --2- 172.21.15.114:0/1802711900 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbaac104730 0x7fbaac107020 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fba9c004000 tx=0x7fba9c004750 comp rx=0 tx=0).stop 2024-02-16T22:11:03.284 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.755+0000 7fbab0908700 1 -- 172.21.15.114:0/1802711900 shutdown_connections 2024-02-16T22:11:03.284 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.755+0000 7fbab0908700 1 --2- 172.21.15.114:0/1802711900 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbaac104730 0x7fbaac107020 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:03.284 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.755+0000 7fbab0908700 1 -- 172.21.15.114:0/1802711900 >> 172.21.15.114:0/1802711900 conn(0x7fbaac100040 msgr2=0x7fbaac18de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:03.284 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.755+0000 7fbab0908700 1 -- 172.21.15.114:0/1802711900 shutdown_connections 2024-02-16T22:11:03.284 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:02.755+0000 7fbab0908700 1 -- 172.21.15.114:0/1802711900 wait complete. 2024-02-16T22:11:03.284 INFO:teuthology.orchestra.run.smithi114.stdout:mon is available 2024-02-16T22:11:03.284 INFO:teuthology.orchestra.run.smithi114.stdout:Assimilating anything we can from ceph.conf... 2024-02-16T22:11:03.387 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:03 smithi114 ceph-mon[28858]: mon.smithi114 is new leader, mons smithi114 in quorum (ranks 0) 2024-02-16T22:11:03.388 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:03 smithi114 ceph-mon[28858]: monmap e1: 1 mons at {smithi114=[v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-16T22:11:03.388 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:03 smithi114 ceph-mon[28858]: fsmap 2024-02-16T22:11:03.388 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:03 smithi114 ceph-mon[28858]: osdmap e1: 0 total, 0 up, 0 in 2024-02-16T22:11:03.388 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:03 smithi114 ceph-mon[28858]: mgrmap e1: no daemons active 2024-02-16T22:11:03.388 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:03 smithi114 ceph-mon[28858]: from='client.? 172.21.15.114:0/1802711900' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-16T22:11:04.294 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-02-16T22:11:04.294 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout [global] 2024-02-16T22:11:04.294 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout fsid = 1dcbaabe-cd18-11ee-95ba-87774f69a715 2024-02-16T22:11:04.294 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.114:3300,v1:172.21.15.114:6789] 2024-02-16T22:11:04.294 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-16T22:11:04.294 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-16T22:11:04.294 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-16T22:11:04.294 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-16T22:11:04.294 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-02-16T22:11:04.294 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-16T22:11:04.294 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-16T22:11:04.295 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-02-16T22:11:04.295 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout [osd] 2024-02-16T22:11:04.295 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-16T22:11:04.295 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-16T22:11:04.295 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-16T22:11:04.295 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-16T22:11:04.295 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-16T22:11:04.295 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.701+0000 7f4e1a6af700 1 Processor -- start 2024-02-16T22:11:04.295 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.701+0000 7f4e1a6af700 1 -- start start 2024-02-16T22:11:04.295 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.702+0000 7f4e1a6af700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e14106ef0 0x7f4e141092e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:04.295 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.702+0000 7f4e1a6af700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e14074640 con 0x7f4e14106ef0 2024-02-16T22:11:04.295 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.702+0000 7f4e13fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e14106ef0 0x7f4e141092e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:04.295 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.702+0000 7f4e13fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e14106ef0 0x7f4e141092e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53032/0 (socket says 172.21.15.114:53032) 2024-02-16T22:11:04.296 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.702+0000 7f4e13fff700 1 -- 172.21.15.114:0/3580453669 learned_addr learned my addr 172.21.15.114:0/3580453669 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:04.296 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.702+0000 7f4e13fff700 1 -- 172.21.15.114:0/3580453669 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e14109820 con 0x7f4e14106ef0 2024-02-16T22:11:04.296 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.702+0000 7f4e13fff700 1 --2- 172.21.15.114:0/3580453669 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e14106ef0 0x7f4e141092e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4e040098a0 tx=0x7f4e04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e99f0b91eaee32a server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:04.296 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.702+0000 7f4e12ffd700 1 -- 172.21.15.114:0/3580453669 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4e04004180 con 0x7f4e14106ef0 2024-02-16T22:11:04.296 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.703+0000 7f4e12ffd700 1 -- 172.21.15.114:0/3580453669 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e040042e0 con 0x7f4e14106ef0 2024-02-16T22:11:04.296 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.703+0000 7f4e12ffd700 1 -- 172.21.15.114:0/3580453669 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4e040036a0 con 0x7f4e14106ef0 2024-02-16T22:11:04.296 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.703+0000 7f4e1a6af700 1 -- 172.21.15.114:0/3580453669 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e14106ef0 msgr2=0x7f4e141092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:04.296 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.703+0000 7f4e1a6af700 1 --2- 172.21.15.114:0/3580453669 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e14106ef0 0x7f4e141092e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4e040098a0 tx=0x7f4e04009bb0 comp rx=0 tx=0).stop 2024-02-16T22:11:04.296 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.703+0000 7f4e1a6af700 1 -- 172.21.15.114:0/3580453669 shutdown_connections 2024-02-16T22:11:04.296 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.703+0000 7f4e1a6af700 1 --2- 172.21.15.114:0/3580453669 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e14106ef0 0x7f4e141092e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:04.296 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.703+0000 7f4e1a6af700 1 -- 172.21.15.114:0/3580453669 >> 172.21.15.114:0/3580453669 conn(0x7f4e141008c0 msgr2=0x7f4e14102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:04.296 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.703+0000 7f4e1a6af700 1 -- 172.21.15.114:0/3580453669 shutdown_connections 2024-02-16T22:11:04.296 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.703+0000 7f4e1a6af700 1 -- 172.21.15.114:0/3580453669 wait complete. 2024-02-16T22:11:04.297 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.704+0000 7f4e1a6af700 1 Processor -- start 2024-02-16T22:11:04.297 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.704+0000 7f4e1a6af700 1 -- start start 2024-02-16T22:11:04.297 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.704+0000 7f4e1a6af700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e14106ef0 0x7f4e14106850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:04.297 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.704+0000 7f4e1a6af700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e14106d90 con 0x7f4e14106ef0 2024-02-16T22:11:04.297 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.704+0000 7f4e13fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e14106ef0 0x7f4e14106850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:04.297 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.704+0000 7f4e13fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e14106ef0 0x7f4e14106850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53048/0 (socket says 172.21.15.114:53048) 2024-02-16T22:11:04.297 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.704+0000 7f4e13fff700 1 -- 172.21.15.114:0/3940748508 learned_addr learned my addr 172.21.15.114:0/3940748508 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:04.297 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.704+0000 7f4e13fff700 1 -- 172.21.15.114:0/3940748508 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e04009550 con 0x7f4e14106ef0 2024-02-16T22:11:04.297 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.704+0000 7f4e13fff700 1 --2- 172.21.15.114:0/3940748508 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e14106ef0 0x7f4e14106850 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4e14073a80 tx=0x7f4e04004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:04.297 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.705+0000 7f4e117fa700 1 -- 172.21.15.114:0/3940748508 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4e04003ce0 con 0x7f4e14106ef0 2024-02-16T22:11:04.297 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.705+0000 7f4e117fa700 1 -- 172.21.15.114:0/3940748508 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e04020070 con 0x7f4e14106ef0 2024-02-16T22:11:04.297 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.705+0000 7f4e1a6af700 1 -- 172.21.15.114:0/3940748508 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e14104ea0 con 0x7f4e14106ef0 2024-02-16T22:11:04.297 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.705+0000 7f4e117fa700 1 -- 172.21.15.114:0/3940748508 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4e04011630 con 0x7f4e14106ef0 2024-02-16T22:11:04.297 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.705+0000 7f4e1a6af700 1 -- 172.21.15.114:0/3940748508 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e14105340 con 0x7f4e14106ef0 2024-02-16T22:11:04.298 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.705+0000 7f4e117fa700 1 -- 172.21.15.114:0/3940748508 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4e04022030 con 0x7f4e14106ef0 2024-02-16T22:11:04.298 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.705+0000 7f4e117fa700 1 -- 172.21.15.114:0/3940748508 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4e04025d00 con 0x7f4e14106ef0 2024-02-16T22:11:04.298 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.706+0000 7f4e1a6af700 1 -- 172.21.15.114:0/3940748508 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e140623c0 con 0x7f4e14106ef0 2024-02-16T22:11:04.298 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.708+0000 7f4e117fa700 1 -- 172.21.15.114:0/3940748508 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f4e04025480 con 0x7f4e14106ef0 2024-02-16T22:11:04.298 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.775+0000 7f4e1a6af700 1 -- 172.21.15.114:0/3940748508 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f4e14193720 con 0x7f4e14106ef0 2024-02-16T22:11:04.298 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.778+0000 7f4e117fa700 1 -- 172.21.15.114:0/3940748508 <== mon.0 v2:172.21.15.114:3300/0 7 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4e04011c20 con 0x7f4e14106ef0 2024-02-16T22:11:04.298 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.779+0000 7f4e117fa700 1 -- 172.21.15.114:0/3940748508 <== mon.0 v2:172.21.15.114:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7f4e0401c500 con 0x7f4e14106ef0 2024-02-16T22:11:04.298 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.779+0000 7f4e1a6af700 1 -- 172.21.15.114:0/3940748508 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e14106ef0 msgr2=0x7f4e14106850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:04.298 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.779+0000 7f4e1a6af700 1 --2- 172.21.15.114:0/3940748508 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e14106ef0 0x7f4e14106850 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4e14073a80 tx=0x7f4e04004750 comp rx=0 tx=0).stop 2024-02-16T22:11:04.298 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.780+0000 7f4e1a6af700 1 -- 172.21.15.114:0/3940748508 shutdown_connections 2024-02-16T22:11:04.298 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.780+0000 7f4e1a6af700 1 --2- 172.21.15.114:0/3940748508 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e14106ef0 0x7f4e14106850 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:04.298 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.780+0000 7f4e1a6af700 1 -- 172.21.15.114:0/3940748508 >> 172.21.15.114:0/3940748508 conn(0x7f4e141008c0 msgr2=0x7f4e14102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:04.298 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.780+0000 7f4e1a6af700 1 -- 172.21.15.114:0/3940748508 shutdown_connections 2024-02-16T22:11:04.298 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:03.780+0000 7f4e1a6af700 1 -- 172.21.15.114:0/3940748508 wait complete. 2024-02-16T22:11:04.298 INFO:teuthology.orchestra.run.smithi114.stdout:Generating new minimal ceph.conf... 2024-02-16T22:11:04.412 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:04 smithi114 ceph-mon[28858]: from='client.? 172.21.15.114:0/3940748508' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-16T22:11:04.412 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:04 smithi114 ceph-mon[28858]: from='client.? 172.21.15.114:0/3940748508' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-16T22:11:05.349 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:05 smithi114 ceph-mon[28858]: from='client.? 172.21.15.114:0/1407300935' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:11:05.357 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.699+0000 7f7576b04700 1 Processor -- start 2024-02-16T22:11:05.357 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.699+0000 7f7576b04700 1 -- start start 2024-02-16T22:11:05.357 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.699+0000 7f7576b04700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7570106420 0x7f7570106840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:05.357 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.699+0000 7f7576b04700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7570106e10 con 0x7f7570106420 2024-02-16T22:11:05.357 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.699+0000 7f75748a0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7570106420 0x7f7570106840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:05.357 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.699+0000 7f75748a0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7570106420 0x7f7570106840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53064/0 (socket says 172.21.15.114:53064) 2024-02-16T22:11:05.357 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.699+0000 7f75748a0700 1 -- 172.21.15.114:0/184507143 learned_addr learned my addr 172.21.15.114:0/184507143 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:05.357 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.699+0000 7f75748a0700 1 -- 172.21.15.114:0/184507143 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7570107620 con 0x7f7570106420 2024-02-16T22:11:05.357 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.700+0000 7f75748a0700 1 --2- 172.21.15.114:0/184507143 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7570106420 0x7f7570106840 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f75640098a0 tx=0x7f7564009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b60376f862f68a5 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:05.357 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.700+0000 7f756f7fe700 1 -- 172.21.15.114:0/184507143 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7564004180 con 0x7f7570106420 2024-02-16T22:11:05.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.700+0000 7f756f7fe700 1 -- 172.21.15.114:0/184507143 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f75640042e0 con 0x7f7570106420 2024-02-16T22:11:05.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.700+0000 7f756f7fe700 1 -- 172.21.15.114:0/184507143 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7564011a50 con 0x7f7570106420 2024-02-16T22:11:05.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.700+0000 7f7576b04700 1 -- 172.21.15.114:0/184507143 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7570106420 msgr2=0x7f7570106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:05.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.700+0000 7f7576b04700 1 --2- 172.21.15.114:0/184507143 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7570106420 0x7f7570106840 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f75640098a0 tx=0x7f7564009bb0 comp rx=0 tx=0).stop 2024-02-16T22:11:05.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.700+0000 7f7576b04700 1 -- 172.21.15.114:0/184507143 shutdown_connections 2024-02-16T22:11:05.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.700+0000 7f7576b04700 1 --2- 172.21.15.114:0/184507143 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7570106420 0x7f7570106840 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:05.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.700+0000 7f7576b04700 1 -- 172.21.15.114:0/184507143 >> 172.21.15.114:0/184507143 conn(0x7f7570101a00 msgr2=0x7f7570103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:05.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.700+0000 7f7576b04700 1 -- 172.21.15.114:0/184507143 shutdown_connections 2024-02-16T22:11:05.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.700+0000 7f7576b04700 1 -- 172.21.15.114:0/184507143 wait complete. 2024-02-16T22:11:05.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.701+0000 7f7576b04700 1 Processor -- start 2024-02-16T22:11:05.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.701+0000 7f7576b04700 1 -- start start 2024-02-16T22:11:05.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.701+0000 7f7576b04700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7570106420 0x7f757019a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:05.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.701+0000 7f7576b04700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757019a5f0 con 0x7f7570106420 2024-02-16T22:11:05.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.701+0000 7f75748a0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7570106420 0x7f757019a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:05.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.701+0000 7f75748a0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7570106420 0x7f757019a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53076/0 (socket says 172.21.15.114:53076) 2024-02-16T22:11:05.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.701+0000 7f75748a0700 1 -- 172.21.15.114:0/1407300935 learned_addr learned my addr 172.21.15.114:0/1407300935 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:05.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.701+0000 7f75748a0700 1 -- 172.21.15.114:0/1407300935 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7564009550 con 0x7f7570106420 2024-02-16T22:11:05.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.702+0000 7f75748a0700 1 --2- 172.21.15.114:0/1407300935 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7570106420 0x7f757019a0b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7564000c00 tx=0x7f7564011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:05.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.702+0000 7f756dffb700 1 -- 172.21.15.114:0/1407300935 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7564003b90 con 0x7f7570106420 2024-02-16T22:11:05.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.702+0000 7f756dffb700 1 -- 172.21.15.114:0/1407300935 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f75640248b0 con 0x7f7570106420 2024-02-16T22:11:05.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.702+0000 7f756dffb700 1 -- 172.21.15.114:0/1407300935 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f756401a8e0 con 0x7f7570106420 2024-02-16T22:11:05.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.702+0000 7f7576b04700 1 -- 172.21.15.114:0/1407300935 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757019a7f0 con 0x7f7570106420 2024-02-16T22:11:05.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.702+0000 7f7576b04700 1 -- 172.21.15.114:0/1407300935 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757019ac90 con 0x7f7570106420 2024-02-16T22:11:05.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.702+0000 7f756dffb700 1 -- 172.21.15.114:0/1407300935 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f756402d3f0 con 0x7f7570106420 2024-02-16T22:11:05.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.702+0000 7f756dffb700 1 -- 172.21.15.114:0/1407300935 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7564019400 con 0x7f7570106420 2024-02-16T22:11:05.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.703+0000 7f7576b04700 1 -- 172.21.15.114:0/1407300935 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75700623c0 con 0x7f7570106420 2024-02-16T22:11:05.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.705+0000 7f756dffb700 1 -- 172.21.15.114:0/1407300935 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f7564024a20 con 0x7f7570106420 2024-02-16T22:11:05.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.771+0000 7f7576b04700 1 -- 172.21.15.114:0/1407300935 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f757010a8d0 con 0x7f7570106420 2024-02-16T22:11:05.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.772+0000 7f756dffb700 1 -- 172.21.15.114:0/1407300935 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7f7564035090 con 0x7f7570106420 2024-02-16T22:11:05.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.772+0000 7f7576b04700 1 -- 172.21.15.114:0/1407300935 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7570106420 msgr2=0x7f757019a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:05.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.772+0000 7f7576b04700 1 --2- 172.21.15.114:0/1407300935 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7570106420 0x7f757019a0b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7564000c00 tx=0x7f7564011ab0 comp rx=0 tx=0).stop 2024-02-16T22:11:05.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.772+0000 7f7576b04700 1 -- 172.21.15.114:0/1407300935 shutdown_connections 2024-02-16T22:11:05.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.772+0000 7f7576b04700 1 --2- 172.21.15.114:0/1407300935 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7570106420 0x7f757019a0b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:05.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.772+0000 7f7576b04700 1 -- 172.21.15.114:0/1407300935 >> 172.21.15.114:0/1407300935 conn(0x7f7570101a00 msgr2=0x7f75701027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:05.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.772+0000 7f7576b04700 1 -- 172.21.15.114:0/1407300935 shutdown_connections 2024-02-16T22:11:05.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:04.772+0000 7f7576b04700 1 -- 172.21.15.114:0/1407300935 wait complete. 2024-02-16T22:11:05.360 INFO:teuthology.orchestra.run.smithi114.stdout:Restarting the monitor... 2024-02-16T22:11:05.603 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:05 smithi114 systemd[1]: Stopping Ceph mon.smithi114 for 1dcbaabe-cd18-11ee-95ba-87774f69a715... 2024-02-16T22:11:05.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:05 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi114[28853]: 2024-02-16T22:11:05.600+0000 7f2969548700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi114 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T22:11:05.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:05 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi114[28853]: 2024-02-16T22:11:05.600+0000 7f2969548700 -1 mon.smithi114@0(leader) e1 *** Got Signal Terminated *** 2024-02-16T22:11:05.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:05 smithi114 podman[29111]: 2024-02-16 22:11:05.755081686 +0000 UTC m=+0.238963670 container died e494c505b2a163fe372427f2d1c651872b96143c6f4a472efb0afb9c629fc7e3 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi114, RELEASE=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-02-16T22:11:06.253 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 podman[29111]: 2024-02-16 22:11:06.059749408 +0000 UTC m=+0.543631391 container cleanup e494c505b2a163fe372427f2d1c651872b96143c6f4a472efb0afb9c629fc7e3 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi114, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.1, org.label-schema.license=GPLv2, RELEASE=HEAD, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-02-16T22:11:06.254 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 bash[29111]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi114 2024-02-16T22:11:06.579 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 podman[29125]: 2024-02-16 22:11:06.252250149 +0000 UTC m=+0.498518024 container remove e494c505b2a163fe372427f2d1c651872b96143c6f4a472efb0afb9c629fc7e3 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi114, RELEASE=HEAD, ceph=True, org.label-schema.build-date=20240214, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T22:11:06.579 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 systemd[1]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@mon.smithi114.service: Deactivated successfully. 2024-02-16T22:11:06.579 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 systemd[1]: Stopped Ceph mon.smithi114 for 1dcbaabe-cd18-11ee-95ba-87774f69a715. 2024-02-16T22:11:06.579 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 systemd[1]: Starting Ceph mon.smithi114 for 1dcbaabe-cd18-11ee-95ba-87774f69a715... 2024-02-16T22:11:06.842 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 podman[29217]: 2024-02-16 22:11:06.578207254 +0000 UTC m=+0.102332164 container create b4c4433aa3a498943b424ae37b066e6165519cec63521ab0fb8159651676d519 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi114, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240214, ceph=True) 2024-02-16T22:11:06.842 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 podman[29217]: 2024-02-16 22:11:06.494174168 +0000 UTC m=+0.018299033 image pull 94eadd990e0a5619d90a7220f97c70d1fe3558d9b0fb9e0c1935661dd49a6d5f quay.io/ceph/ceph:v18.2.1 2024-02-16T22:11:06.842 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 podman[29217]: 2024-02-16 22:11:06.755202456 +0000 UTC m=+0.279327323 container init b4c4433aa3a498943b424ae37b066e6165519cec63521ab0fb8159651676d519 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi114, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.1, org.label-schema.build-date=20240214, ceph=True, RELEASE=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-02-16T22:11:06.842 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 podman[29217]: 2024-02-16 22:11:06.757973364 +0000 UTC m=+0.282098229 container start b4c4433aa3a498943b424ae37b066e6165519cec63521ab0fb8159651676d519 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi114, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-02-16T22:11:06.842 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: set uid:gid to 167:167 (ceph:ceph) 2024-02-16T22:11:06.842 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable), process ceph-mon, pid 2 2024-02-16T22:11:06.842 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: pidfile_write: ignore empty --pid-file 2024-02-16T22:11:06.842 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: load: jerasure load: lrc 2024-02-16T22:11:06.843 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: RocksDB version: 7.9.2 2024-02-16T22:11:06.843 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Git sha 0 2024-02-16T22:11:06.843 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Compile date 2023-12-11 22:07:34 2024-02-16T22:11:06.843 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: DB SUMMARY 2024-02-16T22:11:06.843 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: DB Session ID: KEEERT4XE5DR9Y76SQ1W 2024-02-16T22:11:06.843 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: CURRENT file: CURRENT 2024-02-16T22:11:06.843 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: IDENTITY file: IDENTITY 2024-02-16T22:11:06.843 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-02-16T22:11:06.843 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi114/store.db dir, Total Num: 1, files: 000008.sst 2024-02-16T22:11:06.843 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi114/store.db: 000009.log size: 97516 ; 2024-02-16T22:11:06.843 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.error_if_exists: 0 2024-02-16T22:11:06.843 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.create_if_missing: 0 2024-02-16T22:11:06.844 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.paranoid_checks: 1 2024-02-16T22:11:06.845 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-16T22:11:06.845 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-16T22:11:06.845 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-16T22:11:06.845 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.env: 0x565496e43720 2024-02-16T22:11:06.845 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.fs: PosixFileSystem 2024-02-16T22:11:06.845 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.info_log: 0x56549995d2e0 2024-02-16T22:11:06.846 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.max_file_opening_threads: 16 2024-02-16T22:11:06.846 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.statistics: (nil) 2024-02-16T22:11:06.846 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.use_fsync: 0 2024-02-16T22:11:06.846 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.max_log_file_size: 0 2024-02-16T22:11:06.846 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-16T22:11:06.846 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-16T22:11:06.846 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.keep_log_file_num: 1000 2024-02-16T22:11:06.846 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.recycle_log_file_num: 0 2024-02-16T22:11:06.846 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.allow_fallocate: 1 2024-02-16T22:11:06.846 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.allow_mmap_reads: 0 2024-02-16T22:11:06.846 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.allow_mmap_writes: 0 2024-02-16T22:11:06.846 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.use_direct_reads: 0 2024-02-16T22:11:06.846 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-16T22:11:06.847 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.create_missing_column_families: 0 2024-02-16T22:11:06.847 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.db_log_dir: 2024-02-16T22:11:06.847 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.wal_dir: 2024-02-16T22:11:06.847 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-16T22:11:06.847 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-16T22:11:06.847 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-16T22:11:06.847 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-16T22:11:06.847 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-16T22:11:06.847 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-16T22:11:06.847 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.advise_random_on_open: 1 2024-02-16T22:11:06.847 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.db_write_buffer_size: 0 2024-02-16T22:11:06.847 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.write_buffer_manager: 0x565498bce140 2024-02-16T22:11:06.847 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-16T22:11:06.847 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-16T22:11:06.848 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-16T22:11:06.848 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.rate_limiter: (nil) 2024-02-16T22:11:06.848 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-16T22:11:06.848 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.wal_recovery_mode: 2 2024-02-16T22:11:06.848 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.enable_thread_tracking: 0 2024-02-16T22:11:06.848 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.enable_pipelined_write: 0 2024-02-16T22:11:06.848 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.unordered_write: 0 2024-02-16T22:11:06.848 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-16T22:11:06.848 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-16T22:11:06.848 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-16T22:11:06.848 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-16T22:11:06.848 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.row_cache: None 2024-02-16T22:11:06.848 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.wal_filter: None 2024-02-16T22:11:06.848 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-16T22:11:06.849 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.allow_ingest_behind: 0 2024-02-16T22:11:06.849 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.two_write_queues: 0 2024-02-16T22:11:06.849 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.manual_wal_flush: 0 2024-02-16T22:11:06.849 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.wal_compression: 0 2024-02-16T22:11:06.849 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.atomic_flush: 0 2024-02-16T22:11:06.849 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-16T22:11:06.849 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-16T22:11:06.849 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-16T22:11:06.849 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.log_readahead_size: 0 2024-02-16T22:11:06.849 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-16T22:11:06.849 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.best_efforts_recovery: 0 2024-02-16T22:11:06.849 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-16T22:11:06.849 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-16T22:11:06.849 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.allow_data_in_errors: 0 2024-02-16T22:11:06.849 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.db_host_id: __hostname__ 2024-02-16T22:11:06.850 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-16T22:11:06.850 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.max_background_jobs: 2 2024-02-16T22:11:06.850 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.max_background_compactions: -1 2024-02-16T22:11:06.850 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.max_subcompactions: 1 2024-02-16T22:11:06.850 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-16T22:11:06.850 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-16T22:11:06.850 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-16T22:11:06.850 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.max_total_wal_size: 0 2024-02-16T22:11:06.850 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-16T22:11:06.850 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-16T22:11:06.850 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-16T22:11:06.850 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-16T22:11:06.850 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.max_open_files: -1 2024-02-16T22:11:06.850 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.bytes_per_sync: 0 2024-02-16T22:11:06.850 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-16T22:11:06.851 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-16T22:11:06.851 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.compaction_readahead_size: 0 2024-02-16T22:11:06.851 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.max_background_flushes: -1 2024-02-16T22:11:06.851 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Compression algorithms supported: 2024-02-16T22:11:06.851 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-16T22:11:06.851 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: kZSTD supported: 0 2024-02-16T22:11:06.851 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: kXpressCompression supported: 0 2024-02-16T22:11:06.851 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: kLZ4HCCompression supported: 1 2024-02-16T22:11:06.851 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: kZlibCompression supported: 1 2024-02-16T22:11:06.851 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: kSnappyCompression supported: 1 2024-02-16T22:11:06.851 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: kLZ4Compression supported: 1 2024-02-16T22:11:06.851 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: kBZip2Compression supported: 0 2024-02-16T22:11:06.851 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-16T22:11:06.851 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-16T22:11:06.851 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi114/store.db/MANIFEST-000010 2024-02-16T22:11:06.852 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-16T22:11:06.852 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-16T22:11:06.852 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.merge_operator: 2024-02-16T22:11:06.852 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.compaction_filter: None 2024-02-16T22:11:06.852 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.compaction_filter_factory: None 2024-02-16T22:11:06.852 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.sst_partitioner_factory: None 2024-02-16T22:11:06.852 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-16T22:11:06.852 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-16T22:11:06.852 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56549995d3e0) 2024-02-16T22:11:06.852 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: cache_index_and_filter_blocks: 1 2024-02-16T22:11:06.852 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-16T22:11:06.852 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-16T22:11:06.853 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: pin_top_level_index_and_filter: 1 2024-02-16T22:11:06.853 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: index_type: 0 2024-02-16T22:11:06.853 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: data_block_index_type: 0 2024-02-16T22:11:06.853 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: index_shortening: 1 2024-02-16T22:11:06.853 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-16T22:11:06.853 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: checksum: 4 2024-02-16T22:11:06.853 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: no_block_cache: 0 2024-02-16T22:11:06.853 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: block_cache: 0x565498c6f350 2024-02-16T22:11:06.853 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: block_cache_name: BinnedLRUCache 2024-02-16T22:11:06.853 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: block_cache_options: 2024-02-16T22:11:06.853 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: capacity : 536870912 2024-02-16T22:11:06.853 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: num_shard_bits : 4 2024-02-16T22:11:06.853 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: strict_capacity_limit : 0 2024-02-16T22:11:06.854 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: high_pri_pool_ratio: 0.000 2024-02-16T22:11:06.854 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: block_cache_compressed: (nil) 2024-02-16T22:11:06.854 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: persistent_cache: (nil) 2024-02-16T22:11:06.854 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: block_size: 4096 2024-02-16T22:11:06.854 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: block_size_deviation: 10 2024-02-16T22:11:06.854 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: block_restart_interval: 16 2024-02-16T22:11:06.854 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: index_block_restart_interval: 1 2024-02-16T22:11:06.854 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: metadata_block_size: 4096 2024-02-16T22:11:06.854 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: partition_filters: 0 2024-02-16T22:11:06.854 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: use_delta_encoding: 1 2024-02-16T22:11:06.854 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: filter_policy: bloomfilter 2024-02-16T22:11:06.854 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: whole_key_filtering: 1 2024-02-16T22:11:06.854 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: verify_compression: 0 2024-02-16T22:11:06.854 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: read_amp_bytes_per_bit: 0 2024-02-16T22:11:06.855 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: format_version: 5 2024-02-16T22:11:06.855 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: enable_index_compression: 1 2024-02-16T22:11:06.855 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: block_align: 0 2024-02-16T22:11:06.855 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: max_auto_readahead_size: 262144 2024-02-16T22:11:06.855 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: prepopulate_block_cache: 0 2024-02-16T22:11:06.855 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: initial_auto_readahead_size: 8192 2024-02-16T22:11:06.855 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: num_file_reads_for_auto_readahead: 2 2024-02-16T22:11:06.855 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.write_buffer_size: 33554432 2024-02-16T22:11:06.855 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.max_write_buffer_number: 2 2024-02-16T22:11:06.855 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.compression: NoCompression 2024-02-16T22:11:06.855 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.bottommost_compression: Disabled 2024-02-16T22:11:06.855 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.prefix_extractor: nullptr 2024-02-16T22:11:06.855 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-16T22:11:06.855 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.num_levels: 7 2024-02-16T22:11:06.856 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-16T22:11:06.856 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-16T22:11:06.856 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-16T22:11:06.856 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-16T22:11:06.856 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-16T22:11:06.856 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-16T22:11:06.856 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-16T22:11:06.856 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-16T22:11:06.856 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-16T22:11:06.856 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-16T22:11:06.856 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-16T22:11:06.856 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-16T22:11:06.856 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-16T22:11:06.856 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.compression_opts.level: 32767 2024-02-16T22:11:06.857 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.compression_opts.strategy: 0 2024-02-16T22:11:06.857 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-16T22:11:06.857 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-16T22:11:06.857 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-16T22:11:06.857 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-16T22:11:06.857 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.compression_opts.enabled: false 2024-02-16T22:11:06.857 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-16T22:11:06.857 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-16T22:11:06.857 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-16T22:11:06.857 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-16T22:11:06.857 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.target_file_size_base: 67108864 2024-02-16T22:11:06.857 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-16T22:11:06.858 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-16T22:11:06.858 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-16T22:11:06.858 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-16T22:11:06.858 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-16T22:11:06.858 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-16T22:11:06.858 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-16T22:11:06.858 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-16T22:11:06.858 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-16T22:11:06.858 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-16T22:11:06.858 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-16T22:11:06.858 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-16T22:11:06.858 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-16T22:11:06.858 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-16T22:11:06.858 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.arena_block_size: 1048576 2024-02-16T22:11:06.859 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-16T22:11:06.859 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-16T22:11:06.859 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.disable_auto_compactions: 0 2024-02-16T22:11:06.859 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-16T22:11:06.859 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-16T22:11:06.859 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-16T22:11:06.859 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-16T22:11:06.859 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-16T22:11:06.859 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-16T22:11:06.859 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-16T22:11:06.859 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-16T22:11:06.859 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-16T22:11:06.859 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-16T22:11:06.860 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.table_properties_collectors: 2024-02-16T22:11:06.860 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.inplace_update_support: 0 2024-02-16T22:11:06.860 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-16T22:11:06.860 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-16T22:11:06.860 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-16T22:11:06.860 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-16T22:11:06.860 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.bloom_locality: 0 2024-02-16T22:11:06.860 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.max_successive_merges: 0 2024-02-16T22:11:06.860 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-16T22:11:06.860 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.paranoid_file_checks: 0 2024-02-16T22:11:06.860 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.force_consistency_checks: 1 2024-02-16T22:11:06.860 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.report_bg_io_stats: 0 2024-02-16T22:11:06.860 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.ttl: 2592000 2024-02-16T22:11:06.861 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-16T22:11:06.861 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-16T22:11:06.861 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-16T22:11:06.861 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.enable_blob_files: false 2024-02-16T22:11:06.861 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.min_blob_size: 0 2024-02-16T22:11:06.861 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.blob_file_size: 268435456 2024-02-16T22:11:06.861 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-16T22:11:06.861 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-16T22:11:06.861 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-16T22:11:06.861 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-16T22:11:06.861 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-16T22:11:06.861 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.blob_file_starting_level: 0 2024-02-16T22:11:06.861 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-16T22:11:06.861 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi114/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-02-16T22:11:06.862 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-02-16T22:11:06.862 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 09eac42e-19a1-425c-924d-a6b3942204ce 2024-02-16T22:11:06.862 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708121466786072, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-02-16T22:11:06.862 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-02-16T22:11:06.862 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708121466787031, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 92866, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 310, "table_properties": {"data_size": 90883, "index_size": 265, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13356, "raw_average_key_size": 49, "raw_value_size": 83598, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 269, "num_filter_entries": 269, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708121466, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "09eac42e-19a1-425c-924d-a6b3942204ce", "db_session_id": "KEEERT4XE5DR9Y76SQ1W", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-02-16T22:11:06.862 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708121466787120, "job": 1, "event": "recovery_finished"} 2024-02-16T22:11:06.862 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-02-16T22:11:06.862 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi114/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-16T22:11:06.862 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x565498d0c000 2024-02-16T22:11:06.862 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: DB pointer 0x565498cf8000 2024-02-16T22:11:06.862 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-16T22:11:06.862 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-16T22:11:06.863 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: ** DB Stats ** 2024-02-16T22:11:06.863 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-16T22:11:06.863 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-16T22:11:06.863 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-16T22:11:06.863 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-16T22:11:06.863 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-16T22:11:06.863 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-16T22:11:06.863 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-16T22:11:06.863 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: 2024-02-16T22:11:06.863 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: ** Compaction Stats [default] ** 2024-02-16T22:11:06.863 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-16T22:11:06.863 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-16T22:11:06.863 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: L0 2/0 92.56 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 142.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-16T22:11:06.863 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Sum 2/0 92.56 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 142.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-16T22:11:06.864 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 142.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-16T22:11:06.864 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: 2024-02-16T22:11:06.864 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: ** Compaction Stats [default] ** 2024-02-16T22:11:06.864 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-16T22:11:06.864 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-16T22:11:06.864 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 142.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-16T22:11:06.864 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: 2024-02-16T22:11:06.864 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-16T22:11:06.864 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: 2024-02-16T22:11:06.864 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-16T22:11:06.864 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-16T22:11:06.864 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-16T22:11:06.864 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-16T22:11:06.865 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-16T22:11:06.865 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-16T22:11:06.865 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Cumulative compaction: 0.00 GB write, 30.87 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-16T22:11:06.865 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Interval compaction: 0.00 GB write, 30.87 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-16T22:11:06.865 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-16T22:11:06.865 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Block cache BinnedLRUCache@0x565498c6f350#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-16T22:11:06.865 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-02-16T22:11:06.865 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: 2024-02-16T22:11:06.865 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-16T22:11:06.865 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: starting mon.smithi114 rank 0 at public addrs [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] at bind addrs [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi114 fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 2024-02-16T22:11:06.865 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: mon.smithi114@-1(???) e1 preinit fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 2024-02-16T22:11:06.865 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: mon.smithi114@-1(???).mds e1 new map 2024-02-16T22:11:06.865 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: mon.smithi114@-1(???).mds e1 print_map 2024-02-16T22:11:06.865 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: e1 2024-02-16T22:11:06.866 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:11:06.866 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:11:06.866 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: legacy client fscid: -1 2024-02-16T22:11:06.866 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: 2024-02-16T22:11:06.866 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: No filesystems configured 2024-02-16T22:11:06.866 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: mon.smithi114@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-16T22:11:06.866 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: mon.smithi114@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-16T22:11:06.866 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: mon.smithi114@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-16T22:11:06.866 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: mon.smithi114@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-16T22:11:06.866 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: mon.smithi114@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-16T22:11:06.866 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 bash[29217]: b4c4433aa3a498943b424ae37b066e6165519cec63521ab0fb8159651676d519 2024-02-16T22:11:06.866 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 systemd[1]: Started Ceph mon.smithi114 for 1dcbaabe-cd18-11ee-95ba-87774f69a715. 2024-02-16T22:11:06.867 INFO:teuthology.orchestra.run.smithi114.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-16T22:11:07.104 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: mon.smithi114 is new leader, mons smithi114 in quorum (ranks 0) 2024-02-16T22:11:07.104 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: monmap e1: 1 mons at {smithi114=[v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-16T22:11:07.104 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: fsmap 2024-02-16T22:11:07.104 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: osdmap e1: 0 total, 0 up, 0 in 2024-02-16T22:11:07.104 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:06 smithi114 ceph-mon[29235]: mgrmap e1: no daemons active 2024-02-16T22:11:07.805 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.202+0000 7f585fa3d700 1 Processor -- start 2024-02-16T22:11:07.806 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.202+0000 7f585fa3d700 1 -- start start 2024-02-16T22:11:07.806 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.202+0000 7f585fa3d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5858106f10 0x7f5858109300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:07.806 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.203+0000 7f585fa3d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5858074620 con 0x7f5858106f10 2024-02-16T22:11:07.806 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.203+0000 7f585d7d9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5858106f10 0x7f5858109300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:07.806 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.203+0000 7f585d7d9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5858106f10 0x7f5858109300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53084/0 (socket says 172.21.15.114:53084) 2024-02-16T22:11:07.806 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.203+0000 7f585d7d9700 1 -- 172.21.15.114:0/3694240205 learned_addr learned my addr 172.21.15.114:0/3694240205 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:07.806 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.203+0000 7f585d7d9700 1 -- 172.21.15.114:0/3694240205 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5858109840 con 0x7f5858106f10 2024-02-16T22:11:07.806 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.203+0000 7f585d7d9700 1 --2- 172.21.15.114:0/3694240205 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5858106f10 0x7f5858109300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5850009e80 tx=0x7f585000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d3a03edc2f01c6 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:07.806 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.204+0000 7f5857fff700 1 -- 172.21.15.114:0/3694240205 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f585000e070 con 0x7f5858106f10 2024-02-16T22:11:07.806 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.204+0000 7f5857fff700 1 -- 172.21.15.114:0/3694240205 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5850003e90 con 0x7f5858106f10 2024-02-16T22:11:07.807 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.204+0000 7f5857fff700 1 -- 172.21.15.114:0/3694240205 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5850004380 con 0x7f5858106f10 2024-02-16T22:11:07.807 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.204+0000 7f585fa3d700 1 -- 172.21.15.114:0/3694240205 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5858106f10 msgr2=0x7f5858109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:07.807 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.204+0000 7f585fa3d700 1 --2- 172.21.15.114:0/3694240205 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5858106f10 0x7f5858109300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5850009e80 tx=0x7f585000b330 comp rx=0 tx=0).stop 2024-02-16T22:11:07.807 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.204+0000 7f585fa3d700 1 -- 172.21.15.114:0/3694240205 shutdown_connections 2024-02-16T22:11:07.807 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.204+0000 7f585fa3d700 1 --2- 172.21.15.114:0/3694240205 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5858106f10 0x7f5858109300 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:07.807 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.204+0000 7f585fa3d700 1 -- 172.21.15.114:0/3694240205 >> 172.21.15.114:0/3694240205 conn(0x7f58581008c0 msgr2=0x7f5858102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:07.807 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.204+0000 7f585fa3d700 1 -- 172.21.15.114:0/3694240205 shutdown_connections 2024-02-16T22:11:07.807 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.204+0000 7f585fa3d700 1 -- 172.21.15.114:0/3694240205 wait complete. 2024-02-16T22:11:07.807 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.205+0000 7f585fa3d700 1 Processor -- start 2024-02-16T22:11:07.807 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.205+0000 7f585fa3d700 1 -- start start 2024-02-16T22:11:07.807 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.205+0000 7f585fa3d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5858106f10 0x7f585819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:07.807 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.205+0000 7f585fa3d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f585819a600 con 0x7f5858106f10 2024-02-16T22:11:07.807 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.205+0000 7f585d7d9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5858106f10 0x7f585819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:07.807 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.205+0000 7f585d7d9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5858106f10 0x7f585819a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53088/0 (socket says 172.21.15.114:53088) 2024-02-16T22:11:07.808 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.205+0000 7f585d7d9700 1 -- 172.21.15.114:0/3931054496 learned_addr learned my addr 172.21.15.114:0/3931054496 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:07.808 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.205+0000 7f585d7d9700 1 -- 172.21.15.114:0/3931054496 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5850009b30 con 0x7f5858106f10 2024-02-16T22:11:07.808 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.206+0000 7f585d7d9700 1 --2- 172.21.15.114:0/3931054496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5858106f10 0x7f585819a0c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f585000d010 tx=0x7f5850004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:07.808 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.206+0000 7f58567fc700 1 -- 172.21.15.114:0/3931054496 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f585000e040 con 0x7f5858106f10 2024-02-16T22:11:07.808 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.206+0000 7f585fa3d700 1 -- 172.21.15.114:0/3931054496 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f585819a800 con 0x7f5858106f10 2024-02-16T22:11:07.808 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.206+0000 7f58567fc700 1 -- 172.21.15.114:0/3931054496 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5850003c30 con 0x7f5858106f10 2024-02-16T22:11:07.808 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.206+0000 7f58567fc700 1 -- 172.21.15.114:0/3931054496 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f585001b790 con 0x7f5858106f10 2024-02-16T22:11:07.808 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.206+0000 7f585fa3d700 1 -- 172.21.15.114:0/3931054496 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f585819ad00 con 0x7f5858106f10 2024-02-16T22:11:07.808 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.206+0000 7f58567fc700 1 -- 172.21.15.114:0/3931054496 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5850019040 con 0x7f5858106f10 2024-02-16T22:11:07.808 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.206+0000 7f58567fc700 1 -- 172.21.15.114:0/3931054496 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f585001a6d0 con 0x7f5858106f10 2024-02-16T22:11:07.808 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.207+0000 7f585fa3d700 1 -- 172.21.15.114:0/3931054496 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f585810a560 con 0x7f5858106f10 2024-02-16T22:11:07.808 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.209+0000 7f58567fc700 1 -- 172.21.15.114:0/3931054496 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f5850029060 con 0x7f5858106f10 2024-02-16T22:11:07.808 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.275+0000 7f585fa3d700 1 -- 172.21.15.114:0/3931054496 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f585819da80 con 0x7f5858106f10 2024-02-16T22:11:07.809 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.280+0000 7f58567fc700 1 -- 172.21.15.114:0/3931054496 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f585001f070 con 0x7f5858106f10 2024-02-16T22:11:07.809 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.281+0000 7f585fa3d700 1 -- 172.21.15.114:0/3931054496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5858106f10 msgr2=0x7f585819a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:07.809 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.281+0000 7f585fa3d700 1 --2- 172.21.15.114:0/3931054496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5858106f10 0x7f585819a0c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f585000d010 tx=0x7f5850004430 comp rx=0 tx=0).stop 2024-02-16T22:11:07.809 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.281+0000 7f585fa3d700 1 -- 172.21.15.114:0/3931054496 shutdown_connections 2024-02-16T22:11:07.809 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.281+0000 7f585fa3d700 1 --2- 172.21.15.114:0/3931054496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5858106f10 0x7f585819a0c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:07.809 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.281+0000 7f585fa3d700 1 -- 172.21.15.114:0/3931054496 >> 172.21.15.114:0/3931054496 conn(0x7f58581008c0 msgr2=0x7f5858101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:07.809 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.281+0000 7f585fa3d700 1 -- 172.21.15.114:0/3931054496 shutdown_connections 2024-02-16T22:11:07.809 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:07.281+0000 7f585fa3d700 1 -- 172.21.15.114:0/3931054496 wait complete. 2024-02-16T22:11:07.809 INFO:teuthology.orchestra.run.smithi114.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-16T22:11:07.822 INFO:teuthology.orchestra.run.smithi114.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-16T22:11:07.823 INFO:teuthology.orchestra.run.smithi114.stdout:Creating mgr... 2024-02-16T22:11:07.823 INFO:teuthology.orchestra.run.smithi114.stdout:Verifying port 0.0.0.0:9283 ... 2024-02-16T22:11:07.823 INFO:teuthology.orchestra.run.smithi114.stdout:Verifying port 0.0.0.0:8765 ... 2024-02-16T22:11:07.824 INFO:teuthology.orchestra.run.smithi114.stdout:Verifying port 0.0.0.0:8443 ... 2024-02-16T22:11:08.071 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@mgr.smithi114.vcullz 2024-02-16T22:11:08.072 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@mgr.smithi114.vcullz.service: Unit ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@mgr.smithi114.vcullz.service not loaded. 2024-02-16T22:11:08.221 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715.target.wants/ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@mgr.smithi114.vcullz.service → /etc/systemd/system/ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@.service. 2024-02-16T22:11:08.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:08 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/3931054496' entity='client.admin' 2024-02-16T22:11:08.758 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-16T22:11:08.758 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout disabled 2024-02-16T22:11:08.765 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-16T22:11:08.765 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout inactive 2024-02-16T22:11:08.765 INFO:teuthology.orchestra.run.smithi114.stdout:firewalld.service is not enabled 2024-02-16T22:11:08.765 INFO:teuthology.orchestra.run.smithi114.stdout:Not possible to enable service . firewalld.service is not available 2024-02-16T22:11:08.774 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-16T22:11:08.774 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout disabled 2024-02-16T22:11:08.781 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-16T22:11:08.781 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout inactive 2024-02-16T22:11:08.781 INFO:teuthology.orchestra.run.smithi114.stdout:firewalld.service is not enabled 2024-02-16T22:11:08.781 INFO:teuthology.orchestra.run.smithi114.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-02-16T22:11:08.781 INFO:teuthology.orchestra.run.smithi114.stdout:Waiting for mgr to start... 2024-02-16T22:11:08.781 INFO:teuthology.orchestra.run.smithi114.stdout:Waiting for mgr... 2024-02-16T22:11:09.635 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:09 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/523161095' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-16T22:11:09.837 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-02-16T22:11:09.849 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout { 2024-02-16T22:11:09.849 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "fsid": "1dcbaabe-cd18-11ee-95ba-87774f69a715", 2024-02-16T22:11:09.849 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "health": { 2024-02-16T22:11:09.849 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-16T22:11:09.849 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-16T22:11:09.849 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-16T22:11:09.850 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:11:09.850 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-16T22:11:09.850 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-16T22:11:09.850 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 0 2024-02-16T22:11:09.850 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout ], 2024-02-16T22:11:09.850 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-16T22:11:09.850 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "smithi114" 2024-02-16T22:11:09.850 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout ], 2024-02-16T22:11:09.850 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-02-16T22:11:09.850 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-16T22:11:09.850 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:11:09.850 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-02-16T22:11:09.850 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-16T22:11:09.851 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:11:09.851 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-16T22:11:09.851 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:11:09.851 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-16T22:11:09.851 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-16T22:11:09.851 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-16T22:11:09.851 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-16T22:11:09.851 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-16T22:11:09.851 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-16T22:11:09.851 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:11:09.851 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-16T22:11:09.851 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-16T22:11:09.851 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-16T22:11:09.852 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-16T22:11:09.852 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-16T22:11:09.852 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-16T22:11:09.852 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-16T22:11:09.852 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-16T22:11:09.852 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-16T22:11:09.852 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:11:09.852 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-16T22:11:09.852 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:11:09.852 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-16T22:11:09.852 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-16T22:11:09.853 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:11:09.853 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-16T22:11:09.853 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-16T22:11:09.853 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-16T22:11:09.853 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-16T22:11:09.853 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-16T22:11:09.853 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-16T22:11:09.853 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "restful" 2024-02-16T22:11:09.853 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout ], 2024-02-16T22:11:09.853 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-16T22:11:09.853 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:11:09.853 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-16T22:11:09.853 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:11:09.854 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "modified": "2024-02-16T22:11:02.186641+0000", 2024-02-16T22:11:09.854 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-16T22:11:09.854 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:11:09.854 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-16T22:11:09.854 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout } 2024-02-16T22:11:09.854 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.196+0000 7f76dbad3700 1 Processor -- start 2024-02-16T22:11:09.854 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.196+0000 7f76dbad3700 1 -- start start 2024-02-16T22:11:09.854 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.196+0000 7f76dbad3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76d407be90 0x7f76d407a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:09.854 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.196+0000 7f76dbad3700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d407a8c0 con 0x7f76d407be90 2024-02-16T22:11:09.854 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.196+0000 7f76d986f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76d407be90 0x7f76d407a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:09.854 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.196+0000 7f76d986f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76d407be90 0x7f76d407a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53108/0 (socket says 172.21.15.114:53108) 2024-02-16T22:11:09.854 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.196+0000 7f76d986f700 1 -- 172.21.15.114:0/1399730206 learned_addr learned my addr 172.21.15.114:0/1399730206 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:09.854 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.197+0000 7f76d986f700 1 -- 172.21.15.114:0/1399730206 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d407aa00 con 0x7f76d407be90 2024-02-16T22:11:09.855 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.197+0000 7f76d986f700 1 --2- 172.21.15.114:0/1399730206 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76d407be90 0x7f76d407a2f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f76d00098a0 tx=0x7f76d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8313e4c312095dd server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:09.855 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.197+0000 7f76d886d700 1 -- 172.21.15.114:0/1399730206 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f76d0004180 con 0x7f76d407be90 2024-02-16T22:11:09.855 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.197+0000 7f76d886d700 1 -- 172.21.15.114:0/1399730206 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f76d00042e0 con 0x7f76d407be90 2024-02-16T22:11:09.855 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.197+0000 7f76d886d700 1 -- 172.21.15.114:0/1399730206 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f76d0011a50 con 0x7f76d407be90 2024-02-16T22:11:09.855 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.198+0000 7f76dbad3700 1 -- 172.21.15.114:0/1399730206 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76d407be90 msgr2=0x7f76d407a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:09.855 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.198+0000 7f76dbad3700 1 --2- 172.21.15.114:0/1399730206 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76d407be90 0x7f76d407a2f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f76d00098a0 tx=0x7f76d0009bb0 comp rx=0 tx=0).stop 2024-02-16T22:11:09.855 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.198+0000 7f76dbad3700 1 -- 172.21.15.114:0/1399730206 shutdown_connections 2024-02-16T22:11:09.855 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.198+0000 7f76dbad3700 1 --2- 172.21.15.114:0/1399730206 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76d407be90 0x7f76d407a2f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:09.855 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.198+0000 7f76dbad3700 1 -- 172.21.15.114:0/1399730206 >> 172.21.15.114:0/1399730206 conn(0x7f76d41019d0 msgr2=0x7f76d4103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:09.855 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.198+0000 7f76dbad3700 1 -- 172.21.15.114:0/1399730206 shutdown_connections 2024-02-16T22:11:09.855 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.198+0000 7f76dbad3700 1 -- 172.21.15.114:0/1399730206 wait complete. 2024-02-16T22:11:09.855 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.199+0000 7f76dbad3700 1 Processor -- start 2024-02-16T22:11:09.855 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.199+0000 7f76dbad3700 1 -- start start 2024-02-16T22:11:09.855 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.199+0000 7f76dbad3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76d407be90 0x7f76d419c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:09.856 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.199+0000 7f76dbad3700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d419c820 con 0x7f76d407be90 2024-02-16T22:11:09.856 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.199+0000 7f76d986f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76d407be90 0x7f76d419c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:09.856 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.199+0000 7f76d986f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76d407be90 0x7f76d419c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53110/0 (socket says 172.21.15.114:53110) 2024-02-16T22:11:09.856 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.199+0000 7f76d986f700 1 -- 172.21.15.114:0/523161095 learned_addr learned my addr 172.21.15.114:0/523161095 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:09.856 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.199+0000 7f76d986f700 1 -- 172.21.15.114:0/523161095 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d0009550 con 0x7f76d407be90 2024-02-16T22:11:09.856 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.199+0000 7f76d986f700 1 --2- 172.21.15.114:0/523161095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76d407be90 0x7f76d419c2e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f76d0000c00 tx=0x7f76d0011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:09.856 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.199+0000 7f76caffd700 1 -- 172.21.15.114:0/523161095 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f76d0003b90 con 0x7f76d407be90 2024-02-16T22:11:09.857 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.199+0000 7f76caffd700 1 -- 172.21.15.114:0/523161095 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f76d00248b0 con 0x7f76d407be90 2024-02-16T22:11:09.857 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.199+0000 7f76dbad3700 1 -- 172.21.15.114:0/523161095 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d419ca20 con 0x7f76d407be90 2024-02-16T22:11:09.857 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.199+0000 7f76caffd700 1 -- 172.21.15.114:0/523161095 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f76d001a8e0 con 0x7f76d407be90 2024-02-16T22:11:09.857 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.200+0000 7f76dbad3700 1 -- 172.21.15.114:0/523161095 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d419cec0 con 0x7f76d407be90 2024-02-16T22:11:09.857 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.200+0000 7f76caffd700 1 -- 172.21.15.114:0/523161095 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f76d002d3f0 con 0x7f76d407be90 2024-02-16T22:11:09.857 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.200+0000 7f76caffd700 1 -- 172.21.15.114:0/523161095 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f76d0019520 con 0x7f76d407be90 2024-02-16T22:11:09.857 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.201+0000 7f76c8ff9700 1 -- 172.21.15.114:0/523161095 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76bc0052f0 con 0x7f76d407be90 2024-02-16T22:11:09.857 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.203+0000 7f76caffd700 1 -- 172.21.15.114:0/523161095 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f76d0024db0 con 0x7f76d407be90 2024-02-16T22:11:09.857 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.275+0000 7f76c8ff9700 1 -- 172.21.15.114:0/523161095 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f76bc005c90 con 0x7f76d407be90 2024-02-16T22:11:09.857 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.275+0000 7f76caffd700 1 -- 172.21.15.114:0/523161095 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f76d0024a20 con 0x7f76d407be90 2024-02-16T22:11:09.857 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.276+0000 7f76c8ff9700 1 -- 172.21.15.114:0/523161095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76d407be90 msgr2=0x7f76d419c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:09.857 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.276+0000 7f76c8ff9700 1 --2- 172.21.15.114:0/523161095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76d407be90 0x7f76d419c2e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f76d0000c00 tx=0x7f76d0011ab0 comp rx=0 tx=0).stop 2024-02-16T22:11:09.858 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.276+0000 7f76c8ff9700 1 -- 172.21.15.114:0/523161095 shutdown_connections 2024-02-16T22:11:09.858 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.276+0000 7f76c8ff9700 1 --2- 172.21.15.114:0/523161095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76d407be90 0x7f76d419c2e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:09.858 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.276+0000 7f76c8ff9700 1 -- 172.21.15.114:0/523161095 >> 172.21.15.114:0/523161095 conn(0x7f76d41019d0 msgr2=0x7f76d4102790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:09.858 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.276+0000 7f76c8ff9700 1 -- 172.21.15.114:0/523161095 shutdown_connections 2024-02-16T22:11:09.858 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:09.277+0000 7f76c8ff9700 1 -- 172.21.15.114:0/523161095 wait complete. 2024-02-16T22:11:09.858 INFO:teuthology.orchestra.run.smithi114.stdout:mgr not available, waiting (1/15)... 2024-02-16T22:11:12.930 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:12 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/2430446371' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-16T22:11:12.987 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-02-16T22:11:12.988 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout { 2024-02-16T22:11:12.988 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "fsid": "1dcbaabe-cd18-11ee-95ba-87774f69a715", 2024-02-16T22:11:12.988 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "health": { 2024-02-16T22:11:12.988 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-16T22:11:12.988 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-16T22:11:12.988 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-16T22:11:12.988 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:11:12.988 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-16T22:11:12.988 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-16T22:11:12.988 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 0 2024-02-16T22:11:12.988 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout ], 2024-02-16T22:11:12.988 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-16T22:11:12.988 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "smithi114" 2024-02-16T22:11:12.989 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout ], 2024-02-16T22:11:12.989 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-02-16T22:11:12.989 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-16T22:11:12.989 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:11:12.989 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-02-16T22:11:12.989 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-16T22:11:12.989 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:11:12.989 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-16T22:11:12.989 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:11:12.989 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-16T22:11:12.989 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-16T22:11:12.989 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-16T22:11:12.997 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-16T22:11:12.997 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-16T22:11:12.997 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-16T22:11:12.997 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:11:12.997 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-16T22:11:12.997 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-16T22:11:12.997 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-16T22:11:12.998 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-16T22:11:12.998 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-16T22:11:12.998 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-16T22:11:12.998 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-16T22:11:12.998 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-16T22:11:12.998 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-16T22:11:12.998 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:11:12.998 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-16T22:11:12.998 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:11:12.998 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-16T22:11:12.998 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-16T22:11:12.998 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:11:12.998 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-16T22:11:12.998 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-16T22:11:12.999 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-16T22:11:12.999 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-16T22:11:12.999 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-16T22:11:12.999 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-16T22:11:12.999 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "restful" 2024-02-16T22:11:12.999 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout ], 2024-02-16T22:11:12.999 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-16T22:11:12.999 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:11:12.999 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-16T22:11:12.999 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:11:12.999 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "modified": "2024-02-16T22:11:02.186641+0000", 2024-02-16T22:11:12.999 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-16T22:11:12.999 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:11:12.999 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-16T22:11:13.000 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout } 2024-02-16T22:11:13.000 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.292+0000 7ff1f739d700 1 Processor -- start 2024-02-16T22:11:13.000 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.292+0000 7ff1f739d700 1 -- start start 2024-02-16T22:11:13.000 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.292+0000 7ff1f739d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1f0107a40 0x7ff1f0107e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:13.000 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.292+0000 7ff1f739d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f0108430 con 0x7ff1f0107a40 2024-02-16T22:11:13.000 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.292+0000 7ff1f5139700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1f0107a40 0x7ff1f0107e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:13.000 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.292+0000 7ff1f5139700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1f0107a40 0x7ff1f0107e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33654/0 (socket says 172.21.15.114:33654) 2024-02-16T22:11:13.000 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.292+0000 7ff1f5139700 1 -- 172.21.15.114:0/4183307468 learned_addr learned my addr 172.21.15.114:0/4183307468 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:13.000 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.292+0000 7ff1f5139700 1 -- 172.21.15.114:0/4183307468 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f0108570 con 0x7ff1f0107a40 2024-02-16T22:11:13.000 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.293+0000 7ff1f5139700 1 --2- 172.21.15.114:0/4183307468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1f0107a40 0x7ff1f0107e60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0098a0 tx=0x7ff1ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bb89aec7aa6393 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:13.000 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.293+0000 7ff1e7fff700 1 -- 172.21.15.114:0/4183307468 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff1ec004180 con 0x7ff1f0107a40 2024-02-16T22:11:13.000 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.293+0000 7ff1e7fff700 1 -- 172.21.15.114:0/4183307468 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff1ec0042e0 con 0x7ff1f0107a40 2024-02-16T22:11:13.000 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.293+0000 7ff1e7fff700 1 -- 172.21.15.114:0/4183307468 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff1ec011a50 con 0x7ff1f0107a40 2024-02-16T22:11:13.001 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.293+0000 7ff1f739d700 1 -- 172.21.15.114:0/4183307468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1f0107a40 msgr2=0x7ff1f0107e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:13.001 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.293+0000 7ff1f739d700 1 --2- 172.21.15.114:0/4183307468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1f0107a40 0x7ff1f0107e60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0098a0 tx=0x7ff1ec009bb0 comp rx=0 tx=0).stop 2024-02-16T22:11:13.001 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.293+0000 7ff1f739d700 1 -- 172.21.15.114:0/4183307468 shutdown_connections 2024-02-16T22:11:13.001 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.293+0000 7ff1f739d700 1 --2- 172.21.15.114:0/4183307468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1f0107a40 0x7ff1f0107e60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:13.001 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.293+0000 7ff1f739d700 1 -- 172.21.15.114:0/4183307468 >> 172.21.15.114:0/4183307468 conn(0x7ff1f0100040 msgr2=0x7ff1f01024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:13.001 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.294+0000 7ff1f739d700 1 -- 172.21.15.114:0/4183307468 shutdown_connections 2024-02-16T22:11:13.001 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.294+0000 7ff1f739d700 1 -- 172.21.15.114:0/4183307468 wait complete. 2024-02-16T22:11:13.001 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.294+0000 7ff1f739d700 1 Processor -- start 2024-02-16T22:11:13.001 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.294+0000 7ff1f739d700 1 -- start start 2024-02-16T22:11:13.001 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.294+0000 7ff1f739d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1f0107a40 0x7ff1f019a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:13.001 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.294+0000 7ff1f739d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f019a680 con 0x7ff1f0107a40 2024-02-16T22:11:13.001 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.294+0000 7ff1f5139700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1f0107a40 0x7ff1f019a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:13.001 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.294+0000 7ff1f5139700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1f0107a40 0x7ff1f019a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33664/0 (socket says 172.21.15.114:33664) 2024-02-16T22:11:13.001 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.294+0000 7ff1f5139700 1 -- 172.21.15.114:0/2430446371 learned_addr learned my addr 172.21.15.114:0/2430446371 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:13.001 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.295+0000 7ff1f5139700 1 -- 172.21.15.114:0/2430446371 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec009550 con 0x7ff1f0107a40 2024-02-16T22:11:13.002 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.295+0000 7ff1f5139700 1 --2- 172.21.15.114:0/2430446371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1f0107a40 0x7ff1f019a140 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec009520 tx=0x7ff1ec011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:13.002 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.295+0000 7ff1e67fc700 1 -- 172.21.15.114:0/2430446371 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff1ec003a80 con 0x7ff1f0107a40 2024-02-16T22:11:13.002 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.295+0000 7ff1e67fc700 1 -- 172.21.15.114:0/2430446371 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff1ec003be0 con 0x7ff1f0107a40 2024-02-16T22:11:13.002 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.295+0000 7ff1e67fc700 1 -- 172.21.15.114:0/2430446371 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff1ec0197b0 con 0x7ff1f0107a40 2024-02-16T22:11:13.002 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.295+0000 7ff1f739d700 1 -- 172.21.15.114:0/2430446371 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f019a880 con 0x7ff1f0107a40 2024-02-16T22:11:13.002 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.295+0000 7ff1f739d700 1 -- 172.21.15.114:0/2430446371 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f019ad20 con 0x7ff1f0107a40 2024-02-16T22:11:13.002 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.295+0000 7ff1e67fc700 1 -- 172.21.15.114:0/2430446371 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff1ec019d20 con 0x7ff1f0107a40 2024-02-16T22:11:13.002 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.295+0000 7ff1e67fc700 1 -- 172.21.15.114:0/2430446371 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff1ec024800 con 0x7ff1f0107a40 2024-02-16T22:11:13.002 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.296+0000 7ff1f739d700 1 -- 172.21.15.114:0/2430446371 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1f00623c0 con 0x7ff1f0107a40 2024-02-16T22:11:13.002 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.298+0000 7ff1e67fc700 1 -- 172.21.15.114:0/2430446371 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7ff1ec01ac30 con 0x7ff1f0107a40 2024-02-16T22:11:13.002 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.371+0000 7ff1f739d700 1 -- 172.21.15.114:0/2430446371 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ff1f019d6e0 con 0x7ff1f0107a40 2024-02-16T22:11:13.002 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.371+0000 7ff1e67fc700 1 -- 172.21.15.114:0/2430446371 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7ff1ec024a60 con 0x7ff1f0107a40 2024-02-16T22:11:13.002 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.372+0000 7ff1f739d700 1 -- 172.21.15.114:0/2430446371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1f0107a40 msgr2=0x7ff1f019a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:13.002 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.372+0000 7ff1f739d700 1 --2- 172.21.15.114:0/2430446371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1f0107a40 0x7ff1f019a140 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec009520 tx=0x7ff1ec011ab0 comp rx=0 tx=0).stop 2024-02-16T22:11:13.003 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.372+0000 7ff1f739d700 1 -- 172.21.15.114:0/2430446371 shutdown_connections 2024-02-16T22:11:13.003 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.372+0000 7ff1f739d700 1 --2- 172.21.15.114:0/2430446371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1f0107a40 0x7ff1f019a140 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:13.003 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.372+0000 7ff1f739d700 1 -- 172.21.15.114:0/2430446371 >> 172.21.15.114:0/2430446371 conn(0x7ff1f0100040 msgr2=0x7ff1f0101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:13.003 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.372+0000 7ff1f739d700 1 -- 172.21.15.114:0/2430446371 shutdown_connections 2024-02-16T22:11:13.003 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:12.372+0000 7ff1f739d700 1 -- 172.21.15.114:0/2430446371 wait complete. 2024-02-16T22:11:13.003 INFO:teuthology.orchestra.run.smithi114.stdout:mgr not available, waiting (2/15)... 2024-02-16T22:11:15.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:15 smithi114 ceph-mon[29235]: Activating manager daemon smithi114.vcullz 2024-02-16T22:11:15.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:15 smithi114 ceph-mon[29235]: mgrmap e2: smithi114.vcullz(active, starting, since 0.00341165s) 2024-02-16T22:11:15.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:15 smithi114 ceph-mon[29235]: from='mgr.14100 172.21.15.114:0/715545059' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T22:11:15.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:15 smithi114 ceph-mon[29235]: from='mgr.14100 172.21.15.114:0/715545059' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T22:11:15.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:15 smithi114 ceph-mon[29235]: from='mgr.14100 172.21.15.114:0/715545059' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T22:11:15.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:15 smithi114 ceph-mon[29235]: from='mgr.14100 172.21.15.114:0/715545059' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi114"}]: dispatch 2024-02-16T22:11:15.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:15 smithi114 ceph-mon[29235]: from='mgr.14100 172.21.15.114:0/715545059' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mgr metadata", "who": "smithi114.vcullz", "id": "smithi114.vcullz"}]: dispatch 2024-02-16T22:11:15.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:15 smithi114 ceph-mon[29235]: Manager daemon smithi114.vcullz is now available 2024-02-16T22:11:15.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:15 smithi114 ceph-mon[29235]: from='mgr.14100 172.21.15.114:0/715545059' entity='mgr.smithi114.vcullz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi114.vcullz/mirror_snapshot_schedule"}]: dispatch 2024-02-16T22:11:15.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:15 smithi114 ceph-mon[29235]: from='mgr.14100 172.21.15.114:0/715545059' entity='mgr.smithi114.vcullz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi114.vcullz/trash_purge_schedule"}]: dispatch 2024-02-16T22:11:15.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:15 smithi114 ceph-mon[29235]: from='mgr.14100 172.21.15.114:0/715545059' entity='mgr.smithi114.vcullz' 2024-02-16T22:11:15.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:15 smithi114 ceph-mon[29235]: from='mgr.14100 172.21.15.114:0/715545059' entity='mgr.smithi114.vcullz' 2024-02-16T22:11:16.086 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-02-16T22:11:16.086 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout { 2024-02-16T22:11:16.086 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "fsid": "1dcbaabe-cd18-11ee-95ba-87774f69a715", 2024-02-16T22:11:16.086 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "health": { 2024-02-16T22:11:16.086 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-16T22:11:16.086 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-16T22:11:16.086 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-16T22:11:16.086 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:11:16.086 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-16T22:11:16.086 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-16T22:11:16.086 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 0 2024-02-16T22:11:16.086 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout ], 2024-02-16T22:11:16.086 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-16T22:11:16.087 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "smithi114" 2024-02-16T22:11:16.087 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout ], 2024-02-16T22:11:16.087 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-02-16T22:11:16.087 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-16T22:11:16.087 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:11:16.087 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-02-16T22:11:16.087 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-16T22:11:16.087 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:11:16.087 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-16T22:11:16.087 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:11:16.087 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-16T22:11:16.087 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-16T22:11:16.087 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-16T22:11:16.095 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-16T22:11:16.095 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-16T22:11:16.095 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-16T22:11:16.095 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:11:16.095 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-16T22:11:16.095 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-16T22:11:16.096 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-16T22:11:16.096 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-16T22:11:16.096 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-16T22:11:16.096 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-16T22:11:16.096 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-16T22:11:16.096 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-16T22:11:16.096 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-16T22:11:16.096 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:11:16.096 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-16T22:11:16.096 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:11:16.096 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-16T22:11:16.096 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-16T22:11:16.096 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:11:16.097 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-16T22:11:16.097 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-16T22:11:16.097 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-16T22:11:16.097 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-16T22:11:16.097 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-16T22:11:16.097 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-16T22:11:16.097 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "restful" 2024-02-16T22:11:16.097 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout ], 2024-02-16T22:11:16.097 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-16T22:11:16.097 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:11:16.097 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-16T22:11:16.097 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:11:16.097 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "modified": "2024-02-16T22:11:02.186641+0000", 2024-02-16T22:11:16.097 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-16T22:11:16.098 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:11:16.098 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-16T22:11:16.098 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout } 2024-02-16T22:11:16.098 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.440+0000 7f7cd1dde700 1 Processor -- start 2024-02-16T22:11:16.098 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.440+0000 7f7cd1dde700 1 -- start start 2024-02-16T22:11:16.098 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.440+0000 7f7cd1dde700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ccc104d00 0x7f7ccc105120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:16.098 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.440+0000 7f7cd1dde700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc074620 con 0x7f7ccc104d00 2024-02-16T22:11:16.098 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.440+0000 7f7ccb7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ccc104d00 0x7f7ccc105120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:16.098 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.440+0000 7f7ccb7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ccc104d00 0x7f7ccc105120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33728/0 (socket says 172.21.15.114:33728) 2024-02-16T22:11:16.098 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.440+0000 7f7ccb7fe700 1 -- 172.21.15.114:0/1281144133 learned_addr learned my addr 172.21.15.114:0/1281144133 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:16.098 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.441+0000 7f7ccb7fe700 1 -- 172.21.15.114:0/1281144133 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ccc105660 con 0x7f7ccc104d00 2024-02-16T22:11:16.098 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.441+0000 7f7ccb7fe700 1 --2- 172.21.15.114:0/1281144133 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ccc104d00 0x7f7ccc105120 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc0098a0 tx=0x7f7cbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b57cce91551ba7 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:16.098 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.441+0000 7f7cca7fc700 1 -- 172.21.15.114:0/1281144133 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7cbc004180 con 0x7f7ccc104d00 2024-02-16T22:11:16.098 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.441+0000 7f7cca7fc700 1 -- 172.21.15.114:0/1281144133 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7cbc0042e0 con 0x7f7ccc104d00 2024-02-16T22:11:16.099 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.441+0000 7f7cca7fc700 1 -- 172.21.15.114:0/1281144133 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7cbc011a50 con 0x7f7ccc104d00 2024-02-16T22:11:16.099 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.441+0000 7f7cd1dde700 1 -- 172.21.15.114:0/1281144133 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ccc104d00 msgr2=0x7f7ccc105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:16.099 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.441+0000 7f7cd1dde700 1 --2- 172.21.15.114:0/1281144133 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ccc104d00 0x7f7ccc105120 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc0098a0 tx=0x7f7cbc009bb0 comp rx=0 tx=0).stop 2024-02-16T22:11:16.099 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.441+0000 7f7cd1dde700 1 -- 172.21.15.114:0/1281144133 shutdown_connections 2024-02-16T22:11:16.099 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.441+0000 7f7cd1dde700 1 --2- 172.21.15.114:0/1281144133 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ccc104d00 0x7f7ccc105120 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:16.099 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.441+0000 7f7cd1dde700 1 -- 172.21.15.114:0/1281144133 >> 172.21.15.114:0/1281144133 conn(0x7f7ccc1008c0 msgr2=0x7f7ccc102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:16.099 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.442+0000 7f7cd1dde700 1 -- 172.21.15.114:0/1281144133 shutdown_connections 2024-02-16T22:11:16.099 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.442+0000 7f7cd1dde700 1 -- 172.21.15.114:0/1281144133 wait complete. 2024-02-16T22:11:16.099 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.442+0000 7f7cd1dde700 1 Processor -- start 2024-02-16T22:11:16.099 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.442+0000 7f7cd1dde700 1 -- start start 2024-02-16T22:11:16.099 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.442+0000 7f7cd1dde700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ccc104d00 0x7f7ccc19c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:16.099 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.442+0000 7f7cd1dde700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc19c840 con 0x7f7ccc104d00 2024-02-16T22:11:16.099 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.442+0000 7f7ccb7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ccc104d00 0x7f7ccc19c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:16.099 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.443+0000 7f7ccb7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ccc104d00 0x7f7ccc19c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33744/0 (socket says 172.21.15.114:33744) 2024-02-16T22:11:16.100 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.443+0000 7f7ccb7fe700 1 -- 172.21.15.114:0/3469721311 learned_addr learned my addr 172.21.15.114:0/3469721311 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:16.100 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.443+0000 7f7ccb7fe700 1 -- 172.21.15.114:0/3469721311 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cbc009550 con 0x7f7ccc104d00 2024-02-16T22:11:16.100 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.443+0000 7f7ccb7fe700 1 --2- 172.21.15.114:0/3469721311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ccc104d00 0x7f7ccc19c300 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc003de0 tx=0x7f7cbc011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:16.100 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.443+0000 7f7cc8ff9700 1 -- 172.21.15.114:0/3469721311 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7cbc003930 con 0x7f7ccc104d00 2024-02-16T22:11:16.100 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.443+0000 7f7cd1dde700 1 -- 172.21.15.114:0/3469721311 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ccc19ca40 con 0x7f7ccc104d00 2024-02-16T22:11:16.100 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.443+0000 7f7cc8ff9700 1 -- 172.21.15.114:0/3469721311 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7cbc003a90 con 0x7f7ccc104d00 2024-02-16T22:11:16.100 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.443+0000 7f7cc8ff9700 1 -- 172.21.15.114:0/3469721311 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7cbc019730 con 0x7f7ccc104d00 2024-02-16T22:11:16.100 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.443+0000 7f7cd1dde700 1 -- 172.21.15.114:0/3469721311 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ccc19cee0 con 0x7f7ccc104d00 2024-02-16T22:11:16.100 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.444+0000 7f7cc8ff9700 1 -- 172.21.15.114:0/3469721311 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 2) v1 ==== 45039+0+0 (secure 0 0 0) 0x7f7cbc0199b0 con 0x7f7ccc104d00 2024-02-16T22:11:16.100 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.444+0000 7f7cd1dde700 1 -- 172.21.15.114:0/3469721311 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ccc195be0 con 0x7f7ccc104d00 2024-02-16T22:11:16.100 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.444+0000 7f7cc8ff9700 1 -- 172.21.15.114:0/3469721311 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7cbc04bf80 con 0x7f7ccc104d00 2024-02-16T22:11:16.100 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.446+0000 7f7cc8ff9700 1 -- 172.21.15.114:0/3469721311 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f7cbc0109d0 con 0x7f7ccc104d00 2024-02-16T22:11:16.100 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.516+0000 7f7cd1dde700 1 -- 172.21.15.114:0/3469721311 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f7ccc02cc70 con 0x7f7ccc104d00 2024-02-16T22:11:16.100 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.516+0000 7f7cc8ff9700 1 -- 172.21.15.114:0/3469721311 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f7cbc010380 con 0x7f7ccc104d00 2024-02-16T22:11:16.101 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.517+0000 7f7cd1dde700 1 -- 172.21.15.114:0/3469721311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ccc104d00 msgr2=0x7f7ccc19c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:16.101 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.517+0000 7f7cd1dde700 1 --2- 172.21.15.114:0/3469721311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ccc104d00 0x7f7ccc19c300 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc003de0 tx=0x7f7cbc011ab0 comp rx=0 tx=0).stop 2024-02-16T22:11:16.101 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.517+0000 7f7cd1dde700 1 -- 172.21.15.114:0/3469721311 shutdown_connections 2024-02-16T22:11:16.101 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.517+0000 7f7cd1dde700 1 --2- 172.21.15.114:0/3469721311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ccc104d00 0x7f7ccc19c300 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:16.101 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.517+0000 7f7cd1dde700 1 -- 172.21.15.114:0/3469721311 >> 172.21.15.114:0/3469721311 conn(0x7f7ccc1008c0 msgr2=0x7f7ccc1908c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:16.101 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.517+0000 7f7cd1dde700 1 -- 172.21.15.114:0/3469721311 shutdown_connections 2024-02-16T22:11:16.101 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:15.517+0000 7f7cd1dde700 1 -- 172.21.15.114:0/3469721311 wait complete. 2024-02-16T22:11:16.101 INFO:teuthology.orchestra.run.smithi114.stdout:mgr not available, waiting (3/15)... 2024-02-16T22:11:16.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:16 smithi114 ceph-mon[29235]: from='mgr.14100 172.21.15.114:0/715545059' entity='mgr.smithi114.vcullz' 2024-02-16T22:11:16.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:16 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/3469721311' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-16T22:11:16.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:16 smithi114 ceph-mon[29235]: mgrmap e3: smithi114.vcullz(active, since 1.00707s) 2024-02-16T22:11:19.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:18 smithi114 ceph-mon[29235]: mgrmap e4: smithi114.vcullz(active, since 2s) 2024-02-16T22:11:19.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:18 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/2500003919' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-16T22:11:19.292 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-02-16T22:11:19.292 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout { 2024-02-16T22:11:19.292 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "fsid": "1dcbaabe-cd18-11ee-95ba-87774f69a715", 2024-02-16T22:11:19.292 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "health": { 2024-02-16T22:11:19.292 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-16T22:11:19.293 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-16T22:11:19.293 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-16T22:11:19.293 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:11:19.293 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-16T22:11:19.293 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-16T22:11:19.293 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 0 2024-02-16T22:11:19.293 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout ], 2024-02-16T22:11:19.293 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-16T22:11:19.293 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "smithi114" 2024-02-16T22:11:19.293 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout ], 2024-02-16T22:11:19.293 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-02-16T22:11:19.293 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-16T22:11:19.293 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:11:19.294 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-02-16T22:11:19.294 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-16T22:11:19.294 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:11:19.294 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-16T22:11:19.294 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:11:19.294 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-16T22:11:19.294 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-16T22:11:19.294 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-16T22:11:19.294 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-16T22:11:19.294 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-16T22:11:19.294 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-16T22:11:19.294 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:11:19.294 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-16T22:11:19.294 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-16T22:11:19.295 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-16T22:11:19.295 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-16T22:11:19.295 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-16T22:11:19.295 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-16T22:11:19.295 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-16T22:11:19.295 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-16T22:11:19.295 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-16T22:11:19.295 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:11:19.295 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-16T22:11:19.295 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:11:19.295 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-16T22:11:19.295 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-16T22:11:19.295 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:11:19.295 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-16T22:11:19.295 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-16T22:11:19.296 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-16T22:11:19.296 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-16T22:11:19.296 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-16T22:11:19.296 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-16T22:11:19.296 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "restful" 2024-02-16T22:11:19.296 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout ], 2024-02-16T22:11:19.296 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-16T22:11:19.296 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:11:19.296 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-16T22:11:19.296 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T22:11:19.296 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "modified": "2024-02-16T22:11:02.186641+0000", 2024-02-16T22:11:19.296 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-16T22:11:19.296 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-02-16T22:11:19.296 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-16T22:11:19.297 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout } 2024-02-16T22:11:19.297 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.508+0000 7fe46f318700 1 Processor -- start 2024-02-16T22:11:19.297 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.508+0000 7fe46f318700 1 -- start start 2024-02-16T22:11:19.297 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.508+0000 7fe46f318700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe46807bef0 0x7fe46807a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:19.297 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.508+0000 7fe46f318700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46807a920 con 0x7fe46807bef0 2024-02-16T22:11:19.297 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.508+0000 7fe46d0b4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe46807bef0 0x7fe46807a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:19.297 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.508+0000 7fe46d0b4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe46807bef0 0x7fe46807a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33756/0 (socket says 172.21.15.114:33756) 2024-02-16T22:11:19.297 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.508+0000 7fe46d0b4700 1 -- 172.21.15.114:0/4175990414 learned_addr learned my addr 172.21.15.114:0/4175990414 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:19.297 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.508+0000 7fe46d0b4700 1 -- 172.21.15.114:0/4175990414 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46807aa60 con 0x7fe46807bef0 2024-02-16T22:11:19.297 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.509+0000 7fe46d0b4700 1 --2- 172.21.15.114:0/4175990414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe46807bef0 0x7fe46807a350 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe4600098a0 tx=0x7fe460009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94097351a0fa620 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:19.297 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.509+0000 7fe45ffff700 1 -- 172.21.15.114:0/4175990414 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe460004180 con 0x7fe46807bef0 2024-02-16T22:11:19.297 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.509+0000 7fe45ffff700 1 -- 172.21.15.114:0/4175990414 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe4600042e0 con 0x7fe46807bef0 2024-02-16T22:11:19.297 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.509+0000 7fe45ffff700 1 -- 172.21.15.114:0/4175990414 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe460011a50 con 0x7fe46807bef0 2024-02-16T22:11:19.297 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.509+0000 7fe46f318700 1 -- 172.21.15.114:0/4175990414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe46807bef0 msgr2=0x7fe46807a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:19.297 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.509+0000 7fe46f318700 1 --2- 172.21.15.114:0/4175990414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe46807bef0 0x7fe46807a350 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe4600098a0 tx=0x7fe460009bb0 comp rx=0 tx=0).stop 2024-02-16T22:11:19.298 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.509+0000 7fe46f318700 1 -- 172.21.15.114:0/4175990414 shutdown_connections 2024-02-16T22:11:19.298 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.509+0000 7fe46f318700 1 --2- 172.21.15.114:0/4175990414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe46807bef0 0x7fe46807a350 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:19.298 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.509+0000 7fe46f318700 1 -- 172.21.15.114:0/4175990414 >> 172.21.15.114:0/4175990414 conn(0x7fe468101a00 msgr2=0x7fe468103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:19.298 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.509+0000 7fe46f318700 1 -- 172.21.15.114:0/4175990414 shutdown_connections 2024-02-16T22:11:19.298 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.509+0000 7fe46f318700 1 -- 172.21.15.114:0/4175990414 wait complete. 2024-02-16T22:11:19.298 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.510+0000 7fe46f318700 1 Processor -- start 2024-02-16T22:11:19.298 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.510+0000 7fe46f318700 1 -- start start 2024-02-16T22:11:19.298 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.510+0000 7fe46f318700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe46807bef0 0x7fe4681a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:19.298 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.510+0000 7fe46f318700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4681a0c00 con 0x7fe46807bef0 2024-02-16T22:11:19.298 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.510+0000 7fe46d0b4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe46807bef0 0x7fe4681a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:19.298 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.510+0000 7fe46d0b4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe46807bef0 0x7fe4681a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33766/0 (socket says 172.21.15.114:33766) 2024-02-16T22:11:19.298 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.510+0000 7fe46d0b4700 1 -- 172.21.15.114:0/2500003919 learned_addr learned my addr 172.21.15.114:0/2500003919 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:19.298 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.511+0000 7fe46d0b4700 1 -- 172.21.15.114:0/2500003919 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe460009550 con 0x7fe46807bef0 2024-02-16T22:11:19.298 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.511+0000 7fe46d0b4700 1 --2- 172.21.15.114:0/2500003919 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe46807bef0 0x7fe4681a06c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe460000c00 tx=0x7fe460011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:19.299 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.511+0000 7fe45e7fc700 1 -- 172.21.15.114:0/2500003919 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe460003b90 con 0x7fe46807bef0 2024-02-16T22:11:19.299 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.511+0000 7fe46f318700 1 -- 172.21.15.114:0/2500003919 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4681a0e00 con 0x7fe46807bef0 2024-02-16T22:11:19.299 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.511+0000 7fe45e7fc700 1 -- 172.21.15.114:0/2500003919 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe4600248b0 con 0x7fe46807bef0 2024-02-16T22:11:19.299 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.511+0000 7fe45e7fc700 1 -- 172.21.15.114:0/2500003919 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe46001a8e0 con 0x7fe46807bef0 2024-02-16T22:11:19.299 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.511+0000 7fe46f318700 1 -- 172.21.15.114:0/2500003919 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4681a12a0 con 0x7fe46807bef0 2024-02-16T22:11:19.299 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.512+0000 7fe45e7fc700 1 -- 172.21.15.114:0/2500003919 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7fe46001aba0 con 0x7fe46807bef0 2024-02-16T22:11:19.299 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.512+0000 7fe45e7fc700 1 --2- 172.21.15.114:0/2500003919 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fe4540382d0 0x7fe45403a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:19.299 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.512+0000 7fe46f318700 1 -- 172.21.15.114:0/2500003919 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe468108330 con 0x7fe46807bef0 2024-02-16T22:11:19.299 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.512+0000 7fe45e7fc700 1 -- 172.21.15.114:0/2500003919 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe46004c1d0 con 0x7fe46807bef0 2024-02-16T22:11:19.299 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.512+0000 7fe46c8b3700 1 --2- 172.21.15.114:0/2500003919 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fe4540382d0 0x7fe45403a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:19.299 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.512+0000 7fe46c8b3700 1 --2- 172.21.15.114:0/2500003919 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fe4540382d0 0x7fe45403a790 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe458006fd0 tx=0x7fe458006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:19.299 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.517+0000 7fe45e7fc700 1 -- 172.21.15.114:0/2500003919 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe4600233a0 con 0x7fe46807bef0 2024-02-16T22:11:19.299 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.779+0000 7fe46f318700 1 -- 172.21.15.114:0/2500003919 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fe4681a4080 con 0x7fe46807bef0 2024-02-16T22:11:19.299 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.779+0000 7fe45e7fc700 1 -- 172.21.15.114:0/2500003919 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7fe4600233a0 con 0x7fe46807bef0 2024-02-16T22:11:19.300 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.782+0000 7fe46f318700 1 -- 172.21.15.114:0/2500003919 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fe4540382d0 msgr2=0x7fe45403a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:19.300 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.782+0000 7fe46f318700 1 --2- 172.21.15.114:0/2500003919 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fe4540382d0 0x7fe45403a790 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe458006fd0 tx=0x7fe458006e40 comp rx=0 tx=0).stop 2024-02-16T22:11:19.300 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.782+0000 7fe46f318700 1 -- 172.21.15.114:0/2500003919 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe46807bef0 msgr2=0x7fe4681a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:19.300 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.782+0000 7fe46f318700 1 --2- 172.21.15.114:0/2500003919 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe46807bef0 0x7fe4681a06c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe460000c00 tx=0x7fe460011ab0 comp rx=0 tx=0).stop 2024-02-16T22:11:19.300 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.782+0000 7fe46f318700 1 -- 172.21.15.114:0/2500003919 shutdown_connections 2024-02-16T22:11:19.300 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.782+0000 7fe46f318700 1 --2- 172.21.15.114:0/2500003919 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fe4540382d0 0x7fe45403a790 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:19.300 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.782+0000 7fe46f318700 1 --2- 172.21.15.114:0/2500003919 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe46807bef0 0x7fe4681a06c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:19.300 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.782+0000 7fe46f318700 1 -- 172.21.15.114:0/2500003919 >> 172.21.15.114:0/2500003919 conn(0x7fe468101a00 msgr2=0x7fe4681075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:19.300 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.782+0000 7fe46f318700 1 -- 172.21.15.114:0/2500003919 shutdown_connections 2024-02-16T22:11:19.300 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:18.782+0000 7fe46f318700 1 -- 172.21.15.114:0/2500003919 wait complete. 2024-02-16T22:11:19.300 INFO:teuthology.orchestra.run.smithi114.stdout:mgr is available 2024-02-16T22:11:20.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:19 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/2794010883' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-16T22:11:20.453 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-02-16T22:11:20.453 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout [global] 2024-02-16T22:11:20.453 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout fsid = 1dcbaabe-cd18-11ee-95ba-87774f69a715 2024-02-16T22:11:20.453 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-16T22:11:20.453 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-16T22:11:20.453 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-16T22:11:20.453 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-16T22:11:20.453 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-02-16T22:11:20.453 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-16T22:11:20.453 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-16T22:11:20.453 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-02-16T22:11:20.453 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout [osd] 2024-02-16T22:11:20.454 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-16T22:11:20.454 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-16T22:11:20.454 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-16T22:11:20.454 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-16T22:11:20.454 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-16T22:11:20.454 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.729+0000 7fc361d1d700 1 Processor -- start 2024-02-16T22:11:20.454 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.729+0000 7fc361d1d700 1 -- start start 2024-02-16T22:11:20.454 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.729+0000 7fc361d1d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc35c106420 0x7fc35c106840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:20.454 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.729+0000 7fc361d1d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c106e10 con 0x7fc35c106420 2024-02-16T22:11:20.454 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.729+0000 7fc35b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc35c106420 0x7fc35c106840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:20.454 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.729+0000 7fc35b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc35c106420 0x7fc35c106840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33776/0 (socket says 172.21.15.114:33776) 2024-02-16T22:11:20.454 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.729+0000 7fc35b7fe700 1 -- 172.21.15.114:0/2423736341 learned_addr learned my addr 172.21.15.114:0/2423736341 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:20.454 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.729+0000 7fc35b7fe700 1 -- 172.21.15.114:0/2423736341 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35c107620 con 0x7fc35c106420 2024-02-16T22:11:20.454 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.729+0000 7fc35b7fe700 1 --2- 172.21.15.114:0/2423736341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc35c106420 0x7fc35c106840 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc34c0098a0 tx=0x7fc34c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fbad30245fdb59c server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:20.455 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.730+0000 7fc35affd700 1 -- 172.21.15.114:0/2423736341 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc34c004180 con 0x7fc35c106420 2024-02-16T22:11:20.455 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.730+0000 7fc35affd700 1 -- 172.21.15.114:0/2423736341 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc34c0042e0 con 0x7fc35c106420 2024-02-16T22:11:20.455 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.730+0000 7fc35affd700 1 -- 172.21.15.114:0/2423736341 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc34c011a50 con 0x7fc35c106420 2024-02-16T22:11:20.455 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.730+0000 7fc361d1d700 1 -- 172.21.15.114:0/2423736341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc35c106420 msgr2=0x7fc35c106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:20.455 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.730+0000 7fc361d1d700 1 --2- 172.21.15.114:0/2423736341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc35c106420 0x7fc35c106840 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc34c0098a0 tx=0x7fc34c009bb0 comp rx=0 tx=0).stop 2024-02-16T22:11:20.455 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.730+0000 7fc361d1d700 1 -- 172.21.15.114:0/2423736341 shutdown_connections 2024-02-16T22:11:20.455 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.730+0000 7fc361d1d700 1 --2- 172.21.15.114:0/2423736341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc35c106420 0x7fc35c106840 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:20.455 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.730+0000 7fc361d1d700 1 -- 172.21.15.114:0/2423736341 >> 172.21.15.114:0/2423736341 conn(0x7fc35c101a00 msgr2=0x7fc35c103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:20.455 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.730+0000 7fc361d1d700 1 -- 172.21.15.114:0/2423736341 shutdown_connections 2024-02-16T22:11:20.455 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.730+0000 7fc361d1d700 1 -- 172.21.15.114:0/2423736341 wait complete. 2024-02-16T22:11:20.455 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.731+0000 7fc361d1d700 1 Processor -- start 2024-02-16T22:11:20.455 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.731+0000 7fc361d1d700 1 -- start start 2024-02-16T22:11:20.455 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.731+0000 7fc361d1d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc35c106420 0x7fc35c19a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:20.455 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.731+0000 7fc361d1d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c19a5f0 con 0x7fc35c106420 2024-02-16T22:11:20.455 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.731+0000 7fc35b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc35c106420 0x7fc35c19a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:20.455 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.731+0000 7fc35b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc35c106420 0x7fc35c19a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33778/0 (socket says 172.21.15.114:33778) 2024-02-16T22:11:20.456 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.731+0000 7fc35b7fe700 1 -- 172.21.15.114:0/2794010883 learned_addr learned my addr 172.21.15.114:0/2794010883 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:20.456 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.731+0000 7fc35b7fe700 1 -- 172.21.15.114:0/2794010883 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc34c009550 con 0x7fc35c106420 2024-02-16T22:11:20.456 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.731+0000 7fc35b7fe700 1 --2- 172.21.15.114:0/2794010883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc35c106420 0x7fc35c19a0b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc34c000c00 tx=0x7fc34c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:20.456 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.732+0000 7fc3597fa700 1 -- 172.21.15.114:0/2794010883 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc34c003b90 con 0x7fc35c106420 2024-02-16T22:11:20.456 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.732+0000 7fc3597fa700 1 -- 172.21.15.114:0/2794010883 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc34c0248b0 con 0x7fc35c106420 2024-02-16T22:11:20.456 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.732+0000 7fc361d1d700 1 -- 172.21.15.114:0/2794010883 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc35c19a7f0 con 0x7fc35c106420 2024-02-16T22:11:20.456 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.732+0000 7fc3597fa700 1 -- 172.21.15.114:0/2794010883 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc34c01a8e0 con 0x7fc35c106420 2024-02-16T22:11:20.456 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.732+0000 7fc361d1d700 1 -- 172.21.15.114:0/2794010883 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc35c19ac90 con 0x7fc35c106420 2024-02-16T22:11:20.456 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.732+0000 7fc3597fa700 1 -- 172.21.15.114:0/2794010883 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7fc34c01ab50 con 0x7fc35c106420 2024-02-16T22:11:20.456 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.733+0000 7fc3597fa700 1 --2- 172.21.15.114:0/2794010883 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fc348038320 0x7fc34803a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:20.456 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.733+0000 7fc361d1d700 1 -- 172.21.15.114:0/2794010883 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc35c0623c0 con 0x7fc35c106420 2024-02-16T22:11:20.456 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.733+0000 7fc3597fa700 1 -- 172.21.15.114:0/2794010883 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc34c04c140 con 0x7fc35c106420 2024-02-16T22:11:20.457 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.733+0000 7fc353fff700 1 --2- 172.21.15.114:0/2794010883 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fc348038320 0x7fc34803a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:20.457 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.733+0000 7fc353fff700 1 --2- 172.21.15.114:0/2794010883 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fc348038320 0x7fc34803a7e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc344006fd0 tx=0x7fc344006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:20.457 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.737+0000 7fc3597fa700 1 -- 172.21.15.114:0/2794010883 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc34c0189a0 con 0x7fc35c106420 2024-02-16T22:11:20.457 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.926+0000 7fc361d1d700 1 -- 172.21.15.114:0/2794010883 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fc35c19da70 con 0x7fc35c106420 2024-02-16T22:11:20.457 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.926+0000 7fc3597fa700 1 -- 172.21.15.114:0/2794010883 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fc34c018350 con 0x7fc35c106420 2024-02-16T22:11:20.457 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.928+0000 7fc361d1d700 1 -- 172.21.15.114:0/2794010883 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fc348038320 msgr2=0x7fc34803a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:20.457 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.928+0000 7fc361d1d700 1 --2- 172.21.15.114:0/2794010883 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fc348038320 0x7fc34803a7e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc344006fd0 tx=0x7fc344006e40 comp rx=0 tx=0).stop 2024-02-16T22:11:20.457 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.928+0000 7fc361d1d700 1 -- 172.21.15.114:0/2794010883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc35c106420 msgr2=0x7fc35c19a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:20.457 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.928+0000 7fc361d1d700 1 --2- 172.21.15.114:0/2794010883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc35c106420 0x7fc35c19a0b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc34c000c00 tx=0x7fc34c011ab0 comp rx=0 tx=0).stop 2024-02-16T22:11:20.457 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.929+0000 7fc361d1d700 1 -- 172.21.15.114:0/2794010883 shutdown_connections 2024-02-16T22:11:20.457 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.929+0000 7fc361d1d700 1 --2- 172.21.15.114:0/2794010883 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fc348038320 0x7fc34803a7e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:20.457 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.929+0000 7fc361d1d700 1 --2- 172.21.15.114:0/2794010883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc35c106420 0x7fc35c19a0b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:20.457 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.929+0000 7fc361d1d700 1 -- 172.21.15.114:0/2794010883 >> 172.21.15.114:0/2794010883 conn(0x7fc35c101a00 msgr2=0x7fc35c1027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:20.457 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.929+0000 7fc361d1d700 1 -- 172.21.15.114:0/2794010883 shutdown_connections 2024-02-16T22:11:20.458 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:19.929+0000 7fc361d1d700 1 -- 172.21.15.114:0/2794010883 wait complete. 2024-02-16T22:11:20.458 INFO:teuthology.orchestra.run.smithi114.stdout:Enabling cephadm module... 2024-02-16T22:11:21.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:21 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/438189615' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-16T22:11:21.758 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.885+0000 7fc2d8be0700 1 Processor -- start 2024-02-16T22:11:21.758 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.885+0000 7fc2d8be0700 1 -- start start 2024-02-16T22:11:21.758 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.885+0000 7fc2d8be0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2d4106ea0 0x7fc2d4109290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:21.758 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.885+0000 7fc2d8be0700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d4079ec0 con 0x7fc2d4106ea0 2024-02-16T22:11:21.758 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.885+0000 7fc2d259c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2d4106ea0 0x7fc2d4109290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:21.758 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.885+0000 7fc2d259c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2d4106ea0 0x7fc2d4109290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33792/0 (socket says 172.21.15.114:33792) 2024-02-16T22:11:21.758 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.885+0000 7fc2d259c700 1 -- 172.21.15.114:0/1365424433 learned_addr learned my addr 172.21.15.114:0/1365424433 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:21.759 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.886+0000 7fc2d259c700 1 -- 172.21.15.114:0/1365424433 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d41097d0 con 0x7fc2d4106ea0 2024-02-16T22:11:21.759 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.886+0000 7fc2d259c700 1 --2- 172.21.15.114:0/1365424433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2d4106ea0 0x7fc2d4109290 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc2c40098a0 tx=0x7fc2c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d9b3431fabe64 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:21.759 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.886+0000 7fc2d1d9b700 1 -- 172.21.15.114:0/1365424433 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc2c4004180 con 0x7fc2d4106ea0 2024-02-16T22:11:21.759 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.886+0000 7fc2d1d9b700 1 -- 172.21.15.114:0/1365424433 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc2c40042e0 con 0x7fc2d4106ea0 2024-02-16T22:11:21.759 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.886+0000 7fc2d1d9b700 1 -- 172.21.15.114:0/1365424433 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc2c4011a50 con 0x7fc2d4106ea0 2024-02-16T22:11:21.759 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.886+0000 7fc2d8be0700 1 -- 172.21.15.114:0/1365424433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2d4106ea0 msgr2=0x7fc2d4109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:21.759 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.886+0000 7fc2d8be0700 1 --2- 172.21.15.114:0/1365424433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2d4106ea0 0x7fc2d4109290 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc2c40098a0 tx=0x7fc2c4009bb0 comp rx=0 tx=0).stop 2024-02-16T22:11:21.759 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.887+0000 7fc2d8be0700 1 -- 172.21.15.114:0/1365424433 shutdown_connections 2024-02-16T22:11:21.759 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.887+0000 7fc2d8be0700 1 --2- 172.21.15.114:0/1365424433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2d4106ea0 0x7fc2d4109290 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:21.759 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.887+0000 7fc2d8be0700 1 -- 172.21.15.114:0/1365424433 >> 172.21.15.114:0/1365424433 conn(0x7fc2d41008b0 msgr2=0x7fc2d4102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:21.759 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.887+0000 7fc2d8be0700 1 -- 172.21.15.114:0/1365424433 shutdown_connections 2024-02-16T22:11:21.759 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.887+0000 7fc2d8be0700 1 -- 172.21.15.114:0/1365424433 wait complete. 2024-02-16T22:11:21.759 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.887+0000 7fc2d8be0700 1 Processor -- start 2024-02-16T22:11:21.759 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.888+0000 7fc2d8be0700 1 -- start start 2024-02-16T22:11:21.760 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.888+0000 7fc2d8be0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2d4106ea0 0x7fc2d419a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:21.760 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.888+0000 7fc2d8be0700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d419a640 con 0x7fc2d4106ea0 2024-02-16T22:11:21.760 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.888+0000 7fc2d259c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2d4106ea0 0x7fc2d419a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:21.760 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.888+0000 7fc2d259c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2d4106ea0 0x7fc2d419a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33804/0 (socket says 172.21.15.114:33804) 2024-02-16T22:11:21.760 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.888+0000 7fc2d259c700 1 -- 172.21.15.114:0/438189615 learned_addr learned my addr 172.21.15.114:0/438189615 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:21.760 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.888+0000 7fc2d259c700 1 -- 172.21.15.114:0/438189615 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c4009550 con 0x7fc2d4106ea0 2024-02-16T22:11:21.760 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.888+0000 7fc2d259c700 1 --2- 172.21.15.114:0/438189615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2d4106ea0 0x7fc2d419a100 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4003de0 tx=0x7fc2c4011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:21.760 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.888+0000 7fc2cbfff700 1 -- 172.21.15.114:0/438189615 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc2c4003930 con 0x7fc2d4106ea0 2024-02-16T22:11:21.760 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.888+0000 7fc2cbfff700 1 -- 172.21.15.114:0/438189615 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc2c4003a90 con 0x7fc2d4106ea0 2024-02-16T22:11:21.760 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.888+0000 7fc2d8be0700 1 -- 172.21.15.114:0/438189615 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2d419a840 con 0x7fc2d4106ea0 2024-02-16T22:11:21.760 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.888+0000 7fc2cbfff700 1 -- 172.21.15.114:0/438189615 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc2c4019730 con 0x7fc2d4106ea0 2024-02-16T22:11:21.760 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.888+0000 7fc2d8be0700 1 -- 172.21.15.114:0/438189615 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2d419ace0 con 0x7fc2d4106ea0 2024-02-16T22:11:21.760 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.889+0000 7fc2cbfff700 1 -- 172.21.15.114:0/438189615 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7fc2c40199b0 con 0x7fc2d4106ea0 2024-02-16T22:11:21.760 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.889+0000 7fc2d8be0700 1 -- 172.21.15.114:0/438189615 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2d410a550 con 0x7fc2d4106ea0 2024-02-16T22:11:21.761 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.889+0000 7fc2cbfff700 1 --2- 172.21.15.114:0/438189615 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fc2c00382d0 0x7fc2c003a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:21.761 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.890+0000 7fc2cbfff700 1 -- 172.21.15.114:0/438189615 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc2c404c080 con 0x7fc2d4106ea0 2024-02-16T22:11:21.761 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.890+0000 7fc2c9d9b700 1 --2- 172.21.15.114:0/438189615 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fc2c00382d0 0x7fc2c003a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:21.761 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.890+0000 7fc2c9d9b700 1 --2- 172.21.15.114:0/438189615 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fc2c00382d0 0x7fc2c003a790 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc006fd0 tx=0x7fc2bc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:21.761 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:20.894+0000 7fc2cbfff700 1 -- 172.21.15.114:0/438189615 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc2c407f0e0 con 0x7fc2d4106ea0 2024-02-16T22:11:21.761 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:21.135+0000 7fc2d8be0700 1 -- 172.21.15.114:0/438189615 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fc2d410a770 con 0x7fc2d4106ea0 2024-02-16T22:11:21.761 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:21.188+0000 7fc2cbfff700 1 -- 172.21.15.114:0/438189615 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fc2c400fd60 con 0x7fc2d4106ea0 2024-02-16T22:11:21.761 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:21.189+0000 7fc2cbfff700 1 -- 172.21.15.114:0/438189615 <== mon.0 v2:172.21.15.114:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fc2c404fb90 con 0x7fc2d4106ea0 2024-02-16T22:11:21.761 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:21.191+0000 7fc2d8be0700 1 -- 172.21.15.114:0/438189615 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fc2c00382d0 msgr2=0x7fc2c003a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:21.761 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:21.191+0000 7fc2d8be0700 1 --2- 172.21.15.114:0/438189615 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fc2c00382d0 0x7fc2c003a790 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc006fd0 tx=0x7fc2bc006e40 comp rx=0 tx=0).stop 2024-02-16T22:11:21.761 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:21.191+0000 7fc2d8be0700 1 -- 172.21.15.114:0/438189615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2d4106ea0 msgr2=0x7fc2d419a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:21.761 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:21.191+0000 7fc2d8be0700 1 --2- 172.21.15.114:0/438189615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2d4106ea0 0x7fc2d419a100 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4003de0 tx=0x7fc2c4011ab0 comp rx=0 tx=0).stop 2024-02-16T22:11:21.762 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:21.191+0000 7fc2d8be0700 1 -- 172.21.15.114:0/438189615 shutdown_connections 2024-02-16T22:11:21.762 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:21.191+0000 7fc2d8be0700 1 --2- 172.21.15.114:0/438189615 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fc2c00382d0 0x7fc2c003a790 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:21.762 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:21.191+0000 7fc2d8be0700 1 --2- 172.21.15.114:0/438189615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2d4106ea0 0x7fc2d419a100 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:21.762 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:21.191+0000 7fc2d8be0700 1 -- 172.21.15.114:0/438189615 >> 172.21.15.114:0/438189615 conn(0x7fc2d41008b0 msgr2=0x7fc2d4101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:21.762 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:21.192+0000 7fc2d8be0700 1 -- 172.21.15.114:0/438189615 shutdown_connections 2024-02-16T22:11:21.762 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:21.192+0000 7fc2d8be0700 1 -- 172.21.15.114:0/438189615 wait complete. 2024-02-16T22:11:22.414 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:22 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/438189615' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-16T22:11:22.414 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:22 smithi114 ceph-mon[29235]: mgrmap e5: smithi114.vcullz(active, since 5s) 2024-02-16T22:11:23.124 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout { 2024-02-16T22:11:23.124 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-16T22:11:23.124 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-16T22:11:23.124 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "active_name": "smithi114.vcullz", 2024-02-16T22:11:23.124 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-16T22:11:23.124 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout } 2024-02-16T22:11:23.124 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.207+0000 7fe4d2b88700 1 Processor -- start 2024-02-16T22:11:23.124 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.207+0000 7fe4d2b88700 1 -- start start 2024-02-16T22:11:23.124 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.207+0000 7fe4d2b88700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4cc105840 0x7fe4cc105c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:23.124 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.207+0000 7fe4d2b88700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc106230 con 0x7fe4cc105840 2024-02-16T22:11:23.125 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.207+0000 7fe4d0924700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4cc105840 0x7fe4cc105c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:23.125 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.207+0000 7fe4d0924700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4cc105840 0x7fe4cc105c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42904/0 (socket says 172.21.15.114:42904) 2024-02-16T22:11:23.125 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.207+0000 7fe4d0924700 1 -- 172.21.15.114:0/1654892652 learned_addr learned my addr 172.21.15.114:0/1654892652 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:23.125 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.207+0000 7fe4d0924700 1 -- 172.21.15.114:0/1654892652 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4cc106560 con 0x7fe4cc105840 2024-02-16T22:11:23.125 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.207+0000 7fe4d0924700 1 --2- 172.21.15.114:0/1654892652 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4cc105840 0x7fe4cc105c60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe4c00098a0 tx=0x7fe4c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b6b90c53865d2e server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:23.125 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.208+0000 7fe4cb7fe700 1 -- 172.21.15.114:0/1654892652 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe4c0004180 con 0x7fe4cc105840 2024-02-16T22:11:23.125 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.208+0000 7fe4cb7fe700 1 -- 172.21.15.114:0/1654892652 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe4c00042e0 con 0x7fe4cc105840 2024-02-16T22:11:23.125 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.208+0000 7fe4cb7fe700 1 -- 172.21.15.114:0/1654892652 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe4c0011a50 con 0x7fe4cc105840 2024-02-16T22:11:23.125 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.208+0000 7fe4d2b88700 1 -- 172.21.15.114:0/1654892652 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4cc105840 msgr2=0x7fe4cc105c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:23.125 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.208+0000 7fe4d2b88700 1 --2- 172.21.15.114:0/1654892652 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4cc105840 0x7fe4cc105c60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe4c00098a0 tx=0x7fe4c0009bb0 comp rx=0 tx=0).stop 2024-02-16T22:11:23.125 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.208+0000 7fe4d2b88700 1 -- 172.21.15.114:0/1654892652 shutdown_connections 2024-02-16T22:11:23.125 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.208+0000 7fe4d2b88700 1 --2- 172.21.15.114:0/1654892652 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4cc105840 0x7fe4cc105c60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:23.125 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.208+0000 7fe4d2b88700 1 -- 172.21.15.114:0/1654892652 >> 172.21.15.114:0/1654892652 conn(0x7fe4cc100040 msgr2=0x7fe4cc1024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:23.126 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.208+0000 7fe4d2b88700 1 -- 172.21.15.114:0/1654892652 shutdown_connections 2024-02-16T22:11:23.126 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.208+0000 7fe4d2b88700 1 -- 172.21.15.114:0/1654892652 wait complete. 2024-02-16T22:11:23.126 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.209+0000 7fe4d2b88700 1 Processor -- start 2024-02-16T22:11:23.126 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.209+0000 7fe4d2b88700 1 -- start start 2024-02-16T22:11:23.126 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.209+0000 7fe4d2b88700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4cc105840 0x7fe4cc19a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:23.126 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.209+0000 7fe4d2b88700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc19a670 con 0x7fe4cc105840 2024-02-16T22:11:23.126 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.209+0000 7fe4d0924700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4cc105840 0x7fe4cc19a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:23.126 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.209+0000 7fe4d0924700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4cc105840 0x7fe4cc19a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42908/0 (socket says 172.21.15.114:42908) 2024-02-16T22:11:23.126 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.209+0000 7fe4d0924700 1 -- 172.21.15.114:0/3773231167 learned_addr learned my addr 172.21.15.114:0/3773231167 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:23.126 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.209+0000 7fe4d0924700 1 -- 172.21.15.114:0/3773231167 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c0009550 con 0x7fe4cc105840 2024-02-16T22:11:23.126 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.210+0000 7fe4d0924700 1 --2- 172.21.15.114:0/3773231167 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4cc105840 0x7fe4cc19a130 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0003f80 tx=0x7fe4c0011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:23.126 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.210+0000 7fe4c9ffb700 1 -- 172.21.15.114:0/3773231167 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe4c0003a80 con 0x7fe4cc105840 2024-02-16T22:11:23.126 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.210+0000 7fe4d2b88700 1 -- 172.21.15.114:0/3773231167 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4cc19a870 con 0x7fe4cc105840 2024-02-16T22:11:23.126 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.210+0000 7fe4c9ffb700 1 -- 172.21.15.114:0/3773231167 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe4c0003be0 con 0x7fe4cc105840 2024-02-16T22:11:23.126 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.210+0000 7fe4c9ffb700 1 -- 172.21.15.114:0/3773231167 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe4c0019820 con 0x7fe4cc105840 2024-02-16T22:11:23.127 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.210+0000 7fe4d2b88700 1 -- 172.21.15.114:0/3773231167 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4cc19ad10 con 0x7fe4cc105840 2024-02-16T22:11:23.127 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.211+0000 7fe4d2b88700 1 -- 172.21.15.114:0/3773231167 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4cc0623c0 con 0x7fe4cc105840 2024-02-16T22:11:23.127 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.211+0000 7fe4c9ffb700 1 -- 172.21.15.114:0/3773231167 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fe4c0019aa0 con 0x7fe4cc105840 2024-02-16T22:11:23.127 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.211+0000 7fe4c9ffb700 1 --2- 172.21.15.114:0/3773231167 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fe4bc0382d0 0x7fe4bc03a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:23.127 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.211+0000 7fe4c9ffb700 1 -- 172.21.15.114:0/3773231167 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe4c004c5d0 con 0x7fe4cc105840 2024-02-16T22:11:23.127 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.211+0000 7fe4cbfff700 1 -- 172.21.15.114:0/3773231167 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fe4bc0382d0 msgr2=0x7fe4bc03a790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/2055574123 2024-02-16T22:11:23.127 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.211+0000 7fe4cbfff700 1 --2- 172.21.15.114:0/3773231167 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fe4bc0382d0 0x7fe4bc03a790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T22:11:23.127 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.217+0000 7fe4c9ffb700 1 -- 172.21.15.114:0/3773231167 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe4c002b750 con 0x7fe4cc105840 2024-02-16T22:11:23.127 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.412+0000 7fe4cbfff700 1 -- 172.21.15.114:0/3773231167 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fe4bc0382d0 msgr2=0x7fe4bc03a790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/2055574123 2024-02-16T22:11:23.127 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.412+0000 7fe4cbfff700 1 --2- 172.21.15.114:0/3773231167 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fe4bc0382d0 0x7fe4bc03a790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-16T22:11:23.127 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.467+0000 7fe4d2b88700 1 -- 172.21.15.114:0/3773231167 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fe4cc19de80 con 0x7fe4cc105840 2024-02-16T22:11:23.127 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.468+0000 7fe4c9ffb700 1 -- 172.21.15.114:0/3773231167 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7fe4c00172b0 con 0x7fe4cc105840 2024-02-16T22:11:23.127 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.470+0000 7fe4d2b88700 1 -- 172.21.15.114:0/3773231167 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fe4bc0382d0 msgr2=0x7fe4bc03a790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T22:11:23.128 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.470+0000 7fe4d2b88700 1 --2- 172.21.15.114:0/3773231167 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fe4bc0382d0 0x7fe4bc03a790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:23.128 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.470+0000 7fe4d2b88700 1 -- 172.21.15.114:0/3773231167 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4cc105840 msgr2=0x7fe4cc19a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:23.128 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.470+0000 7fe4d2b88700 1 --2- 172.21.15.114:0/3773231167 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4cc105840 0x7fe4cc19a130 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0003f80 tx=0x7fe4c0011ab0 comp rx=0 tx=0).stop 2024-02-16T22:11:23.128 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.470+0000 7fe4d2b88700 1 -- 172.21.15.114:0/3773231167 shutdown_connections 2024-02-16T22:11:23.128 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.470+0000 7fe4d2b88700 1 --2- 172.21.15.114:0/3773231167 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fe4bc0382d0 0x7fe4bc03a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:23.128 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.470+0000 7fe4d2b88700 1 --2- 172.21.15.114:0/3773231167 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4cc105840 0x7fe4cc19a130 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:23.128 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.470+0000 7fe4d2b88700 1 -- 172.21.15.114:0/3773231167 >> 172.21.15.114:0/3773231167 conn(0x7fe4cc100040 msgr2=0x7fe4cc104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:23.128 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.470+0000 7fe4d2b88700 1 -- 172.21.15.114:0/3773231167 shutdown_connections 2024-02-16T22:11:23.128 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:22.470+0000 7fe4d2b88700 1 -- 172.21.15.114:0/3773231167 wait complete. 2024-02-16T22:11:23.128 INFO:teuthology.orchestra.run.smithi114.stdout:Waiting for the mgr to restart... 2024-02-16T22:11:23.128 INFO:teuthology.orchestra.run.smithi114.stdout:Waiting for mgr epoch 5... 2024-02-16T22:11:23.367 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:23 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/3773231167' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-16T22:11:28.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:27 smithi114 ceph-mon[29235]: Active manager daemon smithi114.vcullz restarted 2024-02-16T22:11:28.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:27 smithi114 ceph-mon[29235]: Activating manager daemon smithi114.vcullz 2024-02-16T22:11:28.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:27 smithi114 ceph-mon[29235]: osdmap e2: 0 total, 0 up, 0 in 2024-02-16T22:11:28.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:27 smithi114 ceph-mon[29235]: mgrmap e6: smithi114.vcullz(active, starting, since 0.00307656s) 2024-02-16T22:11:28.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:27 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi114"}]: dispatch 2024-02-16T22:11:28.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:27 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mgr metadata", "who": "smithi114.vcullz", "id": "smithi114.vcullz"}]: dispatch 2024-02-16T22:11:28.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:27 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T22:11:28.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:27 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T22:11:28.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:27 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T22:11:28.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:27 smithi114 ceph-mon[29235]: Manager daemon smithi114.vcullz is now available 2024-02-16T22:11:28.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:27 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' 2024-02-16T22:11:28.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:27 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' 2024-02-16T22:11:28.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:27 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:11:28.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:27 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:11:28.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:27 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:11:29.122 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:28 smithi114 ceph-mon[29235]: Found migration_current of "None". Setting to last migration. 2024-02-16T22:11:29.122 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:28 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi114.vcullz/mirror_snapshot_schedule"}]: dispatch 2024-02-16T22:11:29.122 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:28 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi114.vcullz/trash_purge_schedule"}]: dispatch 2024-02-16T22:11:29.122 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:28 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' 2024-02-16T22:11:29.122 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:28 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' 2024-02-16T22:11:29.122 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:28 smithi114 ceph-mon[29235]: mgrmap e7: smithi114.vcullz(active, since 1.00565s) 2024-02-16T22:11:29.411 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout { 2024-02-16T22:11:29.412 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-16T22:11:29.412 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-16T22:11:29.412 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout } 2024-02-16T22:11:29.412 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.524+0000 7fed10889700 1 Processor -- start 2024-02-16T22:11:29.412 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.524+0000 7fed10889700 1 -- start start 2024-02-16T22:11:29.412 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.524+0000 7fed10889700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed0c104d10 0x7fed0c105130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:29.412 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.524+0000 7fed10889700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0c079e40 con 0x7fed0c104d10 2024-02-16T22:11:29.412 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.525+0000 7fed0a59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed0c104d10 0x7fed0c105130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:29.412 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.525+0000 7fed0a59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed0c104d10 0x7fed0c105130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42914/0 (socket says 172.21.15.114:42914) 2024-02-16T22:11:29.412 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.525+0000 7fed0a59c700 1 -- 172.21.15.114:0/1815377251 learned_addr learned my addr 172.21.15.114:0/1815377251 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:29.412 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.525+0000 7fed0a59c700 1 -- 172.21.15.114:0/1815377251 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed0c105670 con 0x7fed0c104d10 2024-02-16T22:11:29.412 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.525+0000 7fed0a59c700 1 --2- 172.21.15.114:0/1815377251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed0c104d10 0x7fed0c105130 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fecfc0098a0 tx=0x7fecfc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52ba2a0b2d82c91 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:29.413 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.525+0000 7fed0959a700 1 -- 172.21.15.114:0/1815377251 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fecfc004180 con 0x7fed0c104d10 2024-02-16T22:11:29.413 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.525+0000 7fed0959a700 1 -- 172.21.15.114:0/1815377251 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fecfc0042e0 con 0x7fed0c104d10 2024-02-16T22:11:29.413 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.525+0000 7fed0959a700 1 -- 172.21.15.114:0/1815377251 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fecfc011a50 con 0x7fed0c104d10 2024-02-16T22:11:29.413 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.526+0000 7fed10889700 1 -- 172.21.15.114:0/1815377251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed0c104d10 msgr2=0x7fed0c105130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:29.413 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.526+0000 7fed10889700 1 --2- 172.21.15.114:0/1815377251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed0c104d10 0x7fed0c105130 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fecfc0098a0 tx=0x7fecfc009bb0 comp rx=0 tx=0).stop 2024-02-16T22:11:29.413 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.526+0000 7fed10889700 1 -- 172.21.15.114:0/1815377251 shutdown_connections 2024-02-16T22:11:29.413 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.526+0000 7fed10889700 1 --2- 172.21.15.114:0/1815377251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed0c104d10 0x7fed0c105130 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:29.413 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.526+0000 7fed10889700 1 -- 172.21.15.114:0/1815377251 >> 172.21.15.114:0/1815377251 conn(0x7fed0c1008b0 msgr2=0x7fed0c102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:29.413 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.526+0000 7fed10889700 1 -- 172.21.15.114:0/1815377251 shutdown_connections 2024-02-16T22:11:29.413 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.526+0000 7fed10889700 1 -- 172.21.15.114:0/1815377251 wait complete. 2024-02-16T22:11:29.413 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.527+0000 7fed10889700 1 Processor -- start 2024-02-16T22:11:29.413 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.527+0000 7fed10889700 1 -- start start 2024-02-16T22:11:29.413 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.527+0000 7fed10889700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed0c104d10 0x7fed0c19c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:29.413 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.527+0000 7fed10889700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0c19c850 con 0x7fed0c104d10 2024-02-16T22:11:29.414 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.527+0000 7fed0a59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed0c104d10 0x7fed0c19c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:29.414 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.527+0000 7fed0a59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed0c104d10 0x7fed0c19c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42926/0 (socket says 172.21.15.114:42926) 2024-02-16T22:11:29.414 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.527+0000 7fed0a59c700 1 -- 172.21.15.114:0/2295475760 learned_addr learned my addr 172.21.15.114:0/2295475760 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:29.414 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.527+0000 7fed0a59c700 1 -- 172.21.15.114:0/2295475760 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecfc009550 con 0x7fed0c104d10 2024-02-16T22:11:29.414 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.528+0000 7fed0a59c700 1 --2- 172.21.15.114:0/2295475760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed0c104d10 0x7fed0c19c310 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fecfc003f80 tx=0x7fecfc011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:29.414 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.528+0000 7fecfb7fe700 1 -- 172.21.15.114:0/2295475760 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fecfc003a80 con 0x7fed0c104d10 2024-02-16T22:11:29.414 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.528+0000 7fecfb7fe700 1 -- 172.21.15.114:0/2295475760 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fecfc003be0 con 0x7fed0c104d10 2024-02-16T22:11:29.414 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.528+0000 7fecfb7fe700 1 -- 172.21.15.114:0/2295475760 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fecfc019820 con 0x7fed0c104d10 2024-02-16T22:11:29.414 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.528+0000 7fed10889700 1 -- 172.21.15.114:0/2295475760 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed0c19ca50 con 0x7fed0c104d10 2024-02-16T22:11:29.414 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.528+0000 7fed10889700 1 -- 172.21.15.114:0/2295475760 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed0c19cef0 con 0x7fed0c104d10 2024-02-16T22:11:29.414 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.529+0000 7fecfb7fe700 1 -- 172.21.15.114:0/2295475760 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fecfc019aa0 con 0x7fed0c104d10 2024-02-16T22:11:29.414 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.529+0000 7fecfb7fe700 1 --2- 172.21.15.114:0/2295475760 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fecf40382d0 0x7fecf403a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:29.414 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.529+0000 7fed10889700 1 -- 172.21.15.114:0/2295475760 --> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fed0c106490 con 0x7fecf40382d0 2024-02-16T22:11:29.414 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.529+0000 7fecfb7fe700 1 -- 172.21.15.114:0/2295475760 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fecfc04d500 con 0x7fed0c104d10 2024-02-16T22:11:29.415 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.529+0000 7fed09d9b700 1 -- 172.21.15.114:0/2295475760 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fecf40382d0 msgr2=0x7fecf403a790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/2055574123 2024-02-16T22:11:29.415 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.529+0000 7fed09d9b700 1 --2- 172.21.15.114:0/2295475760 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fecf40382d0 0x7fecf403a790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T22:11:29.415 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.730+0000 7fed09d9b700 1 -- 172.21.15.114:0/2295475760 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fecf40382d0 msgr2=0x7fecf403a790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/2055574123 2024-02-16T22:11:29.415 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:23.730+0000 7fed09d9b700 1 --2- 172.21.15.114:0/2295475760 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fecf40382d0 0x7fecf403a790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-16T22:11:29.415 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:24.131+0000 7fed09d9b700 1 -- 172.21.15.114:0/2295475760 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fecf40382d0 msgr2=0x7fecf403a790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/2055574123 2024-02-16T22:11:29.415 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:24.131+0000 7fed09d9b700 1 --2- 172.21.15.114:0/2295475760 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fecf40382d0 0x7fecf403a790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-16T22:11:29.415 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:24.932+0000 7fed09d9b700 1 -- 172.21.15.114:0/2295475760 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fecf40382d0 msgr2=0x7fecf403a790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/2055574123 2024-02-16T22:11:29.415 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:24.932+0000 7fed09d9b700 1 --2- 172.21.15.114:0/2295475760 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fecf40382d0 0x7fecf403a790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-16T22:11:29.415 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:26.533+0000 7fed09d9b700 1 -- 172.21.15.114:0/2295475760 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fecf40382d0 msgr2=0x7fecf403a790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/2055574123 2024-02-16T22:11:29.415 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:26.533+0000 7fed09d9b700 1 --2- 172.21.15.114:0/2295475760 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fecf40382d0 0x7fecf403a790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-16T22:11:29.415 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:27.863+0000 7fecfb7fe700 1 -- 172.21.15.114:0/2295475760 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mgrmap(e 6) v1 ==== 45050+0+0 (secure 0 0 0) 0x7fecfc016d40 con 0x7fed0c104d10 2024-02-16T22:11:29.415 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:27.863+0000 7fecfb7fe700 1 -- 172.21.15.114:0/2295475760 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fecf40382d0 msgr2=0x7fecf403a790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T22:11:29.415 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:27.863+0000 7fecfb7fe700 1 --2- 172.21.15.114:0/2295475760 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fecf40382d0 0x7fecf403a790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:29.416 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:28.864+0000 7fecfb7fe700 1 -- 172.21.15.114:0/2295475760 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mgrmap(e 7) v1 ==== 45177+0+0 (secure 0 0 0) 0x7fecfc02dce0 con 0x7fed0c104d10 2024-02-16T22:11:29.416 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:28.864+0000 7fecfb7fe700 1 --2- 172.21.15.114:0/2295475760 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fecf403bb70 0x7fecf403df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:29.416 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:28.864+0000 7fecfb7fe700 1 -- 172.21.15.114:0/2295475760 --> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fed0c106490 con 0x7fecf403bb70 2024-02-16T22:11:29.416 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:28.865+0000 7fed09d9b700 1 --2- 172.21.15.114:0/2295475760 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fecf403bb70 0x7fecf403df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:29.416 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:28.865+0000 7fed09d9b700 1 --2- 172.21.15.114:0/2295475760 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fecf403bb70 0x7fecf403df60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fed00003a10 tx=0x7fed00009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:29.416 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:28.866+0000 7fecfb7fe700 1 -- 172.21.15.114:0/2295475760 <== mgr.14120 v2:172.21.15.114:6800/625910309 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fed0c106490 con 0x7fecf403bb70 2024-02-16T22:11:29.416 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:28.872+0000 7fed10889700 1 -- 172.21.15.114:0/2295475760 --> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fed0c19d650 con 0x7fecf403bb70 2024-02-16T22:11:29.416 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:28.872+0000 7fecfb7fe700 1 -- 172.21.15.114:0/2295475760 <== mgr.14120 v2:172.21.15.114:6800/625910309 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fed0c19d650 con 0x7fecf403bb70 2024-02-16T22:11:29.416 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:28.872+0000 7fed10889700 1 -- 172.21.15.114:0/2295475760 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fecf403bb70 msgr2=0x7fecf403df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:29.416 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:28.872+0000 7fed10889700 1 --2- 172.21.15.114:0/2295475760 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fecf403bb70 0x7fecf403df60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fed00003a10 tx=0x7fed00009290 comp rx=0 tx=0).stop 2024-02-16T22:11:29.416 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:28.872+0000 7fed10889700 1 -- 172.21.15.114:0/2295475760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed0c104d10 msgr2=0x7fed0c19c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:29.416 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:28.872+0000 7fed10889700 1 --2- 172.21.15.114:0/2295475760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed0c104d10 0x7fed0c19c310 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fecfc003f80 tx=0x7fecfc011ab0 comp rx=0 tx=0).stop 2024-02-16T22:11:29.416 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:28.873+0000 7fed10889700 1 -- 172.21.15.114:0/2295475760 shutdown_connections 2024-02-16T22:11:29.417 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:28.873+0000 7fed10889700 1 --2- 172.21.15.114:0/2295475760 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fecf403bb70 0x7fecf403df60 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:29.417 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:28.873+0000 7fed10889700 1 --2- 172.21.15.114:0/2295475760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed0c104d10 0x7fed0c19c310 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:29.417 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:28.873+0000 7fed10889700 1 --2- 172.21.15.114:0/2295475760 >> [v2:172.21.15.114:6800/2055574123,v1:172.21.15.114:6801/2055574123] conn(0x7fecf40382d0 0x7fecf403a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:29.417 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:28.873+0000 7fed10889700 1 -- 172.21.15.114:0/2295475760 >> 172.21.15.114:0/2295475760 conn(0x7fed0c1008b0 msgr2=0x7fed0c1908d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:29.417 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:28.873+0000 7fed10889700 1 -- 172.21.15.114:0/2295475760 shutdown_connections 2024-02-16T22:11:29.417 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:28.873+0000 7fed10889700 1 -- 172.21.15.114:0/2295475760 wait complete. 2024-02-16T22:11:29.417 INFO:teuthology.orchestra.run.smithi114.stdout:mgr epoch 5 is available 2024-02-16T22:11:29.417 INFO:teuthology.orchestra.run.smithi114.stdout:Setting orchestrator backend to cephadm... 2024-02-16T22:11:29.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:29 smithi114 ceph-mon[29235]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-16T22:11:29.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:29 smithi114 ceph-mon[29235]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-16T22:11:29.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:29 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:11:30.579 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.781+0000 7f107bf36700 1 Processor -- start 2024-02-16T22:11:30.579 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.781+0000 7f107bf36700 1 -- start start 2024-02-16T22:11:30.579 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.781+0000 7f107bf36700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1074104d10 0x7f1074105130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:30.579 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.781+0000 7f107bf36700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1074079e40 con 0x7f1074104d10 2024-02-16T22:11:30.579 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.781+0000 7f1079cd2700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1074104d10 0x7f1074105130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:30.579 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.781+0000 7f1079cd2700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1074104d10 0x7f1074105130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42978/0 (socket says 172.21.15.114:42978) 2024-02-16T22:11:30.579 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.781+0000 7f1079cd2700 1 -- 172.21.15.114:0/1201161214 learned_addr learned my addr 172.21.15.114:0/1201161214 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:30.579 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.782+0000 7f1079cd2700 1 -- 172.21.15.114:0/1201161214 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1074105670 con 0x7f1074104d10 2024-02-16T22:11:30.580 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.782+0000 7f1079cd2700 1 --2- 172.21.15.114:0/1201161214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1074104d10 0x7f1074105130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f10700098a0 tx=0x7f1070009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d874500c9698f7a1 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:30.580 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.782+0000 7f1078cd0700 1 -- 172.21.15.114:0/1201161214 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1070004180 con 0x7f1074104d10 2024-02-16T22:11:30.580 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.782+0000 7f1078cd0700 1 -- 172.21.15.114:0/1201161214 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f10700042e0 con 0x7f1074104d10 2024-02-16T22:11:30.580 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.782+0000 7f1078cd0700 1 -- 172.21.15.114:0/1201161214 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1070011a50 con 0x7f1074104d10 2024-02-16T22:11:30.580 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.782+0000 7f107bf36700 1 -- 172.21.15.114:0/1201161214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1074104d10 msgr2=0x7f1074105130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:30.580 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.782+0000 7f107bf36700 1 --2- 172.21.15.114:0/1201161214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1074104d10 0x7f1074105130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f10700098a0 tx=0x7f1070009bb0 comp rx=0 tx=0).stop 2024-02-16T22:11:30.580 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.782+0000 7f107bf36700 1 -- 172.21.15.114:0/1201161214 shutdown_connections 2024-02-16T22:11:30.580 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.782+0000 7f107bf36700 1 --2- 172.21.15.114:0/1201161214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1074104d10 0x7f1074105130 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:30.580 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.782+0000 7f107bf36700 1 -- 172.21.15.114:0/1201161214 >> 172.21.15.114:0/1201161214 conn(0x7f10741008b0 msgr2=0x7f1074102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:30.580 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.783+0000 7f107bf36700 1 -- 172.21.15.114:0/1201161214 shutdown_connections 2024-02-16T22:11:30.580 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.783+0000 7f107bf36700 1 -- 172.21.15.114:0/1201161214 wait complete. 2024-02-16T22:11:30.580 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.783+0000 7f107bf36700 1 Processor -- start 2024-02-16T22:11:30.580 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.783+0000 7f107bf36700 1 -- start start 2024-02-16T22:11:30.581 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.783+0000 7f107bf36700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1074104d10 0x7f107419c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:30.581 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.783+0000 7f107bf36700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107419c860 con 0x7f1074104d10 2024-02-16T22:11:30.581 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.783+0000 7f1079cd2700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1074104d10 0x7f107419c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:30.581 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.783+0000 7f1079cd2700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1074104d10 0x7f107419c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42982/0 (socket says 172.21.15.114:42982) 2024-02-16T22:11:30.581 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.783+0000 7f1079cd2700 1 -- 172.21.15.114:0/12007042 learned_addr learned my addr 172.21.15.114:0/12007042 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:30.581 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.784+0000 7f1079cd2700 1 -- 172.21.15.114:0/12007042 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1070009550 con 0x7f1074104d10 2024-02-16T22:11:30.581 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.784+0000 7f1079cd2700 1 --2- 172.21.15.114:0/12007042 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1074104d10 0x7f107419c320 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1070003de0 tx=0x7f1070011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:30.581 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.784+0000 7f106affd700 1 -- 172.21.15.114:0/12007042 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1070003930 con 0x7f1074104d10 2024-02-16T22:11:30.581 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.784+0000 7f106affd700 1 -- 172.21.15.114:0/12007042 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1070003a90 con 0x7f1074104d10 2024-02-16T22:11:30.581 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.784+0000 7f107bf36700 1 -- 172.21.15.114:0/12007042 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f107419ca60 con 0x7f1074104d10 2024-02-16T22:11:30.581 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.784+0000 7f106affd700 1 -- 172.21.15.114:0/12007042 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1070019730 con 0x7f1074104d10 2024-02-16T22:11:30.581 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.784+0000 7f107bf36700 1 -- 172.21.15.114:0/12007042 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f107419cf00 con 0x7f1074104d10 2024-02-16T22:11:30.581 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.785+0000 7f106affd700 1 -- 172.21.15.114:0/12007042 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 7) v1 ==== 45177+0+0 (secure 0 0 0) 0x7f1070019960 con 0x7f1074104d10 2024-02-16T22:11:30.581 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.785+0000 7f107bf36700 1 -- 172.21.15.114:0/12007042 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1074195c00 con 0x7f1074104d10 2024-02-16T22:11:30.582 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.785+0000 7f106affd700 1 --2- 172.21.15.114:0/12007042 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f1060038250 0x7f106003a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:30.582 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.785+0000 7f106affd700 1 -- 172.21.15.114:0/12007042 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f107004c1d0 con 0x7f1074104d10 2024-02-16T22:11:30.582 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.785+0000 7f10794d1700 1 --2- 172.21.15.114:0/12007042 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f1060038250 0x7f106003a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:30.582 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.785+0000 7f10794d1700 1 --2- 172.21.15.114:0/12007042 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f1060038250 0x7f106003a710 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1064006fd0 tx=0x7f1064006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:30.582 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.790+0000 7f106affd700 1 -- 172.21.15.114:0/12007042 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1070010b70 con 0x7f1074104d10 2024-02-16T22:11:30.582 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.915+0000 7f106affd700 1 -- 172.21.15.114:0/12007042 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f107000fda0 con 0x7f1074104d10 2024-02-16T22:11:30.582 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:29.999+0000 7f107bf36700 1 -- 172.21.15.114:0/12007042 --> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f10740611d0 con 0x7f1060038250 2024-02-16T22:11:30.582 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:30.007+0000 7f106affd700 1 -- 172.21.15.114:0/12007042 <== mgr.14120 v2:172.21.15.114:6800/625910309 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f10740611d0 con 0x7f1060038250 2024-02-16T22:11:30.582 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:30.009+0000 7f107bf36700 1 -- 172.21.15.114:0/12007042 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f1060038250 msgr2=0x7f106003a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:30.582 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:30.009+0000 7f107bf36700 1 --2- 172.21.15.114:0/12007042 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f1060038250 0x7f106003a710 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1064006fd0 tx=0x7f1064006e40 comp rx=0 tx=0).stop 2024-02-16T22:11:30.582 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:30.009+0000 7f107bf36700 1 -- 172.21.15.114:0/12007042 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1074104d10 msgr2=0x7f107419c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:30.582 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:30.009+0000 7f107bf36700 1 --2- 172.21.15.114:0/12007042 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1074104d10 0x7f107419c320 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1070003de0 tx=0x7f1070011ab0 comp rx=0 tx=0).stop 2024-02-16T22:11:30.582 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:30.009+0000 7f107bf36700 1 -- 172.21.15.114:0/12007042 shutdown_connections 2024-02-16T22:11:30.582 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:30.009+0000 7f107bf36700 1 --2- 172.21.15.114:0/12007042 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f1060038250 0x7f106003a710 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:30.583 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:30.009+0000 7f107bf36700 1 --2- 172.21.15.114:0/12007042 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1074104d10 0x7f107419c320 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:30.583 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:30.009+0000 7f107bf36700 1 -- 172.21.15.114:0/12007042 >> 172.21.15.114:0/12007042 conn(0x7f10741008b0 msgr2=0x7f10741908d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:30.583 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:30.009+0000 7f107bf36700 1 -- 172.21.15.114:0/12007042 shutdown_connections 2024-02-16T22:11:30.583 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:30.009+0000 7f107bf36700 1 -- 172.21.15.114:0/12007042 wait complete. 2024-02-16T22:11:31.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:30 smithi114 ceph-mon[29235]: [16/Feb/2024:22:11:29] ENGINE Bus STARTING 2024-02-16T22:11:31.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:30 smithi114 ceph-mon[29235]: [16/Feb/2024:22:11:29] ENGINE Serving on http://172.21.15.114:8765 2024-02-16T22:11:31.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:30 smithi114 ceph-mon[29235]: [16/Feb/2024:22:11:29] ENGINE Serving on https://172.21.15.114:7150 2024-02-16T22:11:31.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:30 smithi114 ceph-mon[29235]: [16/Feb/2024:22:11:29] ENGINE Bus STARTED 2024-02-16T22:11:31.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:30 smithi114 ceph-mon[29235]: mgrmap e8: smithi114.vcullz(active, since 2s) 2024-02-16T22:11:31.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:30 smithi114 ceph-mon[29235]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:11:31.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:30 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' 2024-02-16T22:11:31.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:30 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:11:31.794 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-16T22:11:31.794 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.013+0000 7f0075bf4700 1 Processor -- start 2024-02-16T22:11:31.794 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.013+0000 7f0075bf4700 1 -- start start 2024-02-16T22:11:31.794 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.014+0000 7f0075bf4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0070106420 0x7f0070106840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:31.794 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.014+0000 7f0075bf4700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0070106e10 con 0x7f0070106420 2024-02-16T22:11:31.794 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.014+0000 7f006f7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0070106420 0x7f0070106840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:31.794 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.014+0000 7f006f7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0070106420 0x7f0070106840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42998/0 (socket says 172.21.15.114:42998) 2024-02-16T22:11:31.794 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.014+0000 7f006f7fe700 1 -- 172.21.15.114:0/2988797066 learned_addr learned my addr 172.21.15.114:0/2988797066 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:31.794 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.014+0000 7f006f7fe700 1 -- 172.21.15.114:0/2988797066 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0070107620 con 0x7f0070106420 2024-02-16T22:11:31.794 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.014+0000 7f006f7fe700 1 --2- 172.21.15.114:0/2988797066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0070106420 0x7f0070106840 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f00600098a0 tx=0x7f0060009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=497e72977331f847 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:31.794 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.014+0000 7f006e7fc700 1 -- 172.21.15.114:0/2988797066 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0060004180 con 0x7f0070106420 2024-02-16T22:11:31.794 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.014+0000 7f006e7fc700 1 -- 172.21.15.114:0/2988797066 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f00600042e0 con 0x7f0070106420 2024-02-16T22:11:31.795 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.014+0000 7f006e7fc700 1 -- 172.21.15.114:0/2988797066 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0060011a50 con 0x7f0070106420 2024-02-16T22:11:31.795 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.015+0000 7f0075bf4700 1 -- 172.21.15.114:0/2988797066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0070106420 msgr2=0x7f0070106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:31.795 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.015+0000 7f0075bf4700 1 --2- 172.21.15.114:0/2988797066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0070106420 0x7f0070106840 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f00600098a0 tx=0x7f0060009bb0 comp rx=0 tx=0).stop 2024-02-16T22:11:31.795 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.015+0000 7f0075bf4700 1 -- 172.21.15.114:0/2988797066 shutdown_connections 2024-02-16T22:11:31.795 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.015+0000 7f0075bf4700 1 --2- 172.21.15.114:0/2988797066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0070106420 0x7f0070106840 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:31.795 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.015+0000 7f0075bf4700 1 -- 172.21.15.114:0/2988797066 >> 172.21.15.114:0/2988797066 conn(0x7f0070101a00 msgr2=0x7f0070103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:31.795 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.015+0000 7f0075bf4700 1 -- 172.21.15.114:0/2988797066 shutdown_connections 2024-02-16T22:11:31.795 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.015+0000 7f0075bf4700 1 -- 172.21.15.114:0/2988797066 wait complete. 2024-02-16T22:11:31.795 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.016+0000 7f0075bf4700 1 Processor -- start 2024-02-16T22:11:31.795 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.016+0000 7f0075bf4700 1 -- start start 2024-02-16T22:11:31.795 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.016+0000 7f0075bf4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0070106420 0x7f007019a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:31.795 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.016+0000 7f0075bf4700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007019a5f0 con 0x7f0070106420 2024-02-16T22:11:31.795 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.016+0000 7f006f7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0070106420 0x7f007019a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:31.796 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.016+0000 7f006f7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0070106420 0x7f007019a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43004/0 (socket says 172.21.15.114:43004) 2024-02-16T22:11:31.796 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.016+0000 7f006f7fe700 1 -- 172.21.15.114:0/3097758047 learned_addr learned my addr 172.21.15.114:0/3097758047 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:31.796 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.016+0000 7f006f7fe700 1 -- 172.21.15.114:0/3097758047 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0060009550 con 0x7f0070106420 2024-02-16T22:11:31.796 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.016+0000 7f006f7fe700 1 --2- 172.21.15.114:0/3097758047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0070106420 0x7f007019a0b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0060000c00 tx=0x7f0060011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:31.796 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.017+0000 7f006cff9700 1 -- 172.21.15.114:0/3097758047 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0060003b90 con 0x7f0070106420 2024-02-16T22:11:31.796 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.017+0000 7f006cff9700 1 -- 172.21.15.114:0/3097758047 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f00600248b0 con 0x7f0070106420 2024-02-16T22:11:31.796 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.017+0000 7f006cff9700 1 -- 172.21.15.114:0/3097758047 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f006001a8e0 con 0x7f0070106420 2024-02-16T22:11:31.796 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.017+0000 7f0075bf4700 1 -- 172.21.15.114:0/3097758047 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f007019a7f0 con 0x7f0070106420 2024-02-16T22:11:31.796 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.017+0000 7f0075bf4700 1 -- 172.21.15.114:0/3097758047 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f007019ac90 con 0x7f0070106420 2024-02-16T22:11:31.796 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.017+0000 7f006cff9700 1 -- 172.21.15.114:0/3097758047 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f006001aba0 con 0x7f0070106420 2024-02-16T22:11:31.796 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.018+0000 7f0075bf4700 1 -- 172.21.15.114:0/3097758047 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00700623c0 con 0x7f0070106420 2024-02-16T22:11:31.796 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.018+0000 7f006cff9700 1 --2- 172.21.15.114:0/3097758047 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f0058038320 0x7f005803a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:31.796 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.018+0000 7f006cff9700 1 -- 172.21.15.114:0/3097758047 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f006004c560 con 0x7f0070106420 2024-02-16T22:11:31.797 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.018+0000 7f006effd700 1 --2- 172.21.15.114:0/3097758047 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f0058038320 0x7f005803a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:31.797 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.018+0000 7f006effd700 1 --2- 172.21.15.114:0/3097758047 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f0058038320 0x7f005803a7e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0064006fd0 tx=0x7f0064006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:31.797 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.022+0000 7f006cff9700 1 -- 172.21.15.114:0/3097758047 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f00600233a0 con 0x7f0070106420 2024-02-16T22:11:31.797 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.212+0000 7f0075bf4700 1 -- 172.21.15.114:0/3097758047 --> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f007010a820 con 0x7f0058038320 2024-02-16T22:11:31.797 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.212+0000 7f006cff9700 1 -- 172.21.15.114:0/3097758047 <== mgr.14120 v2:172.21.15.114:6800/625910309 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f007010a820 con 0x7f0058038320 2024-02-16T22:11:31.797 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.214+0000 7f0075bf4700 1 -- 172.21.15.114:0/3097758047 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f0058038320 msgr2=0x7f005803a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:31.797 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.214+0000 7f0075bf4700 1 --2- 172.21.15.114:0/3097758047 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f0058038320 0x7f005803a7e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0064006fd0 tx=0x7f0064006e40 comp rx=0 tx=0).stop 2024-02-16T22:11:31.797 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.214+0000 7f0075bf4700 1 -- 172.21.15.114:0/3097758047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0070106420 msgr2=0x7f007019a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:31.797 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.214+0000 7f0075bf4700 1 --2- 172.21.15.114:0/3097758047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0070106420 0x7f007019a0b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0060000c00 tx=0x7f0060011ab0 comp rx=0 tx=0).stop 2024-02-16T22:11:31.797 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.215+0000 7f0075bf4700 1 -- 172.21.15.114:0/3097758047 shutdown_connections 2024-02-16T22:11:31.797 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.215+0000 7f0075bf4700 1 --2- 172.21.15.114:0/3097758047 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f0058038320 0x7f005803a7e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:31.797 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.215+0000 7f0075bf4700 1 --2- 172.21.15.114:0/3097758047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0070106420 0x7f007019a0b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:31.797 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.215+0000 7f0075bf4700 1 -- 172.21.15.114:0/3097758047 >> 172.21.15.114:0/3097758047 conn(0x7f0070101a00 msgr2=0x7f00701027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:31.797 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.215+0000 7f0075bf4700 1 -- 172.21.15.114:0/3097758047 shutdown_connections 2024-02-16T22:11:31.797 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:31.215+0000 7f0075bf4700 1 -- 172.21.15.114:0/3097758047 wait complete. 2024-02-16T22:11:31.797 INFO:teuthology.orchestra.run.smithi114.stdout:Generating ssh key... 2024-02-16T22:11:32.038 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:31 smithi114 ceph-mon[29235]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:11:33.184 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.246+0000 7f6c0f057700 1 Processor -- start 2024-02-16T22:11:33.184 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.246+0000 7f6c0f057700 1 -- start start 2024-02-16T22:11:33.184 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.246+0000 7f6c0f057700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c0807bed0 0x7f6c0807a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:33.184 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.246+0000 7f6c0f057700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0807a900 con 0x7f6c0807bed0 2024-02-16T22:11:33.184 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.246+0000 7f6c0cdf3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c0807bed0 0x7f6c0807a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:33.184 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.246+0000 7f6c0cdf3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c0807bed0 0x7f6c0807a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52952/0 (socket says 172.21.15.114:52952) 2024-02-16T22:11:33.185 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.246+0000 7f6c0cdf3700 1 -- 172.21.15.114:0/3215882497 learned_addr learned my addr 172.21.15.114:0/3215882497 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:33.185 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.247+0000 7f6c0cdf3700 1 -- 172.21.15.114:0/3215882497 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0807aa40 con 0x7f6c0807bed0 2024-02-16T22:11:33.185 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.247+0000 7f6c0cdf3700 1 --2- 172.21.15.114:0/3215882497 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c0807bed0 0x7f6c0807a330 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6c040098a0 tx=0x7f6c04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf916ab7c9a7d2e1 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:33.185 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.247+0000 7f6bff7fe700 1 -- 172.21.15.114:0/3215882497 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6c04004180 con 0x7f6c0807bed0 2024-02-16T22:11:33.185 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.247+0000 7f6bff7fe700 1 -- 172.21.15.114:0/3215882497 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6c040042e0 con 0x7f6c0807bed0 2024-02-16T22:11:33.185 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.247+0000 7f6bff7fe700 1 -- 172.21.15.114:0/3215882497 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6c04011a50 con 0x7f6c0807bed0 2024-02-16T22:11:33.185 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.248+0000 7f6c0f057700 1 -- 172.21.15.114:0/3215882497 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c0807bed0 msgr2=0x7f6c0807a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:33.185 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.248+0000 7f6c0f057700 1 --2- 172.21.15.114:0/3215882497 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c0807bed0 0x7f6c0807a330 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6c040098a0 tx=0x7f6c04009bb0 comp rx=0 tx=0).stop 2024-02-16T22:11:33.185 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.248+0000 7f6c0f057700 1 -- 172.21.15.114:0/3215882497 shutdown_connections 2024-02-16T22:11:33.185 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.248+0000 7f6c0f057700 1 --2- 172.21.15.114:0/3215882497 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c0807bed0 0x7f6c0807a330 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:33.185 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.248+0000 7f6c0f057700 1 -- 172.21.15.114:0/3215882497 >> 172.21.15.114:0/3215882497 conn(0x7f6c08101a00 msgr2=0x7f6c08103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:33.185 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.248+0000 7f6c0f057700 1 -- 172.21.15.114:0/3215882497 shutdown_connections 2024-02-16T22:11:33.185 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.248+0000 7f6c0f057700 1 -- 172.21.15.114:0/3215882497 wait complete. 2024-02-16T22:11:33.185 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.248+0000 7f6c0f057700 1 Processor -- start 2024-02-16T22:11:33.186 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.248+0000 7f6c0f057700 1 -- start start 2024-02-16T22:11:33.186 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.248+0000 7f6c0f057700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c0807bed0 0x7f6c0819c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:33.186 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.248+0000 7f6c0f057700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0819c840 con 0x7f6c0807bed0 2024-02-16T22:11:33.186 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.249+0000 7f6c0cdf3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c0807bed0 0x7f6c0819c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:33.186 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.249+0000 7f6c0cdf3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c0807bed0 0x7f6c0819c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52964/0 (socket says 172.21.15.114:52964) 2024-02-16T22:11:33.186 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.249+0000 7f6c0cdf3700 1 -- 172.21.15.114:0/2232240352 learned_addr learned my addr 172.21.15.114:0/2232240352 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:33.186 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.249+0000 7f6c0cdf3700 1 -- 172.21.15.114:0/2232240352 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c04009550 con 0x7f6c0807bed0 2024-02-16T22:11:33.186 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.249+0000 7f6c0cdf3700 1 --2- 172.21.15.114:0/2232240352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c0807bed0 0x7f6c0819c300 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6c04000c00 tx=0x7f6c04011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:33.186 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.249+0000 7f6bfdffb700 1 -- 172.21.15.114:0/2232240352 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6c04003b90 con 0x7f6c0807bed0 2024-02-16T22:11:33.186 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.249+0000 7f6bfdffb700 1 -- 172.21.15.114:0/2232240352 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6c040248b0 con 0x7f6c0807bed0 2024-02-16T22:11:33.186 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.249+0000 7f6c0f057700 1 -- 172.21.15.114:0/2232240352 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c0819ca40 con 0x7f6c0807bed0 2024-02-16T22:11:33.186 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.249+0000 7f6bfdffb700 1 -- 172.21.15.114:0/2232240352 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6c0401a8e0 con 0x7f6c0807bed0 2024-02-16T22:11:33.186 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.249+0000 7f6c0f057700 1 -- 172.21.15.114:0/2232240352 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c0819cee0 con 0x7f6c0807bed0 2024-02-16T22:11:33.186 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.250+0000 7f6bfdffb700 1 -- 172.21.15.114:0/2232240352 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f6c0401ab50 con 0x7f6c0807bed0 2024-02-16T22:11:33.186 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.250+0000 7f6c0f057700 1 -- 172.21.15.114:0/2232240352 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c0810c760 con 0x7f6c0807bed0 2024-02-16T22:11:33.187 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.250+0000 7f6bfdffb700 1 --2- 172.21.15.114:0/2232240352 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f6bf0038370 0x7f6bf003a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:33.187 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.250+0000 7f6bfdffb700 1 -- 172.21.15.114:0/2232240352 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f6c0404c5e0 con 0x7f6c0807bed0 2024-02-16T22:11:33.187 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.250+0000 7f6bfffff700 1 --2- 172.21.15.114:0/2232240352 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f6bf0038370 0x7f6bf003a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:33.187 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.251+0000 7f6bfffff700 1 --2- 172.21.15.114:0/2232240352 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f6bf0038370 0x7f6bf003a830 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8006fd0 tx=0x7f6bf8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:33.187 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.255+0000 7f6bfdffb700 1 -- 172.21.15.114:0/2232240352 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6c040233a0 con 0x7f6c0807bed0 2024-02-16T22:11:33.187 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.444+0000 7f6c0f057700 1 -- 172.21.15.114:0/2232240352 --> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c08103cc0 con 0x7f6bf0038370 2024-02-16T22:11:33.187 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.593+0000 7f6bfdffb700 1 -- 172.21.15.114:0/2232240352 <== mgr.14120 v2:172.21.15.114:6800/625910309 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6c08103cc0 con 0x7f6bf0038370 2024-02-16T22:11:33.187 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.595+0000 7f6c0f057700 1 -- 172.21.15.114:0/2232240352 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f6bf0038370 msgr2=0x7f6bf003a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:33.187 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.595+0000 7f6c0f057700 1 --2- 172.21.15.114:0/2232240352 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f6bf0038370 0x7f6bf003a830 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8006fd0 tx=0x7f6bf8006e40 comp rx=0 tx=0).stop 2024-02-16T22:11:33.187 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.595+0000 7f6c0f057700 1 -- 172.21.15.114:0/2232240352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c0807bed0 msgr2=0x7f6c0819c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:33.187 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.595+0000 7f6c0f057700 1 --2- 172.21.15.114:0/2232240352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c0807bed0 0x7f6c0819c300 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6c04000c00 tx=0x7f6c04011ab0 comp rx=0 tx=0).stop 2024-02-16T22:11:33.187 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.596+0000 7f6c0f057700 1 -- 172.21.15.114:0/2232240352 shutdown_connections 2024-02-16T22:11:33.187 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.596+0000 7f6c0f057700 1 --2- 172.21.15.114:0/2232240352 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f6bf0038370 0x7f6bf003a830 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:33.187 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.596+0000 7f6c0f057700 1 --2- 172.21.15.114:0/2232240352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c0807bed0 0x7f6c0819c300 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:33.187 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.596+0000 7f6c0f057700 1 -- 172.21.15.114:0/2232240352 >> 172.21.15.114:0/2232240352 conn(0x7f6c08101a00 msgr2=0x7f6c081030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:33.188 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.596+0000 7f6c0f057700 1 -- 172.21.15.114:0/2232240352 shutdown_connections 2024-02-16T22:11:33.188 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:32.596+0000 7f6c0f057700 1 -- 172.21.15.114:0/2232240352 wait complete. 2024-02-16T22:11:33.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:33 smithi114 ceph-mon[29235]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:11:33.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:33 smithi114 ceph-mon[29235]: Generating ssh key... 2024-02-16T22:11:33.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:33 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' 2024-02-16T22:11:33.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:33 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' 2024-02-16T22:11:34.425 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQChsI2oVIGL2COkfQpQTevyqsLNDIIG8Sny9IgRGFwqsK1hSno1iwmFF58iHEvXX8goXr2OhmOeyGDqJDpFJvPOiwS3J1HjvN+sENf0ZH6WBzaoF2ugcGpzlw3+v/55FMZdxq2nABf26+/e8TeQSA7cyIWgI+GppYyZReKWozyEWDif7LS2akO3Ya4ea+zaNZBNapHysVPK/ndQa/iXGHMcQ6mK9jY50Wdq0uCrs3bQKp89o9hugILk1Nm5mQnF051Vp4jkoZfNyitw2fRZS2aRmonzyBAmGxvnDV5Mi8VHfT66X9ciR+U7wQ0UQOTUKMKF9F5j/pIr6OsF7dtsqZAVR0HBsQ8/XkY9e9hrJtNywRC/NypuAHooHnuymxV3zW1m7ZK2uiuEqS3/7VfqPQdACcPdvKpTOd9wi5t+S4abuNfA8PvrXrMOrQd1Cxf1BKd9kQ1GyPC2B8Oh46rks5yH1C1zjWqdJBCSBwVHYA4gNWTyuj2E8lgnKz3MszfeqXU= ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715 2024-02-16T22:11:34.425 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.609+0000 7ff55f7ed700 1 Processor -- start 2024-02-16T22:11:34.425 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.609+0000 7ff55f7ed700 1 -- start start 2024-02-16T22:11:34.425 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.609+0000 7ff55f7ed700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff558106910 0x7ff558106d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:34.425 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.609+0000 7ff55f7ed700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff558107300 con 0x7ff558106910 2024-02-16T22:11:34.425 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.609+0000 7ff55d589700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff558106910 0x7ff558106d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:34.425 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.609+0000 7ff55d589700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff558106910 0x7ff558106d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52968/0 (socket says 172.21.15.114:52968) 2024-02-16T22:11:34.426 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.609+0000 7ff55d589700 1 -- 172.21.15.114:0/2642525778 learned_addr learned my addr 172.21.15.114:0/2642525778 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:34.426 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.610+0000 7ff55d589700 1 -- 172.21.15.114:0/2642525778 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff558107440 con 0x7ff558106910 2024-02-16T22:11:34.426 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.610+0000 7ff55d589700 1 --2- 172.21.15.114:0/2642525778 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff558106910 0x7ff558106d30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff5540098a0 tx=0x7ff554009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df9faebcda96af81 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:34.426 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.610+0000 7ff54ffff700 1 -- 172.21.15.114:0/2642525778 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff554004180 con 0x7ff558106910 2024-02-16T22:11:34.426 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.610+0000 7ff54ffff700 1 -- 172.21.15.114:0/2642525778 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff5540042e0 con 0x7ff558106910 2024-02-16T22:11:34.426 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.610+0000 7ff54ffff700 1 -- 172.21.15.114:0/2642525778 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff554011a50 con 0x7ff558106910 2024-02-16T22:11:34.426 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.611+0000 7ff55f7ed700 1 -- 172.21.15.114:0/2642525778 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff558106910 msgr2=0x7ff558106d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:34.426 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.611+0000 7ff55f7ed700 1 --2- 172.21.15.114:0/2642525778 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff558106910 0x7ff558106d30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff5540098a0 tx=0x7ff554009bb0 comp rx=0 tx=0).stop 2024-02-16T22:11:34.426 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.611+0000 7ff55f7ed700 1 -- 172.21.15.114:0/2642525778 shutdown_connections 2024-02-16T22:11:34.426 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.611+0000 7ff55f7ed700 1 --2- 172.21.15.114:0/2642525778 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff558106910 0x7ff558106d30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:34.426 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.611+0000 7ff55f7ed700 1 -- 172.21.15.114:0/2642525778 >> 172.21.15.114:0/2642525778 conn(0x7ff558100040 msgr2=0x7ff5581024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:34.426 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.611+0000 7ff55f7ed700 1 -- 172.21.15.114:0/2642525778 shutdown_connections 2024-02-16T22:11:34.426 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.611+0000 7ff55f7ed700 1 -- 172.21.15.114:0/2642525778 wait complete. 2024-02-16T22:11:34.426 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.611+0000 7ff55f7ed700 1 Processor -- start 2024-02-16T22:11:34.426 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.611+0000 7ff55f7ed700 1 -- start start 2024-02-16T22:11:34.426 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.611+0000 7ff55f7ed700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff558106910 0x7ff55819c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:34.427 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.611+0000 7ff55f7ed700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55819c850 con 0x7ff558106910 2024-02-16T22:11:34.427 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.612+0000 7ff55d589700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff558106910 0x7ff55819c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:34.427 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.612+0000 7ff55d589700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff558106910 0x7ff55819c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52974/0 (socket says 172.21.15.114:52974) 2024-02-16T22:11:34.427 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.612+0000 7ff55d589700 1 -- 172.21.15.114:0/3446998153 learned_addr learned my addr 172.21.15.114:0/3446998153 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:34.427 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.612+0000 7ff55d589700 1 -- 172.21.15.114:0/3446998153 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff554009550 con 0x7ff558106910 2024-02-16T22:11:34.427 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.612+0000 7ff55d589700 1 --2- 172.21.15.114:0/3446998153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff558106910 0x7ff55819c310 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff554003f80 tx=0x7ff554011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:34.427 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.612+0000 7ff54e7fc700 1 -- 172.21.15.114:0/3446998153 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff554003a80 con 0x7ff558106910 2024-02-16T22:11:34.427 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.612+0000 7ff54e7fc700 1 -- 172.21.15.114:0/3446998153 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff554003be0 con 0x7ff558106910 2024-02-16T22:11:34.427 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.612+0000 7ff54e7fc700 1 -- 172.21.15.114:0/3446998153 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff554019820 con 0x7ff558106910 2024-02-16T22:11:34.427 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.612+0000 7ff55f7ed700 1 -- 172.21.15.114:0/3446998153 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff55819ca50 con 0x7ff558106910 2024-02-16T22:11:34.427 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.612+0000 7ff55f7ed700 1 -- 172.21.15.114:0/3446998153 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff55819cef0 con 0x7ff558106910 2024-02-16T22:11:34.427 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.613+0000 7ff54e7fc700 1 -- 172.21.15.114:0/3446998153 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7ff554019a50 con 0x7ff558106910 2024-02-16T22:11:34.427 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.613+0000 7ff54e7fc700 1 --2- 172.21.15.114:0/3446998153 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7ff544038320 0x7ff54403a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:34.427 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.613+0000 7ff54e7fc700 1 -- 172.21.15.114:0/3446998153 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff55404c500 con 0x7ff558106910 2024-02-16T22:11:34.427 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.613+0000 7ff55f7ed700 1 -- 172.21.15.114:0/3446998153 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff558196000 con 0x7ff558106910 2024-02-16T22:11:34.428 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.613+0000 7ff55cd88700 1 --2- 172.21.15.114:0/3446998153 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7ff544038320 0x7ff54403a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:34.428 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.614+0000 7ff55cd88700 1 --2- 172.21.15.114:0/3446998153 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7ff544038320 0x7ff54403a7e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff548006fd0 tx=0x7ff548006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:34.428 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.618+0000 7ff54e7fc700 1 -- 172.21.15.114:0/3446998153 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff55401a950 con 0x7ff558106910 2024-02-16T22:11:34.428 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.807+0000 7ff55f7ed700 1 -- 172.21.15.114:0/3446998153 --> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5580611d0 con 0x7ff544038320 2024-02-16T22:11:34.428 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.807+0000 7ff54e7fc700 1 -- 172.21.15.114:0/3446998153 <== mgr.14120 v2:172.21.15.114:6800/625910309 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7ff5580611d0 con 0x7ff544038320 2024-02-16T22:11:34.428 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.809+0000 7ff55f7ed700 1 -- 172.21.15.114:0/3446998153 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7ff544038320 msgr2=0x7ff54403a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:34.428 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.809+0000 7ff55f7ed700 1 --2- 172.21.15.114:0/3446998153 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7ff544038320 0x7ff54403a7e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff548006fd0 tx=0x7ff548006e40 comp rx=0 tx=0).stop 2024-02-16T22:11:34.428 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.809+0000 7ff55f7ed700 1 -- 172.21.15.114:0/3446998153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff558106910 msgr2=0x7ff55819c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:34.428 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.809+0000 7ff55f7ed700 1 --2- 172.21.15.114:0/3446998153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff558106910 0x7ff55819c310 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff554003f80 tx=0x7ff554011ab0 comp rx=0 tx=0).stop 2024-02-16T22:11:34.428 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.809+0000 7ff55f7ed700 1 -- 172.21.15.114:0/3446998153 shutdown_connections 2024-02-16T22:11:34.428 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.809+0000 7ff55f7ed700 1 --2- 172.21.15.114:0/3446998153 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7ff544038320 0x7ff54403a7e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:34.428 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.809+0000 7ff55f7ed700 1 --2- 172.21.15.114:0/3446998153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff558106910 0x7ff55819c310 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:34.428 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.809+0000 7ff55f7ed700 1 -- 172.21.15.114:0/3446998153 >> 172.21.15.114:0/3446998153 conn(0x7ff558100040 msgr2=0x7ff558193140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:34.428 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.810+0000 7ff55f7ed700 1 -- 172.21.15.114:0/3446998153 shutdown_connections 2024-02-16T22:11:34.428 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:33.810+0000 7ff55f7ed700 1 -- 172.21.15.114:0/3446998153 wait complete. 2024-02-16T22:11:34.429 INFO:teuthology.orchestra.run.smithi114.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-16T22:11:34.429 INFO:teuthology.orchestra.run.smithi114.stdout:Adding key to root@localhost authorized_keys... 2024-02-16T22:11:34.429 INFO:teuthology.orchestra.run.smithi114.stdout:Adding host smithi114... 2024-02-16T22:11:34.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:34 smithi114 ceph-mon[29235]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:11:35.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:35 smithi114 ceph-mon[29235]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi114", "addr": "172.21.15.114", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:11:36.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:36 smithi114 ceph-mon[29235]: Deploying cephadm binary to smithi114 2024-02-16T22:11:38.191 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout Added host 'smithi114' with addr '172.21.15.114' 2024-02-16T22:11:38.191 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.838+0000 7fd37c0cd700 1 Processor -- start 2024-02-16T22:11:38.191 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.839+0000 7fd37c0cd700 1 -- start start 2024-02-16T22:11:38.192 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.839+0000 7fd37c0cd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd374104ac0 0x7fd374104ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:38.192 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.839+0000 7fd37c0cd700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3741054b0 con 0x7fd374104ac0 2024-02-16T22:11:38.192 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.839+0000 7fd379e69700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd374104ac0 0x7fd374104ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:38.192 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.839+0000 7fd379e69700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd374104ac0 0x7fd374104ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52990/0 (socket says 172.21.15.114:52990) 2024-02-16T22:11:38.192 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.839+0000 7fd379e69700 1 -- 172.21.15.114:0/1922728613 learned_addr learned my addr 172.21.15.114:0/1922728613 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:38.192 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.839+0000 7fd379e69700 1 -- 172.21.15.114:0/1922728613 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd374105cc0 con 0x7fd374104ac0 2024-02-16T22:11:38.192 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.839+0000 7fd379e69700 1 --2- 172.21.15.114:0/1922728613 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd374104ac0 0x7fd374104ee0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd3700098a0 tx=0x7fd370009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f693458aa337112 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:38.192 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.839+0000 7fd378e67700 1 -- 172.21.15.114:0/1922728613 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd370004180 con 0x7fd374104ac0 2024-02-16T22:11:38.192 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.840+0000 7fd378e67700 1 -- 172.21.15.114:0/1922728613 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd3700042e0 con 0x7fd374104ac0 2024-02-16T22:11:38.192 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.840+0000 7fd378e67700 1 -- 172.21.15.114:0/1922728613 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd370011a50 con 0x7fd374104ac0 2024-02-16T22:11:38.192 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.840+0000 7fd37c0cd700 1 -- 172.21.15.114:0/1922728613 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd374104ac0 msgr2=0x7fd374104ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:38.192 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.840+0000 7fd37c0cd700 1 --2- 172.21.15.114:0/1922728613 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd374104ac0 0x7fd374104ee0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd3700098a0 tx=0x7fd370009bb0 comp rx=0 tx=0).stop 2024-02-16T22:11:38.192 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.840+0000 7fd37c0cd700 1 -- 172.21.15.114:0/1922728613 shutdown_connections 2024-02-16T22:11:38.192 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.840+0000 7fd37c0cd700 1 --2- 172.21.15.114:0/1922728613 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd374104ac0 0x7fd374104ee0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:38.192 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.840+0000 7fd37c0cd700 1 -- 172.21.15.114:0/1922728613 >> 172.21.15.114:0/1922728613 conn(0x7fd374100040 msgr2=0x7fd3741024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:38.193 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.840+0000 7fd37c0cd700 1 -- 172.21.15.114:0/1922728613 shutdown_connections 2024-02-16T22:11:38.193 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.840+0000 7fd37c0cd700 1 -- 172.21.15.114:0/1922728613 wait complete. 2024-02-16T22:11:38.193 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.841+0000 7fd37c0cd700 1 Processor -- start 2024-02-16T22:11:38.193 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.841+0000 7fd37c0cd700 1 -- start start 2024-02-16T22:11:38.193 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.841+0000 7fd37c0cd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd374104ac0 0x7fd37419a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:38.193 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.841+0000 7fd37c0cd700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37419a680 con 0x7fd374104ac0 2024-02-16T22:11:38.193 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.841+0000 7fd379e69700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd374104ac0 0x7fd37419a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:38.193 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.841+0000 7fd379e69700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd374104ac0 0x7fd37419a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53006/0 (socket says 172.21.15.114:53006) 2024-02-16T22:11:38.193 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.841+0000 7fd379e69700 1 -- 172.21.15.114:0/3453720028 learned_addr learned my addr 172.21.15.114:0/3453720028 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:38.193 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.841+0000 7fd379e69700 1 -- 172.21.15.114:0/3453720028 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd370009550 con 0x7fd374104ac0 2024-02-16T22:11:38.193 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.841+0000 7fd379e69700 1 --2- 172.21.15.114:0/3453720028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd374104ac0 0x7fd37419a140 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd370009520 tx=0x7fd370011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:38.193 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.841+0000 7fd36affd700 1 -- 172.21.15.114:0/3453720028 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd370003a80 con 0x7fd374104ac0 2024-02-16T22:11:38.193 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.842+0000 7fd37c0cd700 1 -- 172.21.15.114:0/3453720028 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd37419a880 con 0x7fd374104ac0 2024-02-16T22:11:38.193 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.842+0000 7fd36affd700 1 -- 172.21.15.114:0/3453720028 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd370003be0 con 0x7fd374104ac0 2024-02-16T22:11:38.194 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.842+0000 7fd36affd700 1 -- 172.21.15.114:0/3453720028 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd3700197b0 con 0x7fd374104ac0 2024-02-16T22:11:38.194 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.842+0000 7fd37c0cd700 1 -- 172.21.15.114:0/3453720028 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd37419ad20 con 0x7fd374104ac0 2024-02-16T22:11:38.194 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.842+0000 7fd36affd700 1 -- 172.21.15.114:0/3453720028 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fd370019a90 con 0x7fd374104ac0 2024-02-16T22:11:38.194 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.842+0000 7fd37c0cd700 1 -- 172.21.15.114:0/3453720028 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3741061e0 con 0x7fd374104ac0 2024-02-16T22:11:38.194 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.843+0000 7fd36affd700 1 --2- 172.21.15.114:0/3453720028 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fd3600382d0 0x7fd36003a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:38.194 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.843+0000 7fd36affd700 1 -- 172.21.15.114:0/3453720028 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd37004c6b0 con 0x7fd374104ac0 2024-02-16T22:11:38.194 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.843+0000 7fd379668700 1 --2- 172.21.15.114:0/3453720028 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fd3600382d0 0x7fd36003a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:38.194 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.843+0000 7fd379668700 1 --2- 172.21.15.114:0/3453720028 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fd3600382d0 0x7fd36003a790 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd364006fd0 tx=0x7fd364006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:38.194 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:34.847+0000 7fd36affd700 1 -- 172.21.15.114:0/3453720028 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd37002b750 con 0x7fd374104ac0 2024-02-16T22:11:38.194 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:35.037+0000 7fd37c0cd700 1 -- 172.21.15.114:0/3453720028 --> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi114", "addr": "172.21.15.114", "target": ["mon-mgr", ""]}) v1 -- 0x7fd37419db60 con 0x7fd3600382d0 2024-02-16T22:11:38.194 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:37.599+0000 7fd36affd700 1 -- 172.21.15.114:0/3453720028 <== mgr.14120 v2:172.21.15.114:6800/625910309 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fd37419db60 con 0x7fd3600382d0 2024-02-16T22:11:38.194 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:37.601+0000 7fd37c0cd700 1 -- 172.21.15.114:0/3453720028 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fd3600382d0 msgr2=0x7fd36003a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:38.194 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:37.601+0000 7fd37c0cd700 1 --2- 172.21.15.114:0/3453720028 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fd3600382d0 0x7fd36003a790 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd364006fd0 tx=0x7fd364006e40 comp rx=0 tx=0).stop 2024-02-16T22:11:38.194 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:37.601+0000 7fd37c0cd700 1 -- 172.21.15.114:0/3453720028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd374104ac0 msgr2=0x7fd37419a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:38.195 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:37.601+0000 7fd37c0cd700 1 --2- 172.21.15.114:0/3453720028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd374104ac0 0x7fd37419a140 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd370009520 tx=0x7fd370011ab0 comp rx=0 tx=0).stop 2024-02-16T22:11:38.195 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:37.601+0000 7fd37c0cd700 1 -- 172.21.15.114:0/3453720028 shutdown_connections 2024-02-16T22:11:38.195 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:37.601+0000 7fd37c0cd700 1 --2- 172.21.15.114:0/3453720028 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fd3600382d0 0x7fd36003a790 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:38.195 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:37.601+0000 7fd37c0cd700 1 --2- 172.21.15.114:0/3453720028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd374104ac0 0x7fd37419a140 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:38.195 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:37.601+0000 7fd37c0cd700 1 -- 172.21.15.114:0/3453720028 >> 172.21.15.114:0/3453720028 conn(0x7fd374100040 msgr2=0x7fd374100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:38.195 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:37.601+0000 7fd37c0cd700 1 -- 172.21.15.114:0/3453720028 shutdown_connections 2024-02-16T22:11:38.195 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:37.601+0000 7fd37c0cd700 1 -- 172.21.15.114:0/3453720028 wait complete. 2024-02-16T22:11:38.195 INFO:teuthology.orchestra.run.smithi114.stdout:Deploying mon service with default placement... 2024-02-16T22:11:38.808 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:38 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' 2024-02-16T22:11:38.809 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:38 smithi114 ceph-mon[29235]: Added host smithi114 2024-02-16T22:11:38.809 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:38 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:11:39.426 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-16T22:11:39.426 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.652+0000 7fde95418700 1 Processor -- start 2024-02-16T22:11:39.426 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.652+0000 7fde95418700 1 -- start start 2024-02-16T22:11:39.426 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.652+0000 7fde95418700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde90106ea0 0x7fde90109290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:39.426 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.652+0000 7fde95418700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde90079ec0 con 0x7fde90106ea0 2024-02-16T22:11:39.426 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.652+0000 7fde8effd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde90106ea0 0x7fde90109290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:39.426 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.653+0000 7fde8effd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde90106ea0 0x7fde90109290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53016/0 (socket says 172.21.15.114:53016) 2024-02-16T22:11:39.426 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.653+0000 7fde8effd700 1 -- 172.21.15.114:0/59710660 learned_addr learned my addr 172.21.15.114:0/59710660 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:39.426 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.653+0000 7fde8effd700 1 -- 172.21.15.114:0/59710660 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde901097d0 con 0x7fde90106ea0 2024-02-16T22:11:39.426 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.653+0000 7fde8effd700 1 --2- 172.21.15.114:0/59710660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde90106ea0 0x7fde90109290 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fde800098a0 tx=0x7fde80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ad8d04a754ac63 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:39.427 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.653+0000 7fde8e7fc700 1 -- 172.21.15.114:0/59710660 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde80004180 con 0x7fde90106ea0 2024-02-16T22:11:39.427 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.653+0000 7fde8e7fc700 1 -- 172.21.15.114:0/59710660 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fde800042e0 con 0x7fde90106ea0 2024-02-16T22:11:39.427 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.653+0000 7fde8e7fc700 1 -- 172.21.15.114:0/59710660 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde80011a50 con 0x7fde90106ea0 2024-02-16T22:11:39.427 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.654+0000 7fde95418700 1 -- 172.21.15.114:0/59710660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde90106ea0 msgr2=0x7fde90109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:39.427 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.654+0000 7fde95418700 1 --2- 172.21.15.114:0/59710660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde90106ea0 0x7fde90109290 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fde800098a0 tx=0x7fde80009bb0 comp rx=0 tx=0).stop 2024-02-16T22:11:39.427 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.654+0000 7fde95418700 1 -- 172.21.15.114:0/59710660 shutdown_connections 2024-02-16T22:11:39.427 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.654+0000 7fde95418700 1 --2- 172.21.15.114:0/59710660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde90106ea0 0x7fde90109290 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:39.428 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.654+0000 7fde95418700 1 -- 172.21.15.114:0/59710660 >> 172.21.15.114:0/59710660 conn(0x7fde901008b0 msgr2=0x7fde90102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:39.428 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.654+0000 7fde95418700 1 -- 172.21.15.114:0/59710660 shutdown_connections 2024-02-16T22:11:39.428 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.654+0000 7fde95418700 1 -- 172.21.15.114:0/59710660 wait complete. 2024-02-16T22:11:39.428 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.654+0000 7fde95418700 1 Processor -- start 2024-02-16T22:11:39.428 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.654+0000 7fde95418700 1 -- start start 2024-02-16T22:11:39.428 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.655+0000 7fde95418700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde90106ea0 0x7fde9019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:39.429 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.655+0000 7fde95418700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9019a5e0 con 0x7fde90106ea0 2024-02-16T22:11:39.429 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.655+0000 7fde8effd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde90106ea0 0x7fde9019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:39.429 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.655+0000 7fde8effd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde90106ea0 0x7fde9019a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53024/0 (socket says 172.21.15.114:53024) 2024-02-16T22:11:39.429 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.655+0000 7fde8effd700 1 -- 172.21.15.114:0/3575364640 learned_addr learned my addr 172.21.15.114:0/3575364640 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:39.429 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.655+0000 7fde8effd700 1 -- 172.21.15.114:0/3575364640 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde80009550 con 0x7fde90106ea0 2024-02-16T22:11:39.429 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.655+0000 7fde8effd700 1 --2- 172.21.15.114:0/3575364640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde90106ea0 0x7fde9019a0a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fde80003f80 tx=0x7fde80011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:39.429 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.655+0000 7fde8cff9700 1 -- 172.21.15.114:0/3575364640 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde80003a80 con 0x7fde90106ea0 2024-02-16T22:11:39.429 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.655+0000 7fde8cff9700 1 -- 172.21.15.114:0/3575364640 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fde80003be0 con 0x7fde90106ea0 2024-02-16T22:11:39.429 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.655+0000 7fde95418700 1 -- 172.21.15.114:0/3575364640 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde9019a7e0 con 0x7fde90106ea0 2024-02-16T22:11:39.429 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.655+0000 7fde8cff9700 1 -- 172.21.15.114:0/3575364640 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde80019820 con 0x7fde90106ea0 2024-02-16T22:11:39.429 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.655+0000 7fde95418700 1 -- 172.21.15.114:0/3575364640 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde9019ac80 con 0x7fde90106ea0 2024-02-16T22:11:39.429 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.656+0000 7fde8cff9700 1 -- 172.21.15.114:0/3575364640 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fde80019b20 con 0x7fde90106ea0 2024-02-16T22:11:39.429 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.656+0000 7fde95418700 1 -- 172.21.15.114:0/3575364640 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde9010a550 con 0x7fde90106ea0 2024-02-16T22:11:39.430 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.656+0000 7fde8cff9700 1 --2- 172.21.15.114:0/3575364640 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fde7c0382d0 0x7fde7c03a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:39.430 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.656+0000 7fde8cff9700 1 -- 172.21.15.114:0/3575364640 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fde8004c7a0 con 0x7fde90106ea0 2024-02-16T22:11:39.430 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.656+0000 7fde867fc700 1 --2- 172.21.15.114:0/3575364640 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fde7c0382d0 0x7fde7c03a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:39.430 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.657+0000 7fde867fc700 1 --2- 172.21.15.114:0/3575364640 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fde7c0382d0 0x7fde7c03a790 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fde78006fd0 tx=0x7fde78006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:39.430 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.661+0000 7fde8cff9700 1 -- 172.21.15.114:0/3575364640 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fde8002b750 con 0x7fde90106ea0 2024-02-16T22:11:39.430 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.867+0000 7fde95418700 1 -- 172.21.15.114:0/3575364640 --> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7fde90101dc0 con 0x7fde7c0382d0 2024-02-16T22:11:39.430 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.872+0000 7fde8cff9700 1 -- 172.21.15.114:0/3575364640 <== mgr.14120 v2:172.21.15.114:6800/625910309 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fde90101dc0 con 0x7fde7c0382d0 2024-02-16T22:11:39.430 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.874+0000 7fde95418700 1 -- 172.21.15.114:0/3575364640 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fde7c0382d0 msgr2=0x7fde7c03a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:39.430 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.874+0000 7fde95418700 1 --2- 172.21.15.114:0/3575364640 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fde7c0382d0 0x7fde7c03a790 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fde78006fd0 tx=0x7fde78006e40 comp rx=0 tx=0).stop 2024-02-16T22:11:39.430 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.874+0000 7fde95418700 1 -- 172.21.15.114:0/3575364640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde90106ea0 msgr2=0x7fde9019a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:39.430 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.874+0000 7fde95418700 1 --2- 172.21.15.114:0/3575364640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde90106ea0 0x7fde9019a0a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fde80003f80 tx=0x7fde80011ab0 comp rx=0 tx=0).stop 2024-02-16T22:11:39.430 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.874+0000 7fde95418700 1 -- 172.21.15.114:0/3575364640 shutdown_connections 2024-02-16T22:11:39.430 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.874+0000 7fde95418700 1 --2- 172.21.15.114:0/3575364640 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fde7c0382d0 0x7fde7c03a790 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:39.430 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.874+0000 7fde95418700 1 --2- 172.21.15.114:0/3575364640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde90106ea0 0x7fde9019a0a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:39.430 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.874+0000 7fde95418700 1 -- 172.21.15.114:0/3575364640 >> 172.21.15.114:0/3575364640 conn(0x7fde901008b0 msgr2=0x7fde901016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:39.431 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.874+0000 7fde95418700 1 -- 172.21.15.114:0/3575364640 shutdown_connections 2024-02-16T22:11:39.431 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:38.874+0000 7fde95418700 1 -- 172.21.15.114:0/3575364640 wait complete. 2024-02-16T22:11:39.431 INFO:teuthology.orchestra.run.smithi114.stdout:Deploying mgr service with default placement... 2024-02-16T22:11:40.080 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:39 smithi114 ceph-mon[29235]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:11:40.080 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:39 smithi114 ceph-mon[29235]: Saving service mon spec with placement count:5 2024-02-16T22:11:40.080 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:39 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' 2024-02-16T22:11:41.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:41 smithi114 ceph-mon[29235]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:11:41.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:41 smithi114 ceph-mon[29235]: Saving service mgr spec with placement count:2 2024-02-16T22:11:41.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:41 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' 2024-02-16T22:11:41.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:41 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' 2024-02-16T22:11:41.211 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-16T22:11:41.211 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.810+0000 7fba22b8d700 1 Processor -- start 2024-02-16T22:11:41.211 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.810+0000 7fba22b8d700 1 -- start start 2024-02-16T22:11:41.211 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.810+0000 7fba22b8d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba1c106420 0x7fba1c106840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:41.211 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.810+0000 7fba22b8d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c106e10 con 0x7fba1c106420 2024-02-16T22:11:41.211 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.810+0000 7fba20929700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba1c106420 0x7fba1c106840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:41.211 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.810+0000 7fba20929700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba1c106420 0x7fba1c106840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53034/0 (socket says 172.21.15.114:53034) 2024-02-16T22:11:41.211 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.810+0000 7fba20929700 1 -- 172.21.15.114:0/1562093079 learned_addr learned my addr 172.21.15.114:0/1562093079 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:41.211 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.811+0000 7fba20929700 1 -- 172.21.15.114:0/1562093079 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1c107620 con 0x7fba1c106420 2024-02-16T22:11:41.211 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.811+0000 7fba20929700 1 --2- 172.21.15.114:0/1562093079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba1c106420 0x7fba1c106840 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fba100098a0 tx=0x7fba10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40357a646d17d55d server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:41.211 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.811+0000 7fba1b7fe700 1 -- 172.21.15.114:0/1562093079 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fba10004180 con 0x7fba1c106420 2024-02-16T22:11:41.211 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.811+0000 7fba1b7fe700 1 -- 172.21.15.114:0/1562093079 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fba100042e0 con 0x7fba1c106420 2024-02-16T22:11:41.211 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.811+0000 7fba1b7fe700 1 -- 172.21.15.114:0/1562093079 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fba10011a50 con 0x7fba1c106420 2024-02-16T22:11:41.211 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.811+0000 7fba22b8d700 1 -- 172.21.15.114:0/1562093079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba1c106420 msgr2=0x7fba1c106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:41.212 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.811+0000 7fba22b8d700 1 --2- 172.21.15.114:0/1562093079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba1c106420 0x7fba1c106840 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fba100098a0 tx=0x7fba10009bb0 comp rx=0 tx=0).stop 2024-02-16T22:11:41.212 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.811+0000 7fba22b8d700 1 -- 172.21.15.114:0/1562093079 shutdown_connections 2024-02-16T22:11:41.212 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.811+0000 7fba22b8d700 1 --2- 172.21.15.114:0/1562093079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba1c106420 0x7fba1c106840 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:41.212 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.811+0000 7fba22b8d700 1 -- 172.21.15.114:0/1562093079 >> 172.21.15.114:0/1562093079 conn(0x7fba1c101a00 msgr2=0x7fba1c103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:41.212 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.811+0000 7fba22b8d700 1 -- 172.21.15.114:0/1562093079 shutdown_connections 2024-02-16T22:11:41.212 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.811+0000 7fba22b8d700 1 -- 172.21.15.114:0/1562093079 wait complete. 2024-02-16T22:11:41.212 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.812+0000 7fba22b8d700 1 Processor -- start 2024-02-16T22:11:41.212 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.812+0000 7fba22b8d700 1 -- start start 2024-02-16T22:11:41.212 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.812+0000 7fba22b8d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba1c106420 0x7fba1c19a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:41.212 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.812+0000 7fba22b8d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c19a5f0 con 0x7fba1c106420 2024-02-16T22:11:41.212 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.812+0000 7fba20929700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba1c106420 0x7fba1c19a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:41.212 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.812+0000 7fba20929700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba1c106420 0x7fba1c19a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53050/0 (socket says 172.21.15.114:53050) 2024-02-16T22:11:41.212 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.812+0000 7fba20929700 1 -- 172.21.15.114:0/1576758369 learned_addr learned my addr 172.21.15.114:0/1576758369 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:41.212 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.813+0000 7fba20929700 1 -- 172.21.15.114:0/1576758369 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba10009550 con 0x7fba1c106420 2024-02-16T22:11:41.213 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.813+0000 7fba20929700 1 --2- 172.21.15.114:0/1576758369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba1c106420 0x7fba1c19a0b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fba10000c00 tx=0x7fba10011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:41.213 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.813+0000 7fba19ffb700 1 -- 172.21.15.114:0/1576758369 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fba10003b90 con 0x7fba1c106420 2024-02-16T22:11:41.213 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.813+0000 7fba19ffb700 1 -- 172.21.15.114:0/1576758369 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fba100248b0 con 0x7fba1c106420 2024-02-16T22:11:41.213 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.813+0000 7fba22b8d700 1 -- 172.21.15.114:0/1576758369 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba1c19a7f0 con 0x7fba1c106420 2024-02-16T22:11:41.213 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.813+0000 7fba19ffb700 1 -- 172.21.15.114:0/1576758369 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fba1001a8e0 con 0x7fba1c106420 2024-02-16T22:11:41.213 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.813+0000 7fba22b8d700 1 -- 172.21.15.114:0/1576758369 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba1c19ac90 con 0x7fba1c106420 2024-02-16T22:11:41.213 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.814+0000 7fba19ffb700 1 -- 172.21.15.114:0/1576758369 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fba1001aba0 con 0x7fba1c106420 2024-02-16T22:11:41.213 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.814+0000 7fba22b8d700 1 -- 172.21.15.114:0/1576758369 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba1c0623c0 con 0x7fba1c106420 2024-02-16T22:11:41.213 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.814+0000 7fba19ffb700 1 --2- 172.21.15.114:0/1576758369 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fba0c038320 0x7fba0c03a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:41.213 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.814+0000 7fba19ffb700 1 -- 172.21.15.114:0/1576758369 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fba1004c430 con 0x7fba1c106420 2024-02-16T22:11:41.213 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.814+0000 7fba1bfff700 1 --2- 172.21.15.114:0/1576758369 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fba0c038320 0x7fba0c03a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:41.213 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.815+0000 7fba1bfff700 1 --2- 172.21.15.114:0/1576758369 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fba0c038320 0x7fba0c03a7e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fba08006fd0 tx=0x7fba08006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:41.213 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:39.819+0000 7fba19ffb700 1 -- 172.21.15.114:0/1576758369 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fba100233a0 con 0x7fba1c106420 2024-02-16T22:11:41.214 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:40.024+0000 7fba22b8d700 1 -- 172.21.15.114:0/1576758369 --> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7fba1c10a820 con 0x7fba0c038320 2024-02-16T22:11:41.214 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:40.029+0000 7fba19ffb700 1 -- 172.21.15.114:0/1576758369 <== mgr.14120 v2:172.21.15.114:6800/625910309 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fba1c10a820 con 0x7fba0c038320 2024-02-16T22:11:41.214 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:40.031+0000 7fba22b8d700 1 -- 172.21.15.114:0/1576758369 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fba0c038320 msgr2=0x7fba0c03a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:41.214 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:40.031+0000 7fba22b8d700 1 --2- 172.21.15.114:0/1576758369 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fba0c038320 0x7fba0c03a7e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fba08006fd0 tx=0x7fba08006e40 comp rx=0 tx=0).stop 2024-02-16T22:11:41.214 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:40.031+0000 7fba22b8d700 1 -- 172.21.15.114:0/1576758369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba1c106420 msgr2=0x7fba1c19a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:41.214 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:40.031+0000 7fba22b8d700 1 --2- 172.21.15.114:0/1576758369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba1c106420 0x7fba1c19a0b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fba10000c00 tx=0x7fba10011ab0 comp rx=0 tx=0).stop 2024-02-16T22:11:41.214 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:40.031+0000 7fba22b8d700 1 -- 172.21.15.114:0/1576758369 shutdown_connections 2024-02-16T22:11:41.214 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:40.031+0000 7fba22b8d700 1 --2- 172.21.15.114:0/1576758369 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fba0c038320 0x7fba0c03a7e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:41.214 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:40.031+0000 7fba22b8d700 1 --2- 172.21.15.114:0/1576758369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba1c106420 0x7fba1c19a0b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:41.214 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:40.031+0000 7fba22b8d700 1 -- 172.21.15.114:0/1576758369 >> 172.21.15.114:0/1576758369 conn(0x7fba1c101a00 msgr2=0x7fba1c1027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:41.214 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:40.031+0000 7fba22b8d700 1 -- 172.21.15.114:0/1576758369 shutdown_connections 2024-02-16T22:11:41.214 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:40.031+0000 7fba22b8d700 1 -- 172.21.15.114:0/1576758369 wait complete. 2024-02-16T22:11:41.214 INFO:teuthology.orchestra.run.smithi114.stdout:Deploying crash service with default placement... 2024-02-16T22:11:42.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:42 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' 2024-02-16T22:11:42.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:42 smithi114 ceph-mon[29235]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:11:42.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:42 smithi114 ceph-mon[29235]: Saving service crash spec with placement * 2024-02-16T22:11:42.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:42 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' 2024-02-16T22:11:42.998 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-02-16T22:11:42.998 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.680+0000 7f50a7136700 1 Processor -- start 2024-02-16T22:11:42.998 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.680+0000 7f50a7136700 1 -- start start 2024-02-16T22:11:42.998 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.680+0000 7f50a7136700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a0105340 0x7f50a0105760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:42.998 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.680+0000 7f50a7136700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a0105d30 con 0x7f50a0105340 2024-02-16T22:11:42.998 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.680+0000 7f50a4ed2700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a0105340 0x7f50a0105760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:42.998 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.680+0000 7f50a4ed2700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a0105340 0x7f50a0105760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34200/0 (socket says 172.21.15.114:34200) 2024-02-16T22:11:42.998 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.680+0000 7f50a4ed2700 1 -- 172.21.15.114:0/533728611 learned_addr learned my addr 172.21.15.114:0/533728611 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:42.998 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.681+0000 7f50a4ed2700 1 -- 172.21.15.114:0/533728611 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a0106540 con 0x7f50a0105340 2024-02-16T22:11:42.998 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.681+0000 7f50a4ed2700 1 --2- 172.21.15.114:0/533728611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a0105340 0x7f50a0105760 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f509400bb40 tx=0x7f509400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35871fb21e47f9a8 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:42.998 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.681+0000 7f509f7fe700 1 -- 172.21.15.114:0/533728611 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f509400dc00 con 0x7f50a0105340 2024-02-16T22:11:42.999 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.681+0000 7f509f7fe700 1 -- 172.21.15.114:0/533728611 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f50940044e0 con 0x7f50a0105340 2024-02-16T22:11:42.999 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.681+0000 7f509f7fe700 1 -- 172.21.15.114:0/533728611 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5094010470 con 0x7f50a0105340 2024-02-16T22:11:42.999 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.682+0000 7f50a7136700 1 -- 172.21.15.114:0/533728611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a0105340 msgr2=0x7f50a0105760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:42.999 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.682+0000 7f50a7136700 1 --2- 172.21.15.114:0/533728611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a0105340 0x7f50a0105760 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f509400bb40 tx=0x7f509400be50 comp rx=0 tx=0).stop 2024-02-16T22:11:42.999 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.682+0000 7f50a7136700 1 -- 172.21.15.114:0/533728611 shutdown_connections 2024-02-16T22:11:42.999 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.682+0000 7f50a7136700 1 --2- 172.21.15.114:0/533728611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a0105340 0x7f50a0105760 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:42.999 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.682+0000 7f50a7136700 1 -- 172.21.15.114:0/533728611 >> 172.21.15.114:0/533728611 conn(0x7f50a01008c0 msgr2=0x7f50a0102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:42.999 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.682+0000 7f50a7136700 1 -- 172.21.15.114:0/533728611 shutdown_connections 2024-02-16T22:11:42.999 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.682+0000 7f50a7136700 1 -- 172.21.15.114:0/533728611 wait complete. 2024-02-16T22:11:42.999 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.682+0000 7f50a7136700 1 Processor -- start 2024-02-16T22:11:42.999 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.682+0000 7f50a7136700 1 -- start start 2024-02-16T22:11:42.999 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.683+0000 7f50a7136700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a0105340 0x7f50a0197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:42.999 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.683+0000 7f50a7136700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a01984b0 con 0x7f50a0105340 2024-02-16T22:11:42.999 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.683+0000 7f50a4ed2700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a0105340 0x7f50a0197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:43.000 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.683+0000 7f50a4ed2700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a0105340 0x7f50a0197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34202/0 (socket says 172.21.15.114:34202) 2024-02-16T22:11:43.000 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.683+0000 7f50a4ed2700 1 -- 172.21.15.114:0/647114015 learned_addr learned my addr 172.21.15.114:0/647114015 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:43.000 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.683+0000 7f50a4ed2700 1 -- 172.21.15.114:0/647114015 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509400b7f0 con 0x7f50a0105340 2024-02-16T22:11:43.000 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.683+0000 7f50a4ed2700 1 --2- 172.21.15.114:0/647114015 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a0105340 0x7f50a0197f70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f509400b340 tx=0x7f5094004070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:43.000 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.683+0000 7f509dffb700 1 -- 172.21.15.114:0/647114015 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f509401e070 con 0x7f50a0105340 2024-02-16T22:11:43.000 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.683+0000 7f509dffb700 1 -- 172.21.15.114:0/647114015 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f509400de20 con 0x7f50a0105340 2024-02-16T22:11:43.000 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.683+0000 7f50a7136700 1 -- 172.21.15.114:0/647114015 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a01986b0 con 0x7f50a0105340 2024-02-16T22:11:43.000 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.683+0000 7f509dffb700 1 -- 172.21.15.114:0/647114015 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5094022a00 con 0x7f50a0105340 2024-02-16T22:11:43.000 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.684+0000 7f50a7136700 1 -- 172.21.15.114:0/647114015 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a0198b50 con 0x7f50a0105340 2024-02-16T22:11:43.000 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.684+0000 7f509dffb700 1 -- 172.21.15.114:0/647114015 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f5094010730 con 0x7f50a0105340 2024-02-16T22:11:43.000 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.684+0000 7f50a7136700 1 -- 172.21.15.114:0/647114015 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a0191bd0 con 0x7f50a0105340 2024-02-16T22:11:43.000 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.684+0000 7f509dffb700 1 --2- 172.21.15.114:0/647114015 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f50900382d0 0x7f509003a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:43.000 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.685+0000 7f509dffb700 1 -- 172.21.15.114:0/647114015 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f509404e5f0 con 0x7f50a0105340 2024-02-16T22:11:43.001 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.685+0000 7f509ffff700 1 --2- 172.21.15.114:0/647114015 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f50900382d0 0x7f509003a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:43.001 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.685+0000 7f509ffff700 1 --2- 172.21.15.114:0/647114015 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f50900382d0 0x7f509003a790 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f508c006fd0 tx=0x7f508c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:43.001 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.690+0000 7f509dffb700 1 -- 172.21.15.114:0/647114015 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5094021ba0 con 0x7f50a0105340 2024-02-16T22:11:43.001 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.942+0000 7f50a7136700 1 -- 172.21.15.114:0/647114015 --> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f50a00611d0 con 0x7f50900382d0 2024-02-16T22:11:43.001 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.949+0000 7f509dffb700 1 -- 172.21.15.114:0/647114015 <== mgr.14120 v2:172.21.15.114:6800/625910309 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f50a00611d0 con 0x7f50900382d0 2024-02-16T22:11:43.001 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.951+0000 7f50a7136700 1 -- 172.21.15.114:0/647114015 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f50900382d0 msgr2=0x7f509003a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:43.001 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.951+0000 7f50a7136700 1 --2- 172.21.15.114:0/647114015 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f50900382d0 0x7f509003a790 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f508c006fd0 tx=0x7f508c006e40 comp rx=0 tx=0).stop 2024-02-16T22:11:43.001 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.951+0000 7f50a7136700 1 -- 172.21.15.114:0/647114015 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a0105340 msgr2=0x7f50a0197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:43.001 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.952+0000 7f50a7136700 1 --2- 172.21.15.114:0/647114015 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a0105340 0x7f50a0197f70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f509400b340 tx=0x7f5094004070 comp rx=0 tx=0).stop 2024-02-16T22:11:43.001 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.952+0000 7f50a7136700 1 -- 172.21.15.114:0/647114015 shutdown_connections 2024-02-16T22:11:43.001 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.952+0000 7f50a7136700 1 --2- 172.21.15.114:0/647114015 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f50900382d0 0x7f509003a790 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:43.002 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.952+0000 7f50a7136700 1 --2- 172.21.15.114:0/647114015 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a0105340 0x7f50a0197f70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:43.002 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.952+0000 7f50a7136700 1 -- 172.21.15.114:0/647114015 >> 172.21.15.114:0/647114015 conn(0x7f50a01008c0 msgr2=0x7f50a018ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:43.002 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.952+0000 7f50a7136700 1 -- 172.21.15.114:0/647114015 shutdown_connections 2024-02-16T22:11:43.002 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:41.952+0000 7f50a7136700 1 -- 172.21.15.114:0/647114015 wait complete. 2024-02-16T22:11:43.002 INFO:teuthology.orchestra.run.smithi114.stdout:Deploying ceph-exporter service with default placement... 2024-02-16T22:11:43.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:43 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' 2024-02-16T22:11:43.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:43 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' 2024-02-16T22:11:44.204 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-02-16T22:11:44.204 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.428+0000 7f993a6f7700 1 Processor -- start 2024-02-16T22:11:44.204 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.428+0000 7f993a6f7700 1 -- start start 2024-02-16T22:11:44.204 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.428+0000 7f993a6f7700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f993407bed0 0x7f993407a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:44.205 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.428+0000 7f993a6f7700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993407a900 con 0x7f993407bed0 2024-02-16T22:11:44.205 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.429+0000 7f9933fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f993407bed0 0x7f993407a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:44.205 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.429+0000 7f9933fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f993407bed0 0x7f993407a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34212/0 (socket says 172.21.15.114:34212) 2024-02-16T22:11:44.205 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.429+0000 7f9933fff700 1 -- 172.21.15.114:0/3448502099 learned_addr learned my addr 172.21.15.114:0/3448502099 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:44.205 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.429+0000 7f9933fff700 1 -- 172.21.15.114:0/3448502099 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993407aa40 con 0x7f993407bed0 2024-02-16T22:11:44.205 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.429+0000 7f9933fff700 1 --2- 172.21.15.114:0/3448502099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f993407bed0 0x7f993407a330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f99240098a0 tx=0x7f9924009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84fcb543c1d043f7 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:44.205 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.430+0000 7f9932ffd700 1 -- 172.21.15.114:0/3448502099 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9924004180 con 0x7f993407bed0 2024-02-16T22:11:44.205 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.430+0000 7f9932ffd700 1 -- 172.21.15.114:0/3448502099 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f99240042e0 con 0x7f993407bed0 2024-02-16T22:11:44.205 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.430+0000 7f9932ffd700 1 -- 172.21.15.114:0/3448502099 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9924011af0 con 0x7f993407bed0 2024-02-16T22:11:44.205 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.430+0000 7f993a6f7700 1 -- 172.21.15.114:0/3448502099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f993407bed0 msgr2=0x7f993407a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:44.205 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.430+0000 7f993a6f7700 1 --2- 172.21.15.114:0/3448502099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f993407bed0 0x7f993407a330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f99240098a0 tx=0x7f9924009bb0 comp rx=0 tx=0).stop 2024-02-16T22:11:44.205 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.430+0000 7f993a6f7700 1 -- 172.21.15.114:0/3448502099 shutdown_connections 2024-02-16T22:11:44.205 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.430+0000 7f993a6f7700 1 --2- 172.21.15.114:0/3448502099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f993407bed0 0x7f993407a330 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:44.205 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.430+0000 7f993a6f7700 1 -- 172.21.15.114:0/3448502099 >> 172.21.15.114:0/3448502099 conn(0x7f99341019d0 msgr2=0x7f9934103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:44.206 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.430+0000 7f993a6f7700 1 -- 172.21.15.114:0/3448502099 shutdown_connections 2024-02-16T22:11:44.206 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.430+0000 7f993a6f7700 1 -- 172.21.15.114:0/3448502099 wait complete. 2024-02-16T22:11:44.206 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.431+0000 7f993a6f7700 1 Processor -- start 2024-02-16T22:11:44.206 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.431+0000 7f993a6f7700 1 -- start start 2024-02-16T22:11:44.206 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.432+0000 7f993a6f7700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f993407bed0 0x7f993419c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:44.206 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.432+0000 7f993a6f7700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993419c890 con 0x7f993407bed0 2024-02-16T22:11:44.206 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.432+0000 7f9933fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f993407bed0 0x7f993419c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:44.206 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.432+0000 7f9933fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f993407bed0 0x7f993419c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34216/0 (socket says 172.21.15.114:34216) 2024-02-16T22:11:44.206 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.432+0000 7f9933fff700 1 -- 172.21.15.114:0/726552571 learned_addr learned my addr 172.21.15.114:0/726552571 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:44.206 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.432+0000 7f9933fff700 1 -- 172.21.15.114:0/726552571 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9924009550 con 0x7f993407bed0 2024-02-16T22:11:44.206 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.432+0000 7f9933fff700 1 --2- 172.21.15.114:0/726552571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f993407bed0 0x7f993419c350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9924006ba0 tx=0x7f9924011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:44.206 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.432+0000 7f99317fa700 1 -- 172.21.15.114:0/726552571 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9924003b00 con 0x7f993407bed0 2024-02-16T22:11:44.206 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.432+0000 7f99317fa700 1 -- 172.21.15.114:0/726552571 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f992401a8e0 con 0x7f993407bed0 2024-02-16T22:11:44.206 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.432+0000 7f993a6f7700 1 -- 172.21.15.114:0/726552571 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f993419ca90 con 0x7f993407bed0 2024-02-16T22:11:44.206 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.432+0000 7f99317fa700 1 -- 172.21.15.114:0/726552571 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9924019990 con 0x7f993407bed0 2024-02-16T22:11:44.207 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.433+0000 7f993a6f7700 1 -- 172.21.15.114:0/726552571 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993419cf30 con 0x7f993407bed0 2024-02-16T22:11:44.207 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.433+0000 7f99317fa700 1 -- 172.21.15.114:0/726552571 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f9924019c00 con 0x7f993407bed0 2024-02-16T22:11:44.207 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.433+0000 7f993a6f7700 1 -- 172.21.15.114:0/726552571 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f993410c7d0 con 0x7f993407bed0 2024-02-16T22:11:44.207 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.434+0000 7f99317fa700 1 --2- 172.21.15.114:0/726552571 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f991c038320 0x7f991c03a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:44.207 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.434+0000 7f99317fa700 1 -- 172.21.15.114:0/726552571 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f992404d6a0 con 0x7f993407bed0 2024-02-16T22:11:44.207 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.434+0000 7f99337fe700 1 --2- 172.21.15.114:0/726552571 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f991c038320 0x7f991c03a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:44.207 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.434+0000 7f99337fe700 1 --2- 172.21.15.114:0/726552571 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f991c038320 0x7f991c03a7e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9928006fd0 tx=0x7f9928006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:44.207 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.438+0000 7f99317fa700 1 -- 172.21.15.114:0/726552571 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f992402d440 con 0x7f993407bed0 2024-02-16T22:11:44.207 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.658+0000 7f993a6f7700 1 -- 172.21.15.114:0/726552571 --> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f9934103ca0 con 0x7f991c038320 2024-02-16T22:11:44.207 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.665+0000 7f99317fa700 1 -- 172.21.15.114:0/726552571 <== mgr.14120 v2:172.21.15.114:6800/625910309 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f9934103ca0 con 0x7f991c038320 2024-02-16T22:11:44.207 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.668+0000 7f993a6f7700 1 -- 172.21.15.114:0/726552571 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f991c038320 msgr2=0x7f991c03a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:44.207 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.668+0000 7f993a6f7700 1 --2- 172.21.15.114:0/726552571 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f991c038320 0x7f991c03a7e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9928006fd0 tx=0x7f9928006e40 comp rx=0 tx=0).stop 2024-02-16T22:11:44.207 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.668+0000 7f993a6f7700 1 -- 172.21.15.114:0/726552571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f993407bed0 msgr2=0x7f993419c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:44.207 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.668+0000 7f993a6f7700 1 --2- 172.21.15.114:0/726552571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f993407bed0 0x7f993419c350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9924006ba0 tx=0x7f9924011b50 comp rx=0 tx=0).stop 2024-02-16T22:11:44.208 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.668+0000 7f993a6f7700 1 -- 172.21.15.114:0/726552571 shutdown_connections 2024-02-16T22:11:44.208 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.668+0000 7f993a6f7700 1 --2- 172.21.15.114:0/726552571 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f991c038320 0x7f991c03a7e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:44.208 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.668+0000 7f993a6f7700 1 --2- 172.21.15.114:0/726552571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f993407bed0 0x7f993419c350 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:44.208 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.668+0000 7f993a6f7700 1 -- 172.21.15.114:0/726552571 >> 172.21.15.114:0/726552571 conn(0x7f99341019d0 msgr2=0x7f9934103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:44.208 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.668+0000 7f993a6f7700 1 -- 172.21.15.114:0/726552571 shutdown_connections 2024-02-16T22:11:44.208 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:43.668+0000 7f993a6f7700 1 -- 172.21.15.114:0/726552571 wait complete. 2024-02-16T22:11:44.208 INFO:teuthology.orchestra.run.smithi114.stdout:Deploying prometheus service with default placement... 2024-02-16T22:11:44.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:44 smithi114 ceph-mon[29235]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:11:44.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:44 smithi114 ceph-mon[29235]: Saving service ceph-exporter spec with placement * 2024-02-16T22:11:44.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:44 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' 2024-02-16T22:11:45.480 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-02-16T22:11:45.480 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.613+0000 7fb43481c700 1 Processor -- start 2024-02-16T22:11:45.480 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.613+0000 7fb43481c700 1 -- start start 2024-02-16T22:11:45.480 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.613+0000 7fb43481c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb430106420 0x7fb430106840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:45.480 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.613+0000 7fb43481c700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb430106e10 con 0x7fb430106420 2024-02-16T22:11:45.480 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.614+0000 7fb42e59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb430106420 0x7fb430106840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:45.480 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.614+0000 7fb42e59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb430106420 0x7fb430106840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34220/0 (socket says 172.21.15.114:34220) 2024-02-16T22:11:45.480 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.614+0000 7fb42e59c700 1 -- 172.21.15.114:0/2232725836 learned_addr learned my addr 172.21.15.114:0/2232725836 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:45.480 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.614+0000 7fb42e59c700 1 -- 172.21.15.114:0/2232725836 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb430107620 con 0x7fb430106420 2024-02-16T22:11:45.480 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.614+0000 7fb42e59c700 1 --2- 172.21.15.114:0/2232725836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb430106420 0x7fb430106840 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb4200098a0 tx=0x7fb420009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92a26b3580079a2 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:45.481 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.614+0000 7fb42d59a700 1 -- 172.21.15.114:0/2232725836 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb420004180 con 0x7fb430106420 2024-02-16T22:11:45.481 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.614+0000 7fb42d59a700 1 -- 172.21.15.114:0/2232725836 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb4200042e0 con 0x7fb430106420 2024-02-16T22:11:45.481 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.614+0000 7fb42d59a700 1 -- 172.21.15.114:0/2232725836 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb420011af0 con 0x7fb430106420 2024-02-16T22:11:45.481 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.615+0000 7fb43481c700 1 -- 172.21.15.114:0/2232725836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb430106420 msgr2=0x7fb430106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:45.481 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.615+0000 7fb43481c700 1 --2- 172.21.15.114:0/2232725836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb430106420 0x7fb430106840 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb4200098a0 tx=0x7fb420009bb0 comp rx=0 tx=0).stop 2024-02-16T22:11:45.481 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.615+0000 7fb43481c700 1 -- 172.21.15.114:0/2232725836 shutdown_connections 2024-02-16T22:11:45.481 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.615+0000 7fb43481c700 1 --2- 172.21.15.114:0/2232725836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb430106420 0x7fb430106840 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:45.481 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.615+0000 7fb43481c700 1 -- 172.21.15.114:0/2232725836 >> 172.21.15.114:0/2232725836 conn(0x7fb430101a00 msgr2=0x7fb430103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:45.481 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.615+0000 7fb43481c700 1 -- 172.21.15.114:0/2232725836 shutdown_connections 2024-02-16T22:11:45.481 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.615+0000 7fb43481c700 1 -- 172.21.15.114:0/2232725836 wait complete. 2024-02-16T22:11:45.481 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.616+0000 7fb43481c700 1 Processor -- start 2024-02-16T22:11:45.481 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.616+0000 7fb43481c700 1 -- start start 2024-02-16T22:11:45.481 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.616+0000 7fb43481c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb430106420 0x7fb43019a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:45.481 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.616+0000 7fb43481c700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43019a680 con 0x7fb430106420 2024-02-16T22:11:45.481 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.616+0000 7fb42e59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb430106420 0x7fb43019a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:45.482 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.616+0000 7fb42e59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb430106420 0x7fb43019a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34224/0 (socket says 172.21.15.114:34224) 2024-02-16T22:11:45.482 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.616+0000 7fb42e59c700 1 -- 172.21.15.114:0/370965511 learned_addr learned my addr 172.21.15.114:0/370965511 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:45.482 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.616+0000 7fb42e59c700 1 -- 172.21.15.114:0/370965511 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb420009550 con 0x7fb430106420 2024-02-16T22:11:45.482 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.616+0000 7fb42e59c700 1 --2- 172.21.15.114:0/370965511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb430106420 0x7fb43019a140 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb420006ba0 tx=0x7fb420011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:45.482 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.616+0000 7fb41f7fe700 1 -- 172.21.15.114:0/370965511 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb420003b00 con 0x7fb430106420 2024-02-16T22:11:45.482 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.616+0000 7fb41f7fe700 1 -- 172.21.15.114:0/370965511 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb42001a8e0 con 0x7fb430106420 2024-02-16T22:11:45.482 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.616+0000 7fb43481c700 1 -- 172.21.15.114:0/370965511 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb43019a880 con 0x7fb430106420 2024-02-16T22:11:45.482 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.616+0000 7fb41f7fe700 1 -- 172.21.15.114:0/370965511 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb420019990 con 0x7fb430106420 2024-02-16T22:11:45.482 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.617+0000 7fb43481c700 1 -- 172.21.15.114:0/370965511 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43019ad20 con 0x7fb430106420 2024-02-16T22:11:45.482 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.617+0000 7fb41f7fe700 1 -- 172.21.15.114:0/370965511 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fb420019c50 con 0x7fb430106420 2024-02-16T22:11:45.482 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.617+0000 7fb43481c700 1 -- 172.21.15.114:0/370965511 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4300623c0 con 0x7fb430106420 2024-02-16T22:11:45.482 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.617+0000 7fb41f7fe700 1 --2- 172.21.15.114:0/370965511 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fb4180382d0 0x7fb41803a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:45.482 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.618+0000 7fb41f7fe700 1 -- 172.21.15.114:0/370965511 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb42004c570 con 0x7fb430106420 2024-02-16T22:11:45.482 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.618+0000 7fb42dd9b700 1 --2- 172.21.15.114:0/370965511 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fb4180382d0 0x7fb41803a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:45.483 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.618+0000 7fb42dd9b700 1 --2- 172.21.15.114:0/370965511 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fb4180382d0 0x7fb41803a790 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb424006fd0 tx=0x7fb424006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:45.483 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.622+0000 7fb41f7fe700 1 -- 172.21.15.114:0/370965511 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb420018b40 con 0x7fb430106420 2024-02-16T22:11:45.483 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.835+0000 7fb43481c700 1 -- 172.21.15.114:0/370965511 --> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7fb43010a820 con 0x7fb4180382d0 2024-02-16T22:11:45.483 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.840+0000 7fb41f7fe700 1 -- 172.21.15.114:0/370965511 <== mgr.14120 v2:172.21.15.114:6800/625910309 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fb43010a820 con 0x7fb4180382d0 2024-02-16T22:11:45.483 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.842+0000 7fb43481c700 1 -- 172.21.15.114:0/370965511 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fb4180382d0 msgr2=0x7fb41803a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:45.483 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.842+0000 7fb43481c700 1 --2- 172.21.15.114:0/370965511 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fb4180382d0 0x7fb41803a790 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb424006fd0 tx=0x7fb424006e40 comp rx=0 tx=0).stop 2024-02-16T22:11:45.483 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.842+0000 7fb43481c700 1 -- 172.21.15.114:0/370965511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb430106420 msgr2=0x7fb43019a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:45.483 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.842+0000 7fb43481c700 1 --2- 172.21.15.114:0/370965511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb430106420 0x7fb43019a140 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb420006ba0 tx=0x7fb420011b50 comp rx=0 tx=0).stop 2024-02-16T22:11:45.483 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.842+0000 7fb43481c700 1 -- 172.21.15.114:0/370965511 shutdown_connections 2024-02-16T22:11:45.483 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.842+0000 7fb43481c700 1 --2- 172.21.15.114:0/370965511 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fb4180382d0 0x7fb41803a790 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:45.483 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.842+0000 7fb43481c700 1 --2- 172.21.15.114:0/370965511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb430106420 0x7fb43019a140 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:45.483 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.842+0000 7fb43481c700 1 -- 172.21.15.114:0/370965511 >> 172.21.15.114:0/370965511 conn(0x7fb430101a00 msgr2=0x7fb4301030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:45.483 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.842+0000 7fb43481c700 1 -- 172.21.15.114:0/370965511 shutdown_connections 2024-02-16T22:11:45.483 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:44.842+0000 7fb43481c700 1 -- 172.21.15.114:0/370965511 wait complete. 2024-02-16T22:11:45.484 INFO:teuthology.orchestra.run.smithi114.stdout:Deploying grafana service with default placement... 2024-02-16T22:11:46.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:45 smithi114 ceph-mon[29235]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:11:46.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:45 smithi114 ceph-mon[29235]: Saving service prometheus spec with placement count:1 2024-02-16T22:11:46.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:45 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' 2024-02-16T22:11:47.157 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-02-16T22:11:47.157 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.321+0000 7fc4c503e700 1 Processor -- start 2024-02-16T22:11:47.157 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.321+0000 7fc4c503e700 1 -- start start 2024-02-16T22:11:47.158 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.321+0000 7fc4c503e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c0105340 0x7fc4c0105760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:47.158 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.321+0000 7fc4c503e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c0105d30 con 0x7fc4c0105340 2024-02-16T22:11:47.158 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.321+0000 7fc4bed9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c0105340 0x7fc4c0105760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:47.158 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.321+0000 7fc4bed9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c0105340 0x7fc4c0105760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34234/0 (socket says 172.21.15.114:34234) 2024-02-16T22:11:47.158 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.321+0000 7fc4bed9d700 1 -- 172.21.15.114:0/2624955688 learned_addr learned my addr 172.21.15.114:0/2624955688 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:47.158 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.321+0000 7fc4bed9d700 1 -- 172.21.15.114:0/2624955688 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c0106540 con 0x7fc4c0105340 2024-02-16T22:11:47.158 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.322+0000 7fc4bed9d700 1 --2- 172.21.15.114:0/2624955688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c0105340 0x7fc4c0105760 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc4b00098a0 tx=0x7fc4b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca57b1475d4843e server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:47.158 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.322+0000 7fc4bdd9b700 1 -- 172.21.15.114:0/2624955688 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc4b0004180 con 0x7fc4c0105340 2024-02-16T22:11:47.158 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.322+0000 7fc4bdd9b700 1 -- 172.21.15.114:0/2624955688 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc4b00042e0 con 0x7fc4c0105340 2024-02-16T22:11:47.158 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.322+0000 7fc4bdd9b700 1 -- 172.21.15.114:0/2624955688 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc4b0011af0 con 0x7fc4c0105340 2024-02-16T22:11:47.158 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.322+0000 7fc4c503e700 1 -- 172.21.15.114:0/2624955688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c0105340 msgr2=0x7fc4c0105760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:47.158 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.322+0000 7fc4c503e700 1 --2- 172.21.15.114:0/2624955688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c0105340 0x7fc4c0105760 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc4b00098a0 tx=0x7fc4b0009bb0 comp rx=0 tx=0).stop 2024-02-16T22:11:47.158 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.322+0000 7fc4c503e700 1 -- 172.21.15.114:0/2624955688 shutdown_connections 2024-02-16T22:11:47.159 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.322+0000 7fc4c503e700 1 --2- 172.21.15.114:0/2624955688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c0105340 0x7fc4c0105760 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:47.159 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.322+0000 7fc4c503e700 1 -- 172.21.15.114:0/2624955688 >> 172.21.15.114:0/2624955688 conn(0x7fc4c01008c0 msgr2=0x7fc4c0102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:47.159 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.322+0000 7fc4c503e700 1 -- 172.21.15.114:0/2624955688 shutdown_connections 2024-02-16T22:11:47.159 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.322+0000 7fc4c503e700 1 -- 172.21.15.114:0/2624955688 wait complete. 2024-02-16T22:11:47.159 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.323+0000 7fc4c503e700 1 Processor -- start 2024-02-16T22:11:47.159 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.323+0000 7fc4c503e700 1 -- start start 2024-02-16T22:11:47.159 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.323+0000 7fc4c503e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c0105340 0x7fc4c0190530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:47.159 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.323+0000 7fc4c503e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c0190a70 con 0x7fc4c0105340 2024-02-16T22:11:47.159 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.324+0000 7fc4bed9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c0105340 0x7fc4c0190530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:47.159 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.324+0000 7fc4bed9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c0105340 0x7fc4c0190530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34244/0 (socket says 172.21.15.114:34244) 2024-02-16T22:11:47.159 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.324+0000 7fc4bed9d700 1 -- 172.21.15.114:0/240752569 learned_addr learned my addr 172.21.15.114:0/240752569 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:47.159 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.324+0000 7fc4bed9d700 1 -- 172.21.15.114:0/240752569 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b0009550 con 0x7fc4c0105340 2024-02-16T22:11:47.159 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.324+0000 7fc4bed9d700 1 --2- 172.21.15.114:0/240752569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c0105340 0x7fc4c0190530 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0011670 tx=0x7fc4b0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:47.159 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.324+0000 7fc4a7fff700 1 -- 172.21.15.114:0/240752569 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc4b00039c0 con 0x7fc4c0105340 2024-02-16T22:11:47.159 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.324+0000 7fc4c503e700 1 -- 172.21.15.114:0/240752569 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c018eb80 con 0x7fc4c0105340 2024-02-16T22:11:47.160 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.324+0000 7fc4a7fff700 1 -- 172.21.15.114:0/240752569 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc4b0003b20 con 0x7fc4c0105340 2024-02-16T22:11:47.160 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.324+0000 7fc4a7fff700 1 -- 172.21.15.114:0/240752569 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc4b00247f0 con 0x7fc4c0105340 2024-02-16T22:11:47.160 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.324+0000 7fc4c503e700 1 -- 172.21.15.114:0/240752569 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c018f020 con 0x7fc4c0105340 2024-02-16T22:11:47.160 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.325+0000 7fc4a7fff700 1 -- 172.21.15.114:0/240752569 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fc4b0024ab0 con 0x7fc4c0105340 2024-02-16T22:11:47.160 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.325+0000 7fc4a7fff700 1 --2- 172.21.15.114:0/240752569 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fc4a80382e0 0x7fc4a803a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:47.160 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.325+0000 7fc4c503e700 1 -- 172.21.15.114:0/240752569 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c00623c0 con 0x7fc4c0105340 2024-02-16T22:11:47.160 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.325+0000 7fc4a7fff700 1 -- 172.21.15.114:0/240752569 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc4b004c330 con 0x7fc4c0105340 2024-02-16T22:11:47.160 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.325+0000 7fc4be59c700 1 --2- 172.21.15.114:0/240752569 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fc4a80382e0 0x7fc4a803a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:47.160 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.326+0000 7fc4be59c700 1 --2- 172.21.15.114:0/240752569 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fc4a80382e0 0x7fc4a803a7a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4006fd0 tx=0x7fc4b4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:47.160 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.330+0000 7fc4a7fff700 1 -- 172.21.15.114:0/240752569 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc4b0022310 con 0x7fc4c0105340 2024-02-16T22:11:47.160 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.534+0000 7fc4c503e700 1 -- 172.21.15.114:0/240752569 --> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4c0102b10 con 0x7fc4a80382e0 2024-02-16T22:11:47.160 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.539+0000 7fc4a7fff700 1 -- 172.21.15.114:0/240752569 <== mgr.14120 v2:172.21.15.114:6800/625910309 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fc4c0102b10 con 0x7fc4a80382e0 2024-02-16T22:11:47.161 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.541+0000 7fc4c503e700 1 -- 172.21.15.114:0/240752569 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fc4a80382e0 msgr2=0x7fc4a803a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:47.161 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.541+0000 7fc4c503e700 1 --2- 172.21.15.114:0/240752569 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fc4a80382e0 0x7fc4a803a7a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4006fd0 tx=0x7fc4b4006e40 comp rx=0 tx=0).stop 2024-02-16T22:11:47.161 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.541+0000 7fc4c503e700 1 -- 172.21.15.114:0/240752569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c0105340 msgr2=0x7fc4c0190530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:47.161 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.541+0000 7fc4c503e700 1 --2- 172.21.15.114:0/240752569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c0105340 0x7fc4c0190530 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0011670 tx=0x7fc4b0011aa0 comp rx=0 tx=0).stop 2024-02-16T22:11:47.161 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.541+0000 7fc4c503e700 1 -- 172.21.15.114:0/240752569 shutdown_connections 2024-02-16T22:11:47.161 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.541+0000 7fc4c503e700 1 --2- 172.21.15.114:0/240752569 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fc4a80382e0 0x7fc4a803a7a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:47.161 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.541+0000 7fc4c503e700 1 --2- 172.21.15.114:0/240752569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c0105340 0x7fc4c0190530 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:47.161 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.541+0000 7fc4c503e700 1 -- 172.21.15.114:0/240752569 >> 172.21.15.114:0/240752569 conn(0x7fc4c01008c0 msgr2=0x7fc4c00740f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:47.161 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.541+0000 7fc4c503e700 1 -- 172.21.15.114:0/240752569 shutdown_connections 2024-02-16T22:11:47.161 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:46.541+0000 7fc4c503e700 1 -- 172.21.15.114:0/240752569 wait complete. 2024-02-16T22:11:47.161 INFO:teuthology.orchestra.run.smithi114.stdout:Deploying node-exporter service with default placement... 2024-02-16T22:11:47.796 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:47 smithi114 ceph-mon[29235]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:11:47.796 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:47 smithi114 ceph-mon[29235]: Saving service grafana spec with placement count:1 2024-02-16T22:11:47.796 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:47 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' 2024-02-16T22:11:48.406 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-02-16T22:11:48.406 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.565+0000 7fb069090700 1 Processor -- start 2024-02-16T22:11:48.406 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.565+0000 7fb069090700 1 -- start start 2024-02-16T22:11:48.406 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.565+0000 7fb069090700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb06407be70 0x7fb06407a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:48.406 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.565+0000 7fb069090700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06407a8a0 con 0x7fb06407be70 2024-02-16T22:11:48.406 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.565+0000 7fb062d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb06407be70 0x7fb06407a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:48.406 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.565+0000 7fb062d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb06407be70 0x7fb06407a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34256/0 (socket says 172.21.15.114:34256) 2024-02-16T22:11:48.406 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.565+0000 7fb062d9d700 1 -- 172.21.15.114:0/254285218 learned_addr learned my addr 172.21.15.114:0/254285218 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:48.406 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.565+0000 7fb062d9d700 1 -- 172.21.15.114:0/254285218 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb06407a9e0 con 0x7fb06407be70 2024-02-16T22:11:48.406 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.565+0000 7fb062d9d700 1 --2- 172.21.15.114:0/254285218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb06407be70 0x7fb06407a2d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb0540098a0 tx=0x7fb054009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b3061d89e1390a server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:48.406 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.566+0000 7fb061d9b700 1 -- 172.21.15.114:0/254285218 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb054004180 con 0x7fb06407be70 2024-02-16T22:11:48.406 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.566+0000 7fb061d9b700 1 -- 172.21.15.114:0/254285218 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb0540042e0 con 0x7fb06407be70 2024-02-16T22:11:48.407 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.566+0000 7fb061d9b700 1 -- 172.21.15.114:0/254285218 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb054011af0 con 0x7fb06407be70 2024-02-16T22:11:48.407 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.566+0000 7fb069090700 1 -- 172.21.15.114:0/254285218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb06407be70 msgr2=0x7fb06407a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:48.407 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.566+0000 7fb069090700 1 --2- 172.21.15.114:0/254285218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb06407be70 0x7fb06407a2d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb0540098a0 tx=0x7fb054009bb0 comp rx=0 tx=0).stop 2024-02-16T22:11:48.407 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.566+0000 7fb069090700 1 -- 172.21.15.114:0/254285218 shutdown_connections 2024-02-16T22:11:48.407 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.566+0000 7fb069090700 1 --2- 172.21.15.114:0/254285218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb06407be70 0x7fb06407a2d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:48.407 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.566+0000 7fb069090700 1 -- 172.21.15.114:0/254285218 >> 172.21.15.114:0/254285218 conn(0x7fb064101a00 msgr2=0x7fb064103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:48.407 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.566+0000 7fb069090700 1 -- 172.21.15.114:0/254285218 shutdown_connections 2024-02-16T22:11:48.407 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.566+0000 7fb069090700 1 -- 172.21.15.114:0/254285218 wait complete. 2024-02-16T22:11:48.407 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.567+0000 7fb069090700 1 Processor -- start 2024-02-16T22:11:48.407 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.567+0000 7fb069090700 1 -- start start 2024-02-16T22:11:48.407 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.567+0000 7fb069090700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb06407be70 0x7fb06419c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:48.407 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.567+0000 7fb069090700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06419c7e0 con 0x7fb06407be70 2024-02-16T22:11:48.407 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.567+0000 7fb062d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb06407be70 0x7fb06419c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:48.407 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.567+0000 7fb062d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb06407be70 0x7fb06419c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34264/0 (socket says 172.21.15.114:34264) 2024-02-16T22:11:48.408 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.567+0000 7fb062d9d700 1 -- 172.21.15.114:0/544290629 learned_addr learned my addr 172.21.15.114:0/544290629 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:48.408 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.568+0000 7fb062d9d700 1 -- 172.21.15.114:0/544290629 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb054009550 con 0x7fb06407be70 2024-02-16T22:11:48.408 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.568+0000 7fb062d9d700 1 --2- 172.21.15.114:0/544290629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb06407be70 0x7fb06419c2a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb054011ad0 tx=0x7fb054011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:48.408 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.568+0000 7fb04bfff700 1 -- 172.21.15.114:0/544290629 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb054003ab0 con 0x7fb06407be70 2024-02-16T22:11:48.408 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.568+0000 7fb04bfff700 1 -- 172.21.15.114:0/544290629 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb054003c10 con 0x7fb06407be70 2024-02-16T22:11:48.408 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.568+0000 7fb069090700 1 -- 172.21.15.114:0/544290629 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb06419c9e0 con 0x7fb06407be70 2024-02-16T22:11:48.408 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.568+0000 7fb04bfff700 1 -- 172.21.15.114:0/544290629 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb0540198e0 con 0x7fb06407be70 2024-02-16T22:11:48.408 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.568+0000 7fb069090700 1 -- 172.21.15.114:0/544290629 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb06419ce80 con 0x7fb06407be70 2024-02-16T22:11:48.408 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.569+0000 7fb04bfff700 1 -- 172.21.15.114:0/544290629 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fb054019ba0 con 0x7fb06407be70 2024-02-16T22:11:48.408 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.569+0000 7fb04bfff700 1 --2- 172.21.15.114:0/544290629 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fb04c038320 0x7fb04c03a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:48.408 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.569+0000 7fb069090700 1 -- 172.21.15.114:0/544290629 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb06410c7c0 con 0x7fb06407be70 2024-02-16T22:11:48.408 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.569+0000 7fb04bfff700 1 -- 172.21.15.114:0/544290629 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb05404c1d0 con 0x7fb06407be70 2024-02-16T22:11:48.408 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.569+0000 7fb06259c700 1 --2- 172.21.15.114:0/544290629 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fb04c038320 0x7fb04c03a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:48.409 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.570+0000 7fb06259c700 1 --2- 172.21.15.114:0/544290629 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fb04c038320 0x7fb04c03a7e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb058006fd0 tx=0x7fb058006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:48.409 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.574+0000 7fb04bfff700 1 -- 172.21.15.114:0/544290629 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb054010d10 con 0x7fb06407be70 2024-02-16T22:11:48.409 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.776+0000 7fb069090700 1 -- 172.21.15.114:0/544290629 --> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7fb064102ee0 con 0x7fb04c038320 2024-02-16T22:11:48.409 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.781+0000 7fb04bfff700 1 -- 172.21.15.114:0/544290629 <== mgr.14120 v2:172.21.15.114:6800/625910309 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7fb064102ee0 con 0x7fb04c038320 2024-02-16T22:11:48.409 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.783+0000 7fb069090700 1 -- 172.21.15.114:0/544290629 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fb04c038320 msgr2=0x7fb04c03a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:48.409 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.783+0000 7fb069090700 1 --2- 172.21.15.114:0/544290629 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fb04c038320 0x7fb04c03a7e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb058006fd0 tx=0x7fb058006e40 comp rx=0 tx=0).stop 2024-02-16T22:11:48.409 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.783+0000 7fb069090700 1 -- 172.21.15.114:0/544290629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb06407be70 msgr2=0x7fb06419c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:48.409 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.783+0000 7fb069090700 1 --2- 172.21.15.114:0/544290629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb06407be70 0x7fb06419c2a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb054011ad0 tx=0x7fb054011bb0 comp rx=0 tx=0).stop 2024-02-16T22:11:48.409 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.783+0000 7fb069090700 1 -- 172.21.15.114:0/544290629 shutdown_connections 2024-02-16T22:11:48.409 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.783+0000 7fb069090700 1 --2- 172.21.15.114:0/544290629 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fb04c038320 0x7fb04c03a7e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:48.409 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.783+0000 7fb069090700 1 --2- 172.21.15.114:0/544290629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb06407be70 0x7fb06419c2a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:48.409 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.783+0000 7fb069090700 1 -- 172.21.15.114:0/544290629 >> 172.21.15.114:0/544290629 conn(0x7fb064101a00 msgr2=0x7fb0641027d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:48.409 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.784+0000 7fb069090700 1 -- 172.21.15.114:0/544290629 shutdown_connections 2024-02-16T22:11:48.409 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:47.784+0000 7fb069090700 1 -- 172.21.15.114:0/544290629 wait complete. 2024-02-16T22:11:48.410 INFO:teuthology.orchestra.run.smithi114.stdout:Deploying alertmanager service with default placement... 2024-02-16T22:11:49.036 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:48 smithi114 ceph-mon[29235]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:11:49.036 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:48 smithi114 ceph-mon[29235]: Saving service node-exporter spec with placement * 2024-02-16T22:11:49.036 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:48 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' 2024-02-16T22:11:49.036 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:48 smithi114 ceph-mon[29235]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:11:49.663 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-02-16T22:11:49.663 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.808+0000 7fdb12fc5700 1 Processor -- start 2024-02-16T22:11:49.663 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.808+0000 7fdb12fc5700 1 -- start start 2024-02-16T22:11:49.663 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.809+0000 7fdb12fc5700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb0c07bed0 0x7fdb0c07a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:49.663 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.809+0000 7fdb12fc5700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c07a900 con 0x7fdb0c07bed0 2024-02-16T22:11:49.663 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.809+0000 7fdb10d61700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb0c07bed0 0x7fdb0c07a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:49.663 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.809+0000 7fdb10d61700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb0c07bed0 0x7fdb0c07a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34278/0 (socket says 172.21.15.114:34278) 2024-02-16T22:11:49.663 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.809+0000 7fdb10d61700 1 -- 172.21.15.114:0/1083618362 learned_addr learned my addr 172.21.15.114:0/1083618362 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:49.663 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.809+0000 7fdb10d61700 1 -- 172.21.15.114:0/1083618362 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0c07aa40 con 0x7fdb0c07bed0 2024-02-16T22:11:49.663 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.809+0000 7fdb10d61700 1 --2- 172.21.15.114:0/1083618362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb0c07bed0 0x7fdb0c07a330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdb000098a0 tx=0x7fdb00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a546590cc83141 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:49.664 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.809+0000 7fdb0b7fe700 1 -- 172.21.15.114:0/1083618362 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb00004180 con 0x7fdb0c07bed0 2024-02-16T22:11:49.664 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.809+0000 7fdb0b7fe700 1 -- 172.21.15.114:0/1083618362 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdb000042e0 con 0x7fdb0c07bed0 2024-02-16T22:11:49.664 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.809+0000 7fdb0b7fe700 1 -- 172.21.15.114:0/1083618362 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb00011af0 con 0x7fdb0c07bed0 2024-02-16T22:11:49.664 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.810+0000 7fdb12fc5700 1 -- 172.21.15.114:0/1083618362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb0c07bed0 msgr2=0x7fdb0c07a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:49.664 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.810+0000 7fdb12fc5700 1 --2- 172.21.15.114:0/1083618362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb0c07bed0 0x7fdb0c07a330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdb000098a0 tx=0x7fdb00009bb0 comp rx=0 tx=0).stop 2024-02-16T22:11:49.664 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.810+0000 7fdb12fc5700 1 -- 172.21.15.114:0/1083618362 shutdown_connections 2024-02-16T22:11:49.664 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.810+0000 7fdb12fc5700 1 --2- 172.21.15.114:0/1083618362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb0c07bed0 0x7fdb0c07a330 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:49.664 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.810+0000 7fdb12fc5700 1 -- 172.21.15.114:0/1083618362 >> 172.21.15.114:0/1083618362 conn(0x7fdb0c101a00 msgr2=0x7fdb0c103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:49.664 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.810+0000 7fdb12fc5700 1 -- 172.21.15.114:0/1083618362 shutdown_connections 2024-02-16T22:11:49.664 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.810+0000 7fdb12fc5700 1 -- 172.21.15.114:0/1083618362 wait complete. 2024-02-16T22:11:49.664 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.810+0000 7fdb12fc5700 1 Processor -- start 2024-02-16T22:11:49.664 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.810+0000 7fdb12fc5700 1 -- start start 2024-02-16T22:11:49.664 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.811+0000 7fdb12fc5700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb0c07bed0 0x7fdb0c19c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:49.664 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.811+0000 7fdb12fc5700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c19c880 con 0x7fdb0c07bed0 2024-02-16T22:11:49.665 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.811+0000 7fdb10d61700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb0c07bed0 0x7fdb0c19c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:49.665 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.811+0000 7fdb10d61700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb0c07bed0 0x7fdb0c19c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34286/0 (socket says 172.21.15.114:34286) 2024-02-16T22:11:49.665 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.811+0000 7fdb10d61700 1 -- 172.21.15.114:0/2767350908 learned_addr learned my addr 172.21.15.114:0/2767350908 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:49.665 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.811+0000 7fdb10d61700 1 -- 172.21.15.114:0/2767350908 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb00009550 con 0x7fdb0c07bed0 2024-02-16T22:11:49.665 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.811+0000 7fdb10d61700 1 --2- 172.21.15.114:0/2767350908 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb0c07bed0 0x7fdb0c19c340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdb00006ba0 tx=0x7fdb00011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:49.665 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.811+0000 7fdb09ffb700 1 -- 172.21.15.114:0/2767350908 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb00003b00 con 0x7fdb0c07bed0 2024-02-16T22:11:49.665 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.811+0000 7fdb09ffb700 1 -- 172.21.15.114:0/2767350908 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdb0001a8e0 con 0x7fdb0c07bed0 2024-02-16T22:11:49.665 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.811+0000 7fdb12fc5700 1 -- 172.21.15.114:0/2767350908 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb0c19ca80 con 0x7fdb0c07bed0 2024-02-16T22:11:49.665 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.811+0000 7fdb09ffb700 1 -- 172.21.15.114:0/2767350908 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb00019990 con 0x7fdb0c07bed0 2024-02-16T22:11:49.665 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.812+0000 7fdb12fc5700 1 -- 172.21.15.114:0/2767350908 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb0c19cf20 con 0x7fdb0c07bed0 2024-02-16T22:11:49.665 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.812+0000 7fdb09ffb700 1 -- 172.21.15.114:0/2767350908 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fdb00019c50 con 0x7fdb0c07bed0 2024-02-16T22:11:49.665 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.812+0000 7fdb12fc5700 1 -- 172.21.15.114:0/2767350908 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb0c10c7c0 con 0x7fdb0c07bed0 2024-02-16T22:11:49.665 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.812+0000 7fdb09ffb700 1 --2- 172.21.15.114:0/2767350908 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fdafc0382d0 0x7fdafc03a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:49.665 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.812+0000 7fdb09ffb700 1 -- 172.21.15.114:0/2767350908 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdb0004c750 con 0x7fdb0c07bed0 2024-02-16T22:11:49.665 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.813+0000 7fdb0bfff700 1 --2- 172.21.15.114:0/2767350908 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fdafc0382d0 0x7fdafc03a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:49.666 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.813+0000 7fdb0bfff700 1 --2- 172.21.15.114:0/2767350908 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fdafc0382d0 0x7fdafc03a790 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8006fd0 tx=0x7fdaf8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:49.666 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:48.817+0000 7fdb09ffb700 1 -- 172.21.15.114:0/2767350908 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdb00018b40 con 0x7fdb0c07bed0 2024-02-16T22:11:49.666 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:49.016+0000 7fdb12fc5700 1 -- 172.21.15.114:0/2767350908 --> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb0c103cc0 con 0x7fdafc0382d0 2024-02-16T22:11:49.666 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:49.021+0000 7fdb09ffb700 1 -- 172.21.15.114:0/2767350908 <== mgr.14120 v2:172.21.15.114:6800/625910309 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7fdb0c103cc0 con 0x7fdafc0382d0 2024-02-16T22:11:49.666 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:49.023+0000 7fdb12fc5700 1 -- 172.21.15.114:0/2767350908 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fdafc0382d0 msgr2=0x7fdafc03a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:49.666 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:49.023+0000 7fdb12fc5700 1 --2- 172.21.15.114:0/2767350908 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fdafc0382d0 0x7fdafc03a790 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8006fd0 tx=0x7fdaf8006e40 comp rx=0 tx=0).stop 2024-02-16T22:11:49.666 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:49.023+0000 7fdb12fc5700 1 -- 172.21.15.114:0/2767350908 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb0c07bed0 msgr2=0x7fdb0c19c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:49.666 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:49.023+0000 7fdb12fc5700 1 --2- 172.21.15.114:0/2767350908 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb0c07bed0 0x7fdb0c19c340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdb00006ba0 tx=0x7fdb00011b50 comp rx=0 tx=0).stop 2024-02-16T22:11:49.666 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:49.023+0000 7fdb12fc5700 1 -- 172.21.15.114:0/2767350908 shutdown_connections 2024-02-16T22:11:49.666 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:49.023+0000 7fdb12fc5700 1 --2- 172.21.15.114:0/2767350908 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7fdafc0382d0 0x7fdafc03a790 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:49.666 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:49.023+0000 7fdb12fc5700 1 --2- 172.21.15.114:0/2767350908 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb0c07bed0 0x7fdb0c19c340 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:49.666 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:49.023+0000 7fdb12fc5700 1 -- 172.21.15.114:0/2767350908 >> 172.21.15.114:0/2767350908 conn(0x7fdb0c101a00 msgr2=0x7fdb0c1030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:49.666 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:49.024+0000 7fdb12fc5700 1 -- 172.21.15.114:0/2767350908 shutdown_connections 2024-02-16T22:11:49.666 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:49.024+0000 7fdb12fc5700 1 -- 172.21.15.114:0/2767350908 wait complete. 2024-02-16T22:11:50.267 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:50 smithi114 ceph-mon[29235]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:11:50.267 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:50 smithi114 ceph-mon[29235]: Saving service alertmanager spec with placement count:1 2024-02-16T22:11:50.267 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:50 smithi114 ceph-mon[29235]: from='mgr.14120 172.21.15.114:0/2164534274' entity='mgr.smithi114.vcullz' 2024-02-16T22:11:50.938 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.070+0000 7f9766ef5700 1 Processor -- start 2024-02-16T22:11:50.938 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.070+0000 7f9766ef5700 1 -- start start 2024-02-16T22:11:50.938 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.070+0000 7f9766ef5700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9760106ea0 0x7f9760109290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:50.938 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.070+0000 7f9766ef5700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9760079ec0 con 0x7f9760106ea0 2024-02-16T22:11:50.938 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.070+0000 7f9764c91700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9760106ea0 0x7f9760109290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:50.938 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.070+0000 7f9764c91700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9760106ea0 0x7f9760109290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34290/0 (socket says 172.21.15.114:34290) 2024-02-16T22:11:50.938 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.070+0000 7f9764c91700 1 -- 172.21.15.114:0/3281758279 learned_addr learned my addr 172.21.15.114:0/3281758279 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:50.939 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.071+0000 7f9764c91700 1 -- 172.21.15.114:0/3281758279 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97601097d0 con 0x7f9760106ea0 2024-02-16T22:11:50.939 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.071+0000 7f9764c91700 1 --2- 172.21.15.114:0/3281758279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9760106ea0 0x7f9760109290 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f97540098a0 tx=0x7f9754009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744a1ef88b47dcdc server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:50.939 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.071+0000 7f975f7fe700 1 -- 172.21.15.114:0/3281758279 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9754004180 con 0x7f9760106ea0 2024-02-16T22:11:50.939 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.071+0000 7f975f7fe700 1 -- 172.21.15.114:0/3281758279 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f97540042e0 con 0x7f9760106ea0 2024-02-16T22:11:50.939 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.071+0000 7f975f7fe700 1 -- 172.21.15.114:0/3281758279 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9754011af0 con 0x7f9760106ea0 2024-02-16T22:11:50.939 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.071+0000 7f9766ef5700 1 -- 172.21.15.114:0/3281758279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9760106ea0 msgr2=0x7f9760109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:50.939 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.071+0000 7f9766ef5700 1 --2- 172.21.15.114:0/3281758279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9760106ea0 0x7f9760109290 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f97540098a0 tx=0x7f9754009bb0 comp rx=0 tx=0).stop 2024-02-16T22:11:50.939 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.071+0000 7f9766ef5700 1 -- 172.21.15.114:0/3281758279 shutdown_connections 2024-02-16T22:11:50.939 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.071+0000 7f9766ef5700 1 --2- 172.21.15.114:0/3281758279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9760106ea0 0x7f9760109290 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:50.939 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.071+0000 7f9766ef5700 1 -- 172.21.15.114:0/3281758279 >> 172.21.15.114:0/3281758279 conn(0x7f97601008b0 msgr2=0x7f9760102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:50.939 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.071+0000 7f9766ef5700 1 -- 172.21.15.114:0/3281758279 shutdown_connections 2024-02-16T22:11:50.939 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.071+0000 7f9766ef5700 1 -- 172.21.15.114:0/3281758279 wait complete. 2024-02-16T22:11:50.939 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.072+0000 7f9766ef5700 1 Processor -- start 2024-02-16T22:11:50.939 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.072+0000 7f9766ef5700 1 -- start start 2024-02-16T22:11:50.939 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.072+0000 7f9766ef5700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9760106ea0 0x7f976019a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:50.940 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.072+0000 7f9766ef5700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976019a670 con 0x7f9760106ea0 2024-02-16T22:11:50.940 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.072+0000 7f9764c91700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9760106ea0 0x7f976019a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:50.940 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.072+0000 7f9764c91700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9760106ea0 0x7f976019a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34292/0 (socket says 172.21.15.114:34292) 2024-02-16T22:11:50.940 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.072+0000 7f9764c91700 1 -- 172.21.15.114:0/3066551282 learned_addr learned my addr 172.21.15.114:0/3066551282 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:50.940 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.073+0000 7f9764c91700 1 -- 172.21.15.114:0/3066551282 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9754009550 con 0x7f9760106ea0 2024-02-16T22:11:50.940 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.073+0000 7f9764c91700 1 --2- 172.21.15.114:0/3066551282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9760106ea0 0x7f976019a130 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9754011670 tx=0x7f9754011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:50.940 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.073+0000 7f975dffb700 1 -- 172.21.15.114:0/3066551282 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9754003a40 con 0x7f9760106ea0 2024-02-16T22:11:50.940 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.073+0000 7f9766ef5700 1 -- 172.21.15.114:0/3066551282 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f976019a870 con 0x7f9760106ea0 2024-02-16T22:11:50.940 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.073+0000 7f975dffb700 1 -- 172.21.15.114:0/3066551282 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9754003ba0 con 0x7f9760106ea0 2024-02-16T22:11:50.940 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.073+0000 7f975dffb700 1 -- 172.21.15.114:0/3066551282 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9754024790 con 0x7f9760106ea0 2024-02-16T22:11:50.940 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.073+0000 7f9766ef5700 1 -- 172.21.15.114:0/3066551282 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f976019ad10 con 0x7f9760106ea0 2024-02-16T22:11:50.940 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.074+0000 7f975dffb700 1 -- 172.21.15.114:0/3066551282 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f975402d3f0 con 0x7f9760106ea0 2024-02-16T22:11:50.940 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.074+0000 7f9766ef5700 1 -- 172.21.15.114:0/3066551282 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f976010a5c0 con 0x7f9760106ea0 2024-02-16T22:11:50.941 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.074+0000 7f975dffb700 1 --2- 172.21.15.114:0/3066551282 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f9750038320 0x7f975003a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:50.941 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.074+0000 7f975dffb700 1 -- 172.21.15.114:0/3066551282 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f975404c710 con 0x7f9760106ea0 2024-02-16T22:11:50.941 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.074+0000 7f975ffff700 1 --2- 172.21.15.114:0/3066551282 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f9750038320 0x7f975003a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:50.941 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.075+0000 7f975ffff700 1 --2- 172.21.15.114:0/3066551282 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f9750038320 0x7f975003a7e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f974c006fd0 tx=0x7f974c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:50.941 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.079+0000 7f975dffb700 1 -- 172.21.15.114:0/3066551282 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9754010970 con 0x7f9760106ea0 2024-02-16T22:11:50.941 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.265+0000 7f9766ef5700 1 -- 172.21.15.114:0/3066551282 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f976010a7e0 con 0x7f9760106ea0 2024-02-16T22:11:50.941 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.268+0000 7f975dffb700 1 -- 172.21.15.114:0/3066551282 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f975404c080 con 0x7f9760106ea0 2024-02-16T22:11:50.941 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.271+0000 7f9766ef5700 1 -- 172.21.15.114:0/3066551282 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f9750038320 msgr2=0x7f975003a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:50.941 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.271+0000 7f9766ef5700 1 --2- 172.21.15.114:0/3066551282 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f9750038320 0x7f975003a7e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f974c006fd0 tx=0x7f974c006e40 comp rx=0 tx=0).stop 2024-02-16T22:11:50.941 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.271+0000 7f9766ef5700 1 -- 172.21.15.114:0/3066551282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9760106ea0 msgr2=0x7f976019a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:50.941 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.271+0000 7f9766ef5700 1 --2- 172.21.15.114:0/3066551282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9760106ea0 0x7f976019a130 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9754011670 tx=0x7f9754011b50 comp rx=0 tx=0).stop 2024-02-16T22:11:50.941 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.271+0000 7f9766ef5700 1 -- 172.21.15.114:0/3066551282 shutdown_connections 2024-02-16T22:11:50.941 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.271+0000 7f9766ef5700 1 --2- 172.21.15.114:0/3066551282 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f9750038320 0x7f975003a7e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:50.942 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.271+0000 7f9766ef5700 1 --2- 172.21.15.114:0/3066551282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9760106ea0 0x7f976019a130 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:50.942 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.271+0000 7f9766ef5700 1 -- 172.21.15.114:0/3066551282 >> 172.21.15.114:0/3066551282 conn(0x7f97601008b0 msgr2=0x7f9760101fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:50.942 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.271+0000 7f9766ef5700 1 -- 172.21.15.114:0/3066551282 shutdown_connections 2024-02-16T22:11:50.942 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:50.271+0000 7f9766ef5700 1 -- 172.21.15.114:0/3066551282 wait complete. 2024-02-16T22:11:51.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:51 smithi114 ceph-mon[29235]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:11:51.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:51 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/3066551282' entity='client.admin' 2024-02-16T22:11:52.176 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.353+0000 7f7af3f1c700 1 Processor -- start 2024-02-16T22:11:52.176 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.353+0000 7f7af3f1c700 1 -- start start 2024-02-16T22:11:52.176 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.353+0000 7f7af3f1c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7aec104730 0x7f7aec104b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:52.177 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.353+0000 7f7af3f1c700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec105120 con 0x7f7aec104730 2024-02-16T22:11:52.177 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.353+0000 7f7af1cb8700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7aec104730 0x7f7aec104b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:52.177 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.353+0000 7f7af1cb8700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7aec104730 0x7f7aec104b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34302/0 (socket says 172.21.15.114:34302) 2024-02-16T22:11:52.177 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.353+0000 7f7af1cb8700 1 -- 172.21.15.114:0/2249985295 learned_addr learned my addr 172.21.15.114:0/2249985295 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:52.177 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.354+0000 7f7af1cb8700 1 -- 172.21.15.114:0/2249985295 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aec105260 con 0x7f7aec104730 2024-02-16T22:11:52.177 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.354+0000 7f7af1cb8700 1 --2- 172.21.15.114:0/2249985295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7aec104730 0x7f7aec104b50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80098a0 tx=0x7f7ae8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64cb8ac2cad9535b server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:52.177 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.354+0000 7f7af0cb6700 1 -- 172.21.15.114:0/2249985295 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7ae8004180 con 0x7f7aec104730 2024-02-16T22:11:52.177 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.354+0000 7f7af0cb6700 1 -- 172.21.15.114:0/2249985295 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7ae80042e0 con 0x7f7aec104730 2024-02-16T22:11:52.177 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.354+0000 7f7af0cb6700 1 -- 172.21.15.114:0/2249985295 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7ae8011af0 con 0x7f7aec104730 2024-02-16T22:11:52.177 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.354+0000 7f7af3f1c700 1 -- 172.21.15.114:0/2249985295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7aec104730 msgr2=0x7f7aec104b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:52.177 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.354+0000 7f7af3f1c700 1 --2- 172.21.15.114:0/2249985295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7aec104730 0x7f7aec104b50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80098a0 tx=0x7f7ae8009bb0 comp rx=0 tx=0).stop 2024-02-16T22:11:52.177 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.354+0000 7f7af3f1c700 1 -- 172.21.15.114:0/2249985295 shutdown_connections 2024-02-16T22:11:52.177 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.354+0000 7f7af3f1c700 1 --2- 172.21.15.114:0/2249985295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7aec104730 0x7f7aec104b50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:52.177 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.354+0000 7f7af3f1c700 1 -- 172.21.15.114:0/2249985295 >> 172.21.15.114:0/2249985295 conn(0x7f7aec100040 msgr2=0x7f7aec1024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:52.177 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.355+0000 7f7af3f1c700 1 -- 172.21.15.114:0/2249985295 shutdown_connections 2024-02-16T22:11:52.178 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.355+0000 7f7af3f1c700 1 -- 172.21.15.114:0/2249985295 wait complete. 2024-02-16T22:11:52.178 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.355+0000 7f7af3f1c700 1 Processor -- start 2024-02-16T22:11:52.178 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.355+0000 7f7af3f1c700 1 -- start start 2024-02-16T22:11:52.178 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.355+0000 7f7af3f1c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7aec104730 0x7f7aec19a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:52.178 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.355+0000 7f7af3f1c700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec19a6c0 con 0x7f7aec104730 2024-02-16T22:11:52.178 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.355+0000 7f7af1cb8700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7aec104730 0x7f7aec19a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:52.178 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.355+0000 7f7af1cb8700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7aec104730 0x7f7aec19a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34314/0 (socket says 172.21.15.114:34314) 2024-02-16T22:11:52.178 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.355+0000 7f7af1cb8700 1 -- 172.21.15.114:0/920158197 learned_addr learned my addr 172.21.15.114:0/920158197 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:52.178 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.356+0000 7f7af1cb8700 1 -- 172.21.15.114:0/920158197 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae8009550 con 0x7f7aec104730 2024-02-16T22:11:52.178 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.356+0000 7f7af1cb8700 1 --2- 172.21.15.114:0/920158197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7aec104730 0x7f7aec19a180 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8011670 tx=0x7f7ae8011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:52.178 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.356+0000 7f7ae2ffd700 1 -- 172.21.15.114:0/920158197 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7ae80039f0 con 0x7f7aec104730 2024-02-16T22:11:52.178 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.356+0000 7f7ae2ffd700 1 -- 172.21.15.114:0/920158197 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7ae8003b50 con 0x7f7aec104730 2024-02-16T22:11:52.178 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.356+0000 7f7af3f1c700 1 -- 172.21.15.114:0/920158197 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aec19a8c0 con 0x7f7aec104730 2024-02-16T22:11:52.178 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.356+0000 7f7ae2ffd700 1 -- 172.21.15.114:0/920158197 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7ae80247c0 con 0x7f7aec104730 2024-02-16T22:11:52.178 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.356+0000 7f7af3f1c700 1 -- 172.21.15.114:0/920158197 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aec19ad60 con 0x7f7aec104730 2024-02-16T22:11:52.179 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.357+0000 7f7ae2ffd700 1 -- 172.21.15.114:0/920158197 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f7ae802d3f0 con 0x7f7aec104730 2024-02-16T22:11:52.179 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.357+0000 7f7ae2ffd700 1 --2- 172.21.15.114:0/920158197 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f7ad8038370 0x7f7ad803a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:52.179 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.357+0000 7f7ae2ffd700 1 -- 172.21.15.114:0/920158197 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7ae804c360 con 0x7f7aec104730 2024-02-16T22:11:52.179 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.357+0000 7f7af3f1c700 1 -- 172.21.15.114:0/920158197 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aec0623c0 con 0x7f7aec104730 2024-02-16T22:11:52.179 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.357+0000 7f7af14b7700 1 --2- 172.21.15.114:0/920158197 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f7ad8038370 0x7f7ad803a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:52.179 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.358+0000 7f7af14b7700 1 --2- 172.21.15.114:0/920158197 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f7ad8038370 0x7f7ad803a830 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7adc006fd0 tx=0x7f7adc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:52.179 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.362+0000 7f7ae2ffd700 1 -- 172.21.15.114:0/920158197 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7ae802d6f0 con 0x7f7aec104730 2024-02-16T22:11:52.179 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.549+0000 7f7af3f1c700 1 -- 172.21.15.114:0/920158197 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f7aec1083d0 con 0x7f7aec104730 2024-02-16T22:11:52.179 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.553+0000 7f7ae2ffd700 1 -- 172.21.15.114:0/920158197 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f7ae8020220 con 0x7f7aec104730 2024-02-16T22:11:52.179 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.555+0000 7f7af3f1c700 1 -- 172.21.15.114:0/920158197 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f7ad8038370 msgr2=0x7f7ad803a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:52.179 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.555+0000 7f7af3f1c700 1 --2- 172.21.15.114:0/920158197 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f7ad8038370 0x7f7ad803a830 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7adc006fd0 tx=0x7f7adc006e40 comp rx=0 tx=0).stop 2024-02-16T22:11:52.179 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.555+0000 7f7af3f1c700 1 -- 172.21.15.114:0/920158197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7aec104730 msgr2=0x7f7aec19a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:52.179 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.555+0000 7f7af3f1c700 1 --2- 172.21.15.114:0/920158197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7aec104730 0x7f7aec19a180 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8011670 tx=0x7f7ae8011b50 comp rx=0 tx=0).stop 2024-02-16T22:11:52.179 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.555+0000 7f7af3f1c700 1 -- 172.21.15.114:0/920158197 shutdown_connections 2024-02-16T22:11:52.179 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.555+0000 7f7af3f1c700 1 --2- 172.21.15.114:0/920158197 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f7ad8038370 0x7f7ad803a830 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:52.180 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.555+0000 7f7af3f1c700 1 --2- 172.21.15.114:0/920158197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7aec104730 0x7f7aec19a180 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:52.180 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.555+0000 7f7af3f1c700 1 -- 172.21.15.114:0/920158197 >> 172.21.15.114:0/920158197 conn(0x7f7aec100040 msgr2=0x7f7aec101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:52.180 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.555+0000 7f7af3f1c700 1 -- 172.21.15.114:0/920158197 shutdown_connections 2024-02-16T22:11:52.180 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:51.555+0000 7f7af3f1c700 1 -- 172.21.15.114:0/920158197 wait complete. 2024-02-16T22:11:52.180 INFO:teuthology.orchestra.run.smithi114.stdout:Enabling the dashboard module... 2024-02-16T22:11:52.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:52 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/920158197' entity='client.admin' 2024-02-16T22:11:52.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:52 smithi114 ceph-mon[29235]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:11:54.133 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:53 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/2995026646' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-16T22:11:54.391 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.626+0000 7f05c91ee700 1 Processor -- start 2024-02-16T22:11:54.392 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.626+0000 7f05c91ee700 1 -- start start 2024-02-16T22:11:54.392 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.627+0000 7f05c91ee700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05c4106f00 0x7f05c41092f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:54.392 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.627+0000 7f05c91ee700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c4079e60 con 0x7f05c4106f00 2024-02-16T22:11:54.392 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.627+0000 7f05c2d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05c4106f00 0x7f05c41092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:54.392 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.627+0000 7f05c2d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05c4106f00 0x7f05c41092f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60434/0 (socket says 172.21.15.114:60434) 2024-02-16T22:11:54.392 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.627+0000 7f05c2d9d700 1 -- 172.21.15.114:0/4066115986 learned_addr learned my addr 172.21.15.114:0/4066115986 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:54.392 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.627+0000 7f05c2d9d700 1 -- 172.21.15.114:0/4066115986 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c4109830 con 0x7f05c4106f00 2024-02-16T22:11:54.392 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.627+0000 7f05c2d9d700 1 --2- 172.21.15.114:0/4066115986 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05c4106f00 0x7f05c41092f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f05b80098a0 tx=0x7f05b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef43b1904d720cd5 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:54.392 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.627+0000 7f05c1d9b700 1 -- 172.21.15.114:0/4066115986 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05b8004180 con 0x7f05c4106f00 2024-02-16T22:11:54.392 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.627+0000 7f05c1d9b700 1 -- 172.21.15.114:0/4066115986 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f05b80042e0 con 0x7f05c4106f00 2024-02-16T22:11:54.392 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.627+0000 7f05c1d9b700 1 -- 172.21.15.114:0/4066115986 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05b8011af0 con 0x7f05c4106f00 2024-02-16T22:11:54.392 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.628+0000 7f05c91ee700 1 -- 172.21.15.114:0/4066115986 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05c4106f00 msgr2=0x7f05c41092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:54.392 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.628+0000 7f05c91ee700 1 --2- 172.21.15.114:0/4066115986 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05c4106f00 0x7f05c41092f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f05b80098a0 tx=0x7f05b8009bb0 comp rx=0 tx=0).stop 2024-02-16T22:11:54.393 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.628+0000 7f05c91ee700 1 -- 172.21.15.114:0/4066115986 shutdown_connections 2024-02-16T22:11:54.393 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.628+0000 7f05c91ee700 1 --2- 172.21.15.114:0/4066115986 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05c4106f00 0x7f05c41092f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:54.393 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.628+0000 7f05c91ee700 1 -- 172.21.15.114:0/4066115986 >> 172.21.15.114:0/4066115986 conn(0x7f05c41008b0 msgr2=0x7f05c4102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:54.393 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.628+0000 7f05c91ee700 1 -- 172.21.15.114:0/4066115986 shutdown_connections 2024-02-16T22:11:54.393 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.628+0000 7f05c91ee700 1 -- 172.21.15.114:0/4066115986 wait complete. 2024-02-16T22:11:54.393 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.629+0000 7f05c91ee700 1 Processor -- start 2024-02-16T22:11:54.393 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.629+0000 7f05c91ee700 1 -- start start 2024-02-16T22:11:54.393 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.629+0000 7f05c91ee700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05c4106f00 0x7f05c4106860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:54.393 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.629+0000 7f05c91ee700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c4106da0 con 0x7f05c4106f00 2024-02-16T22:11:54.393 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.629+0000 7f05c2d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05c4106f00 0x7f05c4106860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:54.393 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.629+0000 7f05c2d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05c4106f00 0x7f05c4106860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60440/0 (socket says 172.21.15.114:60440) 2024-02-16T22:11:54.393 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.629+0000 7f05c2d9d700 1 -- 172.21.15.114:0/2995026646 learned_addr learned my addr 172.21.15.114:0/2995026646 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:54.393 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.629+0000 7f05c2d9d700 1 -- 172.21.15.114:0/2995026646 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b8009550 con 0x7f05c4106f00 2024-02-16T22:11:54.393 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.630+0000 7f05c2d9d700 1 --2- 172.21.15.114:0/2995026646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05c4106f00 0x7f05c4106860 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f05b8011670 tx=0x7f05b8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:54.394 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.630+0000 7f05abfff700 1 -- 172.21.15.114:0/2995026646 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05b8003a40 con 0x7f05c4106f00 2024-02-16T22:11:54.394 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.630+0000 7f05c91ee700 1 -- 172.21.15.114:0/2995026646 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05c4104eb0 con 0x7f05c4106f00 2024-02-16T22:11:54.394 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.630+0000 7f05abfff700 1 -- 172.21.15.114:0/2995026646 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f05b8003ba0 con 0x7f05c4106f00 2024-02-16T22:11:54.394 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.630+0000 7f05abfff700 1 -- 172.21.15.114:0/2995026646 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05b8024820 con 0x7f05c4106f00 2024-02-16T22:11:54.394 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.630+0000 7f05c91ee700 1 -- 172.21.15.114:0/2995026646 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c4105350 con 0x7f05c4106f00 2024-02-16T22:11:54.394 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.631+0000 7f05abfff700 1 -- 172.21.15.114:0/2995026646 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f05b8024ae0 con 0x7f05c4106f00 2024-02-16T22:11:54.394 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.631+0000 7f05abfff700 1 --2- 172.21.15.114:0/2995026646 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f05ac0382e0 0x7f05ac03a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:54.394 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.631+0000 7f05abfff700 1 -- 172.21.15.114:0/2995026646 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f05b804c300 con 0x7f05c4106f00 2024-02-16T22:11:54.394 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.631+0000 7f05c91ee700 1 -- 172.21.15.114:0/2995026646 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05c410a5c0 con 0x7f05c4106f00 2024-02-16T22:11:54.394 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.631+0000 7f05c259c700 1 --2- 172.21.15.114:0/2995026646 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f05ac0382e0 0x7f05ac03a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:54.394 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.631+0000 7f05c259c700 1 --2- 172.21.15.114:0/2995026646 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f05ac0382e0 0x7f05ac03a7a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f05b4006fd0 tx=0x7f05b4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:54.394 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.636+0000 7f05abfff700 1 -- 172.21.15.114:0/2995026646 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f05b8022310 con 0x7f05c4106f00 2024-02-16T22:11:54.394 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:52.880+0000 7f05c91ee700 1 -- 172.21.15.114:0/2995026646 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f05c410a7e0 con 0x7f05c4106f00 2024-02-16T22:11:54.394 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:53.803+0000 7f05abfff700 1 -- 172.21.15.114:0/2995026646 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f05b801ac70 con 0x7f05c4106f00 2024-02-16T22:11:54.394 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:53.805+0000 7f05abfff700 1 -- 172.21.15.114:0/2995026646 <== mon.0 v2:172.21.15.114:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f05b802a2a0 con 0x7f05c4106f00 2024-02-16T22:11:54.395 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:53.807+0000 7f05c91ee700 1 -- 172.21.15.114:0/2995026646 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f05ac0382e0 msgr2=0x7f05ac03a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:54.395 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:53.807+0000 7f05c91ee700 1 --2- 172.21.15.114:0/2995026646 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f05ac0382e0 0x7f05ac03a7a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f05b4006fd0 tx=0x7f05b4006e40 comp rx=0 tx=0).stop 2024-02-16T22:11:54.395 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:53.807+0000 7f05c91ee700 1 -- 172.21.15.114:0/2995026646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05c4106f00 msgr2=0x7f05c4106860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:54.395 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:53.807+0000 7f05c91ee700 1 --2- 172.21.15.114:0/2995026646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05c4106f00 0x7f05c4106860 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f05b8011670 tx=0x7f05b8011aa0 comp rx=0 tx=0).stop 2024-02-16T22:11:54.395 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:53.807+0000 7f05c91ee700 1 -- 172.21.15.114:0/2995026646 shutdown_connections 2024-02-16T22:11:54.395 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:53.807+0000 7f05c91ee700 1 --2- 172.21.15.114:0/2995026646 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f05ac0382e0 0x7f05ac03a7a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:54.395 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:53.807+0000 7f05c91ee700 1 --2- 172.21.15.114:0/2995026646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05c4106f00 0x7f05c4106860 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:54.395 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:53.807+0000 7f05c91ee700 1 -- 172.21.15.114:0/2995026646 >> 172.21.15.114:0/2995026646 conn(0x7f05c41008b0 msgr2=0x7f05c4101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:54.395 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:53.807+0000 7f05c91ee700 1 -- 172.21.15.114:0/2995026646 shutdown_connections 2024-02-16T22:11:54.395 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:53.807+0000 7f05c91ee700 1 -- 172.21.15.114:0/2995026646 wait complete. 2024-02-16T22:11:54.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:54 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/2995026646' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-16T22:11:54.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:54 smithi114 ceph-mon[29235]: mgrmap e9: smithi114.vcullz(active, since 25s) 2024-02-16T22:11:55.640 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout { 2024-02-16T22:11:55.641 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-16T22:11:55.641 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-16T22:11:55.641 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "active_name": "smithi114.vcullz", 2024-02-16T22:11:55.641 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-16T22:11:55.641 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout } 2024-02-16T22:11:55.641 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.815+0000 7f39e0d87700 1 Processor -- start 2024-02-16T22:11:55.641 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.816+0000 7f39e0d87700 1 -- start start 2024-02-16T22:11:55.641 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.816+0000 7f39e0d87700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39dc105840 0x7f39dc105c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:55.641 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.816+0000 7f39e0d87700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc106230 con 0x7f39dc105840 2024-02-16T22:11:55.641 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.816+0000 7f39da59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39dc105840 0x7f39dc105c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:55.641 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.816+0000 7f39da59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39dc105840 0x7f39dc105c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60484/0 (socket says 172.21.15.114:60484) 2024-02-16T22:11:55.641 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.816+0000 7f39da59c700 1 -- 172.21.15.114:0/3049248328 learned_addr learned my addr 172.21.15.114:0/3049248328 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:55.641 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.817+0000 7f39da59c700 1 -- 172.21.15.114:0/3049248328 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39dc106560 con 0x7f39dc105840 2024-02-16T22:11:55.641 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.817+0000 7f39da59c700 1 --2- 172.21.15.114:0/3049248328 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39dc105840 0x7f39dc105c60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0098a0 tx=0x7f39cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fad867053f5c659 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:55.642 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.817+0000 7f39d959a700 1 -- 172.21.15.114:0/3049248328 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f39cc004180 con 0x7f39dc105840 2024-02-16T22:11:55.642 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.817+0000 7f39d959a700 1 -- 172.21.15.114:0/3049248328 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f39cc0042e0 con 0x7f39dc105840 2024-02-16T22:11:55.642 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.817+0000 7f39d959a700 1 -- 172.21.15.114:0/3049248328 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f39cc011af0 con 0x7f39dc105840 2024-02-16T22:11:55.642 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.817+0000 7f39e0d87700 1 -- 172.21.15.114:0/3049248328 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39dc105840 msgr2=0x7f39dc105c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:55.642 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.817+0000 7f39e0d87700 1 --2- 172.21.15.114:0/3049248328 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39dc105840 0x7f39dc105c60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0098a0 tx=0x7f39cc009bb0 comp rx=0 tx=0).stop 2024-02-16T22:11:55.642 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.818+0000 7f39e0d87700 1 -- 172.21.15.114:0/3049248328 shutdown_connections 2024-02-16T22:11:55.642 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.818+0000 7f39e0d87700 1 --2- 172.21.15.114:0/3049248328 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39dc105840 0x7f39dc105c60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:55.642 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.818+0000 7f39e0d87700 1 -- 172.21.15.114:0/3049248328 >> 172.21.15.114:0/3049248328 conn(0x7f39dc100040 msgr2=0x7f39dc1024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:55.642 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.818+0000 7f39e0d87700 1 -- 172.21.15.114:0/3049248328 shutdown_connections 2024-02-16T22:11:55.642 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.818+0000 7f39e0d87700 1 -- 172.21.15.114:0/3049248328 wait complete. 2024-02-16T22:11:55.642 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.818+0000 7f39e0d87700 1 Processor -- start 2024-02-16T22:11:55.642 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.818+0000 7f39e0d87700 1 -- start start 2024-02-16T22:11:55.642 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.819+0000 7f39e0d87700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39dc105840 0x7f39dc19c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:55.642 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.819+0000 7f39e0d87700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc19c890 con 0x7f39dc105840 2024-02-16T22:11:55.642 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.819+0000 7f39da59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39dc105840 0x7f39dc19c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:11:55.643 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.819+0000 7f39da59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39dc105840 0x7f39dc19c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60494/0 (socket says 172.21.15.114:60494) 2024-02-16T22:11:55.643 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.819+0000 7f39da59c700 1 -- 172.21.15.114:0/138062575 learned_addr learned my addr 172.21.15.114:0/138062575 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:11:55.643 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.819+0000 7f39da59c700 1 -- 172.21.15.114:0/138062575 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39cc009550 con 0x7f39dc105840 2024-02-16T22:11:55.643 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.819+0000 7f39da59c700 1 --2- 172.21.15.114:0/138062575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39dc105840 0x7f39dc19c350 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f39cc011670 tx=0x7f39cc011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:11:55.643 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.819+0000 7f39cb7fe700 1 -- 172.21.15.114:0/138062575 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f39cc003a40 con 0x7f39dc105840 2024-02-16T22:11:55.643 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.819+0000 7f39cb7fe700 1 -- 172.21.15.114:0/138062575 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f39cc003ba0 con 0x7f39dc105840 2024-02-16T22:11:55.643 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.819+0000 7f39e0d87700 1 -- 172.21.15.114:0/138062575 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39dc19ca90 con 0x7f39dc105840 2024-02-16T22:11:55.643 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.819+0000 7f39cb7fe700 1 -- 172.21.15.114:0/138062575 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f39cc024790 con 0x7f39dc105840 2024-02-16T22:11:55.643 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.819+0000 7f39e0d87700 1 -- 172.21.15.114:0/138062575 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39dc19cf30 con 0x7f39dc105840 2024-02-16T22:11:55.643 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.820+0000 7f39cb7fe700 1 -- 172.21.15.114:0/138062575 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f39cc02d3f0 con 0x7f39dc105840 2024-02-16T22:11:55.643 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.820+0000 7f39cb7fe700 1 --2- 172.21.15.114:0/138062575 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f39c4038370 0x7f39c403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:11:55.643 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.820+0000 7f39cb7fe700 1 -- 172.21.15.114:0/138062575 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f39cc04d500 con 0x7f39dc105840 2024-02-16T22:11:55.643 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.820+0000 7f39d9d9b700 1 -- 172.21.15.114:0/138062575 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f39c4038370 msgr2=0x7f39c403a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/625910309 2024-02-16T22:11:55.644 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.820+0000 7f39d9d9b700 1 --2- 172.21.15.114:0/138062575 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f39c4038370 0x7f39c403a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T22:11:55.644 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.820+0000 7f39e0d87700 1 -- 172.21.15.114:0/138062575 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39dc191bf0 con 0x7f39dc105840 2024-02-16T22:11:55.644 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:54.825+0000 7f39cb7fe700 1 -- 172.21.15.114:0/138062575 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f39cc010c00 con 0x7f39dc105840 2024-02-16T22:11:55.644 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:55.021+0000 7f39d9d9b700 1 -- 172.21.15.114:0/138062575 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f39c4038370 msgr2=0x7f39c403a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/625910309 2024-02-16T22:11:55.644 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:55.021+0000 7f39d9d9b700 1 --2- 172.21.15.114:0/138062575 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f39c4038370 0x7f39c403a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-16T22:11:55.644 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:55.069+0000 7f39e0d87700 1 -- 172.21.15.114:0/138062575 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f39dc0619a0 con 0x7f39dc105840 2024-02-16T22:11:55.644 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:55.069+0000 7f39cb7fe700 1 -- 172.21.15.114:0/138062575 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f39cc0107f0 con 0x7f39dc105840 2024-02-16T22:11:55.644 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:55.071+0000 7f39e0d87700 1 -- 172.21.15.114:0/138062575 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f39c4038370 msgr2=0x7f39c403a830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T22:11:55.644 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:55.071+0000 7f39e0d87700 1 --2- 172.21.15.114:0/138062575 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f39c4038370 0x7f39c403a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:55.644 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:55.071+0000 7f39e0d87700 1 -- 172.21.15.114:0/138062575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39dc105840 msgr2=0x7f39dc19c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:11:55.644 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:55.071+0000 7f39e0d87700 1 --2- 172.21.15.114:0/138062575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39dc105840 0x7f39dc19c350 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f39cc011670 tx=0x7f39cc011b50 comp rx=0 tx=0).stop 2024-02-16T22:11:55.644 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:55.071+0000 7f39e0d87700 1 -- 172.21.15.114:0/138062575 shutdown_connections 2024-02-16T22:11:55.644 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:55.072+0000 7f39e0d87700 1 --2- 172.21.15.114:0/138062575 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f39c4038370 0x7f39c403a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:55.644 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:55.072+0000 7f39e0d87700 1 --2- 172.21.15.114:0/138062575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39dc105840 0x7f39dc19c350 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:11:55.644 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:55.072+0000 7f39e0d87700 1 -- 172.21.15.114:0/138062575 >> 172.21.15.114:0/138062575 conn(0x7f39dc100040 msgr2=0x7f39dc18ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:11:55.644 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:55.072+0000 7f39e0d87700 1 -- 172.21.15.114:0/138062575 shutdown_connections 2024-02-16T22:11:55.645 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:55.072+0000 7f39e0d87700 1 -- 172.21.15.114:0/138062575 wait complete. 2024-02-16T22:11:55.645 INFO:teuthology.orchestra.run.smithi114.stdout:Waiting for the mgr to restart... 2024-02-16T22:11:55.645 INFO:teuthology.orchestra.run.smithi114.stdout:Waiting for mgr epoch 9... 2024-02-16T22:11:55.893 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:11:55 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/138062575' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-16T22:12:00.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:00 smithi114 ceph-mon[29235]: Active manager daemon smithi114.vcullz restarted 2024-02-16T22:12:00.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:00 smithi114 ceph-mon[29235]: Activating manager daemon smithi114.vcullz 2024-02-16T22:12:00.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:00 smithi114 ceph-mon[29235]: osdmap e3: 0 total, 0 up, 0 in 2024-02-16T22:12:00.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:00 smithi114 ceph-mon[29235]: mgrmap e10: smithi114.vcullz(active, starting, since 0.00327453s) 2024-02-16T22:12:00.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:00 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi114"}]: dispatch 2024-02-16T22:12:00.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:00 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mgr metadata", "who": "smithi114.vcullz", "id": "smithi114.vcullz"}]: dispatch 2024-02-16T22:12:00.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:00 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T22:12:00.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:00 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T22:12:00.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:00 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T22:12:00.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:00 smithi114 ceph-mon[29235]: Manager daemon smithi114.vcullz is now available 2024-02-16T22:12:00.953 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:00 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:12:00.953 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:00 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:12:01.645 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:01 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi114.vcullz/mirror_snapshot_schedule"}]: dispatch 2024-02-16T22:12:01.646 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:01 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi114.vcullz/trash_purge_schedule"}]: dispatch 2024-02-16T22:12:01.646 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:01 smithi114 ceph-mon[29235]: [16/Feb/2024:22:12:00] ENGINE Bus STARTING 2024-02-16T22:12:01.646 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:01 smithi114 ceph-mon[29235]: [16/Feb/2024:22:12:00] ENGINE Serving on http://172.21.15.114:8765 2024-02-16T22:12:01.646 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:01 smithi114 ceph-mon[29235]: mgrmap e11: smithi114.vcullz(active, since 1.00571s) 2024-02-16T22:12:02.226 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout { 2024-02-16T22:12:02.226 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-16T22:12:02.227 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-16T22:12:02.227 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout } 2024-02-16T22:12:02.227 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.083+0000 7f4acafd9700 1 Processor -- start 2024-02-16T22:12:02.227 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.083+0000 7f4acafd9700 1 -- start start 2024-02-16T22:12:02.227 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.083+0000 7f4acafd9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ac4106f20 0x7f4ac4109310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:02.227 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.083+0000 7f4acafd9700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac4079e60 con 0x7f4ac4106f20 2024-02-16T22:12:02.227 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.084+0000 7f4ac8d75700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ac4106f20 0x7f4ac4109310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:02.227 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.084+0000 7f4ac8d75700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ac4106f20 0x7f4ac4109310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60500/0 (socket says 172.21.15.114:60500) 2024-02-16T22:12:02.227 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.084+0000 7f4ac8d75700 1 -- 172.21.15.114:0/1267697120 learned_addr learned my addr 172.21.15.114:0/1267697120 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:12:02.227 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.084+0000 7f4ac8d75700 1 -- 172.21.15.114:0/1267697120 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac4109850 con 0x7f4ac4106f20 2024-02-16T22:12:02.227 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.084+0000 7f4ac8d75700 1 --2- 172.21.15.114:0/1267697120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ac4106f20 0x7f4ac4109310 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80098a0 tx=0x7f4ab8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=754bf127445cac7b server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:02.227 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.084+0000 7f4ac37fe700 1 -- 172.21.15.114:0/1267697120 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ab8004180 con 0x7f4ac4106f20 2024-02-16T22:12:02.227 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.084+0000 7f4ac37fe700 1 -- 172.21.15.114:0/1267697120 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4ab80042e0 con 0x7f4ac4106f20 2024-02-16T22:12:02.228 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.084+0000 7f4ac37fe700 1 -- 172.21.15.114:0/1267697120 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ab8011af0 con 0x7f4ac4106f20 2024-02-16T22:12:02.228 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.085+0000 7f4acafd9700 1 -- 172.21.15.114:0/1267697120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ac4106f20 msgr2=0x7f4ac4109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:02.228 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.085+0000 7f4acafd9700 1 --2- 172.21.15.114:0/1267697120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ac4106f20 0x7f4ac4109310 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80098a0 tx=0x7f4ab8009bb0 comp rx=0 tx=0).stop 2024-02-16T22:12:02.228 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.085+0000 7f4acafd9700 1 -- 172.21.15.114:0/1267697120 shutdown_connections 2024-02-16T22:12:02.228 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.085+0000 7f4acafd9700 1 --2- 172.21.15.114:0/1267697120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ac4106f20 0x7f4ac4109310 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:02.228 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.085+0000 7f4acafd9700 1 -- 172.21.15.114:0/1267697120 >> 172.21.15.114:0/1267697120 conn(0x7f4ac41008d0 msgr2=0x7f4ac4102d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:02.228 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.085+0000 7f4acafd9700 1 -- 172.21.15.114:0/1267697120 shutdown_connections 2024-02-16T22:12:02.228 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.085+0000 7f4acafd9700 1 -- 172.21.15.114:0/1267697120 wait complete. 2024-02-16T22:12:02.228 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.086+0000 7f4acafd9700 1 Processor -- start 2024-02-16T22:12:02.228 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.086+0000 7f4acafd9700 1 -- start start 2024-02-16T22:12:02.228 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.086+0000 7f4acafd9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ac4106f20 0x7f4ac419a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:02.228 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.086+0000 7f4acafd9700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac419a650 con 0x7f4ac4106f20 2024-02-16T22:12:02.229 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.086+0000 7f4ac8d75700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ac4106f20 0x7f4ac419a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:02.229 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.086+0000 7f4ac8d75700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ac4106f20 0x7f4ac419a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60510/0 (socket says 172.21.15.114:60510) 2024-02-16T22:12:02.229 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.086+0000 7f4ac8d75700 1 -- 172.21.15.114:0/579141482 learned_addr learned my addr 172.21.15.114:0/579141482 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:12:02.229 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.086+0000 7f4ac8d75700 1 -- 172.21.15.114:0/579141482 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab8009550 con 0x7f4ac4106f20 2024-02-16T22:12:02.229 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.086+0000 7f4ac8d75700 1 --2- 172.21.15.114:0/579141482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ac4106f20 0x7f4ac419a110 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8011670 tx=0x7f4ab8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:02.229 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.087+0000 7f4ac1ffb700 1 -- 172.21.15.114:0/579141482 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ab8003a40 con 0x7f4ac4106f20 2024-02-16T22:12:02.229 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.087+0000 7f4ac1ffb700 1 -- 172.21.15.114:0/579141482 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4ab8003ba0 con 0x7f4ac4106f20 2024-02-16T22:12:02.229 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.087+0000 7f4ac1ffb700 1 -- 172.21.15.114:0/579141482 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ab8024820 con 0x7f4ac4106f20 2024-02-16T22:12:02.229 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.087+0000 7f4acafd9700 1 -- 172.21.15.114:0/579141482 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac419a850 con 0x7f4ac4106f20 2024-02-16T22:12:02.229 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.087+0000 7f4acafd9700 1 -- 172.21.15.114:0/579141482 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac419ac30 con 0x7f4ac4106f20 2024-02-16T22:12:02.229 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.088+0000 7f4ac1ffb700 1 -- 172.21.15.114:0/579141482 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f4ab801ac70 con 0x7f4ac4106f20 2024-02-16T22:12:02.229 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.088+0000 7f4ac1ffb700 1 --2- 172.21.15.114:0/579141482 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f4ab4038380 0x7f4ab403a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:02.229 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.088+0000 7f4acafd9700 1 -- 172.21.15.114:0/579141482 --> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ac410a5e0 con 0x7f4ab4038380 2024-02-16T22:12:02.230 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.088+0000 7f4ac1ffb700 1 -- 172.21.15.114:0/579141482 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4ab804c2f0 con 0x7f4ac4106f20 2024-02-16T22:12:02.230 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.088+0000 7f4ac3fff700 1 -- 172.21.15.114:0/579141482 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f4ab4038380 msgr2=0x7f4ab403a840 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/625910309 2024-02-16T22:12:02.230 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.088+0000 7f4ac3fff700 1 --2- 172.21.15.114:0/579141482 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f4ab4038380 0x7f4ab403a840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T22:12:02.230 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.289+0000 7f4ac3fff700 1 -- 172.21.15.114:0/579141482 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f4ab4038380 msgr2=0x7f4ab403a840 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/625910309 2024-02-16T22:12:02.230 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.289+0000 7f4ac3fff700 1 --2- 172.21.15.114:0/579141482 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f4ab4038380 0x7f4ab403a840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-16T22:12:02.230 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.690+0000 7f4ac3fff700 1 -- 172.21.15.114:0/579141482 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f4ab4038380 msgr2=0x7f4ab403a840 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/625910309 2024-02-16T22:12:02.230 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:56.690+0000 7f4ac3fff700 1 --2- 172.21.15.114:0/579141482 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f4ab4038380 0x7f4ab403a840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-16T22:12:02.230 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:57.491+0000 7f4ac3fff700 1 -- 172.21.15.114:0/579141482 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f4ab4038380 msgr2=0x7f4ab403a840 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/625910309 2024-02-16T22:12:02.230 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:57.491+0000 7f4ac3fff700 1 --2- 172.21.15.114:0/579141482 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f4ab4038380 0x7f4ab403a840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-16T22:12:02.230 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:59.092+0000 7f4ac3fff700 1 -- 172.21.15.114:0/579141482 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f4ab4038380 msgr2=0x7f4ab403a840 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/625910309 2024-02-16T22:12:02.230 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:11:59.092+0000 7f4ac3fff700 1 --2- 172.21.15.114:0/579141482 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f4ab4038380 0x7f4ab403a840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-16T22:12:02.230 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:00.494+0000 7f4ac1ffb700 1 -- 172.21.15.114:0/579141482 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mgrmap(e 10) v1 ==== 45063+0+0 (secure 0 0 0) 0x7f4ab801a430 con 0x7f4ac4106f20 2024-02-16T22:12:02.230 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:00.494+0000 7f4ac1ffb700 1 -- 172.21.15.114:0/579141482 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f4ab4038380 msgr2=0x7f4ab403a840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T22:12:02.231 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:00.494+0000 7f4ac1ffb700 1 --2- 172.21.15.114:0/579141482 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f4ab4038380 0x7f4ab403a840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:02.231 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:01.495+0000 7f4ac1ffb700 1 -- 172.21.15.114:0/579141482 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mgrmap(e 11) v1 ==== 45190+0+0 (secure 0 0 0) 0x7f4ab804db00 con 0x7f4ac4106f20 2024-02-16T22:12:02.231 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:01.495+0000 7f4ac1ffb700 1 --2- 172.21.15.114:0/579141482 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f4ab403bdb0 0x7f4ab403e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:02.231 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:01.495+0000 7f4ac1ffb700 1 -- 172.21.15.114:0/579141482 --> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ac410a5e0 con 0x7f4ab403bdb0 2024-02-16T22:12:02.231 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:01.496+0000 7f4ac3fff700 1 --2- 172.21.15.114:0/579141482 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f4ab403bdb0 0x7f4ab403e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:02.231 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:01.496+0000 7f4ac3fff700 1 --2- 172.21.15.114:0/579141482 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f4ab403bdb0 0x7f4ab403e1a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0003a10 tx=0x7f4ab0009270 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:02.231 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:01.497+0000 7f4ac1ffb700 1 -- 172.21.15.114:0/579141482 <== mgr.14164 v2:172.21.15.114:6800/2648536185 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f4ac410a5e0 con 0x7f4ab403bdb0 2024-02-16T22:12:02.231 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:01.503+0000 7f4acafd9700 1 -- 172.21.15.114:0/579141482 --> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f4ac419b450 con 0x7f4ab403bdb0 2024-02-16T22:12:02.231 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:01.503+0000 7f4ac1ffb700 1 -- 172.21.15.114:0/579141482 <== mgr.14164 v2:172.21.15.114:6800/2648536185 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f4ac419b450 con 0x7f4ab403bdb0 2024-02-16T22:12:02.231 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:01.503+0000 7f4acafd9700 1 -- 172.21.15.114:0/579141482 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f4ab403bdb0 msgr2=0x7f4ab403e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:02.231 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:01.503+0000 7f4acafd9700 1 --2- 172.21.15.114:0/579141482 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f4ab403bdb0 0x7f4ab403e1a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0003a10 tx=0x7f4ab0009270 comp rx=0 tx=0).stop 2024-02-16T22:12:02.231 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:01.503+0000 7f4acafd9700 1 -- 172.21.15.114:0/579141482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ac4106f20 msgr2=0x7f4ac419a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:02.231 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:01.503+0000 7f4acafd9700 1 --2- 172.21.15.114:0/579141482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ac4106f20 0x7f4ac419a110 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8011670 tx=0x7f4ab8011aa0 comp rx=0 tx=0).stop 2024-02-16T22:12:02.231 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:01.503+0000 7f4acafd9700 1 -- 172.21.15.114:0/579141482 shutdown_connections 2024-02-16T22:12:02.232 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:01.503+0000 7f4acafd9700 1 --2- 172.21.15.114:0/579141482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ac4106f20 0x7f4ac419a110 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:02.232 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:01.503+0000 7f4acafd9700 1 --2- 172.21.15.114:0/579141482 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f4ab403bdb0 0x7f4ab403e1a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:02.232 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:01.503+0000 7f4acafd9700 1 --2- 172.21.15.114:0/579141482 >> [v2:172.21.15.114:6800/625910309,v1:172.21.15.114:6801/625910309] conn(0x7f4ab4038380 0x7f4ab403a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:02.232 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:01.504+0000 7f4acafd9700 1 -- 172.21.15.114:0/579141482 >> 172.21.15.114:0/579141482 conn(0x7f4ac41008d0 msgr2=0x7f4ac41016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:02.232 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:01.504+0000 7f4acafd9700 1 -- 172.21.15.114:0/579141482 shutdown_connections 2024-02-16T22:12:02.232 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:01.504+0000 7f4acafd9700 1 -- 172.21.15.114:0/579141482 wait complete. 2024-02-16T22:12:02.232 INFO:teuthology.orchestra.run.smithi114.stdout:mgr epoch 9 is available 2024-02-16T22:12:02.232 INFO:teuthology.orchestra.run.smithi114.stdout:Generating a dashboard self-signed certificate... 2024-02-16T22:12:03.100 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:02 smithi114 ceph-mon[29235]: [16/Feb/2024:22:12:00] ENGINE Serving on https://172.21.15.114:7150 2024-02-16T22:12:03.100 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:02 smithi114 ceph-mon[29235]: [16/Feb/2024:22:12:00] ENGINE Bus STARTED 2024-02-16T22:12:03.100 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:02 smithi114 ceph-mon[29235]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-16T22:12:03.100 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:02 smithi114 ceph-mon[29235]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-16T22:12:03.100 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:02 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:03.100 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:02 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:03.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:03 smithi114 ceph-mon[29235]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:12:03.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:03 smithi114 ceph-mon[29235]: mgrmap e12: smithi114.vcullz(active, since 2s) 2024-02-16T22:12:03.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:03 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:03.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:03 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:04.738 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-16T22:12:04.738 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.627+0000 7f8a0203e700 1 Processor -- start 2024-02-16T22:12:04.738 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.627+0000 7f8a0203e700 1 -- start start 2024-02-16T22:12:04.738 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.627+0000 7f8a0203e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89fc106f40 0x7f89fc109330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:04.738 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.627+0000 7f8a0203e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc079e40 con 0x7f89fc106f40 2024-02-16T22:12:04.738 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.628+0000 7f89fb7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89fc106f40 0x7f89fc109330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:04.738 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.628+0000 7f89fb7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89fc106f40 0x7f89fc109330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55722/0 (socket says 172.21.15.114:55722) 2024-02-16T22:12:04.739 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.628+0000 7f89fb7fe700 1 -- 172.21.15.114:0/2049066126 learned_addr learned my addr 172.21.15.114:0/2049066126 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:12:04.739 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.628+0000 7f89fb7fe700 1 -- 172.21.15.114:0/2049066126 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc109870 con 0x7f89fc106f40 2024-02-16T22:12:04.739 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.628+0000 7f89fb7fe700 1 --2- 172.21.15.114:0/2049066126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89fc106f40 0x7f89fc109330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f89ec0098a0 tx=0x7f89ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b4404c57058bd5 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:04.739 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.628+0000 7f89fa7fc700 1 -- 172.21.15.114:0/2049066126 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f89ec004180 con 0x7f89fc106f40 2024-02-16T22:12:04.739 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.628+0000 7f89fa7fc700 1 -- 172.21.15.114:0/2049066126 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f89ec0042e0 con 0x7f89fc106f40 2024-02-16T22:12:04.739 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.628+0000 7f89fa7fc700 1 -- 172.21.15.114:0/2049066126 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f89ec011af0 con 0x7f89fc106f40 2024-02-16T22:12:04.739 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.629+0000 7f8a0203e700 1 -- 172.21.15.114:0/2049066126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89fc106f40 msgr2=0x7f89fc109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:04.739 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.629+0000 7f8a0203e700 1 --2- 172.21.15.114:0/2049066126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89fc106f40 0x7f89fc109330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f89ec0098a0 tx=0x7f89ec009bb0 comp rx=0 tx=0).stop 2024-02-16T22:12:04.739 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.629+0000 7f8a0203e700 1 -- 172.21.15.114:0/2049066126 shutdown_connections 2024-02-16T22:12:04.739 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.629+0000 7f8a0203e700 1 --2- 172.21.15.114:0/2049066126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89fc106f40 0x7f89fc109330 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:04.739 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.629+0000 7f8a0203e700 1 -- 172.21.15.114:0/2049066126 >> 172.21.15.114:0/2049066126 conn(0x7f89fc1008d0 msgr2=0x7f89fc102d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:04.739 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.629+0000 7f8a0203e700 1 -- 172.21.15.114:0/2049066126 shutdown_connections 2024-02-16T22:12:04.739 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.629+0000 7f8a0203e700 1 -- 172.21.15.114:0/2049066126 wait complete. 2024-02-16T22:12:04.739 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.630+0000 7f8a0203e700 1 Processor -- start 2024-02-16T22:12:04.739 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.630+0000 7f8a0203e700 1 -- start start 2024-02-16T22:12:04.739 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.630+0000 7f8a0203e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89fc106f40 0x7f89fc19a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:04.740 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.630+0000 7f8a0203e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc19a640 con 0x7f89fc106f40 2024-02-16T22:12:04.740 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.631+0000 7f89fb7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89fc106f40 0x7f89fc19a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:04.740 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.631+0000 7f89fb7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89fc106f40 0x7f89fc19a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55728/0 (socket says 172.21.15.114:55728) 2024-02-16T22:12:04.740 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.631+0000 7f89fb7fe700 1 -- 172.21.15.114:0/1721265319 learned_addr learned my addr 172.21.15.114:0/1721265319 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:12:04.740 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.631+0000 7f89fb7fe700 1 -- 172.21.15.114:0/1721265319 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ec009550 con 0x7f89fc106f40 2024-02-16T22:12:04.740 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.631+0000 7f89fb7fe700 1 --2- 172.21.15.114:0/1721265319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89fc106f40 0x7f89fc19a100 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f89ec011670 tx=0x7f89ec011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:04.740 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.631+0000 7f89f8ff9700 1 -- 172.21.15.114:0/1721265319 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f89ec003a40 con 0x7f89fc106f40 2024-02-16T22:12:04.740 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.631+0000 7f8a0203e700 1 -- 172.21.15.114:0/1721265319 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89fc19a840 con 0x7f89fc106f40 2024-02-16T22:12:04.740 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.631+0000 7f89f8ff9700 1 -- 172.21.15.114:0/1721265319 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f89ec003ba0 con 0x7f89fc106f40 2024-02-16T22:12:04.740 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.631+0000 7f89f8ff9700 1 -- 172.21.15.114:0/1721265319 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f89ec024790 con 0x7f89fc106f40 2024-02-16T22:12:04.740 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.631+0000 7f8a0203e700 1 -- 172.21.15.114:0/1721265319 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89fc19ace0 con 0x7f89fc106f40 2024-02-16T22:12:04.740 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.632+0000 7f89f8ff9700 1 -- 172.21.15.114:0/1721265319 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 11) v1 ==== 45190+0+0 (secure 0 0 0) 0x7f89ec02d3f0 con 0x7f89fc106f40 2024-02-16T22:12:04.743 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.632+0000 7f8a0203e700 1 -- 172.21.15.114:0/1721265319 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89fc10a5b0 con 0x7f89fc106f40 2024-02-16T22:12:04.743 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.632+0000 7f89f8ff9700 1 --2- 172.21.15.114:0/1721265319 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f89e4038250 0x7f89e403a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:04.743 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.633+0000 7f89f8ff9700 1 -- 172.21.15.114:0/1721265319 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f89ec04c4a0 con 0x7f89fc106f40 2024-02-16T22:12:04.743 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.633+0000 7f89faffd700 1 --2- 172.21.15.114:0/1721265319 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f89e4038250 0x7f89e403a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:04.743 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.633+0000 7f89faffd700 1 --2- 172.21.15.114:0/1721265319 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f89e4038250 0x7f89e403a710 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f89f0006fd0 tx=0x7f89f0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:04.743 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.641+0000 7f89f8ff9700 1 -- 172.21.15.114:0/1721265319 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f89ec0109d0 con 0x7f89fc106f40 2024-02-16T22:12:04.743 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.879+0000 7f8a0203e700 1 -- 172.21.15.114:0/1721265319 --> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f89fc101da0 con 0x7f89e4038250 2024-02-16T22:12:04.743 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:02.911+0000 7f89f8ff9700 1 -- 172.21.15.114:0/1721265319 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f89ec021d60 con 0x7f89fc106f40 2024-02-16T22:12:04.743 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:03.098+0000 7f89f8ff9700 1 -- 172.21.15.114:0/1721265319 <== mgr.14164 v2:172.21.15.114:6800/2648536185 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f89fc101da0 con 0x7f89e4038250 2024-02-16T22:12:04.743 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:03.100+0000 7f8a0203e700 1 -- 172.21.15.114:0/1721265319 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f89e4038250 msgr2=0x7f89e403a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:04.743 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:03.100+0000 7f8a0203e700 1 --2- 172.21.15.114:0/1721265319 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f89e4038250 0x7f89e403a710 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f89f0006fd0 tx=0x7f89f0006e40 comp rx=0 tx=0).stop 2024-02-16T22:12:04.743 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:03.100+0000 7f8a0203e700 1 -- 172.21.15.114:0/1721265319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89fc106f40 msgr2=0x7f89fc19a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:04.743 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:03.100+0000 7f8a0203e700 1 --2- 172.21.15.114:0/1721265319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89fc106f40 0x7f89fc19a100 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f89ec011670 tx=0x7f89ec011b50 comp rx=0 tx=0).stop 2024-02-16T22:12:04.743 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:03.100+0000 7f8a0203e700 1 -- 172.21.15.114:0/1721265319 shutdown_connections 2024-02-16T22:12:04.744 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:03.100+0000 7f8a0203e700 1 --2- 172.21.15.114:0/1721265319 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f89e4038250 0x7f89e403a710 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:04.744 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:03.100+0000 7f8a0203e700 1 --2- 172.21.15.114:0/1721265319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89fc106f40 0x7f89fc19a100 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:04.744 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:03.100+0000 7f8a0203e700 1 -- 172.21.15.114:0/1721265319 >> 172.21.15.114:0/1721265319 conn(0x7f89fc1008d0 msgr2=0x7f89fc101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:04.744 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:03.100+0000 7f8a0203e700 1 -- 172.21.15.114:0/1721265319 shutdown_connections 2024-02-16T22:12:04.744 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:03.100+0000 7f8a0203e700 1 -- 172.21.15.114:0/1721265319 wait complete. 2024-02-16T22:12:04.744 INFO:teuthology.orchestra.run.smithi114.stdout:Creating initial admin user... 2024-02-16T22:12:06.237 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$yAI3buCnHVyu87f8xvnOJueBBe3Psr6.unSYBeF8O0QnbzgADC8.2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1708121525, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-16T22:12:06.237 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.169+0000 7f8958130700 1 Processor -- start 2024-02-16T22:12:06.237 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.169+0000 7f8958130700 1 -- start start 2024-02-16T22:12:06.237 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.169+0000 7f8958130700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8950106420 0x7f8950106840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:06.237 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.169+0000 7f8958130700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8950106e10 con 0x7f8950106420 2024-02-16T22:12:06.237 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.169+0000 7f8955ecc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8950106420 0x7f8950106840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:06.237 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.169+0000 7f8955ecc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8950106420 0x7f8950106840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55734/0 (socket says 172.21.15.114:55734) 2024-02-16T22:12:06.237 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.169+0000 7f8955ecc700 1 -- 172.21.15.114:0/1502287832 learned_addr learned my addr 172.21.15.114:0/1502287832 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:12:06.237 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.170+0000 7f8955ecc700 1 -- 172.21.15.114:0/1502287832 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8950107620 con 0x7f8950106420 2024-02-16T22:12:06.237 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.170+0000 7f8955ecc700 1 --2- 172.21.15.114:0/1502287832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8950106420 0x7f8950106840 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f894c0098a0 tx=0x7f894c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817a2dd2f2c646b3 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:06.237 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.170+0000 7f8954eca700 1 -- 172.21.15.114:0/1502287832 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f894c004180 con 0x7f8950106420 2024-02-16T22:12:06.238 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.170+0000 7f8954eca700 1 -- 172.21.15.114:0/1502287832 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f894c0042e0 con 0x7f8950106420 2024-02-16T22:12:06.238 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.170+0000 7f8954eca700 1 -- 172.21.15.114:0/1502287832 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f894c011af0 con 0x7f8950106420 2024-02-16T22:12:06.238 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.171+0000 7f8958130700 1 -- 172.21.15.114:0/1502287832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8950106420 msgr2=0x7f8950106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:06.238 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.171+0000 7f8958130700 1 --2- 172.21.15.114:0/1502287832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8950106420 0x7f8950106840 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f894c0098a0 tx=0x7f894c009bb0 comp rx=0 tx=0).stop 2024-02-16T22:12:06.238 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.171+0000 7f8958130700 1 -- 172.21.15.114:0/1502287832 shutdown_connections 2024-02-16T22:12:06.238 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.171+0000 7f8958130700 1 --2- 172.21.15.114:0/1502287832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8950106420 0x7f8950106840 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:06.238 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.171+0000 7f8958130700 1 -- 172.21.15.114:0/1502287832 >> 172.21.15.114:0/1502287832 conn(0x7f8950101a00 msgr2=0x7f8950103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:06.238 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.171+0000 7f8958130700 1 -- 172.21.15.114:0/1502287832 shutdown_connections 2024-02-16T22:12:06.238 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.171+0000 7f8958130700 1 -- 172.21.15.114:0/1502287832 wait complete. 2024-02-16T22:12:06.238 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.171+0000 7f8958130700 1 Processor -- start 2024-02-16T22:12:06.238 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.171+0000 7f8958130700 1 -- start start 2024-02-16T22:12:06.238 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.172+0000 7f8958130700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8950106420 0x7f895019a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:06.238 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.172+0000 7f8958130700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895019a670 con 0x7f8950106420 2024-02-16T22:12:06.238 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.172+0000 7f8955ecc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8950106420 0x7f895019a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:06.238 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.172+0000 7f8955ecc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8950106420 0x7f895019a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55746/0 (socket says 172.21.15.114:55746) 2024-02-16T22:12:06.239 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.172+0000 7f8955ecc700 1 -- 172.21.15.114:0/2230807511 learned_addr learned my addr 172.21.15.114:0/2230807511 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:12:06.239 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.172+0000 7f8955ecc700 1 -- 172.21.15.114:0/2230807511 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f894c009550 con 0x7f8950106420 2024-02-16T22:12:06.239 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.172+0000 7f8955ecc700 1 --2- 172.21.15.114:0/2230807511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8950106420 0x7f895019a130 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f894c000c00 tx=0x7f894c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:06.239 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.172+0000 7f8946ffd700 1 -- 172.21.15.114:0/2230807511 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f894c003a70 con 0x7f8950106420 2024-02-16T22:12:06.239 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.172+0000 7f8946ffd700 1 -- 172.21.15.114:0/2230807511 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f894c003bd0 con 0x7f8950106420 2024-02-16T22:12:06.239 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.172+0000 7f8958130700 1 -- 172.21.15.114:0/2230807511 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f895019a870 con 0x7f8950106420 2024-02-16T22:12:06.239 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.172+0000 7f8958130700 1 -- 172.21.15.114:0/2230807511 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f895019ad10 con 0x7f8950106420 2024-02-16T22:12:06.239 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.172+0000 7f8946ffd700 1 -- 172.21.15.114:0/2230807511 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f894c0238c0 con 0x7f8950106420 2024-02-16T22:12:06.239 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.173+0000 7f8946ffd700 1 -- 172.21.15.114:0/2230807511 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f894c023b30 con 0x7f8950106420 2024-02-16T22:12:06.239 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.173+0000 7f8958130700 1 -- 172.21.15.114:0/2230807511 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8934005320 con 0x7f8950106420 2024-02-16T22:12:06.239 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.173+0000 7f8946ffd700 1 --2- 172.21.15.114:0/2230807511 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f893c0383c0 0x7f893c03a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:06.239 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.173+0000 7f8946ffd700 1 -- 172.21.15.114:0/2230807511 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f894c04c2a0 con 0x7f8950106420 2024-02-16T22:12:06.239 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.173+0000 7f89556cb700 1 --2- 172.21.15.114:0/2230807511 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f893c0383c0 0x7f893c03a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:06.239 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.174+0000 7f89556cb700 1 --2- 172.21.15.114:0/2230807511 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f893c0383c0 0x7f893c03a880 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8940006fd0 tx=0x7f8940006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:06.239 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.178+0000 7f8946ffd700 1 -- 172.21.15.114:0/2230807511 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f894c01a430 con 0x7f8950106420 2024-02-16T22:12:06.240 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.380+0000 7f8958130700 1 -- 172.21.15.114:0/2230807511 --> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f8934002430 con 0x7f893c0383c0 2024-02-16T22:12:06.240 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.621+0000 7f8946ffd700 1 -- 172.21.15.114:0/2230807511 <== mgr.14164 v2:172.21.15.114:6800/2648536185 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f8934002430 con 0x7f893c0383c0 2024-02-16T22:12:06.240 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.623+0000 7f8958130700 1 -- 172.21.15.114:0/2230807511 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f893c0383c0 msgr2=0x7f893c03a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:06.240 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.623+0000 7f8958130700 1 --2- 172.21.15.114:0/2230807511 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f893c0383c0 0x7f893c03a880 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8940006fd0 tx=0x7f8940006e40 comp rx=0 tx=0).stop 2024-02-16T22:12:06.240 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.623+0000 7f8958130700 1 -- 172.21.15.114:0/2230807511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8950106420 msgr2=0x7f895019a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:06.240 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.623+0000 7f8958130700 1 --2- 172.21.15.114:0/2230807511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8950106420 0x7f895019a130 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f894c000c00 tx=0x7f894c011b50 comp rx=0 tx=0).stop 2024-02-16T22:12:06.240 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.623+0000 7f8958130700 1 -- 172.21.15.114:0/2230807511 shutdown_connections 2024-02-16T22:12:06.240 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.623+0000 7f8958130700 1 --2- 172.21.15.114:0/2230807511 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f893c0383c0 0x7f893c03a880 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:06.240 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.623+0000 7f8958130700 1 --2- 172.21.15.114:0/2230807511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8950106420 0x7f895019a130 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:06.240 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.623+0000 7f8958130700 1 -- 172.21.15.114:0/2230807511 >> 172.21.15.114:0/2230807511 conn(0x7f8950101a00 msgr2=0x7f89501030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:06.240 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.623+0000 7f8958130700 1 -- 172.21.15.114:0/2230807511 shutdown_connections 2024-02-16T22:12:06.240 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:05.623+0000 7f8958130700 1 -- 172.21.15.114:0/2230807511 wait complete. 2024-02-16T22:12:06.241 INFO:teuthology.orchestra.run.smithi114.stdout:Fetching dashboard port number... 2024-02-16T22:12:06.839 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:06 smithi114 ceph-mon[29235]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:12:06.839 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:06 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:07.401 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 8443 2024-02-16T22:12:07.401 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.639+0000 7f6bf0b10700 1 Processor -- start 2024-02-16T22:12:07.401 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.639+0000 7f6bf0b10700 1 -- start start 2024-02-16T22:12:07.402 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.639+0000 7f6bf0b10700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bec07bef0 0x7f6bec07a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:07.402 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.639+0000 7f6bf0b10700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec07a920 con 0x7f6bec07bef0 2024-02-16T22:12:07.402 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.639+0000 7f6bea59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bec07bef0 0x7f6bec07a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:07.402 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.639+0000 7f6bea59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bec07bef0 0x7f6bec07a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55758/0 (socket says 172.21.15.114:55758) 2024-02-16T22:12:07.402 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.639+0000 7f6bea59c700 1 -- 172.21.15.114:0/739201017 learned_addr learned my addr 172.21.15.114:0/739201017 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:12:07.402 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.639+0000 7f6bea59c700 1 -- 172.21.15.114:0/739201017 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bec07aa60 con 0x7f6bec07bef0 2024-02-16T22:12:07.402 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.640+0000 7f6bea59c700 1 --2- 172.21.15.114:0/739201017 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bec07bef0 0x7f6bec07a350 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc0098a0 tx=0x7f6bdc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cba7fc97d4dce56 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:07.402 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.640+0000 7f6be959a700 1 -- 172.21.15.114:0/739201017 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6bdc004180 con 0x7f6bec07bef0 2024-02-16T22:12:07.402 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.640+0000 7f6be959a700 1 -- 172.21.15.114:0/739201017 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6bdc0042e0 con 0x7f6bec07bef0 2024-02-16T22:12:07.402 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.640+0000 7f6be959a700 1 -- 172.21.15.114:0/739201017 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6bdc011af0 con 0x7f6bec07bef0 2024-02-16T22:12:07.402 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.640+0000 7f6bf0b10700 1 -- 172.21.15.114:0/739201017 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bec07bef0 msgr2=0x7f6bec07a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:07.402 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.640+0000 7f6bf0b10700 1 --2- 172.21.15.114:0/739201017 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bec07bef0 0x7f6bec07a350 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc0098a0 tx=0x7f6bdc009bb0 comp rx=0 tx=0).stop 2024-02-16T22:12:07.402 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.640+0000 7f6bf0b10700 1 -- 172.21.15.114:0/739201017 shutdown_connections 2024-02-16T22:12:07.402 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.640+0000 7f6bf0b10700 1 --2- 172.21.15.114:0/739201017 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bec07bef0 0x7f6bec07a350 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:07.402 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.640+0000 7f6bf0b10700 1 -- 172.21.15.114:0/739201017 >> 172.21.15.114:0/739201017 conn(0x7f6bec1019d0 msgr2=0x7f6bec103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:07.403 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.640+0000 7f6bf0b10700 1 -- 172.21.15.114:0/739201017 shutdown_connections 2024-02-16T22:12:07.403 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.640+0000 7f6bf0b10700 1 -- 172.21.15.114:0/739201017 wait complete. 2024-02-16T22:12:07.403 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.641+0000 7f6bf0b10700 1 Processor -- start 2024-02-16T22:12:07.403 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.641+0000 7f6bf0b10700 1 -- start start 2024-02-16T22:12:07.403 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.641+0000 7f6bf0b10700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bec07bef0 0x7f6bec19c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:07.403 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.641+0000 7f6bf0b10700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec19c850 con 0x7f6bec07bef0 2024-02-16T22:12:07.403 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.641+0000 7f6bea59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bec07bef0 0x7f6bec19c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:07.403 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.641+0000 7f6bea59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bec07bef0 0x7f6bec19c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55764/0 (socket says 172.21.15.114:55764) 2024-02-16T22:12:07.403 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.641+0000 7f6bea59c700 1 -- 172.21.15.114:0/131271408 learned_addr learned my addr 172.21.15.114:0/131271408 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:12:07.403 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.642+0000 7f6bea59c700 1 -- 172.21.15.114:0/131271408 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bdc009550 con 0x7f6bec07bef0 2024-02-16T22:12:07.403 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.642+0000 7f6bea59c700 1 --2- 172.21.15.114:0/131271408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bec07bef0 0x7f6bec19c310 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc000c00 tx=0x7f6bdc011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:07.403 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.642+0000 7f6bdb7fe700 1 -- 172.21.15.114:0/131271408 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6bdc003a70 con 0x7f6bec07bef0 2024-02-16T22:12:07.403 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.642+0000 7f6bdb7fe700 1 -- 172.21.15.114:0/131271408 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6bdc003bd0 con 0x7f6bec07bef0 2024-02-16T22:12:07.403 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.642+0000 7f6bf0b10700 1 -- 172.21.15.114:0/131271408 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bec19ca50 con 0x7f6bec07bef0 2024-02-16T22:12:07.403 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.642+0000 7f6bdb7fe700 1 -- 172.21.15.114:0/131271408 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6bdc0238c0 con 0x7f6bec07bef0 2024-02-16T22:12:07.404 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.642+0000 7f6bf0b10700 1 -- 172.21.15.114:0/131271408 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bec19cef0 con 0x7f6bec07bef0 2024-02-16T22:12:07.404 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.643+0000 7f6bdb7fe700 1 -- 172.21.15.114:0/131271408 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f6bdc023b30 con 0x7f6bec07bef0 2024-02-16T22:12:07.404 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.643+0000 7f6bf0b10700 1 -- 172.21.15.114:0/131271408 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bec10c800 con 0x7f6bec07bef0 2024-02-16T22:12:07.404 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.643+0000 7f6bdb7fe700 1 --2- 172.21.15.114:0/131271408 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f6bd4038460 0x7f6bd403a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:07.404 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.643+0000 7f6bdb7fe700 1 -- 172.21.15.114:0/131271408 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6bdc04c490 con 0x7f6bec07bef0 2024-02-16T22:12:07.404 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.643+0000 7f6be9d9b700 1 --2- 172.21.15.114:0/131271408 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f6bd4038460 0x7f6bd403a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:07.404 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.643+0000 7f6be9d9b700 1 --2- 172.21.15.114:0/131271408 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f6bd4038460 0x7f6bd403a920 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6be0006fd0 tx=0x7f6be0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:07.404 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.648+0000 7f6bdb7fe700 1 -- 172.21.15.114:0/131271408 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6bdc02b300 con 0x7f6bec07bef0 2024-02-16T22:12:07.404 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.837+0000 7f6bf0b10700 1 -- 172.21.15.114:0/131271408 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f6bec19fcd0 con 0x7f6bec07bef0 2024-02-16T22:12:07.404 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.837+0000 7f6bdb7fe700 1 -- 172.21.15.114:0/131271408 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f6bdc010b70 con 0x7f6bec07bef0 2024-02-16T22:12:07.404 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.840+0000 7f6bf0b10700 1 -- 172.21.15.114:0/131271408 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f6bd4038460 msgr2=0x7f6bd403a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:07.404 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.840+0000 7f6bf0b10700 1 --2- 172.21.15.114:0/131271408 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f6bd4038460 0x7f6bd403a920 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6be0006fd0 tx=0x7f6be0006e40 comp rx=0 tx=0).stop 2024-02-16T22:12:07.404 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.840+0000 7f6bf0b10700 1 -- 172.21.15.114:0/131271408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bec07bef0 msgr2=0x7f6bec19c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:07.405 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.840+0000 7f6bf0b10700 1 --2- 172.21.15.114:0/131271408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bec07bef0 0x7f6bec19c310 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc000c00 tx=0x7f6bdc011b50 comp rx=0 tx=0).stop 2024-02-16T22:12:07.405 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.840+0000 7f6bf0b10700 1 -- 172.21.15.114:0/131271408 shutdown_connections 2024-02-16T22:12:07.405 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.840+0000 7f6bf0b10700 1 --2- 172.21.15.114:0/131271408 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f6bd4038460 0x7f6bd403a920 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:07.405 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.840+0000 7f6bf0b10700 1 --2- 172.21.15.114:0/131271408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bec07bef0 0x7f6bec19c310 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:07.405 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.840+0000 7f6bf0b10700 1 -- 172.21.15.114:0/131271408 >> 172.21.15.114:0/131271408 conn(0x7f6bec1019d0 msgr2=0x7f6bec1027d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:07.405 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.840+0000 7f6bf0b10700 1 -- 172.21.15.114:0/131271408 shutdown_connections 2024-02-16T22:12:07.405 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:06.840+0000 7f6bf0b10700 1 -- 172.21.15.114:0/131271408 wait complete. 2024-02-16T22:12:07.410 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-16T22:12:07.410 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout disabled 2024-02-16T22:12:07.418 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-16T22:12:07.418 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout inactive 2024-02-16T22:12:07.418 INFO:teuthology.orchestra.run.smithi114.stdout:firewalld.service is not enabled 2024-02-16T22:12:07.418 INFO:teuthology.orchestra.run.smithi114.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-16T22:12:07.420 INFO:teuthology.orchestra.run.smithi114.stdout:Ceph Dashboard is now available at: 2024-02-16T22:12:07.420 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:12:07.420 INFO:teuthology.orchestra.run.smithi114.stdout: URL: https://smithi114.front.sepia.ceph.com:8443/ 2024-02-16T22:12:07.420 INFO:teuthology.orchestra.run.smithi114.stdout: User: admin 2024-02-16T22:12:07.420 INFO:teuthology.orchestra.run.smithi114.stdout: Password: 650sl1e6he 2024-02-16T22:12:07.420 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:12:07.421 INFO:teuthology.orchestra.run.smithi114.stdout:Saving cluster configuration to /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config directory 2024-02-16T22:12:07.421 INFO:teuthology.orchestra.run.smithi114.stdout:Enabling autotune for osd_memory_target 2024-02-16T22:12:07.658 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:07 smithi114 ceph-mon[29235]: mgrmap e13: smithi114.vcullz(active, since 6s) 2024-02-16T22:12:07.658 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:07 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/131271408' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-16T22:12:08.641 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.844+0000 7f05d8c8a700 1 Processor -- start 2024-02-16T22:12:08.642 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.844+0000 7f05d8c8a700 1 -- start start 2024-02-16T22:12:08.642 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.844+0000 7f05d8c8a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05d407be70 0x7f05d407a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:08.642 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.844+0000 7f05d8c8a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d407a8a0 con 0x7f05d407be70 2024-02-16T22:12:08.642 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.844+0000 7f05d259c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05d407be70 0x7f05d407a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:08.642 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.844+0000 7f05d259c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05d407be70 0x7f05d407a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55768/0 (socket says 172.21.15.114:55768) 2024-02-16T22:12:08.642 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.844+0000 7f05d259c700 1 -- 172.21.15.114:0/3266180876 learned_addr learned my addr 172.21.15.114:0/3266180876 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:12:08.642 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.844+0000 7f05d259c700 1 -- 172.21.15.114:0/3266180876 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d407a9e0 con 0x7f05d407be70 2024-02-16T22:12:08.642 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.845+0000 7f05d259c700 1 --2- 172.21.15.114:0/3266180876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05d407be70 0x7f05d407a2d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f05c80098a0 tx=0x7f05c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f763962178a05a14 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:08.642 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.845+0000 7f05d159a700 1 -- 172.21.15.114:0/3266180876 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05c8004180 con 0x7f05d407be70 2024-02-16T22:12:08.642 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.845+0000 7f05d159a700 1 -- 172.21.15.114:0/3266180876 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f05c80042e0 con 0x7f05d407be70 2024-02-16T22:12:08.642 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.845+0000 7f05d159a700 1 -- 172.21.15.114:0/3266180876 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05c8011af0 con 0x7f05d407be70 2024-02-16T22:12:08.642 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.845+0000 7f05d8c8a700 1 -- 172.21.15.114:0/3266180876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05d407be70 msgr2=0x7f05d407a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:08.643 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.845+0000 7f05d8c8a700 1 --2- 172.21.15.114:0/3266180876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05d407be70 0x7f05d407a2d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f05c80098a0 tx=0x7f05c8009bb0 comp rx=0 tx=0).stop 2024-02-16T22:12:08.643 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.845+0000 7f05d8c8a700 1 -- 172.21.15.114:0/3266180876 shutdown_connections 2024-02-16T22:12:08.643 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.845+0000 7f05d8c8a700 1 --2- 172.21.15.114:0/3266180876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05d407be70 0x7f05d407a2d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:08.643 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.845+0000 7f05d8c8a700 1 -- 172.21.15.114:0/3266180876 >> 172.21.15.114:0/3266180876 conn(0x7f05d4101a00 msgr2=0x7f05d4103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:08.643 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.845+0000 7f05d8c8a700 1 -- 172.21.15.114:0/3266180876 shutdown_connections 2024-02-16T22:12:08.643 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.845+0000 7f05d8c8a700 1 -- 172.21.15.114:0/3266180876 wait complete. 2024-02-16T22:12:08.643 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.846+0000 7f05d8c8a700 1 Processor -- start 2024-02-16T22:12:08.643 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.846+0000 7f05d8c8a700 1 -- start start 2024-02-16T22:12:08.643 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.846+0000 7f05d8c8a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05d407be70 0x7f05d419c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:08.643 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.846+0000 7f05d8c8a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d419c830 con 0x7f05d407be70 2024-02-16T22:12:08.643 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.847+0000 7f05d259c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05d407be70 0x7f05d419c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:08.643 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.847+0000 7f05d259c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05d407be70 0x7f05d419c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55772/0 (socket says 172.21.15.114:55772) 2024-02-16T22:12:08.644 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.847+0000 7f05d259c700 1 -- 172.21.15.114:0/289512116 learned_addr learned my addr 172.21.15.114:0/289512116 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:12:08.644 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.847+0000 7f05d259c700 1 -- 172.21.15.114:0/289512116 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c8009550 con 0x7f05d407be70 2024-02-16T22:12:08.644 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.847+0000 7f05d259c700 1 --2- 172.21.15.114:0/289512116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05d407be70 0x7f05d419c2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f05c8011ad0 tx=0x7f05c8011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:08.644 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.847+0000 7f05c37fe700 1 -- 172.21.15.114:0/289512116 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05c8003ab0 con 0x7f05d407be70 2024-02-16T22:12:08.644 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.847+0000 7f05c37fe700 1 -- 172.21.15.114:0/289512116 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f05c8003c10 con 0x7f05d407be70 2024-02-16T22:12:08.644 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.847+0000 7f05d8c8a700 1 -- 172.21.15.114:0/289512116 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05d419ca30 con 0x7f05d407be70 2024-02-16T22:12:08.644 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.847+0000 7f05c37fe700 1 -- 172.21.15.114:0/289512116 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05c80198e0 con 0x7f05d407be70 2024-02-16T22:12:08.644 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.847+0000 7f05d8c8a700 1 -- 172.21.15.114:0/289512116 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05d419ced0 con 0x7f05d407be70 2024-02-16T22:12:08.644 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.848+0000 7f05c37fe700 1 -- 172.21.15.114:0/289512116 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f05c8019ba0 con 0x7f05d407be70 2024-02-16T22:12:08.644 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.848+0000 7f05d8c8a700 1 -- 172.21.15.114:0/289512116 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05d410c7c0 con 0x7f05d407be70 2024-02-16T22:12:08.644 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.848+0000 7f05c37fe700 1 --2- 172.21.15.114:0/289512116 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f05bc038410 0x7f05bc03a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:08.644 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.848+0000 7f05c37fe700 1 -- 172.21.15.114:0/289512116 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f05c804c350 con 0x7f05d407be70 2024-02-16T22:12:08.644 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.848+0000 7f05d1d9b700 1 --2- 172.21.15.114:0/289512116 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f05bc038410 0x7f05bc03a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:08.644 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.849+0000 7f05d1d9b700 1 --2- 172.21.15.114:0/289512116 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f05bc038410 0x7f05bc03a8d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f05c4006fd0 tx=0x7f05c4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:08.645 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:07.853+0000 7f05c37fe700 1 -- 172.21.15.114:0/289512116 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f05c802b300 con 0x7f05d407be70 2024-02-16T22:12:08.645 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:08.040+0000 7f05d8c8a700 1 -- 172.21.15.114:0/289512116 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f05d419fd10 con 0x7f05d407be70 2024-02-16T22:12:08.645 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:08.041+0000 7f05c37fe700 1 -- 172.21.15.114:0/289512116 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f05c8010b70 con 0x7f05d407be70 2024-02-16T22:12:08.645 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:08.042+0000 7f05d8c8a700 1 -- 172.21.15.114:0/289512116 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f05bc038410 msgr2=0x7f05bc03a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:08.645 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:08.042+0000 7f05d8c8a700 1 --2- 172.21.15.114:0/289512116 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f05bc038410 0x7f05bc03a8d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f05c4006fd0 tx=0x7f05c4006e40 comp rx=0 tx=0).stop 2024-02-16T22:12:08.645 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:08.042+0000 7f05d8c8a700 1 -- 172.21.15.114:0/289512116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05d407be70 msgr2=0x7f05d419c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:08.645 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:08.042+0000 7f05d8c8a700 1 --2- 172.21.15.114:0/289512116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05d407be70 0x7f05d419c2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f05c8011ad0 tx=0x7f05c8011bb0 comp rx=0 tx=0).stop 2024-02-16T22:12:08.645 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:08.042+0000 7f05d8c8a700 1 -- 172.21.15.114:0/289512116 shutdown_connections 2024-02-16T22:12:08.645 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:08.042+0000 7f05d8c8a700 1 --2- 172.21.15.114:0/289512116 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f05bc038410 0x7f05bc03a8d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:08.645 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:08.042+0000 7f05d8c8a700 1 --2- 172.21.15.114:0/289512116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05d407be70 0x7f05d419c2f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:08.645 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:08.042+0000 7f05d8c8a700 1 -- 172.21.15.114:0/289512116 >> 172.21.15.114:0/289512116 conn(0x7f05d4101a00 msgr2=0x7f05d41030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:08.645 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:08.043+0000 7f05d8c8a700 1 -- 172.21.15.114:0/289512116 shutdown_connections 2024-02-16T22:12:08.645 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:08.043+0000 7f05d8c8a700 1 -- 172.21.15.114:0/289512116 wait complete. 2024-02-16T22:12:09.915 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.045+0000 7ffa54623700 1 Processor -- start 2024-02-16T22:12:09.915 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.045+0000 7ffa54623700 1 -- start start 2024-02-16T22:12:09.915 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.046+0000 7ffa54623700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa4c106f50 0x7ffa4c109340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:09.915 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.046+0000 7ffa54623700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c0745e0 con 0x7ffa4c106f50 2024-02-16T22:12:09.915 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.046+0000 7ffa523bf700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa4c106f50 0x7ffa4c109340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:09.915 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.046+0000 7ffa523bf700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa4c106f50 0x7ffa4c109340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55780/0 (socket says 172.21.15.114:55780) 2024-02-16T22:12:09.915 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.046+0000 7ffa523bf700 1 -- 172.21.15.114:0/688696748 learned_addr learned my addr 172.21.15.114:0/688696748 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:12:09.915 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.046+0000 7ffa523bf700 1 -- 172.21.15.114:0/688696748 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa4c074720 con 0x7ffa4c106f50 2024-02-16T22:12:09.916 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.046+0000 7ffa523bf700 1 --2- 172.21.15.114:0/688696748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa4c106f50 0x7ffa4c109340 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c0098a0 tx=0x7ffa3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1171e769d4587630 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:09.916 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.046+0000 7ffa513bd700 1 -- 172.21.15.114:0/688696748 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffa3c004180 con 0x7ffa4c106f50 2024-02-16T22:12:09.916 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.046+0000 7ffa513bd700 1 -- 172.21.15.114:0/688696748 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ffa3c0042e0 con 0x7ffa4c106f50 2024-02-16T22:12:09.916 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.046+0000 7ffa513bd700 1 -- 172.21.15.114:0/688696748 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffa3c011af0 con 0x7ffa4c106f50 2024-02-16T22:12:09.916 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.047+0000 7ffa54623700 1 -- 172.21.15.114:0/688696748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa4c106f50 msgr2=0x7ffa4c109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:09.916 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.047+0000 7ffa54623700 1 --2- 172.21.15.114:0/688696748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa4c106f50 0x7ffa4c109340 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c0098a0 tx=0x7ffa3c009bb0 comp rx=0 tx=0).stop 2024-02-16T22:12:09.916 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.047+0000 7ffa54623700 1 -- 172.21.15.114:0/688696748 shutdown_connections 2024-02-16T22:12:09.916 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.047+0000 7ffa54623700 1 --2- 172.21.15.114:0/688696748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa4c106f50 0x7ffa4c109340 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:09.916 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.047+0000 7ffa54623700 1 -- 172.21.15.114:0/688696748 >> 172.21.15.114:0/688696748 conn(0x7ffa4c1008c0 msgr2=0x7ffa4c102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:09.916 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.047+0000 7ffa54623700 1 -- 172.21.15.114:0/688696748 shutdown_connections 2024-02-16T22:12:09.916 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.047+0000 7ffa54623700 1 -- 172.21.15.114:0/688696748 wait complete. 2024-02-16T22:12:09.916 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.048+0000 7ffa54623700 1 Processor -- start 2024-02-16T22:12:09.916 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.048+0000 7ffa54623700 1 -- start start 2024-02-16T22:12:09.917 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.048+0000 7ffa54623700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa4c106f50 0x7ffa4c1068b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:09.917 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.048+0000 7ffa54623700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c106df0 con 0x7ffa4c106f50 2024-02-16T22:12:09.917 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.048+0000 7ffa523bf700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa4c106f50 0x7ffa4c1068b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:09.917 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.048+0000 7ffa523bf700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa4c106f50 0x7ffa4c1068b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55786/0 (socket says 172.21.15.114:55786) 2024-02-16T22:12:09.917 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.048+0000 7ffa523bf700 1 -- 172.21.15.114:0/883917768 learned_addr learned my addr 172.21.15.114:0/883917768 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:12:09.917 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.048+0000 7ffa523bf700 1 -- 172.21.15.114:0/883917768 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa3c009550 con 0x7ffa4c106f50 2024-02-16T22:12:09.917 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.048+0000 7ffa523bf700 1 --2- 172.21.15.114:0/883917768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa4c106f50 0x7ffa4c1068b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c011670 tx=0x7ffa3c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:09.917 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.048+0000 7ffa437fe700 1 -- 172.21.15.114:0/883917768 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffa3c003a40 con 0x7ffa4c106f50 2024-02-16T22:12:09.917 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.048+0000 7ffa437fe700 1 -- 172.21.15.114:0/883917768 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ffa3c003ba0 con 0x7ffa4c106f50 2024-02-16T22:12:09.917 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.048+0000 7ffa54623700 1 -- 172.21.15.114:0/883917768 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4c104f00 con 0x7ffa4c106f50 2024-02-16T22:12:09.917 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.048+0000 7ffa437fe700 1 -- 172.21.15.114:0/883917768 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffa3c024820 con 0x7ffa4c106f50 2024-02-16T22:12:09.917 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.048+0000 7ffa54623700 1 -- 172.21.15.114:0/883917768 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4c1053a0 con 0x7ffa4c106f50 2024-02-16T22:12:09.917 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.049+0000 7ffa437fe700 1 -- 172.21.15.114:0/883917768 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7ffa3c024ae0 con 0x7ffa4c106f50 2024-02-16T22:12:09.917 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.049+0000 7ffa54623700 1 -- 172.21.15.114:0/883917768 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4c10a5a0 con 0x7ffa4c106f50 2024-02-16T22:12:09.917 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.049+0000 7ffa437fe700 1 --2- 172.21.15.114:0/883917768 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7ffa380383d0 0x7ffa3803a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:09.918 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.049+0000 7ffa437fe700 1 -- 172.21.15.114:0/883917768 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ffa3c04c610 con 0x7ffa4c106f50 2024-02-16T22:12:09.918 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.050+0000 7ffa51bbe700 1 --2- 172.21.15.114:0/883917768 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7ffa380383d0 0x7ffa3803a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:09.918 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.050+0000 7ffa51bbe700 1 --2- 172.21.15.114:0/883917768 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7ffa380383d0 0x7ffa3803a890 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffa48006fd0 tx=0x7ffa48006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:09.918 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.054+0000 7ffa437fe700 1 -- 172.21.15.114:0/883917768 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ffa3c018b80 con 0x7ffa4c106f50 2024-02-16T22:12:09.918 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.345+0000 7ffa54623700 1 -- 172.21.15.114:0/883917768 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7ffa4c10a7c0 con 0x7ffa4c106f50 2024-02-16T22:12:09.918 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.348+0000 7ffa437fe700 1 -- 172.21.15.114:0/883917768 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v34)=0 set mgr/dashboard/cluster/status v34) v1 ==== 153+0+0 (secure 0 0 0) 0x7ffa3c0187c0 con 0x7ffa4c106f50 2024-02-16T22:12:09.918 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.350+0000 7ffa54623700 1 -- 172.21.15.114:0/883917768 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7ffa380383d0 msgr2=0x7ffa3803a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:09.918 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.350+0000 7ffa54623700 1 --2- 172.21.15.114:0/883917768 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7ffa380383d0 0x7ffa3803a890 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffa48006fd0 tx=0x7ffa48006e40 comp rx=0 tx=0).stop 2024-02-16T22:12:09.918 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.350+0000 7ffa54623700 1 -- 172.21.15.114:0/883917768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa4c106f50 msgr2=0x7ffa4c1068b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:09.918 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.350+0000 7ffa54623700 1 --2- 172.21.15.114:0/883917768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa4c106f50 0x7ffa4c1068b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c011670 tx=0x7ffa3c011aa0 comp rx=0 tx=0).stop 2024-02-16T22:12:09.918 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.350+0000 7ffa54623700 1 -- 172.21.15.114:0/883917768 shutdown_connections 2024-02-16T22:12:09.918 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.350+0000 7ffa54623700 1 --2- 172.21.15.114:0/883917768 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7ffa380383d0 0x7ffa3803a890 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:09.918 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.350+0000 7ffa54623700 1 --2- 172.21.15.114:0/883917768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa4c106f50 0x7ffa4c1068b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:09.918 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.350+0000 7ffa54623700 1 -- 172.21.15.114:0/883917768 >> 172.21.15.114:0/883917768 conn(0x7ffa4c1008c0 msgr2=0x7ffa4c101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:09.919 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.350+0000 7ffa54623700 1 -- 172.21.15.114:0/883917768 shutdown_connections 2024-02-16T22:12:09.919 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-02-16T22:12:09.350+0000 7ffa54623700 1 -- 172.21.15.114:0/883917768 wait complete. 2024-02-16T22:12:09.919 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-16T22:12:09.919 INFO:teuthology.orchestra.run.smithi114.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-16T22:12:09.919 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:12:09.919 INFO:teuthology.orchestra.run.smithi114.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-16T22:12:09.919 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:12:09.919 INFO:teuthology.orchestra.run.smithi114.stdout:Or, if you are only running a single cluster on this host: 2024-02-16T22:12:09.919 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:12:09.919 INFO:teuthology.orchestra.run.smithi114.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-16T22:12:09.919 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:12:09.919 INFO:teuthology.orchestra.run.smithi114.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-16T22:12:09.919 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:12:09.919 INFO:teuthology.orchestra.run.smithi114.stdout: ceph telemetry on 2024-02-16T22:12:09.919 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:12:09.919 INFO:teuthology.orchestra.run.smithi114.stdout:For more information see: 2024-02-16T22:12:09.920 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:12:09.920 INFO:teuthology.orchestra.run.smithi114.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-02-16T22:12:09.920 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:12:09.920 INFO:teuthology.orchestra.run.smithi114.stdout:Bootstrap complete. 2024-02-16T22:12:09.948 INFO:tasks.cephadm:Fetching config... 2024-02-16T22:12:09.948 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-02-16T22:12:09.948 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-16T22:12:09.964 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-16T22:12:09.964 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-02-16T22:12:09.964 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-16T22:12:10.019 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-16T22:12:10.019 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-02-16T22:12:10.019 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/keyring of=/dev/stdout 2024-02-16T22:12:10.087 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-16T22:12:10.087 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-02-16T22:12:10.087 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-16T22:12:10.142 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-16T22:12:10.142 DEBUG:teuthology.orchestra.run.smithi114:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQChsI2oVIGL2COkfQpQTevyqsLNDIIG8Sny9IgRGFwqsK1hSno1iwmFF58iHEvXX8goXr2OhmOeyGDqJDpFJvPOiwS3J1HjvN+sENf0ZH6WBzaoF2ugcGpzlw3+v/55FMZdxq2nABf26+/e8TeQSA7cyIWgI+GppYyZReKWozyEWDif7LS2akO3Ya4ea+zaNZBNapHysVPK/ndQa/iXGHMcQ6mK9jY50Wdq0uCrs3bQKp89o9hugILk1Nm5mQnF051Vp4jkoZfNyitw2fRZS2aRmonzyBAmGxvnDV5Mi8VHfT66X9ciR+U7wQ0UQOTUKMKF9F5j/pIr6OsF7dtsqZAVR0HBsQ8/XkY9e9hrJtNywRC/NypuAHooHnuymxV3zW1m7ZK2uiuEqS3/7VfqPQdACcPdvKpTOd9wi5t+S4abuNfA8PvrXrMOrQd1Cxf1BKd9kQ1GyPC2B8Oh46rks5yH1C1zjWqdJBCSBwVHYA4gNWTyuj2E8lgnKz3MszfeqXU= ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-16T22:12:10.219 INFO:teuthology.orchestra.run.smithi114.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQChsI2oVIGL2COkfQpQTevyqsLNDIIG8Sny9IgRGFwqsK1hSno1iwmFF58iHEvXX8goXr2OhmOeyGDqJDpFJvPOiwS3J1HjvN+sENf0ZH6WBzaoF2ugcGpzlw3+v/55FMZdxq2nABf26+/e8TeQSA7cyIWgI+GppYyZReKWozyEWDif7LS2akO3Ya4ea+zaNZBNapHysVPK/ndQa/iXGHMcQ6mK9jY50Wdq0uCrs3bQKp89o9hugILk1Nm5mQnF051Vp4jkoZfNyitw2fRZS2aRmonzyBAmGxvnDV5Mi8VHfT66X9ciR+U7wQ0UQOTUKMKF9F5j/pIr6OsF7dtsqZAVR0HBsQ8/XkY9e9hrJtNywRC/NypuAHooHnuymxV3zW1m7ZK2uiuEqS3/7VfqPQdACcPdvKpTOd9wi5t+S4abuNfA8PvrXrMOrQd1Cxf1BKd9kQ1GyPC2B8Oh46rks5yH1C1zjWqdJBCSBwVHYA4gNWTyuj2E8lgnKz3MszfeqXU= ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715 2024-02-16T22:12:10.233 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQChsI2oVIGL2COkfQpQTevyqsLNDIIG8Sny9IgRGFwqsK1hSno1iwmFF58iHEvXX8goXr2OhmOeyGDqJDpFJvPOiwS3J1HjvN+sENf0ZH6WBzaoF2ugcGpzlw3+v/55FMZdxq2nABf26+/e8TeQSA7cyIWgI+GppYyZReKWozyEWDif7LS2akO3Ya4ea+zaNZBNapHysVPK/ndQa/iXGHMcQ6mK9jY50Wdq0uCrs3bQKp89o9hugILk1Nm5mQnF051Vp4jkoZfNyitw2fRZS2aRmonzyBAmGxvnDV5Mi8VHfT66X9ciR+U7wQ0UQOTUKMKF9F5j/pIr6OsF7dtsqZAVR0HBsQ8/XkY9e9hrJtNywRC/NypuAHooHnuymxV3zW1m7ZK2uiuEqS3/7VfqPQdACcPdvKpTOd9wi5t+S4abuNfA8PvrXrMOrQd1Cxf1BKd9kQ1GyPC2B8Oh46rks5yH1C1zjWqdJBCSBwVHYA4gNWTyuj2E8lgnKz3MszfeqXU= ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-16T22:12:10.279 INFO:teuthology.orchestra.run.smithi123.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQChsI2oVIGL2COkfQpQTevyqsLNDIIG8Sny9IgRGFwqsK1hSno1iwmFF58iHEvXX8goXr2OhmOeyGDqJDpFJvPOiwS3J1HjvN+sENf0ZH6WBzaoF2ugcGpzlw3+v/55FMZdxq2nABf26+/e8TeQSA7cyIWgI+GppYyZReKWozyEWDif7LS2akO3Ya4ea+zaNZBNapHysVPK/ndQa/iXGHMcQ6mK9jY50Wdq0uCrs3bQKp89o9hugILk1Nm5mQnF051Vp4jkoZfNyitw2fRZS2aRmonzyBAmGxvnDV5Mi8VHfT66X9ciR+U7wQ0UQOTUKMKF9F5j/pIr6OsF7dtsqZAVR0HBsQ8/XkY9e9hrJtNywRC/NypuAHooHnuymxV3zW1m7ZK2uiuEqS3/7VfqPQdACcPdvKpTOd9wi5t+S4abuNfA8PvrXrMOrQd1Cxf1BKd9kQ1GyPC2B8Oh46rks5yH1C1zjWqdJBCSBwVHYA4gNWTyuj2E8lgnKz3MszfeqXU= ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715 2024-02-16T22:12:10.294 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-16T22:12:10.502 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:10 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/883917768' entity='client.admin' 2024-02-16T22:12:10.574 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:12:13.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.262+0000 7f37b3da7700 1 -- 172.21.15.114:0/4176819657 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37ac100d50 msgr2=0x7f37ac101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:13.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.262+0000 7f37b3da7700 1 --2- 172.21.15.114:0/4176819657 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37ac100d50 0x7f37ac101170 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009910 tx=0x7f37a8009c20 comp rx=0 tx=0).stop 2024-02-16T22:12:13.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.262+0000 7f37b3da7700 1 -- 172.21.15.114:0/4176819657 shutdown_connections 2024-02-16T22:12:13.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.262+0000 7f37b3da7700 1 --2- 172.21.15.114:0/4176819657 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37ac100d50 0x7f37ac101170 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:13.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.262+0000 7f37b3da7700 1 -- 172.21.15.114:0/4176819657 >> 172.21.15.114:0/4176819657 conn(0x7f37ac0fa4a0 msgr2=0x7f37ac0fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:13.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.262+0000 7f37b3da7700 1 -- 172.21.15.114:0/4176819657 shutdown_connections 2024-02-16T22:12:13.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.263+0000 7f37b3da7700 1 -- 172.21.15.114:0/4176819657 wait complete. 2024-02-16T22:12:13.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.263+0000 7f37b3da7700 1 Processor -- start 2024-02-16T22:12:13.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.263+0000 7f37b3da7700 1 -- start start 2024-02-16T22:12:13.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.263+0000 7f37b3da7700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37ac100d50 0x7f37ac194860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:13.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.263+0000 7f37b3da7700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac194da0 con 0x7f37ac100d50 2024-02-16T22:12:13.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.264+0000 7f37b1b43700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37ac100d50 0x7f37ac194860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:13.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.264+0000 7f37b1b43700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37ac100d50 0x7f37ac194860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35932/0 (socket says 172.21.15.114:35932) 2024-02-16T22:12:13.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.264+0000 7f37b1b43700 1 -- 172.21.15.114:0/213381608 learned_addr learned my addr 172.21.15.114:0/213381608 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:12:13.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.264+0000 7f37b1b43700 1 -- 172.21.15.114:0/213381608 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a80095f0 con 0x7f37ac100d50 2024-02-16T22:12:13.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.264+0000 7f37b1b43700 1 --2- 172.21.15.114:0/213381608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37ac100d50 0x7f37ac194860 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f37a8005b40 tx=0x7f37a8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:13.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.264+0000 7f37a2ffd700 1 -- 172.21.15.114:0/213381608 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f37a801d070 con 0x7f37ac100d50 2024-02-16T22:12:13.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.264+0000 7f37a2ffd700 1 -- 172.21.15.114:0/213381608 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f37a800be10 con 0x7f37ac100d50 2024-02-16T22:12:13.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.264+0000 7f37b3da7700 1 -- 172.21.15.114:0/213381608 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ac194fa0 con 0x7f37ac100d50 2024-02-16T22:12:13.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.264+0000 7f37a2ffd700 1 -- 172.21.15.114:0/213381608 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f37a800f460 con 0x7f37ac100d50 2024-02-16T22:12:13.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.264+0000 7f37b3da7700 1 -- 172.21.15.114:0/213381608 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ac195440 con 0x7f37ac100d50 2024-02-16T22:12:13.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.265+0000 7f37a2ffd700 1 -- 172.21.15.114:0/213381608 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f37a800f6d0 con 0x7f37ac100d50 2024-02-16T22:12:13.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.265+0000 7f37b3da7700 1 -- 172.21.15.114:0/213381608 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37ac1055d0 con 0x7f37ac100d50 2024-02-16T22:12:13.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.265+0000 7f37a2ffd700 1 --2- 172.21.15.114:0/213381608 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f37980383c0 0x7f379803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:13.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.265+0000 7f37a2ffd700 1 -- 172.21.15.114:0/213381608 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f37a804c580 con 0x7f37ac100d50 2024-02-16T22:12:13.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.266+0000 7f37b1342700 1 --2- 172.21.15.114:0/213381608 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f37980383c0 0x7f379803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:13.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.266+0000 7f37b1342700 1 --2- 172.21.15.114:0/213381608 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f37980383c0 0x7f379803a880 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f379c006fd0 tx=0x7f379c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:13.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.270+0000 7f37a2ffd700 1 -- 172.21.15.114:0/213381608 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f37a802a360 con 0x7f37ac100d50 2024-02-16T22:12:13.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.469+0000 7f37b3da7700 1 -- 172.21.15.114:0/213381608 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f37ac1057c0 con 0x7f37ac100d50 2024-02-16T22:12:13.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.472+0000 7f37a2ffd700 1 -- 172.21.15.114:0/213381608 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f37a8027070 con 0x7f37ac100d50 2024-02-16T22:12:13.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.475+0000 7f37b3da7700 1 -- 172.21.15.114:0/213381608 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f37980383c0 msgr2=0x7f379803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:13.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.475+0000 7f37b3da7700 1 --2- 172.21.15.114:0/213381608 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f37980383c0 0x7f379803a880 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f379c006fd0 tx=0x7f379c006e40 comp rx=0 tx=0).stop 2024-02-16T22:12:13.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.475+0000 7f37b3da7700 1 -- 172.21.15.114:0/213381608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37ac100d50 msgr2=0x7f37ac194860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:13.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.475+0000 7f37b3da7700 1 --2- 172.21.15.114:0/213381608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37ac100d50 0x7f37ac194860 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f37a8005b40 tx=0x7f37a8004740 comp rx=0 tx=0).stop 2024-02-16T22:12:13.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.475+0000 7f37b3da7700 1 -- 172.21.15.114:0/213381608 shutdown_connections 2024-02-16T22:12:13.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.475+0000 7f37b3da7700 1 --2- 172.21.15.114:0/213381608 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f37980383c0 0x7f379803a880 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:13.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.475+0000 7f37b3da7700 1 --2- 172.21.15.114:0/213381608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37ac100d50 0x7f37ac194860 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:13.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.475+0000 7f37b3da7700 1 -- 172.21.15.114:0/213381608 >> 172.21.15.114:0/213381608 conn(0x7f37ac0fa4a0 msgr2=0x7f37ac0fb180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:13.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.475+0000 7f37b3da7700 1 -- 172.21.15.114:0/213381608 shutdown_connections 2024-02-16T22:12:13.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:13.475+0000 7f37b3da7700 1 -- 172.21.15.114:0/213381608 wait complete. 2024-02-16T22:12:14.253 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-16T22:12:14.254 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-16T22:12:14.646 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:14 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/213381608' entity='client.admin' 2024-02-16T22:12:14.799 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:12:15.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:15 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:15.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:15 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:15.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:15 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "who": "osd/host:smithi114", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:12:15.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:15 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:15.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:15 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi114", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T22:12:15.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:15 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi114", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-02-16T22:12:15.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:15 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:12:15.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:15 smithi114 ceph-mon[29235]: Deploying daemon ceph-exporter.smithi114 on smithi114 2024-02-16T22:12:16.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.412+0000 7f0428f29700 1 -- 172.21.15.114:0/1582757543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04241008b0 msgr2=0x7f0424100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:16.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.412+0000 7f0428f29700 1 --2- 172.21.15.114:0/1582757543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04241008b0 0x7f0424100cd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0414009910 tx=0x7f0414009c20 comp rx=0 tx=0).stop 2024-02-16T22:12:16.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.412+0000 7f0428f29700 1 -- 172.21.15.114:0/1582757543 shutdown_connections 2024-02-16T22:12:16.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.412+0000 7f0428f29700 1 --2- 172.21.15.114:0/1582757543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04241008b0 0x7f0424100cd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:16.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.412+0000 7f0428f29700 1 -- 172.21.15.114:0/1582757543 >> 172.21.15.114:0/1582757543 conn(0x7f04240fbe30 msgr2=0x7f04240fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:16.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.412+0000 7f0428f29700 1 -- 172.21.15.114:0/1582757543 shutdown_connections 2024-02-16T22:12:16.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.412+0000 7f0428f29700 1 -- 172.21.15.114:0/1582757543 wait complete. 2024-02-16T22:12:16.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.412+0000 7f0428f29700 1 Processor -- start 2024-02-16T22:12:16.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.413+0000 7f0428f29700 1 -- start start 2024-02-16T22:12:16.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.413+0000 7f0428f29700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04241008b0 0x7f0424195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:16.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.413+0000 7f0428f29700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0424195f40 con 0x7f04241008b0 2024-02-16T22:12:16.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.413+0000 7f042259c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04241008b0 0x7f0424195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:16.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.413+0000 7f042259c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04241008b0 0x7f0424195a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35956/0 (socket says 172.21.15.114:35956) 2024-02-16T22:12:16.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.413+0000 7f042259c700 1 -- 172.21.15.114:0/1410165856 learned_addr learned my addr 172.21.15.114:0/1410165856 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:12:16.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.413+0000 7f042259c700 1 -- 172.21.15.114:0/1410165856 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04140095f0 con 0x7f04241008b0 2024-02-16T22:12:16.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.413+0000 7f042259c700 1 --2- 172.21.15.114:0/1410165856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04241008b0 0x7f0424195a00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0414005b40 tx=0x7f0414004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:16.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.413+0000 7f04137fe700 1 -- 172.21.15.114:0/1410165856 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f041401d070 con 0x7f04241008b0 2024-02-16T22:12:16.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.413+0000 7f04137fe700 1 -- 172.21.15.114:0/1410165856 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f041400be10 con 0x7f04241008b0 2024-02-16T22:12:16.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.413+0000 7f0428f29700 1 -- 172.21.15.114:0/1410165856 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0424196140 con 0x7f04241008b0 2024-02-16T22:12:16.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.413+0000 7f04137fe700 1 -- 172.21.15.114:0/1410165856 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f041400f460 con 0x7f04241008b0 2024-02-16T22:12:16.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.413+0000 7f0428f29700 1 -- 172.21.15.114:0/1410165856 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04241965e0 con 0x7f04241008b0 2024-02-16T22:12:16.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.414+0000 7f04137fe700 1 -- 172.21.15.114:0/1410165856 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f041400f5c0 con 0x7f04241008b0 2024-02-16T22:12:16.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.414+0000 7f0428f29700 1 -- 172.21.15.114:0/1410165856 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f042418f4d0 con 0x7f04241008b0 2024-02-16T22:12:16.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.414+0000 7f04137fe700 1 --2- 172.21.15.114:0/1410165856 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f040c038410 0x7f040c03a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:16.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.414+0000 7f04137fe700 1 -- 172.21.15.114:0/1410165856 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f041404c490 con 0x7f04241008b0 2024-02-16T22:12:16.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.415+0000 7f0421d9b700 1 --2- 172.21.15.114:0/1410165856 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f040c038410 0x7f040c03a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:16.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.415+0000 7f0421d9b700 1 --2- 172.21.15.114:0/1410165856 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f040c038410 0x7f040c03a8d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0418006fd0 tx=0x7f0418006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:16.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.419+0000 7f04137fe700 1 -- 172.21.15.114:0/1410165856 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f041402a360 con 0x7f04241008b0 2024-02-16T22:12:16.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.613+0000 7f0428f29700 1 -- 172.21.15.114:0/1410165856 --> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f0424061260 con 0x7f040c038410 2024-02-16T22:12:16.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.617+0000 7f04137fe700 1 -- 172.21.15.114:0/1410165856 <== mgr.14164 v2:172.21.15.114:6800/2648536185 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0424061260 con 0x7f040c038410 2024-02-16T22:12:16.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.619+0000 7f0428f29700 1 -- 172.21.15.114:0/1410165856 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f040c038410 msgr2=0x7f040c03a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:16.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.619+0000 7f0428f29700 1 --2- 172.21.15.114:0/1410165856 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f040c038410 0x7f040c03a8d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0418006fd0 tx=0x7f0418006e40 comp rx=0 tx=0).stop 2024-02-16T22:12:16.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.619+0000 7f0428f29700 1 -- 172.21.15.114:0/1410165856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04241008b0 msgr2=0x7f0424195a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:16.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.619+0000 7f0428f29700 1 --2- 172.21.15.114:0/1410165856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04241008b0 0x7f0424195a00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0414005b40 tx=0x7f0414004740 comp rx=0 tx=0).stop 2024-02-16T22:12:16.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.619+0000 7f0428f29700 1 -- 172.21.15.114:0/1410165856 shutdown_connections 2024-02-16T22:12:16.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.619+0000 7f0428f29700 1 --2- 172.21.15.114:0/1410165856 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f040c038410 0x7f040c03a8d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:16.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.619+0000 7f0428f29700 1 --2- 172.21.15.114:0/1410165856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04241008b0 0x7f0424195a00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:16.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.619+0000 7f0428f29700 1 -- 172.21.15.114:0/1410165856 >> 172.21.15.114:0/1410165856 conn(0x7f04240fbe30 msgr2=0x7f04240fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:16.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.619+0000 7f0428f29700 1 -- 172.21.15.114:0/1410165856 shutdown_connections 2024-02-16T22:12:16.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:16.619+0000 7f0428f29700 1 -- 172.21.15.114:0/1410165856 wait complete. 2024-02-16T22:12:17.882 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:17 smithi114 ceph-mon[29235]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:12:17.882 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:17 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:18.080 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi123 2024-02-16T22:12:18.080 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-02-16T22:12:18.080 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/etc/ceph/ceph.conf 2024-02-16T22:12:18.099 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-02-16T22:12:18.099 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:12:18.157 INFO:tasks.cephadm:Adding host smithi123 to orchestrator... 2024-02-16T22:12:18.157 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph orch host add smithi123 2024-02-16T22:12:18.836 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:12:19.909 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:19 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:19.909 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:19 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:19.909 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:19 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:19.909 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:19 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:19.909 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:19 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi114", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T22:12:19.909 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:19 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi114", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-02-16T22:12:19.909 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:19 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:12:19.909 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:19 smithi114 ceph-mon[29235]: Deploying daemon crash.smithi114 on smithi114 2024-02-16T22:12:21.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:21.448+0000 7f4a5c2d1700 1 -- 172.21.15.114:0/1543237013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a54100890 msgr2=0x7f4a54100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:21.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:21.448+0000 7f4a5c2d1700 1 --2- 172.21.15.114:0/1543237013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a54100890 0x7f4a54100cb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009910 tx=0x7f4a50009c20 comp rx=0 tx=0).stop 2024-02-16T22:12:21.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:21.449+0000 7f4a5c2d1700 1 -- 172.21.15.114:0/1543237013 shutdown_connections 2024-02-16T22:12:21.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:21.449+0000 7f4a5c2d1700 1 --2- 172.21.15.114:0/1543237013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a54100890 0x7f4a54100cb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:21.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:21.449+0000 7f4a5c2d1700 1 -- 172.21.15.114:0/1543237013 >> 172.21.15.114:0/1543237013 conn(0x7f4a540fbe30 msgr2=0x7f4a540fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:21.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:21.449+0000 7f4a5c2d1700 1 -- 172.21.15.114:0/1543237013 shutdown_connections 2024-02-16T22:12:21.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:21.449+0000 7f4a5c2d1700 1 -- 172.21.15.114:0/1543237013 wait complete. 2024-02-16T22:12:21.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:21.449+0000 7f4a5c2d1700 1 Processor -- start 2024-02-16T22:12:21.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:21.449+0000 7f4a5c2d1700 1 -- start start 2024-02-16T22:12:21.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:21.449+0000 7f4a5c2d1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a54100890 0x7f4a541959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:21.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:21.449+0000 7f4a5c2d1700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a54195ef0 con 0x7f4a54100890 2024-02-16T22:12:21.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:21.450+0000 7f4a5a06d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a54100890 0x7f4a541959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:21.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:21.450+0000 7f4a5a06d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a54100890 0x7f4a541959b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51658/0 (socket says 172.21.15.114:51658) 2024-02-16T22:12:21.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:21.450+0000 7f4a5a06d700 1 -- 172.21.15.114:0/1879289709 learned_addr learned my addr 172.21.15.114:0/1879289709 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:12:21.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:21.450+0000 7f4a5a06d700 1 -- 172.21.15.114:0/1879289709 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a500095f0 con 0x7f4a54100890 2024-02-16T22:12:21.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:21.450+0000 7f4a5a06d700 1 --2- 172.21.15.114:0/1879289709 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a54100890 0x7f4a541959b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4a50005950 tx=0x7f4a50009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:21.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:21.450+0000 7f4a4b7fe700 1 -- 172.21.15.114:0/1879289709 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4a5001d070 con 0x7f4a54100890 2024-02-16T22:12:21.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:21.450+0000 7f4a4b7fe700 1 -- 172.21.15.114:0/1879289709 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4a5000be10 con 0x7f4a54100890 2024-02-16T22:12:21.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:21.450+0000 7f4a4b7fe700 1 -- 172.21.15.114:0/1879289709 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4a5000f460 con 0x7f4a54100890 2024-02-16T22:12:21.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:21.450+0000 7f4a5c2d1700 1 -- 172.21.15.114:0/1879289709 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a541960f0 con 0x7f4a54100890 2024-02-16T22:12:21.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:21.450+0000 7f4a5c2d1700 1 -- 172.21.15.114:0/1879289709 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a54196590 con 0x7f4a54100890 2024-02-16T22:12:21.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:21.451+0000 7f4a4b7fe700 1 -- 172.21.15.114:0/1879289709 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f4a5000f690 con 0x7f4a54100890 2024-02-16T22:12:21.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:21.451+0000 7f4a5c2d1700 1 -- 172.21.15.114:0/1879289709 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a5418f4b0 con 0x7f4a54100890 2024-02-16T22:12:21.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:21.451+0000 7f4a4b7fe700 1 --2- 172.21.15.114:0/1879289709 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f4a400383c0 0x7f4a4003a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:21.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:21.451+0000 7f4a4b7fe700 1 -- 172.21.15.114:0/1879289709 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4a5004c590 con 0x7f4a54100890 2024-02-16T22:12:21.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:21.452+0000 7f4a5986c700 1 --2- 172.21.15.114:0/1879289709 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f4a400383c0 0x7f4a4003a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:21.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:21.452+0000 7f4a5986c700 1 --2- 172.21.15.114:0/1879289709 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f4a400383c0 0x7f4a4003a880 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4a44006fd0 tx=0x7f4a44006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:21.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:21.456+0000 7f4a4b7fe700 1 -- 172.21.15.114:0/1879289709 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4a50020ba0 con 0x7f4a54100890 2024-02-16T22:12:21.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:21.666+0000 7f4a5c2d1700 1 -- 172.21.15.114:0/1879289709 --> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi123", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a540611d0 con 0x7f4a400383c0 2024-02-16T22:12:21.669 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:21 smithi114 ceph-mon[29235]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:12:21.669 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:21 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:23.078 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:22 smithi114 ceph-mon[29235]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi123", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:12:23.078 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:22 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:23.078 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:22 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:23.078 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:22 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:23.078 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:22 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:23.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:23 smithi114 ceph-mon[29235]: Deploying daemon node-exporter.smithi114 on smithi114 2024-02-16T22:12:23.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:23 smithi114 ceph-mon[29235]: Deploying cephadm binary to smithi123 2024-02-16T22:12:23.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:23 smithi114 ceph-mon[29235]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:12:24.302 INFO:teuthology.orchestra.run.smithi114.stdout:Added host 'smithi123' with addr '172.21.15.123' 2024-02-16T22:12:24.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:24.298+0000 7f4a4b7fe700 1 -- 172.21.15.114:0/1879289709 <== mgr.14164 v2:172.21.15.114:6800/2648536185 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f4a540611d0 con 0x7f4a400383c0 2024-02-16T22:12:24.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:24.301+0000 7f4a5c2d1700 1 -- 172.21.15.114:0/1879289709 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f4a400383c0 msgr2=0x7f4a4003a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:24.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:24.301+0000 7f4a5c2d1700 1 --2- 172.21.15.114:0/1879289709 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f4a400383c0 0x7f4a4003a880 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4a44006fd0 tx=0x7f4a44006e40 comp rx=0 tx=0).stop 2024-02-16T22:12:24.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:24.301+0000 7f4a5c2d1700 1 -- 172.21.15.114:0/1879289709 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a54100890 msgr2=0x7f4a541959b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:24.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:24.301+0000 7f4a5c2d1700 1 --2- 172.21.15.114:0/1879289709 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a54100890 0x7f4a541959b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4a50005950 tx=0x7f4a50009e60 comp rx=0 tx=0).stop 2024-02-16T22:12:24.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:24.301+0000 7f4a5c2d1700 1 -- 172.21.15.114:0/1879289709 shutdown_connections 2024-02-16T22:12:24.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:24.301+0000 7f4a5c2d1700 1 --2- 172.21.15.114:0/1879289709 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f4a400383c0 0x7f4a4003a880 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:24.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:24.301+0000 7f4a5c2d1700 1 --2- 172.21.15.114:0/1879289709 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a54100890 0x7f4a541959b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:24.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:24.301+0000 7f4a5c2d1700 1 -- 172.21.15.114:0/1879289709 >> 172.21.15.114:0/1879289709 conn(0x7f4a540fbe30 msgr2=0x7f4a540fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:24.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:24.301+0000 7f4a5c2d1700 1 -- 172.21.15.114:0/1879289709 shutdown_connections 2024-02-16T22:12:24.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:24.301+0000 7f4a5c2d1700 1 -- 172.21.15.114:0/1879289709 wait complete. 2024-02-16T22:12:25.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:25 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:25.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:25 smithi114 ceph-mon[29235]: Added host smithi123 2024-02-16T22:12:25.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:25 smithi114 ceph-mon[29235]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:12:25.779 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph orch host ls --format=json 2024-02-16T22:12:26.259 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:12:26.645 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:26 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:26.646 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:26 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:26.646 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:26 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:26.646 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:26 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:26.646 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:26 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:27.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:27.855+0000 7f348d786700 1 -- 172.21.15.114:0/4188883163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34880ff790 msgr2=0x7f34880ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:27.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:27.855+0000 7f348d786700 1 --2- 172.21.15.114:0/4188883163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34880ff790 0x7f34880ffbb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3478009910 tx=0x7f3478009c20 comp rx=0 tx=0).stop 2024-02-16T22:12:27.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:27.855+0000 7f348d786700 1 -- 172.21.15.114:0/4188883163 shutdown_connections 2024-02-16T22:12:27.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:27.855+0000 7f348d786700 1 --2- 172.21.15.114:0/4188883163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34880ff790 0x7f34880ffbb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:27.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:27.855+0000 7f348d786700 1 -- 172.21.15.114:0/4188883163 >> 172.21.15.114:0/4188883163 conn(0x7f34880fad30 msgr2=0x7f34880fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:27.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:27.855+0000 7f348d786700 1 -- 172.21.15.114:0/4188883163 shutdown_connections 2024-02-16T22:12:27.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:27.855+0000 7f348d786700 1 -- 172.21.15.114:0/4188883163 wait complete. 2024-02-16T22:12:27.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:27.855+0000 7f348d786700 1 Processor -- start 2024-02-16T22:12:27.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:27.855+0000 7f348d786700 1 -- start start 2024-02-16T22:12:27.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:27.856+0000 7f348d786700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34880ff790 0x7f3488195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:27.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:27.856+0000 7f348d786700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3488195ed0 con 0x7f34880ff790 2024-02-16T22:12:27.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:27.856+0000 7f3486ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34880ff790 0x7f3488195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:27.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:27.856+0000 7f3486ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34880ff790 0x7f3488195990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51698/0 (socket says 172.21.15.114:51698) 2024-02-16T22:12:27.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:27.856+0000 7f3486ffd700 1 -- 172.21.15.114:0/1512486370 learned_addr learned my addr 172.21.15.114:0/1512486370 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:12:27.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:27.856+0000 7f3486ffd700 1 -- 172.21.15.114:0/1512486370 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34780095f0 con 0x7f34880ff790 2024-02-16T22:12:27.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:27.856+0000 7f3486ffd700 1 --2- 172.21.15.114:0/1512486370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34880ff790 0x7f3488195990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3478005b40 tx=0x7f3478004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:27.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:27.856+0000 7f3484ff9700 1 -- 172.21.15.114:0/1512486370 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f347801d070 con 0x7f34880ff790 2024-02-16T22:12:27.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:27.856+0000 7f3484ff9700 1 -- 172.21.15.114:0/1512486370 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f347800be10 con 0x7f34880ff790 2024-02-16T22:12:27.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:27.856+0000 7f348d786700 1 -- 172.21.15.114:0/1512486370 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34881960d0 con 0x7f34880ff790 2024-02-16T22:12:27.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:27.856+0000 7f3484ff9700 1 -- 172.21.15.114:0/1512486370 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f347800f460 con 0x7f34880ff790 2024-02-16T22:12:27.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:27.856+0000 7f348d786700 1 -- 172.21.15.114:0/1512486370 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3488196570 con 0x7f34880ff790 2024-02-16T22:12:27.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:27.857+0000 7f3484ff9700 1 -- 172.21.15.114:0/1512486370 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f347800f6d0 con 0x7f34880ff790 2024-02-16T22:12:27.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:27.857+0000 7f348d786700 1 -- 172.21.15.114:0/1512486370 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f348818f3c0 con 0x7f34880ff790 2024-02-16T22:12:27.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:27.857+0000 7f3484ff9700 1 --2- 172.21.15.114:0/1512486370 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f34740383c0 0x7f347403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:27.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:27.857+0000 7f3484ff9700 1 -- 172.21.15.114:0/1512486370 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f347804c590 con 0x7f34880ff790 2024-02-16T22:12:27.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:27.857+0000 7f347e7fc700 1 --2- 172.21.15.114:0/1512486370 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f34740383c0 0x7f347403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:27.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:27.858+0000 7f347e7fc700 1 --2- 172.21.15.114:0/1512486370 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f34740383c0 0x7f347403a880 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3470006fd0 tx=0x7f3470006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:27.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:27.862+0000 7f3484ff9700 1 -- 172.21.15.114:0/1512486370 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f347802a360 con 0x7f34880ff790 2024-02-16T22:12:27.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:27 smithi114 ceph-mon[29235]: Deploying daemon alertmanager.smithi114 on smithi114 2024-02-16T22:12:27.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:27 smithi114 ceph-mon[29235]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:12:28.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:28.056+0000 7f348d786700 1 -- 172.21.15.114:0/1512486370 --> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f34880611d0 con 0x7f34740383c0 2024-02-16T22:12:28.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:28.057+0000 7f3484ff9700 1 -- 172.21.15.114:0/1512486370 <== mgr.14164 v2:172.21.15.114:6800/2648536185 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f34880611d0 con 0x7f34740383c0 2024-02-16T22:12:28.059 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:12:28.059 INFO:teuthology.orchestra.run.smithi114.stdout:[{"addr": "172.21.15.114", "hostname": "smithi114", "labels": [], "status": ""}, {"addr": "172.21.15.123", "hostname": "smithi123", "labels": [], "status": ""}] 2024-02-16T22:12:28.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:28.059+0000 7f348d786700 1 -- 172.21.15.114:0/1512486370 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f34740383c0 msgr2=0x7f347403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:28.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:28.059+0000 7f348d786700 1 --2- 172.21.15.114:0/1512486370 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f34740383c0 0x7f347403a880 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3470006fd0 tx=0x7f3470006e40 comp rx=0 tx=0).stop 2024-02-16T22:12:28.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:28.059+0000 7f348d786700 1 -- 172.21.15.114:0/1512486370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34880ff790 msgr2=0x7f3488195990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:28.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:28.059+0000 7f348d786700 1 --2- 172.21.15.114:0/1512486370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34880ff790 0x7f3488195990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3478005b40 tx=0x7f3478004740 comp rx=0 tx=0).stop 2024-02-16T22:12:28.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:28.059+0000 7f348d786700 1 -- 172.21.15.114:0/1512486370 shutdown_connections 2024-02-16T22:12:28.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:28.059+0000 7f348d786700 1 --2- 172.21.15.114:0/1512486370 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f34740383c0 0x7f347403a880 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:28.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:28.059+0000 7f348d786700 1 --2- 172.21.15.114:0/1512486370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34880ff790 0x7f3488195990 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:28.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:28.060+0000 7f348d786700 1 -- 172.21.15.114:0/1512486370 >> 172.21.15.114:0/1512486370 conn(0x7f34880fad30 msgr2=0x7f3488069120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:28.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:28.060+0000 7f348d786700 1 -- 172.21.15.114:0/1512486370 shutdown_connections 2024-02-16T22:12:28.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:28.060+0000 7f348d786700 1 -- 172.21.15.114:0/1512486370 wait complete. 2024-02-16T22:12:29.017 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-16T22:12:29.018 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph osd crush tunables default 2024-02-16T22:12:29.221 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:12:30.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:29 smithi114 ceph-mon[29235]: from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-16T22:12:30.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:29 smithi114 ceph-mon[29235]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:12:31.818 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:31 smithi114 ceph-mon[29235]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:12:31.818 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:31 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:32.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.073+0000 7fa3838f0700 1 -- 172.21.15.114:0/3918357645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa37c101ec0 msgr2=0x7fa37c1022e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:32.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.073+0000 7fa3838f0700 1 --2- 172.21.15.114:0/3918357645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa37c101ec0 0x7fa37c1022e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa378009910 tx=0x7fa378009c20 comp rx=0 tx=0).stop 2024-02-16T22:12:32.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.073+0000 7fa3838f0700 1 -- 172.21.15.114:0/3918357645 shutdown_connections 2024-02-16T22:12:32.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.073+0000 7fa3838f0700 1 --2- 172.21.15.114:0/3918357645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa37c101ec0 0x7fa37c1022e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:32.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.073+0000 7fa3838f0700 1 -- 172.21.15.114:0/3918357645 >> 172.21.15.114:0/3918357645 conn(0x7fa37c0fa4a0 msgr2=0x7fa37c0fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:32.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.073+0000 7fa3838f0700 1 -- 172.21.15.114:0/3918357645 shutdown_connections 2024-02-16T22:12:32.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.073+0000 7fa3838f0700 1 -- 172.21.15.114:0/3918357645 wait complete. 2024-02-16T22:12:32.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.074+0000 7fa3838f0700 1 Processor -- start 2024-02-16T22:12:32.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.074+0000 7fa3838f0700 1 -- start start 2024-02-16T22:12:32.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.074+0000 7fa3838f0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa37c101ec0 0x7fa37c1959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:32.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.074+0000 7fa3838f0700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c195ee0 con 0x7fa37c101ec0 2024-02-16T22:12:32.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.074+0000 7fa38168c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa37c101ec0 0x7fa37c1959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:32.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.074+0000 7fa38168c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa37c101ec0 0x7fa37c1959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59896/0 (socket says 172.21.15.114:59896) 2024-02-16T22:12:32.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.074+0000 7fa38168c700 1 -- 172.21.15.114:0/1676656563 learned_addr learned my addr 172.21.15.114:0/1676656563 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:12:32.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.074+0000 7fa38168c700 1 -- 172.21.15.114:0/1676656563 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3780095f0 con 0x7fa37c101ec0 2024-02-16T22:12:32.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.074+0000 7fa38168c700 1 --2- 172.21.15.114:0/1676656563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa37c101ec0 0x7fa37c1959a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa378005b40 tx=0x7fa378004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:32.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.075+0000 7fa3727fc700 1 -- 172.21.15.114:0/1676656563 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa37801d070 con 0x7fa37c101ec0 2024-02-16T22:12:32.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.075+0000 7fa3838f0700 1 -- 172.21.15.114:0/1676656563 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa37c1960e0 con 0x7fa37c101ec0 2024-02-16T22:12:32.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.075+0000 7fa3727fc700 1 -- 172.21.15.114:0/1676656563 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa37800be10 con 0x7fa37c101ec0 2024-02-16T22:12:32.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.075+0000 7fa3727fc700 1 -- 172.21.15.114:0/1676656563 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa37800f460 con 0x7fa37c101ec0 2024-02-16T22:12:32.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.075+0000 7fa3838f0700 1 -- 172.21.15.114:0/1676656563 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa37c196580 con 0x7fa37c101ec0 2024-02-16T22:12:32.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.075+0000 7fa3727fc700 1 -- 172.21.15.114:0/1676656563 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fa3780227e0 con 0x7fa37c101ec0 2024-02-16T22:12:32.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.076+0000 7fa3838f0700 1 -- 172.21.15.114:0/1676656563 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa37c18f4c0 con 0x7fa37c101ec0 2024-02-16T22:12:32.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.076+0000 7fa3727fc700 1 --2- 172.21.15.114:0/1676656563 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7fa3680383c0 0x7fa36803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:32.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.076+0000 7fa3727fc700 1 -- 172.21.15.114:0/1676656563 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa37804c3b0 con 0x7fa37c101ec0 2024-02-16T22:12:32.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.076+0000 7fa380e8b700 1 --2- 172.21.15.114:0/1676656563 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7fa3680383c0 0x7fa36803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:32.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.076+0000 7fa380e8b700 1 --2- 172.21.15.114:0/1676656563 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7fa3680383c0 0x7fa36803a880 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa36c006fd0 tx=0x7fa36c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:32.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.080+0000 7fa3727fc700 1 -- 172.21.15.114:0/1676656563 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa37802bba0 con 0x7fa37c101ec0 2024-02-16T22:12:32.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.277+0000 7fa3838f0700 1 -- 172.21.15.114:0/1676656563 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fa37c02cc70 con 0x7fa37c101ec0 2024-02-16T22:12:32.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.810+0000 7fa3727fc700 1 -- 172.21.15.114:0/1676656563 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fa378027070 con 0x7fa37c101ec0 2024-02-16T22:12:32.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.812+0000 7fa3838f0700 1 -- 172.21.15.114:0/1676656563 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7fa3680383c0 msgr2=0x7fa36803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:32.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.812+0000 7fa3838f0700 1 --2- 172.21.15.114:0/1676656563 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7fa3680383c0 0x7fa36803a880 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa36c006fd0 tx=0x7fa36c006e40 comp rx=0 tx=0).stop 2024-02-16T22:12:32.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.812+0000 7fa3838f0700 1 -- 172.21.15.114:0/1676656563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa37c101ec0 msgr2=0x7fa37c1959a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:32.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.812+0000 7fa3838f0700 1 --2- 172.21.15.114:0/1676656563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa37c101ec0 0x7fa37c1959a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa378005b40 tx=0x7fa378004740 comp rx=0 tx=0).stop 2024-02-16T22:12:32.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.812+0000 7fa3838f0700 1 -- 172.21.15.114:0/1676656563 shutdown_connections 2024-02-16T22:12:32.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.812+0000 7fa3838f0700 1 --2- 172.21.15.114:0/1676656563 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7fa3680383c0 0x7fa36803a880 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:32.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.812+0000 7fa3838f0700 1 --2- 172.21.15.114:0/1676656563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa37c101ec0 0x7fa37c1959a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:32.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.812+0000 7fa3838f0700 1 -- 172.21.15.114:0/1676656563 >> 172.21.15.114:0/1676656563 conn(0x7fa37c0fa4a0 msgr2=0x7fa37c0fb180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:32.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.812+0000 7fa3838f0700 1 -- 172.21.15.114:0/1676656563 shutdown_connections 2024-02-16T22:12:32.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:12:32.812+0000 7fa3838f0700 1 -- 172.21.15.114:0/1676656563 wait complete. 2024-02-16T22:12:32.815 INFO:teuthology.orchestra.run.smithi114.stderr:adjusted tunables profile to default 2024-02-16T22:12:32.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:32 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/1676656563' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-16T22:12:33.938 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:33 smithi114 ceph-mon[29235]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:12:33.938 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:33 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/1676656563' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-16T22:12:33.938 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:33 smithi114 ceph-mon[29235]: osdmap e4: 0 total, 0 up, 0 in 2024-02-16T22:12:34.709 INFO:tasks.cephadm:Adding mon.smithi114 on smithi114 2024-02-16T22:12:34.709 INFO:tasks.cephadm:Adding mon.smithi123 on smithi123 2024-02-16T22:12:34.709 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph orch apply mon '2;smithi114:172.21.15.114=smithi114;smithi123:172.21.15.123=smithi123' 2024-02-16T22:12:34.912 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:12:34.958 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:12:36.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:35 smithi114 ceph-mon[29235]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:12:36.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.506+0000 7f4fbb579700 1 -- 172.21.15.123:0/2749612480 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fb40ff790 msgr2=0x7f4fb40ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:36.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.506+0000 7f4fbb579700 1 --2- 172.21.15.123:0/2749612480 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fb40ff790 0x7f4fb40ffbb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009910 tx=0x7f4fa4009c20 comp rx=0 tx=0).stop 2024-02-16T22:12:36.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.507+0000 7f4fbb579700 1 -- 172.21.15.123:0/2749612480 shutdown_connections 2024-02-16T22:12:36.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.507+0000 7f4fbb579700 1 --2- 172.21.15.123:0/2749612480 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fb40ff790 0x7f4fb40ffbb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:36.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.507+0000 7f4fbb579700 1 -- 172.21.15.123:0/2749612480 >> 172.21.15.123:0/2749612480 conn(0x7f4fb40fad30 msgr2=0x7f4fb40fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:36.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.507+0000 7f4fbb579700 1 -- 172.21.15.123:0/2749612480 shutdown_connections 2024-02-16T22:12:36.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.507+0000 7f4fbb579700 1 -- 172.21.15.123:0/2749612480 wait complete. 2024-02-16T22:12:36.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.507+0000 7f4fbb579700 1 Processor -- start 2024-02-16T22:12:36.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.507+0000 7f4fbb579700 1 -- start start 2024-02-16T22:12:36.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.507+0000 7f4fbb579700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fb40ff790 0x7f4fb4069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:36.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.507+0000 7f4fbb579700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb41964a0 con 0x7f4fb40ff790 2024-02-16T22:12:36.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.507+0000 7f4fb9315700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fb40ff790 0x7f4fb4069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:36.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.507+0000 7f4fb9315700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fb40ff790 0x7f4fb4069430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.123:33428/0 (socket says 172.21.15.123:33428) 2024-02-16T22:12:36.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.508+0000 7f4fb9315700 1 -- 172.21.15.123:0/1181600935 learned_addr learned my addr 172.21.15.123:0/1181600935 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-02-16T22:12:36.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.508+0000 7f4fb9315700 1 -- 172.21.15.123:0/1181600935 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa40095f0 con 0x7f4fb40ff790 2024-02-16T22:12:36.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.508+0000 7f4fb9315700 1 --2- 172.21.15.123:0/1181600935 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fb40ff790 0x7f4fb4069430 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4005b40 tx=0x7f4fa4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:36.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.508+0000 7f4faa7fc700 1 -- 172.21.15.123:0/1181600935 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4fa401d070 con 0x7f4fb40ff790 2024-02-16T22:12:36.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.508+0000 7f4fbb579700 1 -- 172.21.15.123:0/1181600935 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb419aab0 con 0x7f4fb40ff790 2024-02-16T22:12:36.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.508+0000 7f4faa7fc700 1 -- 172.21.15.123:0/1181600935 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4fa400be10 con 0x7f4fb40ff790 2024-02-16T22:12:36.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.508+0000 7f4faa7fc700 1 -- 172.21.15.123:0/1181600935 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4fa400f460 con 0x7f4fb40ff790 2024-02-16T22:12:36.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.508+0000 7f4fbb579700 1 -- 172.21.15.123:0/1181600935 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb419ae90 con 0x7f4fb40ff790 2024-02-16T22:12:36.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.509+0000 7f4faa7fc700 1 -- 172.21.15.123:0/1181600935 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f4fa400f6d0 con 0x7f4fb40ff790 2024-02-16T22:12:36.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.509+0000 7f4faa7fc700 1 --2- 172.21.15.123:0/1181600935 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f4fa00383c0 0x7f4fa003a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:36.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.509+0000 7f4fbb579700 1 -- 172.21.15.123:0/1181600935 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fb418e3b0 con 0x7f4fb40ff790 2024-02-16T22:12:36.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.509+0000 7f4faa7fc700 1 -- 172.21.15.123:0/1181600935 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4fa404c610 con 0x7f4fb40ff790 2024-02-16T22:12:36.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.509+0000 7f4fb8b14700 1 --2- 172.21.15.123:0/1181600935 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f4fa00383c0 0x7f4fa003a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:36.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.510+0000 7f4fb8b14700 1 --2- 172.21.15.123:0/1181600935 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f4fa00383c0 0x7f4fa003a880 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0006fd0 tx=0x7f4fb0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:36.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.514+0000 7f4faa7fc700 1 -- 172.21.15.123:0/1181600935 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4fa402c440 con 0x7f4fb40ff790 2024-02-16T22:12:36.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.709+0000 7f4fbb579700 1 -- 172.21.15.123:0/1181600935 --> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi114:172.21.15.114=smithi114;smithi123:172.21.15.123=smithi123", "target": ["mon-mgr", ""]}) v1 -- 0x7f4fb4061340 con 0x7f4fa00383c0 2024-02-16T22:12:36.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.715+0000 7f4faa7fc700 1 -- 172.21.15.123:0/1181600935 <== mgr.14164 v2:172.21.15.114:6800/2648536185 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f4fb4061340 con 0x7f4fa00383c0 2024-02-16T22:12:36.716 INFO:teuthology.orchestra.run.smithi123.stdout:Scheduled mon update... 2024-02-16T22:12:36.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.717+0000 7f4fbb579700 1 -- 172.21.15.123:0/1181600935 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f4fa00383c0 msgr2=0x7f4fa003a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:36.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.717+0000 7f4fbb579700 1 --2- 172.21.15.123:0/1181600935 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f4fa00383c0 0x7f4fa003a880 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0006fd0 tx=0x7f4fb0006e40 comp rx=0 tx=0).stop 2024-02-16T22:12:36.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.717+0000 7f4fbb579700 1 -- 172.21.15.123:0/1181600935 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fb40ff790 msgr2=0x7f4fb4069430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:36.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.717+0000 7f4fbb579700 1 --2- 172.21.15.123:0/1181600935 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fb40ff790 0x7f4fb4069430 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4005b40 tx=0x7f4fa4004740 comp rx=0 tx=0).stop 2024-02-16T22:12:36.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.717+0000 7f4fbb579700 1 -- 172.21.15.123:0/1181600935 shutdown_connections 2024-02-16T22:12:36.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.717+0000 7f4fbb579700 1 --2- 172.21.15.123:0/1181600935 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f4fa00383c0 0x7f4fa003a880 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:36.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.717+0000 7f4fbb579700 1 --2- 172.21.15.123:0/1181600935 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fb40ff790 0x7f4fb4069430 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:36.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.717+0000 7f4fbb579700 1 -- 172.21.15.123:0/1181600935 >> 172.21.15.123:0/1181600935 conn(0x7f4fb40fad30 msgr2=0x7f4fb40fba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:36.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.717+0000 7f4fbb579700 1 -- 172.21.15.123:0/1181600935 shutdown_connections 2024-02-16T22:12:36.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:36.717+0000 7f4fbb579700 1 -- 172.21.15.123:0/1181600935 wait complete. 2024-02-16T22:12:37.327 DEBUG:teuthology.orchestra.run.smithi123:mon.smithi123> sudo journalctl -f -n 0 -u ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@mon.smithi123.service 2024-02-16T22:12:37.330 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:12:37.330 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:12:37.557 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:12:37.602 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:12:38.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:37 smithi114 ceph-mon[29235]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:12:38.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:37 smithi114 ceph-mon[29235]: from='client.14197 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi114:172.21.15.114=smithi114;smithi123:172.21.15.123=smithi123", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:12:38.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:37 smithi114 ceph-mon[29235]: Saving service mon spec with placement smithi114:172.21.15.114=smithi114;smithi123:172.21.15.123=smithi123;count:2 2024-02-16T22:12:38.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:37 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:38.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:37 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:38.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:37 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:38.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:37 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:38.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:37 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:38.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:37 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:38.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:37 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:38.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:37 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:38.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:37 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:38.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:37 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-16T22:12:38.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:37 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:38.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:38 smithi114 ceph-mon[29235]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-16T22:12:38.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:38 smithi114 ceph-mon[29235]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-16T22:12:38.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:38 smithi114 ceph-mon[29235]: Deploying daemon grafana.smithi114 on smithi114 2024-02-16T22:12:39.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.004+0000 7fa9244b3700 1 -- 172.21.15.123:0/2144175113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa91c0fef10 msgr2=0x7fa91c0ff330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:39.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.004+0000 7fa9244b3700 1 --2- 172.21.15.123:0/2144175113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa91c0fef10 0x7fa91c0ff330 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00aa20 tx=0x7fa90c00ad30 comp rx=0 tx=0).stop 2024-02-16T22:12:39.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.004+0000 7fa9244b3700 1 -- 172.21.15.123:0/2144175113 shutdown_connections 2024-02-16T22:12:39.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.004+0000 7fa9244b3700 1 --2- 172.21.15.123:0/2144175113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa91c0fef10 0x7fa91c0ff330 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:39.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.004+0000 7fa9244b3700 1 -- 172.21.15.123:0/2144175113 >> 172.21.15.123:0/2144175113 conn(0x7fa91c0fa4b0 msgr2=0x7fa91c0fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:39.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.004+0000 7fa9244b3700 1 -- 172.21.15.123:0/2144175113 shutdown_connections 2024-02-16T22:12:39.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.004+0000 7fa9244b3700 1 -- 172.21.15.123:0/2144175113 wait complete. 2024-02-16T22:12:39.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.005+0000 7fa9244b3700 1 Processor -- start 2024-02-16T22:12:39.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.005+0000 7fa9244b3700 1 -- start start 2024-02-16T22:12:39.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.005+0000 7fa9244b3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa91c0fef10 0x7fa91c1959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:39.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.005+0000 7fa9244b3700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c195f30 con 0x7fa91c0fef10 2024-02-16T22:12:39.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.005+0000 7fa92224f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa91c0fef10 0x7fa91c1959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:39.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.005+0000 7fa92224f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa91c0fef10 0x7fa91c1959f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.123:33444/0 (socket says 172.21.15.123:33444) 2024-02-16T22:12:39.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.005+0000 7fa92224f700 1 -- 172.21.15.123:0/3365761539 learned_addr learned my addr 172.21.15.123:0/3365761539 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-02-16T22:12:39.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.005+0000 7fa92224f700 1 -- 172.21.15.123:0/3365761539 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90c00a700 con 0x7fa91c0fef10 2024-02-16T22:12:39.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.006+0000 7fa92224f700 1 --2- 172.21.15.123:0/3365761539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa91c0fef10 0x7fa91c1959f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa90c005e00 tx=0x7fa90c005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:39.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.006+0000 7fa9137fe700 1 -- 172.21.15.123:0/3365761539 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa90c01d070 con 0x7fa91c0fef10 2024-02-16T22:12:39.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.006+0000 7fa9137fe700 1 -- 172.21.15.123:0/3365761539 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa90c005500 con 0x7fa91c0fef10 2024-02-16T22:12:39.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.006+0000 7fa9244b3700 1 -- 172.21.15.123:0/3365761539 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa91c196130 con 0x7fa91c0fef10 2024-02-16T22:12:39.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.006+0000 7fa9137fe700 1 -- 172.21.15.123:0/3365761539 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa90c00e740 con 0x7fa91c0fef10 2024-02-16T22:12:39.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.006+0000 7fa9244b3700 1 -- 172.21.15.123:0/3365761539 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91c1965d0 con 0x7fa91c0fef10 2024-02-16T22:12:39.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.007+0000 7fa9137fe700 1 -- 172.21.15.123:0/3365761539 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fa90c016470 con 0x7fa91c0fef10 2024-02-16T22:12:39.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.007+0000 7fa9244b3700 1 -- 172.21.15.123:0/3365761539 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa91c18d210 con 0x7fa91c0fef10 2024-02-16T22:12:39.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.007+0000 7fa9137fe700 1 --2- 172.21.15.123:0/3365761539 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7fa9080383c0 0x7fa90803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:39.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.007+0000 7fa9137fe700 1 -- 172.21.15.123:0/3365761539 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa90c012070 con 0x7fa91c0fef10 2024-02-16T22:12:39.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.007+0000 7fa921a4e700 1 --2- 172.21.15.123:0/3365761539 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7fa9080383c0 0x7fa90803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:39.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.007+0000 7fa921a4e700 1 --2- 172.21.15.123:0/3365761539 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7fa9080383c0 0x7fa90803a880 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa918006fd0 tx=0x7fa918006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:39.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.013+0000 7fa9137fe700 1 -- 172.21.15.123:0/3365761539 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa90c016b90 con 0x7fa91c0fef10 2024-02-16T22:12:39.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.275+0000 7fa9244b3700 1 -- 172.21.15.123:0/3365761539 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa91c02cc70 con 0x7fa91c0fef10 2024-02-16T22:12:39.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.275+0000 7fa9137fe700 1 -- 172.21.15.123:0/3365761539 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fa90c02a3a0 con 0x7fa91c0fef10 2024-02-16T22:12:39.280 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:12:39.280 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":1,"fsid":"1dcbaabe-cd18-11ee-95ba-87774f69a715","modified":"2024-02-16T22:10:59.330677Z","created":"2024-02-16T22:10:59.330677Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi114","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:3300","nonce":0},{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:12:39.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.277+0000 7fa9244b3700 1 -- 172.21.15.123:0/3365761539 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7fa9080383c0 msgr2=0x7fa90803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:39.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.277+0000 7fa9244b3700 1 --2- 172.21.15.123:0/3365761539 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7fa9080383c0 0x7fa90803a880 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa918006fd0 tx=0x7fa918006e40 comp rx=0 tx=0).stop 2024-02-16T22:12:39.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.277+0000 7fa9244b3700 1 -- 172.21.15.123:0/3365761539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa91c0fef10 msgr2=0x7fa91c1959f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:39.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.277+0000 7fa9244b3700 1 --2- 172.21.15.123:0/3365761539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa91c0fef10 0x7fa91c1959f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa90c005e00 tx=0x7fa90c005000 comp rx=0 tx=0).stop 2024-02-16T22:12:39.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.277+0000 7fa9244b3700 1 -- 172.21.15.123:0/3365761539 shutdown_connections 2024-02-16T22:12:39.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.277+0000 7fa9244b3700 1 --2- 172.21.15.123:0/3365761539 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7fa9080383c0 0x7fa90803a880 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:39.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.277+0000 7fa9244b3700 1 --2- 172.21.15.123:0/3365761539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa91c0fef10 0x7fa91c1959f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:39.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.277+0000 7fa9244b3700 1 -- 172.21.15.123:0/3365761539 >> 172.21.15.123:0/3365761539 conn(0x7fa91c0fa4b0 msgr2=0x7fa91c0fb170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:39.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.277+0000 7fa9244b3700 1 -- 172.21.15.123:0/3365761539 shutdown_connections 2024-02-16T22:12:39.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:39.277+0000 7fa9244b3700 1 -- 172.21.15.123:0/3365761539 wait complete. 2024-02-16T22:12:39.282 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 1 2024-02-16T22:12:40.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:39 smithi114 ceph-mon[29235]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:12:40.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:39 smithi114 ceph-mon[29235]: from='client.? 172.21.15.123:0/3365761539' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:12:40.704 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:12:40.705 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:12:40.906 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:12:40.952 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:12:41.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:41 smithi114 ceph-mon[29235]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:12:41.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:41 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:42.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.383+0000 7f9397e50700 1 -- 172.21.15.123:0/3756420384 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93901008b0 msgr2=0x7f9390100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:42.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.383+0000 7f9397e50700 1 --2- 172.21.15.123:0/3756420384 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93901008b0 0x7f9390100cd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f938c009910 tx=0x7f938c009c20 comp rx=0 tx=0).stop 2024-02-16T22:12:42.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.383+0000 7f9397e50700 1 -- 172.21.15.123:0/3756420384 shutdown_connections 2024-02-16T22:12:42.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.383+0000 7f9397e50700 1 --2- 172.21.15.123:0/3756420384 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93901008b0 0x7f9390100cd0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:42.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.383+0000 7f9397e50700 1 -- 172.21.15.123:0/3756420384 >> 172.21.15.123:0/3756420384 conn(0x7f93900fbe30 msgr2=0x7f93900fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:42.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.383+0000 7f9397e50700 1 -- 172.21.15.123:0/3756420384 shutdown_connections 2024-02-16T22:12:42.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.383+0000 7f9397e50700 1 -- 172.21.15.123:0/3756420384 wait complete. 2024-02-16T22:12:42.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.384+0000 7f9397e50700 1 Processor -- start 2024-02-16T22:12:42.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.384+0000 7f9397e50700 1 -- start start 2024-02-16T22:12:42.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.384+0000 7f9397e50700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93901008b0 0x7f93901959d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:42.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.384+0000 7f9397e50700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9390195f10 con 0x7f93901008b0 2024-02-16T22:12:42.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.384+0000 7f9395bec700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93901008b0 0x7f93901959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:42.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.384+0000 7f9395bec700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93901008b0 0x7f93901959d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.123:33456/0 (socket says 172.21.15.123:33456) 2024-02-16T22:12:42.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.384+0000 7f9395bec700 1 -- 172.21.15.123:0/2616130309 learned_addr learned my addr 172.21.15.123:0/2616130309 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-02-16T22:12:42.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.384+0000 7f9395bec700 1 -- 172.21.15.123:0/2616130309 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f938c0095f0 con 0x7f93901008b0 2024-02-16T22:12:42.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.385+0000 7f9395bec700 1 --2- 172.21.15.123:0/2616130309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93901008b0 0x7f93901959d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f938c005b40 tx=0x7f938c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:42.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.385+0000 7f9386ffd700 1 -- 172.21.15.123:0/2616130309 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f938c01d070 con 0x7f93901008b0 2024-02-16T22:12:42.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.385+0000 7f9386ffd700 1 -- 172.21.15.123:0/2616130309 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f938c00be10 con 0x7f93901008b0 2024-02-16T22:12:42.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.385+0000 7f9397e50700 1 -- 172.21.15.123:0/2616130309 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9390196110 con 0x7f93901008b0 2024-02-16T22:12:42.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.385+0000 7f9386ffd700 1 -- 172.21.15.123:0/2616130309 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f938c00f460 con 0x7f93901008b0 2024-02-16T22:12:42.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.385+0000 7f9397e50700 1 -- 172.21.15.123:0/2616130309 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93901965b0 con 0x7f93901008b0 2024-02-16T22:12:42.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.386+0000 7f9386ffd700 1 -- 172.21.15.123:0/2616130309 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f938c00f6d0 con 0x7f93901008b0 2024-02-16T22:12:42.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.386+0000 7f9397e50700 1 -- 172.21.15.123:0/2616130309 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f939018f4d0 con 0x7f93901008b0 2024-02-16T22:12:42.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.386+0000 7f9386ffd700 1 --2- 172.21.15.123:0/2616130309 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f937c0383c0 0x7f937c03a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:42.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.386+0000 7f9386ffd700 1 -- 172.21.15.123:0/2616130309 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f938c04c730 con 0x7f93901008b0 2024-02-16T22:12:42.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.386+0000 7f93953eb700 1 --2- 172.21.15.123:0/2616130309 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f937c0383c0 0x7f937c03a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:42.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.386+0000 7f93953eb700 1 --2- 172.21.15.123:0/2616130309 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f937c0383c0 0x7f937c03a880 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9380006fd0 tx=0x7f9380006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:42.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.391+0000 7f9386ffd700 1 -- 172.21.15.123:0/2616130309 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f938c02a360 con 0x7f93901008b0 2024-02-16T22:12:42.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.652+0000 7f9397e50700 1 -- 172.21.15.123:0/2616130309 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f939002cc70 con 0x7f93901008b0 2024-02-16T22:12:42.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.653+0000 7f9386ffd700 1 -- 172.21.15.123:0/2616130309 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f938c027030 con 0x7f93901008b0 2024-02-16T22:12:42.655 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:12:42.655 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":1,"fsid":"1dcbaabe-cd18-11ee-95ba-87774f69a715","modified":"2024-02-16T22:10:59.330677Z","created":"2024-02-16T22:10:59.330677Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi114","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:3300","nonce":0},{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:12:42.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.655+0000 7f9397e50700 1 -- 172.21.15.123:0/2616130309 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f937c0383c0 msgr2=0x7f937c03a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:42.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.655+0000 7f9397e50700 1 --2- 172.21.15.123:0/2616130309 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f937c0383c0 0x7f937c03a880 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9380006fd0 tx=0x7f9380006e40 comp rx=0 tx=0).stop 2024-02-16T22:12:42.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.655+0000 7f9397e50700 1 -- 172.21.15.123:0/2616130309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93901008b0 msgr2=0x7f93901959d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:42.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.655+0000 7f9397e50700 1 --2- 172.21.15.123:0/2616130309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93901008b0 0x7f93901959d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f938c005b40 tx=0x7f938c004740 comp rx=0 tx=0).stop 2024-02-16T22:12:42.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.655+0000 7f9397e50700 1 -- 172.21.15.123:0/2616130309 shutdown_connections 2024-02-16T22:12:42.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.655+0000 7f9397e50700 1 --2- 172.21.15.123:0/2616130309 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f937c0383c0 0x7f937c03a880 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:42.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.655+0000 7f9397e50700 1 --2- 172.21.15.123:0/2616130309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93901008b0 0x7f93901959d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:42.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.655+0000 7f9397e50700 1 -- 172.21.15.123:0/2616130309 >> 172.21.15.123:0/2616130309 conn(0x7f93900fbe30 msgr2=0x7f93900fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:42.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.655+0000 7f9397e50700 1 -- 172.21.15.123:0/2616130309 shutdown_connections 2024-02-16T22:12:42.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:42.655+0000 7f9397e50700 1 -- 172.21.15.123:0/2616130309 wait complete. 2024-02-16T22:12:42.657 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 1 2024-02-16T22:12:43.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:42 smithi114 ceph-mon[29235]: from='client.? 172.21.15.123:0/2616130309' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:12:43.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:43 smithi114 ceph-mon[29235]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:12:44.294 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:12:44.295 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:12:44.496 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:12:44.541 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:12:45.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:45.952+0000 7fdb1a820700 1 -- 172.21.15.123:0/858271384 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb140ff7a0 msgr2=0x7fdb140ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:45.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:45.952+0000 7fdb1a820700 1 --2- 172.21.15.123:0/858271384 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb140ff7a0 0x7fdb140ffbc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdb08009910 tx=0x7fdb08009c20 comp rx=0 tx=0).stop 2024-02-16T22:12:45.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:45.952+0000 7fdb1a820700 1 -- 172.21.15.123:0/858271384 shutdown_connections 2024-02-16T22:12:45.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:45.952+0000 7fdb1a820700 1 --2- 172.21.15.123:0/858271384 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb140ff7a0 0x7fdb140ffbc0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:45.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:45.952+0000 7fdb1a820700 1 -- 172.21.15.123:0/858271384 >> 172.21.15.123:0/858271384 conn(0x7fdb140fad20 msgr2=0x7fdb140fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:45.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:45.952+0000 7fdb1a820700 1 -- 172.21.15.123:0/858271384 shutdown_connections 2024-02-16T22:12:45.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:45.952+0000 7fdb1a820700 1 -- 172.21.15.123:0/858271384 wait complete. 2024-02-16T22:12:45.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:45.953+0000 7fdb1a820700 1 Processor -- start 2024-02-16T22:12:45.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:45.953+0000 7fdb1a820700 1 -- start start 2024-02-16T22:12:45.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:45.953+0000 7fdb1a820700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb140ff7a0 0x7fdb14068b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:45.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:45.953+0000 7fdb1a820700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb141986f0 con 0x7fdb140ff7a0 2024-02-16T22:12:45.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:45.953+0000 7fdb13fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb140ff7a0 0x7fdb14068b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:45.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:45.953+0000 7fdb13fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb140ff7a0 0x7fdb14068b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.123:44042/0 (socket says 172.21.15.123:44042) 2024-02-16T22:12:45.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:45.953+0000 7fdb13fff700 1 -- 172.21.15.123:0/4042620286 learned_addr learned my addr 172.21.15.123:0/4042620286 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-02-16T22:12:45.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:45.953+0000 7fdb13fff700 1 -- 172.21.15.123:0/4042620286 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb080095f0 con 0x7fdb140ff7a0 2024-02-16T22:12:45.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:45.953+0000 7fdb13fff700 1 --2- 172.21.15.123:0/4042620286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb140ff7a0 0x7fdb14068b00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdb08005b40 tx=0x7fdb08004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:45.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:45.954+0000 7fdb117fa700 1 -- 172.21.15.123:0/4042620286 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb0801d070 con 0x7fdb140ff7a0 2024-02-16T22:12:45.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:45.954+0000 7fdb117fa700 1 -- 172.21.15.123:0/4042620286 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdb0800be10 con 0x7fdb140ff7a0 2024-02-16T22:12:45.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:45.954+0000 7fdb1a820700 1 -- 172.21.15.123:0/4042620286 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb14198830 con 0x7fdb140ff7a0 2024-02-16T22:12:45.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:45.954+0000 7fdb117fa700 1 -- 172.21.15.123:0/4042620286 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb0800f460 con 0x7fdb140ff7a0 2024-02-16T22:12:45.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:45.954+0000 7fdb1a820700 1 -- 172.21.15.123:0/4042620286 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb14198cd0 con 0x7fdb140ff7a0 2024-02-16T22:12:45.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:45.955+0000 7fdb117fa700 1 -- 172.21.15.123:0/4042620286 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fdb080227e0 con 0x7fdb140ff7a0 2024-02-16T22:12:45.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:45.955+0000 7fdb1a820700 1 -- 172.21.15.123:0/4042620286 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb1418e3c0 con 0x7fdb140ff7a0 2024-02-16T22:12:45.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:45.955+0000 7fdb117fa700 1 --2- 172.21.15.123:0/4042620286 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7fdafc0383c0 0x7fdafc03a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:45.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:45.955+0000 7fdb117fa700 1 -- 172.21.15.123:0/4042620286 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdb0804c3b0 con 0x7fdb140ff7a0 2024-02-16T22:12:45.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:45.955+0000 7fdb137fe700 1 --2- 172.21.15.123:0/4042620286 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7fdafc0383c0 0x7fdafc03a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:45.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:45.955+0000 7fdb137fe700 1 --2- 172.21.15.123:0/4042620286 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7fdafc0383c0 0x7fdafc03a880 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdb04006fd0 tx=0x7fdb04006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:45.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:45.960+0000 7fdb117fa700 1 -- 172.21.15.123:0/4042620286 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdb0802bba0 con 0x7fdb140ff7a0 2024-02-16T22:12:46.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:45 smithi114 ceph-mon[29235]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:12:46.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:46.225+0000 7fdb1a820700 1 -- 172.21.15.123:0/4042620286 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fdb1402cc70 con 0x7fdb140ff7a0 2024-02-16T22:12:46.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:46.226+0000 7fdb117fa700 1 -- 172.21.15.123:0/4042620286 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fdb08027020 con 0x7fdb140ff7a0 2024-02-16T22:12:46.228 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:12:46.228 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":1,"fsid":"1dcbaabe-cd18-11ee-95ba-87774f69a715","modified":"2024-02-16T22:10:59.330677Z","created":"2024-02-16T22:10:59.330677Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi114","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:3300","nonce":0},{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:12:46.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:46.228+0000 7fdb1a820700 1 -- 172.21.15.123:0/4042620286 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7fdafc0383c0 msgr2=0x7fdafc03a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:46.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:46.228+0000 7fdb1a820700 1 --2- 172.21.15.123:0/4042620286 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7fdafc0383c0 0x7fdafc03a880 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdb04006fd0 tx=0x7fdb04006e40 comp rx=0 tx=0).stop 2024-02-16T22:12:46.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:46.228+0000 7fdb1a820700 1 -- 172.21.15.123:0/4042620286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb140ff7a0 msgr2=0x7fdb14068b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:46.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:46.228+0000 7fdb1a820700 1 --2- 172.21.15.123:0/4042620286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb140ff7a0 0x7fdb14068b00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdb08005b40 tx=0x7fdb08004740 comp rx=0 tx=0).stop 2024-02-16T22:12:46.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:46.228+0000 7fdb1a820700 1 -- 172.21.15.123:0/4042620286 shutdown_connections 2024-02-16T22:12:46.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:46.228+0000 7fdb1a820700 1 --2- 172.21.15.123:0/4042620286 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7fdafc0383c0 0x7fdafc03a880 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:46.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:46.228+0000 7fdb1a820700 1 --2- 172.21.15.123:0/4042620286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb140ff7a0 0x7fdb14068b00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:46.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:46.228+0000 7fdb1a820700 1 -- 172.21.15.123:0/4042620286 >> 172.21.15.123:0/4042620286 conn(0x7fdb140fad20 msgr2=0x7fdb140fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:46.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:46.228+0000 7fdb1a820700 1 -- 172.21.15.123:0/4042620286 shutdown_connections 2024-02-16T22:12:46.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:46.228+0000 7fdb1a820700 1 -- 172.21.15.123:0/4042620286 wait complete. 2024-02-16T22:12:46.231 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 1 2024-02-16T22:12:47.074 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:46 smithi114 ceph-mon[29235]: from='client.? 172.21.15.123:0/4042620286' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:12:47.806 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:12:47.806 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:12:48.022 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:12:48.068 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:12:48.092 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:47 smithi114 ceph-mon[29235]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:12:49.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.508+0000 7f7802126700 1 -- 172.21.15.123:0/500688445 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77fc0ff7a0 msgr2=0x7f77fc0ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:49.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.508+0000 7f7802126700 1 --2- 172.21.15.123:0/500688445 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77fc0ff7a0 0x7f77fc0ffbc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f77f0009910 tx=0x7f77f0009c20 comp rx=0 tx=0).stop 2024-02-16T22:12:49.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.508+0000 7f7802126700 1 -- 172.21.15.123:0/500688445 shutdown_connections 2024-02-16T22:12:49.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.508+0000 7f7802126700 1 --2- 172.21.15.123:0/500688445 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77fc0ff7a0 0x7f77fc0ffbc0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:49.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.508+0000 7f7802126700 1 -- 172.21.15.123:0/500688445 >> 172.21.15.123:0/500688445 conn(0x7f77fc0fad20 msgr2=0x7f77fc0fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:49.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.508+0000 7f7802126700 1 -- 172.21.15.123:0/500688445 shutdown_connections 2024-02-16T22:12:49.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.508+0000 7f7802126700 1 -- 172.21.15.123:0/500688445 wait complete. 2024-02-16T22:12:49.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.509+0000 7f7802126700 1 Processor -- start 2024-02-16T22:12:49.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.509+0000 7f7802126700 1 -- start start 2024-02-16T22:12:49.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.509+0000 7f7802126700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77fc0ff7a0 0x7f77fc1959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:49.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.509+0000 7f7802126700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77fc195f00 con 0x7f77fc0ff7a0 2024-02-16T22:12:49.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.510+0000 7f77fb7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77fc0ff7a0 0x7f77fc1959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:49.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.510+0000 7f77fb7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77fc0ff7a0 0x7f77fc1959c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.123:44052/0 (socket says 172.21.15.123:44052) 2024-02-16T22:12:49.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.510+0000 7f77fb7fe700 1 -- 172.21.15.123:0/2678990055 learned_addr learned my addr 172.21.15.123:0/2678990055 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-02-16T22:12:49.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.510+0000 7f77fb7fe700 1 -- 172.21.15.123:0/2678990055 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f00095f0 con 0x7f77fc0ff7a0 2024-02-16T22:12:49.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.510+0000 7f77fb7fe700 1 --2- 172.21.15.123:0/2678990055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77fc0ff7a0 0x7f77fc1959c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f77f0005b40 tx=0x7f77f0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:49.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.510+0000 7f77f8ff9700 1 -- 172.21.15.123:0/2678990055 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f77f001d070 con 0x7f77fc0ff7a0 2024-02-16T22:12:49.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.510+0000 7f77f8ff9700 1 -- 172.21.15.123:0/2678990055 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f77f000be10 con 0x7f77fc0ff7a0 2024-02-16T22:12:49.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.510+0000 7f7802126700 1 -- 172.21.15.123:0/2678990055 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77fc196100 con 0x7f77fc0ff7a0 2024-02-16T22:12:49.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.510+0000 7f77f8ff9700 1 -- 172.21.15.123:0/2678990055 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f77f000f460 con 0x7f77fc0ff7a0 2024-02-16T22:12:49.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.510+0000 7f7802126700 1 -- 172.21.15.123:0/2678990055 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77fc1965a0 con 0x7f77fc0ff7a0 2024-02-16T22:12:49.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.511+0000 7f77f8ff9700 1 -- 172.21.15.123:0/2678990055 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f77f000b960 con 0x7f77fc0ff7a0 2024-02-16T22:12:49.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.511+0000 7f7802126700 1 -- 172.21.15.123:0/2678990055 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77fc18f3d0 con 0x7f77fc0ff7a0 2024-02-16T22:12:49.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.511+0000 7f77f8ff9700 1 --2- 172.21.15.123:0/2678990055 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f77e40383c0 0x7f77e403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:49.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.511+0000 7f77f8ff9700 1 -- 172.21.15.123:0/2678990055 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f77f004c4b0 con 0x7f77fc0ff7a0 2024-02-16T22:12:49.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.511+0000 7f77faffd700 1 --2- 172.21.15.123:0/2678990055 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f77e40383c0 0x7f77e403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:49.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.512+0000 7f77faffd700 1 --2- 172.21.15.123:0/2678990055 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f77e40383c0 0x7f77e403a880 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f77ec006fd0 tx=0x7f77ec006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:49.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.516+0000 7f77f8ff9700 1 -- 172.21.15.123:0/2678990055 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f77f002a360 con 0x7f77fc0ff7a0 2024-02-16T22:12:49.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.778+0000 7f7802126700 1 -- 172.21.15.123:0/2678990055 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f77fc02cc70 con 0x7f77fc0ff7a0 2024-02-16T22:12:49.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.779+0000 7f77f8ff9700 1 -- 172.21.15.123:0/2678990055 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f77f0027030 con 0x7f77fc0ff7a0 2024-02-16T22:12:49.781 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:12:49.781 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":1,"fsid":"1dcbaabe-cd18-11ee-95ba-87774f69a715","modified":"2024-02-16T22:10:59.330677Z","created":"2024-02-16T22:10:59.330677Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi114","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:3300","nonce":0},{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:12:49.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.781+0000 7f7802126700 1 -- 172.21.15.123:0/2678990055 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f77e40383c0 msgr2=0x7f77e403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:49.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.781+0000 7f7802126700 1 --2- 172.21.15.123:0/2678990055 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f77e40383c0 0x7f77e403a880 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f77ec006fd0 tx=0x7f77ec006e40 comp rx=0 tx=0).stop 2024-02-16T22:12:49.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.781+0000 7f7802126700 1 -- 172.21.15.123:0/2678990055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77fc0ff7a0 msgr2=0x7f77fc1959c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:49.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.781+0000 7f7802126700 1 --2- 172.21.15.123:0/2678990055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77fc0ff7a0 0x7f77fc1959c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f77f0005b40 tx=0x7f77f0004740 comp rx=0 tx=0).stop 2024-02-16T22:12:49.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.781+0000 7f7802126700 1 -- 172.21.15.123:0/2678990055 shutdown_connections 2024-02-16T22:12:49.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.781+0000 7f7802126700 1 --2- 172.21.15.123:0/2678990055 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f77e40383c0 0x7f77e403a880 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:49.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.781+0000 7f7802126700 1 --2- 172.21.15.123:0/2678990055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77fc0ff7a0 0x7f77fc1959c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:49.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.781+0000 7f7802126700 1 -- 172.21.15.123:0/2678990055 >> 172.21.15.123:0/2678990055 conn(0x7f77fc0fad20 msgr2=0x7f77fc069100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:49.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.781+0000 7f7802126700 1 -- 172.21.15.123:0/2678990055 shutdown_connections 2024-02-16T22:12:49.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:49.781+0000 7f7802126700 1 -- 172.21.15.123:0/2678990055 wait complete. 2024-02-16T22:12:49.784 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 1 2024-02-16T22:12:50.115 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:49 smithi114 ceph-mon[29235]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:12:50.115 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:49 smithi114 ceph-mon[29235]: from='client.? 172.21.15.123:0/2678990055' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:12:51.427 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:12:51.427 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:12:51.628 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:12:51.674 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:12:52.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:51 smithi114 ceph-mon[29235]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:12:52.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:51 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:52.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:51 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:52.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:51 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:52.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:51 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:52.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:51 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:52.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:51 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:52.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:51 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:52.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:51 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:53.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:52.999+0000 7f718b32f700 1 -- 172.21.15.123:0/727791393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71841008b0 msgr2=0x7f7184100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:53.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:52.999+0000 7f718b32f700 1 --2- 172.21.15.123:0/727791393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71841008b0 0x7f7184100cd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7180009910 tx=0x7f7180009c20 comp rx=0 tx=0).stop 2024-02-16T22:12:53.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.000+0000 7f718b32f700 1 -- 172.21.15.123:0/727791393 shutdown_connections 2024-02-16T22:12:53.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.000+0000 7f718b32f700 1 --2- 172.21.15.123:0/727791393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71841008b0 0x7f7184100cd0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:53.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.000+0000 7f718b32f700 1 -- 172.21.15.123:0/727791393 >> 172.21.15.123:0/727791393 conn(0x7f71840fbe30 msgr2=0x7f71840fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:53.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.000+0000 7f718b32f700 1 -- 172.21.15.123:0/727791393 shutdown_connections 2024-02-16T22:12:53.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.000+0000 7f718b32f700 1 -- 172.21.15.123:0/727791393 wait complete. 2024-02-16T22:12:53.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.000+0000 7f718b32f700 1 Processor -- start 2024-02-16T22:12:53.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.000+0000 7f718b32f700 1 -- start start 2024-02-16T22:12:53.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.000+0000 7f718b32f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71841008b0 0x7f71841959d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:53.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.000+0000 7f718b32f700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7184195f10 con 0x7f71841008b0 2024-02-16T22:12:53.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.000+0000 7f71890cb700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71841008b0 0x7f71841959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:53.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.000+0000 7f71890cb700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71841008b0 0x7f71841959d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.123:42800/0 (socket says 172.21.15.123:42800) 2024-02-16T22:12:53.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.000+0000 7f71890cb700 1 -- 172.21.15.123:0/3145859798 learned_addr learned my addr 172.21.15.123:0/3145859798 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-02-16T22:12:53.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.001+0000 7f71890cb700 1 -- 172.21.15.123:0/3145859798 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71800095f0 con 0x7f71841008b0 2024-02-16T22:12:53.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.001+0000 7f71890cb700 1 --2- 172.21.15.123:0/3145859798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71841008b0 0x7f71841959d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7180005b40 tx=0x7f7180004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:53.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.001+0000 7f717a7fc700 1 -- 172.21.15.123:0/3145859798 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f718001d070 con 0x7f71841008b0 2024-02-16T22:12:53.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.001+0000 7f717a7fc700 1 -- 172.21.15.123:0/3145859798 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f718000be10 con 0x7f71841008b0 2024-02-16T22:12:53.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.001+0000 7f718b32f700 1 -- 172.21.15.123:0/3145859798 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7184196110 con 0x7f71841008b0 2024-02-16T22:12:53.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.001+0000 7f717a7fc700 1 -- 172.21.15.123:0/3145859798 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f718000f460 con 0x7f71841008b0 2024-02-16T22:12:53.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.001+0000 7f718b32f700 1 -- 172.21.15.123:0/3145859798 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71841965b0 con 0x7f71841008b0 2024-02-16T22:12:53.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.002+0000 7f717a7fc700 1 -- 172.21.15.123:0/3145859798 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f718000f6d0 con 0x7f71841008b0 2024-02-16T22:12:53.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.002+0000 7f718b32f700 1 -- 172.21.15.123:0/3145859798 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f718418f4d0 con 0x7f71841008b0 2024-02-16T22:12:53.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.002+0000 7f717a7fc700 1 --2- 172.21.15.123:0/3145859798 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f71700383c0 0x7f717003a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:53.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.002+0000 7f717a7fc700 1 -- 172.21.15.123:0/3145859798 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f718004c470 con 0x7f71841008b0 2024-02-16T22:12:53.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.002+0000 7f71888ca700 1 --2- 172.21.15.123:0/3145859798 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f71700383c0 0x7f717003a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:53.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.003+0000 7f71888ca700 1 --2- 172.21.15.123:0/3145859798 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f71700383c0 0x7f717003a880 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7174006fd0 tx=0x7f7174006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:53.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.007+0000 7f717a7fc700 1 -- 172.21.15.123:0/3145859798 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f718002ba00 con 0x7f71841008b0 2024-02-16T22:12:53.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:52 smithi114 ceph-mon[29235]: Deploying daemon prometheus.smithi114 on smithi114 2024-02-16T22:12:53.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.267+0000 7f718b32f700 1 -- 172.21.15.123:0/3145859798 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f718402cc70 con 0x7f71841008b0 2024-02-16T22:12:53.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.267+0000 7f717a7fc700 1 -- 172.21.15.123:0/3145859798 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f7180027020 con 0x7f71841008b0 2024-02-16T22:12:53.269 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:12:53.269 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":1,"fsid":"1dcbaabe-cd18-11ee-95ba-87774f69a715","modified":"2024-02-16T22:10:59.330677Z","created":"2024-02-16T22:10:59.330677Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi114","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:3300","nonce":0},{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:12:53.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.270+0000 7f718b32f700 1 -- 172.21.15.123:0/3145859798 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f71700383c0 msgr2=0x7f717003a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:53.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.270+0000 7f718b32f700 1 --2- 172.21.15.123:0/3145859798 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f71700383c0 0x7f717003a880 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7174006fd0 tx=0x7f7174006e40 comp rx=0 tx=0).stop 2024-02-16T22:12:53.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.270+0000 7f718b32f700 1 -- 172.21.15.123:0/3145859798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71841008b0 msgr2=0x7f71841959d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:53.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.270+0000 7f718b32f700 1 --2- 172.21.15.123:0/3145859798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71841008b0 0x7f71841959d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7180005b40 tx=0x7f7180004740 comp rx=0 tx=0).stop 2024-02-16T22:12:53.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.270+0000 7f718b32f700 1 -- 172.21.15.123:0/3145859798 shutdown_connections 2024-02-16T22:12:53.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.270+0000 7f718b32f700 1 --2- 172.21.15.123:0/3145859798 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f71700383c0 0x7f717003a880 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:53.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.270+0000 7f718b32f700 1 --2- 172.21.15.123:0/3145859798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71841008b0 0x7f71841959d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:53.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.270+0000 7f718b32f700 1 -- 172.21.15.123:0/3145859798 >> 172.21.15.123:0/3145859798 conn(0x7f71840fbe30 msgr2=0x7f71840fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:53.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.270+0000 7f718b32f700 1 -- 172.21.15.123:0/3145859798 shutdown_connections 2024-02-16T22:12:53.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:53.270+0000 7f718b32f700 1 -- 172.21.15.123:0/3145859798 wait complete. 2024-02-16T22:12:53.273 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 1 2024-02-16T22:12:53.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:53 smithi114 ceph-mon[29235]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:12:53.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:53 smithi114 ceph-mon[29235]: from='client.? 172.21.15.123:0/3145859798' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:12:54.787 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:12:54.788 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:12:54.987 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:12:55.033 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:12:56.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:55 smithi114 ceph-mon[29235]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:12:56.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:55 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:12:56.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.512+0000 7f73fe6c0700 1 -- 172.21.15.123:0/3221513835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73f81008a0 msgr2=0x7f73f8100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:56.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.512+0000 7f73fe6c0700 1 --2- 172.21.15.123:0/3221513835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73f81008a0 0x7f73f8100cc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f73e8009910 tx=0x7f73e8009c20 comp rx=0 tx=0).stop 2024-02-16T22:12:56.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.512+0000 7f73fe6c0700 1 -- 172.21.15.123:0/3221513835 shutdown_connections 2024-02-16T22:12:56.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.512+0000 7f73fe6c0700 1 --2- 172.21.15.123:0/3221513835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73f81008a0 0x7f73f8100cc0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:56.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.512+0000 7f73fe6c0700 1 -- 172.21.15.123:0/3221513835 >> 172.21.15.123:0/3221513835 conn(0x7f73f80fbe60 msgr2=0x7f73f80fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:56.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.513+0000 7f73fe6c0700 1 -- 172.21.15.123:0/3221513835 shutdown_connections 2024-02-16T22:12:56.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.513+0000 7f73fe6c0700 1 -- 172.21.15.123:0/3221513835 wait complete. 2024-02-16T22:12:56.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.513+0000 7f73fe6c0700 1 Processor -- start 2024-02-16T22:12:56.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.513+0000 7f73fe6c0700 1 -- start start 2024-02-16T22:12:56.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.513+0000 7f73fe6c0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73f81008a0 0x7f73f81959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:56.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.513+0000 7f73fe6c0700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f8195f00 con 0x7f73f81008a0 2024-02-16T22:12:56.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.514+0000 7f73f7fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73f81008a0 0x7f73f81959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:56.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.514+0000 7f73f7fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73f81008a0 0x7f73f81959c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.123:42810/0 (socket says 172.21.15.123:42810) 2024-02-16T22:12:56.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.514+0000 7f73f7fff700 1 -- 172.21.15.123:0/4260856983 learned_addr learned my addr 172.21.15.123:0/4260856983 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-02-16T22:12:56.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.514+0000 7f73f7fff700 1 -- 172.21.15.123:0/4260856983 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e80095f0 con 0x7f73f81008a0 2024-02-16T22:12:56.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.514+0000 7f73f7fff700 1 --2- 172.21.15.123:0/4260856983 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73f81008a0 0x7f73f81959c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f73e8005b40 tx=0x7f73e8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:56.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.514+0000 7f73f57fa700 1 -- 172.21.15.123:0/4260856983 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f73e801d070 con 0x7f73f81008a0 2024-02-16T22:12:56.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.514+0000 7f73f57fa700 1 -- 172.21.15.123:0/4260856983 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f73e800be10 con 0x7f73f81008a0 2024-02-16T22:12:56.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.514+0000 7f73fe6c0700 1 -- 172.21.15.123:0/4260856983 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73f8196100 con 0x7f73f81008a0 2024-02-16T22:12:56.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.514+0000 7f73f57fa700 1 -- 172.21.15.123:0/4260856983 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f73e800f460 con 0x7f73f81008a0 2024-02-16T22:12:56.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.514+0000 7f73fe6c0700 1 -- 172.21.15.123:0/4260856983 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73f81965a0 con 0x7f73f81008a0 2024-02-16T22:12:56.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.515+0000 7f73f57fa700 1 -- 172.21.15.123:0/4260856983 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f73e800f6d0 con 0x7f73f81008a0 2024-02-16T22:12:56.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.515+0000 7f73fe6c0700 1 -- 172.21.15.123:0/4260856983 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73f818f4c0 con 0x7f73f81008a0 2024-02-16T22:12:56.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.515+0000 7f73f57fa700 1 --2- 172.21.15.123:0/4260856983 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f73e40383c0 0x7f73e403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:56.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.515+0000 7f73f57fa700 1 -- 172.21.15.123:0/4260856983 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f73e804c520 con 0x7f73f81008a0 2024-02-16T22:12:56.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.515+0000 7f73f77fe700 1 --2- 172.21.15.123:0/4260856983 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f73e40383c0 0x7f73e403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:56.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.516+0000 7f73f77fe700 1 --2- 172.21.15.123:0/4260856983 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f73e40383c0 0x7f73e403a880 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f73e0006fd0 tx=0x7f73e0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:56.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.520+0000 7f73f57fa700 1 -- 172.21.15.123:0/4260856983 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f73e8020550 con 0x7f73f81008a0 2024-02-16T22:12:56.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.783+0000 7f73fe6c0700 1 -- 172.21.15.123:0/4260856983 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f73f802cc70 con 0x7f73f81008a0 2024-02-16T22:12:56.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.784+0000 7f73f57fa700 1 -- 172.21.15.123:0/4260856983 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f73e8027020 con 0x7f73f81008a0 2024-02-16T22:12:56.787 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:12:56.787 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":1,"fsid":"1dcbaabe-cd18-11ee-95ba-87774f69a715","modified":"2024-02-16T22:10:59.330677Z","created":"2024-02-16T22:10:59.330677Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi114","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:3300","nonce":0},{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:12:56.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.786+0000 7f73fe6c0700 1 -- 172.21.15.123:0/4260856983 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f73e40383c0 msgr2=0x7f73e403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:56.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.786+0000 7f73fe6c0700 1 --2- 172.21.15.123:0/4260856983 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f73e40383c0 0x7f73e403a880 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f73e0006fd0 tx=0x7f73e0006e40 comp rx=0 tx=0).stop 2024-02-16T22:12:56.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.786+0000 7f73fe6c0700 1 -- 172.21.15.123:0/4260856983 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73f81008a0 msgr2=0x7f73f81959c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:56.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.786+0000 7f73fe6c0700 1 --2- 172.21.15.123:0/4260856983 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73f81008a0 0x7f73f81959c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f73e8005b40 tx=0x7f73e8004740 comp rx=0 tx=0).stop 2024-02-16T22:12:56.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.786+0000 7f73fe6c0700 1 -- 172.21.15.123:0/4260856983 shutdown_connections 2024-02-16T22:12:56.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.786+0000 7f73fe6c0700 1 --2- 172.21.15.123:0/4260856983 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f73e40383c0 0x7f73e403a880 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:56.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.786+0000 7f73fe6c0700 1 --2- 172.21.15.123:0/4260856983 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73f81008a0 0x7f73f81959c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:56.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.786+0000 7f73fe6c0700 1 -- 172.21.15.123:0/4260856983 >> 172.21.15.123:0/4260856983 conn(0x7f73f80fbe60 msgr2=0x7f73f80fcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:56.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.786+0000 7f73fe6c0700 1 -- 172.21.15.123:0/4260856983 shutdown_connections 2024-02-16T22:12:56.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:56.786+0000 7f73fe6c0700 1 -- 172.21.15.123:0/4260856983 wait complete. 2024-02-16T22:12:56.790 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 1 2024-02-16T22:12:57.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:56 smithi114 ceph-mon[29235]: from='client.? 172.21.15.123:0/4260856983' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:12:58.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:57 smithi114 ceph-mon[29235]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:12:58.282 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:12:58.282 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:12:58.485 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:12:58.531 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:12:59.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:59.939+0000 7fefd21c1700 1 -- 172.21.15.123:0/2268995691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefcc0ff790 msgr2=0x7fefcc0ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:12:59.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:59.939+0000 7fefd21c1700 1 --2- 172.21.15.123:0/2268995691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefcc0ff790 0x7fefcc0ffbb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fefbc009910 tx=0x7fefbc009c20 comp rx=0 tx=0).stop 2024-02-16T22:12:59.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:59.939+0000 7fefd21c1700 1 -- 172.21.15.123:0/2268995691 shutdown_connections 2024-02-16T22:12:59.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:59.939+0000 7fefd21c1700 1 --2- 172.21.15.123:0/2268995691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefcc0ff790 0x7fefcc0ffbb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:12:59.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:59.939+0000 7fefd21c1700 1 -- 172.21.15.123:0/2268995691 >> 172.21.15.123:0/2268995691 conn(0x7fefcc0fad30 msgr2=0x7fefcc0fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:12:59.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:59.939+0000 7fefd21c1700 1 -- 172.21.15.123:0/2268995691 shutdown_connections 2024-02-16T22:12:59.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:59.939+0000 7fefd21c1700 1 -- 172.21.15.123:0/2268995691 wait complete. 2024-02-16T22:12:59.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:59.940+0000 7fefd21c1700 1 Processor -- start 2024-02-16T22:12:59.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:59.940+0000 7fefd21c1700 1 -- start start 2024-02-16T22:12:59.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:59.940+0000 7fefd21c1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefcc0ff790 0x7fefcc18c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:59.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:59.940+0000 7fefd21c1700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc18ccb0 con 0x7fefcc0ff790 2024-02-16T22:12:59.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:59.940+0000 7fefcb7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefcc0ff790 0x7fefcc18c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:59.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:59.940+0000 7fefcb7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefcc0ff790 0x7fefcc18c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.123:42836/0 (socket says 172.21.15.123:42836) 2024-02-16T22:12:59.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:59.940+0000 7fefcb7fe700 1 -- 172.21.15.123:0/5749660 learned_addr learned my addr 172.21.15.123:0/5749660 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-02-16T22:12:59.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:59.940+0000 7fefcb7fe700 1 -- 172.21.15.123:0/5749660 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefbc0095f0 con 0x7fefcc0ff790 2024-02-16T22:12:59.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:59.941+0000 7fefcb7fe700 1 --2- 172.21.15.123:0/5749660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefcc0ff790 0x7fefcc18c770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fefbc005b40 tx=0x7fefbc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:59.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:59.941+0000 7fefc97fa700 1 -- 172.21.15.123:0/5749660 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fefbc01d070 con 0x7fefcc0ff790 2024-02-16T22:12:59.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:59.941+0000 7fefc97fa700 1 -- 172.21.15.123:0/5749660 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fefbc00be10 con 0x7fefcc0ff790 2024-02-16T22:12:59.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:59.941+0000 7fefc97fa700 1 -- 172.21.15.123:0/5749660 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fefbc00f460 con 0x7fefcc0ff790 2024-02-16T22:12:59.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:59.941+0000 7fefd21c1700 1 -- 172.21.15.123:0/5749660 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefcc18adc0 con 0x7fefcc0ff790 2024-02-16T22:12:59.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:59.941+0000 7fefd21c1700 1 -- 172.21.15.123:0/5749660 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefcc18b260 con 0x7fefcc0ff790 2024-02-16T22:12:59.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:59.942+0000 7fefd21c1700 1 -- 172.21.15.123:0/5749660 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefcc062430 con 0x7fefcc0ff790 2024-02-16T22:12:59.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:59.942+0000 7fefc97fa700 1 -- 172.21.15.123:0/5749660 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fefbc00b960 con 0x7fefcc0ff790 2024-02-16T22:12:59.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:59.942+0000 7fefc97fa700 1 --2- 172.21.15.123:0/5749660 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7fefb80383c0 0x7fefb803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:12:59.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:59.942+0000 7fefc97fa700 1 -- 172.21.15.123:0/5749660 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fefbc04c3f0 con 0x7fefcc0ff790 2024-02-16T22:12:59.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:59.942+0000 7fefc2ffd700 1 --2- 172.21.15.123:0/5749660 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7fefb80383c0 0x7fefb803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:12:59.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:59.942+0000 7fefc2ffd700 1 --2- 172.21.15.123:0/5749660 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7fefb80383c0 0x7fefb803a880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fefb4006fd0 tx=0x7fefb4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:12:59.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:12:59.947+0000 7fefc97fa700 1 -- 172.21.15.123:0/5749660 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fefcc062430 con 0x7fefcc0ff790 2024-02-16T22:13:00.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:12:59 smithi114 ceph-mon[29235]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:13:00.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:00.217+0000 7fefd21c1700 1 -- 172.21.15.123:0/5749660 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fefcc18b5b0 con 0x7fefcc0ff790 2024-02-16T22:13:00.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:00.218+0000 7fefc97fa700 1 -- 172.21.15.123:0/5749660 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fefbc027070 con 0x7fefcc0ff790 2024-02-16T22:13:00.221 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:13:00.221 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":1,"fsid":"1dcbaabe-cd18-11ee-95ba-87774f69a715","modified":"2024-02-16T22:10:59.330677Z","created":"2024-02-16T22:10:59.330677Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi114","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:3300","nonce":0},{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:13:00.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:00.220+0000 7fefd21c1700 1 -- 172.21.15.123:0/5749660 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7fefb80383c0 msgr2=0x7fefb803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:00.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:00.220+0000 7fefd21c1700 1 --2- 172.21.15.123:0/5749660 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7fefb80383c0 0x7fefb803a880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fefb4006fd0 tx=0x7fefb4006e40 comp rx=0 tx=0).stop 2024-02-16T22:13:00.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:00.220+0000 7fefd21c1700 1 -- 172.21.15.123:0/5749660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefcc0ff790 msgr2=0x7fefcc18c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:00.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:00.220+0000 7fefd21c1700 1 --2- 172.21.15.123:0/5749660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefcc0ff790 0x7fefcc18c770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fefbc005b40 tx=0x7fefbc004740 comp rx=0 tx=0).stop 2024-02-16T22:13:00.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:00.220+0000 7fefd21c1700 1 -- 172.21.15.123:0/5749660 shutdown_connections 2024-02-16T22:13:00.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:00.220+0000 7fefd21c1700 1 --2- 172.21.15.123:0/5749660 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7fefb80383c0 0x7fefb803a880 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:00.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:00.220+0000 7fefd21c1700 1 --2- 172.21.15.123:0/5749660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefcc0ff790 0x7fefcc18c770 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:00.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:00.220+0000 7fefd21c1700 1 -- 172.21.15.123:0/5749660 >> 172.21.15.123:0/5749660 conn(0x7fefcc0fad30 msgr2=0x7fefcc0fb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:13:00.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:00.220+0000 7fefd21c1700 1 -- 172.21.15.123:0/5749660 shutdown_connections 2024-02-16T22:13:00.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:00.220+0000 7fefd21c1700 1 -- 172.21.15.123:0/5749660 wait complete. 2024-02-16T22:13:00.224 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 1 2024-02-16T22:13:01.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:00 smithi114 ceph-mon[29235]: from='client.? 172.21.15.123:0/5749660' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:13:01.804 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:13:01.804 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:13:02.007 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:13:02.052 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:13:02.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:01 smithi114 ceph-mon[29235]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:13:03.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.459+0000 7f22a5f7d700 1 -- 172.21.15.123:0/2695590107 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22a0101320 msgr2=0x7f22a0103710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:03.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.459+0000 7f22a5f7d700 1 --2- 172.21.15.123:0/2695590107 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22a0101320 0x7f22a0103710 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f229000b0a0 tx=0x7f229000b3b0 comp rx=0 tx=0).stop 2024-02-16T22:13:03.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.459+0000 7f22a5f7d700 1 -- 172.21.15.123:0/2695590107 shutdown_connections 2024-02-16T22:13:03.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.459+0000 7f22a5f7d700 1 --2- 172.21.15.123:0/2695590107 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22a0101320 0x7f22a0103710 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:03.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.459+0000 7f22a5f7d700 1 -- 172.21.15.123:0/2695590107 >> 172.21.15.123:0/2695590107 conn(0x7f22a00fad30 msgr2=0x7f22a00fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:13:03.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.459+0000 7f22a5f7d700 1 -- 172.21.15.123:0/2695590107 shutdown_connections 2024-02-16T22:13:03.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.459+0000 7f22a5f7d700 1 -- 172.21.15.123:0/2695590107 wait complete. 2024-02-16T22:13:03.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.459+0000 7f22a5f7d700 1 Processor -- start 2024-02-16T22:13:03.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.460+0000 7f22a5f7d700 1 -- start start 2024-02-16T22:13:03.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.460+0000 7f22a5f7d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22a0101320 0x7f22a0100c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:13:03.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.460+0000 7f22a5f7d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a01011c0 con 0x7f22a0101320 2024-02-16T22:13:03.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.460+0000 7f229f7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22a0101320 0x7f22a0100c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:13:03.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.460+0000 7f229f7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22a0101320 0x7f22a0100c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.123:42880/0 (socket says 172.21.15.123:42880) 2024-02-16T22:13:03.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.460+0000 7f229f7fe700 1 -- 172.21.15.123:0/3805012802 learned_addr learned my addr 172.21.15.123:0/3805012802 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-02-16T22:13:03.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.460+0000 7f229f7fe700 1 -- 172.21.15.123:0/3805012802 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2290009d00 con 0x7f22a0101320 2024-02-16T22:13:03.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.460+0000 7f229f7fe700 1 --2- 172.21.15.123:0/3805012802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22a0101320 0x7f22a0100c80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2290015040 tx=0x7f229000bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:13:03.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.461+0000 7f229cff9700 1 -- 172.21.15.123:0/3805012802 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f229000e040 con 0x7f22a0101320 2024-02-16T22:13:03.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.461+0000 7f22a5f7d700 1 -- 172.21.15.123:0/3805012802 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a00ff2d0 con 0x7f22a0101320 2024-02-16T22:13:03.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.461+0000 7f22a5f7d700 1 -- 172.21.15.123:0/3805012802 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a00ff6f0 con 0x7f22a0101320 2024-02-16T22:13:03.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.461+0000 7f229cff9700 1 -- 172.21.15.123:0/3805012802 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f229001d930 con 0x7f22a0101320 2024-02-16T22:13:03.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.461+0000 7f229cff9700 1 -- 172.21.15.123:0/3805012802 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2290027980 con 0x7f22a0101320 2024-02-16T22:13:03.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.462+0000 7f22a5f7d700 1 -- 172.21.15.123:0/3805012802 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a0100100 con 0x7f22a0101320 2024-02-16T22:13:03.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.462+0000 7f229cff9700 1 -- 172.21.15.123:0/3805012802 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f229001d480 con 0x7f22a0101320 2024-02-16T22:13:03.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.462+0000 7f229cff9700 1 --2- 172.21.15.123:0/3805012802 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f22880383c0 0x7f228803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:13:03.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.462+0000 7f229cff9700 1 -- 172.21.15.123:0/3805012802 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f229004c820 con 0x7f22a0101320 2024-02-16T22:13:03.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.462+0000 7f229effd700 1 --2- 172.21.15.123:0/3805012802 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f22880383c0 0x7f228803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:13:03.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.462+0000 7f229effd700 1 --2- 172.21.15.123:0/3805012802 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f22880383c0 0x7f228803a880 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2294006fd0 tx=0x7f2294006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:13:03.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.467+0000 7f229cff9700 1 -- 172.21.15.123:0/3805012802 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f22900265a0 con 0x7f22a0101320 2024-02-16T22:13:03.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.730+0000 7f22a5f7d700 1 -- 172.21.15.123:0/3805012802 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f22a018ee90 con 0x7f22a0101320 2024-02-16T22:13:03.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.731+0000 7f229cff9700 1 -- 172.21.15.123:0/3805012802 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f22900263c0 con 0x7f22a0101320 2024-02-16T22:13:03.734 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:13:03.734 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":1,"fsid":"1dcbaabe-cd18-11ee-95ba-87774f69a715","modified":"2024-02-16T22:10:59.330677Z","created":"2024-02-16T22:10:59.330677Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi114","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:3300","nonce":0},{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:13:03.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.734+0000 7f22a5f7d700 1 -- 172.21.15.123:0/3805012802 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f22880383c0 msgr2=0x7f228803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:03.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.734+0000 7f22a5f7d700 1 --2- 172.21.15.123:0/3805012802 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f22880383c0 0x7f228803a880 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2294006fd0 tx=0x7f2294006e40 comp rx=0 tx=0).stop 2024-02-16T22:13:03.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.734+0000 7f22a5f7d700 1 -- 172.21.15.123:0/3805012802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22a0101320 msgr2=0x7f22a0100c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:03.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.734+0000 7f22a5f7d700 1 --2- 172.21.15.123:0/3805012802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22a0101320 0x7f22a0100c80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2290015040 tx=0x7f229000bb10 comp rx=0 tx=0).stop 2024-02-16T22:13:03.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.734+0000 7f22a5f7d700 1 -- 172.21.15.123:0/3805012802 shutdown_connections 2024-02-16T22:13:03.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.734+0000 7f22a5f7d700 1 --2- 172.21.15.123:0/3805012802 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f22880383c0 0x7f228803a880 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:03.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.734+0000 7f22a5f7d700 1 --2- 172.21.15.123:0/3805012802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22a0101320 0x7f22a0100c80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:03.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.734+0000 7f22a5f7d700 1 -- 172.21.15.123:0/3805012802 >> 172.21.15.123:0/3805012802 conn(0x7f22a00fad30 msgr2=0x7f22a00fb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:13:03.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.734+0000 7f22a5f7d700 1 -- 172.21.15.123:0/3805012802 shutdown_connections 2024-02-16T22:13:03.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:03.734+0000 7f22a5f7d700 1 -- 172.21.15.123:0/3805012802 wait complete. 2024-02-16T22:13:03.737 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 1 2024-02-16T22:13:04.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:03 smithi114 ceph-mon[29235]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:13:04.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:03 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:04.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:03 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:04.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:03 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:04.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:03 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-16T22:13:04.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:03 smithi114 ceph-mon[29235]: from='client.? 172.21.15.123:0/3805012802' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:13:05.322 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:13:05.323 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:13:05.522 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:13:05.568 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:13:05.940 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:05 smithi114 ceph-mon[29235]: from='mgr.14164 172.21.15.114:0/628659527' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-16T22:13:05.940 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:05 smithi114 ceph-mon[29235]: mgrmap e14: smithi114.vcullz(active, since 64s) 2024-02-16T22:13:06.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:06.922+0000 7f7a74e1d700 1 -- 172.21.15.123:0/1774737367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a700ff7a0 msgr2=0x7f7a700ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:06.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:06.922+0000 7f7a74e1d700 1 --2- 172.21.15.123:0/1774737367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a700ff7a0 0x7f7a700ffbc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7a64009910 tx=0x7f7a64009c20 comp rx=0 tx=0).stop 2024-02-16T22:13:06.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:06.922+0000 7f7a74e1d700 1 -- 172.21.15.123:0/1774737367 shutdown_connections 2024-02-16T22:13:06.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:06.922+0000 7f7a74e1d700 1 --2- 172.21.15.123:0/1774737367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a700ff7a0 0x7f7a700ffbc0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:06.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:06.922+0000 7f7a74e1d700 1 -- 172.21.15.123:0/1774737367 >> 172.21.15.123:0/1774737367 conn(0x7f7a700fad20 msgr2=0x7f7a700fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:13:06.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:06.922+0000 7f7a74e1d700 1 -- 172.21.15.123:0/1774737367 shutdown_connections 2024-02-16T22:13:06.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:06.922+0000 7f7a74e1d700 1 -- 172.21.15.123:0/1774737367 wait complete. 2024-02-16T22:13:06.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:06.923+0000 7f7a74e1d700 1 Processor -- start 2024-02-16T22:13:06.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:06.923+0000 7f7a74e1d700 1 -- start start 2024-02-16T22:13:06.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:06.923+0000 7f7a74e1d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a700ff7a0 0x7f7a70197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:13:06.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:06.923+0000 7f7a74e1d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a701980e0 con 0x7f7a700ff7a0 2024-02-16T22:13:06.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:06.923+0000 7f7a6e59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a700ff7a0 0x7f7a70197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:13:06.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:06.923+0000 7f7a6e59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a700ff7a0 0x7f7a70197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.123:42896/0 (socket says 172.21.15.123:42896) 2024-02-16T22:13:06.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:06.923+0000 7f7a6e59c700 1 -- 172.21.15.123:0/1437783473 learned_addr learned my addr 172.21.15.123:0/1437783473 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-02-16T22:13:06.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:06.924+0000 7f7a6e59c700 1 -- 172.21.15.123:0/1437783473 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a640095f0 con 0x7f7a700ff7a0 2024-02-16T22:13:06.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:06.924+0000 7f7a6e59c700 1 --2- 172.21.15.123:0/1437783473 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a700ff7a0 0x7f7a70197ba0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7a64005950 tx=0x7f7a64009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:13:06.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:06.924+0000 7f7a5f7fe700 1 -- 172.21.15.123:0/1437783473 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7a6401d070 con 0x7f7a700ff7a0 2024-02-16T22:13:06.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:06.924+0000 7f7a5f7fe700 1 -- 172.21.15.123:0/1437783473 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7a6400be10 con 0x7f7a700ff7a0 2024-02-16T22:13:06.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:06.924+0000 7f7a5f7fe700 1 -- 172.21.15.123:0/1437783473 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7a6400f460 con 0x7f7a700ff7a0 2024-02-16T22:13:06.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:06.924+0000 7f7a74e1d700 1 -- 172.21.15.123:0/1437783473 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a701982e0 con 0x7f7a700ff7a0 2024-02-16T22:13:06.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:06.924+0000 7f7a74e1d700 1 -- 172.21.15.123:0/1437783473 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a70198780 con 0x7f7a700ff7a0 2024-02-16T22:13:06.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:06.925+0000 7f7a5f7fe700 1 -- 172.21.15.123:0/1437783473 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 14) v1 ==== 45354+0+0 (secure 0 0 0) 0x7f7a6400b960 con 0x7f7a700ff7a0 2024-02-16T22:13:06.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:06.925+0000 7f7a74e1d700 1 -- 172.21.15.123:0/1437783473 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a701917d0 con 0x7f7a700ff7a0 2024-02-16T22:13:06.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:06.925+0000 7f7a5f7fe700 1 --2- 172.21.15.123:0/1437783473 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f7a58038410 0x7f7a5803a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:13:06.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:06.925+0000 7f7a5f7fe700 1 -- 172.21.15.123:0/1437783473 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7a6404c570 con 0x7f7a700ff7a0 2024-02-16T22:13:06.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:06.925+0000 7f7a6dd9b700 1 -- 172.21.15.123:0/1437783473 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f7a58038410 msgr2=0x7f7a5803a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/2648536185 2024-02-16T22:13:06.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:06.925+0000 7f7a6dd9b700 1 --2- 172.21.15.123:0/1437783473 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f7a58038410 0x7f7a5803a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T22:13:06.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:06.930+0000 7f7a5f7fe700 1 -- 172.21.15.123:0/1437783473 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7a6402bb60 con 0x7f7a700ff7a0 2024-02-16T22:13:07.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:07.126+0000 7f7a6dd9b700 1 -- 172.21.15.123:0/1437783473 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f7a58038410 msgr2=0x7f7a5803a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/2648536185 2024-02-16T22:13:07.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:07.126+0000 7f7a6dd9b700 1 --2- 172.21.15.123:0/1437783473 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f7a58038410 0x7f7a5803a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-16T22:13:07.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:07.201+0000 7f7a74e1d700 1 -- 172.21.15.123:0/1437783473 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7a7002cc70 con 0x7f7a700ff7a0 2024-02-16T22:13:07.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:07.202+0000 7f7a5f7fe700 1 -- 172.21.15.123:0/1437783473 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f7a64027030 con 0x7f7a700ff7a0 2024-02-16T22:13:07.204 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:13:07.204 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":1,"fsid":"1dcbaabe-cd18-11ee-95ba-87774f69a715","modified":"2024-02-16T22:10:59.330677Z","created":"2024-02-16T22:10:59.330677Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi114","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:3300","nonce":0},{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:13:07.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:07.204+0000 7f7a74e1d700 1 -- 172.21.15.123:0/1437783473 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f7a58038410 msgr2=0x7f7a5803a8d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T22:13:07.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:07.204+0000 7f7a74e1d700 1 --2- 172.21.15.123:0/1437783473 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f7a58038410 0x7f7a5803a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:07.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:07.204+0000 7f7a74e1d700 1 -- 172.21.15.123:0/1437783473 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a700ff7a0 msgr2=0x7f7a70197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:07.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:07.204+0000 7f7a74e1d700 1 --2- 172.21.15.123:0/1437783473 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a700ff7a0 0x7f7a70197ba0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7a64005950 tx=0x7f7a64009e60 comp rx=0 tx=0).stop 2024-02-16T22:13:07.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:07.204+0000 7f7a74e1d700 1 -- 172.21.15.123:0/1437783473 shutdown_connections 2024-02-16T22:13:07.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:07.204+0000 7f7a74e1d700 1 --2- 172.21.15.123:0/1437783473 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f7a58038410 0x7f7a5803a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:07.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:07.204+0000 7f7a74e1d700 1 --2- 172.21.15.123:0/1437783473 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a700ff7a0 0x7f7a70197ba0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:07.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:07.204+0000 7f7a74e1d700 1 -- 172.21.15.123:0/1437783473 >> 172.21.15.123:0/1437783473 conn(0x7f7a700fad20 msgr2=0x7f7a70068810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:13:07.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:07.204+0000 7f7a74e1d700 1 -- 172.21.15.123:0/1437783473 shutdown_connections 2024-02-16T22:13:07.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:07.204+0000 7f7a74e1d700 1 -- 172.21.15.123:0/1437783473 wait complete. 2024-02-16T22:13:07.207 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 1 2024-02-16T22:13:08.153 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:07 smithi114 ceph-mon[29235]: from='client.? 172.21.15.123:0/1437783473' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:13:08.711 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:13:08.712 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:13:08.922 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:13:08.968 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:13:10.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.426+0000 7f42fa591700 1 -- 172.21.15.123:0/3526513063 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42f40ff7a0 msgr2=0x7f42f40ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:10.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.426+0000 7f42fa591700 1 --2- 172.21.15.123:0/3526513063 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42f40ff7a0 0x7f42f40ffbc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009910 tx=0x7f42e4009c20 comp rx=0 tx=0).stop 2024-02-16T22:13:10.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.426+0000 7f42fa591700 1 -- 172.21.15.123:0/3526513063 shutdown_connections 2024-02-16T22:13:10.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.426+0000 7f42fa591700 1 --2- 172.21.15.123:0/3526513063 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42f40ff7a0 0x7f42f40ffbc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:10.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.426+0000 7f42fa591700 1 -- 172.21.15.123:0/3526513063 >> 172.21.15.123:0/3526513063 conn(0x7f42f40fad20 msgr2=0x7f42f40fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:13:10.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.426+0000 7f42fa591700 1 -- 172.21.15.123:0/3526513063 shutdown_connections 2024-02-16T22:13:10.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.426+0000 7f42fa591700 1 -- 172.21.15.123:0/3526513063 wait complete. 2024-02-16T22:13:10.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.427+0000 7f42fa591700 1 Processor -- start 2024-02-16T22:13:10.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.427+0000 7f42fa591700 1 -- start start 2024-02-16T22:13:10.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.427+0000 7f42fa591700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42f40ff7a0 0x7f42f40693d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:13:10.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.427+0000 7f42fa591700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f4198700 con 0x7f42f40ff7a0 2024-02-16T22:13:10.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.427+0000 7f42f3fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42f40ff7a0 0x7f42f40693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:13:10.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.427+0000 7f42f3fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42f40ff7a0 0x7f42f40693d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.123:42908/0 (socket says 172.21.15.123:42908) 2024-02-16T22:13:10.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.427+0000 7f42f3fff700 1 -- 172.21.15.123:0/4035616187 learned_addr learned my addr 172.21.15.123:0/4035616187 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-02-16T22:13:10.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.427+0000 7f42f3fff700 1 -- 172.21.15.123:0/4035616187 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e40095f0 con 0x7f42f40ff7a0 2024-02-16T22:13:10.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.428+0000 7f42f3fff700 1 --2- 172.21.15.123:0/4035616187 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42f40ff7a0 0x7f42f40693d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f42e4005b40 tx=0x7f42e4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:13:10.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.428+0000 7f42f17fa700 1 -- 172.21.15.123:0/4035616187 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f42e401d070 con 0x7f42f40ff7a0 2024-02-16T22:13:10.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.428+0000 7f42f17fa700 1 -- 172.21.15.123:0/4035616187 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f42e400be10 con 0x7f42f40ff7a0 2024-02-16T22:13:10.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.428+0000 7f42fa591700 1 -- 172.21.15.123:0/4035616187 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f4198840 con 0x7f42f40ff7a0 2024-02-16T22:13:10.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.428+0000 7f42f17fa700 1 -- 172.21.15.123:0/4035616187 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f42e400f460 con 0x7f42f40ff7a0 2024-02-16T22:13:10.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.428+0000 7f42fa591700 1 -- 172.21.15.123:0/4035616187 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f4198ce0 con 0x7f42f40ff7a0 2024-02-16T22:13:10.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.429+0000 7f42f17fa700 1 -- 172.21.15.123:0/4035616187 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 14) v1 ==== 45354+0+0 (secure 0 0 0) 0x7f42e400b960 con 0x7f42f40ff7a0 2024-02-16T22:13:10.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.429+0000 7f42fa591700 1 -- 172.21.15.123:0/4035616187 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f418e3c0 con 0x7f42f40ff7a0 2024-02-16T22:13:10.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.429+0000 7f42f17fa700 1 --2- 172.21.15.123:0/4035616187 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f42dc038410 0x7f42dc03a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:13:10.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.429+0000 7f42f17fa700 1 -- 172.21.15.123:0/4035616187 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f42e404c4f0 con 0x7f42f40ff7a0 2024-02-16T22:13:10.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.429+0000 7f42f37fe700 1 -- 172.21.15.123:0/4035616187 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f42dc038410 msgr2=0x7f42dc03a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/2648536185 2024-02-16T22:13:10.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.429+0000 7f42f37fe700 1 --2- 172.21.15.123:0/4035616187 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f42dc038410 0x7f42dc03a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T22:13:10.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.434+0000 7f42f17fa700 1 -- 172.21.15.123:0/4035616187 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f42e402a360 con 0x7f42f40ff7a0 2024-02-16T22:13:10.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.630+0000 7f42f37fe700 1 -- 172.21.15.123:0/4035616187 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f42dc038410 msgr2=0x7f42dc03a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/2648536185 2024-02-16T22:13:10.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.630+0000 7f42f37fe700 1 --2- 172.21.15.123:0/4035616187 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f42dc038410 0x7f42dc03a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-16T22:13:10.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.698+0000 7f42fa591700 1 -- 172.21.15.123:0/4035616187 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f42f402cc70 con 0x7f42f40ff7a0 2024-02-16T22:13:10.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.699+0000 7f42f17fa700 1 -- 172.21.15.123:0/4035616187 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f42e4027030 con 0x7f42f40ff7a0 2024-02-16T22:13:10.701 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:13:10.701 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":1,"fsid":"1dcbaabe-cd18-11ee-95ba-87774f69a715","modified":"2024-02-16T22:10:59.330677Z","created":"2024-02-16T22:10:59.330677Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi114","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:3300","nonce":0},{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:13:10.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.701+0000 7f42fa591700 1 -- 172.21.15.123:0/4035616187 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f42dc038410 msgr2=0x7f42dc03a8d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T22:13:10.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.701+0000 7f42fa591700 1 --2- 172.21.15.123:0/4035616187 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f42dc038410 0x7f42dc03a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:10.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.701+0000 7f42fa591700 1 -- 172.21.15.123:0/4035616187 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42f40ff7a0 msgr2=0x7f42f40693d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:10.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.701+0000 7f42fa591700 1 --2- 172.21.15.123:0/4035616187 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42f40ff7a0 0x7f42f40693d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f42e4005b40 tx=0x7f42e4004740 comp rx=0 tx=0).stop 2024-02-16T22:13:10.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.701+0000 7f42fa591700 1 -- 172.21.15.123:0/4035616187 shutdown_connections 2024-02-16T22:13:10.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.701+0000 7f42fa591700 1 --2- 172.21.15.123:0/4035616187 >> [v2:172.21.15.114:6800/2648536185,v1:172.21.15.114:6801/2648536185] conn(0x7f42dc038410 0x7f42dc03a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:10.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.701+0000 7f42fa591700 1 --2- 172.21.15.123:0/4035616187 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42f40ff7a0 0x7f42f40693d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:10.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.701+0000 7f42fa591700 1 -- 172.21.15.123:0/4035616187 >> 172.21.15.123:0/4035616187 conn(0x7f42f40fad20 msgr2=0x7f42f40fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:13:10.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.701+0000 7f42fa591700 1 -- 172.21.15.123:0/4035616187 shutdown_connections 2024-02-16T22:13:10.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:10.701+0000 7f42fa591700 1 -- 172.21.15.123:0/4035616187 wait complete. 2024-02-16T22:13:10.703 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 1 2024-02-16T22:13:10.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:10 smithi114 ceph-mon[29235]: from='client.? 172.21.15.123:0/4035616187' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:13:12.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:11 smithi114 ceph-mon[29235]: Active manager daemon smithi114.vcullz restarted 2024-02-16T22:13:12.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:11 smithi114 ceph-mon[29235]: Activating manager daemon smithi114.vcullz 2024-02-16T22:13:12.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:11 smithi114 ceph-mon[29235]: osdmap e5: 0 total, 0 up, 0 in 2024-02-16T22:13:12.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:11 smithi114 ceph-mon[29235]: mgrmap e15: smithi114.vcullz(active, starting, since 0.00321263s) 2024-02-16T22:13:12.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:11 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi114"}]: dispatch 2024-02-16T22:13:12.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:11 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mgr metadata", "who": "smithi114.vcullz", "id": "smithi114.vcullz"}]: dispatch 2024-02-16T22:13:12.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:11 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T22:13:12.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:11 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T22:13:12.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:11 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T22:13:12.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:11 smithi114 ceph-mon[29235]: Manager daemon smithi114.vcullz is now available 2024-02-16T22:13:12.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:11 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:13:12.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:11 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:13:12.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:11 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:12.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:11 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:13:12.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:11 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi114.vcullz/mirror_snapshot_schedule"}]: dispatch 2024-02-16T22:13:12.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:11 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi114.vcullz/trash_purge_schedule"}]: dispatch 2024-02-16T22:13:12.315 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:13:12.315 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:13:12.511 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:13:12.557 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:13:13.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:13 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:13.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:13 smithi114 ceph-mon[29235]: [16/Feb/2024:22:13:12] ENGINE Bus STARTING 2024-02-16T22:13:13.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:13 smithi114 ceph-mon[29235]: [16/Feb/2024:22:13:12] ENGINE Serving on http://172.21.15.114:8765 2024-02-16T22:13:13.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:13 smithi114 ceph-mon[29235]: mgrmap e16: smithi114.vcullz(active, since 1.00496s) 2024-02-16T22:13:13.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:13 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:13.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:13 smithi114 ceph-mon[29235]: [16/Feb/2024:22:13:12] ENGINE Serving on https://172.21.15.114:7150 2024-02-16T22:13:13.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:13 smithi114 ceph-mon[29235]: [16/Feb/2024:22:13:12] ENGINE Bus STARTED 2024-02-16T22:13:14.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.023+0000 7fd75c3ea700 1 -- 172.21.15.123:0/1977044227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7540ff7a0 msgr2=0x7fd7540ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:14.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.023+0000 7fd75c3ea700 1 --2- 172.21.15.123:0/1977044227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7540ff7a0 0x7fd7540ffbc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd744009910 tx=0x7fd744009c20 comp rx=0 tx=0).stop 2024-02-16T22:13:14.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.023+0000 7fd75c3ea700 1 -- 172.21.15.123:0/1977044227 shutdown_connections 2024-02-16T22:13:14.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.023+0000 7fd75c3ea700 1 --2- 172.21.15.123:0/1977044227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7540ff7a0 0x7fd7540ffbc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:14.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.023+0000 7fd75c3ea700 1 -- 172.21.15.123:0/1977044227 >> 172.21.15.123:0/1977044227 conn(0x7fd7540fad20 msgr2=0x7fd7540fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:13:14.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.023+0000 7fd75c3ea700 1 -- 172.21.15.123:0/1977044227 shutdown_connections 2024-02-16T22:13:14.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.023+0000 7fd75c3ea700 1 -- 172.21.15.123:0/1977044227 wait complete. 2024-02-16T22:13:14.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.023+0000 7fd75c3ea700 1 Processor -- start 2024-02-16T22:13:14.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.024+0000 7fd75c3ea700 1 -- start start 2024-02-16T22:13:14.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.024+0000 7fd75c3ea700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7540ff7a0 0x7fd7541959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:13:14.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.024+0000 7fd75c3ea700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd754195f00 con 0x7fd7540ff7a0 2024-02-16T22:13:14.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.024+0000 7fd75a186700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7540ff7a0 0x7fd7541959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:13:14.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.024+0000 7fd75a186700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7540ff7a0 0x7fd7541959c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.123:58202/0 (socket says 172.21.15.123:58202) 2024-02-16T22:13:14.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.024+0000 7fd75a186700 1 -- 172.21.15.123:0/1866913728 learned_addr learned my addr 172.21.15.123:0/1866913728 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-02-16T22:13:14.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.024+0000 7fd75a186700 1 -- 172.21.15.123:0/1866913728 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7440095f0 con 0x7fd7540ff7a0 2024-02-16T22:13:14.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.024+0000 7fd75a186700 1 --2- 172.21.15.123:0/1866913728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7540ff7a0 0x7fd7541959c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd744005b40 tx=0x7fd744004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:13:14.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.025+0000 7fd74b7fe700 1 -- 172.21.15.123:0/1866913728 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd74401d070 con 0x7fd7540ff7a0 2024-02-16T22:13:14.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.025+0000 7fd74b7fe700 1 -- 172.21.15.123:0/1866913728 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd74400be10 con 0x7fd7540ff7a0 2024-02-16T22:13:14.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.025+0000 7fd75c3ea700 1 -- 172.21.15.123:0/1866913728 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd754196100 con 0x7fd7540ff7a0 2024-02-16T22:13:14.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.025+0000 7fd74b7fe700 1 -- 172.21.15.123:0/1866913728 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd74400f460 con 0x7fd7540ff7a0 2024-02-16T22:13:14.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.025+0000 7fd75c3ea700 1 -- 172.21.15.123:0/1866913728 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7541965a0 con 0x7fd7540ff7a0 2024-02-16T22:13:14.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.025+0000 7fd75c3ea700 1 -- 172.21.15.123:0/1866913728 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7541055e0 con 0x7fd7540ff7a0 2024-02-16T22:13:14.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.026+0000 7fd74b7fe700 1 -- 172.21.15.123:0/1866913728 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 45204+0+0 (secure 0 0 0) 0x7fd7440227e0 con 0x7fd7540ff7a0 2024-02-16T22:13:14.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.026+0000 7fd74b7fe700 1 --2- 172.21.15.123:0/1866913728 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd740038270 0x7fd74003a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:13:14.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.026+0000 7fd74b7fe700 1 -- 172.21.15.123:0/1866913728 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fd74404c270 con 0x7fd7540ff7a0 2024-02-16T22:13:14.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.026+0000 7fd759985700 1 --2- 172.21.15.123:0/1866913728 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd740038270 0x7fd74003a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:13:14.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.027+0000 7fd759985700 1 --2- 172.21.15.123:0/1866913728 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd740038270 0x7fd74003a730 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd750006fd0 tx=0x7fd750006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:13:14.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.031+0000 7fd74b7fe700 1 -- 172.21.15.123:0/1866913728 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd74402bba0 con 0x7fd7540ff7a0 2024-02-16T22:13:14.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.091+0000 7fd74b7fe700 1 -- 172.21.15.123:0/1866913728 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7fd74400b960 con 0x7fd7540ff7a0 2024-02-16T22:13:14.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.310+0000 7fd75c3ea700 1 -- 172.21.15.123:0/1866913728 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd754199380 con 0x7fd7540ff7a0 2024-02-16T22:13:14.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.310+0000 7fd74b7fe700 1 -- 172.21.15.123:0/1866913728 <== mon.0 v2:172.21.15.114:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fd744027090 con 0x7fd7540ff7a0 2024-02-16T22:13:14.312 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:13:14.312 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":1,"fsid":"1dcbaabe-cd18-11ee-95ba-87774f69a715","modified":"2024-02-16T22:10:59.330677Z","created":"2024-02-16T22:10:59.330677Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi114","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:3300","nonce":0},{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:13:14.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.313+0000 7fd75c3ea700 1 -- 172.21.15.123:0/1866913728 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd740038270 msgr2=0x7fd74003a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:14.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.313+0000 7fd75c3ea700 1 --2- 172.21.15.123:0/1866913728 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd740038270 0x7fd74003a730 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd750006fd0 tx=0x7fd750006e40 comp rx=0 tx=0).stop 2024-02-16T22:13:14.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.313+0000 7fd75c3ea700 1 -- 172.21.15.123:0/1866913728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7540ff7a0 msgr2=0x7fd7541959c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:14.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.313+0000 7fd75c3ea700 1 --2- 172.21.15.123:0/1866913728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7540ff7a0 0x7fd7541959c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd744005b40 tx=0x7fd744004740 comp rx=0 tx=0).stop 2024-02-16T22:13:14.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.313+0000 7fd75c3ea700 1 -- 172.21.15.123:0/1866913728 shutdown_connections 2024-02-16T22:13:14.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.313+0000 7fd75c3ea700 1 --2- 172.21.15.123:0/1866913728 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd740038270 0x7fd74003a730 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:14.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.313+0000 7fd75c3ea700 1 --2- 172.21.15.123:0/1866913728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7540ff7a0 0x7fd7541959c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:14.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.313+0000 7fd75c3ea700 1 -- 172.21.15.123:0/1866913728 >> 172.21.15.123:0/1866913728 conn(0x7fd7540fad20 msgr2=0x7fd7540fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:13:14.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.313+0000 7fd75c3ea700 1 -- 172.21.15.123:0/1866913728 shutdown_connections 2024-02-16T22:13:14.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:14.313+0000 7fd75c3ea700 1 -- 172.21.15.123:0/1866913728 wait complete. 2024-02-16T22:13:14.315 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 1 2024-02-16T22:13:14.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:14 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:14.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:14 smithi114 ceph-mon[29235]: mgrmap e17: smithi114.vcullz(active, since 2s) 2024-02-16T22:13:14.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:14 smithi114 ceph-mon[29235]: from='client.? 172.21.15.123:0/1866913728' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:13:16.198 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:13:16.199 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:13:16.406 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:13:16.461 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T22:13:18.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.413+0000 7ff57eb6d700 1 -- 172.21.15.123:0/2162570425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5780ff7a0 msgr2=0x7ff5780ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:18.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.413+0000 7ff57eb6d700 1 --2- 172.21.15.123:0/2162570425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5780ff7a0 0x7ff5780ffbc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff56c009910 tx=0x7ff56c009c20 comp rx=0 tx=0).stop 2024-02-16T22:13:18.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.413+0000 7ff57eb6d700 1 -- 172.21.15.123:0/2162570425 shutdown_connections 2024-02-16T22:13:18.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.413+0000 7ff57eb6d700 1 --2- 172.21.15.123:0/2162570425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5780ff7a0 0x7ff5780ffbc0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:18.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.413+0000 7ff57eb6d700 1 -- 172.21.15.123:0/2162570425 >> 172.21.15.123:0/2162570425 conn(0x7ff5780fad20 msgr2=0x7ff5780fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:13:18.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.413+0000 7ff57eb6d700 1 -- 172.21.15.123:0/2162570425 shutdown_connections 2024-02-16T22:13:18.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.413+0000 7ff57eb6d700 1 -- 172.21.15.123:0/2162570425 wait complete. 2024-02-16T22:13:18.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.413+0000 7ff57eb6d700 1 Processor -- start 2024-02-16T22:13:18.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.413+0000 7ff57eb6d700 1 -- start start 2024-02-16T22:13:18.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.414+0000 7ff57eb6d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5780ff7a0 0x7ff5781959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:13:18.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.414+0000 7ff57eb6d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff578195f00 con 0x7ff5780ff7a0 2024-02-16T22:13:18.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.414+0000 7ff57c909700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5780ff7a0 0x7ff5781959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:13:18.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.414+0000 7ff57c909700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5780ff7a0 0x7ff5781959c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.123:58224/0 (socket says 172.21.15.123:58224) 2024-02-16T22:13:18.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.414+0000 7ff57c909700 1 -- 172.21.15.123:0/1041011979 learned_addr learned my addr 172.21.15.123:0/1041011979 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-02-16T22:13:18.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.414+0000 7ff57c909700 1 -- 172.21.15.123:0/1041011979 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff56c0095f0 con 0x7ff5780ff7a0 2024-02-16T22:13:18.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.414+0000 7ff57c909700 1 --2- 172.21.15.123:0/1041011979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5780ff7a0 0x7ff5781959c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff56c005b40 tx=0x7ff56c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:13:18.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.414+0000 7ff575ffb700 1 -- 172.21.15.123:0/1041011979 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff56c01d070 con 0x7ff5780ff7a0 2024-02-16T22:13:18.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.414+0000 7ff575ffb700 1 -- 172.21.15.123:0/1041011979 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff56c00be10 con 0x7ff5780ff7a0 2024-02-16T22:13:18.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.414+0000 7ff57eb6d700 1 -- 172.21.15.123:0/1041011979 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff578196100 con 0x7ff5780ff7a0 2024-02-16T22:13:18.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.414+0000 7ff575ffb700 1 -- 172.21.15.123:0/1041011979 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff56c00f460 con 0x7ff5780ff7a0 2024-02-16T22:13:18.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.415+0000 7ff57eb6d700 1 -- 172.21.15.123:0/1041011979 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5781965a0 con 0x7ff5780ff7a0 2024-02-16T22:13:18.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.415+0000 7ff575ffb700 1 -- 172.21.15.123:0/1041011979 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7ff56c00b960 con 0x7ff5780ff7a0 2024-02-16T22:13:18.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.415+0000 7ff57eb6d700 1 -- 172.21.15.123:0/1041011979 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff57818f3d0 con 0x7ff5780ff7a0 2024-02-16T22:13:18.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.416+0000 7ff575ffb700 1 --2- 172.21.15.123:0/1041011979 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff5680384b0 0x7ff56803a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:13:18.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.416+0000 7ff575ffb700 1 -- 172.21.15.123:0/1041011979 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7ff56c04c6b0 con 0x7ff5780ff7a0 2024-02-16T22:13:18.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.416+0000 7ff577fff700 1 --2- 172.21.15.123:0/1041011979 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff5680384b0 0x7ff56803a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:13:18.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.416+0000 7ff577fff700 1 --2- 172.21.15.123:0/1041011979 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff5680384b0 0x7ff56803a970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff564006fd0 tx=0x7ff564006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:13:18.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.423+0000 7ff575ffb700 1 -- 172.21.15.123:0/1041011979 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff56c02a360 con 0x7ff5780ff7a0 2024-02-16T22:13:18.664 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:18 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:18.664 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:18 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:18.664 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:18 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:18.664 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:18 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:18.664 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:18 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:13:18.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.684+0000 7ff57eb6d700 1 -- 172.21.15.123:0/1041011979 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff57802cc70 con 0x7ff5780ff7a0 2024-02-16T22:13:18.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.684+0000 7ff575ffb700 1 -- 172.21.15.123:0/1041011979 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7ff56c027030 con 0x7ff5780ff7a0 2024-02-16T22:13:18.686 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:13:18.686 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":1,"fsid":"1dcbaabe-cd18-11ee-95ba-87774f69a715","modified":"2024-02-16T22:10:59.330677Z","created":"2024-02-16T22:10:59.330677Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi114","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:3300","nonce":0},{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:13:18.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.686+0000 7ff57eb6d700 1 -- 172.21.15.123:0/1041011979 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff5680384b0 msgr2=0x7ff56803a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:18.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.686+0000 7ff57eb6d700 1 --2- 172.21.15.123:0/1041011979 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff5680384b0 0x7ff56803a970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff564006fd0 tx=0x7ff564006e40 comp rx=0 tx=0).stop 2024-02-16T22:13:18.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.686+0000 7ff57eb6d700 1 -- 172.21.15.123:0/1041011979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5780ff7a0 msgr2=0x7ff5781959c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:18.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.686+0000 7ff57eb6d700 1 --2- 172.21.15.123:0/1041011979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5780ff7a0 0x7ff5781959c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff56c005b40 tx=0x7ff56c004740 comp rx=0 tx=0).stop 2024-02-16T22:13:18.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.686+0000 7ff57eb6d700 1 -- 172.21.15.123:0/1041011979 shutdown_connections 2024-02-16T22:13:18.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.686+0000 7ff57eb6d700 1 --2- 172.21.15.123:0/1041011979 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff5680384b0 0x7ff56803a970 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:18.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.686+0000 7ff57eb6d700 1 --2- 172.21.15.123:0/1041011979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5780ff7a0 0x7ff5781959c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:18.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.686+0000 7ff57eb6d700 1 -- 172.21.15.123:0/1041011979 >> 172.21.15.123:0/1041011979 conn(0x7ff5780fad20 msgr2=0x7ff578069100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:13:18.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.686+0000 7ff57eb6d700 1 -- 172.21.15.123:0/1041011979 shutdown_connections 2024-02-16T22:13:18.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:18.686+0000 7ff57eb6d700 1 -- 172.21.15.123:0/1041011979 wait complete. 2024-02-16T22:13:18.689 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 1 2024-02-16T22:13:19.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:19 smithi114 ceph-mon[29235]: from='client.? 172.21.15.123:0/1041011979' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:13:19.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:19 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:19.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:19 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:20.257 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:13:20.257 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:13:20.449 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:13:21.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:20 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:21.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:20 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:21.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:20 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "who": "osd/host:smithi114", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:13:21.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:20 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:13:21.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:20 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:13:21.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:20 smithi114 ceph-mon[29235]: Updating smithi114:/etc/ceph/ceph.conf 2024-02-16T22:13:21.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:20 smithi114 ceph-mon[29235]: Updating smithi123:/etc/ceph/ceph.conf 2024-02-16T22:13:21.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:20 smithi114 ceph-mon[29235]: Updating smithi114:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:13:21.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:20 smithi114 ceph-mon[29235]: Updating smithi123:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:13:21.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:20 smithi114 ceph-mon[29235]: Updating smithi114:/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:13:21.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:20 smithi114 ceph-mon[29235]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:13:21.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:20 smithi114 ceph-mon[29235]: Updating smithi114:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T22:13:21.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:20 smithi114 ceph-mon[29235]: Updating smithi123:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T22:13:21.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:20 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:21.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:20 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:21.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:20 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:21.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:20 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:21.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:20 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:21.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:20 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi123", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T22:13:21.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:20 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi123", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-02-16T22:13:21.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:20 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:13:22.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:21 smithi114 ceph-mon[29235]: Deploying daemon ceph-exporter.smithi123 on smithi123 2024-02-16T22:13:23.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.158+0000 7fd2c8aa3700 1 -- 172.21.15.123:0/2549384514 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2c4106830 msgr2=0x7fd2c4106c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:23.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.158+0000 7fd2c8aa3700 1 --2- 172.21.15.123:0/2549384514 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2c4106830 0x7fd2c4106c10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00b0a0 tx=0x7fd2bc00b3b0 comp rx=0 tx=0).stop 2024-02-16T22:13:23.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.158+0000 7fd2c8aa3700 1 -- 172.21.15.123:0/2549384514 shutdown_connections 2024-02-16T22:13:23.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.158+0000 7fd2c8aa3700 1 --2- 172.21.15.123:0/2549384514 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2c4106830 0x7fd2c4106c10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:23.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.158+0000 7fd2c8aa3700 1 -- 172.21.15.123:0/2549384514 >> 172.21.15.123:0/2549384514 conn(0x7fd2c40fc380 msgr2=0x7fd2c40fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:13:23.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.158+0000 7fd2c8aa3700 1 -- 172.21.15.123:0/2549384514 shutdown_connections 2024-02-16T22:13:23.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.158+0000 7fd2c8aa3700 1 -- 172.21.15.123:0/2549384514 wait complete. 2024-02-16T22:13:23.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.159+0000 7fd2c8aa3700 1 Processor -- start 2024-02-16T22:13:23.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.159+0000 7fd2c8aa3700 1 -- start start 2024-02-16T22:13:23.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.159+0000 7fd2c8aa3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2c4106830 0x7fd2c419bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:13:23.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.159+0000 7fd2c8aa3700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c419bff0 con 0x7fd2c4106830 2024-02-16T22:13:23.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.159+0000 7fd2c259c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2c4106830 0x7fd2c419bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:13:23.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.159+0000 7fd2c259c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2c4106830 0x7fd2c419bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.123:60072/0 (socket says 172.21.15.123:60072) 2024-02-16T22:13:23.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.159+0000 7fd2c259c700 1 -- 172.21.15.123:0/1264411647 learned_addr learned my addr 172.21.15.123:0/1264411647 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-02-16T22:13:23.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.159+0000 7fd2c259c700 1 -- 172.21.15.123:0/1264411647 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2bc009d00 con 0x7fd2c4106830 2024-02-16T22:13:23.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.159+0000 7fd2c259c700 1 --2- 172.21.15.123:0/1264411647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2c4106830 0x7fd2c419bab0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc015040 tx=0x7fd2bc003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:13:23.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.160+0000 7fd2bb7fe700 1 -- 172.21.15.123:0/1264411647 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd2bc00e040 con 0x7fd2c4106830 2024-02-16T22:13:23.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.160+0000 7fd2c8aa3700 1 -- 172.21.15.123:0/1264411647 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c419c1f0 con 0x7fd2c4106830 2024-02-16T22:13:23.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.160+0000 7fd2bb7fe700 1 -- 172.21.15.123:0/1264411647 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd2bc00bdd0 con 0x7fd2c4106830 2024-02-16T22:13:23.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.160+0000 7fd2bb7fe700 1 -- 172.21.15.123:0/1264411647 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd2bc01d9a0 con 0x7fd2c4106830 2024-02-16T22:13:23.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.160+0000 7fd2c8aa3700 1 -- 172.21.15.123:0/1264411647 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c4195b30 con 0x7fd2c4106830 2024-02-16T22:13:23.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.161+0000 7fd2bb7fe700 1 -- 172.21.15.123:0/1264411647 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7fd2bc01dbe0 con 0x7fd2c4106830 2024-02-16T22:13:23.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.161+0000 7fd2c8aa3700 1 -- 172.21.15.123:0/1264411647 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c40623c0 con 0x7fd2c4106830 2024-02-16T22:13:23.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.161+0000 7fd2bb7fe700 1 --2- 172.21.15.123:0/1264411647 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd2b0038500 0x7fd2b003a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:13:23.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.161+0000 7fd2bb7fe700 1 -- 172.21.15.123:0/1264411647 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fd2bc04c4f0 con 0x7fd2c4106830 2024-02-16T22:13:23.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.161+0000 7fd2c1d9b700 1 --2- 172.21.15.123:0/1264411647 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd2b0038500 0x7fd2b003a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:13:23.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.161+0000 7fd2c1d9b700 1 --2- 172.21.15.123:0/1264411647 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd2b0038500 0x7fd2b003a9c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac006fd0 tx=0x7fd2ac006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:13:23.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.166+0000 7fd2bb7fe700 1 -- 172.21.15.123:0/1264411647 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd2bc011390 con 0x7fd2c4106830 2024-02-16T22:13:23.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.458+0000 7fd2c8aa3700 1 -- 172.21.15.123:0/1264411647 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd2c41964f0 con 0x7fd2c4106830 2024-02-16T22:13:23.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.459+0000 7fd2bb7fe700 1 -- 172.21.15.123:0/1264411647 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fd2bc011390 con 0x7fd2c4106830 2024-02-16T22:13:23.461 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:13:23.461 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":1,"fsid":"1dcbaabe-cd18-11ee-95ba-87774f69a715","modified":"2024-02-16T22:10:59.330677Z","created":"2024-02-16T22:10:59.330677Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi114","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:3300","nonce":0},{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:13:23.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.461+0000 7fd2c8aa3700 1 -- 172.21.15.123:0/1264411647 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd2b0038500 msgr2=0x7fd2b003a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:23.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.461+0000 7fd2c8aa3700 1 --2- 172.21.15.123:0/1264411647 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd2b0038500 0x7fd2b003a9c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac006fd0 tx=0x7fd2ac006e40 comp rx=0 tx=0).stop 2024-02-16T22:13:23.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.461+0000 7fd2c8aa3700 1 -- 172.21.15.123:0/1264411647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2c4106830 msgr2=0x7fd2c419bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:23.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.461+0000 7fd2c8aa3700 1 --2- 172.21.15.123:0/1264411647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2c4106830 0x7fd2c419bab0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc015040 tx=0x7fd2bc003ce0 comp rx=0 tx=0).stop 2024-02-16T22:13:23.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.461+0000 7fd2c8aa3700 1 -- 172.21.15.123:0/1264411647 shutdown_connections 2024-02-16T22:13:23.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.461+0000 7fd2c8aa3700 1 --2- 172.21.15.123:0/1264411647 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd2b0038500 0x7fd2b003a9c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:23.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.461+0000 7fd2c8aa3700 1 --2- 172.21.15.123:0/1264411647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2c4106830 0x7fd2c419bab0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:23.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.461+0000 7fd2c8aa3700 1 -- 172.21.15.123:0/1264411647 >> 172.21.15.123:0/1264411647 conn(0x7fd2c40fc380 msgr2=0x7fd2c4108c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:13:23.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.461+0000 7fd2c8aa3700 1 -- 172.21.15.123:0/1264411647 shutdown_connections 2024-02-16T22:13:23.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:23.461+0000 7fd2c8aa3700 1 -- 172.21.15.123:0/1264411647 wait complete. 2024-02-16T22:13:23.464 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 1 2024-02-16T22:13:23.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:23 smithi114 ceph-mon[29235]: from='client.? 172.21.15.123:0/1264411647' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:13:25.065 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:13:25.065 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:13:26.158 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:13:26.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:25 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:26.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:25 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:26.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:25 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:26.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:25 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:26.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:25 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi123", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T22:13:26.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:25 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi123", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-02-16T22:13:26.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:25 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:13:27.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:26 smithi114 ceph-mon[29235]: Deploying daemon crash.smithi123 on smithi123 2024-02-16T22:13:27.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:27.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:13:28.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.208+0000 7fa84c1cc700 1 -- 172.21.15.123:0/2869260275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa844105710 msgr2=0x7fa844105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:28.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.208+0000 7fa84c1cc700 1 --2- 172.21.15.123:0/2869260275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa844105710 0x7fa844105af0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa840009910 tx=0x7fa840009c20 comp rx=0 tx=0).stop 2024-02-16T22:13:28.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.209+0000 7fa84c1cc700 1 -- 172.21.15.123:0/2869260275 shutdown_connections 2024-02-16T22:13:28.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.209+0000 7fa84c1cc700 1 --2- 172.21.15.123:0/2869260275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa844105710 0x7fa844105af0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:28.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.209+0000 7fa84c1cc700 1 -- 172.21.15.123:0/2869260275 >> 172.21.15.123:0/2869260275 conn(0x7fa8440fb220 msgr2=0x7fa8440fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:13:28.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.209+0000 7fa84c1cc700 1 -- 172.21.15.123:0/2869260275 shutdown_connections 2024-02-16T22:13:28.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.209+0000 7fa84c1cc700 1 -- 172.21.15.123:0/2869260275 wait complete. 2024-02-16T22:13:28.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.209+0000 7fa84c1cc700 1 Processor -- start 2024-02-16T22:13:28.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.209+0000 7fa84c1cc700 1 -- start start 2024-02-16T22:13:28.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.209+0000 7fa84c1cc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa844105710 0x7fa844195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:13:28.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.209+0000 7fa84c1cc700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8441962c0 con 0x7fa844105710 2024-02-16T22:13:28.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.210+0000 7fa849f68700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa844105710 0x7fa844195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:13:28.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.210+0000 7fa849f68700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa844105710 0x7fa844195d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.123:60108/0 (socket says 172.21.15.123:60108) 2024-02-16T22:13:28.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.210+0000 7fa849f68700 1 -- 172.21.15.123:0/856545940 learned_addr learned my addr 172.21.15.123:0/856545940 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-02-16T22:13:28.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.210+0000 7fa849f68700 1 -- 172.21.15.123:0/856545940 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8400095f0 con 0x7fa844105710 2024-02-16T22:13:28.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.210+0000 7fa849f68700 1 --2- 172.21.15.123:0/856545940 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa844105710 0x7fa844195d80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa840005b40 tx=0x7fa840004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:13:28.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.210+0000 7fa83affd700 1 -- 172.21.15.123:0/856545940 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa84001d070 con 0x7fa844105710 2024-02-16T22:13:28.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.210+0000 7fa84c1cc700 1 -- 172.21.15.123:0/856545940 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8441964c0 con 0x7fa844105710 2024-02-16T22:13:28.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.210+0000 7fa83affd700 1 -- 172.21.15.123:0/856545940 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa8400044d0 con 0x7fa844105710 2024-02-16T22:13:28.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.210+0000 7fa83affd700 1 -- 172.21.15.123:0/856545940 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa840003910 con 0x7fa844105710 2024-02-16T22:13:28.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.210+0000 7fa84c1cc700 1 -- 172.21.15.123:0/856545940 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa844196940 con 0x7fa844105710 2024-02-16T22:13:28.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.211+0000 7fa83affd700 1 -- 172.21.15.123:0/856545940 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7fa840003c10 con 0x7fa844105710 2024-02-16T22:13:28.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.211+0000 7fa83affd700 1 --2- 172.21.15.123:0/856545940 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa8300384b0 0x7fa83003a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:13:28.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.211+0000 7fa83affd700 1 -- 172.21.15.123:0/856545940 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fa84004c4b0 con 0x7fa844105710 2024-02-16T22:13:28.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.212+0000 7fa849767700 1 --2- 172.21.15.123:0/856545940 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa8300384b0 0x7fa83003a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:13:28.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.212+0000 7fa84c1cc700 1 -- 172.21.15.123:0/856545940 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8440623c0 con 0x7fa844105710 2024-02-16T22:13:28.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.212+0000 7fa849767700 1 --2- 172.21.15.123:0/856545940 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa8300384b0 0x7fa83003a970 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa834006fd0 tx=0x7fa834006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:13:28.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.217+0000 7fa83affd700 1 -- 172.21.15.123:0/856545940 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa84002a540 con 0x7fa844105710 2024-02-16T22:13:28.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.544+0000 7fa84c1cc700 1 -- 172.21.15.123:0/856545940 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa84419a730 con 0x7fa844105710 2024-02-16T22:13:28.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.545+0000 7fa83affd700 1 -- 172.21.15.123:0/856545940 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fa84002a360 con 0x7fa844105710 2024-02-16T22:13:28.547 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:13:28.547 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":1,"fsid":"1dcbaabe-cd18-11ee-95ba-87774f69a715","modified":"2024-02-16T22:10:59.330677Z","created":"2024-02-16T22:10:59.330677Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi114","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:3300","nonce":0},{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:13:28.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.547+0000 7fa84c1cc700 1 -- 172.21.15.123:0/856545940 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa8300384b0 msgr2=0x7fa83003a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:28.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.547+0000 7fa84c1cc700 1 --2- 172.21.15.123:0/856545940 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa8300384b0 0x7fa83003a970 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa834006fd0 tx=0x7fa834006e40 comp rx=0 tx=0).stop 2024-02-16T22:13:28.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.547+0000 7fa84c1cc700 1 -- 172.21.15.123:0/856545940 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa844105710 msgr2=0x7fa844195d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:28.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.547+0000 7fa84c1cc700 1 --2- 172.21.15.123:0/856545940 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa844105710 0x7fa844195d80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa840005b40 tx=0x7fa840004c80 comp rx=0 tx=0).stop 2024-02-16T22:13:28.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.547+0000 7fa84c1cc700 1 -- 172.21.15.123:0/856545940 shutdown_connections 2024-02-16T22:13:28.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.547+0000 7fa84c1cc700 1 --2- 172.21.15.123:0/856545940 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa8300384b0 0x7fa83003a970 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:28.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.547+0000 7fa84c1cc700 1 --2- 172.21.15.123:0/856545940 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa844105710 0x7fa844195d80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:28.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.547+0000 7fa84c1cc700 1 -- 172.21.15.123:0/856545940 >> 172.21.15.123:0/856545940 conn(0x7fa8440fb220 msgr2=0x7fa844107280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:13:28.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.547+0000 7fa84c1cc700 1 -- 172.21.15.123:0/856545940 shutdown_connections 2024-02-16T22:13:28.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:28.547+0000 7fa84c1cc700 1 -- 172.21.15.123:0/856545940 wait complete. 2024-02-16T22:13:28.550 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 1 2024-02-16T22:13:29.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:28 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:29.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:28 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:29.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:28 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:29.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:28 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:29.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:28 smithi114 ceph-mon[29235]: Deploying daemon node-exporter.smithi123 on smithi123 2024-02-16T22:13:29.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:28 smithi114 ceph-mon[29235]: from='client.? 172.21.15.123:0/856545940' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:13:30.199 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:13:30.200 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:13:31.160 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:13:32.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:32 smithi114 ceph-mon[29235]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:13:32.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:32 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:33.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.399+0000 7f9f79853700 1 -- 172.21.15.123:0/3205635726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f74067fd0 msgr2=0x7f9f740683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:33.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.399+0000 7f9f79853700 1 --2- 172.21.15.123:0/3205635726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f74067fd0 0x7f9f740683b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9f64009910 tx=0x7f9f64009c20 comp rx=0 tx=0).stop 2024-02-16T22:13:33.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.399+0000 7f9f79853700 1 -- 172.21.15.123:0/3205635726 shutdown_connections 2024-02-16T22:13:33.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.399+0000 7f9f79853700 1 --2- 172.21.15.123:0/3205635726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f74067fd0 0x7f9f740683b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:33.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.399+0000 7f9f79853700 1 -- 172.21.15.123:0/3205635726 >> 172.21.15.123:0/3205635726 conn(0x7f9f740752b0 msgr2=0x7f9f740756c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:13:33.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.399+0000 7f9f79853700 1 -- 172.21.15.123:0/3205635726 shutdown_connections 2024-02-16T22:13:33.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.399+0000 7f9f79853700 1 -- 172.21.15.123:0/3205635726 wait complete. 2024-02-16T22:13:33.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.400+0000 7f9f79853700 1 Processor -- start 2024-02-16T22:13:33.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.400+0000 7f9f79853700 1 -- start start 2024-02-16T22:13:33.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.400+0000 7f9f79853700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f74067fd0 0x7f9f74197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:13:33.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.400+0000 7f9f79853700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f74198490 con 0x7f9f74067fd0 2024-02-16T22:13:33.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.401+0000 7f9f72ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f74067fd0 0x7f9f74197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:13:33.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.401+0000 7f9f72ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f74067fd0 0x7f9f74197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.123:56790/0 (socket says 172.21.15.123:56790) 2024-02-16T22:13:33.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.401+0000 7f9f72ffd700 1 -- 172.21.15.123:0/585147868 learned_addr learned my addr 172.21.15.123:0/585147868 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-02-16T22:13:33.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.401+0000 7f9f72ffd700 1 -- 172.21.15.123:0/585147868 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f640095f0 con 0x7f9f74067fd0 2024-02-16T22:13:33.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.401+0000 7f9f72ffd700 1 --2- 172.21.15.123:0/585147868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f74067fd0 0x7f9f74197f50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9f64005b40 tx=0x7f9f64004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:13:33.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.401+0000 7f9f78851700 1 -- 172.21.15.123:0/585147868 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9f6401d070 con 0x7f9f74067fd0 2024-02-16T22:13:33.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.401+0000 7f9f78851700 1 -- 172.21.15.123:0/585147868 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9f640044d0 con 0x7f9f74067fd0 2024-02-16T22:13:33.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.401+0000 7f9f78851700 1 -- 172.21.15.123:0/585147868 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9f64003930 con 0x7f9f74067fd0 2024-02-16T22:13:33.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.401+0000 7f9f79853700 1 -- 172.21.15.123:0/585147868 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f74198690 con 0x7f9f74067fd0 2024-02-16T22:13:33.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.401+0000 7f9f79853700 1 -- 172.21.15.123:0/585147868 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f74198a50 con 0x7f9f74067fd0 2024-02-16T22:13:33.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.402+0000 7f9f79853700 1 -- 172.21.15.123:0/585147868 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f740623c0 con 0x7f9f74067fd0 2024-02-16T22:13:33.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.402+0000 7f9f78851700 1 -- 172.21.15.123:0/585147868 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7f9f6400b960 con 0x7f9f74067fd0 2024-02-16T22:13:33.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.402+0000 7f9f78851700 1 --2- 172.21.15.123:0/585147868 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f9f600384b0 0x7f9f6003a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:13:33.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.402+0000 7f9f78851700 1 -- 172.21.15.123:0/585147868 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f9f6404c510 con 0x7f9f74067fd0 2024-02-16T22:13:33.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.403+0000 7f9f6a7fc700 1 --2- 172.21.15.123:0/585147868 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f9f600384b0 0x7f9f6003a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:13:33.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.403+0000 7f9f6a7fc700 1 --2- 172.21.15.123:0/585147868 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f9f600384b0 0x7f9f6003a970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c006fd0 tx=0x7f9f5c006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:13:33.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.407+0000 7f9f78851700 1 -- 172.21.15.123:0/585147868 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9f6402a540 con 0x7f9f74067fd0 2024-02-16T22:13:33.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.676+0000 7f9f79853700 1 -- 172.21.15.123:0/585147868 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9f7410a030 con 0x7f9f74067fd0 2024-02-16T22:13:33.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.677+0000 7f9f78851700 1 -- 172.21.15.123:0/585147868 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f9f6402a360 con 0x7f9f74067fd0 2024-02-16T22:13:33.679 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:13:33.679 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":1,"fsid":"1dcbaabe-cd18-11ee-95ba-87774f69a715","modified":"2024-02-16T22:10:59.330677Z","created":"2024-02-16T22:10:59.330677Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi114","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:3300","nonce":0},{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:13:33.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.679+0000 7f9f79853700 1 -- 172.21.15.123:0/585147868 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f9f600384b0 msgr2=0x7f9f6003a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:33.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.679+0000 7f9f79853700 1 --2- 172.21.15.123:0/585147868 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f9f600384b0 0x7f9f6003a970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c006fd0 tx=0x7f9f5c006e40 comp rx=0 tx=0).stop 2024-02-16T22:13:33.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.679+0000 7f9f79853700 1 -- 172.21.15.123:0/585147868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f74067fd0 msgr2=0x7f9f74197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:33.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.679+0000 7f9f79853700 1 --2- 172.21.15.123:0/585147868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f74067fd0 0x7f9f74197f50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9f64005b40 tx=0x7f9f64004c80 comp rx=0 tx=0).stop 2024-02-16T22:13:33.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.679+0000 7f9f79853700 1 -- 172.21.15.123:0/585147868 shutdown_connections 2024-02-16T22:13:33.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.679+0000 7f9f79853700 1 --2- 172.21.15.123:0/585147868 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f9f600384b0 0x7f9f6003a970 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:33.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.679+0000 7f9f79853700 1 --2- 172.21.15.123:0/585147868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f74067fd0 0x7f9f74197f50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:33.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.679+0000 7f9f79853700 1 -- 172.21.15.123:0/585147868 >> 172.21.15.123:0/585147868 conn(0x7f9f740752b0 msgr2=0x7f9f740fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:13:33.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.679+0000 7f9f79853700 1 -- 172.21.15.123:0/585147868 shutdown_connections 2024-02-16T22:13:33.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:33.679+0000 7f9f79853700 1 -- 172.21.15.123:0/585147868 wait complete. 2024-02-16T22:13:33.682 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 1 2024-02-16T22:13:33.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:33 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:33.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:33 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:33.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:33 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:33.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:33 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:33.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:33 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi123.inzgjd", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:13:33.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:33 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi123.inzgjd", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-16T22:13:33.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:33 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T22:13:33.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:33 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:13:33.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:33 smithi114 ceph-mon[29235]: Deploying daemon mgr.smithi123.inzgjd on smithi123 2024-02-16T22:13:34.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:34 smithi114 ceph-mon[29235]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:13:34.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:34 smithi114 ceph-mon[29235]: from='client.? 172.21.15.123:0/585147868' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:13:35.588 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:13:35.588 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:13:35.978 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:13:36.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:36 smithi114 ceph-mon[29235]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:13:36.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:36 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:36.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:36 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:36.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:36 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:36.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:36 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:36.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:36 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T22:13:36.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:36 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:13:36.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:36 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:37.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.605+0000 7f4f69233700 1 -- 172.21.15.123:0/1271093813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f64106820 msgr2=0x7f4f64106c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:37.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.605+0000 7f4f69233700 1 --2- 172.21.15.123:0/1271093813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f64106820 0x7f4f64106c00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009910 tx=0x7f4f54009c20 comp rx=0 tx=0).stop 2024-02-16T22:13:37.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.605+0000 7f4f69233700 1 -- 172.21.15.123:0/1271093813 shutdown_connections 2024-02-16T22:13:37.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.605+0000 7f4f69233700 1 --2- 172.21.15.123:0/1271093813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f64106820 0x7f4f64106c00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:37.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.605+0000 7f4f69233700 1 -- 172.21.15.123:0/1271093813 >> 172.21.15.123:0/1271093813 conn(0x7f4f640752b0 msgr2=0x7f4f640756c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:13:37.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.605+0000 7f4f69233700 1 -- 172.21.15.123:0/1271093813 shutdown_connections 2024-02-16T22:13:37.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.605+0000 7f4f69233700 1 -- 172.21.15.123:0/1271093813 wait complete. 2024-02-16T22:13:37.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.606+0000 7f4f69233700 1 Processor -- start 2024-02-16T22:13:37.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.606+0000 7f4f69233700 1 -- start start 2024-02-16T22:13:37.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.606+0000 7f4f69233700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f64106820 0x7f4f640ff800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:13:37.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.606+0000 7f4f69233700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f64101780 con 0x7f4f64106820 2024-02-16T22:13:37.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.607+0000 7f4f62d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f64106820 0x7f4f640ff800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:13:37.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.607+0000 7f4f62d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f64106820 0x7f4f640ff800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.123:56824/0 (socket says 172.21.15.123:56824) 2024-02-16T22:13:37.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.607+0000 7f4f62d9d700 1 -- 172.21.15.123:0/3341323707 learned_addr learned my addr 172.21.15.123:0/3341323707 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-02-16T22:13:37.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.607+0000 7f4f62d9d700 1 -- 172.21.15.123:0/3341323707 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f540095f0 con 0x7f4f64106820 2024-02-16T22:13:37.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.607+0000 7f4f62d9d700 1 --2- 172.21.15.123:0/3341323707 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f64106820 0x7f4f640ff800 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4f54005b40 tx=0x7f4f54004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:13:37.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.607+0000 7f4f4bfff700 1 -- 172.21.15.123:0/3341323707 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4f5401d070 con 0x7f4f64106820 2024-02-16T22:13:37.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.607+0000 7f4f69233700 1 -- 172.21.15.123:0/3341323707 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f640ffd40 con 0x7f4f64106820 2024-02-16T22:13:37.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.607+0000 7f4f4bfff700 1 -- 172.21.15.123:0/3341323707 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4f540044d0 con 0x7f4f64106820 2024-02-16T22:13:37.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.607+0000 7f4f4bfff700 1 -- 172.21.15.123:0/3341323707 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4f540039a0 con 0x7f4f64106820 2024-02-16T22:13:37.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.607+0000 7f4f69233700 1 -- 172.21.15.123:0/3341323707 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f641001c0 con 0x7f4f64106820 2024-02-16T22:13:37.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.608+0000 7f4f4bfff700 1 -- 172.21.15.123:0/3341323707 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7f4f54003b00 con 0x7f4f64106820 2024-02-16T22:13:37.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.608+0000 7f4f69233700 1 -- 172.21.15.123:0/3341323707 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f64105ad0 con 0x7f4f64106820 2024-02-16T22:13:37.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.608+0000 7f4f4bfff700 1 --2- 172.21.15.123:0/3341323707 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f4f4c0384b0 0x7f4f4c03a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:13:37.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.608+0000 7f4f4bfff700 1 -- 172.21.15.123:0/3341323707 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f4f5404c460 con 0x7f4f64106820 2024-02-16T22:13:37.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.609+0000 7f4f6259c700 1 --2- 172.21.15.123:0/3341323707 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f4f4c0384b0 0x7f4f4c03a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:13:37.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.609+0000 7f4f6259c700 1 --2- 172.21.15.123:0/3341323707 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f4f4c0384b0 0x7f4f4c03a970 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4f58006fd0 tx=0x7f4f58006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:13:37.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.614+0000 7f4f4bfff700 1 -- 172.21.15.123:0/3341323707 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4f5402a950 con 0x7f4f64106820 2024-02-16T22:13:37.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.888+0000 7f4f69233700 1 -- 172.21.15.123:0/3341323707 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4f64100500 con 0x7f4f64106820 2024-02-16T22:13:37.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.889+0000 7f4f4bfff700 1 -- 172.21.15.123:0/3341323707 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f4f5402a360 con 0x7f4f64106820 2024-02-16T22:13:37.891 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:13:37.891 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":1,"fsid":"1dcbaabe-cd18-11ee-95ba-87774f69a715","modified":"2024-02-16T22:10:59.330677Z","created":"2024-02-16T22:10:59.330677Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi114","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:3300","nonce":0},{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T22:13:37.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.891+0000 7f4f69233700 1 -- 172.21.15.123:0/3341323707 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f4f4c0384b0 msgr2=0x7f4f4c03a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:37.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.891+0000 7f4f69233700 1 --2- 172.21.15.123:0/3341323707 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f4f4c0384b0 0x7f4f4c03a970 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4f58006fd0 tx=0x7f4f58006e40 comp rx=0 tx=0).stop 2024-02-16T22:13:37.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.891+0000 7f4f69233700 1 -- 172.21.15.123:0/3341323707 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f64106820 msgr2=0x7f4f640ff800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:37.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.891+0000 7f4f69233700 1 --2- 172.21.15.123:0/3341323707 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f64106820 0x7f4f640ff800 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4f54005b40 tx=0x7f4f54004c80 comp rx=0 tx=0).stop 2024-02-16T22:13:37.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.891+0000 7f4f69233700 1 -- 172.21.15.123:0/3341323707 shutdown_connections 2024-02-16T22:13:37.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.891+0000 7f4f69233700 1 --2- 172.21.15.123:0/3341323707 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f4f4c0384b0 0x7f4f4c03a970 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:37.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.891+0000 7f4f69233700 1 --2- 172.21.15.123:0/3341323707 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f64106820 0x7f4f640ff800 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:37.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.891+0000 7f4f69233700 1 -- 172.21.15.123:0/3341323707 >> 172.21.15.123:0/3341323707 conn(0x7f4f640752b0 msgr2=0x7f4f640fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:13:37.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.891+0000 7f4f69233700 1 -- 172.21.15.123:0/3341323707 shutdown_connections 2024-02-16T22:13:37.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:37.891+0000 7f4f69233700 1 -- 172.21.15.123:0/3341323707 wait complete. 2024-02-16T22:13:37.894 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 1 2024-02-16T22:13:37.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:37 smithi114 ceph-mon[29235]: Deploying daemon mon.smithi123 on smithi123 2024-02-16T22:13:38.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:38 smithi114 ceph-mon[29235]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:13:38.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:38 smithi114 ceph-mon[29235]: from='client.? 172.21.15.123:0/3341323707' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:13:40.492 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T22:13:40.493 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T22:13:40.715 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi123/config 2024-02-16T22:13:40.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:40 smithi114 ceph-mon[29235]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:13:41.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:41 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:13:47.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:47 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi114"}]: dispatch 2024-02-16T22:13:47.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:47 smithi114 ceph-mon[29235]: mon.smithi114 calling monitor election 2024-02-16T22:13:47.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:47 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi123"}]: dispatch 2024-02-16T22:13:47.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:47 smithi114 ceph-mon[29235]: from='mgr.? 172.21.15.123:0/1700346899' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi123.inzgjd/crt"}]: dispatch 2024-02-16T22:13:47.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:47 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi123"}]: dispatch 2024-02-16T22:13:47.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:47 smithi114 ceph-mon[29235]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:13:47.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:47 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi123"}]: dispatch 2024-02-16T22:13:47.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:47 smithi114 ceph-mon[29235]: mon.smithi123 calling monitor election 2024-02-16T22:13:47.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:47 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi123"}]: dispatch 2024-02-16T22:13:47.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:47 smithi114 ceph-mon[29235]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:13:47.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:47 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi123"}]: dispatch 2024-02-16T22:13:47.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:47 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi123"}]: dispatch 2024-02-16T22:13:47.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:47 smithi114 ceph-mon[29235]: mon.smithi114 is new leader, mons smithi114,smithi123 in quorum (ranks 0,1) 2024-02-16T22:13:47.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:47 smithi114 ceph-mon[29235]: monmap e2: 2 mons at {smithi114=[v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0],smithi123=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-16T22:13:47.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:47 smithi114 ceph-mon[29235]: fsmap 2024-02-16T22:13:47.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:47 smithi114 ceph-mon[29235]: osdmap e5: 0 total, 0 up, 0 in 2024-02-16T22:13:47.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:47 smithi114 ceph-mon[29235]: mgrmap e17: smithi114.vcullz(active, since 35s) 2024-02-16T22:13:47.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:47 smithi114 ceph-mon[29235]: Standby manager daemon smithi123.inzgjd started 2024-02-16T22:13:47.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:47 smithi114 ceph-mon[29235]: from='mgr.? 172.21.15.123:0/1700346899' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T22:13:47.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:47 smithi114 ceph-mon[29235]: overall HEALTH_OK 2024-02-16T22:13:47.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:47 smithi114 ceph-mon[29235]: from='mgr.? 172.21.15.123:0/1700346899' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi123.inzgjd/key"}]: dispatch 2024-02-16T22:13:47.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:47 smithi114 ceph-mon[29235]: from='mgr.? 172.21.15.123:0/1700346899' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T22:13:47.704 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:47 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:47.704 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:47 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:47.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:47 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:48.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:48 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:48.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:48 smithi123 ceph-mon[37161]: mgrmap e18: smithi114.vcullz(active, since 36s), standbys: smithi123.inzgjd 2024-02-16T22:13:48.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:48 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mgr metadata", "who": "smithi123.inzgjd", "id": "smithi123.inzgjd"}]: dispatch 2024-02-16T22:13:48.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:48 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:48.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:48 smithi123 ceph-mon[37161]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:13:48.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:48 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:48.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:48 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:13:48.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:48 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi123"}]: dispatch 2024-02-16T22:13:48.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.551+0000 7f0ee44cc700 1 -- 172.21.15.123:0/531862545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ec8008490 msgr2=0x7f0ec800a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:48.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.551+0000 7f0ee44cc700 1 --2- 172.21.15.123:0/531862545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ec8008490 0x7f0ec800a8a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0edc100900 tx=0x7f0ed800fe90 comp rx=0 tx=0).stop 2024-02-16T22:13:48.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.552+0000 7f0ee44cc700 1 -- 172.21.15.123:0/531862545 shutdown_connections 2024-02-16T22:13:48.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.552+0000 7f0ee44cc700 1 --2- 172.21.15.123:0/531862545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ec8008490 0x7f0ec800a8a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:48.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.552+0000 7f0ee44cc700 1 -- 172.21.15.123:0/531862545 >> 172.21.15.123:0/531862545 conn(0x7f0edc0fb350 msgr2=0x7f0edc0fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:13:48.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.552+0000 7f0ee44cc700 1 -- 172.21.15.123:0/531862545 shutdown_connections 2024-02-16T22:13:48.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.552+0000 7f0ee44cc700 1 -- 172.21.15.123:0/531862545 wait complete. 2024-02-16T22:13:48.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.552+0000 7f0ee44cc700 1 Processor -- start 2024-02-16T22:13:48.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.552+0000 7f0ee44cc700 1 -- start start 2024-02-16T22:13:48.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.553+0000 7f0ee44cc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ec8008490 0x7f0edc193e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:13:48.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.553+0000 7f0ee44cc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0edc194370 0x7f0edc1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:13:48.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.553+0000 7f0ee44cc700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc194990 con 0x7f0edc194370 2024-02-16T22:13:48.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.553+0000 7f0ee44cc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc194b00 con 0x7f0ec8008490 2024-02-16T22:13:48.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.553+0000 7f0ee2268700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ec8008490 0x7f0edc193e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:13:48.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.553+0000 7f0ee2268700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ec8008490 0x7f0edc193e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55718/0 (socket says 172.21.15.123:55718) 2024-02-16T22:13:48.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.553+0000 7f0ee2268700 1 -- 172.21.15.123:0/1258537335 learned_addr learned my addr 172.21.15.123:0/1258537335 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-02-16T22:13:48.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.553+0000 7f0ee2268700 1 -- 172.21.15.123:0/1258537335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ec8008490 msgr2=0x7f0edc193e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-16T22:13:48.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.553+0000 7f0ee1a67700 1 --2- 172.21.15.123:0/1258537335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0edc194370 0x7f0edc1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:13:48.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.553+0000 7f0ee2268700 1 -- 172.21.15.123:0/1258537335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ec8008490 msgr2=0x7f0edc193e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-16T22:13:48.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.553+0000 7f0ee2268700 1 --2- 172.21.15.123:0/1258537335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ec8008490 0x7f0edc193e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-16T22:13:48.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.553+0000 7f0ee2268700 1 --2- 172.21.15.123:0/1258537335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ec8008490 0x7f0edc193e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T22:13:48.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.553+0000 7f0ee1a67700 1 -- 172.21.15.123:0/1258537335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ec8008490 msgr2=0x7f0edc193e30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T22:13:48.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.553+0000 7f0ee1a67700 1 --2- 172.21.15.123:0/1258537335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ec8008490 0x7f0edc193e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:48.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.553+0000 7f0ee1a67700 1 -- 172.21.15.123:0/1258537335 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed800f8e0 con 0x7f0edc194370 2024-02-16T22:13:48.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.554+0000 7f0ee1a67700 1 --2- 172.21.15.123:0/1258537335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0edc194370 0x7f0edc1987e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc00b4d0 tx=0x7f0ecc00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:13:48.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.554+0000 7f0ed37fe700 1 -- 172.21.15.123:0/1258537335 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ecc011800 con 0x7f0edc194370 2024-02-16T22:13:48.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.554+0000 7f0ee44cc700 1 -- 172.21.15.123:0/1258537335 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0edc198de0 con 0x7f0edc194370 2024-02-16T22:13:48.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.554+0000 7f0ed37fe700 1 -- 172.21.15.123:0/1258537335 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0ecc011e10 con 0x7f0edc194370 2024-02-16T22:13:48.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.554+0000 7f0ed37fe700 1 -- 172.21.15.123:0/1258537335 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ecc00f2e0 con 0x7f0edc194370 2024-02-16T22:13:48.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.554+0000 7f0ee44cc700 1 -- 172.21.15.123:0/1258537335 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0edc199330 con 0x7f0edc194370 2024-02-16T22:13:48.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.555+0000 7f0ee44cc700 1 -- 172.21.15.123:0/1258537335 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0edc0623c0 con 0x7f0edc194370 2024-02-16T22:13:48.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.555+0000 7f0ed37fe700 1 -- 172.21.15.123:0/1258537335 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7f0ecc011960 con 0x7f0edc194370 2024-02-16T22:13:48.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.556+0000 7f0ed37fe700 1 --2- 172.21.15.123:0/1258537335 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f0ec006c5b0 0x7f0ec006ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:13:48.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.556+0000 7f0ed37fe700 1 -- 172.21.15.123:0/1258537335 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f0ecc08a3d0 con 0x7f0edc194370 2024-02-16T22:13:48.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.556+0000 7f0ee2268700 1 --2- 172.21.15.123:0/1258537335 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f0ec006c5b0 0x7f0ec006ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:13:48.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.556+0000 7f0ee2268700 1 --2- 172.21.15.123:0/1258537335 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f0ec006c5b0 0x7f0ec006ea70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8005ed0 tx=0x7f0ed8005da0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:13:48.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.560+0000 7f0ed37fe700 1 -- 172.21.15.123:0/1258537335 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0ecc055bf0 con 0x7f0edc194370 2024-02-16T22:13:48.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:48 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:48.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:48 smithi114 ceph-mon[29235]: mgrmap e18: smithi114.vcullz(active, since 36s), standbys: smithi123.inzgjd 2024-02-16T22:13:48.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:48 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mgr metadata", "who": "smithi123.inzgjd", "id": "smithi123.inzgjd"}]: dispatch 2024-02-16T22:13:48.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:48 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:48.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:48 smithi114 ceph-mon[29235]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:13:48.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:48 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:48.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:48 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:13:48.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:48 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi123"}]: dispatch 2024-02-16T22:13:48.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.846+0000 7f0ee44cc700 1 -- 172.21.15.123:0/1258537335 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0edc19ea80 con 0x7f0edc194370 2024-02-16T22:13:48.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.847+0000 7f0ed37fe700 1 -- 172.21.15.123:0/1258537335 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1026 (secure 0 0 0) 0x7f0ecc015090 con 0x7f0edc194370 2024-02-16T22:13:48.849 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-02-16T22:13:48.849 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":2,"fsid":"1dcbaabe-cd18-11ee-95ba-87774f69a715","modified":"2024-02-16T22:13:42.299381Z","created":"2024-02-16T22:10:59.330677Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi114","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:3300","nonce":0},{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi123","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-16T22:13:48.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.849+0000 7f0ee44cc700 1 -- 172.21.15.123:0/1258537335 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f0ec006c5b0 msgr2=0x7f0ec006ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:48.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.849+0000 7f0ee44cc700 1 --2- 172.21.15.123:0/1258537335 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f0ec006c5b0 0x7f0ec006ea70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8005ed0 tx=0x7f0ed8005da0 comp rx=0 tx=0).stop 2024-02-16T22:13:48.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.849+0000 7f0ee44cc700 1 -- 172.21.15.123:0/1258537335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0edc194370 msgr2=0x7f0edc1987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:48.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.849+0000 7f0ee44cc700 1 --2- 172.21.15.123:0/1258537335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0edc194370 0x7f0edc1987e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc00b4d0 tx=0x7f0ecc00b890 comp rx=0 tx=0).stop 2024-02-16T22:13:48.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.849+0000 7f0ee44cc700 1 -- 172.21.15.123:0/1258537335 shutdown_connections 2024-02-16T22:13:48.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.849+0000 7f0ee44cc700 1 --2- 172.21.15.123:0/1258537335 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f0ec006c5b0 0x7f0ec006ea70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:48.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.849+0000 7f0ee44cc700 1 --2- 172.21.15.123:0/1258537335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ec8008490 0x7f0edc193e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:48.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.849+0000 7f0ee44cc700 1 --2- 172.21.15.123:0/1258537335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0edc194370 0x7f0edc1987e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:48.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.849+0000 7f0ee44cc700 1 -- 172.21.15.123:0/1258537335 >> 172.21.15.123:0/1258537335 conn(0x7f0edc0fb350 msgr2=0x7f0edc101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:13:48.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.849+0000 7f0ee44cc700 1 -- 172.21.15.123:0/1258537335 shutdown_connections 2024-02-16T22:13:48.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:13:48.849+0000 7f0ee44cc700 1 -- 172.21.15.123:0/1258537335 wait complete. 2024-02-16T22:13:48.852 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 2 2024-02-16T22:13:49.496 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:49 smithi123 ceph-mon[37161]: from='client.? 172.21.15.123:0/1258537335' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:13:49.519 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-16T22:13:49.519 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph config generate-minimal-conf 2024-02-16T22:13:49.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:49 smithi114 ceph-mon[29235]: from='client.? 172.21.15.123:0/1258537335' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T22:13:49.735 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:13:50.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:50 smithi114 ceph-mon[29235]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:13:50.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:50 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:50.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:50 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:50.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:50 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:13:50.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:50 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:13:50.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:50 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:50.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:50 smithi123 ceph-mon[37161]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:13:50.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:50 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:50.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:50 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:50.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:50 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:13:50.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:50 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:13:50.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:50 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:51.694 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:51 smithi114 ceph-mon[29235]: Updating smithi114:/etc/ceph/ceph.conf 2024-02-16T22:13:51.694 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:51 smithi114 ceph-mon[29235]: Updating smithi123:/etc/ceph/ceph.conf 2024-02-16T22:13:51.695 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:51 smithi114 ceph-mon[29235]: Updating smithi114:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:13:51.695 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:51 smithi114 ceph-mon[29235]: Updating smithi123:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:13:51.695 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:51 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:51.695 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:51 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:51.695 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:51 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:51.695 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:51 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:51.695 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:51 smithi114 ceph-mon[29235]: Reconfiguring mon.smithi114 (unknown last config time)... 2024-02-16T22:13:51.695 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:51 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T22:13:51.695 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:51 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T22:13:51.695 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:51 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:13:51.695 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:51 smithi114 ceph-mon[29235]: Reconfiguring daemon mon.smithi114 on smithi114 2024-02-16T22:13:51.752 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:51 smithi123 ceph-mon[37161]: Updating smithi114:/etc/ceph/ceph.conf 2024-02-16T22:13:51.752 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:51 smithi123 ceph-mon[37161]: Updating smithi123:/etc/ceph/ceph.conf 2024-02-16T22:13:51.753 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:51 smithi123 ceph-mon[37161]: Updating smithi114:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:13:51.753 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:51 smithi123 ceph-mon[37161]: Updating smithi123:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:13:51.753 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:51 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:51.753 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:51 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:51.753 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:51 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:51.753 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:51 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:51.753 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:51 smithi123 ceph-mon[37161]: Reconfiguring mon.smithi114 (unknown last config time)... 2024-02-16T22:13:51.753 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:51 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T22:13:51.753 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:51 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T22:13:51.753 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:51 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:13:51.753 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:51 smithi123 ceph-mon[37161]: Reconfiguring daemon mon.smithi114 on smithi114 2024-02-16T22:13:52.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.021+0000 7f2cffc5e700 1 -- 172.21.15.114:0/1614218577 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cf8106810 msgr2=0x7f2cf8106bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:52.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.021+0000 7f2cffc5e700 1 --2- 172.21.15.114:0/1614218577 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cf8106810 0x7f2cf8106bf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4009910 tx=0x7f2cf4009c20 comp rx=0 tx=0).stop 2024-02-16T22:13:52.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.021+0000 7f2cffc5e700 1 -- 172.21.15.114:0/1614218577 shutdown_connections 2024-02-16T22:13:52.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.021+0000 7f2cffc5e700 1 --2- 172.21.15.114:0/1614218577 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cf8106810 0x7f2cf8106bf0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:52.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.021+0000 7f2cffc5e700 1 -- 172.21.15.114:0/1614218577 >> 172.21.15.114:0/1614218577 conn(0x7f2cf80fc320 msgr2=0x7f2cf80fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:13:52.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.021+0000 7f2cffc5e700 1 -- 172.21.15.114:0/1614218577 shutdown_connections 2024-02-16T22:13:52.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.021+0000 7f2cffc5e700 1 -- 172.21.15.114:0/1614218577 wait complete. 2024-02-16T22:13:52.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.021+0000 7f2cffc5e700 1 Processor -- start 2024-02-16T22:13:52.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.021+0000 7f2cffc5e700 1 -- start start 2024-02-16T22:13:52.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.021+0000 7f2cffc5e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2cf8106810 0x7f2cf8195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:13:52.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.022+0000 7f2cffc5e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cf81963f0 0x7f2cf819a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:13:52.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.022+0000 7f2cffc5e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf8196a10 con 0x7f2cf81963f0 2024-02-16T22:13:52.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.022+0000 7f2cffc5e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf8196b80 con 0x7f2cf8106810 2024-02-16T22:13:52.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.022+0000 7f2cfd1f9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cf81963f0 0x7f2cf819a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:13:52.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.022+0000 7f2cfd1f9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cf81963f0 0x7f2cf819a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37968/0 (socket says 172.21.15.114:37968) 2024-02-16T22:13:52.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.022+0000 7f2cfd1f9700 1 -- 172.21.15.114:0/406446593 learned_addr learned my addr 172.21.15.114:0/406446593 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:13:52.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.022+0000 7f2cfd9fa700 1 --2- 172.21.15.114:0/406446593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2cf8106810 0x7f2cf8195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:13:52.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.022+0000 7f2cfd1f9700 1 -- 172.21.15.114:0/406446593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2cf8106810 msgr2=0x7f2cf8195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:52.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.022+0000 7f2cfd1f9700 1 --2- 172.21.15.114:0/406446593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2cf8106810 0x7f2cf8195eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:52.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.022+0000 7f2cfd1f9700 1 -- 172.21.15.114:0/406446593 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf40095f0 con 0x7f2cf81963f0 2024-02-16T22:13:52.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.022+0000 7f2cfd1f9700 1 --2- 172.21.15.114:0/406446593 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cf81963f0 0x7f2cf819a860 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2ce800b700 tx=0x7f2ce800bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:13:52.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.022+0000 7f2ceeffd700 1 -- 172.21.15.114:0/406446593 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ce8011800 con 0x7f2cf81963f0 2024-02-16T22:13:52.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.022+0000 7f2cffc5e700 1 -- 172.21.15.114:0/406446593 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf819ae60 con 0x7f2cf81963f0 2024-02-16T22:13:52.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.022+0000 7f2ceeffd700 1 -- 172.21.15.114:0/406446593 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2ce8011e10 con 0x7f2cf81963f0 2024-02-16T22:13:52.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.022+0000 7f2ceeffd700 1 -- 172.21.15.114:0/406446593 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ce800f2e0 con 0x7f2cf81963f0 2024-02-16T22:13:52.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.022+0000 7f2cffc5e700 1 -- 172.21.15.114:0/406446593 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf819b3b0 con 0x7f2cf81963f0 2024-02-16T22:13:52.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.023+0000 7f2cffc5e700 1 -- 172.21.15.114:0/406446593 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cf819b690 con 0x7f2cf81963f0 2024-02-16T22:13:52.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.024+0000 7f2ceeffd700 1 -- 172.21.15.114:0/406446593 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7f2ce8010380 con 0x7f2cf81963f0 2024-02-16T22:13:52.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.024+0000 7f2ceeffd700 1 --2- 172.21.15.114:0/406446593 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f2ce406c5c0 0x7f2ce406ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:13:52.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.024+0000 7f2cfd9fa700 1 --2- 172.21.15.114:0/406446593 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f2ce406c5c0 0x7f2ce406ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:13:52.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.024+0000 7f2ceeffd700 1 -- 172.21.15.114:0/406446593 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f2ce808a400 con 0x7f2cf81963f0 2024-02-16T22:13:52.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.024+0000 7f2cfd9fa700 1 --2- 172.21.15.114:0/406446593 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f2ce406c5c0 0x7f2ce406ea80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4006010 tx=0x7f2cf400b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:13:52.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.029+0000 7f2ceeffd700 1 -- 172.21.15.114:0/406446593 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2ce8055cc0 con 0x7f2cf81963f0 2024-02-16T22:13:52.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.221+0000 7f2cffc5e700 1 -- 172.21.15.114:0/406446593 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f2cf81a0b00 con 0x7f2cf81963f0 2024-02-16T22:13:52.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.221+0000 7f2ceeffd700 1 -- 172.21.15.114:0/406446593 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+227 (secure 0 0 0) 0x7f2ce8089bc0 con 0x7f2cf81963f0 2024-02-16T22:13:52.223 INFO:teuthology.orchestra.run.smithi114.stdout:# minimal ceph.conf for 1dcbaabe-cd18-11ee-95ba-87774f69a715 2024-02-16T22:13:52.224 INFO:teuthology.orchestra.run.smithi114.stdout:[global] 2024-02-16T22:13:52.224 INFO:teuthology.orchestra.run.smithi114.stdout: fsid = 1dcbaabe-cd18-11ee-95ba-87774f69a715 2024-02-16T22:13:52.224 INFO:teuthology.orchestra.run.smithi114.stdout: mon_host = [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] 2024-02-16T22:13:52.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.224+0000 7f2cffc5e700 1 -- 172.21.15.114:0/406446593 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f2ce406c5c0 msgr2=0x7f2ce406ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:52.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.224+0000 7f2cffc5e700 1 --2- 172.21.15.114:0/406446593 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f2ce406c5c0 0x7f2ce406ea80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4006010 tx=0x7f2cf400b540 comp rx=0 tx=0).stop 2024-02-16T22:13:52.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.224+0000 7f2cffc5e700 1 -- 172.21.15.114:0/406446593 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cf81963f0 msgr2=0x7f2cf819a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:13:52.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.224+0000 7f2cffc5e700 1 --2- 172.21.15.114:0/406446593 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cf81963f0 0x7f2cf819a860 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2ce800b700 tx=0x7f2ce800bac0 comp rx=0 tx=0).stop 2024-02-16T22:13:52.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.224+0000 7f2cffc5e700 1 -- 172.21.15.114:0/406446593 shutdown_connections 2024-02-16T22:13:52.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.224+0000 7f2cffc5e700 1 --2- 172.21.15.114:0/406446593 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f2ce406c5c0 0x7f2ce406ea80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:52.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.224+0000 7f2cffc5e700 1 --2- 172.21.15.114:0/406446593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2cf8106810 0x7f2cf8195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:52.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.224+0000 7f2cffc5e700 1 --2- 172.21.15.114:0/406446593 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cf81963f0 0x7f2cf819a860 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:13:52.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.224+0000 7f2cffc5e700 1 -- 172.21.15.114:0/406446593 >> 172.21.15.114:0/406446593 conn(0x7f2cf80fc320 msgr2=0x7f2cf8102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:13:52.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.224+0000 7f2cffc5e700 1 -- 172.21.15.114:0/406446593 shutdown_connections 2024-02-16T22:13:52.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:13:52.225+0000 7f2cffc5e700 1 -- 172.21.15.114:0/406446593 wait complete. 2024-02-16T22:13:52.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:52 smithi123 ceph-mon[37161]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:13:52.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:52 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:52.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:52 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/406446593' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:13:52.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:52 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:52.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:52 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:52.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:52 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi114.vcullz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:13:52.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:52 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T22:13:52.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:52 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:13:52.870 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:52 smithi114 ceph-mon[29235]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:13:52.870 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:52 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:52.870 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:52 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/406446593' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:13:52.870 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:52 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:52.871 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:52 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:52.871 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:52 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi114.vcullz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:13:52.871 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:52 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T22:13:52.871 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:52 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:13:52.983 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-16T22:13:52.983 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-02-16T22:13:52.983 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/ceph/ceph.conf 2024-02-16T22:13:53.013 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-02-16T22:13:53.013 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:13:53.082 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-02-16T22:13:53.082 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.conf 2024-02-16T22:13:53.117 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-02-16T22:13:53.117 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:13:53.189 INFO:tasks.cephadm:Deploying OSDs... 2024-02-16T22:13:53.189 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-02-16T22:13:53.189 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/scratch_devs of=/dev/stdout 2024-02-16T22:13:53.205 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-16T22:13:53.205 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_1 2024-02-16T22:13:53.261 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-16T22:13:53.261 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T22:13:53.261 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-02-16T22:13:53.261 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T22:13:53.261 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T22:13:53.261 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-02-16 22:13:02.979017139 +0000 2024-02-16T22:13:53.261 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-02-16 22:06:47.231663320 +0000 2024-02-16T22:13:53.262 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-02-16 22:06:47.231663320 +0000 2024-02-16T22:13:53.262 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-02-16 22:06:47.231663320 +0000 2024-02-16T22:13:53.262 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-16T22:13:53.328 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-02-16T22:13:53.328 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-02-16T22:13:53.328 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000208914 s, 2.5 MB/s 2024-02-16T22:13:53.329 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-16T22:13:53.386 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_2 2024-02-16T22:13:53.441 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-16T22:13:53.441 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T22:13:53.441 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-02-16T22:13:53.441 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T22:13:53.442 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T22:13:53.442 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-02-16 22:13:02.979017139 +0000 2024-02-16T22:13:53.442 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-02-16 22:06:47.562656202 +0000 2024-02-16T22:13:53.442 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-02-16 22:06:47.562656202 +0000 2024-02-16T22:13:53.442 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-02-16 22:06:47.562656202 +0000 2024-02-16T22:13:53.442 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-16T22:13:53.512 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-02-16T22:13:53.512 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-02-16T22:13:53.512 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000198201 s, 2.6 MB/s 2024-02-16T22:13:53.514 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-16T22:13:53.571 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_3 2024-02-16T22:13:53.626 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-16T22:13:53.626 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T22:13:53.626 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-02-16T22:13:53.626 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T22:13:53.627 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T22:13:53.627 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-02-16 22:13:02.980017117 +0000 2024-02-16T22:13:53.627 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-02-16 22:06:47.899648955 +0000 2024-02-16T22:13:53.627 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-02-16 22:06:47.899648955 +0000 2024-02-16T22:13:53.627 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-02-16 22:06:47.899648955 +0000 2024-02-16T22:13:53.627 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-16T22:13:53.693 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-02-16T22:13:53.693 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-02-16T22:13:53.693 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000179549 s, 2.9 MB/s 2024-02-16T22:13:53.694 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-16T22:13:53.738 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:53 smithi114 ceph-mon[29235]: Reconfiguring mgr.smithi114.vcullz (unknown last config time)... 2024-02-16T22:13:53.738 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:53 smithi114 ceph-mon[29235]: Reconfiguring daemon mgr.smithi114.vcullz on smithi114 2024-02-16T22:13:53.750 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_4 2024-02-16T22:13:53.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:53 smithi123 ceph-mon[37161]: Reconfiguring mgr.smithi114.vcullz (unknown last config time)... 2024-02-16T22:13:53.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:53 smithi123 ceph-mon[37161]: Reconfiguring daemon mgr.smithi114.vcullz on smithi114 2024-02-16T22:13:53.806 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-16T22:13:53.806 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T22:13:53.806 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-02-16T22:13:53.806 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T22:13:53.806 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T22:13:53.806 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-02-16 22:13:02.980017117 +0000 2024-02-16T22:13:53.806 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-02-16 22:06:48.286640633 +0000 2024-02-16T22:13:53.806 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-02-16 22:06:48.286640633 +0000 2024-02-16T22:13:53.806 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-02-16 22:06:48.286640633 +0000 2024-02-16T22:13:53.806 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-16T22:13:53.872 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-02-16T22:13:53.872 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-02-16T22:13:53.873 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.00018918 s, 2.7 MB/s 2024-02-16T22:13:53.874 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-16T22:13:53.930 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-02-16T22:13:53.930 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/scratch_devs of=/dev/stdout 2024-02-16T22:13:53.948 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-16T22:13:53.948 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_1 2024-02-16T22:13:54.005 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-16T22:13:54.005 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T22:13:54.005 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-02-16T22:13:54.005 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T22:13:54.005 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T22:13:54.005 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-02-16 22:13:40.856949473 +0000 2024-02-16T22:13:54.005 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-02-16 22:06:47.226922634 +0000 2024-02-16T22:13:54.005 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-02-16 22:06:47.226922634 +0000 2024-02-16T22:13:54.006 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-02-16 22:06:47.226922634 +0000 2024-02-16T22:13:54.006 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-16T22:13:54.073 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-02-16T22:13:54.073 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-02-16T22:13:54.073 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000203508 s, 2.5 MB/s 2024-02-16T22:13:54.075 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-16T22:13:54.131 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_2 2024-02-16T22:13:54.187 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-16T22:13:54.187 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T22:13:54.187 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-02-16T22:13:54.187 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T22:13:54.188 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T22:13:54.188 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-02-16 22:13:40.856949473 +0000 2024-02-16T22:13:54.188 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-02-16 22:06:47.601913722 +0000 2024-02-16T22:13:54.188 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-02-16 22:06:47.601913722 +0000 2024-02-16T22:13:54.188 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-02-16 22:06:47.601913722 +0000 2024-02-16T22:13:54.188 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-16T22:13:54.254 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-02-16T22:13:54.254 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-02-16T22:13:54.254 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000207143 s, 2.5 MB/s 2024-02-16T22:13:54.256 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-16T22:13:54.313 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_3 2024-02-16T22:13:54.369 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-16T22:13:54.369 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T22:13:54.369 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-02-16T22:13:54.369 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T22:13:54.369 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T22:13:54.369 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-02-16 22:13:40.856949473 +0000 2024-02-16T22:13:54.369 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-02-16 22:06:47.949905453 +0000 2024-02-16T22:13:54.369 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-02-16 22:06:47.949905453 +0000 2024-02-16T22:13:54.369 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-02-16 22:06:47.949905453 +0000 2024-02-16T22:13:54.370 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-16T22:13:54.435 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-02-16T22:13:54.435 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-02-16T22:13:54.435 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000251046 s, 2.0 MB/s 2024-02-16T22:13:54.436 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-16T22:13:54.492 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_4 2024-02-16T22:13:54.547 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-16T22:13:54.547 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T22:13:54.547 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-02-16T22:13:54.547 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T22:13:54.547 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T22:13:54.547 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-02-16 22:13:40.857949449 +0000 2024-02-16T22:13:54.548 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-02-16 22:06:48.365895567 +0000 2024-02-16T22:13:54.548 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-02-16 22:06:48.365895567 +0000 2024-02-16T22:13:54.548 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-02-16 22:06:48.365895567 +0000 2024-02-16T22:13:54.548 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-16T22:13:54.613 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-02-16T22:13:54.613 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-02-16T22:13:54.613 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000206544 s, 2.5 MB/s 2024-02-16T22:13:54.614 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-16T22:13:54.670 INFO:tasks.cephadm:Deploying osd.0 on smithi114 with /dev/vg_nvme/lv_4... 2024-02-16T22:13:54.670 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-16T22:13:54.747 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:54 smithi114 ceph-mon[29235]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:13:54.747 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:54 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:54.747 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:54 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:54.747 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:54 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi114", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T22:13:54.747 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:54 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:13:54.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:54 smithi123 ceph-mon[37161]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:13:54.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:54 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:54.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:54 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:54.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:54 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi114", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T22:13:54.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:54 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:13:55.768 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:55 smithi114 ceph-mon[29235]: Reconfiguring ceph-exporter.smithi114 (monmap changed)... 2024-02-16T22:13:55.768 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:55 smithi114 ceph-mon[29235]: Reconfiguring daemon ceph-exporter.smithi114 on smithi114 2024-02-16T22:13:55.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:55 smithi123 ceph-mon[37161]: Reconfiguring ceph-exporter.smithi114 (monmap changed)... 2024-02-16T22:13:55.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:55 smithi123 ceph-mon[37161]: Reconfiguring daemon ceph-exporter.smithi114 on smithi114 2024-02-16T22:13:55.813 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:13:56.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:56 smithi123 ceph-mon[37161]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:13:56.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:56 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:56.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:56 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:56.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:56 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi114", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T22:13:56.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:56 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:13:56.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:56 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:13:56.916 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:56 smithi114 ceph-mon[29235]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:13:56.916 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:56 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:56.916 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:56 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:13:56.916 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:56 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi114", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T22:13:56.916 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:56 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:13:56.916 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:56 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:13:57.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:57 smithi123 ceph-mon[37161]: Reconfiguring crash.smithi114 (monmap changed)... 2024-02-16T22:13:57.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:57 smithi123 ceph-mon[37161]: Reconfiguring daemon crash.smithi114 on smithi114 2024-02-16T22:13:57.832 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:57 smithi114 ceph-mon[29235]: Reconfiguring crash.smithi114 (monmap changed)... 2024-02-16T22:13:57.832 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:57 smithi114 ceph-mon[29235]: Reconfiguring daemon crash.smithi114 on smithi114 2024-02-16T22:13:57.990 INFO:teuthology.orchestra.run.smithi114.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-16T22:13:57.990 INFO:teuthology.orchestra.run.smithi114.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-16T22:13:57.990 INFO:teuthology.orchestra.run.smithi114.stderr: stderr: 10+0 records in 2024-02-16T22:13:57.990 INFO:teuthology.orchestra.run.smithi114.stderr:10+0 records out 2024-02-16T22:13:57.990 INFO:teuthology.orchestra.run.smithi114.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0119149 s, 880 MB/s 2024-02-16T22:13:57.990 INFO:teuthology.orchestra.run.smithi114.stderr:--> Zapping successful for: 2024-02-16T22:13:58.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:13:58 smithi114 ceph-mon[29235]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:13:58.725 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi114:vg_nvme/lv_4 2024-02-16T22:13:58.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:13:58 smithi123 ceph-mon[37161]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:13:59.265 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:14:00.395 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:00 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:00.395 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:00 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:00.395 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:00 smithi114 ceph-mon[29235]: Reconfiguring alertmanager.smithi114 (dependencies changed)... 2024-02-16T22:14:00.395 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:00 smithi114 ceph-mon[29235]: Reconfiguring daemon alertmanager.smithi114 on smithi114 2024-02-16T22:14:00.395 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:00 smithi114 ceph-mon[29235]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:00.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:00 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:00.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:00 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:00.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:00 smithi123 ceph-mon[37161]: Reconfiguring alertmanager.smithi114 (dependencies changed)... 2024-02-16T22:14:00.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:00 smithi123 ceph-mon[37161]: Reconfiguring daemon alertmanager.smithi114 on smithi114 2024-02-16T22:14:00.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:00 smithi123 ceph-mon[37161]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:02.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:02 smithi114 ceph-mon[29235]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:02.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:02 smithi123 ceph-mon[37161]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:03.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.121+0000 7f38b8f11700 1 -- 172.21.15.114:0/985963401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38b40ffad0 msgr2=0x7f38b40ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:14:03.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.121+0000 7f38b8f11700 1 --2- 172.21.15.114:0/985963401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38b40ffad0 0x7f38b40ffef0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f38a4009910 tx=0x7f38a4009c20 comp rx=0 tx=0).stop 2024-02-16T22:14:03.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.121+0000 7f38b8f11700 1 -- 172.21.15.114:0/985963401 shutdown_connections 2024-02-16T22:14:03.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.121+0000 7f38b8f11700 1 --2- 172.21.15.114:0/985963401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38b4100cd0 0x7f38b4101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:14:03.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.122+0000 7f38b8f11700 1 --2- 172.21.15.114:0/985963401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38b40ffad0 0x7f38b40ffef0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:14:03.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.122+0000 7f38b8f11700 1 -- 172.21.15.114:0/985963401 >> 172.21.15.114:0/985963401 conn(0x7f38b40fb050 msgr2=0x7f38b40fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:14:03.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.122+0000 7f38b8f11700 1 -- 172.21.15.114:0/985963401 shutdown_connections 2024-02-16T22:14:03.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.122+0000 7f38b8f11700 1 -- 172.21.15.114:0/985963401 wait complete. 2024-02-16T22:14:03.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.122+0000 7f38b8f11700 1 Processor -- start 2024-02-16T22:14:03.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.122+0000 7f38b8f11700 1 -- start start 2024-02-16T22:14:03.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.123+0000 7f38b8f11700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38b40ffad0 0x7f38b4069890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:14:03.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.123+0000 7f38b8f11700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38b4100cd0 0x7f38b419b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:14:03.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.123+0000 7f38b8f11700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b419b750 con 0x7f38b4100cd0 2024-02-16T22:14:03.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.123+0000 7f38b8f11700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b419b890 con 0x7f38b40ffad0 2024-02-16T22:14:03.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.123+0000 7f38b1d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38b4100cd0 0x7f38b419b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:14:03.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.123+0000 7f38b1d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38b4100cd0 0x7f38b419b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:56178/0 (socket says 172.21.15.114:56178) 2024-02-16T22:14:03.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.123+0000 7f38b1d9b700 1 -- 172.21.15.114:0/3713133506 learned_addr learned my addr 172.21.15.114:0/3713133506 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:14:03.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.123+0000 7f38b259c700 1 --2- 172.21.15.114:0/3713133506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38b40ffad0 0x7f38b4069890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:14:03.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.123+0000 7f38b1d9b700 1 -- 172.21.15.114:0/3713133506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38b40ffad0 msgr2=0x7f38b4069890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:14:03.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.123+0000 7f38b1d9b700 1 --2- 172.21.15.114:0/3713133506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38b40ffad0 0x7f38b4069890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:14:03.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.123+0000 7f38b1d9b700 1 -- 172.21.15.114:0/3713133506 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a40095f0 con 0x7f38b4100cd0 2024-02-16T22:14:03.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.123+0000 7f38b259c700 1 --2- 172.21.15.114:0/3713133506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38b40ffad0 0x7f38b4069890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T22:14:03.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.123+0000 7f38b1d9b700 1 --2- 172.21.15.114:0/3713133506 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38b4100cd0 0x7f38b419b180 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f389c00b4b0 tx=0x7f389c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:14:03.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.123+0000 7f38ab7fe700 1 -- 172.21.15.114:0/3713133506 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f389c011800 con 0x7f38b4100cd0 2024-02-16T22:14:03.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.123+0000 7f38ab7fe700 1 -- 172.21.15.114:0/3713133506 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f389c011e10 con 0x7f38b4100cd0 2024-02-16T22:14:03.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.123+0000 7f38b8f11700 1 -- 172.21.15.114:0/3713133506 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b419bb70 con 0x7f38b4100cd0 2024-02-16T22:14:03.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.123+0000 7f38ab7fe700 1 -- 172.21.15.114:0/3713133506 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f389c00f2e0 con 0x7f38b4100cd0 2024-02-16T22:14:03.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.123+0000 7f38b8f11700 1 -- 172.21.15.114:0/3713133506 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b419c140 con 0x7f38b4100cd0 2024-02-16T22:14:03.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.124+0000 7f38b8f11700 1 -- 172.21.15.114:0/3713133506 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b41086b0 con 0x7f38b4100cd0 2024-02-16T22:14:03.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.125+0000 7f38ab7fe700 1 -- 172.21.15.114:0/3713133506 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7f389c00f440 con 0x7f38b4100cd0 2024-02-16T22:14:03.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.125+0000 7f38ab7fe700 1 --2- 172.21.15.114:0/3713133506 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f38a006c460 0x7f38a006e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:14:03.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.126+0000 7f38b259c700 1 --2- 172.21.15.114:0/3713133506 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f38a006c460 0x7f38a006e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:14:03.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.126+0000 7f38ab7fe700 1 -- 172.21.15.114:0/3713133506 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f389c08a280 con 0x7f38b4100cd0 2024-02-16T22:14:03.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.126+0000 7f38b259c700 1 --2- 172.21.15.114:0/3713133506 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f38a006c460 0x7f38a006e920 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f38a400b5c0 tx=0x7f38a4005e20 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:14:03.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.131+0000 7f38ab7fe700 1 -- 172.21.15.114:0/3713133506 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f389c0595f0 con 0x7f38b4100cd0 2024-02-16T22:14:03.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:03.330+0000 7f38b8f11700 1 -- 172.21.15.114:0/3713133506 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f38b4105620 con 0x7f38a006c460 2024-02-16T22:14:03.583 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:03 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T22:14:03.583 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:03 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T22:14:03.583 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:03 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:14:03.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:03 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T22:14:03.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:03 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T22:14:03.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:03 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:14:04.781 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:04 smithi114 ceph-mon[29235]: from='client.14252 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:14:04.781 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:04 smithi114 ceph-mon[29235]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:04.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:04 smithi123 ceph-mon[37161]: from='client.14252 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:14:04.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:04 smithi123 ceph-mon[37161]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:06.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:06 smithi114 ceph-mon[29235]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:06.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:06 smithi123 ceph-mon[37161]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:07.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:07 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:07.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:07 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:07.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:07 smithi114 ceph-mon[29235]: Reconfiguring grafana.smithi114 (dependencies changed)... 2024-02-16T22:14:07.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:07 smithi114 ceph-mon[29235]: Reconfiguring daemon grafana.smithi114 on smithi114 2024-02-16T22:14:08.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:07 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:08.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:07 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:08.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:07 smithi123 ceph-mon[37161]: Reconfiguring grafana.smithi114 (dependencies changed)... 2024-02-16T22:14:08.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:07 smithi123 ceph-mon[37161]: Reconfiguring daemon grafana.smithi114 on smithi114 2024-02-16T22:14:08.758 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:08 smithi114 ceph-mon[29235]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:08.758 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:08 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/309242533' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a72ccf1-3f93-4c06-99c3-092a7f9a732d"}]: dispatch 2024-02-16T22:14:08.758 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:08 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/309242533' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a72ccf1-3f93-4c06-99c3-092a7f9a732d"}]': finished 2024-02-16T22:14:08.758 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:08 smithi114 ceph-mon[29235]: osdmap e6: 1 total, 0 up, 1 in 2024-02-16T22:14:08.758 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:08 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:14:08.758 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:08 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/3038888120' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T22:14:09.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:08 smithi123 ceph-mon[37161]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:09.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:08 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/309242533' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a72ccf1-3f93-4c06-99c3-092a7f9a732d"}]: dispatch 2024-02-16T22:14:09.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:08 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/309242533' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a72ccf1-3f93-4c06-99c3-092a7f9a732d"}]': finished 2024-02-16T22:14:09.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:08 smithi123 ceph-mon[37161]: osdmap e6: 1 total, 0 up, 1 in 2024-02-16T22:14:09.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:08 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:14:09.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:08 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/3038888120' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T22:14:10.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:10 smithi114 ceph-mon[29235]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:10.953 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:10 smithi123 ceph-mon[37161]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:11.855 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:11 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:14:12.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:11 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:14:12.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:12 smithi114 ceph-mon[29235]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:13.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:12 smithi123 ceph-mon[37161]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:14.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:14 smithi114 ceph-mon[29235]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:15.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:14 smithi123 ceph-mon[37161]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:16.926 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:16 smithi114 ceph-mon[29235]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:17.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:16 smithi123 ceph-mon[37161]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:18.108 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:18 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:18.108 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:18 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:18.108 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:18 smithi114 ceph-mon[29235]: Reconfiguring prometheus.smithi114 (dependencies changed)... 2024-02-16T22:14:18.108 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:18 smithi114 ceph-mon[29235]: Reconfiguring daemon prometheus.smithi114 on smithi114 2024-02-16T22:14:18.108 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:18 smithi114 ceph-mon[29235]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:18.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:18 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:18.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:18 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:18.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:18 smithi123 ceph-mon[37161]: Reconfiguring prometheus.smithi114 (dependencies changed)... 2024-02-16T22:14:18.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:18 smithi123 ceph-mon[37161]: Reconfiguring daemon prometheus.smithi114 on smithi114 2024-02-16T22:14:18.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:18 smithi123 ceph-mon[37161]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:19.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:19 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-16T22:14:19.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:19 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:14:19.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:19 smithi123 ceph-mon[37161]: Deploying daemon osd.0 on smithi114 2024-02-16T22:14:19.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:19 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-16T22:14:19.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:19 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:14:19.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:19 smithi114 ceph-mon[29235]: Deploying daemon osd.0 on smithi114 2024-02-16T22:14:20.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:20 smithi123 ceph-mon[37161]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:20.331 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:20 smithi114 ceph-mon[29235]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:22.718 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:22 smithi114 ceph-mon[29235]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:22.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:22 smithi123 ceph-mon[37161]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:24.017 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:23 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:24.017 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:23 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:24.017 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:23 smithi114 ceph-mon[29235]: Reconfiguring ceph-exporter.smithi123 (monmap changed)... 2024-02-16T22:14:24.018 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:23 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi123", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T22:14:24.018 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:23 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:14:24.018 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:23 smithi114 ceph-mon[29235]: Reconfiguring daemon ceph-exporter.smithi123 on smithi123 2024-02-16T22:14:24.018 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:23 smithi114 ceph-mon[29235]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:24.043 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:23 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:24.043 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:23 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:24.043 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:23 smithi123 ceph-mon[37161]: Reconfiguring ceph-exporter.smithi123 (monmap changed)... 2024-02-16T22:14:24.043 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:23 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi123", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T22:14:24.043 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:23 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:14:24.043 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:23 smithi123 ceph-mon[37161]: Reconfiguring daemon ceph-exporter.smithi123 on smithi123 2024-02-16T22:14:24.043 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:23 smithi123 ceph-mon[37161]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:25.300 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:25 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:25.300 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:25 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:25.300 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:25 smithi123 ceph-mon[37161]: Reconfiguring crash.smithi123 (monmap changed)... 2024-02-16T22:14:25.300 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:25 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi123", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T22:14:25.300 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:25 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:14:25.300 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:25 smithi123 ceph-mon[37161]: Reconfiguring daemon crash.smithi123 on smithi123 2024-02-16T22:14:25.307 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:25 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:25.307 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:25 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:25.307 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:25 smithi114 ceph-mon[29235]: Reconfiguring crash.smithi123 (monmap changed)... 2024-02-16T22:14:25.307 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:25 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi123", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T22:14:25.308 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:25 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:14:25.308 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:25 smithi114 ceph-mon[29235]: Reconfiguring daemon crash.smithi123 on smithi123 2024-02-16T22:14:26.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:26 smithi114 ceph-mon[29235]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:26.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:26.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:26.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:26 smithi114 ceph-mon[29235]: Reconfiguring mgr.smithi123.inzgjd (monmap changed)... 2024-02-16T22:14:26.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi123.inzgjd", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:14:26.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T22:14:26.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:14:26.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:26 smithi114 ceph-mon[29235]: Reconfiguring daemon mgr.smithi123.inzgjd on smithi123 2024-02-16T22:14:26.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:14:26.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:26 smithi123 ceph-mon[37161]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:26.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:26 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:26.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:26 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:26.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:26 smithi123 ceph-mon[37161]: Reconfiguring mgr.smithi123.inzgjd (monmap changed)... 2024-02-16T22:14:26.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:26 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi123.inzgjd", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:14:26.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:26 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T22:14:26.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:26 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:14:26.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:26 smithi123 ceph-mon[37161]: Reconfiguring daemon mgr.smithi123.inzgjd on smithi123 2024-02-16T22:14:26.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:26 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:14:28.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:27 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:28.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:27 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:28.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:27 smithi123 ceph-mon[37161]: Reconfiguring mon.smithi123 (monmap changed)... 2024-02-16T22:14:28.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:27 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T22:14:28.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:27 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T22:14:28.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:27 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:14:28.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:27 smithi123 ceph-mon[37161]: Reconfiguring daemon mon.smithi123 on smithi123 2024-02-16T22:14:28.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:27 smithi123 ceph-mon[37161]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:28.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:27 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:28.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:27 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:28.309 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:27 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:28.309 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:27 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:28.309 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:27 smithi114 ceph-mon[29235]: Reconfiguring mon.smithi123 (monmap changed)... 2024-02-16T22:14:28.309 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:27 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T22:14:28.309 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:27 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T22:14:28.309 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:27 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:14:28.309 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:27 smithi114 ceph-mon[29235]: Reconfiguring daemon mon.smithi123 on smithi123 2024-02-16T22:14:28.309 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:27 smithi114 ceph-mon[29235]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:28.309 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:27 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:28.309 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:27 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:29.637 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:29 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:29.637 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:29 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:29.638 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:29 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T22:14:29.638 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:29 smithi114 ceph-mon[29235]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T22:14:29.638 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:29 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi114.front.sepia.ceph.com:9093"}]: dispatch 2024-02-16T22:14:29.638 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:29 smithi114 ceph-mon[29235]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi114.front.sepia.ceph.com:9093"}]: dispatch 2024-02-16T22:14:29.638 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:29 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:29.638 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:29 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T22:14:29.638 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:29 smithi114 ceph-mon[29235]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T22:14:29.638 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:29 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi114.front.sepia.ceph.com:3000"}]: dispatch 2024-02-16T22:14:29.638 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:29 smithi114 ceph-mon[29235]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi114.front.sepia.ceph.com:3000"}]: dispatch 2024-02-16T22:14:29.638 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:29 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:29.638 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:29 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T22:14:29.638 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:29 smithi114 ceph-mon[29235]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T22:14:29.638 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:29 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi114.front.sepia.ceph.com:9095"}]: dispatch 2024-02-16T22:14:29.638 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:29 smithi114 ceph-mon[29235]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi114.front.sepia.ceph.com:9095"}]: dispatch 2024-02-16T22:14:29.639 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:29 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:29.639 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:29 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:14:29.666 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:29 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:29.666 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:29 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:29.666 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:29 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T22:14:29.666 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:29 smithi123 ceph-mon[37161]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T22:14:29.666 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:29 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi114.front.sepia.ceph.com:9093"}]: dispatch 2024-02-16T22:14:29.666 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:29 smithi123 ceph-mon[37161]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi114.front.sepia.ceph.com:9093"}]: dispatch 2024-02-16T22:14:29.666 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:29 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:29.666 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:29 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T22:14:29.666 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:29 smithi123 ceph-mon[37161]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T22:14:29.666 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:29 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi114.front.sepia.ceph.com:3000"}]: dispatch 2024-02-16T22:14:29.666 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:29 smithi123 ceph-mon[37161]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi114.front.sepia.ceph.com:3000"}]: dispatch 2024-02-16T22:14:29.666 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:29 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:29.666 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:29 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T22:14:29.666 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:29 smithi123 ceph-mon[37161]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T22:14:29.667 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:29 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi114.front.sepia.ceph.com:9095"}]: dispatch 2024-02-16T22:14:29.667 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:29 smithi123 ceph-mon[37161]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi114.front.sepia.ceph.com:9095"}]: dispatch 2024-02-16T22:14:29.667 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:29 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:29.667 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:29 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:14:30.641 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:30 smithi114 ceph-mon[29235]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:30.641 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:30 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:30.642 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:30 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:30.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:30 smithi123 ceph-mon[37161]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:30.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:30 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:30.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:30 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:30.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:30.815+0000 7f38ab7fe700 1 -- 172.21.15.114:0/3713133506 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f38b4105620 con 0x7f38a006c460 2024-02-16T22:14:30.817 INFO:teuthology.orchestra.run.smithi114.stdout:Created osd(s) 0 on host 'smithi114' 2024-02-16T22:14:30.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:30.817+0000 7f38b8f11700 1 -- 172.21.15.114:0/3713133506 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f38a006c460 msgr2=0x7f38a006e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:14:30.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:30.817+0000 7f38b8f11700 1 --2- 172.21.15.114:0/3713133506 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f38a006c460 0x7f38a006e920 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f38a400b5c0 tx=0x7f38a4005e20 comp rx=0 tx=0).stop 2024-02-16T22:14:30.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:30.817+0000 7f38b8f11700 1 -- 172.21.15.114:0/3713133506 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38b4100cd0 msgr2=0x7f38b419b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:14:30.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:30.817+0000 7f38b8f11700 1 --2- 172.21.15.114:0/3713133506 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38b4100cd0 0x7f38b419b180 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f389c00b4b0 tx=0x7f389c00b870 comp rx=0 tx=0).stop 2024-02-16T22:14:30.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:30.817+0000 7f38b8f11700 1 -- 172.21.15.114:0/3713133506 shutdown_connections 2024-02-16T22:14:30.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:30.817+0000 7f38b8f11700 1 --2- 172.21.15.114:0/3713133506 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f38a006c460 0x7f38a006e920 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:14:30.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:30.817+0000 7f38b8f11700 1 --2- 172.21.15.114:0/3713133506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38b40ffad0 0x7f38b4069890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:14:30.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:30.817+0000 7f38b8f11700 1 --2- 172.21.15.114:0/3713133506 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38b4100cd0 0x7f38b419b180 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:14:30.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:30.817+0000 7f38b8f11700 1 -- 172.21.15.114:0/3713133506 >> 172.21.15.114:0/3713133506 conn(0x7f38b40fb050 msgr2=0x7f38b4103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:14:30.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:30.817+0000 7f38b8f11700 1 -- 172.21.15.114:0/3713133506 shutdown_connections 2024-02-16T22:14:30.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:30.817+0000 7f38b8f11700 1 -- 172.21.15.114:0/3713133506 wait complete. 2024-02-16T22:14:31.625 DEBUG:teuthology.orchestra.run.smithi114:osd.0> sudo journalctl -f -n 0 -u ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@osd.0.service 2024-02-16T22:14:31.627 INFO:tasks.cephadm:Deploying osd.1 on smithi114 with /dev/vg_nvme/lv_3... 2024-02-16T22:14:31.628 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-16T22:14:31.903 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:14:31.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:31 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:31.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:31 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:31.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:31 smithi114 ceph-mon[29235]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:31.952 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:14:31 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0[48477]: 2024-02-16T22:14:31.781+0000 7f6d3ccf1640 -1 osd.0 0 log_to_monitors true 2024-02-16T22:14:32.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:31 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:32.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:31 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:32.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:31 smithi123 ceph-mon[37161]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:32.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:32 smithi114 ceph-mon[29235]: from='osd.0 [v2:172.21.15.114:6802/1379159586,v1:172.21.15.114:6803/1379159586]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-16T22:14:33.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:32 smithi123 ceph-mon[37161]: from='osd.0 [v2:172.21.15.114:6802/1379159586,v1:172.21.15.114:6803/1379159586]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-16T22:14:33.899 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:14:33 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0[48477]: 2024-02-16T22:14:33.793+0000 7f6d31b54700 -1 osd.0 0 waiting for initial osdmap 2024-02-16T22:14:33.899 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:14:33 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0[48477]: 2024-02-16T22:14:33.796+0000 7f6d2a943700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T22:14:33.900 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:33 smithi114 ceph-mon[29235]: from='osd.0 [v2:172.21.15.114:6802/1379159586,v1:172.21.15.114:6803/1379159586]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-16T22:14:33.900 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:33 smithi114 ceph-mon[29235]: osdmap e7: 1 total, 0 up, 1 in 2024-02-16T22:14:33.900 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:33 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:14:33.900 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:33 smithi114 ceph-mon[29235]: from='osd.0 [v2:172.21.15.114:6802/1379159586,v1:172.21.15.114:6803/1379159586]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-02-16T22:14:33.900 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:33 smithi114 ceph-mon[29235]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:34.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:33 smithi123 ceph-mon[37161]: from='osd.0 [v2:172.21.15.114:6802/1379159586,v1:172.21.15.114:6803/1379159586]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-16T22:14:34.058 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:33 smithi123 ceph-mon[37161]: osdmap e7: 1 total, 0 up, 1 in 2024-02-16T22:14:34.058 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:33 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:14:34.058 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:33 smithi123 ceph-mon[37161]: from='osd.0 [v2:172.21.15.114:6802/1379159586,v1:172.21.15.114:6803/1379159586]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-02-16T22:14:34.058 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:33 smithi123 ceph-mon[37161]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:34.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:34 smithi114 ceph-mon[29235]: purged_snaps scrub starts 2024-02-16T22:14:34.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:34 smithi114 ceph-mon[29235]: purged_snaps scrub ok 2024-02-16T22:14:34.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:34 smithi114 ceph-mon[29235]: from='osd.0 [v2:172.21.15.114:6802/1379159586,v1:172.21.15.114:6803/1379159586]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-02-16T22:14:34.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:34 smithi114 ceph-mon[29235]: osdmap e8: 1 total, 0 up, 1 in 2024-02-16T22:14:34.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:34 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:14:34.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:34 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:14:35.034 INFO:teuthology.orchestra.run.smithi114.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-16T22:14:35.034 INFO:teuthology.orchestra.run.smithi114.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-16T22:14:35.034 INFO:teuthology.orchestra.run.smithi114.stderr: stderr: 10+0 records in 2024-02-16T22:14:35.034 INFO:teuthology.orchestra.run.smithi114.stderr:10+0 records out 2024-02-16T22:14:35.034 INFO:teuthology.orchestra.run.smithi114.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128046 s, 819 MB/s 2024-02-16T22:14:35.034 INFO:teuthology.orchestra.run.smithi114.stderr:--> Zapping successful for: 2024-02-16T22:14:35.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:34 smithi123 ceph-mon[37161]: purged_snaps scrub starts 2024-02-16T22:14:35.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:34 smithi123 ceph-mon[37161]: purged_snaps scrub ok 2024-02-16T22:14:35.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:34 smithi123 ceph-mon[37161]: from='osd.0 [v2:172.21.15.114:6802/1379159586,v1:172.21.15.114:6803/1379159586]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-02-16T22:14:35.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:34 smithi123 ceph-mon[37161]: osdmap e8: 1 total, 0 up, 1 in 2024-02-16T22:14:35.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:34 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:14:35.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:34 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:14:35.626 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi114:vg_nvme/lv_3 2024-02-16T22:14:35.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:35 smithi114 ceph-mon[29235]: osd.0 [v2:172.21.15.114:6802/1379159586,v1:172.21.15.114:6803/1379159586] boot 2024-02-16T22:14:35.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:35 smithi114 ceph-mon[29235]: osdmap e9: 1 total, 1 up, 1 in 2024-02-16T22:14:35.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:35 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:14:35.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:35 smithi114 ceph-mon[29235]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:36.017 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:14:36.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:35 smithi123 ceph-mon[37161]: osd.0 [v2:172.21.15.114:6802/1379159586,v1:172.21.15.114:6803/1379159586] boot 2024-02-16T22:14:36.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:35 smithi123 ceph-mon[37161]: osdmap e9: 1 total, 1 up, 1 in 2024-02-16T22:14:36.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:35 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:14:36.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:35 smithi123 ceph-mon[37161]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T22:14:37.070 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:36 smithi114 ceph-mon[29235]: osdmap e10: 1 total, 1 up, 1 in 2024-02-16T22:14:37.070 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:36 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:37.070 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:36 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:37.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:36 smithi123 ceph-mon[37161]: osdmap e10: 1 total, 1 up, 1 in 2024-02-16T22:14:37.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:36 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:37.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:36 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:38.085 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:37 smithi114 ceph-mon[29235]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:14:38.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.287+0000 7f1de72d7700 1 -- 172.21.15.114:0/1153195919 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1de0100bb0 msgr2=0x7f1de0100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:14:38.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.287+0000 7f1de72d7700 1 --2- 172.21.15.114:0/1153195919 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1de0100bb0 0x7f1de0100fd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0009910 tx=0x7f1dd0009c20 comp rx=0 tx=0).stop 2024-02-16T22:14:38.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.287+0000 7f1de72d7700 1 -- 172.21.15.114:0/1153195919 shutdown_connections 2024-02-16T22:14:38.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.287+0000 7f1de72d7700 1 --2- 172.21.15.114:0/1153195919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1de0101db0 0x7f1de0102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:14:38.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.287+0000 7f1de72d7700 1 --2- 172.21.15.114:0/1153195919 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1de0100bb0 0x7f1de0100fd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:14:38.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.287+0000 7f1de72d7700 1 -- 172.21.15.114:0/1153195919 >> 172.21.15.114:0/1153195919 conn(0x7f1de00fc110 msgr2=0x7f1de00fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:14:38.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.287+0000 7f1de72d7700 1 -- 172.21.15.114:0/1153195919 shutdown_connections 2024-02-16T22:14:38.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.287+0000 7f1de72d7700 1 -- 172.21.15.114:0/1153195919 wait complete. 2024-02-16T22:14:38.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.287+0000 7f1de72d7700 1 Processor -- start 2024-02-16T22:14:38.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.287+0000 7f1de72d7700 1 -- start start 2024-02-16T22:14:38.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.288+0000 7f1de72d7700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1de0100bb0 0x7f1de0198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:14:38.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.288+0000 7f1de72d7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1de0101db0 0x7f1de0198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:14:38.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.288+0000 7f1de72d7700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de01990e0 con 0x7f1de0100bb0 2024-02-16T22:14:38.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.288+0000 7f1de72d7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de0199220 con 0x7f1de0101db0 2024-02-16T22:14:38.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.288+0000 7f1de5073700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1de0100bb0 0x7f1de0198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:14:38.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.288+0000 7f1de5073700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1de0100bb0 0x7f1de0198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52658/0 (socket says 172.21.15.114:52658) 2024-02-16T22:14:38.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.288+0000 7f1de5073700 1 -- 172.21.15.114:0/754879014 learned_addr learned my addr 172.21.15.114:0/754879014 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:14:38.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.288+0000 7f1de5073700 1 -- 172.21.15.114:0/754879014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1de0101db0 msgr2=0x7f1de0198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:14:38.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.288+0000 7f1de5073700 1 --2- 172.21.15.114:0/754879014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1de0101db0 0x7f1de0198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:14:38.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.288+0000 7f1de5073700 1 -- 172.21.15.114:0/754879014 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd00095f0 con 0x7f1de0100bb0 2024-02-16T22:14:38.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.288+0000 7f1de5073700 1 --2- 172.21.15.114:0/754879014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1de0100bb0 0x7f1de0198580 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1dd000bda0 tx=0x7f1dd000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:14:38.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.288+0000 7f1dd67fc700 1 -- 172.21.15.114:0/754879014 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1dd001d070 con 0x7f1de0100bb0 2024-02-16T22:14:38.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.288+0000 7f1dd67fc700 1 -- 172.21.15.114:0/754879014 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1dd0004c50 con 0x7f1de0100bb0 2024-02-16T22:14:38.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.288+0000 7f1de72d7700 1 -- 172.21.15.114:0/754879014 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1de019dc70 con 0x7f1de0100bb0 2024-02-16T22:14:38.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.288+0000 7f1dd67fc700 1 -- 172.21.15.114:0/754879014 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1dd000f690 con 0x7f1de0100bb0 2024-02-16T22:14:38.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.288+0000 7f1de72d7700 1 -- 172.21.15.114:0/754879014 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1de019e160 con 0x7f1de0100bb0 2024-02-16T22:14:38.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.289+0000 7f1de72d7700 1 -- 172.21.15.114:0/754879014 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1de0105e90 con 0x7f1de0100bb0 2024-02-16T22:14:38.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.290+0000 7f1dd67fc700 1 -- 172.21.15.114:0/754879014 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7f1dd00229b0 con 0x7f1de0100bb0 2024-02-16T22:14:38.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.290+0000 7f1dd67fc700 1 --2- 172.21.15.114:0/754879014 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f1dcc06c460 0x7f1dcc06e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:14:38.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.290+0000 7f1dd67fc700 1 -- 172.21.15.114:0/754879014 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1982+0+0 (secure 0 0 0) 0x7f1dd008c0a0 con 0x7f1de0100bb0 2024-02-16T22:14:38.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.290+0000 7f1de4872700 1 --2- 172.21.15.114:0/754879014 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f1dcc06c460 0x7f1dcc06e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:14:38.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.291+0000 7f1de4872700 1 --2- 172.21.15.114:0/754879014 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f1dcc06c460 0x7f1dcc06e920 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc00a990 tx=0x7f1ddc005c80 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:14:38.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.295+0000 7f1dd67fc700 1 -- 172.21.15.114:0/754879014 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1dd005b360 con 0x7f1de0100bb0 2024-02-16T22:14:38.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:37 smithi123 ceph-mon[37161]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:14:38.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:38.499+0000 7f1de72d7700 1 -- 172.21.15.114:0/754879014 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f1de00611d0 con 0x7f1dcc06c460 2024-02-16T22:14:39.084 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:38 smithi114 ceph-mon[29235]: from='client.14270 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:14:39.084 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:38 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T22:14:39.084 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:38 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T22:14:39.084 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:38 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:14:39.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:38 smithi123 ceph-mon[37161]: from='client.14270 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:14:39.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:38 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T22:14:39.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:38 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T22:14:39.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:38 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:14:39.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:39 smithi114 ceph-mon[29235]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:14:40.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:39 smithi123 ceph-mon[37161]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:14:41.766 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:41 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:14:41.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:41 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:14:42.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:42 smithi123 ceph-mon[37161]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:14:42.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:42 smithi123 ceph-mon[37161]: Detected new or changed devices on smithi114 2024-02-16T22:14:42.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:42 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:42.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:42 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:42.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:42 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:14:42.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:42 smithi123 ceph-mon[37161]: Adjusting osd_memory_target on smithi114 to 11840M 2024-02-16T22:14:42.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:42 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:42.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:42 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:14:42.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:42 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:14:42.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:42 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:42.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:42 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:14:42.889 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:42 smithi114 ceph-mon[29235]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:14:42.889 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:42 smithi114 ceph-mon[29235]: Detected new or changed devices on smithi114 2024-02-16T22:14:42.889 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:42 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:42.889 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:42 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:42.889 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:42 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:14:42.890 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:42 smithi114 ceph-mon[29235]: Adjusting osd_memory_target on smithi114 to 11840M 2024-02-16T22:14:42.890 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:42 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:42.890 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:42 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:14:42.890 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:42 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:14:42.890 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:42 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:42.890 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:42 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:14:43.661 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:43 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/196080387' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e8829dd-cac2-462f-aca1-86bcfb475a1c"}]: dispatch 2024-02-16T22:14:43.661 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:43 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/196080387' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e8829dd-cac2-462f-aca1-86bcfb475a1c"}]': finished 2024-02-16T22:14:43.662 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:43 smithi114 ceph-mon[29235]: osdmap e11: 2 total, 1 up, 2 in 2024-02-16T22:14:43.662 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:43 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:14:43.662 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:43 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:43.662 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:43 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:44.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:43 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/196080387' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e8829dd-cac2-462f-aca1-86bcfb475a1c"}]: dispatch 2024-02-16T22:14:44.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:43 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/196080387' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e8829dd-cac2-462f-aca1-86bcfb475a1c"}]': finished 2024-02-16T22:14:44.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:43 smithi123 ceph-mon[37161]: osdmap e11: 2 total, 1 up, 2 in 2024-02-16T22:14:44.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:43 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:14:44.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:43 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:44.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:43 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:44.649 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:44 smithi114 ceph-mon[29235]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:14:44.649 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:44 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/3459467980' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T22:14:45.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:44 smithi123 ceph-mon[37161]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:14:45.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:44 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/3459467980' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T22:14:46.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:46 smithi114 ceph-mon[29235]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:14:46.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:46 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:46.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:46 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:46.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:46 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:14:46.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:46 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:14:46.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:46 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:47.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:46 smithi123 ceph-mon[37161]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:14:47.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:46 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:47.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:46 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:47.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:46 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:14:47.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:46 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:14:47.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:46 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:48.086 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:47 smithi114 ceph-mon[29235]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:14:48.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:47 smithi123 ceph-mon[37161]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:14:50.760 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:50 smithi123 ceph-mon[37161]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:14:50.761 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:50 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-16T22:14:50.761 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:50 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:14:50.848 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:50 smithi114 ceph-mon[29235]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:14:50.849 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:50 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-16T22:14:50.849 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:50 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:14:51.798 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:51 smithi114 ceph-mon[29235]: Deploying daemon osd.1 on smithi114 2024-02-16T22:14:51.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:51 smithi123 ceph-mon[37161]: Deploying daemon osd.1 on smithi114 2024-02-16T22:14:52.779 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:52 smithi114 ceph-mon[29235]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:14:52.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:52 smithi123 ceph-mon[37161]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:14:54.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:54 smithi123 ceph-mon[37161]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:14:54.880 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:54 smithi114 ceph-mon[29235]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:14:57.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:56 smithi123 ceph-mon[37161]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:14:57.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:56 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:57.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:56 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:57.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:56 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:14:57.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:56 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:14:57.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:56 smithi114 ceph-mon[29235]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:14:57.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:56 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:57.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:56 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:57.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:56 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:14:57.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:56 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:14:58.676 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:58 smithi114 ceph-mon[29235]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:14:58.677 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:58 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:58.677 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:14:58 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:59.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:58 smithi123 ceph-mon[37161]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:14:59.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:58 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:59.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:14:58 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:14:59.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:59.404+0000 7f1dd67fc700 1 -- 172.21.15.114:0/754879014 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1de00611d0 con 0x7f1dcc06c460 2024-02-16T22:14:59.406 INFO:teuthology.orchestra.run.smithi114.stdout:Created osd(s) 1 on host 'smithi114' 2024-02-16T22:14:59.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:59.406+0000 7f1de72d7700 1 -- 172.21.15.114:0/754879014 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f1dcc06c460 msgr2=0x7f1dcc06e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:14:59.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:59.406+0000 7f1de72d7700 1 --2- 172.21.15.114:0/754879014 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f1dcc06c460 0x7f1dcc06e920 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc00a990 tx=0x7f1ddc005c80 comp rx=0 tx=0).stop 2024-02-16T22:14:59.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:59.406+0000 7f1de72d7700 1 -- 172.21.15.114:0/754879014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1de0100bb0 msgr2=0x7f1de0198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:14:59.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:59.406+0000 7f1de72d7700 1 --2- 172.21.15.114:0/754879014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1de0100bb0 0x7f1de0198580 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1dd000bda0 tx=0x7f1dd000be80 comp rx=0 tx=0).stop 2024-02-16T22:14:59.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:59.406+0000 7f1de72d7700 1 -- 172.21.15.114:0/754879014 shutdown_connections 2024-02-16T22:14:59.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:59.406+0000 7f1de72d7700 1 --2- 172.21.15.114:0/754879014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1de0100bb0 0x7f1de0198580 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:14:59.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:59.406+0000 7f1de72d7700 1 --2- 172.21.15.114:0/754879014 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f1dcc06c460 0x7f1dcc06e920 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:14:59.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:59.406+0000 7f1de72d7700 1 --2- 172.21.15.114:0/754879014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1de0101db0 0x7f1de0198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:14:59.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:59.406+0000 7f1de72d7700 1 -- 172.21.15.114:0/754879014 >> 172.21.15.114:0/754879014 conn(0x7f1de00fc110 msgr2=0x7f1de0102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:14:59.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:59.406+0000 7f1de72d7700 1 -- 172.21.15.114:0/754879014 shutdown_connections 2024-02-16T22:14:59.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:14:59.406+0000 7f1de72d7700 1 -- 172.21.15.114:0/754879014 wait complete. 2024-02-16T22:15:00.198 DEBUG:teuthology.orchestra.run.smithi114:osd.1> sudo journalctl -f -n 0 -u ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@osd.1.service 2024-02-16T22:15:00.200 INFO:tasks.cephadm:Deploying osd.2 on smithi114 with /dev/vg_nvme/lv_2... 2024-02-16T22:15:00.200 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-16T22:15:00.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:00 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:00.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:00 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:00.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:00 smithi114 ceph-mon[29235]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:15:00.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:00 smithi114 ceph-mon[29235]: from='osd.1 [v2:172.21.15.114:6810/1548199763,v1:172.21.15.114:6811/1548199763]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-16T22:15:00.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:00 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:00.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:00 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:00.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:00 smithi123 ceph-mon[37161]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:15:00.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:00 smithi123 ceph-mon[37161]: from='osd.1 [v2:172.21.15.114:6810/1548199763,v1:172.21.15.114:6811/1548199763]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-16T22:15:01.312 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:15:01.467 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:01 smithi114 ceph-mon[29235]: from='osd.1 [v2:172.21.15.114:6810/1548199763,v1:172.21.15.114:6811/1548199763]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-16T22:15:01.467 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:01 smithi114 ceph-mon[29235]: osdmap e12: 2 total, 1 up, 2 in 2024-02-16T22:15:01.467 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:01 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:15:01.467 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:01 smithi114 ceph-mon[29235]: from='osd.1 [v2:172.21.15.114:6810/1548199763,v1:172.21.15.114:6811/1548199763]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-02-16T22:15:01.468 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:15:01 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1[55938]: 2024-02-16T22:15:01.382+0000 7f8a9e957700 -1 osd.1 0 waiting for initial osdmap 2024-02-16T22:15:01.468 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:15:01 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1[55938]: 2024-02-16T22:15:01.386+0000 7f8a99f4b700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T22:15:01.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:01 smithi123 ceph-mon[37161]: from='osd.1 [v2:172.21.15.114:6810/1548199763,v1:172.21.15.114:6811/1548199763]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-16T22:15:01.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:01 smithi123 ceph-mon[37161]: osdmap e12: 2 total, 1 up, 2 in 2024-02-16T22:15:01.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:01 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:15:01.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:01 smithi123 ceph-mon[37161]: from='osd.1 [v2:172.21.15.114:6810/1548199763,v1:172.21.15.114:6811/1548199763]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-02-16T22:15:02.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:02 smithi114 ceph-mon[29235]: from='osd.1 [v2:172.21.15.114:6810/1548199763,v1:172.21.15.114:6811/1548199763]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-02-16T22:15:02.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:02 smithi114 ceph-mon[29235]: osdmap e13: 2 total, 1 up, 2 in 2024-02-16T22:15:02.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:02 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:15:02.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:02 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:15:02.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:02 smithi114 ceph-mon[29235]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:15:02.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:02 smithi123 ceph-mon[37161]: from='osd.1 [v2:172.21.15.114:6810/1548199763,v1:172.21.15.114:6811/1548199763]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-02-16T22:15:02.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:02 smithi123 ceph-mon[37161]: osdmap e13: 2 total, 1 up, 2 in 2024-02-16T22:15:02.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:02 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:15:02.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:02 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:15:02.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:02 smithi123 ceph-mon[37161]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-16T22:15:03.642 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:03 smithi114 ceph-mon[29235]: purged_snaps scrub starts 2024-02-16T22:15:03.642 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:03 smithi114 ceph-mon[29235]: purged_snaps scrub ok 2024-02-16T22:15:03.642 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:03 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:15:03.642 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:03 smithi114 ceph-mon[29235]: osd.1 [v2:172.21.15.114:6810/1548199763,v1:172.21.15.114:6811/1548199763] boot 2024-02-16T22:15:03.642 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:03 smithi114 ceph-mon[29235]: osdmap e14: 2 total, 2 up, 2 in 2024-02-16T22:15:03.642 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:03 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:15:03.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:03 smithi123 ceph-mon[37161]: purged_snaps scrub starts 2024-02-16T22:15:03.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:03 smithi123 ceph-mon[37161]: purged_snaps scrub ok 2024-02-16T22:15:03.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:03 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:15:03.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:03 smithi123 ceph-mon[37161]: osd.1 [v2:172.21.15.114:6810/1548199763,v1:172.21.15.114:6811/1548199763] boot 2024-02-16T22:15:03.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:03 smithi123 ceph-mon[37161]: osdmap e14: 2 total, 2 up, 2 in 2024-02-16T22:15:03.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:03 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:15:04.176 INFO:teuthology.orchestra.run.smithi114.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-16T22:15:04.176 INFO:teuthology.orchestra.run.smithi114.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-16T22:15:04.176 INFO:teuthology.orchestra.run.smithi114.stderr: stderr: 10+0 records in 2024-02-16T22:15:04.176 INFO:teuthology.orchestra.run.smithi114.stderr:10+0 records out 2024-02-16T22:15:04.176 INFO:teuthology.orchestra.run.smithi114.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0125893 s, 833 MB/s 2024-02-16T22:15:04.176 INFO:teuthology.orchestra.run.smithi114.stderr:--> Zapping successful for: 2024-02-16T22:15:04.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:04 smithi114 ceph-mon[29235]: osdmap e15: 2 total, 2 up, 2 in 2024-02-16T22:15:04.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:04 smithi114 ceph-mon[29235]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T22:15:04.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:04 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:04.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:04 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:04.710 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:04 smithi123 ceph-mon[37161]: osdmap e15: 2 total, 2 up, 2 in 2024-02-16T22:15:04.710 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:04 smithi123 ceph-mon[37161]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T22:15:04.710 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:04 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:04.711 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:04 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:04.841 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi114:vg_nvme/lv_2 2024-02-16T22:15:06.073 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:15:06.607 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:06 smithi114 ceph-mon[29235]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T22:15:06.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:06 smithi123 ceph-mon[37161]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T22:15:08.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:07 smithi114 ceph-mon[29235]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T22:15:08.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:07 smithi123 ceph-mon[37161]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T22:15:08.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.770+0000 7f51e82be700 1 -- 172.21.15.114:0/4009725158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51e0100bc0 msgr2=0x7f51e0100fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:15:08.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.770+0000 7f51e82be700 1 --2- 172.21.15.114:0/4009725158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51e0100bc0 0x7f51e0100fe0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f51dc009910 tx=0x7f51dc009c20 comp rx=0 tx=0).stop 2024-02-16T22:15:08.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.770+0000 7f51e82be700 1 -- 172.21.15.114:0/4009725158 shutdown_connections 2024-02-16T22:15:08.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.770+0000 7f51e82be700 1 --2- 172.21.15.114:0/4009725158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51e0101dc0 0x7f51e0102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:15:08.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.770+0000 7f51e82be700 1 --2- 172.21.15.114:0/4009725158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51e0100bc0 0x7f51e0100fe0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:15:08.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.770+0000 7f51e82be700 1 -- 172.21.15.114:0/4009725158 >> 172.21.15.114:0/4009725158 conn(0x7f51e00fc140 msgr2=0x7f51e00fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:15:08.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.770+0000 7f51e82be700 1 -- 172.21.15.114:0/4009725158 shutdown_connections 2024-02-16T22:15:08.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.770+0000 7f51e82be700 1 -- 172.21.15.114:0/4009725158 wait complete. 2024-02-16T22:15:08.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.770+0000 7f51e82be700 1 Processor -- start 2024-02-16T22:15:08.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.770+0000 7f51e82be700 1 -- start start 2024-02-16T22:15:08.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.771+0000 7f51e82be700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51e0100bc0 0x7f51e0074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:15:08.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.771+0000 7f51e82be700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51e0101dc0 0x7f51e0073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:15:08.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.771+0000 7f51e82be700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e00736c0 con 0x7f51e0101dc0 2024-02-16T22:15:08.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.771+0000 7f51e82be700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e0073800 con 0x7f51e0100bc0 2024-02-16T22:15:08.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.771+0000 7f51e5859700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51e0101dc0 0x7f51e0073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:15:08.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.771+0000 7f51e5859700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51e0101dc0 0x7f51e0073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42648/0 (socket says 172.21.15.114:42648) 2024-02-16T22:15:08.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.771+0000 7f51e5859700 1 -- 172.21.15.114:0/1411550780 learned_addr learned my addr 172.21.15.114:0/1411550780 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:15:08.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.771+0000 7f51e605a700 1 --2- 172.21.15.114:0/1411550780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51e0100bc0 0x7f51e0074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:15:08.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.771+0000 7f51e5859700 1 -- 172.21.15.114:0/1411550780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51e0100bc0 msgr2=0x7f51e0074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:15:08.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.771+0000 7f51e5859700 1 --2- 172.21.15.114:0/1411550780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51e0100bc0 0x7f51e0074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:15:08.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.771+0000 7f51e5859700 1 -- 172.21.15.114:0/1411550780 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc0095f0 con 0x7f51e0101dc0 2024-02-16T22:15:08.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.771+0000 7f51e5859700 1 --2- 172.21.15.114:0/1411550780 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51e0101dc0 0x7f51e0073180 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f51d000d900 tx=0x7f51d000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:15:08.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.771+0000 7f51d77fe700 1 -- 172.21.15.114:0/1411550780 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51d00041f0 con 0x7f51e0101dc0 2024-02-16T22:15:08.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.771+0000 7f51d77fe700 1 -- 172.21.15.114:0/1411550780 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f51d0004ce0 con 0x7f51e0101dc0 2024-02-16T22:15:08.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.771+0000 7f51e82be700 1 -- 172.21.15.114:0/1411550780 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e0073ae0 con 0x7f51e0101dc0 2024-02-16T22:15:08.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.771+0000 7f51d77fe700 1 -- 172.21.15.114:0/1411550780 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51d000b750 con 0x7f51e0101dc0 2024-02-16T22:15:08.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.771+0000 7f51e82be700 1 -- 172.21.15.114:0/1411550780 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e019e3d0 con 0x7f51e0101dc0 2024-02-16T22:15:08.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.772+0000 7f51e82be700 1 -- 172.21.15.114:0/1411550780 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51e0109720 con 0x7f51e0101dc0 2024-02-16T22:15:08.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.773+0000 7f51d77fe700 1 -- 172.21.15.114:0/1411550780 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7f51d0004e50 con 0x7f51e0101dc0 2024-02-16T22:15:08.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.773+0000 7f51d77fe700 1 --2- 172.21.15.114:0/1411550780 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f51cc06c460 0x7f51cc06e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:15:08.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.773+0000 7f51e605a700 1 --2- 172.21.15.114:0/1411550780 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f51cc06c460 0x7f51cc06e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:15:08.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.773+0000 7f51d77fe700 1 -- 172.21.15.114:0/1411550780 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2414+0+0 (secure 0 0 0) 0x7f51d008aab0 con 0x7f51e0101dc0 2024-02-16T22:15:08.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.774+0000 7f51e605a700 1 --2- 172.21.15.114:0/1411550780 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f51cc06c460 0x7f51cc06e920 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f51dc006010 tx=0x7f51dc005ec0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:15:08.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:08.778+0000 7f51d77fe700 1 -- 172.21.15.114:0/1411550780 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f51d00599f0 con 0x7f51e0101dc0 2024-02-16T22:15:09.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:09.020+0000 7f51e82be700 1 -- 172.21.15.114:0/1411550780 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f51e0106710 con 0x7f51cc06c460 2024-02-16T22:15:09.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:09 smithi123 ceph-mon[37161]: Detected new or changed devices on smithi114 2024-02-16T22:15:09.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:09 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:09.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:09 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:09.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:09 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:15:09.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:09 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:15:09.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:09 smithi123 ceph-mon[37161]: Adjusting osd_memory_target on smithi114 to 5920M 2024-02-16T22:15:09.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:09 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:09.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:09 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:15:09.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:09 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:15:09.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:09 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:09.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:09 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:15:09.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:09 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T22:15:09.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:09 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T22:15:09.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:09 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:15:09.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:09 smithi114 ceph-mon[29235]: Detected new or changed devices on smithi114 2024-02-16T22:15:09.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:09 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:09.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:09 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:09.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:09 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:15:09.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:09 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:15:09.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:09 smithi114 ceph-mon[29235]: Adjusting osd_memory_target on smithi114 to 5920M 2024-02-16T22:15:09.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:09 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:09.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:09 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:15:09.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:09 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:15:09.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:09 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:09.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:09 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:15:09.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:09 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T22:15:09.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:09 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T22:15:09.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:09 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:15:10.416 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:10 smithi123 ceph-mon[37161]: from='client.14288 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:15:10.416 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:10 smithi123 ceph-mon[37161]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T22:15:10.416 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:10 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:10.416 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:10 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:10.646 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:10 smithi114 ceph-mon[29235]: from='client.14288 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:15:10.646 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:10 smithi114 ceph-mon[29235]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T22:15:10.646 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:10 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:10.646 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:10 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:11.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:11 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:15:11.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:11 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:15:12.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:12 smithi114 ceph-mon[29235]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T22:15:12.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:12 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/3746520397' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44f9fce8-5ea8-4741-9936-6720df91b628"}]: dispatch 2024-02-16T22:15:12.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:12 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/3746520397' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "44f9fce8-5ea8-4741-9936-6720df91b628"}]': finished 2024-02-16T22:15:12.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:12 smithi114 ceph-mon[29235]: osdmap e16: 3 total, 2 up, 3 in 2024-02-16T22:15:12.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:12 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:15:12.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:12 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/2890833065' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T22:15:13.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:12 smithi123 ceph-mon[37161]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T22:15:13.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:12 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/3746520397' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44f9fce8-5ea8-4741-9936-6720df91b628"}]: dispatch 2024-02-16T22:15:13.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:12 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/3746520397' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "44f9fce8-5ea8-4741-9936-6720df91b628"}]': finished 2024-02-16T22:15:13.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:12 smithi123 ceph-mon[37161]: osdmap e16: 3 total, 2 up, 3 in 2024-02-16T22:15:13.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:12 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:15:13.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:12 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/2890833065' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T22:15:14.579 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:14 smithi114 ceph-mon[29235]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T22:15:14.579 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:14 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:14.579 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:14 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:14.580 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:14 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:15:14.580 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:14 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:15:14.580 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:14 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:15.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:14 smithi123 ceph-mon[37161]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T22:15:15.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:14 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:15.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:14 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:15.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:14 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:15:15.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:14 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:15:15.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:14 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:16.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:16 smithi114 ceph-mon[29235]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T22:15:17.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:16 smithi123 ceph-mon[37161]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T22:15:18.094 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:17 smithi114 ceph-mon[29235]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T22:15:18.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:17 smithi123 ceph-mon[37161]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T22:15:19.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:19 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-16T22:15:19.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:19 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:15:19.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:19 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-16T22:15:19.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:19 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:15:20.327 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:20 smithi114 ceph-mon[29235]: Deploying daemon osd.2 on smithi114 2024-02-16T22:15:20.327 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:20 smithi114 ceph-mon[29235]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T22:15:20.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:20 smithi123 ceph-mon[37161]: Deploying daemon osd.2 on smithi114 2024-02-16T22:15:20.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:20 smithi123 ceph-mon[37161]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T22:15:22.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:22 smithi114 ceph-mon[29235]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T22:15:22.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:22 smithi123 ceph-mon[37161]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T22:15:24.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:24 smithi114 ceph-mon[29235]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T22:15:24.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:24 smithi123 ceph-mon[37161]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T22:15:25.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:25 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:25.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:25 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:25.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:25 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:15:26.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:25 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:26.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:25 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:26.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:25 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:15:26.700 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:26 smithi114 ceph-mon[29235]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T22:15:26.700 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:15:27.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:26 smithi123 ceph-mon[37161]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T22:15:27.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:26 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:15:27.826 INFO:teuthology.orchestra.run.smithi114.stdout:Created osd(s) 2 on host 'smithi114' 2024-02-16T22:15:27.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:27.824+0000 7f51d77fe700 1 -- 172.21.15.114:0/1411550780 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f51e0106710 con 0x7f51cc06c460 2024-02-16T22:15:27.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:27.826+0000 7f51e82be700 1 -- 172.21.15.114:0/1411550780 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f51cc06c460 msgr2=0x7f51cc06e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:15:27.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:27.826+0000 7f51e82be700 1 --2- 172.21.15.114:0/1411550780 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f51cc06c460 0x7f51cc06e920 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f51dc006010 tx=0x7f51dc005ec0 comp rx=0 tx=0).stop 2024-02-16T22:15:27.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:27.827+0000 7f51e82be700 1 -- 172.21.15.114:0/1411550780 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51e0101dc0 msgr2=0x7f51e0073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:15:27.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:27.827+0000 7f51e82be700 1 --2- 172.21.15.114:0/1411550780 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51e0101dc0 0x7f51e0073180 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f51d000d900 tx=0x7f51d000dc10 comp rx=0 tx=0).stop 2024-02-16T22:15:27.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:27.827+0000 7f51e82be700 1 -- 172.21.15.114:0/1411550780 shutdown_connections 2024-02-16T22:15:27.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:27.827+0000 7f51e82be700 1 --2- 172.21.15.114:0/1411550780 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f51cc06c460 0x7f51cc06e920 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:15:27.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:27.827+0000 7f51e82be700 1 --2- 172.21.15.114:0/1411550780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51e0100bc0 0x7f51e0074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:15:27.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:27.827+0000 7f51e82be700 1 --2- 172.21.15.114:0/1411550780 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51e0101dc0 0x7f51e0073180 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:15:27.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:27.827+0000 7f51e82be700 1 -- 172.21.15.114:0/1411550780 >> 172.21.15.114:0/1411550780 conn(0x7f51e00fc140 msgr2=0x7f51e0104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:15:27.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:27.827+0000 7f51e82be700 1 -- 172.21.15.114:0/1411550780 shutdown_connections 2024-02-16T22:15:27.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:15:27.827+0000 7f51e82be700 1 -- 172.21.15.114:0/1411550780 wait complete. 2024-02-16T22:15:28.192 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:27 smithi114 ceph-mon[29235]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T22:15:28.192 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:27 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:28.192 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:27 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:28.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:27 smithi123 ceph-mon[37161]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T22:15:28.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:27 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:28.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:27 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:28.508 DEBUG:teuthology.orchestra.run.smithi114:osd.2> sudo journalctl -f -n 0 -u ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@osd.2.service 2024-02-16T22:15:28.510 INFO:tasks.cephadm:Deploying osd.3 on smithi123 with /dev/vg_nvme/lv_4... 2024-02-16T22:15:28.510 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-16T22:15:28.724 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi123/config 2024-02-16T22:15:28.845 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:15:28 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2[63791]: 2024-02-16T22:15:28.554+0000 7fed1b826640 -1 osd.2 0 log_to_monitors true 2024-02-16T22:15:29.096 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:28 smithi123 ceph-mon[37161]: from='osd.2 [v2:172.21.15.114:6818/1293302101,v1:172.21.15.114:6819/1293302101]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-16T22:15:29.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:28 smithi114 ceph-mon[29235]: from='osd.2 [v2:172.21.15.114:6818/1293302101,v1:172.21.15.114:6819/1293302101]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-16T22:15:30.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:29 smithi123 ceph-mon[37161]: from='osd.2 [v2:172.21.15.114:6818/1293302101,v1:172.21.15.114:6819/1293302101]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-16T22:15:30.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:29 smithi123 ceph-mon[37161]: osdmap e17: 3 total, 2 up, 3 in 2024-02-16T22:15:30.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:29 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:15:30.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:29 smithi123 ceph-mon[37161]: from='osd.2 [v2:172.21.15.114:6818/1293302101,v1:172.21.15.114:6819/1293302101]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-02-16T22:15:30.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:29 smithi123 ceph-mon[37161]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T22:15:30.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:29 smithi114 ceph-mon[29235]: from='osd.2 [v2:172.21.15.114:6818/1293302101,v1:172.21.15.114:6819/1293302101]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-16T22:15:30.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:29 smithi114 ceph-mon[29235]: osdmap e17: 3 total, 2 up, 3 in 2024-02-16T22:15:30.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:29 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:15:30.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:29 smithi114 ceph-mon[29235]: from='osd.2 [v2:172.21.15.114:6818/1293302101,v1:172.21.15.114:6819/1293302101]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-02-16T22:15:30.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:29 smithi114 ceph-mon[29235]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T22:15:30.202 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:15:29 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2[63791]: 2024-02-16T22:15:29.856+0000 7fed10689700 -1 osd.2 0 waiting for initial osdmap 2024-02-16T22:15:30.202 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:15:29 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2[63791]: 2024-02-16T22:15:29.864+0000 7fed0cc7f700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T22:15:30.650 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-16T22:15:30.650 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-16T22:15:30.650 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2024-02-16T22:15:30.650 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2024-02-16T22:15:30.651 INFO:teuthology.orchestra.run.smithi123.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0117108 s, 895 MB/s 2024-02-16T22:15:30.651 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 2024-02-16T22:15:30.651 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2024-02-16T22:15:31.172 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:30 smithi114 ceph-mon[29235]: from='osd.2 [v2:172.21.15.114:6818/1293302101,v1:172.21.15.114:6819/1293302101]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-02-16T22:15:31.172 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:30 smithi114 ceph-mon[29235]: osdmap e18: 3 total, 2 up, 3 in 2024-02-16T22:15:31.172 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:30 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:15:31.172 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:30 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:15:31.172 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:30 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:15:31.200 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:30 smithi123 ceph-mon[37161]: from='osd.2 [v2:172.21.15.114:6818/1293302101,v1:172.21.15.114:6819/1293302101]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-02-16T22:15:31.200 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:30 smithi123 ceph-mon[37161]: osdmap e18: 3 total, 2 up, 3 in 2024-02-16T22:15:31.200 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:30 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:15:31.200 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:30 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:15:31.200 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:30 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:15:31.221 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi123:vg_nvme/lv_4 2024-02-16T22:15:31.429 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi123/config 2024-02-16T22:15:32.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:31 smithi123 ceph-mon[37161]: purged_snaps scrub starts 2024-02-16T22:15:32.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:31 smithi123 ceph-mon[37161]: purged_snaps scrub ok 2024-02-16T22:15:32.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:31 smithi123 ceph-mon[37161]: osd.2 [v2:172.21.15.114:6818/1293302101,v1:172.21.15.114:6819/1293302101] boot 2024-02-16T22:15:32.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:31 smithi123 ceph-mon[37161]: osdmap e19: 3 total, 3 up, 3 in 2024-02-16T22:15:32.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:31 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:15:32.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:31 smithi123 ceph-mon[37161]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T22:15:32.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:31 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-16T22:15:32.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:31 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:32.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:31 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:32.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:31 smithi114 ceph-mon[29235]: purged_snaps scrub starts 2024-02-16T22:15:32.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:31 smithi114 ceph-mon[29235]: purged_snaps scrub ok 2024-02-16T22:15:32.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:31 smithi114 ceph-mon[29235]: osd.2 [v2:172.21.15.114:6818/1293302101,v1:172.21.15.114:6819/1293302101] boot 2024-02-16T22:15:32.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:31 smithi114 ceph-mon[29235]: osdmap e19: 3 total, 3 up, 3 in 2024-02-16T22:15:32.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:31 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:15:32.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:31 smithi114 ceph-mon[29235]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-16T22:15:32.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:31 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-16T22:15:32.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:31 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:32.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:31 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:32.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.969+0000 7fd4fe13a700 1 -- 172.21.15.123:0/1694223967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4f8101dc0 msgr2=0x7fd4f8102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:15:32.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.969+0000 7fd4fe13a700 1 --2- 172.21.15.123:0/1694223967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4f8101dc0 0x7fd4f8102220 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec009910 tx=0x7fd4ec009c20 comp rx=0 tx=0).stop 2024-02-16T22:15:32.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.969+0000 7fd4fe13a700 1 -- 172.21.15.123:0/1694223967 shutdown_connections 2024-02-16T22:15:32.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.969+0000 7fd4fe13a700 1 --2- 172.21.15.123:0/1694223967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4f8101dc0 0x7fd4f8102220 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:15:32.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.969+0000 7fd4fe13a700 1 --2- 172.21.15.123:0/1694223967 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4f8100bc0 0x7fd4f8100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:15:32.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.969+0000 7fd4fe13a700 1 -- 172.21.15.123:0/1694223967 >> 172.21.15.123:0/1694223967 conn(0x7fd4f80fc140 msgr2=0x7fd4f80fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:15:32.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.969+0000 7fd4fe13a700 1 -- 172.21.15.123:0/1694223967 shutdown_connections 2024-02-16T22:15:32.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.969+0000 7fd4fe13a700 1 -- 172.21.15.123:0/1694223967 wait complete. 2024-02-16T22:15:32.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.969+0000 7fd4fe13a700 1 Processor -- start 2024-02-16T22:15:32.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.969+0000 7fd4fe13a700 1 -- start start 2024-02-16T22:15:32.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.970+0000 7fd4fe13a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4f8100bc0 0x7fd4f8196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:15:32.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.970+0000 7fd4fe13a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4f8101dc0 0x7fd4f8196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:15:32.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.970+0000 7fd4fe13a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f8196ea0 con 0x7fd4f8100bc0 2024-02-16T22:15:32.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.970+0000 7fd4fe13a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f8196fe0 con 0x7fd4f8101dc0 2024-02-16T22:15:32.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.970+0000 7fd4f6ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4f8101dc0 0x7fd4f8196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:15:32.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.970+0000 7fd4f6ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4f8101dc0 0x7fd4f8196880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60276/0 (socket says 172.21.15.123:60276) 2024-02-16T22:15:32.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.970+0000 7fd4f6ffd700 1 -- 172.21.15.123:0/405372705 learned_addr learned my addr 172.21.15.123:0/405372705 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-02-16T22:15:32.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.970+0000 7fd4f77fe700 1 --2- 172.21.15.123:0/405372705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4f8100bc0 0x7fd4f8196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:15:32.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.970+0000 7fd4f6ffd700 1 -- 172.21.15.123:0/405372705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4f8100bc0 msgr2=0x7fd4f8196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:15:32.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.970+0000 7fd4f6ffd700 1 --2- 172.21.15.123:0/405372705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4f8100bc0 0x7fd4f8196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:15:32.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.970+0000 7fd4f6ffd700 1 -- 172.21.15.123:0/405372705 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ec0095f0 con 0x7fd4f8101dc0 2024-02-16T22:15:32.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.970+0000 7fd4f6ffd700 1 --2- 172.21.15.123:0/405372705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4f8101dc0 0x7fd4f8196880 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec000c00 tx=0x7fd4ec00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:15:32.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.970+0000 7fd4f4ff9700 1 -- 172.21.15.123:0/405372705 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4ec01d070 con 0x7fd4f8101dc0 2024-02-16T22:15:32.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.970+0000 7fd4f4ff9700 1 -- 172.21.15.123:0/405372705 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd4ec005dc0 con 0x7fd4f8101dc0 2024-02-16T22:15:32.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.970+0000 7fd4f4ff9700 1 -- 172.21.15.123:0/405372705 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4ec0226a0 con 0x7fd4f8101dc0 2024-02-16T22:15:32.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.970+0000 7fd4fe13a700 1 -- 172.21.15.123:0/405372705 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f819ba30 con 0x7fd4f8101dc0 2024-02-16T22:15:32.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.970+0000 7fd4fe13a700 1 -- 172.21.15.123:0/405372705 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f819bed0 con 0x7fd4f8101dc0 2024-02-16T22:15:32.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.971+0000 7fd4fe13a700 1 -- 172.21.15.123:0/405372705 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f8109720 con 0x7fd4f8101dc0 2024-02-16T22:15:32.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.972+0000 7fd4f4ff9700 1 -- 172.21.15.123:0/405372705 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7fd4ec0038f0 con 0x7fd4f8101dc0 2024-02-16T22:15:32.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.972+0000 7fd4f4ff9700 1 --2- 172.21.15.123:0/405372705 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd4e006c460 0x7fd4e006e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:15:32.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.972+0000 7fd4f4ff9700 1 -- 172.21.15.123:0/405372705 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3232+0+0 (secure 0 0 0) 0x7fd4ec08c390 con 0x7fd4f8101dc0 2024-02-16T22:15:32.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.972+0000 7fd4f77fe700 1 --2- 172.21.15.123:0/405372705 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd4e006c460 0x7fd4e006e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:15:32.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.973+0000 7fd4f77fe700 1 --2- 172.21.15.123:0/405372705 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd4e006c460 0x7fd4e006e920 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8101c20 tx=0x7fd4e8009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:15:32.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:32.977+0000 7fd4f4ff9700 1 -- 172.21.15.123:0/405372705 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd4ec05afa0 con 0x7fd4f8101dc0 2024-02-16T22:15:33.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:33.172+0000 7fd4fe13a700 1 -- 172.21.15.123:0/405372705 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4f8106710 con 0x7fd4e006c460 2024-02-16T22:15:33.174 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:32 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-16T22:15:33.174 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:32 smithi123 ceph-mon[37161]: osdmap e20: 3 total, 3 up, 3 in 2024-02-16T22:15:33.174 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:32 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-16T22:15:33.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:32 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-16T22:15:33.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:32 smithi114 ceph-mon[29235]: osdmap e20: 3 total, 3 up, 3 in 2024-02-16T22:15:33.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:32 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-16T22:15:33.751 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:15:33 smithi114 sudo[65636]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-16T22:15:33.751 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:15:33 smithi114 sudo[65636]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T22:15:33.751 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:15:33 smithi114 sudo[65636]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T22:15:33.751 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:15:33 smithi114 sudo[65636]: pam_unix(sudo:session): session closed for user root 2024-02-16T22:15:34.036 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:15:33 smithi114 sudo[65660]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-16T22:15:34.037 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:15:33 smithi114 sudo[65660]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T22:15:34.037 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:15:33 smithi114 sudo[65660]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T22:15:34.037 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:15:33 smithi114 sudo[65660]: pam_unix(sudo:session): session closed for user root 2024-02-16T22:15:34.037 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:33 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-16T22:15:34.037 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:33 smithi114 ceph-mon[29235]: osdmap e21: 3 total, 3 up, 3 in 2024-02-16T22:15:34.037 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:33 smithi114 ceph-mon[29235]: from='client.24131 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:15:34.037 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:33 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T22:15:34.038 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:33 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T22:15:34.038 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:33 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:15:34.038 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:33 smithi114 ceph-mon[29235]: pgmap v80: 1 pgs: 1 active+clean; 449 KiB data, 79 MiB used, 268 GiB / 268 GiB avail 2024-02-16T22:15:34.038 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:33 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:34.282 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:33 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-16T22:15:34.282 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:33 smithi123 ceph-mon[37161]: osdmap e21: 3 total, 3 up, 3 in 2024-02-16T22:15:34.282 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:33 smithi123 ceph-mon[37161]: from='client.24131 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:15:34.283 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:33 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T22:15:34.283 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:33 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T22:15:34.283 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:33 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:15:34.283 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:33 smithi123 ceph-mon[37161]: pgmap v80: 1 pgs: 1 active+clean; 449 KiB data, 79 MiB used, 268 GiB / 268 GiB avail 2024-02-16T22:15:34.283 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:33 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:34.303 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:15:34 smithi114 sudo[65668]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-16T22:15:34.303 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:15:34 smithi114 sudo[65668]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T22:15:34.303 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:15:34 smithi114 sudo[65668]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T22:15:34.303 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:15:34 smithi114 sudo[65668]: pam_unix(sudo:session): session closed for user root 2024-02-16T22:15:34.560 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:15:34 smithi114 sudo[65694]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-16T22:15:34.560 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:15:34 smithi114 sudo[65694]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T22:15:34.560 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:15:34 smithi114 sudo[65694]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T22:15:34.560 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:15:34 smithi114 sudo[65694]: pam_unix(sudo:session): session closed for user root 2024-02-16T22:15:34.820 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:15:34 smithi114 sudo[67962]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-16T22:15:34.821 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:15:34 smithi114 sudo[67962]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T22:15:34.821 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:15:34 smithi114 sudo[67962]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T22:15:34.821 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:15:34 smithi114 sudo[67962]: pam_unix(sudo:session): session closed for user root 2024-02-16T22:15:35.084 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:15:34 smithi114 sudo[67969]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-16T22:15:35.084 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:15:34 smithi114 sudo[67969]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T22:15:35.084 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:15:34 smithi114 sudo[67969]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T22:15:35.084 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:15:34 smithi114 sudo[67969]: pam_unix(sudo:session): session closed for user root 2024-02-16T22:15:35.085 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:35 smithi114 sudo[67972]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-16T22:15:35.085 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:35 smithi114 sudo[67972]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T22:15:35.085 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:35 smithi114 sudo[67972]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T22:15:35.085 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:35 smithi114 sudo[67972]: pam_unix(sudo:session): session closed for user root 2024-02-16T22:15:35.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:35.350+0000 7fd4f4ff9700 1 -- 172.21.15.123:0/405372705 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fd4ec0578e0 con 0x7fd4f8101dc0 2024-02-16T22:15:35.599 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:35 smithi123 ceph-mon[37161]: osdmap e22: 3 total, 3 up, 3 in 2024-02-16T22:15:35.599 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:35 smithi123 ceph-mon[37161]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-16T22:15:35.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:35 smithi114 sudo[67976]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-16T22:15:35.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:35 smithi114 sudo[67976]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T22:15:35.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:35 smithi114 sudo[67976]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T22:15:35.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:35 smithi114 sudo[67976]: pam_unix(sudo:session): session closed for user root 2024-02-16T22:15:35.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:35 smithi114 ceph-mon[29235]: osdmap e22: 3 total, 3 up, 3 in 2024-02-16T22:15:35.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:35 smithi114 ceph-mon[29235]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-16T22:15:35.851 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:35 smithi123 sudo[40853]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-16T22:15:35.851 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:35 smithi123 sudo[40853]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T22:15:35.851 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:35 smithi123 sudo[40853]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T22:15:35.851 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:35 smithi123 sudo[40853]: pam_unix(sudo:session): session closed for user root 2024-02-16T22:15:36.248 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:35 smithi123 sudo[40856]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-16T22:15:36.248 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:35 smithi123 sudo[40856]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T22:15:36.248 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:35 smithi123 sudo[40856]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T22:15:36.248 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:35 smithi123 sudo[40856]: pam_unix(sudo:session): session closed for user root 2024-02-16T22:15:36.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:36 smithi123 ceph-mon[37161]: Detected new or changed devices on smithi114 2024-02-16T22:15:36.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:36 smithi123 ceph-mon[37161]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-16T22:15:36.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:36 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi114"}]: dispatch 2024-02-16T22:15:36.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:36 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi123"}]: dispatch 2024-02-16T22:15:36.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:36 smithi123 ceph-mon[37161]: mgrmap e19: smithi114.vcullz(active, since 2m), standbys: smithi123.inzgjd 2024-02-16T22:15:36.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:36 smithi123 ceph-mon[37161]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-16T22:15:36.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:36 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:36.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:36 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi114"}]: dispatch 2024-02-16T22:15:36.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:36 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi123"}]: dispatch 2024-02-16T22:15:36.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:36 smithi123 ceph-mon[37161]: pgmap v82: 1 pgs: 1 active+clean; 449 KiB data, 79 MiB used, 268 GiB / 268 GiB avail 2024-02-16T22:15:36.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:36 smithi123 ceph-mon[37161]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-16T22:15:36.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:36 smithi123 ceph-mon[37161]: from='client.? 172.21.15.123:0/644565584' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6eb4941-bef4-4722-9050-cd6c041f9fea"}]: dispatch 2024-02-16T22:15:36.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:36 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:36.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:36 smithi123 ceph-mon[37161]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6eb4941-bef4-4722-9050-cd6c041f9fea"}]: dispatch 2024-02-16T22:15:36.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:36 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:15:36.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:36 smithi123 ceph-mon[37161]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6eb4941-bef4-4722-9050-cd6c041f9fea"}]': finished 2024-02-16T22:15:36.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:36 smithi123 ceph-mon[37161]: osdmap e23: 4 total, 3 up, 4 in 2024-02-16T22:15:36.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:36 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:15:36.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:36 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:15:36.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:36 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:15:36.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:36 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:36.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:36 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:15:36.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:36 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:15:36.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:36 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:36.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:36 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:36.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:36 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:15:36.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:36 smithi114 ceph-mon[29235]: Detected new or changed devices on smithi114 2024-02-16T22:15:36.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:36 smithi114 ceph-mon[29235]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-16T22:15:36.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:36 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi114"}]: dispatch 2024-02-16T22:15:36.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:36 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi123"}]: dispatch 2024-02-16T22:15:36.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:36 smithi114 ceph-mon[29235]: mgrmap e19: smithi114.vcullz(active, since 2m), standbys: smithi123.inzgjd 2024-02-16T22:15:36.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:36 smithi114 ceph-mon[29235]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-16T22:15:36.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:36 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:36.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:36 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi114"}]: dispatch 2024-02-16T22:15:36.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:36 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi123"}]: dispatch 2024-02-16T22:15:36.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:36 smithi114 ceph-mon[29235]: pgmap v82: 1 pgs: 1 active+clean; 449 KiB data, 79 MiB used, 268 GiB / 268 GiB avail 2024-02-16T22:15:36.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:36 smithi114 ceph-mon[29235]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-16T22:15:36.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:36 smithi114 ceph-mon[29235]: from='client.? 172.21.15.123:0/644565584' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6eb4941-bef4-4722-9050-cd6c041f9fea"}]: dispatch 2024-02-16T22:15:36.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:36 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:36.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:36 smithi114 ceph-mon[29235]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6eb4941-bef4-4722-9050-cd6c041f9fea"}]: dispatch 2024-02-16T22:15:36.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:36 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:15:36.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:36 smithi114 ceph-mon[29235]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6eb4941-bef4-4722-9050-cd6c041f9fea"}]': finished 2024-02-16T22:15:36.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:36 smithi114 ceph-mon[29235]: osdmap e23: 4 total, 3 up, 4 in 2024-02-16T22:15:36.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:36 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:15:36.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:36 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:15:36.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:36 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:15:36.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:36 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:36.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:36 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:15:36.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:36 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:15:36.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:36 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:36.704 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:36 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:36.704 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:36 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:15:37.655 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:37 smithi114 ceph-mon[29235]: Adjusting osd_memory_target on smithi114 to 3946M 2024-02-16T22:15:37.655 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:37 smithi114 ceph-mon[29235]: from='client.? 172.21.15.123:0/1466554802' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T22:15:37.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:37 smithi123 ceph-mon[37161]: Adjusting osd_memory_target on smithi114 to 3946M 2024-02-16T22:15:37.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:37 smithi123 ceph-mon[37161]: from='client.? 172.21.15.123:0/1466554802' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T22:15:38.548 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:38 smithi114 ceph-mon[29235]: pgmap v84: 1 pgs: 1 active+clean; 449 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-16T22:15:38.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:38 smithi123 ceph-mon[37161]: pgmap v84: 1 pgs: 1 active+clean; 449 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-16T22:15:40.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:40 smithi123 ceph-mon[37161]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 218 KiB/s wr, 14 op/s 2024-02-16T22:15:40.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:40 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:40.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:40 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:40.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:40 smithi114 ceph-mon[29235]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 218 KiB/s wr, 14 op/s 2024-02-16T22:15:40.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:40 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:40.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:40 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:41.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:41 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:15:41.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:41 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:15:41.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:41 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:41.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:41 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:15:41.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:41 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:15:41.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:41 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:15:41.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:41 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:41.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:41 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:15:42.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:42 smithi123 ceph-mon[37161]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 163 KiB/s wr, 11 op/s 2024-02-16T22:15:42.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:42 smithi114 ceph-mon[29235]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 163 KiB/s wr, 11 op/s 2024-02-16T22:15:43.797 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:43 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-16T22:15:43.797 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:43 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:15:43.852 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:43 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-16T22:15:43.853 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:43 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:15:44.525 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:44 smithi123 ceph-mon[37161]: Deploying daemon osd.3 on smithi123 2024-02-16T22:15:44.525 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:44 smithi123 ceph-mon[37161]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.8 KiB/s rd, 137 KiB/s wr, 9 op/s 2024-02-16T22:15:44.872 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:44 smithi114 ceph-mon[29235]: Deploying daemon osd.3 on smithi123 2024-02-16T22:15:44.872 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:44 smithi114 ceph-mon[29235]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.8 KiB/s rd, 137 KiB/s wr, 9 op/s 2024-02-16T22:15:46.694 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:46 smithi123 ceph-mon[37161]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-02-16T22:15:46.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:46 smithi114 ceph-mon[29235]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-02-16T22:15:48.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:47 smithi123 ceph-mon[37161]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.3 KiB/s rd, 113 KiB/s wr, 7 op/s 2024-02-16T22:15:48.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:47 smithi114 ceph-mon[29235]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.3 KiB/s rd, 113 KiB/s wr, 7 op/s 2024-02-16T22:15:49.653 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:49 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:49.653 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:49 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:49.653 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:49 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:15:49.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:49 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:49.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:49 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:49.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:49 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:15:50.443 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:50 smithi123 ceph-mon[37161]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-02-16T22:15:50.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:50 smithi114 ceph-mon[29235]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-02-16T22:15:52.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:52.542+0000 7fd4f4ff9700 1 -- 172.21.15.123:0/405372705 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd4f8106710 con 0x7fd4e006c460 2024-02-16T22:15:52.545 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 3 on host 'smithi123' 2024-02-16T22:15:52.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:52.544+0000 7fd4fe13a700 1 -- 172.21.15.123:0/405372705 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd4e006c460 msgr2=0x7fd4e006e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:15:52.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:52.544+0000 7fd4fe13a700 1 --2- 172.21.15.123:0/405372705 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd4e006c460 0x7fd4e006e920 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8101c20 tx=0x7fd4e8009260 comp rx=0 tx=0).stop 2024-02-16T22:15:52.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:52.544+0000 7fd4fe13a700 1 -- 172.21.15.123:0/405372705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4f8101dc0 msgr2=0x7fd4f8196880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:15:52.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:52.544+0000 7fd4fe13a700 1 --2- 172.21.15.123:0/405372705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4f8101dc0 0x7fd4f8196880 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec000c00 tx=0x7fd4ec00f690 comp rx=0 tx=0).stop 2024-02-16T22:15:52.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:52.544+0000 7fd4fe13a700 1 -- 172.21.15.123:0/405372705 shutdown_connections 2024-02-16T22:15:52.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:52.544+0000 7fd4fe13a700 1 --2- 172.21.15.123:0/405372705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4f8100bc0 0x7fd4f8196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:15:52.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:52.544+0000 7fd4fe13a700 1 --2- 172.21.15.123:0/405372705 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd4e006c460 0x7fd4e006e920 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:15:52.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:52.544+0000 7fd4fe13a700 1 --2- 172.21.15.123:0/405372705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4f8101dc0 0x7fd4f8196880 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:15:52.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:52.544+0000 7fd4fe13a700 1 -- 172.21.15.123:0/405372705 >> 172.21.15.123:0/405372705 conn(0x7fd4f80fc140 msgr2=0x7fd4f8104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:15:52.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:52.544+0000 7fd4fe13a700 1 -- 172.21.15.123:0/405372705 shutdown_connections 2024-02-16T22:15:52.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:52.544+0000 7fd4fe13a700 1 -- 172.21.15.123:0/405372705 wait complete. 2024-02-16T22:15:52.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:52 smithi123 ceph-mon[37161]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-16T22:15:52.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:52 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:52.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:52 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:52.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:52 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:15:52.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:52 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:15:52.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:52 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:52.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:52 smithi123 ceph-mon[37161]: from='osd.3 [v2:172.21.15.123:6800/3211307893,v1:172.21.15.123:6801/3211307893]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T22:15:52.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:52 smithi123 ceph-mon[37161]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T22:15:52.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:52 smithi114 ceph-mon[29235]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-16T22:15:52.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:52 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:52.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:52 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:52.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:52 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:15:52.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:52 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:15:52.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:52 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:52.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:52 smithi114 ceph-mon[29235]: from='osd.3 [v2:172.21.15.123:6800/3211307893,v1:172.21.15.123:6801/3211307893]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T22:15:52.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:52 smithi114 ceph-mon[29235]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T22:15:53.127 DEBUG:teuthology.orchestra.run.smithi123:osd.3> sudo journalctl -f -n 0 -u ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@osd.3.service 2024-02-16T22:15:53.130 INFO:tasks.cephadm:Deploying osd.4 on smithi123 with /dev/vg_nvme/lv_3... 2024-02-16T22:15:53.130 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-16T22:15:53.685 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:53 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:53.685 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:53 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:53.685 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:53 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:15:53.685 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:53 smithi123 ceph-mon[37161]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-16T22:15:53.685 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:53 smithi123 ceph-mon[37161]: osdmap e24: 4 total, 3 up, 4 in 2024-02-16T22:15:53.685 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:53 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:15:53.685 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:53 smithi123 ceph-mon[37161]: from='osd.3 [v2:172.21.15.123:6800/3211307893,v1:172.21.15.123:6801/3211307893]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-02-16T22:15:53.685 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:53 smithi123 ceph-mon[37161]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-02-16T22:15:53.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:53 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:53.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:53 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:53.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:53 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:15:53.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:53 smithi114 ceph-mon[29235]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-16T22:15:53.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:53 smithi114 ceph-mon[29235]: osdmap e24: 4 total, 3 up, 4 in 2024-02-16T22:15:53.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:53 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:15:53.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:53 smithi114 ceph-mon[29235]: from='osd.3 [v2:172.21.15.123:6800/3211307893,v1:172.21.15.123:6801/3211307893]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-02-16T22:15:53.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:53 smithi114 ceph-mon[29235]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-02-16T22:15:54.052 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:15:53 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3[43094]: 2024-02-16T22:15:53.852+0000 7f3ecd14f700 -1 osd.3 0 waiting for initial osdmap 2024-02-16T22:15:54.052 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:15:53 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3[43094]: 2024-02-16T22:15:53.858+0000 7f3ec7741700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T22:15:54.599 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:54 smithi123 ceph-mon[37161]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-16T22:15:54.599 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:54 smithi123 ceph-mon[37161]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-02-16T22:15:54.599 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:54 smithi123 ceph-mon[37161]: osdmap e25: 4 total, 3 up, 4 in 2024-02-16T22:15:54.599 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:54 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:15:54.599 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:54 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:15:54.796 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi123/config 2024-02-16T22:15:54.947 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:54 smithi114 ceph-mon[29235]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-16T22:15:54.947 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:54 smithi114 ceph-mon[29235]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-02-16T22:15:54.947 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:54 smithi114 ceph-mon[29235]: osdmap e25: 4 total, 3 up, 4 in 2024-02-16T22:15:54.948 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:54 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:15:54.948 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:54 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:15:56.082 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:55 smithi123 ceph-mon[37161]: purged_snaps scrub starts 2024-02-16T22:15:56.082 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:55 smithi123 ceph-mon[37161]: purged_snaps scrub ok 2024-02-16T22:15:56.082 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:55 smithi123 ceph-mon[37161]: osd.3 [v2:172.21.15.123:6800/3211307893,v1:172.21.15.123:6801/3211307893] boot 2024-02-16T22:15:56.082 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:55 smithi123 ceph-mon[37161]: osdmap e26: 4 total, 4 up, 4 in 2024-02-16T22:15:56.082 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:55 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:15:56.082 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:55 smithi123 ceph-mon[37161]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-16T22:15:56.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:55 smithi114 ceph-mon[29235]: purged_snaps scrub starts 2024-02-16T22:15:56.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:55 smithi114 ceph-mon[29235]: purged_snaps scrub ok 2024-02-16T22:15:56.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:55 smithi114 ceph-mon[29235]: osd.3 [v2:172.21.15.123:6800/3211307893,v1:172.21.15.123:6801/3211307893] boot 2024-02-16T22:15:56.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:55 smithi114 ceph-mon[29235]: osdmap e26: 4 total, 4 up, 4 in 2024-02-16T22:15:56.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:55 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:15:56.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:55 smithi114 ceph-mon[29235]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-16T22:15:57.038 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:56 smithi123 ceph-mon[37161]: osdmap e27: 4 total, 4 up, 4 in 2024-02-16T22:15:57.038 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:56 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:15:57.038 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:56 smithi123 ceph-mon[37161]: Detected new or changed devices on smithi123 2024-02-16T22:15:57.038 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:56 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:57.038 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:56 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:57.038 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:56 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:15:57.038 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:56 smithi123 ceph-mon[37161]: Adjusting osd_memory_target on smithi123 to 14912M 2024-02-16T22:15:57.038 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:56 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:57.038 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:56 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:15:57.038 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:56 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:15:57.039 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:56 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:57.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:56 smithi114 ceph-mon[29235]: osdmap e27: 4 total, 4 up, 4 in 2024-02-16T22:15:57.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:56 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:15:57.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:56 smithi114 ceph-mon[29235]: Detected new or changed devices on smithi123 2024-02-16T22:15:57.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:56 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:57.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:56 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:57.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:56 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:15:57.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:56 smithi114 ceph-mon[29235]: Adjusting osd_memory_target on smithi123 to 14912M 2024-02-16T22:15:57.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:56 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:57.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:56 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:15:57.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:56 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:15:57.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:56 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:15:57.352 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-16T22:15:57.352 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-16T22:15:57.352 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2024-02-16T22:15:57.352 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2024-02-16T22:15:57.352 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128302 s, 817 MB/s 2024-02-16T22:15:57.352 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2024-02-16T22:15:57.983 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:57 smithi123 ceph-mon[37161]: osdmap e28: 4 total, 4 up, 4 in 2024-02-16T22:15:57.983 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:57 smithi123 ceph-mon[37161]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T22:15:58.005 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi123:vg_nvme/lv_3 2024-02-16T22:15:58.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:57 smithi114 ceph-mon[29235]: osdmap e28: 4 total, 4 up, 4 in 2024-02-16T22:15:58.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:57 smithi114 ceph-mon[29235]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T22:15:58.212 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi123/config 2024-02-16T22:15:59.039 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:58 smithi123 ceph-mon[37161]: osdmap e29: 4 total, 4 up, 4 in 2024-02-16T22:15:59.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:58 smithi114 ceph-mon[29235]: osdmap e29: 4 total, 4 up, 4 in 2024-02-16T22:15:59.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.807+0000 7f5d165c5700 1 -- 172.21.15.123:0/3810659347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d10101640 msgr2=0x7f5d10103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:15:59.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.807+0000 7f5d165c5700 1 --2- 172.21.15.123:0/3810659347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d10101640 0x7f5d10103a30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5d00009910 tx=0x7f5d00009c20 comp rx=0 tx=0).stop 2024-02-16T22:15:59.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.807+0000 7f5d165c5700 1 -- 172.21.15.123:0/3810659347 shutdown_connections 2024-02-16T22:15:59.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.807+0000 7f5d165c5700 1 --2- 172.21.15.123:0/3810659347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d10103f70 0x7f5d10106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:15:59.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.807+0000 7f5d165c5700 1 --2- 172.21.15.123:0/3810659347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d10101640 0x7f5d10103a30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:15:59.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.807+0000 7f5d165c5700 1 -- 172.21.15.123:0/3810659347 >> 172.21.15.123:0/3810659347 conn(0x7f5d100fb030 msgr2=0x7f5d100fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:15:59.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.807+0000 7f5d165c5700 1 -- 172.21.15.123:0/3810659347 shutdown_connections 2024-02-16T22:15:59.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.807+0000 7f5d165c5700 1 -- 172.21.15.123:0/3810659347 wait complete. 2024-02-16T22:15:59.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.808+0000 7f5d165c5700 1 Processor -- start 2024-02-16T22:15:59.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.808+0000 7f5d165c5700 1 -- start start 2024-02-16T22:15:59.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.808+0000 7f5d165c5700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d10101640 0x7f5d101962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:15:59.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.808+0000 7f5d165c5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d10103f70 0x7f5d10196820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:15:59.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.808+0000 7f5d165c5700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d10196e40 con 0x7f5d10101640 2024-02-16T22:15:59.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.808+0000 7f5d165c5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d10196f80 con 0x7f5d10103f70 2024-02-16T22:15:59.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.808+0000 7f5d0f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d10103f70 0x7f5d10196820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:15:59.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.808+0000 7f5d0f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d10103f70 0x7f5d10196820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48138/0 (socket says 172.21.15.123:48138) 2024-02-16T22:15:59.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.808+0000 7f5d0f7fe700 1 -- 172.21.15.123:0/1346136999 learned_addr learned my addr 172.21.15.123:0/1346136999 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-02-16T22:15:59.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.809+0000 7f5d0ffff700 1 --2- 172.21.15.123:0/1346136999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d10101640 0x7f5d101962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:15:59.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.809+0000 7f5d0f7fe700 1 -- 172.21.15.123:0/1346136999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d10101640 msgr2=0x7f5d101962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:15:59.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.809+0000 7f5d0f7fe700 1 --2- 172.21.15.123:0/1346136999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d10101640 0x7f5d101962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:15:59.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.809+0000 7f5d0f7fe700 1 -- 172.21.15.123:0/1346136999 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d000095f0 con 0x7f5d10103f70 2024-02-16T22:15:59.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.809+0000 7f5d0f7fe700 1 --2- 172.21.15.123:0/1346136999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d10103f70 0x7f5d10196820 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400d8d0 tx=0x7f5d0400dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:15:59.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.809+0000 7f5d0d7fa700 1 -- 172.21.15.123:0/1346136999 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d040098a0 con 0x7f5d10103f70 2024-02-16T22:15:59.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.809+0000 7f5d0d7fa700 1 -- 172.21.15.123:0/1346136999 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5d04010460 con 0x7f5d10103f70 2024-02-16T22:15:59.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.809+0000 7f5d165c5700 1 -- 172.21.15.123:0/1346136999 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d1019ba30 con 0x7f5d10103f70 2024-02-16T22:15:59.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.810+0000 7f5d0d7fa700 1 -- 172.21.15.123:0/1346136999 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d0400f5f0 con 0x7f5d10103f70 2024-02-16T22:15:59.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.810+0000 7f5d165c5700 1 -- 172.21.15.123:0/1346136999 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d1019bf80 con 0x7f5d10103f70 2024-02-16T22:15:59.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.811+0000 7f5d0d7fa700 1 -- 172.21.15.123:0/1346136999 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f5d0400f820 con 0x7f5d10103f70 2024-02-16T22:15:59.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.811+0000 7f5d165c5700 1 -- 172.21.15.123:0/1346136999 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d10190520 con 0x7f5d10103f70 2024-02-16T22:15:59.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.811+0000 7f5d0d7fa700 1 --2- 172.21.15.123:0/1346136999 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f5cf806c4e0 0x7f5cf806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:15:59.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.811+0000 7f5d0d7fa700 1 -- 172.21.15.123:0/1346136999 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3774+0+0 (secure 0 0 0) 0x7f5d0408b050 con 0x7f5d10103f70 2024-02-16T22:15:59.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.811+0000 7f5d0ffff700 1 --2- 172.21.15.123:0/1346136999 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f5cf806c4e0 0x7f5cf806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:15:59.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.812+0000 7f5d0ffff700 1 --2- 172.21.15.123:0/1346136999 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f5cf806c4e0 0x7f5cf806e9a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5d00006010 tx=0x7f5d0000b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:15:59.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:15:59.816+0000 7f5d0d7fa700 1 -- 172.21.15.123:0/1346136999 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5d040599d0 con 0x7f5d10103f70 2024-02-16T22:16:00.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:00.014+0000 7f5d165c5700 1 -- 172.21.15.123:0/1346136999 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d100611d0 con 0x7f5cf806c4e0 2024-02-16T22:16:00.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:59 smithi114 ceph-mon[29235]: osdmap e30: 4 total, 4 up, 4 in 2024-02-16T22:16:00.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:15:59 smithi114 ceph-mon[29235]: pgmap v102: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-16T22:16:00.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:59 smithi123 ceph-mon[37161]: osdmap e30: 4 total, 4 up, 4 in 2024-02-16T22:16:00.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:15:59 smithi123 ceph-mon[37161]: pgmap v102: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-16T22:16:01.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:00 smithi123 ceph-mon[37161]: from='client.24155 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:16:01.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:00 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T22:16:01.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:00 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T22:16:01.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:00 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:16:01.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:00 smithi114 ceph-mon[29235]: from='client.24155 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:16:01.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:00 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T22:16:01.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:00 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T22:16:01.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:00 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:16:02.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:01 smithi123 ceph-mon[37161]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-02-16T22:16:02.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:01 smithi114 ceph-mon[29235]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-02-16T22:16:03.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:02 smithi114 ceph-mon[29235]: from='client.? 172.21.15.123:0/3229152985' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9caa348-26f4-4ed5-a64e-59231fc48546"}]: dispatch 2024-02-16T22:16:03.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:02 smithi114 ceph-mon[29235]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9caa348-26f4-4ed5-a64e-59231fc48546"}]: dispatch 2024-02-16T22:16:03.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:02 smithi114 ceph-mon[29235]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9caa348-26f4-4ed5-a64e-59231fc48546"}]': finished 2024-02-16T22:16:03.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:02 smithi114 ceph-mon[29235]: osdmap e31: 5 total, 4 up, 5 in 2024-02-16T22:16:03.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:02 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:16:03.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:02 smithi123 ceph-mon[37161]: from='client.? 172.21.15.123:0/3229152985' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9caa348-26f4-4ed5-a64e-59231fc48546"}]: dispatch 2024-02-16T22:16:03.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:02 smithi123 ceph-mon[37161]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9caa348-26f4-4ed5-a64e-59231fc48546"}]: dispatch 2024-02-16T22:16:03.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:02 smithi123 ceph-mon[37161]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9caa348-26f4-4ed5-a64e-59231fc48546"}]': finished 2024-02-16T22:16:03.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:02 smithi123 ceph-mon[37161]: osdmap e31: 5 total, 4 up, 5 in 2024-02-16T22:16:03.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:02 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:16:04.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:03 smithi114 ceph-mon[29235]: from='client.? 172.21.15.123:0/1723764962' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T22:16:04.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:03 smithi114 ceph-mon[29235]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-16T22:16:04.245 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:03 smithi123 ceph-mon[37161]: from='client.? 172.21.15.123:0/1723764962' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T22:16:04.245 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:03 smithi123 ceph-mon[37161]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-16T22:16:06.738 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:06 smithi123 ceph-mon[37161]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-02-16T22:16:06.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:06 smithi114 ceph-mon[29235]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-02-16T22:16:08.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:07 smithi114 ceph-mon[29235]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T22:16:08.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:07 smithi123 ceph-mon[37161]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T22:16:10.688 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:10 smithi123 ceph-mon[37161]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T22:16:10.688 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:10 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-16T22:16:10.688 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:10 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:16:10.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:10 smithi114 ceph-mon[29235]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T22:16:10.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:10 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-16T22:16:10.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:10 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:16:11.738 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:11 smithi123 ceph-mon[37161]: Deploying daemon osd.4 on smithi123 2024-02-16T22:16:11.738 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:11 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:16:11.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:11 smithi114 ceph-mon[29235]: Deploying daemon osd.4 on smithi123 2024-02-16T22:16:11.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:11 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:16:12.532 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:12 smithi123 ceph-mon[37161]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T22:16:12.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:12 smithi114 ceph-mon[29235]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T22:16:14.590 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:14 smithi123 ceph-mon[37161]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T22:16:14.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:14 smithi114 ceph-mon[29235]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T22:16:16.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:16 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:16.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:16 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:16.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:16 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:16:16.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:16 smithi114 ceph-mon[29235]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T22:16:16.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:16 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:16.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:16 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:16.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:16 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:16:16.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:16 smithi123 ceph-mon[37161]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T22:16:18.044 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:17 smithi123 ceph-mon[37161]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T22:16:18.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:17 smithi114 ceph-mon[29235]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T22:16:19.028 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 4 on host 'smithi123' 2024-02-16T22:16:19.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:19.026+0000 7f5d0d7fa700 1 -- 172.21.15.123:0/1346136999 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5d100611d0 con 0x7f5cf806c4e0 2024-02-16T22:16:19.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:19.029+0000 7f5d165c5700 1 -- 172.21.15.123:0/1346136999 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f5cf806c4e0 msgr2=0x7f5cf806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:16:19.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:19.029+0000 7f5d165c5700 1 --2- 172.21.15.123:0/1346136999 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f5cf806c4e0 0x7f5cf806e9a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5d00006010 tx=0x7f5d0000b540 comp rx=0 tx=0).stop 2024-02-16T22:16:19.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:19.029+0000 7f5d165c5700 1 -- 172.21.15.123:0/1346136999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d10103f70 msgr2=0x7f5d10196820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:16:19.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:19.029+0000 7f5d165c5700 1 --2- 172.21.15.123:0/1346136999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d10103f70 0x7f5d10196820 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400d8d0 tx=0x7f5d0400dbe0 comp rx=0 tx=0).stop 2024-02-16T22:16:19.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:19.029+0000 7f5d165c5700 1 -- 172.21.15.123:0/1346136999 shutdown_connections 2024-02-16T22:16:19.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:19.029+0000 7f5d165c5700 1 --2- 172.21.15.123:0/1346136999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d10101640 0x7f5d101962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:19.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:19.029+0000 7f5d165c5700 1 --2- 172.21.15.123:0/1346136999 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f5cf806c4e0 0x7f5cf806e9a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:19.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:19.029+0000 7f5d165c5700 1 --2- 172.21.15.123:0/1346136999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d10103f70 0x7f5d10196820 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:19.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:19.029+0000 7f5d165c5700 1 -- 172.21.15.123:0/1346136999 >> 172.21.15.123:0/1346136999 conn(0x7f5d100fb030 msgr2=0x7f5d100fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:16:19.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:19.029+0000 7f5d165c5700 1 -- 172.21.15.123:0/1346136999 shutdown_connections 2024-02-16T22:16:19.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:19.029+0000 7f5d165c5700 1 -- 172.21.15.123:0/1346136999 wait complete. 2024-02-16T22:16:19.629 DEBUG:teuthology.orchestra.run.smithi123:osd.4> sudo journalctl -f -n 0 -u ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@osd.4.service 2024-02-16T22:16:19.632 INFO:tasks.cephadm:Deploying osd.5 on smithi123 with /dev/vg_nvme/lv_2... 2024-02-16T22:16:19.632 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-16T22:16:19.878 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi123/config 2024-02-16T22:16:20.134 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:20 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:20.134 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:20 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:20.134 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:20 smithi123 ceph-mon[37161]: from='osd.4 [v2:172.21.15.123:6808/3994684204,v1:172.21.15.123:6809/3994684204]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T22:16:20.134 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:20 smithi123 ceph-mon[37161]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T22:16:20.134 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:20 smithi123 ceph-mon[37161]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T22:16:20.134 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:20 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:20.134 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:20 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:20.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:20 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:20.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:20 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:20.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:20 smithi114 ceph-mon[29235]: from='osd.4 [v2:172.21.15.123:6808/3994684204,v1:172.21.15.123:6809/3994684204]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T22:16:20.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:20 smithi114 ceph-mon[29235]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T22:16:20.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:20 smithi114 ceph-mon[29235]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T22:16:20.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:20 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:20.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:20 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:21.232 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:21 smithi123 ceph-mon[37161]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-16T22:16:21.233 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:21 smithi123 ceph-mon[37161]: osdmap e32: 5 total, 4 up, 5 in 2024-02-16T22:16:21.233 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:21 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:16:21.233 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:21 smithi123 ceph-mon[37161]: from='osd.4 [v2:172.21.15.123:6808/3994684204,v1:172.21.15.123:6809/3994684204]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-02-16T22:16:21.233 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:21 smithi123 ceph-mon[37161]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-02-16T22:16:21.233 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:16:21 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4[49978]: 2024-02-16T22:16:21.032+0000 7fd5fa707700 -1 osd.4 0 waiting for initial osdmap 2024-02-16T22:16:21.233 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:16:21 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4[49978]: 2024-02-16T22:16:21.039+0000 7fd5f2cf5700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T22:16:21.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:21 smithi114 ceph-mon[29235]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-16T22:16:21.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:21 smithi114 ceph-mon[29235]: osdmap e32: 5 total, 4 up, 5 in 2024-02-16T22:16:21.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:21 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:16:21.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:21 smithi114 ceph-mon[29235]: from='osd.4 [v2:172.21.15.123:6808/3994684204,v1:172.21.15.123:6809/3994684204]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-02-16T22:16:21.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:21 smithi114 ceph-mon[29235]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-02-16T22:16:21.739 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-16T22:16:21.739 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-16T22:16:21.739 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2024-02-16T22:16:21.739 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2024-02-16T22:16:21.739 INFO:teuthology.orchestra.run.smithi123.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0130384 s, 804 MB/s 2024-02-16T22:16:21.739 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2024-02-16T22:16:22.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:22 smithi123 ceph-mon[37161]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-02-16T22:16:22.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:22 smithi123 ceph-mon[37161]: osdmap e33: 5 total, 4 up, 5 in 2024-02-16T22:16:22.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:22 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:16:22.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:22 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:16:22.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:22 smithi123 ceph-mon[37161]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T22:16:22.359 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi123:vg_nvme/lv_2 2024-02-16T22:16:22.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:22 smithi114 ceph-mon[29235]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-02-16T22:16:22.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:22 smithi114 ceph-mon[29235]: osdmap e33: 5 total, 4 up, 5 in 2024-02-16T22:16:22.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:22 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:16:22.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:22 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:16:22.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:22 smithi114 ceph-mon[29235]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-16T22:16:23.201 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:23 smithi123 ceph-mon[37161]: purged_snaps scrub starts 2024-02-16T22:16:23.202 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:23 smithi123 ceph-mon[37161]: purged_snaps scrub ok 2024-02-16T22:16:23.202 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:23 smithi123 ceph-mon[37161]: osd.4 [v2:172.21.15.123:6808/3994684204,v1:172.21.15.123:6809/3994684204] boot 2024-02-16T22:16:23.202 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:23 smithi123 ceph-mon[37161]: osdmap e34: 5 total, 5 up, 5 in 2024-02-16T22:16:23.202 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:23 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:16:23.243 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi123/config 2024-02-16T22:16:23.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:23 smithi114 ceph-mon[29235]: purged_snaps scrub starts 2024-02-16T22:16:23.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:23 smithi114 ceph-mon[29235]: purged_snaps scrub ok 2024-02-16T22:16:23.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:23 smithi114 ceph-mon[29235]: osd.4 [v2:172.21.15.123:6808/3994684204,v1:172.21.15.123:6809/3994684204] boot 2024-02-16T22:16:23.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:23 smithi114 ceph-mon[29235]: osdmap e34: 5 total, 5 up, 5 in 2024-02-16T22:16:23.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:23 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:16:24.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:24 smithi123 ceph-mon[37161]: osdmap e35: 5 total, 5 up, 5 in 2024-02-16T22:16:24.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:24 smithi123 ceph-mon[37161]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T22:16:24.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:24 smithi114 ceph-mon[29235]: osdmap e35: 5 total, 5 up, 5 in 2024-02-16T22:16:24.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:24 smithi114 ceph-mon[29235]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T22:16:25.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.276+0000 7f66e714a700 1 -- 172.21.15.123:0/3176290417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e0103f70 msgr2=0x7f66e0106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:16:25.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.276+0000 7f66e714a700 1 --2- 172.21.15.123:0/3176290417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e0103f70 0x7f66e0106360 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f66cc009910 tx=0x7f66cc009c20 comp rx=0 tx=0).stop 2024-02-16T22:16:25.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.276+0000 7f66e714a700 1 -- 172.21.15.123:0/3176290417 shutdown_connections 2024-02-16T22:16:25.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.276+0000 7f66e714a700 1 --2- 172.21.15.123:0/3176290417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e0103f70 0x7f66e0106360 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:25.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.276+0000 7f66e714a700 1 --2- 172.21.15.123:0/3176290417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66e0101640 0x7f66e0103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:25.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.276+0000 7f66e714a700 1 -- 172.21.15.123:0/3176290417 >> 172.21.15.123:0/3176290417 conn(0x7f66e00fb050 msgr2=0x7f66e00fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:16:25.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.276+0000 7f66e714a700 1 -- 172.21.15.123:0/3176290417 shutdown_connections 2024-02-16T22:16:25.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.276+0000 7f66e714a700 1 -- 172.21.15.123:0/3176290417 wait complete. 2024-02-16T22:16:25.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.277+0000 7f66e714a700 1 Processor -- start 2024-02-16T22:16:25.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.277+0000 7f66e714a700 1 -- start start 2024-02-16T22:16:25.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.277+0000 7f66e714a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66e0101640 0x7f66e0198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:16:25.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.277+0000 7f66e714a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e0103f70 0x7f66e0198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:16:25.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.277+0000 7f66e714a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e01990c0 con 0x7f66e0101640 2024-02-16T22:16:25.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.277+0000 7f66e714a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e0199200 con 0x7f66e0103f70 2024-02-16T22:16:25.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.278+0000 7f66dffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e0103f70 0x7f66e0198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:16:25.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.278+0000 7f66e4ee6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66e0101640 0x7f66e0198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:16:25.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.278+0000 7f66e4ee6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66e0101640 0x7f66e0198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.123:54912/0 (socket says 172.21.15.123:54912) 2024-02-16T22:16:25.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.278+0000 7f66e4ee6700 1 -- 172.21.15.123:0/3983619930 learned_addr learned my addr 172.21.15.123:0/3983619930 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-02-16T22:16:25.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.278+0000 7f66dffff700 1 -- 172.21.15.123:0/3983619930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66e0101640 msgr2=0x7f66e0198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:16:25.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.278+0000 7f66dffff700 1 --2- 172.21.15.123:0/3983619930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66e0101640 0x7f66e0198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:25.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.278+0000 7f66dffff700 1 -- 172.21.15.123:0/3983619930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66cc0095f0 con 0x7f66e0103f70 2024-02-16T22:16:25.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.278+0000 7f66e4ee6700 1 --2- 172.21.15.123:0/3983619930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66e0101640 0x7f66e0198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:16:25.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.278+0000 7f66dffff700 1 --2- 172.21.15.123:0/3983619930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e0103f70 0x7f66e0198aa0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f66cc009910 tx=0x7f66cc00bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:16:25.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.278+0000 7f66ddffb700 1 -- 172.21.15.123:0/3983619930 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66cc01d070 con 0x7f66e0103f70 2024-02-16T22:16:25.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.279+0000 7f66e714a700 1 -- 172.21.15.123:0/3983619930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e019dc50 con 0x7f66e0103f70 2024-02-16T22:16:25.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.279+0000 7f66ddffb700 1 -- 172.21.15.123:0/3983619930 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f66cc00f460 con 0x7f66e0103f70 2024-02-16T22:16:25.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.279+0000 7f66ddffb700 1 -- 172.21.15.123:0/3983619930 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66cc021620 con 0x7f66e0103f70 2024-02-16T22:16:25.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.279+0000 7f66e714a700 1 -- 172.21.15.123:0/3983619930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e019e140 con 0x7f66e0103f70 2024-02-16T22:16:25.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.280+0000 7f66e714a700 1 -- 172.21.15.123:0/3983619930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66e00fcc20 con 0x7f66e0103f70 2024-02-16T22:16:25.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.280+0000 7f66ddffb700 1 -- 172.21.15.123:0/3983619930 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f66cc00fa40 con 0x7f66e0103f70 2024-02-16T22:16:25.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.281+0000 7f66ddffb700 1 --2- 172.21.15.123:0/3983619930 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f66d006c4e0 0x7f66d006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:16:25.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.281+0000 7f66ddffb700 1 -- 172.21.15.123:0/3983619930 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4206+0+0 (secure 0 0 0) 0x7f66cc08c530 con 0x7f66e0103f70 2024-02-16T22:16:25.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.281+0000 7f66e4ee6700 1 --2- 172.21.15.123:0/3983619930 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f66d006c4e0 0x7f66d006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:16:25.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.282+0000 7f66e4ee6700 1 --2- 172.21.15.123:0/3983619930 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f66d006c4e0 0x7f66d006e9a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f66d4005950 tx=0x7f66d400b500 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:16:25.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.288+0000 7f66ddffb700 1 -- 172.21.15.123:0/3983619930 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f66cc05ac90 con 0x7f66e0103f70 2024-02-16T22:16:25.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:25.493+0000 7f66e714a700 1 -- 172.21.15.123:0/3983619930 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f66e00611d0 con 0x7f66d006c4e0 2024-02-16T22:16:26.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:26 smithi123 ceph-mon[37161]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T22:16:26.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:26 smithi123 ceph-mon[37161]: from='client.24173 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:16:26.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:26 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T22:16:26.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:26 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T22:16:26.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:26 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:16:26.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:26 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:26.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:26 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:26.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:26 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:16:26.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:26 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:16:26.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:26 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:26.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:26 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:16:26.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:26 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:16:26.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:26 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:26.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:26 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:16:26.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:26 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:16:26.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:26 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:16:26.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:26 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:26.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:26 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:16:26.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:26 smithi114 ceph-mon[29235]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T22:16:26.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:26 smithi114 ceph-mon[29235]: from='client.24173 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:16:26.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T22:16:26.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T22:16:26.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:16:26.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:26.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:26.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:16:26.953 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:16:26.953 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:26.953 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:16:26.953 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:16:26.953 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:26.953 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:16:26.953 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:16:26.953 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:16:26.953 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:26.953 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:16:27.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:27 smithi123 ceph-mon[37161]: Detected new or changed devices on smithi123 2024-02-16T22:16:27.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:27 smithi123 ceph-mon[37161]: Adjusting osd_memory_target on smithi123 to 7456M 2024-02-16T22:16:27.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:27 smithi114 ceph-mon[29235]: Detected new or changed devices on smithi123 2024-02-16T22:16:27.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:27 smithi114 ceph-mon[29235]: Adjusting osd_memory_target on smithi123 to 7456M 2024-02-16T22:16:28.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:28 smithi123 ceph-mon[37161]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T22:16:28.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:28 smithi123 ceph-mon[37161]: from='client.? 172.21.15.123:0/1460424251' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79d81af1-2361-430d-bc76-35ef74aa6bad"}]: dispatch 2024-02-16T22:16:28.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:28 smithi123 ceph-mon[37161]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79d81af1-2361-430d-bc76-35ef74aa6bad"}]: dispatch 2024-02-16T22:16:28.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:28 smithi123 ceph-mon[37161]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79d81af1-2361-430d-bc76-35ef74aa6bad"}]': finished 2024-02-16T22:16:28.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:28 smithi123 ceph-mon[37161]: osdmap e36: 6 total, 5 up, 6 in 2024-02-16T22:16:28.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:28 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:16:28.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:28 smithi114 ceph-mon[29235]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T22:16:28.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:28 smithi114 ceph-mon[29235]: from='client.? 172.21.15.123:0/1460424251' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79d81af1-2361-430d-bc76-35ef74aa6bad"}]: dispatch 2024-02-16T22:16:28.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:28 smithi114 ceph-mon[29235]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79d81af1-2361-430d-bc76-35ef74aa6bad"}]: dispatch 2024-02-16T22:16:28.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:28 smithi114 ceph-mon[29235]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79d81af1-2361-430d-bc76-35ef74aa6bad"}]': finished 2024-02-16T22:16:28.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:28 smithi114 ceph-mon[29235]: osdmap e36: 6 total, 5 up, 6 in 2024-02-16T22:16:28.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:28 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:16:29.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:29 smithi123 ceph-mon[37161]: from='client.? 172.21.15.123:0/3569695303' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T22:16:29.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:29 smithi114 ceph-mon[29235]: from='client.? 172.21.15.123:0/3569695303' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T22:16:30.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:30 smithi123 ceph-mon[37161]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T22:16:30.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:30 smithi114 ceph-mon[29235]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T22:16:32.748 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:32 smithi123 ceph-mon[37161]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T22:16:32.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:32 smithi114 ceph-mon[29235]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T22:16:34.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:34 smithi123 ceph-mon[37161]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T22:16:34.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:34 smithi114 ceph-mon[29235]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T22:16:36.787 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:36 smithi123 ceph-mon[37161]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T22:16:36.787 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:36 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-16T22:16:36.787 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:36 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:16:36.787 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:36 smithi123 ceph-mon[37161]: Deploying daemon osd.5 on smithi123 2024-02-16T22:16:36.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:36 smithi114 ceph-mon[29235]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T22:16:36.953 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:36 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-16T22:16:36.953 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:36 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:16:36.953 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:36 smithi114 ceph-mon[29235]: Deploying daemon osd.5 on smithi123 2024-02-16T22:16:38.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:37 smithi114 ceph-mon[29235]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T22:16:38.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:37 smithi123 ceph-mon[37161]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T22:16:40.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:40 smithi114 ceph-mon[29235]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T22:16:40.732 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:40 smithi123 ceph-mon[37161]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T22:16:42.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:41 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:42.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:41 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:42.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:41 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:16:42.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:41 smithi123 ceph-mon[37161]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T22:16:42.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:41 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:16:42.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:41 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:42.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:41 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:42.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:41 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:16:42.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:41 smithi114 ceph-mon[29235]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T22:16:42.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:41 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:16:44.644 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:44 smithi123 ceph-mon[37161]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T22:16:44.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:44 smithi114 ceph-mon[29235]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T22:16:44.962 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 5 on host 'smithi123' 2024-02-16T22:16:44.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:44.960+0000 7f66ddffb700 1 -- 172.21.15.123:0/3983619930 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f66e00611d0 con 0x7f66d006c4e0 2024-02-16T22:16:44.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:44.962+0000 7f66e714a700 1 -- 172.21.15.123:0/3983619930 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f66d006c4e0 msgr2=0x7f66d006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:16:44.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:44.962+0000 7f66e714a700 1 --2- 172.21.15.123:0/3983619930 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f66d006c4e0 0x7f66d006e9a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f66d4005950 tx=0x7f66d400b500 comp rx=0 tx=0).stop 2024-02-16T22:16:44.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:44.962+0000 7f66e714a700 1 -- 172.21.15.123:0/3983619930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e0103f70 msgr2=0x7f66e0198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:16:44.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:44.962+0000 7f66e714a700 1 --2- 172.21.15.123:0/3983619930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e0103f70 0x7f66e0198aa0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f66cc009910 tx=0x7f66cc00bd70 comp rx=0 tx=0).stop 2024-02-16T22:16:44.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:44.963+0000 7f66e714a700 1 -- 172.21.15.123:0/3983619930 shutdown_connections 2024-02-16T22:16:44.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:44.963+0000 7f66e714a700 1 --2- 172.21.15.123:0/3983619930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66e0101640 0x7f66e0198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:44.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:44.963+0000 7f66e714a700 1 --2- 172.21.15.123:0/3983619930 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f66d006c4e0 0x7f66d006e9a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:44.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:44.963+0000 7f66e714a700 1 --2- 172.21.15.123:0/3983619930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e0103f70 0x7f66e0198aa0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:44.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:44.963+0000 7f66e714a700 1 -- 172.21.15.123:0/3983619930 >> 172.21.15.123:0/3983619930 conn(0x7f66e00fb050 msgr2=0x7f66e0100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:16:44.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:44.963+0000 7f66e714a700 1 -- 172.21.15.123:0/3983619930 shutdown_connections 2024-02-16T22:16:44.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:44.963+0000 7f66e714a700 1 -- 172.21.15.123:0/3983619930 wait complete. 2024-02-16T22:16:45.610 DEBUG:teuthology.orchestra.run.smithi123:osd.5> sudo journalctl -f -n 0 -u ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@osd.5.service 2024-02-16T22:16:45.613 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-02-16T22:16:45.613 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph osd stat -f json 2024-02-16T22:16:45.643 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:45 smithi114 ceph-mon[29235]: from='osd.5 [v2:172.21.15.123:6816/2122343545,v1:172.21.15.123:6817/2122343545]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T22:16:45.644 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:45 smithi114 ceph-mon[29235]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T22:16:45.644 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:45 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:45.644 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:45 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:45.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:45 smithi123 ceph-mon[37161]: from='osd.5 [v2:172.21.15.123:6816/2122343545,v1:172.21.15.123:6817/2122343545]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T22:16:45.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:45 smithi123 ceph-mon[37161]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T22:16:45.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:45 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:45.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:45 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:45.833 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:16:46.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:46 smithi114 ceph-mon[29235]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T22:16:46.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:46 smithi114 ceph-mon[29235]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-16T22:16:46.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:46 smithi114 ceph-mon[29235]: osdmap e37: 6 total, 5 up, 6 in 2024-02-16T22:16:46.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:46 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:16:46.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:46 smithi114 ceph-mon[29235]: from='osd.5 [v2:172.21.15.123:6816/2122343545,v1:172.21.15.123:6817/2122343545]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-02-16T22:16:46.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:46 smithi114 ceph-mon[29235]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-02-16T22:16:46.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:46 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:46.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:46 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:46.776 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:46 smithi123 ceph-mon[37161]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T22:16:46.777 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:46 smithi123 ceph-mon[37161]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-16T22:16:46.777 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:46 smithi123 ceph-mon[37161]: osdmap e37: 6 total, 5 up, 6 in 2024-02-16T22:16:46.777 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:46 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:16:46.777 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:46 smithi123 ceph-mon[37161]: from='osd.5 [v2:172.21.15.123:6816/2122343545,v1:172.21.15.123:6817/2122343545]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-02-16T22:16:46.777 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:46 smithi123 ceph-mon[37161]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-02-16T22:16:46.777 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:46 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:46.777 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:46 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:46.777 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:16:46 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[56928]: 2024-02-16T22:16:46.520+0000 7f3d4478f700 -1 osd.5 0 waiting for initial osdmap 2024-02-16T22:16:46.778 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:16:46 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[56928]: 2024-02-16T22:16:46.528+0000 7f3d40d85700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T22:16:47.542 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:47 smithi114 ceph-mon[29235]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-02-16T22:16:47.542 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:47 smithi114 ceph-mon[29235]: osdmap e38: 6 total, 5 up, 6 in 2024-02-16T22:16:47.542 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:47 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:16:47.542 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:47 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:16:47.630 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:47 smithi123 ceph-mon[37161]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-02-16T22:16:47.630 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:47 smithi123 ceph-mon[37161]: osdmap e38: 6 total, 5 up, 6 in 2024-02-16T22:16:47.630 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:47 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:16:47.630 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:47 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:16:47.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.638+0000 7f51be9f5700 1 -- 172.21.15.114:0/2881089947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51b8101d80 msgr2=0x7f51b81021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:16:47.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.638+0000 7f51be9f5700 1 --2- 172.21.15.114:0/2881089947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51b8101d80 0x7f51b81021e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009910 tx=0x7f51ac009c20 comp rx=0 tx=0).stop 2024-02-16T22:16:47.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.638+0000 7f51be9f5700 1 -- 172.21.15.114:0/2881089947 shutdown_connections 2024-02-16T22:16:47.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.638+0000 7f51be9f5700 1 --2- 172.21.15.114:0/2881089947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51b8101d80 0x7f51b81021e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:47.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.638+0000 7f51be9f5700 1 --2- 172.21.15.114:0/2881089947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51b8100b80 0x7f51b8100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:47.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.638+0000 7f51be9f5700 1 -- 172.21.15.114:0/2881089947 >> 172.21.15.114:0/2881089947 conn(0x7f51b80fc100 msgr2=0x7f51b80fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:16:47.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.638+0000 7f51be9f5700 1 -- 172.21.15.114:0/2881089947 shutdown_connections 2024-02-16T22:16:47.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.638+0000 7f51be9f5700 1 -- 172.21.15.114:0/2881089947 wait complete. 2024-02-16T22:16:47.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.638+0000 7f51be9f5700 1 Processor -- start 2024-02-16T22:16:47.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.638+0000 7f51be9f5700 1 -- start start 2024-02-16T22:16:47.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.639+0000 7f51be9f5700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51b8100b80 0x7f51b8196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:16:47.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.639+0000 7f51be9f5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51b8101d80 0x7f51b8196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:16:47.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.639+0000 7f51be9f5700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b8196e70 con 0x7f51b8100b80 2024-02-16T22:16:47.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.639+0000 7f51be9f5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b8196fb0 con 0x7f51b8101d80 2024-02-16T22:16:47.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.639+0000 7f51b7fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51b8100b80 0x7f51b8196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:16:47.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.639+0000 7f51b7fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51b8100b80 0x7f51b8196310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48466/0 (socket says 172.21.15.114:48466) 2024-02-16T22:16:47.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.639+0000 7f51b7fff700 1 -- 172.21.15.114:0/149877985 learned_addr learned my addr 172.21.15.114:0/149877985 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:16:47.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.639+0000 7f51b77fe700 1 --2- 172.21.15.114:0/149877985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51b8101d80 0x7f51b8196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:16:47.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.639+0000 7f51b7fff700 1 -- 172.21.15.114:0/149877985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51b8101d80 msgr2=0x7f51b8196850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:16:47.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.639+0000 7f51b7fff700 1 --2- 172.21.15.114:0/149877985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51b8101d80 0x7f51b8196850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:47.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.639+0000 7f51b7fff700 1 -- 172.21.15.114:0/149877985 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac0095f0 con 0x7f51b8100b80 2024-02-16T22:16:47.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.639+0000 7f51b7fff700 1 --2- 172.21.15.114:0/149877985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51b8100b80 0x7f51b8196310 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f51a800d900 tx=0x7f51a800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:16:47.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.639+0000 7f51b57fa700 1 -- 172.21.15.114:0/149877985 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51a800b920 con 0x7f51b8100b80 2024-02-16T22:16:47.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.639+0000 7f51b57fa700 1 -- 172.21.15.114:0/149877985 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f51a8010460 con 0x7f51b8100b80 2024-02-16T22:16:47.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.639+0000 7f51be9f5700 1 -- 172.21.15.114:0/149877985 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b819ba60 con 0x7f51b8100b80 2024-02-16T22:16:47.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.640+0000 7f51b57fa700 1 -- 172.21.15.114:0/149877985 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51a800f590 con 0x7f51b8100b80 2024-02-16T22:16:47.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.640+0000 7f51be9f5700 1 -- 172.21.15.114:0/149877985 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b819bfb0 con 0x7f51b8100b80 2024-02-16T22:16:47.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.640+0000 7f51be9f5700 1 -- 172.21.15.114:0/149877985 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b8109870 con 0x7f51b8100b80 2024-02-16T22:16:47.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.641+0000 7f51b57fa700 1 -- 172.21.15.114:0/149877985 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f51a80105d0 con 0x7f51b8100b80 2024-02-16T22:16:47.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.641+0000 7f51b57fa700 1 --2- 172.21.15.114:0/149877985 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f51a006c4e0 0x7f51a006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:16:47.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.641+0000 7f51b57fa700 1 -- 172.21.15.114:0/149877985 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f51a808b0a0 con 0x7f51b8100b80 2024-02-16T22:16:47.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.641+0000 7f51b77fe700 1 --2- 172.21.15.114:0/149877985 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f51a006c4e0 0x7f51a006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:16:47.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.642+0000 7f51b77fe700 1 --2- 172.21.15.114:0/149877985 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f51a006c4e0 0x7f51a006e9a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f51ac00b5c0 tx=0x7f51ac005ee0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:16:47.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.646+0000 7f51b57fa700 1 -- 172.21.15.114:0/149877985 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f51a80596d0 con 0x7f51b8100b80 2024-02-16T22:16:47.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.836+0000 7f51be9f5700 1 -- 172.21.15.114:0/149877985 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f51b80733e0 con 0x7f51b8100b80 2024-02-16T22:16:47.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.836+0000 7f51b57fa700 1 -- 172.21.15.114:0/149877985 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v39) v1 ==== 74+0+130 (secure 0 0 0) 0x7f51a8059260 con 0x7f51b8100b80 2024-02-16T22:16:47.838 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:16:47.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.838+0000 7f51be9f5700 1 -- 172.21.15.114:0/149877985 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f51a006c4e0 msgr2=0x7f51a006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:16:47.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.838+0000 7f51be9f5700 1 --2- 172.21.15.114:0/149877985 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f51a006c4e0 0x7f51a006e9a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f51ac00b5c0 tx=0x7f51ac005ee0 comp rx=0 tx=0).stop 2024-02-16T22:16:47.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.838+0000 7f51be9f5700 1 -- 172.21.15.114:0/149877985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51b8100b80 msgr2=0x7f51b8196310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:16:47.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.838+0000 7f51be9f5700 1 --2- 172.21.15.114:0/149877985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51b8100b80 0x7f51b8196310 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f51a800d900 tx=0x7f51a800dc10 comp rx=0 tx=0).stop 2024-02-16T22:16:47.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.838+0000 7f51be9f5700 1 -- 172.21.15.114:0/149877985 shutdown_connections 2024-02-16T22:16:47.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.838+0000 7f51be9f5700 1 --2- 172.21.15.114:0/149877985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51b8100b80 0x7f51b8196310 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:47.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.838+0000 7f51be9f5700 1 --2- 172.21.15.114:0/149877985 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f51a006c4e0 0x7f51a006e9a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:47.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.838+0000 7f51be9f5700 1 --2- 172.21.15.114:0/149877985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51b8101d80 0x7f51b8196850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:47.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.839+0000 7f51be9f5700 1 -- 172.21.15.114:0/149877985 >> 172.21.15.114:0/149877985 conn(0x7f51b80fc100 msgr2=0x7f51b8104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:16:47.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.839+0000 7f51be9f5700 1 -- 172.21.15.114:0/149877985 shutdown_connections 2024-02-16T22:16:47.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:47.839+0000 7f51be9f5700 1 -- 172.21.15.114:0/149877985 wait complete. 2024-02-16T22:16:48.516 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":39,"num_osds":6,"num_up_osds":6,"osd_up_since":1708121807,"num_in_osds":6,"osd_in_since":1708121788,"num_remapped_pgs":0} 2024-02-16T22:16:48.516 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-16T22:16:48.768 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:16:48.798 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:48 smithi114 ceph-mon[29235]: purged_snaps scrub starts 2024-02-16T22:16:48.799 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:48 smithi114 ceph-mon[29235]: purged_snaps scrub ok 2024-02-16T22:16:48.799 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:48 smithi114 ceph-mon[29235]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T22:16:48.799 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:48 smithi114 ceph-mon[29235]: osd.5 [v2:172.21.15.123:6816/2122343545,v1:172.21.15.123:6817/2122343545] boot 2024-02-16T22:16:48.799 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:48 smithi114 ceph-mon[29235]: osdmap e39: 6 total, 6 up, 6 in 2024-02-16T22:16:48.799 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:48 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:16:48.799 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:48 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/149877985' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-16T22:16:48.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:48 smithi123 ceph-mon[37161]: purged_snaps scrub starts 2024-02-16T22:16:48.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:48 smithi123 ceph-mon[37161]: purged_snaps scrub ok 2024-02-16T22:16:48.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:48 smithi123 ceph-mon[37161]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-16T22:16:48.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:48 smithi123 ceph-mon[37161]: osd.5 [v2:172.21.15.123:6816/2122343545,v1:172.21.15.123:6817/2122343545] boot 2024-02-16T22:16:48.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:48 smithi123 ceph-mon[37161]: osdmap e39: 6 total, 6 up, 6 in 2024-02-16T22:16:48.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:48 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:16:48.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:48 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/149877985' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-16T22:16:50.170 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:49 smithi114 ceph-mon[29235]: osdmap e40: 6 total, 6 up, 6 in 2024-02-16T22:16:50.171 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:49 smithi114 ceph-mon[29235]: Detected new or changed devices on smithi123 2024-02-16T22:16:50.171 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:49 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:50.171 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:49 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:50.171 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:49 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:16:50.171 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:49 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:16:50.171 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:49 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:16:50.171 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:49 smithi114 ceph-mon[29235]: Adjusting osd_memory_target on smithi123 to 4970M 2024-02-16T22:16:50.171 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:49 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:50.171 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:49 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:16:50.171 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:49 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:16:50.171 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:49 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:50.171 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:49 smithi114 ceph-mon[29235]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:16:50.172 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:49 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:16:50.172 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:49 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:16:50.172 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:49 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:16:50.172 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:49 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:50.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:49 smithi123 ceph-mon[37161]: osdmap e40: 6 total, 6 up, 6 in 2024-02-16T22:16:50.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:49 smithi123 ceph-mon[37161]: Detected new or changed devices on smithi123 2024-02-16T22:16:50.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:49 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:50.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:49 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:50.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:49 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:16:50.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:49 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:16:50.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:49 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:16:50.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:49 smithi123 ceph-mon[37161]: Adjusting osd_memory_target on smithi123 to 4970M 2024-02-16T22:16:50.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:49 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:50.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:49 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:16:50.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:49 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:16:50.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:49 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:50.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:49 smithi123 ceph-mon[37161]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:16:50.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:49 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:16:50.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:49 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:16:50.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:49 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:16:50.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:49 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:16:50.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.495+0000 7f30c24cb700 1 -- 172.21.15.114:0/1213907059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30bc100dc0 msgr2=0x7f30bc1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:16:50.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.495+0000 7f30c24cb700 1 --2- 172.21.15.114:0/1213907059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30bc100dc0 0x7f30bc1011a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f30a4009910 tx=0x7f30a4009c20 comp rx=0 tx=0).stop 2024-02-16T22:16:50.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.495+0000 7f30c24cb700 1 -- 172.21.15.114:0/1213907059 shutdown_connections 2024-02-16T22:16:50.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.495+0000 7f30c24cb700 1 --2- 172.21.15.114:0/1213907059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30bc101770 0x7f30bc1057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:50.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.495+0000 7f30c24cb700 1 --2- 172.21.15.114:0/1213907059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30bc100dc0 0x7f30bc1011a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:50.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.495+0000 7f30c24cb700 1 -- 172.21.15.114:0/1213907059 >> 172.21.15.114:0/1213907059 conn(0x7f30bc0fc650 msgr2=0x7f30bc0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:16:50.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.495+0000 7f30c24cb700 1 -- 172.21.15.114:0/1213907059 shutdown_connections 2024-02-16T22:16:50.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.495+0000 7f30c24cb700 1 -- 172.21.15.114:0/1213907059 wait complete. 2024-02-16T22:16:50.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.496+0000 7f30c24cb700 1 Processor -- start 2024-02-16T22:16:50.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.496+0000 7f30c24cb700 1 -- start start 2024-02-16T22:16:50.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.496+0000 7f30c24cb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30bc100dc0 0x7f30bc073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:16:50.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.496+0000 7f30c24cb700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30bc101770 0x7f30bc0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:16:50.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.496+0000 7f30c24cb700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc073b10 con 0x7f30bc101770 2024-02-16T22:16:50.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.496+0000 7f30c24cb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc073c50 con 0x7f30bc100dc0 2024-02-16T22:16:50.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.496+0000 7f30bb7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30bc101770 0x7f30bc0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:16:50.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.496+0000 7f30bb7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30bc101770 0x7f30bc0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48496/0 (socket says 172.21.15.114:48496) 2024-02-16T22:16:50.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.496+0000 7f30bb7fe700 1 -- 172.21.15.114:0/1877903558 learned_addr learned my addr 172.21.15.114:0/1877903558 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:16:50.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.496+0000 7f30bbfff700 1 --2- 172.21.15.114:0/1877903558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30bc100dc0 0x7f30bc073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:16:50.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.497+0000 7f30bb7fe700 1 -- 172.21.15.114:0/1877903558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30bc100dc0 msgr2=0x7f30bc073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:16:50.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.497+0000 7f30bb7fe700 1 --2- 172.21.15.114:0/1877903558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30bc100dc0 0x7f30bc073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:50.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.497+0000 7f30bb7fe700 1 -- 172.21.15.114:0/1877903558 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a40095f0 con 0x7f30bc101770 2024-02-16T22:16:50.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.497+0000 7f30bb7fe700 1 --2- 172.21.15.114:0/1877903558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30bc101770 0x7f30bc0735d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00b480 tx=0x7f30ac00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:16:50.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.497+0000 7f30b97fa700 1 -- 172.21.15.114:0/1877903558 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30ac011800 con 0x7f30bc101770 2024-02-16T22:16:50.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.497+0000 7f30b97fa700 1 -- 172.21.15.114:0/1877903558 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f30ac011e10 con 0x7f30bc101770 2024-02-16T22:16:50.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.497+0000 7f30c24cb700 1 -- 172.21.15.114:0/1877903558 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30bc1a2a60 con 0x7f30bc101770 2024-02-16T22:16:50.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.497+0000 7f30b97fa700 1 -- 172.21.15.114:0/1877903558 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30ac010ea0 con 0x7f30bc101770 2024-02-16T22:16:50.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.497+0000 7f30c24cb700 1 -- 172.21.15.114:0/1877903558 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30bc1a2e10 con 0x7f30bc101770 2024-02-16T22:16:50.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.498+0000 7f30c24cb700 1 -- 172.21.15.114:0/1877903558 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30bc109160 con 0x7f30bc101770 2024-02-16T22:16:50.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.498+0000 7f30b97fa700 1 -- 172.21.15.114:0/1877903558 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f30ac010380 con 0x7f30bc101770 2024-02-16T22:16:50.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.499+0000 7f30b97fa700 1 --2- 172.21.15.114:0/1877903558 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f30a806c4e0 0x7f30a806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:16:50.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.499+0000 7f30b97fa700 1 -- 172.21.15.114:0/1877903558 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f30ac08aff0 con 0x7f30bc101770 2024-02-16T22:16:50.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.499+0000 7f30bbfff700 1 --2- 172.21.15.114:0/1877903558 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f30a806c4e0 0x7f30a806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:16:50.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.499+0000 7f30bbfff700 1 --2- 172.21.15.114:0/1877903558 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f30a806c4e0 0x7f30a806e9a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f30a4006010 tx=0x7f30a401a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:16:50.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.503+0000 7f30b97fa700 1 -- 172.21.15.114:0/1877903558 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f30ac055b10 con 0x7f30bc101770 2024-02-16T22:16:50.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.694+0000 7f30c24cb700 1 -- 172.21.15.114:0/1877903558 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30bc04ea90 con 0x7f30bc101770 2024-02-16T22:16:50.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.695+0000 7f30b97fa700 1 -- 172.21.15.114:0/1877903558 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11398 (secure 0 0 0) 0x7f30ac059130 con 0x7f30bc101770 2024-02-16T22:16:50.697 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:16:50.698 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":40,"fsid":"1dcbaabe-cd18-11ee-95ba-87774f69a715","created":"2024-02-16T22:11:02.186164+0000","modified":"2024-02-16T22:16:48.855087+0000","last_up_change":"2024-02-16T22:16:47.516304+0000","last_in_change":"2024-02-16T22:16:28.224489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-16T22:15:31.749346+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3a72ccf1-3f93-4c06-99c3-092a7f9a732d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1379159586},{"type":"v1","addr":"172.21.15.114:6803","nonce":1379159586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1379159586},{"type":"v1","addr":"172.21.15.114:6805","nonce":1379159586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1379159586},{"type":"v1","addr":"172.21.15.114:6809","nonce":1379159586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1379159586},{"type":"v1","addr":"172.21.15.114:6807","nonce":1379159586}]},"public_addr":"172.21.15.114:6803/1379159586","cluster_addr":"172.21.15.114:6805/1379159586","heartbeat_back_addr":"172.21.15.114:6809/1379159586","heartbeat_front_addr":"172.21.15.114:6807/1379159586","state":["exists","up"]},{"osd":1,"uuid":"1e8829dd-cac2-462f-aca1-86bcfb475a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1548199763},{"type":"v1","addr":"172.21.15.114:6811","nonce":1548199763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1548199763},{"type":"v1","addr":"172.21.15.114:6813","nonce":1548199763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1548199763},{"type":"v1","addr":"172.21.15.114:6817","nonce":1548199763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1548199763},{"type":"v1","addr":"172.21.15.114:6815","nonce":1548199763}]},"public_addr":"172.21.15.114:6811/1548199763","cluster_addr":"172.21.15.114:6813/1548199763","heartbeat_back_addr":"172.21.15.114:6817/1548199763","heartbeat_front_addr":"172.21.15.114:6815/1548199763","state":["exists","up"]},{"osd":2,"uuid":"44f9fce8-5ea8-4741-9936-6720df91b628","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1293302101},{"type":"v1","addr":"172.21.15.114:6819","nonce":1293302101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1293302101},{"type":"v1","addr":"172.21.15.114:6821","nonce":1293302101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1293302101},{"type":"v1","addr":"172.21.15.114:6825","nonce":1293302101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1293302101},{"type":"v1","addr":"172.21.15.114:6823","nonce":1293302101}]},"public_addr":"172.21.15.114:6819/1293302101","cluster_addr":"172.21.15.114:6821/1293302101","heartbeat_back_addr":"172.21.15.114:6825/1293302101","heartbeat_front_addr":"172.21.15.114:6823/1293302101","state":["exists","up"]},{"osd":3,"uuid":"b6eb4941-bef4-4722-9050-cd6c041f9fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3211307893},{"type":"v1","addr":"172.21.15.123:6801","nonce":3211307893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3211307893},{"type":"v1","addr":"172.21.15.123:6803","nonce":3211307893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3211307893},{"type":"v1","addr":"172.21.15.123:6807","nonce":3211307893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3211307893},{"type":"v1","addr":"172.21.15.123:6805","nonce":3211307893}]},"public_addr":"172.21.15.123:6801/3211307893","cluster_addr":"172.21.15.123:6803/3211307893","heartbeat_back_addr":"172.21.15.123:6807/3211307893","heartbeat_front_addr":"172.21.15.123:6805/3211307893","state":["exists","up"]},{"osd":4,"uuid":"e9caa348-26f4-4ed5-a64e-59231fc48546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3994684204},{"type":"v1","addr":"172.21.15.123:6809","nonce":3994684204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3994684204},{"type":"v1","addr":"172.21.15.123:6811","nonce":3994684204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3994684204},{"type":"v1","addr":"172.21.15.123:6815","nonce":3994684204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3994684204},{"type":"v1","addr":"172.21.15.123:6813","nonce":3994684204}]},"public_addr":"172.21.15.123:6809/3994684204","cluster_addr":"172.21.15.123:6811/3994684204","heartbeat_back_addr":"172.21.15.123:6815/3994684204","heartbeat_front_addr":"172.21.15.123:6813/3994684204","state":["exists","up"]},{"osd":5,"uuid":"79d81af1-2361-430d-bc76-35ef74aa6bad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2122343545},{"type":"v1","addr":"172.21.15.123:6817","nonce":2122343545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":2122343545},{"type":"v1","addr":"172.21.15.123:6819","nonce":2122343545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":2122343545},{"type":"v1","addr":"172.21.15.123:6823","nonce":2122343545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2122343545},{"type":"v1","addr":"172.21.15.123:6821","nonce":2122343545}]},"public_addr":"172.21.15.123:6817/2122343545","cluster_addr":"172.21.15.123:6819/2122343545","heartbeat_back_addr":"172.21.15.123:6823/2122343545","heartbeat_front_addr":"172.21.15.123:6821/2122343545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:14:32.818023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:15:00.597924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:15:29.574022+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:15:53.159458+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:16:20.058254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:16:45.789725+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:0/2762476246":"2024-02-17T22:13:11.405214+0000","172.21.15.114:6801/2648536185":"2024-02-17T22:13:11.405214+0000","172.21.15.114:0/1034649687":"2024-02-17T22:13:11.405214+0000","172.21.15.114:0/1349416361":"2024-02-17T22:12:00.492971+0000","172.21.15.114:0/499892900":"2024-02-17T22:12:00.492971+0000","172.21.15.114:0/744140874":"2024-02-17T22:12:00.492971+0000","172.21.15.114:6800/2648536185":"2024-02-17T22:13:11.405214+0000","172.21.15.114:6801/625910309":"2024-02-17T22:12:00.492971+0000","172.21.15.114:6800/2055574123":"2024-02-17T22:11:27.861007+0000","172.21.15.114:0/3971798486":"2024-02-17T22:11:27.861007+0000","172.21.15.114:0/4029631970":"2024-02-17T22:11:27.861007+0000","172.21.15.114:0/3211326148":"2024-02-17T22:13:11.405214+0000","172.21.15.114:6800/625910309":"2024-02-17T22:12:00.492971+0000","172.21.15.114:6801/2055574123":"2024-02-17T22:11:27.861007+0000","172.21.15.114:0/2825621999":"2024-02-17T22:11:27.861007+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-16T22:16:50.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.697+0000 7f30c24cb700 1 -- 172.21.15.114:0/1877903558 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f30a806c4e0 msgr2=0x7f30a806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:16:50.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.697+0000 7f30c24cb700 1 --2- 172.21.15.114:0/1877903558 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f30a806c4e0 0x7f30a806e9a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f30a4006010 tx=0x7f30a401a040 comp rx=0 tx=0).stop 2024-02-16T22:16:50.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.697+0000 7f30c24cb700 1 -- 172.21.15.114:0/1877903558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30bc101770 msgr2=0x7f30bc0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:16:50.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.697+0000 7f30c24cb700 1 --2- 172.21.15.114:0/1877903558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30bc101770 0x7f30bc0735d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00b480 tx=0x7f30ac00b790 comp rx=0 tx=0).stop 2024-02-16T22:16:50.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.697+0000 7f30c24cb700 1 -- 172.21.15.114:0/1877903558 shutdown_connections 2024-02-16T22:16:50.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.697+0000 7f30c24cb700 1 --2- 172.21.15.114:0/1877903558 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f30a806c4e0 0x7f30a806e9a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:50.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.697+0000 7f30c24cb700 1 --2- 172.21.15.114:0/1877903558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30bc100dc0 0x7f30bc073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:50.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.697+0000 7f30c24cb700 1 --2- 172.21.15.114:0/1877903558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30bc101770 0x7f30bc0735d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:50.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.697+0000 7f30c24cb700 1 -- 172.21.15.114:0/1877903558 >> 172.21.15.114:0/1877903558 conn(0x7f30bc0fc650 msgr2=0x7f30bc0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:16:50.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.697+0000 7f30c24cb700 1 -- 172.21.15.114:0/1877903558 shutdown_connections 2024-02-16T22:16:50.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:50.697+0000 7f30c24cb700 1 -- 172.21.15.114:0/1877903558 wait complete. 2024-02-16T22:16:51.374 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-16T22:15:31.749346+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-16T22:16:51.374 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-16T22:16:51.583 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:16:51.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:51 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/1877903558' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T22:16:51.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:51 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/1877903558' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T22:16:52.575 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:52 smithi114 ceph-mon[29235]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:16:52.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:52 smithi123 ceph-mon[37161]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:16:53.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.087+0000 7fd65118d700 1 -- 172.21.15.114:0/1886458437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd64c105a40 msgr2=0x7fd64c105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:16:53.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.087+0000 7fd65118d700 1 --2- 172.21.15.114:0/1886458437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd64c105a40 0x7fd64c105e20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd63c009910 tx=0x7fd63c009c20 comp rx=0 tx=0).stop 2024-02-16T22:16:53.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.087+0000 7fd65118d700 1 -- 172.21.15.114:0/1886458437 shutdown_connections 2024-02-16T22:16:53.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.087+0000 7fd65118d700 1 --2- 172.21.15.114:0/1886458437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd64c0ffbd0 0x7fd64c100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:53.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.087+0000 7fd65118d700 1 --2- 172.21.15.114:0/1886458437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd64c105a40 0x7fd64c105e20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:53.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.087+0000 7fd65118d700 1 -- 172.21.15.114:0/1886458437 >> 172.21.15.114:0/1886458437 conn(0x7fd64c0755a0 msgr2=0x7fd64c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:16:53.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.087+0000 7fd65118d700 1 -- 172.21.15.114:0/1886458437 shutdown_connections 2024-02-16T22:16:53.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.087+0000 7fd65118d700 1 -- 172.21.15.114:0/1886458437 wait complete. 2024-02-16T22:16:53.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.087+0000 7fd65118d700 1 Processor -- start 2024-02-16T22:16:53.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.088+0000 7fd65118d700 1 -- start start 2024-02-16T22:16:53.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.088+0000 7fd65118d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd64c0ffbd0 0x7fd64c101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:16:53.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.088+0000 7fd65118d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd64c105a40 0x7fd64c101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:16:53.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.088+0000 7fd65118d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c103330 con 0x7fd64c0ffbd0 2024-02-16T22:16:53.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.088+0000 7fd65118d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c1034a0 con 0x7fd64c105a40 2024-02-16T22:16:53.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.088+0000 7fd64ad9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd64c0ffbd0 0x7fd64c101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:16:53.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.088+0000 7fd64ad9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd64c0ffbd0 0x7fd64c101610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38298/0 (socket says 172.21.15.114:38298) 2024-02-16T22:16:53.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.088+0000 7fd64ad9d700 1 -- 172.21.15.114:0/3260845001 learned_addr learned my addr 172.21.15.114:0/3260845001 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:16:53.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.088+0000 7fd64ad9d700 1 -- 172.21.15.114:0/3260845001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd64c105a40 msgr2=0x7fd64c101b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:16:53.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.088+0000 7fd64ad9d700 1 --2- 172.21.15.114:0/3260845001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd64c105a40 0x7fd64c101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:53.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.088+0000 7fd64ad9d700 1 -- 172.21.15.114:0/3260845001 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c0095f0 con 0x7fd64c0ffbd0 2024-02-16T22:16:53.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.088+0000 7fd64ad9d700 1 --2- 172.21.15.114:0/3260845001 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd64c0ffbd0 0x7fd64c101610 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00bda0 tx=0x7fd63c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:16:53.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.088+0000 7fd648d99700 1 -- 172.21.15.114:0/3260845001 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd63c01d070 con 0x7fd64c0ffbd0 2024-02-16T22:16:53.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.088+0000 7fd65118d700 1 -- 172.21.15.114:0/3260845001 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd64c102090 con 0x7fd64c0ffbd0 2024-02-16T22:16:53.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.088+0000 7fd648d99700 1 -- 172.21.15.114:0/3260845001 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd63c022470 con 0x7fd64c0ffbd0 2024-02-16T22:16:53.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.088+0000 7fd648d99700 1 -- 172.21.15.114:0/3260845001 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd63c00f650 con 0x7fd64c0ffbd0 2024-02-16T22:16:53.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.088+0000 7fd65118d700 1 -- 172.21.15.114:0/3260845001 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd64c1a7090 con 0x7fd64c0ffbd0 2024-02-16T22:16:53.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.089+0000 7fd65118d700 1 -- 172.21.15.114:0/3260845001 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd64c109050 con 0x7fd64c0ffbd0 2024-02-16T22:16:53.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.090+0000 7fd648d99700 1 -- 172.21.15.114:0/3260845001 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fd63c0225e0 con 0x7fd64c0ffbd0 2024-02-16T22:16:53.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.090+0000 7fd648d99700 1 --2- 172.21.15.114:0/3260845001 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd63806c4e0 0x7fd63806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:16:53.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.090+0000 7fd648d99700 1 -- 172.21.15.114:0/3260845001 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fd63c08ca60 con 0x7fd64c0ffbd0 2024-02-16T22:16:53.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.090+0000 7fd64259c700 1 --2- 172.21.15.114:0/3260845001 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd63806c4e0 0x7fd63806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:16:53.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.091+0000 7fd64259c700 1 --2- 172.21.15.114:0/3260845001 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd63806c4e0 0x7fd63806e9a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd634005fd0 tx=0x7fd634005d00 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:16:53.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.095+0000 7fd648d99700 1 -- 172.21.15.114:0/3260845001 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd63c057630 con 0x7fd64c0ffbd0 2024-02-16T22:16:53.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.283+0000 7fd65118d700 1 -- 172.21.15.114:0/3260845001 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd64c102b40 con 0x7fd64c0ffbd0 2024-02-16T22:16:53.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.284+0000 7fd648d99700 1 -- 172.21.15.114:0/3260845001 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd63c05ac50 con 0x7fd64c0ffbd0 2024-02-16T22:16:53.286 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2024-02-16T22:16:53.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.287+0000 7fd65118d700 1 -- 172.21.15.114:0/3260845001 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd63806c4e0 msgr2=0x7fd63806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:16:53.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.287+0000 7fd65118d700 1 --2- 172.21.15.114:0/3260845001 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd63806c4e0 0x7fd63806e9a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd634005fd0 tx=0x7fd634005d00 comp rx=0 tx=0).stop 2024-02-16T22:16:53.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.287+0000 7fd65118d700 1 -- 172.21.15.114:0/3260845001 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd64c0ffbd0 msgr2=0x7fd64c101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:16:53.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.287+0000 7fd65118d700 1 --2- 172.21.15.114:0/3260845001 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd64c0ffbd0 0x7fd64c101610 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00bda0 tx=0x7fd63c00be80 comp rx=0 tx=0).stop 2024-02-16T22:16:53.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.287+0000 7fd65118d700 1 -- 172.21.15.114:0/3260845001 shutdown_connections 2024-02-16T22:16:53.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.287+0000 7fd65118d700 1 --2- 172.21.15.114:0/3260845001 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd64c0ffbd0 0x7fd64c101610 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:53.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.287+0000 7fd65118d700 1 --2- 172.21.15.114:0/3260845001 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd63806c4e0 0x7fd63806e9a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:53.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.287+0000 7fd65118d700 1 --2- 172.21.15.114:0/3260845001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd64c105a40 0x7fd64c101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:53.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.287+0000 7fd65118d700 1 -- 172.21.15.114:0/3260845001 >> 172.21.15.114:0/3260845001 conn(0x7fd64c0755a0 msgr2=0x7fd64c0685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:16:53.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.287+0000 7fd65118d700 1 -- 172.21.15.114:0/3260845001 shutdown_connections 2024-02-16T22:16:53.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:53.287+0000 7fd65118d700 1 -- 172.21.15.114:0/3260845001 wait complete. 2024-02-16T22:16:53.582 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:53 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/3260845001' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T22:16:53.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:53 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/3260845001' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T22:16:53.953 INFO:tasks.cephadm:Setting up client nodes... 2024-02-16T22:16:53.954 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-16T22:16:54.158 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:16:54.618 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:54 smithi114 ceph-mon[29235]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:16:54.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:54 smithi123 ceph-mon[37161]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:16:55.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.824+0000 7f514d927700 1 -- 172.21.15.114:0/1590711260 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51480ff290 msgr2=0x7f51480ff6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:16:55.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.824+0000 7f514d927700 1 --2- 172.21.15.114:0/1590711260 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51480ff290 0x7f51480ff6b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f513800aa20 tx=0x7f513800ad30 comp rx=0 tx=0).stop 2024-02-16T22:16:55.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.824+0000 7f514d927700 1 -- 172.21.15.114:0/1590711260 shutdown_connections 2024-02-16T22:16:55.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.824+0000 7f514d927700 1 --2- 172.21.15.114:0/1590711260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5148100490 0x7f51481008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:55.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.825+0000 7f514d927700 1 --2- 172.21.15.114:0/1590711260 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51480ff290 0x7f51480ff6b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:55.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.825+0000 7f514d927700 1 -- 172.21.15.114:0/1590711260 >> 172.21.15.114:0/1590711260 conn(0x7f51480fa810 msgr2=0x7f51480fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:16:55.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.825+0000 7f514d927700 1 -- 172.21.15.114:0/1590711260 shutdown_connections 2024-02-16T22:16:55.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.825+0000 7f514d927700 1 -- 172.21.15.114:0/1590711260 wait complete. 2024-02-16T22:16:55.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.825+0000 7f514d927700 1 Processor -- start 2024-02-16T22:16:55.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.825+0000 7f514d927700 1 -- start start 2024-02-16T22:16:55.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.825+0000 7f514d927700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51480ff290 0x7f5148196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:16:55.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.825+0000 7f514d927700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5148100490 0x7f51481969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:16:55.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.825+0000 7f514d927700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5148196fe0 con 0x7f5148100490 2024-02-16T22:16:55.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.825+0000 7f514d927700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5148197120 con 0x7f51480ff290 2024-02-16T22:16:55.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.826+0000 7f51467fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5148100490 0x7f51481969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:16:55.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.826+0000 7f51467fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5148100490 0x7f51481969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38322/0 (socket says 172.21.15.114:38322) 2024-02-16T22:16:55.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.826+0000 7f51467fc700 1 -- 172.21.15.114:0/4004522947 learned_addr learned my addr 172.21.15.114:0/4004522947 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:16:55.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.826+0000 7f5146ffd700 1 --2- 172.21.15.114:0/4004522947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51480ff290 0x7f5148196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:16:55.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.826+0000 7f51467fc700 1 -- 172.21.15.114:0/4004522947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51480ff290 msgr2=0x7f5148196480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:16:55.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.826+0000 7f51467fc700 1 --2- 172.21.15.114:0/4004522947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51480ff290 0x7f5148196480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:55.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.826+0000 7f51467fc700 1 -- 172.21.15.114:0/4004522947 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513800a700 con 0x7f5148100490 2024-02-16T22:16:55.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.826+0000 7f51467fc700 1 --2- 172.21.15.114:0/4004522947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5148100490 0x7f51481969c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f513c00c8a0 tx=0x7f513c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:16:55.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.826+0000 7f514c925700 1 -- 172.21.15.114:0/4004522947 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f513c0078e0 con 0x7f5148100490 2024-02-16T22:16:55.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.826+0000 7f514c925700 1 -- 172.21.15.114:0/4004522947 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f513c00f450 con 0x7f5148100490 2024-02-16T22:16:55.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.826+0000 7f514c925700 1 -- 172.21.15.114:0/4004522947 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f513c00e5e0 con 0x7f5148100490 2024-02-16T22:16:55.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.826+0000 7f514d927700 1 -- 172.21.15.114:0/4004522947 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f514819bbd0 con 0x7f5148100490 2024-02-16T22:16:55.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.826+0000 7f514d927700 1 -- 172.21.15.114:0/4004522947 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f514819c120 con 0x7f5148100490 2024-02-16T22:16:55.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.827+0000 7f514d927700 1 -- 172.21.15.114:0/4004522947 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5148107560 con 0x7f5148100490 2024-02-16T22:16:55.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.828+0000 7f514c925700 1 -- 172.21.15.114:0/4004522947 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f513c00f5c0 con 0x7f5148100490 2024-02-16T22:16:55.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.828+0000 7f514c925700 1 --2- 172.21.15.114:0/4004522947 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f513006c4e0 0x7f513006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:16:55.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.828+0000 7f514c925700 1 -- 172.21.15.114:0/4004522947 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f513c08b380 con 0x7f5148100490 2024-02-16T22:16:55.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.828+0000 7f5146ffd700 1 --2- 172.21.15.114:0/4004522947 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f513006c4e0 0x7f513006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:16:55.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.829+0000 7f5146ffd700 1 --2- 172.21.15.114:0/4004522947 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f513006c4e0 0x7f513006e9a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f513800d010 tx=0x7f5138005de0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:16:55.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:55.832+0000 7f514c925700 1 -- 172.21.15.114:0/4004522947 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f513c0599b0 con 0x7f5148100490 2024-02-16T22:16:56.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:56.101+0000 7f514d927700 1 -- 172.21.15.114:0/4004522947 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f514819c880 con 0x7f5148100490 2024-02-16T22:16:56.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:56.105+0000 7f514c925700 1 -- 172.21.15.114:0/4004522947 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7f513c059540 con 0x7f5148100490 2024-02-16T22:16:56.107 INFO:teuthology.orchestra.run.smithi114.stdout:[client.0] 2024-02-16T22:16:56.107 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQDY3s9lbRwUBhAApe2PrtBPQToFixgr+5lQ7g== 2024-02-16T22:16:56.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:56.107+0000 7f514d927700 1 -- 172.21.15.114:0/4004522947 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f513006c4e0 msgr2=0x7f513006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:16:56.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:56.107+0000 7f514d927700 1 --2- 172.21.15.114:0/4004522947 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f513006c4e0 0x7f513006e9a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f513800d010 tx=0x7f5138005de0 comp rx=0 tx=0).stop 2024-02-16T22:16:56.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:56.107+0000 7f514d927700 1 -- 172.21.15.114:0/4004522947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5148100490 msgr2=0x7f51481969c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:16:56.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:56.107+0000 7f514d927700 1 --2- 172.21.15.114:0/4004522947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5148100490 0x7f51481969c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f513c00c8a0 tx=0x7f513c00cbb0 comp rx=0 tx=0).stop 2024-02-16T22:16:56.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:56.107+0000 7f514d927700 1 -- 172.21.15.114:0/4004522947 shutdown_connections 2024-02-16T22:16:56.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:56.107+0000 7f514d927700 1 --2- 172.21.15.114:0/4004522947 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f513006c4e0 0x7f513006e9a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:56.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:56.107+0000 7f514d927700 1 --2- 172.21.15.114:0/4004522947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51480ff290 0x7f5148196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:56.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:56.107+0000 7f514d927700 1 --2- 172.21.15.114:0/4004522947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5148100490 0x7f51481969c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:56.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:56.107+0000 7f514d927700 1 -- 172.21.15.114:0/4004522947 >> 172.21.15.114:0/4004522947 conn(0x7f51480fa810 msgr2=0x7f51480fc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:16:56.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:56.107+0000 7f514d927700 1 -- 172.21.15.114:0/4004522947 shutdown_connections 2024-02-16T22:16:56.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:16:56.107+0000 7f514d927700 1 -- 172.21.15.114:0/4004522947 wait complete. 2024-02-16T22:16:56.788 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:56 smithi114 ceph-mon[29235]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:16:56.788 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:56 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/4004522947' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-16T22:16:56.788 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:56 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/4004522947' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-16T22:16:56.788 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:56 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:16:56.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:56 smithi123 ceph-mon[37161]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:16:56.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:56 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/4004522947' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-16T22:16:56.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:56 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/4004522947' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-16T22:16:56.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:56 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:16:56.809 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-02-16T22:16:56.809 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-16T22:16:56.809 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-16T22:16:56.851 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-16T22:16:57.066 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi123/config 2024-02-16T22:16:58.181 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:57 smithi123 ceph-mon[37161]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:16:58.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:57 smithi114 ceph-mon[29235]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:16:58.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.626+0000 7ff964139700 1 -- 172.21.15.123:0/377906352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff95c101eb0 msgr2=0x7ff95c102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:16:58.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.626+0000 7ff964139700 1 --2- 172.21.15.123:0/377906352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff95c101eb0 0x7ff95c102310 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff94c009910 tx=0x7ff94c009c20 comp rx=0 tx=0).stop 2024-02-16T22:16:58.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.626+0000 7ff964139700 1 -- 172.21.15.123:0/377906352 shutdown_connections 2024-02-16T22:16:58.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.626+0000 7ff964139700 1 --2- 172.21.15.123:0/377906352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff95c101eb0 0x7ff95c102310 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:58.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.626+0000 7ff964139700 1 --2- 172.21.15.123:0/377906352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff95c0feed0 0x7ff95c0ff2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:58.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.626+0000 7ff964139700 1 -- 172.21.15.123:0/377906352 >> 172.21.15.123:0/377906352 conn(0x7ff95c0fa810 msgr2=0x7ff95c0fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:16:58.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.626+0000 7ff964139700 1 -- 172.21.15.123:0/377906352 shutdown_connections 2024-02-16T22:16:58.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.626+0000 7ff964139700 1 -- 172.21.15.123:0/377906352 wait complete. 2024-02-16T22:16:58.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.626+0000 7ff964139700 1 Processor -- start 2024-02-16T22:16:58.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.626+0000 7ff964139700 1 -- start start 2024-02-16T22:16:58.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.627+0000 7ff964139700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff95c0feed0 0x7ff95c196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:16:58.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.627+0000 7ff964139700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff95c101eb0 0x7ff95c1969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:16:58.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.627+0000 7ff964139700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c196ff0 con 0x7ff95c101eb0 2024-02-16T22:16:58.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.627+0000 7ff964139700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c197130 con 0x7ff95c0feed0 2024-02-16T22:16:58.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.627+0000 7ff961ed5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff95c0feed0 0x7ff95c196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:16:58.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.627+0000 7ff961ed5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff95c0feed0 0x7ff95c196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39222/0 (socket says 172.21.15.123:39222) 2024-02-16T22:16:58.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.627+0000 7ff961ed5700 1 -- 172.21.15.123:0/2757444443 learned_addr learned my addr 172.21.15.123:0/2757444443 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-02-16T22:16:58.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.627+0000 7ff961ed5700 1 -- 172.21.15.123:0/2757444443 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff95c101eb0 msgr2=0x7ff95c1969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:16:58.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.627+0000 7ff961ed5700 1 --2- 172.21.15.123:0/2757444443 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff95c101eb0 0x7ff95c1969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:58.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.627+0000 7ff961ed5700 1 -- 172.21.15.123:0/2757444443 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94c0095f0 con 0x7ff95c0feed0 2024-02-16T22:16:58.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.627+0000 7ff961ed5700 1 --2- 172.21.15.123:0/2757444443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff95c0feed0 0x7ff95c196490 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff95800b480 tx=0x7ff95800b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:16:58.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.627+0000 7ff952ffd700 1 -- 172.21.15.123:0/2757444443 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9580118c0 con 0x7ff95c0feed0 2024-02-16T22:16:58.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.627+0000 7ff952ffd700 1 -- 172.21.15.123:0/2757444443 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff958010380 con 0x7ff95c0feed0 2024-02-16T22:16:58.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.627+0000 7ff952ffd700 1 -- 172.21.15.123:0/2757444443 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff95800f4f0 con 0x7ff95c0feed0 2024-02-16T22:16:58.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.627+0000 7ff964139700 1 -- 172.21.15.123:0/2757444443 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff95c19bbe0 con 0x7ff95c0feed0 2024-02-16T22:16:58.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.627+0000 7ff964139700 1 -- 172.21.15.123:0/2757444443 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff95c19c1b0 con 0x7ff95c0feed0 2024-02-16T22:16:58.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.628+0000 7ff964139700 1 -- 172.21.15.123:0/2757444443 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff95c109810 con 0x7ff95c0feed0 2024-02-16T22:16:58.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.629+0000 7ff952ffd700 1 -- 172.21.15.123:0/2757444443 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7ff9580104f0 con 0x7ff95c0feed0 2024-02-16T22:16:58.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.629+0000 7ff952ffd700 1 --2- 172.21.15.123:0/2757444443 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff94806c4e0 0x7ff94806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:16:58.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.629+0000 7ff952ffd700 1 -- 172.21.15.123:0/2757444443 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff95808b090 con 0x7ff95c0feed0 2024-02-16T22:16:58.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.630+0000 7ff9616d4700 1 --2- 172.21.15.123:0/2757444443 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff94806c4e0 0x7ff94806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:16:58.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.630+0000 7ff9616d4700 1 --2- 172.21.15.123:0/2757444443 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff94806c4e0 0x7ff94806e9a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff94c00b5c0 tx=0x7ff94c005960 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:16:58.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.634+0000 7ff952ffd700 1 -- 172.21.15.123:0/2757444443 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff9580596c0 con 0x7ff95c0feed0 2024-02-16T22:16:58.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.894+0000 7ff964139700 1 -- 172.21.15.123:0/2757444443 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7ff95c19c850 con 0x7ff95c0feed0 2024-02-16T22:16:58.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.900+0000 7ff952ffd700 1 -- 172.21.15.123:0/2757444443 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7ff958059250 con 0x7ff95c0feed0 2024-02-16T22:16:58.901 INFO:teuthology.orchestra.run.smithi123.stdout:[client.1] 2024-02-16T22:16:58.901 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDa3s9lmStsNRAAdkeHsSRN0V8Tr6JP1q8yQw== 2024-02-16T22:16:58.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.902+0000 7ff964139700 1 -- 172.21.15.123:0/2757444443 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff94806c4e0 msgr2=0x7ff94806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:16:58.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.902+0000 7ff964139700 1 --2- 172.21.15.123:0/2757444443 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff94806c4e0 0x7ff94806e9a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff94c00b5c0 tx=0x7ff94c005960 comp rx=0 tx=0).stop 2024-02-16T22:16:58.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.902+0000 7ff964139700 1 -- 172.21.15.123:0/2757444443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff95c0feed0 msgr2=0x7ff95c196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:16:58.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.902+0000 7ff964139700 1 --2- 172.21.15.123:0/2757444443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff95c0feed0 0x7ff95c196490 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff95800b480 tx=0x7ff95800b840 comp rx=0 tx=0).stop 2024-02-16T22:16:58.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.902+0000 7ff964139700 1 -- 172.21.15.123:0/2757444443 shutdown_connections 2024-02-16T22:16:58.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.902+0000 7ff964139700 1 --2- 172.21.15.123:0/2757444443 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff94806c4e0 0x7ff94806e9a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:58.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.902+0000 7ff964139700 1 --2- 172.21.15.123:0/2757444443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff95c0feed0 0x7ff95c196490 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:58.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.902+0000 7ff964139700 1 --2- 172.21.15.123:0/2757444443 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff95c101eb0 0x7ff95c1969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:16:58.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.902+0000 7ff964139700 1 -- 172.21.15.123:0/2757444443 >> 172.21.15.123:0/2757444443 conn(0x7ff95c0fa810 msgr2=0x7ff95c1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:16:58.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.902+0000 7ff964139700 1 -- 172.21.15.123:0/2757444443 shutdown_connections 2024-02-16T22:16:58.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:16:58.902+0000 7ff964139700 1 -- 172.21.15.123:0/2757444443 wait complete. 2024-02-16T22:16:59.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:58 smithi114 ceph-mon[29235]: from='client.? 172.21.15.123:0/2757444443' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-16T22:16:59.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:58 smithi114 ceph-mon[29235]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-16T22:16:59.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:58 smithi114 ceph-mon[29235]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-16T22:16:59.256 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:58 smithi123 ceph-mon[37161]: from='client.? 172.21.15.123:0/2757444443' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-16T22:16:59.256 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:58 smithi123 ceph-mon[37161]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-16T22:16:59.256 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:58 smithi123 ceph-mon[37161]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-16T22:16:59.578 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-02-16T22:16:59.578 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-02-16T22:16:59.578 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-16T22:16:59.620 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-16T22:16:59.620 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-16T22:16:59.620 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph mgr dump --format=json 2024-02-16T22:16:59.835 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:17:00.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:16:59 smithi114 ceph-mon[29235]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:00.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:16:59 smithi123 ceph-mon[37161]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:01.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.387+0000 7f1a84cf3700 1 -- 172.21.15.114:0/3899183416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a801026c0 msgr2=0x7f1a8010abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:01.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.387+0000 7f1a84cf3700 1 --2- 172.21.15.114:0/3899183416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a801026c0 0x7f1a8010abb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1a74009910 tx=0x7f1a74009c20 comp rx=0 tx=0).stop 2024-02-16T22:17:01.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.387+0000 7f1a84cf3700 1 -- 172.21.15.114:0/3899183416 shutdown_connections 2024-02-16T22:17:01.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.387+0000 7f1a84cf3700 1 --2- 172.21.15.114:0/3899183416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a801026c0 0x7f1a8010abb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:01.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.387+0000 7f1a84cf3700 1 --2- 172.21.15.114:0/3899183416 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a80101da0 0x7f1a80102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:01.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.387+0000 7f1a84cf3700 1 -- 172.21.15.114:0/3899183416 >> 172.21.15.114:0/3899183416 conn(0x7f1a800fb550 msgr2=0x7f1a800fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:01.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.387+0000 7f1a84cf3700 1 -- 172.21.15.114:0/3899183416 shutdown_connections 2024-02-16T22:17:01.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.387+0000 7f1a84cf3700 1 -- 172.21.15.114:0/3899183416 wait complete. 2024-02-16T22:17:01.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.388+0000 7f1a84cf3700 1 Processor -- start 2024-02-16T22:17:01.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.388+0000 7f1a84cf3700 1 -- start start 2024-02-16T22:17:01.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.388+0000 7f1a84cf3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a80101da0 0x7f1a8019c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:01.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.388+0000 7f1a84cf3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a801026c0 0x7f1a8019ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:01.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.388+0000 7f1a84cf3700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8019d070 con 0x7f1a801026c0 2024-02-16T22:17:01.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.388+0000 7f1a84cf3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a801965b0 con 0x7f1a80101da0 2024-02-16T22:17:01.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.388+0000 7f1a7dd9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a801026c0 0x7f1a8019ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:01.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.388+0000 7f1a7dd9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a801026c0 0x7f1a8019ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38344/0 (socket says 172.21.15.114:38344) 2024-02-16T22:17:01.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.388+0000 7f1a7dd9b700 1 -- 172.21.15.114:0/2096576087 learned_addr learned my addr 172.21.15.114:0/2096576087 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:17:01.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.388+0000 7f1a7e59c700 1 --2- 172.21.15.114:0/2096576087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a80101da0 0x7f1a8019c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:01.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.388+0000 7f1a7dd9b700 1 -- 172.21.15.114:0/2096576087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a80101da0 msgr2=0x7f1a8019c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:01.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.388+0000 7f1a7dd9b700 1 --2- 172.21.15.114:0/2096576087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a80101da0 0x7f1a8019c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:01.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.388+0000 7f1a7dd9b700 1 -- 172.21.15.114:0/2096576087 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a740095f0 con 0x7f1a801026c0 2024-02-16T22:17:01.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.389+0000 7f1a7dd9b700 1 --2- 172.21.15.114:0/2096576087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a801026c0 0x7f1a8019ca70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1a740092e0 tx=0x7f1a7400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:01.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.389+0000 7f1a6f7fe700 1 -- 172.21.15.114:0/2096576087 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a7401d070 con 0x7f1a801026c0 2024-02-16T22:17:01.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.389+0000 7f1a6f7fe700 1 -- 172.21.15.114:0/2096576087 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1a74004fa0 con 0x7f1a801026c0 2024-02-16T22:17:01.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.389+0000 7f1a84cf3700 1 -- 172.21.15.114:0/2096576087 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a80196830 con 0x7f1a801026c0 2024-02-16T22:17:01.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.389+0000 7f1a6f7fe700 1 -- 172.21.15.114:0/2096576087 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a7400f690 con 0x7f1a801026c0 2024-02-16T22:17:01.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.389+0000 7f1a84cf3700 1 -- 172.21.15.114:0/2096576087 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a80196d20 con 0x7f1a801026c0 2024-02-16T22:17:01.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.390+0000 7f1a84cf3700 1 -- 172.21.15.114:0/2096576087 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a801082b0 con 0x7f1a801026c0 2024-02-16T22:17:01.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.390+0000 7f1a6f7fe700 1 -- 172.21.15.114:0/2096576087 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f1a74022470 con 0x7f1a801026c0 2024-02-16T22:17:01.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.391+0000 7f1a6f7fe700 1 --2- 172.21.15.114:0/2096576087 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f1a6806c4e0 0x7f1a6806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:01.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.391+0000 7f1a6f7fe700 1 -- 172.21.15.114:0/2096576087 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f1a7408cd80 con 0x7f1a801026c0 2024-02-16T22:17:01.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.391+0000 7f1a7e59c700 1 --2- 172.21.15.114:0/2096576087 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f1a6806c4e0 0x7f1a6806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:01.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.391+0000 7f1a7e59c700 1 --2- 172.21.15.114:0/2096576087 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f1a6806c4e0 0x7f1a6806e9a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000ac50 tx=0x7f1a7000a380 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:01.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.395+0000 7f1a6f7fe700 1 -- 172.21.15.114:0/2096576087 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1a7405b3b0 con 0x7f1a801026c0 2024-02-16T22:17:01.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.640+0000 7f1a84cf3700 1 -- 172.21.15.114:0/2096576087 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f1a8004ea90 con 0x7f1a801026c0 2024-02-16T22:17:01.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.644+0000 7f1a6f7fe700 1 -- 172.21.15.114:0/2096576087 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+173047 (secure 0 0 0) 0x7f1a74027090 con 0x7f1a801026c0 2024-02-16T22:17:01.648 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:17:01.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.647+0000 7f1a84cf3700 1 -- 172.21.15.114:0/2096576087 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f1a6806c4e0 msgr2=0x7f1a6806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:01.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.647+0000 7f1a84cf3700 1 --2- 172.21.15.114:0/2096576087 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f1a6806c4e0 0x7f1a6806e9a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000ac50 tx=0x7f1a7000a380 comp rx=0 tx=0).stop 2024-02-16T22:17:01.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.647+0000 7f1a84cf3700 1 -- 172.21.15.114:0/2096576087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a801026c0 msgr2=0x7f1a8019ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:01.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.647+0000 7f1a84cf3700 1 --2- 172.21.15.114:0/2096576087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a801026c0 0x7f1a8019ca70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1a740092e0 tx=0x7f1a7400bd70 comp rx=0 tx=0).stop 2024-02-16T22:17:01.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.647+0000 7f1a84cf3700 1 -- 172.21.15.114:0/2096576087 shutdown_connections 2024-02-16T22:17:01.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.647+0000 7f1a84cf3700 1 --2- 172.21.15.114:0/2096576087 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f1a6806c4e0 0x7f1a6806e9a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:01.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.647+0000 7f1a84cf3700 1 --2- 172.21.15.114:0/2096576087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a80101da0 0x7f1a8019c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:01.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.647+0000 7f1a84cf3700 1 --2- 172.21.15.114:0/2096576087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a801026c0 0x7f1a8019ca70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:01.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.647+0000 7f1a84cf3700 1 -- 172.21.15.114:0/2096576087 >> 172.21.15.114:0/2096576087 conn(0x7f1a800fb550 msgr2=0x7f1a801053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:01.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.647+0000 7f1a84cf3700 1 -- 172.21.15.114:0/2096576087 shutdown_connections 2024-02-16T22:17:01.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:01.647+0000 7f1a84cf3700 1 -- 172.21.15.114:0/2096576087 wait complete. 2024-02-16T22:17:02.352 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":19,"active_gid":14215,"active_name":"smithi114.vcullz","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":3153243195},{"type":"v1","addr":"172.21.15.114:6801","nonce":3153243195}]},"active_addr":"172.21.15.114:6801/3153243195","active_change":"2024-02-16T22:13:11.405326+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14238,"name":"smithi123.inzgjd","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.114:8443/","prometheus":"http://172.21.15.114:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.114:0","nonce":2393113501}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.114:0","nonce":693887539}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.114:0","nonce":4031283053}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.114:0","nonce":3991553908}]}]} 2024-02-16T22:17:02.355 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-16T22:17:02.355 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-16T22:17:02.356 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-16T22:17:02.566 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:17:02.686 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:02 smithi114 ceph-mon[29235]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:02.686 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:02 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/2096576087' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-16T22:17:02.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:02 smithi123 ceph-mon[37161]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:02.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:02 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/2096576087' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-16T22:17:04.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.201+0000 7fad5fb74700 1 -- 172.21.15.114:0/2976510153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad581026c0 msgr2=0x7fad5810abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:04.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.201+0000 7fad5fb74700 1 --2- 172.21.15.114:0/2976510153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad581026c0 0x7fad5810abb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fad48009910 tx=0x7fad48009c20 comp rx=0 tx=0).stop 2024-02-16T22:17:04.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.201+0000 7fad5fb74700 1 -- 172.21.15.114:0/2976510153 shutdown_connections 2024-02-16T22:17:04.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.201+0000 7fad5fb74700 1 --2- 172.21.15.114:0/2976510153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad581026c0 0x7fad5810abb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:04.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.201+0000 7fad5fb74700 1 --2- 172.21.15.114:0/2976510153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad58101da0 0x7fad58102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:04.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.201+0000 7fad5fb74700 1 -- 172.21.15.114:0/2976510153 >> 172.21.15.114:0/2976510153 conn(0x7fad580fb550 msgr2=0x7fad580fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:04.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.201+0000 7fad5fb74700 1 -- 172.21.15.114:0/2976510153 shutdown_connections 2024-02-16T22:17:04.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.202+0000 7fad5fb74700 1 -- 172.21.15.114:0/2976510153 wait complete. 2024-02-16T22:17:04.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.202+0000 7fad5fb74700 1 Processor -- start 2024-02-16T22:17:04.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.202+0000 7fad5fb74700 1 -- start start 2024-02-16T22:17:04.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.202+0000 7fad5fb74700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad58101da0 0x7fad5819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:04.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.202+0000 7fad5fb74700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad581026c0 0x7fad5819ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:04.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.202+0000 7fad5fb74700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5819d100 con 0x7fad58101da0 2024-02-16T22:17:04.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.202+0000 7fad5fb74700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad581965b0 con 0x7fad581026c0 2024-02-16T22:17:04.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.202+0000 7fad5d910700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad58101da0 0x7fad5819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:04.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.203+0000 7fad5d910700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad58101da0 0x7fad5819c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60208/0 (socket says 172.21.15.114:60208) 2024-02-16T22:17:04.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.203+0000 7fad5d910700 1 -- 172.21.15.114:0/3905859006 learned_addr learned my addr 172.21.15.114:0/3905859006 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:17:04.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.203+0000 7fad5d10f700 1 --2- 172.21.15.114:0/3905859006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad581026c0 0x7fad5819ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:04.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.203+0000 7fad5d910700 1 -- 172.21.15.114:0/3905859006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad581026c0 msgr2=0x7fad5819ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:04.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.203+0000 7fad5d910700 1 --2- 172.21.15.114:0/3905859006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad581026c0 0x7fad5819ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:04.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.203+0000 7fad5d910700 1 -- 172.21.15.114:0/3905859006 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad480095f0 con 0x7fad58101da0 2024-02-16T22:17:04.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.203+0000 7fad5d910700 1 --2- 172.21.15.114:0/3905859006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad58101da0 0x7fad5819c530 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fad5400d8d0 tx=0x7fad5400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:04.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.203+0000 7fad4effd700 1 -- 172.21.15.114:0/3905859006 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad54009940 con 0x7fad58101da0 2024-02-16T22:17:04.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.203+0000 7fad4effd700 1 -- 172.21.15.114:0/3905859006 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fad54010460 con 0x7fad58101da0 2024-02-16T22:17:04.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.203+0000 7fad4effd700 1 -- 172.21.15.114:0/3905859006 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad5400f5f0 con 0x7fad58101da0 2024-02-16T22:17:04.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.203+0000 7fad5fb74700 1 -- 172.21.15.114:0/3905859006 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad58196890 con 0x7fad58101da0 2024-02-16T22:17:04.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.203+0000 7fad5fb74700 1 -- 172.21.15.114:0/3905859006 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad58196d90 con 0x7fad58101da0 2024-02-16T22:17:04.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.204+0000 7fad5fb74700 1 -- 172.21.15.114:0/3905859006 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad581082b0 con 0x7fad58101da0 2024-02-16T22:17:04.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.204+0000 7fad4effd700 1 -- 172.21.15.114:0/3905859006 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fad540105d0 con 0x7fad58101da0 2024-02-16T22:17:04.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.205+0000 7fad4effd700 1 --2- 172.21.15.114:0/3905859006 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fad4406c4e0 0x7fad4406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:04.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.205+0000 7fad5d10f700 1 --2- 172.21.15.114:0/3905859006 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fad4406c4e0 0x7fad4406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:04.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.205+0000 7fad4effd700 1 -- 172.21.15.114:0/3905859006 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fad5408b4f0 con 0x7fad58101da0 2024-02-16T22:17:04.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.205+0000 7fad5d10f700 1 --2- 172.21.15.114:0/3905859006 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fad4406c4e0 0x7fad4406e9a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fad580fcc90 tx=0x7fad4801a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:04.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.209+0000 7fad4effd700 1 -- 172.21.15.114:0/3905859006 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fad54059aa0 con 0x7fad58101da0 2024-02-16T22:17:04.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.398+0000 7fad5fb74700 1 -- 172.21.15.114:0/3905859006 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad58197080 con 0x7fad58101da0 2024-02-16T22:17:04.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.399+0000 7fad4effd700 1 -- 172.21.15.114:0/3905859006 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11398 (secure 0 0 0) 0x7fad54059630 con 0x7fad58101da0 2024-02-16T22:17:04.401 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:17:04.401 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":40,"fsid":"1dcbaabe-cd18-11ee-95ba-87774f69a715","created":"2024-02-16T22:11:02.186164+0000","modified":"2024-02-16T22:16:48.855087+0000","last_up_change":"2024-02-16T22:16:47.516304+0000","last_in_change":"2024-02-16T22:16:28.224489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-16T22:15:31.749346+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3a72ccf1-3f93-4c06-99c3-092a7f9a732d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1379159586},{"type":"v1","addr":"172.21.15.114:6803","nonce":1379159586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1379159586},{"type":"v1","addr":"172.21.15.114:6805","nonce":1379159586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1379159586},{"type":"v1","addr":"172.21.15.114:6809","nonce":1379159586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1379159586},{"type":"v1","addr":"172.21.15.114:6807","nonce":1379159586}]},"public_addr":"172.21.15.114:6803/1379159586","cluster_addr":"172.21.15.114:6805/1379159586","heartbeat_back_addr":"172.21.15.114:6809/1379159586","heartbeat_front_addr":"172.21.15.114:6807/1379159586","state":["exists","up"]},{"osd":1,"uuid":"1e8829dd-cac2-462f-aca1-86bcfb475a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1548199763},{"type":"v1","addr":"172.21.15.114:6811","nonce":1548199763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1548199763},{"type":"v1","addr":"172.21.15.114:6813","nonce":1548199763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1548199763},{"type":"v1","addr":"172.21.15.114:6817","nonce":1548199763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1548199763},{"type":"v1","addr":"172.21.15.114:6815","nonce":1548199763}]},"public_addr":"172.21.15.114:6811/1548199763","cluster_addr":"172.21.15.114:6813/1548199763","heartbeat_back_addr":"172.21.15.114:6817/1548199763","heartbeat_front_addr":"172.21.15.114:6815/1548199763","state":["exists","up"]},{"osd":2,"uuid":"44f9fce8-5ea8-4741-9936-6720df91b628","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1293302101},{"type":"v1","addr":"172.21.15.114:6819","nonce":1293302101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1293302101},{"type":"v1","addr":"172.21.15.114:6821","nonce":1293302101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1293302101},{"type":"v1","addr":"172.21.15.114:6825","nonce":1293302101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1293302101},{"type":"v1","addr":"172.21.15.114:6823","nonce":1293302101}]},"public_addr":"172.21.15.114:6819/1293302101","cluster_addr":"172.21.15.114:6821/1293302101","heartbeat_back_addr":"172.21.15.114:6825/1293302101","heartbeat_front_addr":"172.21.15.114:6823/1293302101","state":["exists","up"]},{"osd":3,"uuid":"b6eb4941-bef4-4722-9050-cd6c041f9fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3211307893},{"type":"v1","addr":"172.21.15.123:6801","nonce":3211307893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3211307893},{"type":"v1","addr":"172.21.15.123:6803","nonce":3211307893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3211307893},{"type":"v1","addr":"172.21.15.123:6807","nonce":3211307893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3211307893},{"type":"v1","addr":"172.21.15.123:6805","nonce":3211307893}]},"public_addr":"172.21.15.123:6801/3211307893","cluster_addr":"172.21.15.123:6803/3211307893","heartbeat_back_addr":"172.21.15.123:6807/3211307893","heartbeat_front_addr":"172.21.15.123:6805/3211307893","state":["exists","up"]},{"osd":4,"uuid":"e9caa348-26f4-4ed5-a64e-59231fc48546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3994684204},{"type":"v1","addr":"172.21.15.123:6809","nonce":3994684204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3994684204},{"type":"v1","addr":"172.21.15.123:6811","nonce":3994684204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3994684204},{"type":"v1","addr":"172.21.15.123:6815","nonce":3994684204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3994684204},{"type":"v1","addr":"172.21.15.123:6813","nonce":3994684204}]},"public_addr":"172.21.15.123:6809/3994684204","cluster_addr":"172.21.15.123:6811/3994684204","heartbeat_back_addr":"172.21.15.123:6815/3994684204","heartbeat_front_addr":"172.21.15.123:6813/3994684204","state":["exists","up"]},{"osd":5,"uuid":"79d81af1-2361-430d-bc76-35ef74aa6bad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2122343545},{"type":"v1","addr":"172.21.15.123:6817","nonce":2122343545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":2122343545},{"type":"v1","addr":"172.21.15.123:6819","nonce":2122343545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":2122343545},{"type":"v1","addr":"172.21.15.123:6823","nonce":2122343545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2122343545},{"type":"v1","addr":"172.21.15.123:6821","nonce":2122343545}]},"public_addr":"172.21.15.123:6817/2122343545","cluster_addr":"172.21.15.123:6819/2122343545","heartbeat_back_addr":"172.21.15.123:6823/2122343545","heartbeat_front_addr":"172.21.15.123:6821/2122343545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:14:32.818023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:15:00.597924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:15:29.574022+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:15:53.159458+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:16:20.058254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:16:45.789725+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:0/2762476246":"2024-02-17T22:13:11.405214+0000","172.21.15.114:6801/2648536185":"2024-02-17T22:13:11.405214+0000","172.21.15.114:0/1034649687":"2024-02-17T22:13:11.405214+0000","172.21.15.114:0/1349416361":"2024-02-17T22:12:00.492971+0000","172.21.15.114:0/499892900":"2024-02-17T22:12:00.492971+0000","172.21.15.114:0/744140874":"2024-02-17T22:12:00.492971+0000","172.21.15.114:6800/2648536185":"2024-02-17T22:13:11.405214+0000","172.21.15.114:6801/625910309":"2024-02-17T22:12:00.492971+0000","172.21.15.114:6800/2055574123":"2024-02-17T22:11:27.861007+0000","172.21.15.114:0/3971798486":"2024-02-17T22:11:27.861007+0000","172.21.15.114:0/4029631970":"2024-02-17T22:11:27.861007+0000","172.21.15.114:0/3211326148":"2024-02-17T22:13:11.405214+0000","172.21.15.114:6800/625910309":"2024-02-17T22:12:00.492971+0000","172.21.15.114:6801/2055574123":"2024-02-17T22:11:27.861007+0000","172.21.15.114:0/2825621999":"2024-02-17T22:11:27.861007+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-16T22:17:04.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.401+0000 7fad5fb74700 1 -- 172.21.15.114:0/3905859006 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fad4406c4e0 msgr2=0x7fad4406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:04.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.401+0000 7fad5fb74700 1 --2- 172.21.15.114:0/3905859006 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fad4406c4e0 0x7fad4406e9a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fad580fcc90 tx=0x7fad4801a040 comp rx=0 tx=0).stop 2024-02-16T22:17:04.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.401+0000 7fad5fb74700 1 -- 172.21.15.114:0/3905859006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad58101da0 msgr2=0x7fad5819c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:04.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.401+0000 7fad5fb74700 1 --2- 172.21.15.114:0/3905859006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad58101da0 0x7fad5819c530 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fad5400d8d0 tx=0x7fad5400dc90 comp rx=0 tx=0).stop 2024-02-16T22:17:04.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.401+0000 7fad5fb74700 1 -- 172.21.15.114:0/3905859006 shutdown_connections 2024-02-16T22:17:04.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.401+0000 7fad5fb74700 1 --2- 172.21.15.114:0/3905859006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad58101da0 0x7fad5819c530 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:04.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.401+0000 7fad5fb74700 1 --2- 172.21.15.114:0/3905859006 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fad4406c4e0 0x7fad4406e9a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:04.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.401+0000 7fad5fb74700 1 --2- 172.21.15.114:0/3905859006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad581026c0 0x7fad5819ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:04.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.401+0000 7fad5fb74700 1 -- 172.21.15.114:0/3905859006 >> 172.21.15.114:0/3905859006 conn(0x7fad580fb550 msgr2=0x7fad581053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:04.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.401+0000 7fad5fb74700 1 -- 172.21.15.114:0/3905859006 shutdown_connections 2024-02-16T22:17:04.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:04.401+0000 7fad5fb74700 1 -- 172.21.15.114:0/3905859006 wait complete. 2024-02-16T22:17:04.690 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:04 smithi114 ceph-mon[29235]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:04.690 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:04 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/3905859006' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T22:17:04.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:04 smithi123 ceph-mon[37161]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:04.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:04 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/3905859006' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T22:17:04.971 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-16T22:17:04.971 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-16T22:17:05.178 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:17:06.626 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:06 smithi114 ceph-mon[29235]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:06.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:06 smithi123 ceph-mon[37161]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:06.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.848+0000 7f0f88d9c700 1 -- 172.21.15.114:0/646591020 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f84101770 msgr2=0x7f0f841057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:06.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.848+0000 7f0f88d9c700 1 --2- 172.21.15.114:0/646591020 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f84101770 0x7f0f841057c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0f78009910 tx=0x7f0f78009c20 comp rx=0 tx=0).stop 2024-02-16T22:17:06.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.848+0000 7f0f88d9c700 1 -- 172.21.15.114:0/646591020 shutdown_connections 2024-02-16T22:17:06.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.848+0000 7f0f88d9c700 1 --2- 172.21.15.114:0/646591020 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f84101770 0x7f0f841057c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:06.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.848+0000 7f0f88d9c700 1 --2- 172.21.15.114:0/646591020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f84100dc0 0x7f0f841011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:06.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.848+0000 7f0f88d9c700 1 -- 172.21.15.114:0/646591020 >> 172.21.15.114:0/646591020 conn(0x7f0f840fc650 msgr2=0x7f0f840fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:06.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.849+0000 7f0f88d9c700 1 -- 172.21.15.114:0/646591020 shutdown_connections 2024-02-16T22:17:06.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.849+0000 7f0f88d9c700 1 -- 172.21.15.114:0/646591020 wait complete. 2024-02-16T22:17:06.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.849+0000 7f0f88d9c700 1 Processor -- start 2024-02-16T22:17:06.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.849+0000 7f0f88d9c700 1 -- start start 2024-02-16T22:17:06.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.849+0000 7f0f88d9c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f84100dc0 0x7f0f8419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:06.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.849+0000 7f0f88d9c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f84101770 0x7f0f8419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:06.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.849+0000 7f0f88d9c700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8419d0a0 con 0x7f0f84101770 2024-02-16T22:17:06.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.849+0000 7f0f88d9c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f841965e0 con 0x7f0f84100dc0 2024-02-16T22:17:06.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.850+0000 7f0f81d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f84101770 0x7f0f8419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:06.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.850+0000 7f0f81d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f84101770 0x7f0f8419caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60218/0 (socket says 172.21.15.114:60218) 2024-02-16T22:17:06.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.850+0000 7f0f81d9b700 1 -- 172.21.15.114:0/943737361 learned_addr learned my addr 172.21.15.114:0/943737361 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:17:06.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.850+0000 7f0f8259c700 1 --2- 172.21.15.114:0/943737361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f84100dc0 0x7f0f8419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:06.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.850+0000 7f0f81d9b700 1 -- 172.21.15.114:0/943737361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f84100dc0 msgr2=0x7f0f8419c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:06.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.850+0000 7f0f81d9b700 1 --2- 172.21.15.114:0/943737361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f84100dc0 0x7f0f8419c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:06.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.850+0000 7f0f81d9b700 1 -- 172.21.15.114:0/943737361 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f780095f0 con 0x7f0f84101770 2024-02-16T22:17:06.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.850+0000 7f0f81d9b700 1 --2- 172.21.15.114:0/943737361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f84101770 0x7f0f8419caa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800bda0 tx=0x7f0f7800bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:06.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.850+0000 7f0f737fe700 1 -- 172.21.15.114:0/943737361 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f7801d070 con 0x7f0f84101770 2024-02-16T22:17:06.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.850+0000 7f0f88d9c700 1 -- 172.21.15.114:0/943737361 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f84196860 con 0x7f0f84101770 2024-02-16T22:17:06.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.850+0000 7f0f737fe700 1 -- 172.21.15.114:0/943737361 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0f78004fa0 con 0x7f0f84101770 2024-02-16T22:17:06.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.850+0000 7f0f737fe700 1 -- 172.21.15.114:0/943737361 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f7800f690 con 0x7f0f84101770 2024-02-16T22:17:06.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.850+0000 7f0f88d9c700 1 -- 172.21.15.114:0/943737361 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f84196d50 con 0x7f0f84101770 2024-02-16T22:17:06.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.851+0000 7f0f88d9c700 1 -- 172.21.15.114:0/943737361 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f841091c0 con 0x7f0f84101770 2024-02-16T22:17:06.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.852+0000 7f0f737fe700 1 -- 172.21.15.114:0/943737361 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f0f780228c0 con 0x7f0f84101770 2024-02-16T22:17:06.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.852+0000 7f0f737fe700 1 --2- 172.21.15.114:0/943737361 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f0f6c06c4e0 0x7f0f6c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:06.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.852+0000 7f0f737fe700 1 -- 172.21.15.114:0/943737361 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f0f7808cc40 con 0x7f0f84101770 2024-02-16T22:17:06.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.852+0000 7f0f8259c700 1 --2- 172.21.15.114:0/943737361 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f0f6c06c4e0 0x7f0f6c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:06.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.853+0000 7f0f8259c700 1 --2- 172.21.15.114:0/943737361 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f0f6c06c4e0 0x7f0f6c06e9a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0f7400ac50 tx=0x7f0f7400a380 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:06.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:06.856+0000 7f0f737fe700 1 -- 172.21.15.114:0/943737361 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0f7805b1f0 con 0x7f0f84101770 2024-02-16T22:17:07.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:07.048+0000 7f0f88d9c700 1 -- 172.21.15.114:0/943737361 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f8404ea90 con 0x7f0f84101770 2024-02-16T22:17:07.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:07.049+0000 7f0f737fe700 1 -- 172.21.15.114:0/943737361 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11398 (secure 0 0 0) 0x7f0f7805ad80 con 0x7f0f84101770 2024-02-16T22:17:07.051 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:17:07.051 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":40,"fsid":"1dcbaabe-cd18-11ee-95ba-87774f69a715","created":"2024-02-16T22:11:02.186164+0000","modified":"2024-02-16T22:16:48.855087+0000","last_up_change":"2024-02-16T22:16:47.516304+0000","last_in_change":"2024-02-16T22:16:28.224489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-16T22:15:31.749346+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3a72ccf1-3f93-4c06-99c3-092a7f9a732d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1379159586},{"type":"v1","addr":"172.21.15.114:6803","nonce":1379159586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1379159586},{"type":"v1","addr":"172.21.15.114:6805","nonce":1379159586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1379159586},{"type":"v1","addr":"172.21.15.114:6809","nonce":1379159586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1379159586},{"type":"v1","addr":"172.21.15.114:6807","nonce":1379159586}]},"public_addr":"172.21.15.114:6803/1379159586","cluster_addr":"172.21.15.114:6805/1379159586","heartbeat_back_addr":"172.21.15.114:6809/1379159586","heartbeat_front_addr":"172.21.15.114:6807/1379159586","state":["exists","up"]},{"osd":1,"uuid":"1e8829dd-cac2-462f-aca1-86bcfb475a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1548199763},{"type":"v1","addr":"172.21.15.114:6811","nonce":1548199763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1548199763},{"type":"v1","addr":"172.21.15.114:6813","nonce":1548199763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1548199763},{"type":"v1","addr":"172.21.15.114:6817","nonce":1548199763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1548199763},{"type":"v1","addr":"172.21.15.114:6815","nonce":1548199763}]},"public_addr":"172.21.15.114:6811/1548199763","cluster_addr":"172.21.15.114:6813/1548199763","heartbeat_back_addr":"172.21.15.114:6817/1548199763","heartbeat_front_addr":"172.21.15.114:6815/1548199763","state":["exists","up"]},{"osd":2,"uuid":"44f9fce8-5ea8-4741-9936-6720df91b628","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1293302101},{"type":"v1","addr":"172.21.15.114:6819","nonce":1293302101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1293302101},{"type":"v1","addr":"172.21.15.114:6821","nonce":1293302101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1293302101},{"type":"v1","addr":"172.21.15.114:6825","nonce":1293302101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1293302101},{"type":"v1","addr":"172.21.15.114:6823","nonce":1293302101}]},"public_addr":"172.21.15.114:6819/1293302101","cluster_addr":"172.21.15.114:6821/1293302101","heartbeat_back_addr":"172.21.15.114:6825/1293302101","heartbeat_front_addr":"172.21.15.114:6823/1293302101","state":["exists","up"]},{"osd":3,"uuid":"b6eb4941-bef4-4722-9050-cd6c041f9fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3211307893},{"type":"v1","addr":"172.21.15.123:6801","nonce":3211307893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3211307893},{"type":"v1","addr":"172.21.15.123:6803","nonce":3211307893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3211307893},{"type":"v1","addr":"172.21.15.123:6807","nonce":3211307893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3211307893},{"type":"v1","addr":"172.21.15.123:6805","nonce":3211307893}]},"public_addr":"172.21.15.123:6801/3211307893","cluster_addr":"172.21.15.123:6803/3211307893","heartbeat_back_addr":"172.21.15.123:6807/3211307893","heartbeat_front_addr":"172.21.15.123:6805/3211307893","state":["exists","up"]},{"osd":4,"uuid":"e9caa348-26f4-4ed5-a64e-59231fc48546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3994684204},{"type":"v1","addr":"172.21.15.123:6809","nonce":3994684204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3994684204},{"type":"v1","addr":"172.21.15.123:6811","nonce":3994684204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3994684204},{"type":"v1","addr":"172.21.15.123:6815","nonce":3994684204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3994684204},{"type":"v1","addr":"172.21.15.123:6813","nonce":3994684204}]},"public_addr":"172.21.15.123:6809/3994684204","cluster_addr":"172.21.15.123:6811/3994684204","heartbeat_back_addr":"172.21.15.123:6815/3994684204","heartbeat_front_addr":"172.21.15.123:6813/3994684204","state":["exists","up"]},{"osd":5,"uuid":"79d81af1-2361-430d-bc76-35ef74aa6bad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2122343545},{"type":"v1","addr":"172.21.15.123:6817","nonce":2122343545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":2122343545},{"type":"v1","addr":"172.21.15.123:6819","nonce":2122343545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":2122343545},{"type":"v1","addr":"172.21.15.123:6823","nonce":2122343545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2122343545},{"type":"v1","addr":"172.21.15.123:6821","nonce":2122343545}]},"public_addr":"172.21.15.123:6817/2122343545","cluster_addr":"172.21.15.123:6819/2122343545","heartbeat_back_addr":"172.21.15.123:6823/2122343545","heartbeat_front_addr":"172.21.15.123:6821/2122343545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:14:32.818023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:15:00.597924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:15:29.574022+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:15:53.159458+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:16:20.058254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:16:45.789725+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:0/2762476246":"2024-02-17T22:13:11.405214+0000","172.21.15.114:6801/2648536185":"2024-02-17T22:13:11.405214+0000","172.21.15.114:0/1034649687":"2024-02-17T22:13:11.405214+0000","172.21.15.114:0/1349416361":"2024-02-17T22:12:00.492971+0000","172.21.15.114:0/499892900":"2024-02-17T22:12:00.492971+0000","172.21.15.114:0/744140874":"2024-02-17T22:12:00.492971+0000","172.21.15.114:6800/2648536185":"2024-02-17T22:13:11.405214+0000","172.21.15.114:6801/625910309":"2024-02-17T22:12:00.492971+0000","172.21.15.114:6800/2055574123":"2024-02-17T22:11:27.861007+0000","172.21.15.114:0/3971798486":"2024-02-17T22:11:27.861007+0000","172.21.15.114:0/4029631970":"2024-02-17T22:11:27.861007+0000","172.21.15.114:0/3211326148":"2024-02-17T22:13:11.405214+0000","172.21.15.114:6800/625910309":"2024-02-17T22:12:00.492971+0000","172.21.15.114:6801/2055574123":"2024-02-17T22:11:27.861007+0000","172.21.15.114:0/2825621999":"2024-02-17T22:11:27.861007+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-16T22:17:07.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:07.051+0000 7f0f88d9c700 1 -- 172.21.15.114:0/943737361 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f0f6c06c4e0 msgr2=0x7f0f6c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:07.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:07.051+0000 7f0f88d9c700 1 --2- 172.21.15.114:0/943737361 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f0f6c06c4e0 0x7f0f6c06e9a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0f7400ac50 tx=0x7f0f7400a380 comp rx=0 tx=0).stop 2024-02-16T22:17:07.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:07.051+0000 7f0f88d9c700 1 -- 172.21.15.114:0/943737361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f84101770 msgr2=0x7f0f8419caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:07.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:07.051+0000 7f0f88d9c700 1 --2- 172.21.15.114:0/943737361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f84101770 0x7f0f8419caa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800bda0 tx=0x7f0f7800bdd0 comp rx=0 tx=0).stop 2024-02-16T22:17:07.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:07.051+0000 7f0f88d9c700 1 -- 172.21.15.114:0/943737361 shutdown_connections 2024-02-16T22:17:07.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:07.051+0000 7f0f88d9c700 1 --2- 172.21.15.114:0/943737361 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f0f6c06c4e0 0x7f0f6c06e9a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:07.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:07.051+0000 7f0f88d9c700 1 --2- 172.21.15.114:0/943737361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f84100dc0 0x7f0f8419c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:07.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:07.051+0000 7f0f88d9c700 1 --2- 172.21.15.114:0/943737361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f84101770 0x7f0f8419caa0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:07.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:07.051+0000 7f0f88d9c700 1 -- 172.21.15.114:0/943737361 >> 172.21.15.114:0/943737361 conn(0x7f0f840fc650 msgr2=0x7f0f840fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:07.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:07.051+0000 7f0f88d9c700 1 -- 172.21.15.114:0/943737361 shutdown_connections 2024-02-16T22:17:07.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:07.051+0000 7f0f88d9c700 1 -- 172.21.15.114:0/943737361 wait complete. 2024-02-16T22:17:07.705 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-16T22:17:07.710 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-16T22:17:07.710 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-16T22:17:07.711 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-16T22:17:07.711 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-16T22:17:07.712 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-16T22:17:07.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:07 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/943737361' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T22:17:07.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:07 smithi114 ceph-mon[29235]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:07.973 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:17:08.021 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:17:08.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:07 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/943737361' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T22:17:08.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:07 smithi123 ceph-mon[37161]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:08.583 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:17:08.584 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:17:08.603 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:17:09.094 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:17:10.516 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:10 smithi114 ceph-mon[29235]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:10.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:10 smithi123 ceph-mon[37161]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:11.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:11 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:17:11.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:11 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:17:11.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.939+0000 7f3f902d7700 1 -- 172.21.15.114:0/2844871205 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f88101780 msgr2=0x7f3f881057d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:11.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.939+0000 7f3f902d7700 1 --2- 172.21.15.114:0/2844871205 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f88101780 0x7f3f881057d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3f84009910 tx=0x7f3f84009c20 comp rx=0 tx=0).stop 2024-02-16T22:17:11.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.939+0000 7f3f902d7700 1 -- 172.21.15.114:0/2844871205 shutdown_connections 2024-02-16T22:17:11.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.939+0000 7f3f902d7700 1 --2- 172.21.15.114:0/2844871205 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f88101780 0x7f3f881057d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:11.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.939+0000 7f3f902d7700 1 --2- 172.21.15.114:0/2844871205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f88100dd0 0x7f3f881011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:11.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.939+0000 7f3f902d7700 1 -- 172.21.15.114:0/2844871205 >> 172.21.15.114:0/2844871205 conn(0x7f3f880fc640 msgr2=0x7f3f880fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:11.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.939+0000 7f3f902d7700 1 -- 172.21.15.114:0/2844871205 shutdown_connections 2024-02-16T22:17:11.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.939+0000 7f3f902d7700 1 -- 172.21.15.114:0/2844871205 wait complete. 2024-02-16T22:17:11.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.940+0000 7f3f902d7700 1 Processor -- start 2024-02-16T22:17:11.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.940+0000 7f3f902d7700 1 -- start start 2024-02-16T22:17:11.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.940+0000 7f3f902d7700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f88100dd0 0x7f3f88073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:11.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.940+0000 7f3f902d7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f88101780 0x7f3f880735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:11.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.940+0000 7f3f902d7700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f88073b10 con 0x7f3f88100dd0 2024-02-16T22:17:11.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.940+0000 7f3f902d7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f88073c50 con 0x7f3f88101780 2024-02-16T22:17:11.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.940+0000 7f3f8e073700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f88100dd0 0x7f3f88073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:11.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.940+0000 7f3f8e073700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f88100dd0 0x7f3f88073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47838/0 (socket says 172.21.15.114:47838) 2024-02-16T22:17:11.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.940+0000 7f3f8e073700 1 -- 172.21.15.114:0/1884919060 learned_addr learned my addr 172.21.15.114:0/1884919060 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:17:11.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.940+0000 7f3f8d872700 1 --2- 172.21.15.114:0/1884919060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f88101780 0x7f3f880735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:11.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.941+0000 7f3f8e073700 1 -- 172.21.15.114:0/1884919060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f88101780 msgr2=0x7f3f880735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:11.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.941+0000 7f3f8e073700 1 --2- 172.21.15.114:0/1884919060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f88101780 0x7f3f880735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:11.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.941+0000 7f3f8e073700 1 -- 172.21.15.114:0/1884919060 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f840095f0 con 0x7f3f88100dd0 2024-02-16T22:17:11.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.941+0000 7f3f8e073700 1 --2- 172.21.15.114:0/1884919060 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f88100dd0 0x7f3f88073090 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800d8d0 tx=0x7f3f7800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:11.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.941+0000 7f3f7f7fe700 1 -- 172.21.15.114:0/1884919060 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f780098a0 con 0x7f3f88100dd0 2024-02-16T22:17:11.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.941+0000 7f3f7f7fe700 1 -- 172.21.15.114:0/1884919060 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3f78010460 con 0x7f3f88100dd0 2024-02-16T22:17:11.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.941+0000 7f3f902d7700 1 -- 172.21.15.114:0/1884919060 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f881a2990 con 0x7f3f88100dd0 2024-02-16T22:17:11.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.941+0000 7f3f7f7fe700 1 -- 172.21.15.114:0/1884919060 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f7800f5f0 con 0x7f3f88100dd0 2024-02-16T22:17:11.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.941+0000 7f3f902d7700 1 -- 172.21.15.114:0/1884919060 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f881a2da0 con 0x7f3f88100dd0 2024-02-16T22:17:11.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.942+0000 7f3f902d7700 1 -- 172.21.15.114:0/1884919060 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3f8810dff0 con 0x7f3f88100dd0 2024-02-16T22:17:11.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.942+0000 7f3f7f7fe700 1 -- 172.21.15.114:0/1884919060 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f3f78009a00 con 0x7f3f88100dd0 2024-02-16T22:17:11.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.943+0000 7f3f7f7fe700 1 --2- 172.21.15.114:0/1884919060 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f3f7406c4e0 0x7f3f7406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:11.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.943+0000 7f3f7f7fe700 1 -- 172.21.15.114:0/1884919060 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f3f7808b3d0 con 0x7f3f88100dd0 2024-02-16T22:17:11.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.943+0000 7f3f8d872700 1 --2- 172.21.15.114:0/1884919060 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f3f7406c4e0 0x7f3f7406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:11.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.943+0000 7f3f7f7fe700 1 --2- 172.21.15.114:0/1884919060 >> [v2:172.21.15.114:6810/1548199763,v1:172.21.15.114:6811/1548199763] conn(0x7f3f740721c0 0x7f3f740745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:11.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.943+0000 7f3f7f7fe700 1 -- 172.21.15.114:0/1884919060 --> [v2:172.21.15.114:6810/1548199763,v1:172.21.15.114:6811/1548199763] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3f74074c90 con 0x7f3f740721c0 2024-02-16T22:17:11.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.943+0000 7f3f7f7fe700 1 -- 172.21.15.114:0/1884919060 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f7808daa0 con 0x7f3f88100dd0 2024-02-16T22:17:11.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.943+0000 7f3f8e874700 1 --2- 172.21.15.114:0/1884919060 >> [v2:172.21.15.114:6810/1548199763,v1:172.21.15.114:6811/1548199763] conn(0x7f3f740721c0 0x7f3f740745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:11.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.943+0000 7f3f8d872700 1 --2- 172.21.15.114:0/1884919060 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f3f7406c4e0 0x7f3f7406e9a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3f88074880 tx=0x7f3f8401a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:11.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.943+0000 7f3f8e874700 1 --2- 172.21.15.114:0/1884919060 >> [v2:172.21.15.114:6810/1548199763,v1:172.21.15.114:6811/1548199763] conn(0x7f3f740721c0 0x7f3f740745e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:11.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.944+0000 7f3f7f7fe700 1 -- 172.21.15.114:0/1884919060 <== osd.1 v2:172.21.15.114:6810/1548199763 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3f74074c90 con 0x7f3f740721c0 2024-02-16T22:17:11.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.960+0000 7f3f902d7700 1 -- 172.21.15.114:0/1884919060 --> [v2:172.21.15.114:6810/1548199763,v1:172.21.15.114:6811/1548199763] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3f8804ea90 con 0x7f3f740721c0 2024-02-16T22:17:11.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.960+0000 7f3f7f7fe700 1 -- 172.21.15.114:0/1884919060 <== osd.1 v2:172.21.15.114:6810/1548199763 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3f8804ea90 con 0x7f3f740721c0 2024-02-16T22:17:11.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.960+0000 7f3f902d7700 1 -- 172.21.15.114:0/1884919060 >> [v2:172.21.15.114:6810/1548199763,v1:172.21.15.114:6811/1548199763] conn(0x7f3f740721c0 msgr2=0x7f3f740745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:11.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.960+0000 7f3f902d7700 1 --2- 172.21.15.114:0/1884919060 >> [v2:172.21.15.114:6810/1548199763,v1:172.21.15.114:6811/1548199763] conn(0x7f3f740721c0 0x7f3f740745e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:11.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.960+0000 7f3f902d7700 1 -- 172.21.15.114:0/1884919060 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f3f7406c4e0 msgr2=0x7f3f7406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:11.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.960+0000 7f3f902d7700 1 --2- 172.21.15.114:0/1884919060 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f3f7406c4e0 0x7f3f7406e9a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3f88074880 tx=0x7f3f8401a040 comp rx=0 tx=0).stop 2024-02-16T22:17:11.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.960+0000 7f3f902d7700 1 -- 172.21.15.114:0/1884919060 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f88100dd0 msgr2=0x7f3f88073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:11.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.960+0000 7f3f902d7700 1 --2- 172.21.15.114:0/1884919060 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f88100dd0 0x7f3f88073090 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800d8d0 tx=0x7f3f7800dbe0 comp rx=0 tx=0).stop 2024-02-16T22:17:11.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.960+0000 7f3f902d7700 1 -- 172.21.15.114:0/1884919060 shutdown_connections 2024-02-16T22:17:11.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.960+0000 7f3f902d7700 1 --2- 172.21.15.114:0/1884919060 >> [v2:172.21.15.114:6810/1548199763,v1:172.21.15.114:6811/1548199763] conn(0x7f3f740721c0 0x7f3f740745e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:11.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.960+0000 7f3f902d7700 1 --2- 172.21.15.114:0/1884919060 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f88100dd0 0x7f3f88073090 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:11.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.960+0000 7f3f902d7700 1 --2- 172.21.15.114:0/1884919060 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f3f7406c4e0 0x7f3f7406e9a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:11.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.960+0000 7f3f902d7700 1 --2- 172.21.15.114:0/1884919060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f88101780 0x7f3f880735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:11.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.960+0000 7f3f902d7700 1 -- 172.21.15.114:0/1884919060 >> 172.21.15.114:0/1884919060 conn(0x7f3f880fc640 msgr2=0x7f3f880fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:11.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.960+0000 7f3f902d7700 1 -- 172.21.15.114:0/1884919060 shutdown_connections 2024-02-16T22:17:11.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:11.960+0000 7f3f902d7700 1 -- 172.21.15.114:0/1884919060 wait complete. 2024-02-16T22:17:12.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:12 smithi114 ceph-mon[29235]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:12.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:12 smithi123 ceph-mon[37161]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:14.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:14 smithi114 ceph-mon[29235]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:14.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:14 smithi123 ceph-mon[37161]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:14.833 INFO:teuthology.orchestra.run.smithi114.stdout:60129542172 2024-02-16T22:17:14.834 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-16T22:17:16.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.628+0000 7f5925b73700 1 -- 172.21.15.114:0/2143399328 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5920100e00 msgr2=0x7f59201011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:16.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.628+0000 7f5925b73700 1 --2- 172.21.15.114:0/2143399328 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5920100e00 0x7f59201011e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5910009910 tx=0x7f5910009c20 comp rx=0 tx=0).stop 2024-02-16T22:17:16.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.628+0000 7f5925b73700 1 -- 172.21.15.114:0/2143399328 shutdown_connections 2024-02-16T22:17:16.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.628+0000 7f5925b73700 1 --2- 172.21.15.114:0/2143399328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59201017b0 0x7f5920105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:16.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.628+0000 7f5925b73700 1 --2- 172.21.15.114:0/2143399328 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5920100e00 0x7f59201011e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:16.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.628+0000 7f5925b73700 1 -- 172.21.15.114:0/2143399328 >> 172.21.15.114:0/2143399328 conn(0x7f59200fc6b0 msgr2=0x7f59200fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:16.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.628+0000 7f5925b73700 1 -- 172.21.15.114:0/2143399328 shutdown_connections 2024-02-16T22:17:16.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.628+0000 7f5925b73700 1 -- 172.21.15.114:0/2143399328 wait complete. 2024-02-16T22:17:16.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.629+0000 7f5925b73700 1 Processor -- start 2024-02-16T22:17:16.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.629+0000 7f5925b73700 1 -- start start 2024-02-16T22:17:16.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.629+0000 7f5925b73700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5920100e00 0x7f592019c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:16.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.629+0000 7f5925b73700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f59201017b0 0x7f592019caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:16.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.629+0000 7f5925b73700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592019d0f0 con 0x7f59201017b0 2024-02-16T22:17:16.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.629+0000 7f5925b73700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5920196630 con 0x7f5920100e00 2024-02-16T22:17:16.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.629+0000 7f5916ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f59201017b0 0x7f592019caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:16.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.629+0000 7f5916ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f59201017b0 0x7f592019caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47864/0 (socket says 172.21.15.114:47864) 2024-02-16T22:17:16.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.629+0000 7f5916ffd700 1 -- 172.21.15.114:0/3252147446 learned_addr learned my addr 172.21.15.114:0/3252147446 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:17:16.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.630+0000 7f591f7fe700 1 --2- 172.21.15.114:0/3252147446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5920100e00 0x7f592019c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:16.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.630+0000 7f5916ffd700 1 -- 172.21.15.114:0/3252147446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5920100e00 msgr2=0x7f592019c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:16.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.630+0000 7f5916ffd700 1 --2- 172.21.15.114:0/3252147446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5920100e00 0x7f592019c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:16.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.630+0000 7f5916ffd700 1 -- 172.21.15.114:0/3252147446 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59100095f0 con 0x7f59201017b0 2024-02-16T22:17:16.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.630+0000 7f591f7fe700 1 --2- 172.21.15.114:0/3252147446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5920100e00 0x7f592019c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:17:16.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.630+0000 7f5916ffd700 1 --2- 172.21.15.114:0/3252147446 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f59201017b0 0x7f592019caf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f590800c960 tx=0x7f590800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:16.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.630+0000 7f591d7fa700 1 -- 172.21.15.114:0/3252147446 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5908007a70 con 0x7f59201017b0 2024-02-16T22:17:16.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.630+0000 7f591d7fa700 1 -- 172.21.15.114:0/3252147446 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5908007bd0 con 0x7f59201017b0 2024-02-16T22:17:16.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.630+0000 7f5925b73700 1 -- 172.21.15.114:0/3252147446 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5920196910 con 0x7f59201017b0 2024-02-16T22:17:16.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.630+0000 7f591d7fa700 1 -- 172.21.15.114:0/3252147446 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5908018630 con 0x7f59201017b0 2024-02-16T22:17:16.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.630+0000 7f5925b73700 1 -- 172.21.15.114:0/3252147446 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5920196e60 con 0x7f59201017b0 2024-02-16T22:17:16.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.631+0000 7f5925b73700 1 -- 172.21.15.114:0/3252147446 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f592010e010 con 0x7f59201017b0 2024-02-16T22:17:16.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.631+0000 7f591d7fa700 1 -- 172.21.15.114:0/3252147446 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f590800fa90 con 0x7f59201017b0 2024-02-16T22:17:16.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.632+0000 7f591d7fa700 1 --2- 172.21.15.114:0/3252147446 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f590c06c4e0 0x7f590c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:16.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.632+0000 7f591d7fa700 1 -- 172.21.15.114:0/3252147446 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f590808b430 con 0x7f59201017b0 2024-02-16T22:17:16.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.632+0000 7f591f7fe700 1 --2- 172.21.15.114:0/3252147446 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f590c06c4e0 0x7f590c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:16.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.632+0000 7f591d7fa700 1 --2- 172.21.15.114:0/3252147446 >> [v2:172.21.15.123:6808/3994684204,v1:172.21.15.123:6809/3994684204] conn(0x7f590c0721c0 0x7f590c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:16.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.632+0000 7f591d7fa700 1 -- 172.21.15.114:0/3252147446 --> [v2:172.21.15.123:6808/3994684204,v1:172.21.15.123:6809/3994684204] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f590c074c90 con 0x7f590c0721c0 2024-02-16T22:17:16.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.632+0000 7f591d7fa700 1 -- 172.21.15.114:0/3252147446 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f590808db00 con 0x7f59201017b0 2024-02-16T22:17:16.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.632+0000 7f591f7fe700 1 --2- 172.21.15.114:0/3252147446 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f590c06c4e0 0x7f590c06e9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5910005b40 tx=0x7f591001a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:16.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.632+0000 7f591ffff700 1 --2- 172.21.15.114:0/3252147446 >> [v2:172.21.15.123:6808/3994684204,v1:172.21.15.123:6809/3994684204] conn(0x7f590c0721c0 0x7f590c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:16.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.633+0000 7f591ffff700 1 --2- 172.21.15.114:0/3252147446 >> [v2:172.21.15.123:6808/3994684204,v1:172.21.15.123:6809/3994684204] conn(0x7f590c0721c0 0x7f590c0745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:16.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.633+0000 7f591d7fa700 1 -- 172.21.15.114:0/3252147446 <== osd.4 v2:172.21.15.123:6808/3994684204 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f590c074c90 con 0x7f590c0721c0 2024-02-16T22:17:16.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.649+0000 7f5925b73700 1 -- 172.21.15.114:0/3252147446 --> [v2:172.21.15.123:6808/3994684204,v1:172.21.15.123:6809/3994684204] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f592004ea90 con 0x7f590c0721c0 2024-02-16T22:17:16.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.649+0000 7f591d7fa700 1 -- 172.21.15.114:0/3252147446 <== osd.4 v2:172.21.15.123:6808/3994684204 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f592004ea90 con 0x7f590c0721c0 2024-02-16T22:17:16.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.649+0000 7f5925b73700 1 -- 172.21.15.114:0/3252147446 >> [v2:172.21.15.123:6808/3994684204,v1:172.21.15.123:6809/3994684204] conn(0x7f590c0721c0 msgr2=0x7f590c0745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:16.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.649+0000 7f5925b73700 1 --2- 172.21.15.114:0/3252147446 >> [v2:172.21.15.123:6808/3994684204,v1:172.21.15.123:6809/3994684204] conn(0x7f590c0721c0 0x7f590c0745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:16.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.650+0000 7f5925b73700 1 -- 172.21.15.114:0/3252147446 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f590c06c4e0 msgr2=0x7f590c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:16.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.650+0000 7f5925b73700 1 --2- 172.21.15.114:0/3252147446 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f590c06c4e0 0x7f590c06e9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5910005b40 tx=0x7f591001a040 comp rx=0 tx=0).stop 2024-02-16T22:17:16.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.650+0000 7f5925b73700 1 -- 172.21.15.114:0/3252147446 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f59201017b0 msgr2=0x7f592019caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:16.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.650+0000 7f5925b73700 1 --2- 172.21.15.114:0/3252147446 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f59201017b0 0x7f592019caf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f590800c960 tx=0x7f590800cd20 comp rx=0 tx=0).stop 2024-02-16T22:17:16.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.650+0000 7f5925b73700 1 -- 172.21.15.114:0/3252147446 shutdown_connections 2024-02-16T22:17:16.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.650+0000 7f5925b73700 1 --2- 172.21.15.114:0/3252147446 >> [v2:172.21.15.123:6808/3994684204,v1:172.21.15.123:6809/3994684204] conn(0x7f590c0721c0 0x7f590c0745e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:16.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.650+0000 7f5925b73700 1 --2- 172.21.15.114:0/3252147446 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f590c06c4e0 0x7f590c06e9a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:16.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.650+0000 7f5925b73700 1 --2- 172.21.15.114:0/3252147446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5920100e00 0x7f592019c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:16.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.650+0000 7f5925b73700 1 --2- 172.21.15.114:0/3252147446 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f59201017b0 0x7f592019caf0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:16.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.650+0000 7f5925b73700 1 -- 172.21.15.114:0/3252147446 >> 172.21.15.114:0/3252147446 conn(0x7f59200fc6b0 msgr2=0x7f59200feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:16.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.650+0000 7f5925b73700 1 -- 172.21.15.114:0/3252147446 shutdown_connections 2024-02-16T22:17:16.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.650+0000 7f5925b73700 1 -- 172.21.15.114:0/3252147446 wait complete. 2024-02-16T22:17:16.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:16 smithi123 ceph-mon[37161]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:16.806 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:16 smithi114 ceph-mon[29235]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:16.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.822+0000 7f57e354a700 1 -- 172.21.15.114:0/2880223785 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57dc0ff510 msgr2=0x7f57dc0ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:16.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.822+0000 7f57e354a700 1 --2- 172.21.15.114:0/2880223785 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57dc0ff510 0x7f57dc0ff8f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f57d400b0a0 tx=0x7f57d400b3b0 comp rx=0 tx=0).stop 2024-02-16T22:17:16.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.822+0000 7f57e354a700 1 -- 172.21.15.114:0/2880223785 shutdown_connections 2024-02-16T22:17:16.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.822+0000 7f57e354a700 1 --2- 172.21.15.114:0/2880223785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57dc0ffec0 0x7f57dc1057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:16.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.822+0000 7f57e354a700 1 --2- 172.21.15.114:0/2880223785 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57dc0ff510 0x7f57dc0ff8f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:16.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.822+0000 7f57e354a700 1 -- 172.21.15.114:0/2880223785 >> 172.21.15.114:0/2880223785 conn(0x7f57dc074e00 msgr2=0x7f57dc075210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:16.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.822+0000 7f57e354a700 1 -- 172.21.15.114:0/2880223785 shutdown_connections 2024-02-16T22:17:16.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.822+0000 7f57e354a700 1 -- 172.21.15.114:0/2880223785 wait complete. 2024-02-16T22:17:16.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.823+0000 7f57e354a700 1 Processor -- start 2024-02-16T22:17:16.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.823+0000 7f57e354a700 1 -- start start 2024-02-16T22:17:16.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.823+0000 7f57e354a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57dc0ff510 0x7f57dc100490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:16.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.823+0000 7f57e354a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57dc0ffec0 0x7f57dc1009d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:16.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.823+0000 7f57e354a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc100f10 con 0x7f57dc0ffec0 2024-02-16T22:17:16.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.823+0000 7f57e354a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc101050 con 0x7f57dc0ff510 2024-02-16T22:17:16.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.823+0000 7f57e0ae5700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57dc0ffec0 0x7f57dc1009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:16.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.823+0000 7f57e0ae5700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57dc0ffec0 0x7f57dc1009d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47880/0 (socket says 172.21.15.114:47880) 2024-02-16T22:17:16.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.823+0000 7f57e0ae5700 1 -- 172.21.15.114:0/3703643242 learned_addr learned my addr 172.21.15.114:0/3703643242 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:17:16.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.823+0000 7f57e12e6700 1 --2- 172.21.15.114:0/3703643242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57dc0ff510 0x7f57dc100490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:16.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.823+0000 7f57e0ae5700 1 -- 172.21.15.114:0/3703643242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57dc0ff510 msgr2=0x7f57dc100490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:16.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.823+0000 7f57e0ae5700 1 --2- 172.21.15.114:0/3703643242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57dc0ff510 0x7f57dc100490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:16.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.823+0000 7f57e0ae5700 1 -- 172.21.15.114:0/3703643242 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d4009d00 con 0x7f57dc0ffec0 2024-02-16T22:17:16.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.824+0000 7f57e0ae5700 1 --2- 172.21.15.114:0/3703643242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57dc0ffec0 0x7f57dc1009d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f57d000d900 tx=0x7f57d000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:16.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.824+0000 7f57da7fc700 1 -- 172.21.15.114:0/3703643242 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57d0009900 con 0x7f57dc0ffec0 2024-02-16T22:17:16.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.824+0000 7f57da7fc700 1 -- 172.21.15.114:0/3703643242 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f57d0010460 con 0x7f57dc0ffec0 2024-02-16T22:17:16.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.824+0000 7f57da7fc700 1 -- 172.21.15.114:0/3703643242 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57d000f5f0 con 0x7f57dc0ffec0 2024-02-16T22:17:16.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.824+0000 7f57e354a700 1 -- 172.21.15.114:0/3703643242 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57dc1a2970 con 0x7f57dc0ffec0 2024-02-16T22:17:16.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.824+0000 7f57e354a700 1 -- 172.21.15.114:0/3703643242 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57dc1a2d80 con 0x7f57dc0ffec0 2024-02-16T22:17:16.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.825+0000 7f57e354a700 1 -- 172.21.15.114:0/3703643242 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f57dc10e010 con 0x7f57dc0ffec0 2024-02-16T22:17:16.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.825+0000 7f57da7fc700 1 -- 172.21.15.114:0/3703643242 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f57d00105d0 con 0x7f57dc0ffec0 2024-02-16T22:17:16.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.826+0000 7f57da7fc700 1 --2- 172.21.15.114:0/3703643242 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f57c406c4e0 0x7f57c406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:16.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.826+0000 7f57da7fc700 1 -- 172.21.15.114:0/3703643242 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f57d008b3a0 con 0x7f57dc0ffec0 2024-02-16T22:17:16.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.826+0000 7f57e12e6700 1 --2- 172.21.15.114:0/3703643242 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f57c406c4e0 0x7f57c406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:16.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.826+0000 7f57da7fc700 1 --2- 172.21.15.114:0/3703643242 >> [v2:172.21.15.114:6802/1379159586,v1:172.21.15.114:6803/1379159586] conn(0x7f57c40721c0 0x7f57c40745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:16.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.826+0000 7f57da7fc700 1 -- 172.21.15.114:0/3703643242 --> [v2:172.21.15.114:6802/1379159586,v1:172.21.15.114:6803/1379159586] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f57c4074c90 con 0x7f57c40721c0 2024-02-16T22:17:16.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.826+0000 7f57da7fc700 1 -- 172.21.15.114:0/3703643242 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57d008b820 con 0x7f57dc0ffec0 2024-02-16T22:17:16.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.826+0000 7f57e1ae7700 1 --2- 172.21.15.114:0/3703643242 >> [v2:172.21.15.114:6802/1379159586,v1:172.21.15.114:6803/1379159586] conn(0x7f57c40721c0 0x7f57c40745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:16.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.826+0000 7f57e12e6700 1 --2- 172.21.15.114:0/3703643242 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f57c406c4e0 0x7f57c406e9a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f57d4000f80 tx=0x7f57d40060c0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:16.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.826+0000 7f57e1ae7700 1 --2- 172.21.15.114:0/3703643242 >> [v2:172.21.15.114:6802/1379159586,v1:172.21.15.114:6803/1379159586] conn(0x7f57c40721c0 0x7f57c40745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:16.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.827+0000 7f57da7fc700 1 -- 172.21.15.114:0/3703643242 <== osd.0 v2:172.21.15.114:6802/1379159586 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f57c4074c90 con 0x7f57c40721c0 2024-02-16T22:17:16.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.842+0000 7f57e354a700 1 -- 172.21.15.114:0/3703643242 --> [v2:172.21.15.114:6802/1379159586,v1:172.21.15.114:6803/1379159586] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f57dc04ea90 con 0x7f57c40721c0 2024-02-16T22:17:16.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.843+0000 7f57da7fc700 1 -- 172.21.15.114:0/3703643242 <== osd.0 v2:172.21.15.114:6802/1379159586 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f57dc04ea90 con 0x7f57c40721c0 2024-02-16T22:17:16.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.843+0000 7f57e354a700 1 -- 172.21.15.114:0/3703643242 >> [v2:172.21.15.114:6802/1379159586,v1:172.21.15.114:6803/1379159586] conn(0x7f57c40721c0 msgr2=0x7f57c40745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:16.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.843+0000 7f57e354a700 1 --2- 172.21.15.114:0/3703643242 >> [v2:172.21.15.114:6802/1379159586,v1:172.21.15.114:6803/1379159586] conn(0x7f57c40721c0 0x7f57c40745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:16.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.843+0000 7f57e354a700 1 -- 172.21.15.114:0/3703643242 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f57c406c4e0 msgr2=0x7f57c406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:16.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.843+0000 7f57e354a700 1 --2- 172.21.15.114:0/3703643242 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f57c406c4e0 0x7f57c406e9a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f57d4000f80 tx=0x7f57d40060c0 comp rx=0 tx=0).stop 2024-02-16T22:17:16.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.843+0000 7f57e354a700 1 -- 172.21.15.114:0/3703643242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57dc0ffec0 msgr2=0x7f57dc1009d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:16.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.843+0000 7f57e354a700 1 --2- 172.21.15.114:0/3703643242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57dc0ffec0 0x7f57dc1009d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f57d000d900 tx=0x7f57d000dc10 comp rx=0 tx=0).stop 2024-02-16T22:17:16.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.843+0000 7f57e354a700 1 -- 172.21.15.114:0/3703643242 shutdown_connections 2024-02-16T22:17:16.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.843+0000 7f57e354a700 1 --2- 172.21.15.114:0/3703643242 >> [v2:172.21.15.114:6802/1379159586,v1:172.21.15.114:6803/1379159586] conn(0x7f57c40721c0 0x7f57c40745e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:16.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.843+0000 7f57e354a700 1 --2- 172.21.15.114:0/3703643242 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f57c406c4e0 0x7f57c406e9a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:16.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.843+0000 7f57e354a700 1 --2- 172.21.15.114:0/3703643242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57dc0ff510 0x7f57dc100490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:16.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.843+0000 7f57e354a700 1 --2- 172.21.15.114:0/3703643242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57dc0ffec0 0x7f57dc1009d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:16.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.843+0000 7f57e354a700 1 -- 172.21.15.114:0/3703643242 >> 172.21.15.114:0/3703643242 conn(0x7f57dc074e00 msgr2=0x7f57dc0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:16.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.843+0000 7f57e354a700 1 -- 172.21.15.114:0/3703643242 shutdown_connections 2024-02-16T22:17:16.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:16.843+0000 7f57e354a700 1 -- 172.21.15.114:0/3703643242 wait complete. 2024-02-16T22:17:17.485 INFO:teuthology.orchestra.run.smithi114.stdout:146028888077 2024-02-16T22:17:17.486 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-16T22:17:18.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:17 smithi114 ceph-mon[29235]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:18.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:17 smithi123 ceph-mon[37161]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:18.355 INFO:teuthology.orchestra.run.smithi114.stdout:38654705700 2024-02-16T22:17:18.355 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-16T22:17:18.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.597+0000 7fa46dfad700 1 -- 172.21.15.114:0/277993348 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa468100da0 msgr2=0x7fa468101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:18.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.597+0000 7fa46dfad700 1 --2- 172.21.15.114:0/277993348 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa468100da0 0x7fa468101180 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa460009910 tx=0x7fa460009c20 comp rx=0 tx=0).stop 2024-02-16T22:17:18.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.597+0000 7fa46dfad700 1 -- 172.21.15.114:0/277993348 shutdown_connections 2024-02-16T22:17:18.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.597+0000 7fa46dfad700 1 --2- 172.21.15.114:0/277993348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa468101750 0x7fa4681057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:18.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.597+0000 7fa46dfad700 1 --2- 172.21.15.114:0/277993348 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa468100da0 0x7fa468101180 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:18.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.597+0000 7fa46dfad700 1 -- 172.21.15.114:0/277993348 >> 172.21.15.114:0/277993348 conn(0x7fa4680fc630 msgr2=0x7fa4680fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:18.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.597+0000 7fa46dfad700 1 -- 172.21.15.114:0/277993348 shutdown_connections 2024-02-16T22:17:18.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.597+0000 7fa46dfad700 1 -- 172.21.15.114:0/277993348 wait complete. 2024-02-16T22:17:18.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.597+0000 7fa46dfad700 1 Processor -- start 2024-02-16T22:17:18.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.598+0000 7fa46dfad700 1 -- start start 2024-02-16T22:17:18.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.598+0000 7fa46dfad700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa468100da0 0x7fa46819c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:18.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.598+0000 7fa46dfad700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa468101750 0x7fa46819cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:18.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.598+0000 7fa46dfad700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46819d170 con 0x7fa468100da0 2024-02-16T22:17:18.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.598+0000 7fa46dfad700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa468196620 con 0x7fa468101750 2024-02-16T22:17:18.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.598+0000 7fa4677fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa468100da0 0x7fa46819c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:18.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.598+0000 7fa4677fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa468100da0 0x7fa46819c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47908/0 (socket says 172.21.15.114:47908) 2024-02-16T22:17:18.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.598+0000 7fa4677fe700 1 -- 172.21.15.114:0/1037595591 learned_addr learned my addr 172.21.15.114:0/1037595591 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:17:18.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.598+0000 7fa4677fe700 1 -- 172.21.15.114:0/1037595591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa468101750 msgr2=0x7fa46819cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:18.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.598+0000 7fa4677fe700 1 --2- 172.21.15.114:0/1037595591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa468101750 0x7fa46819cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:18.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.598+0000 7fa4677fe700 1 -- 172.21.15.114:0/1037595591 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4600095f0 con 0x7fa468100da0 2024-02-16T22:17:18.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.598+0000 7fa4677fe700 1 --2- 172.21.15.114:0/1037595591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa468100da0 0x7fa46819c5a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa460006010 tx=0x7fa46000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:18.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.599+0000 7fa464ff9700 1 -- 172.21.15.114:0/1037595591 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa46001d070 con 0x7fa468100da0 2024-02-16T22:17:18.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.599+0000 7fa464ff9700 1 -- 172.21.15.114:0/1037595591 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa46000f460 con 0x7fa468100da0 2024-02-16T22:17:18.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.599+0000 7fa464ff9700 1 -- 172.21.15.114:0/1037595591 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa460021640 con 0x7fa468100da0 2024-02-16T22:17:18.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.599+0000 7fa46dfad700 1 -- 172.21.15.114:0/1037595591 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4681968a0 con 0x7fa468100da0 2024-02-16T22:17:18.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.599+0000 7fa46dfad700 1 -- 172.21.15.114:0/1037595591 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa468196d90 con 0x7fa468100da0 2024-02-16T22:17:18.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.599+0000 7fa46dfad700 1 -- 172.21.15.114:0/1037595591 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa46810e0e0 con 0x7fa468100da0 2024-02-16T22:17:18.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.600+0000 7fa464ff9700 1 -- 172.21.15.114:0/1037595591 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fa46000faf0 con 0x7fa468100da0 2024-02-16T22:17:18.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.601+0000 7fa464ff9700 1 --2- 172.21.15.114:0/1037595591 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa45406c4e0 0x7fa45406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:18.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.601+0000 7fa466ffd700 1 --2- 172.21.15.114:0/1037595591 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa45406c4e0 0x7fa45406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:18.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.601+0000 7fa464ff9700 1 -- 172.21.15.114:0/1037595591 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fa46008c870 con 0x7fa468100da0 2024-02-16T22:17:18.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.601+0000 7fa464ff9700 1 --2- 172.21.15.114:0/1037595591 >> [v2:172.21.15.123:6800/3211307893,v1:172.21.15.123:6801/3211307893] conn(0x7fa4540721c0 0x7fa4540745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:18.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.601+0000 7fa464ff9700 1 -- 172.21.15.114:0/1037595591 --> [v2:172.21.15.123:6800/3211307893,v1:172.21.15.123:6801/3211307893] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa454074c90 con 0x7fa4540721c0 2024-02-16T22:17:18.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.601+0000 7fa464ff9700 1 -- 172.21.15.114:0/1037595591 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4600912a0 con 0x7fa468100da0 2024-02-16T22:17:18.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.601+0000 7fa467fff700 1 --2- 172.21.15.114:0/1037595591 >> [v2:172.21.15.123:6800/3211307893,v1:172.21.15.123:6801/3211307893] conn(0x7fa4540721c0 0x7fa4540745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:18.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.601+0000 7fa466ffd700 1 --2- 172.21.15.114:0/1037595591 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa45406c4e0 0x7fa45406e9a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa4680fcdb0 tx=0x7fa450009250 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:18.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.602+0000 7fa467fff700 1 --2- 172.21.15.114:0/1037595591 >> [v2:172.21.15.123:6800/3211307893,v1:172.21.15.123:6801/3211307893] conn(0x7fa4540721c0 0x7fa4540745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:18.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.603+0000 7fa464ff9700 1 -- 172.21.15.114:0/1037595591 <== osd.3 v2:172.21.15.123:6800/3211307893 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa454074c90 con 0x7fa4540721c0 2024-02-16T22:17:18.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.618+0000 7fa46dfad700 1 -- 172.21.15.114:0/1037595591 --> [v2:172.21.15.123:6800/3211307893,v1:172.21.15.123:6801/3211307893] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa46804ea90 con 0x7fa4540721c0 2024-02-16T22:17:18.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.619+0000 7fa464ff9700 1 -- 172.21.15.114:0/1037595591 <== osd.3 v2:172.21.15.123:6800/3211307893 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa46804ea90 con 0x7fa4540721c0 2024-02-16T22:17:18.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.619+0000 7fa46dfad700 1 -- 172.21.15.114:0/1037595591 >> [v2:172.21.15.123:6800/3211307893,v1:172.21.15.123:6801/3211307893] conn(0x7fa4540721c0 msgr2=0x7fa4540745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:18.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.619+0000 7fa46dfad700 1 --2- 172.21.15.114:0/1037595591 >> [v2:172.21.15.123:6800/3211307893,v1:172.21.15.123:6801/3211307893] conn(0x7fa4540721c0 0x7fa4540745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:18.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.619+0000 7fa46dfad700 1 -- 172.21.15.114:0/1037595591 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa45406c4e0 msgr2=0x7fa45406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:18.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.619+0000 7fa46dfad700 1 --2- 172.21.15.114:0/1037595591 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa45406c4e0 0x7fa45406e9a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa4680fcdb0 tx=0x7fa450009250 comp rx=0 tx=0).stop 2024-02-16T22:17:18.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.619+0000 7fa46dfad700 1 -- 172.21.15.114:0/1037595591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa468100da0 msgr2=0x7fa46819c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:18.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.619+0000 7fa46dfad700 1 --2- 172.21.15.114:0/1037595591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa468100da0 0x7fa46819c5a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa460006010 tx=0x7fa46000be20 comp rx=0 tx=0).stop 2024-02-16T22:17:18.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.619+0000 7fa46dfad700 1 -- 172.21.15.114:0/1037595591 shutdown_connections 2024-02-16T22:17:18.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.619+0000 7fa46dfad700 1 --2- 172.21.15.114:0/1037595591 >> [v2:172.21.15.123:6800/3211307893,v1:172.21.15.123:6801/3211307893] conn(0x7fa4540721c0 0x7fa4540745e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:18.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.619+0000 7fa46dfad700 1 --2- 172.21.15.114:0/1037595591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa468100da0 0x7fa46819c5a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:18.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.619+0000 7fa46dfad700 1 --2- 172.21.15.114:0/1037595591 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa45406c4e0 0x7fa45406e9a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:18.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.619+0000 7fa46dfad700 1 --2- 172.21.15.114:0/1037595591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa468101750 0x7fa46819cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:18.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.619+0000 7fa46dfad700 1 -- 172.21.15.114:0/1037595591 >> 172.21.15.114:0/1037595591 conn(0x7fa4680fc630 msgr2=0x7fa4680fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:18.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.619+0000 7fa46dfad700 1 -- 172.21.15.114:0/1037595591 shutdown_connections 2024-02-16T22:17:18.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:18.619+0000 7fa46dfad700 1 -- 172.21.15.114:0/1037595591 wait complete. 2024-02-16T22:17:19.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.402+0000 7fb0d7d71700 1 -- 172.21.15.114:0/1850563337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0d0105a40 msgr2=0x7fb0d0105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:19.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.402+0000 7fb0d7d71700 1 --2- 172.21.15.114:0/1850563337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0d0105a40 0x7fb0d0105e20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb0c000b0a0 tx=0x7fb0c000b3b0 comp rx=0 tx=0).stop 2024-02-16T22:17:19.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.402+0000 7fb0d7d71700 1 -- 172.21.15.114:0/1850563337 shutdown_connections 2024-02-16T22:17:19.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.402+0000 7fb0d7d71700 1 --2- 172.21.15.114:0/1850563337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0d00ffa40 0x7fb0d00ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:19.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.402+0000 7fb0d7d71700 1 --2- 172.21.15.114:0/1850563337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0d0105a40 0x7fb0d0105e20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:19.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.402+0000 7fb0d7d71700 1 -- 172.21.15.114:0/1850563337 >> 172.21.15.114:0/1850563337 conn(0x7fb0d00fb570 msgr2=0x7fb0d00fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:19.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.402+0000 7fb0d7d71700 1 -- 172.21.15.114:0/1850563337 shutdown_connections 2024-02-16T22:17:19.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.402+0000 7fb0d7d71700 1 -- 172.21.15.114:0/1850563337 wait complete. 2024-02-16T22:17:19.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.403+0000 7fb0d7d71700 1 Processor -- start 2024-02-16T22:17:19.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.403+0000 7fb0d7d71700 1 -- start start 2024-02-16T22:17:19.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.403+0000 7fb0d7d71700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0d00ffa40 0x7fb0d0195670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:19.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.404+0000 7fb0d7d71700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0d0105a40 0x7fb0d0195bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:19.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.404+0000 7fb0d7d71700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d0196290 con 0x7fb0d0105a40 2024-02-16T22:17:19.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.404+0000 7fb0d7d71700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d019a020 con 0x7fb0d00ffa40 2024-02-16T22:17:19.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.404+0000 7fb0d530c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0d0105a40 0x7fb0d0195bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:19.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.404+0000 7fb0d530c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0d0105a40 0x7fb0d0195bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47924/0 (socket says 172.21.15.114:47924) 2024-02-16T22:17:19.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.404+0000 7fb0d530c700 1 -- 172.21.15.114:0/2067101031 learned_addr learned my addr 172.21.15.114:0/2067101031 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:17:19.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.404+0000 7fb0d5b0d700 1 --2- 172.21.15.114:0/2067101031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0d00ffa40 0x7fb0d0195670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:19.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.404+0000 7fb0d530c700 1 -- 172.21.15.114:0/2067101031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0d00ffa40 msgr2=0x7fb0d0195670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:19.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.404+0000 7fb0d530c700 1 --2- 172.21.15.114:0/2067101031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0d00ffa40 0x7fb0d0195670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:19.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.404+0000 7fb0d530c700 1 -- 172.21.15.114:0/2067101031 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c0009d00 con 0x7fb0d0105a40 2024-02-16T22:17:19.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.404+0000 7fb0d530c700 1 --2- 172.21.15.114:0/2067101031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0d0105a40 0x7fb0d0195bb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400d900 tx=0x7fb0c400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:19.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.404+0000 7fb0ceffd700 1 -- 172.21.15.114:0/2067101031 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0c40041f0 con 0x7fb0d0105a40 2024-02-16T22:17:19.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.404+0000 7fb0ceffd700 1 -- 172.21.15.114:0/2067101031 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb0c4004ce0 con 0x7fb0d0105a40 2024-02-16T22:17:19.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.405+0000 7fb0d7d71700 1 -- 172.21.15.114:0/2067101031 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0d019a300 con 0x7fb0d0105a40 2024-02-16T22:17:19.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.405+0000 7fb0ceffd700 1 -- 172.21.15.114:0/2067101031 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0c400b750 con 0x7fb0d0105a40 2024-02-16T22:17:19.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.405+0000 7fb0d7d71700 1 -- 172.21.15.114:0/2067101031 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0d019a850 con 0x7fb0d0105a40 2024-02-16T22:17:19.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.406+0000 7fb0ceffd700 1 -- 172.21.15.114:0/2067101031 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fb0c400b970 con 0x7fb0d0105a40 2024-02-16T22:17:19.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.406+0000 7fb0d7d71700 1 -- 172.21.15.114:0/2067101031 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb0d010cf40 con 0x7fb0d0105a40 2024-02-16T22:17:19.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.407+0000 7fb0ceffd700 1 --2- 172.21.15.114:0/2067101031 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fb0b806c2e0 0x7fb0b806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:19.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.407+0000 7fb0ceffd700 1 -- 172.21.15.114:0/2067101031 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fb0c408ca90 con 0x7fb0d0105a40 2024-02-16T22:17:19.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.407+0000 7fb0d5b0d700 1 --2- 172.21.15.114:0/2067101031 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fb0b806c2e0 0x7fb0b806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:19.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.407+0000 7fb0ceffd700 1 --2- 172.21.15.114:0/2067101031 >> [v2:172.21.15.114:6818/1293302101,v1:172.21.15.114:6819/1293302101] conn(0x7fb0b8071e50 0x7fb0b8074270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:19.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.407+0000 7fb0ceffd700 1 -- 172.21.15.114:0/2067101031 --> [v2:172.21.15.114:6818/1293302101,v1:172.21.15.114:6819/1293302101] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb0b8074920 con 0x7fb0b8071e50 2024-02-16T22:17:19.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.407+0000 7fb0ceffd700 1 -- 172.21.15.114:0/2067101031 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0c408ce30 con 0x7fb0d0105a40 2024-02-16T22:17:19.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.407+0000 7fb0d630e700 1 --2- 172.21.15.114:0/2067101031 >> [v2:172.21.15.114:6818/1293302101,v1:172.21.15.114:6819/1293302101] conn(0x7fb0b8071e50 0x7fb0b8074270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:19.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.407+0000 7fb0d5b0d700 1 --2- 172.21.15.114:0/2067101031 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fb0b806c2e0 0x7fb0b806e7a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00060b0 tx=0x7fb0c0006000 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:19.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.408+0000 7fb0d630e700 1 --2- 172.21.15.114:0/2067101031 >> [v2:172.21.15.114:6818/1293302101,v1:172.21.15.114:6819/1293302101] conn(0x7fb0b8071e50 0x7fb0b8074270 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:19.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.409+0000 7fb0ceffd700 1 -- 172.21.15.114:0/2067101031 <== osd.2 v2:172.21.15.114:6818/1293302101 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb0b8074920 con 0x7fb0b8071e50 2024-02-16T22:17:19.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.424+0000 7fb0d7d71700 1 -- 172.21.15.114:0/2067101031 --> [v2:172.21.15.114:6818/1293302101,v1:172.21.15.114:6819/1293302101] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb0d004ea90 con 0x7fb0b8071e50 2024-02-16T22:17:19.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.424+0000 7fb0ceffd700 1 -- 172.21.15.114:0/2067101031 <== osd.2 v2:172.21.15.114:6818/1293302101 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb0d004ea90 con 0x7fb0b8071e50 2024-02-16T22:17:19.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.424+0000 7fb0d7d71700 1 -- 172.21.15.114:0/2067101031 >> [v2:172.21.15.114:6818/1293302101,v1:172.21.15.114:6819/1293302101] conn(0x7fb0b8071e50 msgr2=0x7fb0b8074270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:19.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.424+0000 7fb0d7d71700 1 --2- 172.21.15.114:0/2067101031 >> [v2:172.21.15.114:6818/1293302101,v1:172.21.15.114:6819/1293302101] conn(0x7fb0b8071e50 0x7fb0b8074270 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:19.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.424+0000 7fb0d7d71700 1 -- 172.21.15.114:0/2067101031 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fb0b806c2e0 msgr2=0x7fb0b806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:19.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.424+0000 7fb0d7d71700 1 --2- 172.21.15.114:0/2067101031 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fb0b806c2e0 0x7fb0b806e7a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00060b0 tx=0x7fb0c0006000 comp rx=0 tx=0).stop 2024-02-16T22:17:19.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.424+0000 7fb0d7d71700 1 -- 172.21.15.114:0/2067101031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0d0105a40 msgr2=0x7fb0d0195bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:19.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.424+0000 7fb0d7d71700 1 --2- 172.21.15.114:0/2067101031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0d0105a40 0x7fb0d0195bb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400d900 tx=0x7fb0c400dcc0 comp rx=0 tx=0).stop 2024-02-16T22:17:19.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.425+0000 7fb0d7d71700 1 -- 172.21.15.114:0/2067101031 shutdown_connections 2024-02-16T22:17:19.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.425+0000 7fb0d7d71700 1 --2- 172.21.15.114:0/2067101031 >> [v2:172.21.15.114:6818/1293302101,v1:172.21.15.114:6819/1293302101] conn(0x7fb0b8071e50 0x7fb0b8074270 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:19.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.425+0000 7fb0d7d71700 1 --2- 172.21.15.114:0/2067101031 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fb0b806c2e0 0x7fb0b806e7a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:19.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.425+0000 7fb0d7d71700 1 --2- 172.21.15.114:0/2067101031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0d00ffa40 0x7fb0d0195670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:19.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.425+0000 7fb0d7d71700 1 --2- 172.21.15.114:0/2067101031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0d0105a40 0x7fb0d0195bb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:19.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.425+0000 7fb0d7d71700 1 -- 172.21.15.114:0/2067101031 >> 172.21.15.114:0/2067101031 conn(0x7fb0d00fb570 msgr2=0x7fb0d00fc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:19.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.425+0000 7fb0d7d71700 1 -- 172.21.15.114:0/2067101031 shutdown_connections 2024-02-16T22:17:19.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:19.425+0000 7fb0d7d71700 1 -- 172.21.15.114:0/2067101031 wait complete. 2024-02-16T22:17:20.135 INFO:teuthology.orchestra.run.smithi114.stdout:81604378649 2024-02-16T22:17:20.135 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-16T22:17:20.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.616+0000 7f53b3f5f700 1 -- 172.21.15.114:0/4126909510 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53ac105a40 msgr2=0x7f53ac105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:20.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.616+0000 7f53b3f5f700 1 --2- 172.21.15.114:0/4126909510 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53ac105a40 0x7f53ac105e20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f53a8009910 tx=0x7f53a8009c20 comp rx=0 tx=0).stop 2024-02-16T22:17:20.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.616+0000 7f53b3f5f700 1 -- 172.21.15.114:0/4126909510 shutdown_connections 2024-02-16T22:17:20.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.616+0000 7f53b3f5f700 1 --2- 172.21.15.114:0/4126909510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53ac0ffbd0 0x7f53ac100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:20.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.616+0000 7f53b3f5f700 1 --2- 172.21.15.114:0/4126909510 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53ac105a40 0x7f53ac105e20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:20.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.616+0000 7f53b3f5f700 1 -- 172.21.15.114:0/4126909510 >> 172.21.15.114:0/4126909510 conn(0x7f53ac0755a0 msgr2=0x7f53ac0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:20.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.617+0000 7f53b3f5f700 1 -- 172.21.15.114:0/4126909510 shutdown_connections 2024-02-16T22:17:20.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.617+0000 7f53b3f5f700 1 -- 172.21.15.114:0/4126909510 wait complete. 2024-02-16T22:17:20.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.617+0000 7f53b3f5f700 1 Processor -- start 2024-02-16T22:17:20.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.617+0000 7f53b3f5f700 1 -- start start 2024-02-16T22:17:20.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.618+0000 7f53b3f5f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53ac0ffbd0 0x7f53ac19c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:20.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.618+0000 7f53b3f5f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53ac105a40 0x7f53ac19c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:20.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.618+0000 7f53b3f5f700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac19d060 con 0x7f53ac105a40 2024-02-16T22:17:20.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.618+0000 7f53b3f5f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac196520 con 0x7f53ac0ffbd0 2024-02-16T22:17:20.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.618+0000 7f53b1cfb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53ac0ffbd0 0x7f53ac19c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:20.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.618+0000 7f53b1cfb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53ac0ffbd0 0x7f53ac19c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.114:34534/0 (socket says 172.21.15.114:34534) 2024-02-16T22:17:20.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.618+0000 7f53b1cfb700 1 -- 172.21.15.114:0/1469006485 learned_addr learned my addr 172.21.15.114:0/1469006485 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:17:20.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.618+0000 7f53b14fa700 1 --2- 172.21.15.114:0/1469006485 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53ac105a40 0x7f53ac19c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:20.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.618+0000 7f53b1cfb700 1 -- 172.21.15.114:0/1469006485 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53ac105a40 msgr2=0x7f53ac19c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:20.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.618+0000 7f53b1cfb700 1 --2- 172.21.15.114:0/1469006485 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53ac105a40 0x7f53ac19c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:20.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.618+0000 7f53b1cfb700 1 -- 172.21.15.114:0/1469006485 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a80095f0 con 0x7f53ac0ffbd0 2024-02-16T22:17:20.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.618+0000 7f53b14fa700 1 --2- 172.21.15.114:0/1469006485 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53ac105a40 0x7f53ac19c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:17:20.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.619+0000 7f53b1cfb700 1 --2- 172.21.15.114:0/1469006485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53ac0ffbd0 0x7f53ac19c440 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f53a800fa00 tx=0x7f53a800fae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:20.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.619+0000 7f53a2ffd700 1 -- 172.21.15.114:0/1469006485 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53a801d070 con 0x7f53ac0ffbd0 2024-02-16T22:17:20.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.619+0000 7f53b3f5f700 1 -- 172.21.15.114:0/1469006485 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53ac1967a0 con 0x7f53ac0ffbd0 2024-02-16T22:17:20.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.619+0000 7f53a2ffd700 1 -- 172.21.15.114:0/1469006485 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f53a8005640 con 0x7f53ac0ffbd0 2024-02-16T22:17:20.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.619+0000 7f53b3f5f700 1 -- 172.21.15.114:0/1469006485 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ac196c90 con 0x7f53ac0ffbd0 2024-02-16T22:17:20.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.619+0000 7f53a2ffd700 1 -- 172.21.15.114:0/1469006485 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53a80226a0 con 0x7f53ac0ffbd0 2024-02-16T22:17:20.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.620+0000 7f53b3f5f700 1 -- 172.21.15.114:0/1469006485 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5390000ff0 con 0x7f53ac0ffbd0 2024-02-16T22:17:20.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.621+0000 7f53a2ffd700 1 -- 172.21.15.114:0/1469006485 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f53a8003680 con 0x7f53ac0ffbd0 2024-02-16T22:17:20.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.621+0000 7f53a2ffd700 1 --2- 172.21.15.114:0/1469006485 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f539806c4e0 0x7f539806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:20.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.621+0000 7f53a2ffd700 1 -- 172.21.15.114:0/1469006485 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f53a808c850 con 0x7f53ac0ffbd0 2024-02-16T22:17:20.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.621+0000 7f53a2ffd700 1 --2- 172.21.15.114:0/1469006485 >> [v2:172.21.15.123:6816/2122343545,v1:172.21.15.123:6817/2122343545] conn(0x7f53980721c0 0x7f53980745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:20.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.622+0000 7f53a2ffd700 1 -- 172.21.15.114:0/1469006485 --> [v2:172.21.15.123:6816/2122343545,v1:172.21.15.123:6817/2122343545] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5398074c90 con 0x7f53980721c0 2024-02-16T22:17:20.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.622+0000 7f53a2ffd700 1 -- 172.21.15.114:0/1469006485 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53a80912a0 con 0x7f53ac0ffbd0 2024-02-16T22:17:20.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.622+0000 7f53b14fa700 1 --2- 172.21.15.114:0/1469006485 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f539806c4e0 0x7f539806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:20.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.622+0000 7f53b24fc700 1 --2- 172.21.15.114:0/1469006485 >> [v2:172.21.15.123:6816/2122343545,v1:172.21.15.123:6817/2122343545] conn(0x7f53980721c0 0x7f53980745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:20.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.622+0000 7f53b14fa700 1 --2- 172.21.15.114:0/1469006485 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f539806c4e0 0x7f539806e9a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f53ac197d10 tx=0x7f539c009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:20.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.623+0000 7f53b24fc700 1 --2- 172.21.15.114:0/1469006485 >> [v2:172.21.15.123:6816/2122343545,v1:172.21.15.123:6817/2122343545] conn(0x7f53980721c0 0x7f53980745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:20.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.623+0000 7f53a2ffd700 1 -- 172.21.15.114:0/1469006485 <== osd.5 v2:172.21.15.123:6816/2122343545 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5398074c90 con 0x7f53980721c0 2024-02-16T22:17:20.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.646+0000 7f53b3f5f700 1 -- 172.21.15.114:0/1469006485 --> [v2:172.21.15.123:6816/2122343545,v1:172.21.15.123:6817/2122343545] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5390002ce0 con 0x7f53980721c0 2024-02-16T22:17:20.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.646+0000 7f53a2ffd700 1 -- 172.21.15.114:0/1469006485 <== osd.5 v2:172.21.15.123:6816/2122343545 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5390002ce0 con 0x7f53980721c0 2024-02-16T22:17:20.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.647+0000 7f53b3f5f700 1 -- 172.21.15.114:0/1469006485 >> [v2:172.21.15.123:6816/2122343545,v1:172.21.15.123:6817/2122343545] conn(0x7f53980721c0 msgr2=0x7f53980745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:20.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.647+0000 7f53b3f5f700 1 --2- 172.21.15.114:0/1469006485 >> [v2:172.21.15.123:6816/2122343545,v1:172.21.15.123:6817/2122343545] conn(0x7f53980721c0 0x7f53980745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:20.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.647+0000 7f53b3f5f700 1 -- 172.21.15.114:0/1469006485 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f539806c4e0 msgr2=0x7f539806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:20.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.647+0000 7f53b3f5f700 1 --2- 172.21.15.114:0/1469006485 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f539806c4e0 0x7f539806e9a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f53ac197d10 tx=0x7f539c009260 comp rx=0 tx=0).stop 2024-02-16T22:17:20.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.647+0000 7f53b3f5f700 1 -- 172.21.15.114:0/1469006485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53ac0ffbd0 msgr2=0x7f53ac19c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:20.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.647+0000 7f53b3f5f700 1 --2- 172.21.15.114:0/1469006485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53ac0ffbd0 0x7f53ac19c440 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f53a800fa00 tx=0x7f53a800fae0 comp rx=0 tx=0).stop 2024-02-16T22:17:20.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.647+0000 7f53b3f5f700 1 -- 172.21.15.114:0/1469006485 shutdown_connections 2024-02-16T22:17:20.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.647+0000 7f53b3f5f700 1 --2- 172.21.15.114:0/1469006485 >> [v2:172.21.15.123:6816/2122343545,v1:172.21.15.123:6817/2122343545] conn(0x7f53980721c0 0x7f53980745e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:20.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.647+0000 7f53b3f5f700 1 --2- 172.21.15.114:0/1469006485 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f539806c4e0 0x7f539806e9a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:20.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.647+0000 7f53b3f5f700 1 --2- 172.21.15.114:0/1469006485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53ac0ffbd0 0x7f53ac19c440 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:20.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.647+0000 7f53b3f5f700 1 --2- 172.21.15.114:0/1469006485 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53ac105a40 0x7f53ac19c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:20.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.647+0000 7f53b3f5f700 1 -- 172.21.15.114:0/1469006485 >> 172.21.15.114:0/1469006485 conn(0x7f53ac0755a0 msgr2=0x7f53ac0685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:20.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.647+0000 7f53b3f5f700 1 -- 172.21.15.114:0/1469006485 shutdown_connections 2024-02-16T22:17:20.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:20.647+0000 7f53b3f5f700 1 -- 172.21.15.114:0/1469006485 wait complete. 2024-02-16T22:17:20.763 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:20 smithi114 ceph-mon[29235]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:20.769 INFO:teuthology.orchestra.run.smithi114.stdout:111669149715 2024-02-16T22:17:20.769 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-16T22:17:20.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:20 smithi123 ceph-mon[37161]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:20.989 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:17:21.384 INFO:teuthology.orchestra.run.smithi114.stdout:167503724553 2024-02-16T22:17:21.384 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-16T22:17:21.416 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:17:22.700 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:17:22.700 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:17:22.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:22 smithi114 ceph-mon[29235]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:22.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:22 smithi123 ceph-mon[37161]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:24.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:24 smithi114 ceph-mon[29235]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:24.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:24 smithi123 ceph-mon[37161]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:25.527 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:17:25.548 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:17:25.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.754+0000 7f0d630d8700 1 -- 172.21.15.114:0/1549784966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d5c105a40 msgr2=0x7f0d5c105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:25.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.754+0000 7f0d630d8700 1 --2- 172.21.15.114:0/1549784966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d5c105a40 0x7f0d5c105e20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009910 tx=0x7f0d50009c20 comp rx=0 tx=0).stop 2024-02-16T22:17:25.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.754+0000 7f0d630d8700 1 -- 172.21.15.114:0/1549784966 shutdown_connections 2024-02-16T22:17:25.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.754+0000 7f0d630d8700 1 --2- 172.21.15.114:0/1549784966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d5c0ffbd0 0x7f0d5c100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:25.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.754+0000 7f0d630d8700 1 --2- 172.21.15.114:0/1549784966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d5c105a40 0x7f0d5c105e20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:25.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.754+0000 7f0d630d8700 1 -- 172.21.15.114:0/1549784966 >> 172.21.15.114:0/1549784966 conn(0x7f0d5c0755a0 msgr2=0x7f0d5c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:25.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.754+0000 7f0d630d8700 1 -- 172.21.15.114:0/1549784966 shutdown_connections 2024-02-16T22:17:25.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.754+0000 7f0d630d8700 1 -- 172.21.15.114:0/1549784966 wait complete. 2024-02-16T22:17:25.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.755+0000 7f0d630d8700 1 Processor -- start 2024-02-16T22:17:25.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.755+0000 7f0d630d8700 1 -- start start 2024-02-16T22:17:25.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.755+0000 7f0d630d8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d5c0ffbd0 0x7f0d5c1976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:25.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.755+0000 7f0d630d8700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d5c105a40 0x7f0d5c197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:25.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.755+0000 7f0d630d8700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c198300 con 0x7f0d5c105a40 2024-02-16T22:17:25.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.755+0000 7f0d630d8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c198470 con 0x7f0d5c0ffbd0 2024-02-16T22:17:25.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.755+0000 7f0d5bfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d5c105a40 0x7f0d5c197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:25.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.755+0000 7f0d5bfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d5c105a40 0x7f0d5c197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40406/0 (socket says 172.21.15.114:40406) 2024-02-16T22:17:25.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.755+0000 7f0d5bfff700 1 -- 172.21.15.114:0/2171955078 learned_addr learned my addr 172.21.15.114:0/2171955078 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:17:25.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.755+0000 7f0d60e74700 1 --2- 172.21.15.114:0/2171955078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d5c0ffbd0 0x7f0d5c1976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:25.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.755+0000 7f0d5bfff700 1 -- 172.21.15.114:0/2171955078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d5c0ffbd0 msgr2=0x7f0d5c1976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:25.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.755+0000 7f0d5bfff700 1 --2- 172.21.15.114:0/2171955078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d5c0ffbd0 0x7f0d5c1976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:25.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.755+0000 7f0d5bfff700 1 -- 172.21.15.114:0/2171955078 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d500095f0 con 0x7f0d5c105a40 2024-02-16T22:17:25.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.755+0000 7f0d5bfff700 1 --2- 172.21.15.114:0/2171955078 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d5c105a40 0x7f0d5c197c20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800d900 tx=0x7f0d4800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:25.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.756+0000 7f0d59ffb700 1 -- 172.21.15.114:0/2171955078 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d48004a00 con 0x7f0d5c105a40 2024-02-16T22:17:25.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.756+0000 7f0d59ffb700 1 -- 172.21.15.114:0/2171955078 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0d480054d0 con 0x7f0d5c105a40 2024-02-16T22:17:25.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.756+0000 7f0d630d8700 1 -- 172.21.15.114:0/2171955078 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d5c1a0750 con 0x7f0d5c105a40 2024-02-16T22:17:25.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.756+0000 7f0d59ffb700 1 -- 172.21.15.114:0/2171955078 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d48009d00 con 0x7f0d5c105a40 2024-02-16T22:17:25.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.756+0000 7f0d630d8700 1 -- 172.21.15.114:0/2171955078 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d5c1a0ca0 con 0x7f0d5c105a40 2024-02-16T22:17:25.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.757+0000 7f0d630d8700 1 -- 172.21.15.114:0/2171955078 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d5c108040 con 0x7f0d5c105a40 2024-02-16T22:17:25.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.757+0000 7f0d59ffb700 1 -- 172.21.15.114:0/2171955078 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f0d48005020 con 0x7f0d5c105a40 2024-02-16T22:17:25.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.757+0000 7f0d59ffb700 1 --2- 172.21.15.114:0/2171955078 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f0d4c06c4e0 0x7f0d4c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:25.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.758+0000 7f0d59ffb700 1 -- 172.21.15.114:0/2171955078 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f0d4808b440 con 0x7f0d5c105a40 2024-02-16T22:17:25.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.758+0000 7f0d60e74700 1 --2- 172.21.15.114:0/2171955078 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f0d4c06c4e0 0x7f0d4c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:25.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.758+0000 7f0d60e74700 1 --2- 172.21.15.114:0/2171955078 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f0d4c06c4e0 0x7f0d4c06e9a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0d5000b5c0 tx=0x7f0d5001a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:25.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.762+0000 7f0d59ffb700 1 -- 172.21.15.114:0/2171955078 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0d48055f60 con 0x7f0d5c105a40 2024-02-16T22:17:25.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.957+0000 7f0d630d8700 1 -- 172.21.15.114:0/2171955078 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f0d5c068af0 con 0x7f0d5c105a40 2024-02-16T22:17:25.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.958+0000 7f0d59ffb700 1 -- 172.21.15.114:0/2171955078 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0d48009e60 con 0x7f0d5c105a40 2024-02-16T22:17:25.960 INFO:teuthology.orchestra.run.smithi114.stdout:146028888079 2024-02-16T22:17:25.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.960+0000 7f0d630d8700 1 -- 172.21.15.114:0/2171955078 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f0d4c06c4e0 msgr2=0x7f0d4c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:25.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.960+0000 7f0d630d8700 1 --2- 172.21.15.114:0/2171955078 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f0d4c06c4e0 0x7f0d4c06e9a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0d5000b5c0 tx=0x7f0d5001a040 comp rx=0 tx=0).stop 2024-02-16T22:17:25.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.960+0000 7f0d630d8700 1 -- 172.21.15.114:0/2171955078 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d5c105a40 msgr2=0x7f0d5c197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:25.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.960+0000 7f0d630d8700 1 --2- 172.21.15.114:0/2171955078 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d5c105a40 0x7f0d5c197c20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800d900 tx=0x7f0d4800dc10 comp rx=0 tx=0).stop 2024-02-16T22:17:25.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.960+0000 7f0d630d8700 1 -- 172.21.15.114:0/2171955078 shutdown_connections 2024-02-16T22:17:25.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.960+0000 7f0d630d8700 1 --2- 172.21.15.114:0/2171955078 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f0d4c06c4e0 0x7f0d4c06e9a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:25.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.960+0000 7f0d630d8700 1 --2- 172.21.15.114:0/2171955078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d5c0ffbd0 0x7f0d5c1976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:25.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.960+0000 7f0d630d8700 1 --2- 172.21.15.114:0/2171955078 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d5c105a40 0x7f0d5c197c20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:25.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.960+0000 7f0d630d8700 1 -- 172.21.15.114:0/2171955078 >> 172.21.15.114:0/2171955078 conn(0x7f0d5c0755a0 msgr2=0x7f0d5c0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:25.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.960+0000 7f0d630d8700 1 -- 172.21.15.114:0/2171955078 shutdown_connections 2024-02-16T22:17:25.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:25.960+0000 7f0d630d8700 1 -- 172.21.15.114:0/2171955078 wait complete. 2024-02-16T22:17:26.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.792+0000 7f00b9788700 1 -- 172.21.15.114:0/1870338603 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00b40ffa40 msgr2=0x7f00b40ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:26.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.792+0000 7f00b9788700 1 --2- 172.21.15.114:0/1870338603 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00b40ffa40 0x7f00b40ffec0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f00a8009910 tx=0x7f00a8009c20 comp rx=0 tx=0).stop 2024-02-16T22:17:26.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.792+0000 7f00b9788700 1 -- 172.21.15.114:0/1870338603 shutdown_connections 2024-02-16T22:17:26.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.792+0000 7f00b9788700 1 --2- 172.21.15.114:0/1870338603 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00b40ffa40 0x7f00b40ffec0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:26.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.792+0000 7f00b9788700 1 --2- 172.21.15.114:0/1870338603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00b4105a40 0x7f00b4105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:26.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.792+0000 7f00b9788700 1 -- 172.21.15.114:0/1870338603 >> 172.21.15.114:0/1870338603 conn(0x7f00b40fb570 msgr2=0x7f00b40fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:26.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.792+0000 7f00b9788700 1 -- 172.21.15.114:0/1870338603 shutdown_connections 2024-02-16T22:17:26.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.792+0000 7f00b9788700 1 -- 172.21.15.114:0/1870338603 wait complete. 2024-02-16T22:17:26.795 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:26 smithi114 ceph-mon[29235]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:26.795 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:26 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/2171955078' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-16T22:17:26.795 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:17:26.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.793+0000 7f00b9788700 1 Processor -- start 2024-02-16T22:17:26.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.793+0000 7f00b9788700 1 -- start start 2024-02-16T22:17:26.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.793+0000 7f00b9788700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00b40ffa40 0x7f00b41975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:26.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.793+0000 7f00b9788700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00b4105a40 0x7f00b4197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:26.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.793+0000 7f00b9788700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b4198120 con 0x7f00b4105a40 2024-02-16T22:17:26.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.793+0000 7f00b9788700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b4198260 con 0x7f00b40ffa40 2024-02-16T22:17:26.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.793+0000 7f00b27fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00b4105a40 0x7f00b4197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:26.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.793+0000 7f00b27fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00b4105a40 0x7f00b4197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40434/0 (socket says 172.21.15.114:40434) 2024-02-16T22:17:26.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.793+0000 7f00b27fc700 1 -- 172.21.15.114:0/429478289 learned_addr learned my addr 172.21.15.114:0/429478289 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:17:26.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.793+0000 7f00b2ffd700 1 --2- 172.21.15.114:0/429478289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00b40ffa40 0x7f00b41975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:26.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.794+0000 7f00b27fc700 1 -- 172.21.15.114:0/429478289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00b40ffa40 msgr2=0x7f00b41975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:26.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.794+0000 7f00b27fc700 1 --2- 172.21.15.114:0/429478289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00b40ffa40 0x7f00b41975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:26.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.794+0000 7f00b27fc700 1 -- 172.21.15.114:0/429478289 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a80095f0 con 0x7f00b4105a40 2024-02-16T22:17:26.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.794+0000 7f00b2ffd700 1 --2- 172.21.15.114:0/429478289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00b40ffa40 0x7f00b41975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T22:17:26.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.794+0000 7f00b27fc700 1 --2- 172.21.15.114:0/429478289 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00b4105a40 0x7f00b4197b00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f00a8006010 tx=0x7f00a800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:26.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.794+0000 7f009bfff700 1 -- 172.21.15.114:0/429478289 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00a801d070 con 0x7f00b4105a40 2024-02-16T22:17:26.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.794+0000 7f009bfff700 1 -- 172.21.15.114:0/429478289 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f00a8004f50 con 0x7f00b4105a40 2024-02-16T22:17:26.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.794+0000 7f00b9788700 1 -- 172.21.15.114:0/429478289 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00b41a0630 con 0x7f00b4105a40 2024-02-16T22:17:26.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.794+0000 7f009bfff700 1 -- 172.21.15.114:0/429478289 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00a8021640 con 0x7f00b4105a40 2024-02-16T22:17:26.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.794+0000 7f00b9788700 1 -- 172.21.15.114:0/429478289 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b41a0a40 con 0x7f00b4105a40 2024-02-16T22:17:26.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.795+0000 7f00b9788700 1 -- 172.21.15.114:0/429478289 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00b4107fd0 con 0x7f00b4105a40 2024-02-16T22:17:26.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.795+0000 7f009bfff700 1 -- 172.21.15.114:0/429478289 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f00a8004aa0 con 0x7f00b4105a40 2024-02-16T22:17:26.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.796+0000 7f009bfff700 1 --2- 172.21.15.114:0/429478289 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f009c06c4e0 0x7f009c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:26.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.796+0000 7f009bfff700 1 -- 172.21.15.114:0/429478289 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f00a808c660 con 0x7f00b4105a40 2024-02-16T22:17:26.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.796+0000 7f00b2ffd700 1 --2- 172.21.15.114:0/429478289 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f009c06c4e0 0x7f009c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:26.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.796+0000 7f00b2ffd700 1 --2- 172.21.15.114:0/429478289 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f009c06c4e0 0x7f009c06e9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0094f0 tx=0x7f00ac009040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:26.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:26 smithi123 ceph-mon[37161]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:26.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:26 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/2171955078' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-16T22:17:26.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:26 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:17:26.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.800+0000 7f009bfff700 1 -- 172.21.15.114:0/429478289 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f00a8057320 con 0x7f00b4105a40 2024-02-16T22:17:27.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.999+0000 7f00b9788700 1 -- 172.21.15.114:0/429478289 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f00b404ea90 con 0x7f00b4105a40 2024-02-16T22:17:27.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:26.999+0000 7f009bfff700 1 -- 172.21.15.114:0/429478289 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f00a800fe60 con 0x7f00b4105a40 2024-02-16T22:17:27.002 INFO:teuthology.orchestra.run.smithi114.stdout:81604378650 2024-02-16T22:17:27.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:27.001+0000 7f00b9788700 1 -- 172.21.15.114:0/429478289 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f009c06c4e0 msgr2=0x7f009c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:27.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:27.001+0000 7f00b9788700 1 --2- 172.21.15.114:0/429478289 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f009c06c4e0 0x7f009c06e9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0094f0 tx=0x7f00ac009040 comp rx=0 tx=0).stop 2024-02-16T22:17:27.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:27.001+0000 7f00b9788700 1 -- 172.21.15.114:0/429478289 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00b4105a40 msgr2=0x7f00b4197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:27.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:27.001+0000 7f00b9788700 1 --2- 172.21.15.114:0/429478289 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00b4105a40 0x7f00b4197b00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f00a8006010 tx=0x7f00a800be20 comp rx=0 tx=0).stop 2024-02-16T22:17:27.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:27.001+0000 7f00b9788700 1 -- 172.21.15.114:0/429478289 shutdown_connections 2024-02-16T22:17:27.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:27.001+0000 7f00b9788700 1 --2- 172.21.15.114:0/429478289 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f009c06c4e0 0x7f009c06e9a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:27.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:27.001+0000 7f00b9788700 1 --2- 172.21.15.114:0/429478289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00b40ffa40 0x7f00b41975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:27.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:27.002+0000 7f00b9788700 1 --2- 172.21.15.114:0/429478289 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00b4105a40 0x7f00b4197b00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:27.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:27.002+0000 7f00b9788700 1 -- 172.21.15.114:0/429478289 >> 172.21.15.114:0/429478289 conn(0x7f00b40fb570 msgr2=0x7f00b40fc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:27.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:27.002+0000 7f00b9788700 1 -- 172.21.15.114:0/429478289 shutdown_connections 2024-02-16T22:17:27.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:27.002+0000 7f00b9788700 1 -- 172.21.15.114:0/429478289 wait complete. 2024-02-16T22:17:28.129 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:27 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/429478289' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-16T22:17:28.129 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:27 smithi114 ceph-mon[29235]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:28.149 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378649 got 81604378650 for osd.2 2024-02-16T22:17:28.149 DEBUG:teuthology.parallel:result is None 2024-02-16T22:17:28.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:27 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/429478289' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-16T22:17:28.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:27 smithi123 ceph-mon[37161]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:28.688 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888077 got 146028888079 for osd.4 2024-02-16T22:17:28.688 DEBUG:teuthology.parallel:result is None 2024-02-16T22:17:30.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.312+0000 7f3381e66700 1 -- 172.21.15.114:0/4241247005 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f337c0ffbd0 msgr2=0x7f337c100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:30.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.312+0000 7f3381e66700 1 --2- 172.21.15.114:0/4241247005 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f337c0ffbd0 0x7f337c100050 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3370009910 tx=0x7f3370009c20 comp rx=0 tx=0).stop 2024-02-16T22:17:30.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.312+0000 7f3381e66700 1 -- 172.21.15.114:0/4241247005 shutdown_connections 2024-02-16T22:17:30.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.312+0000 7f3381e66700 1 --2- 172.21.15.114:0/4241247005 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f337c0ffbd0 0x7f337c100050 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:30.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.312+0000 7f3381e66700 1 --2- 172.21.15.114:0/4241247005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f337c105a40 0x7f337c105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:30.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.312+0000 7f3381e66700 1 -- 172.21.15.114:0/4241247005 >> 172.21.15.114:0/4241247005 conn(0x7f337c0755a0 msgr2=0x7f337c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:30.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.312+0000 7f3381e66700 1 -- 172.21.15.114:0/4241247005 shutdown_connections 2024-02-16T22:17:30.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.312+0000 7f3381e66700 1 -- 172.21.15.114:0/4241247005 wait complete. 2024-02-16T22:17:30.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.312+0000 7f3381e66700 1 Processor -- start 2024-02-16T22:17:30.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.313+0000 7f3381e66700 1 -- start start 2024-02-16T22:17:30.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.313+0000 7f3381e66700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f337c0ffbd0 0x7f337c195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:30.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.313+0000 7f3381e66700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f337c105a40 0x7f337c195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:30.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.313+0000 7f3381e66700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337c196240 con 0x7f337c105a40 2024-02-16T22:17:30.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.313+0000 7f3381e66700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337c199fd0 con 0x7f337c0ffbd0 2024-02-16T22:17:30.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.313+0000 7f337affd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f337c105a40 0x7f337c195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:30.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.313+0000 7f337affd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f337c105a40 0x7f337c195b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40478/0 (socket says 172.21.15.114:40478) 2024-02-16T22:17:30.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.313+0000 7f337affd700 1 -- 172.21.15.114:0/1357337539 learned_addr learned my addr 172.21.15.114:0/1357337539 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:17:30.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.313+0000 7f337b7fe700 1 --2- 172.21.15.114:0/1357337539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f337c0ffbd0 0x7f337c195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:30.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.313+0000 7f337affd700 1 -- 172.21.15.114:0/1357337539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f337c0ffbd0 msgr2=0x7f337c195620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:30.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.313+0000 7f337affd700 1 --2- 172.21.15.114:0/1357337539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f337c0ffbd0 0x7f337c195620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:30.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.313+0000 7f337affd700 1 -- 172.21.15.114:0/1357337539 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33700095f0 con 0x7f337c105a40 2024-02-16T22:17:30.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.313+0000 7f337affd700 1 --2- 172.21.15.114:0/1357337539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f337c105a40 0x7f337c195b60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3370006010 tx=0x7f337000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:30.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.314+0000 7f3378ff9700 1 -- 172.21.15.114:0/1357337539 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f337001d070 con 0x7f337c105a40 2024-02-16T22:17:30.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.314+0000 7f3381e66700 1 -- 172.21.15.114:0/1357337539 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f337c19a250 con 0x7f337c105a40 2024-02-16T22:17:30.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.314+0000 7f3378ff9700 1 -- 172.21.15.114:0/1357337539 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3370004f50 con 0x7f337c105a40 2024-02-16T22:17:30.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.314+0000 7f3378ff9700 1 -- 172.21.15.114:0/1357337539 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3370021640 con 0x7f337c105a40 2024-02-16T22:17:30.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.314+0000 7f3381e66700 1 -- 172.21.15.114:0/1357337539 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f337c19a740 con 0x7f337c105a40 2024-02-16T22:17:30.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.315+0000 7f3381e66700 1 -- 172.21.15.114:0/1357337539 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f337c107fd0 con 0x7f337c105a40 2024-02-16T22:17:30.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.315+0000 7f3378ff9700 1 -- 172.21.15.114:0/1357337539 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f337002b440 con 0x7f337c105a40 2024-02-16T22:17:30.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.315+0000 7f3378ff9700 1 --2- 172.21.15.114:0/1357337539 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f336406c4e0 0x7f336406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:30.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.316+0000 7f3378ff9700 1 -- 172.21.15.114:0/1357337539 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f337008c6e0 con 0x7f337c105a40 2024-02-16T22:17:30.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.316+0000 7f337b7fe700 1 --2- 172.21.15.114:0/1357337539 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f336406c4e0 0x7f336406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:30.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.316+0000 7f337b7fe700 1 --2- 172.21.15.114:0/1357337539 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f336406c4e0 0x7f336406e9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f336c005950 tx=0x7f336c0058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:30.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.320+0000 7f3378ff9700 1 -- 172.21.15.114:0/1357337539 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3370057200 con 0x7f337c105a40 2024-02-16T22:17:30.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.519+0000 7f3381e66700 1 -- 172.21.15.114:0/1357337539 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f337c19aae0 con 0x7f337c105a40 2024-02-16T22:17:30.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.520+0000 7f3378ff9700 1 -- 172.21.15.114:0/1357337539 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f337005a820 con 0x7f337c105a40 2024-02-16T22:17:30.522 INFO:teuthology.orchestra.run.smithi114.stdout:38654705702 2024-02-16T22:17:30.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.522+0000 7f3381e66700 1 -- 172.21.15.114:0/1357337539 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f336406c4e0 msgr2=0x7f336406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:30.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.522+0000 7f3381e66700 1 --2- 172.21.15.114:0/1357337539 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f336406c4e0 0x7f336406e9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f336c005950 tx=0x7f336c0058e0 comp rx=0 tx=0).stop 2024-02-16T22:17:30.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.522+0000 7f3381e66700 1 -- 172.21.15.114:0/1357337539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f337c105a40 msgr2=0x7f337c195b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:30.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.522+0000 7f3381e66700 1 --2- 172.21.15.114:0/1357337539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f337c105a40 0x7f337c195b60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3370006010 tx=0x7f337000be20 comp rx=0 tx=0).stop 2024-02-16T22:17:30.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.522+0000 7f3381e66700 1 -- 172.21.15.114:0/1357337539 shutdown_connections 2024-02-16T22:17:30.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.522+0000 7f3381e66700 1 --2- 172.21.15.114:0/1357337539 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f336406c4e0 0x7f336406e9a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:30.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.522+0000 7f3381e66700 1 --2- 172.21.15.114:0/1357337539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f337c0ffbd0 0x7f337c195620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:30.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.522+0000 7f3381e66700 1 --2- 172.21.15.114:0/1357337539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f337c105a40 0x7f337c195b60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:30.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.522+0000 7f3381e66700 1 -- 172.21.15.114:0/1357337539 >> 172.21.15.114:0/1357337539 conn(0x7f337c0755a0 msgr2=0x7f337c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:30.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.522+0000 7f3381e66700 1 -- 172.21.15.114:0/1357337539 shutdown_connections 2024-02-16T22:17:30.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.522+0000 7f3381e66700 1 -- 172.21.15.114:0/1357337539 wait complete. 2024-02-16T22:17:30.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.550+0000 7f2f26461700 1 -- 172.21.15.114:0/1419576275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f20105a40 msgr2=0x7f2f20105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:30.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.550+0000 7f2f26461700 1 --2- 172.21.15.114:0/1419576275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f20105a40 0x7f2f20105e20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009910 tx=0x7f2f10009c20 comp rx=0 tx=0).stop 2024-02-16T22:17:30.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.550+0000 7f2f26461700 1 -- 172.21.15.114:0/1419576275 shutdown_connections 2024-02-16T22:17:30.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.550+0000 7f2f26461700 1 --2- 172.21.15.114:0/1419576275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f200ffbd0 0x7f2f20100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:30.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.550+0000 7f2f26461700 1 --2- 172.21.15.114:0/1419576275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f20105a40 0x7f2f20105e20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:30.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.550+0000 7f2f26461700 1 -- 172.21.15.114:0/1419576275 >> 172.21.15.114:0/1419576275 conn(0x7f2f200755a0 msgr2=0x7f2f200759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:30.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.550+0000 7f2f26461700 1 -- 172.21.15.114:0/1419576275 shutdown_connections 2024-02-16T22:17:30.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.550+0000 7f2f26461700 1 -- 172.21.15.114:0/1419576275 wait complete. 2024-02-16T22:17:30.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.551+0000 7f2f26461700 1 Processor -- start 2024-02-16T22:17:30.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.551+0000 7f2f26461700 1 -- start start 2024-02-16T22:17:30.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.552+0000 7f2f26461700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f200ffbd0 0x7f2f20196100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:30.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.552+0000 7f2f26461700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f20105a40 0x7f2f20194340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:30.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.552+0000 7f2f1ffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f200ffbd0 0x7f2f20196100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:30.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.552+0000 7f2f26461700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f20194a20 con 0x7f2f200ffbd0 2024-02-16T22:17:30.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.552+0000 7f2f1ffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f200ffbd0 0x7f2f20196100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40490/0 (socket says 172.21.15.114:40490) 2024-02-16T22:17:30.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.552+0000 7f2f1ffff700 1 -- 172.21.15.114:0/2353098492 learned_addr learned my addr 172.21.15.114:0/2353098492 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:17:30.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.552+0000 7f2f26461700 1 -- 172.21.15.114:0/2353098492 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f20194b90 con 0x7f2f20105a40 2024-02-16T22:17:30.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.552+0000 7f2f1f7fe700 1 --2- 172.21.15.114:0/2353098492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f20105a40 0x7f2f20194340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:30.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.552+0000 7f2f1ffff700 1 -- 172.21.15.114:0/2353098492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f20105a40 msgr2=0x7f2f20194340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:30.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.552+0000 7f2f1ffff700 1 --2- 172.21.15.114:0/2353098492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f20105a40 0x7f2f20194340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:30.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.552+0000 7f2f1ffff700 1 -- 172.21.15.114:0/2353098492 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f100095f0 con 0x7f2f200ffbd0 2024-02-16T22:17:30.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.552+0000 7f2f1ffff700 1 --2- 172.21.15.114:0/2353098492 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f200ffbd0 0x7f2f20196100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2f10006010 tx=0x7f2f100048c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:30.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.553+0000 7f2f1d7fa700 1 -- 172.21.15.114:0/2353098492 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f1001d070 con 0x7f2f200ffbd0 2024-02-16T22:17:30.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.553+0000 7f2f26461700 1 -- 172.21.15.114:0/2353098492 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f20194e10 con 0x7f2f200ffbd0 2024-02-16T22:17:30.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.553+0000 7f2f1d7fa700 1 -- 172.21.15.114:0/2353098492 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2f10005dc0 con 0x7f2f200ffbd0 2024-02-16T22:17:30.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.553+0000 7f2f1d7fa700 1 -- 172.21.15.114:0/2353098492 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f1000f690 con 0x7f2f200ffbd0 2024-02-16T22:17:30.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.553+0000 7f2f26461700 1 -- 172.21.15.114:0/2353098492 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f201a6f30 con 0x7f2f200ffbd0 2024-02-16T22:17:30.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.554+0000 7f2f26461700 1 -- 172.21.15.114:0/2353098492 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f2004ea90 con 0x7f2f200ffbd0 2024-02-16T22:17:30.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.554+0000 7f2f1d7fa700 1 -- 172.21.15.114:0/2353098492 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f2f10022470 con 0x7f2f200ffbd0 2024-02-16T22:17:30.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.555+0000 7f2f1d7fa700 1 --2- 172.21.15.114:0/2353098492 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f2f0806c4e0 0x7f2f0806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:30.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.555+0000 7f2f1d7fa700 1 -- 172.21.15.114:0/2353098492 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f2f1008cbf0 con 0x7f2f200ffbd0 2024-02-16T22:17:30.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.555+0000 7f2f1f7fe700 1 --2- 172.21.15.114:0/2353098492 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f2f0806c4e0 0x7f2f0806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:30.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.555+0000 7f2f1f7fe700 1 --2- 172.21.15.114:0/2353098492 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f2f0806c4e0 0x7f2f0806e9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2f20195920 tx=0x7f2f1400b410 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:30.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.559+0000 7f2f1d7fa700 1 -- 172.21.15.114:0/2353098492 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2f10057960 con 0x7f2f200ffbd0 2024-02-16T22:17:30.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.759+0000 7f2f26461700 1 -- 172.21.15.114:0/2353098492 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2f201a71c0 con 0x7f2f200ffbd0 2024-02-16T22:17:30.761 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:30 smithi114 ceph-mon[29235]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:30.761 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:30 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/516562533' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-16T22:17:30.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.759+0000 7f2f1d7fa700 1 -- 172.21.15.114:0/2353098492 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2f10027020 con 0x7f2f200ffbd0 2024-02-16T22:17:30.762 INFO:teuthology.orchestra.run.smithi114.stdout:60129542176 2024-02-16T22:17:30.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.761+0000 7f2f26461700 1 -- 172.21.15.114:0/2353098492 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f2f0806c4e0 msgr2=0x7f2f0806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:30.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.761+0000 7f2f26461700 1 --2- 172.21.15.114:0/2353098492 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f2f0806c4e0 0x7f2f0806e9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2f20195920 tx=0x7f2f1400b410 comp rx=0 tx=0).stop 2024-02-16T22:17:30.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.761+0000 7f2f26461700 1 -- 172.21.15.114:0/2353098492 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f200ffbd0 msgr2=0x7f2f20196100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:30.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.761+0000 7f2f26461700 1 --2- 172.21.15.114:0/2353098492 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f200ffbd0 0x7f2f20196100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2f10006010 tx=0x7f2f100048c0 comp rx=0 tx=0).stop 2024-02-16T22:17:30.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.761+0000 7f2f26461700 1 -- 172.21.15.114:0/2353098492 shutdown_connections 2024-02-16T22:17:30.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.761+0000 7f2f26461700 1 --2- 172.21.15.114:0/2353098492 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f200ffbd0 0x7f2f20196100 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:30.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.761+0000 7f2f26461700 1 --2- 172.21.15.114:0/2353098492 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f2f0806c4e0 0x7f2f0806e9a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:30.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.761+0000 7f2f26461700 1 --2- 172.21.15.114:0/2353098492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f20105a40 0x7f2f20194340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:30.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.761+0000 7f2f26461700 1 -- 172.21.15.114:0/2353098492 >> 172.21.15.114:0/2353098492 conn(0x7f2f200755a0 msgr2=0x7f2f20068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:30.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.761+0000 7f2f26461700 1 -- 172.21.15.114:0/2353098492 shutdown_connections 2024-02-16T22:17:30.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.762+0000 7f2f26461700 1 -- 172.21.15.114:0/2353098492 wait complete. 2024-02-16T22:17:30.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:30 smithi123 ceph-mon[37161]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:30.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:30 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/516562533' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-16T22:17:30.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.212+0000 7fe72b88a700 1 -- 172.21.15.114:0/1549211243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe724100de0 msgr2=0x7fe7241011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:30.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.212+0000 7fe72b88a700 1 --2- 172.21.15.114:0/1549211243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe724100de0 0x7fe7241011c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe720009910 tx=0x7fe720009c20 comp rx=0 tx=0).stop 2024-02-16T22:17:30.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.212+0000 7fe72b88a700 1 -- 172.21.15.114:0/1549211243 shutdown_connections 2024-02-16T22:17:30.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.212+0000 7fe72b88a700 1 --2- 172.21.15.114:0/1549211243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe724101790 0x7fe7241057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:30.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.212+0000 7fe72b88a700 1 --2- 172.21.15.114:0/1549211243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe724100de0 0x7fe7241011c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:30.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.212+0000 7fe72b88a700 1 -- 172.21.15.114:0/1549211243 >> 172.21.15.114:0/1549211243 conn(0x7fe7240fc670 msgr2=0x7fe7240fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:30.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.212+0000 7fe72b88a700 1 -- 172.21.15.114:0/1549211243 shutdown_connections 2024-02-16T22:17:30.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.212+0000 7fe72b88a700 1 -- 172.21.15.114:0/1549211243 wait complete. 2024-02-16T22:17:30.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.213+0000 7fe72b88a700 1 Processor -- start 2024-02-16T22:17:30.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.213+0000 7fe72b88a700 1 -- start start 2024-02-16T22:17:30.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.213+0000 7fe72b88a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe724100de0 0x7fe724073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:30.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.213+0000 7fe72b88a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe724101790 0x7fe7240735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:30.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.214+0000 7fe72b88a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe724073b10 con 0x7fe724101790 2024-02-16T22:17:30.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.214+0000 7fe72b88a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe724073c50 con 0x7fe724100de0 2024-02-16T22:17:30.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.214+0000 7fe728e25700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe724101790 0x7fe7240735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:30.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.214+0000 7fe728e25700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe724101790 0x7fe7240735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40462/0 (socket says 172.21.15.114:40462) 2024-02-16T22:17:30.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.214+0000 7fe728e25700 1 -- 172.21.15.114:0/516562533 learned_addr learned my addr 172.21.15.114:0/516562533 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:17:30.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.214+0000 7fe729626700 1 --2- 172.21.15.114:0/516562533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe724100de0 0x7fe724073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:30.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.214+0000 7fe728e25700 1 -- 172.21.15.114:0/516562533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe724100de0 msgr2=0x7fe724073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:30.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.214+0000 7fe728e25700 1 --2- 172.21.15.114:0/516562533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe724100de0 0x7fe724073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:30.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.214+0000 7fe728e25700 1 -- 172.21.15.114:0/516562533 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7200095f0 con 0x7fe724101790 2024-02-16T22:17:30.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.214+0000 7fe728e25700 1 --2- 172.21.15.114:0/516562533 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe724101790 0x7fe7240735d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe71400c960 tx=0x7fe71400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:30.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.214+0000 7fe71a7fc700 1 -- 172.21.15.114:0/516562533 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe714007a70 con 0x7fe724101790 2024-02-16T22:17:30.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.214+0000 7fe71a7fc700 1 -- 172.21.15.114:0/516562533 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe714007bd0 con 0x7fe724101790 2024-02-16T22:17:30.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.214+0000 7fe72b88a700 1 -- 172.21.15.114:0/516562533 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7241a2930 con 0x7fe724101790 2024-02-16T22:17:30.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.214+0000 7fe71a7fc700 1 -- 172.21.15.114:0/516562533 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe714018630 con 0x7fe724101790 2024-02-16T22:17:30.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.214+0000 7fe72b88a700 1 -- 172.21.15.114:0/516562533 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7241a2c90 con 0x7fe724101790 2024-02-16T22:17:30.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.215+0000 7fe72b88a700 1 -- 172.21.15.114:0/516562533 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe724109180 con 0x7fe724101790 2024-02-16T22:17:30.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.216+0000 7fe71a7fc700 1 -- 172.21.15.114:0/516562533 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fe714018790 con 0x7fe724101790 2024-02-16T22:17:30.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.216+0000 7fe71a7fc700 1 --2- 172.21.15.114:0/516562533 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fe71006c4e0 0x7fe71006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:30.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.216+0000 7fe71a7fc700 1 -- 172.21.15.114:0/516562533 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe71408b460 con 0x7fe724101790 2024-02-16T22:17:30.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.216+0000 7fe729626700 1 --2- 172.21.15.114:0/516562533 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fe71006c4e0 0x7fe71006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:30.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.217+0000 7fe729626700 1 --2- 172.21.15.114:0/516562533 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fe71006c4e0 0x7fe71006e9a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe72000b5c0 tx=0x7fe72001a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:30.885 INFO:teuthology.orchestra.run.smithi114.stdout:111669149717 2024-02-16T22:17:30.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.220+0000 7fe71a7fc700 1 -- 172.21.15.114:0/516562533 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe714059a90 con 0x7fe724101790 2024-02-16T22:17:30.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.419+0000 7fe72b88a700 1 -- 172.21.15.114:0/516562533 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe72404ea90 con 0x7fe724101790 2024-02-16T22:17:30.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.419+0000 7fe71a7fc700 1 -- 172.21.15.114:0/516562533 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe714059620 con 0x7fe724101790 2024-02-16T22:17:30.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.422+0000 7fe72b88a700 1 -- 172.21.15.114:0/516562533 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fe71006c4e0 msgr2=0x7fe71006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:30.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.422+0000 7fe72b88a700 1 --2- 172.21.15.114:0/516562533 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fe71006c4e0 0x7fe71006e9a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe72000b5c0 tx=0x7fe72001a040 comp rx=0 tx=0).stop 2024-02-16T22:17:30.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.422+0000 7fe72b88a700 1 -- 172.21.15.114:0/516562533 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe724101790 msgr2=0x7fe7240735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:30.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.422+0000 7fe72b88a700 1 --2- 172.21.15.114:0/516562533 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe724101790 0x7fe7240735d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe71400c960 tx=0x7fe71400cd20 comp rx=0 tx=0).stop 2024-02-16T22:17:30.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.422+0000 7fe72b88a700 1 -- 172.21.15.114:0/516562533 shutdown_connections 2024-02-16T22:17:30.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.422+0000 7fe72b88a700 1 --2- 172.21.15.114:0/516562533 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fe71006c4e0 0x7fe71006e9a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:30.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.422+0000 7fe72b88a700 1 --2- 172.21.15.114:0/516562533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe724100de0 0x7fe724073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:30.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.422+0000 7fe72b88a700 1 --2- 172.21.15.114:0/516562533 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe724101790 0x7fe7240735d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:30.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.422+0000 7fe72b88a700 1 -- 172.21.15.114:0/516562533 >> 172.21.15.114:0/516562533 conn(0x7fe7240fc670 msgr2=0x7fe7240fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:30.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.422+0000 7fe72b88a700 1 -- 172.21.15.114:0/516562533 shutdown_connections 2024-02-16T22:17:30.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:30.422+0000 7fe72b88a700 1 -- 172.21.15.114:0/516562533 wait complete. 2024-02-16T22:17:31.474 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705700 got 38654705702 for osd.0 2024-02-16T22:17:31.474 DEBUG:teuthology.parallel:result is None 2024-02-16T22:17:31.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:31 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/1357337539' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-16T22:17:31.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:31 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/2353098492' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-16T22:17:31.905 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:31 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/1357337539' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-16T22:17:31.905 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:31 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/2353098492' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-16T22:17:32.542 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149715 got 111669149717 for osd.3 2024-02-16T22:17:32.542 DEBUG:teuthology.parallel:result is None 2024-02-16T22:17:32.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:32 smithi123 ceph-mon[37161]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:32.836 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:32 smithi114 ceph-mon[29235]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:32.862 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542172 got 60129542176 for osd.1 2024-02-16T22:17:32.862 DEBUG:teuthology.parallel:result is None 2024-02-16T22:17:33.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.850+0000 7f9df268a700 1 -- 172.21.15.114:0/3509006545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9dec069020 msgr2=0x7f9dec105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:33.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.850+0000 7f9df268a700 1 --2- 172.21.15.114:0/3509006545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9dec069020 0x7f9dec105790 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc009910 tx=0x7f9ddc009c20 comp rx=0 tx=0).stop 2024-02-16T22:17:33.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.850+0000 7f9df268a700 1 -- 172.21.15.114:0/3509006545 shutdown_connections 2024-02-16T22:17:33.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.850+0000 7f9df268a700 1 --2- 172.21.15.114:0/3509006545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9dec069020 0x7f9dec105790 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:33.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.850+0000 7f9df268a700 1 --2- 172.21.15.114:0/3509006545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dec068670 0x7f9dec068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:33.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.850+0000 7f9df268a700 1 -- 172.21.15.114:0/3509006545 >> 172.21.15.114:0/3509006545 conn(0x7f9dec0755a0 msgr2=0x7f9dec0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:33.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.850+0000 7f9df268a700 1 -- 172.21.15.114:0/3509006545 shutdown_connections 2024-02-16T22:17:33.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.850+0000 7f9df268a700 1 -- 172.21.15.114:0/3509006545 wait complete. 2024-02-16T22:17:33.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.851+0000 7f9df268a700 1 Processor -- start 2024-02-16T22:17:33.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.851+0000 7f9df268a700 1 -- start start 2024-02-16T22:17:33.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.851+0000 7f9df268a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dec068670 0x7f9dec19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:33.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.851+0000 7f9df268a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9dec069020 0x7f9dec19d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:33.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.851+0000 7f9df268a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec19d980 con 0x7f9dec069020 2024-02-16T22:17:33.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.851+0000 7f9df268a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec1a1710 con 0x7f9dec068670 2024-02-16T22:17:33.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.851+0000 7f9deb7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9dec069020 0x7f9dec19d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:33.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.851+0000 7f9deb7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9dec069020 0x7f9dec19d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57754/0 (socket says 172.21.15.114:57754) 2024-02-16T22:17:33.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.851+0000 7f9deb7fe700 1 -- 172.21.15.114:0/1567594584 learned_addr learned my addr 172.21.15.114:0/1567594584 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:17:33.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.851+0000 7f9debfff700 1 --2- 172.21.15.114:0/1567594584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dec068670 0x7f9dec19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:33.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.851+0000 7f9deb7fe700 1 -- 172.21.15.114:0/1567594584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dec068670 msgr2=0x7f9dec19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:33.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.851+0000 7f9deb7fe700 1 --2- 172.21.15.114:0/1567594584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dec068670 0x7f9dec19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:33.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.851+0000 7f9deb7fe700 1 -- 172.21.15.114:0/1567594584 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ddc0095f0 con 0x7f9dec069020 2024-02-16T22:17:33.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.851+0000 7f9debfff700 1 --2- 172.21.15.114:0/1567594584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dec068670 0x7f9dec19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:17:33.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.852+0000 7f9deb7fe700 1 --2- 172.21.15.114:0/1567594584 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9dec069020 0x7f9dec19d2a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc0092e0 tx=0x7f9ddc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:33.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.852+0000 7f9de97fa700 1 -- 172.21.15.114:0/1567594584 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ddc01d070 con 0x7f9dec069020 2024-02-16T22:17:33.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.852+0000 7f9df268a700 1 -- 172.21.15.114:0/1567594584 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dec1a1990 con 0x7f9dec069020 2024-02-16T22:17:33.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.852+0000 7f9de97fa700 1 -- 172.21.15.114:0/1567594584 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9ddc00f460 con 0x7f9dec069020 2024-02-16T22:17:33.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.852+0000 7f9de97fa700 1 -- 172.21.15.114:0/1567594584 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ddc017630 con 0x7f9dec069020 2024-02-16T22:17:33.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.852+0000 7f9df268a700 1 -- 172.21.15.114:0/1567594584 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dec1a1e80 con 0x7f9dec069020 2024-02-16T22:17:33.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.853+0000 7f9df268a700 1 -- 172.21.15.114:0/1567594584 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dec04ea90 con 0x7f9dec069020 2024-02-16T22:17:33.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.853+0000 7f9de97fa700 1 -- 172.21.15.114:0/1567594584 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f9ddc003a60 con 0x7f9dec069020 2024-02-16T22:17:33.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.854+0000 7f9de97fa700 1 --2- 172.21.15.114:0/1567594584 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f9dd806c4e0 0x7f9dd806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:33.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.854+0000 7f9de97fa700 1 -- 172.21.15.114:0/1567594584 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f9ddc08c9b0 con 0x7f9dec069020 2024-02-16T22:17:33.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.854+0000 7f9debfff700 1 --2- 172.21.15.114:0/1567594584 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f9dd806c4e0 0x7f9dd806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:33.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.854+0000 7f9debfff700 1 --2- 172.21.15.114:0/1567594584 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f9dd806c4e0 0x7f9dd806e9a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400bcf0 tx=0x7f9dd400b2c0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:33.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:33.858+0000 7f9de97fa700 1 -- 172.21.15.114:0/1567594584 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9ddc0574d0 con 0x7f9dec069020 2024-02-16T22:17:34.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:34.047+0000 7f9df268a700 1 -- 172.21.15.114:0/1567594584 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f9dec19e0c0 con 0x7f9dec069020 2024-02-16T22:17:34.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:34.048+0000 7f9de97fa700 1 -- 172.21.15.114:0/1567594584 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9ddc05aaf0 con 0x7f9dec069020 2024-02-16T22:17:34.050 INFO:teuthology.orchestra.run.smithi114.stdout:167503724555 2024-02-16T22:17:34.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:34.050+0000 7f9df268a700 1 -- 172.21.15.114:0/1567594584 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f9dd806c4e0 msgr2=0x7f9dd806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:34.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:34.050+0000 7f9df268a700 1 --2- 172.21.15.114:0/1567594584 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f9dd806c4e0 0x7f9dd806e9a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400bcf0 tx=0x7f9dd400b2c0 comp rx=0 tx=0).stop 2024-02-16T22:17:34.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:34.050+0000 7f9df268a700 1 -- 172.21.15.114:0/1567594584 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9dec069020 msgr2=0x7f9dec19d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:34.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:34.050+0000 7f9df268a700 1 --2- 172.21.15.114:0/1567594584 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9dec069020 0x7f9dec19d2a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc0092e0 tx=0x7f9ddc00be20 comp rx=0 tx=0).stop 2024-02-16T22:17:34.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:34.050+0000 7f9df268a700 1 -- 172.21.15.114:0/1567594584 shutdown_connections 2024-02-16T22:17:34.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:34.050+0000 7f9df268a700 1 --2- 172.21.15.114:0/1567594584 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f9dd806c4e0 0x7f9dd806e9a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:34.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:34.050+0000 7f9df268a700 1 --2- 172.21.15.114:0/1567594584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dec068670 0x7f9dec19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:34.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:34.050+0000 7f9df268a700 1 --2- 172.21.15.114:0/1567594584 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9dec069020 0x7f9dec19d2a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:34.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:34.050+0000 7f9df268a700 1 -- 172.21.15.114:0/1567594584 >> 172.21.15.114:0/1567594584 conn(0x7f9dec0755a0 msgr2=0x7f9dec105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:34.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:34.050+0000 7f9df268a700 1 -- 172.21.15.114:0/1567594584 shutdown_connections 2024-02-16T22:17:34.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:34.050+0000 7f9df268a700 1 -- 172.21.15.114:0/1567594584 wait complete. 2024-02-16T22:17:34.623 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:34 smithi114 ceph-mon[29235]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:34.623 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:34 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/1567594584' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-16T22:17:34.645 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724553 got 167503724555 for osd.5 2024-02-16T22:17:34.645 DEBUG:teuthology.parallel:result is None 2024-02-16T22:17:34.645 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-16T22:17:34.646 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-16T22:17:34.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:34 smithi123 ceph-mon[37161]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:34.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:34 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/1567594584' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-16T22:17:34.854 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:17:36.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.480+0000 7f04f31a6700 1 -- 172.21.15.114:0/1384631431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04ec0ff530 msgr2=0x7f04ec0ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:36.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.480+0000 7f04f31a6700 1 --2- 172.21.15.114:0/1384631431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04ec0ff530 0x7f04ec0ff910 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f04e0009910 tx=0x7f04e0009c20 comp rx=0 tx=0).stop 2024-02-16T22:17:36.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.480+0000 7f04f31a6700 1 -- 172.21.15.114:0/1384631431 shutdown_connections 2024-02-16T22:17:36.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.480+0000 7f04f31a6700 1 --2- 172.21.15.114:0/1384631431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04ec0ffee0 0x7f04ec103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:36.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.480+0000 7f04f31a6700 1 --2- 172.21.15.114:0/1384631431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04ec0ff530 0x7f04ec0ff910 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:36.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.480+0000 7f04f31a6700 1 -- 172.21.15.114:0/1384631431 >> 172.21.15.114:0/1384631431 conn(0x7f04ec074df0 msgr2=0x7f04ec075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:36.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.480+0000 7f04f31a6700 1 -- 172.21.15.114:0/1384631431 shutdown_connections 2024-02-16T22:17:36.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.480+0000 7f04f31a6700 1 -- 172.21.15.114:0/1384631431 wait complete. 2024-02-16T22:17:36.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.481+0000 7f04f31a6700 1 Processor -- start 2024-02-16T22:17:36.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.481+0000 7f04f31a6700 1 -- start start 2024-02-16T22:17:36.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.481+0000 7f04f31a6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04ec0ff530 0x7f04ec197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:36.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.481+0000 7f04f31a6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04ec0ffee0 0x7f04ec197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:36.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.481+0000 7f04f31a6700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ec198270 con 0x7f04ec0ff530 2024-02-16T22:17:36.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.481+0000 7f04f31a6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ec1983b0 con 0x7f04ec0ffee0 2024-02-16T22:17:36.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.481+0000 7f04f0f42700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04ec0ff530 0x7f04ec197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:36.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.481+0000 7f04f0f42700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04ec0ff530 0x7f04ec197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57782/0 (socket says 172.21.15.114:57782) 2024-02-16T22:17:36.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.481+0000 7f04f0f42700 1 -- 172.21.15.114:0/4216907142 learned_addr learned my addr 172.21.15.114:0/4216907142 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:17:36.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.482+0000 7f04ebfff700 1 --2- 172.21.15.114:0/4216907142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04ec0ffee0 0x7f04ec197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:36.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.482+0000 7f04f0f42700 1 -- 172.21.15.114:0/4216907142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04ec0ffee0 msgr2=0x7f04ec197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:36.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.482+0000 7f04f0f42700 1 --2- 172.21.15.114:0/4216907142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04ec0ffee0 0x7f04ec197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:36.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.482+0000 7f04f0f42700 1 -- 172.21.15.114:0/4216907142 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e00095f0 con 0x7f04ec0ff530 2024-02-16T22:17:36.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.482+0000 7f04f0f42700 1 --2- 172.21.15.114:0/4216907142 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04ec0ff530 0x7f04ec197710 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f04e0006010 tx=0x7f04e000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:36.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.482+0000 7f04e9ffb700 1 -- 172.21.15.114:0/4216907142 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04e001d070 con 0x7f04ec0ff530 2024-02-16T22:17:36.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.482+0000 7f04e9ffb700 1 -- 172.21.15.114:0/4216907142 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f04e0004fa0 con 0x7f04ec0ff530 2024-02-16T22:17:36.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.482+0000 7f04e9ffb700 1 -- 172.21.15.114:0/4216907142 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04e000f690 con 0x7f04ec0ff530 2024-02-16T22:17:36.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.482+0000 7f04f31a6700 1 -- 172.21.15.114:0/4216907142 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04ec1a0780 con 0x7f04ec0ff530 2024-02-16T22:17:36.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.482+0000 7f04f31a6700 1 -- 172.21.15.114:0/4216907142 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04ec1a0b90 con 0x7f04ec0ff530 2024-02-16T22:17:36.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.483+0000 7f04f31a6700 1 -- 172.21.15.114:0/4216907142 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04ec107820 con 0x7f04ec0ff530 2024-02-16T22:17:36.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.484+0000 7f04e9ffb700 1 -- 172.21.15.114:0/4216907142 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f04e0022470 con 0x7f04ec0ff530 2024-02-16T22:17:36.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.484+0000 7f04e9ffb700 1 --2- 172.21.15.114:0/4216907142 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f04dc06c4e0 0x7f04dc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:36.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.484+0000 7f04e9ffb700 1 -- 172.21.15.114:0/4216907142 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f04e008cb90 con 0x7f04ec0ff530 2024-02-16T22:17:36.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.484+0000 7f04ebfff700 1 --2- 172.21.15.114:0/4216907142 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f04dc06c4e0 0x7f04dc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:36.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.484+0000 7f04ebfff700 1 --2- 172.21.15.114:0/4216907142 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f04dc06c4e0 0x7f04dc06e9a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f04ec10aa00 tx=0x7f04d8009190 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:36.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.488+0000 7f04e9ffb700 1 -- 172.21.15.114:0/4216907142 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f04e0057850 con 0x7f04ec0ff530 2024-02-16T22:17:36.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.675+0000 7f04f31a6700 1 -- 172.21.15.114:0/4216907142 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f04ec0611d0 con 0x7f04dc06c4e0 2024-02-16T22:17:36.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.677+0000 7f04e9ffb700 1 -- 172.21.15.114:0/4216907142 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27799 (secure 0 0 0) 0x7f04ec0611d0 con 0x7f04dc06c4e0 2024-02-16T22:17:36.681 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:17:36.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.679+0000 7f04f31a6700 1 -- 172.21.15.114:0/4216907142 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f04dc06c4e0 msgr2=0x7f04dc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:36.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.679+0000 7f04f31a6700 1 --2- 172.21.15.114:0/4216907142 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f04dc06c4e0 0x7f04dc06e9a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f04ec10aa00 tx=0x7f04d8009190 comp rx=0 tx=0).stop 2024-02-16T22:17:36.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.679+0000 7f04f31a6700 1 -- 172.21.15.114:0/4216907142 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04ec0ff530 msgr2=0x7f04ec197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:36.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.679+0000 7f04f31a6700 1 --2- 172.21.15.114:0/4216907142 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04ec0ff530 0x7f04ec197710 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f04e0006010 tx=0x7f04e000bd70 comp rx=0 tx=0).stop 2024-02-16T22:17:36.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.679+0000 7f04f31a6700 1 -- 172.21.15.114:0/4216907142 shutdown_connections 2024-02-16T22:17:36.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.679+0000 7f04f31a6700 1 --2- 172.21.15.114:0/4216907142 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04ec0ff530 0x7f04ec197710 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:36.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.679+0000 7f04f31a6700 1 --2- 172.21.15.114:0/4216907142 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f04dc06c4e0 0x7f04dc06e9a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:36.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.679+0000 7f04f31a6700 1 --2- 172.21.15.114:0/4216907142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04ec0ffee0 0x7f04ec197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:36.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.679+0000 7f04f31a6700 1 -- 172.21.15.114:0/4216907142 >> 172.21.15.114:0/4216907142 conn(0x7f04ec074df0 msgr2=0x7f04ec0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:36.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.679+0000 7f04f31a6700 1 -- 172.21.15.114:0/4216907142 shutdown_connections 2024-02-16T22:17:36.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:36.679+0000 7f04f31a6700 1 -- 172.21.15.114:0/4216907142 wait complete. 2024-02-16T22:17:36.682 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-02-16T22:17:36.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:36 smithi123 ceph-mon[37161]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:36.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:36 smithi114 ceph-mon[29235]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:37.283 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-02-16T22:17:35.454430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165588,"kb_used_data":3876,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305324,"statfs":{"total":575970213888,"available":575800651776,"internally_reserved":0,"allocated":3969024,"data_stored":2609166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002444"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":187,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-02-16T22:16:47.519247+0000","last_change":"2024-02-16T22:16:22.032783+0000","last_active":"2024-02-16T22:16:47.519247+0000","last_peered":"2024-02-16T22:16:47.519247+0000","last_clean":"2024-02-16T22:16:47.519247+0000","last_became_active":"2024-02-16T22:16:22.032528+0000","last_became_peered":"2024-02-16T22:16:22.032528+0000","last_unstale":"2024-02-16T22:16:47.519247+0000","last_undegraded":"2024-02-16T22:16:47.519247+0000","last_fullsized":"2024-02-16T22:16:47.519247+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-16T22:15:31.863386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-16T22:15:31.863386+0000","last_clean_scrub_stamp":"2024-02-16T22:15:31.863386+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T07:07:16.638085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724556,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]}]},{"osd":4,"up_from":34,"seq":146028888081,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 22:17:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.52400000000000002}]},{"osd":1,"last update":"Fri Feb 16 22:17:24 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.50700000000000001}]},{"osd":2,"last update":"Fri Feb 16 22:17:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.48699999999999999}]},{"osd":3,"last update":"Fri Feb 16 22:17:24 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.46899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]}]},{"osd":3,"up_from":26,"seq":111669149718,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 22:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.52500000000000002}]},{"osd":1,"last update":"Fri Feb 16 22:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.504},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.40500000000000003}]},{"osd":2,"last update":"Fri Feb 16 22:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.45600000000000002}]},{"osd":4,"last update":"Fri Feb 16 22:17:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.377},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.311}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 22:16:31 2024","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"last":0.26700000000000002}]},{"osd":1,"last update":"Fri Feb 16 22:16:31 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.22600000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.497,"5min":0.497,"15min":0.497},"last":0.253}]},{"osd":3,"last update":"Fri Feb 16 22:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":4.3129999999999997}]},{"osd":4,"last update":"Fri Feb 16 22:17:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":4.3259999999999996},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.47499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]}]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 16 22:17:05 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.314,"15min":0.308},"min":{"1min":0.219,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.32800000000000001,"15min":0.32000000000000001},"min":{"1min":0.25600000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.307}]},{"osd":2,"last update":"Fri Feb 16 22:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.33000000000000002}]},{"osd":3,"last update":"Fri Feb 16 22:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.503},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.47499999999999998}]},{"osd":4,"last update":"Fri Feb 16 22:17:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.53300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 22:17:05 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.32400000000000001,"15min":0.32100000000000001},"min":{"1min":0.214,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.52700000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.32000000000000001,"15min":0.315},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.49099999999999999,"5min":0.503,"15min":0.503},"last":0.42599999999999999}]},{"osd":2,"last update":"Fri Feb 16 22:16:32 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.443},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.41199999999999998}]},{"osd":3,"last update":"Fri Feb 16 22:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.51100000000000001}]},{"osd":4,"last update":"Fri Feb 16 22:17:22 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-16T22:17:37.284 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-16T22:17:37.488 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:17:37.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:37 smithi123 ceph-mon[37161]: from='client.14428 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-16T22:17:37.823 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:37 smithi114 ceph-mon[29235]: from='client.14428 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-16T22:17:38.670 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:38 smithi114 ceph-mon[29235]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:38.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:38 smithi123 ceph-mon[37161]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:39.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.071+0000 7fc10d9e8700 1 -- 172.21.15.114:0/1194234781 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc108101790 msgr2=0x7fc1081057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:39.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.071+0000 7fc10d9e8700 1 --2- 172.21.15.114:0/1194234781 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc108101790 0x7fc1081057e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc009910 tx=0x7fc0fc009c20 comp rx=0 tx=0).stop 2024-02-16T22:17:39.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.071+0000 7fc10d9e8700 1 -- 172.21.15.114:0/1194234781 shutdown_connections 2024-02-16T22:17:39.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.071+0000 7fc10d9e8700 1 --2- 172.21.15.114:0/1194234781 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc108101790 0x7fc1081057e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:39.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.071+0000 7fc10d9e8700 1 --2- 172.21.15.114:0/1194234781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc108100de0 0x7fc1081011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:39.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.071+0000 7fc10d9e8700 1 -- 172.21.15.114:0/1194234781 >> 172.21.15.114:0/1194234781 conn(0x7fc1080fc6b0 msgr2=0x7fc1080fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:39.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.071+0000 7fc10d9e8700 1 -- 172.21.15.114:0/1194234781 shutdown_connections 2024-02-16T22:17:39.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.071+0000 7fc10d9e8700 1 -- 172.21.15.114:0/1194234781 wait complete. 2024-02-16T22:17:39.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.071+0000 7fc10d9e8700 1 Processor -- start 2024-02-16T22:17:39.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.072+0000 7fc10d9e8700 1 -- start start 2024-02-16T22:17:39.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.072+0000 7fc10d9e8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc108100de0 0x7fc10819c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:39.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.072+0000 7fc10d9e8700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc108101790 0x7fc10819caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:39.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.072+0000 7fc10d9e8700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10819d180 con 0x7fc108101790 2024-02-16T22:17:39.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.072+0000 7fc10d9e8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc108196630 con 0x7fc108100de0 2024-02-16T22:17:39.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.072+0000 7fc1067fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc108101790 0x7fc10819caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:39.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.072+0000 7fc1067fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc108101790 0x7fc10819caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57808/0 (socket says 172.21.15.114:57808) 2024-02-16T22:17:39.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.072+0000 7fc1067fc700 1 -- 172.21.15.114:0/1971318534 learned_addr learned my addr 172.21.15.114:0/1971318534 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:17:39.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.072+0000 7fc106ffd700 1 --2- 172.21.15.114:0/1971318534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc108100de0 0x7fc10819c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:39.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.072+0000 7fc1067fc700 1 -- 172.21.15.114:0/1971318534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc108100de0 msgr2=0x7fc10819c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:39.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.072+0000 7fc1067fc700 1 --2- 172.21.15.114:0/1971318534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc108100de0 0x7fc10819c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:39.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.072+0000 7fc1067fc700 1 -- 172.21.15.114:0/1971318534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0fc0095f0 con 0x7fc108101790 2024-02-16T22:17:39.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.072+0000 7fc106ffd700 1 --2- 172.21.15.114:0/1971318534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc108100de0 0x7fc10819c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T22:17:39.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.072+0000 7fc1067fc700 1 --2- 172.21.15.114:0/1971318534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc108101790 0x7fc10819caf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc0092e0 tx=0x7fc0fc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:39.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.072+0000 7fc10c9e6700 1 -- 172.21.15.114:0/1971318534 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0fc01d070 con 0x7fc108101790 2024-02-16T22:17:39.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.073+0000 7fc10c9e6700 1 -- 172.21.15.114:0/1971318534 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc0fc022470 con 0x7fc108101790 2024-02-16T22:17:39.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.073+0000 7fc10d9e8700 1 -- 172.21.15.114:0/1971318534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1081968b0 con 0x7fc108101790 2024-02-16T22:17:39.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.073+0000 7fc10c9e6700 1 -- 172.21.15.114:0/1971318534 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0fc00f690 con 0x7fc108101790 2024-02-16T22:17:39.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.073+0000 7fc10d9e8700 1 -- 172.21.15.114:0/1971318534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc108196da0 con 0x7fc108101790 2024-02-16T22:17:39.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.073+0000 7fc10d9e8700 1 -- 172.21.15.114:0/1971318534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc108109130 con 0x7fc108101790 2024-02-16T22:17:39.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.074+0000 7fc10c9e6700 1 -- 172.21.15.114:0/1971318534 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fc0fc0225e0 con 0x7fc108101790 2024-02-16T22:17:39.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.074+0000 7fc10c9e6700 1 --2- 172.21.15.114:0/1971318534 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fc0f006c4e0 0x7fc0f006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:39.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.075+0000 7fc106ffd700 1 --2- 172.21.15.114:0/1971318534 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fc0f006c4e0 0x7fc0f006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:39.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.075+0000 7fc10c9e6700 1 -- 172.21.15.114:0/1971318534 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc0fc08cb00 con 0x7fc108101790 2024-02-16T22:17:39.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.075+0000 7fc106ffd700 1 --2- 172.21.15.114:0/1971318534 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fc0f006c4e0 0x7fc0f006e9a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8007900 tx=0x7fc0f8008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:39.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.079+0000 7fc10c9e6700 1 -- 172.21.15.114:0/1971318534 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc0fc05b160 con 0x7fc108101790 2024-02-16T22:17:39.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.267+0000 7fc10d9e8700 1 -- 172.21.15.114:0/1971318534 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc108196fc0 con 0x7fc0f006c4e0 2024-02-16T22:17:39.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.269+0000 7fc10c9e6700 1 -- 172.21.15.114:0/1971318534 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27895 (secure 0 0 0) 0x7fc108196fc0 con 0x7fc0f006c4e0 2024-02-16T22:17:39.272 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:17:39.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.272+0000 7fc10d9e8700 1 -- 172.21.15.114:0/1971318534 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fc0f006c4e0 msgr2=0x7fc0f006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:39.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.272+0000 7fc10d9e8700 1 --2- 172.21.15.114:0/1971318534 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fc0f006c4e0 0x7fc0f006e9a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8007900 tx=0x7fc0f8008040 comp rx=0 tx=0).stop 2024-02-16T22:17:39.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.272+0000 7fc10d9e8700 1 -- 172.21.15.114:0/1971318534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc108101790 msgr2=0x7fc10819caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:39.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.272+0000 7fc10d9e8700 1 --2- 172.21.15.114:0/1971318534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc108101790 0x7fc10819caf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc0092e0 tx=0x7fc0fc00be20 comp rx=0 tx=0).stop 2024-02-16T22:17:39.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.272+0000 7fc10d9e8700 1 -- 172.21.15.114:0/1971318534 shutdown_connections 2024-02-16T22:17:39.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.272+0000 7fc10d9e8700 1 --2- 172.21.15.114:0/1971318534 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fc0f006c4e0 0x7fc0f006e9a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:39.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.272+0000 7fc10d9e8700 1 --2- 172.21.15.114:0/1971318534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc108100de0 0x7fc10819c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:39.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.272+0000 7fc10d9e8700 1 --2- 172.21.15.114:0/1971318534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc108101790 0x7fc10819caf0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:39.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.272+0000 7fc10d9e8700 1 -- 172.21.15.114:0/1971318534 >> 172.21.15.114:0/1971318534 conn(0x7fc1080fc6b0 msgr2=0x7fc1080fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:39.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.272+0000 7fc10d9e8700 1 -- 172.21.15.114:0/1971318534 shutdown_connections 2024-02-16T22:17:39.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:39.272+0000 7fc10d9e8700 1 -- 172.21.15.114:0/1971318534 wait complete. 2024-02-16T22:17:39.275 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-02-16T22:17:39.888 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-02-16T22:17:37.454729+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165588,"kb_used_data":3876,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305324,"statfs":{"total":575970213888,"available":575800651776,"internally_reserved":0,"allocated":3969024,"data_stored":2609166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002275"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":187,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-02-16T22:16:47.519247+0000","last_change":"2024-02-16T22:16:22.032783+0000","last_active":"2024-02-16T22:16:47.519247+0000","last_peered":"2024-02-16T22:16:47.519247+0000","last_clean":"2024-02-16T22:16:47.519247+0000","last_became_active":"2024-02-16T22:16:22.032528+0000","last_became_peered":"2024-02-16T22:16:22.032528+0000","last_unstale":"2024-02-16T22:16:47.519247+0000","last_undegraded":"2024-02-16T22:16:47.519247+0000","last_fullsized":"2024-02-16T22:16:47.519247+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-16T22:15:31.863386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-16T22:15:31.863386+0000","last_clean_scrub_stamp":"2024-02-16T22:15:31.863386+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T07:07:16.638085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724556,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]}]},{"osd":4,"up_from":34,"seq":146028888081,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 22:17:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.52400000000000002}]},{"osd":1,"last update":"Fri Feb 16 22:17:24 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.50700000000000001}]},{"osd":2,"last update":"Fri Feb 16 22:17:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.48699999999999999}]},{"osd":3,"last update":"Fri Feb 16 22:17:24 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.46899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]}]},{"osd":3,"up_from":26,"seq":111669149719,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 22:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.52400000000000002}]},{"osd":1,"last update":"Fri Feb 16 22:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.45600000000000002}]},{"osd":2,"last update":"Fri Feb 16 22:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.497},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.47999999999999998}]},{"osd":4,"last update":"Fri Feb 16 22:17:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.36699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 22:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.45200000000000001,"5min":0.498,"15min":0.498},"last":0.247},{"interface":"front","average":{"1min":0.316,"5min":0.34499999999999997,"15min":0.34999999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.31900000000000001}]},{"osd":1,"last update":"Fri Feb 16 22:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.35299999999999998,"15min":0.35999999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.36499999999999999,"15min":0.36699999999999999},"min":{"1min":0.247,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.28000000000000003}]},{"osd":3,"last update":"Fri Feb 16 22:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.501}]},{"osd":4,"last update":"Fri Feb 16 22:17:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.50900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]}]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 16 22:17:05 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.314,"15min":0.308},"min":{"1min":0.219,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.376,"5min":0.32800000000000001,"15min":0.32000000000000001},"min":{"1min":0.25600000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.318}]},{"osd":2,"last update":"Fri Feb 16 22:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.35999999999999999}]},{"osd":3,"last update":"Fri Feb 16 22:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.67200000000000004}]},{"osd":4,"last update":"Fri Feb 16 22:17:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.71299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 22:17:05 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.32400000000000001,"15min":0.32100000000000001},"min":{"1min":0.214,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.52700000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.32000000000000001,"15min":0.315},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.49099999999999999,"5min":0.503,"15min":0.503},"last":0.42599999999999999}]},{"osd":2,"last update":"Fri Feb 16 22:16:32 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.443},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.41199999999999998}]},{"osd":3,"last update":"Fri Feb 16 22:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.51100000000000001}]},{"osd":4,"last update":"Fri Feb 16 22:17:22 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-16T22:17:39.889 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-16T22:17:39.889 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-16T22:17:39.889 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-16T22:17:39.890 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph health --format=json 2024-02-16T22:17:40.094 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:17:40.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:40 smithi123 ceph-mon[37161]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-16T22:17:40.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:40 smithi123 ceph-mon[37161]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:40.882 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:40 smithi114 ceph-mon[29235]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-16T22:17:40.882 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:40 smithi114 ceph-mon[29235]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:41.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.682+0000 7ff218e0e700 1 -- 172.21.15.114:0/4159441580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff214100240 msgr2=0x7ff21410cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:41.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.682+0000 7ff218e0e700 1 --2- 172.21.15.114:0/4159441580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff214100240 0x7ff21410cf90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff208009910 tx=0x7ff208009c20 comp rx=0 tx=0).stop 2024-02-16T22:17:41.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.682+0000 7ff218e0e700 1 -- 172.21.15.114:0/4159441580 shutdown_connections 2024-02-16T22:17:41.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.682+0000 7ff218e0e700 1 --2- 172.21.15.114:0/4159441580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff214100240 0x7ff21410cf90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:41.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.682+0000 7ff218e0e700 1 --2- 172.21.15.114:0/4159441580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2140ff920 0x7ff2140ffd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:41.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.682+0000 7ff218e0e700 1 -- 172.21.15.114:0/4159441580 >> 172.21.15.114:0/4159441580 conn(0x7ff2140fb560 msgr2=0x7ff2140fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:41.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.682+0000 7ff218e0e700 1 -- 172.21.15.114:0/4159441580 shutdown_connections 2024-02-16T22:17:41.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.682+0000 7ff218e0e700 1 -- 172.21.15.114:0/4159441580 wait complete. 2024-02-16T22:17:41.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.683+0000 7ff218e0e700 1 Processor -- start 2024-02-16T22:17:41.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.683+0000 7ff218e0e700 1 -- start start 2024-02-16T22:17:41.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.683+0000 7ff218e0e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2140ff920 0x7ff214198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:41.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.683+0000 7ff218e0e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff214100240 0x7ff214198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:41.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.683+0000 7ff218e0e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff214199560 con 0x7ff214100240 2024-02-16T22:17:41.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.683+0000 7ff218e0e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21419d370 con 0x7ff2140ff920 2024-02-16T22:17:41.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.683+0000 7ff211d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff214100240 0x7ff214198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:41.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.683+0000 7ff211d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff214100240 0x7ff214198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49934/0 (socket says 172.21.15.114:49934) 2024-02-16T22:17:41.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.683+0000 7ff211d9b700 1 -- 172.21.15.114:0/1923414209 learned_addr learned my addr 172.21.15.114:0/1923414209 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:17:41.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.683+0000 7ff21259c700 1 --2- 172.21.15.114:0/1923414209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2140ff920 0x7ff214198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:41.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.684+0000 7ff211d9b700 1 -- 172.21.15.114:0/1923414209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2140ff920 msgr2=0x7ff214198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:41.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.684+0000 7ff211d9b700 1 --2- 172.21.15.114:0/1923414209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2140ff920 0x7ff214198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:41.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.684+0000 7ff211d9b700 1 -- 172.21.15.114:0/1923414209 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2080095f0 con 0x7ff214100240 2024-02-16T22:17:41.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.684+0000 7ff21259c700 1 --2- 172.21.15.114:0/1923414209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2140ff920 0x7ff214198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:17:41.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.684+0000 7ff211d9b700 1 --2- 172.21.15.114:0/1923414209 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff214100240 0x7ff214198ed0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff2080092e0 tx=0x7ff20800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:41.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.684+0000 7ff2037fe700 1 -- 172.21.15.114:0/1923414209 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff20801d070 con 0x7ff214100240 2024-02-16T22:17:41.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.684+0000 7ff2037fe700 1 -- 172.21.15.114:0/1923414209 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff20800f460 con 0x7ff214100240 2024-02-16T22:17:41.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.684+0000 7ff218e0e700 1 -- 172.21.15.114:0/1923414209 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff21419d5f0 con 0x7ff214100240 2024-02-16T22:17:41.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.684+0000 7ff2037fe700 1 -- 172.21.15.114:0/1923414209 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff208017630 con 0x7ff214100240 2024-02-16T22:17:41.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.684+0000 7ff218e0e700 1 -- 172.21.15.114:0/1923414209 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff21419dae0 con 0x7ff214100240 2024-02-16T22:17:41.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.685+0000 7ff218e0e700 1 -- 172.21.15.114:0/1923414209 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff21410a700 con 0x7ff214100240 2024-02-16T22:17:41.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.685+0000 7ff2037fe700 1 -- 172.21.15.114:0/1923414209 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7ff20800f5d0 con 0x7ff214100240 2024-02-16T22:17:41.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.686+0000 7ff2037fe700 1 --2- 172.21.15.114:0/1923414209 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff1fc06c4e0 0x7ff1fc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:41.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.686+0000 7ff21259c700 1 --2- 172.21.15.114:0/1923414209 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff1fc06c4e0 0x7ff1fc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:41.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.686+0000 7ff2037fe700 1 -- 172.21.15.114:0/1923414209 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff20808c970 con 0x7ff214100240 2024-02-16T22:17:41.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.686+0000 7ff21259c700 1 --2- 172.21.15.114:0/1923414209 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff1fc06c4e0 0x7ff1fc06e9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff204005950 tx=0x7ff2040058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:41.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.690+0000 7ff2037fe700 1 -- 172.21.15.114:0/1923414209 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff20805af20 con 0x7ff214100240 2024-02-16T22:17:41.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:41 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:17:41.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.919+0000 7ff218e0e700 1 -- 172.21.15.114:0/1923414209 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff21404ea90 con 0x7ff214100240 2024-02-16T22:17:41.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.920+0000 7ff2037fe700 1 -- 172.21.15.114:0/1923414209 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7ff20805aab0 con 0x7ff214100240 2024-02-16T22:17:41.921 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:41 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:17:41.921 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:17:41.921 INFO:teuthology.orchestra.run.smithi114.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-16T22:17:41.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.922+0000 7ff218e0e700 1 -- 172.21.15.114:0/1923414209 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff1fc06c4e0 msgr2=0x7ff1fc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:41.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.922+0000 7ff218e0e700 1 --2- 172.21.15.114:0/1923414209 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff1fc06c4e0 0x7ff1fc06e9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff204005950 tx=0x7ff2040058e0 comp rx=0 tx=0).stop 2024-02-16T22:17:41.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.922+0000 7ff218e0e700 1 -- 172.21.15.114:0/1923414209 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff214100240 msgr2=0x7ff214198ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:41.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.922+0000 7ff218e0e700 1 --2- 172.21.15.114:0/1923414209 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff214100240 0x7ff214198ed0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff2080092e0 tx=0x7ff20800be20 comp rx=0 tx=0).stop 2024-02-16T22:17:41.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.922+0000 7ff218e0e700 1 -- 172.21.15.114:0/1923414209 shutdown_connections 2024-02-16T22:17:41.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.922+0000 7ff218e0e700 1 --2- 172.21.15.114:0/1923414209 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff1fc06c4e0 0x7ff1fc06e9a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:41.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.922+0000 7ff218e0e700 1 --2- 172.21.15.114:0/1923414209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2140ff920 0x7ff214198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:41.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.922+0000 7ff218e0e700 1 --2- 172.21.15.114:0/1923414209 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff214100240 0x7ff214198ed0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:41.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.922+0000 7ff218e0e700 1 -- 172.21.15.114:0/1923414209 >> 172.21.15.114:0/1923414209 conn(0x7ff2140fb560 msgr2=0x7ff2140fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:41.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.922+0000 7ff218e0e700 1 -- 172.21.15.114:0/1923414209 shutdown_connections 2024-02-16T22:17:41.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:41.922+0000 7ff218e0e700 1 -- 172.21.15.114:0/1923414209 wait complete. 2024-02-16T22:17:42.544 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-16T22:17:42.545 INFO:tasks.cephadm:Setup complete, yielding 2024-02-16T22:17:42.545 INFO:teuthology.run_tasks:Running task print... 2024-02-16T22:17:42.558 INFO:teuthology.task.print:**** done end installing v18.2.1 cephadm ... 2024-02-16T22:17:42.558 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T22:17:42.574 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi114.front.sepia.ceph.com 2024-02-16T22:17:42.574 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-02-16T22:17:42.781 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:17:42.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:42 smithi123 ceph-mon[37161]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:42.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:42 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/1923414209' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-16T22:17:42.902 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:42 smithi114 ceph-mon[29235]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:42.902 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:42 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/1923414209' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-16T22:17:44.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.489+0000 7fc5dd4ff700 1 -- 172.21.15.114:0/4067971417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5d8100cb0 msgr2=0x7fc5d8101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:44.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.489+0000 7fc5dd4ff700 1 --2- 172.21.15.114:0/4067971417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5d8100cb0 0x7fc5d8101110 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009910 tx=0x7fc5cc009c20 comp rx=0 tx=0).stop 2024-02-16T22:17:44.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.489+0000 7fc5dd4ff700 1 -- 172.21.15.114:0/4067971417 shutdown_connections 2024-02-16T22:17:44.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.489+0000 7fc5dd4ff700 1 --2- 172.21.15.114:0/4067971417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5d8100cb0 0x7fc5d8101110 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:44.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.489+0000 7fc5dd4ff700 1 --2- 172.21.15.114:0/4067971417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5d80ffab0 0x7fc5d80ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:44.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.489+0000 7fc5dd4ff700 1 -- 172.21.15.114:0/4067971417 >> 172.21.15.114:0/4067971417 conn(0x7fc5d80fb030 msgr2=0x7fc5d80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:44.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.489+0000 7fc5dd4ff700 1 -- 172.21.15.114:0/4067971417 shutdown_connections 2024-02-16T22:17:44.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.489+0000 7fc5dd4ff700 1 -- 172.21.15.114:0/4067971417 wait complete. 2024-02-16T22:17:44.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.490+0000 7fc5dd4ff700 1 Processor -- start 2024-02-16T22:17:44.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.490+0000 7fc5dd4ff700 1 -- start start 2024-02-16T22:17:44.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.490+0000 7fc5dd4ff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5d80ffab0 0x7fc5d8068f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:44.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.490+0000 7fc5dd4ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5d8100cb0 0x7fc5d819b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:44.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.490+0000 7fc5dd4ff700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d819b690 con 0x7fc5d80ffab0 2024-02-16T22:17:44.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.490+0000 7fc5dd4ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d819b7d0 con 0x7fc5d8100cb0 2024-02-16T22:17:44.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.490+0000 7fc5d6ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5d80ffab0 0x7fc5d8068f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:44.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.490+0000 7fc5d6ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5d80ffab0 0x7fc5d8068f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49950/0 (socket says 172.21.15.114:49950) 2024-02-16T22:17:44.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.490+0000 7fc5d6ffd700 1 -- 172.21.15.114:0/381453863 learned_addr learned my addr 172.21.15.114:0/381453863 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:17:44.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.490+0000 7fc5d67fc700 1 --2- 172.21.15.114:0/381453863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5d8100cb0 0x7fc5d819b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:44.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.490+0000 7fc5d6ffd700 1 -- 172.21.15.114:0/381453863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5d8100cb0 msgr2=0x7fc5d819b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:44.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.490+0000 7fc5d6ffd700 1 --2- 172.21.15.114:0/381453863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5d8100cb0 0x7fc5d819b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:44.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.490+0000 7fc5d6ffd700 1 -- 172.21.15.114:0/381453863 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5cc0095f0 con 0x7fc5d80ffab0 2024-02-16T22:17:44.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.490+0000 7fc5d67fc700 1 --2- 172.21.15.114:0/381453863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5d8100cb0 0x7fc5d819b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T22:17:44.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.491+0000 7fc5d6ffd700 1 --2- 172.21.15.114:0/381453863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5d80ffab0 0x7fc5d8068f70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800c8f0 tx=0x7fc5c800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:44.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.491+0000 7fc5bffff700 1 -- 172.21.15.114:0/381453863 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5c8007a70 con 0x7fc5d80ffab0 2024-02-16T22:17:44.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.491+0000 7fc5dd4ff700 1 -- 172.21.15.114:0/381453863 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d819bab0 con 0x7fc5d80ffab0 2024-02-16T22:17:44.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.491+0000 7fc5bffff700 1 -- 172.21.15.114:0/381453863 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc5c8007bd0 con 0x7fc5d80ffab0 2024-02-16T22:17:44.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.491+0000 7fc5bffff700 1 -- 172.21.15.114:0/381453863 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5c8018610 con 0x7fc5d80ffab0 2024-02-16T22:17:44.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.491+0000 7fc5dd4ff700 1 -- 172.21.15.114:0/381453863 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d819c000 con 0x7fc5d80ffab0 2024-02-16T22:17:44.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.492+0000 7fc5dd4ff700 1 -- 172.21.15.114:0/381453863 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d8108610 con 0x7fc5d80ffab0 2024-02-16T22:17:44.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.492+0000 7fc5bffff700 1 -- 172.21.15.114:0/381453863 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fc5c800f450 con 0x7fc5d80ffab0 2024-02-16T22:17:44.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.493+0000 7fc5bffff700 1 --2- 172.21.15.114:0/381453863 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fc5c006c4e0 0x7fc5c006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:44.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.493+0000 7fc5d67fc700 1 --2- 172.21.15.114:0/381453863 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fc5c006c4e0 0x7fc5c006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:44.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.493+0000 7fc5bffff700 1 -- 172.21.15.114:0/381453863 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc5c808b4a0 con 0x7fc5d80ffab0 2024-02-16T22:17:44.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.493+0000 7fc5d67fc700 1 --2- 172.21.15.114:0/381453863 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fc5c006c4e0 0x7fc5c006e9a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc00b5c0 tx=0x7fc5cc005e20 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:44.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.497+0000 7fc5bffff700 1 -- 172.21.15.114:0/381453863 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc5c8059b00 con 0x7fc5d80ffab0 2024-02-16T22:17:44.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.685+0000 7fc5dd4ff700 1 -- 172.21.15.114:0/381453863 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7fc5d819cb10 con 0x7fc5d80ffab0 2024-02-16T22:17:44.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.689+0000 7fc5bffff700 1 -- 172.21.15.114:0/381453863 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7fc5c8059690 con 0x7fc5d80ffab0 2024-02-16T22:17:44.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.692+0000 7fc5dd4ff700 1 -- 172.21.15.114:0/381453863 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fc5c006c4e0 msgr2=0x7fc5c006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:44.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.692+0000 7fc5dd4ff700 1 --2- 172.21.15.114:0/381453863 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fc5c006c4e0 0x7fc5c006e9a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc00b5c0 tx=0x7fc5cc005e20 comp rx=0 tx=0).stop 2024-02-16T22:17:44.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.692+0000 7fc5dd4ff700 1 -- 172.21.15.114:0/381453863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5d80ffab0 msgr2=0x7fc5d8068f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:44.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.692+0000 7fc5dd4ff700 1 --2- 172.21.15.114:0/381453863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5d80ffab0 0x7fc5d8068f70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800c8f0 tx=0x7fc5c800ccb0 comp rx=0 tx=0).stop 2024-02-16T22:17:44.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.692+0000 7fc5dd4ff700 1 -- 172.21.15.114:0/381453863 shutdown_connections 2024-02-16T22:17:44.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.692+0000 7fc5dd4ff700 1 --2- 172.21.15.114:0/381453863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5d80ffab0 0x7fc5d8068f70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:44.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.692+0000 7fc5dd4ff700 1 --2- 172.21.15.114:0/381453863 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fc5c006c4e0 0x7fc5c006e9a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:44.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.692+0000 7fc5dd4ff700 1 --2- 172.21.15.114:0/381453863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5d8100cb0 0x7fc5d819b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:44.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.692+0000 7fc5dd4ff700 1 -- 172.21.15.114:0/381453863 >> 172.21.15.114:0/381453863 conn(0x7fc5d80fb030 msgr2=0x7fc5d8103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:44.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.692+0000 7fc5dd4ff700 1 -- 172.21.15.114:0/381453863 shutdown_connections 2024-02-16T22:17:44.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:44.692+0000 7fc5dd4ff700 1 -- 172.21.15.114:0/381453863 wait complete. 2024-02-16T22:17:44.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:44 smithi123 ceph-mon[37161]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:44.809 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:44 smithi114 ceph-mon[29235]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:45.375 INFO:teuthology.run_tasks:Running task print... 2024-02-16T22:17:45.383 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-02-16T22:17:45.383 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T22:17:45.393 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi114.front.sepia.ceph.com 2024-02-16T22:17:45.393 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- bash -c 'ceph orch status' 2024-02-16T22:17:45.612 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:17:45.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:45 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/381453863' entity='client.admin' 2024-02-16T22:17:45.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:45 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:17:45.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:45 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:17:45.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:45 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:17:45.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:45 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:17:45.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:45 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/381453863' entity='client.admin' 2024-02-16T22:17:45.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:45 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:17:45.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:45 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:17:45.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:45 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:17:45.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:45 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:17:46.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:46 smithi114 ceph-mon[29235]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:47.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:46 smithi123 ceph-mon[37161]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:47.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.155+0000 7fe34cd50700 1 -- 172.21.15.114:0/4282180936 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe3480ffac0 msgr2=0x7fe3480ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:47.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.155+0000 7fe34cd50700 1 --2- 172.21.15.114:0/4282180936 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe3480ffac0 0x7fe3480ffee0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe338009910 tx=0x7fe338009c20 comp rx=0 tx=0).stop 2024-02-16T22:17:47.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.155+0000 7fe34cd50700 1 -- 172.21.15.114:0/4282180936 shutdown_connections 2024-02-16T22:17:47.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.155+0000 7fe34cd50700 1 --2- 172.21.15.114:0/4282180936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe348100cc0 0x7fe348101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:47.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.155+0000 7fe34cd50700 1 --2- 172.21.15.114:0/4282180936 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe3480ffac0 0x7fe3480ffee0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:47.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.155+0000 7fe34cd50700 1 -- 172.21.15.114:0/4282180936 >> 172.21.15.114:0/4282180936 conn(0x7fe3480fb060 msgr2=0x7fe3480fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:47.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.155+0000 7fe34cd50700 1 -- 172.21.15.114:0/4282180936 shutdown_connections 2024-02-16T22:17:47.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.155+0000 7fe34cd50700 1 -- 172.21.15.114:0/4282180936 wait complete. 2024-02-16T22:17:47.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.155+0000 7fe34cd50700 1 Processor -- start 2024-02-16T22:17:47.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.156+0000 7fe34cd50700 1 -- start start 2024-02-16T22:17:47.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.156+0000 7fe34cd50700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3480ffac0 0x7fe348069930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:47.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.156+0000 7fe34cd50700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe348100cc0 0x7fe34819b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:47.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.156+0000 7fe34cd50700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34819b730 con 0x7fe348100cc0 2024-02-16T22:17:47.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.156+0000 7fe34cd50700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34819b8a0 con 0x7fe3480ffac0 2024-02-16T22:17:47.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.156+0000 7fe345d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe348100cc0 0x7fe34819b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:47.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.156+0000 7fe345d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe348100cc0 0x7fe34819b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49982/0 (socket says 172.21.15.114:49982) 2024-02-16T22:17:47.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.156+0000 7fe345d9b700 1 -- 172.21.15.114:0/3241903868 learned_addr learned my addr 172.21.15.114:0/3241903868 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:17:47.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.156+0000 7fe34659c700 1 --2- 172.21.15.114:0/3241903868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3480ffac0 0x7fe348069930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:47.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.156+0000 7fe345d9b700 1 -- 172.21.15.114:0/3241903868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3480ffac0 msgr2=0x7fe348069930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:47.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.156+0000 7fe345d9b700 1 --2- 172.21.15.114:0/3241903868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3480ffac0 0x7fe348069930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:47.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.156+0000 7fe345d9b700 1 -- 172.21.15.114:0/3241903868 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3380095f0 con 0x7fe348100cc0 2024-02-16T22:17:47.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.156+0000 7fe345d9b700 1 --2- 172.21.15.114:0/3241903868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe348100cc0 0x7fe34819b160 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00b4b0 tx=0x7fe33c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:47.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.156+0000 7fe3377fe700 1 -- 172.21.15.114:0/3241903868 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe33c011800 con 0x7fe348100cc0 2024-02-16T22:17:47.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.156+0000 7fe3377fe700 1 -- 172.21.15.114:0/3241903868 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe33c011e10 con 0x7fe348100cc0 2024-02-16T22:17:47.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.156+0000 7fe34cd50700 1 -- 172.21.15.114:0/3241903868 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe34819bb80 con 0x7fe348100cc0 2024-02-16T22:17:47.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.156+0000 7fe3377fe700 1 -- 172.21.15.114:0/3241903868 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe33c00f2e0 con 0x7fe348100cc0 2024-02-16T22:17:47.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.157+0000 7fe34cd50700 1 -- 172.21.15.114:0/3241903868 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe34819c150 con 0x7fe348100cc0 2024-02-16T22:17:47.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.157+0000 7fe34cd50700 1 -- 172.21.15.114:0/3241903868 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3481086a0 con 0x7fe348100cc0 2024-02-16T22:17:47.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.158+0000 7fe3377fe700 1 -- 172.21.15.114:0/3241903868 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fe33c010380 con 0x7fe348100cc0 2024-02-16T22:17:47.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.159+0000 7fe3377fe700 1 --2- 172.21.15.114:0/3241903868 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fe33006c4e0 0x7fe33006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:47.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.159+0000 7fe34659c700 1 --2- 172.21.15.114:0/3241903868 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fe33006c4e0 0x7fe33006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:47.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.159+0000 7fe3377fe700 1 -- 172.21.15.114:0/3241903868 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe33c08b050 con 0x7fe348100cc0 2024-02-16T22:17:47.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.159+0000 7fe34659c700 1 --2- 172.21.15.114:0/3241903868 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fe33006c4e0 0x7fe33006e9a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe338006010 tx=0x7fe338005920 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:47.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.163+0000 7fe3377fe700 1 -- 172.21.15.114:0/3241903868 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe33c059680 con 0x7fe348100cc0 2024-02-16T22:17:47.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.367+0000 7fe34cd50700 1 -- 172.21.15.114:0/3241903868 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe348105610 con 0x7fe33006c4e0 2024-02-16T22:17:47.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.368+0000 7fe3377fe700 1 -- 172.21.15.114:0/3241903868 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fe348105610 con 0x7fe33006c4e0 2024-02-16T22:17:47.370 INFO:teuthology.orchestra.run.smithi114.stdout:Backend: cephadm 2024-02-16T22:17:47.370 INFO:teuthology.orchestra.run.smithi114.stdout:Available: Yes 2024-02-16T22:17:47.370 INFO:teuthology.orchestra.run.smithi114.stdout:Paused: No 2024-02-16T22:17:47.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.370+0000 7fe34cd50700 1 -- 172.21.15.114:0/3241903868 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fe33006c4e0 msgr2=0x7fe33006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:47.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.370+0000 7fe34cd50700 1 --2- 172.21.15.114:0/3241903868 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fe33006c4e0 0x7fe33006e9a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe338006010 tx=0x7fe338005920 comp rx=0 tx=0).stop 2024-02-16T22:17:47.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.370+0000 7fe34cd50700 1 -- 172.21.15.114:0/3241903868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe348100cc0 msgr2=0x7fe34819b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:47.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.370+0000 7fe34cd50700 1 --2- 172.21.15.114:0/3241903868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe348100cc0 0x7fe34819b160 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00b4b0 tx=0x7fe33c00b870 comp rx=0 tx=0).stop 2024-02-16T22:17:47.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.370+0000 7fe34cd50700 1 -- 172.21.15.114:0/3241903868 shutdown_connections 2024-02-16T22:17:47.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.370+0000 7fe34cd50700 1 --2- 172.21.15.114:0/3241903868 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fe33006c4e0 0x7fe33006e9a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:47.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.370+0000 7fe34cd50700 1 --2- 172.21.15.114:0/3241903868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3480ffac0 0x7fe348069930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:47.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.370+0000 7fe34cd50700 1 --2- 172.21.15.114:0/3241903868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe348100cc0 0x7fe34819b160 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:47.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.370+0000 7fe34cd50700 1 -- 172.21.15.114:0/3241903868 >> 172.21.15.114:0/3241903868 conn(0x7fe3480fb060 msgr2=0x7fe348103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:47.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.370+0000 7fe34cd50700 1 -- 172.21.15.114:0/3241903868 shutdown_connections 2024-02-16T22:17:47.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:47.370+0000 7fe34cd50700 1 -- 172.21.15.114:0/3241903868 wait complete. 2024-02-16T22:17:47.994 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:47 smithi114 ceph-mon[29235]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:17:47.994 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:47 smithi114 ceph-mon[29235]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:48.016 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- bash -c 'ceph orch ps' 2024-02-16T22:17:48.226 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:17:48.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:47 smithi123 ceph-mon[37161]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:17:48.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:47 smithi123 ceph-mon[37161]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:49.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.904+0000 7fd870f7f700 1 -- 172.21.15.114:0/1267066275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd86c100d50 msgr2=0x7fd86c1011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:49.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.904+0000 7fd870f7f700 1 --2- 172.21.15.114:0/1267066275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd86c100d50 0x7fd86c1011d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd860009910 tx=0x7fd860009c20 comp rx=0 tx=0).stop 2024-02-16T22:17:49.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.904+0000 7fd870f7f700 1 -- 172.21.15.114:0/1267066275 shutdown_connections 2024-02-16T22:17:49.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.904+0000 7fd870f7f700 1 --2- 172.21.15.114:0/1267066275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd86c100d50 0x7fd86c1011d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:49.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.904+0000 7fd870f7f700 1 --2- 172.21.15.114:0/1267066275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd86c100030 0x7fd86c100450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:49.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.904+0000 7fd870f7f700 1 -- 172.21.15.114:0/1267066275 >> 172.21.15.114:0/1267066275 conn(0x7fd86c0fa810 msgr2=0x7fd86c0fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:49.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.904+0000 7fd870f7f700 1 -- 172.21.15.114:0/1267066275 shutdown_connections 2024-02-16T22:17:49.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.904+0000 7fd870f7f700 1 -- 172.21.15.114:0/1267066275 wait complete. 2024-02-16T22:17:49.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.905+0000 7fd870f7f700 1 Processor -- start 2024-02-16T22:17:49.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.905+0000 7fd870f7f700 1 -- start start 2024-02-16T22:17:49.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.905+0000 7fd870f7f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd86c100030 0x7fd86c0ff710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:49.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.905+0000 7fd870f7f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd86c100d50 0x7fd86c19b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:49.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.905+0000 7fd870f7f700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c19b670 con 0x7fd86c100030 2024-02-16T22:17:49.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.905+0000 7fd870f7f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c19b7e0 con 0x7fd86c100d50 2024-02-16T22:17:49.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.905+0000 7fd86a59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd86c100030 0x7fd86c0ff710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:49.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.905+0000 7fd86a59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd86c100030 0x7fd86c0ff710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50010/0 (socket says 172.21.15.114:50010) 2024-02-16T22:17:49.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.905+0000 7fd86a59c700 1 -- 172.21.15.114:0/1054707763 learned_addr learned my addr 172.21.15.114:0/1054707763 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:17:49.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.905+0000 7fd869d9b700 1 --2- 172.21.15.114:0/1054707763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd86c100d50 0x7fd86c19b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:49.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.905+0000 7fd86a59c700 1 -- 172.21.15.114:0/1054707763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd86c100d50 msgr2=0x7fd86c19b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:49.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.905+0000 7fd86a59c700 1 --2- 172.21.15.114:0/1054707763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd86c100d50 0x7fd86c19b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:49.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.905+0000 7fd86a59c700 1 -- 172.21.15.114:0/1054707763 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8600095f0 con 0x7fd86c100030 2024-02-16T22:17:49.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.906+0000 7fd86a59c700 1 --2- 172.21.15.114:0/1054707763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd86c100030 0x7fd86c0ff710 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd86400cfd0 tx=0x7fd86400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:49.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.906+0000 7fd85b7fe700 1 -- 172.21.15.114:0/1054707763 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd86400ad70 con 0x7fd86c100030 2024-02-16T22:17:49.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.906+0000 7fd870f7f700 1 -- 172.21.15.114:0/1054707763 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd86c19bac0 con 0x7fd86c100030 2024-02-16T22:17:49.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.906+0000 7fd85b7fe700 1 -- 172.21.15.114:0/1054707763 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd864007e50 con 0x7fd86c100030 2024-02-16T22:17:49.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.906+0000 7fd85b7fe700 1 -- 172.21.15.114:0/1054707763 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd864012650 con 0x7fd86c100030 2024-02-16T22:17:49.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.906+0000 7fd870f7f700 1 -- 172.21.15.114:0/1054707763 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd86c19bfe0 con 0x7fd86c100030 2024-02-16T22:17:49.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.907+0000 7fd870f7f700 1 -- 172.21.15.114:0/1054707763 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd86c1086a0 con 0x7fd86c100030 2024-02-16T22:17:49.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.907+0000 7fd85b7fe700 1 -- 172.21.15.114:0/1054707763 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fd864003a00 con 0x7fd86c100030 2024-02-16T22:17:49.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.908+0000 7fd85b7fe700 1 --2- 172.21.15.114:0/1054707763 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd85406c4e0 0x7fd85406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:49.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.908+0000 7fd85b7fe700 1 -- 172.21.15.114:0/1054707763 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fd86408b2a0 con 0x7fd86c100030 2024-02-16T22:17:49.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.908+0000 7fd869d9b700 1 --2- 172.21.15.114:0/1054707763 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd85406c4e0 0x7fd85406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:49.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.908+0000 7fd869d9b700 1 --2- 172.21.15.114:0/1054707763 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd85406c4e0 0x7fd85406e9a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd860006010 tx=0x7fd860005280 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:49.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:49.912+0000 7fd85b7fe700 1 -- 172.21.15.114:0/1054707763 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd864059850 con 0x7fd86c100030 2024-02-16T22:17:50.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:50.110+0000 7fd870f7f700 1 -- 172.21.15.114:0/1054707763 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd86c1056a0 con 0x7fd85406c4e0 2024-02-16T22:17:50.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:50.119+0000 7fd85b7fe700 1 -- 172.21.15.114:0/1054707763 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2820 (secure 0 0 0) 0x7fd86c1056a0 con 0x7fd85406c4e0 2024-02-16T22:17:50.121 INFO:teuthology.orchestra.run.smithi114.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:17:50.121 INFO:teuthology.orchestra.run.smithi114.stdout:alertmanager.smithi114 smithi114 *:9093,9094 running (3m) 2m ago 5m 25.5M - 0.25.0 c8568f914cd2 e7404fea400e 2024-02-16T22:17:50.121 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi114 smithi114 running (5m) 2m ago 5m 7075k - 18.2.1 94eadd990e0a 7fd07162cd96 2024-02-16T22:17:50.122 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi123 smithi123 running (4m) 64s ago 4m 6953k - 18.2.1 94eadd990e0a 7687f5e99621 2024-02-16T22:17:50.122 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi114 smithi114 running (5m) 2m ago 5m 6660k - 18.2.1 94eadd990e0a fc8fbfe4fdd5 2024-02-16T22:17:50.122 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi123 smithi123 running (4m) 64s ago 4m 6665k - 18.2.1 94eadd990e0a a6249642b9bb 2024-02-16T22:17:50.122 INFO:teuthology.orchestra.run.smithi114.stdout:grafana.smithi114 smithi114 *:3000 running (3m) 2m ago 4m 86.7M - 9.4.7 954c08fa6188 db1c26c3bc20 2024-02-16T22:17:50.122 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi114.vcullz smithi114 *:9283,8765,8443 running (6m) 2m ago 6m 481M - 18.2.1 94eadd990e0a c8f21783728d 2024-02-16T22:17:50.122 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi123.inzgjd smithi123 *:8443,9283,8765 running (4m) 64s ago 4m 442M - 18.2.1 94eadd990e0a 7f193d913aba 2024-02-16T22:17:50.122 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi114 smithi114 running (6m) 2m ago 6m 48.1M 2048M 18.2.1 94eadd990e0a b4c4433aa3a4 2024-02-16T22:17:50.122 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi123 smithi123 running (4m) 64s ago 4m 40.3M 2048M 18.2.1 94eadd990e0a c217fc30c152 2024-02-16T22:17:50.122 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi114 smithi114 *:9100 running (5m) 2m ago 5m 16.0M - 1.5.0 0da6a335fe13 744a090e06d0 2024-02-16T22:17:50.122 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi123 smithi123 *:9100 running (4m) 64s ago 4m 16.1M - 1.5.0 0da6a335fe13 3994b8c29b2e 2024-02-16T22:17:50.122 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 smithi114 running (3m) 2m ago 3m 40.4M 3946M 18.2.1 94eadd990e0a 24ca2e9714e0 2024-02-16T22:17:50.122 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 smithi114 running (2m) 2m ago 2m 40.9M 3946M 18.2.1 94eadd990e0a 32bb191b88e7 2024-02-16T22:17:50.122 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 smithi114 running (2m) 2m ago 2m 43.5M 3946M 18.2.1 94eadd990e0a 6608718d0818 2024-02-16T22:17:50.122 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 smithi123 running (2m) 64s ago 2m 43.2M 4970M 18.2.1 94eadd990e0a 575db379de3d 2024-02-16T22:17:50.122 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 smithi123 running (95s) 64s ago 94s 38.3M 4970M 18.2.1 94eadd990e0a 7e266866cd9c 2024-02-16T22:17:50.122 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 smithi123 running (69s) 64s ago 69s 10.8M 4970M 18.2.1 94eadd990e0a 24e5bef19fda 2024-02-16T22:17:50.123 INFO:teuthology.orchestra.run.smithi114.stdout:prometheus.smithi114 smithi114 *:9095 running (3m) 2m ago 4m 38.3M - 2.43.0 a07b618ecd1d 51185086d582 2024-02-16T22:17:50.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:50.121+0000 7fd870f7f700 1 -- 172.21.15.114:0/1054707763 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd85406c4e0 msgr2=0x7fd85406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:50.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:50.121+0000 7fd870f7f700 1 --2- 172.21.15.114:0/1054707763 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd85406c4e0 0x7fd85406e9a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd860006010 tx=0x7fd860005280 comp rx=0 tx=0).stop 2024-02-16T22:17:50.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:50.121+0000 7fd870f7f700 1 -- 172.21.15.114:0/1054707763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd86c100030 msgr2=0x7fd86c0ff710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:50.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:50.121+0000 7fd870f7f700 1 --2- 172.21.15.114:0/1054707763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd86c100030 0x7fd86c0ff710 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd86400cfd0 tx=0x7fd86400a340 comp rx=0 tx=0).stop 2024-02-16T22:17:50.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:50.121+0000 7fd870f7f700 1 -- 172.21.15.114:0/1054707763 shutdown_connections 2024-02-16T22:17:50.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:50.121+0000 7fd870f7f700 1 --2- 172.21.15.114:0/1054707763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd86c100030 0x7fd86c0ff710 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:50.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:50.121+0000 7fd870f7f700 1 --2- 172.21.15.114:0/1054707763 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd85406c4e0 0x7fd85406e9a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:50.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:50.121+0000 7fd870f7f700 1 --2- 172.21.15.114:0/1054707763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd86c100d50 0x7fd86c19b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:50.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:50.121+0000 7fd870f7f700 1 -- 172.21.15.114:0/1054707763 >> 172.21.15.114:0/1054707763 conn(0x7fd86c0fa810 msgr2=0x7fd86c103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:50.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:50.121+0000 7fd870f7f700 1 -- 172.21.15.114:0/1054707763 shutdown_connections 2024-02-16T22:17:50.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:50.121+0000 7fd870f7f700 1 -- 172.21.15.114:0/1054707763 wait complete. 2024-02-16T22:17:50.692 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:50 smithi114 ceph-mon[29235]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:50.713 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- bash -c 'ceph orch ls' 2024-02-16T22:17:50.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:50 smithi123 ceph-mon[37161]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:50.920 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:17:51.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:51 smithi114 ceph-mon[29235]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:17:51.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:51 smithi123 ceph-mon[37161]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:17:52.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.505+0000 7f9732998700 1 -- 172.21.15.114:0/2747548012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f972c100bc0 msgr2=0x7f972c100fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:52.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.505+0000 7f9732998700 1 --2- 172.21.15.114:0/2747548012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f972c100bc0 0x7f972c100fe0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f971c009910 tx=0x7f971c009c20 comp rx=0 tx=0).stop 2024-02-16T22:17:52.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.505+0000 7f9732998700 1 -- 172.21.15.114:0/2747548012 shutdown_connections 2024-02-16T22:17:52.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.505+0000 7f9732998700 1 --2- 172.21.15.114:0/2747548012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f972c101dc0 0x7f972c102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:52.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.505+0000 7f9732998700 1 --2- 172.21.15.114:0/2747548012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f972c100bc0 0x7f972c100fe0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:52.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.505+0000 7f9732998700 1 -- 172.21.15.114:0/2747548012 >> 172.21.15.114:0/2747548012 conn(0x7f972c0fc140 msgr2=0x7f972c0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:52.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.505+0000 7f9732998700 1 -- 172.21.15.114:0/2747548012 shutdown_connections 2024-02-16T22:17:52.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.505+0000 7f9732998700 1 -- 172.21.15.114:0/2747548012 wait complete. 2024-02-16T22:17:52.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.506+0000 7f9732998700 1 Processor -- start 2024-02-16T22:17:52.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.506+0000 7f9732998700 1 -- start start 2024-02-16T22:17:52.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.506+0000 7f9732998700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f972c100bc0 0x7f972c1963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:52.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.506+0000 7f9732998700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f972c101dc0 0x7f972c196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:52.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.506+0000 7f9732998700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c196f20 con 0x7f972c101dc0 2024-02-16T22:17:52.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.506+0000 7f9732998700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c197060 con 0x7f972c100bc0 2024-02-16T22:17:52.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.506+0000 7f972b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f972c101dc0 0x7f972c196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:52.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.506+0000 7f972b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f972c101dc0 0x7f972c196900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38558/0 (socket says 172.21.15.114:38558) 2024-02-16T22:17:52.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.506+0000 7f972b7fe700 1 -- 172.21.15.114:0/2786424029 learned_addr learned my addr 172.21.15.114:0/2786424029 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:17:52.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.506+0000 7f972bfff700 1 --2- 172.21.15.114:0/2786424029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f972c100bc0 0x7f972c1963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:52.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.507+0000 7f972b7fe700 1 -- 172.21.15.114:0/2786424029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f972c100bc0 msgr2=0x7f972c1963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:52.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.507+0000 7f972b7fe700 1 --2- 172.21.15.114:0/2786424029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f972c100bc0 0x7f972c1963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:52.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.507+0000 7f972b7fe700 1 -- 172.21.15.114:0/2786424029 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f971c0095f0 con 0x7f972c101dc0 2024-02-16T22:17:52.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.507+0000 7f972b7fe700 1 --2- 172.21.15.114:0/2786424029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f972c101dc0 0x7f972c196900 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f972000d900 tx=0x7f972000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:52.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.507+0000 7f97297fa700 1 -- 172.21.15.114:0/2786424029 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9720004a00 con 0x7f972c101dc0 2024-02-16T22:17:52.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.507+0000 7f9732998700 1 -- 172.21.15.114:0/2786424029 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f972c19bb10 con 0x7f972c101dc0 2024-02-16T22:17:52.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.507+0000 7f97297fa700 1 -- 172.21.15.114:0/2786424029 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f97200054d0 con 0x7f972c101dc0 2024-02-16T22:17:52.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.507+0000 7f97297fa700 1 -- 172.21.15.114:0/2786424029 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9720009d00 con 0x7f972c101dc0 2024-02-16T22:17:52.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.507+0000 7f9732998700 1 -- 172.21.15.114:0/2786424029 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f972c19c060 con 0x7f972c101dc0 2024-02-16T22:17:52.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.508+0000 7f9732998700 1 -- 172.21.15.114:0/2786424029 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f972c109740 con 0x7f972c101dc0 2024-02-16T22:17:52.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.508+0000 7f97297fa700 1 -- 172.21.15.114:0/2786424029 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f9720005020 con 0x7f972c101dc0 2024-02-16T22:17:52.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.509+0000 7f97297fa700 1 --2- 172.21.15.114:0/2786424029 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f971406c4e0 0x7f971406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:52.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.509+0000 7f972bfff700 1 --2- 172.21.15.114:0/2786424029 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f971406c4e0 0x7f971406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:52.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.509+0000 7f97297fa700 1 -- 172.21.15.114:0/2786424029 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f972008b3f0 con 0x7f972c101dc0 2024-02-16T22:17:52.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.509+0000 7f972bfff700 1 --2- 172.21.15.114:0/2786424029 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f971406c4e0 0x7f971406e9a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f971c006010 tx=0x7f971c00b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:52.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.513+0000 7f97297fa700 1 -- 172.21.15.114:0/2786424029 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f97200599a0 con 0x7f972c101dc0 2024-02-16T22:17:52.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:52 smithi114 ceph-mon[29235]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:52.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.713+0000 7f9732998700 1 -- 172.21.15.114:0/2786424029 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f972c106710 con 0x7f971406c4e0 2024-02-16T22:17:52.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.717+0000 7f97297fa700 1 -- 172.21.15.114:0/2786424029 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1310 (secure 0 0 0) 0x7f972c106710 con 0x7f971406c4e0 2024-02-16T22:17:52.719 INFO:teuthology.orchestra.run.smithi114.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-02-16T22:17:52.719 INFO:teuthology.orchestra.run.smithi114.stdout:alertmanager ?:9093,9094 1/1 2m ago 6m count:1 2024-02-16T22:17:52.719 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter 2/2 2m ago 6m * 2024-02-16T22:17:52.720 INFO:teuthology.orchestra.run.smithi114.stdout:crash 2/2 2m ago 6m * 2024-02-16T22:17:52.720 INFO:teuthology.orchestra.run.smithi114.stdout:grafana ?:3000 1/1 2m ago 6m count:1 2024-02-16T22:17:52.720 INFO:teuthology.orchestra.run.smithi114.stdout:mgr 2/2 2m ago 6m count:2 2024-02-16T22:17:52.720 INFO:teuthology.orchestra.run.smithi114.stdout:mon 2/2 2m ago 5m smithi114:172.21.15.114=smithi114;smithi123:172.21.15.123=smithi123;count:2 2024-02-16T22:17:52.720 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter ?:9100 2/2 2m ago 6m * 2024-02-16T22:17:52.720 INFO:teuthology.orchestra.run.smithi114.stdout:osd 6 2m ago - 2024-02-16T22:17:52.720 INFO:teuthology.orchestra.run.smithi114.stdout:prometheus ?:9095 1/1 2m ago 6m count:1 2024-02-16T22:17:52.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.719+0000 7f9732998700 1 -- 172.21.15.114:0/2786424029 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f971406c4e0 msgr2=0x7f971406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:52.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.719+0000 7f9732998700 1 --2- 172.21.15.114:0/2786424029 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f971406c4e0 0x7f971406e9a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f971c006010 tx=0x7f971c00b540 comp rx=0 tx=0).stop 2024-02-16T22:17:52.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.719+0000 7f9732998700 1 -- 172.21.15.114:0/2786424029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f972c101dc0 msgr2=0x7f972c196900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:52.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.719+0000 7f9732998700 1 --2- 172.21.15.114:0/2786424029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f972c101dc0 0x7f972c196900 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f972000d900 tx=0x7f972000dc10 comp rx=0 tx=0).stop 2024-02-16T22:17:52.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.719+0000 7f9732998700 1 -- 172.21.15.114:0/2786424029 shutdown_connections 2024-02-16T22:17:52.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.719+0000 7f9732998700 1 --2- 172.21.15.114:0/2786424029 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f971406c4e0 0x7f971406e9a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:52.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.719+0000 7f9732998700 1 --2- 172.21.15.114:0/2786424029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f972c100bc0 0x7f972c1963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:52.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.719+0000 7f9732998700 1 --2- 172.21.15.114:0/2786424029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f972c101dc0 0x7f972c196900 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:52.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.719+0000 7f9732998700 1 -- 172.21.15.114:0/2786424029 >> 172.21.15.114:0/2786424029 conn(0x7f972c0fc140 msgr2=0x7f972c104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:52.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.720+0000 7f9732998700 1 -- 172.21.15.114:0/2786424029 shutdown_connections 2024-02-16T22:17:52.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:52.720+0000 7f9732998700 1 -- 172.21.15.114:0/2786424029 wait complete. 2024-02-16T22:17:52.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:52 smithi123 ceph-mon[37161]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:53.360 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- bash -c 'ceph orch host ls' 2024-02-16T22:17:53.566 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:17:53.689 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:53 smithi114 ceph-mon[29235]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:17:53.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:53 smithi123 ceph-mon[37161]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:17:54.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:54 smithi123 ceph-mon[37161]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:54.838 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:54 smithi114 ceph-mon[29235]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:55.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.269+0000 7ff495857700 1 -- 172.21.15.114:0/3261160772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff490100cb0 msgr2=0x7ff490101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:55.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.269+0000 7ff495857700 1 --2- 172.21.15.114:0/3261160772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff490100cb0 0x7ff490101110 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff484009910 tx=0x7ff484009c20 comp rx=0 tx=0).stop 2024-02-16T22:17:55.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.269+0000 7ff495857700 1 -- 172.21.15.114:0/3261160772 shutdown_connections 2024-02-16T22:17:55.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.269+0000 7ff495857700 1 --2- 172.21.15.114:0/3261160772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff490100cb0 0x7ff490101110 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:55.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.269+0000 7ff495857700 1 --2- 172.21.15.114:0/3261160772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4900ffab0 0x7ff4900ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:55.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.269+0000 7ff495857700 1 -- 172.21.15.114:0/3261160772 >> 172.21.15.114:0/3261160772 conn(0x7ff4900fb030 msgr2=0x7ff4900fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:55.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.269+0000 7ff495857700 1 -- 172.21.15.114:0/3261160772 shutdown_connections 2024-02-16T22:17:55.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.269+0000 7ff495857700 1 -- 172.21.15.114:0/3261160772 wait complete. 2024-02-16T22:17:55.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.270+0000 7ff495857700 1 Processor -- start 2024-02-16T22:17:55.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.270+0000 7ff495857700 1 -- start start 2024-02-16T22:17:55.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.270+0000 7ff495857700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4900ffab0 0x7ff490196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:55.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.270+0000 7ff495857700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff490100cb0 0x7ff4901968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:55.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.270+0000 7ff495857700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff490196ec0 con 0x7ff490100cb0 2024-02-16T22:17:55.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.270+0000 7ff495857700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff490197000 con 0x7ff4900ffab0 2024-02-16T22:17:55.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.270+0000 7ff48e7fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff490100cb0 0x7ff4901968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:55.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.270+0000 7ff48e7fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff490100cb0 0x7ff4901968a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38568/0 (socket says 172.21.15.114:38568) 2024-02-16T22:17:55.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.270+0000 7ff48e7fc700 1 -- 172.21.15.114:0/2132964861 learned_addr learned my addr 172.21.15.114:0/2132964861 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:17:55.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.271+0000 7ff48effd700 1 --2- 172.21.15.114:0/2132964861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4900ffab0 0x7ff490196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:55.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.271+0000 7ff48e7fc700 1 -- 172.21.15.114:0/2132964861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4900ffab0 msgr2=0x7ff490196360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:55.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.271+0000 7ff48e7fc700 1 --2- 172.21.15.114:0/2132964861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4900ffab0 0x7ff490196360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:55.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.271+0000 7ff48e7fc700 1 -- 172.21.15.114:0/2132964861 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4840095f0 con 0x7ff490100cb0 2024-02-16T22:17:55.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.271+0000 7ff48e7fc700 1 --2- 172.21.15.114:0/2132964861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff490100cb0 0x7ff4901968a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff4840092e0 tx=0x7ff48400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:55.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.271+0000 7ff494855700 1 -- 172.21.15.114:0/2132964861 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff48401d070 con 0x7ff490100cb0 2024-02-16T22:17:55.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.271+0000 7ff494855700 1 -- 172.21.15.114:0/2132964861 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff48400f460 con 0x7ff490100cb0 2024-02-16T22:17:55.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.271+0000 7ff495857700 1 -- 172.21.15.114:0/2132964861 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff49019ba50 con 0x7ff490100cb0 2024-02-16T22:17:55.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.271+0000 7ff494855700 1 -- 172.21.15.114:0/2132964861 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff484021620 con 0x7ff490100cb0 2024-02-16T22:17:55.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.271+0000 7ff495857700 1 -- 172.21.15.114:0/2132964861 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff49019bfc0 con 0x7ff490100cb0 2024-02-16T22:17:55.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.272+0000 7ff495857700 1 -- 172.21.15.114:0/2132964861 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff490109710 con 0x7ff490100cb0 2024-02-16T22:17:55.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.272+0000 7ff494855700 1 -- 172.21.15.114:0/2132964861 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7ff48400fa90 con 0x7ff490100cb0 2024-02-16T22:17:55.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.273+0000 7ff494855700 1 --2- 172.21.15.114:0/2132964861 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff47806c4e0 0x7ff47806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:55.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.273+0000 7ff48effd700 1 --2- 172.21.15.114:0/2132964861 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff47806c4e0 0x7ff47806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:55.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.273+0000 7ff494855700 1 -- 172.21.15.114:0/2132964861 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff48408c870 con 0x7ff490100cb0 2024-02-16T22:17:55.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.273+0000 7ff48effd700 1 --2- 172.21.15.114:0/2132964861 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff47806c4e0 0x7ff47806e9a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff490100b10 tx=0x7ff48000a380 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:55.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.277+0000 7ff494855700 1 -- 172.21.15.114:0/2132964861 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff48405ae20 con 0x7ff490100cb0 2024-02-16T22:17:55.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.474+0000 7ff495857700 1 -- 172.21.15.114:0/2132964861 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff490105600 con 0x7ff47806c4e0 2024-02-16T22:17:55.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.476+0000 7ff494855700 1 -- 172.21.15.114:0/2132964861 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7ff490105600 con 0x7ff47806c4e0 2024-02-16T22:17:55.478 INFO:teuthology.orchestra.run.smithi114.stdout:HOST ADDR LABELS STATUS 2024-02-16T22:17:55.478 INFO:teuthology.orchestra.run.smithi114.stdout:smithi114 172.21.15.114 2024-02-16T22:17:55.478 INFO:teuthology.orchestra.run.smithi114.stdout:smithi123 172.21.15.123 2024-02-16T22:17:55.478 INFO:teuthology.orchestra.run.smithi114.stdout:2 hosts in cluster 2024-02-16T22:17:55.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.478+0000 7ff495857700 1 -- 172.21.15.114:0/2132964861 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff47806c4e0 msgr2=0x7ff47806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:55.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.478+0000 7ff495857700 1 --2- 172.21.15.114:0/2132964861 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff47806c4e0 0x7ff47806e9a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff490100b10 tx=0x7ff48000a380 comp rx=0 tx=0).stop 2024-02-16T22:17:55.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.478+0000 7ff495857700 1 -- 172.21.15.114:0/2132964861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff490100cb0 msgr2=0x7ff4901968a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:55.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.478+0000 7ff495857700 1 --2- 172.21.15.114:0/2132964861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff490100cb0 0x7ff4901968a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff4840092e0 tx=0x7ff48400bd70 comp rx=0 tx=0).stop 2024-02-16T22:17:55.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.478+0000 7ff495857700 1 -- 172.21.15.114:0/2132964861 shutdown_connections 2024-02-16T22:17:55.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.478+0000 7ff495857700 1 --2- 172.21.15.114:0/2132964861 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff47806c4e0 0x7ff47806e9a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:55.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.478+0000 7ff495857700 1 --2- 172.21.15.114:0/2132964861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4900ffab0 0x7ff490196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:55.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.478+0000 7ff495857700 1 --2- 172.21.15.114:0/2132964861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff490100cb0 0x7ff4901968a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:55.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.478+0000 7ff495857700 1 -- 172.21.15.114:0/2132964861 >> 172.21.15.114:0/2132964861 conn(0x7ff4900fb030 msgr2=0x7ff490103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:55.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.478+0000 7ff495857700 1 -- 172.21.15.114:0/2132964861 shutdown_connections 2024-02-16T22:17:55.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:55.478+0000 7ff495857700 1 -- 172.21.15.114:0/2132964861 wait complete. 2024-02-16T22:17:56.066 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- bash -c 'ceph orch device ls' 2024-02-16T22:17:56.270 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:17:56.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:56 smithi114 ceph-mon[29235]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:56.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:56 smithi114 ceph-mon[29235]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:17:56.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:56 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:17:56.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:56 smithi123 ceph-mon[37161]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:56.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:56 smithi123 ceph-mon[37161]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:17:56.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:56 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:17:57.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.839+0000 7f24981cd700 1 -- 172.21.15.114:0/973516043 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24900ffab0 msgr2=0x7f24900ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:57.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.839+0000 7f24981cd700 1 --2- 172.21.15.114:0/973516043 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24900ffab0 0x7f24900ffed0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f248c009910 tx=0x7f248c009c20 comp rx=0 tx=0).stop 2024-02-16T22:17:57.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.839+0000 7f24981cd700 1 -- 172.21.15.114:0/973516043 shutdown_connections 2024-02-16T22:17:57.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.839+0000 7f24981cd700 1 --2- 172.21.15.114:0/973516043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2490100cb0 0x7f2490101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:57.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.839+0000 7f24981cd700 1 --2- 172.21.15.114:0/973516043 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24900ffab0 0x7f24900ffed0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:57.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.839+0000 7f24981cd700 1 -- 172.21.15.114:0/973516043 >> 172.21.15.114:0/973516043 conn(0x7f24900fb030 msgr2=0x7f24900fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:57.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.839+0000 7f24981cd700 1 -- 172.21.15.114:0/973516043 shutdown_connections 2024-02-16T22:17:57.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.839+0000 7f24981cd700 1 -- 172.21.15.114:0/973516043 wait complete. 2024-02-16T22:17:57.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.839+0000 7f24981cd700 1 Processor -- start 2024-02-16T22:17:57.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.839+0000 7f24981cd700 1 -- start start 2024-02-16T22:17:57.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.840+0000 7f24981cd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24900ffab0 0x7f2490196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:57.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.840+0000 7f24981cd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2490100cb0 0x7f24901969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:57.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.840+0000 7f24981cd700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2490196fe0 con 0x7f24900ffab0 2024-02-16T22:17:57.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.840+0000 7f24981cd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2490197120 con 0x7f2490100cb0 2024-02-16T22:17:57.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.840+0000 7f2495f69700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24900ffab0 0x7f2490196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:57.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.840+0000 7f2495f69700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24900ffab0 0x7f2490196480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38586/0 (socket says 172.21.15.114:38586) 2024-02-16T22:17:57.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.840+0000 7f2495f69700 1 -- 172.21.15.114:0/1620435995 learned_addr learned my addr 172.21.15.114:0/1620435995 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:17:57.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.840+0000 7f2495f69700 1 -- 172.21.15.114:0/1620435995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2490100cb0 msgr2=0x7f24901969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:57.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.840+0000 7f2495f69700 1 --2- 172.21.15.114:0/1620435995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2490100cb0 0x7f24901969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:57.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.840+0000 7f2495f69700 1 -- 172.21.15.114:0/1620435995 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248c0095f0 con 0x7f24900ffab0 2024-02-16T22:17:57.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.840+0000 7f2495f69700 1 --2- 172.21.15.114:0/1620435995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24900ffab0 0x7f2490196480 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f248c00bd70 tx=0x7f248c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:57.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.840+0000 7f2486ffd700 1 -- 172.21.15.114:0/1620435995 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f248c01d070 con 0x7f24900ffab0 2024-02-16T22:17:57.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.840+0000 7f24981cd700 1 -- 172.21.15.114:0/1620435995 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f249019bb70 con 0x7f24900ffab0 2024-02-16T22:17:57.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.840+0000 7f2486ffd700 1 -- 172.21.15.114:0/1620435995 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f248c022470 con 0x7f24900ffab0 2024-02-16T22:17:57.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.840+0000 7f2486ffd700 1 -- 172.21.15.114:0/1620435995 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f248c00f690 con 0x7f24900ffab0 2024-02-16T22:17:57.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.840+0000 7f24981cd700 1 -- 172.21.15.114:0/1620435995 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f249019c0e0 con 0x7f24900ffab0 2024-02-16T22:17:57.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.841+0000 7f24981cd700 1 -- 172.21.15.114:0/1620435995 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2490108690 con 0x7f24900ffab0 2024-02-16T22:17:57.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.842+0000 7f2486ffd700 1 -- 172.21.15.114:0/1620435995 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f248c0225e0 con 0x7f24900ffab0 2024-02-16T22:17:57.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.842+0000 7f2486ffd700 1 --2- 172.21.15.114:0/1620435995 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f247c06c4e0 0x7f247c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:17:57.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.843+0000 7f2486ffd700 1 -- 172.21.15.114:0/1620435995 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f248c08cca0 con 0x7f24900ffab0 2024-02-16T22:17:57.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.843+0000 7f2495768700 1 --2- 172.21.15.114:0/1620435995 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f247c06c4e0 0x7f247c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:17:57.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.843+0000 7f2495768700 1 --2- 172.21.15.114:0/1620435995 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f247c06c4e0 0x7f247c06e9a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f24800097b0 tx=0x7f2480006d20 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:17:57.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:57.847+0000 7f2486ffd700 1 -- 172.21.15.114:0/1620435995 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f248c091070 con 0x7f24900ffab0 2024-02-16T22:17:57.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:57 smithi114 ceph-mon[29235]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:58.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:58.051+0000 7f24981cd700 1 -- 172.21.15.114:0/1620435995 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f2490068b20 con 0x7f247c06c4e0 2024-02-16T22:17:58.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:58.053+0000 7f2486ffd700 1 -- 172.21.15.114:0/1620435995 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7f2490068b20 con 0x7f247c06c4e0 2024-02-16T22:17:58.055 INFO:teuthology.orchestra.run.smithi114.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-02-16T22:17:58.055 INFO:teuthology.orchestra.run.smithi114.stdout:smithi114 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400WB400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-02-16T22:17:58.055 INFO:teuthology.orchestra.run.smithi114.stdout:smithi123 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400YK400BGN 372G No 68s ago Has a FileSystem, LVM detected 2024-02-16T22:17:58.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:58.055+0000 7f24981cd700 1 -- 172.21.15.114:0/1620435995 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f247c06c4e0 msgr2=0x7f247c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:58.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:58.055+0000 7f24981cd700 1 --2- 172.21.15.114:0/1620435995 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f247c06c4e0 0x7f247c06e9a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f24800097b0 tx=0x7f2480006d20 comp rx=0 tx=0).stop 2024-02-16T22:17:58.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:58.055+0000 7f24981cd700 1 -- 172.21.15.114:0/1620435995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24900ffab0 msgr2=0x7f2490196480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:17:58.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:58.055+0000 7f24981cd700 1 --2- 172.21.15.114:0/1620435995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24900ffab0 0x7f2490196480 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f248c00bd70 tx=0x7f248c00be50 comp rx=0 tx=0).stop 2024-02-16T22:17:58.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:58.055+0000 7f24981cd700 1 -- 172.21.15.114:0/1620435995 shutdown_connections 2024-02-16T22:17:58.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:58.055+0000 7f24981cd700 1 --2- 172.21.15.114:0/1620435995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24900ffab0 0x7f2490196480 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:58.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:58.055+0000 7f24981cd700 1 --2- 172.21.15.114:0/1620435995 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f247c06c4e0 0x7f247c06e9a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:58.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:58.055+0000 7f24981cd700 1 --2- 172.21.15.114:0/1620435995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2490100cb0 0x7f24901969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:17:58.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:58.055+0000 7f24981cd700 1 -- 172.21.15.114:0/1620435995 >> 172.21.15.114:0/1620435995 conn(0x7f24900fb030 msgr2=0x7f2490068410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:17:58.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:58.055+0000 7f24981cd700 1 -- 172.21.15.114:0/1620435995 shutdown_connections 2024-02-16T22:17:58.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:17:58.055+0000 7f24981cd700 1 -- 172.21.15.114:0/1620435995 wait complete. 2024-02-16T22:17:58.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:57 smithi123 ceph-mon[37161]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:17:58.727 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-02-16T22:17:58.737 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T22:17:58.745 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi114.front.sepia.ceph.com 2024-02-16T22:17:58.745 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-02-16T22:17:58.950 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:17:58.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:58 smithi114 ceph-mon[29235]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:17:59.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:58 smithi123 ceph-mon[37161]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:18:00.187 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:17:59 smithi114 ceph-mon[29235]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:18:00.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:17:59 smithi123 ceph-mon[37161]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:18:00.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.581+0000 7f01fac86700 1 -- 172.21.15.114:0/2472725217 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01f4100be0 msgr2=0x7f01f4101000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:00.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.581+0000 7f01fac86700 1 --2- 172.21.15.114:0/2472725217 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01f4100be0 0x7f01f4101000 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00b0a0 tx=0x7f01ec00b3b0 comp rx=0 tx=0).stop 2024-02-16T22:18:00.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.581+0000 7f01fac86700 1 -- 172.21.15.114:0/2472725217 shutdown_connections 2024-02-16T22:18:00.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.581+0000 7f01fac86700 1 --2- 172.21.15.114:0/2472725217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01f4101de0 0x7f01f4102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:00.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.581+0000 7f01fac86700 1 --2- 172.21.15.114:0/2472725217 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01f4100be0 0x7f01f4101000 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:00.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.581+0000 7f01fac86700 1 -- 172.21.15.114:0/2472725217 >> 172.21.15.114:0/2472725217 conn(0x7f01f40fc140 msgr2=0x7f01f40fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:00.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.581+0000 7f01fac86700 1 -- 172.21.15.114:0/2472725217 shutdown_connections 2024-02-16T22:18:00.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.581+0000 7f01fac86700 1 -- 172.21.15.114:0/2472725217 wait complete. 2024-02-16T22:18:00.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.582+0000 7f01fac86700 1 Processor -- start 2024-02-16T22:18:00.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.582+0000 7f01fac86700 1 -- start start 2024-02-16T22:18:00.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.582+0000 7f01fac86700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01f4100be0 0x7f01f4074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:00.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.582+0000 7f01fac86700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01f4101de0 0x7f01f4073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:00.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.582+0000 7f01fac86700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f40736c0 con 0x7f01f4101de0 2024-02-16T22:18:00.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.582+0000 7f01fac86700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f4073800 con 0x7f01f4100be0 2024-02-16T22:18:00.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.582+0000 7f01f3fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01f4101de0 0x7f01f4073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:00.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.582+0000 7f01f3fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01f4101de0 0x7f01f4073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38598/0 (socket says 172.21.15.114:38598) 2024-02-16T22:18:00.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.582+0000 7f01f3fff700 1 -- 172.21.15.114:0/2628100347 learned_addr learned my addr 172.21.15.114:0/2628100347 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:18:00.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.582+0000 7f01f8a22700 1 --2- 172.21.15.114:0/2628100347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01f4100be0 0x7f01f4074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:00.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.582+0000 7f01f3fff700 1 -- 172.21.15.114:0/2628100347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01f4100be0 msgr2=0x7f01f4074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:00.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.582+0000 7f01f3fff700 1 --2- 172.21.15.114:0/2628100347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01f4100be0 0x7f01f4074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:00.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.582+0000 7f01f3fff700 1 -- 172.21.15.114:0/2628100347 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec009d00 con 0x7f01f4101de0 2024-02-16T22:18:00.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.583+0000 7f01f3fff700 1 --2- 172.21.15.114:0/2628100347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01f4101de0 0x7f01f4073180 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f01e800c930 tx=0x7f01e800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:00.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.583+0000 7f01f1ffb700 1 -- 172.21.15.114:0/2628100347 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01e8007ad0 con 0x7f01f4101de0 2024-02-16T22:18:00.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.583+0000 7f01f1ffb700 1 -- 172.21.15.114:0/2628100347 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f01e800ce80 con 0x7f01f4101de0 2024-02-16T22:18:00.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.583+0000 7f01fac86700 1 -- 172.21.15.114:0/2628100347 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f4073ae0 con 0x7f01f4101de0 2024-02-16T22:18:00.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.583+0000 7f01f1ffb700 1 -- 172.21.15.114:0/2628100347 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01e8018630 con 0x7f01f4101de0 2024-02-16T22:18:00.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.583+0000 7f01fac86700 1 -- 172.21.15.114:0/2628100347 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f419e3f0 con 0x7f01f4101de0 2024-02-16T22:18:00.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.584+0000 7f01fac86700 1 -- 172.21.15.114:0/2628100347 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f4109740 con 0x7f01f4101de0 2024-02-16T22:18:00.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.584+0000 7f01f1ffb700 1 -- 172.21.15.114:0/2628100347 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f01e8007c30 con 0x7f01f4101de0 2024-02-16T22:18:00.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.585+0000 7f01f1ffb700 1 --2- 172.21.15.114:0/2628100347 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f01dc06c4e0 0x7f01dc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:00.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.585+0000 7f01f8a22700 1 --2- 172.21.15.114:0/2628100347 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f01dc06c4e0 0x7f01dc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:00.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.585+0000 7f01f1ffb700 1 -- 172.21.15.114:0/2628100347 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f01e808b470 con 0x7f01f4101de0 2024-02-16T22:18:00.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.585+0000 7f01f8a22700 1 --2- 172.21.15.114:0/2628100347 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f01dc06c4e0 0x7f01dc06e9a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009fd0 tx=0x7f01ec0060c0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:00.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.589+0000 7f01f1ffb700 1 -- 172.21.15.114:0/2628100347 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f01e8059a20 con 0x7f01f4101de0 2024-02-16T22:18:00.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:00.803+0000 7f01fac86700 1 -- 172.21.15.114:0/2628100347 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f01f4106730 con 0x7f01dc06c4e0 2024-02-16T22:18:01.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:01 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-02-16T22:18:01.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:01 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-02-16T22:18:02.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:02.127+0000 7f01f1ffb700 1 -- 172.21.15.114:0/2628100347 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f01f4106730 con 0x7f01dc06c4e0 2024-02-16T22:18:02.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:02.131+0000 7f01fac86700 1 -- 172.21.15.114:0/2628100347 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f01dc06c4e0 msgr2=0x7f01dc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:02.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:02.131+0000 7f01fac86700 1 --2- 172.21.15.114:0/2628100347 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f01dc06c4e0 0x7f01dc06e9a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009fd0 tx=0x7f01ec0060c0 comp rx=0 tx=0).stop 2024-02-16T22:18:02.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:02.131+0000 7f01fac86700 1 -- 172.21.15.114:0/2628100347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01f4101de0 msgr2=0x7f01f4073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:02.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:02.131+0000 7f01fac86700 1 --2- 172.21.15.114:0/2628100347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01f4101de0 0x7f01f4073180 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f01e800c930 tx=0x7f01e800cc40 comp rx=0 tx=0).stop 2024-02-16T22:18:02.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:02.131+0000 7f01fac86700 1 -- 172.21.15.114:0/2628100347 shutdown_connections 2024-02-16T22:18:02.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:02.131+0000 7f01fac86700 1 --2- 172.21.15.114:0/2628100347 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f01dc06c4e0 0x7f01dc06e9a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:02.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:02.131+0000 7f01fac86700 1 --2- 172.21.15.114:0/2628100347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01f4100be0 0x7f01f4074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:02.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:02.131+0000 7f01fac86700 1 --2- 172.21.15.114:0/2628100347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01f4101de0 0x7f01f4073180 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:02.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:02.131+0000 7f01fac86700 1 -- 172.21.15.114:0/2628100347 >> 172.21.15.114:0/2628100347 conn(0x7f01f40fc140 msgr2=0x7f01f4105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:02.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:02.131+0000 7f01fac86700 1 -- 172.21.15.114:0/2628100347 shutdown_connections 2024-02-16T22:18:02.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:02.132+0000 7f01fac86700 1 -- 172.21.15.114:0/2628100347 wait complete. 2024-02-16T22:18:02.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:02 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi114[29231]: 2024-02-16T22:18:02.107+0000 7fcd9ecfb700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T22:18:02.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:02 smithi114 ceph-mon[29235]: from='client.14464 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:18:02.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:02 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-02-16T22:18:02.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:02 smithi114 ceph-mon[29235]: osdmap e41: 6 total, 6 up, 6 in 2024-02-16T22:18:02.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:02 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-02-16T22:18:02.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:02 smithi114 ceph-mon[29235]: pgmap v174: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:18:02.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:02 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-02-16T22:18:02.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:02 smithi114 ceph-mon[29235]: osdmap e42: 6 total, 6 up, 6 in 2024-02-16T22:18:02.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:02 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-02-16T22:18:02.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:02 smithi114 ceph-mon[29235]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T22:18:02.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:02 smithi114 ceph-mon[29235]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-16T22:18:02.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:02 smithi114 ceph-mon[29235]: osdmap e43: 6 total, 6 up, 6 in 2024-02-16T22:18:02.453 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:02 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-02-16T22:18:02.453 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:02 smithi114 ceph-mon[29235]: fsmap cephfs:0 2024-02-16T22:18:02.453 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:02 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:02.453 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:02 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:18:02.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:02 smithi123 ceph-mon[37161]: from='client.14464 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:18:02.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:02 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-02-16T22:18:02.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:02 smithi123 ceph-mon[37161]: osdmap e41: 6 total, 6 up, 6 in 2024-02-16T22:18:02.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:02 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-02-16T22:18:02.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:02 smithi123 ceph-mon[37161]: pgmap v174: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:18:02.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:02 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-02-16T22:18:02.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:02 smithi123 ceph-mon[37161]: osdmap e42: 6 total, 6 up, 6 in 2024-02-16T22:18:02.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:02 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-02-16T22:18:02.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:02 smithi123 ceph-mon[37161]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T22:18:02.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:02 smithi123 ceph-mon[37161]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-16T22:18:02.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:02 smithi123 ceph-mon[37161]: osdmap e43: 6 total, 6 up, 6 in 2024-02-16T22:18:02.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:02 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-02-16T22:18:02.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:02 smithi123 ceph-mon[37161]: fsmap cephfs:0 2024-02-16T22:18:02.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:02 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:02.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:02 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:18:02.889 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-16T22:18:03.581 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:03 smithi114 ceph-mon[29235]: Saving service mds.cephfs spec with placement count:4 2024-02-16T22:18:03.581 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:03 smithi114 ceph-mon[29235]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-16T22:18:03.581 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:03 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:18:03.582 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:03 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:18:03.582 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:03 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:03.582 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:03 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi114.zzryke", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:18:03.582 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:03 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi114.zzryke", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T22:18:03.582 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:03 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:18:03.582 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:03 smithi114 ceph-mon[29235]: Deploying daemon mds.cephfs.smithi114.zzryke on smithi114 2024-02-16T22:18:03.582 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:03 smithi114 ceph-mon[29235]: osdmap e44: 6 total, 6 up, 6 in 2024-02-16T22:18:03.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:03 smithi123 ceph-mon[37161]: Saving service mds.cephfs spec with placement count:4 2024-02-16T22:18:03.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:03 smithi123 ceph-mon[37161]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-16T22:18:03.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:03 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:18:03.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:03 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:18:03.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:03 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:03.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:03 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi114.zzryke", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:18:03.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:03 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi114.zzryke", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T22:18:03.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:03 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:18:03.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:03 smithi123 ceph-mon[37161]: Deploying daemon mds.cephfs.smithi114.zzryke on smithi114 2024-02-16T22:18:03.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:03 smithi123 ceph-mon[37161]: osdmap e44: 6 total, 6 up, 6 in 2024-02-16T22:18:04.173 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:18:04.663 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:04 smithi114 ceph-mon[29235]: pgmap v178: 65 pgs: 5 creating+peering, 19 active+clean, 41 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:18:04.663 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:04 smithi114 ceph-mon[29235]: osdmap e45: 6 total, 6 up, 6 in 2024-02-16T22:18:04.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:04 smithi123 ceph-mon[37161]: pgmap v178: 65 pgs: 5 creating+peering, 19 active+clean, 41 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:18:04.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:04 smithi123 ceph-mon[37161]: osdmap e45: 6 total, 6 up, 6 in 2024-02-16T22:18:05.600 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:05 smithi114 ceph-mon[29235]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-16T22:18:05.648 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:05 smithi123 ceph-mon[37161]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-16T22:18:06.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.272+0000 7f62d65b7700 1 -- 172.21.15.114:0/2421592477 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62d00ffac0 msgr2=0x7f62d00ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:06.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.272+0000 7f62d65b7700 1 --2- 172.21.15.114:0/2421592477 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62d00ffac0 0x7f62d00ffee0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009910 tx=0x7f62c0009c20 comp rx=0 tx=0).stop 2024-02-16T22:18:06.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.272+0000 7f62d65b7700 1 -- 172.21.15.114:0/2421592477 shutdown_connections 2024-02-16T22:18:06.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.272+0000 7f62d65b7700 1 --2- 172.21.15.114:0/2421592477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62d0100cc0 0x7f62d0101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:06.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.272+0000 7f62d65b7700 1 --2- 172.21.15.114:0/2421592477 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62d00ffac0 0x7f62d00ffee0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:06.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.272+0000 7f62d65b7700 1 -- 172.21.15.114:0/2421592477 >> 172.21.15.114:0/2421592477 conn(0x7f62d00fb060 msgr2=0x7f62d00fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:06.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.272+0000 7f62d65b7700 1 -- 172.21.15.114:0/2421592477 shutdown_connections 2024-02-16T22:18:06.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.272+0000 7f62d65b7700 1 -- 172.21.15.114:0/2421592477 wait complete. 2024-02-16T22:18:06.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.272+0000 7f62d65b7700 1 Processor -- start 2024-02-16T22:18:06.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.272+0000 7f62d65b7700 1 -- start start 2024-02-16T22:18:06.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.273+0000 7f62d65b7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62d00ffac0 0x7f62d01952a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:06.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.273+0000 7f62d65b7700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62d0100cc0 0x7f62d01957e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:06.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.273+0000 7f62d65b7700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d0195e00 con 0x7f62d0100cc0 2024-02-16T22:18:06.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.273+0000 7f62d65b7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d0195f40 con 0x7f62d00ffac0 2024-02-16T22:18:06.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.273+0000 7f62cf7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62d0100cc0 0x7f62d01957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:06.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.273+0000 7f62cf7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62d0100cc0 0x7f62d01957e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55252/0 (socket says 172.21.15.114:55252) 2024-02-16T22:18:06.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.273+0000 7f62cf7fe700 1 -- 172.21.15.114:0/3262921737 learned_addr learned my addr 172.21.15.114:0/3262921737 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:18:06.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.273+0000 7f62cffff700 1 --2- 172.21.15.114:0/3262921737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62d00ffac0 0x7f62d01952a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:06.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.273+0000 7f62cf7fe700 1 -- 172.21.15.114:0/3262921737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62d00ffac0 msgr2=0x7f62d01952a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:06.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.273+0000 7f62cf7fe700 1 --2- 172.21.15.114:0/3262921737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62d00ffac0 0x7f62d01952a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:06.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.273+0000 7f62cf7fe700 1 -- 172.21.15.114:0/3262921737 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c00095f0 con 0x7f62d0100cc0 2024-02-16T22:18:06.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.273+0000 7f62cf7fe700 1 --2- 172.21.15.114:0/3262921737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62d0100cc0 0x7f62d01957e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f62c400d8d0 tx=0x7f62c400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:06.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.274+0000 7f62cd7fa700 1 -- 172.21.15.114:0/3262921737 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62c40098a0 con 0x7f62d0100cc0 2024-02-16T22:18:06.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.274+0000 7f62d65b7700 1 -- 172.21.15.114:0/3262921737 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d019a9f0 con 0x7f62d0100cc0 2024-02-16T22:18:06.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.274+0000 7f62cd7fa700 1 -- 172.21.15.114:0/3262921737 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f62c4010460 con 0x7f62d0100cc0 2024-02-16T22:18:06.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.274+0000 7f62cd7fa700 1 -- 172.21.15.114:0/3262921737 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62c400f5f0 con 0x7f62d0100cc0 2024-02-16T22:18:06.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.274+0000 7f62d65b7700 1 -- 172.21.15.114:0/3262921737 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d019af40 con 0x7f62d0100cc0 2024-02-16T22:18:06.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.275+0000 7f62d65b7700 1 -- 172.21.15.114:0/3262921737 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d0108620 con 0x7f62d0100cc0 2024-02-16T22:18:06.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.275+0000 7f62cd7fa700 1 -- 172.21.15.114:0/3262921737 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f62c4009a00 con 0x7f62d0100cc0 2024-02-16T22:18:06.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.276+0000 7f62cd7fa700 1 --2- 172.21.15.114:0/3262921737 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f62b806c4e0 0x7f62b806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:06.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.276+0000 7f62cffff700 1 --2- 172.21.15.114:0/3262921737 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f62b806c4e0 0x7f62b806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:06.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.276+0000 7f62cd7fa700 1 -- 172.21.15.114:0/3262921737 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f62c408b710 con 0x7f62d0100cc0 2024-02-16T22:18:06.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.276+0000 7f62cffff700 1 --2- 172.21.15.114:0/3262921737 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f62b806c4e0 0x7f62b806e9a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f62c0006010 tx=0x7f62c000b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:06.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.280+0000 7f62cd7fa700 1 -- 172.21.15.114:0/3262921737 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f62c4059980 con 0x7f62d0100cc0 2024-02-16T22:18:06.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:06 smithi114 ceph-mon[29235]: pgmap v180: 65 pgs: 5 creating+activating, 5 creating+peering, 27 active+clean, 28 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:18:06.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:06 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:06.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:06 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:06.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:06 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:06.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:06 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi123.vhlqbe", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:18:06.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:06 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi123.vhlqbe", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T22:18:06.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:06 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:18:06.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.522+0000 7f62d65b7700 1 -- 172.21.15.114:0/3262921737 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f62d01087e0 con 0x7f62d0100cc0 2024-02-16T22:18:06.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.522+0000 7f62cd7fa700 1 -- 172.21.15.114:0/3262921737 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 4 v4) v1 ==== 75+0+1264 (secure 0 0 0) 0x7f62c4059510 con 0x7f62d0100cc0 2024-02-16T22:18:06.524 INFO:teuthology.orchestra.run.smithi114.stdout:e4 2024-02-16T22:18:06.524 INFO:teuthology.orchestra.run.smithi114.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:18:06.525 INFO:teuthology.orchestra.run.smithi114.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:18:06.525 INFO:teuthology.orchestra.run.smithi114.stdout:legacy client fscid: 1 2024-02-16T22:18:06.525 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:18:06.525 INFO:teuthology.orchestra.run.smithi114.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:18:06.525 INFO:teuthology.orchestra.run.smithi114.stdout:fs_name cephfs 2024-02-16T22:18:06.525 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 4 2024-02-16T22:18:06.525 INFO:teuthology.orchestra.run.smithi114.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:18:06.525 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-02-16T22:18:02.108825+0000 2024-02-16T22:18:06.525 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-02-16T22:18:06.355505+0000 2024-02-16T22:18:06.525 INFO:teuthology.orchestra.run.smithi114.stdout:tableserver 0 2024-02-16T22:18:06.525 INFO:teuthology.orchestra.run.smithi114.stdout:root 0 2024-02-16T22:18:06.525 INFO:teuthology.orchestra.run.smithi114.stdout:session_timeout 60 2024-02-16T22:18:06.525 INFO:teuthology.orchestra.run.smithi114.stdout:session_autoclose 300 2024-02-16T22:18:06.525 INFO:teuthology.orchestra.run.smithi114.stdout:max_file_size 1099511627776 2024-02-16T22:18:06.525 INFO:teuthology.orchestra.run.smithi114.stdout:required_client_features {} 2024-02-16T22:18:06.526 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure 0 2024-02-16T22:18:06.526 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure_osd_epoch 0 2024-02-16T22:18:06.526 INFO:teuthology.orchestra.run.smithi114.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:18:06.526 INFO:teuthology.orchestra.run.smithi114.stdout:max_mds 1 2024-02-16T22:18:06.526 INFO:teuthology.orchestra.run.smithi114.stdout:in 0 2024-02-16T22:18:06.526 INFO:teuthology.orchestra.run.smithi114.stdout:up {0=14468} 2024-02-16T22:18:06.526 INFO:teuthology.orchestra.run.smithi114.stdout:failed 2024-02-16T22:18:06.526 INFO:teuthology.orchestra.run.smithi114.stdout:damaged 2024-02-16T22:18:06.526 INFO:teuthology.orchestra.run.smithi114.stdout:stopped 2024-02-16T22:18:06.526 INFO:teuthology.orchestra.run.smithi114.stdout:data_pools [3] 2024-02-16T22:18:06.526 INFO:teuthology.orchestra.run.smithi114.stdout:metadata_pool 2 2024-02-16T22:18:06.526 INFO:teuthology.orchestra.run.smithi114.stdout:inline_data disabled 2024-02-16T22:18:06.526 INFO:teuthology.orchestra.run.smithi114.stdout:balancer 2024-02-16T22:18:06.526 INFO:teuthology.orchestra.run.smithi114.stdout:bal_rank_mask -1 2024-02-16T22:18:06.526 INFO:teuthology.orchestra.run.smithi114.stdout:standby_count_wanted 0 2024-02-16T22:18:06.527 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.zzryke{0:14468} state up:creating seq 1 addr [v2:172.21.15.114:6826/277953535,v1:172.21.15.114:6827/277953535] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:18:06.527 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:18:06.527 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:18:06.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.525+0000 7f62d65b7700 1 -- 172.21.15.114:0/3262921737 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f62b806c4e0 msgr2=0x7f62b806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:06.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.525+0000 7f62d65b7700 1 --2- 172.21.15.114:0/3262921737 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f62b806c4e0 0x7f62b806e9a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f62c0006010 tx=0x7f62c000b540 comp rx=0 tx=0).stop 2024-02-16T22:18:06.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.525+0000 7f62d65b7700 1 -- 172.21.15.114:0/3262921737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62d0100cc0 msgr2=0x7f62d01957e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:06.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.525+0000 7f62d65b7700 1 --2- 172.21.15.114:0/3262921737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62d0100cc0 0x7f62d01957e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f62c400d8d0 tx=0x7f62c400dbe0 comp rx=0 tx=0).stop 2024-02-16T22:18:06.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.525+0000 7f62d65b7700 1 -- 172.21.15.114:0/3262921737 shutdown_connections 2024-02-16T22:18:06.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.525+0000 7f62d65b7700 1 --2- 172.21.15.114:0/3262921737 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f62b806c4e0 0x7f62b806e9a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:06.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.525+0000 7f62d65b7700 1 --2- 172.21.15.114:0/3262921737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62d00ffac0 0x7f62d01952a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:06.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.525+0000 7f62d65b7700 1 --2- 172.21.15.114:0/3262921737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62d0100cc0 0x7f62d01957e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:06.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.525+0000 7f62d65b7700 1 -- 172.21.15.114:0/3262921737 >> 172.21.15.114:0/3262921737 conn(0x7f62d00fb060 msgr2=0x7f62d0103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:06.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.525+0000 7f62d65b7700 1 -- 172.21.15.114:0/3262921737 shutdown_connections 2024-02-16T22:18:06.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:06.525+0000 7f62d65b7700 1 -- 172.21.15.114:0/3262921737 wait complete. 2024-02-16T22:18:06.528 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 4 2024-02-16T22:18:06.670 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:06 smithi123 ceph-mon[37161]: pgmap v180: 65 pgs: 5 creating+activating, 5 creating+peering, 27 active+clean, 28 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:18:06.670 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:06 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:06.670 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:06 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:06.670 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:06 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:06.670 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:06 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi123.vhlqbe", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:18:06.670 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:06 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi123.vhlqbe", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T22:18:06.670 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:06 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:18:07.280 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T22:18:07.290 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi114.front.sepia.ceph.com 2024-02-16T22:18:07.290 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- bash -c 'ceph fs set cephfs max_mds 1' 2024-02-16T22:18:07.504 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:18:07.625 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:07 smithi114 ceph-mon[29235]: Deploying daemon mds.cephfs.smithi123.vhlqbe on smithi123 2024-02-16T22:18:07.625 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:07 smithi114 ceph-mon[29235]: mds.? [v2:172.21.15.114:6826/277953535,v1:172.21.15.114:6827/277953535] up:boot 2024-02-16T22:18:07.626 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:07 smithi114 ceph-mon[29235]: daemon mds.cephfs.smithi114.zzryke assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-02-16T22:18:07.626 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:07 smithi114 ceph-mon[29235]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T22:18:07.626 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:07 smithi114 ceph-mon[29235]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-16T22:18:07.626 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:07 smithi114 ceph-mon[29235]: Cluster is now healthy 2024-02-16T22:18:07.626 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:07 smithi114 ceph-mon[29235]: fsmap cephfs:0 1 up:standby 2024-02-16T22:18:07.626 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:07 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi114.zzryke"}]: dispatch 2024-02-16T22:18:07.626 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:07 smithi114 ceph-mon[29235]: fsmap cephfs:1 {0=cephfs.smithi114.zzryke=up:creating} 2024-02-16T22:18:07.626 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:07 smithi114 ceph-mon[29235]: daemon mds.cephfs.smithi114.zzryke is now active in filesystem cephfs as rank 0 2024-02-16T22:18:07.626 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:07 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/3262921737' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:18:07.664 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:07 smithi123 ceph-mon[37161]: Deploying daemon mds.cephfs.smithi123.vhlqbe on smithi123 2024-02-16T22:18:07.664 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:07 smithi123 ceph-mon[37161]: mds.? [v2:172.21.15.114:6826/277953535,v1:172.21.15.114:6827/277953535] up:boot 2024-02-16T22:18:07.664 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:07 smithi123 ceph-mon[37161]: daemon mds.cephfs.smithi114.zzryke assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-02-16T22:18:07.664 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:07 smithi123 ceph-mon[37161]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T22:18:07.664 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:07 smithi123 ceph-mon[37161]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-16T22:18:07.664 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:07 smithi123 ceph-mon[37161]: Cluster is now healthy 2024-02-16T22:18:07.664 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:07 smithi123 ceph-mon[37161]: fsmap cephfs:0 1 up:standby 2024-02-16T22:18:07.664 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:07 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi114.zzryke"}]: dispatch 2024-02-16T22:18:07.664 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:07 smithi123 ceph-mon[37161]: fsmap cephfs:1 {0=cephfs.smithi114.zzryke=up:creating} 2024-02-16T22:18:07.665 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:07 smithi123 ceph-mon[37161]: daemon mds.cephfs.smithi114.zzryke is now active in filesystem cephfs as rank 0 2024-02-16T22:18:07.665 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:07 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/3262921737' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:18:08.650 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:08 smithi114 ceph-mon[29235]: mds.? [v2:172.21.15.114:6826/277953535,v1:172.21.15.114:6827/277953535] up:active 2024-02-16T22:18:08.650 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:08 smithi114 ceph-mon[29235]: fsmap cephfs:1 {0=cephfs.smithi114.zzryke=up:active} 2024-02-16T22:18:08.651 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:08 smithi114 ceph-mon[29235]: pgmap v181: 65 pgs: 5 creating+activating, 60 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 0 B/s wr, 2 op/s 2024-02-16T22:18:08.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:08 smithi123 ceph-mon[37161]: mds.? [v2:172.21.15.114:6826/277953535,v1:172.21.15.114:6827/277953535] up:active 2024-02-16T22:18:08.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:08 smithi123 ceph-mon[37161]: fsmap cephfs:1 {0=cephfs.smithi114.zzryke=up:active} 2024-02-16T22:18:08.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:08 smithi123 ceph-mon[37161]: pgmap v181: 65 pgs: 5 creating+activating, 60 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 0 B/s wr, 2 op/s 2024-02-16T22:18:09.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.010+0000 7fc9107af700 1 -- 172.21.15.114:0/3973910872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc908103f70 msgr2=0x7fc908106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:09.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.010+0000 7fc9107af700 1 --2- 172.21.15.114:0/3973910872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc908103f70 0x7fc908106360 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8009910 tx=0x7fc8f8009c20 comp rx=0 tx=0).stop 2024-02-16T22:18:09.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.010+0000 7fc9107af700 1 -- 172.21.15.114:0/3973910872 shutdown_connections 2024-02-16T22:18:09.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.010+0000 7fc9107af700 1 --2- 172.21.15.114:0/3973910872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc908103f70 0x7fc908106360 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:09.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.010+0000 7fc9107af700 1 --2- 172.21.15.114:0/3973910872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc908101640 0x7fc908103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:09.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.010+0000 7fc9107af700 1 -- 172.21.15.114:0/3973910872 >> 172.21.15.114:0/3973910872 conn(0x7fc9080fb050 msgr2=0x7fc9080fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:09.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.010+0000 7fc9107af700 1 -- 172.21.15.114:0/3973910872 shutdown_connections 2024-02-16T22:18:09.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.010+0000 7fc9107af700 1 -- 172.21.15.114:0/3973910872 wait complete. 2024-02-16T22:18:09.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.010+0000 7fc9107af700 1 Processor -- start 2024-02-16T22:18:09.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.011+0000 7fc9107af700 1 -- start start 2024-02-16T22:18:09.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.011+0000 7fc9107af700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc908101640 0x7fc908196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:09.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.011+0000 7fc9107af700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc908103f70 0x7fc908196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:09.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.011+0000 7fc9107af700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc908196e80 con 0x7fc908103f70 2024-02-16T22:18:09.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.011+0000 7fc9107af700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc908196fc0 con 0x7fc908101640 2024-02-16T22:18:09.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.011+0000 7fc90dd4a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc908103f70 0x7fc908196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:09.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.011+0000 7fc90dd4a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc908103f70 0x7fc908196860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55272/0 (socket says 172.21.15.114:55272) 2024-02-16T22:18:09.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.011+0000 7fc90dd4a700 1 -- 172.21.15.114:0/1599105629 learned_addr learned my addr 172.21.15.114:0/1599105629 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:18:09.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.011+0000 7fc90e54b700 1 --2- 172.21.15.114:0/1599105629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc908101640 0x7fc908196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:09.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.011+0000 7fc90dd4a700 1 -- 172.21.15.114:0/1599105629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc908101640 msgr2=0x7fc908196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:09.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.011+0000 7fc90dd4a700 1 --2- 172.21.15.114:0/1599105629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc908101640 0x7fc908196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:09.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.011+0000 7fc90dd4a700 1 -- 172.21.15.114:0/1599105629 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f80095f0 con 0x7fc908103f70 2024-02-16T22:18:09.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.011+0000 7fc90dd4a700 1 --2- 172.21.15.114:0/1599105629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc908103f70 0x7fc908196860 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8006010 tx=0x7fc8f800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:09.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.011+0000 7fc8ff7fe700 1 -- 172.21.15.114:0/1599105629 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8f801d070 con 0x7fc908103f70 2024-02-16T22:18:09.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.012+0000 7fc8ff7fe700 1 -- 172.21.15.114:0/1599105629 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc8f800f460 con 0x7fc908103f70 2024-02-16T22:18:09.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.012+0000 7fc9107af700 1 -- 172.21.15.114:0/1599105629 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc90819ba10 con 0x7fc908103f70 2024-02-16T22:18:09.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.012+0000 7fc8ff7fe700 1 -- 172.21.15.114:0/1599105629 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8f8021640 con 0x7fc908103f70 2024-02-16T22:18:09.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.012+0000 7fc9107af700 1 -- 172.21.15.114:0/1599105629 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90819bf00 con 0x7fc908103f70 2024-02-16T22:18:09.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.013+0000 7fc9107af700 1 -- 172.21.15.114:0/1599105629 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc908190520 con 0x7fc908103f70 2024-02-16T22:18:09.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.013+0000 7fc8ff7fe700 1 -- 172.21.15.114:0/1599105629 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fc8f800fab0 con 0x7fc908103f70 2024-02-16T22:18:09.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.014+0000 7fc8ff7fe700 1 --2- 172.21.15.114:0/1599105629 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fc8f406c4e0 0x7fc8f406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:09.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.014+0000 7fc8ff7fe700 1 -- 172.21.15.114:0/1599105629 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fc8f808ca40 con 0x7fc908103f70 2024-02-16T22:18:09.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.014+0000 7fc90e54b700 1 --2- 172.21.15.114:0/1599105629 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fc8f406c4e0 0x7fc8f406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:09.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.014+0000 7fc90e54b700 1 --2- 172.21.15.114:0/1599105629 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fc8f406c4e0 0x7fc8f406e9a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc904005950 tx=0x7fc90400b410 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:09.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.019+0000 7fc8ff7fe700 1 -- 172.21.15.114:0/1599105629 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc8f805ad30 con 0x7fc908103f70 2024-02-16T22:18:09.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.274+0000 7fc9107af700 1 -- 172.21.15.114:0/1599105629 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7fc908061a40 con 0x7fc908103f70 2024-02-16T22:18:09.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.368+0000 7fc8ff7fe700 1 -- 172.21.15.114:0/1599105629 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7fc8f805a8c0 con 0x7fc908103f70 2024-02-16T22:18:09.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.370+0000 7fc9107af700 1 -- 172.21.15.114:0/1599105629 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fc8f406c4e0 msgr2=0x7fc8f406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:09.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.370+0000 7fc9107af700 1 --2- 172.21.15.114:0/1599105629 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fc8f406c4e0 0x7fc8f406e9a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc904005950 tx=0x7fc90400b410 comp rx=0 tx=0).stop 2024-02-16T22:18:09.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.370+0000 7fc9107af700 1 -- 172.21.15.114:0/1599105629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc908103f70 msgr2=0x7fc908196860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:09.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.370+0000 7fc9107af700 1 --2- 172.21.15.114:0/1599105629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc908103f70 0x7fc908196860 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8006010 tx=0x7fc8f800bd70 comp rx=0 tx=0).stop 2024-02-16T22:18:09.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.370+0000 7fc9107af700 1 -- 172.21.15.114:0/1599105629 shutdown_connections 2024-02-16T22:18:09.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.370+0000 7fc9107af700 1 --2- 172.21.15.114:0/1599105629 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fc8f406c4e0 0x7fc8f406e9a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:09.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.370+0000 7fc9107af700 1 --2- 172.21.15.114:0/1599105629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc908101640 0x7fc908196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:09.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.370+0000 7fc9107af700 1 --2- 172.21.15.114:0/1599105629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc908103f70 0x7fc908196860 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:09.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.370+0000 7fc9107af700 1 -- 172.21.15.114:0/1599105629 >> 172.21.15.114:0/1599105629 conn(0x7fc9080fb050 msgr2=0x7fc9080fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:09.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.370+0000 7fc9107af700 1 -- 172.21.15.114:0/1599105629 shutdown_connections 2024-02-16T22:18:09.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:09.371+0000 7fc9107af700 1 -- 172.21.15.114:0/1599105629 wait complete. 2024-02-16T22:18:09.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:09 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:09.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:09 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:09.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:09 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:09.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:09 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi114.fdxpey", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:18:09.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:09 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi114.fdxpey", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T22:18:09.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:09 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:18:09.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:09 smithi114 ceph-mon[29235]: Deploying daemon mds.cephfs.smithi114.fdxpey on smithi114 2024-02-16T22:18:09.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:09 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/1599105629' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-02-16T22:18:09.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:09 smithi114 ceph-mon[29235]: mds.? [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] up:boot 2024-02-16T22:18:09.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:09 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/1599105629' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-02-16T22:18:09.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:09 smithi114 ceph-mon[29235]: fsmap cephfs:1 {0=cephfs.smithi114.zzryke=up:active} 1 up:standby 2024-02-16T22:18:09.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:09 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi123.vhlqbe"}]: dispatch 2024-02-16T22:18:09.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:09 smithi114 ceph-mon[29235]: fsmap cephfs:1 {0=cephfs.smithi114.zzryke=up:active} 1 up:standby 2024-02-16T22:18:10.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:09 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:10.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:09 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:10.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:09 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:10.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:09 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi114.fdxpey", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:18:10.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:09 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi114.fdxpey", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T22:18:10.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:09 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:18:10.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:09 smithi123 ceph-mon[37161]: Deploying daemon mds.cephfs.smithi114.fdxpey on smithi114 2024-02-16T22:18:10.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:09 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/1599105629' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-02-16T22:18:10.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:09 smithi123 ceph-mon[37161]: mds.? [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] up:boot 2024-02-16T22:18:10.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:09 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/1599105629' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-02-16T22:18:10.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:09 smithi123 ceph-mon[37161]: fsmap cephfs:1 {0=cephfs.smithi114.zzryke=up:active} 1 up:standby 2024-02-16T22:18:10.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:09 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi123.vhlqbe"}]: dispatch 2024-02-16T22:18:10.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:09 smithi123 ceph-mon[37161]: fsmap cephfs:1 {0=cephfs.smithi114.zzryke=up:active} 1 up:standby 2024-02-16T22:18:10.789 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T22:18:10.799 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi114.front.sepia.ceph.com 2024-02-16T22:18:10.800 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2024-02-16T22:18:10.827 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:10 smithi114 ceph-mon[29235]: pgmap v182: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 418 B/s wr, 3 op/s 2024-02-16T22:18:11.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:10 smithi123 ceph-mon[37161]: pgmap v182: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 418 B/s wr, 3 op/s 2024-02-16T22:18:11.432 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:18:12.519 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:12 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:12.519 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:12 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:12.519 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:12 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:12.519 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:12 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi123.gopcfa", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:18:12.519 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:12 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi123.gopcfa", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T22:18:12.519 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:12 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:18:12.519 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:12 smithi114 ceph-mon[29235]: Deploying daemon mds.cephfs.smithi123.gopcfa on smithi123 2024-02-16T22:18:12.519 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:12 smithi114 ceph-mon[29235]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s wr, 5 op/s 2024-02-16T22:18:12.519 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:12 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:18:12.519 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:12 smithi114 ceph-mon[29235]: mds.? [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] up:boot 2024-02-16T22:18:12.520 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:12 smithi114 ceph-mon[29235]: fsmap cephfs:1 {0=cephfs.smithi114.zzryke=up:active} 2 up:standby 2024-02-16T22:18:12.520 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:12 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi114.fdxpey"}]: dispatch 2024-02-16T22:18:12.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:12 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:12.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:12 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:12.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:12 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:12.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:12 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi123.gopcfa", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:18:12.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:12 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi123.gopcfa", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T22:18:12.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:12 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:18:12.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:12 smithi123 ceph-mon[37161]: Deploying daemon mds.cephfs.smithi123.gopcfa on smithi123 2024-02-16T22:18:12.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:12 smithi123 ceph-mon[37161]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s wr, 5 op/s 2024-02-16T22:18:12.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:12 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:18:12.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:12 smithi123 ceph-mon[37161]: mds.? [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] up:boot 2024-02-16T22:18:12.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:12 smithi123 ceph-mon[37161]: fsmap cephfs:1 {0=cephfs.smithi114.zzryke=up:active} 2 up:standby 2024-02-16T22:18:12.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:12 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi114.fdxpey"}]: dispatch 2024-02-16T22:18:12.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.974+0000 7fb22c882700 1 -- 172.21.15.114:0/1897052823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2280ffab0 msgr2=0x7fb2280ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:12.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.974+0000 7fb22c882700 1 --2- 172.21.15.114:0/1897052823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2280ffab0 0x7fb2280ffed0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb218009910 tx=0x7fb218009c20 comp rx=0 tx=0).stop 2024-02-16T22:18:12.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.974+0000 7fb22c882700 1 -- 172.21.15.114:0/1897052823 shutdown_connections 2024-02-16T22:18:12.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.974+0000 7fb22c882700 1 --2- 172.21.15.114:0/1897052823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb228100cb0 0x7fb228101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:12.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.974+0000 7fb22c882700 1 --2- 172.21.15.114:0/1897052823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2280ffab0 0x7fb2280ffed0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:12.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.974+0000 7fb22c882700 1 -- 172.21.15.114:0/1897052823 >> 172.21.15.114:0/1897052823 conn(0x7fb2280fb050 msgr2=0x7fb2280fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:12.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.974+0000 7fb22c882700 1 -- 172.21.15.114:0/1897052823 shutdown_connections 2024-02-16T22:18:12.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.974+0000 7fb22c882700 1 -- 172.21.15.114:0/1897052823 wait complete. 2024-02-16T22:18:12.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.974+0000 7fb22c882700 1 Processor -- start 2024-02-16T22:18:12.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.975+0000 7fb22c882700 1 -- start start 2024-02-16T22:18:12.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.975+0000 7fb22c882700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2280ffab0 0x7fb228196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:12.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.975+0000 7fb22c882700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb228100cb0 0x7fb2281968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:12.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.975+0000 7fb22c882700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb228196ed0 con 0x7fb228100cb0 2024-02-16T22:18:12.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.975+0000 7fb22c882700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb228197010 con 0x7fb2280ffab0 2024-02-16T22:18:12.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.975+0000 7fb21dd9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb228100cb0 0x7fb2281968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:12.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.975+0000 7fb21dd9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb228100cb0 0x7fb2281968b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38172/0 (socket says 172.21.15.114:38172) 2024-02-16T22:18:12.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.975+0000 7fb21dd9b700 1 -- 172.21.15.114:0/2169560664 learned_addr learned my addr 172.21.15.114:0/2169560664 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:18:12.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.975+0000 7fb22659c700 1 --2- 172.21.15.114:0/2169560664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2280ffab0 0x7fb228196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:12.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.975+0000 7fb21dd9b700 1 -- 172.21.15.114:0/2169560664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2280ffab0 msgr2=0x7fb228196370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:12.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.975+0000 7fb21dd9b700 1 --2- 172.21.15.114:0/2169560664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2280ffab0 0x7fb228196370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:12.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.975+0000 7fb21dd9b700 1 -- 172.21.15.114:0/2169560664 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2180095f0 con 0x7fb228100cb0 2024-02-16T22:18:12.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.975+0000 7fb22659c700 1 --2- 172.21.15.114:0/2169560664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2280ffab0 0x7fb228196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:18:12.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.975+0000 7fb21dd9b700 1 --2- 172.21.15.114:0/2169560664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb228100cb0 0x7fb2281968b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb21000d8d0 tx=0x7fb21000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:12.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.975+0000 7fb21ffff700 1 -- 172.21.15.114:0/2169560664 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb210009940 con 0x7fb228100cb0 2024-02-16T22:18:12.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.976+0000 7fb22c882700 1 -- 172.21.15.114:0/2169560664 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb22819bac0 con 0x7fb228100cb0 2024-02-16T22:18:12.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.976+0000 7fb21ffff700 1 -- 172.21.15.114:0/2169560664 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb210010460 con 0x7fb228100cb0 2024-02-16T22:18:12.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.976+0000 7fb21ffff700 1 -- 172.21.15.114:0/2169560664 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb21000f5f0 con 0x7fb228100cb0 2024-02-16T22:18:12.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.976+0000 7fb22c882700 1 -- 172.21.15.114:0/2169560664 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb22819c010 con 0x7fb228100cb0 2024-02-16T22:18:12.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.977+0000 7fb22c882700 1 -- 172.21.15.114:0/2169560664 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb228108610 con 0x7fb228100cb0 2024-02-16T22:18:12.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.977+0000 7fb21ffff700 1 -- 172.21.15.114:0/2169560664 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fb2100105d0 con 0x7fb228100cb0 2024-02-16T22:18:12.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.977+0000 7fb21ffff700 1 --2- 172.21.15.114:0/2169560664 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fb21406c4e0 0x7fb21406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:12.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.978+0000 7fb21ffff700 1 -- 172.21.15.114:0/2169560664 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fb21008b820 con 0x7fb228100cb0 2024-02-16T22:18:12.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.978+0000 7fb22659c700 1 --2- 172.21.15.114:0/2169560664 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fb21406c4e0 0x7fb21406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:12.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.978+0000 7fb22659c700 1 --2- 172.21.15.114:0/2169560664 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fb21406c4e0 0x7fb21406e9a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb218006010 tx=0x7fb218005a90 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:12.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:12.982+0000 7fb21ffff700 1 -- 172.21.15.114:0/2169560664 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb210059a90 con 0x7fb228100cb0 2024-02-16T22:18:13.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:13.216+0000 7fb22c882700 1 -- 172.21.15.114:0/2169560664 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fb22819c7c0 con 0x7fb228100cb0 2024-02-16T22:18:13.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:13.402+0000 7fb21ffff700 1 -- 172.21.15.114:0/2169560664 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb210059620 con 0x7fb228100cb0 2024-02-16T22:18:13.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:13.404+0000 7fb22c882700 1 -- 172.21.15.114:0/2169560664 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fb21406c4e0 msgr2=0x7fb21406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:13.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:13.404+0000 7fb22c882700 1 --2- 172.21.15.114:0/2169560664 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fb21406c4e0 0x7fb21406e9a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb218006010 tx=0x7fb218005a90 comp rx=0 tx=0).stop 2024-02-16T22:18:13.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:13.404+0000 7fb22c882700 1 -- 172.21.15.114:0/2169560664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb228100cb0 msgr2=0x7fb2281968b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:13.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:13.404+0000 7fb22c882700 1 --2- 172.21.15.114:0/2169560664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb228100cb0 0x7fb2281968b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb21000d8d0 tx=0x7fb21000dc90 comp rx=0 tx=0).stop 2024-02-16T22:18:13.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:13.405+0000 7fb22c882700 1 -- 172.21.15.114:0/2169560664 shutdown_connections 2024-02-16T22:18:13.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:13.405+0000 7fb22c882700 1 --2- 172.21.15.114:0/2169560664 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fb21406c4e0 0x7fb21406e9a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:13.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:13.405+0000 7fb22c882700 1 --2- 172.21.15.114:0/2169560664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2280ffab0 0x7fb228196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:13.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:13.405+0000 7fb22c882700 1 --2- 172.21.15.114:0/2169560664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb228100cb0 0x7fb2281968b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:13.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:13.405+0000 7fb22c882700 1 -- 172.21.15.114:0/2169560664 >> 172.21.15.114:0/2169560664 conn(0x7fb2280fb050 msgr2=0x7fb228068c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:13.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:13.405+0000 7fb22c882700 1 -- 172.21.15.114:0/2169560664 shutdown_connections 2024-02-16T22:18:13.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:13.405+0000 7fb22c882700 1 -- 172.21.15.114:0/2169560664 wait complete. 2024-02-16T22:18:13.470 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:13 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/2169560664' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-02-16T22:18:13.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:13 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/2169560664' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-02-16T22:18:14.058 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T22:18:14.067 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi114.front.sepia.ceph.com 2024-02-16T22:18:14.068 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- bash -c 'ceph fs set cephfs inline_data true --yes-i-really-really-mean-it' 2024-02-16T22:18:14.319 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:18:14.671 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:14 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/2169560664' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-02-16T22:18:14.671 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:14 smithi114 ceph-mon[29235]: fsmap cephfs:1 {0=cephfs.smithi114.zzryke=up:active} 2 up:standby 2024-02-16T22:18:14.671 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:14 smithi114 ceph-mon[29235]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-02-16T22:18:14.671 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:14 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:14.672 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:14 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:14.672 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:14 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:14.672 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:14 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:14.672 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:14 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:14.672 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:14 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:18:14.766 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:14 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/2169560664' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-02-16T22:18:14.767 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:14 smithi123 ceph-mon[37161]: fsmap cephfs:1 {0=cephfs.smithi114.zzryke=up:active} 2 up:standby 2024-02-16T22:18:14.767 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:14 smithi123 ceph-mon[37161]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-02-16T22:18:14.767 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:14 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:14.767 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:14 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:14.767 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:14 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:14.767 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:14 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:14.767 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:14 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:14.767 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:14 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:18:15.642 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:15 smithi114 ceph-mon[29235]: mds.? [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] up:boot 2024-02-16T22:18:15.642 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:15 smithi114 ceph-mon[29235]: mds.? [v2:172.21.15.114:6826/277953535,v1:172.21.15.114:6827/277953535] up:active 2024-02-16T22:18:15.643 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:15 smithi114 ceph-mon[29235]: fsmap cephfs:1 {0=cephfs.smithi114.zzryke=up:active} 3 up:standby 2024-02-16T22:18:15.643 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:15 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi123.gopcfa"}]: dispatch 2024-02-16T22:18:15.730 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:15 smithi123 ceph-mon[37161]: mds.? [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] up:boot 2024-02-16T22:18:15.730 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:15 smithi123 ceph-mon[37161]: mds.? [v2:172.21.15.114:6826/277953535,v1:172.21.15.114:6827/277953535] up:active 2024-02-16T22:18:15.730 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:15 smithi123 ceph-mon[37161]: fsmap cephfs:1 {0=cephfs.smithi114.zzryke=up:active} 3 up:standby 2024-02-16T22:18:15.730 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:15 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi123.gopcfa"}]: dispatch 2024-02-16T22:18:15.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.934+0000 7fd1c4a03700 1 -- 172.21.15.114:0/170651886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1c0101c30 msgr2=0x7fd1c0102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:15.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.934+0000 7fd1c4a03700 1 --2- 172.21.15.114:0/170651886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1c0101c30 0x7fd1c0102090 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4009910 tx=0x7fd1b4009c20 comp rx=0 tx=0).stop 2024-02-16T22:18:15.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.934+0000 7fd1c4a03700 1 -- 172.21.15.114:0/170651886 shutdown_connections 2024-02-16T22:18:15.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.934+0000 7fd1c4a03700 1 --2- 172.21.15.114:0/170651886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1c0101c30 0x7fd1c0102090 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:15.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.934+0000 7fd1c4a03700 1 --2- 172.21.15.114:0/170651886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1c0100a30 0x7fd1c0100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:15.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.934+0000 7fd1c4a03700 1 -- 172.21.15.114:0/170651886 >> 172.21.15.114:0/170651886 conn(0x7fd1c00fbfd0 msgr2=0x7fd1c00fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:15.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.934+0000 7fd1c4a03700 1 -- 172.21.15.114:0/170651886 shutdown_connections 2024-02-16T22:18:15.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.934+0000 7fd1c4a03700 1 -- 172.21.15.114:0/170651886 wait complete. 2024-02-16T22:18:15.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.934+0000 7fd1c4a03700 1 Processor -- start 2024-02-16T22:18:15.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.935+0000 7fd1c4a03700 1 -- start start 2024-02-16T22:18:15.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.935+0000 7fd1c4a03700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1c0100a30 0x7fd1c01961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:15.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.935+0000 7fd1c4a03700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1c0101c30 0x7fd1c01966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:15.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.935+0000 7fd1c4a03700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c0196d10 con 0x7fd1c0100a30 2024-02-16T22:18:15.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.935+0000 7fd1c4a03700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c0196e50 con 0x7fd1c0101c30 2024-02-16T22:18:15.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.935+0000 7fd1bed9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1c0100a30 0x7fd1c01961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:15.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.935+0000 7fd1bed9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1c0100a30 0x7fd1c01961b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38198/0 (socket says 172.21.15.114:38198) 2024-02-16T22:18:15.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.935+0000 7fd1bed9d700 1 -- 172.21.15.114:0/3229326169 learned_addr learned my addr 172.21.15.114:0/3229326169 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:18:15.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.935+0000 7fd1be59c700 1 --2- 172.21.15.114:0/3229326169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1c0101c30 0x7fd1c01966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:15.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.935+0000 7fd1bed9d700 1 -- 172.21.15.114:0/3229326169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1c0101c30 msgr2=0x7fd1c01966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:15.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.935+0000 7fd1bed9d700 1 --2- 172.21.15.114:0/3229326169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1c0101c30 0x7fd1c01966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:15.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.935+0000 7fd1bed9d700 1 -- 172.21.15.114:0/3229326169 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b40095f0 con 0x7fd1c0100a30 2024-02-16T22:18:15.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.935+0000 7fd1be59c700 1 --2- 172.21.15.114:0/3229326169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1c0101c30 0x7fd1c01966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:18:15.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.936+0000 7fd1bed9d700 1 --2- 172.21.15.114:0/3229326169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1c0100a30 0x7fd1c01961b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd1b000e8f0 tx=0x7fd1b000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:15.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.936+0000 7fd1a7fff700 1 -- 172.21.15.114:0/3229326169 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1b0009790 con 0x7fd1c0100a30 2024-02-16T22:18:15.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.936+0000 7fd1a7fff700 1 -- 172.21.15.114:0/3229326169 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd1b000c810 con 0x7fd1c0100a30 2024-02-16T22:18:15.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.936+0000 7fd1c4a03700 1 -- 172.21.15.114:0/3229326169 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c019b900 con 0x7fd1c0100a30 2024-02-16T22:18:15.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.936+0000 7fd1a7fff700 1 -- 172.21.15.114:0/3229326169 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1b0003d20 con 0x7fd1c0100a30 2024-02-16T22:18:15.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.936+0000 7fd1c4a03700 1 -- 172.21.15.114:0/3229326169 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c019be50 con 0x7fd1c0100a30 2024-02-16T22:18:15.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.937+0000 7fd1c4a03700 1 -- 172.21.15.114:0/3229326169 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c0109590 con 0x7fd1c0100a30 2024-02-16T22:18:15.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.937+0000 7fd1a7fff700 1 -- 172.21.15.114:0/3229326169 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fd1b0018430 con 0x7fd1c0100a30 2024-02-16T22:18:15.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.938+0000 7fd1a7fff700 1 --2- 172.21.15.114:0/3229326169 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd1a806c4e0 0x7fd1a806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:15.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.938+0000 7fd1a7fff700 1 -- 172.21.15.114:0/3229326169 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fd1b0014070 con 0x7fd1c0100a30 2024-02-16T22:18:15.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.938+0000 7fd1be59c700 1 --2- 172.21.15.114:0/3229326169 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd1a806c4e0 0x7fd1a806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:15.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.938+0000 7fd1be59c700 1 --2- 172.21.15.114:0/3229326169 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd1a806c4e0 0x7fd1a806e9a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4009fd0 tx=0x7fd1b4005940 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:15.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:15.943+0000 7fd1a7fff700 1 -- 172.21.15.114:0/3229326169 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd1b0059db0 con 0x7fd1c0100a30 2024-02-16T22:18:16.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:16.196+0000 7fd1c4a03700 1 -- 172.21.15.114:0/3229326169 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd1c01097b0 con 0x7fd1c0100a30 2024-02-16T22:18:16.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:16.423+0000 7fd1a7fff700 1 -- 172.21.15.114:0/3229326169 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]=0 inline data enabled v12) v1 ==== 168+0+0 (secure 0 0 0) 0x7fd1b0059940 con 0x7fd1c0100a30 2024-02-16T22:18:16.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:16.426+0000 7fd1c4a03700 1 -- 172.21.15.114:0/3229326169 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd1a806c4e0 msgr2=0x7fd1a806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:16.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:16.426+0000 7fd1c4a03700 1 --2- 172.21.15.114:0/3229326169 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd1a806c4e0 0x7fd1a806e9a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4009fd0 tx=0x7fd1b4005940 comp rx=0 tx=0).stop 2024-02-16T22:18:16.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:16.426+0000 7fd1c4a03700 1 -- 172.21.15.114:0/3229326169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1c0100a30 msgr2=0x7fd1c01961b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:16.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:16.426+0000 7fd1c4a03700 1 --2- 172.21.15.114:0/3229326169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1c0100a30 0x7fd1c01961b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd1b000e8f0 tx=0x7fd1b000ec00 comp rx=0 tx=0).stop 2024-02-16T22:18:16.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:16.426+0000 7fd1c4a03700 1 -- 172.21.15.114:0/3229326169 shutdown_connections 2024-02-16T22:18:16.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:16.426+0000 7fd1c4a03700 1 --2- 172.21.15.114:0/3229326169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1c0100a30 0x7fd1c01961b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:16.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:16.426+0000 7fd1c4a03700 1 --2- 172.21.15.114:0/3229326169 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fd1a806c4e0 0x7fd1a806e9a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:16.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:16.426+0000 7fd1c4a03700 1 --2- 172.21.15.114:0/3229326169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1c0101c30 0x7fd1c01966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:16.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:16.426+0000 7fd1c4a03700 1 -- 172.21.15.114:0/3229326169 >> 172.21.15.114:0/3229326169 conn(0x7fd1c00fbfd0 msgr2=0x7fd1c0104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:16.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:16.426+0000 7fd1c4a03700 1 -- 172.21.15.114:0/3229326169 shutdown_connections 2024-02-16T22:18:16.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:16.426+0000 7fd1c4a03700 1 -- 172.21.15.114:0/3229326169 wait complete. 2024-02-16T22:18:16.429 INFO:teuthology.orchestra.run.smithi114.stderr:inline data enabled 2024-02-16T22:18:16.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:16 smithi114 ceph-mon[29235]: mds.? [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] up:standby 2024-02-16T22:18:16.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:16 smithi114 ceph-mon[29235]: fsmap cephfs:1 {0=cephfs.smithi114.zzryke=up:active} 3 up:standby 2024-02-16T22:18:16.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:16 smithi114 ceph-mon[29235]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 3 op/s 2024-02-16T22:18:16.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:16 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/3229326169' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-16T22:18:16.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:16 smithi123 ceph-mon[37161]: mds.? [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] up:standby 2024-02-16T22:18:16.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:16 smithi123 ceph-mon[37161]: fsmap cephfs:1 {0=cephfs.smithi114.zzryke=up:active} 3 up:standby 2024-02-16T22:18:16.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:16 smithi123 ceph-mon[37161]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 3 op/s 2024-02-16T22:18:16.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:16 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/3229326169' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-16T22:18:17.391 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T22:18:17.403 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi114.front.sepia.ceph.com 2024-02-16T22:18:17.404 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-16T22:18:17.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:17 smithi114 ceph-mon[29235]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-02-16T22:18:17.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:17 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/3229326169' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-02-16T22:18:17.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:17 smithi114 ceph-mon[29235]: fsmap cephfs:1 {0=cephfs.smithi114.zzryke=up:active} 3 up:standby 2024-02-16T22:18:17.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:17 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:17.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:17 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:17.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:17 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:17.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:17 smithi123 ceph-mon[37161]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-02-16T22:18:17.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:17 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/3229326169' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-02-16T22:18:17.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:17 smithi123 ceph-mon[37161]: fsmap cephfs:1 {0=cephfs.smithi114.zzryke=up:active} 3 up:standby 2024-02-16T22:18:17.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:17 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:17.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:17 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:17.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:17 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:18.291 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:18:18.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:18 smithi114 ceph-mon[29235]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 3 op/s 2024-02-16T22:18:18.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:18 smithi114 ceph-mon[29235]: mds.? [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] up:standby 2024-02-16T22:18:18.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:18 smithi114 ceph-mon[29235]: fsmap cephfs:1 {0=cephfs.smithi114.zzryke=up:active} 3 up:standby 2024-02-16T22:18:18.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:18 smithi123 ceph-mon[37161]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 3 op/s 2024-02-16T22:18:18.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:18 smithi123 ceph-mon[37161]: mds.? [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] up:standby 2024-02-16T22:18:18.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:18 smithi123 ceph-mon[37161]: fsmap cephfs:1 {0=cephfs.smithi114.zzryke=up:active} 3 up:standby 2024-02-16T22:18:19.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:19 smithi114 ceph-mon[29235]: mds.? [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] up:standby 2024-02-16T22:18:19.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:19 smithi114 ceph-mon[29235]: fsmap cephfs:1 {0=cephfs.smithi114.zzryke=up:active} 3 up:standby 2024-02-16T22:18:19.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:19 smithi123 ceph-mon[37161]: mds.? [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] up:standby 2024-02-16T22:18:19.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:19 smithi123 ceph-mon[37161]: fsmap cephfs:1 {0=cephfs.smithi114.zzryke=up:active} 3 up:standby 2024-02-16T22:18:20.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:20 smithi114 ceph-mon[29235]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 2 op/s 2024-02-16T22:18:20.772 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:20 smithi123 ceph-mon[37161]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 2 op/s 2024-02-16T22:18:21.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.006+0000 7f31791d4700 1 -- 172.21.15.114:0/1895138266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3174100cb0 msgr2=0x7f3174101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:21.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.006+0000 7f31791d4700 1 --2- 172.21.15.114:0/1895138266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3174100cb0 0x7f3174101110 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3168009910 tx=0x7f3168009c20 comp rx=0 tx=0).stop 2024-02-16T22:18:21.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.006+0000 7f31791d4700 1 -- 172.21.15.114:0/1895138266 shutdown_connections 2024-02-16T22:18:21.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.006+0000 7f31791d4700 1 --2- 172.21.15.114:0/1895138266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3174100cb0 0x7f3174101110 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:21.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.006+0000 7f31791d4700 1 --2- 172.21.15.114:0/1895138266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31740ffab0 0x7f31740ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:21.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.006+0000 7f31791d4700 1 -- 172.21.15.114:0/1895138266 >> 172.21.15.114:0/1895138266 conn(0x7f31740fb030 msgr2=0x7f31740fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:21.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.007+0000 7f31791d4700 1 -- 172.21.15.114:0/1895138266 shutdown_connections 2024-02-16T22:18:21.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.007+0000 7f31791d4700 1 -- 172.21.15.114:0/1895138266 wait complete. 2024-02-16T22:18:21.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.007+0000 7f31791d4700 1 Processor -- start 2024-02-16T22:18:21.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.008+0000 7f31791d4700 1 -- start start 2024-02-16T22:18:21.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.008+0000 7f31791d4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31740ffab0 0x7f31741952e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:21.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.008+0000 7f31791d4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3174100cb0 0x7f3174195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:21.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.008+0000 7f31791d4700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3174195e40 con 0x7f31740ffab0 2024-02-16T22:18:21.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.008+0000 7f31791d4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3174195f80 con 0x7f3174100cb0 2024-02-16T22:18:21.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.008+0000 7f3172d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31740ffab0 0x7f31741952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:21.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.008+0000 7f3172d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31740ffab0 0x7f31741952e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38220/0 (socket says 172.21.15.114:38220) 2024-02-16T22:18:21.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.008+0000 7f3172d9d700 1 -- 172.21.15.114:0/1087898199 learned_addr learned my addr 172.21.15.114:0/1087898199 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:18:21.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.008+0000 7f317259c700 1 --2- 172.21.15.114:0/1087898199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3174100cb0 0x7f3174195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:21.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.008+0000 7f3172d9d700 1 -- 172.21.15.114:0/1087898199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3174100cb0 msgr2=0x7f3174195820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:21.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.008+0000 7f3172d9d700 1 --2- 172.21.15.114:0/1087898199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3174100cb0 0x7f3174195820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:21.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.008+0000 7f3172d9d700 1 -- 172.21.15.114:0/1087898199 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31680095f0 con 0x7f31740ffab0 2024-02-16T22:18:21.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.008+0000 7f317259c700 1 --2- 172.21.15.114:0/1087898199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3174100cb0 0x7f3174195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:18:21.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.009+0000 7f3172d9d700 1 --2- 172.21.15.114:0/1087898199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31740ffab0 0x7f31741952e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f316400b480 tx=0x7f316400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:21.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.009+0000 7f315bfff700 1 -- 172.21.15.114:0/1087898199 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31640118c0 con 0x7f31740ffab0 2024-02-16T22:18:21.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.009+0000 7f315bfff700 1 -- 172.21.15.114:0/1087898199 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3164010380 con 0x7f31740ffab0 2024-02-16T22:18:21.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.009+0000 7f31791d4700 1 -- 172.21.15.114:0/1087898199 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f317419aa30 con 0x7f31740ffab0 2024-02-16T22:18:21.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.009+0000 7f315bfff700 1 -- 172.21.15.114:0/1087898199 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f316400f4f0 con 0x7f31740ffab0 2024-02-16T22:18:21.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.009+0000 7f31791d4700 1 -- 172.21.15.114:0/1087898199 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f317419b000 con 0x7f31740ffab0 2024-02-16T22:18:21.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.010+0000 7f31791d4700 1 -- 172.21.15.114:0/1087898199 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31741086b0 con 0x7f31740ffab0 2024-02-16T22:18:21.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.010+0000 7f315bfff700 1 -- 172.21.15.114:0/1087898199 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f31640104f0 con 0x7f31740ffab0 2024-02-16T22:18:21.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.011+0000 7f315bfff700 1 --2- 172.21.15.114:0/1087898199 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f315c06c4e0 0x7f315c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:21.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.011+0000 7f317259c700 1 --2- 172.21.15.114:0/1087898199 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f315c06c4e0 0x7f315c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:21.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.011+0000 7f315bfff700 1 -- 172.21.15.114:0/1087898199 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f316408b3c0 con 0x7f31740ffab0 2024-02-16T22:18:21.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.011+0000 7f317259c700 1 --2- 172.21.15.114:0/1087898199 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f315c06c4e0 0x7f315c06e9a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3168006010 tx=0x7f316800b560 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:21.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.015+0000 7f315bfff700 1 -- 172.21.15.114:0/1087898199 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f31640596b0 con 0x7f31740ffab0 2024-02-16T22:18:21.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.271+0000 7f31791d4700 1 -- 172.21.15.114:0/1087898199 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3174108870 con 0x7f31740ffab0 2024-02-16T22:18:21.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.271+0000 7f315bfff700 1 -- 172.21.15.114:0/1087898199 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1825 (secure 0 0 0) 0x7f3164015740 con 0x7f31740ffab0 2024-02-16T22:18:21.274 INFO:teuthology.orchestra.run.smithi114.stdout:e14 2024-02-16T22:18:21.274 INFO:teuthology.orchestra.run.smithi114.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:18:21.274 INFO:teuthology.orchestra.run.smithi114.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:18:21.274 INFO:teuthology.orchestra.run.smithi114.stdout:legacy client fscid: 1 2024-02-16T22:18:21.274 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:18:21.274 INFO:teuthology.orchestra.run.smithi114.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:18:21.274 INFO:teuthology.orchestra.run.smithi114.stdout:fs_name cephfs 2024-02-16T22:18:21.274 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 12 2024-02-16T22:18:21.274 INFO:teuthology.orchestra.run.smithi114.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:18:21.274 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-02-16T22:18:02.108825+0000 2024-02-16T22:18:21.274 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-02-16T22:18:16.413371+0000 2024-02-16T22:18:21.274 INFO:teuthology.orchestra.run.smithi114.stdout:tableserver 0 2024-02-16T22:18:21.274 INFO:teuthology.orchestra.run.smithi114.stdout:root 0 2024-02-16T22:18:21.274 INFO:teuthology.orchestra.run.smithi114.stdout:session_timeout 60 2024-02-16T22:18:21.274 INFO:teuthology.orchestra.run.smithi114.stdout:session_autoclose 300 2024-02-16T22:18:21.275 INFO:teuthology.orchestra.run.smithi114.stdout:max_file_size 1099511627776 2024-02-16T22:18:21.275 INFO:teuthology.orchestra.run.smithi114.stdout:required_client_features {} 2024-02-16T22:18:21.275 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure 0 2024-02-16T22:18:21.275 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure_osd_epoch 0 2024-02-16T22:18:21.275 INFO:teuthology.orchestra.run.smithi114.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:18:21.275 INFO:teuthology.orchestra.run.smithi114.stdout:max_mds 1 2024-02-16T22:18:21.275 INFO:teuthology.orchestra.run.smithi114.stdout:in 0 2024-02-16T22:18:21.275 INFO:teuthology.orchestra.run.smithi114.stdout:up {0=14468} 2024-02-16T22:18:21.275 INFO:teuthology.orchestra.run.smithi114.stdout:failed 2024-02-16T22:18:21.275 INFO:teuthology.orchestra.run.smithi114.stdout:damaged 2024-02-16T22:18:21.275 INFO:teuthology.orchestra.run.smithi114.stdout:stopped 2024-02-16T22:18:21.275 INFO:teuthology.orchestra.run.smithi114.stdout:data_pools [3] 2024-02-16T22:18:21.275 INFO:teuthology.orchestra.run.smithi114.stdout:metadata_pool 2 2024-02-16T22:18:21.275 INFO:teuthology.orchestra.run.smithi114.stdout:inline_data enabled 2024-02-16T22:18:21.275 INFO:teuthology.orchestra.run.smithi114.stdout:balancer 2024-02-16T22:18:21.275 INFO:teuthology.orchestra.run.smithi114.stdout:bal_rank_mask -1 2024-02-16T22:18:21.275 INFO:teuthology.orchestra.run.smithi114.stdout:standby_count_wanted 1 2024-02-16T22:18:21.276 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.zzryke{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.114:6826/277953535,v1:172.21.15.114:6827/277953535] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:18:21.276 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:18:21.276 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:18:21.276 INFO:teuthology.orchestra.run.smithi114.stdout:Standby daemons: 2024-02-16T22:18:21.276 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:18:21.276 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.fdxpey{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:18:21.276 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.vhlqbe{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:18:21.276 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.gopcfa{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:18:21.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.274+0000 7f31791d4700 1 -- 172.21.15.114:0/1087898199 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f315c06c4e0 msgr2=0x7f315c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:21.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.274+0000 7f31791d4700 1 --2- 172.21.15.114:0/1087898199 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f315c06c4e0 0x7f315c06e9a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3168006010 tx=0x7f316800b560 comp rx=0 tx=0).stop 2024-02-16T22:18:21.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.274+0000 7f31791d4700 1 -- 172.21.15.114:0/1087898199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31740ffab0 msgr2=0x7f31741952e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:21.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.274+0000 7f31791d4700 1 --2- 172.21.15.114:0/1087898199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31740ffab0 0x7f31741952e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f316400b480 tx=0x7f316400b840 comp rx=0 tx=0).stop 2024-02-16T22:18:21.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.274+0000 7f31791d4700 1 -- 172.21.15.114:0/1087898199 shutdown_connections 2024-02-16T22:18:21.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.274+0000 7f31791d4700 1 --2- 172.21.15.114:0/1087898199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31740ffab0 0x7f31741952e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:21.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.274+0000 7f31791d4700 1 --2- 172.21.15.114:0/1087898199 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f315c06c4e0 0x7f315c06e9a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:21.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.274+0000 7f31791d4700 1 --2- 172.21.15.114:0/1087898199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3174100cb0 0x7f3174195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:21.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.274+0000 7f31791d4700 1 -- 172.21.15.114:0/1087898199 >> 172.21.15.114:0/1087898199 conn(0x7f31740fb030 msgr2=0x7f3174103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:21.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.274+0000 7f31791d4700 1 -- 172.21.15.114:0/1087898199 shutdown_connections 2024-02-16T22:18:21.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:21.274+0000 7f31791d4700 1 -- 172.21.15.114:0/1087898199 wait complete. 2024-02-16T22:18:21.278 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 14 2024-02-16T22:18:21.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:21 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/1087898199' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:18:21.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:21 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/1087898199' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:18:21.993 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-02-16T22:18:22.440 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:18:22.793 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:22 smithi114 ceph-mon[29235]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 938 B/s wr, 1 op/s 2024-02-16T22:18:22.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:22 smithi123 ceph-mon[37161]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 938 B/s wr, 1 op/s 2024-02-16T22:18:23.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.968+0000 7f4e90407700 1 -- 172.21.15.114:0/1822442635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e880ffcf0 msgr2=0x7f4e88100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:23.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.968+0000 7f4e90407700 1 --2- 172.21.15.114:0/1822442635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e880ffcf0 0x7f4e88100170 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4e78009910 tx=0x7f4e78009c20 comp rx=0 tx=0).stop 2024-02-16T22:18:23.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.968+0000 7f4e90407700 1 -- 172.21.15.114:0/1822442635 shutdown_connections 2024-02-16T22:18:23.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.969+0000 7f4e90407700 1 --2- 172.21.15.114:0/1822442635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e880ffcf0 0x7f4e88100170 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:23.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.969+0000 7f4e90407700 1 --2- 172.21.15.114:0/1822442635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e88101090 0x7f4e881014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:23.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.969+0000 7f4e90407700 1 -- 172.21.15.114:0/1822442635 >> 172.21.15.114:0/1822442635 conn(0x7f4e880fa7c0 msgr2=0x7f4e880fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:23.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.969+0000 7f4e90407700 1 -- 172.21.15.114:0/1822442635 shutdown_connections 2024-02-16T22:18:23.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.969+0000 7f4e90407700 1 -- 172.21.15.114:0/1822442635 wait complete. 2024-02-16T22:18:23.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.969+0000 7f4e90407700 1 Processor -- start 2024-02-16T22:18:23.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.969+0000 7f4e90407700 1 -- start start 2024-02-16T22:18:23.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.969+0000 7f4e90407700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e880ffcf0 0x7f4e88198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:23.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.969+0000 7f4e90407700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e88101090 0x7f4e88198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:23.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.969+0000 7f4e90407700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e881990f0 con 0x7f4e88101090 2024-02-16T22:18:23.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.969+0000 7f4e90407700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e88199230 con 0x7f4e880ffcf0 2024-02-16T22:18:23.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.969+0000 7f4e8d9a2700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e88101090 0x7f4e88198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:23.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.969+0000 7f4e8d9a2700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e88101090 0x7f4e88198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50752/0 (socket says 172.21.15.114:50752) 2024-02-16T22:18:23.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.969+0000 7f4e8d9a2700 1 -- 172.21.15.114:0/192160703 learned_addr learned my addr 172.21.15.114:0/192160703 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:18:23.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.969+0000 7f4e8e1a3700 1 --2- 172.21.15.114:0/192160703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e880ffcf0 0x7f4e88198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:23.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.970+0000 7f4e8d9a2700 1 -- 172.21.15.114:0/192160703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e880ffcf0 msgr2=0x7f4e88198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:23.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.970+0000 7f4e8d9a2700 1 --2- 172.21.15.114:0/192160703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e880ffcf0 0x7f4e88198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:23.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.970+0000 7f4e8d9a2700 1 -- 172.21.15.114:0/192160703 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e780095f0 con 0x7f4e88101090 2024-02-16T22:18:23.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.970+0000 7f4e8d9a2700 1 --2- 172.21.15.114:0/192160703 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e88101090 0x7f4e88198ad0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800bda0 tx=0x7f4e7800bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:23.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.970+0000 7f4e7f7fe700 1 -- 172.21.15.114:0/192160703 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e7801d070 con 0x7f4e88101090 2024-02-16T22:18:23.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.970+0000 7f4e90407700 1 -- 172.21.15.114:0/192160703 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e8819dc80 con 0x7f4e88101090 2024-02-16T22:18:23.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.970+0000 7f4e7f7fe700 1 -- 172.21.15.114:0/192160703 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4e7800f460 con 0x7f4e88101090 2024-02-16T22:18:23.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.970+0000 7f4e7f7fe700 1 -- 172.21.15.114:0/192160703 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e78021640 con 0x7f4e88101090 2024-02-16T22:18:23.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.970+0000 7f4e90407700 1 -- 172.21.15.114:0/192160703 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e8819e170 con 0x7f4e88101090 2024-02-16T22:18:23.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.971+0000 7f4e90407700 1 -- 172.21.15.114:0/192160703 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e88108650 con 0x7f4e88101090 2024-02-16T22:18:23.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.972+0000 7f4e7f7fe700 1 -- 172.21.15.114:0/192160703 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f4e7800f5d0 con 0x7f4e88101090 2024-02-16T22:18:23.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.972+0000 7f4e7f7fe700 1 --2- 172.21.15.114:0/192160703 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f4e7406c4e0 0x7f4e7406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:23.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.972+0000 7f4e8e1a3700 1 --2- 172.21.15.114:0/192160703 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f4e7406c4e0 0x7f4e7406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:23.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.972+0000 7f4e7f7fe700 1 -- 172.21.15.114:0/192160703 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f4e7808cac0 con 0x7f4e88101090 2024-02-16T22:18:23.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.973+0000 7f4e8e1a3700 1 --2- 172.21.15.114:0/192160703 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f4e7406c4e0 0x7f4e7406e9a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4e88067ee0 tx=0x7f4e84007400 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:23.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:23.977+0000 7f4e7f7fe700 1 -- 172.21.15.114:0/192160703 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4e7805ad30 con 0x7f4e88101090 2024-02-16T22:18:24.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:24.224+0000 7f4e90407700 1 -- 172.21.15.114:0/192160703 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e88108840 con 0x7f4e88101090 2024-02-16T22:18:24.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:24.224+0000 7f4e7f7fe700 1 -- 172.21.15.114:0/192160703 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f4e7805a8c0 con 0x7f4e88101090 2024-02-16T22:18:24.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:24.226+0000 7f4e90407700 1 -- 172.21.15.114:0/192160703 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f4e7406c4e0 msgr2=0x7f4e7406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:24.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:24.226+0000 7f4e90407700 1 --2- 172.21.15.114:0/192160703 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f4e7406c4e0 0x7f4e7406e9a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4e88067ee0 tx=0x7f4e84007400 comp rx=0 tx=0).stop 2024-02-16T22:18:24.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:24.226+0000 7f4e90407700 1 -- 172.21.15.114:0/192160703 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e88101090 msgr2=0x7f4e88198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:24.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:24.226+0000 7f4e90407700 1 --2- 172.21.15.114:0/192160703 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e88101090 0x7f4e88198ad0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800bda0 tx=0x7f4e7800bdd0 comp rx=0 tx=0).stop 2024-02-16T22:18:24.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:24.226+0000 7f4e90407700 1 -- 172.21.15.114:0/192160703 shutdown_connections 2024-02-16T22:18:24.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:24.226+0000 7f4e90407700 1 --2- 172.21.15.114:0/192160703 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f4e7406c4e0 0x7f4e7406e9a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:24.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:24.226+0000 7f4e90407700 1 --2- 172.21.15.114:0/192160703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e880ffcf0 0x7f4e88198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:24.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:24.226+0000 7f4e90407700 1 --2- 172.21.15.114:0/192160703 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e88101090 0x7f4e88198ad0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:24.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:24.226+0000 7f4e90407700 1 -- 172.21.15.114:0/192160703 >> 172.21.15.114:0/192160703 conn(0x7f4e880fa7c0 msgr2=0x7f4e880febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:24.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:24.227+0000 7f4e90407700 1 -- 172.21.15.114:0/192160703 shutdown_connections 2024-02-16T22:18:24.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:24.227+0000 7f4e90407700 1 -- 172.21.15.114:0/192160703 wait complete. 2024-02-16T22:18:24.230 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 14 2024-02-16T22:18:24.241 INFO:teuthology.orchestra.run.smithi114.stdout:true 2024-02-16T22:18:24.770 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:24 smithi123 ceph-mon[37161]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-02-16T22:18:24.770 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:24 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:24.771 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:24 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:24.771 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:24 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:18:24.771 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:24 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:18:24.771 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:24 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:24.771 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:24 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:18:24.771 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:24 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/192160703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-16T22:18:24.855 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:24 smithi114 ceph-mon[29235]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-02-16T22:18:24.856 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:24 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:24.856 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:24 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:24.856 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:24 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:18:24.856 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:24 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:18:24.856 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:24 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:24.856 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:24 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:18:24.856 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:24 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/192160703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-16T22:18:24.879 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-02-16T22:18:25.620 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:18:26.791 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:26 smithi114 ceph-mon[29235]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:18:26.792 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:26.792 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:26.792 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:26.792 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:18:26.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:26 smithi123 ceph-mon[37161]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:18:26.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:26 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:26.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:26 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:26.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:26 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:26.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:26 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:18:28.157 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:27 smithi114 ceph-mon[29235]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:18:28.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:27 smithi123 ceph-mon[37161]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:18:28.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.469+0000 7ffb4834b700 1 -- 172.21.15.114:0/3196684364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb40100bc0 msgr2=0x7ffb40100fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:28.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.469+0000 7ffb4834b700 1 --2- 172.21.15.114:0/3196684364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb40100bc0 0x7ffb40100fe0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c009910 tx=0x7ffb3c009c20 comp rx=0 tx=0).stop 2024-02-16T22:18:28.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.469+0000 7ffb4834b700 1 -- 172.21.15.114:0/3196684364 shutdown_connections 2024-02-16T22:18:28.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.469+0000 7ffb4834b700 1 --2- 172.21.15.114:0/3196684364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb40101dc0 0x7ffb40102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:28.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.469+0000 7ffb4834b700 1 --2- 172.21.15.114:0/3196684364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb40100bc0 0x7ffb40100fe0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:28.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.469+0000 7ffb4834b700 1 -- 172.21.15.114:0/3196684364 >> 172.21.15.114:0/3196684364 conn(0x7ffb400fc140 msgr2=0x7ffb400fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:28.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.469+0000 7ffb4834b700 1 -- 172.21.15.114:0/3196684364 shutdown_connections 2024-02-16T22:18:28.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.469+0000 7ffb4834b700 1 -- 172.21.15.114:0/3196684364 wait complete. 2024-02-16T22:18:28.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.470+0000 7ffb4834b700 1 Processor -- start 2024-02-16T22:18:28.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.470+0000 7ffb4834b700 1 -- start start 2024-02-16T22:18:28.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.470+0000 7ffb4834b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb40100bc0 0x7ffb40074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:28.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.470+0000 7ffb4834b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb40101dc0 0x7ffb40073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:28.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.470+0000 7ffb4834b700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb400736c0 con 0x7ffb40100bc0 2024-02-16T22:18:28.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.470+0000 7ffb4834b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb40073800 con 0x7ffb40101dc0 2024-02-16T22:18:28.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.470+0000 7ffb460e7700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb40100bc0 0x7ffb40074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:28.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.470+0000 7ffb460e7700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb40100bc0 0x7ffb40074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50776/0 (socket says 172.21.15.114:50776) 2024-02-16T22:18:28.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.470+0000 7ffb460e7700 1 -- 172.21.15.114:0/1063967789 learned_addr learned my addr 172.21.15.114:0/1063967789 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:18:28.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.470+0000 7ffb458e6700 1 --2- 172.21.15.114:0/1063967789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb40101dc0 0x7ffb40073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:28.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.470+0000 7ffb460e7700 1 -- 172.21.15.114:0/1063967789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb40101dc0 msgr2=0x7ffb40073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:28.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.470+0000 7ffb460e7700 1 --2- 172.21.15.114:0/1063967789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb40101dc0 0x7ffb40073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:28.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.470+0000 7ffb460e7700 1 -- 172.21.15.114:0/1063967789 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3c0095f0 con 0x7ffb40100bc0 2024-02-16T22:18:28.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.470+0000 7ffb460e7700 1 --2- 172.21.15.114:0/1063967789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb40100bc0 0x7ffb40074b30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c000c00 tx=0x7ffb3c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:28.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.470+0000 7ffb377fe700 1 -- 172.21.15.114:0/1063967789 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb3c01d070 con 0x7ffb40100bc0 2024-02-16T22:18:28.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.470+0000 7ffb4834b700 1 -- 172.21.15.114:0/1063967789 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb40073a80 con 0x7ffb40100bc0 2024-02-16T22:18:28.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.471+0000 7ffb377fe700 1 -- 172.21.15.114:0/1063967789 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ffb3c005dc0 con 0x7ffb40100bc0 2024-02-16T22:18:28.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.471+0000 7ffb377fe700 1 -- 172.21.15.114:0/1063967789 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb3c022470 con 0x7ffb40100bc0 2024-02-16T22:18:28.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.471+0000 7ffb4834b700 1 -- 172.21.15.114:0/1063967789 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb40073f70 con 0x7ffb40100bc0 2024-02-16T22:18:28.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.472+0000 7ffb4834b700 1 -- 172.21.15.114:0/1063967789 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb40109720 con 0x7ffb40100bc0 2024-02-16T22:18:28.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.472+0000 7ffb377fe700 1 -- 172.21.15.114:0/1063967789 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7ffb3c003810 con 0x7ffb40100bc0 2024-02-16T22:18:28.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.472+0000 7ffb377fe700 1 --2- 172.21.15.114:0/1063967789 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ffb2c06c4e0 0x7ffb2c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:28.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.473+0000 7ffb377fe700 1 -- 172.21.15.114:0/1063967789 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7ffb3c08cdc0 con 0x7ffb40100bc0 2024-02-16T22:18:28.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.473+0000 7ffb458e6700 1 --2- 172.21.15.114:0/1063967789 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ffb2c06c4e0 0x7ffb2c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:28.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.473+0000 7ffb458e6700 1 --2- 172.21.15.114:0/1063967789 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ffb2c06c4e0 0x7ffb2c06e9a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ffb30009c60 tx=0x7ffb30009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:28.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.477+0000 7ffb377fe700 1 -- 172.21.15.114:0/1063967789 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ffb3c05b030 con 0x7ffb40100bc0 2024-02-16T22:18:28.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.754+0000 7ffb4834b700 1 -- 172.21.15.114:0/1063967789 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7ffb4019e810 con 0x7ffb40100bc0 2024-02-16T22:18:28.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.754+0000 7ffb377fe700 1 -- 172.21.15.114:0/1063967789 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v14) v1 ==== 78+0+83 (secure 0 0 0) 0x7ffb3c05abc0 con 0x7ffb40100bc0 2024-02-16T22:18:28.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.757+0000 7ffb4834b700 1 -- 172.21.15.114:0/1063967789 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ffb2c06c4e0 msgr2=0x7ffb2c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:28.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.757+0000 7ffb4834b700 1 --2- 172.21.15.114:0/1063967789 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ffb2c06c4e0 0x7ffb2c06e9a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ffb30009c60 tx=0x7ffb30009260 comp rx=0 tx=0).stop 2024-02-16T22:18:28.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.757+0000 7ffb4834b700 1 -- 172.21.15.114:0/1063967789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb40100bc0 msgr2=0x7ffb40074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:28.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.757+0000 7ffb4834b700 1 --2- 172.21.15.114:0/1063967789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb40100bc0 0x7ffb40074b30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c000c00 tx=0x7ffb3c00f690 comp rx=0 tx=0).stop 2024-02-16T22:18:28.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.757+0000 7ffb4834b700 1 -- 172.21.15.114:0/1063967789 shutdown_connections 2024-02-16T22:18:28.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.757+0000 7ffb4834b700 1 --2- 172.21.15.114:0/1063967789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb40100bc0 0x7ffb40074b30 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:28.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.757+0000 7ffb4834b700 1 --2- 172.21.15.114:0/1063967789 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ffb2c06c4e0 0x7ffb2c06e9a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:28.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.757+0000 7ffb4834b700 1 --2- 172.21.15.114:0/1063967789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb40101dc0 0x7ffb40073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:28.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.757+0000 7ffb4834b700 1 -- 172.21.15.114:0/1063967789 >> 172.21.15.114:0/1063967789 conn(0x7ffb400fc140 msgr2=0x7ffb40104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:28.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.757+0000 7ffb4834b700 1 -- 172.21.15.114:0/1063967789 shutdown_connections 2024-02-16T22:18:28.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:28.757+0000 7ffb4834b700 1 -- 172.21.15.114:0/1063967789 wait complete. 2024-02-16T22:18:28.769 INFO:teuthology.orchestra.run.smithi114.stdout:true 2024-02-16T22:18:29.008 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:28 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/1063967789' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-02-16T22:18:29.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:28 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/1063967789' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-02-16T22:18:29.720 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-02-16T22:18:29.734 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-16T22:18:29.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.848+0000 7fc546d5c640 1 -- 172.21.15.114:0/3458499657 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5400fefb0 msgr2=0x7fc5400ff3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:29.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.848+0000 7fc546d5c640 1 --2- 172.21.15.114:0/3458499657 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5400fefb0 0x7fc5400ff3d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc528009880 tx=0x7fc52802bce0 comp rx=0 tx=0).stop 2024-02-16T22:18:29.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.848+0000 7fc546d5c640 1 -- 172.21.15.114:0/3458499657 shutdown_connections 2024-02-16T22:18:29.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.848+0000 7fc546d5c640 1 --2- 172.21.15.114:0/3458499657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5401001a0 0x7fc540100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:29.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.848+0000 7fc546d5c640 1 --2- 172.21.15.114:0/3458499657 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5400fefb0 0x7fc5400ff3d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:29.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.848+0000 7fc546d5c640 1 -- 172.21.15.114:0/3458499657 >> 172.21.15.114:0/3458499657 conn(0x7fc5400fa4e0 msgr2=0x7fc5400fc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:29.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.848+0000 7fc546d5c640 1 -- 172.21.15.114:0/3458499657 shutdown_connections 2024-02-16T22:18:29.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.848+0000 7fc546d5c640 1 -- 172.21.15.114:0/3458499657 wait complete. 2024-02-16T22:18:29.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.848+0000 7fc546d5c640 1 Processor -- start 2024-02-16T22:18:29.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.848+0000 7fc546d5c640 1 -- start start 2024-02-16T22:18:29.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.848+0000 7fc546d5c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5400fefb0 0x7fc5401958d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:29.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.848+0000 7fc546d5c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5401001a0 0x7fc540195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:29.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.848+0000 7fc546d5c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5401963e0 con 0x7fc5400fefb0 2024-02-16T22:18:29.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.848+0000 7fc546d5c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc540196550 con 0x7fc5401001a0 2024-02-16T22:18:29.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.848+0000 7fc544af8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5400fefb0 0x7fc5401958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:29.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.848+0000 7fc544af8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5400fefb0 0x7fc5401958d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50788/0 (socket says 172.21.15.114:50788) 2024-02-16T22:18:29.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.848+0000 7fc544af8640 1 -- 172.21.15.114:0/1227572614 learned_addr learned my addr 172.21.15.114:0/1227572614 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:18:29.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.849+0000 7fc53ffff640 1 --2- 172.21.15.114:0/1227572614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5401001a0 0x7fc540195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:29.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.849+0000 7fc544af8640 1 -- 172.21.15.114:0/1227572614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5401001a0 msgr2=0x7fc540195e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:29.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.849+0000 7fc544af8640 1 --2- 172.21.15.114:0/1227572614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5401001a0 0x7fc540195e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:29.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.849+0000 7fc544af8640 1 -- 172.21.15.114:0/1227572614 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc528009560 con 0x7fc5400fefb0 2024-02-16T22:18:29.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.849+0000 7fc544af8640 1 --2- 172.21.15.114:0/1227572614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5400fefb0 0x7fc5401958d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc528000c00 tx=0x7fc528004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:29.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.849+0000 7fc53dffb640 1 -- 172.21.15.114:0/1227572614 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc528004520 con 0x7fc5400fefb0 2024-02-16T22:18:29.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.849+0000 7fc53dffb640 1 -- 172.21.15.114:0/1227572614 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc5280054b0 con 0x7fc5400fefb0 2024-02-16T22:18:29.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.849+0000 7fc546d5c640 1 -- 172.21.15.114:0/1227572614 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54019af80 con 0x7fc5400fefb0 2024-02-16T22:18:29.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.849+0000 7fc53dffb640 1 -- 172.21.15.114:0/1227572614 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc528034440 con 0x7fc5400fefb0 2024-02-16T22:18:29.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.849+0000 7fc546d5c640 1 -- 172.21.15.114:0/1227572614 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc54019b400 con 0x7fc5400fefb0 2024-02-16T22:18:29.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.850+0000 7fc546d5c640 1 -- 172.21.15.114:0/1227572614 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc540107d20 con 0x7fc5400fefb0 2024-02-16T22:18:29.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.851+0000 7fc53dffb640 1 -- 172.21.15.114:0/1227572614 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fc528005000 con 0x7fc5400fefb0 2024-02-16T22:18:29.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.851+0000 7fc53dffb640 1 --2- 172.21.15.114:0/1227572614 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fc52c06c510 0x7fc52c06e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:29.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.851+0000 7fc53dffb640 1 -- 172.21.15.114:0/1227572614 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fc5280adeb0 con 0x7fc5400fefb0 2024-02-16T22:18:29.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.851+0000 7fc53ffff640 1 --2- 172.21.15.114:0/1227572614 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fc52c06c510 0x7fc52c06e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:29.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.851+0000 7fc53ffff640 1 --2- 172.21.15.114:0/1227572614 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fc52c06c510 0x7fc52c06e9d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc540196de0 tx=0x7fc53000a4c0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:29.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:29.855+0000 7fc53dffb640 1 -- 172.21.15.114:0/1227572614 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc528078690 con 0x7fc5400fefb0 2024-02-16T22:18:29.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:29 smithi114 ceph-mon[29235]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:18:30.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.044+0000 7fc546d5c640 1 -- 172.21.15.114:0/1227572614 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc54019b710 con 0x7fc5400fefb0 2024-02-16T22:18:30.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.045+0000 7fc53dffb640 1 -- 172.21.15.114:0/1227572614 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+15071 (secure 0 0 0) 0x7fc52807bcb0 con 0x7fc5400fefb0 2024-02-16T22:18:30.047 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:18:30.047 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":45,"fsid":"1dcbaabe-cd18-11ee-95ba-87774f69a715","created":"2024-02-16T22:11:02.186164+0000","modified":"2024-02-16T22:18:04.113145+0000","last_up_change":"2024-02-16T22:16:47.516304+0000","last_in_change":"2024-02-16T22:16:28.224489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-16T22:15:31.749346+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-16T22:18:01.042512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-16T22:18:01.336617+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3a72ccf1-3f93-4c06-99c3-092a7f9a732d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1379159586},{"type":"v1","addr":"172.21.15.114:6803","nonce":1379159586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1379159586},{"type":"v1","addr":"172.21.15.114:6805","nonce":1379159586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1379159586},{"type":"v1","addr":"172.21.15.114:6809","nonce":1379159586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1379159586},{"type":"v1","addr":"172.21.15.114:6807","nonce":1379159586}]},"public_addr":"172.21.15.114:6803/1379159586","cluster_addr":"172.21.15.114:6805/1379159586","heartbeat_back_addr":"172.21.15.114:6809/1379159586","heartbeat_front_addr":"172.21.15.114:6807/1379159586","state":["exists","up"]},{"osd":1,"uuid":"1e8829dd-cac2-462f-aca1-86bcfb475a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1548199763},{"type":"v1","addr":"172.21.15.114:6811","nonce":1548199763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1548199763},{"type":"v1","addr":"172.21.15.114:6813","nonce":1548199763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1548199763},{"type":"v1","addr":"172.21.15.114:6817","nonce":1548199763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1548199763},{"type":"v1","addr":"172.21.15.114:6815","nonce":1548199763}]},"public_addr":"172.21.15.114:6811/1548199763","cluster_addr":"172.21.15.114:6813/1548199763","heartbeat_back_addr":"172.21.15.114:6817/1548199763","heartbeat_front_addr":"172.21.15.114:6815/1548199763","state":["exists","up"]},{"osd":2,"uuid":"44f9fce8-5ea8-4741-9936-6720df91b628","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1293302101},{"type":"v1","addr":"172.21.15.114:6819","nonce":1293302101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1293302101},{"type":"v1","addr":"172.21.15.114:6821","nonce":1293302101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1293302101},{"type":"v1","addr":"172.21.15.114:6825","nonce":1293302101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1293302101},{"type":"v1","addr":"172.21.15.114:6823","nonce":1293302101}]},"public_addr":"172.21.15.114:6819/1293302101","cluster_addr":"172.21.15.114:6821/1293302101","heartbeat_back_addr":"172.21.15.114:6825/1293302101","heartbeat_front_addr":"172.21.15.114:6823/1293302101","state":["exists","up"]},{"osd":3,"uuid":"b6eb4941-bef4-4722-9050-cd6c041f9fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3211307893},{"type":"v1","addr":"172.21.15.123:6801","nonce":3211307893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3211307893},{"type":"v1","addr":"172.21.15.123:6803","nonce":3211307893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3211307893},{"type":"v1","addr":"172.21.15.123:6807","nonce":3211307893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3211307893},{"type":"v1","addr":"172.21.15.123:6805","nonce":3211307893}]},"public_addr":"172.21.15.123:6801/3211307893","cluster_addr":"172.21.15.123:6803/3211307893","heartbeat_back_addr":"172.21.15.123:6807/3211307893","heartbeat_front_addr":"172.21.15.123:6805/3211307893","state":["exists","up"]},{"osd":4,"uuid":"e9caa348-26f4-4ed5-a64e-59231fc48546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3994684204},{"type":"v1","addr":"172.21.15.123:6809","nonce":3994684204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3994684204},{"type":"v1","addr":"172.21.15.123:6811","nonce":3994684204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3994684204},{"type":"v1","addr":"172.21.15.123:6815","nonce":3994684204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3994684204},{"type":"v1","addr":"172.21.15.123:6813","nonce":3994684204}]},"public_addr":"172.21.15.123:6809/3994684204","cluster_addr":"172.21.15.123:6811/3994684204","heartbeat_back_addr":"172.21.15.123:6815/3994684204","heartbeat_front_addr":"172.21.15.123:6813/3994684204","state":["exists","up"]},{"osd":5,"uuid":"79d81af1-2361-430d-bc76-35ef74aa6bad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2122343545},{"type":"v1","addr":"172.21.15.123:6817","nonce":2122343545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":2122343545},{"type":"v1","addr":"172.21.15.123:6819","nonce":2122343545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":2122343545},{"type":"v1","addr":"172.21.15.123:6823","nonce":2122343545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2122343545},{"type":"v1","addr":"172.21.15.123:6821","nonce":2122343545}]},"public_addr":"172.21.15.123:6817/2122343545","cluster_addr":"172.21.15.123:6819/2122343545","heartbeat_back_addr":"172.21.15.123:6823/2122343545","heartbeat_front_addr":"172.21.15.123:6821/2122343545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:14:32.818023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:15:00.597924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:15:29.574022+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:15:53.159458+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:16:20.058254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:16:45.789725+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:0/2762476246":"2024-02-17T22:13:11.405214+0000","172.21.15.114:6801/2648536185":"2024-02-17T22:13:11.405214+0000","172.21.15.114:0/1034649687":"2024-02-17T22:13:11.405214+0000","172.21.15.114:0/1349416361":"2024-02-17T22:12:00.492971+0000","172.21.15.114:0/499892900":"2024-02-17T22:12:00.492971+0000","172.21.15.114:0/744140874":"2024-02-17T22:12:00.492971+0000","172.21.15.114:6800/2648536185":"2024-02-17T22:13:11.405214+0000","172.21.15.114:6801/625910309":"2024-02-17T22:12:00.492971+0000","172.21.15.114:6800/2055574123":"2024-02-17T22:11:27.861007+0000","172.21.15.114:0/3971798486":"2024-02-17T22:11:27.861007+0000","172.21.15.114:0/4029631970":"2024-02-17T22:11:27.861007+0000","172.21.15.114:0/3211326148":"2024-02-17T22:13:11.405214+0000","172.21.15.114:6800/625910309":"2024-02-17T22:12:00.492971+0000","172.21.15.114:6801/2055574123":"2024-02-17T22:11:27.861007+0000","172.21.15.114:0/2825621999":"2024-02-17T22:11:27.861007+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-16T22:18:30.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.048+0000 7fc546d5c640 1 -- 172.21.15.114:0/1227572614 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fc52c06c510 msgr2=0x7fc52c06e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:30.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.048+0000 7fc546d5c640 1 --2- 172.21.15.114:0/1227572614 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fc52c06c510 0x7fc52c06e9d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc540196de0 tx=0x7fc53000a4c0 comp rx=0 tx=0).stop 2024-02-16T22:18:30.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.048+0000 7fc546d5c640 1 -- 172.21.15.114:0/1227572614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5400fefb0 msgr2=0x7fc5401958d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:30.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.048+0000 7fc546d5c640 1 --2- 172.21.15.114:0/1227572614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5400fefb0 0x7fc5401958d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc528000c00 tx=0x7fc528004f00 comp rx=0 tx=0).stop 2024-02-16T22:18:30.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.048+0000 7fc546d5c640 1 -- 172.21.15.114:0/1227572614 shutdown_connections 2024-02-16T22:18:30.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.048+0000 7fc546d5c640 1 --2- 172.21.15.114:0/1227572614 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fc52c06c510 0x7fc52c06e9d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:30.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.048+0000 7fc546d5c640 1 --2- 172.21.15.114:0/1227572614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5401001a0 0x7fc540195e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:30.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.048+0000 7fc546d5c640 1 --2- 172.21.15.114:0/1227572614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5400fefb0 0x7fc5401958d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:30.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.048+0000 7fc546d5c640 1 -- 172.21.15.114:0/1227572614 >> 172.21.15.114:0/1227572614 conn(0x7fc5400fa4e0 msgr2=0x7fc540067350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:30.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.048+0000 7fc546d5c640 1 -- 172.21.15.114:0/1227572614 shutdown_connections 2024-02-16T22:18:30.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.048+0000 7fc546d5c640 1 -- 172.21.15.114:0/1227572614 wait complete. 2024-02-16T22:18:30.060 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-16T22:15:31.749346+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-16T22:18:01.042512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-16T22:18:01.336617+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-16T22:18:30.060 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-16T22:18:30.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.166+0000 7f749e539640 1 -- 172.21.15.114:0/4293936646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7498100180 msgr2=0x7f7498100600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:30.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.166+0000 7f749e539640 1 --2- 172.21.15.114:0/4293936646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7498100180 0x7f7498100600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f748c009880 tx=0x7f748c02bce0 comp rx=0 tx=0).stop 2024-02-16T22:18:30.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.166+0000 7f749e539640 1 -- 172.21.15.114:0/4293936646 shutdown_connections 2024-02-16T22:18:30.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.166+0000 7f749e539640 1 --2- 172.21.15.114:0/4293936646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7498100180 0x7f7498100600 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:30.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.166+0000 7f749e539640 1 --2- 172.21.15.114:0/4293936646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74980fef90 0x7f74980ff3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:30.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.166+0000 7f749e539640 1 -- 172.21.15.114:0/4293936646 >> 172.21.15.114:0/4293936646 conn(0x7f74980fa4e0 msgr2=0x7f74980fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:30.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.167+0000 7f749e539640 1 -- 172.21.15.114:0/4293936646 shutdown_connections 2024-02-16T22:18:30.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.167+0000 7f749e539640 1 -- 172.21.15.114:0/4293936646 wait complete. 2024-02-16T22:18:30.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.167+0000 7f749e539640 1 Processor -- start 2024-02-16T22:18:30.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.167+0000 7f749e539640 1 -- start start 2024-02-16T22:18:30.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.167+0000 7f749e539640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74980fef90 0x7f7498067ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:30.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.167+0000 7f749e539640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7498100180 0x7f749819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:30.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.167+0000 7f749e539640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749819ac70 con 0x7f7498100180 2024-02-16T22:18:30.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.167+0000 7f749e539640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749819adb0 con 0x7f74980fef90 2024-02-16T22:18:30.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.168+0000 7f74977fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7498100180 0x7f749819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:30.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.168+0000 7f74977fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7498100180 0x7f749819a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50802/0 (socket says 172.21.15.114:50802) 2024-02-16T22:18:30.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.168+0000 7f74977fe640 1 -- 172.21.15.114:0/3380637035 learned_addr learned my addr 172.21.15.114:0/3380637035 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:18:30.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.168+0000 7f7497fff640 1 --2- 172.21.15.114:0/3380637035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74980fef90 0x7f7498067ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:30.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.168+0000 7f74977fe640 1 -- 172.21.15.114:0/3380637035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74980fef90 msgr2=0x7f7498067ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:30.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.168+0000 7f74977fe640 1 --2- 172.21.15.114:0/3380637035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74980fef90 0x7f7498067ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:30.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.168+0000 7f74977fe640 1 -- 172.21.15.114:0/3380637035 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748c009560 con 0x7f7498100180 2024-02-16T22:18:30.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.168+0000 7f74977fe640 1 --2- 172.21.15.114:0/3380637035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7498100180 0x7f749819a6a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f748c009430 tx=0x7f748c02cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:30.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.168+0000 7f74957fa640 1 -- 172.21.15.114:0/3380637035 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f748c004520 con 0x7f7498100180 2024-02-16T22:18:30.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.168+0000 7f749e539640 1 -- 172.21.15.114:0/3380637035 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f749819b000 con 0x7f7498100180 2024-02-16T22:18:30.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.168+0000 7f74957fa640 1 -- 172.21.15.114:0/3380637035 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f748c0054b0 con 0x7f7498100180 2024-02-16T22:18:30.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.168+0000 7f74957fa640 1 -- 172.21.15.114:0/3380637035 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f748c034440 con 0x7f7498100180 2024-02-16T22:18:30.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.168+0000 7f749e539640 1 -- 172.21.15.114:0/3380637035 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f749819b500 con 0x7f7498100180 2024-02-16T22:18:30.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.169+0000 7f749e539640 1 -- 172.21.15.114:0/3380637035 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7498107ce0 con 0x7f7498100180 2024-02-16T22:18:30.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.170+0000 7f74957fa640 1 -- 172.21.15.114:0/3380637035 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f748c005000 con 0x7f7498100180 2024-02-16T22:18:30.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.170+0000 7f74957fa640 1 --2- 172.21.15.114:0/3380637035 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f748006c450 0x7f748006e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:30.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.170+0000 7f74957fa640 1 -- 172.21.15.114:0/3380637035 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f748c0adf40 con 0x7f7498100180 2024-02-16T22:18:30.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.170+0000 7f7497fff640 1 --2- 172.21.15.114:0/3380637035 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f748006c450 0x7f748006e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:30.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.171+0000 7f7497fff640 1 --2- 172.21.15.114:0/3380637035 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f748006c450 0x7f748006e910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f74880058c0 tx=0x7f748800a4c0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:30.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.174+0000 7f74957fa640 1 -- 172.21.15.114:0/3380637035 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f748c0e10e0 con 0x7f7498100180 2024-02-16T22:18:30.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:29 smithi123 ceph-mon[37161]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:18:30.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.343+0000 7f749e539640 1 -- 172.21.15.114:0/3380637035 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f749819b720 con 0x7f7498100180 2024-02-16T22:18:30.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.344+0000 7f74957fa640 1 -- 172.21.15.114:0/3380637035 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7f748c078720 con 0x7f7498100180 2024-02-16T22:18:30.345 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2024-02-16T22:18:30.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.346+0000 7f749e539640 1 -- 172.21.15.114:0/3380637035 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f748006c450 msgr2=0x7f748006e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:30.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.346+0000 7f749e539640 1 --2- 172.21.15.114:0/3380637035 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f748006c450 0x7f748006e910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f74880058c0 tx=0x7f748800a4c0 comp rx=0 tx=0).stop 2024-02-16T22:18:30.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.346+0000 7f749e539640 1 -- 172.21.15.114:0/3380637035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7498100180 msgr2=0x7f749819a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:30.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.346+0000 7f749e539640 1 --2- 172.21.15.114:0/3380637035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7498100180 0x7f749819a6a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f748c009430 tx=0x7f748c02cec0 comp rx=0 tx=0).stop 2024-02-16T22:18:30.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.346+0000 7f749e539640 1 -- 172.21.15.114:0/3380637035 shutdown_connections 2024-02-16T22:18:30.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.346+0000 7f749e539640 1 --2- 172.21.15.114:0/3380637035 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f748006c450 0x7f748006e910 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:30.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.346+0000 7f749e539640 1 --2- 172.21.15.114:0/3380637035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7498100180 0x7f749819a6a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:30.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.346+0000 7f749e539640 1 --2- 172.21.15.114:0/3380637035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74980fef90 0x7f7498067ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:30.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.346+0000 7f749e539640 1 -- 172.21.15.114:0/3380637035 >> 172.21.15.114:0/3380637035 conn(0x7f74980fa4e0 msgr2=0x7f74981033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:30.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.346+0000 7f749e539640 1 -- 172.21.15.114:0/3380637035 shutdown_connections 2024-02-16T22:18:30.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.346+0000 7f749e539640 1 -- 172.21.15.114:0/3380637035 wait complete. 2024-02-16T22:18:30.360 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-16T22:18:30.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.472+0000 7f7ad2a5f640 1 -- 172.21.15.114:0/855032602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7acc1000c0 msgr2=0x7f7acc1004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:30.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.472+0000 7f7ad2a5f640 1 --2- 172.21.15.114:0/855032602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7acc1000c0 0x7f7acc1004e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009880 tx=0x7f7abc02bce0 comp rx=0 tx=0).stop 2024-02-16T22:18:30.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.472+0000 7f7ad2a5f640 1 -- 172.21.15.114:0/855032602 shutdown_connections 2024-02-16T22:18:30.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.472+0000 7f7ad2a5f640 1 --2- 172.21.15.114:0/855032602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7acc1012b0 0x7f7acc101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:30.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.472+0000 7f7ad2a5f640 1 --2- 172.21.15.114:0/855032602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7acc1000c0 0x7f7acc1004e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:30.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.472+0000 7f7ad2a5f640 1 -- 172.21.15.114:0/855032602 >> 172.21.15.114:0/855032602 conn(0x7f7acc0fb650 msgr2=0x7f7acc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:30.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.472+0000 7f7ad2a5f640 1 -- 172.21.15.114:0/855032602 shutdown_connections 2024-02-16T22:18:30.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.472+0000 7f7ad2a5f640 1 -- 172.21.15.114:0/855032602 wait complete. 2024-02-16T22:18:30.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.473+0000 7f7ad2a5f640 1 Processor -- start 2024-02-16T22:18:30.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.473+0000 7f7ad2a5f640 1 -- start start 2024-02-16T22:18:30.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.473+0000 7f7ad2a5f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7acc1000c0 0x7f7acc1959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:30.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.473+0000 7f7ad2a5f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7acc1012b0 0x7f7acc195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:30.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.473+0000 7f7ad2a5f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc1964b0 con 0x7f7acc1000c0 2024-02-16T22:18:30.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.473+0000 7f7ad2a5f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc196620 con 0x7f7acc1012b0 2024-02-16T22:18:30.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.473+0000 7f7acbfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7acc1000c0 0x7f7acc1959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:30.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.473+0000 7f7acbfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7acc1000c0 0x7f7acc1959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50822/0 (socket says 172.21.15.114:50822) 2024-02-16T22:18:30.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.473+0000 7f7acbfff640 1 -- 172.21.15.114:0/1431463387 learned_addr learned my addr 172.21.15.114:0/1431463387 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:18:30.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.473+0000 7f7acb7fe640 1 --2- 172.21.15.114:0/1431463387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7acc1012b0 0x7f7acc195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:30.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.473+0000 7f7acbfff640 1 -- 172.21.15.114:0/1431463387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7acc1012b0 msgr2=0x7f7acc195ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:30.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.473+0000 7f7acbfff640 1 --2- 172.21.15.114:0/1431463387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7acc1012b0 0x7f7acc195ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:30.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.473+0000 7f7acbfff640 1 -- 172.21.15.114:0/1431463387 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7abc009560 con 0x7f7acc1000c0 2024-02-16T22:18:30.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.473+0000 7f7acb7fe640 1 --2- 172.21.15.114:0/1431463387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7acc1012b0 0x7f7acc195ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:18:30.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.473+0000 7f7acbfff640 1 --2- 172.21.15.114:0/1431463387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7acc1000c0 0x7f7acc1959a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009230 tx=0x7f7abc004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:30.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.474+0000 7f7ac97fa640 1 -- 172.21.15.114:0/1431463387 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7abc0044e0 con 0x7f7acc1000c0 2024-02-16T22:18:30.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.474+0000 7f7ad2a5f640 1 -- 172.21.15.114:0/1431463387 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7acc19b050 con 0x7f7acc1000c0 2024-02-16T22:18:30.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.474+0000 7f7ac97fa640 1 -- 172.21.15.114:0/1431463387 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7abc0054b0 con 0x7f7acc1000c0 2024-02-16T22:18:30.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.474+0000 7f7ad2a5f640 1 -- 172.21.15.114:0/1431463387 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7acc19b550 con 0x7f7acc1000c0 2024-02-16T22:18:30.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.474+0000 7f7ac97fa640 1 -- 172.21.15.114:0/1431463387 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7abc034440 con 0x7f7acc1000c0 2024-02-16T22:18:30.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.475+0000 7f7ad2a5f640 1 -- 172.21.15.114:0/1431463387 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7acc108e50 con 0x7f7acc1000c0 2024-02-16T22:18:30.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.475+0000 7f7ac97fa640 1 -- 172.21.15.114:0/1431463387 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f7abc003810 con 0x7f7acc1000c0 2024-02-16T22:18:30.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.476+0000 7f7ac97fa640 1 --2- 172.21.15.114:0/1431463387 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f7ab406c4a0 0x7f7ab406e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:30.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.476+0000 7f7ac97fa640 1 -- 172.21.15.114:0/1431463387 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f7abc0add10 con 0x7f7acc1000c0 2024-02-16T22:18:30.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.476+0000 7f7acb7fe640 1 --2- 172.21.15.114:0/1431463387 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f7ab406c4a0 0x7f7ab406e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:30.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.476+0000 7f7acb7fe640 1 --2- 172.21.15.114:0/1431463387 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f7ab406c4a0 0x7f7ab406e960 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7acc196eb0 tx=0x7f7ac0005f90 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:30.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.482+0000 7f7ac97fa640 1 -- 172.21.15.114:0/1431463387 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7abc0784d0 con 0x7f7acc1000c0 2024-02-16T22:18:30.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.661+0000 7f7ad2a5f640 1 -- 172.21.15.114:0/1431463387 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f7acc19b880 con 0x7f7acc1000c0 2024-02-16T22:18:30.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.662+0000 7f7ac97fa640 1 -- 172.21.15.114:0/1431463387 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v45) v1 ==== 107+0+11 (secure 0 0 0) 0x7f7abc041020 con 0x7f7acc1000c0 2024-02-16T22:18:30.664 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-02-16T22:18:30.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.664+0000 7f7ad2a5f640 1 -- 172.21.15.114:0/1431463387 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f7ab406c4a0 msgr2=0x7f7ab406e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:30.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.664+0000 7f7ad2a5f640 1 --2- 172.21.15.114:0/1431463387 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f7ab406c4a0 0x7f7ab406e960 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7acc196eb0 tx=0x7f7ac0005f90 comp rx=0 tx=0).stop 2024-02-16T22:18:30.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.664+0000 7f7ad2a5f640 1 -- 172.21.15.114:0/1431463387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7acc1000c0 msgr2=0x7f7acc1959a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:30.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.664+0000 7f7ad2a5f640 1 --2- 172.21.15.114:0/1431463387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7acc1000c0 0x7f7acc1959a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009230 tx=0x7f7abc004dc0 comp rx=0 tx=0).stop 2024-02-16T22:18:30.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.664+0000 7f7ad2a5f640 1 -- 172.21.15.114:0/1431463387 shutdown_connections 2024-02-16T22:18:30.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.664+0000 7f7ad2a5f640 1 --2- 172.21.15.114:0/1431463387 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f7ab406c4a0 0x7f7ab406e960 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:30.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.664+0000 7f7ad2a5f640 1 --2- 172.21.15.114:0/1431463387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7acc1012b0 0x7f7acc195ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:30.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.664+0000 7f7ad2a5f640 1 --2- 172.21.15.114:0/1431463387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7acc1000c0 0x7f7acc1959a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:30.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.664+0000 7f7ad2a5f640 1 -- 172.21.15.114:0/1431463387 >> 172.21.15.114:0/1431463387 conn(0x7f7acc0fb650 msgr2=0x7f7acc1044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:30.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.664+0000 7f7ad2a5f640 1 -- 172.21.15.114:0/1431463387 shutdown_connections 2024-02-16T22:18:30.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.664+0000 7f7ad2a5f640 1 -- 172.21.15.114:0/1431463387 wait complete. 2024-02-16T22:18:30.677 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-16T22:18:30.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.779+0000 7fb9de7fd640 1 -- 172.21.15.114:0/1641346397 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9d80fef90 msgr2=0x7fb9d80ff3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:30.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.779+0000 7fb9de7fd640 1 --2- 172.21.15.114:0/1641346397 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9d80fef90 0x7fb9d80ff3b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8009880 tx=0x7fb9c802bce0 comp rx=0 tx=0).stop 2024-02-16T22:18:30.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.779+0000 7fb9de7fd640 1 -- 172.21.15.114:0/1641346397 shutdown_connections 2024-02-16T22:18:30.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.779+0000 7fb9de7fd640 1 --2- 172.21.15.114:0/1641346397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9d8100180 0x7fb9d8100600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:30.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.779+0000 7fb9de7fd640 1 --2- 172.21.15.114:0/1641346397 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9d80fef90 0x7fb9d80ff3b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:30.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.779+0000 7fb9de7fd640 1 -- 172.21.15.114:0/1641346397 >> 172.21.15.114:0/1641346397 conn(0x7fb9d80fa4e0 msgr2=0x7fb9d80fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:30.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.779+0000 7fb9de7fd640 1 -- 172.21.15.114:0/1641346397 shutdown_connections 2024-02-16T22:18:30.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.779+0000 7fb9de7fd640 1 -- 172.21.15.114:0/1641346397 wait complete. 2024-02-16T22:18:30.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.780+0000 7fb9de7fd640 1 Processor -- start 2024-02-16T22:18:30.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.780+0000 7fb9de7fd640 1 -- start start 2024-02-16T22:18:30.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.780+0000 7fb9de7fd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9d80fef90 0x7fb9d8195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:30.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.780+0000 7fb9de7fd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9d8100180 0x7fb9d8195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:30.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.780+0000 7fb9de7fd640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d81964a0 con 0x7fb9d8100180 2024-02-16T22:18:30.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.780+0000 7fb9de7fd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d8196610 con 0x7fb9d80fef90 2024-02-16T22:18:30.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.780+0000 7fb9d77fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9d8100180 0x7fb9d8195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:30.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.780+0000 7fb9d77fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9d8100180 0x7fb9d8195ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50834/0 (socket says 172.21.15.114:50834) 2024-02-16T22:18:30.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.780+0000 7fb9d77fe640 1 -- 172.21.15.114:0/3381425599 learned_addr learned my addr 172.21.15.114:0/3381425599 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:18:30.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.780+0000 7fb9d77fe640 1 -- 172.21.15.114:0/3381425599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9d80fef90 msgr2=0x7fb9d8195990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:30.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.780+0000 7fb9d77fe640 1 --2- 172.21.15.114:0/3381425599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9d80fef90 0x7fb9d8195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:30.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.780+0000 7fb9d77fe640 1 -- 172.21.15.114:0/3381425599 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c8009560 con 0x7fb9d8100180 2024-02-16T22:18:30.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.781+0000 7fb9d77fe640 1 --2- 172.21.15.114:0/3381425599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9d8100180 0x7fb9d8195ed0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc00c8f0 tx=0x7fb9cc00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:30.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.781+0000 7fb9d57fa640 1 -- 172.21.15.114:0/3381425599 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9cc007b80 con 0x7fb9d8100180 2024-02-16T22:18:30.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.781+0000 7fb9de7fd640 1 -- 172.21.15.114:0/3381425599 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9d819b0a0 con 0x7fb9d8100180 2024-02-16T22:18:30.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.781+0000 7fb9d57fa640 1 -- 172.21.15.114:0/3381425599 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb9cc007ce0 con 0x7fb9d8100180 2024-02-16T22:18:30.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.781+0000 7fb9d57fa640 1 -- 172.21.15.114:0/3381425599 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9cc00f450 con 0x7fb9d8100180 2024-02-16T22:18:30.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.781+0000 7fb9de7fd640 1 -- 172.21.15.114:0/3381425599 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d819b600 con 0x7fb9d8100180 2024-02-16T22:18:30.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.782+0000 7fb9de7fd640 1 -- 172.21.15.114:0/3381425599 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9d8107ce0 con 0x7fb9d8100180 2024-02-16T22:18:30.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.782+0000 7fb9d57fa640 1 -- 172.21.15.114:0/3381425599 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fb9cc004f60 con 0x7fb9d8100180 2024-02-16T22:18:30.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.783+0000 7fb9d57fa640 1 --2- 172.21.15.114:0/3381425599 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fb9c006c450 0x7fb9c006e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:30.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.783+0000 7fb9d57fa640 1 -- 172.21.15.114:0/3381425599 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fb9cc08b620 con 0x7fb9d8100180 2024-02-16T22:18:30.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.783+0000 7fb9d7fff640 1 --2- 172.21.15.114:0/3381425599 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fb9c006c450 0x7fb9c006e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:30.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.783+0000 7fb9d7fff640 1 --2- 172.21.15.114:0/3381425599 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fb9c006c450 0x7fb9c006e910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb9c802efd0 tx=0x7fb9c8000bc0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:30.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.787+0000 7fb9d57fa640 1 -- 172.21.15.114:0/3381425599 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb9cc055e00 con 0x7fb9d8100180 2024-02-16T22:18:30.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:30 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/1227572614' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T22:18:30.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:30 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/3380637035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T22:18:30.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:30 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:30.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:30 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:30.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:30 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:18:30.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:30 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:18:30.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:30 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:30.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:30 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/1431463387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T22:18:30.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.958+0000 7fb9de7fd640 1 -- 172.21.15.114:0/3381425599 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fb9d819bab0 con 0x7fb9d8100180 2024-02-16T22:18:30.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.958+0000 7fb9d57fa640 1 -- 172.21.15.114:0/3381425599 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v45) v1 ==== 107+0+11 (secure 0 0 0) 0x7fb9cc059420 con 0x7fb9d8100180 2024-02-16T22:18:30.960 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-02-16T22:18:30.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.960+0000 7fb9de7fd640 1 -- 172.21.15.114:0/3381425599 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fb9c006c450 msgr2=0x7fb9c006e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:30.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.960+0000 7fb9de7fd640 1 --2- 172.21.15.114:0/3381425599 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fb9c006c450 0x7fb9c006e910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb9c802efd0 tx=0x7fb9c8000bc0 comp rx=0 tx=0).stop 2024-02-16T22:18:30.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.960+0000 7fb9de7fd640 1 -- 172.21.15.114:0/3381425599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9d8100180 msgr2=0x7fb9d8195ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:30.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.960+0000 7fb9de7fd640 1 --2- 172.21.15.114:0/3381425599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9d8100180 0x7fb9d8195ed0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc00c8f0 tx=0x7fb9cc00cdb0 comp rx=0 tx=0).stop 2024-02-16T22:18:30.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.960+0000 7fb9de7fd640 1 -- 172.21.15.114:0/3381425599 shutdown_connections 2024-02-16T22:18:30.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.960+0000 7fb9de7fd640 1 --2- 172.21.15.114:0/3381425599 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fb9c006c450 0x7fb9c006e910 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:30.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.960+0000 7fb9de7fd640 1 --2- 172.21.15.114:0/3381425599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9d8100180 0x7fb9d8195ed0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:30.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.960+0000 7fb9de7fd640 1 --2- 172.21.15.114:0/3381425599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9d80fef90 0x7fb9d8195990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:30.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.960+0000 7fb9de7fd640 1 -- 172.21.15.114:0/3381425599 >> 172.21.15.114:0/3381425599 conn(0x7fb9d80fa4e0 msgr2=0x7fb9d8067350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:30.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.960+0000 7fb9de7fd640 1 -- 172.21.15.114:0/3381425599 shutdown_connections 2024-02-16T22:18:30.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:30.960+0000 7fb9de7fd640 1 -- 172.21.15.114:0/3381425599 wait complete. 2024-02-16T22:18:30.973 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-16T22:18:31.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.079+0000 7fa04dff0640 1 -- 172.21.15.114:0/445666290 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa048101290 msgr2=0x7fa048101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:31.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.079+0000 7fa04dff0640 1 --2- 172.21.15.114:0/445666290 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa048101290 0x7fa048101710 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa03c009880 tx=0x7fa03c02bce0 comp rx=0 tx=0).stop 2024-02-16T22:18:31.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.079+0000 7fa04dff0640 1 -- 172.21.15.114:0/445666290 shutdown_connections 2024-02-16T22:18:31.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.079+0000 7fa04dff0640 1 --2- 172.21.15.114:0/445666290 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa048101290 0x7fa048101710 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:31.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.079+0000 7fa04dff0640 1 --2- 172.21.15.114:0/445666290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0481000a0 0x7fa0481004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:31.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.079+0000 7fa04dff0640 1 -- 172.21.15.114:0/445666290 >> 172.21.15.114:0/445666290 conn(0x7fa0480fb610 msgr2=0x7fa0480fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:31.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.079+0000 7fa04dff0640 1 -- 172.21.15.114:0/445666290 shutdown_connections 2024-02-16T22:18:31.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.079+0000 7fa04dff0640 1 -- 172.21.15.114:0/445666290 wait complete. 2024-02-16T22:18:31.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.079+0000 7fa04dff0640 1 Processor -- start 2024-02-16T22:18:31.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.080+0000 7fa04dff0640 1 -- start start 2024-02-16T22:18:31.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.080+0000 7fa04dff0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0481000a0 0x7fa0481958a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:31.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.080+0000 7fa04dff0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa048101290 0x7fa048195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:31.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.080+0000 7fa04dff0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0481963b0 con 0x7fa0481000a0 2024-02-16T22:18:31.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.080+0000 7fa04dff0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa048196520 con 0x7fa048101290 2024-02-16T22:18:31.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.080+0000 7fa0477fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0481000a0 0x7fa0481958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:31.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.080+0000 7fa0477fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0481000a0 0x7fa0481958a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50864/0 (socket says 172.21.15.114:50864) 2024-02-16T22:18:31.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.080+0000 7fa0477fe640 1 -- 172.21.15.114:0/2390757437 learned_addr learned my addr 172.21.15.114:0/2390757437 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:18:31.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.080+0000 7fa0477fe640 1 -- 172.21.15.114:0/2390757437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa048101290 msgr2=0x7fa048195de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:31.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.080+0000 7fa0477fe640 1 --2- 172.21.15.114:0/2390757437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa048101290 0x7fa048195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:31.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.080+0000 7fa0477fe640 1 -- 172.21.15.114:0/2390757437 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa03c009560 con 0x7fa0481000a0 2024-02-16T22:18:31.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.080+0000 7fa0477fe640 1 --2- 172.21.15.114:0/2390757437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0481000a0 0x7fa0481958a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa03800c8f0 tx=0x7fa03800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:31.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.080+0000 7fa044ff9640 1 -- 172.21.15.114:0/2390757437 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0380041f0 con 0x7fa0481000a0 2024-02-16T22:18:31.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.080+0000 7fa044ff9640 1 -- 172.21.15.114:0/2390757437 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa038007e60 con 0x7fa0481000a0 2024-02-16T22:18:31.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.080+0000 7fa04dff0640 1 -- 172.21.15.114:0/2390757437 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa04819afb0 con 0x7fa0481000a0 2024-02-16T22:18:31.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.080+0000 7fa044ff9640 1 -- 172.21.15.114:0/2390757437 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0380049d0 con 0x7fa0481000a0 2024-02-16T22:18:31.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.080+0000 7fa04dff0640 1 -- 172.21.15.114:0/2390757437 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa04819b4e0 con 0x7fa0481000a0 2024-02-16T22:18:31.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.081+0000 7fa04dff0640 1 -- 172.21.15.114:0/2390757437 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa048108ca0 con 0x7fa0481000a0 2024-02-16T22:18:31.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.082+0000 7fa044ff9640 1 -- 172.21.15.114:0/2390757437 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fa038004350 con 0x7fa0481000a0 2024-02-16T22:18:31.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.082+0000 7fa044ff9640 1 --2- 172.21.15.114:0/2390757437 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa03006c510 0x7fa03006e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:31.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.083+0000 7fa046ffd640 1 --2- 172.21.15.114:0/2390757437 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa03006c510 0x7fa03006e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:31.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.083+0000 7fa044ff9640 1 -- 172.21.15.114:0/2390757437 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fa03808b600 con 0x7fa0481000a0 2024-02-16T22:18:31.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.083+0000 7fa046ffd640 1 --2- 172.21.15.114:0/2390757437 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa03006c510 0x7fa03006e9d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa03c02c7c0 tx=0x7fa03c000bc0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:31.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.087+0000 7fa044ff9640 1 -- 172.21.15.114:0/2390757437 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa038055de0 con 0x7fa0481000a0 2024-02-16T22:18:31.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:30 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/1227572614' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T22:18:31.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:30 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/3380637035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T22:18:31.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:30 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:31.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:30 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:31.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:30 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:18:31.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:30 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:18:31.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:30 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:31.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:30 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/1431463387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T22:18:31.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.303+0000 7fa04dff0640 1 -- 172.21.15.114:0/2390757437 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa048108ed0 con 0x7fa0481000a0 2024-02-16T22:18:31.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.303+0000 7fa044ff9640 1 -- 172.21.15.114:0/2390757437 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4740 (secure 0 0 0) 0x7fa038059400 con 0x7fa0481000a0 2024-02-16T22:18:31.305 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:18:31.305 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14484,"name":"cephfs.smithi114.fdxpey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.114:6829/865692293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":865692293},{"type":"v1","addr":"172.21.15.114:6829","nonce":865692293}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24269,"name":"cephfs.smithi123.vhlqbe","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.123:6825/418014612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":418014612},{"type":"v1","addr":"172.21.15.123:6825","nonce":418014612}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24285,"name":"cephfs.smithi123.gopcfa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.123:6827/2812517585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2812517585},{"type":"v1","addr":"172.21.15.123:6827","nonce":2812517585}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:18:02.108825+0000","modified":"2024-02-16T22:18:16.413371+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14468":{"gid":14468,"name":"cephfs.smithi114.zzryke","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6827/277953535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":277953535},{"type":"v1","addr":"172.21.15.114:6827","nonce":277953535}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T22:18:31.306 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 14 2024-02-16T22:18:31.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.305+0000 7fa04dff0640 1 -- 172.21.15.114:0/2390757437 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa03006c510 msgr2=0x7fa03006e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:31.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.305+0000 7fa04dff0640 1 --2- 172.21.15.114:0/2390757437 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa03006c510 0x7fa03006e9d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa03c02c7c0 tx=0x7fa03c000bc0 comp rx=0 tx=0).stop 2024-02-16T22:18:31.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.305+0000 7fa04dff0640 1 -- 172.21.15.114:0/2390757437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0481000a0 msgr2=0x7fa0481958a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:31.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.305+0000 7fa04dff0640 1 --2- 172.21.15.114:0/2390757437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0481000a0 0x7fa0481958a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa03800c8f0 tx=0x7fa03800cdb0 comp rx=0 tx=0).stop 2024-02-16T22:18:31.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.305+0000 7fa04dff0640 1 -- 172.21.15.114:0/2390757437 shutdown_connections 2024-02-16T22:18:31.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.305+0000 7fa04dff0640 1 --2- 172.21.15.114:0/2390757437 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa03006c510 0x7fa03006e9d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:31.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.305+0000 7fa04dff0640 1 --2- 172.21.15.114:0/2390757437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa048101290 0x7fa048195de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:31.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.305+0000 7fa04dff0640 1 --2- 172.21.15.114:0/2390757437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0481000a0 0x7fa0481958a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:31.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.305+0000 7fa04dff0640 1 -- 172.21.15.114:0/2390757437 >> 172.21.15.114:0/2390757437 conn(0x7fa0480fb610 msgr2=0x7fa0481044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:31.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.305+0000 7fa04dff0640 1 -- 172.21.15.114:0/2390757437 shutdown_connections 2024-02-16T22:18:31.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:31.305+0000 7fa04dff0640 1 -- 172.21.15.114:0/2390757437 wait complete. 2024-02-16T22:18:31.318 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 12, 'max_mds': 1, 'flags': 18} 2024-02-16T22:18:31.318 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-02-16T22:18:31.343 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-02-16T22:18:31.343 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-02-16T22:18:31.343 INFO:tasks.ceph_fuse:client.0 config is {} 2024-02-16T22:18:31.343 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-16T22:18:31.344 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-16T22:18:31.344 INFO:tasks.ceph_fuse:client.1 config is {} 2024-02-16T22:18:31.344 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-16T22:18:31.344 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-16T22:18:31.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:31.344 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-02-16T22:18:31.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:31.364 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link delete ceph-brx 2024-02-16T22:18:31.441 INFO:teuthology.orchestra.run.smithi123.stderr:Cannot find device "ceph-brx" 2024-02-16T22:18:31.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T22:18:31.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:31.443 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list 2024-02-16T22:18:31.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:31.459 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link delete ceph-brx 2024-02-16T22:18:31.529 INFO:teuthology.orchestra.run.smithi114.stderr:Cannot find device "ceph-brx" 2024-02-16T22:18:31.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T22:18:31.531 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-02-16T22:18:31.531 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph fs ls 2024-02-16T22:18:31.777 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:18:32.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:31 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/3381425599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T22:18:32.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:31 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/2390757437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-16T22:18:32.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:31 smithi114 ceph-mon[29235]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:18:32.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:31 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/3381425599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T22:18:32.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:31 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/2390757437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-16T22:18:32.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:31 smithi123 ceph-mon[37161]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:18:33.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.270+0000 7f2c9201a700 1 -- 172.21.15.114:0/3155653667 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c8c069020 msgr2=0x7f2c8c105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:33.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.270+0000 7f2c9201a700 1 --2- 172.21.15.114:0/3155653667 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c8c069020 0x7f2c8c105790 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009910 tx=0x7f2c80009c20 comp rx=0 tx=0).stop 2024-02-16T22:18:33.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.270+0000 7f2c9201a700 1 -- 172.21.15.114:0/3155653667 shutdown_connections 2024-02-16T22:18:33.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.270+0000 7f2c9201a700 1 --2- 172.21.15.114:0/3155653667 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c8c069020 0x7f2c8c105790 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:33.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.270+0000 7f2c9201a700 1 --2- 172.21.15.114:0/3155653667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c8c068670 0x7f2c8c068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:33.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.270+0000 7f2c9201a700 1 -- 172.21.15.114:0/3155653667 >> 172.21.15.114:0/3155653667 conn(0x7f2c8c0755a0 msgr2=0x7f2c8c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:33.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.270+0000 7f2c9201a700 1 -- 172.21.15.114:0/3155653667 shutdown_connections 2024-02-16T22:18:33.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.270+0000 7f2c9201a700 1 -- 172.21.15.114:0/3155653667 wait complete. 2024-02-16T22:18:33.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.271+0000 7f2c9201a700 1 Processor -- start 2024-02-16T22:18:33.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.271+0000 7f2c9201a700 1 -- start start 2024-02-16T22:18:33.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.271+0000 7f2c9201a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c8c068670 0x7f2c8c0ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:33.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.271+0000 7f2c9201a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c8c069020 0x7f2c8c1001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:33.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.271+0000 7f2c9201a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c100720 con 0x7f2c8c069020 2024-02-16T22:18:33.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.271+0000 7f2c9201a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c100860 con 0x7f2c8c068670 2024-02-16T22:18:33.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.272+0000 7f2c8affd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c8c069020 0x7f2c8c1001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:33.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.272+0000 7f2c8affd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c8c069020 0x7f2c8c1001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39876/0 (socket says 172.21.15.114:39876) 2024-02-16T22:18:33.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.272+0000 7f2c8affd700 1 -- 172.21.15.114:0/1026098931 learned_addr learned my addr 172.21.15.114:0/1026098931 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:18:33.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.272+0000 7f2c8b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c8c068670 0x7f2c8c0ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:33.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.272+0000 7f2c8affd700 1 -- 172.21.15.114:0/1026098931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c8c068670 msgr2=0x7f2c8c0ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:33.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.272+0000 7f2c8affd700 1 --2- 172.21.15.114:0/1026098931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c8c068670 0x7f2c8c0ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:33.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.272+0000 7f2c8affd700 1 -- 172.21.15.114:0/1026098931 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c800095f0 con 0x7f2c8c069020 2024-02-16T22:18:33.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.272+0000 7f2c8affd700 1 --2- 172.21.15.114:0/1026098931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c8c069020 0x7f2c8c1001e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2c800092e0 tx=0x7f2c8000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:33.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.272+0000 7f2c88ff9700 1 -- 172.21.15.114:0/1026098931 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c8001d070 con 0x7f2c8c069020 2024-02-16T22:18:33.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.272+0000 7f2c9201a700 1 -- 172.21.15.114:0/1026098931 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c8c100ae0 con 0x7f2c8c069020 2024-02-16T22:18:33.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.272+0000 7f2c88ff9700 1 -- 172.21.15.114:0/1026098931 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2c80005dc0 con 0x7f2c8c069020 2024-02-16T22:18:33.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.272+0000 7f2c9201a700 1 -- 172.21.15.114:0/1026098931 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c8c1a2c00 con 0x7f2c8c069020 2024-02-16T22:18:33.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.272+0000 7f2c88ff9700 1 -- 172.21.15.114:0/1026098931 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c80017400 con 0x7f2c8c069020 2024-02-16T22:18:33.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.273+0000 7f2c9201a700 1 -- 172.21.15.114:0/1026098931 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c8c1090a0 con 0x7f2c8c069020 2024-02-16T22:18:33.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.274+0000 7f2c88ff9700 1 -- 172.21.15.114:0/1026098931 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f2c8000f460 con 0x7f2c8c069020 2024-02-16T22:18:33.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.274+0000 7f2c88ff9700 1 --2- 172.21.15.114:0/1026098931 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f2c7406c4e0 0x7f2c7406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:33.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.274+0000 7f2c8b7fe700 1 --2- 172.21.15.114:0/1026098931 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f2c7406c4e0 0x7f2c7406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:33.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.274+0000 7f2c88ff9700 1 -- 172.21.15.114:0/1026098931 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f2c8008cc30 con 0x7f2c8c069020 2024-02-16T22:18:33.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.274+0000 7f2c8b7fe700 1 --2- 172.21.15.114:0/1026098931 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f2c7406c4e0 0x7f2c7406e9a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c005950 tx=0x7f2c7c00b410 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:33.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.279+0000 7f2c88ff9700 1 -- 172.21.15.114:0/1026098931 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2c80057490 con 0x7f2c8c069020 2024-02-16T22:18:33.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.511+0000 7f2c9201a700 1 -- 172.21.15.114:0/1026098931 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2c8c04ea90 con 0x7f2c8c069020 2024-02-16T22:18:33.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.511+0000 7f2c88ff9700 1 -- 172.21.15.114:0/1026098931 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+83 (secure 0 0 0) 0x7f2c8005aab0 con 0x7f2c8c069020 2024-02-16T22:18:33.514 INFO:teuthology.orchestra.run.smithi114.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-02-16T22:18:33.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.513+0000 7f2c9201a700 1 -- 172.21.15.114:0/1026098931 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f2c7406c4e0 msgr2=0x7f2c7406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:33.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.513+0000 7f2c9201a700 1 --2- 172.21.15.114:0/1026098931 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f2c7406c4e0 0x7f2c7406e9a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c005950 tx=0x7f2c7c00b410 comp rx=0 tx=0).stop 2024-02-16T22:18:33.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.513+0000 7f2c9201a700 1 -- 172.21.15.114:0/1026098931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c8c069020 msgr2=0x7f2c8c1001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:33.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.513+0000 7f2c9201a700 1 --2- 172.21.15.114:0/1026098931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c8c069020 0x7f2c8c1001e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2c800092e0 tx=0x7f2c8000bd70 comp rx=0 tx=0).stop 2024-02-16T22:18:33.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.514+0000 7f2c9201a700 1 -- 172.21.15.114:0/1026098931 shutdown_connections 2024-02-16T22:18:33.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.514+0000 7f2c9201a700 1 --2- 172.21.15.114:0/1026098931 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f2c7406c4e0 0x7f2c7406e9a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:33.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.514+0000 7f2c9201a700 1 --2- 172.21.15.114:0/1026098931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c8c068670 0x7f2c8c0ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:33.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.514+0000 7f2c9201a700 1 --2- 172.21.15.114:0/1026098931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c8c069020 0x7f2c8c1001e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:33.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.514+0000 7f2c9201a700 1 -- 172.21.15.114:0/1026098931 >> 172.21.15.114:0/1026098931 conn(0x7f2c8c0755a0 msgr2=0x7f2c8c105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:33.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.514+0000 7f2c9201a700 1 -- 172.21.15.114:0/1026098931 shutdown_connections 2024-02-16T22:18:33.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:33.514+0000 7f2c9201a700 1 -- 172.21.15.114:0/1026098931 wait complete. 2024-02-16T22:18:33.919 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:33 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/1026098931' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-16T22:18:34.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:33 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/1026098931' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-16T22:18:34.135 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-16T22:18:34.136 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-16T22:18:34.136 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi114.front.sepia.ceph.com 2024-02-16T22:18:34.136 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-16T22:18:34.136 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-16T22:18:34.136 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-16T22:18:34.136 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-16T22:18:34.136 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-16T22:18:34.136 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-16T22:18:34.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:34.136 DEBUG:teuthology.orchestra.run.smithi114:> ip addr 2024-02-16T22:18:34.153 INFO:teuthology.orchestra.run.smithi114.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-16T22:18:34.153 INFO:teuthology.orchestra.run.smithi114.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-16T22:18:34.153 INFO:teuthology.orchestra.run.smithi114.stdout: inet 127.0.0.1/8 scope host lo 2024-02-16T22:18:34.153 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-02-16T22:18:34.153 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 ::1/128 scope host 2024-02-16T22:18:34.154 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-02-16T22:18:34.154 INFO:teuthology.orchestra.run.smithi114.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-16T22:18:34.154 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:d8:a8:92 brd ff:ff:ff:ff:ff:ff 2024-02-16T22:18:34.154 INFO:teuthology.orchestra.run.smithi114.stdout: altname enp7s0f0 2024-02-16T22:18:34.154 INFO:teuthology.orchestra.run.smithi114.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-16T22:18:34.154 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:d8:a8:93 brd ff:ff:ff:ff:ff:ff 2024-02-16T22:18:34.154 INFO:teuthology.orchestra.run.smithi114.stdout: altname enp7s0f1 2024-02-16T22:18:34.154 INFO:teuthology.orchestra.run.smithi114.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-16T22:18:34.154 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:8f:cb:62 brd ff:ff:ff:ff:ff:ff 2024-02-16T22:18:34.154 INFO:teuthology.orchestra.run.smithi114.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-16T22:18:34.154 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:8f:cb:63 brd ff:ff:ff:ff:ff:ff 2024-02-16T22:18:34.154 INFO:teuthology.orchestra.run.smithi114.stdout: inet 172.21.15.114/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-16T22:18:34.155 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft 42256sec preferred_lft 42256sec 2024-02-16T22:18:34.155 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 fe80::ec4:7aff:fe8f:cb63/64 scope link 2024-02-16T22:18:34.155 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-02-16T22:18:34.155 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-16T22:18:34.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:34.155 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec bash -c ' 2024-02-16T22:18:34.155 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-02-16T22:18:34.155 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link add name ceph-brx type bridge 2024-02-16T22:18:34.155 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip addr flush dev ceph-brx 2024-02-16T22:18:34.155 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set ceph-brx up 2024-02-16T22:18:34.156 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-16T22:18:34.156 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-02-16T22:18:34.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:34.328 DEBUG:teuthology.orchestra.run.smithi114:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-16T22:18:34.354 INFO:teuthology.orchestra.run.smithi114.stdout:1 2024-02-16T22:18:34.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:34.356 DEBUG:teuthology.orchestra.run.smithi114:> route 2024-02-16T22:18:34.430 INFO:teuthology.orchestra.run.smithi114.stdout:Kernel IP routing table 2024-02-16T22:18:34.430 INFO:teuthology.orchestra.run.smithi114.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-16T22:18:34.430 INFO:teuthology.orchestra.run.smithi114.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-16T22:18:34.431 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-16T22:18:34.431 INFO:teuthology.orchestra.run.smithi114.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-16T22:18:34.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:34.431 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec bash -c ' 2024-02-16T22:18:34.431 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-02-16T22:18:34.431 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-16T22:18:34.431 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-16T22:18:34.431 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-16T22:18:34.432 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-02-16T22:18:34.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:34.720 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list 2024-02-16T22:18:34.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:34.738 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list-id 2024-02-16T22:18:34.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:34.795 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec bash -c ' 2024-02-16T22:18:34.795 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-02-16T22:18:34.795 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-16T22:18:34.795 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-02-16T22:18:34.795 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-02-16T22:18:34.861 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:34 smithi114 ceph-mon[29235]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:18:34.879 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-02-16T22:18:34.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:34.879 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec bash -c ' 2024-02-16T22:18:34.879 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-02-16T22:18:34.879 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-02-16T22:18:34.879 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-16T22:18:34.880 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-02-16T22:18:34.880 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-02-16T22:18:34.880 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-02-16T22:18:34.880 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-02-16T22:18:35.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:35.021 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec bash -c ' 2024-02-16T22:18:35.021 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-02-16T22:18:35.021 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set brx.0 up 2024-02-16T22:18:35.021 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set dev brx.0 master ceph-brx 2024-02-16T22:18:35.022 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-02-16T22:18:35.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:34 smithi123 ceph-mon[37161]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:18:35.062 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-02-16T22:18:35.062 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-16T22:18:35.062 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-16T22:18:35.117 INFO:teuthology.orchestra.run.smithi114.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-16T22:18:35.118 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-16T22:18:35.118 DEBUG:teuthology.orchestra.run.smithi114:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-16T22:18:35.172 DEBUG:teuthology.orchestra.run.smithi114:> sudo modprobe fuse 2024-02-16T22:18:35.240 DEBUG:teuthology.orchestra.run.smithi114:> cat /proc/self/mounts | awk '{print $2}' 2024-02-16T22:18:35.295 INFO:teuthology.orchestra.run.smithi114.stdout:/proc 2024-02-16T22:18:35.295 INFO:teuthology.orchestra.run.smithi114.stdout:/sys 2024-02-16T22:18:35.295 INFO:teuthology.orchestra.run.smithi114.stdout:/dev 2024-02-16T22:18:35.295 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/kernel/security 2024-02-16T22:18:35.295 INFO:teuthology.orchestra.run.smithi114.stdout:/dev/shm 2024-02-16T22:18:35.295 INFO:teuthology.orchestra.run.smithi114.stdout:/dev/pts 2024-02-16T22:18:35.295 INFO:teuthology.orchestra.run.smithi114.stdout:/run 2024-02-16T22:18:35.295 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/fs/cgroup 2024-02-16T22:18:35.295 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/fs/pstore 2024-02-16T22:18:35.295 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/fs/bpf 2024-02-16T22:18:35.296 INFO:teuthology.orchestra.run.smithi114.stdout:/ 2024-02-16T22:18:35.296 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/fs/selinux 2024-02-16T22:18:35.296 INFO:teuthology.orchestra.run.smithi114.stdout:/proc/sys/fs/binfmt_misc 2024-02-16T22:18:35.296 INFO:teuthology.orchestra.run.smithi114.stdout:/dev/mqueue 2024-02-16T22:18:35.296 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/kernel/debug 2024-02-16T22:18:35.296 INFO:teuthology.orchestra.run.smithi114.stdout:/dev/hugepages 2024-02-16T22:18:35.296 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/kernel/tracing 2024-02-16T22:18:35.296 INFO:teuthology.orchestra.run.smithi114.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-16T22:18:35.296 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/fs/fuse/connections 2024-02-16T22:18:35.296 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/kernel/config 2024-02-16T22:18:35.296 INFO:teuthology.orchestra.run.smithi114.stdout:/run/credentials/systemd-sysctl.service 2024-02-16T22:18:35.296 INFO:teuthology.orchestra.run.smithi114.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-16T22:18:35.296 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/nfs/rpc_pipefs 2024-02-16T22:18:35.296 INFO:teuthology.orchestra.run.smithi114.stdout:/run/user/1000 2024-02-16T22:18:35.297 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/ceph 2024-02-16T22:18:35.297 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay 2024-02-16T22:18:35.297 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/194843ed558c8c0cf937199010dc7f876fc72e6e25a3067f9a64317ba8ac5fe1/merged 2024-02-16T22:18:35.297 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/b564bdffff69096962faebc80a02987a35cd164f79088dd114952e231c8e22b1/merged 2024-02-16T22:18:35.297 INFO:teuthology.orchestra.run.smithi114.stdout:/run/user/0 2024-02-16T22:18:35.297 INFO:teuthology.orchestra.run.smithi114.stdout:/proc/sys/fs/binfmt_misc 2024-02-16T22:18:35.297 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/b0646b1aff588c99dabbc7603027ba4f914dab1860dd96037dae4f387c0b3872/merged 2024-02-16T22:18:35.297 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/1fe1ccd7bba1c7725b77496d07418898da7e812b4efa15ef1fa2120c772034ff/merged 2024-02-16T22:18:35.297 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/ea051f38dd05063aafa6abb7eed1c4f97836c7d3f7d9cec891001a5fccd5cf58/merged 2024-02-16T22:18:35.297 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/f7c6974cfb8aa010436839a11655de690b4ccaa45f1be348610269058b1876b3/merged 2024-02-16T22:18:35.297 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/5d51fa1c0d84749e9e0f0e4143c6cddcfdbe3d0c9b2969157208e1c2a3c5bb9f/merged 2024-02-16T22:18:35.297 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/c27f0edbf087652969b617b5de8157106708b26fdc8055441be4fe82168081f5/merged 2024-02-16T22:18:35.297 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/358cb3e02284828c40629ad36659a2a72a5c530b54ab3622357826321cc1d5d7/merged 2024-02-16T22:18:35.297 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/d72764e1057f4b603cb7ebec76cf63cacdd18d58eb0934db50237685c5a7c0d7/merged 2024-02-16T22:18:35.297 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/c61786980452e4a520755475e85a0dd732f90d6f9526470960b371167f33286b/merged 2024-02-16T22:18:35.298 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/5272bd17e0fd578d1259c8276778ac5bb0cc63e1524328a853192f986ff41ff5/merged 2024-02-16T22:18:35.298 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/252d1aa8f3a92b1fae054dc590dbf4006198e5a8a8b0e213919e1fa0437aa55d/merged 2024-02-16T22:18:35.298 INFO:teuthology.orchestra.run.smithi114.stdout:/run/netns 2024-02-16T22:18:35.298 INFO:teuthology.orchestra.run.smithi114.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-16T22:18:35.298 INFO:teuthology.orchestra.run.smithi114.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-16T22:18:35.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:35.298 DEBUG:teuthology.orchestra.run.smithi114:> ls /sys/fs/fuse/connections 2024-02-16T22:18:35.350 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-02-16T22:18:35.351 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-02-16T22:18:35.393 DEBUG:teuthology.orchestra.run.smithi114:> sudo modprobe fuse 2024-02-16T22:18:35.418 DEBUG:teuthology.orchestra.run.smithi114:> cat /proc/self/mounts | awk '{print $2}' 2024-02-16T22:18:35.474 INFO:teuthology.orchestra.run.smithi114.stdout:/proc 2024-02-16T22:18:35.475 INFO:teuthology.orchestra.run.smithi114.stdout:/sys 2024-02-16T22:18:35.475 INFO:teuthology.orchestra.run.smithi114.stdout:/dev 2024-02-16T22:18:35.475 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/kernel/security 2024-02-16T22:18:35.475 INFO:teuthology.orchestra.run.smithi114.stdout:/dev/shm 2024-02-16T22:18:35.475 INFO:teuthology.orchestra.run.smithi114.stdout:/dev/pts 2024-02-16T22:18:35.475 INFO:teuthology.orchestra.run.smithi114.stdout:/run 2024-02-16T22:18:35.475 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/fs/cgroup 2024-02-16T22:18:35.475 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/fs/pstore 2024-02-16T22:18:35.475 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/fs/bpf 2024-02-16T22:18:35.475 INFO:teuthology.orchestra.run.smithi114.stdout:/ 2024-02-16T22:18:35.475 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/fs/selinux 2024-02-16T22:18:35.475 INFO:teuthology.orchestra.run.smithi114.stdout:/proc/sys/fs/binfmt_misc 2024-02-16T22:18:35.475 INFO:teuthology.orchestra.run.smithi114.stdout:/dev/mqueue 2024-02-16T22:18:35.475 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/kernel/debug 2024-02-16T22:18:35.475 INFO:teuthology.orchestra.run.smithi114.stdout:/dev/hugepages 2024-02-16T22:18:35.476 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/kernel/tracing 2024-02-16T22:18:35.476 INFO:teuthology.orchestra.run.smithi114.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-16T22:18:35.476 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/fs/fuse/connections 2024-02-16T22:18:35.476 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/kernel/config 2024-02-16T22:18:35.476 INFO:teuthology.orchestra.run.smithi114.stdout:/run/credentials/systemd-sysctl.service 2024-02-16T22:18:35.476 INFO:teuthology.orchestra.run.smithi114.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-16T22:18:35.476 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/nfs/rpc_pipefs 2024-02-16T22:18:35.476 INFO:teuthology.orchestra.run.smithi114.stdout:/run/user/1000 2024-02-16T22:18:35.476 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/ceph 2024-02-16T22:18:35.476 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay 2024-02-16T22:18:35.476 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/194843ed558c8c0cf937199010dc7f876fc72e6e25a3067f9a64317ba8ac5fe1/merged 2024-02-16T22:18:35.476 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/b564bdffff69096962faebc80a02987a35cd164f79088dd114952e231c8e22b1/merged 2024-02-16T22:18:35.476 INFO:teuthology.orchestra.run.smithi114.stdout:/run/user/0 2024-02-16T22:18:35.476 INFO:teuthology.orchestra.run.smithi114.stdout:/proc/sys/fs/binfmt_misc 2024-02-16T22:18:35.477 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/b0646b1aff588c99dabbc7603027ba4f914dab1860dd96037dae4f387c0b3872/merged 2024-02-16T22:18:35.477 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/1fe1ccd7bba1c7725b77496d07418898da7e812b4efa15ef1fa2120c772034ff/merged 2024-02-16T22:18:35.477 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/ea051f38dd05063aafa6abb7eed1c4f97836c7d3f7d9cec891001a5fccd5cf58/merged 2024-02-16T22:18:35.477 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/f7c6974cfb8aa010436839a11655de690b4ccaa45f1be348610269058b1876b3/merged 2024-02-16T22:18:35.477 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/5d51fa1c0d84749e9e0f0e4143c6cddcfdbe3d0c9b2969157208e1c2a3c5bb9f/merged 2024-02-16T22:18:35.477 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/c27f0edbf087652969b617b5de8157106708b26fdc8055441be4fe82168081f5/merged 2024-02-16T22:18:35.477 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/358cb3e02284828c40629ad36659a2a72a5c530b54ab3622357826321cc1d5d7/merged 2024-02-16T22:18:35.477 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/d72764e1057f4b603cb7ebec76cf63cacdd18d58eb0934db50237685c5a7c0d7/merged 2024-02-16T22:18:35.477 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/c61786980452e4a520755475e85a0dd732f90d6f9526470960b371167f33286b/merged 2024-02-16T22:18:35.477 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/5272bd17e0fd578d1259c8276778ac5bb0cc63e1524328a853192f986ff41ff5/merged 2024-02-16T22:18:35.477 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/252d1aa8f3a92b1fae054dc590dbf4006198e5a8a8b0e213919e1fa0437aa55d/merged 2024-02-16T22:18:35.477 INFO:teuthology.orchestra.run.smithi114.stdout:/run/netns 2024-02-16T22:18:35.477 INFO:teuthology.orchestra.run.smithi114.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-16T22:18:35.477 INFO:teuthology.orchestra.run.smithi114.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-16T22:18:35.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:35.478 DEBUG:teuthology.orchestra.run.smithi114:> ls /sys/fs/fuse/connections 2024-02-16T22:18:35.498 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi114.stderr:2024-02-16T22:18:35.496+0000 7fc36bf80480 -1 init, newargv = 0x558445104390 newargc=15 2024-02-16T22:18:35.498 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi114.stderr:ceph-fuse[77657]: starting ceph client 2024-02-16T22:18:35.502 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi114.stderr:ceph-fuse[77657]: starting fuse 2024-02-16T22:18:35.512 INFO:teuthology.orchestra.run.smithi114.stdout:159 2024-02-16T22:18:35.513 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [159] 2024-02-16T22:18:35.513 DEBUG:teuthology.orchestra.run.smithi114:> sudo stdin-killer -- python3 -c ' 2024-02-16T22:18:35.513 DEBUG:teuthology.orchestra.run.smithi114:> import glob 2024-02-16T22:18:35.513 DEBUG:teuthology.orchestra.run.smithi114:> import re 2024-02-16T22:18:35.513 DEBUG:teuthology.orchestra.run.smithi114:> import os 2024-02-16T22:18:35.513 DEBUG:teuthology.orchestra.run.smithi114:> import subprocess 2024-02-16T22:18:35.513 DEBUG:teuthology.orchestra.run.smithi114:> 2024-02-16T22:18:35.513 DEBUG:teuthology.orchestra.run.smithi114:> def _find_admin_socket(client_name): 2024-02-16T22:18:35.513 DEBUG:teuthology.orchestra.run.smithi114:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-02-16T22:18:35.513 DEBUG:teuthology.orchestra.run.smithi114:> files = glob.glob(asok_path) 2024-02-16T22:18:35.513 DEBUG:teuthology.orchestra.run.smithi114:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-02-16T22:18:35.513 DEBUG:teuthology.orchestra.run.smithi114:> 2024-02-16T22:18:35.514 DEBUG:teuthology.orchestra.run.smithi114:> # Given a non-glob path, it better be there 2024-02-16T22:18:35.514 DEBUG:teuthology.orchestra.run.smithi114:> if "*" not in asok_path: 2024-02-16T22:18:35.514 DEBUG:teuthology.orchestra.run.smithi114:> assert(len(files) == 1) 2024-02-16T22:18:35.514 DEBUG:teuthology.orchestra.run.smithi114:> return files[0] 2024-02-16T22:18:35.514 DEBUG:teuthology.orchestra.run.smithi114:> 2024-02-16T22:18:35.514 DEBUG:teuthology.orchestra.run.smithi114:> for f in files: 2024-02-16T22:18:35.514 DEBUG:teuthology.orchestra.run.smithi114:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-02-16T22:18:35.514 DEBUG:teuthology.orchestra.run.smithi114:> if os.path.exists("/proc/{0}".format(pid)): 2024-02-16T22:18:35.514 DEBUG:teuthology.orchestra.run.smithi114:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-02-16T22:18:35.514 DEBUG:teuthology.orchestra.run.smithi114:> contents = proc_f.read() 2024-02-16T22:18:35.514 DEBUG:teuthology.orchestra.run.smithi114:> if mountpoint in contents: 2024-02-16T22:18:35.514 DEBUG:teuthology.orchestra.run.smithi114:> return f 2024-02-16T22:18:35.514 DEBUG:teuthology.orchestra.run.smithi114:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-02-16T22:18:35.514 DEBUG:teuthology.orchestra.run.smithi114:> 2024-02-16T22:18:35.514 DEBUG:teuthology.orchestra.run.smithi114:> print(_find_admin_socket("client.0")) 2024-02-16T22:18:35.514 DEBUG:teuthology.orchestra.run.smithi114:> ' 2024-02-16T22:18:35.633 INFO:teuthology.orchestra.run.smithi114.stdout:/var/run/ceph/ceph-client.0.77657.asok 2024-02-16T22:18:35.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-02-16T22:18:35.643 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.77657.asok 2024-02-16T22:18:35.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:35.644 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.77657.asok status 2024-02-16T22:18:35.779 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:18:35.779 INFO:teuthology.orchestra.run.smithi114.stdout: "metadata": { 2024-02-16T22:18:35.779 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_sha1": "7fe91d5d5842e04be3b4f514d6dd990c54b29c76", 2024-02-16T22:18:35.779 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-02-16T22:18:35.779 INFO:teuthology.orchestra.run.smithi114.stdout: "entity_id": "0", 2024-02-16T22:18:35.779 INFO:teuthology.orchestra.run.smithi114.stdout: "hostname": "smithi114", 2024-02-16T22:18:35.779 INFO:teuthology.orchestra.run.smithi114.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-02-16T22:18:35.780 INFO:teuthology.orchestra.run.smithi114.stdout: "pid": "77657", 2024-02-16T22:18:35.780 INFO:teuthology.orchestra.run.smithi114.stdout: "root": "/" 2024-02-16T22:18:35.780 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:18:35.780 INFO:teuthology.orchestra.run.smithi114.stdout: "dentry_count": 0, 2024-02-16T22:18:35.780 INFO:teuthology.orchestra.run.smithi114.stdout: "dentry_pinned_count": 0, 2024-02-16T22:18:35.780 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 14536, 2024-02-16T22:18:35.780 INFO:teuthology.orchestra.run.smithi114.stdout: "inst": { 2024-02-16T22:18:35.780 INFO:teuthology.orchestra.run.smithi114.stdout: "name": { 2024-02-16T22:18:35.780 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "client", 2024-02-16T22:18:35.780 INFO:teuthology.orchestra.run.smithi114.stdout: "num": 14536 2024-02-16T22:18:35.780 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:18:35.780 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": { 2024-02-16T22:18:35.780 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "v1", 2024-02-16T22:18:35.781 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": "192.168.0.1:0", 2024-02-16T22:18:35.781 INFO:teuthology.orchestra.run.smithi114.stdout: "nonce": 21264991 2024-02-16T22:18:35.781 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-02-16T22:18:35.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:18:35.781 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": { 2024-02-16T22:18:35.781 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "v1", 2024-02-16T22:18:35.781 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": "192.168.0.1:0", 2024-02-16T22:18:35.781 INFO:teuthology.orchestra.run.smithi114.stdout: "nonce": 21264991 2024-02-16T22:18:35.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:18:35.781 INFO:teuthology.orchestra.run.smithi114.stdout: "inst_str": "client.14536 192.168.0.1:0/21264991", 2024-02-16T22:18:35.781 INFO:teuthology.orchestra.run.smithi114.stdout: "addr_str": "192.168.0.1:0/21264991", 2024-02-16T22:18:35.781 INFO:teuthology.orchestra.run.smithi114.stdout: "inode_count": 1, 2024-02-16T22:18:35.781 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_epoch": 12, 2024-02-16T22:18:35.781 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_epoch": 45, 2024-02-16T22:18:35.781 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_epoch_barrier": 0, 2024-02-16T22:18:35.782 INFO:teuthology.orchestra.run.smithi114.stdout: "blocklisted": false, 2024-02-16T22:18:35.782 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs" 2024-02-16T22:18:35.782 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:18:35.788 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- ceph fs ls 2024-02-16T22:18:35.994 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:18:36.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:36 smithi114 ceph-mon[29235]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:18:37.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:36 smithi123 ceph-mon[37161]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:18:37.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.648+0000 7f7f97fcf700 1 -- 172.21.15.114:0/3237575177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f90100dc0 msgr2=0x7f7f901011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:37.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.648+0000 7f7f97fcf700 1 --2- 172.21.15.114:0/3237575177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f90100dc0 0x7f7f901011a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c009910 tx=0x7f7f8c009c20 comp rx=0 tx=0).stop 2024-02-16T22:18:37.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.648+0000 7f7f97fcf700 1 -- 172.21.15.114:0/3237575177 shutdown_connections 2024-02-16T22:18:37.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.648+0000 7f7f97fcf700 1 --2- 172.21.15.114:0/3237575177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f90101770 0x7f7f901057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:37.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.648+0000 7f7f97fcf700 1 --2- 172.21.15.114:0/3237575177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f90100dc0 0x7f7f901011a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:37.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.648+0000 7f7f97fcf700 1 -- 172.21.15.114:0/3237575177 >> 172.21.15.114:0/3237575177 conn(0x7f7f900fc650 msgr2=0x7f7f900fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:37.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.648+0000 7f7f97fcf700 1 -- 172.21.15.114:0/3237575177 shutdown_connections 2024-02-16T22:18:37.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.648+0000 7f7f97fcf700 1 -- 172.21.15.114:0/3237575177 wait complete. 2024-02-16T22:18:37.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.649+0000 7f7f97fcf700 1 Processor -- start 2024-02-16T22:18:37.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.649+0000 7f7f97fcf700 1 -- start start 2024-02-16T22:18:37.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.649+0000 7f7f97fcf700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f90100dc0 0x7f7f90073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:37.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.649+0000 7f7f97fcf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f90101770 0x7f7f900735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:37.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.649+0000 7f7f97fcf700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f90073b10 con 0x7f7f90100dc0 2024-02-16T22:18:37.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.649+0000 7f7f97fcf700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f90073c50 con 0x7f7f90101770 2024-02-16T22:18:37.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.649+0000 7f7f95d6b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f90100dc0 0x7f7f90073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:37.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.649+0000 7f7f95d6b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f90100dc0 0x7f7f90073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39896/0 (socket says 172.21.15.114:39896) 2024-02-16T22:18:37.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.649+0000 7f7f95d6b700 1 -- 172.21.15.114:0/3083932958 learned_addr learned my addr 172.21.15.114:0/3083932958 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:18:37.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.649+0000 7f7f9556a700 1 --2- 172.21.15.114:0/3083932958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f90101770 0x7f7f900735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:37.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.649+0000 7f7f95d6b700 1 -- 172.21.15.114:0/3083932958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f90101770 msgr2=0x7f7f900735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:37.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.649+0000 7f7f95d6b700 1 --2- 172.21.15.114:0/3083932958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f90101770 0x7f7f900735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:37.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.649+0000 7f7f95d6b700 1 -- 172.21.15.114:0/3083932958 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c0095f0 con 0x7f7f90100dc0 2024-02-16T22:18:37.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.649+0000 7f7f9556a700 1 --2- 172.21.15.114:0/3083932958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f90101770 0x7f7f900735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:18:37.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.650+0000 7f7f95d6b700 1 --2- 172.21.15.114:0/3083932958 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f90100dc0 0x7f7f90073090 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c006010 tx=0x7f7f8c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:37.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.650+0000 7f7f86ffd700 1 -- 172.21.15.114:0/3083932958 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f8c01d070 con 0x7f7f90100dc0 2024-02-16T22:18:37.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.650+0000 7f7f86ffd700 1 -- 172.21.15.114:0/3083932958 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7f8c004f50 con 0x7f7f90100dc0 2024-02-16T22:18:37.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.650+0000 7f7f86ffd700 1 -- 172.21.15.114:0/3083932958 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f8c021410 con 0x7f7f90100dc0 2024-02-16T22:18:37.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.650+0000 7f7f97fcf700 1 -- 172.21.15.114:0/3083932958 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f90073ed0 con 0x7f7f90100dc0 2024-02-16T22:18:37.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.650+0000 7f7f97fcf700 1 -- 172.21.15.114:0/3083932958 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f901a2c20 con 0x7f7f90100dc0 2024-02-16T22:18:37.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.651+0000 7f7f97fcf700 1 -- 172.21.15.114:0/3083932958 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f90109160 con 0x7f7f90100dc0 2024-02-16T22:18:37.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.651+0000 7f7f86ffd700 1 -- 172.21.15.114:0/3083932958 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f7f8c00f460 con 0x7f7f90100dc0 2024-02-16T22:18:37.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.652+0000 7f7f86ffd700 1 --2- 172.21.15.114:0/3083932958 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f7f7c06c4e0 0x7f7f7c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:37.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.652+0000 7f7f9556a700 1 --2- 172.21.15.114:0/3083932958 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f7f7c06c4e0 0x7f7f7c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:37.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.652+0000 7f7f86ffd700 1 -- 172.21.15.114:0/3083932958 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f7f8c08ca10 con 0x7f7f90100dc0 2024-02-16T22:18:37.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.652+0000 7f7f9556a700 1 --2- 172.21.15.114:0/3083932958 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f7f7c06c4e0 0x7f7f7c06e9a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7f80005950 tx=0x7f7f800058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:37.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.656+0000 7f7f86ffd700 1 -- 172.21.15.114:0/3083932958 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7f8c0571f0 con 0x7f7f90100dc0 2024-02-16T22:18:37.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.891+0000 7f7f97fcf700 1 -- 172.21.15.114:0/3083932958 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7f9004ea90 con 0x7f7f90100dc0 2024-02-16T22:18:37.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.891+0000 7f7f86ffd700 1 -- 172.21.15.114:0/3083932958 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+83 (secure 0 0 0) 0x7f7f8c05a810 con 0x7f7f90100dc0 2024-02-16T22:18:37.893 INFO:teuthology.orchestra.run.smithi114.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-02-16T22:18:37.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.893+0000 7f7f97fcf700 1 -- 172.21.15.114:0/3083932958 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f7f7c06c4e0 msgr2=0x7f7f7c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:37.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.893+0000 7f7f97fcf700 1 --2- 172.21.15.114:0/3083932958 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f7f7c06c4e0 0x7f7f7c06e9a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7f80005950 tx=0x7f7f800058e0 comp rx=0 tx=0).stop 2024-02-16T22:18:37.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.893+0000 7f7f97fcf700 1 -- 172.21.15.114:0/3083932958 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f90100dc0 msgr2=0x7f7f90073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.893+0000 7f7f97fcf700 1 --2- 172.21.15.114:0/3083932958 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f90100dc0 0x7f7f90073090 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c006010 tx=0x7f7f8c00be20 comp rx=0 tx=0).stop 2024-02-16T22:18:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.893+0000 7f7f97fcf700 1 -- 172.21.15.114:0/3083932958 shutdown_connections 2024-02-16T22:18:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.893+0000 7f7f97fcf700 1 --2- 172.21.15.114:0/3083932958 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f90100dc0 0x7f7f90073090 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.893+0000 7f7f97fcf700 1 --2- 172.21.15.114:0/3083932958 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f7f7c06c4e0 0x7f7f7c06e9a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.893+0000 7f7f97fcf700 1 --2- 172.21.15.114:0/3083932958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f90101770 0x7f7f900735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.893+0000 7f7f97fcf700 1 -- 172.21.15.114:0/3083932958 >> 172.21.15.114:0/3083932958 conn(0x7f7f900fc650 msgr2=0x7f7f900fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.893+0000 7f7f97fcf700 1 -- 172.21.15.114:0/3083932958 shutdown_connections 2024-02-16T22:18:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:37.893+0000 7f7f97fcf700 1 -- 172.21.15.114:0/3083932958 wait complete. 2024-02-16T22:18:37.906 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:37 smithi114 ceph-mon[29235]: pgmap v196: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-16T22:18:38.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:37 smithi123 ceph-mon[37161]: pgmap v196: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-16T22:18:38.538 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-16T22:18:38.538 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-16T22:18:38.538 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi123.front.sepia.ceph.com 2024-02-16T22:18:38.538 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-02-16T22:18:38.538 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-16T22:18:38.538 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-16T22:18:38.538 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-16T22:18:38.539 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-16T22:18:38.539 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-16T22:18:38.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:38.539 DEBUG:teuthology.orchestra.run.smithi123:> ip addr 2024-02-16T22:18:38.558 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-16T22:18:38.558 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-16T22:18:38.558 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2024-02-16T22:18:38.558 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-02-16T22:18:38.558 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2024-02-16T22:18:38.558 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-02-16T22:18:38.558 INFO:teuthology.orchestra.run.smithi123.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-16T22:18:38.558 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6c brd ff:ff:ff:ff:ff:ff 2024-02-16T22:18:38.558 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f0 2024-02-16T22:18:38.559 INFO:teuthology.orchestra.run.smithi123.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-16T22:18:38.559 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6d brd ff:ff:ff:ff:ff:ff 2024-02-16T22:18:38.559 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f1 2024-02-16T22:18:38.559 INFO:teuthology.orchestra.run.smithi123.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-16T22:18:38.559 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f0 brd ff:ff:ff:ff:ff:ff 2024-02-16T22:18:38.559 INFO:teuthology.orchestra.run.smithi123.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-16T22:18:38.559 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f1 brd ff:ff:ff:ff:ff:ff 2024-02-16T22:18:38.559 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.21.15.123/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-16T22:18:38.559 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft 42221sec preferred_lft 42221sec 2024-02-16T22:18:38.559 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::ec4:7aff:fe8f:d3f1/64 scope link 2024-02-16T22:18:38.559 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-02-16T22:18:38.559 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-16T22:18:38.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:38.560 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2024-02-16T22:18:38.560 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-02-16T22:18:38.560 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add name ceph-brx type bridge 2024-02-16T22:18:38.560 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr flush dev ceph-brx 2024-02-16T22:18:38.560 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set ceph-brx up 2024-02-16T22:18:38.560 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-16T22:18:38.560 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-02-16T22:18:38.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:38.731 DEBUG:teuthology.orchestra.run.smithi123:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-16T22:18:38.759 INFO:teuthology.orchestra.run.smithi123.stdout:1 2024-02-16T22:18:38.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:38.761 DEBUG:teuthology.orchestra.run.smithi123:> route 2024-02-16T22:18:38.830 INFO:teuthology.orchestra.run.smithi123.stdout:Kernel IP routing table 2024-02-16T22:18:38.830 INFO:teuthology.orchestra.run.smithi123.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-16T22:18:38.831 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-16T22:18:38.831 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-16T22:18:38.831 INFO:teuthology.orchestra.run.smithi123.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-16T22:18:38.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:38.831 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2024-02-16T22:18:38.831 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-02-16T22:18:38.831 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-16T22:18:38.831 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-16T22:18:38.831 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-16T22:18:38.831 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-02-16T22:18:39.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:39.104 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-02-16T22:18:39.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:39.120 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list-id 2024-02-16T22:18:39.164 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:38 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/3083932958' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-16T22:18:39.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:39.175 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2024-02-16T22:18:39.175 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-02-16T22:18:39.175 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-16T22:18:39.175 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-02-16T22:18:39.175 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-02-16T22:18:39.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:38 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/3083932958' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-16T22:18:39.258 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-02-16T22:18:39.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:39.259 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2024-02-16T22:18:39.259 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-02-16T22:18:39.259 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-02-16T22:18:39.259 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-16T22:18:39.259 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-02-16T22:18:39.259 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-02-16T22:18:39.259 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-02-16T22:18:39.259 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-02-16T22:18:39.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:39.397 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2024-02-16T22:18:39.397 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-02-16T22:18:39.397 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set brx.0 up 2024-02-16T22:18:39.397 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set dev brx.0 master ceph-brx 2024-02-16T22:18:39.398 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-02-16T22:18:39.439 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-02-16T22:18:39.439 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-16T22:18:39.440 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-02-16T22:18:39.495 INFO:teuthology.orchestra.run.smithi123.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-02-16T22:18:39.495 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-16T22:18:39.495 DEBUG:teuthology.orchestra.run.smithi123:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-02-16T22:18:39.550 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2024-02-16T22:18:39.618 DEBUG:teuthology.orchestra.run.smithi123:> cat /proc/self/mounts | awk '{print $2}' 2024-02-16T22:18:39.673 INFO:teuthology.orchestra.run.smithi123.stdout:/proc 2024-02-16T22:18:39.673 INFO:teuthology.orchestra.run.smithi123.stdout:/sys 2024-02-16T22:18:39.674 INFO:teuthology.orchestra.run.smithi123.stdout:/dev 2024-02-16T22:18:39.674 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/security 2024-02-16T22:18:39.674 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/shm 2024-02-16T22:18:39.674 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/pts 2024-02-16T22:18:39.674 INFO:teuthology.orchestra.run.smithi123.stdout:/run 2024-02-16T22:18:39.674 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/cgroup 2024-02-16T22:18:39.674 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/pstore 2024-02-16T22:18:39.674 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/bpf 2024-02-16T22:18:39.674 INFO:teuthology.orchestra.run.smithi123.stdout:/ 2024-02-16T22:18:39.674 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/selinux 2024-02-16T22:18:39.674 INFO:teuthology.orchestra.run.smithi123.stdout:/proc/sys/fs/binfmt_misc 2024-02-16T22:18:39.674 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/hugepages 2024-02-16T22:18:39.675 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/mqueue 2024-02-16T22:18:39.675 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/debug 2024-02-16T22:18:39.675 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/tracing 2024-02-16T22:18:39.675 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-16T22:18:39.675 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/fuse/connections 2024-02-16T22:18:39.675 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/config 2024-02-16T22:18:39.675 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-sysctl.service 2024-02-16T22:18:39.675 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-16T22:18:39.675 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/nfs/rpc_pipefs 2024-02-16T22:18:39.675 INFO:teuthology.orchestra.run.smithi123.stdout:/run/user/1000 2024-02-16T22:18:39.675 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph 2024-02-16T22:18:39.675 INFO:teuthology.orchestra.run.smithi123.stdout:/run/user/0 2024-02-16T22:18:39.675 INFO:teuthology.orchestra.run.smithi123.stdout:/proc/sys/fs/binfmt_misc 2024-02-16T22:18:39.675 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/containers/storage/overlay 2024-02-16T22:18:39.675 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/containers/storage/overlay/0fde9569be0afe3f38f1eaf87db08f6c7132d0018ce6ea62a22c60645801ff73/merged 2024-02-16T22:18:39.676 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/containers/storage/overlay/1ac55c27341edb2795a4040bd75d0c8b66bba38f3cbf4f82037d45648a436b9c/merged 2024-02-16T22:18:39.676 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/containers/storage/overlay/f2c2ec06c58b25775aa8494397b16faf280723adb76f905b7ab7b1770c6d2c49/merged 2024-02-16T22:18:39.676 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/containers/storage/overlay/3110648102b021b5cad71ecc8d819b621260bc2c4356b6db14a204a44a14163c/merged 2024-02-16T22:18:39.676 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/containers/storage/overlay/3d1c02267cce8296e0d19907258427179e7ebb2457045e0cb1d12e75580e83f1/merged 2024-02-16T22:18:39.676 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/containers/storage/overlay/c35dc615dedd7337615a3888711ed240607e04a8209cedf699c8411158017aef/merged 2024-02-16T22:18:39.676 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/containers/storage/overlay/fae48d7f2524f2c49835a0d641e9f47a5c0edd05b8ec7cc8862a4a1c6477e889/merged 2024-02-16T22:18:39.676 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/containers/storage/overlay/f8ac744f5c5583bba6c27820e19ab54890be5fb2a86a3179e6e006902a124602/merged 2024-02-16T22:18:39.676 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/containers/storage/overlay/0c45429b0feebc63a25274a3f16b9969a0993ec71e0fc8569d6dbdd6f972cc0a/merged 2024-02-16T22:18:39.676 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/containers/storage/overlay/2d8240a62ee72aa1ef8e7992b6fb07e2dfedb2d70803f26729106b11fc1a5fd4/merged 2024-02-16T22:18:39.676 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns 2024-02-16T22:18:39.676 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-16T22:18:39.676 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-16T22:18:39.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:39.677 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2024-02-16T22:18:39.728 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-02-16T22:18:39.728 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-02-16T22:18:39.771 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2024-02-16T22:18:39.798 DEBUG:teuthology.orchestra.run.smithi123:> cat /proc/self/mounts | awk '{print $2}' 2024-02-16T22:18:39.856 INFO:teuthology.orchestra.run.smithi123.stdout:/proc 2024-02-16T22:18:39.856 INFO:teuthology.orchestra.run.smithi123.stdout:/sys 2024-02-16T22:18:39.857 INFO:teuthology.orchestra.run.smithi123.stdout:/dev 2024-02-16T22:18:39.857 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/security 2024-02-16T22:18:39.857 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/shm 2024-02-16T22:18:39.857 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/pts 2024-02-16T22:18:39.857 INFO:teuthology.orchestra.run.smithi123.stdout:/run 2024-02-16T22:18:39.857 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/cgroup 2024-02-16T22:18:39.857 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/pstore 2024-02-16T22:18:39.857 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/bpf 2024-02-16T22:18:39.857 INFO:teuthology.orchestra.run.smithi123.stdout:/ 2024-02-16T22:18:39.857 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/selinux 2024-02-16T22:18:39.857 INFO:teuthology.orchestra.run.smithi123.stdout:/proc/sys/fs/binfmt_misc 2024-02-16T22:18:39.857 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/hugepages 2024-02-16T22:18:39.857 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/mqueue 2024-02-16T22:18:39.857 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/debug 2024-02-16T22:18:39.858 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/tracing 2024-02-16T22:18:39.858 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-16T22:18:39.858 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/fuse/connections 2024-02-16T22:18:39.858 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/config 2024-02-16T22:18:39.858 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-sysctl.service 2024-02-16T22:18:39.858 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-16T22:18:39.858 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/nfs/rpc_pipefs 2024-02-16T22:18:39.858 INFO:teuthology.orchestra.run.smithi123.stdout:/run/user/1000 2024-02-16T22:18:39.858 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph 2024-02-16T22:18:39.858 INFO:teuthology.orchestra.run.smithi123.stdout:/run/user/0 2024-02-16T22:18:39.858 INFO:teuthology.orchestra.run.smithi123.stdout:/proc/sys/fs/binfmt_misc 2024-02-16T22:18:39.858 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/containers/storage/overlay 2024-02-16T22:18:39.858 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/containers/storage/overlay/0fde9569be0afe3f38f1eaf87db08f6c7132d0018ce6ea62a22c60645801ff73/merged 2024-02-16T22:18:39.858 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/containers/storage/overlay/1ac55c27341edb2795a4040bd75d0c8b66bba38f3cbf4f82037d45648a436b9c/merged 2024-02-16T22:18:39.859 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/containers/storage/overlay/f2c2ec06c58b25775aa8494397b16faf280723adb76f905b7ab7b1770c6d2c49/merged 2024-02-16T22:18:39.859 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/containers/storage/overlay/3110648102b021b5cad71ecc8d819b621260bc2c4356b6db14a204a44a14163c/merged 2024-02-16T22:18:39.859 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/containers/storage/overlay/3d1c02267cce8296e0d19907258427179e7ebb2457045e0cb1d12e75580e83f1/merged 2024-02-16T22:18:39.859 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/containers/storage/overlay/c35dc615dedd7337615a3888711ed240607e04a8209cedf699c8411158017aef/merged 2024-02-16T22:18:39.859 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/containers/storage/overlay/fae48d7f2524f2c49835a0d641e9f47a5c0edd05b8ec7cc8862a4a1c6477e889/merged 2024-02-16T22:18:39.859 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/containers/storage/overlay/f8ac744f5c5583bba6c27820e19ab54890be5fb2a86a3179e6e006902a124602/merged 2024-02-16T22:18:39.859 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/containers/storage/overlay/0c45429b0feebc63a25274a3f16b9969a0993ec71e0fc8569d6dbdd6f972cc0a/merged 2024-02-16T22:18:39.859 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/containers/storage/overlay/2d8240a62ee72aa1ef8e7992b6fb07e2dfedb2d70803f26729106b11fc1a5fd4/merged 2024-02-16T22:18:39.859 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns 2024-02-16T22:18:39.859 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-16T22:18:39.859 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-16T22:18:39.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:39.859 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2024-02-16T22:18:39.881 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi123.stderr:2024-02-16T22:18:39.879+0000 7f9109a3a480 -1 init, newargv = 0x555c2b3243f0 newargc=15 2024-02-16T22:18:39.881 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi123.stderr:ceph-fuse[63203]: starting ceph client 2024-02-16T22:18:39.886 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi123.stderr:ceph-fuse[63203]: starting fuse 2024-02-16T22:18:39.901 INFO:teuthology.orchestra.run.smithi123.stdout:126 2024-02-16T22:18:39.901 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [126] 2024-02-16T22:18:39.901 DEBUG:teuthology.orchestra.run.smithi123:> sudo stdin-killer -- python3 -c ' 2024-02-16T22:18:39.901 DEBUG:teuthology.orchestra.run.smithi123:> import glob 2024-02-16T22:18:39.901 DEBUG:teuthology.orchestra.run.smithi123:> import re 2024-02-16T22:18:39.901 DEBUG:teuthology.orchestra.run.smithi123:> import os 2024-02-16T22:18:39.901 DEBUG:teuthology.orchestra.run.smithi123:> import subprocess 2024-02-16T22:18:39.901 DEBUG:teuthology.orchestra.run.smithi123:> 2024-02-16T22:18:39.901 DEBUG:teuthology.orchestra.run.smithi123:> def _find_admin_socket(client_name): 2024-02-16T22:18:39.901 DEBUG:teuthology.orchestra.run.smithi123:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-02-16T22:18:39.901 DEBUG:teuthology.orchestra.run.smithi123:> files = glob.glob(asok_path) 2024-02-16T22:18:39.901 DEBUG:teuthology.orchestra.run.smithi123:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-02-16T22:18:39.902 DEBUG:teuthology.orchestra.run.smithi123:> 2024-02-16T22:18:39.902 DEBUG:teuthology.orchestra.run.smithi123:> # Given a non-glob path, it better be there 2024-02-16T22:18:39.902 DEBUG:teuthology.orchestra.run.smithi123:> if "*" not in asok_path: 2024-02-16T22:18:39.902 DEBUG:teuthology.orchestra.run.smithi123:> assert(len(files) == 1) 2024-02-16T22:18:39.902 DEBUG:teuthology.orchestra.run.smithi123:> return files[0] 2024-02-16T22:18:39.902 DEBUG:teuthology.orchestra.run.smithi123:> 2024-02-16T22:18:39.902 DEBUG:teuthology.orchestra.run.smithi123:> for f in files: 2024-02-16T22:18:39.902 DEBUG:teuthology.orchestra.run.smithi123:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-02-16T22:18:39.902 DEBUG:teuthology.orchestra.run.smithi123:> if os.path.exists("/proc/{0}".format(pid)): 2024-02-16T22:18:39.902 DEBUG:teuthology.orchestra.run.smithi123:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-02-16T22:18:39.902 DEBUG:teuthology.orchestra.run.smithi123:> contents = proc_f.read() 2024-02-16T22:18:39.902 DEBUG:teuthology.orchestra.run.smithi123:> if mountpoint in contents: 2024-02-16T22:18:39.902 DEBUG:teuthology.orchestra.run.smithi123:> return f 2024-02-16T22:18:39.902 DEBUG:teuthology.orchestra.run.smithi123:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-02-16T22:18:39.902 DEBUG:teuthology.orchestra.run.smithi123:> 2024-02-16T22:18:39.902 DEBUG:teuthology.orchestra.run.smithi123:> print(_find_admin_socket("client.1")) 2024-02-16T22:18:39.902 DEBUG:teuthology.orchestra.run.smithi123:> ' 2024-02-16T22:18:40.025 INFO:teuthology.orchestra.run.smithi123.stdout:/var/run/ceph/ceph-client.1.63203.asok 2024-02-16T22:18:40.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-02-16T22:18:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-02-16T22:18:40.035 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.63203.asok 2024-02-16T22:18:40.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:40.036 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.63203.asok status 2024-02-16T22:18:40.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:39 smithi123 ceph-mon[37161]: pgmap v197: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-16T22:18:40.148 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-02-16T22:18:40.148 INFO:teuthology.orchestra.run.smithi123.stdout: "metadata": { 2024-02-16T22:18:40.148 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_sha1": "7fe91d5d5842e04be3b4f514d6dd990c54b29c76", 2024-02-16T22:18:40.148 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-02-16T22:18:40.149 INFO:teuthology.orchestra.run.smithi123.stdout: "entity_id": "1", 2024-02-16T22:18:40.149 INFO:teuthology.orchestra.run.smithi123.stdout: "hostname": "smithi123", 2024-02-16T22:18:40.149 INFO:teuthology.orchestra.run.smithi123.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-02-16T22:18:40.149 INFO:teuthology.orchestra.run.smithi123.stdout: "pid": "63203", 2024-02-16T22:18:40.149 INFO:teuthology.orchestra.run.smithi123.stdout: "root": "/" 2024-02-16T22:18:40.149 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-02-16T22:18:40.149 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_count": 0, 2024-02-16T22:18:40.149 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_pinned_count": 0, 2024-02-16T22:18:40.149 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 24321, 2024-02-16T22:18:40.149 INFO:teuthology.orchestra.run.smithi123.stdout: "inst": { 2024-02-16T22:18:40.149 INFO:teuthology.orchestra.run.smithi123.stdout: "name": { 2024-02-16T22:18:40.149 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "client", 2024-02-16T22:18:40.149 INFO:teuthology.orchestra.run.smithi123.stdout: "num": 24321 2024-02-16T22:18:40.149 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-02-16T22:18:40.149 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2024-02-16T22:18:40.150 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-02-16T22:18:40.150 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.1:0", 2024-02-16T22:18:40.150 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2666785881 2024-02-16T22:18:40.150 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-02-16T22:18:40.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-02-16T22:18:40.150 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2024-02-16T22:18:40.150 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-02-16T22:18:40.150 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.1:0", 2024-02-16T22:18:40.150 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2666785881 2024-02-16T22:18:40.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-02-16T22:18:40.150 INFO:teuthology.orchestra.run.smithi123.stdout: "inst_str": "client.24321 192.168.0.1:0/2666785881", 2024-02-16T22:18:40.150 INFO:teuthology.orchestra.run.smithi123.stdout: "addr_str": "192.168.0.1:0/2666785881", 2024-02-16T22:18:40.150 INFO:teuthology.orchestra.run.smithi123.stdout: "inode_count": 1, 2024-02-16T22:18:40.150 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_epoch": 12, 2024-02-16T22:18:40.150 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch": 45, 2024-02-16T22:18:40.150 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch_barrier": 0, 2024-02-16T22:18:40.151 INFO:teuthology.orchestra.run.smithi123.stdout: "blocklisted": false, 2024-02-16T22:18:40.151 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs" 2024-02-16T22:18:40.151 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-02-16T22:18:40.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:40.157 DEBUG:teuthology.orchestra.run.smithi114:> stat --file-system '--printf=%T 2024-02-16T22:18:40.157 DEBUG:teuthology.orchestra.run.smithi114:> ' -- /home/ubuntu/cephtest/mnt.0 2024-02-16T22:18:40.177 INFO:teuthology.orchestra.run.smithi114.stdout:fuseblk 2024-02-16T22:18:40.177 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-02-16T22:18:40.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:40.177 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-16T22:18:40.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:39 smithi114 ceph-mon[29235]: pgmap v197: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-16T22:18:40.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:40.236 DEBUG:teuthology.orchestra.run.smithi123:> stat --file-system '--printf=%T 2024-02-16T22:18:40.236 DEBUG:teuthology.orchestra.run.smithi123:> ' -- /home/ubuntu/cephtest/mnt.1 2024-02-16T22:18:40.251 INFO:teuthology.orchestra.run.smithi123.stdout:fuseblk 2024-02-16T22:18:40.251 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-02-16T22:18:40.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:18:40.251 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-02-16T22:18:40.320 INFO:teuthology.run_tasks:Running task print... 2024-02-16T22:18:40.329 INFO:teuthology.task.print:**** done client 2024-02-16T22:18:40.329 INFO:teuthology.run_tasks:Running task parallel... 2024-02-16T22:18:40.339 INFO:teuthology.task.parallel:starting parallel... 2024-02-16T22:18:40.339 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-02-16T22:18:40.340 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-02-16T22:18:40.340 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi114.front.sepia.ceph.com 2024-02-16T22:18:40.340 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs false || true' 2024-02-16T22:18:40.341 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-02-16T22:18:40.341 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-02-16T22:18:40.344 INFO:tasks.workunit:Pulling workunits from ref ed7ab725e1a43615009dfb74a42bf33d80081bc8 2024-02-16T22:18:40.345 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-16T22:18:40.345 INFO:tasks.workunit:timeout=3h 2024-02-16T22:18:40.345 INFO:tasks.workunit:cleanup=True 2024-02-16T22:18:40.346 DEBUG:teuthology.orchestra.run.smithi114:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-16T22:18:40.364 INFO:teuthology.orchestra.run.smithi114.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-02-16T22:18:40.364 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-02-16T22:18:40.364 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 9fh/159d Inode: 1 Links: 2 2024-02-16T22:18:40.364 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T22:18:40.364 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:fusefs_t:s0 2024-02-16T22:18:40.364 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-02-16T22:18:40.364 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-02-16 22:18:06.360911174 +0000 2024-02-16T22:18:40.364 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-02-16 22:18:40.232301115 +0000 2024-02-16T22:18:40.364 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: - 2024-02-16T22:18:40.364 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-02-16T22:18:40.365 DEBUG:teuthology.orchestra.run.smithi114:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-02-16T22:18:40.437 DEBUG:teuthology.orchestra.run.smithi123:> stat -- /home/ubuntu/cephtest/mnt.1 2024-02-16T22:18:40.453 INFO:teuthology.orchestra.run.smithi123.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-02-16T22:18:40.453 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-02-16T22:18:40.454 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 7eh/126d Inode: 1 Links: 3 2024-02-16T22:18:40.454 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T22:18:40.454 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:fusefs_t:s0 2024-02-16T22:18:40.454 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-02-16T22:18:40.454 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-02-16 22:18:40.431370697 +0000 2024-02-16T22:18:40.454 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-02-16 22:18:40.431370697 +0000 2024-02-16T22:18:40.454 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2024-02-16T22:18:40.454 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-02-16T22:18:40.454 DEBUG:teuthology.orchestra.run.smithi123:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-02-16T22:18:40.525 DEBUG:teuthology.orchestra.run.smithi114:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/batrick/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout ed7ab725e1a43615009dfb74a42bf33d80081bc8 2024-02-16T22:18:40.525 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://github.com/batrick/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout ed7ab725e1a43615009dfb74a42bf33d80081bc8 2024-02-16T22:18:40.540 INFO:tasks.workunit.client.0.smithi114.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-16T22:18:40.564 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:18:40.580 INFO:tasks.workunit.client.1.smithi123.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-02-16T22:18:41.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:41 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:18:41.908 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:41 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:18:42.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.498+0000 7f3926e14700 1 -- 172.21.15.114:0/748042556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39200ffab0 msgr2=0x7f39200ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:42.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.498+0000 7f3926e14700 1 --2- 172.21.15.114:0/748042556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39200ffab0 0x7f39200ffed0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3914009910 tx=0x7f3914009c20 comp rx=0 tx=0).stop 2024-02-16T22:18:42.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.498+0000 7f3926e14700 1 -- 172.21.15.114:0/748042556 shutdown_connections 2024-02-16T22:18:42.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.498+0000 7f3926e14700 1 --2- 172.21.15.114:0/748042556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3920100cb0 0x7f3920101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:42.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.498+0000 7f3926e14700 1 --2- 172.21.15.114:0/748042556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39200ffab0 0x7f39200ffed0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:42.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.498+0000 7f3926e14700 1 -- 172.21.15.114:0/748042556 >> 172.21.15.114:0/748042556 conn(0x7f39200fb030 msgr2=0x7f39200fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:42.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.498+0000 7f3926e14700 1 -- 172.21.15.114:0/748042556 shutdown_connections 2024-02-16T22:18:42.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.498+0000 7f3926e14700 1 -- 172.21.15.114:0/748042556 wait complete. 2024-02-16T22:18:42.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.499+0000 7f3926e14700 1 Processor -- start 2024-02-16T22:18:42.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.499+0000 7f3926e14700 1 -- start start 2024-02-16T22:18:42.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.499+0000 7f3926e14700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39200ffab0 0x7f3920068f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:42.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.499+0000 7f3926e14700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3920100cb0 0x7f392019b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:42.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.499+0000 7f3926e14700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392019b650 con 0x7f3920100cb0 2024-02-16T22:18:42.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.499+0000 7f3926e14700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392019b7c0 con 0x7f39200ffab0 2024-02-16T22:18:42.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.499+0000 7f391ffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3920100cb0 0x7f392019b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:42.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.499+0000 7f391ffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3920100cb0 0x7f392019b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48918/0 (socket says 172.21.15.114:48918) 2024-02-16T22:18:42.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.499+0000 7f391ffff700 1 -- 172.21.15.114:0/2213552158 learned_addr learned my addr 172.21.15.114:0/2213552158 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:18:42.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.499+0000 7f3924bb0700 1 --2- 172.21.15.114:0/2213552158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39200ffab0 0x7f3920068f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:42.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.499+0000 7f391ffff700 1 -- 172.21.15.114:0/2213552158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39200ffab0 msgr2=0x7f3920068f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:42.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.499+0000 7f391ffff700 1 --2- 172.21.15.114:0/2213552158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39200ffab0 0x7f3920068f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:42.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.499+0000 7f391ffff700 1 -- 172.21.15.114:0/2213552158 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39140095f0 con 0x7f3920100cb0 2024-02-16T22:18:42.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.500+0000 7f391ffff700 1 --2- 172.21.15.114:0/2213552158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3920100cb0 0x7f392019b080 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f390c00d8d0 tx=0x7f390c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:42.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.500+0000 7f391dffb700 1 -- 172.21.15.114:0/2213552158 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f390c0098a0 con 0x7f3920100cb0 2024-02-16T22:18:42.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.500+0000 7f3926e14700 1 -- 172.21.15.114:0/2213552158 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f392019baa0 con 0x7f3920100cb0 2024-02-16T22:18:42.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.500+0000 7f391dffb700 1 -- 172.21.15.114:0/2213552158 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f390c010460 con 0x7f3920100cb0 2024-02-16T22:18:42.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.500+0000 7f391dffb700 1 -- 172.21.15.114:0/2213552158 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f390c00f5f0 con 0x7f3920100cb0 2024-02-16T22:18:42.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.500+0000 7f3926e14700 1 -- 172.21.15.114:0/2213552158 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f392019bff0 con 0x7f3920100cb0 2024-02-16T22:18:42.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.501+0000 7f3926e14700 1 -- 172.21.15.114:0/2213552158 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3920108610 con 0x7f3920100cb0 2024-02-16T22:18:42.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.501+0000 7f391dffb700 1 -- 172.21.15.114:0/2213552158 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f390c009a00 con 0x7f3920100cb0 2024-02-16T22:18:42.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.502+0000 7f391dffb700 1 --2- 172.21.15.114:0/2213552158 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f391006c4e0 0x7f391006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:42.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.502+0000 7f391dffb700 1 -- 172.21.15.114:0/2213552158 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f390c08b7b0 con 0x7f3920100cb0 2024-02-16T22:18:42.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.502+0000 7f3924bb0700 1 --2- 172.21.15.114:0/2213552158 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f391006c4e0 0x7f391006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:42.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.502+0000 7f3924bb0700 1 --2- 172.21.15.114:0/2213552158 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f391006c4e0 0x7f391006e9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3914006010 tx=0x7f391400b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:42.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.506+0000 7f391dffb700 1 -- 172.21.15.114:0/2213552158 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f390c059980 con 0x7f3920100cb0 2024-02-16T22:18:42.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.697+0000 7f3926e14700 1 -- 172.21.15.114:0/2213552158 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f39201087d0 con 0x7f3920100cb0 2024-02-16T22:18:42.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.702+0000 7f391dffb700 1 -- 172.21.15.114:0/2213552158 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v22)=0 v22) v1 ==== 126+0+0 (secure 0 0 0) 0x7f390c059510 con 0x7f3920100cb0 2024-02-16T22:18:42.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.704+0000 7f3926e14700 1 -- 172.21.15.114:0/2213552158 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f391006c4e0 msgr2=0x7f391006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:42.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.704+0000 7f3926e14700 1 --2- 172.21.15.114:0/2213552158 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f391006c4e0 0x7f391006e9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3914006010 tx=0x7f391400b540 comp rx=0 tx=0).stop 2024-02-16T22:18:42.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.704+0000 7f3926e14700 1 -- 172.21.15.114:0/2213552158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3920100cb0 msgr2=0x7f392019b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:42.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.704+0000 7f3926e14700 1 --2- 172.21.15.114:0/2213552158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3920100cb0 0x7f392019b080 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f390c00d8d0 tx=0x7f390c00dbe0 comp rx=0 tx=0).stop 2024-02-16T22:18:42.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.704+0000 7f3926e14700 1 -- 172.21.15.114:0/2213552158 shutdown_connections 2024-02-16T22:18:42.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.704+0000 7f3926e14700 1 --2- 172.21.15.114:0/2213552158 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f391006c4e0 0x7f391006e9a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:42.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.705+0000 7f3926e14700 1 --2- 172.21.15.114:0/2213552158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39200ffab0 0x7f3920068f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:42.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.705+0000 7f3926e14700 1 --2- 172.21.15.114:0/2213552158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3920100cb0 0x7f392019b080 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:42.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.705+0000 7f3926e14700 1 -- 172.21.15.114:0/2213552158 >> 172.21.15.114:0/2213552158 conn(0x7f39200fb030 msgr2=0x7f3920103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:42.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.705+0000 7f3926e14700 1 -- 172.21.15.114:0/2213552158 shutdown_connections 2024-02-16T22:18:42.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:42.705+0000 7f3926e14700 1 -- 172.21.15.114:0/2213552158 wait complete. 2024-02-16T22:18:42.747 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:42 smithi114 ceph-mon[29235]: pgmap v198: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-02-16T22:18:42.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:42 smithi123 ceph-mon[37161]: pgmap v198: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-02-16T22:18:43.413 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-02-16T22:18:43.414 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi114.front.sepia.ceph.com 2024-02-16T22:18:43.414 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-02-16T22:18:43.665 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:18:44.045 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:43 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/2213552158' entity='client.admin' 2024-02-16T22:18:44.045 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:43 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:18:44.046 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:43 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:18:44.046 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:43 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:18:44.046 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:43 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:44.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:43 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/2213552158' entity='client.admin' 2024-02-16T22:18:44.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:43 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:18:44.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:43 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:18:44.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:43 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:18:44.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:43 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:45.025 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:44 smithi114 ceph-mon[29235]: pgmap v199: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-02-16T22:18:45.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:44 smithi123 ceph-mon[37161]: pgmap v199: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-02-16T22:18:45.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.385+0000 7fa76886b700 1 -- 172.21.15.114:0/1320826252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa764100c00 msgr2=0x7fa764101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:45.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.385+0000 7fa76886b700 1 --2- 172.21.15.114:0/1320826252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa764100c00 0x7fa764101020 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa754009910 tx=0x7fa754009c20 comp rx=0 tx=0).stop 2024-02-16T22:18:45.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.385+0000 7fa76886b700 1 -- 172.21.15.114:0/1320826252 shutdown_connections 2024-02-16T22:18:45.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.385+0000 7fa76886b700 1 --2- 172.21.15.114:0/1320826252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa764101e00 0x7fa764102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:45.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.385+0000 7fa76886b700 1 --2- 172.21.15.114:0/1320826252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa764100c00 0x7fa764101020 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:45.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.385+0000 7fa76886b700 1 -- 172.21.15.114:0/1320826252 >> 172.21.15.114:0/1320826252 conn(0x7fa7640fc180 msgr2=0x7fa7640fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:45.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.385+0000 7fa76886b700 1 -- 172.21.15.114:0/1320826252 shutdown_connections 2024-02-16T22:18:45.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.385+0000 7fa76886b700 1 -- 172.21.15.114:0/1320826252 wait complete. 2024-02-16T22:18:45.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.386+0000 7fa76886b700 1 Processor -- start 2024-02-16T22:18:45.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.386+0000 7fa76886b700 1 -- start start 2024-02-16T22:18:45.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.386+0000 7fa76886b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa764100c00 0x7fa764196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:45.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.386+0000 7fa76886b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa764101e00 0x7fa7641968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:45.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.386+0000 7fa76886b700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa764196ef0 con 0x7fa764100c00 2024-02-16T22:18:45.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.386+0000 7fa76886b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa764197030 con 0x7fa764101e00 2024-02-16T22:18:45.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.387+0000 7fa76259c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa764100c00 0x7fa764196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:45.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.387+0000 7fa761d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa764101e00 0x7fa7641968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:45.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.387+0000 7fa76259c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa764100c00 0x7fa764196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48940/0 (socket says 172.21.15.114:48940) 2024-02-16T22:18:45.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.387+0000 7fa761d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa764101e00 0x7fa7641968d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.114:45486/0 (socket says 172.21.15.114:45486) 2024-02-16T22:18:45.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.387+0000 7fa76259c700 1 -- 172.21.15.114:0/2469745635 learned_addr learned my addr 172.21.15.114:0/2469745635 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:18:45.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.387+0000 7fa76259c700 1 -- 172.21.15.114:0/2469745635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa764101e00 msgr2=0x7fa7641968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:45.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.387+0000 7fa76259c700 1 --2- 172.21.15.114:0/2469745635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa764101e00 0x7fa7641968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:45.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.387+0000 7fa76259c700 1 -- 172.21.15.114:0/2469745635 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7540095f0 con 0x7fa764100c00 2024-02-16T22:18:45.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.387+0000 7fa76259c700 1 --2- 172.21.15.114:0/2469745635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa764100c00 0x7fa764196390 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa754000c00 tx=0x7fa75400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:45.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.387+0000 7fa7537fe700 1 -- 172.21.15.114:0/2469745635 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa75401d070 con 0x7fa764100c00 2024-02-16T22:18:45.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.387+0000 7fa76886b700 1 -- 172.21.15.114:0/2469745635 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa76419ba80 con 0x7fa764100c00 2024-02-16T22:18:45.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.387+0000 7fa7537fe700 1 -- 172.21.15.114:0/2469745635 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa754005dc0 con 0x7fa764100c00 2024-02-16T22:18:45.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.387+0000 7fa7537fe700 1 -- 172.21.15.114:0/2469745635 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa754022470 con 0x7fa764100c00 2024-02-16T22:18:45.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.387+0000 7fa76886b700 1 -- 172.21.15.114:0/2469745635 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa76419bf70 con 0x7fa764100c00 2024-02-16T22:18:45.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.388+0000 7fa76886b700 1 -- 172.21.15.114:0/2469745635 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa764109760 con 0x7fa764100c00 2024-02-16T22:18:45.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.389+0000 7fa7537fe700 1 -- 172.21.15.114:0/2469745635 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fa754003810 con 0x7fa764100c00 2024-02-16T22:18:45.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.389+0000 7fa7537fe700 1 --2- 172.21.15.114:0/2469745635 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa74c06c4e0 0x7fa74c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:45.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.389+0000 7fa7537fe700 1 -- 172.21.15.114:0/2469745635 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fa75408cdc0 con 0x7fa764100c00 2024-02-16T22:18:45.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.389+0000 7fa761d9b700 1 --2- 172.21.15.114:0/2469745635 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa74c06c4e0 0x7fa74c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:45.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.390+0000 7fa761d9b700 1 --2- 172.21.15.114:0/2469745635 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa74c06c4e0 0x7fa74c06e9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa758009c60 tx=0x7fa758009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:45.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.396+0000 7fa7537fe700 1 -- 172.21.15.114:0/2469745635 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa75405b030 con 0x7fa764100c00 2024-02-16T22:18:45.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.589+0000 7fa76886b700 1 -- 172.21.15.114:0/2469745635 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7fa764109950 con 0x7fa764100c00 2024-02-16T22:18:45.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.589+0000 7fa7537fe700 1 -- 172.21.15.114:0/2469745635 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v22)=0 v22) v1 ==== 155+0+0 (secure 0 0 0) 0x7fa75405abc0 con 0x7fa764100c00 2024-02-16T22:18:45.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.591+0000 7fa76886b700 1 -- 172.21.15.114:0/2469745635 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa74c06c4e0 msgr2=0x7fa74c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:45.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.591+0000 7fa76886b700 1 --2- 172.21.15.114:0/2469745635 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa74c06c4e0 0x7fa74c06e9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa758009c60 tx=0x7fa758009260 comp rx=0 tx=0).stop 2024-02-16T22:18:45.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.591+0000 7fa76886b700 1 -- 172.21.15.114:0/2469745635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa764100c00 msgr2=0x7fa764196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:45.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.591+0000 7fa76886b700 1 --2- 172.21.15.114:0/2469745635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa764100c00 0x7fa764196390 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa754000c00 tx=0x7fa75400f690 comp rx=0 tx=0).stop 2024-02-16T22:18:45.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.591+0000 7fa76886b700 1 -- 172.21.15.114:0/2469745635 shutdown_connections 2024-02-16T22:18:45.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.591+0000 7fa76886b700 1 --2- 172.21.15.114:0/2469745635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa764100c00 0x7fa764196390 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:45.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.591+0000 7fa76886b700 1 --2- 172.21.15.114:0/2469745635 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa74c06c4e0 0x7fa74c06e9a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:45.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.591+0000 7fa76886b700 1 --2- 172.21.15.114:0/2469745635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa764101e00 0x7fa7641968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:45.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.591+0000 7fa76886b700 1 -- 172.21.15.114:0/2469745635 >> 172.21.15.114:0/2469745635 conn(0x7fa7640fc180 msgr2=0x7fa764105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:45.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.591+0000 7fa76886b700 1 -- 172.21.15.114:0/2469745635 shutdown_connections 2024-02-16T22:18:45.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:45.591+0000 7fa76886b700 1 -- 172.21.15.114:0/2469745635 wait complete. 2024-02-16T22:18:46.320 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-02-16T22:18:46.547 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:18:46.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:46 smithi114 ceph-mon[29235]: pgmap v200: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-02-16T22:18:47.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:46 smithi123 ceph-mon[37161]: pgmap v200: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-02-16T22:18:48.194 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:47 smithi114 ceph-mon[29235]: pgmap v201: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-02-16T22:18:48.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:47 smithi123 ceph-mon[37161]: pgmap v201: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-02-16T22:18:48.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.469+0000 7f0fa7eba700 1 -- 172.21.15.114:0/231500183 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fa0100470 msgr2=0x7f0fa01008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:48.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.469+0000 7f0fa7eba700 1 --2- 172.21.15.114:0/231500183 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fa0100470 0x7f0fa01008d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00aa20 tx=0x7f0f9c00ad30 comp rx=0 tx=0).stop 2024-02-16T22:18:48.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.469+0000 7f0fa7eba700 1 -- 172.21.15.114:0/231500183 shutdown_connections 2024-02-16T22:18:48.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.469+0000 7f0fa7eba700 1 --2- 172.21.15.114:0/231500183 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fa0100470 0x7f0fa01008d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:48.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.469+0000 7f0fa7eba700 1 --2- 172.21.15.114:0/231500183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fa00ff270 0x7f0fa00ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:48.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.469+0000 7f0fa7eba700 1 -- 172.21.15.114:0/231500183 >> 172.21.15.114:0/231500183 conn(0x7f0fa00fa810 msgr2=0x7f0fa00fcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:48.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.469+0000 7f0fa7eba700 1 -- 172.21.15.114:0/231500183 shutdown_connections 2024-02-16T22:18:48.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.469+0000 7f0fa7eba700 1 -- 172.21.15.114:0/231500183 wait complete. 2024-02-16T22:18:48.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.470+0000 7f0fa7eba700 1 Processor -- start 2024-02-16T22:18:48.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.470+0000 7f0fa7eba700 1 -- start start 2024-02-16T22:18:48.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.470+0000 7f0fa7eba700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fa00ff270 0x7f0fa0196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:48.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.470+0000 7f0fa7eba700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fa0100470 0x7f0fa0196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:48.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.470+0000 7f0fa7eba700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa0196f70 con 0x7f0fa0100470 2024-02-16T22:18:48.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.470+0000 7f0fa7eba700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa01970b0 con 0x7f0fa00ff270 2024-02-16T22:18:48.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.470+0000 7f0fa5455700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fa0100470 0x7f0fa0196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:48.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.470+0000 7f0fa5455700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fa0100470 0x7f0fa0196950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48960/0 (socket says 172.21.15.114:48960) 2024-02-16T22:18:48.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.470+0000 7f0fa5455700 1 -- 172.21.15.114:0/3357539695 learned_addr learned my addr 172.21.15.114:0/3357539695 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:18:48.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.470+0000 7f0fa5c56700 1 --2- 172.21.15.114:0/3357539695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fa00ff270 0x7f0fa0196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:48.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.470+0000 7f0fa5455700 1 -- 172.21.15.114:0/3357539695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fa00ff270 msgr2=0x7f0fa0196410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:48.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.470+0000 7f0fa5455700 1 --2- 172.21.15.114:0/3357539695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fa00ff270 0x7f0fa0196410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:48.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.470+0000 7f0fa5455700 1 -- 172.21.15.114:0/3357539695 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f9c00a700 con 0x7f0fa0100470 2024-02-16T22:18:48.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.471+0000 7f0fa5455700 1 --2- 172.21.15.114:0/3357539695 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fa0100470 0x7f0fa0196950 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00fd70 tx=0x7f0f9c00fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:48.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.471+0000 7f0f96ffd700 1 -- 172.21.15.114:0/3357539695 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f9c00ccc0 con 0x7f0fa0100470 2024-02-16T22:18:48.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.471+0000 7f0f96ffd700 1 -- 172.21.15.114:0/3357539695 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0f9c0050f0 con 0x7f0fa0100470 2024-02-16T22:18:48.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.471+0000 7f0fa7eba700 1 -- 172.21.15.114:0/3357539695 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fa019bb00 con 0x7f0fa0100470 2024-02-16T22:18:48.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.471+0000 7f0f96ffd700 1 -- 172.21.15.114:0/3357539695 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f9c016770 con 0x7f0fa0100470 2024-02-16T22:18:48.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.471+0000 7f0fa7eba700 1 -- 172.21.15.114:0/3357539695 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fa019c070 con 0x7f0fa0100470 2024-02-16T22:18:48.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.472+0000 7f0fa7eba700 1 -- 172.21.15.114:0/3357539695 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fa0190360 con 0x7f0fa0100470 2024-02-16T22:18:48.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.472+0000 7f0f96ffd700 1 -- 172.21.15.114:0/3357539695 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f0f9c0169e0 con 0x7f0fa0100470 2024-02-16T22:18:48.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.473+0000 7f0f96ffd700 1 --2- 172.21.15.114:0/3357539695 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f0f8c06c4e0 0x7f0f8c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:48.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.473+0000 7f0f96ffd700 1 -- 172.21.15.114:0/3357539695 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f0f9c012070 con 0x7f0fa0100470 2024-02-16T22:18:48.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.473+0000 7f0fa5c56700 1 --2- 172.21.15.114:0/3357539695 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f0f8c06c4e0 0x7f0f8c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:48.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.473+0000 7f0fa5c56700 1 --2- 172.21.15.114:0/3357539695 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f0f8c06c4e0 0x7f0f8c06e9a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0fa01002d0 tx=0x7f0f90009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:48.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.477+0000 7f0f96ffd700 1 -- 172.21.15.114:0/3357539695 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0f9c05b7e0 con 0x7f0fa0100470 2024-02-16T22:18:48.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.697+0000 7f0fa7eba700 1 -- 172.21.15.114:0/3357539695 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f0fa002cc70 con 0x7f0fa0100470 2024-02-16T22:18:48.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.698+0000 7f0f96ffd700 1 -- 172.21.15.114:0/3357539695 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v22)=0 v22) v1 ==== 163+0+0 (secure 0 0 0) 0x7f0f9c05b370 con 0x7f0fa0100470 2024-02-16T22:18:48.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.700+0000 7f0fa7eba700 1 -- 172.21.15.114:0/3357539695 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f0f8c06c4e0 msgr2=0x7f0f8c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:48.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.700+0000 7f0fa7eba700 1 --2- 172.21.15.114:0/3357539695 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f0f8c06c4e0 0x7f0f8c06e9a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0fa01002d0 tx=0x7f0f90009260 comp rx=0 tx=0).stop 2024-02-16T22:18:48.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.700+0000 7f0fa7eba700 1 -- 172.21.15.114:0/3357539695 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fa0100470 msgr2=0x7f0fa0196950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:48.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.700+0000 7f0fa7eba700 1 --2- 172.21.15.114:0/3357539695 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fa0100470 0x7f0fa0196950 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00fd70 tx=0x7f0f9c00fda0 comp rx=0 tx=0).stop 2024-02-16T22:18:48.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.700+0000 7f0fa7eba700 1 -- 172.21.15.114:0/3357539695 shutdown_connections 2024-02-16T22:18:48.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.700+0000 7f0fa7eba700 1 --2- 172.21.15.114:0/3357539695 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f0f8c06c4e0 0x7f0f8c06e9a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:48.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.700+0000 7f0fa7eba700 1 --2- 172.21.15.114:0/3357539695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fa00ff270 0x7f0fa0196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:48.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.700+0000 7f0fa7eba700 1 --2- 172.21.15.114:0/3357539695 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fa0100470 0x7f0fa0196950 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:48.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.700+0000 7f0fa7eba700 1 -- 172.21.15.114:0/3357539695 >> 172.21.15.114:0/3357539695 conn(0x7f0fa00fa810 msgr2=0x7f0fa01038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:48.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.700+0000 7f0fa7eba700 1 -- 172.21.15.114:0/3357539695 shutdown_connections 2024-02-16T22:18:48.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:48.700+0000 7f0fa7eba700 1 -- 172.21.15.114:0/3357539695 wait complete. 2024-02-16T22:18:49.352 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph config set global log_to_journald false --force' 2024-02-16T22:18:49.568 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:18:50.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:50 smithi123 ceph-mon[37161]: pgmap v202: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-02-16T22:18:50.847 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:50 smithi114 ceph-mon[29235]: pgmap v202: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-02-16T22:18:51.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.411+0000 7f0fd9620700 1 -- 172.21.15.114:0/1171148233 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fd4100bd0 msgr2=0x7f0fd4100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:51.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.411+0000 7f0fd9620700 1 --2- 172.21.15.114:0/1171148233 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fd4100bd0 0x7f0fd4100ff0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009910 tx=0x7f0fc4009c20 comp rx=0 tx=0).stop 2024-02-16T22:18:51.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.411+0000 7f0fd9620700 1 -- 172.21.15.114:0/1171148233 shutdown_connections 2024-02-16T22:18:51.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.411+0000 7f0fd9620700 1 --2- 172.21.15.114:0/1171148233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fd4101dd0 0x7f0fd4102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:51.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.411+0000 7f0fd9620700 1 --2- 172.21.15.114:0/1171148233 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fd4100bd0 0x7f0fd4100ff0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:51.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.411+0000 7f0fd9620700 1 -- 172.21.15.114:0/1171148233 >> 172.21.15.114:0/1171148233 conn(0x7f0fd40fc190 msgr2=0x7f0fd40fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:51.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.411+0000 7f0fd9620700 1 -- 172.21.15.114:0/1171148233 shutdown_connections 2024-02-16T22:18:51.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.411+0000 7f0fd9620700 1 -- 172.21.15.114:0/1171148233 wait complete. 2024-02-16T22:18:51.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.412+0000 7f0fd9620700 1 Processor -- start 2024-02-16T22:18:51.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.412+0000 7f0fd9620700 1 -- start start 2024-02-16T22:18:51.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.413+0000 7f0fd9620700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fd4100bd0 0x7f0fd4198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:51.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.413+0000 7f0fd9620700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fd4101dd0 0x7f0fd4198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:51.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.413+0000 7f0fd9620700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd41990d0 con 0x7f0fd4100bd0 2024-02-16T22:18:51.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.413+0000 7f0fd9620700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd4199210 con 0x7f0fd4101dd0 2024-02-16T22:18:51.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.413+0000 7f0fd27fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fd4101dd0 0x7f0fd4198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:51.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.413+0000 7f0fd27fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fd4101dd0 0x7f0fd4198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.114:45540/0 (socket says 172.21.15.114:45540) 2024-02-16T22:18:51.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.413+0000 7f0fd27fc700 1 -- 172.21.15.114:0/1336701844 learned_addr learned my addr 172.21.15.114:0/1336701844 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:18:51.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.413+0000 7f0fd2ffd700 1 --2- 172.21.15.114:0/1336701844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fd4100bd0 0x7f0fd4198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:51.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.413+0000 7f0fd27fc700 1 -- 172.21.15.114:0/1336701844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fd4100bd0 msgr2=0x7f0fd4198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:51.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.413+0000 7f0fd27fc700 1 --2- 172.21.15.114:0/1336701844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fd4100bd0 0x7f0fd4198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:51.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.413+0000 7f0fd27fc700 1 -- 172.21.15.114:0/1336701844 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc40095f0 con 0x7f0fd4101dd0 2024-02-16T22:18:51.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.413+0000 7f0fd27fc700 1 --2- 172.21.15.114:0/1336701844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fd4101dd0 0x7f0fd4198ab0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0fc800e920 tx=0x7f0fc800ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:51.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.414+0000 7f0fbbfff700 1 -- 172.21.15.114:0/1336701844 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fc800c9d0 con 0x7f0fd4101dd0 2024-02-16T22:18:51.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.414+0000 7f0fbbfff700 1 -- 172.21.15.114:0/1336701844 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0fc8010430 con 0x7f0fd4101dd0 2024-02-16T22:18:51.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.414+0000 7f0fd9620700 1 -- 172.21.15.114:0/1336701844 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd419dcc0 con 0x7f0fd4101dd0 2024-02-16T22:18:51.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.414+0000 7f0fbbfff700 1 -- 172.21.15.114:0/1336701844 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fc8018650 con 0x7f0fd4101dd0 2024-02-16T22:18:51.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.414+0000 7f0fd9620700 1 -- 172.21.15.114:0/1336701844 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd419e210 con 0x7f0fd4101dd0 2024-02-16T22:18:51.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.415+0000 7f0fd9620700 1 -- 172.21.15.114:0/1336701844 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd4105eb0 con 0x7f0fd4101dd0 2024-02-16T22:18:51.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.415+0000 7f0fbbfff700 1 -- 172.21.15.114:0/1336701844 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f0fc80188c0 con 0x7f0fd4101dd0 2024-02-16T22:18:51.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.416+0000 7f0fbbfff700 1 --2- 172.21.15.114:0/1336701844 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f0fbc06c4e0 0x7f0fbc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:51.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.416+0000 7f0fbbfff700 1 -- 172.21.15.114:0/1336701844 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f0fc8014070 con 0x7f0fd4101dd0 2024-02-16T22:18:51.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.416+0000 7f0fd2ffd700 1 --2- 172.21.15.114:0/1336701844 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f0fbc06c4e0 0x7f0fbc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:51.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.416+0000 7f0fd2ffd700 1 --2- 172.21.15.114:0/1336701844 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f0fbc06c4e0 0x7f0fbc06e9a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400b6f0 tx=0x7f0fc4005940 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:51.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.420+0000 7f0fbbfff700 1 -- 172.21.15.114:0/1336701844 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0fc80520e0 con 0x7f0fd4101dd0 2024-02-16T22:18:51.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.625+0000 7f0fd9620700 1 -- 172.21.15.114:0/1336701844 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f0fd402cc70 con 0x7f0fd4101dd0 2024-02-16T22:18:51.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.625+0000 7f0fbbfff700 1 -- 172.21.15.114:0/1336701844 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v22)=0 v22) v1 ==== 135+0+0 (secure 0 0 0) 0x7f0fc8059cf0 con 0x7f0fd4101dd0 2024-02-16T22:18:51.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.627+0000 7f0fd9620700 1 -- 172.21.15.114:0/1336701844 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f0fbc06c4e0 msgr2=0x7f0fbc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:51.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.627+0000 7f0fd9620700 1 --2- 172.21.15.114:0/1336701844 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f0fbc06c4e0 0x7f0fbc06e9a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400b6f0 tx=0x7f0fc4005940 comp rx=0 tx=0).stop 2024-02-16T22:18:51.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.627+0000 7f0fd9620700 1 -- 172.21.15.114:0/1336701844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fd4101dd0 msgr2=0x7f0fd4198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:51.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.627+0000 7f0fd9620700 1 --2- 172.21.15.114:0/1336701844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fd4101dd0 0x7f0fd4198ab0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0fc800e920 tx=0x7f0fc800ec30 comp rx=0 tx=0).stop 2024-02-16T22:18:51.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.628+0000 7f0fd9620700 1 -- 172.21.15.114:0/1336701844 shutdown_connections 2024-02-16T22:18:51.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.628+0000 7f0fd9620700 1 --2- 172.21.15.114:0/1336701844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fd4100bd0 0x7f0fd4198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:51.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.628+0000 7f0fd9620700 1 --2- 172.21.15.114:0/1336701844 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f0fbc06c4e0 0x7f0fbc06e9a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:51.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.628+0000 7f0fd9620700 1 --2- 172.21.15.114:0/1336701844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fd4101dd0 0x7f0fd4198ab0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:51.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.628+0000 7f0fd9620700 1 -- 172.21.15.114:0/1336701844 >> 172.21.15.114:0/1336701844 conn(0x7f0fd40fc190 msgr2=0x7f0fd4102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:51.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.628+0000 7f0fd9620700 1 -- 172.21.15.114:0/1336701844 shutdown_connections 2024-02-16T22:18:51.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:51.628+0000 7f0fd9620700 1 -- 172.21.15.114:0/1336701844 wait complete. 2024-02-16T22:18:52.427 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-02-16T22:18:52.638 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:18:52.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:52 smithi114 ceph-mon[29235]: pgmap v203: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-02-16T22:18:52.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:52 smithi123 ceph-mon[37161]: pgmap v203: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-02-16T22:18:54.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.563+0000 7f0094f7c700 1 -- 172.21.15.114:0/1469153963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0090100bd0 msgr2=0x7f0090100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:54.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.563+0000 7f0094f7c700 1 --2- 172.21.15.114:0/1469153963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0090100bd0 0x7f0090100ff0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0088009910 tx=0x7f0088009c20 comp rx=0 tx=0).stop 2024-02-16T22:18:54.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.563+0000 7f0094f7c700 1 -- 172.21.15.114:0/1469153963 shutdown_connections 2024-02-16T22:18:54.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.563+0000 7f0094f7c700 1 --2- 172.21.15.114:0/1469153963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0090101dd0 0x7f0090102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:54.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.563+0000 7f0094f7c700 1 --2- 172.21.15.114:0/1469153963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0090100bd0 0x7f0090100ff0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:54.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.563+0000 7f0094f7c700 1 -- 172.21.15.114:0/1469153963 >> 172.21.15.114:0/1469153963 conn(0x7f00900fc190 msgr2=0x7f00900fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:54.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.563+0000 7f0094f7c700 1 -- 172.21.15.114:0/1469153963 shutdown_connections 2024-02-16T22:18:54.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.563+0000 7f0094f7c700 1 -- 172.21.15.114:0/1469153963 wait complete. 2024-02-16T22:18:54.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.564+0000 7f0094f7c700 1 Processor -- start 2024-02-16T22:18:54.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.564+0000 7f0094f7c700 1 -- start start 2024-02-16T22:18:54.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.564+0000 7f0094f7c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0090100bd0 0x7f0090196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:54.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.564+0000 7f0094f7c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0090101dd0 0x7f00901968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:54.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.564+0000 7f0094f7c700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0090196ed0 con 0x7f0090100bd0 2024-02-16T22:18:54.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.564+0000 7f0094f7c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0090197010 con 0x7f0090101dd0 2024-02-16T22:18:54.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.564+0000 7f008e59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0090100bd0 0x7f0090196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:54.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.564+0000 7f008e59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0090100bd0 0x7f0090196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53564/0 (socket says 172.21.15.114:53564) 2024-02-16T22:18:54.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.564+0000 7f008e59c700 1 -- 172.21.15.114:0/1693716733 learned_addr learned my addr 172.21.15.114:0/1693716733 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:18:54.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.564+0000 7f008dd9b700 1 --2- 172.21.15.114:0/1693716733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0090101dd0 0x7f00901968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:54.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.564+0000 7f008e59c700 1 -- 172.21.15.114:0/1693716733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0090101dd0 msgr2=0x7f00901968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:54.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.564+0000 7f008e59c700 1 --2- 172.21.15.114:0/1693716733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0090101dd0 0x7f00901968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:54.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.564+0000 7f008e59c700 1 -- 172.21.15.114:0/1693716733 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00880095f0 con 0x7f0090100bd0 2024-02-16T22:18:54.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.565+0000 7f008e59c700 1 --2- 172.21.15.114:0/1693716733 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0090100bd0 0x7f0090196370 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0088000c00 tx=0x7f008800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:54.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.565+0000 7f007f7fe700 1 -- 172.21.15.114:0/1693716733 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f008801d070 con 0x7f0090100bd0 2024-02-16T22:18:54.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.565+0000 7f0094f7c700 1 -- 172.21.15.114:0/1693716733 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f009019ba60 con 0x7f0090100bd0 2024-02-16T22:18:54.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.565+0000 7f007f7fe700 1 -- 172.21.15.114:0/1693716733 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0088005dc0 con 0x7f0090100bd0 2024-02-16T22:18:54.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.565+0000 7f007f7fe700 1 -- 172.21.15.114:0/1693716733 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0088022470 con 0x7f0090100bd0 2024-02-16T22:18:54.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.565+0000 7f0094f7c700 1 -- 172.21.15.114:0/1693716733 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f009019bf50 con 0x7f0090100bd0 2024-02-16T22:18:54.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.566+0000 7f0094f7c700 1 -- 172.21.15.114:0/1693716733 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0090109750 con 0x7f0090100bd0 2024-02-16T22:18:54.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.566+0000 7f007f7fe700 1 -- 172.21.15.114:0/1693716733 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f0088003810 con 0x7f0090100bd0 2024-02-16T22:18:54.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.567+0000 7f007f7fe700 1 --2- 172.21.15.114:0/1693716733 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f007806c4e0 0x7f007806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:54.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.567+0000 7f007f7fe700 1 -- 172.21.15.114:0/1693716733 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f008808cda0 con 0x7f0090100bd0 2024-02-16T22:18:54.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.567+0000 7f008dd9b700 1 --2- 172.21.15.114:0/1693716733 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f007806c4e0 0x7f007806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:54.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.567+0000 7f008dd9b700 1 --2- 172.21.15.114:0/1693716733 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f007806c4e0 0x7f007806e9a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0084009c60 tx=0x7f0084009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:54.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.571+0000 7f007f7fe700 1 -- 172.21.15.114:0/1693716733 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f008805b010 con 0x7f0090100bd0 2024-02-16T22:18:54.573 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:54 smithi114 ceph-mon[29235]: pgmap v204: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-02-16T22:18:54.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.782+0000 7f0094f7c700 1 -- 172.21.15.114:0/1693716733 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275", "target": ["mon-mgr", ""]}) v1 -- 0x7f00901067c0 con 0x7f007806c4e0 2024-02-16T22:18:54.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.788+0000 7f007f7fe700 1 -- 172.21.15.114:0/1693716733 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f00901067c0 con 0x7f007806c4e0 2024-02-16T22:18:54.790 INFO:teuthology.orchestra.run.smithi114.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T22:18:54.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.790+0000 7f0094f7c700 1 -- 172.21.15.114:0/1693716733 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f007806c4e0 msgr2=0x7f007806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:54.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.790+0000 7f0094f7c700 1 --2- 172.21.15.114:0/1693716733 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f007806c4e0 0x7f007806e9a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0084009c60 tx=0x7f0084009260 comp rx=0 tx=0).stop 2024-02-16T22:18:54.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.790+0000 7f0094f7c700 1 -- 172.21.15.114:0/1693716733 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0090100bd0 msgr2=0x7f0090196370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:54.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.790+0000 7f0094f7c700 1 --2- 172.21.15.114:0/1693716733 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0090100bd0 0x7f0090196370 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0088000c00 tx=0x7f008800f690 comp rx=0 tx=0).stop 2024-02-16T22:18:54.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.790+0000 7f0094f7c700 1 -- 172.21.15.114:0/1693716733 shutdown_connections 2024-02-16T22:18:54.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.790+0000 7f0094f7c700 1 --2- 172.21.15.114:0/1693716733 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0090100bd0 0x7f0090196370 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:54.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.790+0000 7f0094f7c700 1 --2- 172.21.15.114:0/1693716733 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f007806c4e0 0x7f007806e9a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:54.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.790+0000 7f0094f7c700 1 --2- 172.21.15.114:0/1693716733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0090101dd0 0x7f00901968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:54.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.790+0000 7f0094f7c700 1 -- 172.21.15.114:0/1693716733 >> 172.21.15.114:0/1693716733 conn(0x7f00900fc190 msgr2=0x7f0090105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:54.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.790+0000 7f0094f7c700 1 -- 172.21.15.114:0/1693716733 shutdown_connections 2024-02-16T22:18:54.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:54.790+0000 7f0094f7c700 1 -- 172.21.15.114:0/1693716733 wait complete. 2024-02-16T22:18:54.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:54 smithi123 ceph-mon[37161]: pgmap v204: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-02-16T22:18:55.511 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-02-16T22:18:55.512 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi114.front.sepia.ceph.com 2024-02-16T22:18:55.512 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-02-16T22:18:55.735 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:18:56.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:55 smithi123 ceph-mon[37161]: from='client.14564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:18:56.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:55 smithi123 ceph-mon[37161]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T22:18:56.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:55 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:56.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:55 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:18:56.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:55 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:18:56.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:55 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:18:56.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:55 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:56.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:55 smithi123 ceph-mon[37161]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T22:18:56.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:55 smithi123 ceph-mon[37161]: pgmap v205: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-16T22:18:56.083 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:55 smithi114 ceph-mon[29235]: from='client.14564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:18:56.084 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:55 smithi114 ceph-mon[29235]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T22:18:56.084 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:55 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:56.084 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:55 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:18:56.084 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:55 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:18:56.084 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:55 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:18:56.084 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:55 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:18:56.084 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:55 smithi114 ceph-mon[29235]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T22:18:56.084 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:55 smithi114 ceph-mon[29235]: pgmap v205: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-16T22:18:57.643 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:57 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:18:57.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.648+0000 7fe1c04eb700 1 -- 172.21.15.114:0/4118109263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1b80ffab0 msgr2=0x7fe1b80ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:57.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.648+0000 7fe1c04eb700 1 --2- 172.21.15.114:0/4118109263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1b80ffab0 0x7fe1b80ffed0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4009910 tx=0x7fe1b4009c20 comp rx=0 tx=0).stop 2024-02-16T22:18:57.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.648+0000 7fe1c04eb700 1 -- 172.21.15.114:0/4118109263 shutdown_connections 2024-02-16T22:18:57.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.648+0000 7fe1c04eb700 1 --2- 172.21.15.114:0/4118109263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1b8100cb0 0x7fe1b8101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:57.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.648+0000 7fe1c04eb700 1 --2- 172.21.15.114:0/4118109263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1b80ffab0 0x7fe1b80ffed0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:57.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.648+0000 7fe1c04eb700 1 -- 172.21.15.114:0/4118109263 >> 172.21.15.114:0/4118109263 conn(0x7fe1b80fb030 msgr2=0x7fe1b80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:57.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.648+0000 7fe1c04eb700 1 -- 172.21.15.114:0/4118109263 shutdown_connections 2024-02-16T22:18:57.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.648+0000 7fe1c04eb700 1 -- 172.21.15.114:0/4118109263 wait complete. 2024-02-16T22:18:57.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.649+0000 7fe1c04eb700 1 Processor -- start 2024-02-16T22:18:57.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.649+0000 7fe1c04eb700 1 -- start start 2024-02-16T22:18:57.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.649+0000 7fe1c04eb700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1b80ffab0 0x7fe1b81952e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:57.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.649+0000 7fe1c04eb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1b8100cb0 0x7fe1b8195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:57.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.649+0000 7fe1c04eb700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b8195e40 con 0x7fe1b80ffab0 2024-02-16T22:18:57.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.649+0000 7fe1c04eb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b8195f80 con 0x7fe1b8100cb0 2024-02-16T22:18:57.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.649+0000 7fe1be287700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1b80ffab0 0x7fe1b81952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:57.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.649+0000 7fe1be287700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1b80ffab0 0x7fe1b81952e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53574/0 (socket says 172.21.15.114:53574) 2024-02-16T22:18:57.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.649+0000 7fe1be287700 1 -- 172.21.15.114:0/3327600226 learned_addr learned my addr 172.21.15.114:0/3327600226 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:18:57.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.649+0000 7fe1bda86700 1 --2- 172.21.15.114:0/3327600226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1b8100cb0 0x7fe1b8195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:57.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.649+0000 7fe1be287700 1 -- 172.21.15.114:0/3327600226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1b8100cb0 msgr2=0x7fe1b8195820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:57.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.649+0000 7fe1be287700 1 --2- 172.21.15.114:0/3327600226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1b8100cb0 0x7fe1b8195820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:57.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.650+0000 7fe1be287700 1 -- 172.21.15.114:0/3327600226 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b40095f0 con 0x7fe1b80ffab0 2024-02-16T22:18:57.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.650+0000 7fe1be287700 1 --2- 172.21.15.114:0/3327600226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1b80ffab0 0x7fe1b81952e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe1b400bda0 tx=0x7fe1b400bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:57.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.650+0000 7fe1af7fe700 1 -- 172.21.15.114:0/3327600226 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1b401f070 con 0x7fe1b80ffab0 2024-02-16T22:18:57.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.650+0000 7fe1af7fe700 1 -- 172.21.15.114:0/3327600226 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe1b4019610 con 0x7fe1b80ffab0 2024-02-16T22:18:57.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.650+0000 7fe1af7fe700 1 -- 172.21.15.114:0/3327600226 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1b40246e0 con 0x7fe1b80ffab0 2024-02-16T22:18:57.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.650+0000 7fe1c04eb700 1 -- 172.21.15.114:0/3327600226 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b8068410 con 0x7fe1b80ffab0 2024-02-16T22:18:57.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.650+0000 7fe1c04eb700 1 -- 172.21.15.114:0/3327600226 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b8068980 con 0x7fe1b80ffab0 2024-02-16T22:18:57.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.651+0000 7fe1c04eb700 1 -- 172.21.15.114:0/3327600226 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b8108690 con 0x7fe1b80ffab0 2024-02-16T22:18:57.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.651+0000 7fe1af7fe700 1 -- 172.21.15.114:0/3327600226 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fe1b4019780 con 0x7fe1b80ffab0 2024-02-16T22:18:57.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.652+0000 7fe1af7fe700 1 --2- 172.21.15.114:0/3327600226 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fe1a406c4e0 0x7fe1a406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:57.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.652+0000 7fe1af7fe700 1 -- 172.21.15.114:0/3327600226 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fe1b408f100 con 0x7fe1b80ffab0 2024-02-16T22:18:57.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.652+0000 7fe1bda86700 1 --2- 172.21.15.114:0/3327600226 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fe1a406c4e0 0x7fe1a406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:57.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.652+0000 7fe1bda86700 1 --2- 172.21.15.114:0/3327600226 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fe1a406c4e0 0x7fe1a406e9a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8009a10 tx=0x7fe1a8009190 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:57.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.656+0000 7fe1af7fe700 1 -- 172.21.15.114:0/3327600226 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe1b405d510 con 0x7fe1b80ffab0 2024-02-16T22:18:57.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:57 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:18:57.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.906+0000 7fe1c04eb700 1 -- 172.21.15.114:0/3327600226 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1b8105600 con 0x7fe1a406c4e0 2024-02-16T22:18:57.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.907+0000 7fe1af7fe700 1 -- 172.21.15.114:0/3327600226 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fe1b8105600 con 0x7fe1a406c4e0 2024-02-16T22:18:57.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.910+0000 7fe1c04eb700 1 -- 172.21.15.114:0/3327600226 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fe1a406c4e0 msgr2=0x7fe1a406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:57.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.910+0000 7fe1c04eb700 1 --2- 172.21.15.114:0/3327600226 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fe1a406c4e0 0x7fe1a406e9a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8009a10 tx=0x7fe1a8009190 comp rx=0 tx=0).stop 2024-02-16T22:18:57.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.910+0000 7fe1c04eb700 1 -- 172.21.15.114:0/3327600226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1b80ffab0 msgr2=0x7fe1b81952e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:57.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.910+0000 7fe1c04eb700 1 --2- 172.21.15.114:0/3327600226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1b80ffab0 0x7fe1b81952e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe1b400bda0 tx=0x7fe1b400bdd0 comp rx=0 tx=0).stop 2024-02-16T22:18:57.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.910+0000 7fe1c04eb700 1 -- 172.21.15.114:0/3327600226 shutdown_connections 2024-02-16T22:18:57.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.910+0000 7fe1c04eb700 1 --2- 172.21.15.114:0/3327600226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1b80ffab0 0x7fe1b81952e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:57.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.910+0000 7fe1c04eb700 1 --2- 172.21.15.114:0/3327600226 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fe1a406c4e0 0x7fe1a406e9a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:57.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.910+0000 7fe1c04eb700 1 --2- 172.21.15.114:0/3327600226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1b8100cb0 0x7fe1b8195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:57.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.910+0000 7fe1c04eb700 1 -- 172.21.15.114:0/3327600226 >> 172.21.15.114:0/3327600226 conn(0x7fe1b80fb030 msgr2=0x7fe1b8103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:57.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.910+0000 7fe1c04eb700 1 -- 172.21.15.114:0/3327600226 shutdown_connections 2024-02-16T22:18:57.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:57.910+0000 7fe1c04eb700 1 -- 172.21.15.114:0/3327600226 wait complete. 2024-02-16T22:18:57.926 INFO:teuthology.orchestra.run.smithi114.stdout:true 2024-02-16T22:18:58.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.021+0000 7f6e71967700 1 -- 172.21.15.114:0/1087358981 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e6c104020 msgr2=0x7f6e6c106410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:58.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.021+0000 7f6e71967700 1 --2- 172.21.15.114:0/1087358981 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e6c104020 0x7f6e6c106410 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009910 tx=0x7f6e60009c20 comp rx=0 tx=0).stop 2024-02-16T22:18:58.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.021+0000 7f6e71967700 1 -- 172.21.15.114:0/1087358981 shutdown_connections 2024-02-16T22:18:58.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.021+0000 7f6e71967700 1 --2- 172.21.15.114:0/1087358981 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e6c104020 0x7f6e6c106410 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:58.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.021+0000 7f6e71967700 1 --2- 172.21.15.114:0/1087358981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e6c1016f0 0x7f6e6c103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:58.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.021+0000 7f6e71967700 1 -- 172.21.15.114:0/1087358981 >> 172.21.15.114:0/1087358981 conn(0x7f6e6c0fb080 msgr2=0x7f6e6c0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:58.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.021+0000 7f6e71967700 1 -- 172.21.15.114:0/1087358981 shutdown_connections 2024-02-16T22:18:58.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.022+0000 7f6e71967700 1 -- 172.21.15.114:0/1087358981 wait complete. 2024-02-16T22:18:58.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.022+0000 7f6e71967700 1 Processor -- start 2024-02-16T22:18:58.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.022+0000 7f6e71967700 1 -- start start 2024-02-16T22:18:58.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.022+0000 7f6e71967700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e6c1016f0 0x7f6e6c101050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:58.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.023+0000 7f6e71967700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e6c104020 0x7f6e6c0ff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:58.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.023+0000 7f6e71967700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c0ffbe0 con 0x7f6e6c104020 2024-02-16T22:18:58.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.023+0000 7f6e71967700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c0ffd20 con 0x7f6e6c1016f0 2024-02-16T22:18:58.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.023+0000 7f6e6a7fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e6c104020 0x7f6e6c0ff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:58.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.023+0000 7f6e6a7fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e6c104020 0x7f6e6c0ff6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53594/0 (socket says 172.21.15.114:53594) 2024-02-16T22:18:58.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.023+0000 7f6e6a7fc700 1 -- 172.21.15.114:0/2962926389 learned_addr learned my addr 172.21.15.114:0/2962926389 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:18:58.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.023+0000 7f6e6affd700 1 --2- 172.21.15.114:0/2962926389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e6c1016f0 0x7f6e6c101050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:58.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.023+0000 7f6e6a7fc700 1 -- 172.21.15.114:0/2962926389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e6c1016f0 msgr2=0x7f6e6c101050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:58.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.023+0000 7f6e6a7fc700 1 --2- 172.21.15.114:0/2962926389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e6c1016f0 0x7f6e6c101050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:58.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.023+0000 7f6e6a7fc700 1 -- 172.21.15.114:0/2962926389 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e600095f0 con 0x7f6e6c104020 2024-02-16T22:18:58.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.023+0000 7f6e6affd700 1 --2- 172.21.15.114:0/2962926389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e6c1016f0 0x7f6e6c101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:18:58.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.023+0000 7f6e6a7fc700 1 --2- 172.21.15.114:0/2962926389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e6c104020 0x7f6e6c0ff6a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6e60006010 tx=0x7f6e6000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:58.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.023+0000 7f6e70965700 1 -- 172.21.15.114:0/2962926389 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e6001d070 con 0x7f6e6c104020 2024-02-16T22:18:58.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.024+0000 7f6e70965700 1 -- 172.21.15.114:0/2962926389 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6e6000f460 con 0x7f6e6c104020 2024-02-16T22:18:58.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.024+0000 7f6e70965700 1 -- 172.21.15.114:0/2962926389 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e60021640 con 0x7f6e6c104020 2024-02-16T22:18:58.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.024+0000 7f6e71967700 1 -- 172.21.15.114:0/2962926389 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e6c0fffa0 con 0x7f6e6c104020 2024-02-16T22:18:58.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.024+0000 7f6e71967700 1 -- 172.21.15.114:0/2962926389 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e6c1003d0 con 0x7f6e6c104020 2024-02-16T22:18:58.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.025+0000 7f6e71967700 1 -- 172.21.15.114:0/2962926389 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e6c190550 con 0x7f6e6c104020 2024-02-16T22:18:58.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.025+0000 7f6e70965700 1 -- 172.21.15.114:0/2962926389 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f6e6000fab0 con 0x7f6e6c104020 2024-02-16T22:18:58.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.026+0000 7f6e70965700 1 --2- 172.21.15.114:0/2962926389 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f6e5406c4e0 0x7f6e5406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:58.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.026+0000 7f6e70965700 1 -- 172.21.15.114:0/2962926389 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f6e6008ca40 con 0x7f6e6c104020 2024-02-16T22:18:58.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.026+0000 7f6e6affd700 1 --2- 172.21.15.114:0/2962926389 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f6e5406c4e0 0x7f6e5406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:58.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.026+0000 7f6e6affd700 1 --2- 172.21.15.114:0/2962926389 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f6e5406c4e0 0x7f6e5406e9a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c005950 tx=0x7f6e5c00b410 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:58.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.030+0000 7f6e70965700 1 -- 172.21.15.114:0/2962926389 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6e6005ad30 con 0x7f6e6c104020 2024-02-16T22:18:58.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.262+0000 7f6e71967700 1 -- 172.21.15.114:0/2962926389 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e6c0611d0 con 0x7f6e5406c4e0 2024-02-16T22:18:58.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.263+0000 7f6e70965700 1 -- 172.21.15.114:0/2962926389 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f6e6c0611d0 con 0x7f6e5406c4e0 2024-02-16T22:18:58.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.265+0000 7f6e71967700 1 -- 172.21.15.114:0/2962926389 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f6e5406c4e0 msgr2=0x7f6e5406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:58.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.265+0000 7f6e71967700 1 --2- 172.21.15.114:0/2962926389 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f6e5406c4e0 0x7f6e5406e9a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c005950 tx=0x7f6e5c00b410 comp rx=0 tx=0).stop 2024-02-16T22:18:58.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.265+0000 7f6e71967700 1 -- 172.21.15.114:0/2962926389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e6c104020 msgr2=0x7f6e6c0ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:58.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.265+0000 7f6e71967700 1 --2- 172.21.15.114:0/2962926389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e6c104020 0x7f6e6c0ff6a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6e60006010 tx=0x7f6e6000bd70 comp rx=0 tx=0).stop 2024-02-16T22:18:58.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.265+0000 7f6e71967700 1 -- 172.21.15.114:0/2962926389 shutdown_connections 2024-02-16T22:18:58.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.265+0000 7f6e71967700 1 --2- 172.21.15.114:0/2962926389 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f6e5406c4e0 0x7f6e5406e9a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:58.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.265+0000 7f6e71967700 1 --2- 172.21.15.114:0/2962926389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e6c1016f0 0x7f6e6c101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:58.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.265+0000 7f6e71967700 1 --2- 172.21.15.114:0/2962926389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e6c104020 0x7f6e6c0ff6a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:58.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.265+0000 7f6e71967700 1 -- 172.21.15.114:0/2962926389 >> 172.21.15.114:0/2962926389 conn(0x7f6e6c0fb080 msgr2=0x7f6e6c0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:58.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.266+0000 7f6e71967700 1 -- 172.21.15.114:0/2962926389 shutdown_connections 2024-02-16T22:18:58.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.266+0000 7f6e71967700 1 -- 172.21.15.114:0/2962926389 wait complete. 2024-02-16T22:18:58.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.375+0000 7f2209ddb700 1 -- 172.21.15.114:0/678321263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2204100cb0 msgr2=0x7f2204101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:58.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.375+0000 7f2209ddb700 1 --2- 172.21.15.114:0/678321263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2204100cb0 0x7f2204101110 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f21f8009910 tx=0x7f21f8009c20 comp rx=0 tx=0).stop 2024-02-16T22:18:58.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.375+0000 7f2209ddb700 1 -- 172.21.15.114:0/678321263 shutdown_connections 2024-02-16T22:18:58.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.375+0000 7f2209ddb700 1 --2- 172.21.15.114:0/678321263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2204100cb0 0x7f2204101110 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:58.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.375+0000 7f2209ddb700 1 --2- 172.21.15.114:0/678321263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22040ffab0 0x7f22040ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:58.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.375+0000 7f2209ddb700 1 -- 172.21.15.114:0/678321263 >> 172.21.15.114:0/678321263 conn(0x7f22040fb030 msgr2=0x7f22040fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:58.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.375+0000 7f2209ddb700 1 -- 172.21.15.114:0/678321263 shutdown_connections 2024-02-16T22:18:58.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.375+0000 7f2209ddb700 1 -- 172.21.15.114:0/678321263 wait complete. 2024-02-16T22:18:58.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.375+0000 7f2209ddb700 1 Processor -- start 2024-02-16T22:18:58.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.376+0000 7f2209ddb700 1 -- start start 2024-02-16T22:18:58.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.376+0000 7f2209ddb700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22040ffab0 0x7f2204195260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:58.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.376+0000 7f2209ddb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2204100cb0 0x7f22041957a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:58.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.376+0000 7f2209ddb700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2204195dc0 con 0x7f22040ffab0 2024-02-16T22:18:58.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.376+0000 7f2209ddb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2204195f00 con 0x7f2204100cb0 2024-02-16T22:18:58.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.376+0000 7f22037fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22040ffab0 0x7f2204195260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:58.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.376+0000 7f22037fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22040ffab0 0x7f2204195260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53602/0 (socket says 172.21.15.114:53602) 2024-02-16T22:18:58.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.376+0000 7f22037fe700 1 -- 172.21.15.114:0/3374852891 learned_addr learned my addr 172.21.15.114:0/3374852891 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:18:58.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.376+0000 7f2202ffd700 1 --2- 172.21.15.114:0/3374852891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2204100cb0 0x7f22041957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:58.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.377+0000 7f22037fe700 1 -- 172.21.15.114:0/3374852891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2204100cb0 msgr2=0x7f22041957a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:58.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.377+0000 7f22037fe700 1 --2- 172.21.15.114:0/3374852891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2204100cb0 0x7f22041957a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:58.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.377+0000 7f22037fe700 1 -- 172.21.15.114:0/3374852891 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f80095f0 con 0x7f22040ffab0 2024-02-16T22:18:58.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.377+0000 7f22037fe700 1 --2- 172.21.15.114:0/3374852891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22040ffab0 0x7f2204195260 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f21f400d900 tx=0x7f21f400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:58.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.377+0000 7f2200ff9700 1 -- 172.21.15.114:0/3374852891 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21f400b920 con 0x7f22040ffab0 2024-02-16T22:18:58.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.377+0000 7f2200ff9700 1 -- 172.21.15.114:0/3374852891 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f21f4010460 con 0x7f22040ffab0 2024-02-16T22:18:58.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.377+0000 7f2209ddb700 1 -- 172.21.15.114:0/3374852891 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2204068470 con 0x7f22040ffab0 2024-02-16T22:18:58.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.377+0000 7f2200ff9700 1 -- 172.21.15.114:0/3374852891 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21f400f590 con 0x7f22040ffab0 2024-02-16T22:18:58.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.378+0000 7f2209ddb700 1 -- 172.21.15.114:0/3374852891 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22040689c0 con 0x7f22040ffab0 2024-02-16T22:18:58.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.378+0000 7f2209ddb700 1 -- 172.21.15.114:0/3374852891 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2204108610 con 0x7f22040ffab0 2024-02-16T22:18:58.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.379+0000 7f2200ff9700 1 -- 172.21.15.114:0/3374852891 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f21f40105d0 con 0x7f22040ffab0 2024-02-16T22:18:58.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.380+0000 7f2200ff9700 1 --2- 172.21.15.114:0/3374852891 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f21ec06c4e0 0x7f21ec06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:58.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.380+0000 7f2200ff9700 1 -- 172.21.15.114:0/3374852891 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f21f408b4a0 con 0x7f22040ffab0 2024-02-16T22:18:58.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.380+0000 7f2202ffd700 1 --2- 172.21.15.114:0/3374852891 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f21ec06c4e0 0x7f21ec06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:58.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.380+0000 7f2202ffd700 1 --2- 172.21.15.114:0/3374852891 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f21ec06c4e0 0x7f21ec06e9a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f21f800b5c0 tx=0x7f21f8005ee0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:58.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.385+0000 7f2200ff9700 1 -- 172.21.15.114:0/3374852891 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f21f4059710 con 0x7f22040ffab0 2024-02-16T22:18:58.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.597+0000 7f2209ddb700 1 -- 172.21.15.114:0/3374852891 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2204105600 con 0x7f21ec06c4e0 2024-02-16T22:18:58.599 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:58 smithi114 ceph-mon[29235]: pgmap v206: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-16T22:18:58.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.609+0000 7f2200ff9700 1 -- 172.21.15.114:0/3374852891 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3480 (secure 0 0 0) 0x7f2204105600 con 0x7f21ec06c4e0 2024-02-16T22:18:58.611 INFO:teuthology.orchestra.run.smithi114.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:18:58.611 INFO:teuthology.orchestra.run.smithi114.stdout:alertmanager.smithi114 smithi114 *:9093,9094 running (4m) 27s ago 6m 24.4M - 0.25.0 c8568f914cd2 e7404fea400e 2024-02-16T22:18:58.611 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi114 smithi114 running (6m) 27s ago 6m 7440k - 18.2.1 94eadd990e0a 7fd07162cd96 2024-02-16T22:18:58.612 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi123 smithi123 running (5m) 32s ago 5m 7168k - 18.2.1 94eadd990e0a 7687f5e99621 2024-02-16T22:18:58.612 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi114 smithi114 running (6m) 27s ago 6m 6660k - 18.2.1 94eadd990e0a fc8fbfe4fdd5 2024-02-16T22:18:58.612 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi123 smithi123 running (5m) 32s ago 5m 6665k - 18.2.1 94eadd990e0a a6249642b9bb 2024-02-16T22:18:58.612 INFO:teuthology.orchestra.run.smithi114.stdout:grafana.smithi114 smithi114 *:3000 running (4m) 27s ago 6m 88.1M - 9.4.7 954c08fa6188 db1c26c3bc20 2024-02-16T22:18:58.612 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.fdxpey smithi114 running (47s) 27s ago 47s 13.2M - 18.2.1 94eadd990e0a 704b40c144fb 2024-02-16T22:18:58.612 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.zzryke smithi114 running (53s) 27s ago 52s 16.4M - 18.2.1 94eadd990e0a 1f795d23c9a4 2024-02-16T22:18:58.612 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.gopcfa smithi123 running (45s) 32s ago 44s 13.2M - 18.2.1 94eadd990e0a 166134b7532a 2024-02-16T22:18:58.612 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.vhlqbe smithi123 running (50s) 32s ago 50s 14.7M - 18.2.1 94eadd990e0a 243c8bb96fa8 2024-02-16T22:18:58.612 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi114.vcullz smithi114 *:9283,8765,8443 running (7m) 27s ago 7m 488M - 18.2.1 94eadd990e0a c8f21783728d 2024-02-16T22:18:58.612 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi123.inzgjd smithi123 *:8443,9283,8765 running (5m) 32s ago 5m 443M - 18.2.1 94eadd990e0a 7f193d913aba 2024-02-16T22:18:58.612 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi114 smithi114 running (7m) 27s ago 7m 49.9M 2048M 18.2.1 94eadd990e0a b4c4433aa3a4 2024-02-16T22:18:58.612 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi123 smithi123 running (5m) 32s ago 5m 34.9M 2048M 18.2.1 94eadd990e0a c217fc30c152 2024-02-16T22:18:58.613 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi114 smithi114 *:9100 running (6m) 27s ago 6m 15.6M - 1.5.0 0da6a335fe13 744a090e06d0 2024-02-16T22:18:58.613 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi123 smithi123 *:9100 running (5m) 32s ago 5m 15.7M - 1.5.0 0da6a335fe13 3994b8c29b2e 2024-02-16T22:18:58.613 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 smithi114 running (4m) 27s ago 4m 47.7M 3946M 18.2.1 94eadd990e0a 24ca2e9714e0 2024-02-16T22:18:58.613 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 smithi114 running (4m) 27s ago 4m 47.8M 3946M 18.2.1 94eadd990e0a 32bb191b88e7 2024-02-16T22:18:58.613 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 smithi114 running (3m) 27s ago 3m 48.1M 3946M 18.2.1 94eadd990e0a 6608718d0818 2024-02-16T22:18:58.613 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 smithi123 running (3m) 32s ago 3m 48.8M 4970M 18.2.1 94eadd990e0a 575db379de3d 2024-02-16T22:18:58.613 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 smithi123 running (2m) 32s ago 2m 43.3M 4970M 18.2.1 94eadd990e0a 7e266866cd9c 2024-02-16T22:18:58.613 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 smithi123 running (2m) 32s ago 2m 42.1M 4970M 18.2.1 94eadd990e0a 24e5bef19fda 2024-02-16T22:18:58.613 INFO:teuthology.orchestra.run.smithi114.stdout:prometheus.smithi114 smithi114 *:9095 running (4m) 27s ago 5m 55.9M - 2.43.0 a07b618ecd1d 51185086d582 2024-02-16T22:18:58.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.612+0000 7f2209ddb700 1 -- 172.21.15.114:0/3374852891 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f21ec06c4e0 msgr2=0x7f21ec06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:58.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.612+0000 7f2209ddb700 1 --2- 172.21.15.114:0/3374852891 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f21ec06c4e0 0x7f21ec06e9a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f21f800b5c0 tx=0x7f21f8005ee0 comp rx=0 tx=0).stop 2024-02-16T22:18:58.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.612+0000 7f2209ddb700 1 -- 172.21.15.114:0/3374852891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22040ffab0 msgr2=0x7f2204195260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:58.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.612+0000 7f2209ddb700 1 --2- 172.21.15.114:0/3374852891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22040ffab0 0x7f2204195260 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f21f400d900 tx=0x7f21f400dc10 comp rx=0 tx=0).stop 2024-02-16T22:18:58.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.612+0000 7f2209ddb700 1 -- 172.21.15.114:0/3374852891 shutdown_connections 2024-02-16T22:18:58.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.612+0000 7f2209ddb700 1 --2- 172.21.15.114:0/3374852891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22040ffab0 0x7f2204195260 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:58.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.612+0000 7f2209ddb700 1 --2- 172.21.15.114:0/3374852891 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f21ec06c4e0 0x7f21ec06e9a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:58.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.612+0000 7f2209ddb700 1 --2- 172.21.15.114:0/3374852891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2204100cb0 0x7f22041957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:58.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.612+0000 7f2209ddb700 1 -- 172.21.15.114:0/3374852891 >> 172.21.15.114:0/3374852891 conn(0x7f22040fb030 msgr2=0x7f2204103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:58.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.612+0000 7f2209ddb700 1 -- 172.21.15.114:0/3374852891 shutdown_connections 2024-02-16T22:18:58.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.612+0000 7f2209ddb700 1 -- 172.21.15.114:0/3374852891 wait complete. 2024-02-16T22:18:58.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.724+0000 7fec21008700 1 -- 172.21.15.114:0/99840076 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec1c101640 msgr2=0x7fec1c103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:58.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.724+0000 7fec21008700 1 --2- 172.21.15.114:0/99840076 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec1c101640 0x7fec1c103a30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fec0c009910 tx=0x7fec0c009c20 comp rx=0 tx=0).stop 2024-02-16T22:18:58.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.724+0000 7fec21008700 1 -- 172.21.15.114:0/99840076 shutdown_connections 2024-02-16T22:18:58.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.724+0000 7fec21008700 1 --2- 172.21.15.114:0/99840076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec1c103f70 0x7fec1c106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:58.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.724+0000 7fec21008700 1 --2- 172.21.15.114:0/99840076 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec1c101640 0x7fec1c103a30 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:58.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.724+0000 7fec21008700 1 -- 172.21.15.114:0/99840076 >> 172.21.15.114:0/99840076 conn(0x7fec1c0fb030 msgr2=0x7fec1c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:58.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.724+0000 7fec21008700 1 -- 172.21.15.114:0/99840076 shutdown_connections 2024-02-16T22:18:58.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.724+0000 7fec21008700 1 -- 172.21.15.114:0/99840076 wait complete. 2024-02-16T22:18:58.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.725+0000 7fec21008700 1 Processor -- start 2024-02-16T22:18:58.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.725+0000 7fec21008700 1 -- start start 2024-02-16T22:18:58.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.725+0000 7fec21008700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec1c101640 0x7fec1c196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:58.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.725+0000 7fec21008700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec1c103f70 0x7fec1c196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:58.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.725+0000 7fec21008700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c196e60 con 0x7fec1c101640 2024-02-16T22:18:58.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.725+0000 7fec21008700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c196fa0 con 0x7fec1c103f70 2024-02-16T22:18:58.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.725+0000 7fec1ad9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec1c101640 0x7fec1c196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:58.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.725+0000 7fec1ad9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec1c101640 0x7fec1c196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53634/0 (socket says 172.21.15.114:53634) 2024-02-16T22:18:58.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.725+0000 7fec1ad9d700 1 -- 172.21.15.114:0/2706526623 learned_addr learned my addr 172.21.15.114:0/2706526623 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:18:58.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.725+0000 7fec1a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec1c103f70 0x7fec1c196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:58.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.726+0000 7fec1ad9d700 1 -- 172.21.15.114:0/2706526623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec1c103f70 msgr2=0x7fec1c196840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:58.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.726+0000 7fec1ad9d700 1 --2- 172.21.15.114:0/2706526623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec1c103f70 0x7fec1c196840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:58.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.726+0000 7fec1ad9d700 1 -- 172.21.15.114:0/2706526623 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec0c0095f0 con 0x7fec1c101640 2024-02-16T22:18:58.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.726+0000 7fec1a59c700 1 --2- 172.21.15.114:0/2706526623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec1c103f70 0x7fec1c196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:18:58.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.726+0000 7fec1ad9d700 1 --2- 172.21.15.114:0/2706526623 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec1c101640 0x7fec1c196300 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fec0c006010 tx=0x7fec0c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:58.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.726+0000 7fec03fff700 1 -- 172.21.15.114:0/2706526623 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec0c01d070 con 0x7fec1c101640 2024-02-16T22:18:58.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.726+0000 7fec21008700 1 -- 172.21.15.114:0/2706526623 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec1c19b9f0 con 0x7fec1c101640 2024-02-16T22:18:58.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.726+0000 7fec03fff700 1 -- 172.21.15.114:0/2706526623 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fec0c00f460 con 0x7fec1c101640 2024-02-16T22:18:58.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.726+0000 7fec03fff700 1 -- 172.21.15.114:0/2706526623 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec0c021640 con 0x7fec1c101640 2024-02-16T22:18:58.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.726+0000 7fec21008700 1 -- 172.21.15.114:0/2706526623 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec1c19bee0 con 0x7fec1c101640 2024-02-16T22:18:58.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.727+0000 7fec21008700 1 -- 172.21.15.114:0/2706526623 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec1c190520 con 0x7fec1c101640 2024-02-16T22:18:58.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.728+0000 7fec03fff700 1 -- 172.21.15.114:0/2706526623 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fec0c00fab0 con 0x7fec1c101640 2024-02-16T22:18:58.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.728+0000 7fec03fff700 1 --2- 172.21.15.114:0/2706526623 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fec0406c4e0 0x7fec0406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:58.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.728+0000 7fec03fff700 1 -- 172.21.15.114:0/2706526623 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fec0c08ca70 con 0x7fec1c101640 2024-02-16T22:18:58.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.728+0000 7fec1a59c700 1 --2- 172.21.15.114:0/2706526623 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fec0406c4e0 0x7fec0406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:58.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.729+0000 7fec1a59c700 1 --2- 172.21.15.114:0/2706526623 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fec0406c4e0 0x7fec0406e9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fec10010420 tx=0x7fec100073d0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:58.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:58.733+0000 7fec03fff700 1 -- 172.21.15.114:0/2706526623 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fec0c05ace0 con 0x7fec1c101640 2024-02-16T22:18:58.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:58 smithi123 ceph-mon[37161]: pgmap v206: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-16T22:18:59.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.044+0000 7fec21008700 1 -- 172.21.15.114:0/2706526623 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fec1c061a40 con 0x7fec1c101640 2024-02-16T22:18:59.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.044+0000 7fec03fff700 1 -- 172.21.15.114:0/2706526623 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7fec0c05a870 con 0x7fec1c101640 2024-02-16T22:18:59.046 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:18:59.047 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": { 2024-02-16T22:18:59.047 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-02-16T22:18:59.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:18:59.047 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": { 2024-02-16T22:18:59.047 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-02-16T22:18:59.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:18:59.047 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": { 2024-02-16T22:18:59.047 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-02-16T22:18:59.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:18:59.047 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-02-16T22:18:59.047 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-02-16T22:18:59.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:18:59.048 INFO:teuthology.orchestra.run.smithi114.stdout: "overall": { 2024-02-16T22:18:59.048 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 14 2024-02-16T22:18:59.048 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-02-16T22:18:59.048 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:18:59.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.047+0000 7fec21008700 1 -- 172.21.15.114:0/2706526623 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fec0406c4e0 msgr2=0x7fec0406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:59.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.047+0000 7fec21008700 1 --2- 172.21.15.114:0/2706526623 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fec0406c4e0 0x7fec0406e9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fec10010420 tx=0x7fec100073d0 comp rx=0 tx=0).stop 2024-02-16T22:18:59.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.047+0000 7fec21008700 1 -- 172.21.15.114:0/2706526623 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec1c101640 msgr2=0x7fec1c196300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:59.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.047+0000 7fec21008700 1 --2- 172.21.15.114:0/2706526623 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec1c101640 0x7fec1c196300 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fec0c006010 tx=0x7fec0c00bd70 comp rx=0 tx=0).stop 2024-02-16T22:18:59.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.047+0000 7fec21008700 1 -- 172.21.15.114:0/2706526623 shutdown_connections 2024-02-16T22:18:59.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.047+0000 7fec21008700 1 --2- 172.21.15.114:0/2706526623 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec1c101640 0x7fec1c196300 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:59.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.047+0000 7fec21008700 1 --2- 172.21.15.114:0/2706526623 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fec0406c4e0 0x7fec0406e9a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:59.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.047+0000 7fec21008700 1 --2- 172.21.15.114:0/2706526623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec1c103f70 0x7fec1c196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:59.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.047+0000 7fec21008700 1 -- 172.21.15.114:0/2706526623 >> 172.21.15.114:0/2706526623 conn(0x7fec1c0fb030 msgr2=0x7fec1c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:59.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.047+0000 7fec21008700 1 -- 172.21.15.114:0/2706526623 shutdown_connections 2024-02-16T22:18:59.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.047+0000 7fec21008700 1 -- 172.21.15.114:0/2706526623 wait complete. 2024-02-16T22:18:59.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.155+0000 7efcf24de700 1 -- 172.21.15.114:0/999117236 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcec101140 msgr2=0x7efcec101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:59.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.155+0000 7efcf24de700 1 --2- 172.21.15.114:0/999117236 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcec101140 0x7efcec101560 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efcdc009910 tx=0x7efcdc009c20 comp rx=0 tx=0).stop 2024-02-16T22:18:59.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.155+0000 7efcf24de700 1 -- 172.21.15.114:0/999117236 shutdown_connections 2024-02-16T22:18:59.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.155+0000 7efcf24de700 1 --2- 172.21.15.114:0/999117236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcec0fec40 0x7efcec0ff0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:59.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.155+0000 7efcf24de700 1 --2- 172.21.15.114:0/999117236 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcec101140 0x7efcec101560 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:59.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.155+0000 7efcf24de700 1 -- 172.21.15.114:0/999117236 >> 172.21.15.114:0/999117236 conn(0x7efcec0fa7c0 msgr2=0x7efcec0fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:59.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.156+0000 7efcf24de700 1 -- 172.21.15.114:0/999117236 shutdown_connections 2024-02-16T22:18:59.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.156+0000 7efcf24de700 1 -- 172.21.15.114:0/999117236 wait complete. 2024-02-16T22:18:59.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.156+0000 7efcf24de700 1 Processor -- start 2024-02-16T22:18:59.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.156+0000 7efcf24de700 1 -- start start 2024-02-16T22:18:59.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.156+0000 7efcf24de700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcec0fec40 0x7efcec198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:59.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.157+0000 7efcf24de700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcec101140 0x7efcec198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:59.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.157+0000 7efcf24de700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcec1990e0 con 0x7efcec101140 2024-02-16T22:18:59.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.157+0000 7efcf24de700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcec199220 con 0x7efcec0fec40 2024-02-16T22:18:59.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.157+0000 7efceb7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcec101140 0x7efcec198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:59.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.157+0000 7efceb7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcec101140 0x7efcec198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53650/0 (socket says 172.21.15.114:53650) 2024-02-16T22:18:59.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.157+0000 7efceb7fe700 1 -- 172.21.15.114:0/3173978428 learned_addr learned my addr 172.21.15.114:0/3173978428 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:18:59.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.157+0000 7efcebfff700 1 --2- 172.21.15.114:0/3173978428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcec0fec40 0x7efcec198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:59.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.157+0000 7efceb7fe700 1 -- 172.21.15.114:0/3173978428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcec0fec40 msgr2=0x7efcec198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:59.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.157+0000 7efceb7fe700 1 --2- 172.21.15.114:0/3173978428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcec0fec40 0x7efcec198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:59.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.157+0000 7efceb7fe700 1 -- 172.21.15.114:0/3173978428 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcdc0095f0 con 0x7efcec101140 2024-02-16T22:18:59.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.157+0000 7efceb7fe700 1 --2- 172.21.15.114:0/3173978428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcec101140 0x7efcec198ac0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7efce000d8d0 tx=0x7efce000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:59.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.157+0000 7efce97fa700 1 -- 172.21.15.114:0/3173978428 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efce00098a0 con 0x7efcec101140 2024-02-16T22:18:59.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.157+0000 7efce97fa700 1 -- 172.21.15.114:0/3173978428 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efce0010460 con 0x7efcec101140 2024-02-16T22:18:59.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.157+0000 7efcf24de700 1 -- 172.21.15.114:0/3173978428 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcec19dcd0 con 0x7efcec101140 2024-02-16T22:18:59.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.157+0000 7efce97fa700 1 -- 172.21.15.114:0/3173978428 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efce000f5f0 con 0x7efcec101140 2024-02-16T22:18:59.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.157+0000 7efcf24de700 1 -- 172.21.15.114:0/3173978428 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcec19e220 con 0x7efcec101140 2024-02-16T22:18:59.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.158+0000 7efcf24de700 1 -- 172.21.15.114:0/3173978428 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcec10b9d0 con 0x7efcec101140 2024-02-16T22:18:59.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.159+0000 7efce97fa700 1 -- 172.21.15.114:0/3173978428 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7efce0009a00 con 0x7efcec101140 2024-02-16T22:18:59.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.159+0000 7efce97fa700 1 --2- 172.21.15.114:0/3173978428 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7efcd406c4e0 0x7efcd406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:59.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.160+0000 7efce97fa700 1 -- 172.21.15.114:0/3173978428 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7efce008b6d0 con 0x7efcec101140 2024-02-16T22:18:59.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.160+0000 7efcebfff700 1 --2- 172.21.15.114:0/3173978428 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7efcd406c4e0 0x7efcd406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:59.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.160+0000 7efcebfff700 1 --2- 172.21.15.114:0/3173978428 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7efcd406c4e0 0x7efcd406e9a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efcdc006010 tx=0x7efcdc00b560 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:59.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.164+0000 7efce97fa700 1 -- 172.21.15.114:0/3173978428 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7efce0059940 con 0x7efcec101140 2024-02-16T22:18:59.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.420+0000 7efcf24de700 1 -- 172.21.15.114:0/3173978428 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7efcec10bbc0 con 0x7efcec101140 2024-02-16T22:18:59.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.421+0000 7efce97fa700 1 -- 172.21.15.114:0/3173978428 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1825 (secure 0 0 0) 0x7efce0016740 con 0x7efcec101140 2024-02-16T22:18:59.423 INFO:teuthology.orchestra.run.smithi114.stdout:e14 2024-02-16T22:18:59.423 INFO:teuthology.orchestra.run.smithi114.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:18:59.423 INFO:teuthology.orchestra.run.smithi114.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:18:59.423 INFO:teuthology.orchestra.run.smithi114.stdout:legacy client fscid: 1 2024-02-16T22:18:59.423 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:18:59.423 INFO:teuthology.orchestra.run.smithi114.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:18:59.423 INFO:teuthology.orchestra.run.smithi114.stdout:fs_name cephfs 2024-02-16T22:18:59.423 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 12 2024-02-16T22:18:59.423 INFO:teuthology.orchestra.run.smithi114.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:18:59.423 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-02-16T22:18:02.108825+0000 2024-02-16T22:18:59.424 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-02-16T22:18:16.413371+0000 2024-02-16T22:18:59.424 INFO:teuthology.orchestra.run.smithi114.stdout:tableserver 0 2024-02-16T22:18:59.424 INFO:teuthology.orchestra.run.smithi114.stdout:root 0 2024-02-16T22:18:59.424 INFO:teuthology.orchestra.run.smithi114.stdout:session_timeout 60 2024-02-16T22:18:59.424 INFO:teuthology.orchestra.run.smithi114.stdout:session_autoclose 300 2024-02-16T22:18:59.424 INFO:teuthology.orchestra.run.smithi114.stdout:max_file_size 1099511627776 2024-02-16T22:18:59.424 INFO:teuthology.orchestra.run.smithi114.stdout:required_client_features {} 2024-02-16T22:18:59.424 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure 0 2024-02-16T22:18:59.424 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure_osd_epoch 0 2024-02-16T22:18:59.424 INFO:teuthology.orchestra.run.smithi114.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:18:59.424 INFO:teuthology.orchestra.run.smithi114.stdout:max_mds 1 2024-02-16T22:18:59.424 INFO:teuthology.orchestra.run.smithi114.stdout:in 0 2024-02-16T22:18:59.424 INFO:teuthology.orchestra.run.smithi114.stdout:up {0=14468} 2024-02-16T22:18:59.424 INFO:teuthology.orchestra.run.smithi114.stdout:failed 2024-02-16T22:18:59.425 INFO:teuthology.orchestra.run.smithi114.stdout:damaged 2024-02-16T22:18:59.425 INFO:teuthology.orchestra.run.smithi114.stdout:stopped 2024-02-16T22:18:59.425 INFO:teuthology.orchestra.run.smithi114.stdout:data_pools [3] 2024-02-16T22:18:59.425 INFO:teuthology.orchestra.run.smithi114.stdout:metadata_pool 2 2024-02-16T22:18:59.425 INFO:teuthology.orchestra.run.smithi114.stdout:inline_data enabled 2024-02-16T22:18:59.425 INFO:teuthology.orchestra.run.smithi114.stdout:balancer 2024-02-16T22:18:59.425 INFO:teuthology.orchestra.run.smithi114.stdout:bal_rank_mask -1 2024-02-16T22:18:59.425 INFO:teuthology.orchestra.run.smithi114.stdout:standby_count_wanted 1 2024-02-16T22:18:59.425 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.zzryke{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.114:6826/277953535,v1:172.21.15.114:6827/277953535] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:18:59.425 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:18:59.425 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:18:59.425 INFO:teuthology.orchestra.run.smithi114.stdout:Standby daemons: 2024-02-16T22:18:59.425 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:18:59.425 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.fdxpey{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:18:59.426 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.vhlqbe{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:18:59.426 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.gopcfa{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:18:59.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.423+0000 7efcf24de700 1 -- 172.21.15.114:0/3173978428 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7efcd406c4e0 msgr2=0x7efcd406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:59.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.423+0000 7efcf24de700 1 --2- 172.21.15.114:0/3173978428 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7efcd406c4e0 0x7efcd406e9a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efcdc006010 tx=0x7efcdc00b560 comp rx=0 tx=0).stop 2024-02-16T22:18:59.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.423+0000 7efcf24de700 1 -- 172.21.15.114:0/3173978428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcec101140 msgr2=0x7efcec198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:59.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.423+0000 7efcf24de700 1 --2- 172.21.15.114:0/3173978428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcec101140 0x7efcec198ac0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7efce000d8d0 tx=0x7efce000dbe0 comp rx=0 tx=0).stop 2024-02-16T22:18:59.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.423+0000 7efcf24de700 1 -- 172.21.15.114:0/3173978428 shutdown_connections 2024-02-16T22:18:59.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.423+0000 7efcf24de700 1 --2- 172.21.15.114:0/3173978428 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7efcd406c4e0 0x7efcd406e9a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:59.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.423+0000 7efcf24de700 1 --2- 172.21.15.114:0/3173978428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcec0fec40 0x7efcec198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:59.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.423+0000 7efcf24de700 1 --2- 172.21.15.114:0/3173978428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcec101140 0x7efcec198ac0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:59.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.423+0000 7efcf24de700 1 -- 172.21.15.114:0/3173978428 >> 172.21.15.114:0/3173978428 conn(0x7efcec0fa7c0 msgr2=0x7efcec108340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:59.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.423+0000 7efcf24de700 1 -- 172.21.15.114:0/3173978428 shutdown_connections 2024-02-16T22:18:59.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.423+0000 7efcf24de700 1 -- 172.21.15.114:0/3173978428 wait complete. 2024-02-16T22:18:59.427 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 14 2024-02-16T22:18:59.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.543+0000 7fc004c9e700 1 -- 172.21.15.114:0/3930986374 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc000103f70 msgr2=0x7fc000106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:59.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.543+0000 7fc004c9e700 1 --2- 172.21.15.114:0/3930986374 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc000103f70 0x7fc000106360 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8009910 tx=0x7fbfe8009c20 comp rx=0 tx=0).stop 2024-02-16T22:18:59.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.544+0000 7fc004c9e700 1 -- 172.21.15.114:0/3930986374 shutdown_connections 2024-02-16T22:18:59.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.544+0000 7fc004c9e700 1 --2- 172.21.15.114:0/3930986374 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc000103f70 0x7fc000106360 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:59.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.544+0000 7fc004c9e700 1 --2- 172.21.15.114:0/3930986374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc000101640 0x7fc000103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:59.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.544+0000 7fc004c9e700 1 -- 172.21.15.114:0/3930986374 >> 172.21.15.114:0/3930986374 conn(0x7fc0000fb030 msgr2=0x7fc0000fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:59.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.544+0000 7fc004c9e700 1 -- 172.21.15.114:0/3930986374 shutdown_connections 2024-02-16T22:18:59.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.544+0000 7fc004c9e700 1 -- 172.21.15.114:0/3930986374 wait complete. 2024-02-16T22:18:59.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.544+0000 7fc004c9e700 1 Processor -- start 2024-02-16T22:18:59.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.545+0000 7fc004c9e700 1 -- start start 2024-02-16T22:18:59.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.545+0000 7fc004c9e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc000101640 0x7fc000100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:59.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.545+0000 7fc004c9e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc000103f70 0x7fc0000ff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:59.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.545+0000 7fc004c9e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0000ffb30 con 0x7fc000103f70 2024-02-16T22:18:59.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.545+0000 7fc004c9e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0000ffc70 con 0x7fc000101640 2024-02-16T22:18:59.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.545+0000 7fbffe59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc000103f70 0x7fc0000ff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:59.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.545+0000 7fbffe59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc000103f70 0x7fc0000ff5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53676/0 (socket says 172.21.15.114:53676) 2024-02-16T22:18:59.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.545+0000 7fbffe59c700 1 -- 172.21.15.114:0/1295558684 learned_addr learned my addr 172.21.15.114:0/1295558684 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:18:59.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.545+0000 7fbff7fff700 1 --2- 172.21.15.114:0/1295558684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc000101640 0x7fc000100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:59.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.545+0000 7fbffe59c700 1 -- 172.21.15.114:0/1295558684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc000101640 msgr2=0x7fc000100fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:59.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.545+0000 7fbffe59c700 1 --2- 172.21.15.114:0/1295558684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc000101640 0x7fc000100fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:59.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.545+0000 7fbffe59c700 1 -- 172.21.15.114:0/1295558684 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe80095f0 con 0x7fc000103f70 2024-02-16T22:18:59.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.546+0000 7fbffe59c700 1 --2- 172.21.15.114:0/1295558684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc000103f70 0x7fc0000ff5f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8005b40 tx=0x7fbfe800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:59.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.546+0000 7fbff77fe700 1 -- 172.21.15.114:0/1295558684 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfe801d070 con 0x7fc000103f70 2024-02-16T22:18:59.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.546+0000 7fbff77fe700 1 -- 172.21.15.114:0/1295558684 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbfe8022470 con 0x7fc000103f70 2024-02-16T22:18:59.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.546+0000 7fc004c9e700 1 -- 172.21.15.114:0/1295558684 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0000ffef0 con 0x7fc000103f70 2024-02-16T22:18:59.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.546+0000 7fbff77fe700 1 -- 172.21.15.114:0/1295558684 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfe800f690 con 0x7fc000103f70 2024-02-16T22:18:59.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.546+0000 7fc004c9e700 1 -- 172.21.15.114:0/1295558684 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0001003e0 con 0x7fc000103f70 2024-02-16T22:18:59.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.547+0000 7fc004c9e700 1 -- 172.21.15.114:0/1295558684 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc000190520 con 0x7fc000103f70 2024-02-16T22:18:59.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.548+0000 7fbff77fe700 1 -- 172.21.15.114:0/1295558684 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fbfe8022a10 con 0x7fc000103f70 2024-02-16T22:18:59.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.548+0000 7fbff77fe700 1 --2- 172.21.15.114:0/1295558684 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fbfec06c4e0 0x7fbfec06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:59.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.548+0000 7fbff77fe700 1 -- 172.21.15.114:0/1295558684 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fbfe808cf30 con 0x7fc000103f70 2024-02-16T22:18:59.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.548+0000 7fbff7fff700 1 --2- 172.21.15.114:0/1295558684 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fbfec06c4e0 0x7fbfec06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:59.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.548+0000 7fbff7fff700 1 --2- 172.21.15.114:0/1295558684 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fbfec06c4e0 0x7fbfec06e9a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbff0005950 tx=0x7fbff000a300 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:59.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.553+0000 7fbff77fe700 1 -- 172.21.15.114:0/1295558684 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbfe805b1b0 con 0x7fc000103f70 2024-02-16T22:18:59.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:59 smithi114 ceph-mon[29235]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:18:59.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:59 smithi114 ceph-mon[29235]: from='client.14572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:18:59.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:59 smithi114 ceph-mon[29235]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:18:59.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:18:59 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/2706526623' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:18:59.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.783+0000 7fc004c9e700 1 -- 172.21.15.114:0/1295558684 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0000611d0 con 0x7fbfec06c4e0 2024-02-16T22:18:59.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.784+0000 7fbff77fe700 1 -- 172.21.15.114:0/1295558684 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fc0000611d0 con 0x7fbfec06c4e0 2024-02-16T22:18:59.786 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:18:59.787 INFO:teuthology.orchestra.run.smithi114.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275", 2024-02-16T22:18:59.787 INFO:teuthology.orchestra.run.smithi114.stdout: "in_progress": true, 2024-02-16T22:18:59.787 INFO:teuthology.orchestra.run.smithi114.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:18:59.787 INFO:teuthology.orchestra.run.smithi114.stdout: "services_complete": [], 2024-02-16T22:18:59.787 INFO:teuthology.orchestra.run.smithi114.stdout: "progress": "", 2024-02-16T22:18:59.787 INFO:teuthology.orchestra.run.smithi114.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 image", 2024-02-16T22:18:59.787 INFO:teuthology.orchestra.run.smithi114.stdout: "is_paused": false 2024-02-16T22:18:59.787 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:18:59.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.787+0000 7fc004c9e700 1 -- 172.21.15.114:0/1295558684 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fbfec06c4e0 msgr2=0x7fbfec06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:59.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.787+0000 7fc004c9e700 1 --2- 172.21.15.114:0/1295558684 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fbfec06c4e0 0x7fbfec06e9a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbff0005950 tx=0x7fbff000a300 comp rx=0 tx=0).stop 2024-02-16T22:18:59.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.787+0000 7fc004c9e700 1 -- 172.21.15.114:0/1295558684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc000103f70 msgr2=0x7fc0000ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:59.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.787+0000 7fc004c9e700 1 --2- 172.21.15.114:0/1295558684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc000103f70 0x7fc0000ff5f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8005b40 tx=0x7fbfe800bd70 comp rx=0 tx=0).stop 2024-02-16T22:18:59.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.787+0000 7fc004c9e700 1 -- 172.21.15.114:0/1295558684 shutdown_connections 2024-02-16T22:18:59.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.787+0000 7fc004c9e700 1 --2- 172.21.15.114:0/1295558684 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fbfec06c4e0 0x7fbfec06e9a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:59.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.787+0000 7fc004c9e700 1 --2- 172.21.15.114:0/1295558684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc000101640 0x7fc000100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:59.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.787+0000 7fc004c9e700 1 --2- 172.21.15.114:0/1295558684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc000103f70 0x7fc0000ff5f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:59.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.787+0000 7fc004c9e700 1 -- 172.21.15.114:0/1295558684 >> 172.21.15.114:0/1295558684 conn(0x7fc0000fb030 msgr2=0x7fc0000fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:59.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.787+0000 7fc004c9e700 1 -- 172.21.15.114:0/1295558684 shutdown_connections 2024-02-16T22:18:59.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.787+0000 7fc004c9e700 1 -- 172.21.15.114:0/1295558684 wait complete. 2024-02-16T22:18:59.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:59 smithi123 ceph-mon[37161]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:18:59.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:59 smithi123 ceph-mon[37161]: from='client.14572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:18:59.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:59 smithi123 ceph-mon[37161]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:18:59.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:18:59 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/2706526623' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:18:59.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.894+0000 7f3367b54700 1 -- 172.21.15.114:0/300327705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3360100cb0 msgr2=0x7f3360101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:59.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.894+0000 7f3367b54700 1 --2- 172.21.15.114:0/300327705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3360100cb0 0x7f3360101110 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3350009910 tx=0x7f3350009c20 comp rx=0 tx=0).stop 2024-02-16T22:18:59.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.894+0000 7f3367b54700 1 -- 172.21.15.114:0/300327705 shutdown_connections 2024-02-16T22:18:59.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.894+0000 7f3367b54700 1 --2- 172.21.15.114:0/300327705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3360100cb0 0x7f3360101110 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:59.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.894+0000 7f3367b54700 1 --2- 172.21.15.114:0/300327705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33600ffab0 0x7f33600ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:59.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.894+0000 7f3367b54700 1 -- 172.21.15.114:0/300327705 >> 172.21.15.114:0/300327705 conn(0x7f33600fb030 msgr2=0x7f33600fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:18:59.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.894+0000 7f3367b54700 1 -- 172.21.15.114:0/300327705 shutdown_connections 2024-02-16T22:18:59.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.894+0000 7f3367b54700 1 -- 172.21.15.114:0/300327705 wait complete. 2024-02-16T22:18:59.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.895+0000 7f3367b54700 1 Processor -- start 2024-02-16T22:18:59.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.895+0000 7f3367b54700 1 -- start start 2024-02-16T22:18:59.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.895+0000 7f3367b54700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33600ffab0 0x7f3360068f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:59.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.895+0000 7f3367b54700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3360100cb0 0x7f336019b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:59.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.895+0000 7f3367b54700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336019b710 con 0x7f3360100cb0 2024-02-16T22:18:59.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.895+0000 7f3367b54700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336019b850 con 0x7f33600ffab0 2024-02-16T22:18:59.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.895+0000 7f33650ef700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3360100cb0 0x7f336019b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:59.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.895+0000 7f33650ef700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3360100cb0 0x7f336019b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53690/0 (socket says 172.21.15.114:53690) 2024-02-16T22:18:59.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.895+0000 7f33650ef700 1 -- 172.21.15.114:0/1686231725 learned_addr learned my addr 172.21.15.114:0/1686231725 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:18:59.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.895+0000 7f33658f0700 1 --2- 172.21.15.114:0/1686231725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33600ffab0 0x7f3360068f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:59.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.895+0000 7f33650ef700 1 -- 172.21.15.114:0/1686231725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33600ffab0 msgr2=0x7f3360068f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:18:59.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.895+0000 7f33650ef700 1 --2- 172.21.15.114:0/1686231725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33600ffab0 0x7f3360068f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:18:59.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.895+0000 7f33650ef700 1 -- 172.21.15.114:0/1686231725 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33500095f0 con 0x7f3360100cb0 2024-02-16T22:18:59.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.896+0000 7f33650ef700 1 --2- 172.21.15.114:0/1686231725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3360100cb0 0x7f336019b140 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3350006010 tx=0x7f335000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:59.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.896+0000 7f3356ffd700 1 -- 172.21.15.114:0/1686231725 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f335001d070 con 0x7f3360100cb0 2024-02-16T22:18:59.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.896+0000 7f3367b54700 1 -- 172.21.15.114:0/1686231725 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f336019bad0 con 0x7f3360100cb0 2024-02-16T22:18:59.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.896+0000 7f3356ffd700 1 -- 172.21.15.114:0/1686231725 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f335000f460 con 0x7f3360100cb0 2024-02-16T22:18:59.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.896+0000 7f3356ffd700 1 -- 172.21.15.114:0/1686231725 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3350021640 con 0x7f3360100cb0 2024-02-16T22:18:59.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.896+0000 7f3367b54700 1 -- 172.21.15.114:0/1686231725 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f336019c040 con 0x7f3360100cb0 2024-02-16T22:18:59.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.897+0000 7f3367b54700 1 -- 172.21.15.114:0/1686231725 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33601086b0 con 0x7f3360100cb0 2024-02-16T22:18:59.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.897+0000 7f3356ffd700 1 -- 172.21.15.114:0/1686231725 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f3350004b20 con 0x7f3360100cb0 2024-02-16T22:18:59.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.898+0000 7f3356ffd700 1 --2- 172.21.15.114:0/1686231725 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f334c06c4e0 0x7f334c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:18:59.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.898+0000 7f33658f0700 1 --2- 172.21.15.114:0/1686231725 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f334c06c4e0 0x7f334c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:18:59.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.898+0000 7f3356ffd700 1 -- 172.21.15.114:0/1686231725 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f335008cbf0 con 0x7f3360100cb0 2024-02-16T22:18:59.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.898+0000 7f33658f0700 1 --2- 172.21.15.114:0/1686231725 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f334c06c4e0 0x7f334c06e9a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f335c006fd0 tx=0x7f335c008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:18:59.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:18:59.903+0000 7f3356ffd700 1 -- 172.21.15.114:0/1686231725 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f335005afa0 con 0x7f3360100cb0 2024-02-16T22:19:00.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:00.220+0000 7f3367b54700 1 -- 172.21.15.114:0/1686231725 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f336019c6d0 con 0x7f3360100cb0 2024-02-16T22:19:00.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:00.221+0000 7f3356ffd700 1 -- 172.21.15.114:0/1686231725 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f33500260e0 con 0x7f3360100cb0 2024-02-16T22:19:00.223 INFO:teuthology.orchestra.run.smithi114.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:19:00.223 INFO:teuthology.orchestra.run.smithi114.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:19:00.223 INFO:teuthology.orchestra.run.smithi114.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:19:00.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:00.223+0000 7f3367b54700 1 -- 172.21.15.114:0/1686231725 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f334c06c4e0 msgr2=0x7f334c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:19:00.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:00.223+0000 7f3367b54700 1 --2- 172.21.15.114:0/1686231725 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f334c06c4e0 0x7f334c06e9a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f335c006fd0 tx=0x7f335c008040 comp rx=0 tx=0).stop 2024-02-16T22:19:00.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:00.223+0000 7f3367b54700 1 -- 172.21.15.114:0/1686231725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3360100cb0 msgr2=0x7f336019b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:19:00.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:00.223+0000 7f3367b54700 1 --2- 172.21.15.114:0/1686231725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3360100cb0 0x7f336019b140 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3350006010 tx=0x7f335000be20 comp rx=0 tx=0).stop 2024-02-16T22:19:00.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:00.223+0000 7f3367b54700 1 -- 172.21.15.114:0/1686231725 shutdown_connections 2024-02-16T22:19:00.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:00.223+0000 7f3367b54700 1 --2- 172.21.15.114:0/1686231725 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f334c06c4e0 0x7f334c06e9a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:00.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:00.223+0000 7f3367b54700 1 --2- 172.21.15.114:0/1686231725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33600ffab0 0x7f3360068f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:00.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:00.223+0000 7f3367b54700 1 --2- 172.21.15.114:0/1686231725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3360100cb0 0x7f336019b140 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:00.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:00.223+0000 7f3367b54700 1 -- 172.21.15.114:0/1686231725 >> 172.21.15.114:0/1686231725 conn(0x7f33600fb030 msgr2=0x7f3360103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:19:00.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:00.223+0000 7f3367b54700 1 -- 172.21.15.114:0/1686231725 shutdown_connections 2024-02-16T22:19:00.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:00.223+0000 7f3367b54700 1 -- 172.21.15.114:0/1686231725 wait complete. 2024-02-16T22:19:00.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:00 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/3173978428' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:19:00.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:00 smithi114 ceph-mon[29235]: pgmap v207: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-16T22:19:00.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:00 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/1686231725' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:19:00.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:00 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/3173978428' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:19:00.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:00 smithi123 ceph-mon[37161]: pgmap v207: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-16T22:19:00.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:00 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/1686231725' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:19:01.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:01 smithi123 ceph-mon[37161]: from='client.14588 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:19:01.699 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:01 smithi114 ceph-mon[29235]: from='client.14588 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:19:02.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:02 smithi114 ceph-mon[29235]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-02-16T22:19:02.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:02 smithi123 ceph-mon[37161]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-02-16T22:19:04.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:04 smithi123 ceph-mon[37161]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:04.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:04 smithi114 ceph-mon[29235]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:06.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:06 smithi123 ceph-mon[37161]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:06.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:06 smithi114 ceph-mon[29235]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:08.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:07 smithi114 ceph-mon[29235]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:08.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:07 smithi123 ceph-mon[37161]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:10.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:10 smithi123 ceph-mon[37161]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:10.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:10 smithi114 ceph-mon[29235]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:11.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:11 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:19:11.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:11 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:19:12.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:12 smithi123 ceph-mon[37161]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:12.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:12 smithi114 ceph-mon[29235]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:14.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:14 smithi123 ceph-mon[37161]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:14.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:14 smithi114 ceph-mon[29235]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:16.555 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:16 smithi123 ceph-mon[37161]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:16.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:16 smithi114 ceph-mon[29235]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:18.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:17 smithi114 ceph-mon[29235]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:18.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:17 smithi123 ceph-mon[37161]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:20.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:20 smithi123 ceph-mon[37161]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:20.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:20 smithi114 ceph-mon[29235]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:22.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:22 smithi123 ceph-mon[37161]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:22.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:22 smithi114 ceph-mon[29235]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:24.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:24 smithi123 ceph-mon[37161]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:24.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:24 smithi114 ceph-mon[29235]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:26.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:26 smithi123 ceph-mon[37161]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:26.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:26 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:19:26.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:26 smithi114 ceph-mon[29235]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:26.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:19:28.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:27 smithi114 ceph-mon[29235]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:28.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:27 smithi123 ceph-mon[37161]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:30.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.333+0000 7f22563d7700 1 -- 172.21.15.114:0/3651230913 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22500feed0 msgr2=0x7f22500ff2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:19:30.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.333+0000 7f22563d7700 1 --2- 172.21.15.114:0/3651230913 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22500feed0 0x7f22500ff2f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2244009910 tx=0x7f2244009c20 comp rx=0 tx=0).stop 2024-02-16T22:19:30.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.333+0000 7f22563d7700 1 -- 172.21.15.114:0/3651230913 shutdown_connections 2024-02-16T22:19:30.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.333+0000 7f22563d7700 1 --2- 172.21.15.114:0/3651230913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2250101eb0 0x7f2250102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:30.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.333+0000 7f22563d7700 1 --2- 172.21.15.114:0/3651230913 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22500feed0 0x7f22500ff2f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:30.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.333+0000 7f22563d7700 1 -- 172.21.15.114:0/3651230913 >> 172.21.15.114:0/3651230913 conn(0x7f22500fa810 msgr2=0x7f22500fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:19:30.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.333+0000 7f22563d7700 1 -- 172.21.15.114:0/3651230913 shutdown_connections 2024-02-16T22:19:30.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.333+0000 7f22563d7700 1 -- 172.21.15.114:0/3651230913 wait complete. 2024-02-16T22:19:30.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.334+0000 7f22563d7700 1 Processor -- start 2024-02-16T22:19:30.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.334+0000 7f22563d7700 1 -- start start 2024-02-16T22:19:30.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.334+0000 7f22563d7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22500feed0 0x7f2250196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:19:30.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.334+0000 7f22563d7700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2250101eb0 0x7f2250196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:19:30.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.334+0000 7f22563d7700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2250196fa0 con 0x7f2250101eb0 2024-02-16T22:19:30.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.334+0000 7f22563d7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22501970e0 con 0x7f22500feed0 2024-02-16T22:19:30.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.334+0000 7f224f7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2250101eb0 0x7f2250196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:19:30.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.334+0000 7f224ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22500feed0 0x7f2250196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:19:30.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.334+0000 7f224f7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2250101eb0 0x7f2250196980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60944/0 (socket says 172.21.15.114:60944) 2024-02-16T22:19:30.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.334+0000 7f224f7fe700 1 -- 172.21.15.114:0/770015636 learned_addr learned my addr 172.21.15.114:0/770015636 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:19:30.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.335+0000 7f224f7fe700 1 -- 172.21.15.114:0/770015636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22500feed0 msgr2=0x7f2250196440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:19:30.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.335+0000 7f224f7fe700 1 --2- 172.21.15.114:0/770015636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22500feed0 0x7f2250196440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:30.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.335+0000 7f224f7fe700 1 -- 172.21.15.114:0/770015636 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22440095f0 con 0x7f2250101eb0 2024-02-16T22:19:30.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.335+0000 7f224f7fe700 1 --2- 172.21.15.114:0/770015636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2250101eb0 0x7f2250196980 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f224000d8d0 tx=0x7f224000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:19:30.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.335+0000 7f224d7fa700 1 -- 172.21.15.114:0/770015636 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22400098a0 con 0x7f2250101eb0 2024-02-16T22:19:30.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.335+0000 7f224d7fa700 1 -- 172.21.15.114:0/770015636 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2240010460 con 0x7f2250101eb0 2024-02-16T22:19:30.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.335+0000 7f22563d7700 1 -- 172.21.15.114:0/770015636 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f225019bb90 con 0x7f2250101eb0 2024-02-16T22:19:30.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.335+0000 7f224d7fa700 1 -- 172.21.15.114:0/770015636 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f224000f5f0 con 0x7f2250101eb0 2024-02-16T22:19:30.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.335+0000 7f22563d7700 1 -- 172.21.15.114:0/770015636 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f225019c0e0 con 0x7f2250101eb0 2024-02-16T22:19:30.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.336+0000 7f22563d7700 1 -- 172.21.15.114:0/770015636 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2250109770 con 0x7f2250101eb0 2024-02-16T22:19:30.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.336+0000 7f224d7fa700 1 -- 172.21.15.114:0/770015636 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f2240009a00 con 0x7f2250101eb0 2024-02-16T22:19:30.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.337+0000 7f224d7fa700 1 --2- 172.21.15.114:0/770015636 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f223806c4e0 0x7f223806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:19:30.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.337+0000 7f224d7fa700 1 -- 172.21.15.114:0/770015636 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f224008b6d0 con 0x7f2250101eb0 2024-02-16T22:19:30.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.337+0000 7f224ffff700 1 --2- 172.21.15.114:0/770015636 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f223806c4e0 0x7f223806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:19:30.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.337+0000 7f224ffff700 1 --2- 172.21.15.114:0/770015636 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f223806c4e0 0x7f223806e9a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2244005090 tx=0x7f224401a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:19:30.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.341+0000 7f224d7fa700 1 -- 172.21.15.114:0/770015636 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2240059940 con 0x7f2250101eb0 2024-02-16T22:19:30.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.571+0000 7f22563d7700 1 -- 172.21.15.114:0/770015636 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2250106800 con 0x7f223806c4e0 2024-02-16T22:19:30.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.572+0000 7f224d7fa700 1 -- 172.21.15.114:0/770015636 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f2250106800 con 0x7f223806c4e0 2024-02-16T22:19:30.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.574+0000 7f22563d7700 1 -- 172.21.15.114:0/770015636 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f223806c4e0 msgr2=0x7f223806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:19:30.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.574+0000 7f22563d7700 1 --2- 172.21.15.114:0/770015636 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f223806c4e0 0x7f223806e9a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2244005090 tx=0x7f224401a040 comp rx=0 tx=0).stop 2024-02-16T22:19:30.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.574+0000 7f22563d7700 1 -- 172.21.15.114:0/770015636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2250101eb0 msgr2=0x7f2250196980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:19:30.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.574+0000 7f22563d7700 1 --2- 172.21.15.114:0/770015636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2250101eb0 0x7f2250196980 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f224000d8d0 tx=0x7f224000dbe0 comp rx=0 tx=0).stop 2024-02-16T22:19:30.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.575+0000 7f22563d7700 1 -- 172.21.15.114:0/770015636 shutdown_connections 2024-02-16T22:19:30.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.575+0000 7f22563d7700 1 --2- 172.21.15.114:0/770015636 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f223806c4e0 0x7f223806e9a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:30.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.575+0000 7f22563d7700 1 --2- 172.21.15.114:0/770015636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22500feed0 0x7f2250196440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:30.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.575+0000 7f22563d7700 1 --2- 172.21.15.114:0/770015636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2250101eb0 0x7f2250196980 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:30.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.575+0000 7f22563d7700 1 -- 172.21.15.114:0/770015636 >> 172.21.15.114:0/770015636 conn(0x7f22500fa810 msgr2=0x7f22501050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:19:30.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.575+0000 7f22563d7700 1 -- 172.21.15.114:0/770015636 shutdown_connections 2024-02-16T22:19:30.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.575+0000 7f22563d7700 1 -- 172.21.15.114:0/770015636 wait complete. 2024-02-16T22:19:30.587 INFO:teuthology.orchestra.run.smithi114.stdout:true 2024-02-16T22:19:30.587 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:30 smithi114 ceph-mon[29235]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:30.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.681+0000 7ff4d134c700 1 -- 172.21.15.114:0/4135382791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4cc0fee80 msgr2=0x7ff4cc0ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:19:30.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.681+0000 7ff4d134c700 1 --2- 172.21.15.114:0/4135382791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4cc0fee80 0x7ff4cc0ff2a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009910 tx=0x7ff4bc009c20 comp rx=0 tx=0).stop 2024-02-16T22:19:30.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.681+0000 7ff4d134c700 1 -- 172.21.15.114:0/4135382791 shutdown_connections 2024-02-16T22:19:30.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.681+0000 7ff4d134c700 1 --2- 172.21.15.114:0/4135382791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4cc101e60 0x7ff4cc1022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:30.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.681+0000 7ff4d134c700 1 --2- 172.21.15.114:0/4135382791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4cc0fee80 0x7ff4cc0ff2a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:30.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.681+0000 7ff4d134c700 1 -- 172.21.15.114:0/4135382791 >> 172.21.15.114:0/4135382791 conn(0x7ff4cc0fa7c0 msgr2=0x7ff4cc0fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:19:30.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.681+0000 7ff4d134c700 1 -- 172.21.15.114:0/4135382791 shutdown_connections 2024-02-16T22:19:30.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.681+0000 7ff4d134c700 1 -- 172.21.15.114:0/4135382791 wait complete. 2024-02-16T22:19:30.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.682+0000 7ff4d134c700 1 Processor -- start 2024-02-16T22:19:30.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.682+0000 7ff4d134c700 1 -- start start 2024-02-16T22:19:30.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.682+0000 7ff4d134c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4cc0fee80 0x7ff4cc1963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:19:30.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.682+0000 7ff4d134c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4cc101e60 0x7ff4cc1968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:19:30.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.682+0000 7ff4d134c700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc196f00 con 0x7ff4cc0fee80 2024-02-16T22:19:30.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.682+0000 7ff4d134c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc197040 con 0x7ff4cc101e60 2024-02-16T22:19:30.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.682+0000 7ff4caffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4cc0fee80 0x7ff4cc1963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:19:30.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.682+0000 7ff4caffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4cc0fee80 0x7ff4cc1963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60972/0 (socket says 172.21.15.114:60972) 2024-02-16T22:19:30.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.682+0000 7ff4caffd700 1 -- 172.21.15.114:0/2466402529 learned_addr learned my addr 172.21.15.114:0/2466402529 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:19:30.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.682+0000 7ff4ca7fc700 1 --2- 172.21.15.114:0/2466402529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4cc101e60 0x7ff4cc1968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:19:30.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.683+0000 7ff4caffd700 1 -- 172.21.15.114:0/2466402529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4cc101e60 msgr2=0x7ff4cc1968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:19:30.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.683+0000 7ff4caffd700 1 --2- 172.21.15.114:0/2466402529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4cc101e60 0x7ff4cc1968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:30.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.683+0000 7ff4caffd700 1 -- 172.21.15.114:0/2466402529 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4bc0095f0 con 0x7ff4cc0fee80 2024-02-16T22:19:30.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.683+0000 7ff4ca7fc700 1 --2- 172.21.15.114:0/2466402529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4cc101e60 0x7ff4cc1968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:19:30.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.683+0000 7ff4caffd700 1 --2- 172.21.15.114:0/2466402529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4cc0fee80 0x7ff4cc1963a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc000c00 tx=0x7ff4bc00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:19:30.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.683+0000 7ff4b3fff700 1 -- 172.21.15.114:0/2466402529 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4bc01d070 con 0x7ff4cc0fee80 2024-02-16T22:19:30.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.683+0000 7ff4b3fff700 1 -- 172.21.15.114:0/2466402529 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff4bc005dc0 con 0x7ff4cc0fee80 2024-02-16T22:19:30.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.683+0000 7ff4b3fff700 1 -- 172.21.15.114:0/2466402529 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4bc0226a0 con 0x7ff4cc0fee80 2024-02-16T22:19:30.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.683+0000 7ff4d134c700 1 -- 172.21.15.114:0/2466402529 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4cc19ba90 con 0x7ff4cc0fee80 2024-02-16T22:19:30.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.683+0000 7ff4d134c700 1 -- 172.21.15.114:0/2466402529 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4cc19bf30 con 0x7ff4cc0fee80 2024-02-16T22:19:30.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.684+0000 7ff4d134c700 1 -- 172.21.15.114:0/2466402529 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4cc109720 con 0x7ff4cc0fee80 2024-02-16T22:19:30.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.684+0000 7ff4b3fff700 1 -- 172.21.15.114:0/2466402529 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7ff4bc003910 con 0x7ff4cc0fee80 2024-02-16T22:19:30.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.685+0000 7ff4b3fff700 1 --2- 172.21.15.114:0/2466402529 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff4b406c4e0 0x7ff4b406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:19:30.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.685+0000 7ff4b3fff700 1 -- 172.21.15.114:0/2466402529 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7ff4bc08ce60 con 0x7ff4cc0fee80 2024-02-16T22:19:30.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.685+0000 7ff4ca7fc700 1 --2- 172.21.15.114:0/2466402529 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff4b406c4e0 0x7ff4b406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:19:30.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.685+0000 7ff4ca7fc700 1 --2- 172.21.15.114:0/2466402529 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff4b406c4e0 0x7ff4b406e9a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0009cb0 tx=0x7ff4c0009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:19:30.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.689+0000 7ff4b3fff700 1 -- 172.21.15.114:0/2466402529 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff4bc05b0d0 con 0x7ff4cc0fee80 2024-02-16T22:19:30.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:30 smithi123 ceph-mon[37161]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:30.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.913+0000 7ff4d134c700 1 -- 172.21.15.114:0/2466402529 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4cc1067b0 con 0x7ff4b406c4e0 2024-02-16T22:19:30.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.914+0000 7ff4b3fff700 1 -- 172.21.15.114:0/2466402529 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7ff4cc1067b0 con 0x7ff4b406c4e0 2024-02-16T22:19:30.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.916+0000 7ff4d134c700 1 -- 172.21.15.114:0/2466402529 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff4b406c4e0 msgr2=0x7ff4b406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:19:30.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.916+0000 7ff4d134c700 1 --2- 172.21.15.114:0/2466402529 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff4b406c4e0 0x7ff4b406e9a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0009cb0 tx=0x7ff4c0009260 comp rx=0 tx=0).stop 2024-02-16T22:19:30.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.916+0000 7ff4d134c700 1 -- 172.21.15.114:0/2466402529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4cc0fee80 msgr2=0x7ff4cc1963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:19:30.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.916+0000 7ff4d134c700 1 --2- 172.21.15.114:0/2466402529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4cc0fee80 0x7ff4cc1963a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc000c00 tx=0x7ff4bc00f740 comp rx=0 tx=0).stop 2024-02-16T22:19:30.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.916+0000 7ff4d134c700 1 -- 172.21.15.114:0/2466402529 shutdown_connections 2024-02-16T22:19:30.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.916+0000 7ff4d134c700 1 --2- 172.21.15.114:0/2466402529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4cc0fee80 0x7ff4cc1963a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:30.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.916+0000 7ff4d134c700 1 --2- 172.21.15.114:0/2466402529 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff4b406c4e0 0x7ff4b406e9a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:30.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.916+0000 7ff4d134c700 1 --2- 172.21.15.114:0/2466402529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4cc101e60 0x7ff4cc1968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:30.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.916+0000 7ff4d134c700 1 -- 172.21.15.114:0/2466402529 >> 172.21.15.114:0/2466402529 conn(0x7ff4cc0fa7c0 msgr2=0x7ff4cc105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:19:30.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.916+0000 7ff4d134c700 1 -- 172.21.15.114:0/2466402529 shutdown_connections 2024-02-16T22:19:30.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:30.916+0000 7ff4d134c700 1 -- 172.21.15.114:0/2466402529 wait complete. 2024-02-16T22:19:31.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.021+0000 7f2ba1a66700 1 -- 172.21.15.114:0/3514602615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b9c1016e0 msgr2=0x7f2b9c103ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:19:31.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.021+0000 7f2ba1a66700 1 --2- 172.21.15.114:0/3514602615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b9c1016e0 0x7f2b9c103ad0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c009910 tx=0x7f2b8c009c20 comp rx=0 tx=0).stop 2024-02-16T22:19:31.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.021+0000 7f2ba1a66700 1 -- 172.21.15.114:0/3514602615 shutdown_connections 2024-02-16T22:19:31.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.021+0000 7f2ba1a66700 1 --2- 172.21.15.114:0/3514602615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b9c104010 0x7f2b9c106400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:31.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.021+0000 7f2ba1a66700 1 --2- 172.21.15.114:0/3514602615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b9c1016e0 0x7f2b9c103ad0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:31.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.021+0000 7f2ba1a66700 1 -- 172.21.15.114:0/3514602615 >> 172.21.15.114:0/3514602615 conn(0x7f2b9c0fb030 msgr2=0x7f2b9c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:19:31.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.021+0000 7f2ba1a66700 1 -- 172.21.15.114:0/3514602615 shutdown_connections 2024-02-16T22:19:31.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.021+0000 7f2ba1a66700 1 -- 172.21.15.114:0/3514602615 wait complete. 2024-02-16T22:19:31.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.021+0000 7f2ba1a66700 1 Processor -- start 2024-02-16T22:19:31.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.022+0000 7f2ba1a66700 1 -- start start 2024-02-16T22:19:31.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.022+0000 7f2ba1a66700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b9c1016e0 0x7f2b9c196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:19:31.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.022+0000 7f2ba1a66700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b9c104010 0x7f2b9c196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:19:31.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.022+0000 7f2ba1a66700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c196e80 con 0x7f2b9c1016e0 2024-02-16T22:19:31.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.022+0000 7f2ba1a66700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c196fc0 con 0x7f2b9c104010 2024-02-16T22:19:31.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.022+0000 7f2b9b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b9c1016e0 0x7f2b9c196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:19:31.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.022+0000 7f2b9b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b9c1016e0 0x7f2b9c196320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60986/0 (socket says 172.21.15.114:60986) 2024-02-16T22:19:31.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.022+0000 7f2b9b7fe700 1 -- 172.21.15.114:0/3714290517 learned_addr learned my addr 172.21.15.114:0/3714290517 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:19:31.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.022+0000 7f2b9affd700 1 --2- 172.21.15.114:0/3714290517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b9c104010 0x7f2b9c196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:19:31.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.022+0000 7f2b9b7fe700 1 -- 172.21.15.114:0/3714290517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b9c104010 msgr2=0x7f2b9c196860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:19:31.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.022+0000 7f2b9b7fe700 1 --2- 172.21.15.114:0/3714290517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b9c104010 0x7f2b9c196860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:31.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.022+0000 7f2b9b7fe700 1 -- 172.21.15.114:0/3714290517 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b8c0095f0 con 0x7f2b9c1016e0 2024-02-16T22:19:31.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.022+0000 7f2b9b7fe700 1 --2- 172.21.15.114:0/3714290517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b9c1016e0 0x7f2b9c196320 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c00b5c0 tx=0x7f2b8c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:19:31.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.022+0000 7f2b98ff9700 1 -- 172.21.15.114:0/3714290517 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b8c01d070 con 0x7f2b9c1016e0 2024-02-16T22:19:31.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.022+0000 7f2b98ff9700 1 -- 172.21.15.114:0/3714290517 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2b8c00f460 con 0x7f2b9c1016e0 2024-02-16T22:19:31.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.022+0000 7f2ba1a66700 1 -- 172.21.15.114:0/3714290517 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b9c19ba10 con 0x7f2b9c1016e0 2024-02-16T22:19:31.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.022+0000 7f2b98ff9700 1 -- 172.21.15.114:0/3714290517 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b8c021640 con 0x7f2b9c1016e0 2024-02-16T22:19:31.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.023+0000 7f2ba1a66700 1 -- 172.21.15.114:0/3714290517 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b9c19bf00 con 0x7f2b9c1016e0 2024-02-16T22:19:31.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.023+0000 7f2ba1a66700 1 -- 172.21.15.114:0/3714290517 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b9c190530 con 0x7f2b9c1016e0 2024-02-16T22:19:31.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.024+0000 7f2b98ff9700 1 -- 172.21.15.114:0/3714290517 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f2b8c00fa60 con 0x7f2b9c1016e0 2024-02-16T22:19:31.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.024+0000 7f2b98ff9700 1 --2- 172.21.15.114:0/3714290517 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f2b8406c4e0 0x7f2b8406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:19:31.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.025+0000 7f2b98ff9700 1 -- 172.21.15.114:0/3714290517 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f2b8c08ca40 con 0x7f2b9c1016e0 2024-02-16T22:19:31.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.025+0000 7f2b9affd700 1 --2- 172.21.15.114:0/3714290517 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f2b8406c4e0 0x7f2b8406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:19:31.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.025+0000 7f2b9affd700 1 --2- 172.21.15.114:0/3714290517 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f2b8406c4e0 0x7f2b8406e9a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2b90005950 tx=0x7f2b9000b410 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:19:31.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.029+0000 7f2b98ff9700 1 -- 172.21.15.114:0/3714290517 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2b8c05ad30 con 0x7f2b9c1016e0 2024-02-16T22:19:31.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.242+0000 7f2ba1a66700 1 -- 172.21.15.114:0/3714290517 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b9c0611d0 con 0x7f2b8406c4e0 2024-02-16T22:19:31.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.253+0000 7f2b98ff9700 1 -- 172.21.15.114:0/3714290517 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3480 (secure 0 0 0) 0x7f2b9c0611d0 con 0x7f2b8406c4e0 2024-02-16T22:19:31.255 INFO:teuthology.orchestra.run.smithi114.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:19:31.255 INFO:teuthology.orchestra.run.smithi114.stdout:alertmanager.smithi114 smithi114 *:9093,9094 running (5m) 60s ago 6m 24.4M - 0.25.0 c8568f914cd2 e7404fea400e 2024-02-16T22:19:31.255 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi114 smithi114 running (7m) 60s ago 7m 7440k - 18.2.1 94eadd990e0a 7fd07162cd96 2024-02-16T22:19:31.255 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi123 smithi123 running (6m) 65s ago 6m 7168k - 18.2.1 94eadd990e0a 7687f5e99621 2024-02-16T22:19:31.255 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi114 smithi114 running (7m) 60s ago 7m 6660k - 18.2.1 94eadd990e0a fc8fbfe4fdd5 2024-02-16T22:19:31.255 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi123 smithi123 running (6m) 65s ago 6m 6665k - 18.2.1 94eadd990e0a a6249642b9bb 2024-02-16T22:19:31.256 INFO:teuthology.orchestra.run.smithi114.stdout:grafana.smithi114 smithi114 *:3000 running (5m) 60s ago 6m 88.1M - 9.4.7 954c08fa6188 db1c26c3bc20 2024-02-16T22:19:31.256 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.fdxpey smithi114 running (80s) 60s ago 79s 13.2M - 18.2.1 94eadd990e0a 704b40c144fb 2024-02-16T22:19:31.256 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.zzryke smithi114 running (85s) 60s ago 85s 16.4M - 18.2.1 94eadd990e0a 1f795d23c9a4 2024-02-16T22:19:31.256 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.gopcfa smithi123 running (77s) 65s ago 77s 13.2M - 18.2.1 94eadd990e0a 166134b7532a 2024-02-16T22:19:31.256 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.vhlqbe smithi123 running (83s) 65s ago 82s 14.7M - 18.2.1 94eadd990e0a 243c8bb96fa8 2024-02-16T22:19:31.256 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi114.vcullz smithi114 *:9283,8765,8443 running (8m) 60s ago 8m 488M - 18.2.1 94eadd990e0a c8f21783728d 2024-02-16T22:19:31.256 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi123.inzgjd smithi123 *:8443,9283,8765 running (5m) 65s ago 5m 443M - 18.2.1 94eadd990e0a 7f193d913aba 2024-02-16T22:19:31.256 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi114 smithi114 running (8m) 60s ago 8m 49.9M 2048M 18.2.1 94eadd990e0a b4c4433aa3a4 2024-02-16T22:19:31.256 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi123 smithi123 running (5m) 65s ago 5m 34.9M 2048M 18.2.1 94eadd990e0a c217fc30c152 2024-02-16T22:19:31.256 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi114 smithi114 *:9100 running (7m) 60s ago 7m 15.6M - 1.5.0 0da6a335fe13 744a090e06d0 2024-02-16T22:19:31.256 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi123 smithi123 *:9100 running (5m) 65s ago 5m 15.7M - 1.5.0 0da6a335fe13 3994b8c29b2e 2024-02-16T22:19:31.256 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 smithi114 running (5m) 60s ago 5m 47.7M 3946M 18.2.1 94eadd990e0a 24ca2e9714e0 2024-02-16T22:19:31.256 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 smithi114 running (4m) 60s ago 4m 47.8M 3946M 18.2.1 94eadd990e0a 32bb191b88e7 2024-02-16T22:19:31.256 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 smithi114 running (4m) 60s ago 4m 48.1M 3946M 18.2.1 94eadd990e0a 6608718d0818 2024-02-16T22:19:31.256 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 smithi123 running (3m) 65s ago 3m 48.8M 4970M 18.2.1 94eadd990e0a 575db379de3d 2024-02-16T22:19:31.256 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 smithi123 running (3m) 65s ago 3m 43.3M 4970M 18.2.1 94eadd990e0a 7e266866cd9c 2024-02-16T22:19:31.257 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 smithi123 running (2m) 65s ago 2m 42.1M 4970M 18.2.1 94eadd990e0a 24e5bef19fda 2024-02-16T22:19:31.257 INFO:teuthology.orchestra.run.smithi114.stdout:prometheus.smithi114 smithi114 *:9095 running (5m) 60s ago 6m 55.9M - 2.43.0 a07b618ecd1d 51185086d582 2024-02-16T22:19:31.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.257+0000 7f2ba1a66700 1 -- 172.21.15.114:0/3714290517 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f2b8406c4e0 msgr2=0x7f2b8406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:19:31.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.257+0000 7f2ba1a66700 1 --2- 172.21.15.114:0/3714290517 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f2b8406c4e0 0x7f2b8406e9a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2b90005950 tx=0x7f2b9000b410 comp rx=0 tx=0).stop 2024-02-16T22:19:31.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.257+0000 7f2ba1a66700 1 -- 172.21.15.114:0/3714290517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b9c1016e0 msgr2=0x7f2b9c196320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:19:31.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.257+0000 7f2ba1a66700 1 --2- 172.21.15.114:0/3714290517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b9c1016e0 0x7f2b9c196320 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c00b5c0 tx=0x7f2b8c00bd70 comp rx=0 tx=0).stop 2024-02-16T22:19:31.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.257+0000 7f2ba1a66700 1 -- 172.21.15.114:0/3714290517 shutdown_connections 2024-02-16T22:19:31.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.257+0000 7f2ba1a66700 1 --2- 172.21.15.114:0/3714290517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b9c1016e0 0x7f2b9c196320 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:31.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.257+0000 7f2ba1a66700 1 --2- 172.21.15.114:0/3714290517 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f2b8406c4e0 0x7f2b8406e9a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:31.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.257+0000 7f2ba1a66700 1 --2- 172.21.15.114:0/3714290517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b9c104010 0x7f2b9c196860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:31.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.257+0000 7f2ba1a66700 1 -- 172.21.15.114:0/3714290517 >> 172.21.15.114:0/3714290517 conn(0x7f2b9c0fb030 msgr2=0x7f2b9c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:19:31.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.257+0000 7f2ba1a66700 1 -- 172.21.15.114:0/3714290517 shutdown_connections 2024-02-16T22:19:31.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.257+0000 7f2ba1a66700 1 -- 172.21.15.114:0/3714290517 wait complete. 2024-02-16T22:19:31.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.361+0000 7f991d3a4700 1 -- 172.21.15.114:0/2228443330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9918100be0 msgr2=0x7f9918101000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:19:31.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.361+0000 7f991d3a4700 1 --2- 172.21.15.114:0/2228443330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9918100be0 0x7f9918101000 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9900009910 tx=0x7f9900009c20 comp rx=0 tx=0).stop 2024-02-16T22:19:31.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.361+0000 7f991d3a4700 1 -- 172.21.15.114:0/2228443330 shutdown_connections 2024-02-16T22:19:31.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.361+0000 7f991d3a4700 1 --2- 172.21.15.114:0/2228443330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9918101de0 0x7f9918102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:31.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.361+0000 7f991d3a4700 1 --2- 172.21.15.114:0/2228443330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9918100be0 0x7f9918101000 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:31.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.361+0000 7f991d3a4700 1 -- 172.21.15.114:0/2228443330 >> 172.21.15.114:0/2228443330 conn(0x7f99180fc140 msgr2=0x7f99180fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:19:31.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.361+0000 7f991d3a4700 1 -- 172.21.15.114:0/2228443330 shutdown_connections 2024-02-16T22:19:31.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.361+0000 7f991d3a4700 1 -- 172.21.15.114:0/2228443330 wait complete. 2024-02-16T22:19:31.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.361+0000 7f991d3a4700 1 Processor -- start 2024-02-16T22:19:31.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.361+0000 7f991d3a4700 1 -- start start 2024-02-16T22:19:31.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.362+0000 7f991d3a4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9918100be0 0x7f9918196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:19:31.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.362+0000 7f991d3a4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9918101de0 0x7f99181968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:19:31.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.362+0000 7f991d3a4700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9918196ef0 con 0x7f9918100be0 2024-02-16T22:19:31.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.362+0000 7f991d3a4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9918197030 con 0x7f9918101de0 2024-02-16T22:19:31.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.362+0000 7f990effd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9918100be0 0x7f9918196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:19:31.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.362+0000 7f990effd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9918100be0 0x7f9918196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:32782/0 (socket says 172.21.15.114:32782) 2024-02-16T22:19:31.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.362+0000 7f990effd700 1 -- 172.21.15.114:0/195865086 learned_addr learned my addr 172.21.15.114:0/195865086 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:19:31.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.362+0000 7f9916ffd700 1 --2- 172.21.15.114:0/195865086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9918101de0 0x7f99181968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:19:31.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.362+0000 7f990effd700 1 -- 172.21.15.114:0/195865086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9918101de0 msgr2=0x7f99181968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:19:31.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.362+0000 7f990effd700 1 --2- 172.21.15.114:0/195865086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9918101de0 0x7f99181968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:31.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.362+0000 7f990effd700 1 -- 172.21.15.114:0/195865086 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99000095f0 con 0x7f9918100be0 2024-02-16T22:19:31.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.362+0000 7f990effd700 1 --2- 172.21.15.114:0/195865086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9918100be0 0x7f9918196390 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9900000c00 tx=0x7f990000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:19:31.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.362+0000 7f9914ff9700 1 -- 172.21.15.114:0/195865086 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f990001d070 con 0x7f9918100be0 2024-02-16T22:19:31.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.362+0000 7f991d3a4700 1 -- 172.21.15.114:0/195865086 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991819ba80 con 0x7f9918100be0 2024-02-16T22:19:31.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.362+0000 7f9914ff9700 1 -- 172.21.15.114:0/195865086 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9900005dc0 con 0x7f9918100be0 2024-02-16T22:19:31.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.362+0000 7f9914ff9700 1 -- 172.21.15.114:0/195865086 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9900022470 con 0x7f9918100be0 2024-02-16T22:19:31.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.362+0000 7f991d3a4700 1 -- 172.21.15.114:0/195865086 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f991819bf70 con 0x7f9918100be0 2024-02-16T22:19:31.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.363+0000 7f991d3a4700 1 -- 172.21.15.114:0/195865086 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99181098d0 con 0x7f9918100be0 2024-02-16T22:19:31.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.364+0000 7f9914ff9700 1 -- 172.21.15.114:0/195865086 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f9900003810 con 0x7f9918100be0 2024-02-16T22:19:31.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.364+0000 7f9914ff9700 1 --2- 172.21.15.114:0/195865086 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f98f806c4e0 0x7f98f806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:19:31.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.364+0000 7f9914ff9700 1 -- 172.21.15.114:0/195865086 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f990008cda0 con 0x7f9918100be0 2024-02-16T22:19:31.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.364+0000 7f9916ffd700 1 --2- 172.21.15.114:0/195865086 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f98f806c4e0 0x7f98f806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:19:31.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.365+0000 7f9916ffd700 1 --2- 172.21.15.114:0/195865086 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f98f806c4e0 0x7f98f806e9a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9908009c60 tx=0x7f9908009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:19:31.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.369+0000 7f9914ff9700 1 -- 172.21.15.114:0/195865086 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f990005b010 con 0x7f9918100be0 2024-02-16T22:19:31.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.654+0000 7f991d3a4700 1 -- 172.21.15.114:0/195865086 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f991819c5e0 con 0x7f9918100be0 2024-02-16T22:19:31.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.655+0000 7f9914ff9700 1 -- 172.21.15.114:0/195865086 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7f990005aba0 con 0x7f9918100be0 2024-02-16T22:19:31.657 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:19:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": { 2024-02-16T22:19:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-02-16T22:19:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:19:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": { 2024-02-16T22:19:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-02-16T22:19:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:19:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": { 2024-02-16T22:19:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-02-16T22:19:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:19:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-02-16T22:19:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-02-16T22:19:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:19:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: "overall": { 2024-02-16T22:19:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 14 2024-02-16T22:19:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-02-16T22:19:31.658 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:19:31.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.657+0000 7f991d3a4700 1 -- 172.21.15.114:0/195865086 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f98f806c4e0 msgr2=0x7f98f806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:19:31.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.657+0000 7f991d3a4700 1 --2- 172.21.15.114:0/195865086 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f98f806c4e0 0x7f98f806e9a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9908009c60 tx=0x7f9908009260 comp rx=0 tx=0).stop 2024-02-16T22:19:31.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.657+0000 7f991d3a4700 1 -- 172.21.15.114:0/195865086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9918100be0 msgr2=0x7f9918196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:19:31.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.657+0000 7f991d3a4700 1 --2- 172.21.15.114:0/195865086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9918100be0 0x7f9918196390 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9900000c00 tx=0x7f990000f690 comp rx=0 tx=0).stop 2024-02-16T22:19:31.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.657+0000 7f991d3a4700 1 -- 172.21.15.114:0/195865086 shutdown_connections 2024-02-16T22:19:31.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.657+0000 7f991d3a4700 1 --2- 172.21.15.114:0/195865086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9918100be0 0x7f9918196390 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:31.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.657+0000 7f991d3a4700 1 --2- 172.21.15.114:0/195865086 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f98f806c4e0 0x7f98f806e9a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:31.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.657+0000 7f991d3a4700 1 --2- 172.21.15.114:0/195865086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9918101de0 0x7f99181968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:31.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.657+0000 7f991d3a4700 1 -- 172.21.15.114:0/195865086 >> 172.21.15.114:0/195865086 conn(0x7f99180fc140 msgr2=0x7f9918105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:19:31.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.657+0000 7f991d3a4700 1 -- 172.21.15.114:0/195865086 shutdown_connections 2024-02-16T22:19:31.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.657+0000 7f991d3a4700 1 -- 172.21.15.114:0/195865086 wait complete. 2024-02-16T22:19:31.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.762+0000 7f97cdd54700 1 -- 172.21.15.114:0/4093184822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97c80ffab0 msgr2=0x7f97c80ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:19:31.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.762+0000 7f97cdd54700 1 --2- 172.21.15.114:0/4093184822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97c80ffab0 0x7f97c80ffed0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f97b8009910 tx=0x7f97b8009c20 comp rx=0 tx=0).stop 2024-02-16T22:19:31.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.762+0000 7f97cdd54700 1 -- 172.21.15.114:0/4093184822 shutdown_connections 2024-02-16T22:19:31.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.762+0000 7f97cdd54700 1 --2- 172.21.15.114:0/4093184822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97c8100cb0 0x7f97c8101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:31.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.762+0000 7f97cdd54700 1 --2- 172.21.15.114:0/4093184822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97c80ffab0 0x7f97c80ffed0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:31.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.762+0000 7f97cdd54700 1 -- 172.21.15.114:0/4093184822 >> 172.21.15.114:0/4093184822 conn(0x7f97c80fb030 msgr2=0x7f97c80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:19:31.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.762+0000 7f97cdd54700 1 -- 172.21.15.114:0/4093184822 shutdown_connections 2024-02-16T22:19:31.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.762+0000 7f97cdd54700 1 -- 172.21.15.114:0/4093184822 wait complete. 2024-02-16T22:19:31.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.763+0000 7f97cdd54700 1 Processor -- start 2024-02-16T22:19:31.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.763+0000 7f97cdd54700 1 -- start start 2024-02-16T22:19:31.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.763+0000 7f97cdd54700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97c80ffab0 0x7f97c8196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:19:31.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.763+0000 7f97cdd54700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97c8100cb0 0x7f97c81968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:19:31.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.763+0000 7f97cdd54700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c8196ee0 con 0x7f97c80ffab0 2024-02-16T22:19:31.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.763+0000 7f97cdd54700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c8197020 con 0x7f97c8100cb0 2024-02-16T22:19:31.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.763+0000 7f97c77fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97c80ffab0 0x7f97c8196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:19:31.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.763+0000 7f97c77fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97c80ffab0 0x7f97c8196380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51756/0 (socket says 172.21.15.114:51756) 2024-02-16T22:19:31.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.763+0000 7f97c77fe700 1 -- 172.21.15.114:0/1974549340 learned_addr learned my addr 172.21.15.114:0/1974549340 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:19:31.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.763+0000 7f97c6ffd700 1 --2- 172.21.15.114:0/1974549340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97c8100cb0 0x7f97c81968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:19:31.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.763+0000 7f97c77fe700 1 -- 172.21.15.114:0/1974549340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97c8100cb0 msgr2=0x7f97c81968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:19:31.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.763+0000 7f97c77fe700 1 --2- 172.21.15.114:0/1974549340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97c8100cb0 0x7f97c81968c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:31.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.763+0000 7f97c77fe700 1 -- 172.21.15.114:0/1974549340 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b80095f0 con 0x7f97c80ffab0 2024-02-16T22:19:31.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.764+0000 7f97c77fe700 1 --2- 172.21.15.114:0/1974549340 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97c80ffab0 0x7f97c8196380 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f97b8000c00 tx=0x7f97b800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:19:31.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.764+0000 7f97c4ff9700 1 -- 172.21.15.114:0/1974549340 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97b801d070 con 0x7f97c80ffab0 2024-02-16T22:19:31.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.764+0000 7f97c4ff9700 1 -- 172.21.15.114:0/1974549340 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f97b8005dc0 con 0x7f97c80ffab0 2024-02-16T22:19:31.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.764+0000 7f97c4ff9700 1 -- 172.21.15.114:0/1974549340 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97b8022470 con 0x7f97c80ffab0 2024-02-16T22:19:31.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.764+0000 7f97cdd54700 1 -- 172.21.15.114:0/1974549340 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c819ba70 con 0x7f97c80ffab0 2024-02-16T22:19:31.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.764+0000 7f97cdd54700 1 -- 172.21.15.114:0/1974549340 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c819bf60 con 0x7f97c80ffab0 2024-02-16T22:19:31.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.765+0000 7f97cdd54700 1 -- 172.21.15.114:0/1974549340 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c8108630 con 0x7f97c80ffab0 2024-02-16T22:19:31.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.765+0000 7f97c4ff9700 1 -- 172.21.15.114:0/1974549340 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f97b8003810 con 0x7f97c80ffab0 2024-02-16T22:19:31.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.766+0000 7f97c4ff9700 1 --2- 172.21.15.114:0/1974549340 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f97b006c4e0 0x7f97b006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:19:31.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.766+0000 7f97c6ffd700 1 --2- 172.21.15.114:0/1974549340 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f97b006c4e0 0x7f97b006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:19:31.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.766+0000 7f97c4ff9700 1 -- 172.21.15.114:0/1974549340 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f97b808cc70 con 0x7f97c80ffab0 2024-02-16T22:19:31.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.766+0000 7f97c6ffd700 1 --2- 172.21.15.114:0/1974549340 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f97b006c4e0 0x7f97b006e9a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f97bc009c60 tx=0x7f97bc009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:19:31.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:31.770+0000 7f97c4ff9700 1 -- 172.21.15.114:0/1974549340 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f97b805af60 con 0x7f97c80ffab0 2024-02-16T22:19:31.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:31 smithi123 ceph-mon[37161]: from='client.14596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:19:31.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:31 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:19:31.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:31 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:19:31.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:31 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:19:31.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:31 smithi114 ceph-mon[29235]: from='client.14596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:19:31.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:31 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:19:31.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:31 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:19:31.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:31 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:19:32.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.024+0000 7f97cdd54700 1 -- 172.21.15.114:0/1974549340 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f97c819c5f0 con 0x7f97c80ffab0 2024-02-16T22:19:32.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.024+0000 7f97c4ff9700 1 -- 172.21.15.114:0/1974549340 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1825 (secure 0 0 0) 0x7f97b805aaf0 con 0x7f97c80ffab0 2024-02-16T22:19:32.026 INFO:teuthology.orchestra.run.smithi114.stdout:e14 2024-02-16T22:19:32.026 INFO:teuthology.orchestra.run.smithi114.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:19:32.026 INFO:teuthology.orchestra.run.smithi114.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:19:32.026 INFO:teuthology.orchestra.run.smithi114.stdout:legacy client fscid: 1 2024-02-16T22:19:32.026 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:19:32.027 INFO:teuthology.orchestra.run.smithi114.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:19:32.027 INFO:teuthology.orchestra.run.smithi114.stdout:fs_name cephfs 2024-02-16T22:19:32.027 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 12 2024-02-16T22:19:32.027 INFO:teuthology.orchestra.run.smithi114.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:19:32.027 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-02-16T22:18:02.108825+0000 2024-02-16T22:19:32.027 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-02-16T22:18:16.413371+0000 2024-02-16T22:19:32.027 INFO:teuthology.orchestra.run.smithi114.stdout:tableserver 0 2024-02-16T22:19:32.027 INFO:teuthology.orchestra.run.smithi114.stdout:root 0 2024-02-16T22:19:32.027 INFO:teuthology.orchestra.run.smithi114.stdout:session_timeout 60 2024-02-16T22:19:32.027 INFO:teuthology.orchestra.run.smithi114.stdout:session_autoclose 300 2024-02-16T22:19:32.027 INFO:teuthology.orchestra.run.smithi114.stdout:max_file_size 1099511627776 2024-02-16T22:19:32.027 INFO:teuthology.orchestra.run.smithi114.stdout:required_client_features {} 2024-02-16T22:19:32.027 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure 0 2024-02-16T22:19:32.027 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure_osd_epoch 0 2024-02-16T22:19:32.027 INFO:teuthology.orchestra.run.smithi114.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:19:32.027 INFO:teuthology.orchestra.run.smithi114.stdout:max_mds 1 2024-02-16T22:19:32.028 INFO:teuthology.orchestra.run.smithi114.stdout:in 0 2024-02-16T22:19:32.028 INFO:teuthology.orchestra.run.smithi114.stdout:up {0=14468} 2024-02-16T22:19:32.028 INFO:teuthology.orchestra.run.smithi114.stdout:failed 2024-02-16T22:19:32.028 INFO:teuthology.orchestra.run.smithi114.stdout:damaged 2024-02-16T22:19:32.028 INFO:teuthology.orchestra.run.smithi114.stdout:stopped 2024-02-16T22:19:32.028 INFO:teuthology.orchestra.run.smithi114.stdout:data_pools [3] 2024-02-16T22:19:32.028 INFO:teuthology.orchestra.run.smithi114.stdout:metadata_pool 2 2024-02-16T22:19:32.028 INFO:teuthology.orchestra.run.smithi114.stdout:inline_data enabled 2024-02-16T22:19:32.028 INFO:teuthology.orchestra.run.smithi114.stdout:balancer 2024-02-16T22:19:32.028 INFO:teuthology.orchestra.run.smithi114.stdout:bal_rank_mask -1 2024-02-16T22:19:32.028 INFO:teuthology.orchestra.run.smithi114.stdout:standby_count_wanted 1 2024-02-16T22:19:32.028 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.zzryke{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.114:6826/277953535,v1:172.21.15.114:6827/277953535] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:19:32.028 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:19:32.028 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:19:32.029 INFO:teuthology.orchestra.run.smithi114.stdout:Standby daemons: 2024-02-16T22:19:32.029 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:19:32.029 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.fdxpey{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:19:32.029 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.vhlqbe{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:19:32.029 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.gopcfa{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:19:32.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.026+0000 7f97cdd54700 1 -- 172.21.15.114:0/1974549340 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f97b006c4e0 msgr2=0x7f97b006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:19:32.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.026+0000 7f97cdd54700 1 --2- 172.21.15.114:0/1974549340 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f97b006c4e0 0x7f97b006e9a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f97bc009c60 tx=0x7f97bc009260 comp rx=0 tx=0).stop 2024-02-16T22:19:32.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.026+0000 7f97cdd54700 1 -- 172.21.15.114:0/1974549340 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97c80ffab0 msgr2=0x7f97c8196380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:19:32.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.026+0000 7f97cdd54700 1 --2- 172.21.15.114:0/1974549340 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97c80ffab0 0x7f97c8196380 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f97b8000c00 tx=0x7f97b800f690 comp rx=0 tx=0).stop 2024-02-16T22:19:32.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.026+0000 7f97cdd54700 1 -- 172.21.15.114:0/1974549340 shutdown_connections 2024-02-16T22:19:32.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.026+0000 7f97cdd54700 1 --2- 172.21.15.114:0/1974549340 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97c80ffab0 0x7f97c8196380 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:32.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.026+0000 7f97cdd54700 1 --2- 172.21.15.114:0/1974549340 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f97b006c4e0 0x7f97b006e9a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:32.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.026+0000 7f97cdd54700 1 --2- 172.21.15.114:0/1974549340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97c8100cb0 0x7f97c81968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:32.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.026+0000 7f97cdd54700 1 -- 172.21.15.114:0/1974549340 >> 172.21.15.114:0/1974549340 conn(0x7f97c80fb030 msgr2=0x7f97c8068410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:19:32.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.026+0000 7f97cdd54700 1 -- 172.21.15.114:0/1974549340 shutdown_connections 2024-02-16T22:19:32.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.026+0000 7f97cdd54700 1 -- 172.21.15.114:0/1974549340 wait complete. 2024-02-16T22:19:32.030 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 14 2024-02-16T22:19:32.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.131+0000 7f517a6d9700 1 -- 172.21.15.114:0/1395690670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5174100ce0 msgr2=0x7f5174101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:19:32.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.131+0000 7f517a6d9700 1 --2- 172.21.15.114:0/1395690670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5174100ce0 0x7f5174101140 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5168009910 tx=0x7f5168009c20 comp rx=0 tx=0).stop 2024-02-16T22:19:32.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.131+0000 7f517a6d9700 1 -- 172.21.15.114:0/1395690670 shutdown_connections 2024-02-16T22:19:32.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.131+0000 7f517a6d9700 1 --2- 172.21.15.114:0/1395690670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5174100ce0 0x7f5174101140 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:32.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.131+0000 7f517a6d9700 1 --2- 172.21.15.114:0/1395690670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51740ffae0 0x7f51740fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:32.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.131+0000 7f517a6d9700 1 -- 172.21.15.114:0/1395690670 >> 172.21.15.114:0/1395690670 conn(0x7f51740fb080 msgr2=0x7f51740fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:19:32.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.131+0000 7f517a6d9700 1 -- 172.21.15.114:0/1395690670 shutdown_connections 2024-02-16T22:19:32.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.131+0000 7f517a6d9700 1 -- 172.21.15.114:0/1395690670 wait complete. 2024-02-16T22:19:32.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.132+0000 7f517a6d9700 1 Processor -- start 2024-02-16T22:19:32.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.132+0000 7f517a6d9700 1 -- start start 2024-02-16T22:19:32.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.132+0000 7f517a6d9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51740ffae0 0x7f5174196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:19:32.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.132+0000 7f517a6d9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5174100ce0 0x7f51741968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:19:32.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.132+0000 7f517a6d9700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5174196ed0 con 0x7f51740ffae0 2024-02-16T22:19:32.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.132+0000 7f517a6d9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5174197010 con 0x7f5174100ce0 2024-02-16T22:19:32.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.132+0000 7f5173fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51740ffae0 0x7f5174196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:19:32.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.132+0000 7f5173fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51740ffae0 0x7f5174196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51784/0 (socket says 172.21.15.114:51784) 2024-02-16T22:19:32.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.132+0000 7f5173fff700 1 -- 172.21.15.114:0/1887082258 learned_addr learned my addr 172.21.15.114:0/1887082258 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:19:32.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.132+0000 7f51737fe700 1 --2- 172.21.15.114:0/1887082258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5174100ce0 0x7f51741968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:19:32.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.133+0000 7f5173fff700 1 -- 172.21.15.114:0/1887082258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5174100ce0 msgr2=0x7f51741968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:19:32.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.133+0000 7f5173fff700 1 --2- 172.21.15.114:0/1887082258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5174100ce0 0x7f51741968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:32.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.133+0000 7f5173fff700 1 -- 172.21.15.114:0/1887082258 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51680095f0 con 0x7f51740ffae0 2024-02-16T22:19:32.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.133+0000 7f5173fff700 1 --2- 172.21.15.114:0/1887082258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51740ffae0 0x7f5174196370 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f516400c8f0 tx=0x7f516400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:19:32.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.133+0000 7f51717fa700 1 -- 172.21.15.114:0/1887082258 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51640078e0 con 0x7f51740ffae0 2024-02-16T22:19:32.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.133+0000 7f517a6d9700 1 -- 172.21.15.114:0/1887082258 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f517419bac0 con 0x7f51740ffae0 2024-02-16T22:19:32.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.133+0000 7f51717fa700 1 -- 172.21.15.114:0/1887082258 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f516400f450 con 0x7f51740ffae0 2024-02-16T22:19:32.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.133+0000 7f51717fa700 1 -- 172.21.15.114:0/1887082258 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5164018610 con 0x7f51740ffae0 2024-02-16T22:19:32.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.133+0000 7f517a6d9700 1 -- 172.21.15.114:0/1887082258 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517419c010 con 0x7f51740ffae0 2024-02-16T22:19:32.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.134+0000 7f517a6d9700 1 -- 172.21.15.114:0/1887082258 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5174108640 con 0x7f51740ffae0 2024-02-16T22:19:32.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.135+0000 7f51717fa700 1 -- 172.21.15.114:0/1887082258 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f516400f5c0 con 0x7f51740ffae0 2024-02-16T22:19:32.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.135+0000 7f51717fa700 1 --2- 172.21.15.114:0/1887082258 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f515c06c4e0 0x7f515c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:19:32.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.135+0000 7f51717fa700 1 -- 172.21.15.114:0/1887082258 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f516408b6f0 con 0x7f51740ffae0 2024-02-16T22:19:32.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.135+0000 7f51737fe700 1 --2- 172.21.15.114:0/1887082258 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f515c06c4e0 0x7f515c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:19:32.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.136+0000 7f51737fe700 1 --2- 172.21.15.114:0/1887082258 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f515c06c4e0 0x7f515c06e9a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f516800b5c0 tx=0x7f51680052a0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:19:32.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.140+0000 7f51717fa700 1 -- 172.21.15.114:0/1887082258 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5164059960 con 0x7f51740ffae0 2024-02-16T22:19:32.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.388+0000 7f517a6d9700 1 -- 172.21.15.114:0/1887082258 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5174069440 con 0x7f515c06c4e0 2024-02-16T22:19:32.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.390+0000 7f51717fa700 1 -- 172.21.15.114:0/1887082258 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f5174069440 con 0x7f515c06c4e0 2024-02-16T22:19:32.392 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:19:32.392 INFO:teuthology.orchestra.run.smithi114.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:19:32.392 INFO:teuthology.orchestra.run.smithi114.stdout: "in_progress": true, 2024-02-16T22:19:32.392 INFO:teuthology.orchestra.run.smithi114.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:19:32.392 INFO:teuthology.orchestra.run.smithi114.stdout: "services_complete": [], 2024-02-16T22:19:32.392 INFO:teuthology.orchestra.run.smithi114.stdout: "progress": "0/23 daemons upgraded", 2024-02-16T22:19:32.392 INFO:teuthology.orchestra.run.smithi114.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 image on host smithi123", 2024-02-16T22:19:32.392 INFO:teuthology.orchestra.run.smithi114.stdout: "is_paused": false 2024-02-16T22:19:32.393 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:19:32.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.392+0000 7f517a6d9700 1 -- 172.21.15.114:0/1887082258 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f515c06c4e0 msgr2=0x7f515c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:19:32.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.392+0000 7f517a6d9700 1 --2- 172.21.15.114:0/1887082258 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f515c06c4e0 0x7f515c06e9a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f516800b5c0 tx=0x7f51680052a0 comp rx=0 tx=0).stop 2024-02-16T22:19:32.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.392+0000 7f517a6d9700 1 -- 172.21.15.114:0/1887082258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51740ffae0 msgr2=0x7f5174196370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:19:32.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.392+0000 7f517a6d9700 1 --2- 172.21.15.114:0/1887082258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51740ffae0 0x7f5174196370 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f516400c8f0 tx=0x7f516400cc00 comp rx=0 tx=0).stop 2024-02-16T22:19:32.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.392+0000 7f517a6d9700 1 -- 172.21.15.114:0/1887082258 shutdown_connections 2024-02-16T22:19:32.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.392+0000 7f517a6d9700 1 --2- 172.21.15.114:0/1887082258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51740ffae0 0x7f5174196370 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:32.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.392+0000 7f517a6d9700 1 --2- 172.21.15.114:0/1887082258 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f515c06c4e0 0x7f515c06e9a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:32.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.392+0000 7f517a6d9700 1 --2- 172.21.15.114:0/1887082258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5174100ce0 0x7f51741968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:32.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.392+0000 7f517a6d9700 1 -- 172.21.15.114:0/1887082258 >> 172.21.15.114:0/1887082258 conn(0x7f51740fb080 msgr2=0x7f5174068d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:19:32.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.392+0000 7f517a6d9700 1 -- 172.21.15.114:0/1887082258 shutdown_connections 2024-02-16T22:19:32.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.392+0000 7f517a6d9700 1 -- 172.21.15.114:0/1887082258 wait complete. 2024-02-16T22:19:32.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.498+0000 7f2024b2a700 1 -- 172.21.15.114:0/2810797847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2020100cb0 msgr2=0x7f2020101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:19:32.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.498+0000 7f2024b2a700 1 --2- 172.21.15.114:0/2810797847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2020100cb0 0x7f2020101110 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2008009910 tx=0x7f2008009c20 comp rx=0 tx=0).stop 2024-02-16T22:19:32.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.498+0000 7f2024b2a700 1 -- 172.21.15.114:0/2810797847 shutdown_connections 2024-02-16T22:19:32.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.498+0000 7f2024b2a700 1 --2- 172.21.15.114:0/2810797847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2020100cb0 0x7f2020101110 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:32.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.498+0000 7f2024b2a700 1 --2- 172.21.15.114:0/2810797847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20200ffab0 0x7f20200ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:32.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.498+0000 7f2024b2a700 1 -- 172.21.15.114:0/2810797847 >> 172.21.15.114:0/2810797847 conn(0x7f20200fb030 msgr2=0x7f20200fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:19:32.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.498+0000 7f2024b2a700 1 -- 172.21.15.114:0/2810797847 shutdown_connections 2024-02-16T22:19:32.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.498+0000 7f2024b2a700 1 -- 172.21.15.114:0/2810797847 wait complete. 2024-02-16T22:19:32.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.499+0000 7f2024b2a700 1 Processor -- start 2024-02-16T22:19:32.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.499+0000 7f2024b2a700 1 -- start start 2024-02-16T22:19:32.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.499+0000 7f2024b2a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20200ffab0 0x7f2020195260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:19:32.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.499+0000 7f2024b2a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2020100cb0 0x7f20201957a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:19:32.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.499+0000 7f2024b2a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2020195dc0 con 0x7f20200ffab0 2024-02-16T22:19:32.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.499+0000 7f2024b2a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2020195f00 con 0x7f2020100cb0 2024-02-16T22:19:32.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.499+0000 7f201e59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20200ffab0 0x7f2020195260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:19:32.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.499+0000 7f201e59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20200ffab0 0x7f2020195260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51798/0 (socket says 172.21.15.114:51798) 2024-02-16T22:19:32.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.499+0000 7f201e59c700 1 -- 172.21.15.114:0/883736292 learned_addr learned my addr 172.21.15.114:0/883736292 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:19:32.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.499+0000 7f201dd9b700 1 --2- 172.21.15.114:0/883736292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2020100cb0 0x7f20201957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:19:32.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.499+0000 7f201e59c700 1 -- 172.21.15.114:0/883736292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2020100cb0 msgr2=0x7f20201957a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:19:32.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.499+0000 7f201e59c700 1 --2- 172.21.15.114:0/883736292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2020100cb0 0x7f20201957a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:32.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.499+0000 7f201e59c700 1 -- 172.21.15.114:0/883736292 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20080095f0 con 0x7f20200ffab0 2024-02-16T22:19:32.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.499+0000 7f201e59c700 1 --2- 172.21.15.114:0/883736292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20200ffab0 0x7f2020195260 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f201800efc0 tx=0x7f201800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:19:32.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.500+0000 7f20177fe700 1 -- 172.21.15.114:0/883736292 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2018018070 con 0x7f20200ffab0 2024-02-16T22:19:32.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.500+0000 7f20177fe700 1 -- 172.21.15.114:0/883736292 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f201800f040 con 0x7f20200ffab0 2024-02-16T22:19:32.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.500+0000 7f2024b2a700 1 -- 172.21.15.114:0/883736292 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202019a950 con 0x7f20200ffab0 2024-02-16T22:19:32.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.500+0000 7f20177fe700 1 -- 172.21.15.114:0/883736292 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2018004490 con 0x7f20200ffab0 2024-02-16T22:19:32.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.500+0000 7f2024b2a700 1 -- 172.21.15.114:0/883736292 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f202019aea0 con 0x7f20200ffab0 2024-02-16T22:19:32.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.501+0000 7f2024b2a700 1 -- 172.21.15.114:0/883736292 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2020108610 con 0x7f20200ffab0 2024-02-16T22:19:32.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.501+0000 7f20177fe700 1 -- 172.21.15.114:0/883736292 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f2018004710 con 0x7f20200ffab0 2024-02-16T22:19:32.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.501+0000 7f20177fe700 1 --2- 172.21.15.114:0/883736292 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f200c06c4e0 0x7f200c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:19:32.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.502+0000 7f20177fe700 1 -- 172.21.15.114:0/883736292 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f201808ba30 con 0x7f20200ffab0 2024-02-16T22:19:32.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.502+0000 7f201dd9b700 1 --2- 172.21.15.114:0/883736292 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f200c06c4e0 0x7f200c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:19:32.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.502+0000 7f201dd9b700 1 --2- 172.21.15.114:0/883736292 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f200c06c4e0 0x7f200c06e9a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2008009fd0 tx=0x7f2008005940 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:19:32.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.506+0000 7f20177fe700 1 -- 172.21.15.114:0/883736292 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2018059ca0 con 0x7f20200ffab0 2024-02-16T22:19:32.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:32 smithi114 ceph-mon[29235]: Upgrade: Target is version 19.0.0-1450-gf78a58c0 (unknown) 2024-02-16T22:19:32.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:32 smithi114 ceph-mon[29235]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, digests ['quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526'] 2024-02-16T22:19:32.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:32 smithi114 ceph-mon[29235]: Upgrade: Need to upgrade myself (mgr.smithi114.vcullz) 2024-02-16T22:19:32.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:32 smithi114 ceph-mon[29235]: from='client.14600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:19:32.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:32 smithi114 ceph-mon[29235]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 on smithi123 2024-02-16T22:19:32.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:32 smithi114 ceph-mon[29235]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:19:32.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:32 smithi114 ceph-mon[29235]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:32.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:32 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/195865086' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:19:32.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:32 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/1974549340' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:19:32.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.792+0000 7f2024b2a700 1 -- 172.21.15.114:0/883736292 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f202019b4e0 con 0x7f20200ffab0 2024-02-16T22:19:32.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.792+0000 7f20177fe700 1 -- 172.21.15.114:0/883736292 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f2018059830 con 0x7f20200ffab0 2024-02-16T22:19:32.794 INFO:teuthology.orchestra.run.smithi114.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:19:32.794 INFO:teuthology.orchestra.run.smithi114.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:19:32.794 INFO:teuthology.orchestra.run.smithi114.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:19:32.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.794+0000 7f2024b2a700 1 -- 172.21.15.114:0/883736292 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f200c06c4e0 msgr2=0x7f200c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:19:32.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.794+0000 7f2024b2a700 1 --2- 172.21.15.114:0/883736292 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f200c06c4e0 0x7f200c06e9a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2008009fd0 tx=0x7f2008005940 comp rx=0 tx=0).stop 2024-02-16T22:19:32.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.794+0000 7f2024b2a700 1 -- 172.21.15.114:0/883736292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20200ffab0 msgr2=0x7f2020195260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:19:32.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.794+0000 7f2024b2a700 1 --2- 172.21.15.114:0/883736292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20200ffab0 0x7f2020195260 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f201800efc0 tx=0x7f201800c3a0 comp rx=0 tx=0).stop 2024-02-16T22:19:32.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.794+0000 7f2024b2a700 1 -- 172.21.15.114:0/883736292 shutdown_connections 2024-02-16T22:19:32.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.794+0000 7f2024b2a700 1 --2- 172.21.15.114:0/883736292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20200ffab0 0x7f2020195260 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:32.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.794+0000 7f2024b2a700 1 --2- 172.21.15.114:0/883736292 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f200c06c4e0 0x7f200c06e9a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:32.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.794+0000 7f2024b2a700 1 --2- 172.21.15.114:0/883736292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2020100cb0 0x7f20201957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:19:32.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.794+0000 7f2024b2a700 1 -- 172.21.15.114:0/883736292 >> 172.21.15.114:0/883736292 conn(0x7f20200fb030 msgr2=0x7f2020103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:19:32.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.794+0000 7f2024b2a700 1 -- 172.21.15.114:0/883736292 shutdown_connections 2024-02-16T22:19:32.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:19:32.794+0000 7f2024b2a700 1 -- 172.21.15.114:0/883736292 wait complete. 2024-02-16T22:19:32.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:32 smithi123 ceph-mon[37161]: Upgrade: Target is version 19.0.0-1450-gf78a58c0 (unknown) 2024-02-16T22:19:32.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:32 smithi123 ceph-mon[37161]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, digests ['quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526'] 2024-02-16T22:19:32.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:32 smithi123 ceph-mon[37161]: Upgrade: Need to upgrade myself (mgr.smithi114.vcullz) 2024-02-16T22:19:32.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:32 smithi123 ceph-mon[37161]: from='client.14600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:19:32.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:32 smithi123 ceph-mon[37161]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 on smithi123 2024-02-16T22:19:32.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:32 smithi123 ceph-mon[37161]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:19:32.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:32 smithi123 ceph-mon[37161]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:32.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:32 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/195865086' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:19:32.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:32 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/1974549340' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:19:33.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:33 smithi123 ceph-mon[37161]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:19:33.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:33 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/883736292' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:19:33.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:33 smithi114 ceph-mon[29235]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:19:33.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:33 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/883736292' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:19:34.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:34 smithi123 ceph-mon[37161]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:34.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:34 smithi114 ceph-mon[29235]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:36.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:36 smithi123 ceph-mon[37161]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:36.830 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:36 smithi114 ceph-mon[29235]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:38.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:37 smithi114 ceph-mon[29235]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:38.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:37 smithi123 ceph-mon[37161]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:40.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:40 smithi123 ceph-mon[37161]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:40.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:40 smithi114 ceph-mon[29235]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:41.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:41 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:19:41.877 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:41 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:19:42.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:42 smithi123 ceph-mon[37161]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:42.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:42 smithi114 ceph-mon[29235]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:44.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:44 smithi123 ceph-mon[37161]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:44.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:44 smithi114 ceph-mon[29235]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:46.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:46 smithi123 ceph-mon[37161]: pgmap v230: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:46.924 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:46 smithi114 ceph-mon[29235]: pgmap v230: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:48.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:47 smithi114 ceph-mon[29235]: pgmap v231: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:48.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:47 smithi123 ceph-mon[37161]: pgmap v231: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:50.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:50 smithi123 ceph-mon[37161]: pgmap v232: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:50.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:50 smithi114 ceph-mon[29235]: pgmap v232: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:52.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:52 smithi123 ceph-mon[37161]: pgmap v233: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:52.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:52 smithi114 ceph-mon[29235]: pgmap v233: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:54.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:54 smithi123 ceph-mon[37161]: pgmap v234: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:54.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:54 smithi114 ceph-mon[29235]: pgmap v234: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:56.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:56 smithi123 ceph-mon[37161]: pgmap v235: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:56.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:56 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:19:56.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:56 smithi114 ceph-mon[29235]: pgmap v235: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:56.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:56 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:19:58.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:19:57 smithi114 ceph-mon[29235]: pgmap v236: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:19:58.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:19:57 smithi123 ceph-mon[37161]: pgmap v236: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:00.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:00 smithi123 ceph-mon[37161]: pgmap v237: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:00.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:00 smithi123 ceph-mon[37161]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:20:00.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:00 smithi123 ceph-mon[37161]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:20:00.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:00 smithi123 ceph-mon[37161]: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:20:00.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:00 smithi114 ceph-mon[29235]: pgmap v237: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:00.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:00 smithi114 ceph-mon[29235]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:20:00.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:00 smithi114 ceph-mon[29235]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:20:00.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:00 smithi114 ceph-mon[29235]: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:20:02.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:02 smithi123 ceph-mon[37161]: pgmap v238: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:02.914 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:02 smithi114 ceph-mon[29235]: pgmap v238: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:02.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.913+0000 7fa6ba155700 1 -- 172.21.15.114:0/32818118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6b4101e90 msgr2=0x7fa6b41022f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:02.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.914+0000 7fa6ba155700 1 --2- 172.21.15.114:0/32818118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6b4101e90 0x7fa6b41022f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8009910 tx=0x7fa6a8009c20 comp rx=0 tx=0).stop 2024-02-16T22:20:02.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.914+0000 7fa6ba155700 1 -- 172.21.15.114:0/32818118 shutdown_connections 2024-02-16T22:20:02.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.914+0000 7fa6ba155700 1 --2- 172.21.15.114:0/32818118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6b4101e90 0x7fa6b41022f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:02.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.914+0000 7fa6ba155700 1 --2- 172.21.15.114:0/32818118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6b40feeb0 0x7fa6b40ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:02.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.914+0000 7fa6ba155700 1 -- 172.21.15.114:0/32818118 >> 172.21.15.114:0/32818118 conn(0x7fa6b40fa810 msgr2=0x7fa6b40fcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:20:02.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.914+0000 7fa6ba155700 1 -- 172.21.15.114:0/32818118 shutdown_connections 2024-02-16T22:20:02.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.914+0000 7fa6ba155700 1 -- 172.21.15.114:0/32818118 wait complete. 2024-02-16T22:20:02.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.914+0000 7fa6ba155700 1 Processor -- start 2024-02-16T22:20:02.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.915+0000 7fa6ba155700 1 -- start start 2024-02-16T22:20:02.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.915+0000 7fa6ba155700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6b40feeb0 0x7fa6b41963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:02.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.915+0000 7fa6ba155700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6b4101e90 0x7fa6b41968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:02.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.915+0000 7fa6ba155700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b4196f00 con 0x7fa6b40feeb0 2024-02-16T22:20:02.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.915+0000 7fa6ba155700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b4197040 con 0x7fa6b4101e90 2024-02-16T22:20:02.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.915+0000 7fa6b37fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6b40feeb0 0x7fa6b41963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:02.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.915+0000 7fa6b37fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6b40feeb0 0x7fa6b41963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51704/0 (socket says 172.21.15.114:51704) 2024-02-16T22:20:02.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.915+0000 7fa6b37fe700 1 -- 172.21.15.114:0/100395099 learned_addr learned my addr 172.21.15.114:0/100395099 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:20:02.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.915+0000 7fa6b2ffd700 1 --2- 172.21.15.114:0/100395099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6b4101e90 0x7fa6b41968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:02.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.915+0000 7fa6b37fe700 1 -- 172.21.15.114:0/100395099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6b4101e90 msgr2=0x7fa6b41968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:02.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.915+0000 7fa6b37fe700 1 --2- 172.21.15.114:0/100395099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6b4101e90 0x7fa6b41968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:02.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.915+0000 7fa6b37fe700 1 -- 172.21.15.114:0/100395099 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a80095f0 con 0x7fa6b40feeb0 2024-02-16T22:20:02.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.915+0000 7fa6b37fe700 1 --2- 172.21.15.114:0/100395099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6b40feeb0 0x7fa6b41963a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400efc0 tx=0x7fa6a400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:20:02.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.916+0000 7fa6b0ff9700 1 -- 172.21.15.114:0/100395099 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6a4018070 con 0x7fa6b40feeb0 2024-02-16T22:20:02.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.916+0000 7fa6b0ff9700 1 -- 172.21.15.114:0/100395099 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa6a400f040 con 0x7fa6b40feeb0 2024-02-16T22:20:02.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.916+0000 7fa6ba155700 1 -- 172.21.15.114:0/100395099 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b419ba90 con 0x7fa6b40feeb0 2024-02-16T22:20:02.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.916+0000 7fa6b0ff9700 1 -- 172.21.15.114:0/100395099 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6a4004490 con 0x7fa6b40feeb0 2024-02-16T22:20:02.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.916+0000 7fa6ba155700 1 -- 172.21.15.114:0/100395099 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b419bfe0 con 0x7fa6b40feeb0 2024-02-16T22:20:02.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.917+0000 7fa6ba155700 1 -- 172.21.15.114:0/100395099 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6b4109750 con 0x7fa6b40feeb0 2024-02-16T22:20:02.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.917+0000 7fa6b0ff9700 1 -- 172.21.15.114:0/100395099 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fa6a4004710 con 0x7fa6b40feeb0 2024-02-16T22:20:02.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.918+0000 7fa6b0ff9700 1 --2- 172.21.15.114:0/100395099 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa69c06c4e0 0x7fa69c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:02.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.918+0000 7fa6b0ff9700 1 -- 172.21.15.114:0/100395099 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fa6a408ba20 con 0x7fa6b40feeb0 2024-02-16T22:20:02.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.918+0000 7fa6b2ffd700 1 --2- 172.21.15.114:0/100395099 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa69c06c4e0 0x7fa69c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:02.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.918+0000 7fa6b2ffd700 1 --2- 172.21.15.114:0/100395099 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa69c06c4e0 0x7fa69c06e9a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8005b40 tx=0x7fa6a8005a90 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:20:02.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:02.922+0000 7fa6b0ff9700 1 -- 172.21.15.114:0/100395099 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa6a4059c90 con 0x7fa6b40feeb0 2024-02-16T22:20:03.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.130+0000 7fa6ba155700 1 -- 172.21.15.114:0/100395099 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6b41067e0 con 0x7fa69c06c4e0 2024-02-16T22:20:03.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.132+0000 7fa6b0ff9700 1 -- 172.21.15.114:0/100395099 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7fa6b41067e0 con 0x7fa69c06c4e0 2024-02-16T22:20:03.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.134+0000 7fa6ba155700 1 -- 172.21.15.114:0/100395099 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa69c06c4e0 msgr2=0x7fa69c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:03.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.134+0000 7fa6ba155700 1 --2- 172.21.15.114:0/100395099 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa69c06c4e0 0x7fa69c06e9a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8005b40 tx=0x7fa6a8005a90 comp rx=0 tx=0).stop 2024-02-16T22:20:03.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.134+0000 7fa6ba155700 1 -- 172.21.15.114:0/100395099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6b40feeb0 msgr2=0x7fa6b41963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:03.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.134+0000 7fa6ba155700 1 --2- 172.21.15.114:0/100395099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6b40feeb0 0x7fa6b41963a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400efc0 tx=0x7fa6a400c3a0 comp rx=0 tx=0).stop 2024-02-16T22:20:03.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.134+0000 7fa6ba155700 1 -- 172.21.15.114:0/100395099 shutdown_connections 2024-02-16T22:20:03.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.134+0000 7fa6ba155700 1 --2- 172.21.15.114:0/100395099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6b40feeb0 0x7fa6b41963a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:03.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.134+0000 7fa6ba155700 1 --2- 172.21.15.114:0/100395099 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa69c06c4e0 0x7fa69c06e9a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:03.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.134+0000 7fa6ba155700 1 --2- 172.21.15.114:0/100395099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6b4101e90 0x7fa6b41968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:03.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.134+0000 7fa6ba155700 1 -- 172.21.15.114:0/100395099 >> 172.21.15.114:0/100395099 conn(0x7fa6b40fa810 msgr2=0x7fa6b41050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:20:03.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.134+0000 7fa6ba155700 1 -- 172.21.15.114:0/100395099 shutdown_connections 2024-02-16T22:20:03.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.134+0000 7fa6ba155700 1 -- 172.21.15.114:0/100395099 wait complete. 2024-02-16T22:20:03.145 INFO:teuthology.orchestra.run.smithi114.stdout:true 2024-02-16T22:20:03.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.241+0000 7f86f405a700 1 -- 172.21.15.114:0/2899964292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86ec101dc0 msgr2=0x7f86ec102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:03.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.241+0000 7f86f405a700 1 --2- 172.21.15.114:0/2899964292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86ec101dc0 0x7f86ec102220 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f86dc009910 tx=0x7f86dc009c20 comp rx=0 tx=0).stop 2024-02-16T22:20:03.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.241+0000 7f86f405a700 1 -- 172.21.15.114:0/2899964292 shutdown_connections 2024-02-16T22:20:03.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.241+0000 7f86f405a700 1 --2- 172.21.15.114:0/2899964292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86ec101dc0 0x7f86ec102220 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:03.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.241+0000 7f86f405a700 1 --2- 172.21.15.114:0/2899964292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86ec100bc0 0x7f86ec100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:03.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.241+0000 7f86f405a700 1 -- 172.21.15.114:0/2899964292 >> 172.21.15.114:0/2899964292 conn(0x7f86ec0fc140 msgr2=0x7f86ec0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:20:03.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.241+0000 7f86f405a700 1 -- 172.21.15.114:0/2899964292 shutdown_connections 2024-02-16T22:20:03.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.241+0000 7f86f405a700 1 -- 172.21.15.114:0/2899964292 wait complete. 2024-02-16T22:20:03.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.241+0000 7f86f405a700 1 Processor -- start 2024-02-16T22:20:03.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.241+0000 7f86f405a700 1 -- start start 2024-02-16T22:20:03.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.241+0000 7f86f405a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86ec100bc0 0x7f86ec196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:03.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.241+0000 7f86f405a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86ec101dc0 0x7f86ec1968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:03.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.241+0000 7f86f405a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec196ed0 con 0x7f86ec100bc0 2024-02-16T22:20:03.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.242+0000 7f86f405a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec197010 con 0x7f86ec101dc0 2024-02-16T22:20:03.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.242+0000 7f86f1df6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86ec100bc0 0x7f86ec196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:03.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.242+0000 7f86f1df6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86ec100bc0 0x7f86ec196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51720/0 (socket says 172.21.15.114:51720) 2024-02-16T22:20:03.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.242+0000 7f86f1df6700 1 -- 172.21.15.114:0/2455620829 learned_addr learned my addr 172.21.15.114:0/2455620829 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:20:03.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.242+0000 7f86f15f5700 1 --2- 172.21.15.114:0/2455620829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86ec101dc0 0x7f86ec1968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:03.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.242+0000 7f86f1df6700 1 -- 172.21.15.114:0/2455620829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86ec101dc0 msgr2=0x7f86ec1968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:03.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.242+0000 7f86f1df6700 1 --2- 172.21.15.114:0/2455620829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86ec101dc0 0x7f86ec1968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:03.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.242+0000 7f86f1df6700 1 -- 172.21.15.114:0/2455620829 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86dc0095f0 con 0x7f86ec100bc0 2024-02-16T22:20:03.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.242+0000 7f86f1df6700 1 --2- 172.21.15.114:0/2455620829 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86ec100bc0 0x7f86ec196370 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f86e800c8f0 tx=0x7f86e800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:20:03.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.242+0000 7f86e2ffd700 1 -- 172.21.15.114:0/2455620829 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86e80078e0 con 0x7f86ec100bc0 2024-02-16T22:20:03.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.242+0000 7f86e2ffd700 1 -- 172.21.15.114:0/2455620829 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f86e800f450 con 0x7f86ec100bc0 2024-02-16T22:20:03.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.242+0000 7f86f405a700 1 -- 172.21.15.114:0/2455620829 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86ec19bac0 con 0x7f86ec100bc0 2024-02-16T22:20:03.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.242+0000 7f86e2ffd700 1 -- 172.21.15.114:0/2455620829 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86e8018610 con 0x7f86ec100bc0 2024-02-16T22:20:03.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.242+0000 7f86f405a700 1 -- 172.21.15.114:0/2455620829 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86ec19c010 con 0x7f86ec100bc0 2024-02-16T22:20:03.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.243+0000 7f86f405a700 1 -- 172.21.15.114:0/2455620829 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86ec109720 con 0x7f86ec100bc0 2024-02-16T22:20:03.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.244+0000 7f86e2ffd700 1 -- 172.21.15.114:0/2455620829 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f86e800f5c0 con 0x7f86ec100bc0 2024-02-16T22:20:03.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.244+0000 7f86e2ffd700 1 --2- 172.21.15.114:0/2455620829 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f86d806c4e0 0x7f86d806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:03.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.244+0000 7f86e2ffd700 1 -- 172.21.15.114:0/2455620829 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f86e808b740 con 0x7f86ec100bc0 2024-02-16T22:20:03.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.244+0000 7f86f15f5700 1 --2- 172.21.15.114:0/2455620829 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f86d806c4e0 0x7f86d806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:03.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.245+0000 7f86f15f5700 1 --2- 172.21.15.114:0/2455620829 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f86d806c4e0 0x7f86d806e9a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f86dc006010 tx=0x7f86dc00b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:20:03.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.249+0000 7f86e2ffd700 1 -- 172.21.15.114:0/2455620829 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f86e80599b0 con 0x7f86ec100bc0 2024-02-16T22:20:03.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.456+0000 7f86f405a700 1 -- 172.21.15.114:0/2455620829 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f86ec106710 con 0x7f86d806c4e0 2024-02-16T22:20:03.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.457+0000 7f86e2ffd700 1 -- 172.21.15.114:0/2455620829 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f86ec106710 con 0x7f86d806c4e0 2024-02-16T22:20:03.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.460+0000 7f86f405a700 1 -- 172.21.15.114:0/2455620829 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f86d806c4e0 msgr2=0x7f86d806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:03.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.460+0000 7f86f405a700 1 --2- 172.21.15.114:0/2455620829 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f86d806c4e0 0x7f86d806e9a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f86dc006010 tx=0x7f86dc00b540 comp rx=0 tx=0).stop 2024-02-16T22:20:03.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.460+0000 7f86f405a700 1 -- 172.21.15.114:0/2455620829 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86ec100bc0 msgr2=0x7f86ec196370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:03.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.460+0000 7f86f405a700 1 --2- 172.21.15.114:0/2455620829 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86ec100bc0 0x7f86ec196370 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f86e800c8f0 tx=0x7f86e800cc00 comp rx=0 tx=0).stop 2024-02-16T22:20:03.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.460+0000 7f86f405a700 1 -- 172.21.15.114:0/2455620829 shutdown_connections 2024-02-16T22:20:03.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.460+0000 7f86f405a700 1 --2- 172.21.15.114:0/2455620829 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86ec100bc0 0x7f86ec196370 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:03.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.460+0000 7f86f405a700 1 --2- 172.21.15.114:0/2455620829 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f86d806c4e0 0x7f86d806e9a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:03.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.460+0000 7f86f405a700 1 --2- 172.21.15.114:0/2455620829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86ec101dc0 0x7f86ec1968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:03.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.460+0000 7f86f405a700 1 -- 172.21.15.114:0/2455620829 >> 172.21.15.114:0/2455620829 conn(0x7f86ec0fc140 msgr2=0x7f86ec104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:20:03.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.460+0000 7f86f405a700 1 -- 172.21.15.114:0/2455620829 shutdown_connections 2024-02-16T22:20:03.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.460+0000 7f86f405a700 1 -- 172.21.15.114:0/2455620829 wait complete. 2024-02-16T22:20:03.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.566+0000 7ff555e35700 1 -- 172.21.15.114:0/994281317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5500ffab0 msgr2=0x7ff5500ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:03.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.566+0000 7ff555e35700 1 --2- 172.21.15.114:0/994281317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5500ffab0 0x7ff5500ffed0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff540009910 tx=0x7ff540009c20 comp rx=0 tx=0).stop 2024-02-16T22:20:03.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.566+0000 7ff555e35700 1 -- 172.21.15.114:0/994281317 shutdown_connections 2024-02-16T22:20:03.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.566+0000 7ff555e35700 1 --2- 172.21.15.114:0/994281317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff550100cb0 0x7ff550101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:03.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.566+0000 7ff555e35700 1 --2- 172.21.15.114:0/994281317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5500ffab0 0x7ff5500ffed0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:03.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.566+0000 7ff555e35700 1 -- 172.21.15.114:0/994281317 >> 172.21.15.114:0/994281317 conn(0x7ff5500fb030 msgr2=0x7ff5500fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:20:03.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.566+0000 7ff555e35700 1 -- 172.21.15.114:0/994281317 shutdown_connections 2024-02-16T22:20:03.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.566+0000 7ff555e35700 1 -- 172.21.15.114:0/994281317 wait complete. 2024-02-16T22:20:03.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.566+0000 7ff555e35700 1 Processor -- start 2024-02-16T22:20:03.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.567+0000 7ff555e35700 1 -- start start 2024-02-16T22:20:03.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.567+0000 7ff555e35700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5500ffab0 0x7ff550196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:03.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.567+0000 7ff555e35700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff550100cb0 0x7ff550196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:03.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.567+0000 7ff555e35700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff550196e80 con 0x7ff5500ffab0 2024-02-16T22:20:03.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.567+0000 7ff555e35700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff550196fc0 con 0x7ff550100cb0 2024-02-16T22:20:03.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.567+0000 7ff54f7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5500ffab0 0x7ff550196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:03.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.567+0000 7ff54f7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5500ffab0 0x7ff550196320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51744/0 (socket says 172.21.15.114:51744) 2024-02-16T22:20:03.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.567+0000 7ff54f7fe700 1 -- 172.21.15.114:0/3922088915 learned_addr learned my addr 172.21.15.114:0/3922088915 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:20:03.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.567+0000 7ff54effd700 1 --2- 172.21.15.114:0/3922088915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff550100cb0 0x7ff550196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:03.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.567+0000 7ff54effd700 1 -- 172.21.15.114:0/3922088915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5500ffab0 msgr2=0x7ff550196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:03.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.567+0000 7ff54effd700 1 --2- 172.21.15.114:0/3922088915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5500ffab0 0x7ff550196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:03.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.567+0000 7ff54effd700 1 -- 172.21.15.114:0/3922088915 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5400095f0 con 0x7ff550100cb0 2024-02-16T22:20:03.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.567+0000 7ff54f7fe700 1 --2- 172.21.15.114:0/3922088915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5500ffab0 0x7ff550196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:20:03.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.568+0000 7ff54effd700 1 --2- 172.21.15.114:0/3922088915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff550100cb0 0x7ff550196860 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff54400e970 tx=0x7ff54400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:20:03.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.568+0000 7ff54cff9700 1 -- 172.21.15.114:0/3922088915 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff54400c9d0 con 0x7ff550100cb0 2024-02-16T22:20:03.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.568+0000 7ff54cff9700 1 -- 172.21.15.114:0/3922088915 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff544010430 con 0x7ff550100cb0 2024-02-16T22:20:03.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.568+0000 7ff555e35700 1 -- 172.21.15.114:0/3922088915 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff55019ba70 con 0x7ff550100cb0 2024-02-16T22:20:03.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.568+0000 7ff54cff9700 1 -- 172.21.15.114:0/3922088915 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff544018650 con 0x7ff550100cb0 2024-02-16T22:20:03.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.568+0000 7ff555e35700 1 -- 172.21.15.114:0/3922088915 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff55019bfc0 con 0x7ff550100cb0 2024-02-16T22:20:03.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.569+0000 7ff555e35700 1 -- 172.21.15.114:0/3922088915 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff550108610 con 0x7ff550100cb0 2024-02-16T22:20:03.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.569+0000 7ff54cff9700 1 -- 172.21.15.114:0/3922088915 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7ff54400cb30 con 0x7ff550100cb0 2024-02-16T22:20:03.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.570+0000 7ff54cff9700 1 --2- 172.21.15.114:0/3922088915 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff53806c4e0 0x7ff53806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:03.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.570+0000 7ff54cff9700 1 -- 172.21.15.114:0/3922088915 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7ff544014070 con 0x7ff550100cb0 2024-02-16T22:20:03.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.570+0000 7ff54f7fe700 1 --2- 172.21.15.114:0/3922088915 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff53806c4e0 0x7ff53806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:03.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.570+0000 7ff54f7fe700 1 --2- 172.21.15.114:0/3922088915 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff53806c4e0 0x7ff53806e9a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff540006010 tx=0x7ff540005a90 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:20:03.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.574+0000 7ff54cff9700 1 -- 172.21.15.114:0/3922088915 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff5440520e0 con 0x7ff550100cb0 2024-02-16T22:20:03.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.774+0000 7ff555e35700 1 -- 172.21.15.114:0/3922088915 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff550068ae0 con 0x7ff53806c4e0 2024-02-16T22:20:03.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.785+0000 7ff54cff9700 1 -- 172.21.15.114:0/3922088915 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3528 (secure 0 0 0) 0x7ff550068ae0 con 0x7ff53806c4e0 2024-02-16T22:20:03.787 INFO:teuthology.orchestra.run.smithi114.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:20:03.787 INFO:teuthology.orchestra.run.smithi114.stdout:alertmanager.smithi114 smithi114 *:9093,9094 running (5m) 93s ago 7m 24.4M - 0.25.0 c8568f914cd2 e7404fea400e 2024-02-16T22:20:03.787 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi114 smithi114 running (7m) 93s ago 7m 7440k - 18.2.1 94eadd990e0a 7fd07162cd96 2024-02-16T22:20:03.787 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi123 smithi123 running (6m) 97s ago 6m 7168k - 18.2.1 94eadd990e0a 7687f5e99621 2024-02-16T22:20:03.787 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi114 smithi114 running (7m) 93s ago 7m 6660k - 18.2.1 94eadd990e0a fc8fbfe4fdd5 2024-02-16T22:20:03.787 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi123 smithi123 running (6m) 97s ago 6m 6665k - 18.2.1 94eadd990e0a a6249642b9bb 2024-02-16T22:20:03.787 INFO:teuthology.orchestra.run.smithi114.stdout:grafana.smithi114 smithi114 *:3000 running (5m) 93s ago 7m 88.1M - 9.4.7 954c08fa6188 db1c26c3bc20 2024-02-16T22:20:03.787 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.fdxpey smithi114 running (112s) 93s ago 112s 13.2M - 18.2.1 94eadd990e0a 704b40c144fb 2024-02-16T22:20:03.787 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.zzryke smithi114 running (118s) 93s ago 117s 16.4M - 18.2.1 94eadd990e0a 1f795d23c9a4 2024-02-16T22:20:03.787 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.gopcfa smithi123 running (110s) 97s ago 109s 13.2M - 18.2.1 94eadd990e0a 166134b7532a 2024-02-16T22:20:03.787 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.vhlqbe smithi123 running (115s) 97s ago 115s 14.7M - 18.2.1 94eadd990e0a 243c8bb96fa8 2024-02-16T22:20:03.787 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi114.vcullz smithi114 *:9283,8765,8443 running (8m) 93s ago 8m 488M - 18.2.1 94eadd990e0a c8f21783728d 2024-02-16T22:20:03.787 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi123.inzgjd smithi123 *:8443,9283,8765 running (6m) 97s ago 6m 443M - 18.2.1 94eadd990e0a 7f193d913aba 2024-02-16T22:20:03.787 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi114 smithi114 running (8m) 93s ago 9m 49.9M 2048M 18.2.1 94eadd990e0a b4c4433aa3a4 2024-02-16T22:20:03.788 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi123 smithi123 running (6m) 97s ago 6m 34.9M 2048M 18.2.1 94eadd990e0a c217fc30c152 2024-02-16T22:20:03.788 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi114 smithi114 *:9100 running (7m) 93s ago 7m 15.6M - 1.5.0 0da6a335fe13 744a090e06d0 2024-02-16T22:20:03.788 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi123 smithi123 *:9100 running (6m) 97s ago 6m 15.7M - 1.5.0 0da6a335fe13 3994b8c29b2e 2024-02-16T22:20:03.788 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 smithi114 running (5m) 93s ago 5m 47.7M 3946M 18.2.1 94eadd990e0a 24ca2e9714e0 2024-02-16T22:20:03.788 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 smithi114 running (5m) 93s ago 5m 47.8M 3946M 18.2.1 94eadd990e0a 32bb191b88e7 2024-02-16T22:20:03.788 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 smithi114 running (4m) 93s ago 4m 48.1M 3946M 18.2.1 94eadd990e0a 6608718d0818 2024-02-16T22:20:03.788 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 smithi123 running (4m) 97s ago 4m 48.8M 4970M 18.2.1 94eadd990e0a 575db379de3d 2024-02-16T22:20:03.788 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 smithi123 running (3m) 97s ago 3m 43.3M 4970M 18.2.1 94eadd990e0a 7e266866cd9c 2024-02-16T22:20:03.788 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 smithi123 running (3m) 97s ago 3m 42.1M 4970M 18.2.1 94eadd990e0a 24e5bef19fda 2024-02-16T22:20:03.788 INFO:teuthology.orchestra.run.smithi114.stdout:prometheus.smithi114 smithi114 *:9095 running (5m) 93s ago 7m 55.9M - 2.43.0 a07b618ecd1d 51185086d582 2024-02-16T22:20:03.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.787+0000 7ff555e35700 1 -- 172.21.15.114:0/3922088915 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff53806c4e0 msgr2=0x7ff53806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:03.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.787+0000 7ff555e35700 1 --2- 172.21.15.114:0/3922088915 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff53806c4e0 0x7ff53806e9a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff540006010 tx=0x7ff540005a90 comp rx=0 tx=0).stop 2024-02-16T22:20:03.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.787+0000 7ff555e35700 1 -- 172.21.15.114:0/3922088915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff550100cb0 msgr2=0x7ff550196860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:03.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.787+0000 7ff555e35700 1 --2- 172.21.15.114:0/3922088915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff550100cb0 0x7ff550196860 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff54400e970 tx=0x7ff54400ec80 comp rx=0 tx=0).stop 2024-02-16T22:20:03.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.787+0000 7ff555e35700 1 -- 172.21.15.114:0/3922088915 shutdown_connections 2024-02-16T22:20:03.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.787+0000 7ff555e35700 1 --2- 172.21.15.114:0/3922088915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5500ffab0 0x7ff550196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:03.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.787+0000 7ff555e35700 1 --2- 172.21.15.114:0/3922088915 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff53806c4e0 0x7ff53806e9a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:03.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.787+0000 7ff555e35700 1 --2- 172.21.15.114:0/3922088915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff550100cb0 0x7ff550196860 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:03.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.787+0000 7ff555e35700 1 -- 172.21.15.114:0/3922088915 >> 172.21.15.114:0/3922088915 conn(0x7ff5500fb030 msgr2=0x7ff5500683d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:20:03.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.787+0000 7ff555e35700 1 -- 172.21.15.114:0/3922088915 shutdown_connections 2024-02-16T22:20:03.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.787+0000 7ff555e35700 1 -- 172.21.15.114:0/3922088915 wait complete. 2024-02-16T22:20:03.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.889+0000 7f54be355700 1 -- 172.21.15.114:0/83983415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f54b80ffad0 msgr2=0x7f54b80ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:03.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.889+0000 7f54be355700 1 --2- 172.21.15.114:0/83983415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f54b80ffad0 0x7f54b80ffef0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009910 tx=0x7f54a8009c20 comp rx=0 tx=0).stop 2024-02-16T22:20:03.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.889+0000 7f54be355700 1 -- 172.21.15.114:0/83983415 shutdown_connections 2024-02-16T22:20:03.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.889+0000 7f54be355700 1 --2- 172.21.15.114:0/83983415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54b8100cd0 0x7f54b8101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:03.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.889+0000 7f54be355700 1 --2- 172.21.15.114:0/83983415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f54b80ffad0 0x7f54b80ffef0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:03.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.889+0000 7f54be355700 1 -- 172.21.15.114:0/83983415 >> 172.21.15.114:0/83983415 conn(0x7f54b80fb050 msgr2=0x7f54b80fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:20:03.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.889+0000 7f54be355700 1 -- 172.21.15.114:0/83983415 shutdown_connections 2024-02-16T22:20:03.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.890+0000 7f54be355700 1 -- 172.21.15.114:0/83983415 wait complete. 2024-02-16T22:20:03.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.890+0000 7f54be355700 1 Processor -- start 2024-02-16T22:20:03.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.890+0000 7f54be355700 1 -- start start 2024-02-16T22:20:03.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.890+0000 7f54be355700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f54b80ffad0 0x7f54b8103a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:03.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.890+0000 7f54be355700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54b8100cd0 0x7f54b81020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:03.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.890+0000 7f54be355700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b81025f0 con 0x7f54b80ffad0 2024-02-16T22:20:03.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.890+0000 7f54be355700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b8102730 con 0x7f54b8100cd0 2024-02-16T22:20:03.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.891+0000 7f54b7fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f54b80ffad0 0x7f54b8103a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:03.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.891+0000 7f54b7fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f54b80ffad0 0x7f54b8103a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51766/0 (socket says 172.21.15.114:51766) 2024-02-16T22:20:03.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.891+0000 7f54b7fff700 1 -- 172.21.15.114:0/1132250022 learned_addr learned my addr 172.21.15.114:0/1132250022 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:20:03.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.891+0000 7f54b77fe700 1 --2- 172.21.15.114:0/1132250022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54b8100cd0 0x7f54b81020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:03.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.891+0000 7f54b7fff700 1 -- 172.21.15.114:0/1132250022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54b8100cd0 msgr2=0x7f54b81020b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:03.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.891+0000 7f54b7fff700 1 --2- 172.21.15.114:0/1132250022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54b8100cd0 0x7f54b81020b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:03.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.891+0000 7f54b7fff700 1 -- 172.21.15.114:0/1132250022 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a80095f0 con 0x7f54b80ffad0 2024-02-16T22:20:03.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.891+0000 7f54b7fff700 1 --2- 172.21.15.114:0/1132250022 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f54b80ffad0 0x7f54b8103a60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f54a800bda0 tx=0x7f54a800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:20:03.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.891+0000 7f54b57fa700 1 -- 172.21.15.114:0/1132250022 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54a801d070 con 0x7f54b80ffad0 2024-02-16T22:20:03.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.891+0000 7f54b57fa700 1 -- 172.21.15.114:0/1132250022 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f54a800f460 con 0x7f54b80ffad0 2024-02-16T22:20:03.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.891+0000 7f54b57fa700 1 -- 172.21.15.114:0/1132250022 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54a8021640 con 0x7f54b80ffad0 2024-02-16T22:20:03.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.891+0000 7f54be355700 1 -- 172.21.15.114:0/1132250022 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b81029b0 con 0x7f54b80ffad0 2024-02-16T22:20:03.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.891+0000 7f54be355700 1 -- 172.21.15.114:0/1132250022 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b8102e50 con 0x7f54b80ffad0 2024-02-16T22:20:03.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.892+0000 7f54be355700 1 -- 172.21.15.114:0/1132250022 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54b8108630 con 0x7f54b80ffad0 2024-02-16T22:20:03.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.893+0000 7f54b57fa700 1 -- 172.21.15.114:0/1132250022 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f54a800fa00 con 0x7f54b80ffad0 2024-02-16T22:20:03.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.893+0000 7f54b57fa700 1 --2- 172.21.15.114:0/1132250022 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f54a006c4e0 0x7f54a006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:03.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.893+0000 7f54b57fa700 1 -- 172.21.15.114:0/1132250022 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f54a808cb20 con 0x7f54b80ffad0 2024-02-16T22:20:03.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.893+0000 7f54b77fe700 1 --2- 172.21.15.114:0/1132250022 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f54a006c4e0 0x7f54a006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:03.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.894+0000 7f54b77fe700 1 --2- 172.21.15.114:0/1132250022 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f54a006c4e0 0x7f54a006e9a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f54ac005950 tx=0x7f54ac0058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:20:03.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:03.898+0000 7f54b57fa700 1 -- 172.21.15.114:0/1132250022 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f54a805ad90 con 0x7f54b80ffad0 2024-02-16T22:20:04.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.171+0000 7f54be355700 1 -- 172.21.15.114:0/1132250022 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f54b819e7e0 con 0x7f54b80ffad0 2024-02-16T22:20:04.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.171+0000 7f54b57fa700 1 -- 172.21.15.114:0/1132250022 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7f54a805a920 con 0x7f54b80ffad0 2024-02-16T22:20:04.174 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:20:04.174 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": { 2024-02-16T22:20:04.174 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-02-16T22:20:04.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:20:04.174 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": { 2024-02-16T22:20:04.174 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-02-16T22:20:04.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:20:04.174 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": { 2024-02-16T22:20:04.174 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-02-16T22:20:04.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:20:04.174 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-02-16T22:20:04.174 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-02-16T22:20:04.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:20:04.175 INFO:teuthology.orchestra.run.smithi114.stdout: "overall": { 2024-02-16T22:20:04.175 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 14 2024-02-16T22:20:04.175 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-02-16T22:20:04.175 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:20:04.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.173+0000 7f54be355700 1 -- 172.21.15.114:0/1132250022 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f54a006c4e0 msgr2=0x7f54a006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:04.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.173+0000 7f54be355700 1 --2- 172.21.15.114:0/1132250022 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f54a006c4e0 0x7f54a006e9a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f54ac005950 tx=0x7f54ac0058e0 comp rx=0 tx=0).stop 2024-02-16T22:20:04.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.173+0000 7f54be355700 1 -- 172.21.15.114:0/1132250022 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f54b80ffad0 msgr2=0x7f54b8103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:04.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.173+0000 7f54be355700 1 --2- 172.21.15.114:0/1132250022 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f54b80ffad0 0x7f54b8103a60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f54a800bda0 tx=0x7f54a800be80 comp rx=0 tx=0).stop 2024-02-16T22:20:04.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.173+0000 7f54be355700 1 -- 172.21.15.114:0/1132250022 shutdown_connections 2024-02-16T22:20:04.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.173+0000 7f54be355700 1 --2- 172.21.15.114:0/1132250022 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f54b80ffad0 0x7f54b8103a60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:04.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.173+0000 7f54be355700 1 --2- 172.21.15.114:0/1132250022 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f54a006c4e0 0x7f54a006e9a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:04.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.173+0000 7f54be355700 1 --2- 172.21.15.114:0/1132250022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54b8100cd0 0x7f54b81020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:04.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.173+0000 7f54be355700 1 -- 172.21.15.114:0/1132250022 >> 172.21.15.114:0/1132250022 conn(0x7f54b80fb050 msgr2=0x7f54b8068d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:20:04.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.173+0000 7f54be355700 1 -- 172.21.15.114:0/1132250022 shutdown_connections 2024-02-16T22:20:04.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.174+0000 7f54be355700 1 -- 172.21.15.114:0/1132250022 wait complete. 2024-02-16T22:20:04.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.277+0000 7fa356cf7700 1 -- 172.21.15.114:0/3021296705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa350100cc0 msgr2=0x7fa350101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:04.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.277+0000 7fa356cf7700 1 --2- 172.21.15.114:0/3021296705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa350100cc0 0x7fa350101120 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa338009910 tx=0x7fa338009c20 comp rx=0 tx=0).stop 2024-02-16T22:20:04.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.277+0000 7fa356cf7700 1 -- 172.21.15.114:0/3021296705 shutdown_connections 2024-02-16T22:20:04.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.277+0000 7fa356cf7700 1 --2- 172.21.15.114:0/3021296705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa350100cc0 0x7fa350101120 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:04.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.277+0000 7fa356cf7700 1 --2- 172.21.15.114:0/3021296705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3500ffac0 0x7fa3500ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:04.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.277+0000 7fa356cf7700 1 -- 172.21.15.114:0/3021296705 >> 172.21.15.114:0/3021296705 conn(0x7fa3500fb060 msgr2=0x7fa3500fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:20:04.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.277+0000 7fa356cf7700 1 -- 172.21.15.114:0/3021296705 shutdown_connections 2024-02-16T22:20:04.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.277+0000 7fa356cf7700 1 -- 172.21.15.114:0/3021296705 wait complete. 2024-02-16T22:20:04.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.278+0000 7fa356cf7700 1 Processor -- start 2024-02-16T22:20:04.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.278+0000 7fa356cf7700 1 -- start start 2024-02-16T22:20:04.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.278+0000 7fa356cf7700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3500ffac0 0x7fa350069910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:04.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.278+0000 7fa356cf7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa350100cc0 0x7fa35019b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:04.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.278+0000 7fa356cf7700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35019b6a0 con 0x7fa3500ffac0 2024-02-16T22:20:04.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.278+0000 7fa356cf7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35019b7e0 con 0x7fa350100cc0 2024-02-16T22:20:04.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.278+0000 7fa354a93700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3500ffac0 0x7fa350069910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:04.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.278+0000 7fa354a93700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3500ffac0 0x7fa350069910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51782/0 (socket says 172.21.15.114:51782) 2024-02-16T22:20:04.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.278+0000 7fa354a93700 1 -- 172.21.15.114:0/420840076 learned_addr learned my addr 172.21.15.114:0/420840076 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:20:04.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.278+0000 7fa34ffff700 1 --2- 172.21.15.114:0/420840076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa350100cc0 0x7fa35019b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:04.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.278+0000 7fa354a93700 1 -- 172.21.15.114:0/420840076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa350100cc0 msgr2=0x7fa35019b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:04.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.278+0000 7fa354a93700 1 --2- 172.21.15.114:0/420840076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa350100cc0 0x7fa35019b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:04.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.278+0000 7fa354a93700 1 -- 172.21.15.114:0/420840076 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3380095f0 con 0x7fa3500ffac0 2024-02-16T22:20:04.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.278+0000 7fa354a93700 1 --2- 172.21.15.114:0/420840076 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3500ffac0 0x7fa350069910 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa34800d900 tx=0x7fa34800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:20:04.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.279+0000 7fa34dffb700 1 -- 172.21.15.114:0/420840076 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa34800b920 con 0x7fa3500ffac0 2024-02-16T22:20:04.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.279+0000 7fa34dffb700 1 -- 172.21.15.114:0/420840076 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa348010460 con 0x7fa3500ffac0 2024-02-16T22:20:04.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.279+0000 7fa356cf7700 1 -- 172.21.15.114:0/420840076 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa35019bac0 con 0x7fa3500ffac0 2024-02-16T22:20:04.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.279+0000 7fa34dffb700 1 -- 172.21.15.114:0/420840076 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa34800f5f0 con 0x7fa3500ffac0 2024-02-16T22:20:04.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.279+0000 7fa356cf7700 1 -- 172.21.15.114:0/420840076 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa35019bfe0 con 0x7fa3500ffac0 2024-02-16T22:20:04.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.280+0000 7fa356cf7700 1 -- 172.21.15.114:0/420840076 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa350108620 con 0x7fa3500ffac0 2024-02-16T22:20:04.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.280+0000 7fa34dffb700 1 -- 172.21.15.114:0/420840076 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fa3480105d0 con 0x7fa3500ffac0 2024-02-16T22:20:04.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.281+0000 7fa34dffb700 1 --2- 172.21.15.114:0/420840076 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa33c06c4e0 0x7fa33c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:04.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.281+0000 7fa34dffb700 1 -- 172.21.15.114:0/420840076 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fa34808b410 con 0x7fa3500ffac0 2024-02-16T22:20:04.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.281+0000 7fa34ffff700 1 --2- 172.21.15.114:0/420840076 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa33c06c4e0 0x7fa33c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:04.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.281+0000 7fa34ffff700 1 --2- 172.21.15.114:0/420840076 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa33c06c4e0 0x7fa33c06e9a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa33800b5c0 tx=0x7fa338005ee0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:20:04.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.285+0000 7fa34dffb700 1 -- 172.21.15.114:0/420840076 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa348059680 con 0x7fa3500ffac0 2024-02-16T22:20:04.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.525+0000 7fa356cf7700 1 -- 172.21.15.114:0/420840076 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa350108810 con 0x7fa3500ffac0 2024-02-16T22:20:04.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.525+0000 7fa34dffb700 1 -- 172.21.15.114:0/420840076 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1825 (secure 0 0 0) 0x7fa348016740 con 0x7fa3500ffac0 2024-02-16T22:20:04.527 INFO:teuthology.orchestra.run.smithi114.stdout:e14 2024-02-16T22:20:04.527 INFO:teuthology.orchestra.run.smithi114.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:20:04.528 INFO:teuthology.orchestra.run.smithi114.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:20:04.528 INFO:teuthology.orchestra.run.smithi114.stdout:legacy client fscid: 1 2024-02-16T22:20:04.528 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:20:04.528 INFO:teuthology.orchestra.run.smithi114.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:20:04.528 INFO:teuthology.orchestra.run.smithi114.stdout:fs_name cephfs 2024-02-16T22:20:04.528 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 12 2024-02-16T22:20:04.528 INFO:teuthology.orchestra.run.smithi114.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:20:04.528 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-02-16T22:18:02.108825+0000 2024-02-16T22:20:04.528 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-02-16T22:18:16.413371+0000 2024-02-16T22:20:04.528 INFO:teuthology.orchestra.run.smithi114.stdout:tableserver 0 2024-02-16T22:20:04.528 INFO:teuthology.orchestra.run.smithi114.stdout:root 0 2024-02-16T22:20:04.528 INFO:teuthology.orchestra.run.smithi114.stdout:session_timeout 60 2024-02-16T22:20:04.528 INFO:teuthology.orchestra.run.smithi114.stdout:session_autoclose 300 2024-02-16T22:20:04.528 INFO:teuthology.orchestra.run.smithi114.stdout:max_file_size 1099511627776 2024-02-16T22:20:04.529 INFO:teuthology.orchestra.run.smithi114.stdout:required_client_features {} 2024-02-16T22:20:04.529 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure 0 2024-02-16T22:20:04.529 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure_osd_epoch 0 2024-02-16T22:20:04.529 INFO:teuthology.orchestra.run.smithi114.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:20:04.529 INFO:teuthology.orchestra.run.smithi114.stdout:max_mds 1 2024-02-16T22:20:04.529 INFO:teuthology.orchestra.run.smithi114.stdout:in 0 2024-02-16T22:20:04.529 INFO:teuthology.orchestra.run.smithi114.stdout:up {0=14468} 2024-02-16T22:20:04.529 INFO:teuthology.orchestra.run.smithi114.stdout:failed 2024-02-16T22:20:04.529 INFO:teuthology.orchestra.run.smithi114.stdout:damaged 2024-02-16T22:20:04.529 INFO:teuthology.orchestra.run.smithi114.stdout:stopped 2024-02-16T22:20:04.529 INFO:teuthology.orchestra.run.smithi114.stdout:data_pools [3] 2024-02-16T22:20:04.529 INFO:teuthology.orchestra.run.smithi114.stdout:metadata_pool 2 2024-02-16T22:20:04.529 INFO:teuthology.orchestra.run.smithi114.stdout:inline_data enabled 2024-02-16T22:20:04.529 INFO:teuthology.orchestra.run.smithi114.stdout:balancer 2024-02-16T22:20:04.529 INFO:teuthology.orchestra.run.smithi114.stdout:bal_rank_mask -1 2024-02-16T22:20:04.530 INFO:teuthology.orchestra.run.smithi114.stdout:standby_count_wanted 1 2024-02-16T22:20:04.530 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.zzryke{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.114:6826/277953535,v1:172.21.15.114:6827/277953535] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:20:04.530 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:20:04.530 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:20:04.530 INFO:teuthology.orchestra.run.smithi114.stdout:Standby daemons: 2024-02-16T22:20:04.530 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:20:04.530 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.fdxpey{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:20:04.530 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.vhlqbe{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:20:04.530 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.gopcfa{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:20:04.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.528+0000 7fa356cf7700 1 -- 172.21.15.114:0/420840076 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa33c06c4e0 msgr2=0x7fa33c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:04.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.528+0000 7fa356cf7700 1 --2- 172.21.15.114:0/420840076 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa33c06c4e0 0x7fa33c06e9a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa33800b5c0 tx=0x7fa338005ee0 comp rx=0 tx=0).stop 2024-02-16T22:20:04.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.528+0000 7fa356cf7700 1 -- 172.21.15.114:0/420840076 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3500ffac0 msgr2=0x7fa350069910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:04.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.528+0000 7fa356cf7700 1 --2- 172.21.15.114:0/420840076 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3500ffac0 0x7fa350069910 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa34800d900 tx=0x7fa34800dc10 comp rx=0 tx=0).stop 2024-02-16T22:20:04.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.528+0000 7fa356cf7700 1 -- 172.21.15.114:0/420840076 shutdown_connections 2024-02-16T22:20:04.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.528+0000 7fa356cf7700 1 --2- 172.21.15.114:0/420840076 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3500ffac0 0x7fa350069910 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:04.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.528+0000 7fa356cf7700 1 --2- 172.21.15.114:0/420840076 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa33c06c4e0 0x7fa33c06e9a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:04.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.528+0000 7fa356cf7700 1 --2- 172.21.15.114:0/420840076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa350100cc0 0x7fa35019b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:04.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.528+0000 7fa356cf7700 1 -- 172.21.15.114:0/420840076 >> 172.21.15.114:0/420840076 conn(0x7fa3500fb060 msgr2=0x7fa350103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:20:04.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.528+0000 7fa356cf7700 1 -- 172.21.15.114:0/420840076 shutdown_connections 2024-02-16T22:20:04.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.528+0000 7fa356cf7700 1 -- 172.21.15.114:0/420840076 wait complete. 2024-02-16T22:20:04.532 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 14 2024-02-16T22:20:04.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.632+0000 7f9338b25700 1 -- 172.21.15.114:0/3917279776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9334100bb0 msgr2=0x7f9334100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:04.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.632+0000 7f9338b25700 1 --2- 172.21.15.114:0/3917279776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9334100bb0 0x7f9334100fd0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9324009910 tx=0x7f9324009c20 comp rx=0 tx=0).stop 2024-02-16T22:20:04.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.632+0000 7f9338b25700 1 -- 172.21.15.114:0/3917279776 shutdown_connections 2024-02-16T22:20:04.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.632+0000 7f9338b25700 1 --2- 172.21.15.114:0/3917279776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9334101db0 0x7f9334102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:04.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.632+0000 7f9338b25700 1 --2- 172.21.15.114:0/3917279776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9334100bb0 0x7f9334100fd0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:04.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.632+0000 7f9338b25700 1 -- 172.21.15.114:0/3917279776 >> 172.21.15.114:0/3917279776 conn(0x7f93340fc110 msgr2=0x7f93340fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:20:04.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.633+0000 7f9338b25700 1 -- 172.21.15.114:0/3917279776 shutdown_connections 2024-02-16T22:20:04.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.633+0000 7f9338b25700 1 -- 172.21.15.114:0/3917279776 wait complete. 2024-02-16T22:20:04.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.633+0000 7f9338b25700 1 Processor -- start 2024-02-16T22:20:04.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.633+0000 7f9338b25700 1 -- start start 2024-02-16T22:20:04.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.633+0000 7f9338b25700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9334100bb0 0x7f9334196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:04.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.633+0000 7f9338b25700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9334101db0 0x7f93341968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:04.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.634+0000 7f9338b25700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9334196ef0 con 0x7f9334101db0 2024-02-16T22:20:04.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.634+0000 7f9338b25700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9334197030 con 0x7f9334100bb0 2024-02-16T22:20:04.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.634+0000 7f9331d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9334101db0 0x7f93341968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:04.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.634+0000 7f933259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9334100bb0 0x7f9334196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:04.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.634+0000 7f9331d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9334101db0 0x7f93341968d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51800/0 (socket says 172.21.15.114:51800) 2024-02-16T22:20:04.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.634+0000 7f9331d9b700 1 -- 172.21.15.114:0/4162870766 learned_addr learned my addr 172.21.15.114:0/4162870766 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:20:04.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.634+0000 7f9331d9b700 1 -- 172.21.15.114:0/4162870766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9334100bb0 msgr2=0x7f9334196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:04.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.634+0000 7f9331d9b700 1 --2- 172.21.15.114:0/4162870766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9334100bb0 0x7f9334196390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:04.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.634+0000 7f9331d9b700 1 -- 172.21.15.114:0/4162870766 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93240095f0 con 0x7f9334101db0 2024-02-16T22:20:04.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.634+0000 7f9331d9b700 1 --2- 172.21.15.114:0/4162870766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9334101db0 0x7f93341968d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f932800d8d0 tx=0x7f932800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:20:04.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.634+0000 7f93237fe700 1 -- 172.21.15.114:0/4162870766 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93280098a0 con 0x7f9334101db0 2024-02-16T22:20:04.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.634+0000 7f9338b25700 1 -- 172.21.15.114:0/4162870766 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f933419bae0 con 0x7f9334101db0 2024-02-16T22:20:04.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.634+0000 7f93237fe700 1 -- 172.21.15.114:0/4162870766 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9328010460 con 0x7f9334101db0 2024-02-16T22:20:04.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.634+0000 7f93237fe700 1 -- 172.21.15.114:0/4162870766 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f932800f5f0 con 0x7f9334101db0 2024-02-16T22:20:04.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.634+0000 7f9338b25700 1 -- 172.21.15.114:0/4162870766 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f933419c030 con 0x7f9334101db0 2024-02-16T22:20:04.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.635+0000 7f9338b25700 1 -- 172.21.15.114:0/4162870766 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9334109710 con 0x7f9334101db0 2024-02-16T22:20:04.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.636+0000 7f93237fe700 1 -- 172.21.15.114:0/4162870766 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f9328009a00 con 0x7f9334101db0 2024-02-16T22:20:04.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.636+0000 7f93237fe700 1 --2- 172.21.15.114:0/4162870766 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f931c06c4e0 0x7f931c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:04.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.636+0000 7f93237fe700 1 -- 172.21.15.114:0/4162870766 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f932808b730 con 0x7f9334101db0 2024-02-16T22:20:04.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.636+0000 7f933259c700 1 --2- 172.21.15.114:0/4162870766 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f931c06c4e0 0x7f931c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:04.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.637+0000 7f933259c700 1 --2- 172.21.15.114:0/4162870766 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f931c06c4e0 0x7f931c06e9a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9324006010 tx=0x7f932400b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:20:04.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.641+0000 7f93237fe700 1 -- 172.21.15.114:0/4162870766 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9328059940 con 0x7f9334101db0 2024-02-16T22:20:04.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:04 smithi123 ceph-mon[37161]: from='client.14624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:20:04.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:04 smithi123 ceph-mon[37161]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:20:04.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:04 smithi123 ceph-mon[37161]: pgmap v239: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:04.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:04 smithi123 ceph-mon[37161]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:20:04.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:04 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/1132250022' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:20:04.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:04 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/420840076' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:20:04.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.848+0000 7f9338b25700 1 -- 172.21.15.114:0/4162870766 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9334106700 con 0x7f931c06c4e0 2024-02-16T22:20:04.851 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:04 smithi114 ceph-mon[29235]: from='client.14624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:20:04.851 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:04 smithi114 ceph-mon[29235]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:20:04.851 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:04 smithi114 ceph-mon[29235]: pgmap v239: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:04.851 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:04 smithi114 ceph-mon[29235]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:20:04.851 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:04 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/1132250022' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:20:04.851 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:04 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/420840076' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:20:04.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.850+0000 7f93237fe700 1 -- 172.21.15.114:0/4162870766 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f9334106700 con 0x7f931c06c4e0 2024-02-16T22:20:04.852 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:20:04.852 INFO:teuthology.orchestra.run.smithi114.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:20:04.852 INFO:teuthology.orchestra.run.smithi114.stdout: "in_progress": true, 2024-02-16T22:20:04.852 INFO:teuthology.orchestra.run.smithi114.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:20:04.852 INFO:teuthology.orchestra.run.smithi114.stdout: "services_complete": [], 2024-02-16T22:20:04.852 INFO:teuthology.orchestra.run.smithi114.stdout: "progress": "0/23 daemons upgraded", 2024-02-16T22:20:04.852 INFO:teuthology.orchestra.run.smithi114.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 image on host smithi123", 2024-02-16T22:20:04.852 INFO:teuthology.orchestra.run.smithi114.stdout: "is_paused": false 2024-02-16T22:20:04.852 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:20:04.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.852+0000 7f9338b25700 1 -- 172.21.15.114:0/4162870766 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f931c06c4e0 msgr2=0x7f931c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:04.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.852+0000 7f9338b25700 1 --2- 172.21.15.114:0/4162870766 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f931c06c4e0 0x7f931c06e9a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9324006010 tx=0x7f932400b540 comp rx=0 tx=0).stop 2024-02-16T22:20:04.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.852+0000 7f9338b25700 1 -- 172.21.15.114:0/4162870766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9334101db0 msgr2=0x7f93341968d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:04.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.852+0000 7f9338b25700 1 --2- 172.21.15.114:0/4162870766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9334101db0 0x7f93341968d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f932800d8d0 tx=0x7f932800dbe0 comp rx=0 tx=0).stop 2024-02-16T22:20:04.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.852+0000 7f9338b25700 1 -- 172.21.15.114:0/4162870766 shutdown_connections 2024-02-16T22:20:04.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.852+0000 7f9338b25700 1 --2- 172.21.15.114:0/4162870766 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f931c06c4e0 0x7f931c06e9a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:04.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.852+0000 7f9338b25700 1 --2- 172.21.15.114:0/4162870766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9334100bb0 0x7f9334196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:04.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.852+0000 7f9338b25700 1 --2- 172.21.15.114:0/4162870766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9334101db0 0x7f93341968d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:04.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.852+0000 7f9338b25700 1 -- 172.21.15.114:0/4162870766 >> 172.21.15.114:0/4162870766 conn(0x7f93340fc110 msgr2=0x7f9334104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:20:04.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.852+0000 7f9338b25700 1 -- 172.21.15.114:0/4162870766 shutdown_connections 2024-02-16T22:20:04.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.852+0000 7f9338b25700 1 -- 172.21.15.114:0/4162870766 wait complete. 2024-02-16T22:20:04.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.956+0000 7ff9727fc700 1 -- 172.21.15.114:0/2395436492 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff96c007b00 con 0x7ff9741016e0 2024-02-16T22:20:04.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.956+0000 7ff979ed0700 1 -- 172.21.15.114:0/2395436492 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9741016e0 msgr2=0x7ff974103ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:04.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.956+0000 7ff979ed0700 1 --2- 172.21.15.114:0/2395436492 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9741016e0 0x7ff974103ad0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00b0a0 tx=0x7ff96c00b3b0 comp rx=0 tx=0).stop 2024-02-16T22:20:04.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.956+0000 7ff979ed0700 1 -- 172.21.15.114:0/2395436492 shutdown_connections 2024-02-16T22:20:04.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.956+0000 7ff979ed0700 1 --2- 172.21.15.114:0/2395436492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff974104010 0x7ff974106400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:04.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.956+0000 7ff979ed0700 1 --2- 172.21.15.114:0/2395436492 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9741016e0 0x7ff974103ad0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:04.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.956+0000 7ff979ed0700 1 -- 172.21.15.114:0/2395436492 >> 172.21.15.114:0/2395436492 conn(0x7ff9740fb030 msgr2=0x7ff9740fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:20:04.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.956+0000 7ff979ed0700 1 -- 172.21.15.114:0/2395436492 shutdown_connections 2024-02-16T22:20:04.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.956+0000 7ff979ed0700 1 -- 172.21.15.114:0/2395436492 wait complete. 2024-02-16T22:20:04.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.956+0000 7ff979ed0700 1 Processor -- start 2024-02-16T22:20:04.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.957+0000 7ff979ed0700 1 -- start start 2024-02-16T22:20:04.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.957+0000 7ff979ed0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9741016e0 0x7ff974101040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:04.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.957+0000 7ff979ed0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff974104010 0x7ff9740ff690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:04.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.957+0000 7ff979ed0700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9740ffbd0 con 0x7ff9741016e0 2024-02-16T22:20:04.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.957+0000 7ff979ed0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9740ffd10 con 0x7ff974104010 2024-02-16T22:20:04.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.957+0000 7ff9737fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9741016e0 0x7ff974101040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:04.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.957+0000 7ff9737fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9741016e0 0x7ff974101040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51804/0 (socket says 172.21.15.114:51804) 2024-02-16T22:20:04.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.957+0000 7ff9737fe700 1 -- 172.21.15.114:0/1376185016 learned_addr learned my addr 172.21.15.114:0/1376185016 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:20:04.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.957+0000 7ff972ffd700 1 --2- 172.21.15.114:0/1376185016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff974104010 0x7ff9740ff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:04.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.957+0000 7ff9737fe700 1 -- 172.21.15.114:0/1376185016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff974104010 msgr2=0x7ff9740ff690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:04.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.957+0000 7ff9737fe700 1 --2- 172.21.15.114:0/1376185016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff974104010 0x7ff9740ff690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:04.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.957+0000 7ff9737fe700 1 -- 172.21.15.114:0/1376185016 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff96c009d00 con 0x7ff9741016e0 2024-02-16T22:20:04.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.957+0000 7ff9737fe700 1 --2- 172.21.15.114:0/1376185016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9741016e0 0x7ff974101040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00d010 tx=0x7ff96c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:20:04.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.958+0000 7ff970ff9700 1 -- 172.21.15.114:0/1376185016 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff96c00e040 con 0x7ff9741016e0 2024-02-16T22:20:04.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.958+0000 7ff970ff9700 1 -- 172.21.15.114:0/1376185016 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff96c003de0 con 0x7ff9741016e0 2024-02-16T22:20:04.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.958+0000 7ff979ed0700 1 -- 172.21.15.114:0/1376185016 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9740fff90 con 0x7ff9741016e0 2024-02-16T22:20:04.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.958+0000 7ff970ff9700 1 -- 172.21.15.114:0/1376185016 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff96c01d840 con 0x7ff9741016e0 2024-02-16T22:20:04.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.958+0000 7ff979ed0700 1 -- 172.21.15.114:0/1376185016 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff974100480 con 0x7ff9741016e0 2024-02-16T22:20:04.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.959+0000 7ff979ed0700 1 -- 172.21.15.114:0/1376185016 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff974190530 con 0x7ff9741016e0 2024-02-16T22:20:04.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.959+0000 7ff970ff9700 1 -- 172.21.15.114:0/1376185016 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7ff96c004810 con 0x7ff9741016e0 2024-02-16T22:20:04.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.960+0000 7ff970ff9700 1 --2- 172.21.15.114:0/1376185016 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff96006c4e0 0x7ff96006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:04.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.960+0000 7ff972ffd700 1 --2- 172.21.15.114:0/1376185016 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff96006c4e0 0x7ff96006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:04.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.960+0000 7ff970ff9700 1 -- 172.21.15.114:0/1376185016 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7ff96c08d1d0 con 0x7ff9741016e0 2024-02-16T22:20:04.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.960+0000 7ff972ffd700 1 --2- 172.21.15.114:0/1376185016 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff96006c4e0 0x7ff96006e9a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009c60 tx=0x7ff95c009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:20:04.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:04.964+0000 7ff970ff9700 1 -- 172.21.15.114:0/1376185016 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff96c05b440 con 0x7ff9741016e0 2024-02-16T22:20:05.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:05.240+0000 7ff979ed0700 1 -- 172.21.15.114:0/1376185016 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff97402cc70 con 0x7ff9741016e0 2024-02-16T22:20:05.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:05.240+0000 7ff970ff9700 1 -- 172.21.15.114:0/1376185016 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7ff96c05afd0 con 0x7ff9741016e0 2024-02-16T22:20:05.242 INFO:teuthology.orchestra.run.smithi114.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:20:05.242 INFO:teuthology.orchestra.run.smithi114.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:20:05.242 INFO:teuthology.orchestra.run.smithi114.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:20:05.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:05.242+0000 7ff979ed0700 1 -- 172.21.15.114:0/1376185016 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff96006c4e0 msgr2=0x7ff96006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:05.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:05.242+0000 7ff979ed0700 1 --2- 172.21.15.114:0/1376185016 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff96006c4e0 0x7ff96006e9a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009c60 tx=0x7ff95c009260 comp rx=0 tx=0).stop 2024-02-16T22:20:05.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:05.242+0000 7ff979ed0700 1 -- 172.21.15.114:0/1376185016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9741016e0 msgr2=0x7ff974101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:05.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:05.242+0000 7ff979ed0700 1 --2- 172.21.15.114:0/1376185016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9741016e0 0x7ff974101040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00d010 tx=0x7ff96c009750 comp rx=0 tx=0).stop 2024-02-16T22:20:05.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:05.242+0000 7ff979ed0700 1 -- 172.21.15.114:0/1376185016 shutdown_connections 2024-02-16T22:20:05.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:05.242+0000 7ff979ed0700 1 --2- 172.21.15.114:0/1376185016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9741016e0 0x7ff974101040 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:05.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:05.242+0000 7ff979ed0700 1 --2- 172.21.15.114:0/1376185016 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff96006c4e0 0x7ff96006e9a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:05.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:05.242+0000 7ff979ed0700 1 --2- 172.21.15.114:0/1376185016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff974104010 0x7ff9740ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:05.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:05.242+0000 7ff979ed0700 1 -- 172.21.15.114:0/1376185016 >> 172.21.15.114:0/1376185016 conn(0x7ff9740fb030 msgr2=0x7ff9740fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:20:05.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:05.242+0000 7ff979ed0700 1 -- 172.21.15.114:0/1376185016 shutdown_connections 2024-02-16T22:20:05.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:05.242+0000 7ff979ed0700 1 -- 172.21.15.114:0/1376185016 wait complete. 2024-02-16T22:20:05.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:05 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/1376185016' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:20:05.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:05 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/1376185016' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:20:06.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:06 smithi123 ceph-mon[37161]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:20:06.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:06 smithi123 ceph-mon[37161]: pgmap v240: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:06.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:06 smithi114 ceph-mon[29235]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:20:06.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:06 smithi114 ceph-mon[29235]: pgmap v240: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:08.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:07 smithi114 ceph-mon[29235]: pgmap v241: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:08.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:07 smithi123 ceph-mon[37161]: pgmap v241: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:10.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:10 smithi123 ceph-mon[37161]: pgmap v242: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:10.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:10 smithi114 ceph-mon[29235]: pgmap v242: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:11.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:11 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:20:11.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:11 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:20:12.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:12 smithi123 ceph-mon[37161]: pgmap v243: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:12.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:12 smithi114 ceph-mon[29235]: pgmap v243: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:14.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:14 smithi123 ceph-mon[37161]: pgmap v244: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:14.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:14 smithi114 ceph-mon[29235]: pgmap v244: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:16.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:16 smithi123 ceph-mon[37161]: pgmap v245: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:16.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:16 smithi114 ceph-mon[29235]: pgmap v245: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:18.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:17 smithi114 ceph-mon[29235]: pgmap v246: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:18.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:17 smithi123 ceph-mon[37161]: pgmap v246: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:20.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:20 smithi123 ceph-mon[37161]: pgmap v247: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:20.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:20 smithi114 ceph-mon[29235]: pgmap v247: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:22.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:22 smithi123 ceph-mon[37161]: pgmap v248: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:22.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:22 smithi114 ceph-mon[29235]: pgmap v248: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:24.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:24 smithi123 ceph-mon[37161]: pgmap v249: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:24.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:24 smithi114 ceph-mon[29235]: pgmap v249: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:26.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:26 smithi123 ceph-mon[37161]: pgmap v250: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:26.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:26 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:20:26.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:26 smithi114 ceph-mon[29235]: pgmap v250: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:26.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:26 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:20:28.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:27 smithi114 ceph-mon[29235]: pgmap v251: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:28.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:27 smithi123 ceph-mon[37161]: pgmap v251: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:30.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:30 smithi123 ceph-mon[37161]: pgmap v252: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:30.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:30 smithi114 ceph-mon[29235]: pgmap v252: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:32.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:32 smithi114 ceph-mon[29235]: pgmap v253: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:32.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:32 smithi123 ceph-mon[37161]: pgmap v253: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:34.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:34 smithi123 ceph-mon[37161]: pgmap v254: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:34.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:34 smithi114 ceph-mon[29235]: pgmap v254: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:35.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.381+0000 7f3d0dd22700 1 -- 172.21.15.114:0/2570309326 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d08100cb0 msgr2=0x7f3d08101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:35.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.381+0000 7f3d0dd22700 1 --2- 172.21.15.114:0/2570309326 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d08100cb0 0x7f3d08101110 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc009910 tx=0x7f3cfc009c20 comp rx=0 tx=0).stop 2024-02-16T22:20:35.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.381+0000 7f3d0dd22700 1 -- 172.21.15.114:0/2570309326 shutdown_connections 2024-02-16T22:20:35.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.381+0000 7f3d0dd22700 1 --2- 172.21.15.114:0/2570309326 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d08100cb0 0x7f3d08101110 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:35.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.381+0000 7f3d0dd22700 1 --2- 172.21.15.114:0/2570309326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d080ffab0 0x7f3d080ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:35.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.381+0000 7f3d0dd22700 1 -- 172.21.15.114:0/2570309326 >> 172.21.15.114:0/2570309326 conn(0x7f3d080fb030 msgr2=0x7f3d080fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:20:35.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.381+0000 7f3d0dd22700 1 -- 172.21.15.114:0/2570309326 shutdown_connections 2024-02-16T22:20:35.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.381+0000 7f3d0dd22700 1 -- 172.21.15.114:0/2570309326 wait complete. 2024-02-16T22:20:35.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.382+0000 7f3d0dd22700 1 Processor -- start 2024-02-16T22:20:35.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.382+0000 7f3d0dd22700 1 -- start start 2024-02-16T22:20:35.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.382+0000 7f3d0dd22700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d080ffab0 0x7f3d08196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:35.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.382+0000 7f3d0dd22700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d08100cb0 0x7f3d08196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:35.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.382+0000 7f3d0dd22700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d08196ea0 con 0x7f3d08100cb0 2024-02-16T22:20:35.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.382+0000 7f3d0dd22700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d08196fe0 con 0x7f3d080ffab0 2024-02-16T22:20:35.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.382+0000 7f3d077fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d080ffab0 0x7f3d08196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:35.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.382+0000 7f3d077fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d080ffab0 0x7f3d08196340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.114:40872/0 (socket says 172.21.15.114:40872) 2024-02-16T22:20:35.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.382+0000 7f3d077fe700 1 -- 172.21.15.114:0/2890222722 learned_addr learned my addr 172.21.15.114:0/2890222722 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:20:35.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.382+0000 7f3d06ffd700 1 --2- 172.21.15.114:0/2890222722 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d08100cb0 0x7f3d08196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:35.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.383+0000 7f3d06ffd700 1 -- 172.21.15.114:0/2890222722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d080ffab0 msgr2=0x7f3d08196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:35.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.383+0000 7f3d06ffd700 1 --2- 172.21.15.114:0/2890222722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d080ffab0 0x7f3d08196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:35.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.383+0000 7f3d06ffd700 1 -- 172.21.15.114:0/2890222722 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cfc0095f0 con 0x7f3d08100cb0 2024-02-16T22:20:35.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.383+0000 7f3d06ffd700 1 --2- 172.21.15.114:0/2890222722 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d08100cb0 0x7f3d08196880 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc000c00 tx=0x7f3cfc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:20:35.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.383+0000 7f3d04ff9700 1 -- 172.21.15.114:0/2890222722 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cfc01d070 con 0x7f3d08100cb0 2024-02-16T22:20:35.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.383+0000 7f3d0dd22700 1 -- 172.21.15.114:0/2890222722 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d0819ba30 con 0x7f3d08100cb0 2024-02-16T22:20:35.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.383+0000 7f3d0dd22700 1 -- 172.21.15.114:0/2890222722 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d0819bf20 con 0x7f3d08100cb0 2024-02-16T22:20:35.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.383+0000 7f3d04ff9700 1 -- 172.21.15.114:0/2890222722 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3cfc005dc0 con 0x7f3d08100cb0 2024-02-16T22:20:35.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.383+0000 7f3d04ff9700 1 -- 172.21.15.114:0/2890222722 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cfc017400 con 0x7f3d08100cb0 2024-02-16T22:20:35.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.384+0000 7f3d0dd22700 1 -- 172.21.15.114:0/2890222722 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d08108610 con 0x7f3d08100cb0 2024-02-16T22:20:35.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.385+0000 7f3d04ff9700 1 -- 172.21.15.114:0/2890222722 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f3cfc003810 con 0x7f3d08100cb0 2024-02-16T22:20:35.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.385+0000 7f3d04ff9700 1 --2- 172.21.15.114:0/2890222722 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f3cf006c530 0x7f3cf006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:35.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.386+0000 7f3d077fe700 1 --2- 172.21.15.114:0/2890222722 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f3cf006c530 0x7f3cf006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:35.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.386+0000 7f3d04ff9700 1 -- 172.21.15.114:0/2890222722 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f3cfc08cc10 con 0x7f3d08100cb0 2024-02-16T22:20:35.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.386+0000 7f3d077fe700 1 --2- 172.21.15.114:0/2890222722 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f3cf006c530 0x7f3cf006e9f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8005950 tx=0x7f3cf8009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:20:35.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.393+0000 7f3d04ff9700 1 -- 172.21.15.114:0/2890222722 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3cfc05ae60 con 0x7f3d08100cb0 2024-02-16T22:20:35.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.800+0000 7f3d0dd22700 1 -- 172.21.15.114:0/2890222722 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d0819c670 con 0x7f3cf006c530 2024-02-16T22:20:35.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.802+0000 7f3d04ff9700 1 -- 172.21.15.114:0/2890222722 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f3d0819c670 con 0x7f3cf006c530 2024-02-16T22:20:35.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.805+0000 7f3d0dd22700 1 -- 172.21.15.114:0/2890222722 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f3cf006c530 msgr2=0x7f3cf006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:35.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.805+0000 7f3d0dd22700 1 --2- 172.21.15.114:0/2890222722 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f3cf006c530 0x7f3cf006e9f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8005950 tx=0x7f3cf8009260 comp rx=0 tx=0).stop 2024-02-16T22:20:35.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.805+0000 7f3d0dd22700 1 -- 172.21.15.114:0/2890222722 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d08100cb0 msgr2=0x7f3d08196880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:35.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.805+0000 7f3d0dd22700 1 --2- 172.21.15.114:0/2890222722 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d08100cb0 0x7f3d08196880 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc000c00 tx=0x7f3cfc00f690 comp rx=0 tx=0).stop 2024-02-16T22:20:35.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.805+0000 7f3d0dd22700 1 -- 172.21.15.114:0/2890222722 shutdown_connections 2024-02-16T22:20:35.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.805+0000 7f3d0dd22700 1 --2- 172.21.15.114:0/2890222722 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f3cf006c530 0x7f3cf006e9f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:35.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.805+0000 7f3d0dd22700 1 --2- 172.21.15.114:0/2890222722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d080ffab0 0x7f3d08196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:35.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.805+0000 7f3d0dd22700 1 --2- 172.21.15.114:0/2890222722 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d08100cb0 0x7f3d08196880 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:35.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.805+0000 7f3d0dd22700 1 -- 172.21.15.114:0/2890222722 >> 172.21.15.114:0/2890222722 conn(0x7f3d080fb030 msgr2=0x7f3d08068410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:20:35.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.805+0000 7f3d0dd22700 1 -- 172.21.15.114:0/2890222722 shutdown_connections 2024-02-16T22:20:35.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.805+0000 7f3d0dd22700 1 -- 172.21.15.114:0/2890222722 wait complete. 2024-02-16T22:20:35.817 INFO:teuthology.orchestra.run.smithi114.stdout:true 2024-02-16T22:20:35.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.961+0000 7fb9709a9700 1 -- 172.21.15.114:0/3051447179 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb96c100cb0 msgr2=0x7fb96c101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:35.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.961+0000 7fb9709a9700 1 --2- 172.21.15.114:0/3051447179 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb96c100cb0 0x7fb96c101110 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb960009910 tx=0x7fb960009c20 comp rx=0 tx=0).stop 2024-02-16T22:20:35.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.961+0000 7fb9709a9700 1 -- 172.21.15.114:0/3051447179 shutdown_connections 2024-02-16T22:20:35.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.961+0000 7fb9709a9700 1 --2- 172.21.15.114:0/3051447179 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb96c100cb0 0x7fb96c101110 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:35.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.961+0000 7fb9709a9700 1 --2- 172.21.15.114:0/3051447179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb96c0ffab0 0x7fb96c0ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:35.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.961+0000 7fb9709a9700 1 -- 172.21.15.114:0/3051447179 >> 172.21.15.114:0/3051447179 conn(0x7fb96c0fb030 msgr2=0x7fb96c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:20:35.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.961+0000 7fb9709a9700 1 -- 172.21.15.114:0/3051447179 shutdown_connections 2024-02-16T22:20:35.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.961+0000 7fb9709a9700 1 -- 172.21.15.114:0/3051447179 wait complete. 2024-02-16T22:20:35.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.962+0000 7fb9709a9700 1 Processor -- start 2024-02-16T22:20:35.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.962+0000 7fb9709a9700 1 -- start start 2024-02-16T22:20:35.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.962+0000 7fb9709a9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb96c0ffab0 0x7fb96c068f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:35.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.962+0000 7fb9709a9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb96c100cb0 0x7fb96c19b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:35.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.962+0000 7fb9709a9700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c19b670 con 0x7fb96c0ffab0 2024-02-16T22:20:35.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.962+0000 7fb9709a9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c19b7e0 con 0x7fb96c100cb0 2024-02-16T22:20:35.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.962+0000 7fb96a59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb96c0ffab0 0x7fb96c068f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:35.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.962+0000 7fb96a59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb96c0ffab0 0x7fb96c068f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60728/0 (socket says 172.21.15.114:60728) 2024-02-16T22:20:35.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.962+0000 7fb96a59c700 1 -- 172.21.15.114:0/3350467438 learned_addr learned my addr 172.21.15.114:0/3350467438 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:20:35.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.962+0000 7fb969d9b700 1 --2- 172.21.15.114:0/3350467438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb96c100cb0 0x7fb96c19b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:35.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.963+0000 7fb96a59c700 1 -- 172.21.15.114:0/3350467438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb96c100cb0 msgr2=0x7fb96c19b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:35.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.963+0000 7fb96a59c700 1 --2- 172.21.15.114:0/3350467438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb96c100cb0 0x7fb96c19b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:35.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.963+0000 7fb96a59c700 1 -- 172.21.15.114:0/3350467438 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9600095f0 con 0x7fb96c0ffab0 2024-02-16T22:20:35.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.963+0000 7fb96a59c700 1 --2- 172.21.15.114:0/3350467438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb96c0ffab0 0x7fb96c068f90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb95c00e8f0 tx=0x7fb95c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:20:35.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.963+0000 7fb95b7fe700 1 -- 172.21.15.114:0/3350467438 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb95c009790 con 0x7fb96c0ffab0 2024-02-16T22:20:35.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.963+0000 7fb95b7fe700 1 -- 172.21.15.114:0/3350467438 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb95c00c810 con 0x7fb96c0ffab0 2024-02-16T22:20:35.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.963+0000 7fb9709a9700 1 -- 172.21.15.114:0/3350467438 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb96c19bac0 con 0x7fb96c0ffab0 2024-02-16T22:20:35.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.963+0000 7fb95b7fe700 1 -- 172.21.15.114:0/3350467438 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb95c003d20 con 0x7fb96c0ffab0 2024-02-16T22:20:35.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.963+0000 7fb9709a9700 1 -- 172.21.15.114:0/3350467438 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb96c19bf90 con 0x7fb96c0ffab0 2024-02-16T22:20:35.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.964+0000 7fb9709a9700 1 -- 172.21.15.114:0/3350467438 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb96c108610 con 0x7fb96c0ffab0 2024-02-16T22:20:35.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.965+0000 7fb95b7fe700 1 -- 172.21.15.114:0/3350467438 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fb95c0041f0 con 0x7fb96c0ffab0 2024-02-16T22:20:35.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.966+0000 7fb95b7fe700 1 --2- 172.21.15.114:0/3350467438 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fb95406c530 0x7fb95406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:35.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.966+0000 7fb95b7fe700 1 -- 172.21.15.114:0/3350467438 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fb95c014070 con 0x7fb96c0ffab0 2024-02-16T22:20:35.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.966+0000 7fb969d9b700 1 --2- 172.21.15.114:0/3350467438 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fb95406c530 0x7fb95406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:35.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.966+0000 7fb969d9b700 1 --2- 172.21.15.114:0/3350467438 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fb95406c530 0x7fb95406e9f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb960009fd0 tx=0x7fb960005940 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:20:35.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:35.970+0000 7fb95b7fe700 1 -- 172.21.15.114:0/3350467438 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb95c059d70 con 0x7fb96c0ffab0 2024-02-16T22:20:36.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.262+0000 7fb9709a9700 1 -- 172.21.15.114:0/3350467438 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb96c105600 con 0x7fb95406c530 2024-02-16T22:20:36.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.264+0000 7fb95b7fe700 1 -- 172.21.15.114:0/3350467438 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7fb96c105600 con 0x7fb95406c530 2024-02-16T22:20:36.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.266+0000 7fb9709a9700 1 -- 172.21.15.114:0/3350467438 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fb95406c530 msgr2=0x7fb95406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:36.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.266+0000 7fb9709a9700 1 --2- 172.21.15.114:0/3350467438 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fb95406c530 0x7fb95406e9f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb960009fd0 tx=0x7fb960005940 comp rx=0 tx=0).stop 2024-02-16T22:20:36.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.266+0000 7fb9709a9700 1 -- 172.21.15.114:0/3350467438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb96c0ffab0 msgr2=0x7fb96c068f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:36.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.266+0000 7fb9709a9700 1 --2- 172.21.15.114:0/3350467438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb96c0ffab0 0x7fb96c068f90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb95c00e8f0 tx=0x7fb95c00ec00 comp rx=0 tx=0).stop 2024-02-16T22:20:36.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.266+0000 7fb9709a9700 1 -- 172.21.15.114:0/3350467438 shutdown_connections 2024-02-16T22:20:36.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.266+0000 7fb9709a9700 1 --2- 172.21.15.114:0/3350467438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb96c0ffab0 0x7fb96c068f90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:36.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.266+0000 7fb9709a9700 1 --2- 172.21.15.114:0/3350467438 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fb95406c530 0x7fb95406e9f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:36.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.266+0000 7fb9709a9700 1 --2- 172.21.15.114:0/3350467438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb96c100cb0 0x7fb96c19b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:36.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.266+0000 7fb9709a9700 1 -- 172.21.15.114:0/3350467438 >> 172.21.15.114:0/3350467438 conn(0x7fb96c0fb030 msgr2=0x7fb96c103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:20:36.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.267+0000 7fb9709a9700 1 -- 172.21.15.114:0/3350467438 shutdown_connections 2024-02-16T22:20:36.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.267+0000 7fb9709a9700 1 -- 172.21.15.114:0/3350467438 wait complete. 2024-02-16T22:20:36.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.428+0000 7f195cf85700 1 -- 172.21.15.114:0/2782900786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1958100cb0 msgr2=0x7f1958101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:36.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.428+0000 7f195cf85700 1 --2- 172.21.15.114:0/2782900786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1958100cb0 0x7f1958101110 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f194c009910 tx=0x7f194c009c20 comp rx=0 tx=0).stop 2024-02-16T22:20:36.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.428+0000 7f195cf85700 1 -- 172.21.15.114:0/2782900786 shutdown_connections 2024-02-16T22:20:36.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.428+0000 7f195cf85700 1 --2- 172.21.15.114:0/2782900786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1958100cb0 0x7f1958101110 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:36.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.428+0000 7f195cf85700 1 --2- 172.21.15.114:0/2782900786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19580ffab0 0x7f19580ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:36.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.428+0000 7f195cf85700 1 -- 172.21.15.114:0/2782900786 >> 172.21.15.114:0/2782900786 conn(0x7f19580fb030 msgr2=0x7f19580fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:20:36.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.428+0000 7f195cf85700 1 -- 172.21.15.114:0/2782900786 shutdown_connections 2024-02-16T22:20:36.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.428+0000 7f195cf85700 1 -- 172.21.15.114:0/2782900786 wait complete. 2024-02-16T22:20:36.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.429+0000 7f195cf85700 1 Processor -- start 2024-02-16T22:20:36.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.429+0000 7f195cf85700 1 -- start start 2024-02-16T22:20:36.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.429+0000 7f195cf85700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19580ffab0 0x7f1958196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:36.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.429+0000 7f195cf85700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1958196ad0 0x7f195819bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:36.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.429+0000 7f195cf85700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1958196fe0 con 0x7f1958196ad0 2024-02-16T22:20:36.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.429+0000 7f195cf85700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1958197150 con 0x7f19580ffab0 2024-02-16T22:20:36.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.429+0000 7f195659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19580ffab0 0x7f1958196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:36.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.429+0000 7f195659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19580ffab0 0x7f1958196590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.114:40912/0 (socket says 172.21.15.114:40912) 2024-02-16T22:20:36.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.429+0000 7f195659c700 1 -- 172.21.15.114:0/3616018568 learned_addr learned my addr 172.21.15.114:0/3616018568 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:20:36.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.429+0000 7f1955d9b700 1 --2- 172.21.15.114:0/3616018568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1958196ad0 0x7f195819bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:36.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.430+0000 7f1955d9b700 1 -- 172.21.15.114:0/3616018568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19580ffab0 msgr2=0x7f1958196590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:36.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.430+0000 7f1955d9b700 1 --2- 172.21.15.114:0/3616018568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19580ffab0 0x7f1958196590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:36.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.430+0000 7f1955d9b700 1 -- 172.21.15.114:0/3616018568 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c0095f0 con 0x7f1958196ad0 2024-02-16T22:20:36.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.430+0000 7f1955d9b700 1 --2- 172.21.15.114:0/3616018568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1958196ad0 0x7f195819bb40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f194c008000 tx=0x7f194c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:20:36.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.430+0000 7f19477fe700 1 -- 172.21.15.114:0/3616018568 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f194c01d070 con 0x7f1958196ad0 2024-02-16T22:20:36.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.430+0000 7f195cf85700 1 -- 172.21.15.114:0/3616018568 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f195819c080 con 0x7f1958196ad0 2024-02-16T22:20:36.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.430+0000 7f19477fe700 1 -- 172.21.15.114:0/3616018568 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f194c005640 con 0x7f1958196ad0 2024-02-16T22:20:36.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.431+0000 7f19477fe700 1 -- 172.21.15.114:0/3616018568 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f194c022470 con 0x7f1958196ad0 2024-02-16T22:20:36.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.431+0000 7f195cf85700 1 -- 172.21.15.114:0/3616018568 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f195819c570 con 0x7f1958196ad0 2024-02-16T22:20:36.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.432+0000 7f195cf85700 1 -- 172.21.15.114:0/3616018568 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1958108610 con 0x7f1958196ad0 2024-02-16T22:20:36.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.433+0000 7f19477fe700 1 -- 172.21.15.114:0/3616018568 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f194c003680 con 0x7f1958196ad0 2024-02-16T22:20:36.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.433+0000 7f19477fe700 1 --2- 172.21.15.114:0/3616018568 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f194006c4e0 0x7f194006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:36.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.433+0000 7f19477fe700 1 -- 172.21.15.114:0/3616018568 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f194c08c9e0 con 0x7f1958196ad0 2024-02-16T22:20:36.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.433+0000 7f195659c700 1 --2- 172.21.15.114:0/3616018568 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f194006c4e0 0x7f194006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:36.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.434+0000 7f195659c700 1 --2- 172.21.15.114:0/3616018568 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f194006c4e0 0x7f194006e9a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1948005950 tx=0x7f1948009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:20:36.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.438+0000 7f19477fe700 1 -- 172.21.15.114:0/3616018568 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f194c05ac50 con 0x7f1958196ad0 2024-02-16T22:20:36.554 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:36 smithi114 ceph-mon[29235]: pgmap v255: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:36.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.717+0000 7f195cf85700 1 -- 172.21.15.114:0/3616018568 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1958068b20 con 0x7f194006c4e0 2024-02-16T22:20:36.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.735+0000 7f19477fe700 1 -- 172.21.15.114:0/3616018568 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3456 (secure 0 0 0) 0x7f1958068b20 con 0x7f194006c4e0 2024-02-16T22:20:36.737 INFO:teuthology.orchestra.run.smithi114.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:20:36.737 INFO:teuthology.orchestra.run.smithi114.stdout:alertmanager.smithi114 smithi114 *:9093,9094 running (6m) 2m ago 7m 24.4M - 0.25.0 c8568f914cd2 e7404fea400e 2024-02-16T22:20:36.737 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi114 smithi114 running (8m) 2m ago 8m 7440k - 18.2.1 94eadd990e0a 7fd07162cd96 2024-02-16T22:20:36.737 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi123 smithi123 running (7m) 2m ago 7m 7168k - 18.2.1 94eadd990e0a 7687f5e99621 2024-02-16T22:20:36.737 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi114 smithi114 running (8m) 2m ago 8m 6660k - 18.2.1 94eadd990e0a fc8fbfe4fdd5 2024-02-16T22:20:36.737 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi123 smithi123 running (7m) 2m ago 7m 6665k - 18.2.1 94eadd990e0a a6249642b9bb 2024-02-16T22:20:36.738 INFO:teuthology.orchestra.run.smithi114.stdout:grafana.smithi114 smithi114 *:3000 running (6m) 2m ago 7m 88.1M - 9.4.7 954c08fa6188 db1c26c3bc20 2024-02-16T22:20:36.738 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.fdxpey smithi114 running (2m) 2m ago 2m 13.2M - 18.2.1 94eadd990e0a 704b40c144fb 2024-02-16T22:20:36.738 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.zzryke smithi114 running (2m) 2m ago 2m 16.4M - 18.2.1 94eadd990e0a 1f795d23c9a4 2024-02-16T22:20:36.738 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.gopcfa smithi123 running (2m) 2m ago 2m 13.2M - 18.2.1 94eadd990e0a 166134b7532a 2024-02-16T22:20:36.738 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.vhlqbe smithi123 running (2m) 2m ago 2m 14.7M - 18.2.1 94eadd990e0a 243c8bb96fa8 2024-02-16T22:20:36.738 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi114.vcullz smithi114 *:9283,8765,8443 running (9m) 2m ago 9m 488M - 18.2.1 94eadd990e0a c8f21783728d 2024-02-16T22:20:36.738 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi123.inzgjd smithi123 *:8443,9283,8765 running (7m) 2m ago 7m 443M - 18.2.1 94eadd990e0a 7f193d913aba 2024-02-16T22:20:36.738 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi114 smithi114 running (9m) 2m ago 9m 49.9M 2048M 18.2.1 94eadd990e0a b4c4433aa3a4 2024-02-16T22:20:36.738 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi123 smithi123 running (6m) 2m ago 6m 34.9M 2048M 18.2.1 94eadd990e0a c217fc30c152 2024-02-16T22:20:36.738 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi114 smithi114 *:9100 running (8m) 2m ago 8m 15.6M - 1.5.0 0da6a335fe13 744a090e06d0 2024-02-16T22:20:36.738 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi123 smithi123 *:9100 running (7m) 2m ago 7m 15.7M - 1.5.0 0da6a335fe13 3994b8c29b2e 2024-02-16T22:20:36.738 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 smithi114 running (6m) 2m ago 6m 47.7M 3946M 18.2.1 94eadd990e0a 24ca2e9714e0 2024-02-16T22:20:36.738 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 smithi114 running (5m) 2m ago 5m 47.8M 3946M 18.2.1 94eadd990e0a 32bb191b88e7 2024-02-16T22:20:36.738 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 smithi114 running (5m) 2m ago 5m 48.1M 3946M 18.2.1 94eadd990e0a 6608718d0818 2024-02-16T22:20:36.739 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 smithi123 running (4m) 2m ago 4m 48.8M 4970M 18.2.1 94eadd990e0a 575db379de3d 2024-02-16T22:20:36.739 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 smithi123 running (4m) 2m ago 4m 43.3M 4970M 18.2.1 94eadd990e0a 7e266866cd9c 2024-02-16T22:20:36.739 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 smithi123 running (3m) 2m ago 3m 42.1M 4970M 18.2.1 94eadd990e0a 24e5bef19fda 2024-02-16T22:20:36.739 INFO:teuthology.orchestra.run.smithi114.stdout:prometheus.smithi114 smithi114 *:9095 running (6m) 2m ago 7m 55.9M - 2.43.0 a07b618ecd1d 51185086d582 2024-02-16T22:20:36.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.737+0000 7f195cf85700 1 -- 172.21.15.114:0/3616018568 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f194006c4e0 msgr2=0x7f194006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:36.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.737+0000 7f195cf85700 1 --2- 172.21.15.114:0/3616018568 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f194006c4e0 0x7f194006e9a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1948005950 tx=0x7f1948009260 comp rx=0 tx=0).stop 2024-02-16T22:20:36.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.737+0000 7f195cf85700 1 -- 172.21.15.114:0/3616018568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1958196ad0 msgr2=0x7f195819bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:36.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.737+0000 7f195cf85700 1 --2- 172.21.15.114:0/3616018568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1958196ad0 0x7f195819bb40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f194c008000 tx=0x7f194c00fa00 comp rx=0 tx=0).stop 2024-02-16T22:20:36.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.738+0000 7f195cf85700 1 -- 172.21.15.114:0/3616018568 shutdown_connections 2024-02-16T22:20:36.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.738+0000 7f195cf85700 1 --2- 172.21.15.114:0/3616018568 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f194006c4e0 0x7f194006e9a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:36.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.738+0000 7f195cf85700 1 --2- 172.21.15.114:0/3616018568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19580ffab0 0x7f1958196590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:36.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.738+0000 7f195cf85700 1 --2- 172.21.15.114:0/3616018568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1958196ad0 0x7f195819bb40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:36.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.738+0000 7f195cf85700 1 -- 172.21.15.114:0/3616018568 >> 172.21.15.114:0/3616018568 conn(0x7f19580fb030 msgr2=0x7f1958068410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:20:36.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.738+0000 7f195cf85700 1 -- 172.21.15.114:0/3616018568 shutdown_connections 2024-02-16T22:20:36.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.738+0000 7f195cf85700 1 -- 172.21.15.114:0/3616018568 wait complete. 2024-02-16T22:20:36.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:36 smithi123 ceph-mon[37161]: pgmap v255: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:36.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.844+0000 7f061666a700 1 -- 172.21.15.114:0/2144876699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06100ffe10 msgr2=0x7f0610100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:36.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.844+0000 7f061666a700 1 --2- 172.21.15.114:0/2144876699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06100ffe10 0x7f0610100290 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0604009910 tx=0x7f0604009c20 comp rx=0 tx=0).stop 2024-02-16T22:20:36.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.844+0000 7f061666a700 1 -- 172.21.15.114:0/2144876699 shutdown_connections 2024-02-16T22:20:36.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.844+0000 7f061666a700 1 --2- 172.21.15.114:0/2144876699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06100ffe10 0x7f0610100290 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:36.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.844+0000 7f061666a700 1 --2- 172.21.15.114:0/2144876699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06100ff4b0 0x7f06100ff8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:36.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.844+0000 7f061666a700 1 -- 172.21.15.114:0/2144876699 >> 172.21.15.114:0/2144876699 conn(0x7f06100fb050 msgr2=0x7f06100fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:20:36.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.844+0000 7f061666a700 1 -- 172.21.15.114:0/2144876699 shutdown_connections 2024-02-16T22:20:36.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.844+0000 7f061666a700 1 -- 172.21.15.114:0/2144876699 wait complete. 2024-02-16T22:20:36.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.845+0000 7f061666a700 1 Processor -- start 2024-02-16T22:20:36.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.845+0000 7f061666a700 1 -- start start 2024-02-16T22:20:36.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.846+0000 7f061666a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06100ff4b0 0x7f0610198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:36.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.846+0000 7f061666a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06100ffe10 0x7f0610198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:36.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.846+0000 7f061666a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0610199090 con 0x7f06100ff4b0 2024-02-16T22:20:36.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.846+0000 7f061666a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06101991d0 con 0x7f06100ffe10 2024-02-16T22:20:36.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.846+0000 7f060ffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06100ff4b0 0x7f0610198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:36.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.846+0000 7f060ffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06100ff4b0 0x7f0610198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60764/0 (socket says 172.21.15.114:60764) 2024-02-16T22:20:36.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.846+0000 7f060ffff700 1 -- 172.21.15.114:0/1418329375 learned_addr learned my addr 172.21.15.114:0/1418329375 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:20:36.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.846+0000 7f060f7fe700 1 --2- 172.21.15.114:0/1418329375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06100ffe10 0x7f0610198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:36.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.846+0000 7f060ffff700 1 -- 172.21.15.114:0/1418329375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06100ffe10 msgr2=0x7f0610198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:36.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.846+0000 7f060ffff700 1 --2- 172.21.15.114:0/1418329375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06100ffe10 0x7f0610198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:36.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.846+0000 7f060ffff700 1 -- 172.21.15.114:0/1418329375 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06040095f0 con 0x7f06100ff4b0 2024-02-16T22:20:36.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.847+0000 7f060ffff700 1 --2- 172.21.15.114:0/1418329375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06100ff4b0 0x7f0610198530 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f060000d8d0 tx=0x7f060000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:20:36.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.847+0000 7f060d7fa700 1 -- 172.21.15.114:0/1418329375 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06000098a0 con 0x7f06100ff4b0 2024-02-16T22:20:36.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.847+0000 7f060d7fa700 1 -- 172.21.15.114:0/1418329375 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0600010460 con 0x7f06100ff4b0 2024-02-16T22:20:36.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.847+0000 7f060d7fa700 1 -- 172.21.15.114:0/1418329375 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f060000f5f0 con 0x7f06100ff4b0 2024-02-16T22:20:36.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.847+0000 7f061666a700 1 -- 172.21.15.114:0/1418329375 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f061019dc80 con 0x7f06100ff4b0 2024-02-16T22:20:36.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.847+0000 7f061666a700 1 -- 172.21.15.114:0/1418329375 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f061019e110 con 0x7f06100ff4b0 2024-02-16T22:20:36.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.848+0000 7f061666a700 1 -- 172.21.15.114:0/1418329375 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f061010b910 con 0x7f06100ff4b0 2024-02-16T22:20:36.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.849+0000 7f060d7fa700 1 -- 172.21.15.114:0/1418329375 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f0600009a00 con 0x7f06100ff4b0 2024-02-16T22:20:36.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.850+0000 7f060d7fa700 1 --2- 172.21.15.114:0/1418329375 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f05f806c600 0x7f05f806eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:36.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.850+0000 7f060d7fa700 1 -- 172.21.15.114:0/1418329375 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f060008b640 con 0x7f06100ff4b0 2024-02-16T22:20:36.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.850+0000 7f060f7fe700 1 --2- 172.21.15.114:0/1418329375 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f05f806c600 0x7f05f806eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:36.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.851+0000 7f060f7fe700 1 --2- 172.21.15.114:0/1418329375 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f05f806c600 0x7f05f806eac0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f060400b5c0 tx=0x7f0604005ee0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:20:36.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:36.857+0000 7f060d7fa700 1 -- 172.21.15.114:0/1418329375 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f06000599c0 con 0x7f06100ff4b0 2024-02-16T22:20:37.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.143+0000 7f061666a700 1 -- 172.21.15.114:0/1418329375 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f061019e7b0 con 0x7f06100ff4b0 2024-02-16T22:20:37.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.143+0000 7f060d7fa700 1 -- 172.21.15.114:0/1418329375 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7f0600059550 con 0x7f06100ff4b0 2024-02-16T22:20:37.145 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:20:37.145 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": { 2024-02-16T22:20:37.146 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-02-16T22:20:37.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:20:37.146 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": { 2024-02-16T22:20:37.146 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-02-16T22:20:37.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:20:37.146 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": { 2024-02-16T22:20:37.146 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-02-16T22:20:37.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:20:37.146 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-02-16T22:20:37.146 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-02-16T22:20:37.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:20:37.146 INFO:teuthology.orchestra.run.smithi114.stdout: "overall": { 2024-02-16T22:20:37.146 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 14 2024-02-16T22:20:37.146 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-02-16T22:20:37.147 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:20:37.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.145+0000 7f061666a700 1 -- 172.21.15.114:0/1418329375 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f05f806c600 msgr2=0x7f05f806eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:37.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.146+0000 7f061666a700 1 --2- 172.21.15.114:0/1418329375 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f05f806c600 0x7f05f806eac0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f060400b5c0 tx=0x7f0604005ee0 comp rx=0 tx=0).stop 2024-02-16T22:20:37.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.146+0000 7f061666a700 1 -- 172.21.15.114:0/1418329375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06100ff4b0 msgr2=0x7f0610198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:37.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.146+0000 7f061666a700 1 --2- 172.21.15.114:0/1418329375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06100ff4b0 0x7f0610198530 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f060000d8d0 tx=0x7f060000dbe0 comp rx=0 tx=0).stop 2024-02-16T22:20:37.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.146+0000 7f061666a700 1 -- 172.21.15.114:0/1418329375 shutdown_connections 2024-02-16T22:20:37.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.146+0000 7f061666a700 1 --2- 172.21.15.114:0/1418329375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06100ff4b0 0x7f0610198530 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:37.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.146+0000 7f061666a700 1 --2- 172.21.15.114:0/1418329375 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f05f806c600 0x7f05f806eac0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:37.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.146+0000 7f061666a700 1 --2- 172.21.15.114:0/1418329375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06100ffe10 0x7f0610198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:37.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.146+0000 7f061666a700 1 -- 172.21.15.114:0/1418329375 >> 172.21.15.114:0/1418329375 conn(0x7f06100fb050 msgr2=0x7f0610107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:20:37.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.146+0000 7f061666a700 1 -- 172.21.15.114:0/1418329375 shutdown_connections 2024-02-16T22:20:37.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.146+0000 7f061666a700 1 -- 172.21.15.114:0/1418329375 wait complete. 2024-02-16T22:20:37.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.255+0000 7f677a4a5700 1 -- 172.21.15.114:0/2030384282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67740ff490 msgr2=0x7f67740ff8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:37.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.255+0000 7f677a4a5700 1 --2- 172.21.15.114:0/2030384282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67740ff490 0x7f67740ff8b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6764009910 tx=0x7f6764009c20 comp rx=0 tx=0).stop 2024-02-16T22:20:37.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.255+0000 7f677a4a5700 1 -- 172.21.15.114:0/2030384282 shutdown_connections 2024-02-16T22:20:37.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.255+0000 7f677a4a5700 1 --2- 172.21.15.114:0/2030384282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67740ffdf0 0x7f6774100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:37.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.255+0000 7f677a4a5700 1 --2- 172.21.15.114:0/2030384282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67740ff490 0x7f67740ff8b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:37.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.255+0000 7f677a4a5700 1 -- 172.21.15.114:0/2030384282 >> 172.21.15.114:0/2030384282 conn(0x7f67740fb030 msgr2=0x7f67740fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:20:37.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.255+0000 7f677a4a5700 1 -- 172.21.15.114:0/2030384282 shutdown_connections 2024-02-16T22:20:37.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.255+0000 7f677a4a5700 1 -- 172.21.15.114:0/2030384282 wait complete. 2024-02-16T22:20:37.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.256+0000 7f677a4a5700 1 Processor -- start 2024-02-16T22:20:37.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.256+0000 7f677a4a5700 1 -- start start 2024-02-16T22:20:37.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.256+0000 7f677a4a5700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67740ff490 0x7f6774105010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:37.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.256+0000 7f677a4a5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67740ffdf0 0x7f6774105550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:37.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.256+0000 7f677a4a5700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6774101b00 con 0x7f67740ff490 2024-02-16T22:20:37.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.256+0000 7f677a4a5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6774101c70 con 0x7f67740ffdf0 2024-02-16T22:20:37.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.256+0000 7f6773fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67740ff490 0x7f6774105010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:37.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.256+0000 7f67737fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67740ffdf0 0x7f6774105550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:37.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.256+0000 7f67737fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67740ffdf0 0x7f6774105550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.114:40952/0 (socket says 172.21.15.114:40952) 2024-02-16T22:20:37.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.256+0000 7f67737fe700 1 -- 172.21.15.114:0/2724475273 learned_addr learned my addr 172.21.15.114:0/2724475273 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:20:37.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.256+0000 7f6773fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67740ff490 0x7f6774105010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60790/0 (socket says 172.21.15.114:60790) 2024-02-16T22:20:37.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.257+0000 7f67737fe700 1 -- 172.21.15.114:0/2724475273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67740ff490 msgr2=0x7f6774105010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:37.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.257+0000 7f67737fe700 1 --2- 172.21.15.114:0/2724475273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67740ff490 0x7f6774105010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:37.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.257+0000 7f67737fe700 1 -- 172.21.15.114:0/2724475273 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67640095f0 con 0x7f67740ffdf0 2024-02-16T22:20:37.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.257+0000 7f67737fe700 1 --2- 172.21.15.114:0/2724475273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67740ffdf0 0x7f6774105550 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f676800e920 tx=0x7f676800ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:20:37.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.257+0000 7f67717fa700 1 -- 172.21.15.114:0/2724475273 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f676800c9d0 con 0x7f67740ffdf0 2024-02-16T22:20:37.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.257+0000 7f677a4a5700 1 -- 172.21.15.114:0/2724475273 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6774101f50 con 0x7f67740ffdf0 2024-02-16T22:20:37.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.257+0000 7f677a4a5700 1 -- 172.21.15.114:0/2724475273 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67741024a0 con 0x7f67740ffdf0 2024-02-16T22:20:37.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.257+0000 7f67717fa700 1 -- 172.21.15.114:0/2724475273 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6768010430 con 0x7f67740ffdf0 2024-02-16T22:20:37.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.257+0000 7f67717fa700 1 -- 172.21.15.114:0/2724475273 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6768018650 con 0x7f67740ffdf0 2024-02-16T22:20:37.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.259+0000 7f677a4a5700 1 -- 172.21.15.114:0/2724475273 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f677410b910 con 0x7f67740ffdf0 2024-02-16T22:20:37.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.259+0000 7f67717fa700 1 -- 172.21.15.114:0/2724475273 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f67680105a0 con 0x7f67740ffdf0 2024-02-16T22:20:37.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.259+0000 7f67717fa700 1 --2- 172.21.15.114:0/2724475273 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f675c06c4e0 0x7f675c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:37.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.260+0000 7f67717fa700 1 -- 172.21.15.114:0/2724475273 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f6768014070 con 0x7f67740ffdf0 2024-02-16T22:20:37.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.260+0000 7f6773fff700 1 --2- 172.21.15.114:0/2724475273 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f675c06c4e0 0x7f675c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:37.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.260+0000 7f6773fff700 1 --2- 172.21.15.114:0/2724475273 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f675c06c4e0 0x7f675c06e9a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6764009fd0 tx=0x7f6764005940 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:20:37.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.268+0000 7f67717fa700 1 -- 172.21.15.114:0/2724475273 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f67680520e0 con 0x7f67740ffdf0 2024-02-16T22:20:37.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.541+0000 7f677a4a5700 1 -- 172.21.15.114:0/2724475273 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f67741a2d90 con 0x7f67740ffdf0 2024-02-16T22:20:37.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.541+0000 7f67717fa700 1 -- 172.21.15.114:0/2724475273 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1825 (secure 0 0 0) 0x7f6768059b30 con 0x7f67740ffdf0 2024-02-16T22:20:37.543 INFO:teuthology.orchestra.run.smithi114.stdout:e14 2024-02-16T22:20:37.543 INFO:teuthology.orchestra.run.smithi114.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:20:37.544 INFO:teuthology.orchestra.run.smithi114.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:20:37.544 INFO:teuthology.orchestra.run.smithi114.stdout:legacy client fscid: 1 2024-02-16T22:20:37.544 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:20:37.544 INFO:teuthology.orchestra.run.smithi114.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:20:37.544 INFO:teuthology.orchestra.run.smithi114.stdout:fs_name cephfs 2024-02-16T22:20:37.544 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 12 2024-02-16T22:20:37.544 INFO:teuthology.orchestra.run.smithi114.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:20:37.544 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-02-16T22:18:02.108825+0000 2024-02-16T22:20:37.544 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-02-16T22:18:16.413371+0000 2024-02-16T22:20:37.544 INFO:teuthology.orchestra.run.smithi114.stdout:tableserver 0 2024-02-16T22:20:37.544 INFO:teuthology.orchestra.run.smithi114.stdout:root 0 2024-02-16T22:20:37.544 INFO:teuthology.orchestra.run.smithi114.stdout:session_timeout 60 2024-02-16T22:20:37.544 INFO:teuthology.orchestra.run.smithi114.stdout:session_autoclose 300 2024-02-16T22:20:37.545 INFO:teuthology.orchestra.run.smithi114.stdout:max_file_size 1099511627776 2024-02-16T22:20:37.545 INFO:teuthology.orchestra.run.smithi114.stdout:required_client_features {} 2024-02-16T22:20:37.545 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure 0 2024-02-16T22:20:37.545 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure_osd_epoch 0 2024-02-16T22:20:37.545 INFO:teuthology.orchestra.run.smithi114.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:20:37.545 INFO:teuthology.orchestra.run.smithi114.stdout:max_mds 1 2024-02-16T22:20:37.545 INFO:teuthology.orchestra.run.smithi114.stdout:in 0 2024-02-16T22:20:37.545 INFO:teuthology.orchestra.run.smithi114.stdout:up {0=14468} 2024-02-16T22:20:37.545 INFO:teuthology.orchestra.run.smithi114.stdout:failed 2024-02-16T22:20:37.545 INFO:teuthology.orchestra.run.smithi114.stdout:damaged 2024-02-16T22:20:37.545 INFO:teuthology.orchestra.run.smithi114.stdout:stopped 2024-02-16T22:20:37.545 INFO:teuthology.orchestra.run.smithi114.stdout:data_pools [3] 2024-02-16T22:20:37.545 INFO:teuthology.orchestra.run.smithi114.stdout:metadata_pool 2 2024-02-16T22:20:37.545 INFO:teuthology.orchestra.run.smithi114.stdout:inline_data enabled 2024-02-16T22:20:37.545 INFO:teuthology.orchestra.run.smithi114.stdout:balancer 2024-02-16T22:20:37.545 INFO:teuthology.orchestra.run.smithi114.stdout:bal_rank_mask -1 2024-02-16T22:20:37.546 INFO:teuthology.orchestra.run.smithi114.stdout:standby_count_wanted 1 2024-02-16T22:20:37.546 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.zzryke{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.114:6826/277953535,v1:172.21.15.114:6827/277953535] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:20:37.546 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:20:37.546 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:20:37.546 INFO:teuthology.orchestra.run.smithi114.stdout:Standby daemons: 2024-02-16T22:20:37.546 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:20:37.546 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.fdxpey{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:20:37.546 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.vhlqbe{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:20:37.546 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.gopcfa{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:20:37.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.544+0000 7f677a4a5700 1 -- 172.21.15.114:0/2724475273 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f675c06c4e0 msgr2=0x7f675c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:37.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.544+0000 7f677a4a5700 1 --2- 172.21.15.114:0/2724475273 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f675c06c4e0 0x7f675c06e9a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6764009fd0 tx=0x7f6764005940 comp rx=0 tx=0).stop 2024-02-16T22:20:37.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.544+0000 7f677a4a5700 1 -- 172.21.15.114:0/2724475273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67740ffdf0 msgr2=0x7f6774105550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:37.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.544+0000 7f677a4a5700 1 --2- 172.21.15.114:0/2724475273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67740ffdf0 0x7f6774105550 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f676800e920 tx=0x7f676800ec30 comp rx=0 tx=0).stop 2024-02-16T22:20:37.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.545+0000 7f677a4a5700 1 -- 172.21.15.114:0/2724475273 shutdown_connections 2024-02-16T22:20:37.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.545+0000 7f677a4a5700 1 --2- 172.21.15.114:0/2724475273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67740ff490 0x7f6774105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:37.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.545+0000 7f677a4a5700 1 --2- 172.21.15.114:0/2724475273 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f675c06c4e0 0x7f675c06e9a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:37.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.545+0000 7f677a4a5700 1 --2- 172.21.15.114:0/2724475273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67740ffdf0 0x7f6774105550 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:37.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.545+0000 7f677a4a5700 1 -- 172.21.15.114:0/2724475273 >> 172.21.15.114:0/2724475273 conn(0x7f67740fb030 msgr2=0x7f6774107ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:20:37.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.545+0000 7f677a4a5700 1 -- 172.21.15.114:0/2724475273 shutdown_connections 2024-02-16T22:20:37.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.545+0000 7f677a4a5700 1 -- 172.21.15.114:0/2724475273 wait complete. 2024-02-16T22:20:37.548 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 14 2024-02-16T22:20:37.551 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:37 smithi123 ceph-mon[37161]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:20:37.551 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:37 smithi123 ceph-mon[37161]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:20:37.551 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:37 smithi123 ceph-mon[37161]: from='client.14660 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:20:37.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:37 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/1418329375' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:20:37.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:37 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/2724475273' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:20:37.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.663+0000 7f2fdfaa1700 1 -- 172.21.15.114:0/3035845215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fd80ffcb0 msgr2=0x7f2fd8100110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:37.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.663+0000 7f2fdfaa1700 1 --2- 172.21.15.114:0/3035845215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fd80ffcb0 0x7f2fd8100110 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009910 tx=0x7f2fc8009c20 comp rx=0 tx=0).stop 2024-02-16T22:20:37.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.663+0000 7f2fdfaa1700 1 -- 172.21.15.114:0/3035845215 shutdown_connections 2024-02-16T22:20:37.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.663+0000 7f2fdfaa1700 1 --2- 172.21.15.114:0/3035845215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fd80ffcb0 0x7f2fd8100110 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:37.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.663+0000 7f2fdfaa1700 1 --2- 172.21.15.114:0/3035845215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fd81010a0 0x7f2fd81014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:37.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.663+0000 7f2fdfaa1700 1 -- 172.21.15.114:0/3035845215 >> 172.21.15.114:0/3035845215 conn(0x7f2fd80fa7c0 msgr2=0x7f2fd80fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:20:37.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.663+0000 7f2fdfaa1700 1 -- 172.21.15.114:0/3035845215 shutdown_connections 2024-02-16T22:20:37.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.663+0000 7f2fdfaa1700 1 -- 172.21.15.114:0/3035845215 wait complete. 2024-02-16T22:20:37.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.664+0000 7f2fdfaa1700 1 Processor -- start 2024-02-16T22:20:37.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.664+0000 7f2fdfaa1700 1 -- start start 2024-02-16T22:20:37.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.664+0000 7f2fdfaa1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fd80ffcb0 0x7f2fd81963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:37.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.664+0000 7f2fdfaa1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fd81010a0 0x7f2fd81968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:37.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.664+0000 7f2fdfaa1700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd8196f00 con 0x7f2fd81010a0 2024-02-16T22:20:37.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.664+0000 7f2fdfaa1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd8197040 con 0x7f2fd80ffcb0 2024-02-16T22:20:37.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.664+0000 7f2fdd03c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fd81010a0 0x7f2fd81968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:37.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.664+0000 7f2fdd03c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fd81010a0 0x7f2fd81968e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60812/0 (socket says 172.21.15.114:60812) 2024-02-16T22:20:37.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.664+0000 7f2fdd03c700 1 -- 172.21.15.114:0/2750267252 learned_addr learned my addr 172.21.15.114:0/2750267252 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:20:37.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.665+0000 7f2fdd83d700 1 --2- 172.21.15.114:0/2750267252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fd80ffcb0 0x7f2fd81963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:37.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.665+0000 7f2fdd03c700 1 -- 172.21.15.114:0/2750267252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fd80ffcb0 msgr2=0x7f2fd81963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:37.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.665+0000 7f2fdd03c700 1 --2- 172.21.15.114:0/2750267252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fd80ffcb0 0x7f2fd81963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:37.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.665+0000 7f2fdd03c700 1 -- 172.21.15.114:0/2750267252 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc80095f0 con 0x7f2fd81010a0 2024-02-16T22:20:37.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.665+0000 7f2fdd03c700 1 --2- 172.21.15.114:0/2750267252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fd81010a0 0x7f2fd81968e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800b5c0 tx=0x7f2fc800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:20:37.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.665+0000 7f2fceffd700 1 -- 172.21.15.114:0/2750267252 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fc801d070 con 0x7f2fd81010a0 2024-02-16T22:20:37.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.666+0000 7f2fceffd700 1 -- 172.21.15.114:0/2750267252 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2fc800f460 con 0x7f2fd81010a0 2024-02-16T22:20:37.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.666+0000 7f2fdfaa1700 1 -- 172.21.15.114:0/2750267252 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd8105f00 con 0x7f2fd81010a0 2024-02-16T22:20:37.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.666+0000 7f2fceffd700 1 -- 172.21.15.114:0/2750267252 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fc8021640 con 0x7f2fd81010a0 2024-02-16T22:20:37.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.666+0000 7f2fdfaa1700 1 -- 172.21.15.114:0/2750267252 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd81063f0 con 0x7f2fd81010a0 2024-02-16T22:20:37.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.668+0000 7f2fdfaa1700 1 -- 172.21.15.114:0/2750267252 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd8109720 con 0x7f2fd81010a0 2024-02-16T22:20:37.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.668+0000 7f2fceffd700 1 -- 172.21.15.114:0/2750267252 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f2fc800f5d0 con 0x7f2fd81010a0 2024-02-16T22:20:37.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.668+0000 7f2fceffd700 1 --2- 172.21.15.114:0/2750267252 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f2fc406c4e0 0x7f2fc406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:37.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.669+0000 7f2fceffd700 1 -- 172.21.15.114:0/2750267252 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f2fc808cb60 con 0x7f2fd81010a0 2024-02-16T22:20:37.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.669+0000 7f2fdd83d700 1 --2- 172.21.15.114:0/2750267252 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f2fc406c4e0 0x7f2fc406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:37.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.669+0000 7f2fdd83d700 1 --2- 172.21.15.114:0/2750267252 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f2fc406c4e0 0x7f2fc406e9a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4005950 tx=0x7f2fd40058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:20:37.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.673+0000 7f2fceffd700 1 -- 172.21.15.114:0/2750267252 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2fc805ae50 con 0x7f2fd81010a0 2024-02-16T22:20:37.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.889+0000 7f2fdfaa1700 1 -- 172.21.15.114:0/2750267252 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fd8105610 con 0x7f2fc406c4e0 2024-02-16T22:20:37.891 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:37 smithi114 ceph-mon[29235]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:20:37.891 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:37 smithi114 ceph-mon[29235]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:20:37.891 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:37 smithi114 ceph-mon[29235]: from='client.14660 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:20:37.891 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:37 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/1418329375' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:20:37.891 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:37 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/2724475273' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:20:37.893 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:20:37.893 INFO:teuthology.orchestra.run.smithi114.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:20:37.893 INFO:teuthology.orchestra.run.smithi114.stdout: "in_progress": true, 2024-02-16T22:20:37.893 INFO:teuthology.orchestra.run.smithi114.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:20:37.894 INFO:teuthology.orchestra.run.smithi114.stdout: "services_complete": [], 2024-02-16T22:20:37.894 INFO:teuthology.orchestra.run.smithi114.stdout: "progress": "0/23 daemons upgraded", 2024-02-16T22:20:37.894 INFO:teuthology.orchestra.run.smithi114.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 image on host smithi123", 2024-02-16T22:20:37.894 INFO:teuthology.orchestra.run.smithi114.stdout: "is_paused": false 2024-02-16T22:20:37.894 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:20:37.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.891+0000 7f2fceffd700 1 -- 172.21.15.114:0/2750267252 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f2fd8105610 con 0x7f2fc406c4e0 2024-02-16T22:20:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.894+0000 7f2fdfaa1700 1 -- 172.21.15.114:0/2750267252 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f2fc406c4e0 msgr2=0x7f2fc406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.894+0000 7f2fdfaa1700 1 --2- 172.21.15.114:0/2750267252 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f2fc406c4e0 0x7f2fc406e9a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4005950 tx=0x7f2fd40058e0 comp rx=0 tx=0).stop 2024-02-16T22:20:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.894+0000 7f2fdfaa1700 1 -- 172.21.15.114:0/2750267252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fd81010a0 msgr2=0x7f2fd81968e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.894+0000 7f2fdfaa1700 1 --2- 172.21.15.114:0/2750267252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fd81010a0 0x7f2fd81968e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800b5c0 tx=0x7f2fc800be20 comp rx=0 tx=0).stop 2024-02-16T22:20:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.894+0000 7f2fdfaa1700 1 -- 172.21.15.114:0/2750267252 shutdown_connections 2024-02-16T22:20:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.894+0000 7f2fdfaa1700 1 --2- 172.21.15.114:0/2750267252 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f2fc406c4e0 0x7f2fc406e9a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.894+0000 7f2fdfaa1700 1 --2- 172.21.15.114:0/2750267252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fd80ffcb0 0x7f2fd81963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.894+0000 7f2fdfaa1700 1 --2- 172.21.15.114:0/2750267252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fd81010a0 0x7f2fd81968e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.894+0000 7f2fdfaa1700 1 -- 172.21.15.114:0/2750267252 >> 172.21.15.114:0/2750267252 conn(0x7f2fd80fa7c0 msgr2=0x7f2fd8103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:20:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.894+0000 7f2fdfaa1700 1 -- 172.21.15.114:0/2750267252 shutdown_connections 2024-02-16T22:20:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:37.894+0000 7f2fdfaa1700 1 -- 172.21.15.114:0/2750267252 wait complete. 2024-02-16T22:20:38.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.001+0000 7fb05e8cf700 1 -- 172.21.15.114:0/2307472905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb058101640 msgr2=0x7fb058103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:38.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.001+0000 7fb05e8cf700 1 --2- 172.21.15.114:0/2307472905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb058101640 0x7fb058103a30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb048009910 tx=0x7fb048009c20 comp rx=0 tx=0).stop 2024-02-16T22:20:38.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.001+0000 7fb05e8cf700 1 -- 172.21.15.114:0/2307472905 shutdown_connections 2024-02-16T22:20:38.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.001+0000 7fb05e8cf700 1 --2- 172.21.15.114:0/2307472905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb058103f70 0x7fb058106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:38.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.001+0000 7fb05e8cf700 1 --2- 172.21.15.114:0/2307472905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb058101640 0x7fb058103a30 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:38.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.001+0000 7fb05e8cf700 1 -- 172.21.15.114:0/2307472905 >> 172.21.15.114:0/2307472905 conn(0x7fb0580fb030 msgr2=0x7fb0580fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:20:38.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.001+0000 7fb05e8cf700 1 -- 172.21.15.114:0/2307472905 shutdown_connections 2024-02-16T22:20:38.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.001+0000 7fb05e8cf700 1 -- 172.21.15.114:0/2307472905 wait complete. 2024-02-16T22:20:38.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.002+0000 7fb05e8cf700 1 Processor -- start 2024-02-16T22:20:38.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.002+0000 7fb05e8cf700 1 -- start start 2024-02-16T22:20:38.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.002+0000 7fb05e8cf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb058101640 0x7fb058196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:38.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.002+0000 7fb05e8cf700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb058103f70 0x7fb058196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:38.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.003+0000 7fb05e8cf700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb058196e60 con 0x7fb058103f70 2024-02-16T22:20:38.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.003+0000 7fb0577fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb058103f70 0x7fb058196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:38.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.003+0000 7fb0577fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb058103f70 0x7fb058196840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60836/0 (socket says 172.21.15.114:60836) 2024-02-16T22:20:38.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.003+0000 7fb0577fe700 1 -- 172.21.15.114:0/3279584963 learned_addr learned my addr 172.21.15.114:0/3279584963 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:20:38.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.003+0000 7fb05e8cf700 1 -- 172.21.15.114:0/3279584963 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb058196fa0 con 0x7fb058101640 2024-02-16T22:20:38.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.003+0000 7fb057fff700 1 --2- 172.21.15.114:0/3279584963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb058101640 0x7fb058196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:38.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.003+0000 7fb0577fe700 1 -- 172.21.15.114:0/3279584963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb058101640 msgr2=0x7fb058196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:38.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.003+0000 7fb0577fe700 1 --2- 172.21.15.114:0/3279584963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb058101640 0x7fb058196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:38.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.003+0000 7fb0577fe700 1 -- 172.21.15.114:0/3279584963 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04c009520 con 0x7fb058103f70 2024-02-16T22:20:38.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.003+0000 7fb0577fe700 1 --2- 172.21.15.114:0/3279584963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb058103f70 0x7fb058196840 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00eab0 tx=0x7fb04c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:20:38.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.003+0000 7fb0557fa700 1 -- 172.21.15.114:0/3279584963 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb04c00cc60 con 0x7fb058103f70 2024-02-16T22:20:38.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.004+0000 7fb05e8cf700 1 -- 172.21.15.114:0/3279584963 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0480095f0 con 0x7fb058103f70 2024-02-16T22:20:38.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.004+0000 7fb0557fa700 1 -- 172.21.15.114:0/3279584963 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb04c0044d0 con 0x7fb058103f70 2024-02-16T22:20:38.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.004+0000 7fb0557fa700 1 -- 172.21.15.114:0/3279584963 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb04c005250 con 0x7fb058103f70 2024-02-16T22:20:38.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.004+0000 7fb05e8cf700 1 -- 172.21.15.114:0/3279584963 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb05819bdb0 con 0x7fb058103f70 2024-02-16T22:20:38.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.005+0000 7fb05e8cf700 1 -- 172.21.15.114:0/3279584963 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb058190520 con 0x7fb058103f70 2024-02-16T22:20:38.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.006+0000 7fb0557fa700 1 -- 172.21.15.114:0/3279584963 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fb04c004930 con 0x7fb058103f70 2024-02-16T22:20:38.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.006+0000 7fb0557fa700 1 --2- 172.21.15.114:0/3279584963 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fb04006c4e0 0x7fb04006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:20:38.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.006+0000 7fb0557fa700 1 -- 172.21.15.114:0/3279584963 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fb04c014070 con 0x7fb058103f70 2024-02-16T22:20:38.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.007+0000 7fb057fff700 1 --2- 172.21.15.114:0/3279584963 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fb04006c4e0 0x7fb04006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:20:38.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.007+0000 7fb057fff700 1 --2- 172.21.15.114:0/3279584963 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fb04006c4e0 0x7fb04006e9a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb048006010 tx=0x7fb048009f90 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:20:38.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.011+0000 7fb0557fa700 1 -- 172.21.15.114:0/3279584963 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb04c059ac0 con 0x7fb058103f70 2024-02-16T22:20:38.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.292+0000 7fb05e8cf700 1 -- 172.21.15.114:0/3279584963 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb05802cc70 con 0x7fb058103f70 2024-02-16T22:20:38.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.292+0000 7fb0557fa700 1 -- 172.21.15.114:0/3279584963 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fb04c059650 con 0x7fb058103f70 2024-02-16T22:20:38.294 INFO:teuthology.orchestra.run.smithi114.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:20:38.294 INFO:teuthology.orchestra.run.smithi114.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:20:38.295 INFO:teuthology.orchestra.run.smithi114.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:20:38.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.294+0000 7fb05e8cf700 1 -- 172.21.15.114:0/3279584963 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fb04006c4e0 msgr2=0x7fb04006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:38.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.294+0000 7fb05e8cf700 1 --2- 172.21.15.114:0/3279584963 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fb04006c4e0 0x7fb04006e9a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb048006010 tx=0x7fb048009f90 comp rx=0 tx=0).stop 2024-02-16T22:20:38.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.294+0000 7fb05e8cf700 1 -- 172.21.15.114:0/3279584963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb058103f70 msgr2=0x7fb058196840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:20:38.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.294+0000 7fb05e8cf700 1 --2- 172.21.15.114:0/3279584963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb058103f70 0x7fb058196840 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00eab0 tx=0x7fb04c00ee70 comp rx=0 tx=0).stop 2024-02-16T22:20:38.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.295+0000 7fb05e8cf700 1 -- 172.21.15.114:0/3279584963 shutdown_connections 2024-02-16T22:20:38.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.295+0000 7fb05e8cf700 1 --2- 172.21.15.114:0/3279584963 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fb04006c4e0 0x7fb04006e9a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:38.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.295+0000 7fb05e8cf700 1 --2- 172.21.15.114:0/3279584963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb058101640 0x7fb058196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:20:38.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.295+0000 7fb05e8cf700 1 --2- 172.21.15.114:0/3279584963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb058103f70 0x7fb058196840 secure :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00eab0 tx=0x7fb04c00ee70 comp rx=0 tx=0).stop 2024-02-16T22:20:38.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.295+0000 7fb05e8cf700 1 -- 172.21.15.114:0/3279584963 >> 172.21.15.114:0/3279584963 conn(0x7fb0580fb030 msgr2=0x7fb0580fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:20:38.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.295+0000 7fb05e8cf700 1 -- 172.21.15.114:0/3279584963 shutdown_connections 2024-02-16T22:20:38.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:20:38.295+0000 7fb05e8cf700 1 -- 172.21.15.114:0/3279584963 wait complete. 2024-02-16T22:20:38.551 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:38 smithi114 ceph-mon[29235]: pgmap v256: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:38.551 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:38 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/3279584963' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:20:38.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:38 smithi123 ceph-mon[37161]: pgmap v256: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:38.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:38 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/3279584963' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:20:39.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:39 smithi114 ceph-mon[29235]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:20:40.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:39 smithi123 ceph-mon[37161]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:20:40.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:40 smithi114 ceph-mon[29235]: pgmap v257: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:40.953 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:40 smithi123 ceph-mon[37161]: pgmap v257: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:41.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:41 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:20:42.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:41 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:20:42.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:42 smithi114 ceph-mon[29235]: pgmap v258: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:43.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:42 smithi123 ceph-mon[37161]: pgmap v258: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:44.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:44 smithi114 ceph-mon[29235]: pgmap v259: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:45.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:44 smithi123 ceph-mon[37161]: pgmap v259: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:46.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:46 smithi114 ceph-mon[29235]: pgmap v260: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:47.046 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:46 smithi123 ceph-mon[37161]: pgmap v260: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:48.045 INFO:tasks.workunit.client.1.smithi123.stderr:Note: switching to 'ed7ab725e1a43615009dfb74a42bf33d80081bc8'. 2024-02-16T22:20:48.045 INFO:tasks.workunit.client.1.smithi123.stderr: 2024-02-16T22:20:48.045 INFO:tasks.workunit.client.1.smithi123.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-16T22:20:48.046 INFO:tasks.workunit.client.1.smithi123.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-16T22:20:48.046 INFO:tasks.workunit.client.1.smithi123.stderr:state without impacting any branches by switching back to a branch. 2024-02-16T22:20:48.046 INFO:tasks.workunit.client.1.smithi123.stderr: 2024-02-16T22:20:48.046 INFO:tasks.workunit.client.1.smithi123.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-16T22:20:48.046 INFO:tasks.workunit.client.1.smithi123.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-16T22:20:48.046 INFO:tasks.workunit.client.1.smithi123.stderr: 2024-02-16T22:20:48.046 INFO:tasks.workunit.client.1.smithi123.stderr: git switch -c 2024-02-16T22:20:48.046 INFO:tasks.workunit.client.1.smithi123.stderr: 2024-02-16T22:20:48.046 INFO:tasks.workunit.client.1.smithi123.stderr:Or undo this operation with: 2024-02-16T22:20:48.046 INFO:tasks.workunit.client.1.smithi123.stderr: 2024-02-16T22:20:48.046 INFO:tasks.workunit.client.1.smithi123.stderr: git switch - 2024-02-16T22:20:48.046 INFO:tasks.workunit.client.1.smithi123.stderr: 2024-02-16T22:20:48.046 INFO:tasks.workunit.client.1.smithi123.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-16T22:20:48.047 INFO:tasks.workunit.client.1.smithi123.stderr: 2024-02-16T22:20:48.047 INFO:tasks.workunit.client.1.smithi123.stderr:HEAD is now at ed7ab725e1a qa: test fuse/kclient for mds upgrade seq 2024-02-16T22:20:48.050 DEBUG:teuthology.orchestra.run.smithi123:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-02-16T22:20:48.116 INFO:tasks.workunit.client.1.smithi123.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-16T22:20:48.118 INFO:tasks.workunit.client.1.smithi123.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-02-16T22:20:48.118 INFO:tasks.workunit.client.1.smithi123.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-16T22:20:48.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:47 smithi114 ceph-mon[29235]: pgmap v261: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:48.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:47 smithi123 ceph-mon[37161]: pgmap v261: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:49.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:49 smithi114 ceph-mon[29235]: Upgrade: Updating mgr.smithi123.inzgjd 2024-02-16T22:20:49.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:49 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:20:49.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:49 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi123.inzgjd", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:20:49.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:49 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T22:20:49.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:49 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:20:49.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:49 smithi114 ceph-mon[29235]: Deploying daemon mgr.smithi123.inzgjd on smithi123 2024-02-16T22:20:49.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:49 smithi123 ceph-mon[37161]: Upgrade: Updating mgr.smithi123.inzgjd 2024-02-16T22:20:49.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:49 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:20:49.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:49 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi123.inzgjd", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:20:49.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:49 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T22:20:49.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:49 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:20:49.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:49 smithi123 ceph-mon[37161]: Deploying daemon mgr.smithi123.inzgjd on smithi123 2024-02-16T22:20:50.004 INFO:tasks.workunit.client.1.smithi123.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-16T22:20:50.163 INFO:tasks.workunit.client.1.smithi123.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-16T22:20:50.204 INFO:tasks.workunit.client.1.smithi123.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-02-16T22:20:50.206 INFO:tasks.workunit.client.1.smithi123.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-02-16T22:20:50.206 INFO:tasks.workunit.client.1.smithi123.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-16T22:20:50.371 INFO:tasks.workunit.client.1.smithi123.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-02-16T22:20:50.375 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-02-16T22:20:50.375 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-02-16T22:20:50.384 INFO:tasks.workunit.client.0.smithi114.stderr:Note: switching to 'ed7ab725e1a43615009dfb74a42bf33d80081bc8'. 2024-02-16T22:20:50.384 INFO:tasks.workunit.client.0.smithi114.stderr: 2024-02-16T22:20:50.384 INFO:tasks.workunit.client.0.smithi114.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-16T22:20:50.384 INFO:tasks.workunit.client.0.smithi114.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-16T22:20:50.384 INFO:tasks.workunit.client.0.smithi114.stderr:state without impacting any branches by switching back to a branch. 2024-02-16T22:20:50.384 INFO:tasks.workunit.client.0.smithi114.stderr: 2024-02-16T22:20:50.384 INFO:tasks.workunit.client.0.smithi114.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-16T22:20:50.384 INFO:tasks.workunit.client.0.smithi114.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-16T22:20:50.385 INFO:tasks.workunit.client.0.smithi114.stderr: 2024-02-16T22:20:50.385 INFO:tasks.workunit.client.0.smithi114.stderr: git switch -c 2024-02-16T22:20:50.385 INFO:tasks.workunit.client.0.smithi114.stderr: 2024-02-16T22:20:50.385 INFO:tasks.workunit.client.0.smithi114.stderr:Or undo this operation with: 2024-02-16T22:20:50.385 INFO:tasks.workunit.client.0.smithi114.stderr: 2024-02-16T22:20:50.385 INFO:tasks.workunit.client.0.smithi114.stderr: git switch - 2024-02-16T22:20:50.385 INFO:tasks.workunit.client.0.smithi114.stderr: 2024-02-16T22:20:50.385 INFO:tasks.workunit.client.0.smithi114.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-16T22:20:50.385 INFO:tasks.workunit.client.0.smithi114.stderr: 2024-02-16T22:20:50.385 INFO:tasks.workunit.client.0.smithi114.stderr:HEAD is now at ed7ab725e1a qa: test fuse/kclient for mds upgrade seq 2024-02-16T22:20:50.388 DEBUG:teuthology.orchestra.run.smithi114:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-16T22:20:50.410 INFO:tasks.workunit.client.0.smithi114.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-16T22:20:50.412 INFO:tasks.workunit.client.0.smithi114.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-16T22:20:50.412 INFO:tasks.workunit.client.0.smithi114.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-16T22:20:50.431 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-02-16T22:20:50.433 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-02-16T22:20:50.433 DEBUG:teuthology.orchestra.run.smithi123:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ed7ab725e1a43615009dfb74a42bf33d80081bc8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-02-16T22:20:50.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:50 smithi114 ceph-mon[29235]: pgmap v262: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:50.498 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:50 smithi123 ceph-mon[37161]: pgmap v262: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:50.499 INFO:tasks.workunit.client.1.smithi123.stderr:+ mkdir -p fsstress 2024-02-16T22:20:50.501 INFO:tasks.workunit.client.1.smithi123.stderr:+ pushd fsstress 2024-02-16T22:20:50.503 INFO:tasks.workunit.client.1.smithi123.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-02-16T22:20:50.503 INFO:tasks.workunit.client.1.smithi123.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-02-16T22:20:51.275 INFO:tasks.workunit.client.0.smithi114.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-16T22:20:51.350 INFO:tasks.workunit.client.0.smithi114.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-16T22:20:51.389 INFO:tasks.workunit.client.0.smithi114.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-16T22:20:51.391 INFO:tasks.workunit.client.0.smithi114.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-16T22:20:51.391 INFO:tasks.workunit.client.0.smithi114.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-16T22:20:51.430 INFO:tasks.workunit.client.0.smithi114.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-16T22:20:51.433 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-02-16T22:20:51.433 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-16T22:20:51.489 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-02-16T22:20:51.490 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-02-16T22:20:51.490 DEBUG:teuthology.orchestra.run.smithi114:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ed7ab725e1a43615009dfb74a42bf33d80081bc8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-02-16T22:20:51.553 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir -p fsstress 2024-02-16T22:20:51.557 INFO:tasks.workunit.client.0.smithi114.stderr:+ pushd fsstress 2024-02-16T22:20:51.558 INFO:tasks.workunit.client.0.smithi114.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-02-16T22:20:51.558 INFO:tasks.workunit.client.0.smithi114.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-02-16T22:20:51.698 INFO:tasks.workunit.client.1.smithi123.stderr:+ tar xzf ltp-full.tgz 2024-02-16T22:20:52.092 INFO:tasks.workunit.client.0.smithi114.stderr:+ tar xzf ltp-full.tgz 2024-02-16T22:20:52.727 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:52 smithi123 ceph-mon[37161]: pgmap v263: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:52.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:52 smithi114 ceph-mon[29235]: pgmap v263: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:20:54.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:53 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:20:54.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:53 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:20:54.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:53 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:20:54.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:53 smithi123 ceph-mon[37161]: pgmap v264: 65 pgs: 65 active+clean; 7.9 MiB data, 188 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 628 KiB/s wr, 10 op/s 2024-02-16T22:20:54.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:53 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:20:54.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:53 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:20:54.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:53 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:20:54.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:53 smithi114 ceph-mon[29235]: pgmap v264: 65 pgs: 65 active+clean; 7.9 MiB data, 188 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 628 KiB/s wr, 10 op/s 2024-02-16T22:20:56.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:56 smithi123 ceph-mon[37161]: pgmap v265: 65 pgs: 65 active+clean; 17 MiB data, 204 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.3 MiB/s wr, 20 op/s 2024-02-16T22:20:56.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:56 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:20:56.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:56 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:20:56.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:56 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:20:56.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:56 smithi114 ceph-mon[29235]: pgmap v265: 65 pgs: 65 active+clean; 17 MiB data, 204 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.3 MiB/s wr, 20 op/s 2024-02-16T22:20:56.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:56 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:20:56.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:56 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:20:56.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:56 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:20:58.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:58 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:20:58.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:58 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:20:58.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:20:58 smithi123 ceph-mon[37161]: pgmap v266: 65 pgs: 65 active+clean; 29 MiB data, 243 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.4 MiB/s wr, 62 op/s 2024-02-16T22:20:58.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:58 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:20:58.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:58 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:20:58.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:20:58 smithi114 ceph-mon[29235]: pgmap v266: 65 pgs: 65 active+clean; 29 MiB data, 243 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.4 MiB/s wr, 62 op/s 2024-02-16T22:21:00.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:00 smithi123 ceph-mon[37161]: pgmap v267: 65 pgs: 65 active+clean; 36 MiB data, 269 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.0 MiB/s wr, 113 op/s 2024-02-16T22:21:00.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:00 smithi114 ceph-mon[29235]: pgmap v267: 65 pgs: 65 active+clean; 36 MiB data, 269 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.0 MiB/s wr, 113 op/s 2024-02-16T22:21:01.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:01 smithi123 ceph-mon[37161]: Standby manager daemon smithi123.inzgjd restarted 2024-02-16T22:21:01.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:01 smithi123 ceph-mon[37161]: Standby manager daemon smithi123.inzgjd started 2024-02-16T22:21:01.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:01 smithi123 ceph-mon[37161]: from='mgr.? 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi123.inzgjd/crt"}]: dispatch 2024-02-16T22:21:01.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:01 smithi123 ceph-mon[37161]: from='mgr.? 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T22:21:01.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:01 smithi123 ceph-mon[37161]: from='mgr.? 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi123.inzgjd/key"}]: dispatch 2024-02-16T22:21:01.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:01 smithi123 ceph-mon[37161]: from='mgr.? 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T22:21:01.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:01 smithi114 ceph-mon[29235]: Standby manager daemon smithi123.inzgjd restarted 2024-02-16T22:21:01.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:01 smithi114 ceph-mon[29235]: Standby manager daemon smithi123.inzgjd started 2024-02-16T22:21:01.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:01 smithi114 ceph-mon[29235]: from='mgr.? 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi123.inzgjd/crt"}]: dispatch 2024-02-16T22:21:01.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:01 smithi114 ceph-mon[29235]: from='mgr.? 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T22:21:01.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:01 smithi114 ceph-mon[29235]: from='mgr.? 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi123.inzgjd/key"}]: dispatch 2024-02-16T22:21:01.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:01 smithi114 ceph-mon[29235]: from='mgr.? 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T22:21:02.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:02 smithi123 ceph-mon[37161]: pgmap v268: 65 pgs: 65 active+clean; 46 MiB data, 297 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.8 MiB/s wr, 145 op/s 2024-02-16T22:21:02.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:02 smithi123 ceph-mon[37161]: mgrmap e20: smithi114.vcullz(active, since 7m), standbys: smithi123.inzgjd 2024-02-16T22:21:02.817 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:02 smithi114 ceph-mon[29235]: pgmap v268: 65 pgs: 65 active+clean; 46 MiB data, 297 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.8 MiB/s wr, 145 op/s 2024-02-16T22:21:02.817 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:02 smithi114 ceph-mon[29235]: mgrmap e20: smithi114.vcullz(active, since 7m), standbys: smithi123.inzgjd 2024-02-16T22:21:04.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:04 smithi114 ceph-mon[29235]: pgmap v269: 65 pgs: 65 active+clean; 50 MiB data, 328 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.2 MiB/s wr, 156 op/s 2024-02-16T22:21:05.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:04 smithi123 ceph-mon[37161]: pgmap v269: 65 pgs: 65 active+clean; 50 MiB data, 328 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.2 MiB/s wr, 156 op/s 2024-02-16T22:21:06.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:06 smithi114 ceph-mon[29235]: pgmap v270: 65 pgs: 65 active+clean; 53 MiB data, 349 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.8 MiB/s wr, 153 op/s 2024-02-16T22:21:07.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:06 smithi123 ceph-mon[37161]: pgmap v270: 65 pgs: 65 active+clean; 53 MiB data, 349 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.8 MiB/s wr, 153 op/s 2024-02-16T22:21:08.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:07 smithi114 ceph-mon[29235]: pgmap v271: 65 pgs: 65 active+clean; 76 MiB data, 396 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 5.0 MiB/s wr, 199 op/s 2024-02-16T22:21:08.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:07 smithi123 ceph-mon[37161]: pgmap v271: 65 pgs: 65 active+clean; 76 MiB data, 396 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 5.0 MiB/s wr, 199 op/s 2024-02-16T22:21:08.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.432+0000 7fe96b49d700 1 -- 172.21.15.114:0/2683051079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe964103fb0 msgr2=0x7fe9641063a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:08.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.432+0000 7fe96b49d700 1 --2- 172.21.15.114:0/2683051079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe964103fb0 0x7fe9641063a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe954009910 tx=0x7fe954009c20 comp rx=0 tx=0).stop 2024-02-16T22:21:08.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.432+0000 7fe96b49d700 1 -- 172.21.15.114:0/2683051079 shutdown_connections 2024-02-16T22:21:08.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.432+0000 7fe96b49d700 1 --2- 172.21.15.114:0/2683051079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe964103fb0 0x7fe9641063a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:08.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.432+0000 7fe96b49d700 1 --2- 172.21.15.114:0/2683051079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe964101680 0x7fe964103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:08.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.432+0000 7fe96b49d700 1 -- 172.21.15.114:0/2683051079 >> 172.21.15.114:0/2683051079 conn(0x7fe9640fb030 msgr2=0x7fe9640fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:21:08.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.433+0000 7fe96b49d700 1 -- 172.21.15.114:0/2683051079 shutdown_connections 2024-02-16T22:21:08.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.433+0000 7fe96b49d700 1 -- 172.21.15.114:0/2683051079 wait complete. 2024-02-16T22:21:08.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.433+0000 7fe96b49d700 1 Processor -- start 2024-02-16T22:21:08.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.433+0000 7fe96b49d700 1 -- start start 2024-02-16T22:21:08.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.433+0000 7fe96b49d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe964101680 0x7fe964100fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:08.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.433+0000 7fe96b49d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe964103fb0 0x7fe9640ff630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:08.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.433+0000 7fe96b49d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9640ffb70 con 0x7fe964101680 2024-02-16T22:21:08.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.434+0000 7fe96b49d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9640ffcb0 con 0x7fe964103fb0 2024-02-16T22:21:08.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.434+0000 7fe969239700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe964101680 0x7fe964100fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:08.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.434+0000 7fe969239700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe964101680 0x7fe964100fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48400/0 (socket says 172.21.15.114:48400) 2024-02-16T22:21:08.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.434+0000 7fe969239700 1 -- 172.21.15.114:0/3606584369 learned_addr learned my addr 172.21.15.114:0/3606584369 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:21:08.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.434+0000 7fe968a38700 1 --2- 172.21.15.114:0/3606584369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe964103fb0 0x7fe9640ff630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:08.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.434+0000 7fe969239700 1 -- 172.21.15.114:0/3606584369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe964103fb0 msgr2=0x7fe9640ff630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:08.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.434+0000 7fe969239700 1 --2- 172.21.15.114:0/3606584369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe964103fb0 0x7fe9640ff630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:08.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.434+0000 7fe969239700 1 -- 172.21.15.114:0/3606584369 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9540095f0 con 0x7fe964101680 2024-02-16T22:21:08.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.434+0000 7fe969239700 1 --2- 172.21.15.114:0/3606584369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe964101680 0x7fe964100fe0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe96000e920 tx=0x7fe96000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:21:08.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.434+0000 7fe95a7fc700 1 -- 172.21.15.114:0/3606584369 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe96000c9d0 con 0x7fe964101680 2024-02-16T22:21:08.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.434+0000 7fe96b49d700 1 -- 172.21.15.114:0/3606584369 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9640fff90 con 0x7fe964101680 2024-02-16T22:21:08.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.434+0000 7fe96b49d700 1 -- 172.21.15.114:0/3606584369 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe96419e350 con 0x7fe964101680 2024-02-16T22:21:08.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.434+0000 7fe95a7fc700 1 -- 172.21.15.114:0/3606584369 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe960010430 con 0x7fe964101680 2024-02-16T22:21:08.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.435+0000 7fe95a7fc700 1 -- 172.21.15.114:0/3606584369 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe960018650 con 0x7fe964101680 2024-02-16T22:21:08.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.436+0000 7fe96b49d700 1 -- 172.21.15.114:0/3606584369 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe964190530 con 0x7fe964101680 2024-02-16T22:21:08.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.437+0000 7fe95a7fc700 1 -- 172.21.15.114:0/3606584369 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 20) v1 ==== 90831+0+0 (secure 0 0 0) 0x7fe9600105a0 con 0x7fe964101680 2024-02-16T22:21:08.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.437+0000 7fe95a7fc700 1 --2- 172.21.15.114:0/3606584369 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fe95006cd00 0x7fe95006f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:08.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.438+0000 7fe95a7fc700 1 -- 172.21.15.114:0/3606584369 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fe960014070 con 0x7fe964101680 2024-02-16T22:21:08.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.438+0000 7fe968a38700 1 --2- 172.21.15.114:0/3606584369 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fe95006cd00 0x7fe95006f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:08.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.438+0000 7fe968a38700 1 --2- 172.21.15.114:0/3606584369 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fe95006cd00 0x7fe95006f1c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe954009fd0 tx=0x7fe954005940 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:21:08.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.445+0000 7fe95a7fc700 1 -- 172.21.15.114:0/3606584369 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe9600520e0 con 0x7fe964101680 2024-02-16T22:21:08.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.761+0000 7fe96b49d700 1 -- 172.21.15.114:0/3606584369 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9640611d0 con 0x7fe95006cd00 2024-02-16T22:21:08.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.763+0000 7fe95a7fc700 1 -- 172.21.15.114:0/3606584369 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7fe9640611d0 con 0x7fe95006cd00 2024-02-16T22:21:08.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.766+0000 7fe96b49d700 1 -- 172.21.15.114:0/3606584369 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fe95006cd00 msgr2=0x7fe95006f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:08.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.766+0000 7fe96b49d700 1 --2- 172.21.15.114:0/3606584369 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fe95006cd00 0x7fe95006f1c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe954009fd0 tx=0x7fe954005940 comp rx=0 tx=0).stop 2024-02-16T22:21:08.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.766+0000 7fe96b49d700 1 -- 172.21.15.114:0/3606584369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe964101680 msgr2=0x7fe964100fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:08.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.766+0000 7fe96b49d700 1 --2- 172.21.15.114:0/3606584369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe964101680 0x7fe964100fe0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe96000e920 tx=0x7fe96000ec30 comp rx=0 tx=0).stop 2024-02-16T22:21:08.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.766+0000 7fe96b49d700 1 -- 172.21.15.114:0/3606584369 shutdown_connections 2024-02-16T22:21:08.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.766+0000 7fe96b49d700 1 --2- 172.21.15.114:0/3606584369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe964101680 0x7fe964100fe0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:08.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.766+0000 7fe96b49d700 1 --2- 172.21.15.114:0/3606584369 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fe95006cd00 0x7fe95006f1c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:08.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.766+0000 7fe96b49d700 1 --2- 172.21.15.114:0/3606584369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe964103fb0 0x7fe9640ff630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:08.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.766+0000 7fe96b49d700 1 -- 172.21.15.114:0/3606584369 >> 172.21.15.114:0/3606584369 conn(0x7fe9640fb030 msgr2=0x7fe9640fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:21:08.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.766+0000 7fe96b49d700 1 -- 172.21.15.114:0/3606584369 shutdown_connections 2024-02-16T22:21:08.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.766+0000 7fe96b49d700 1 -- 172.21.15.114:0/3606584369 wait complete. 2024-02-16T22:21:08.778 INFO:teuthology.orchestra.run.smithi114.stdout:true 2024-02-16T22:21:08.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.884+0000 7fa59e8f9700 1 -- 172.21.15.114:0/888922386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa598103f70 msgr2=0x7fa598106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:08.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.884+0000 7fa59e8f9700 1 --2- 172.21.15.114:0/888922386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa598103f70 0x7fa598106360 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa58c009910 tx=0x7fa58c009c20 comp rx=0 tx=0).stop 2024-02-16T22:21:08.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.884+0000 7fa59e8f9700 1 -- 172.21.15.114:0/888922386 shutdown_connections 2024-02-16T22:21:08.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.884+0000 7fa59e8f9700 1 --2- 172.21.15.114:0/888922386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa598103f70 0x7fa598106360 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:08.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.884+0000 7fa59e8f9700 1 --2- 172.21.15.114:0/888922386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa598101640 0x7fa598103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:08.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.884+0000 7fa59e8f9700 1 -- 172.21.15.114:0/888922386 >> 172.21.15.114:0/888922386 conn(0x7fa5980fb030 msgr2=0x7fa5980fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:21:08.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.884+0000 7fa59e8f9700 1 -- 172.21.15.114:0/888922386 shutdown_connections 2024-02-16T22:21:08.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.884+0000 7fa59e8f9700 1 -- 172.21.15.114:0/888922386 wait complete. 2024-02-16T22:21:08.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.884+0000 7fa59e8f9700 1 Processor -- start 2024-02-16T22:21:08.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.885+0000 7fa59e8f9700 1 -- start start 2024-02-16T22:21:08.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.885+0000 7fa59e8f9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa598101640 0x7fa598100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:08.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.885+0000 7fa59e8f9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa598103f70 0x7fa5980ff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:08.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.885+0000 7fa59e8f9700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5980ffb30 con 0x7fa598101640 2024-02-16T22:21:08.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.885+0000 7fa59e8f9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5980ffc70 con 0x7fa598103f70 2024-02-16T22:21:08.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.885+0000 7fa597fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa598101640 0x7fa598100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:08.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.885+0000 7fa597fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa598101640 0x7fa598100fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48416/0 (socket says 172.21.15.114:48416) 2024-02-16T22:21:08.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.885+0000 7fa597fff700 1 -- 172.21.15.114:0/2794806489 learned_addr learned my addr 172.21.15.114:0/2794806489 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:21:08.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.885+0000 7fa5977fe700 1 --2- 172.21.15.114:0/2794806489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa598103f70 0x7fa5980ff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:08.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.885+0000 7fa597fff700 1 -- 172.21.15.114:0/2794806489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa598103f70 msgr2=0x7fa5980ff5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:08.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.885+0000 7fa597fff700 1 --2- 172.21.15.114:0/2794806489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa598103f70 0x7fa5980ff5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:08.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.885+0000 7fa597fff700 1 -- 172.21.15.114:0/2794806489 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa58c0095f0 con 0x7fa598101640 2024-02-16T22:21:08.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.886+0000 7fa597fff700 1 --2- 172.21.15.114:0/2794806489 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa598101640 0x7fa598100fa0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa58800e920 tx=0x7fa58800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:21:08.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.886+0000 7fa5957fa700 1 -- 172.21.15.114:0/2794806489 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa58800c9d0 con 0x7fa598101640 2024-02-16T22:21:08.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.886+0000 7fa59e8f9700 1 -- 172.21.15.114:0/2794806489 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5980fff50 con 0x7fa598101640 2024-02-16T22:21:08.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.886+0000 7fa5957fa700 1 -- 172.21.15.114:0/2794806489 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa588010430 con 0x7fa598101640 2024-02-16T22:21:08.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.886+0000 7fa5957fa700 1 -- 172.21.15.114:0/2794806489 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa588018650 con 0x7fa598101640 2024-02-16T22:21:08.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.886+0000 7fa59e8f9700 1 -- 172.21.15.114:0/2794806489 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa59819e390 con 0x7fa598101640 2024-02-16T22:21:08.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.887+0000 7fa59e8f9700 1 -- 172.21.15.114:0/2794806489 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa598190520 con 0x7fa598101640 2024-02-16T22:21:08.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.887+0000 7fa5957fa700 1 -- 172.21.15.114:0/2794806489 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 20) v1 ==== 90831+0+0 (secure 0 0 0) 0x7fa5880105a0 con 0x7fa598101640 2024-02-16T22:21:08.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.888+0000 7fa5957fa700 1 --2- 172.21.15.114:0/2794806489 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa58006cd00 0x7fa58006f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:08.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.888+0000 7fa5977fe700 1 --2- 172.21.15.114:0/2794806489 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa58006cd00 0x7fa58006f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:08.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.888+0000 7fa5957fa700 1 -- 172.21.15.114:0/2794806489 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fa588014070 con 0x7fa598101640 2024-02-16T22:21:08.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.888+0000 7fa5977fe700 1 --2- 172.21.15.114:0/2794806489 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa58006cd00 0x7fa58006f1c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa58c00b6f0 tx=0x7fa58c005a90 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:21:08.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:08.892+0000 7fa5957fa700 1 -- 172.21.15.114:0/2794806489 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa5880520e0 con 0x7fa598101640 2024-02-16T22:21:09.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:08 smithi114 ceph-mon[29235]: from='client.14684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:21:09.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.218+0000 7fa59e8f9700 1 -- 172.21.15.114:0/2794806489 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5980611d0 con 0x7fa58006cd00 2024-02-16T22:21:09.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.220+0000 7fa5957fa700 1 -- 172.21.15.114:0/2794806489 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7fa5980611d0 con 0x7fa58006cd00 2024-02-16T22:21:09.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.223+0000 7fa59e8f9700 1 -- 172.21.15.114:0/2794806489 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa58006cd00 msgr2=0x7fa58006f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:09.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.223+0000 7fa59e8f9700 1 --2- 172.21.15.114:0/2794806489 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa58006cd00 0x7fa58006f1c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa58c00b6f0 tx=0x7fa58c005a90 comp rx=0 tx=0).stop 2024-02-16T22:21:09.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.223+0000 7fa59e8f9700 1 -- 172.21.15.114:0/2794806489 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa598101640 msgr2=0x7fa598100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:09.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.223+0000 7fa59e8f9700 1 --2- 172.21.15.114:0/2794806489 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa598101640 0x7fa598100fa0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa58800e920 tx=0x7fa58800ec30 comp rx=0 tx=0).stop 2024-02-16T22:21:09.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.223+0000 7fa59e8f9700 1 -- 172.21.15.114:0/2794806489 shutdown_connections 2024-02-16T22:21:09.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.223+0000 7fa59e8f9700 1 --2- 172.21.15.114:0/2794806489 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa598101640 0x7fa598100fa0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:09.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.223+0000 7fa59e8f9700 1 --2- 172.21.15.114:0/2794806489 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7fa58006cd00 0x7fa58006f1c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:09.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.223+0000 7fa59e8f9700 1 --2- 172.21.15.114:0/2794806489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa598103f70 0x7fa5980ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:09.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.223+0000 7fa59e8f9700 1 -- 172.21.15.114:0/2794806489 >> 172.21.15.114:0/2794806489 conn(0x7fa5980fb030 msgr2=0x7fa5980fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:21:09.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.223+0000 7fa59e8f9700 1 -- 172.21.15.114:0/2794806489 shutdown_connections 2024-02-16T22:21:09.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.223+0000 7fa59e8f9700 1 -- 172.21.15.114:0/2794806489 wait complete. 2024-02-16T22:21:09.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:08 smithi123 ceph-mon[37161]: from='client.14684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:21:09.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.355+0000 7f62fe876700 1 -- 172.21.15.114:0/365441117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62f8103f70 msgr2=0x7f62f8106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:09.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.355+0000 7f62fe876700 1 --2- 172.21.15.114:0/365441117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62f8103f70 0x7f62f8106360 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f62ec009910 tx=0x7f62ec009c20 comp rx=0 tx=0).stop 2024-02-16T22:21:09.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.355+0000 7f62fe876700 1 -- 172.21.15.114:0/365441117 shutdown_connections 2024-02-16T22:21:09.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.355+0000 7f62fe876700 1 --2- 172.21.15.114:0/365441117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62f8103f70 0x7f62f8106360 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:09.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.355+0000 7f62fe876700 1 --2- 172.21.15.114:0/365441117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f8101640 0x7f62f8103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:09.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.355+0000 7f62fe876700 1 -- 172.21.15.114:0/365441117 >> 172.21.15.114:0/365441117 conn(0x7f62f80fb030 msgr2=0x7f62f80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:21:09.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.355+0000 7f62fe876700 1 -- 172.21.15.114:0/365441117 shutdown_connections 2024-02-16T22:21:09.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.355+0000 7f62fe876700 1 -- 172.21.15.114:0/365441117 wait complete. 2024-02-16T22:21:09.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.356+0000 7f62fe876700 1 Processor -- start 2024-02-16T22:21:09.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.356+0000 7f62fe876700 1 -- start start 2024-02-16T22:21:09.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.356+0000 7f62fe876700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f8101640 0x7f62f8196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:09.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.356+0000 7f62fe876700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62f8103f70 0x7f62f8196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:09.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.356+0000 7f62fe876700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f8196e90 con 0x7f62f8103f70 2024-02-16T22:21:09.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.356+0000 7f62fe876700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f8196fd0 con 0x7f62f8101640 2024-02-16T22:21:09.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.357+0000 7f62f77fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62f8103f70 0x7f62f8196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:09.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.357+0000 7f62f77fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62f8103f70 0x7f62f8196870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48424/0 (socket says 172.21.15.114:48424) 2024-02-16T22:21:09.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.357+0000 7f62f7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f8101640 0x7f62f8196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:09.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.357+0000 7f62f77fe700 1 -- 172.21.15.114:0/2477444379 learned_addr learned my addr 172.21.15.114:0/2477444379 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:21:09.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.357+0000 7f62f77fe700 1 -- 172.21.15.114:0/2477444379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f8101640 msgr2=0x7f62f8196330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:09.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.357+0000 7f62f77fe700 1 --2- 172.21.15.114:0/2477444379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f8101640 0x7f62f8196330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:09.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.357+0000 7f62f77fe700 1 -- 172.21.15.114:0/2477444379 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec0095f0 con 0x7f62f8103f70 2024-02-16T22:21:09.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.357+0000 7f62f7fff700 1 --2- 172.21.15.114:0/2477444379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f8101640 0x7f62f8196330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:21:09.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.357+0000 7f62f77fe700 1 --2- 172.21.15.114:0/2477444379 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62f8103f70 0x7f62f8196870 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f62ec000c00 tx=0x7f62ec00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:21:09.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.357+0000 7f62f57fa700 1 -- 172.21.15.114:0/2477444379 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62ec01d070 con 0x7f62f8103f70 2024-02-16T22:21:09.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.357+0000 7f62fe876700 1 -- 172.21.15.114:0/2477444379 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62f819ba20 con 0x7f62f8103f70 2024-02-16T22:21:09.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.357+0000 7f62f57fa700 1 -- 172.21.15.114:0/2477444379 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f62ec005dc0 con 0x7f62f8103f70 2024-02-16T22:21:09.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.358+0000 7f62f57fa700 1 -- 172.21.15.114:0/2477444379 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62ec0226a0 con 0x7f62f8103f70 2024-02-16T22:21:09.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.358+0000 7f62fe876700 1 -- 172.21.15.114:0/2477444379 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62f819bf10 con 0x7f62f8103f70 2024-02-16T22:21:09.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.359+0000 7f62fe876700 1 -- 172.21.15.114:0/2477444379 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62f8190520 con 0x7f62f8103f70 2024-02-16T22:21:09.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.359+0000 7f62f57fa700 1 -- 172.21.15.114:0/2477444379 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 20) v1 ==== 90831+0+0 (secure 0 0 0) 0x7f62ec022800 con 0x7f62f8103f70 2024-02-16T22:21:09.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.359+0000 7f62f57fa700 1 --2- 172.21.15.114:0/2477444379 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f62e006cd00 0x7f62e006f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:09.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.360+0000 7f62f57fa700 1 -- 172.21.15.114:0/2477444379 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f62ec08d7f0 con 0x7f62f8103f70 2024-02-16T22:21:09.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.360+0000 7f62f7fff700 1 --2- 172.21.15.114:0/2477444379 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f62e006cd00 0x7f62e006f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:09.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.360+0000 7f62f7fff700 1 --2- 172.21.15.114:0/2477444379 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f62e006cd00 0x7f62e006f1c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f62e8011490 tx=0x7f62e8009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:21:09.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.365+0000 7f62f57fa700 1 -- 172.21.15.114:0/2477444379 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f62ec05ba60 con 0x7f62f8103f70 2024-02-16T22:21:09.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.621+0000 7f62fe876700 1 -- 172.21.15.114:0/2477444379 --> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f62f80611d0 con 0x7f62e006cd00 2024-02-16T22:21:09.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.633+0000 7f62f57fa700 1 -- 172.21.15.114:0/2477444379 <== mgr.14215 v2:172.21.15.114:6800/3153243195 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f62f80611d0 con 0x7f62e006cd00 2024-02-16T22:21:09.635 INFO:teuthology.orchestra.run.smithi114.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:21:09.635 INFO:teuthology.orchestra.run.smithi114.stdout:alertmanager.smithi114 smithi114 *:9093,9094 running (7m) 2m ago 8m 24.4M - 0.25.0 c8568f914cd2 e7404fea400e 2024-02-16T22:21:09.635 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi114 smithi114 running (8m) 2m ago 8m 7440k - 18.2.1 94eadd990e0a 7fd07162cd96 2024-02-16T22:21:09.636 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi123 smithi123 running (7m) 13s ago 7m 7377k - 18.2.1 94eadd990e0a 7687f5e99621 2024-02-16T22:21:09.636 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi114 smithi114 running (8m) 2m ago 8m 6660k - 18.2.1 94eadd990e0a fc8fbfe4fdd5 2024-02-16T22:21:09.636 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi123 smithi123 running (7m) 13s ago 7m 6665k - 18.2.1 94eadd990e0a a6249642b9bb 2024-02-16T22:21:09.636 INFO:teuthology.orchestra.run.smithi114.stdout:grafana.smithi114 smithi114 *:3000 running (6m) 2m ago 8m 88.1M - 9.4.7 954c08fa6188 db1c26c3bc20 2024-02-16T22:21:09.636 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.fdxpey smithi114 running (2m) 2m ago 2m 13.2M - 18.2.1 94eadd990e0a 704b40c144fb 2024-02-16T22:21:09.636 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.zzryke smithi114 running (3m) 2m ago 3m 16.4M - 18.2.1 94eadd990e0a 1f795d23c9a4 2024-02-16T22:21:09.636 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.gopcfa smithi123 running (2m) 13s ago 2m 14.5M - 18.2.1 94eadd990e0a 166134b7532a 2024-02-16T22:21:09.636 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.vhlqbe smithi123 running (3m) 13s ago 3m 16.0M - 18.2.1 94eadd990e0a 243c8bb96fa8 2024-02-16T22:21:09.636 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi114.vcullz smithi114 *:9283,8765,8443 running (10m) 2m ago 10m 488M - 18.2.1 94eadd990e0a c8f21783728d 2024-02-16T22:21:09.636 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi123.inzgjd smithi123 *:8443,9283,8765 running (17s) 13s ago 7m 55.0M - 19.0.0-1450-gf78a58c0 f081f847070a c9d6ba733f8b 2024-02-16T22:21:09.636 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi114 smithi114 running (10m) 2m ago 10m 49.9M 2048M 18.2.1 94eadd990e0a b4c4433aa3a4 2024-02-16T22:21:09.636 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi123 smithi123 running (7m) 13s ago 7m 35.7M 2048M 18.2.1 94eadd990e0a c217fc30c152 2024-02-16T22:21:09.637 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi114 smithi114 *:9100 running (8m) 2m ago 8m 15.6M - 1.5.0 0da6a335fe13 744a090e06d0 2024-02-16T22:21:09.637 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi123 smithi123 *:9100 running (7m) 13s ago 7m 15.1M - 1.5.0 0da6a335fe13 3994b8c29b2e 2024-02-16T22:21:09.637 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 smithi114 running (6m) 2m ago 6m 47.7M 3946M 18.2.1 94eadd990e0a 24ca2e9714e0 2024-02-16T22:21:09.637 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 smithi114 running (6m) 2m ago 6m 47.8M 3946M 18.2.1 94eadd990e0a 32bb191b88e7 2024-02-16T22:21:09.637 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 smithi114 running (5m) 2m ago 5m 48.1M 3946M 18.2.1 94eadd990e0a 6608718d0818 2024-02-16T22:21:09.637 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 smithi123 running (5m) 13s ago 5m 58.9M 4970M 18.2.1 94eadd990e0a 575db379de3d 2024-02-16T22:21:09.637 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 smithi123 running (4m) 13s ago 4m 52.6M 4970M 18.2.1 94eadd990e0a 7e266866cd9c 2024-02-16T22:21:09.637 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 smithi123 running (4m) 13s ago 4m 51.2M 4970M 18.2.1 94eadd990e0a 24e5bef19fda 2024-02-16T22:21:09.637 INFO:teuthology.orchestra.run.smithi114.stdout:prometheus.smithi114 smithi114 *:9095 running (6m) 2m ago 8m 55.9M - 2.43.0 a07b618ecd1d 51185086d582 2024-02-16T22:21:09.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.636+0000 7f62fe876700 1 -- 172.21.15.114:0/2477444379 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f62e006cd00 msgr2=0x7f62e006f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:09.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.636+0000 7f62fe876700 1 --2- 172.21.15.114:0/2477444379 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f62e006cd00 0x7f62e006f1c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f62e8011490 tx=0x7f62e8009260 comp rx=0 tx=0).stop 2024-02-16T22:21:09.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.636+0000 7f62fe876700 1 -- 172.21.15.114:0/2477444379 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62f8103f70 msgr2=0x7f62f8196870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:09.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.636+0000 7f62fe876700 1 --2- 172.21.15.114:0/2477444379 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62f8103f70 0x7f62f8196870 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f62ec000c00 tx=0x7f62ec00f690 comp rx=0 tx=0).stop 2024-02-16T22:21:09.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.636+0000 7f62fe876700 1 -- 172.21.15.114:0/2477444379 shutdown_connections 2024-02-16T22:21:09.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.636+0000 7f62fe876700 1 --2- 172.21.15.114:0/2477444379 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f62e006cd00 0x7f62e006f1c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:09.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.636+0000 7f62fe876700 1 --2- 172.21.15.114:0/2477444379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f8101640 0x7f62f8196330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:09.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.636+0000 7f62fe876700 1 --2- 172.21.15.114:0/2477444379 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62f8103f70 0x7f62f8196870 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:09.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.636+0000 7f62fe876700 1 -- 172.21.15.114:0/2477444379 >> 172.21.15.114:0/2477444379 conn(0x7f62f80fb030 msgr2=0x7f62f80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:21:09.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.636+0000 7f62fe876700 1 -- 172.21.15.114:0/2477444379 shutdown_connections 2024-02-16T22:21:09.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.636+0000 7f62fe876700 1 -- 172.21.15.114:0/2477444379 wait complete. 2024-02-16T22:21:09.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.769+0000 7f3fb1a67700 1 -- 172.21.15.114:0/614076003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3fac100bc0 msgr2=0x7f3fac100fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:09.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.769+0000 7f3fb1a67700 1 --2- 172.21.15.114:0/614076003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3fac100bc0 0x7f3fac100fe0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c009910 tx=0x7f3f9c009c20 comp rx=0 tx=0).stop 2024-02-16T22:21:09.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.769+0000 7f3fb1a67700 1 -- 172.21.15.114:0/614076003 shutdown_connections 2024-02-16T22:21:09.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.769+0000 7f3fb1a67700 1 --2- 172.21.15.114:0/614076003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fac101dc0 0x7f3fac102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:09.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.769+0000 7f3fb1a67700 1 --2- 172.21.15.114:0/614076003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3fac100bc0 0x7f3fac100fe0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:09.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.769+0000 7f3fb1a67700 1 -- 172.21.15.114:0/614076003 >> 172.21.15.114:0/614076003 conn(0x7f3fac0fc140 msgr2=0x7f3fac0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:21:09.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.769+0000 7f3fb1a67700 1 -- 172.21.15.114:0/614076003 shutdown_connections 2024-02-16T22:21:09.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.769+0000 7f3fb1a67700 1 -- 172.21.15.114:0/614076003 wait complete. 2024-02-16T22:21:09.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.769+0000 7f3fb1a67700 1 Processor -- start 2024-02-16T22:21:09.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.770+0000 7f3fb1a67700 1 -- start start 2024-02-16T22:21:09.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.770+0000 7f3fb1a67700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3fac100bc0 0x7f3fac074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:09.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.770+0000 7f3fb1a67700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fac101dc0 0x7f3fac073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:09.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.770+0000 7f3fb1a67700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac0736c0 con 0x7f3fac100bc0 2024-02-16T22:21:09.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.770+0000 7f3fb1a67700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac073800 con 0x7f3fac101dc0 2024-02-16T22:21:09.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.770+0000 7f3fab7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3fac100bc0 0x7f3fac074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:09.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.770+0000 7f3faaffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fac101dc0 0x7f3fac073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:09.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.770+0000 7f3fab7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3fac100bc0 0x7f3fac074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48450/0 (socket says 172.21.15.114:48450) 2024-02-16T22:21:09.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.770+0000 7f3fab7fe700 1 -- 172.21.15.114:0/2902849354 learned_addr learned my addr 172.21.15.114:0/2902849354 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:21:09.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.770+0000 7f3faaffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fac101dc0 0x7f3fac073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.114:59974/0 (socket says 172.21.15.114:59974) 2024-02-16T22:21:09.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.770+0000 7f3faaffd700 1 -- 172.21.15.114:0/2902849354 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3fac100bc0 msgr2=0x7f3fac074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:09.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.770+0000 7f3faaffd700 1 --2- 172.21.15.114:0/2902849354 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3fac100bc0 0x7f3fac074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:09.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.770+0000 7f3faaffd700 1 -- 172.21.15.114:0/2902849354 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f9c0095f0 con 0x7f3fac101dc0 2024-02-16T22:21:09.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.770+0000 7f3faaffd700 1 --2- 172.21.15.114:0/2902849354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fac101dc0 0x7f3fac073180 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3fa000e970 tx=0x7f3fa000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:21:09.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.771+0000 7f3fa8ff9700 1 -- 172.21.15.114:0/2902849354 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fa000c9d0 con 0x7f3fac101dc0 2024-02-16T22:21:09.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.771+0000 7f3fa8ff9700 1 -- 172.21.15.114:0/2902849354 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3fa0010430 con 0x7f3fac101dc0 2024-02-16T22:21:09.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.771+0000 7f3fa8ff9700 1 -- 172.21.15.114:0/2902849354 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fa0018650 con 0x7f3fac101dc0 2024-02-16T22:21:09.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.771+0000 7f3fb1a67700 1 -- 172.21.15.114:0/2902849354 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fac073ae0 con 0x7f3fac101dc0 2024-02-16T22:21:09.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.771+0000 7f3fb1a67700 1 -- 172.21.15.114:0/2902849354 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fac19e3f0 con 0x7f3fac101dc0 2024-02-16T22:21:09.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.772+0000 7f3fb1a67700 1 -- 172.21.15.114:0/2902849354 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fac109720 con 0x7f3fac101dc0 2024-02-16T22:21:09.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.772+0000 7f3fa8ff9700 1 -- 172.21.15.114:0/2902849354 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) v1 ==== 90831+0+0 (secure 0 0 0) 0x7f3fa00105a0 con 0x7f3fac101dc0 2024-02-16T22:21:09.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.773+0000 7f3fa8ff9700 1 --2- 172.21.15.114:0/2902849354 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f3f9406cd00 0x7f3f9406f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:09.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.773+0000 7f3fa8ff9700 1 -- 172.21.15.114:0/2902849354 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f3fa0014070 con 0x7f3fac101dc0 2024-02-16T22:21:09.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.773+0000 7f3fab7fe700 1 --2- 172.21.15.114:0/2902849354 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f3f9406cd00 0x7f3f9406f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:09.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.773+0000 7f3fab7fe700 1 --2- 172.21.15.114:0/2902849354 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f3f9406cd00 0x7f3f9406f1c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c009fd0 tx=0x7f3f9c005940 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:21:09.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:09.778+0000 7f3fa8ff9700 1 -- 172.21.15.114:0/2902849354 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3fa00520e0 con 0x7f3fac101dc0 2024-02-16T22:21:10.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.162+0000 7f3fb1a67700 1 -- 172.21.15.114:0/2902849354 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3fac1098e0 con 0x7f3fac101dc0 2024-02-16T22:21:10.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.162+0000 7f3fa8ff9700 1 -- 172.21.15.114:0/2902849354 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+766 (secure 0 0 0) 0x7f3fa005a6b0 con 0x7f3fac101dc0 2024-02-16T22:21:10.165 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:21:10.165 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": { 2024-02-16T22:21:10.165 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-02-16T22:21:10.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:21:10.165 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": { 2024-02-16T22:21:10.165 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-02-16T22:21:10.165 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T22:21:10.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:21:10.165 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": { 2024-02-16T22:21:10.165 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-02-16T22:21:10.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:21:10.165 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-02-16T22:21:10.165 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-02-16T22:21:10.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:21:10.166 INFO:teuthology.orchestra.run.smithi114.stdout: "overall": { 2024-02-16T22:21:10.166 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 13, 2024-02-16T22:21:10.166 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T22:21:10.166 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-02-16T22:21:10.166 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:21:10.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.165+0000 7f3fb1a67700 1 -- 172.21.15.114:0/2902849354 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f3f9406cd00 msgr2=0x7f3f9406f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:10.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.165+0000 7f3fb1a67700 1 --2- 172.21.15.114:0/2902849354 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f3f9406cd00 0x7f3f9406f1c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c009fd0 tx=0x7f3f9c005940 comp rx=0 tx=0).stop 2024-02-16T22:21:10.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.165+0000 7f3fb1a67700 1 -- 172.21.15.114:0/2902849354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fac101dc0 msgr2=0x7f3fac073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:10.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.165+0000 7f3fb1a67700 1 --2- 172.21.15.114:0/2902849354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fac101dc0 0x7f3fac073180 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3fa000e970 tx=0x7f3fa000ec80 comp rx=0 tx=0).stop 2024-02-16T22:21:10.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.165+0000 7f3fb1a67700 1 -- 172.21.15.114:0/2902849354 shutdown_connections 2024-02-16T22:21:10.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.165+0000 7f3fb1a67700 1 --2- 172.21.15.114:0/2902849354 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3fac100bc0 0x7f3fac074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:10.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.165+0000 7f3fb1a67700 1 --2- 172.21.15.114:0/2902849354 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7f3f9406cd00 0x7f3f9406f1c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:10.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.165+0000 7f3fb1a67700 1 --2- 172.21.15.114:0/2902849354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fac101dc0 0x7f3fac073180 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:10.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.165+0000 7f3fb1a67700 1 -- 172.21.15.114:0/2902849354 >> 172.21.15.114:0/2902849354 conn(0x7f3fac0fc140 msgr2=0x7f3fac104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:21:10.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.165+0000 7f3fb1a67700 1 -- 172.21.15.114:0/2902849354 shutdown_connections 2024-02-16T22:21:10.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.165+0000 7f3fb1a67700 1 -- 172.21.15.114:0/2902849354 wait complete. 2024-02-16T22:21:10.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:09 smithi114 ceph-mon[29235]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:21:10.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:09 smithi114 ceph-mon[29235]: pgmap v272: 65 pgs: 65 active+clean; 84 MiB data, 442 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.7 MiB/s wr, 213 op/s 2024-02-16T22:21:10.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:09 smithi114 ceph-mon[29235]: from='client.14692 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:21:10.203 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:09 smithi123 ceph-mon[37161]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:21:10.203 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:09 smithi123 ceph-mon[37161]: pgmap v272: 65 pgs: 65 active+clean; 84 MiB data, 442 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.7 MiB/s wr, 213 op/s 2024-02-16T22:21:10.203 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:09 smithi123 ceph-mon[37161]: from='client.14692 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:21:10.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.313+0000 7ff66bda3700 1 -- 172.21.15.114:0/2158504618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff664100bb0 msgr2=0x7ff664100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:10.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.313+0000 7ff66bda3700 1 --2- 172.21.15.114:0/2158504618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff664100bb0 0x7ff664100fd0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff660009910 tx=0x7ff660009c20 comp rx=0 tx=0).stop 2024-02-16T22:21:10.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.313+0000 7ff66bda3700 1 -- 172.21.15.114:0/2158504618 shutdown_connections 2024-02-16T22:21:10.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.313+0000 7ff66bda3700 1 --2- 172.21.15.114:0/2158504618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff664101db0 0x7ff664102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:10.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.313+0000 7ff66bda3700 1 --2- 172.21.15.114:0/2158504618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff664100bb0 0x7ff664100fd0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:10.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.313+0000 7ff66bda3700 1 -- 172.21.15.114:0/2158504618 >> 172.21.15.114:0/2158504618 conn(0x7ff6640fc110 msgr2=0x7ff6640fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:21:10.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.313+0000 7ff66bda3700 1 -- 172.21.15.114:0/2158504618 shutdown_connections 2024-02-16T22:21:10.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.313+0000 7ff66bda3700 1 -- 172.21.15.114:0/2158504618 wait complete. 2024-02-16T22:21:10.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.314+0000 7ff66bda3700 1 Processor -- start 2024-02-16T22:21:10.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.314+0000 7ff66bda3700 1 -- start start 2024-02-16T22:21:10.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.314+0000 7ff66bda3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff664100bb0 0x7ff664074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:10.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.314+0000 7ff66bda3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff664101db0 0x7ff664073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:10.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.314+0000 7ff66bda3700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6640736c0 con 0x7ff664101db0 2024-02-16T22:21:10.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.314+0000 7ff66bda3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff664073800 con 0x7ff664100bb0 2024-02-16T22:21:10.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.314+0000 7ff66933e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff664101db0 0x7ff664073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:10.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.314+0000 7ff669b3f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff664100bb0 0x7ff664074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:10.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.315+0000 7ff66933e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff664101db0 0x7ff664073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48470/0 (socket says 172.21.15.114:48470) 2024-02-16T22:21:10.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.315+0000 7ff66933e700 1 -- 172.21.15.114:0/3838205903 learned_addr learned my addr 172.21.15.114:0/3838205903 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:21:10.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.315+0000 7ff66933e700 1 -- 172.21.15.114:0/3838205903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff664100bb0 msgr2=0x7ff664074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:10.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.315+0000 7ff66933e700 1 --2- 172.21.15.114:0/3838205903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff664100bb0 0x7ff664074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:10.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.315+0000 7ff66933e700 1 -- 172.21.15.114:0/3838205903 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6600095f0 con 0x7ff664101db0 2024-02-16T22:21:10.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.315+0000 7ff669b3f700 1 --2- 172.21.15.114:0/3838205903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff664100bb0 0x7ff664074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:21:10.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.315+0000 7ff66933e700 1 --2- 172.21.15.114:0/3838205903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff664101db0 0x7ff664073180 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff65400d8d0 tx=0x7ff65400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:21:10.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.315+0000 7ff65affd700 1 -- 172.21.15.114:0/3838205903 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6540098a0 con 0x7ff664101db0 2024-02-16T22:21:10.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.315+0000 7ff65affd700 1 -- 172.21.15.114:0/3838205903 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff654010460 con 0x7ff664101db0 2024-02-16T22:21:10.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.315+0000 7ff66bda3700 1 -- 172.21.15.114:0/3838205903 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff664073ae0 con 0x7ff664101db0 2024-02-16T22:21:10.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.315+0000 7ff65affd700 1 -- 172.21.15.114:0/3838205903 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff65400f5f0 con 0x7ff664101db0 2024-02-16T22:21:10.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.315+0000 7ff66bda3700 1 -- 172.21.15.114:0/3838205903 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff66419e3f0 con 0x7ff664101db0 2024-02-16T22:21:10.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.317+0000 7ff66bda3700 1 -- 172.21.15.114:0/3838205903 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff664109710 con 0x7ff664101db0 2024-02-16T22:21:10.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.317+0000 7ff65affd700 1 -- 172.21.15.114:0/3838205903 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 20) v1 ==== 90831+0+0 (secure 0 0 0) 0x7ff654009a00 con 0x7ff664101db0 2024-02-16T22:21:10.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.318+0000 7ff65affd700 1 --2- 172.21.15.114:0/3838205903 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff65006cd00 0x7ff65006f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:10.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.318+0000 7ff65affd700 1 -- 172.21.15.114:0/3838205903 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7ff65408c070 con 0x7ff664101db0 2024-02-16T22:21:10.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.318+0000 7ff669b3f700 1 --2- 172.21.15.114:0/3838205903 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff65006cd00 0x7ff65006f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:10.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.318+0000 7ff669b3f700 1 --2- 172.21.15.114:0/3838205903 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff65006cd00 0x7ff65006f1c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff660006010 tx=0x7ff66000b930 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:21:10.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.324+0000 7ff65affd700 1 -- 172.21.15.114:0/3838205903 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff65405a360 con 0x7ff664101db0 2024-02-16T22:21:10.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.412+0000 7ff65affd700 1 -- 172.21.15.114:0/3838205903 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mgrmap(e 21) v1 ==== 45594+0+0 (secure 0 0 0) 0x7ff654056ca0 con 0x7ff664101db0 2024-02-16T22:21:10.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.412+0000 7ff65affd700 1 -- 172.21.15.114:0/3838205903 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff65006cd00 msgr2=0x7ff65006f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:10.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.412+0000 7ff65affd700 1 --2- 172.21.15.114:0/3838205903 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff65006cd00 0x7ff65006f1c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff660006010 tx=0x7ff66000b930 comp rx=0 tx=0).stop 2024-02-16T22:21:10.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.682+0000 7ff66bda3700 1 -- 172.21.15.114:0/3838205903 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff6641098d0 con 0x7ff664101db0 2024-02-16T22:21:10.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.683+0000 7ff65affd700 1 -- 172.21.15.114:0/3838205903 <== mon.0 v2:172.21.15.114:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1825 (secure 0 0 0) 0x7ff65408a7a0 con 0x7ff664101db0 2024-02-16T22:21:10.685 INFO:teuthology.orchestra.run.smithi114.stdout:e14 2024-02-16T22:21:10.685 INFO:teuthology.orchestra.run.smithi114.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:21:10.685 INFO:teuthology.orchestra.run.smithi114.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:21:10.685 INFO:teuthology.orchestra.run.smithi114.stdout:legacy client fscid: 1 2024-02-16T22:21:10.685 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:21:10.685 INFO:teuthology.orchestra.run.smithi114.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:21:10.685 INFO:teuthology.orchestra.run.smithi114.stdout:fs_name cephfs 2024-02-16T22:21:10.685 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 12 2024-02-16T22:21:10.686 INFO:teuthology.orchestra.run.smithi114.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:21:10.686 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-02-16T22:18:02.108825+0000 2024-02-16T22:21:10.686 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-02-16T22:18:16.413371+0000 2024-02-16T22:21:10.686 INFO:teuthology.orchestra.run.smithi114.stdout:tableserver 0 2024-02-16T22:21:10.686 INFO:teuthology.orchestra.run.smithi114.stdout:root 0 2024-02-16T22:21:10.686 INFO:teuthology.orchestra.run.smithi114.stdout:session_timeout 60 2024-02-16T22:21:10.686 INFO:teuthology.orchestra.run.smithi114.stdout:session_autoclose 300 2024-02-16T22:21:10.686 INFO:teuthology.orchestra.run.smithi114.stdout:max_file_size 1099511627776 2024-02-16T22:21:10.686 INFO:teuthology.orchestra.run.smithi114.stdout:required_client_features {} 2024-02-16T22:21:10.686 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure 0 2024-02-16T22:21:10.686 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure_osd_epoch 0 2024-02-16T22:21:10.686 INFO:teuthology.orchestra.run.smithi114.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:21:10.686 INFO:teuthology.orchestra.run.smithi114.stdout:max_mds 1 2024-02-16T22:21:10.686 INFO:teuthology.orchestra.run.smithi114.stdout:in 0 2024-02-16T22:21:10.686 INFO:teuthology.orchestra.run.smithi114.stdout:up {0=14468} 2024-02-16T22:21:10.686 INFO:teuthology.orchestra.run.smithi114.stdout:failed 2024-02-16T22:21:10.687 INFO:teuthology.orchestra.run.smithi114.stdout:damaged 2024-02-16T22:21:10.687 INFO:teuthology.orchestra.run.smithi114.stdout:stopped 2024-02-16T22:21:10.687 INFO:teuthology.orchestra.run.smithi114.stdout:data_pools [3] 2024-02-16T22:21:10.687 INFO:teuthology.orchestra.run.smithi114.stdout:metadata_pool 2 2024-02-16T22:21:10.687 INFO:teuthology.orchestra.run.smithi114.stdout:inline_data enabled 2024-02-16T22:21:10.687 INFO:teuthology.orchestra.run.smithi114.stdout:balancer 2024-02-16T22:21:10.687 INFO:teuthology.orchestra.run.smithi114.stdout:bal_rank_mask -1 2024-02-16T22:21:10.687 INFO:teuthology.orchestra.run.smithi114.stdout:standby_count_wanted 1 2024-02-16T22:21:10.687 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.zzryke{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.114:6826/277953535,v1:172.21.15.114:6827/277953535] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:21:10.687 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:21:10.687 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:21:10.687 INFO:teuthology.orchestra.run.smithi114.stdout:Standby daemons: 2024-02-16T22:21:10.687 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:21:10.687 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.fdxpey{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:21:10.687 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.vhlqbe{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:21:10.687 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.gopcfa{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:21:10.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.686+0000 7ff66bda3700 1 -- 172.21.15.114:0/3838205903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff664101db0 msgr2=0x7ff664073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:10.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.686+0000 7ff66bda3700 1 --2- 172.21.15.114:0/3838205903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff664101db0 0x7ff664073180 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff65400d8d0 tx=0x7ff65400dbe0 comp rx=0 tx=0).stop 2024-02-16T22:21:10.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.686+0000 7ff66bda3700 1 -- 172.21.15.114:0/3838205903 shutdown_connections 2024-02-16T22:21:10.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.686+0000 7ff66bda3700 1 --2- 172.21.15.114:0/3838205903 >> [v2:172.21.15.114:6800/3153243195,v1:172.21.15.114:6801/3153243195] conn(0x7ff65006cd00 0x7ff65006f1c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:10.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.686+0000 7ff66bda3700 1 --2- 172.21.15.114:0/3838205903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff664100bb0 0x7ff664074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:10.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.686+0000 7ff66bda3700 1 --2- 172.21.15.114:0/3838205903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff664101db0 0x7ff664073180 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:10.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.686+0000 7ff66bda3700 1 -- 172.21.15.114:0/3838205903 >> 172.21.15.114:0/3838205903 conn(0x7ff6640fc110 msgr2=0x7ff664104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:21:10.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.686+0000 7ff66bda3700 1 -- 172.21.15.114:0/3838205903 shutdown_connections 2024-02-16T22:21:10.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.686+0000 7ff66bda3700 1 -- 172.21.15.114:0/3838205903 wait complete. 2024-02-16T22:21:10.689 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 14 2024-02-16T22:21:10.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.837+0000 7fd911078700 1 -- 172.21.15.114:0/55059748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd90c103f70 msgr2=0x7fd90c106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:10.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.837+0000 7fd911078700 1 --2- 172.21.15.114:0/55059748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd90c103f70 0x7fd90c106360 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd900009910 tx=0x7fd900009c20 comp rx=0 tx=0).stop 2024-02-16T22:21:10.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.837+0000 7fd911078700 1 -- 172.21.15.114:0/55059748 shutdown_connections 2024-02-16T22:21:10.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.837+0000 7fd911078700 1 --2- 172.21.15.114:0/55059748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd90c103f70 0x7fd90c106360 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:10.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.837+0000 7fd911078700 1 --2- 172.21.15.114:0/55059748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd90c101640 0x7fd90c103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:10.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.837+0000 7fd911078700 1 -- 172.21.15.114:0/55059748 >> 172.21.15.114:0/55059748 conn(0x7fd90c0fb030 msgr2=0x7fd90c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:21:10.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.838+0000 7fd911078700 1 -- 172.21.15.114:0/55059748 shutdown_connections 2024-02-16T22:21:10.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.838+0000 7fd911078700 1 -- 172.21.15.114:0/55059748 wait complete. 2024-02-16T22:21:10.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.839+0000 7fd911078700 1 Processor -- start 2024-02-16T22:21:10.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.839+0000 7fd911078700 1 -- start start 2024-02-16T22:21:10.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.839+0000 7fd911078700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd90c101640 0x7fd90c196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:10.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.839+0000 7fd911078700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd90c103f70 0x7fd90c196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:10.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.839+0000 7fd911078700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c196e80 con 0x7fd90c103f70 2024-02-16T22:21:10.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.839+0000 7fd911078700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c196fc0 con 0x7fd90c101640 2024-02-16T22:21:10.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.840+0000 7fd90a59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd90c103f70 0x7fd90c196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:10.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.840+0000 7fd90ad9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd90c101640 0x7fd90c196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:10.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.840+0000 7fd90a59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd90c103f70 0x7fd90c196860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48502/0 (socket says 172.21.15.114:48502) 2024-02-16T22:21:10.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.840+0000 7fd90a59c700 1 -- 172.21.15.114:0/2252802319 learned_addr learned my addr 172.21.15.114:0/2252802319 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:21:10.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.840+0000 7fd90a59c700 1 -- 172.21.15.114:0/2252802319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd90c101640 msgr2=0x7fd90c196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:10.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.840+0000 7fd90a59c700 1 --2- 172.21.15.114:0/2252802319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd90c101640 0x7fd90c196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:10.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.840+0000 7fd90a59c700 1 -- 172.21.15.114:0/2252802319 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9000095f0 con 0x7fd90c103f70 2024-02-16T22:21:10.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.840+0000 7fd90a59c700 1 --2- 172.21.15.114:0/2252802319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd90c103f70 0x7fd90c196860 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd900000c00 tx=0x7fd90000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:21:10.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.841+0000 7fd8f3fff700 1 -- 172.21.15.114:0/2252802319 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd90001d070 con 0x7fd90c103f70 2024-02-16T22:21:10.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.841+0000 7fd8f3fff700 1 -- 172.21.15.114:0/2252802319 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd900005dc0 con 0x7fd90c103f70 2024-02-16T22:21:10.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.841+0000 7fd8f3fff700 1 -- 172.21.15.114:0/2252802319 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9000226a0 con 0x7fd90c103f70 2024-02-16T22:21:10.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.841+0000 7fd911078700 1 -- 172.21.15.114:0/2252802319 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd90c19ba10 con 0x7fd90c103f70 2024-02-16T22:21:10.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.841+0000 7fd911078700 1 -- 172.21.15.114:0/2252802319 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd90c19beb0 con 0x7fd90c103f70 2024-02-16T22:21:10.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.842+0000 7fd911078700 1 -- 172.21.15.114:0/2252802319 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd90c190540 con 0x7fd90c103f70 2024-02-16T22:21:10.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.842+0000 7fd8f3fff700 1 -- 172.21.15.114:0/2252802319 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 21) v1 ==== 45594+0+0 (secure 0 0 0) 0x7fd9000038f0 con 0x7fd90c103f70 2024-02-16T22:21:10.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.843+0000 7fd8f3fff700 1 -- 172.21.15.114:0/2252802319 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5726+0+0 (secure 0 0 0) 0x7fd90004db10 con 0x7fd90c103f70 2024-02-16T22:21:10.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:10.850+0000 7fd8f3fff700 1 -- 172.21.15.114:0/2252802319 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd900020370 con 0x7fd90c103f70 2024-02-16T22:21:11.170 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/2902849354' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:21:11.170 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:21:11.170 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:21:11.170 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:21:11.170 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:21:11.170 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:21:11.170 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:21:11.170 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mgr fail", "who": "smithi114.vcullz"}]: dispatch 2024-02-16T22:21:11.170 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "mon metadata", "id": "smithi114"}]: dispatch 2024-02-16T22:21:11.170 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "mon metadata", "id": "smithi123"}]: dispatch 2024-02-16T22:21:11.170 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi114.zzryke"}]: dispatch 2024-02-16T22:21:11.171 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi114.fdxpey"}]: dispatch 2024-02-16T22:21:11.171 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi123.vhlqbe"}]: dispatch 2024-02-16T22:21:11.171 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi123.gopcfa"}]: dispatch 2024-02-16T22:21:11.171 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "mgr metadata", "who": "smithi123.inzgjd", "id": "smithi123.inzgjd"}]: dispatch 2024-02-16T22:21:11.171 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:21:11.171 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:21:11.171 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:21:11.171 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:21:11.171 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:21:11.171 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: osdmap e46: 6 total, 6 up, 6 in 2024-02-16T22:21:11.171 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:21:11.171 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "mgr fail", "who": "smithi114.vcullz"}]': finished 2024-02-16T22:21:11.171 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: mgrmap e21: smithi123.inzgjd(active, starting, since 0.00929902s) 2024-02-16T22:21:11.171 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T22:21:11.171 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T22:21:11.171 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T22:21:11.172 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: Manager daemon smithi123.inzgjd is now available 2024-02-16T22:21:11.172 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:21:11.172 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:21:11.172 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:21:11.172 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi123.inzgjd/mirror_snapshot_schedule"}]: dispatch 2024-02-16T22:21:11.172 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi123.inzgjd/mirror_snapshot_schedule"}]: dispatch 2024-02-16T22:21:11.172 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi123.inzgjd/trash_purge_schedule"}]: dispatch 2024-02-16T22:21:11.172 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi123.inzgjd/trash_purge_schedule"}]: dispatch 2024-02-16T22:21:11.172 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:10 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/3838205903' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:21:11.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/2902849354' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:21:11.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:21:11.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:21:11.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:21:11.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:21:11.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' 2024-02-16T22:21:11.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:21:11.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mgr fail", "who": "smithi114.vcullz"}]: dispatch 2024-02-16T22:21:11.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "mon metadata", "id": "smithi114"}]: dispatch 2024-02-16T22:21:11.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "mon metadata", "id": "smithi123"}]: dispatch 2024-02-16T22:21:11.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi114.zzryke"}]: dispatch 2024-02-16T22:21:11.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi114.fdxpey"}]: dispatch 2024-02-16T22:21:11.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi123.vhlqbe"}]: dispatch 2024-02-16T22:21:11.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi123.gopcfa"}]: dispatch 2024-02-16T22:21:11.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "mgr metadata", "who": "smithi123.inzgjd", "id": "smithi123.inzgjd"}]: dispatch 2024-02-16T22:21:11.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:21:11.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:21:11.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:21:11.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:21:11.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:21:11.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: osdmap e46: 6 total, 6 up, 6 in 2024-02-16T22:21:11.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:21:11.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.14215 172.21.15.114:0/3483089389' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "mgr fail", "who": "smithi114.vcullz"}]': finished 2024-02-16T22:21:11.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: mgrmap e21: smithi123.inzgjd(active, starting, since 0.00929902s) 2024-02-16T22:21:11.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T22:21:11.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T22:21:11.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T22:21:11.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: Manager daemon smithi123.inzgjd is now available 2024-02-16T22:21:11.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:21:11.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:21:11.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:21:11.305 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi123.inzgjd/mirror_snapshot_schedule"}]: dispatch 2024-02-16T22:21:11.305 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi123.inzgjd/mirror_snapshot_schedule"}]: dispatch 2024-02-16T22:21:11.305 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi123.inzgjd/trash_purge_schedule"}]: dispatch 2024-02-16T22:21:11.305 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi123.inzgjd/trash_purge_schedule"}]: dispatch 2024-02-16T22:21:11.305 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:10 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/3838205903' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:21:11.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.412+0000 7fd8f3fff700 1 -- 172.21.15.114:0/2252802319 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mgrmap(e 22) v1 ==== 45915+0+0 (secure 0 0 0) 0x7fd900022800 con 0x7fd90c103f70 2024-02-16T22:21:11.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.412+0000 7fd8f3fff700 1 --2- 172.21.15.114:0/2252802319 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fd8f403dba0 0x7fd8f403ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:11.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.412+0000 7fd8f3fff700 1 -- 172.21.15.114:0/2252802319 --> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8f40404d0 con 0x7fd8f403dba0 2024-02-16T22:21:11.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.423+0000 7fd90ad9d700 1 --2- 172.21.15.114:0/2252802319 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fd8f403dba0 0x7fd8f403ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:11.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.427+0000 7fd90ad9d700 1 --2- 172.21.15.114:0/2252802319 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fd8f403dba0 0x7fd8f403ff90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc009c60 tx=0x7fd8fc009260 comp rx=0 tx=0).ready entity=mgr.24441 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:21:11.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.439+0000 7fd8f3fff700 1 -- 172.21.15.114:0/2252802319 <== mgr.24441 v2:172.21.15.123:6828/1539029039 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fd8f40404d0 con 0x7fd8f403dba0 2024-02-16T22:21:11.441 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:21:11.441 INFO:teuthology.orchestra.run.smithi114.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:21:11.442 INFO:teuthology.orchestra.run.smithi114.stdout: "in_progress": true, 2024-02-16T22:21:11.442 INFO:teuthology.orchestra.run.smithi114.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:21:11.442 INFO:teuthology.orchestra.run.smithi114.stdout: "services_complete": [], 2024-02-16T22:21:11.442 INFO:teuthology.orchestra.run.smithi114.stdout: "progress": "1/23 daemons upgraded", 2024-02-16T22:21:11.442 INFO:teuthology.orchestra.run.smithi114.stdout: "message": "", 2024-02-16T22:21:11.442 INFO:teuthology.orchestra.run.smithi114.stdout: "is_paused": false 2024-02-16T22:21:11.442 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:21:11.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.442+0000 7fd911078700 1 -- 172.21.15.114:0/2252802319 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fd8f403dba0 msgr2=0x7fd8f403ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:11.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.442+0000 7fd911078700 1 --2- 172.21.15.114:0/2252802319 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fd8f403dba0 0x7fd8f403ff90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc009c60 tx=0x7fd8fc009260 comp rx=0 tx=0).stop 2024-02-16T22:21:11.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.442+0000 7fd911078700 1 -- 172.21.15.114:0/2252802319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd90c103f70 msgr2=0x7fd90c196860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:11.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.442+0000 7fd911078700 1 --2- 172.21.15.114:0/2252802319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd90c103f70 0x7fd90c196860 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd900000c00 tx=0x7fd90000f690 comp rx=0 tx=0).stop 2024-02-16T22:21:11.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.442+0000 7fd911078700 1 -- 172.21.15.114:0/2252802319 shutdown_connections 2024-02-16T22:21:11.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.442+0000 7fd911078700 1 --2- 172.21.15.114:0/2252802319 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fd8f403dba0 0x7fd8f403ff90 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:11.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.442+0000 7fd911078700 1 --2- 172.21.15.114:0/2252802319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd90c101640 0x7fd90c196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:11.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.442+0000 7fd911078700 1 --2- 172.21.15.114:0/2252802319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd90c103f70 0x7fd90c196860 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:11.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.442+0000 7fd911078700 1 -- 172.21.15.114:0/2252802319 >> 172.21.15.114:0/2252802319 conn(0x7fd90c0fb030 msgr2=0x7fd90c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:21:11.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.442+0000 7fd911078700 1 -- 172.21.15.114:0/2252802319 shutdown_connections 2024-02-16T22:21:11.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.442+0000 7fd911078700 1 -- 172.21.15.114:0/2252802319 wait complete. 2024-02-16T22:21:11.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.596+0000 7f6aa9304700 1 -- 172.21.15.114:0/3816768552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6aa4100cb0 msgr2=0x7f6aa4101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:11.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.596+0000 7f6aa9304700 1 --2- 172.21.15.114:0/3816768552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6aa4100cb0 0x7f6aa4101110 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009910 tx=0x7f6a98009c20 comp rx=0 tx=0).stop 2024-02-16T22:21:11.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.596+0000 7f6aa9304700 1 -- 172.21.15.114:0/3816768552 shutdown_connections 2024-02-16T22:21:11.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.596+0000 7f6aa9304700 1 --2- 172.21.15.114:0/3816768552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6aa4100cb0 0x7f6aa4101110 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:11.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.596+0000 7f6aa9304700 1 --2- 172.21.15.114:0/3816768552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa40ffab0 0x7f6aa40ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:11.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.596+0000 7f6aa9304700 1 -- 172.21.15.114:0/3816768552 >> 172.21.15.114:0/3816768552 conn(0x7f6aa40fb050 msgr2=0x7f6aa40fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:21:11.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.596+0000 7f6aa9304700 1 -- 172.21.15.114:0/3816768552 shutdown_connections 2024-02-16T22:21:11.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.596+0000 7f6aa9304700 1 -- 172.21.15.114:0/3816768552 wait complete. 2024-02-16T22:21:11.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.597+0000 7f6aa9304700 1 Processor -- start 2024-02-16T22:21:11.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.597+0000 7f6aa9304700 1 -- start start 2024-02-16T22:21:11.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.597+0000 7f6aa9304700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6aa40ffab0 0x7f6aa4194120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:11.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.597+0000 7f6aa9304700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa4100cb0 0x7f6aa4194660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:11.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.597+0000 7f6aa9304700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa4194c80 con 0x7f6aa40ffab0 2024-02-16T22:21:11.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.597+0000 7f6aa9304700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa4194dc0 con 0x7f6aa4100cb0 2024-02-16T22:21:11.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.597+0000 7f6aa27fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa4100cb0 0x7f6aa4194660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:11.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.598+0000 7f6aa27fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa4100cb0 0x7f6aa4194660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.114:58106/0 (socket says 172.21.15.114:58106) 2024-02-16T22:21:11.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.598+0000 7f6aa2ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6aa40ffab0 0x7f6aa4194120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:11.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.598+0000 7f6aa27fc700 1 -- 172.21.15.114:0/356851413 learned_addr learned my addr 172.21.15.114:0/356851413 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:21:11.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.598+0000 7f6aa27fc700 1 -- 172.21.15.114:0/356851413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6aa40ffab0 msgr2=0x7f6aa4194120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:11.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.598+0000 7f6aa27fc700 1 --2- 172.21.15.114:0/356851413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6aa40ffab0 0x7f6aa4194120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:11.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.598+0000 7f6aa27fc700 1 -- 172.21.15.114:0/356851413 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9400b730 con 0x7f6aa4100cb0 2024-02-16T22:21:11.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.598+0000 7f6aa27fc700 1 --2- 172.21.15.114:0/356851413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa4100cb0 0x7f6aa4194660 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800f690 tx=0x7f6a9800f6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:21:11.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.598+0000 7f6a8bfff700 1 -- 172.21.15.114:0/356851413 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a9801d070 con 0x7f6aa4100cb0 2024-02-16T22:21:11.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.598+0000 7f6aa9304700 1 -- 172.21.15.114:0/356851413 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a980095f0 con 0x7f6aa4100cb0 2024-02-16T22:21:11.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.598+0000 7f6a8bfff700 1 -- 172.21.15.114:0/356851413 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6a98005dc0 con 0x7f6aa4100cb0 2024-02-16T22:21:11.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.599+0000 7f6aa9304700 1 -- 172.21.15.114:0/356851413 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa4199ab0 con 0x7f6aa4100cb0 2024-02-16T22:21:11.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.599+0000 7f6a8bfff700 1 -- 172.21.15.114:0/356851413 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a980226a0 con 0x7f6aa4100cb0 2024-02-16T22:21:11.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.599+0000 7f6a8bfff700 1 -- 172.21.15.114:0/356851413 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 22) v1 ==== 45915+0+0 (secure 0 0 0) 0x7f6a98022800 con 0x7f6aa4100cb0 2024-02-16T22:21:11.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.600+0000 7f6a8bfff700 1 --2- 172.21.15.114:0/356851413 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f6a8c038d00 0x7f6a8c03b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:11.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.600+0000 7f6a8bfff700 1 -- 172.21.15.114:0/356851413 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5726+0+0 (secure 0 0 0) 0x7f6a9804e300 con 0x7f6aa4100cb0 2024-02-16T22:21:11.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.600+0000 7f6aa9304700 1 -- 172.21.15.114:0/356851413 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa4107500 con 0x7f6aa4100cb0 2024-02-16T22:21:11.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.600+0000 7f6aa2ffd700 1 --2- 172.21.15.114:0/356851413 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f6a8c038d00 0x7f6a8c03b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:11.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.601+0000 7f6aa2ffd700 1 --2- 172.21.15.114:0/356851413 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f6a8c038d00 0x7f6a8c03b1c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4100b10 tx=0x7f6a9400d040 comp rx=0 tx=0).ready entity=mgr.24441 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:21:11.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:11.609+0000 7f6a8bfff700 1 -- 172.21.15.114:0/356851413 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182869 (secure 0 0 0) 0x7f6a98022c10 con 0x7f6aa4100cb0 2024-02-16T22:21:12.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:12.072+0000 7f6aa9304700 1 -- 172.21.15.114:0/356851413 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6aa419a520 con 0x7f6aa4100cb0 2024-02-16T22:21:12.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:12.073+0000 7f6a8bfff700 1 -- 172.21.15.114:0/356851413 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f6a98041090 con 0x7f6aa4100cb0 2024-02-16T22:21:12.075 INFO:teuthology.orchestra.run.smithi114.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:21:12.075 INFO:teuthology.orchestra.run.smithi114.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:21:12.075 INFO:teuthology.orchestra.run.smithi114.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:21:12.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:12.076+0000 7f6aa9304700 1 -- 172.21.15.114:0/356851413 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f6a8c038d00 msgr2=0x7f6a8c03b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:12.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:12.076+0000 7f6aa9304700 1 --2- 172.21.15.114:0/356851413 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f6a8c038d00 0x7f6a8c03b1c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4100b10 tx=0x7f6a9400d040 comp rx=0 tx=0).stop 2024-02-16T22:21:12.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:12.076+0000 7f6aa9304700 1 -- 172.21.15.114:0/356851413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa4100cb0 msgr2=0x7f6aa4194660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:12.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:12.076+0000 7f6aa9304700 1 --2- 172.21.15.114:0/356851413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa4100cb0 0x7f6aa4194660 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800f690 tx=0x7f6a9800f6c0 comp rx=0 tx=0).stop 2024-02-16T22:21:12.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:12.077+0000 7f6aa9304700 1 -- 172.21.15.114:0/356851413 shutdown_connections 2024-02-16T22:21:12.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:12.077+0000 7f6aa9304700 1 --2- 172.21.15.114:0/356851413 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f6a8c038d00 0x7f6a8c03b1c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:12.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:12.077+0000 7f6aa9304700 1 --2- 172.21.15.114:0/356851413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6aa40ffab0 0x7f6aa4194120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:12.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:12.077+0000 7f6aa9304700 1 --2- 172.21.15.114:0/356851413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa4100cb0 0x7f6aa4194660 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:12.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:12.077+0000 7f6aa9304700 1 -- 172.21.15.114:0/356851413 >> 172.21.15.114:0/356851413 conn(0x7f6aa40fb050 msgr2=0x7f6aa4103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:21:12.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:12.077+0000 7f6aa9304700 1 -- 172.21.15.114:0/356851413 shutdown_connections 2024-02-16T22:21:12.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:12.077+0000 7f6aa9304700 1 -- 172.21.15.114:0/356851413 wait complete. 2024-02-16T22:21:12.680 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:12 smithi114 ceph-mon[29235]: [16/Feb/2024:22:21:10] ENGINE Bus STARTING 2024-02-16T22:21:12.680 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:12 smithi114 ceph-mon[29235]: Deploying cephadm binary to smithi123 2024-02-16T22:21:12.680 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:12 smithi114 ceph-mon[29235]: Deploying cephadm binary to smithi114 2024-02-16T22:21:12.680 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:12 smithi114 ceph-mon[29235]: [16/Feb/2024:22:21:11] ENGINE Serving on https://172.21.15.123:7150 2024-02-16T22:21:12.680 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:12 smithi114 ceph-mon[29235]: [16/Feb/2024:22:21:11] ENGINE Serving on http://172.21.15.123:8765 2024-02-16T22:21:12.680 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:12 smithi114 ceph-mon[29235]: [16/Feb/2024:22:21:11] ENGINE Bus STARTED 2024-02-16T22:21:12.681 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:12 smithi114 ceph-mon[29235]: mgrmap e22: smithi123.inzgjd(active, since 1.00926s) 2024-02-16T22:21:12.681 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:12 smithi114 ceph-mon[29235]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:21:12.681 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:12 smithi114 ceph-mon[29235]: pgmap v3: 65 pgs: 65 active+clean; 103 MiB data, 515 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:21:12.681 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:12 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/356851413' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:21:12.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:12 smithi123 ceph-mon[37161]: [16/Feb/2024:22:21:10] ENGINE Bus STARTING 2024-02-16T22:21:12.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:12 smithi123 ceph-mon[37161]: Deploying cephadm binary to smithi123 2024-02-16T22:21:12.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:12 smithi123 ceph-mon[37161]: Deploying cephadm binary to smithi114 2024-02-16T22:21:12.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:12 smithi123 ceph-mon[37161]: [16/Feb/2024:22:21:11] ENGINE Serving on https://172.21.15.123:7150 2024-02-16T22:21:12.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:12 smithi123 ceph-mon[37161]: [16/Feb/2024:22:21:11] ENGINE Serving on http://172.21.15.123:8765 2024-02-16T22:21:12.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:12 smithi123 ceph-mon[37161]: [16/Feb/2024:22:21:11] ENGINE Bus STARTED 2024-02-16T22:21:12.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:12 smithi123 ceph-mon[37161]: mgrmap e22: smithi123.inzgjd(active, since 1.00926s) 2024-02-16T22:21:12.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:12 smithi123 ceph-mon[37161]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:21:12.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:12 smithi123 ceph-mon[37161]: pgmap v3: 65 pgs: 65 active+clean; 103 MiB data, 515 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:21:12.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:12 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/356851413' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:21:13.509 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:13 smithi114 ceph-mon[29235]: pgmap v4: 65 pgs: 65 active+clean; 103 MiB data, 515 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:21:13.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:13 smithi123 ceph-mon[37161]: pgmap v4: 65 pgs: 65 active+clean; 103 MiB data, 515 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:21:14.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:14 smithi114 ceph-mon[29235]: mgrmap e23: smithi123.inzgjd(active, since 3s) 2024-02-16T22:21:14.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:14 smithi123 ceph-mon[37161]: mgrmap e23: smithi123.inzgjd(active, since 3s) 2024-02-16T22:21:15.636 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:15 smithi114 ceph-mon[29235]: pgmap v5: 65 pgs: 65 active+clean; 103 MiB data, 515 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:21:15.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:15 smithi123 ceph-mon[37161]: pgmap v5: 65 pgs: 65 active+clean; 103 MiB data, 515 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:21:17.762 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:17 smithi123 ceph-mon[37161]: pgmap v6: 65 pgs: 65 active+clean; 103 MiB data, 515 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:21:17.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:17 smithi114 ceph-mon[29235]: pgmap v6: 65 pgs: 65 active+clean; 103 MiB data, 515 MiB used, 536 GiB / 536 GiB avail 2024-02-16T22:21:19.766 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:19 smithi114 ceph-mon[29235]: pgmap v7: 65 pgs: 65 active+clean; 116 MiB data, 562 MiB used, 536 GiB / 536 GiB avail; 29 KiB/s rd, 2.1 MiB/s wr, 129 op/s 2024-02-16T22:21:19.766 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:19 smithi114 ceph-mon[29235]: Standby manager daemon smithi114.vcullz started 2024-02-16T22:21:19.766 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:19 smithi114 ceph-mon[29235]: from='mgr.? 172.21.15.114:0/1587210374' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi114.vcullz/crt"}]: dispatch 2024-02-16T22:21:19.766 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:19 smithi114 ceph-mon[29235]: from='mgr.? 172.21.15.114:0/1587210374' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T22:21:19.766 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:19 smithi114 ceph-mon[29235]: from='mgr.? 172.21.15.114:0/1587210374' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi114.vcullz/key"}]: dispatch 2024-02-16T22:21:19.766 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:19 smithi114 ceph-mon[29235]: from='mgr.? 172.21.15.114:0/1587210374' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T22:21:19.766 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:19 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:19.766 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:19 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:19.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:19 smithi123 ceph-mon[37161]: pgmap v7: 65 pgs: 65 active+clean; 116 MiB data, 562 MiB used, 536 GiB / 536 GiB avail; 29 KiB/s rd, 2.1 MiB/s wr, 129 op/s 2024-02-16T22:21:19.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:19 smithi123 ceph-mon[37161]: Standby manager daemon smithi114.vcullz started 2024-02-16T22:21:19.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:19 smithi123 ceph-mon[37161]: from='mgr.? 172.21.15.114:0/1587210374' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi114.vcullz/crt"}]: dispatch 2024-02-16T22:21:19.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:19 smithi123 ceph-mon[37161]: from='mgr.? 172.21.15.114:0/1587210374' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T22:21:19.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:19 smithi123 ceph-mon[37161]: from='mgr.? 172.21.15.114:0/1587210374' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi114.vcullz/key"}]: dispatch 2024-02-16T22:21:19.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:19 smithi123 ceph-mon[37161]: from='mgr.? 172.21.15.114:0/1587210374' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T22:21:19.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:19 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:19.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:19 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:20.801 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:20 smithi114 ceph-mon[29235]: mgrmap e24: smithi123.inzgjd(active, since 9s), standbys: smithi114.vcullz 2024-02-16T22:21:20.801 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:20 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "mgr metadata", "who": "smithi114.vcullz", "id": "smithi114.vcullz"}]: dispatch 2024-02-16T22:21:20.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:20 smithi123 ceph-mon[37161]: mgrmap e24: smithi123.inzgjd(active, since 9s), standbys: smithi114.vcullz 2024-02-16T22:21:20.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:20 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "mgr metadata", "who": "smithi114.vcullz", "id": "smithi114.vcullz"}]: dispatch 2024-02-16T22:21:21.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:21 smithi114 ceph-mon[29235]: pgmap v8: 65 pgs: 65 active+clean; 116 MiB data, 562 MiB used, 536 GiB / 536 GiB avail; 23 KiB/s rd, 1.6 MiB/s wr, 100 op/s 2024-02-16T22:21:21.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:21 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:21.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:21 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:22.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:21 smithi123 ceph-mon[37161]: pgmap v8: 65 pgs: 65 active+clean; 116 MiB data, 562 MiB used, 536 GiB / 536 GiB avail; 23 KiB/s rd, 1.6 MiB/s wr, 100 op/s 2024-02-16T22:21:22.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:21 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:22.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:21 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:23.045 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:22 smithi114 ceph-mon[29235]: pgmap v9: 65 pgs: 65 active+clean; 134 MiB data, 626 MiB used, 536 GiB / 536 GiB avail; 19 KiB/s rd, 2.9 MiB/s wr, 186 op/s 2024-02-16T22:21:23.046 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:22 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:23.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:22 smithi123 ceph-mon[37161]: pgmap v9: 65 pgs: 65 active+clean; 134 MiB data, 626 MiB used, 536 GiB / 536 GiB avail; 19 KiB/s rd, 2.9 MiB/s wr, 186 op/s 2024-02-16T22:21:23.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:22 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:24.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:23 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:24.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:23 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:25.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:25 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:25.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:25 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:25.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:25 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:21:25.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:25 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:21:25.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:25 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:21:25.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:25 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:21:25.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:25 smithi123 ceph-mon[37161]: pgmap v10: 65 pgs: 65 active+clean; 134 MiB data, 626 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 2.7 MiB/s wr, 170 op/s 2024-02-16T22:21:25.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:25 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:21:25.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:25 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:21:25.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:25 smithi123 ceph-mon[37161]: Adjusting osd_memory_target on smithi114 to 1216M 2024-02-16T22:21:25.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:25 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:25.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:25 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:25.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:25 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:25.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:25 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:21:25.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:25 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:21:25.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:25 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:21:25.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:25 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:21:25.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:25 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:21:25.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:25 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:21:25.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:25 smithi123 ceph-mon[37161]: Adjusting osd_memory_target on smithi123 to 2240M 2024-02-16T22:21:25.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:25 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:25.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:25 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:21:25.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:25 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:21:25.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:25 smithi123 ceph-mon[37161]: Updating smithi114:/etc/ceph/ceph.conf 2024-02-16T22:21:25.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:25 smithi123 ceph-mon[37161]: Updating smithi123:/etc/ceph/ceph.conf 2024-02-16T22:21:25.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:25 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:25.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:25 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:25.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:25 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:21:25.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:25 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:21:25.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:25 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:21:25.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:25 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:21:25.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:25 smithi114 ceph-mon[29235]: pgmap v10: 65 pgs: 65 active+clean; 134 MiB data, 626 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 2.7 MiB/s wr, 170 op/s 2024-02-16T22:21:25.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:25 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:21:25.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:25 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:21:25.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:25 smithi114 ceph-mon[29235]: Adjusting osd_memory_target on smithi114 to 1216M 2024-02-16T22:21:25.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:25 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:25.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:25 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:25.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:25 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:25.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:25 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:21:25.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:25 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:21:25.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:25 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:21:25.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:25 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:21:25.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:25 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:21:25.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:25 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-16T22:21:25.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:25 smithi114 ceph-mon[29235]: Adjusting osd_memory_target on smithi123 to 2240M 2024-02-16T22:21:25.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:25 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:25.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:25 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:21:25.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:25 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:21:25.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:25 smithi114 ceph-mon[29235]: Updating smithi114:/etc/ceph/ceph.conf 2024-02-16T22:21:25.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:25 smithi114 ceph-mon[29235]: Updating smithi123:/etc/ceph/ceph.conf 2024-02-16T22:21:26.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:26 smithi114 ceph-mon[29235]: Updating smithi123:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:21:26.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:26 smithi114 ceph-mon[29235]: Updating smithi114:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:21:26.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:26 smithi114 ceph-mon[29235]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:21:26.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:26 smithi114 ceph-mon[29235]: Updating smithi114:/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:21:26.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:26 smithi114 ceph-mon[29235]: Updating smithi123:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T22:21:26.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:26 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:21:26.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:26 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:26.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:26 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:26.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:26 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:26.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:26 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:26.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:26 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:26.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:26 smithi123 ceph-mon[37161]: Updating smithi123:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:21:26.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:26 smithi123 ceph-mon[37161]: Updating smithi114:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:21:26.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:26 smithi123 ceph-mon[37161]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:21:26.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:26 smithi123 ceph-mon[37161]: Updating smithi114:/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:21:26.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:26 smithi123 ceph-mon[37161]: Updating smithi123:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T22:21:26.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:26 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:21:26.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:26 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:26.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:26 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:26.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:26 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:26.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:26 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:26.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:26 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:27.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:27 smithi114 ceph-mon[29235]: Updating smithi114:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T22:21:27.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:27 smithi114 ceph-mon[29235]: Reconfiguring prometheus.smithi114 (dependencies changed)... 2024-02-16T22:21:27.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:27 smithi114 ceph-mon[29235]: pgmap v11: 65 pgs: 65 active+clean; 134 MiB data, 626 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 2.7 MiB/s wr, 170 op/s 2024-02-16T22:21:27.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:27 smithi114 ceph-mon[29235]: Reconfiguring daemon prometheus.smithi114 on smithi114 2024-02-16T22:21:27.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:27 smithi123 ceph-mon[37161]: Updating smithi114:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T22:21:27.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:27 smithi123 ceph-mon[37161]: Reconfiguring prometheus.smithi114 (dependencies changed)... 2024-02-16T22:21:27.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:27 smithi123 ceph-mon[37161]: pgmap v11: 65 pgs: 65 active+clean; 134 MiB data, 626 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 2.7 MiB/s wr, 170 op/s 2024-02-16T22:21:27.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:27 smithi123 ceph-mon[37161]: Reconfiguring daemon prometheus.smithi114 on smithi114 2024-02-16T22:21:29.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:29 smithi114 ceph-mon[29235]: pgmap v12: 65 pgs: 65 active+clean; 158 MiB data, 717 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 4.7 MiB/s wr, 306 op/s 2024-02-16T22:21:29.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:29 smithi123 ceph-mon[37161]: pgmap v12: 65 pgs: 65 active+clean; 158 MiB data, 717 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 4.7 MiB/s wr, 306 op/s 2024-02-16T22:21:31.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:31 smithi123 ceph-mon[37161]: pgmap v13: 65 pgs: 65 active+clean; 158 MiB data, 717 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.6 MiB/s wr, 230 op/s 2024-02-16T22:21:31.865 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:31 smithi114 ceph-mon[29235]: pgmap v13: 65 pgs: 65 active+clean; 158 MiB data, 717 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.6 MiB/s wr, 230 op/s 2024-02-16T22:21:33.084 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:32 smithi114 ceph-mon[29235]: pgmap v14: 65 pgs: 65 active+clean; 174 MiB data, 755 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.9 MiB/s wr, 379 op/s 2024-02-16T22:21:33.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:32 smithi123 ceph-mon[37161]: pgmap v14: 65 pgs: 65 active+clean; 174 MiB data, 755 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.9 MiB/s wr, 379 op/s 2024-02-16T22:21:35.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:35 smithi123 ceph-mon[37161]: pgmap v15: 65 pgs: 65 active+clean; 174 MiB data, 755 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.4 MiB/s wr, 284 op/s 2024-02-16T22:21:35.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:35 smithi114 ceph-mon[29235]: pgmap v15: 65 pgs: 65 active+clean; 174 MiB data, 755 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.4 MiB/s wr, 284 op/s 2024-02-16T22:21:37.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:37 smithi123 ceph-mon[37161]: pgmap v16: 65 pgs: 65 active+clean; 174 MiB data, 755 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.4 MiB/s wr, 284 op/s 2024-02-16T22:21:37.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:37 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:37.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:37 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:37.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:37 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T22:21:37.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:37 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:21:37.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:37 smithi114 ceph-mon[29235]: pgmap v16: 65 pgs: 65 active+clean; 174 MiB data, 755 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.4 MiB/s wr, 284 op/s 2024-02-16T22:21:37.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:37 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:37.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:37 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:37.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:37 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T22:21:37.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:37 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:21:38.744 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:38 smithi114 ceph-mon[29235]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T22:21:38.745 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:38 smithi114 ceph-mon[29235]: Upgrade: Need to upgrade myself (mgr.smithi123.inzgjd) 2024-02-16T22:21:38.745 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:38 smithi114 ceph-mon[29235]: Upgrade: Need to upgrade myself (mgr.smithi123.inzgjd) 2024-02-16T22:21:38.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:38 smithi123 ceph-mon[37161]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T22:21:38.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:38 smithi123 ceph-mon[37161]: Upgrade: Need to upgrade myself (mgr.smithi123.inzgjd) 2024-02-16T22:21:38.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:38 smithi123 ceph-mon[37161]: Upgrade: Need to upgrade myself (mgr.smithi123.inzgjd) 2024-02-16T22:21:39.506 INFO:tasks.workunit.client.0.smithi114.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-02-16T22:21:39.511 INFO:tasks.workunit.client.0.smithi114.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-02-16T22:21:39.511 INFO:tasks.workunit.client.0.smithi114.stderr:+ make 2024-02-16T22:21:39.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:39 smithi123 ceph-mon[37161]: pgmap v17: 65 pgs: 65 active+clean; 197 MiB data, 794 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 5.4 MiB/s wr, 477 op/s 2024-02-16T22:21:39.872 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:39 smithi114 ceph-mon[29235]: pgmap v17: 65 pgs: 65 active+clean; 197 MiB data, 794 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 5.4 MiB/s wr, 477 op/s 2024-02-16T22:21:40.121 INFO:tasks.workunit.client.0.smithi114.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-02-16T22:21:41.086 INFO:tasks.workunit.client.1.smithi123.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-02-16T22:21:41.091 INFO:tasks.workunit.client.1.smithi123.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-02-16T22:21:41.091 INFO:tasks.workunit.client.1.smithi123.stderr:+ make 2024-02-16T22:21:41.321 INFO:tasks.workunit.client.1.smithi123.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-02-16T22:21:41.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:41 smithi123 ceph-mon[37161]: pgmap v18: 65 pgs: 65 active+clean; 197 MiB data, 794 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.4 MiB/s wr, 342 op/s 2024-02-16T22:21:41.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:41 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:21:41.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:41 smithi114 ceph-mon[29235]: pgmap v18: 65 pgs: 65 active+clean; 197 MiB data, 794 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.4 MiB/s wr, 342 op/s 2024-02-16T22:21:41.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:41 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:21:42.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.195+0000 7f49b3097700 1 -- 172.21.15.114:0/2301503455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49ac100cd0 msgr2=0x7f49ac101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:42.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.195+0000 7f49b3097700 1 --2- 172.21.15.114:0/2301503455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49ac100cd0 0x7f49ac101130 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4998009910 tx=0x7f4998009c20 comp rx=0 tx=0).stop 2024-02-16T22:21:42.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.195+0000 7f49b3097700 1 -- 172.21.15.114:0/2301503455 shutdown_connections 2024-02-16T22:21:42.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.195+0000 7f49b3097700 1 --2- 172.21.15.114:0/2301503455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49ac100cd0 0x7f49ac101130 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:42.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.195+0000 7f49b3097700 1 --2- 172.21.15.114:0/2301503455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49ac0ffad0 0x7f49ac0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:42.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.195+0000 7f49b3097700 1 -- 172.21.15.114:0/2301503455 >> 172.21.15.114:0/2301503455 conn(0x7f49ac0fb050 msgr2=0x7f49ac0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:21:42.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.195+0000 7f49b3097700 1 -- 172.21.15.114:0/2301503455 shutdown_connections 2024-02-16T22:21:42.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.195+0000 7f49b3097700 1 -- 172.21.15.114:0/2301503455 wait complete. 2024-02-16T22:21:42.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.195+0000 7f49b3097700 1 Processor -- start 2024-02-16T22:21:42.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.195+0000 7f49b3097700 1 -- start start 2024-02-16T22:21:42.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.195+0000 7f49b3097700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49ac0ffad0 0x7f49ac196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:42.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.195+0000 7f49b3097700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49ac100cd0 0x7f49ac1968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:42.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.195+0000 7f49b3097700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac196ec0 con 0x7f49ac0ffad0 2024-02-16T22:21:42.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.195+0000 7f49b3097700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac197000 con 0x7f49ac100cd0 2024-02-16T22:21:42.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.196+0000 7f49b0e33700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49ac0ffad0 0x7f49ac196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:42.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.196+0000 7f49b0e33700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49ac0ffad0 0x7f49ac196360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39356/0 (socket says 172.21.15.114:39356) 2024-02-16T22:21:42.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.196+0000 7f49b0e33700 1 -- 172.21.15.114:0/2966622370 learned_addr learned my addr 172.21.15.114:0/2966622370 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:21:42.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.196+0000 7f49abfff700 1 --2- 172.21.15.114:0/2966622370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49ac100cd0 0x7f49ac1968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:42.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.196+0000 7f49abfff700 1 -- 172.21.15.114:0/2966622370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49ac0ffad0 msgr2=0x7f49ac196360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:42.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.196+0000 7f49abfff700 1 --2- 172.21.15.114:0/2966622370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49ac0ffad0 0x7f49ac196360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:42.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.196+0000 7f49abfff700 1 -- 172.21.15.114:0/2966622370 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49980095f0 con 0x7f49ac100cd0 2024-02-16T22:21:42.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.196+0000 7f49b0e33700 1 --2- 172.21.15.114:0/2966622370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49ac0ffad0 0x7f49ac196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:21:42.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.196+0000 7f49abfff700 1 --2- 172.21.15.114:0/2966622370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49ac100cd0 0x7f49ac1968a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4998000c00 tx=0x7f499800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:21:42.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.196+0000 7f49a9ffb700 1 -- 172.21.15.114:0/2966622370 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f499801d070 con 0x7f49ac100cd0 2024-02-16T22:21:42.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.196+0000 7f49b3097700 1 -- 172.21.15.114:0/2966622370 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49ac19ba50 con 0x7f49ac100cd0 2024-02-16T22:21:42.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.196+0000 7f49a9ffb700 1 -- 172.21.15.114:0/2966622370 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4998005dc0 con 0x7f49ac100cd0 2024-02-16T22:21:42.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.196+0000 7f49a9ffb700 1 -- 172.21.15.114:0/2966622370 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4998017400 con 0x7f49ac100cd0 2024-02-16T22:21:42.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.197+0000 7f49b3097700 1 -- 172.21.15.114:0/2966622370 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49ac19bf40 con 0x7f49ac100cd0 2024-02-16T22:21:42.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.198+0000 7f49b3097700 1 -- 172.21.15.114:0/2966622370 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49ac108630 con 0x7f49ac100cd0 2024-02-16T22:21:42.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.198+0000 7f49a9ffb700 1 -- 172.21.15.114:0/2966622370 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) v1 ==== 90933+0+0 (secure 0 0 0) 0x7f4998003810 con 0x7f49ac100cd0 2024-02-16T22:21:42.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.199+0000 7f49a9ffb700 1 --2- 172.21.15.114:0/2966622370 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f499c06ce40 0x7f499c06f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:42.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.199+0000 7f49a9ffb700 1 -- 172.21.15.114:0/2966622370 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5726+0+0 (secure 0 0 0) 0x7f499808daf0 con 0x7f49ac100cd0 2024-02-16T22:21:42.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.199+0000 7f49b0e33700 1 --2- 172.21.15.114:0/2966622370 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f499c06ce40 0x7f499c06f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:42.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.199+0000 7f49b0e33700 1 --2- 172.21.15.114:0/2966622370 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f499c06ce40 0x7f499c06f300 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f49a0005950 tx=0x7f49a0009260 comp rx=0 tx=0).ready entity=mgr.24441 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:21:42.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.204+0000 7f49a9ffb700 1 -- 172.21.15.114:0/2966622370 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182869 (secure 0 0 0) 0x7f499805b1f0 con 0x7f49ac100cd0 2024-02-16T22:21:42.237 INFO:tasks.workunit.client.0.smithi114.stderr:++ readlink -f fsstress 2024-02-16T22:21:42.238 INFO:tasks.workunit.client.0.smithi114.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-02-16T22:21:42.239 INFO:tasks.workunit.client.0.smithi114.stderr:+ popd 2024-02-16T22:21:42.239 INFO:tasks.workunit.client.0.smithi114.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-02-16T22:21:42.239 INFO:tasks.workunit.client.0.smithi114.stderr:+ popd 2024-02-16T22:21:42.240 INFO:tasks.workunit.client.0.smithi114.stdout:~/cephtest/mnt.0/client.0/tmp 2024-02-16T22:21:42.240 INFO:tasks.workunit.client.0.smithi114.stderr:++ mktemp -d -p . 2024-02-16T22:21:42.242 INFO:tasks.workunit.client.0.smithi114.stderr:+ T=./tmp.WttGWFrPDP 2024-02-16T22:21:42.242 INFO:tasks.workunit.client.0.smithi114.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.WttGWFrPDP -l 1 -n 1000 -p 10 -v 2024-02-16T22:21:42.245 INFO:tasks.workunit.client.0.smithi114.stdout:seed = 1707893609 2024-02-16T22:21:42.249 INFO:tasks.workunit.client.0.smithi114.stdout:1/0: write - no filename 2024-02-16T22:21:42.250 INFO:tasks.workunit.client.0.smithi114.stdout:0/0: mknod c0 0 2024-02-16T22:21:42.252 INFO:tasks.workunit.client.0.smithi114.stdout:0/1: mkdir d1 0 2024-02-16T22:21:42.252 INFO:tasks.workunit.client.0.smithi114.stdout:1/1: creat f0 x:0 0 0 2024-02-16T22:21:42.254 INFO:tasks.workunit.client.0.smithi114.stdout:3/0: chown . 11 1 2024-02-16T22:21:42.254 INFO:tasks.workunit.client.0.smithi114.stdout:3/1: stat - no entries 2024-02-16T22:21:42.254 INFO:tasks.workunit.client.0.smithi114.stdout:3/2: truncate - no filename 2024-02-16T22:21:42.254 INFO:tasks.workunit.client.0.smithi114.stdout:3/3: rename - no filename 2024-02-16T22:21:42.254 INFO:tasks.workunit.client.0.smithi114.stdout:3/4: dread - no filename 2024-02-16T22:21:42.254 INFO:tasks.workunit.client.0.smithi114.stdout:3/5: chown . 13709 1 2024-02-16T22:21:42.254 INFO:tasks.workunit.client.0.smithi114.stdout:3/6: truncate - no filename 2024-02-16T22:21:42.254 INFO:tasks.workunit.client.0.smithi114.stdout:1/2: rename f0 to f1 0 2024-02-16T22:21:42.255 INFO:tasks.workunit.client.0.smithi114.stdout:2/0: creat f0 x:0 0 0 2024-02-16T22:21:42.255 INFO:tasks.workunit.client.0.smithi114.stdout:2/1: rmdir - no directory 2024-02-16T22:21:42.255 INFO:tasks.workunit.client.0.smithi114.stdout:2/2: chown f0 6356 1 2024-02-16T22:21:42.256 INFO:tasks.workunit.client.0.smithi114.stdout:0/2: creat d1/f2 x:0 0 0 2024-02-16T22:21:42.257 INFO:tasks.workunit.client.0.smithi114.stdout:4/0: unlink - no file 2024-02-16T22:21:42.257 INFO:tasks.workunit.client.0.smithi114.stdout:4/1: rename - no filename 2024-02-16T22:21:42.258 INFO:tasks.workunit.client.0.smithi114.stdout:4/2: rename - no filename 2024-02-16T22:21:42.258 INFO:tasks.workunit.client.0.smithi114.stdout:4/3: fdatasync - no filename 2024-02-16T22:21:42.258 INFO:tasks.workunit.client.0.smithi114.stdout:0/3: truncate d1/f2 652831 0 2024-02-16T22:21:42.258 INFO:tasks.workunit.client.0.smithi114.stdout:1/3: mknod c2 0 2024-02-16T22:21:42.258 INFO:tasks.workunit.client.0.smithi114.stdout:0/4: rename d1 to d1/d3 22 2024-02-16T22:21:42.259 INFO:tasks.workunit.client.0.smithi114.stdout:2/3: creat f1 x:0 0 0 2024-02-16T22:21:42.260 INFO:tasks.workunit.client.0.smithi114.stdout:3/7: creat f0 x:0 0 0 2024-02-16T22:21:42.261 INFO:tasks.workunit.client.0.smithi114.stdout:3/8: fdatasync f0 0 2024-02-16T22:21:42.263 INFO:tasks.workunit.client.0.smithi114.stdout:1/4: symlink l3 0 2024-02-16T22:21:42.264 INFO:tasks.workunit.client.0.smithi114.stdout:4/4: creat f0 x:0 0 0 2024-02-16T22:21:42.265 INFO:tasks.workunit.client.0.smithi114.stdout:3/9: creat f1 x:0 0 0 2024-02-16T22:21:42.266 INFO:tasks.workunit.client.0.smithi114.stdout:1/5: mkdir d4 0 2024-02-16T22:21:42.267 INFO:tasks.workunit.client.0.smithi114.stdout:5/0: mkdir d0 0 2024-02-16T22:21:42.268 INFO:tasks.workunit.client.0.smithi114.stdout:5/1: dwrite - no filename 2024-02-16T22:21:42.268 INFO:tasks.workunit.client.0.smithi114.stdout:5/2: dwrite - no filename 2024-02-16T22:21:42.269 INFO:tasks.workunit.client.0.smithi114.stdout:3/10: symlink l2 0 2024-02-16T22:21:42.269 INFO:tasks.workunit.client.0.smithi114.stdout:5/3: chown d0 89768648 1 2024-02-16T22:21:42.269 INFO:tasks.workunit.client.0.smithi114.stdout:5/4: truncate - no filename 2024-02-16T22:21:42.270 INFO:tasks.workunit.client.0.smithi114.stdout:4/5: rename f0 to f1 0 2024-02-16T22:21:42.270 INFO:tasks.workunit.client.0.smithi114.stdout:5/5: chown d0 24 1 2024-02-16T22:21:42.272 INFO:tasks.workunit.client.0.smithi114.stdout:7/0: truncate - no filename 2024-02-16T22:21:42.272 INFO:tasks.workunit.client.0.smithi114.stdout:7/1: read - no filename 2024-02-16T22:21:42.272 INFO:tasks.workunit.client.0.smithi114.stdout:7/2: dwrite - no filename 2024-02-16T22:21:42.272 INFO:tasks.workunit.client.0.smithi114.stdout:7/3: truncate - no filename 2024-02-16T22:21:42.272 INFO:tasks.workunit.client.0.smithi114.stdout:7/4: fsync - no filename 2024-02-16T22:21:42.272 INFO:tasks.workunit.client.0.smithi114.stdout:7/5: dread - no filename 2024-02-16T22:21:42.273 INFO:tasks.workunit.client.0.smithi114.stdout:7/6: rmdir - no directory 2024-02-16T22:21:42.273 INFO:tasks.workunit.client.0.smithi114.stdout:6/0: symlink l0 0 2024-02-16T22:21:42.275 INFO:tasks.workunit.client.0.smithi114.stdout:6/1: chown l0 340 1 2024-02-16T22:21:42.277 INFO:tasks.workunit.client.0.smithi114.stdout:8/0: write - no filename 2024-02-16T22:21:42.277 INFO:tasks.workunit.client.0.smithi114.stdout:8/1: read - no filename 2024-02-16T22:21:42.277 INFO:tasks.workunit.client.0.smithi114.stdout:8/2: fsync - no filename 2024-02-16T22:21:42.277 INFO:tasks.workunit.client.0.smithi114.stdout:2/4: dwrite f0 [0,4194304] 0 2024-02-16T22:21:42.278 INFO:tasks.workunit.client.0.smithi114.stdout:6/2: chown l0 1017 1 2024-02-16T22:21:42.279 INFO:tasks.workunit.client.0.smithi114.stdout:4/6: write f1 [578511,3380] 0 2024-02-16T22:21:42.279 INFO:tasks.workunit.client.0.smithi114.stdout:0/5: dwrite d1/f2 [0,4194304] 0 2024-02-16T22:21:42.279 INFO:tasks.workunit.client.0.smithi114.stdout:3/11: rename l2 to l3 0 2024-02-16T22:21:42.280 INFO:tasks.workunit.client.0.smithi114.stdout:6/3: chown l0 22484 1 2024-02-16T22:21:42.280 INFO:tasks.workunit.client.0.smithi114.stdout:6/4: dwrite - no filename 2024-02-16T22:21:42.280 INFO:tasks.workunit.client.0.smithi114.stdout:6/5: fsync - no filename 2024-02-16T22:21:42.280 INFO:tasks.workunit.client.0.smithi114.stdout:6/6: dread - no filename 2024-02-16T22:21:42.281 INFO:tasks.workunit.client.0.smithi114.stdout:5/6: getdents d0 0 2024-02-16T22:21:42.283 INFO:tasks.workunit.client.0.smithi114.stdout:9/0: link - no file 2024-02-16T22:21:42.283 INFO:tasks.workunit.client.0.smithi114.stdout:9/1: write - no filename 2024-02-16T22:21:42.283 INFO:tasks.workunit.client.0.smithi114.stdout:1/6: link f1 d4/f5 0 2024-02-16T22:21:42.299 INFO:tasks.workunit.client.0.smithi114.stdout:1/7: chown d4 2022572 1 2024-02-16T22:21:42.299 INFO:tasks.workunit.client.0.smithi114.stdout:7/7: mknod c0 0 2024-02-16T22:21:42.299 INFO:tasks.workunit.client.0.smithi114.stdout:7/8: dread - no filename 2024-02-16T22:21:42.300 INFO:tasks.workunit.client.0.smithi114.stdout:2/5: symlink l2 0 2024-02-16T22:21:42.302 INFO:tasks.workunit.client.0.smithi114.stdout:7/9: stat c0 0 2024-02-16T22:21:42.302 INFO:tasks.workunit.client.0.smithi114.stdout:7/10: dread - no filename 2024-02-16T22:21:42.302 INFO:tasks.workunit.client.0.smithi114.stdout:1/8: dread - d4/f5 zero size 2024-02-16T22:21:42.302 INFO:tasks.workunit.client.0.smithi114.stdout:2/6: readlink l2 0 2024-02-16T22:21:42.302 INFO:tasks.workunit.client.0.smithi114.stdout:8/3: creat f0 x:0 0 0 2024-02-16T22:21:42.305 INFO:tasks.workunit.client.0.smithi114.stdout:8/4: write f0 [757832,42155] 0 2024-02-16T22:21:42.306 INFO:tasks.workunit.client.0.smithi114.stdout:6/7: rename l0 to l1 0 2024-02-16T22:21:42.306 INFO:tasks.workunit.client.0.smithi114.stdout:6/8: dread - no filename 2024-02-16T22:21:42.306 INFO:tasks.workunit.client.0.smithi114.stdout:6/9: fdatasync - no filename 2024-02-16T22:21:42.310 INFO:tasks.workunit.client.0.smithi114.stdout:8/5: write f0 [1089912,109907] 0 2024-02-16T22:21:42.312 INFO:tasks.workunit.client.0.smithi114.stdout:3/12: dwrite f1 [0,4194304] 0 2024-02-16T22:21:42.314 INFO:tasks.workunit.client.0.smithi114.stdout:4/7: dwrite f1 [0,4194304] 0 2024-02-16T22:21:42.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.491+0000 7f49b3097700 1 -- 172.21.15.114:0/2966622370 --> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f49ac069440 con 0x7f499c06ce40 2024-02-16T22:21:42.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.492+0000 7f49a9ffb700 1 -- 172.21.15.114:0/2966622370 <== mgr.24441 v2:172.21.15.123:6828/1539029039 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f49ac069440 con 0x7f499c06ce40 2024-02-16T22:21:42.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.495+0000 7f49b3097700 1 -- 172.21.15.114:0/2966622370 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f499c06ce40 msgr2=0x7f499c06f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:42.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.495+0000 7f49b3097700 1 --2- 172.21.15.114:0/2966622370 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f499c06ce40 0x7f499c06f300 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f49a0005950 tx=0x7f49a0009260 comp rx=0 tx=0).stop 2024-02-16T22:21:42.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.495+0000 7f49b3097700 1 -- 172.21.15.114:0/2966622370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49ac100cd0 msgr2=0x7f49ac1968a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:42.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.495+0000 7f49b3097700 1 --2- 172.21.15.114:0/2966622370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49ac100cd0 0x7f49ac1968a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4998000c00 tx=0x7f499800f690 comp rx=0 tx=0).stop 2024-02-16T22:21:42.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.495+0000 7f49b3097700 1 -- 172.21.15.114:0/2966622370 shutdown_connections 2024-02-16T22:21:42.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.495+0000 7f49b3097700 1 --2- 172.21.15.114:0/2966622370 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f499c06ce40 0x7f499c06f300 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:42.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.495+0000 7f49b3097700 1 --2- 172.21.15.114:0/2966622370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49ac0ffad0 0x7f49ac196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:42.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.495+0000 7f49b3097700 1 --2- 172.21.15.114:0/2966622370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49ac100cd0 0x7f49ac1968a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:42.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.495+0000 7f49b3097700 1 -- 172.21.15.114:0/2966622370 >> 172.21.15.114:0/2966622370 conn(0x7f49ac0fb050 msgr2=0x7f49ac068d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:21:42.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.495+0000 7f49b3097700 1 -- 172.21.15.114:0/2966622370 shutdown_connections 2024-02-16T22:21:42.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.495+0000 7f49b3097700 1 -- 172.21.15.114:0/2966622370 wait complete. 2024-02-16T22:21:42.507 INFO:teuthology.orchestra.run.smithi114.stdout:true 2024-02-16T22:21:42.515 INFO:tasks.workunit.client.0.smithi114.stdout:7/11: rename c0 to c1 0 2024-02-16T22:21:42.517 INFO:tasks.workunit.client.0.smithi114.stdout:5/7: mknod d0/c1 0 2024-02-16T22:21:42.533 INFO:tasks.workunit.client.0.smithi114.stdout:9/2: creat f0 x:0 0 0 2024-02-16T22:21:42.534 INFO:tasks.workunit.client.0.smithi114.stdout:9/3: truncate f0 1030986 0 2024-02-16T22:21:42.534 INFO:tasks.workunit.client.0.smithi114.stdout:1/9: fsync f1 0 2024-02-16T22:21:42.535 INFO:tasks.workunit.client.0.smithi114.stdout:6/10: creat f2 x:0 0 0 2024-02-16T22:21:42.535 INFO:tasks.workunit.client.0.smithi114.stdout:6/11: dread - f2 zero size 2024-02-16T22:21:42.536 INFO:tasks.workunit.client.0.smithi114.stdout:6/12: stat f2 0 2024-02-16T22:21:42.536 INFO:tasks.workunit.client.0.smithi114.stdout:8/6: creat f1 x:0 0 0 2024-02-16T22:21:42.537 INFO:tasks.workunit.client.0.smithi114.stdout:3/13: mkdir d4 0 2024-02-16T22:21:42.538 INFO:tasks.workunit.client.0.smithi114.stdout:3/14: truncate f0 62911 0 2024-02-16T22:21:42.539 INFO:tasks.workunit.client.0.smithi114.stdout:4/8: creat f2 x:0 0 0 2024-02-16T22:21:42.542 INFO:tasks.workunit.client.0.smithi114.stdout:2/7: rename f1 to f3 0 2024-02-16T22:21:42.543 INFO:tasks.workunit.client.0.smithi114.stdout:7/12: creat f2 x:0 0 0 2024-02-16T22:21:42.544 INFO:tasks.workunit.client.0.smithi114.stdout:7/13: truncate f2 426058 0 2024-02-16T22:21:42.545 INFO:tasks.workunit.client.0.smithi114.stdout:5/8: mknod d0/c2 0 2024-02-16T22:21:42.545 INFO:tasks.workunit.client.0.smithi114.stdout:5/9: dread - no filename 2024-02-16T22:21:42.545 INFO:tasks.workunit.client.0.smithi114.stdout:5/10: dwrite - no filename 2024-02-16T22:21:42.545 INFO:tasks.workunit.client.0.smithi114.stdout:5/11: rename d0 to d0/d3 22 2024-02-16T22:21:42.545 INFO:tasks.workunit.client.0.smithi114.stdout:7/14: dread f2 [0,4194304] 0 2024-02-16T22:21:42.547 INFO:tasks.workunit.client.0.smithi114.stdout:9/4: rename f0 to f1 0 2024-02-16T22:21:42.548 INFO:tasks.workunit.client.0.smithi114.stdout:9/5: chown f1 14 1 2024-02-16T22:21:42.549 INFO:tasks.workunit.client.0.smithi114.stdout:9/6: dread f1 [0,4194304] 0 2024-02-16T22:21:42.550 INFO:tasks.workunit.client.0.smithi114.stdout:6/13: rename f2 to f3 0 2024-02-16T22:21:42.551 INFO:tasks.workunit.client.0.smithi114.stdout:8/7: mknod c2 0 2024-02-16T22:21:42.552 INFO:tasks.workunit.client.0.smithi114.stdout:8/8: write f1 [463557,90907] 0 2024-02-16T22:21:42.565 INFO:tasks.workunit.client.0.smithi114.stdout:7/15: mkdir d3 0 2024-02-16T22:21:42.568 INFO:tasks.workunit.client.0.smithi114.stdout:2/8: dwrite f3 [0,4194304] 0 2024-02-16T22:21:42.572 INFO:tasks.workunit.client.0.smithi114.stdout:2/9: dread f3 [0,4194304] 0 2024-02-16T22:21:42.573 INFO:tasks.workunit.client.0.smithi114.stdout:9/7: mkdir d2 0 2024-02-16T22:21:42.574 INFO:tasks.workunit.client.0.smithi114.stdout:2/10: write f0 [2303834,118518] 0 2024-02-16T22:21:42.576 INFO:tasks.workunit.client.0.smithi114.stdout:9/8: dread f1 [0,4194304] 0 2024-02-16T22:21:42.576 INFO:tasks.workunit.client.0.smithi114.stdout:6/14: fdatasync f3 0 2024-02-16T22:21:42.577 INFO:tasks.workunit.client.0.smithi114.stdout:9/9: chown d2 20279072 1 2024-02-16T22:21:42.578 INFO:tasks.workunit.client.0.smithi114.stdout:6/15: write f3 [866524,105408] 0 2024-02-16T22:21:42.580 INFO:tasks.workunit.client.0.smithi114.stdout:8/9: unlink f1 0 2024-02-16T22:21:42.581 INFO:tasks.workunit.client.0.smithi114.stdout:3/15: symlink d4/l5 0 2024-02-16T22:21:42.583 INFO:tasks.workunit.client.0.smithi114.stdout:8/10: write f0 [2219789,31815] 0 2024-02-16T22:21:42.583 INFO:tasks.workunit.client.0.smithi114.stdout:3/16: fsync f1 0 2024-02-16T22:21:42.585 INFO:tasks.workunit.client.0.smithi114.stdout:2/11: dwrite f0 [0,4194304] 0 2024-02-16T22:21:42.587 INFO:tasks.workunit.client.0.smithi114.stdout:3/17: chown f0 0 1 2024-02-16T22:21:42.591 INFO:tasks.workunit.client.0.smithi114.stdout:6/16: dwrite f3 [0,4194304] 0 2024-02-16T22:21:42.591 INFO:tasks.workunit.client.0.smithi114.stdout:2/12: write f0 [3965389,61060] 0 2024-02-16T22:21:42.592 INFO:tasks.workunit.client.0.smithi114.stdout:3/18: write f1 [267044,32788] 0 2024-02-16T22:21:42.596 INFO:tasks.workunit.client.0.smithi114.stdout:2/13: write f3 [605255,56634] 0 2024-02-16T22:21:42.601 INFO:tasks.workunit.client.0.smithi114.stdout:8/11: dread f0 [0,4194304] 0 2024-02-16T22:21:42.608 INFO:tasks.workunit.client.0.smithi114.stdout:7/16: dwrite f2 [0,4194304] 0 2024-02-16T22:21:42.611 INFO:tasks.workunit.client.0.smithi114.stdout:6/17: dread f3 [0,4194304] 0 2024-02-16T22:21:42.612 INFO:tasks.workunit.client.1.smithi123.stderr:++ readlink -f fsstress 2024-02-16T22:21:42.615 INFO:tasks.workunit.client.1.smithi123.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-02-16T22:21:42.616 INFO:tasks.workunit.client.1.smithi123.stderr:+ popd 2024-02-16T22:21:42.616 INFO:tasks.workunit.client.1.smithi123.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-02-16T22:21:42.616 INFO:tasks.workunit.client.1.smithi123.stderr:+ popd 2024-02-16T22:21:42.617 INFO:tasks.workunit.client.1.smithi123.stdout:~/cephtest/mnt.1/client.1/tmp 2024-02-16T22:21:42.617 INFO:tasks.workunit.client.1.smithi123.stderr:++ mktemp -d -p . 2024-02-16T22:21:42.617 INFO:tasks.workunit.client.0.smithi114.stdout:3/19: dwrite f0 [0,4194304] 0 2024-02-16T22:21:42.618 INFO:tasks.workunit.client.0.smithi114.stdout:8/12: dread f0 [0,4194304] 0 2024-02-16T22:21:42.620 INFO:tasks.workunit.client.0.smithi114.stdout:8/13: truncate f0 2725418 0 2024-02-16T22:21:42.622 INFO:tasks.workunit.client.0.smithi114.stdout:6/18: write f3 [2418233,23848] 0 2024-02-16T22:21:42.624 INFO:tasks.workunit.client.0.smithi114.stdout:9/10: dwrite f1 [0,4194304] 0 2024-02-16T22:21:42.630 INFO:tasks.workunit.client.0.smithi114.stdout:7/17: dwrite f2 [0,4194304] 0 2024-02-16T22:21:42.633 INFO:tasks.workunit.client.0.smithi114.stdout:7/18: write f2 [970851,104786] 0 2024-02-16T22:21:42.640 INFO:tasks.workunit.client.0.smithi114.stdout:7/19: dread f2 [0,4194304] 0 2024-02-16T22:21:42.648 INFO:tasks.workunit.client.0.smithi114.stdout:3/20: creat d4/f6 x:0 0 0 2024-02-16T22:21:42.648 INFO:tasks.workunit.client.0.smithi114.stdout:3/21: stat d4 0 2024-02-16T22:21:42.650 INFO:tasks.workunit.client.0.smithi114.stdout:8/14: creat f3 x:0 0 0 2024-02-16T22:21:42.654 INFO:tasks.workunit.client.0.smithi114.stdout:6/19: symlink l4 0 2024-02-16T22:21:42.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.655+0000 7fb6336c8700 1 -- 172.21.15.114:0/3084650754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb62c0ff240 msgr2=0x7fb62c0ff660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:42.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.655+0000 7fb6336c8700 1 --2- 172.21.15.114:0/3084650754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb62c0ff240 0x7fb62c0ff660 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb628009910 tx=0x7fb628009c20 comp rx=0 tx=0).stop 2024-02-16T22:21:42.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.655+0000 7fb6336c8700 1 -- 172.21.15.114:0/3084650754 shutdown_connections 2024-02-16T22:21:42.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.655+0000 7fb6336c8700 1 --2- 172.21.15.114:0/3084650754 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb62c100440 0x7fb62c1008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:42.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.655+0000 7fb6336c8700 1 --2- 172.21.15.114:0/3084650754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb62c0ff240 0x7fb62c0ff660 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:42.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.655+0000 7fb6336c8700 1 -- 172.21.15.114:0/3084650754 >> 172.21.15.114:0/3084650754 conn(0x7fb62c0fa7c0 msgr2=0x7fb62c0fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:21:42.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.655+0000 7fb6336c8700 1 -- 172.21.15.114:0/3084650754 shutdown_connections 2024-02-16T22:21:42.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.655+0000 7fb6336c8700 1 -- 172.21.15.114:0/3084650754 wait complete. 2024-02-16T22:21:42.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.656+0000 7fb6336c8700 1 Processor -- start 2024-02-16T22:21:42.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.656+0000 7fb6336c8700 1 -- start start 2024-02-16T22:21:42.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.656+0000 7fb6336c8700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb62c0ff240 0x7fb62c1952c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:42.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.656+0000 7fb6336c8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb62c100440 0x7fb62c195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:42.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.656+0000 7fb6336c8700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c195e20 con 0x7fb62c0ff240 2024-02-16T22:21:42.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.656+0000 7fb6336c8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c195f60 con 0x7fb62c100440 2024-02-16T22:21:42.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.656+0000 7fb631464700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb62c0ff240 0x7fb62c1952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:42.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.656+0000 7fb630c63700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb62c100440 0x7fb62c195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:42.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.656+0000 7fb630c63700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb62c100440 0x7fb62c195800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.114:49252/0 (socket says 172.21.15.114:49252) 2024-02-16T22:21:42.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.656+0000 7fb630c63700 1 -- 172.21.15.114:0/551576182 learned_addr learned my addr 172.21.15.114:0/551576182 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:21:42.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.656+0000 7fb631464700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb62c0ff240 0x7fb62c1952c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39378/0 (socket says 172.21.15.114:39378) 2024-02-16T22:21:42.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.657+0000 7fb630c63700 1 -- 172.21.15.114:0/551576182 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb62c0ff240 msgr2=0x7fb62c1952c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:42.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.657+0000 7fb630c63700 1 --2- 172.21.15.114:0/551576182 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb62c0ff240 0x7fb62c1952c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:42.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.657+0000 7fb630c63700 1 -- 172.21.15.114:0/551576182 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6280095f0 con 0x7fb62c100440 2024-02-16T22:21:42.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.657+0000 7fb631464700 1 --2- 172.21.15.114:0/551576182 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb62c0ff240 0x7fb62c1952c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:21:42.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.657+0000 7fb630c63700 1 --2- 172.21.15.114:0/551576182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb62c100440 0x7fb62c195800 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00e920 tx=0x7fb61c00ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:21:42.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.657+0000 7fb6227fc700 1 -- 172.21.15.114:0/551576182 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb61c00c9d0 con 0x7fb62c100440 2024-02-16T22:21:42.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.657+0000 7fb6227fc700 1 -- 172.21.15.114:0/551576182 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb61c010430 con 0x7fb62c100440 2024-02-16T22:21:42.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.657+0000 7fb6336c8700 1 -- 172.21.15.114:0/551576182 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb62c19aa10 con 0x7fb62c100440 2024-02-16T22:21:42.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.657+0000 7fb6336c8700 1 -- 172.21.15.114:0/551576182 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb62c19af60 con 0x7fb62c100440 2024-02-16T22:21:42.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.657+0000 7fb6227fc700 1 -- 172.21.15.114:0/551576182 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb61c018650 con 0x7fb62c100440 2024-02-16T22:21:42.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.658+0000 7fb6336c8700 1 -- 172.21.15.114:0/551576182 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb62c107da0 con 0x7fb62c100440 2024-02-16T22:21:42.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.659+0000 7fb6227fc700 1 -- 172.21.15.114:0/551576182 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) v1 ==== 90933+0+0 (secure 0 0 0) 0x7fb61c0105a0 con 0x7fb62c100440 2024-02-16T22:21:42.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.660+0000 7fb6227fc700 1 --2- 172.21.15.114:0/551576182 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fb61806ce40 0x7fb61806f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:42.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.660+0000 7fb6227fc700 1 -- 172.21.15.114:0/551576182 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5726+0+0 (secure 0 0 0) 0x7fb61c014070 con 0x7fb62c100440 2024-02-16T22:21:42.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.660+0000 7fb631464700 1 --2- 172.21.15.114:0/551576182 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fb61806ce40 0x7fb61806f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:42.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.660+0000 7fb631464700 1 --2- 172.21.15.114:0/551576182 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fb61806ce40 0x7fb61806f300 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb62800b6f0 tx=0x7fb628005940 comp rx=0 tx=0).ready entity=mgr.24441 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:21:42.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.664+0000 7fb6227fc700 1 -- 172.21.15.114:0/551576182 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182869 (secure 0 0 0) 0x7fb61c052a70 con 0x7fb62c100440 2024-02-16T22:21:42.670 INFO:tasks.workunit.client.0.smithi114.stdout:6/20: fdatasync f3 0 2024-02-16T22:21:42.671 INFO:tasks.workunit.client.0.smithi114.stdout:6/21: chown l4 41934 1 2024-02-16T22:21:42.675 INFO:tasks.workunit.client.0.smithi114.stdout:8/15: dwrite f0 [0,4194304] 0 2024-02-16T22:21:42.677 INFO:tasks.workunit.client.0.smithi114.stdout:7/20: mknod d3/c4 0 2024-02-16T22:21:42.683 INFO:tasks.workunit.client.0.smithi114.stdout:8/16: mkdir d4 0 2024-02-16T22:21:42.690 INFO:tasks.workunit.client.0.smithi114.stdout:8/17: unlink c2 0 2024-02-16T22:21:42.692 INFO:tasks.workunit.client.0.smithi114.stdout:6/22: link l4 l5 0 2024-02-16T22:21:42.692 INFO:tasks.workunit.client.0.smithi114.stdout:7/21: dwrite f2 [0,4194304] 0 2024-02-16T22:21:42.697 INFO:tasks.workunit.client.0.smithi114.stdout:6/23: stat l1 0 2024-02-16T22:21:42.701 INFO:tasks.workunit.client.0.smithi114.stdout:8/18: symlink d4/l5 0 2024-02-16T22:21:42.702 INFO:tasks.workunit.client.0.smithi114.stdout:8/19: dread - f3 zero size 2024-02-16T22:21:42.704 INFO:tasks.workunit.client.0.smithi114.stdout:6/24: mkdir d6 0 2024-02-16T22:21:42.706 INFO:tasks.workunit.client.0.smithi114.stdout:7/22: fdatasync f2 0 2024-02-16T22:21:42.706 INFO:tasks.workunit.client.0.smithi114.stdout:7/23: fdatasync f2 0 2024-02-16T22:21:42.707 INFO:tasks.workunit.client.0.smithi114.stdout:8/20: creat d4/f6 x:0 0 0 2024-02-16T22:21:42.707 INFO:tasks.workunit.client.0.smithi114.stdout:8/21: dread - f3 zero size 2024-02-16T22:21:42.711 INFO:tasks.workunit.client.0.smithi114.stdout:8/22: unlink f3 0 2024-02-16T22:21:42.712 INFO:tasks.workunit.client.0.smithi114.stdout:7/24: dwrite f2 [0,4194304] 0 2024-02-16T22:21:42.714 INFO:tasks.workunit.client.0.smithi114.stdout:6/25: symlink d6/l7 0 2024-02-16T22:21:42.721 INFO:tasks.workunit.client.0.smithi114.stdout:7/25: creat d3/f5 x:0 0 0 2024-02-16T22:21:42.725 INFO:tasks.workunit.client.0.smithi114.stdout:6/26: rename l1 to d6/l8 0 2024-02-16T22:21:42.731 INFO:tasks.workunit.client.0.smithi114.stdout:7/26: creat d3/f6 x:0 0 0 2024-02-16T22:21:42.732 INFO:tasks.workunit.client.0.smithi114.stdout:6/27: symlink d6/l9 0 2024-02-16T22:21:42.733 INFO:tasks.workunit.client.0.smithi114.stdout:6/28: write f3 [346281,58563] 0 2024-02-16T22:21:42.734 INFO:tasks.workunit.client.0.smithi114.stdout:6/29: write f3 [3181477,27161] 0 2024-02-16T22:21:42.734 INFO:tasks.workunit.client.0.smithi114.stdout:7/27: symlink d3/l7 0 2024-02-16T22:21:42.740 INFO:tasks.workunit.client.0.smithi114.stdout:7/28: dwrite f2 [0,4194304] 0 2024-02-16T22:21:42.749 INFO:tasks.workunit.client.0.smithi114.stdout:7/29: dwrite d3/f6 [0,4194304] 0 2024-02-16T22:21:42.751 INFO:tasks.workunit.client.0.smithi114.stdout:7/30: write f2 [3484643,94953] 0 2024-02-16T22:21:42.758 INFO:tasks.workunit.client.0.smithi114.stdout:7/31: link f2 d3/f8 0 2024-02-16T22:21:42.767 INFO:tasks.workunit.client.0.smithi114.stdout:7/32: dwrite f2 [0,4194304] 0 2024-02-16T22:21:42.915 INFO:tasks.workunit.client.0.smithi114.stdout:0/6: truncate d1/f2 3284250 0 2024-02-16T22:21:42.927 INFO:tasks.workunit.client.0.smithi114.stdout:5/12: getdents d0 0 2024-02-16T22:21:42.928 INFO:tasks.workunit.client.0.smithi114.stdout:2/14: rename f3 to f4 0 2024-02-16T22:21:42.929 INFO:tasks.workunit.client.0.smithi114.stdout:1/10: truncate f1 1024214 0 2024-02-16T22:21:42.932 INFO:tasks.workunit.client.0.smithi114.stdout:5/13: mknod d0/c4 0 2024-02-16T22:21:42.934 INFO:tasks.workunit.client.0.smithi114.stdout:2/15: mknod c5 0 2024-02-16T22:21:42.937 INFO:tasks.workunit.client.0.smithi114.stdout:4/9: write f1 [4314003,23149] 0 2024-02-16T22:21:42.939 INFO:tasks.workunit.client.0.smithi114.stdout:5/14: mkdir d0/d5 0 2024-02-16T22:21:42.939 INFO:tasks.workunit.client.0.smithi114.stdout:5/15: fdatasync - no filename 2024-02-16T22:21:42.940 INFO:tasks.workunit.client.0.smithi114.stdout:2/16: creat f6 x:0 0 0 2024-02-16T22:21:42.940 INFO:tasks.workunit.client.0.smithi114.stdout:2/17: write f6 [855850,14303] 0 2024-02-16T22:21:42.942 INFO:tasks.workunit.client.0.smithi114.stdout:1/11: mknod d4/c6 0 2024-02-16T22:21:42.944 INFO:tasks.workunit.client.0.smithi114.stdout:4/10: symlink l3 0 2024-02-16T22:21:42.945 INFO:tasks.workunit.client.0.smithi114.stdout:5/16: symlink d0/l6 0 2024-02-16T22:21:42.945 INFO:tasks.workunit.client.0.smithi114.stdout:5/17: write - no filename 2024-02-16T22:21:42.946 INFO:tasks.workunit.client.0.smithi114.stdout:5/18: chown d0/c2 19681079 1 2024-02-16T22:21:42.946 INFO:tasks.workunit.client.0.smithi114.stdout:5/19: dwrite - no filename 2024-02-16T22:21:42.946 INFO:tasks.workunit.client.0.smithi114.stdout:5/20: chown d0/c4 2119079333 1 2024-02-16T22:21:42.947 INFO:tasks.workunit.client.0.smithi114.stdout:2/18: symlink l7 0 2024-02-16T22:21:42.948 INFO:tasks.workunit.client.0.smithi114.stdout:2/19: write f6 [422628,115612] 0 2024-02-16T22:21:42.950 INFO:tasks.workunit.client.0.smithi114.stdout:1/12: mknod d4/c7 0 2024-02-16T22:21:42.953 INFO:tasks.workunit.client.0.smithi114.stdout:5/21: creat d0/f7 x:0 0 0 2024-02-16T22:21:42.953 INFO:tasks.workunit.client.0.smithi114.stdout:5/22: dread - d0/f7 zero size 2024-02-16T22:21:42.956 INFO:tasks.workunit.client.0.smithi114.stdout:2/20: creat f8 x:0 0 0 2024-02-16T22:21:42.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.956+0000 7fb6336c8700 1 -- 172.21.15.114:0/551576182 --> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb62c104780 con 0x7fb61806ce40 2024-02-16T22:21:42.958 INFO:tasks.workunit.client.0.smithi114.stdout:1/13: symlink d4/l8 0 2024-02-16T22:21:42.958 INFO:tasks.workunit.client.0.smithi114.stdout:1/14: readlink d4/l8 0 2024-02-16T22:21:42.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.957+0000 7fb6227fc700 1 -- 172.21.15.114:0/551576182 <== mgr.24441 v2:172.21.15.123:6828/1539029039 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7fb62c104780 con 0x7fb61806ce40 2024-02-16T22:21:42.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.960+0000 7fb6336c8700 1 -- 172.21.15.114:0/551576182 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fb61806ce40 msgr2=0x7fb61806f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:42.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.960+0000 7fb6336c8700 1 --2- 172.21.15.114:0/551576182 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fb61806ce40 0x7fb61806f300 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb62800b6f0 tx=0x7fb628005940 comp rx=0 tx=0).stop 2024-02-16T22:21:42.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.960+0000 7fb6336c8700 1 -- 172.21.15.114:0/551576182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb62c100440 msgr2=0x7fb62c195800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:42.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.960+0000 7fb6336c8700 1 --2- 172.21.15.114:0/551576182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb62c100440 0x7fb62c195800 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00e920 tx=0x7fb61c00ec30 comp rx=0 tx=0).stop 2024-02-16T22:21:42.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.960+0000 7fb6336c8700 1 -- 172.21.15.114:0/551576182 shutdown_connections 2024-02-16T22:21:42.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.960+0000 7fb6336c8700 1 --2- 172.21.15.114:0/551576182 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fb61806ce40 0x7fb61806f300 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:42.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.960+0000 7fb6336c8700 1 --2- 172.21.15.114:0/551576182 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb62c0ff240 0x7fb62c1952c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:42.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.960+0000 7fb6336c8700 1 --2- 172.21.15.114:0/551576182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb62c100440 0x7fb62c195800 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:42.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.960+0000 7fb6336c8700 1 -- 172.21.15.114:0/551576182 >> 172.21.15.114:0/551576182 conn(0x7fb62c0fa7c0 msgr2=0x7fb62c0fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:21:42.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.960+0000 7fb6336c8700 1 -- 172.21.15.114:0/551576182 shutdown_connections 2024-02-16T22:21:42.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:42.960+0000 7fb6336c8700 1 -- 172.21.15.114:0/551576182 wait complete. 2024-02-16T22:21:42.963 INFO:tasks.workunit.client.0.smithi114.stdout:2/21: dwrite f6 [0,4194304] 0 2024-02-16T22:21:42.964 INFO:tasks.workunit.client.0.smithi114.stdout:2/22: write f6 [2057214,14539] 0 2024-02-16T22:21:42.968 INFO:tasks.workunit.client.0.smithi114.stdout:2/23: dread f0 [0,4194304] 0 2024-02-16T22:21:42.969 INFO:tasks.workunit.client.0.smithi114.stdout:2/24: chown f4 106 1 2024-02-16T22:21:42.976 INFO:tasks.workunit.client.0.smithi114.stdout:2/25: dwrite f0 [0,4194304] 0 2024-02-16T22:21:42.977 INFO:tasks.workunit.client.0.smithi114.stdout:2/26: chown l7 6 1 2024-02-16T22:21:42.977 INFO:tasks.workunit.client.0.smithi114.stdout:2/27: write f6 [898038,102477] 0 2024-02-16T22:21:42.982 INFO:tasks.workunit.client.0.smithi114.stdout:2/28: mkdir d9 0 2024-02-16T22:21:42.985 INFO:tasks.workunit.client.0.smithi114.stdout:2/29: unlink f6 0 2024-02-16T22:21:43.010 INFO:tasks.workunit.client.0.smithi114.stdout:9/11: truncate f1 1022314 0 2024-02-16T22:21:43.014 INFO:tasks.workunit.client.0.smithi114.stdout:9/12: symlink d2/l3 0 2024-02-16T22:21:43.015 INFO:tasks.workunit.client.0.smithi114.stdout:3/22: truncate f1 2219782 0 2024-02-16T22:21:43.021 INFO:tasks.workunit.client.0.smithi114.stdout:9/13: fdatasync f1 0 2024-02-16T22:21:43.022 INFO:tasks.workunit.client.0.smithi114.stdout:3/23: dwrite f0 [0,4194304] 0 2024-02-16T22:21:43.047 INFO:tasks.workunit.client.0.smithi114.stdout:8/23: truncate f0 1085897 0 2024-02-16T22:21:43.050 INFO:tasks.workunit.client.0.smithi114.stdout:8/24: creat d4/f7 x:0 0 0 2024-02-16T22:21:43.051 INFO:tasks.workunit.client.0.smithi114.stdout:6/30: getdents d6 0 2024-02-16T22:21:43.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:42 smithi123 ceph-mon[37161]: pgmap v19: 65 pgs: 65 active+clean; 221 MiB data, 883 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 5.4 MiB/s wr, 441 op/s 2024-02-16T22:21:43.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:42 smithi123 ceph-mon[37161]: from='client.24495 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:21:43.055 INFO:tasks.workunit.client.0.smithi114.stdout:6/31: mkdir d6/da 0 2024-02-16T22:21:43.056 INFO:tasks.workunit.client.0.smithi114.stdout:8/25: dwrite d4/f7 [0,4194304] 0 2024-02-16T22:21:43.061 INFO:tasks.workunit.client.0.smithi114.stdout:8/26: creat d4/f8 x:0 0 0 2024-02-16T22:21:43.062 INFO:tasks.workunit.client.0.smithi114.stdout:6/32: symlink d6/da/lb 0 2024-02-16T22:21:43.065 INFO:tasks.workunit.client.0.smithi114.stdout:8/27: creat d4/f9 x:0 0 0 2024-02-16T22:21:43.071 INFO:tasks.workunit.client.0.smithi114.stdout:6/33: dwrite f3 [0,4194304] 0 2024-02-16T22:21:43.077 INFO:tasks.workunit.client.1.smithi123.stderr:+ T=./tmp.2GodLqhxmo 2024-02-16T22:21:43.077 INFO:tasks.workunit.client.1.smithi123.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.2GodLqhxmo -l 1 -n 1000 -p 10 -v 2024-02-16T22:21:43.083 INFO:tasks.workunit.client.1.smithi123.stdout:seed = 1708038748 2024-02-16T22:21:43.088 INFO:tasks.workunit.client.1.smithi123.stdout:0/0: rmdir - no directory 2024-02-16T22:21:43.088 INFO:tasks.workunit.client.1.smithi123.stdout:0/1: dread - no filename 2024-02-16T22:21:43.088 INFO:tasks.workunit.client.1.smithi123.stdout:0/2: truncate - no filename 2024-02-16T22:21:43.088 INFO:tasks.workunit.client.1.smithi123.stdout:0/3: fdatasync - no filename 2024-02-16T22:21:43.090 INFO:tasks.workunit.client.1.smithi123.stdout:1/0: chown . 21 1 2024-02-16T22:21:43.090 INFO:tasks.workunit.client.1.smithi123.stdout:1/1: dwrite - no filename 2024-02-16T22:21:43.090 INFO:tasks.workunit.client.1.smithi123.stdout:1/2: write - no filename 2024-02-16T22:21:43.090 INFO:tasks.workunit.client.1.smithi123.stdout:1/3: unlink - no file 2024-02-16T22:21:43.091 INFO:tasks.workunit.client.1.smithi123.stdout:2/0: dread - no filename 2024-02-16T22:21:43.092 INFO:tasks.workunit.client.1.smithi123.stdout:2/1: read - no filename 2024-02-16T22:21:43.092 INFO:tasks.workunit.client.1.smithi123.stdout:2/2: unlink - no file 2024-02-16T22:21:43.092 INFO:tasks.workunit.client.1.smithi123.stdout:2/3: dwrite - no filename 2024-02-16T22:21:43.092 INFO:tasks.workunit.client.1.smithi123.stdout:2/4: write - no filename 2024-02-16T22:21:43.092 INFO:tasks.workunit.client.1.smithi123.stdout:2/5: truncate - no filename 2024-02-16T22:21:43.092 INFO:tasks.workunit.client.1.smithi123.stdout:2/6: chown . 68237768 1 2024-02-16T22:21:43.092 INFO:tasks.workunit.client.1.smithi123.stdout:2/7: truncate - no filename 2024-02-16T22:21:43.092 INFO:tasks.workunit.client.1.smithi123.stdout:0/4: mkdir d0 0 2024-02-16T22:21:43.095 INFO:tasks.workunit.client.1.smithi123.stdout:3/0: fsync - no filename 2024-02-16T22:21:43.095 INFO:tasks.workunit.client.1.smithi123.stdout:3/1: chown . 284 1 2024-02-16T22:21:43.095 INFO:tasks.workunit.client.1.smithi123.stdout:3/2: dread - no filename 2024-02-16T22:21:43.096 INFO:tasks.workunit.client.1.smithi123.stdout:1/4: creat f0 x:0 0 0 2024-02-16T22:21:43.098 INFO:tasks.workunit.client.1.smithi123.stdout:2/8: symlink l0 0 2024-02-16T22:21:43.098 INFO:tasks.workunit.client.1.smithi123.stdout:2/9: write - no filename 2024-02-16T22:21:43.098 INFO:tasks.workunit.client.1.smithi123.stdout:2/10: truncate - no filename 2024-02-16T22:21:43.098 INFO:tasks.workunit.client.1.smithi123.stdout:2/11: dwrite - no filename 2024-02-16T22:21:43.098 INFO:tasks.workunit.client.1.smithi123.stdout:2/12: dwrite - no filename 2024-02-16T22:21:43.101 INFO:tasks.workunit.client.1.smithi123.stdout:5/0: dwrite - no filename 2024-02-16T22:21:43.102 INFO:tasks.workunit.client.1.smithi123.stdout:2/13: rename l0 to l1 0 2024-02-16T22:21:43.103 INFO:tasks.workunit.client.1.smithi123.stdout:3/3: mkdir d0 0 2024-02-16T22:21:43.104 INFO:tasks.workunit.client.1.smithi123.stdout:3/4: dwrite - no filename 2024-02-16T22:21:43.104 INFO:tasks.workunit.client.1.smithi123.stdout:4/0: symlink l0 0 2024-02-16T22:21:43.107 INFO:tasks.workunit.client.1.smithi123.stdout:2/14: creat f2 x:0 0 0 2024-02-16T22:21:43.107 INFO:tasks.workunit.client.1.smithi123.stdout:2/15: write f2 [479485,61644] 0 2024-02-16T22:21:43.109 INFO:tasks.workunit.client.1.smithi123.stdout:4/1: rename l0 to l1 0 2024-02-16T22:21:43.110 INFO:tasks.workunit.client.1.smithi123.stdout:7/0: chown . 4113768 1 2024-02-16T22:21:43.110 INFO:tasks.workunit.client.1.smithi123.stdout:7/1: dread - no filename 2024-02-16T22:21:43.111 INFO:tasks.workunit.client.1.smithi123.stdout:7/2: truncate - no filename 2024-02-16T22:21:43.111 INFO:tasks.workunit.client.1.smithi123.stdout:7/3: dwrite - no filename 2024-02-16T22:21:43.112 INFO:tasks.workunit.client.1.smithi123.stdout:3/5: creat d0/f1 x:0 0 0 2024-02-16T22:21:43.112 INFO:tasks.workunit.client.1.smithi123.stdout:4/2: symlink l2 0 2024-02-16T22:21:43.112 INFO:tasks.workunit.client.1.smithi123.stdout:3/6: dread - d0/f1 zero size 2024-02-16T22:21:43.113 INFO:tasks.workunit.client.1.smithi123.stdout:8/0: readlink - no filename 2024-02-16T22:21:43.113 INFO:tasks.workunit.client.1.smithi123.stdout:8/1: fdatasync - no filename 2024-02-16T22:21:43.114 INFO:tasks.workunit.client.1.smithi123.stdout:6/0: mkdir d0 0 2024-02-16T22:21:43.114 INFO:tasks.workunit.client.1.smithi123.stdout:6/1: stat d0 0 2024-02-16T22:21:43.114 INFO:tasks.workunit.client.1.smithi123.stdout:6/2: chown d0 460509 1 2024-02-16T22:21:43.114 INFO:tasks.workunit.client.1.smithi123.stdout:6/3: read - no filename 2024-02-16T22:21:43.114 INFO:tasks.workunit.client.1.smithi123.stdout:6/4: write - no filename 2024-02-16T22:21:43.114 INFO:tasks.workunit.client.1.smithi123.stdout:6/5: dread - no filename 2024-02-16T22:21:43.114 INFO:tasks.workunit.client.1.smithi123.stdout:6/6: fsync - no filename 2024-02-16T22:21:43.114 INFO:tasks.workunit.client.1.smithi123.stdout:6/7: fdatasync - no filename 2024-02-16T22:21:43.115 INFO:tasks.workunit.client.1.smithi123.stdout:6/8: link - no file 2024-02-16T22:21:43.116 INFO:tasks.workunit.client.1.smithi123.stdout:4/3: creat f3 x:0 0 0 2024-02-16T22:21:43.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.114+0000 7fed834ff700 1 -- 172.21.15.114:0/3053133020 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed7c101de0 msgr2=0x7fed7c102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:43.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.114+0000 7fed834ff700 1 --2- 172.21.15.114:0/3053133020 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed7c101de0 0x7fed7c102240 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fed6c009910 tx=0x7fed6c009c20 comp rx=0 tx=0).stop 2024-02-16T22:21:43.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.114+0000 7fed834ff700 1 -- 172.21.15.114:0/3053133020 shutdown_connections 2024-02-16T22:21:43.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.114+0000 7fed834ff700 1 --2- 172.21.15.114:0/3053133020 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed7c101de0 0x7fed7c102240 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:43.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.114+0000 7fed834ff700 1 --2- 172.21.15.114:0/3053133020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed7c100be0 0x7fed7c101000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:43.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.114+0000 7fed834ff700 1 -- 172.21.15.114:0/3053133020 >> 172.21.15.114:0/3053133020 conn(0x7fed7c0fc140 msgr2=0x7fed7c0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:21:43.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.114+0000 7fed834ff700 1 -- 172.21.15.114:0/3053133020 shutdown_connections 2024-02-16T22:21:43.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.114+0000 7fed834ff700 1 -- 172.21.15.114:0/3053133020 wait complete. 2024-02-16T22:21:43.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.115+0000 7fed834ff700 1 Processor -- start 2024-02-16T22:21:43.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.115+0000 7fed834ff700 1 -- start start 2024-02-16T22:21:43.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.115+0000 7fed834ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed7c100be0 0x7fed7c074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:43.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.116+0000 7fed834ff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed7c101de0 0x7fed7c073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:43.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.116+0000 7fed834ff700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c0736c0 con 0x7fed7c101de0 2024-02-16T22:21:43.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.116+0000 7fed834ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c073800 con 0x7fed7c100be0 2024-02-16T22:21:43.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.116+0000 7fed80a9a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed7c101de0 0x7fed7c073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:43.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.116+0000 7fed8129b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed7c100be0 0x7fed7c074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:43.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.116+0000 7fed80a9a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed7c101de0 0x7fed7c073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39392/0 (socket says 172.21.15.114:39392) 2024-02-16T22:21:43.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.116+0000 7fed80a9a700 1 -- 172.21.15.114:0/2034343462 learned_addr learned my addr 172.21.15.114:0/2034343462 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:21:43.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.116+0000 7fed80a9a700 1 -- 172.21.15.114:0/2034343462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed7c100be0 msgr2=0x7fed7c074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:43.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.116+0000 7fed80a9a700 1 --2- 172.21.15.114:0/2034343462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed7c100be0 0x7fed7c074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:43.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.116+0000 7fed80a9a700 1 -- 172.21.15.114:0/2034343462 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6c0095f0 con 0x7fed7c101de0 2024-02-16T22:21:43.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.116+0000 7fed8129b700 1 --2- 172.21.15.114:0/2034343462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed7c100be0 0x7fed7c074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:21:43.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.116+0000 7fed80a9a700 1 --2- 172.21.15.114:0/2034343462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed7c101de0 0x7fed7c073180 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fed6c0092e0 tx=0x7fed6c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:21:43.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.116+0000 7fed727fc700 1 -- 172.21.15.114:0/2034343462 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed6c01d070 con 0x7fed7c101de0 2024-02-16T22:21:43.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.116+0000 7fed834ff700 1 -- 172.21.15.114:0/2034343462 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed7c073a80 con 0x7fed7c101de0 2024-02-16T22:21:43.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.116+0000 7fed727fc700 1 -- 172.21.15.114:0/2034343462 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fed6c005dc0 con 0x7fed7c101de0 2024-02-16T22:21:43.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.116+0000 7fed727fc700 1 -- 172.21.15.114:0/2034343462 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed6c017630 con 0x7fed7c101de0 2024-02-16T22:21:43.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.116+0000 7fed834ff700 1 -- 172.21.15.114:0/2034343462 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed7c073f70 con 0x7fed7c101de0 2024-02-16T22:21:43.121 INFO:tasks.workunit.client.1.smithi123.stdout:7/4: getdents . 0 2024-02-16T22:21:43.121 INFO:tasks.workunit.client.1.smithi123.stdout:7/5: write - no filename 2024-02-16T22:21:43.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.118+0000 7fed834ff700 1 -- 172.21.15.114:0/2034343462 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed7c109740 con 0x7fed7c101de0 2024-02-16T22:21:43.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.118+0000 7fed727fc700 1 -- 172.21.15.114:0/2034343462 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) v1 ==== 90933+0+0 (secure 0 0 0) 0x7fed6c00faf0 con 0x7fed7c101de0 2024-02-16T22:21:43.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.118+0000 7fed727fc700 1 --2- 172.21.15.114:0/2034343462 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fed6806ce40 0x7fed6806f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:43.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.119+0000 7fed727fc700 1 -- 172.21.15.114:0/2034343462 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5726+0+0 (secure 0 0 0) 0x7fed6c08da60 con 0x7fed7c101de0 2024-02-16T22:21:43.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.119+0000 7fed8129b700 1 --2- 172.21.15.114:0/2034343462 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fed6806ce40 0x7fed6806f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:43.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.119+0000 7fed8129b700 1 --2- 172.21.15.114:0/2034343462 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fed6806ce40 0x7fed6806f300 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fed7c101c40 tx=0x7fed78009260 comp rx=0 tx=0).ready entity=mgr.24441 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:21:43.122 INFO:tasks.workunit.client.1.smithi123.stdout:8/2: mkdir d0 0 2024-02-16T22:21:43.122 INFO:tasks.workunit.client.1.smithi123.stdout:8/3: chown d0 29658767 1 2024-02-16T22:21:43.123 INFO:tasks.workunit.client.1.smithi123.stdout:6/9: creat d0/f1 x:0 0 0 2024-02-16T22:21:43.123 INFO:tasks.workunit.client.1.smithi123.stdout:6/10: fdatasync d0/f1 0 2024-02-16T22:21:43.124 INFO:tasks.workunit.client.1.smithi123.stdout:6/11: chown d0 2083079966 1 2024-02-16T22:21:43.124 INFO:tasks.workunit.client.1.smithi123.stdout:9/0: symlink l0 0 2024-02-16T22:21:43.124 INFO:tasks.workunit.client.1.smithi123.stdout:9/1: fsync - no filename 2024-02-16T22:21:43.125 INFO:tasks.workunit.client.1.smithi123.stdout:6/12: write d0/f1 [293214,13623] 0 2024-02-16T22:21:43.125 INFO:tasks.workunit.client.1.smithi123.stdout:7/6: creat f0 x:0 0 0 2024-02-16T22:21:43.126 INFO:tasks.workunit.client.1.smithi123.stdout:7/7: read - f0 zero size 2024-02-16T22:21:43.126 INFO:tasks.workunit.client.1.smithi123.stdout:6/13: write d0/f1 [1219496,80324] 0 2024-02-16T22:21:43.127 INFO:tasks.workunit.client.1.smithi123.stdout:9/2: rename l0 to l1 0 2024-02-16T22:21:43.127 INFO:tasks.workunit.client.1.smithi123.stdout:9/3: write - no filename 2024-02-16T22:21:43.128 INFO:tasks.workunit.client.1.smithi123.stdout:8/4: symlink d0/l1 0 2024-02-16T22:21:43.128 INFO:tasks.workunit.client.1.smithi123.stdout:8/5: chown d0 372656 1 2024-02-16T22:21:43.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.127+0000 7fed727fc700 1 -- 172.21.15.114:0/2034343462 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182869 (secure 0 0 0) 0x7fed6c05b200 con 0x7fed7c101de0 2024-02-16T22:21:43.129 INFO:tasks.workunit.client.1.smithi123.stdout:7/8: symlink l1 0 2024-02-16T22:21:43.130 INFO:tasks.workunit.client.1.smithi123.stdout:7/9: chown l1 376 1 2024-02-16T22:21:43.130 INFO:tasks.workunit.client.1.smithi123.stdout:9/4: creat f2 x:0 0 0 2024-02-16T22:21:43.130 INFO:tasks.workunit.client.1.smithi123.stdout:9/5: stat f2 0 2024-02-16T22:21:43.131 INFO:tasks.workunit.client.1.smithi123.stdout:9/6: chown f2 1356375 1 2024-02-16T22:21:43.131 INFO:tasks.workunit.client.1.smithi123.stdout:7/10: rename f0 to f2 0 2024-02-16T22:21:43.133 INFO:tasks.workunit.client.1.smithi123.stdout:9/7: mknod c3 0 2024-02-16T22:21:43.134 INFO:tasks.workunit.client.1.smithi123.stdout:8/6: link d0/l1 d0/l2 0 2024-02-16T22:21:43.134 INFO:tasks.workunit.client.1.smithi123.stdout:8/7: read - no filename 2024-02-16T22:21:43.134 INFO:tasks.workunit.client.1.smithi123.stdout:8/8: dread - no filename 2024-02-16T22:21:43.137 INFO:tasks.workunit.client.1.smithi123.stdout:8/9: chown d0/l2 2537868 1 2024-02-16T22:21:43.137 INFO:tasks.workunit.client.1.smithi123.stdout:8/10: read - no filename 2024-02-16T22:21:43.140 INFO:tasks.workunit.client.1.smithi123.stdout:8/11: creat d0/f3 x:0 0 0 2024-02-16T22:21:43.141 INFO:tasks.workunit.client.0.smithi114.stdout:0/7: write d1/f2 [1522569,2876] 0 2024-02-16T22:21:43.143 INFO:tasks.workunit.client.0.smithi114.stdout:0/8: symlink d1/l4 0 2024-02-16T22:21:43.144 INFO:tasks.workunit.client.1.smithi123.stdout:9/8: dwrite f2 [0,4194304] 0 2024-02-16T22:21:43.144 INFO:tasks.workunit.client.1.smithi123.stdout:7/11: dwrite f2 [0,4194304] 0 2024-02-16T22:21:43.145 INFO:tasks.workunit.client.0.smithi114.stdout:0/9: read d1/f2 [2174861,101027] 0 2024-02-16T22:21:43.145 INFO:tasks.workunit.client.1.smithi123.stdout:8/12: creat d0/f4 x:0 0 0 2024-02-16T22:21:43.145 INFO:tasks.workunit.client.1.smithi123.stdout:9/9: read f2 [2671288,124725] 0 2024-02-16T22:21:43.145 INFO:tasks.workunit.client.1.smithi123.stdout:8/13: rename d0 to d0/d5 22 2024-02-16T22:21:43.148 INFO:tasks.workunit.client.1.smithi123.stdout:7/12: mkdir d3 0 2024-02-16T22:21:43.150 INFO:tasks.workunit.client.0.smithi114.stdout:2/30: fsync f4 0 2024-02-16T22:21:43.152 INFO:tasks.workunit.client.0.smithi114.stdout:1/15: dread d4/f5 [0,4194304] 0 2024-02-16T22:21:43.154 INFO:tasks.workunit.client.0.smithi114.stdout:2/31: dread f4 [0,4194304] 0 2024-02-16T22:21:43.155 INFO:tasks.workunit.client.1.smithi123.stdout:7/13: mknod d3/c4 0 2024-02-16T22:21:43.155 INFO:tasks.workunit.client.0.smithi114.stdout:1/16: rmdir d4 39 2024-02-16T22:21:43.179 INFO:tasks.workunit.client.0.smithi114.stdout:1/17: chown d4 161953 1 2024-02-16T22:21:43.180 INFO:tasks.workunit.client.0.smithi114.stdout:4/11: getdents . 0 2024-02-16T22:21:43.181 INFO:tasks.workunit.client.0.smithi114.stdout:4/12: write f2 [805273,33907] 0 2024-02-16T22:21:43.182 INFO:tasks.workunit.client.0.smithi114.stdout:4/13: truncate f2 1763001 0 2024-02-16T22:21:43.184 INFO:tasks.workunit.client.0.smithi114.stdout:1/18: fdatasync f1 0 2024-02-16T22:21:43.185 INFO:tasks.workunit.client.0.smithi114.stdout:2/32: rmdir d9 0 2024-02-16T22:21:43.186 INFO:tasks.workunit.client.0.smithi114.stdout:2/33: write f8 [889567,41100] 0 2024-02-16T22:21:43.187 INFO:tasks.workunit.client.0.smithi114.stdout:4/14: creat f4 x:0 0 0 2024-02-16T22:21:43.187 INFO:tasks.workunit.client.0.smithi114.stdout:2/34: fdatasync f0 0 2024-02-16T22:21:43.189 INFO:tasks.workunit.client.0.smithi114.stdout:1/19: rmdir d4 39 2024-02-16T22:21:43.191 INFO:tasks.workunit.client.0.smithi114.stdout:2/35: creat fa x:0 0 0 2024-02-16T22:21:43.200 INFO:tasks.workunit.client.0.smithi114.stdout:4/15: rename f1 to f5 0 2024-02-16T22:21:43.200 INFO:tasks.workunit.client.1.smithi123.stdout:9/10: fdatasync f2 0 2024-02-16T22:21:43.200 INFO:tasks.workunit.client.0.smithi114.stdout:4/16: chown f4 7125 1 2024-02-16T22:21:43.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:42 smithi114 ceph-mon[29235]: pgmap v19: 65 pgs: 65 active+clean; 221 MiB data, 883 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 5.4 MiB/s wr, 441 op/s 2024-02-16T22:21:43.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:42 smithi114 ceph-mon[29235]: from='client.24495 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:21:43.203 INFO:tasks.workunit.client.0.smithi114.stdout:4/17: symlink l6 0 2024-02-16T22:21:43.203 INFO:tasks.workunit.client.0.smithi114.stdout:4/18: rmdir - no directory 2024-02-16T22:21:43.203 INFO:tasks.workunit.client.0.smithi114.stdout:2/36: dread f4 [0,4194304] 0 2024-02-16T22:21:43.204 INFO:tasks.workunit.client.0.smithi114.stdout:4/19: truncate f5 4391167 0 2024-02-16T22:21:43.206 INFO:tasks.workunit.client.0.smithi114.stdout:2/37: symlink lb 0 2024-02-16T22:21:43.208 INFO:tasks.workunit.client.0.smithi114.stdout:2/38: creat fc x:0 0 0 2024-02-16T22:21:43.211 INFO:tasks.workunit.client.0.smithi114.stdout:4/20: dwrite f4 [0,4194304] 0 2024-02-16T22:21:43.215 INFO:tasks.workunit.client.0.smithi114.stdout:2/39: dread f0 [0,4194304] 0 2024-02-16T22:21:43.218 INFO:tasks.workunit.client.0.smithi114.stdout:2/40: mkdir dd 0 2024-02-16T22:21:43.219 INFO:tasks.workunit.client.0.smithi114.stdout:4/21: dwrite f2 [0,4194304] 0 2024-02-16T22:21:43.225 INFO:tasks.workunit.client.0.smithi114.stdout:2/41: mknod dd/ce 0 2024-02-16T22:21:43.231 INFO:tasks.workunit.client.0.smithi114.stdout:4/22: dwrite f5 [0,4194304] 0 2024-02-16T22:21:43.240 INFO:tasks.workunit.client.0.smithi114.stdout:8/28: dread f0 [0,4194304] 0 2024-02-16T22:21:43.240 INFO:tasks.workunit.client.0.smithi114.stdout:8/29: readlink d4/l5 0 2024-02-16T22:21:43.243 INFO:tasks.workunit.client.0.smithi114.stdout:8/30: creat d4/fa x:0 0 0 2024-02-16T22:21:43.245 INFO:tasks.workunit.client.0.smithi114.stdout:8/31: creat d4/fb x:0 0 0 2024-02-16T22:21:43.246 INFO:tasks.workunit.client.0.smithi114.stdout:8/32: write d4/fa [292048,19562] 0 2024-02-16T22:21:43.259 INFO:tasks.workunit.client.1.smithi123.stdout:4/4: chown l1 0 1 2024-02-16T22:21:43.260 INFO:tasks.workunit.client.1.smithi123.stdout:3/7: getdents d0 0 2024-02-16T22:21:43.261 INFO:tasks.workunit.client.1.smithi123.stdout:3/8: write d0/f1 [1013536,29404] 0 2024-02-16T22:21:43.264 INFO:tasks.workunit.client.1.smithi123.stdout:3/9: symlink d0/l2 0 2024-02-16T22:21:43.265 INFO:tasks.workunit.client.1.smithi123.stdout:3/10: fsync d0/f1 0 2024-02-16T22:21:43.267 INFO:tasks.workunit.client.1.smithi123.stdout:4/5: dwrite f3 [0,4194304] 0 2024-02-16T22:21:43.267 INFO:tasks.workunit.client.1.smithi123.stdout:9/11: fsync f2 0 2024-02-16T22:21:43.268 INFO:tasks.workunit.client.1.smithi123.stdout:9/12: write f2 [4000599,128833] 0 2024-02-16T22:21:43.268 INFO:tasks.workunit.client.1.smithi123.stdout:3/11: unlink d0/f1 0 2024-02-16T22:21:43.268 INFO:tasks.workunit.client.1.smithi123.stdout:3/12: dread - no filename 2024-02-16T22:21:43.271 INFO:tasks.workunit.client.1.smithi123.stdout:4/6: dread f3 [0,4194304] 0 2024-02-16T22:21:43.271 INFO:tasks.workunit.client.1.smithi123.stdout:4/7: chown f3 1037144 1 2024-02-16T22:21:43.272 INFO:tasks.workunit.client.1.smithi123.stdout:9/13: readlink l1 0 2024-02-16T22:21:43.274 INFO:tasks.workunit.client.1.smithi123.stdout:7/14: write f2 [4476556,128608] 0 2024-02-16T22:21:43.275 INFO:tasks.workunit.client.1.smithi123.stdout:4/8: creat f4 x:0 0 0 2024-02-16T22:21:43.276 INFO:tasks.workunit.client.1.smithi123.stdout:4/9: chown l2 45967606 1 2024-02-16T22:21:43.276 INFO:tasks.workunit.client.1.smithi123.stdout:4/10: stat f4 0 2024-02-16T22:21:43.276 INFO:tasks.workunit.client.1.smithi123.stdout:4/11: dread - f4 zero size 2024-02-16T22:21:43.277 INFO:tasks.workunit.client.1.smithi123.stdout:9/14: mkdir d4 0 2024-02-16T22:21:43.277 INFO:tasks.workunit.client.1.smithi123.stdout:9/15: readlink l1 0 2024-02-16T22:21:43.278 INFO:tasks.workunit.client.1.smithi123.stdout:7/15: mkdir d3/d5 0 2024-02-16T22:21:43.279 INFO:tasks.workunit.client.1.smithi123.stdout:8/14: rmdir d0 39 2024-02-16T22:21:43.280 INFO:tasks.workunit.client.1.smithi123.stdout:9/16: dread f2 [0,4194304] 0 2024-02-16T22:21:43.281 INFO:tasks.workunit.client.0.smithi114.stdout:0/10: truncate d1/f2 2748264 0 2024-02-16T22:21:43.282 INFO:tasks.workunit.client.1.smithi123.stdout:4/12: mkdir d5 0 2024-02-16T22:21:43.284 INFO:tasks.workunit.client.0.smithi114.stdout:0/11: fdatasync d1/f2 0 2024-02-16T22:21:43.291 INFO:tasks.workunit.client.0.smithi114.stdout:1/20: truncate d4/f5 2014519 0 2024-02-16T22:21:43.292 INFO:tasks.workunit.client.1.smithi123.stdout:8/15: dwrite d0/f3 [0,4194304] 0 2024-02-16T22:21:43.294 INFO:tasks.workunit.client.0.smithi114.stdout:1/21: mkdir d4/d9 0 2024-02-16T22:21:43.295 INFO:tasks.workunit.client.0.smithi114.stdout:1/22: chown d4/c6 6 1 2024-02-16T22:21:43.295 INFO:tasks.workunit.client.0.smithi114.stdout:1/23: chown l3 1 1 2024-02-16T22:21:43.296 INFO:tasks.workunit.client.1.smithi123.stdout:9/17: creat d4/f5 x:0 0 0 2024-02-16T22:21:43.296 INFO:tasks.workunit.client.1.smithi123.stdout:4/13: symlink d5/l6 0 2024-02-16T22:21:43.297 INFO:tasks.workunit.client.1.smithi123.stdout:4/14: dread - f4 zero size 2024-02-16T22:21:43.300 INFO:tasks.workunit.client.1.smithi123.stdout:4/15: dread f3 [0,4194304] 0 2024-02-16T22:21:43.301 INFO:tasks.workunit.client.1.smithi123.stdout:8/16: chown d0/l2 1 1 2024-02-16T22:21:43.306 INFO:tasks.workunit.client.1.smithi123.stdout:9/18: creat d4/f6 x:0 0 0 2024-02-16T22:21:43.306 INFO:tasks.workunit.client.1.smithi123.stdout:9/19: dread - d4/f5 zero size 2024-02-16T22:21:43.306 INFO:tasks.workunit.client.1.smithi123.stdout:9/20: read - d4/f5 zero size 2024-02-16T22:21:43.307 INFO:tasks.workunit.client.1.smithi123.stdout:9/21: truncate d4/f5 784193 0 2024-02-16T22:21:43.308 INFO:tasks.workunit.client.1.smithi123.stdout:9/22: write d4/f5 [529503,61730] 0 2024-02-16T22:21:43.308 INFO:tasks.workunit.client.1.smithi123.stdout:4/16: write f3 [3932386,9284] 0 2024-02-16T22:21:43.309 INFO:tasks.workunit.client.1.smithi123.stdout:4/17: write f3 [1524870,115276] 0 2024-02-16T22:21:43.309 INFO:tasks.workunit.client.1.smithi123.stdout:9/23: write d4/f6 [19266,62030] 0 2024-02-16T22:21:43.315 INFO:tasks.workunit.client.0.smithi114.stdout:1/24: creat d4/fa x:0 0 0 2024-02-16T22:21:43.316 INFO:tasks.workunit.client.0.smithi114.stdout:4/23: write f4 [4816589,109952] 0 2024-02-16T22:21:43.318 INFO:tasks.workunit.client.1.smithi123.stdout:4/18: dwrite f3 [0,4194304] 0 2024-02-16T22:21:43.318 INFO:tasks.workunit.client.1.smithi123.stdout:9/24: dwrite d4/f5 [0,4194304] 0 2024-02-16T22:21:43.319 INFO:tasks.workunit.client.0.smithi114.stdout:8/33: getdents d4 0 2024-02-16T22:21:43.319 INFO:tasks.workunit.client.1.smithi123.stdout:4/19: chown f4 7 1 2024-02-16T22:21:43.319 INFO:tasks.workunit.client.1.smithi123.stdout:9/25: write d4/f6 [46936,16393] 0 2024-02-16T22:21:43.322 INFO:tasks.workunit.client.1.smithi123.stdout:4/20: dread f3 [0,4194304] 0 2024-02-16T22:21:43.323 INFO:tasks.workunit.client.1.smithi123.stdout:4/21: readlink l1 0 2024-02-16T22:21:43.323 INFO:tasks.workunit.client.1.smithi123.stdout:4/22: write f4 [148027,42852] 0 2024-02-16T22:21:43.323 INFO:tasks.workunit.client.1.smithi123.stdout:4/23: chown d5/l6 0 1 2024-02-16T22:21:43.325 INFO:tasks.workunit.client.0.smithi114.stdout:9/14: dwrite f1 [0,4194304] 0 2024-02-16T22:21:43.325 INFO:tasks.workunit.client.0.smithi114.stdout:3/24: dwrite f1 [0,4194304] 0 2024-02-16T22:21:43.326 INFO:tasks.workunit.client.0.smithi114.stdout:9/15: readlink d2/l3 0 2024-02-16T22:21:43.327 INFO:tasks.workunit.client.0.smithi114.stdout:9/16: chown f1 921335 1 2024-02-16T22:21:43.329 INFO:tasks.workunit.client.0.smithi114.stdout:9/17: write f1 [1376183,91326] 0 2024-02-16T22:21:43.329 INFO:tasks.workunit.client.1.smithi123.stdout:3/13: rmdir d0 39 2024-02-16T22:21:43.329 INFO:tasks.workunit.client.1.smithi123.stdout:3/14: dread - no filename 2024-02-16T22:21:43.329 INFO:tasks.workunit.client.1.smithi123.stdout:3/15: dwrite - no filename 2024-02-16T22:21:43.337 INFO:tasks.workunit.client.1.smithi123.stdout:9/26: mknod d4/c7 0 2024-02-16T22:21:43.350 INFO:tasks.workunit.client.1.smithi123.stdout:7/16: truncate f2 1030982 0 2024-02-16T22:21:43.355 INFO:tasks.workunit.client.1.smithi123.stdout:4/24: link f4 d5/f7 0 2024-02-16T22:21:43.357 INFO:tasks.workunit.client.1.smithi123.stdout:3/16: creat d0/f3 x:0 0 0 2024-02-16T22:21:43.358 INFO:tasks.workunit.client.1.smithi123.stdout:3/17: write d0/f3 [935252,58158] 0 2024-02-16T22:21:43.358 INFO:tasks.workunit.client.1.smithi123.stdout:3/18: chown d0 12 1 2024-02-16T22:21:43.359 INFO:tasks.workunit.client.1.smithi123.stdout:7/17: symlink d3/l6 0 2024-02-16T22:21:43.362 INFO:tasks.workunit.client.1.smithi123.stdout:4/25: symlink d5/l8 0 2024-02-16T22:21:43.364 INFO:tasks.workunit.client.1.smithi123.stdout:3/19: symlink d0/l4 0 2024-02-16T22:21:43.364 INFO:tasks.workunit.client.1.smithi123.stdout:3/20: chown d0/l2 51302106 1 2024-02-16T22:21:43.369 INFO:tasks.workunit.client.1.smithi123.stdout:4/26: unlink l2 0 2024-02-16T22:21:43.371 INFO:tasks.workunit.client.1.smithi123.stdout:8/17: truncate d0/f3 2771663 0 2024-02-16T22:21:43.374 INFO:tasks.workunit.client.1.smithi123.stdout:4/27: mkdir d5/d9 0 2024-02-16T22:21:43.377 INFO:tasks.workunit.client.1.smithi123.stdout:9/27: getdents d4 0 2024-02-16T22:21:43.377 INFO:tasks.workunit.client.1.smithi123.stdout:8/18: rmdir d0 39 2024-02-16T22:21:43.380 INFO:tasks.workunit.client.1.smithi123.stdout:3/21: link d0/l4 d0/l5 0 2024-02-16T22:21:43.383 INFO:tasks.workunit.client.1.smithi123.stdout:9/28: symlink d4/l8 0 2024-02-16T22:21:43.384 INFO:tasks.workunit.client.1.smithi123.stdout:4/28: dwrite f3 [0,4194304] 0 2024-02-16T22:21:43.387 INFO:tasks.workunit.client.0.smithi114.stdout:0/12: truncate d1/f2 657780 0 2024-02-16T22:21:43.388 INFO:tasks.workunit.client.1.smithi123.stdout:3/22: creat d0/f6 x:0 0 0 2024-02-16T22:21:43.389 INFO:tasks.workunit.client.1.smithi123.stdout:9/29: dwrite d4/f5 [0,4194304] 0 2024-02-16T22:21:43.389 INFO:tasks.workunit.client.1.smithi123.stdout:3/23: write d0/f3 [1849421,102524] 0 2024-02-16T22:21:43.394 INFO:tasks.workunit.client.1.smithi123.stdout:4/29: symlink d5/la 0 2024-02-16T22:21:43.396 INFO:tasks.workunit.client.1.smithi123.stdout:8/19: creat d0/f6 x:0 0 0 2024-02-16T22:21:43.396 INFO:tasks.workunit.client.1.smithi123.stdout:4/30: dread f4 [0,4194304] 0 2024-02-16T22:21:43.399 INFO:tasks.workunit.client.1.smithi123.stdout:9/30: symlink d4/l9 0 2024-02-16T22:21:43.400 INFO:tasks.workunit.client.1.smithi123.stdout:9/31: dread d4/f6 [0,4194304] 0 2024-02-16T22:21:43.401 INFO:tasks.workunit.client.1.smithi123.stdout:9/32: chown d4/l9 1644 1 2024-02-16T22:21:43.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.427+0000 7fed834ff700 1 -- 172.21.15.114:0/2034343462 --> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fed7c106730 con 0x7fed6806ce40 2024-02-16T22:21:43.431 INFO:tasks.workunit.client.1.smithi123.stdout:3/24: unlink d0/l2 0 2024-02-16T22:21:43.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.438+0000 7fed727fc700 1 -- 172.21.15.114:0/2034343462 <== mgr.24441 v2:172.21.15.123:6828/1539029039 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fed7c106730 con 0x7fed6806ce40 2024-02-16T22:21:43.440 INFO:teuthology.orchestra.run.smithi114.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:21:43.441 INFO:teuthology.orchestra.run.smithi114.stdout:alertmanager.smithi114 smithi114 *:9093,9094 running (7m) 20s ago 9m 24.0M - 0.25.0 c8568f914cd2 e7404fea400e 2024-02-16T22:21:43.441 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi114 smithi114 running (9m) 20s ago 9m 7666k - 18.2.1 94eadd990e0a 7fd07162cd96 2024-02-16T22:21:43.441 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi123 smithi123 running (8m) 24s ago 8m 7402k - 18.2.1 94eadd990e0a 7687f5e99621 2024-02-16T22:21:43.441 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi114 smithi114 running (9m) 20s ago 9m 6660k - 18.2.1 94eadd990e0a fc8fbfe4fdd5 2024-02-16T22:21:43.441 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi123 smithi123 running (8m) 24s ago 8m 6665k - 18.2.1 94eadd990e0a a6249642b9bb 2024-02-16T22:21:43.441 INFO:teuthology.orchestra.run.smithi114.stdout:grafana.smithi114 smithi114 *:3000 running (7m) 20s ago 8m 89.8M - 9.4.7 954c08fa6188 db1c26c3bc20 2024-02-16T22:21:43.441 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.fdxpey smithi114 running (3m) 20s ago 3m 15.3M - 18.2.1 94eadd990e0a 704b40c144fb 2024-02-16T22:21:43.441 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.zzryke smithi114 running (3m) 20s ago 3m 150M - 18.2.1 94eadd990e0a 1f795d23c9a4 2024-02-16T22:21:43.441 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.gopcfa smithi123 running (3m) 24s ago 3m 15.2M - 18.2.1 94eadd990e0a 166134b7532a 2024-02-16T22:21:43.441 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.vhlqbe smithi123 running (3m) 24s ago 3m 16.7M - 18.2.1 94eadd990e0a 243c8bb96fa8 2024-02-16T22:21:43.441 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi114.vcullz smithi114 *:9283,8765,8443 running (10m) 20s ago 10m 363M - 18.2.1 94eadd990e0a c8f21783728d 2024-02-16T22:21:43.441 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi123.inzgjd smithi123 *:8443,9283,8765 running (51s) 24s ago 8m 487M - 19.0.0-1450-gf78a58c0 f081f847070a c9d6ba733f8b 2024-02-16T22:21:43.441 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi114 smithi114 running (10m) 20s ago 10m 52.5M 2048M 18.2.1 94eadd990e0a b4c4433aa3a4 2024-02-16T22:21:43.441 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi123 smithi123 running (8m) 24s ago 8m 37.2M 2048M 18.2.1 94eadd990e0a c217fc30c152 2024-02-16T22:21:43.442 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi114 smithi114 *:9100 running (9m) 20s ago 9m 15.7M - 1.5.0 0da6a335fe13 744a090e06d0 2024-02-16T22:21:43.442 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi123 smithi123 *:9100 running (8m) 24s ago 8m 15.2M - 1.5.0 0da6a335fe13 3994b8c29b2e 2024-02-16T22:21:43.442 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 smithi114 running (7m) 20s ago 7m 74.7M 1216M 18.2.1 94eadd990e0a 24ca2e9714e0 2024-02-16T22:21:43.442 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 smithi114 running (6m) 20s ago 6m 77.5M 1216M 18.2.1 94eadd990e0a 32bb191b88e7 2024-02-16T22:21:43.442 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 smithi114 running (6m) 20s ago 6m 70.5M 1216M 18.2.1 94eadd990e0a 6608718d0818 2024-02-16T22:21:43.442 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 smithi123 running (5m) 24s ago 5m 86.3M 2240M 18.2.1 94eadd990e0a 575db379de3d 2024-02-16T22:21:43.442 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 smithi123 running (5m) 24s ago 5m 73.9M 2240M 18.2.1 94eadd990e0a 7e266866cd9c 2024-02-16T22:21:43.442 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 smithi123 running (5m) 24s ago 5m 69.8M 2240M 18.2.1 94eadd990e0a 24e5bef19fda 2024-02-16T22:21:43.442 INFO:teuthology.orchestra.run.smithi114.stdout:prometheus.smithi114 smithi114 *:9095 starting - - - - 2024-02-16T22:21:43.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.440+0000 7fed834ff700 1 -- 172.21.15.114:0/2034343462 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fed6806ce40 msgr2=0x7fed6806f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:43.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.440+0000 7fed834ff700 1 --2- 172.21.15.114:0/2034343462 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fed6806ce40 0x7fed6806f300 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fed7c101c40 tx=0x7fed78009260 comp rx=0 tx=0).stop 2024-02-16T22:21:43.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.441+0000 7fed834ff700 1 -- 172.21.15.114:0/2034343462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed7c101de0 msgr2=0x7fed7c073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:43.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.441+0000 7fed834ff700 1 --2- 172.21.15.114:0/2034343462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed7c101de0 0x7fed7c073180 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fed6c0092e0 tx=0x7fed6c00f690 comp rx=0 tx=0).stop 2024-02-16T22:21:43.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.441+0000 7fed834ff700 1 -- 172.21.15.114:0/2034343462 shutdown_connections 2024-02-16T22:21:43.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.441+0000 7fed834ff700 1 --2- 172.21.15.114:0/2034343462 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fed6806ce40 0x7fed6806f300 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:43.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.441+0000 7fed834ff700 1 --2- 172.21.15.114:0/2034343462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed7c100be0 0x7fed7c074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:43.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.441+0000 7fed834ff700 1 --2- 172.21.15.114:0/2034343462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed7c101de0 0x7fed7c073180 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:43.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.441+0000 7fed834ff700 1 -- 172.21.15.114:0/2034343462 >> 172.21.15.114:0/2034343462 conn(0x7fed7c0fc140 msgr2=0x7fed7c105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:21:43.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.441+0000 7fed834ff700 1 -- 172.21.15.114:0/2034343462 shutdown_connections 2024-02-16T22:21:43.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.441+0000 7fed834ff700 1 -- 172.21.15.114:0/2034343462 wait complete. 2024-02-16T22:21:43.450 INFO:tasks.workunit.client.1.smithi123.stdout:8/20: creat d0/f7 x:0 0 0 2024-02-16T22:21:43.454 INFO:tasks.workunit.client.1.smithi123.stdout:9/33: rmdir d4 39 2024-02-16T22:21:43.456 INFO:tasks.workunit.client.1.smithi123.stdout:3/25: creat d0/f7 x:0 0 0 2024-02-16T22:21:43.469 INFO:tasks.workunit.client.1.smithi123.stdout:4/31: rmdir d5/d9 0 2024-02-16T22:21:43.469 INFO:tasks.workunit.client.1.smithi123.stdout:4/32: read d5/f7 [24549,128028] 0 2024-02-16T22:21:43.470 INFO:tasks.workunit.client.1.smithi123.stdout:4/33: write f3 [2829396,12285] 0 2024-02-16T22:21:43.471 INFO:tasks.workunit.client.1.smithi123.stdout:9/34: dread d4/f6 [0,4194304] 0 2024-02-16T22:21:43.472 INFO:tasks.workunit.client.1.smithi123.stdout:8/21: link d0/l2 d0/l8 0 2024-02-16T22:21:43.480 INFO:tasks.workunit.client.1.smithi123.stdout:9/35: creat d4/fa x:0 0 0 2024-02-16T22:21:43.481 INFO:tasks.workunit.client.1.smithi123.stdout:8/22: mknod d0/c9 0 2024-02-16T22:21:43.484 INFO:tasks.workunit.client.1.smithi123.stdout:9/36: fsync d4/f6 0 2024-02-16T22:21:43.485 INFO:tasks.workunit.client.1.smithi123.stdout:9/37: readlink d4/l8 0 2024-02-16T22:21:43.488 INFO:tasks.workunit.client.1.smithi123.stdout:9/38: creat d4/fb x:0 0 0 2024-02-16T22:21:43.490 INFO:tasks.workunit.client.1.smithi123.stdout:8/23: dwrite d0/f3 [0,4194304] 0 2024-02-16T22:21:43.491 INFO:tasks.workunit.client.1.smithi123.stdout:9/39: symlink d4/lc 0 2024-02-16T22:21:43.494 INFO:tasks.workunit.client.1.smithi123.stdout:4/34: getdents d5 0 2024-02-16T22:21:43.509 INFO:tasks.workunit.client.1.smithi123.stdout:8/24: link d0/l2 d0/la 0 2024-02-16T22:21:43.509 INFO:tasks.workunit.client.1.smithi123.stdout:8/25: read - d0/f6 zero size 2024-02-16T22:21:43.520 INFO:tasks.workunit.client.1.smithi123.stdout:9/40: dwrite f2 [0,4194304] 0 2024-02-16T22:21:43.521 INFO:tasks.workunit.client.1.smithi123.stdout:9/41: readlink l1 0 2024-02-16T22:21:43.522 INFO:tasks.workunit.client.1.smithi123.stdout:4/35: dwrite f4 [0,4194304] 0 2024-02-16T22:21:43.525 INFO:tasks.workunit.client.1.smithi123.stdout:9/42: mknod d4/cd 0 2024-02-16T22:21:43.527 INFO:tasks.workunit.client.1.smithi123.stdout:4/36: dread f3 [0,4194304] 0 2024-02-16T22:21:43.529 INFO:tasks.workunit.client.1.smithi123.stdout:9/43: creat d4/fe x:0 0 0 2024-02-16T22:21:43.534 INFO:tasks.workunit.client.1.smithi123.stdout:9/44: mknod d4/cf 0 2024-02-16T22:21:43.537 INFO:tasks.workunit.client.1.smithi123.stdout:4/37: dwrite f3 [0,4194304] 0 2024-02-16T22:21:43.537 INFO:tasks.workunit.client.1.smithi123.stdout:9/45: creat d4/f10 x:0 0 0 2024-02-16T22:21:43.542 INFO:tasks.workunit.client.1.smithi123.stdout:4/38: creat d5/fb x:0 0 0 2024-02-16T22:21:43.548 INFO:tasks.workunit.client.1.smithi123.stdout:4/39: dwrite d5/f7 [0,4194304] 0 2024-02-16T22:21:43.553 INFO:tasks.workunit.client.1.smithi123.stdout:4/40: rename d5/fb to d5/fc 0 2024-02-16T22:21:43.554 INFO:tasks.workunit.client.1.smithi123.stdout:9/46: link c3 d4/c11 0 2024-02-16T22:21:43.561 INFO:tasks.workunit.client.1.smithi123.stdout:4/41: dwrite f4 [4194304,4194304] 0 2024-02-16T22:21:43.563 INFO:tasks.workunit.client.1.smithi123.stdout:9/47: dwrite d4/f10 [0,4194304] 0 2024-02-16T22:21:43.567 INFO:tasks.workunit.client.1.smithi123.stdout:9/48: creat d4/f12 x:0 0 0 2024-02-16T22:21:43.570 INFO:tasks.workunit.client.1.smithi123.stdout:0/5: sync 2024-02-16T22:21:43.570 INFO:tasks.workunit.client.1.smithi123.stdout:0/6: dread - no filename 2024-02-16T22:21:43.570 INFO:tasks.workunit.client.1.smithi123.stdout:0/7: truncate - no filename 2024-02-16T22:21:43.570 INFO:tasks.workunit.client.1.smithi123.stdout:1/5: sync 2024-02-16T22:21:43.579 INFO:tasks.workunit.client.1.smithi123.stdout:9/49: truncate d4/fb 280725 0 2024-02-16T22:21:43.579 INFO:tasks.workunit.client.1.smithi123.stdout:9/50: fdatasync d4/f6 0 2024-02-16T22:21:43.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.587+0000 7fe9af6d5700 1 -- 172.21.15.114:0/1254372476 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9a80737f0 msgr2=0x7fe9a8073c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:43.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.587+0000 7fe9af6d5700 1 --2- 172.21.15.114:0/1254372476 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9a80737f0 0x7fe9a8073c70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe998009910 tx=0x7fe998009c20 comp rx=0 tx=0).stop 2024-02-16T22:21:43.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.588+0000 7fe9af6d5700 1 -- 172.21.15.114:0/1254372476 shutdown_connections 2024-02-16T22:21:43.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.588+0000 7fe9af6d5700 1 --2- 172.21.15.114:0/1254372476 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9a80737f0 0x7fe9a8073c70 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:43.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.588+0000 7fe9af6d5700 1 --2- 172.21.15.114:0/1254372476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9a8074dc0 0x7fe9a8073220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:43.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.588+0000 7fe9af6d5700 1 -- 172.21.15.114:0/1254372476 >> 172.21.15.114:0/1254372476 conn(0x7fe9a80fc110 msgr2=0x7fe9a80fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:21:43.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.588+0000 7fe9af6d5700 1 -- 172.21.15.114:0/1254372476 shutdown_connections 2024-02-16T22:21:43.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.588+0000 7fe9af6d5700 1 -- 172.21.15.114:0/1254372476 wait complete. 2024-02-16T22:21:43.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.588+0000 7fe9af6d5700 1 Processor -- start 2024-02-16T22:21:43.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.589+0000 7fe9af6d5700 1 -- start start 2024-02-16T22:21:43.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.589+0000 7fe9af6d5700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9a80737f0 0x7fe9a81985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:43.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.589+0000 7fe9af6d5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9a8074dc0 0x7fe9a8198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:43.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.589+0000 7fe9af6d5700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a8199110 con 0x7fe9a80737f0 2024-02-16T22:21:43.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.589+0000 7fe9af6d5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a8199250 con 0x7fe9a8074dc0 2024-02-16T22:21:43.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.589+0000 7fe9ad471700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9a80737f0 0x7fe9a81985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:43.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.589+0000 7fe9ad471700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9a80737f0 0x7fe9a81985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39418/0 (socket says 172.21.15.114:39418) 2024-02-16T22:21:43.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.589+0000 7fe9ad471700 1 -- 172.21.15.114:0/3369193630 learned_addr learned my addr 172.21.15.114:0/3369193630 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:21:43.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.589+0000 7fe9acc70700 1 --2- 172.21.15.114:0/3369193630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9a8074dc0 0x7fe9a8198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:43.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.589+0000 7fe9ad471700 1 -- 172.21.15.114:0/3369193630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9a8074dc0 msgr2=0x7fe9a8198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:43.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.589+0000 7fe9ad471700 1 --2- 172.21.15.114:0/3369193630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9a8074dc0 0x7fe9a8198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:43.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.589+0000 7fe9ad471700 1 -- 172.21.15.114:0/3369193630 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9980095f0 con 0x7fe9a80737f0 2024-02-16T22:21:43.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.589+0000 7fe9acc70700 1 --2- 172.21.15.114:0/3369193630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9a8074dc0 0x7fe9a8198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:21:43.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.589+0000 7fe9ad471700 1 --2- 172.21.15.114:0/3369193630 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9a80737f0 0x7fe9a81985b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400e8f0 tx=0x7fe9a400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:21:43.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.590+0000 7fe99e7fc700 1 -- 172.21.15.114:0/3369193630 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9a4009790 con 0x7fe9a80737f0 2024-02-16T22:21:43.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.590+0000 7fe9af6d5700 1 -- 172.21.15.114:0/3369193630 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a819dd00 con 0x7fe9a80737f0 2024-02-16T22:21:43.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.590+0000 7fe9af6d5700 1 -- 172.21.15.114:0/3369193630 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a819e1f0 con 0x7fe9a80737f0 2024-02-16T22:21:43.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.590+0000 7fe99e7fc700 1 -- 172.21.15.114:0/3369193630 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe9a400c810 con 0x7fe9a80737f0 2024-02-16T22:21:43.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.590+0000 7fe99e7fc700 1 -- 172.21.15.114:0/3369193630 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9a40055b0 con 0x7fe9a80737f0 2024-02-16T22:21:43.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.591+0000 7fe9af6d5700 1 -- 172.21.15.114:0/3369193630 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe98c005320 con 0x7fe9a80737f0 2024-02-16T22:21:43.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.592+0000 7fe99e7fc700 1 -- 172.21.15.114:0/3369193630 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) v1 ==== 90933+0+0 (secure 0 0 0) 0x7fe9a400c980 con 0x7fe9a80737f0 2024-02-16T22:21:43.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.592+0000 7fe99e7fc700 1 --2- 172.21.15.114:0/3369193630 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fe99406ce40 0x7fe99406f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:43.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.592+0000 7fe99e7fc700 1 -- 172.21.15.114:0/3369193630 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5726+0+0 (secure 0 0 0) 0x7fe9a4014070 con 0x7fe9a80737f0 2024-02-16T22:21:43.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.592+0000 7fe9acc70700 1 --2- 172.21.15.114:0/3369193630 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fe99406ce40 0x7fe99406f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:43.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.593+0000 7fe9acc70700 1 --2- 172.21.15.114:0/3369193630 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fe99406ce40 0x7fe99406f300 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe99800b6f0 tx=0x7fe998005940 comp rx=0 tx=0).ready entity=mgr.24441 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:21:43.597 INFO:tasks.workunit.client.1.smithi123.stdout:4/42: rmdir d5 39 2024-02-16T22:21:43.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.597+0000 7fe99e7fc700 1 -- 172.21.15.114:0/3369193630 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182869 (secure 0 0 0) 0x7fe9a4059f90 con 0x7fe9a80737f0 2024-02-16T22:21:43.614 INFO:tasks.workunit.client.1.smithi123.stdout:5/1: sync 2024-02-16T22:21:43.614 INFO:tasks.workunit.client.1.smithi123.stdout:2/16: sync 2024-02-16T22:21:43.614 INFO:tasks.workunit.client.1.smithi123.stdout:5/2: readlink - no filename 2024-02-16T22:21:43.615 INFO:tasks.workunit.client.1.smithi123.stdout:5/3: dwrite - no filename 2024-02-16T22:21:43.695 INFO:tasks.workunit.client.1.smithi123.stdout:7/18: sync 2024-02-16T22:21:43.696 INFO:tasks.workunit.client.1.smithi123.stdout:6/14: sync 2024-02-16T22:21:43.696 INFO:tasks.workunit.client.1.smithi123.stdout:3/26: sync 2024-02-16T22:21:43.712 INFO:tasks.workunit.client.0.smithi114.stdout:4/24: mknod c7 0 2024-02-16T22:21:43.713 INFO:tasks.workunit.client.0.smithi114.stdout:8/34: creat d4/fc x:0 0 0 2024-02-16T22:21:43.716 INFO:tasks.workunit.client.0.smithi114.stdout:3/25: creat d4/f7 x:0 0 0 2024-02-16T22:21:43.718 INFO:tasks.workunit.client.0.smithi114.stdout:9/18: creat d2/f4 x:0 0 0 2024-02-16T22:21:43.719 INFO:tasks.workunit.client.0.smithi114.stdout:9/19: chown d2/l3 1 1 2024-02-16T22:21:43.725 INFO:tasks.workunit.client.0.smithi114.stdout:9/20: dwrite d2/f4 [0,4194304] 0 2024-02-16T22:21:43.727 INFO:tasks.workunit.client.0.smithi114.stdout:4/25: mknod c8 0 2024-02-16T22:21:43.727 INFO:tasks.workunit.client.0.smithi114.stdout:4/26: write f4 [456195,31124] 0 2024-02-16T22:21:43.728 INFO:tasks.workunit.client.0.smithi114.stdout:4/27: chown c7 0 1 2024-02-16T22:21:43.730 INFO:tasks.workunit.client.0.smithi114.stdout:8/35: symlink d4/ld 0 2024-02-16T22:21:43.730 INFO:tasks.workunit.client.0.smithi114.stdout:8/36: dread - d4/fc zero size 2024-02-16T22:21:43.733 INFO:tasks.workunit.client.0.smithi114.stdout:3/26: mknod d4/c8 0 2024-02-16T22:21:43.734 INFO:tasks.workunit.client.0.smithi114.stdout:3/27: read f1 [3122263,110274] 0 2024-02-16T22:21:43.735 INFO:tasks.workunit.client.1.smithi123.stdout:8/26: sync 2024-02-16T22:21:43.748 INFO:tasks.workunit.client.0.smithi114.stdout:3/28: rename d4/l5 to d4/l9 0 2024-02-16T22:21:43.752 INFO:tasks.workunit.client.0.smithi114.stdout:3/29: dread f0 [0,4194304] 0 2024-02-16T22:21:43.752 INFO:tasks.workunit.client.0.smithi114.stdout:9/21: link d2/f4 d2/f5 0 2024-02-16T22:21:43.850 INFO:tasks.workunit.client.1.smithi123.stdout:9/51: rename d4/lc to d4/l13 0 2024-02-16T22:21:43.851 INFO:tasks.workunit.client.1.smithi123.stdout:2/17: symlink l3 0 2024-02-16T22:21:43.852 INFO:tasks.workunit.client.1.smithi123.stdout:7/19: rmdir d3 39 2024-02-16T22:21:43.853 INFO:tasks.workunit.client.1.smithi123.stdout:3/27: symlink d0/l8 0 2024-02-16T22:21:43.853 INFO:tasks.workunit.client.1.smithi123.stdout:4/43: dread d5/f7 [0,4194304] 0 2024-02-16T22:21:43.854 INFO:tasks.workunit.client.1.smithi123.stdout:8/27: mknod d0/cb 0 2024-02-16T22:21:43.855 INFO:tasks.workunit.client.1.smithi123.stdout:4/44: write f4 [7563157,115697] 0 2024-02-16T22:21:43.856 INFO:tasks.workunit.client.1.smithi123.stdout:9/52: symlink d4/l14 0 2024-02-16T22:21:43.858 INFO:tasks.workunit.client.1.smithi123.stdout:5/4: symlink l0 0 2024-02-16T22:21:43.858 INFO:tasks.workunit.client.1.smithi123.stdout:5/5: write - no filename 2024-02-16T22:21:43.858 INFO:tasks.workunit.client.1.smithi123.stdout:5/6: dread - no filename 2024-02-16T22:21:43.860 INFO:tasks.workunit.client.1.smithi123.stdout:9/53: write d4/f5 [2248476,55805] 0 2024-02-16T22:21:43.860 INFO:tasks.workunit.client.1.smithi123.stdout:2/18: dwrite f2 [0,4194304] 0 2024-02-16T22:21:43.861 INFO:tasks.workunit.client.1.smithi123.stdout:6/15: link d0/f1 d0/f2 0 2024-02-16T22:21:43.862 INFO:tasks.workunit.client.1.smithi123.stdout:7/20: read f2 [982008,66118] 0 2024-02-16T22:21:43.862 INFO:tasks.workunit.client.1.smithi123.stdout:3/28: creat d0/f9 x:0 0 0 2024-02-16T22:21:43.863 INFO:tasks.workunit.client.1.smithi123.stdout:0/8: rmdir d0 0 2024-02-16T22:21:43.864 INFO:tasks.workunit.client.1.smithi123.stdout:0/9: write - no filename 2024-02-16T22:21:43.864 INFO:tasks.workunit.client.1.smithi123.stdout:3/29: readlink d0/l8 0 2024-02-16T22:21:43.864 INFO:tasks.workunit.client.1.smithi123.stdout:7/21: read f2 [1029042,12212] 0 2024-02-16T22:21:43.867 INFO:tasks.workunit.client.1.smithi123.stdout:4/45: dwrite f3 [0,4194304] 0 2024-02-16T22:21:43.869 INFO:tasks.workunit.client.1.smithi123.stdout:5/7: creat f1 x:0 0 0 2024-02-16T22:21:43.871 INFO:tasks.workunit.client.1.smithi123.stdout:6/16: fdatasync d0/f1 0 2024-02-16T22:21:43.872 INFO:tasks.workunit.client.1.smithi123.stdout:1/6: getdents . 0 2024-02-16T22:21:43.874 INFO:tasks.workunit.client.1.smithi123.stdout:0/10: mkdir d1 0 2024-02-16T22:21:43.874 INFO:tasks.workunit.client.1.smithi123.stdout:0/11: unlink - no file 2024-02-16T22:21:43.874 INFO:tasks.workunit.client.1.smithi123.stdout:0/12: read - no filename 2024-02-16T22:21:43.874 INFO:tasks.workunit.client.1.smithi123.stdout:0/13: dread - no filename 2024-02-16T22:21:43.874 INFO:tasks.workunit.client.1.smithi123.stdout:0/14: dread - no filename 2024-02-16T22:21:43.875 INFO:tasks.workunit.client.1.smithi123.stdout:3/30: dwrite d0/f7 [0,4194304] 0 2024-02-16T22:21:43.875 INFO:tasks.workunit.client.1.smithi123.stdout:3/31: chown d0/f9 11 1 2024-02-16T22:21:43.876 INFO:tasks.workunit.client.1.smithi123.stdout:3/32: write d0/f3 [771024,117665] 0 2024-02-16T22:21:43.877 INFO:tasks.workunit.client.1.smithi123.stdout:2/19: fsync f2 0 2024-02-16T22:21:43.878 INFO:tasks.workunit.client.1.smithi123.stdout:4/46: rename d5/l8 to d5/ld 0 2024-02-16T22:21:43.878 INFO:tasks.workunit.client.1.smithi123.stdout:2/20: fsync f2 0 2024-02-16T22:21:43.879 INFO:tasks.workunit.client.1.smithi123.stdout:4/47: truncate f3 4851243 0 2024-02-16T22:21:43.882 INFO:tasks.workunit.client.1.smithi123.stdout:6/17: mknod d0/c3 0 2024-02-16T22:21:43.888 INFO:tasks.workunit.client.1.smithi123.stdout:2/21: mkdir d4 0 2024-02-16T22:21:43.891 INFO:tasks.workunit.client.1.smithi123.stdout:4/48: rename l1 to d5/le 0 2024-02-16T22:21:43.893 INFO:tasks.workunit.client.1.smithi123.stdout:0/15: symlink d1/l2 0 2024-02-16T22:21:43.893 INFO:tasks.workunit.client.1.smithi123.stdout:0/16: stat d1 0 2024-02-16T22:21:43.893 INFO:tasks.workunit.client.1.smithi123.stdout:0/17: dwrite - no filename 2024-02-16T22:21:43.893 INFO:tasks.workunit.client.1.smithi123.stdout:0/18: chown d1/l2 3256 1 2024-02-16T22:21:43.894 INFO:tasks.workunit.client.1.smithi123.stdout:6/18: rename d0/f1 to d0/f4 0 2024-02-16T22:21:43.894 INFO:tasks.workunit.client.1.smithi123.stdout:2/22: dwrite f2 [0,4194304] 0 2024-02-16T22:21:43.895 INFO:tasks.workunit.client.1.smithi123.stdout:7/22: mknod d3/d5/c7 0 2024-02-16T22:21:43.895 INFO:tasks.workunit.client.1.smithi123.stdout:2/23: read f2 [828601,2021] 0 2024-02-16T22:21:43.897 INFO:tasks.workunit.client.1.smithi123.stdout:3/33: unlink d0/l4 0 2024-02-16T22:21:43.899 INFO:tasks.workunit.client.1.smithi123.stdout:3/34: write d0/f3 [1990923,8351] 0 2024-02-16T22:21:43.900 INFO:tasks.workunit.client.1.smithi123.stdout:4/49: creat d5/ff x:0 0 0 2024-02-16T22:21:43.901 INFO:tasks.workunit.client.1.smithi123.stdout:2/24: dread f2 [0,4194304] 0 2024-02-16T22:21:43.901 INFO:tasks.workunit.client.1.smithi123.stdout:3/35: write d0/f7 [3775857,87931] 0 2024-02-16T22:21:43.902 INFO:tasks.workunit.client.1.smithi123.stdout:0/19: symlink d1/l3 0 2024-02-16T22:21:43.902 INFO:tasks.workunit.client.1.smithi123.stdout:0/20: dread - no filename 2024-02-16T22:21:43.903 INFO:tasks.workunit.client.1.smithi123.stdout:6/19: creat d0/f5 x:0 0 0 2024-02-16T22:21:43.903 INFO:tasks.workunit.client.1.smithi123.stdout:6/20: chown d0 257647521 1 2024-02-16T22:21:43.904 INFO:tasks.workunit.client.1.smithi123.stdout:6/21: write d0/f5 [698317,27140] 0 2024-02-16T22:21:43.906 INFO:tasks.workunit.client.1.smithi123.stdout:7/23: creat d3/f8 x:0 0 0 2024-02-16T22:21:43.906 INFO:tasks.workunit.client.1.smithi123.stdout:4/50: dread d5/f7 [4194304,4194304] 0 2024-02-16T22:21:43.907 INFO:tasks.workunit.client.1.smithi123.stdout:7/24: dread f2 [0,4194304] 0 2024-02-16T22:21:43.908 INFO:tasks.workunit.client.1.smithi123.stdout:7/25: dread f2 [0,4194304] 0 2024-02-16T22:21:43.908 INFO:tasks.workunit.client.1.smithi123.stdout:7/26: chown d3 507164 1 2024-02-16T22:21:43.911 INFO:tasks.workunit.client.1.smithi123.stdout:0/21: mkdir d1/d4 0 2024-02-16T22:21:43.911 INFO:tasks.workunit.client.1.smithi123.stdout:0/22: stat d1/l3 0 2024-02-16T22:21:43.911 INFO:tasks.workunit.client.1.smithi123.stdout:0/23: dwrite - no filename 2024-02-16T22:21:43.911 INFO:tasks.workunit.client.1.smithi123.stdout:0/24: dread - no filename 2024-02-16T22:21:43.911 INFO:tasks.workunit.client.1.smithi123.stdout:0/25: dwrite - no filename 2024-02-16T22:21:43.911 INFO:tasks.workunit.client.1.smithi123.stdout:0/26: fdatasync - no filename 2024-02-16T22:21:43.911 INFO:tasks.workunit.client.1.smithi123.stdout:0/27: chown d1 293811958 1 2024-02-16T22:21:43.912 INFO:tasks.workunit.client.1.smithi123.stdout:0/28: write - no filename 2024-02-16T22:21:43.912 INFO:tasks.workunit.client.1.smithi123.stdout:3/36: symlink d0/la 0 2024-02-16T22:21:43.914 INFO:tasks.workunit.client.1.smithi123.stdout:6/22: mkdir d0/d6 0 2024-02-16T22:21:43.915 INFO:tasks.workunit.client.1.smithi123.stdout:4/51: rename d5/l6 to d5/l10 0 2024-02-16T22:21:43.916 INFO:tasks.workunit.client.1.smithi123.stdout:2/25: mknod d4/c5 0 2024-02-16T22:21:43.918 INFO:tasks.workunit.client.0.smithi114.stdout:1/25: write f1 [111510,52108] 0 2024-02-16T22:21:43.919 INFO:tasks.workunit.client.0.smithi114.stdout:0/13: truncate d1/f2 1693926 0 2024-02-16T22:21:43.921 INFO:tasks.workunit.client.1.smithi123.stdout:3/37: creat d0/fb x:0 0 0 2024-02-16T22:21:43.921 INFO:tasks.workunit.client.1.smithi123.stdout:3/38: truncate d0/f7 5073931 0 2024-02-16T22:21:43.922 INFO:tasks.workunit.client.0.smithi114.stdout:0/14: symlink d1/l5 0 2024-02-16T22:21:43.922 INFO:tasks.workunit.client.0.smithi114.stdout:0/15: chown c0 199689592 1 2024-02-16T22:21:43.923 INFO:tasks.workunit.client.0.smithi114.stdout:8/37: getdents d4 0 2024-02-16T22:21:43.924 INFO:tasks.workunit.client.0.smithi114.stdout:8/38: read - d4/f8 zero size 2024-02-16T22:21:43.924 INFO:tasks.workunit.client.1.smithi123.stdout:4/52: write d5/fc [598134,64377] 0 2024-02-16T22:21:43.926 INFO:tasks.workunit.client.1.smithi123.stdout:2/26: creat d4/f6 x:0 0 0 2024-02-16T22:21:43.927 INFO:tasks.workunit.client.1.smithi123.stdout:2/27: chown l3 170607451 1 2024-02-16T22:21:43.927 INFO:tasks.workunit.client.1.smithi123.stdout:4/53: dread d5/f7 [0,4194304] 0 2024-02-16T22:21:43.930 INFO:tasks.workunit.client.1.smithi123.stdout:0/29: mkdir d1/d4/d5 0 2024-02-16T22:21:43.930 INFO:tasks.workunit.client.1.smithi123.stdout:0/30: read - no filename 2024-02-16T22:21:43.930 INFO:tasks.workunit.client.1.smithi123.stdout:0/31: readlink d1/l2 0 2024-02-16T22:21:43.931 INFO:tasks.workunit.client.0.smithi114.stdout:8/39: rename d4/fb to d4/fe 0 2024-02-16T22:21:43.931 INFO:tasks.workunit.client.0.smithi114.stdout:8/40: chown f0 1530717930 1 2024-02-16T22:21:43.932 INFO:tasks.workunit.client.1.smithi123.stdout:0/32: chown d1/d4 23 1 2024-02-16T22:21:43.932 INFO:tasks.workunit.client.1.smithi123.stdout:0/33: fdatasync - no filename 2024-02-16T22:21:43.932 INFO:tasks.workunit.client.1.smithi123.stdout:0/34: chown d1/l2 1116155 1 2024-02-16T22:21:43.933 INFO:tasks.workunit.client.0.smithi114.stdout:4/28: write f2 [4121027,103869] 0 2024-02-16T22:21:43.938 INFO:tasks.workunit.client.0.smithi114.stdout:4/29: creat f9 x:0 0 0 2024-02-16T22:21:43.939 INFO:tasks.workunit.client.0.smithi114.stdout:3/30: rmdir d4 39 2024-02-16T22:21:43.940 INFO:tasks.workunit.client.0.smithi114.stdout:8/41: dwrite d4/fc [0,4194304] 0 2024-02-16T22:21:43.942 INFO:tasks.workunit.client.1.smithi123.stdout:4/54: dwrite f4 [0,4194304] 0 2024-02-16T22:21:43.944 INFO:tasks.workunit.client.0.smithi114.stdout:4/30: dread f4 [4194304,4194304] 0 2024-02-16T22:21:43.947 INFO:tasks.workunit.client.1.smithi123.stdout:3/39: chown d0/l5 1351 1 2024-02-16T22:21:43.947 INFO:tasks.workunit.client.1.smithi123.stdout:3/40: dread - d0/f9 zero size 2024-02-16T22:21:43.948 INFO:tasks.workunit.client.0.smithi114.stdout:8/42: mkdir d4/df 0 2024-02-16T22:21:43.952 INFO:tasks.workunit.client.0.smithi114.stdout:4/31: mknod ca 0 2024-02-16T22:21:43.953 INFO:tasks.workunit.client.0.smithi114.stdout:3/31: readlink d4/l9 0 2024-02-16T22:21:43.955 INFO:tasks.workunit.client.1.smithi123.stdout:3/41: dwrite d0/fb [0,4194304] 0 2024-02-16T22:21:43.957 INFO:tasks.workunit.client.0.smithi114.stdout:4/32: mknod cb 0 2024-02-16T22:21:43.959 INFO:tasks.workunit.client.0.smithi114.stdout:8/43: creat d4/df/f10 x:0 0 0 2024-02-16T22:21:43.961 INFO:tasks.workunit.client.0.smithi114.stdout:8/44: chown d4/f6 3 1 2024-02-16T22:21:43.961 INFO:tasks.workunit.client.1.smithi123.stdout:0/35: rmdir d1/d4/d5 0 2024-02-16T22:21:43.961 INFO:tasks.workunit.client.1.smithi123.stdout:0/36: fsync - no filename 2024-02-16T22:21:43.962 INFO:tasks.workunit.client.0.smithi114.stdout:3/32: dwrite d4/f7 [0,4194304] 0 2024-02-16T22:21:43.962 INFO:tasks.workunit.client.0.smithi114.stdout:8/45: read - d4/df/f10 zero size 2024-02-16T22:21:43.963 INFO:tasks.workunit.client.1.smithi123.stdout:3/42: dwrite d0/f7 [0,4194304] 0 2024-02-16T22:21:43.964 INFO:tasks.workunit.client.0.smithi114.stdout:4/33: symlink lc 0 2024-02-16T22:21:43.964 INFO:tasks.workunit.client.0.smithi114.stdout:4/34: write f5 [4056995,32048] 0 2024-02-16T22:21:43.966 INFO:tasks.workunit.client.0.smithi114.stdout:4/35: read - f9 zero size 2024-02-16T22:21:43.966 INFO:tasks.workunit.client.0.smithi114.stdout:4/36: rmdir - no directory 2024-02-16T22:21:43.966 INFO:tasks.workunit.client.0.smithi114.stdout:4/37: rmdir - no directory 2024-02-16T22:21:43.971 INFO:tasks.workunit.client.1.smithi123.stdout:8/28: sync 2024-02-16T22:21:43.972 INFO:tasks.workunit.client.1.smithi123.stdout:8/29: chown d0/f7 3395 1 2024-02-16T22:21:43.972 INFO:tasks.workunit.client.0.smithi114.stdout:3/33: rename d4/f7 to d4/fa 0 2024-02-16T22:21:43.976 INFO:tasks.workunit.client.1.smithi123.stdout:3/43: creat d0/fc x:0 0 0 2024-02-16T22:21:43.979 INFO:tasks.workunit.client.1.smithi123.stdout:8/30: dwrite d0/f6 [0,4194304] 0 2024-02-16T22:21:43.980 INFO:tasks.workunit.client.1.smithi123.stdout:3/44: unlink d0/f9 0 2024-02-16T22:21:43.984 INFO:tasks.workunit.client.0.smithi114.stdout:4/38: dwrite f4 [0,4194304] 0 2024-02-16T22:21:43.985 INFO:tasks.workunit.client.0.smithi114.stdout:4/39: dread - f9 zero size 2024-02-16T22:21:43.985 INFO:tasks.workunit.client.0.smithi114.stdout:4/40: rmdir - no directory 2024-02-16T22:21:43.986 INFO:tasks.workunit.client.1.smithi123.stdout:8/31: creat d0/fc x:0 0 0 2024-02-16T22:21:43.990 INFO:tasks.workunit.client.0.smithi114.stdout:4/41: mkdir dd 0 2024-02-16T22:21:43.991 INFO:tasks.workunit.client.1.smithi123.stdout:8/32: creat d0/fd x:0 0 0 2024-02-16T22:21:43.995 INFO:tasks.workunit.client.1.smithi123.stdout:8/33: mknod d0/ce 0 2024-02-16T22:21:43.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.995+0000 7fe9af6d5700 1 -- 172.21.15.114:0/3369193630 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe98c005cc0 con 0x7fe9a80737f0 2024-02-16T22:21:43.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:43.995+0000 7fe99e7fc700 1 -- 172.21.15.114:0/3369193630 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+766 (secure 0 0 0) 0x7fe9a4059b20 con 0x7fe9a80737f0 2024-02-16T22:21:43.998 INFO:tasks.workunit.client.1.smithi123.stdout:8/34: readlink d0/l2 0 2024-02-16T22:21:43.998 INFO:tasks.workunit.client.1.smithi123.stdout:8/35: dread - d0/fc zero size 2024-02-16T22:21:43.998 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:21:43.998 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": { 2024-02-16T22:21:43.998 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-02-16T22:21:43.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:21:43.999 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": { 2024-02-16T22:21:43.999 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-02-16T22:21:43.999 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T22:21:43.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:21:43.999 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": { 2024-02-16T22:21:43.999 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-02-16T22:21:43.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:21:43.999 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-02-16T22:21:43.999 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-02-16T22:21:43.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:21:43.999 INFO:teuthology.orchestra.run.smithi114.stdout: "overall": { 2024-02-16T22:21:44.000 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 13, 2024-02-16T22:21:44.000 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T22:21:44.000 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-02-16T22:21:44.000 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:21:44.001 INFO:tasks.workunit.client.1.smithi123.stdout:8/36: write d0/fd [964411,118443] 0 2024-02-16T22:21:44.001 INFO:tasks.workunit.client.0.smithi114.stdout:4/42: dwrite f2 [4194304,4194304] 0 2024-02-16T22:21:44.001 INFO:tasks.workunit.client.0.smithi114.stdout:4/43: chown f2 18159 1 2024-02-16T22:21:44.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.000+0000 7fe9af6d5700 1 -- 172.21.15.114:0/3369193630 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fe99406ce40 msgr2=0x7fe99406f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:44.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.000+0000 7fe9af6d5700 1 --2- 172.21.15.114:0/3369193630 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fe99406ce40 0x7fe99406f300 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe99800b6f0 tx=0x7fe998005940 comp rx=0 tx=0).stop 2024-02-16T22:21:44.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.000+0000 7fe9af6d5700 1 -- 172.21.15.114:0/3369193630 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9a80737f0 msgr2=0x7fe9a81985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:44.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.000+0000 7fe9af6d5700 1 --2- 172.21.15.114:0/3369193630 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9a80737f0 0x7fe9a81985b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400e8f0 tx=0x7fe9a400ec00 comp rx=0 tx=0).stop 2024-02-16T22:21:44.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.000+0000 7fe9af6d5700 1 -- 172.21.15.114:0/3369193630 shutdown_connections 2024-02-16T22:21:44.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.000+0000 7fe9af6d5700 1 --2- 172.21.15.114:0/3369193630 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fe99406ce40 0x7fe99406f300 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe99800b6f0 tx=0x7fe998005940 comp rx=0 tx=0).stop 2024-02-16T22:21:44.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.000+0000 7fe9af6d5700 1 --2- 172.21.15.114:0/3369193630 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9a80737f0 0x7fe9a81985b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:44.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.000+0000 7fe9af6d5700 1 --2- 172.21.15.114:0/3369193630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9a8074dc0 0x7fe9a8198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:44.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.000+0000 7fe9af6d5700 1 -- 172.21.15.114:0/3369193630 >> 172.21.15.114:0/3369193630 conn(0x7fe9a80fc110 msgr2=0x7fe9a8106990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:21:44.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.001+0000 7fe9af6d5700 1 -- 172.21.15.114:0/3369193630 shutdown_connections 2024-02-16T22:21:44.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.002+0000 7fe9af6d5700 1 -- 172.21.15.114:0/3369193630 wait complete. 2024-02-16T22:21:44.004 INFO:tasks.workunit.client.1.smithi123.stdout:1/7: sync 2024-02-16T22:21:44.004 INFO:tasks.workunit.client.1.smithi123.stdout:1/8: rmdir - no directory 2024-02-16T22:21:44.004 INFO:tasks.workunit.client.1.smithi123.stdout:1/9: fdatasync f0 0 2024-02-16T22:21:44.006 INFO:tasks.workunit.client.1.smithi123.stdout:8/37: dwrite d0/f7 [0,4194304] 0 2024-02-16T22:21:44.006 INFO:tasks.workunit.client.1.smithi123.stdout:1/10: chown f0 24 1 2024-02-16T22:21:44.019 INFO:tasks.workunit.client.1.smithi123.stdout:9/54: getdents d4 0 2024-02-16T22:21:44.019 INFO:tasks.workunit.client.1.smithi123.stdout:9/55: read d4/f10 [3222559,31518] 0 2024-02-16T22:21:44.021 INFO:tasks.workunit.client.1.smithi123.stdout:9/56: symlink d4/l15 0 2024-02-16T22:21:44.022 INFO:tasks.workunit.client.1.smithi123.stdout:9/57: write f2 [2520263,121989] 0 2024-02-16T22:21:44.022 INFO:tasks.workunit.client.1.smithi123.stdout:9/58: dread - d4/fe zero size 2024-02-16T22:21:44.023 INFO:tasks.workunit.client.1.smithi123.stdout:9/59: write d4/fa [19375,128636] 0 2024-02-16T22:21:44.024 INFO:tasks.workunit.client.1.smithi123.stdout:9/60: chown l1 220265 1 2024-02-16T22:21:44.024 INFO:tasks.workunit.client.1.smithi123.stdout:9/61: write d4/f6 [704713,112295] 0 2024-02-16T22:21:44.025 INFO:tasks.workunit.client.1.smithi123.stdout:9/62: write d4/fa [1123644,40759] 0 2024-02-16T22:21:44.026 INFO:tasks.workunit.client.1.smithi123.stdout:5/8: getdents . 0 2024-02-16T22:21:44.029 INFO:tasks.workunit.client.1.smithi123.stdout:9/63: unlink d4/f5 0 2024-02-16T22:21:44.030 INFO:tasks.workunit.client.1.smithi123.stdout:5/9: unlink f1 0 2024-02-16T22:21:44.030 INFO:tasks.workunit.client.1.smithi123.stdout:5/10: readlink l0 0 2024-02-16T22:21:44.034 INFO:tasks.workunit.client.1.smithi123.stdout:9/64: rename d4/fb to d4/f16 0 2024-02-16T22:21:44.037 INFO:tasks.workunit.client.1.smithi123.stdout:5/11: symlink l2 0 2024-02-16T22:21:44.037 INFO:tasks.workunit.client.1.smithi123.stdout:5/12: write - no filename 2024-02-16T22:21:44.040 INFO:tasks.workunit.client.1.smithi123.stdout:9/65: dwrite d4/fe [0,4194304] 0 2024-02-16T22:21:44.044 INFO:tasks.workunit.client.1.smithi123.stdout:9/66: mkdir d4/d17 0 2024-02-16T22:21:44.045 INFO:tasks.workunit.client.1.smithi123.stdout:7/27: getdents d3/d5 0 2024-02-16T22:21:44.046 INFO:tasks.workunit.client.1.smithi123.stdout:5/13: link l0 l3 0 2024-02-16T22:21:44.052 INFO:tasks.workunit.client.1.smithi123.stdout:9/67: dwrite d4/fa [0,4194304] 0 2024-02-16T22:21:44.052 INFO:tasks.workunit.client.1.smithi123.stdout:5/14: symlink l4 0 2024-02-16T22:21:44.052 INFO:tasks.workunit.client.1.smithi123.stdout:5/15: truncate - no filename 2024-02-16T22:21:44.052 INFO:tasks.workunit.client.1.smithi123.stdout:5/16: dread - no filename 2024-02-16T22:21:44.056 INFO:tasks.workunit.client.1.smithi123.stdout:7/28: link d3/f8 d3/d5/f9 0 2024-02-16T22:21:44.058 INFO:tasks.workunit.client.1.smithi123.stdout:7/29: dread f2 [0,4194304] 0 2024-02-16T22:21:44.059 INFO:tasks.workunit.client.1.smithi123.stdout:9/68: dwrite d4/f6 [0,4194304] 0 2024-02-16T22:21:44.060 INFO:tasks.workunit.client.1.smithi123.stdout:6/23: getdents d0 0 2024-02-16T22:21:44.061 INFO:tasks.workunit.client.1.smithi123.stdout:6/24: read d0/f2 [122732,22817] 0 2024-02-16T22:21:44.063 INFO:tasks.workunit.client.1.smithi123.stdout:7/30: mknod d3/d5/ca 0 2024-02-16T22:21:44.064 INFO:tasks.workunit.client.0.smithi114.stdout:0/16: dread d1/f2 [0,4194304] 0 2024-02-16T22:21:44.065 INFO:tasks.workunit.client.1.smithi123.stdout:7/31: dread f2 [0,4194304] 0 2024-02-16T22:21:44.067 INFO:tasks.workunit.client.1.smithi123.stdout:6/25: unlink d0/f4 0 2024-02-16T22:21:44.067 INFO:tasks.workunit.client.1.smithi123.stdout:2/28: fsync d4/f6 0 2024-02-16T22:21:44.069 INFO:tasks.workunit.client.0.smithi114.stdout:1/26: truncate d4/f5 1411181 0 2024-02-16T22:21:44.069 INFO:tasks.workunit.client.1.smithi123.stdout:9/69: mkdir d4/d17/d18 0 2024-02-16T22:21:44.070 INFO:tasks.workunit.client.1.smithi123.stdout:9/70: dread - d4/f12 zero size 2024-02-16T22:21:44.070 INFO:tasks.workunit.client.0.smithi114.stdout:0/17: dread d1/f2 [0,4194304] 0 2024-02-16T22:21:44.070 INFO:tasks.workunit.client.0.smithi114.stdout:1/27: truncate d4/fa 527418 0 2024-02-16T22:21:44.071 INFO:tasks.workunit.client.0.smithi114.stdout:0/18: stat d1/f2 0 2024-02-16T22:21:44.071 INFO:tasks.workunit.client.0.smithi114.stdout:1/28: chown d4/l8 4427651 1 2024-02-16T22:21:44.072 INFO:tasks.workunit.client.1.smithi123.stdout:7/32: mknod d3/cb 0 2024-02-16T22:21:44.072 INFO:tasks.workunit.client.0.smithi114.stdout:0/19: chown d1/l5 65 1 2024-02-16T22:21:44.072 INFO:tasks.workunit.client.0.smithi114.stdout:1/29: chown d4 9471379 1 2024-02-16T22:21:44.073 INFO:tasks.workunit.client.1.smithi123.stdout:2/29: creat d4/f7 x:0 0 0 2024-02-16T22:21:44.074 INFO:tasks.workunit.client.1.smithi123.stdout:2/30: dread - d4/f6 zero size 2024-02-16T22:21:44.074 INFO:tasks.workunit.client.1.smithi123.stdout:2/31: chown d4/f6 299 1 2024-02-16T22:21:44.075 INFO:tasks.workunit.client.1.smithi123.stdout:9/71: symlink d4/l19 0 2024-02-16T22:21:44.076 INFO:tasks.workunit.client.0.smithi114.stdout:8/46: getdents d4 0 2024-02-16T22:21:44.077 INFO:tasks.workunit.client.0.smithi114.stdout:0/20: symlink d1/l6 0 2024-02-16T22:21:44.079 INFO:tasks.workunit.client.1.smithi123.stdout:6/26: mknod d0/d6/c7 0 2024-02-16T22:21:44.080 INFO:tasks.workunit.client.1.smithi123.stdout:6/27: chown d0 61875961 1 2024-02-16T22:21:44.080 INFO:tasks.workunit.client.0.smithi114.stdout:8/47: mkdir d4/d11 0 2024-02-16T22:21:44.081 INFO:tasks.workunit.client.0.smithi114.stdout:8/48: readlink d4/ld 0 2024-02-16T22:21:44.081 INFO:tasks.workunit.client.0.smithi114.stdout:8/49: dread - d4/f8 zero size 2024-02-16T22:21:44.081 INFO:tasks.workunit.client.1.smithi123.stdout:0/37: getdents d1/d4 0 2024-02-16T22:21:44.083 INFO:tasks.workunit.client.1.smithi123.stdout:2/32: mkdir d4/d8 0 2024-02-16T22:21:44.083 INFO:tasks.workunit.client.1.smithi123.stdout:6/28: dread d0/f5 [0,4194304] 0 2024-02-16T22:21:44.085 INFO:tasks.workunit.client.0.smithi114.stdout:0/21: rename d1/f2 to d1/f7 0 2024-02-16T22:21:44.085 INFO:tasks.workunit.client.1.smithi123.stdout:4/55: truncate f4 3160204 0 2024-02-16T22:21:44.088 INFO:tasks.workunit.client.0.smithi114.stdout:8/50: dwrite d4/f6 [0,4194304] 0 2024-02-16T22:21:44.090 INFO:tasks.workunit.client.0.smithi114.stdout:0/22: creat d1/f8 x:0 0 0 2024-02-16T22:21:44.092 INFO:tasks.workunit.client.0.smithi114.stdout:8/51: symlink d4/df/l12 0 2024-02-16T22:21:44.094 INFO:tasks.workunit.client.1.smithi123.stdout:6/29: creat d0/d6/f8 x:0 0 0 2024-02-16T22:21:44.094 INFO:tasks.workunit.client.1.smithi123.stdout:6/30: read - d0/d6/f8 zero size 2024-02-16T22:21:44.095 INFO:tasks.workunit.client.0.smithi114.stdout:0/23: unlink d1/l5 0 2024-02-16T22:21:44.096 INFO:tasks.workunit.client.1.smithi123.stdout:6/31: truncate d0/d6/f8 569211 0 2024-02-16T22:21:44.096 INFO:tasks.workunit.client.0.smithi114.stdout:0/24: dread - d1/f8 zero size 2024-02-16T22:21:44.099 INFO:tasks.workunit.client.1.smithi123.stdout:4/56: symlink d5/l11 0 2024-02-16T22:21:44.101 INFO:tasks.workunit.client.0.smithi114.stdout:8/52: symlink d4/l13 0 2024-02-16T22:21:44.103 INFO:tasks.workunit.client.1.smithi123.stdout:9/72: link d4/f12 d4/d17/f1a 0 2024-02-16T22:21:44.104 INFO:tasks.workunit.client.0.smithi114.stdout:8/53: read d4/fa [193109,23783] 0 2024-02-16T22:21:44.105 INFO:tasks.workunit.client.0.smithi114.stdout:3/34: dwrite f0 [0,4194304] 0 2024-02-16T22:21:44.106 INFO:tasks.workunit.client.1.smithi123.stdout:3/45: truncate d0/f3 1407566 0 2024-02-16T22:21:44.107 INFO:tasks.workunit.client.0.smithi114.stdout:0/25: mknod d1/c9 0 2024-02-16T22:21:44.107 INFO:tasks.workunit.client.0.smithi114.stdout:0/26: chown d1/f8 0 1 2024-02-16T22:21:44.109 INFO:tasks.workunit.client.0.smithi114.stdout:3/35: dread f0 [0,4194304] 0 2024-02-16T22:21:44.112 INFO:tasks.workunit.client.0.smithi114.stdout:4/44: truncate f5 163731 0 2024-02-16T22:21:44.112 INFO:tasks.workunit.client.1.smithi123.stdout:2/33: creat d4/d8/f9 x:0 0 0 2024-02-16T22:21:44.113 INFO:tasks.workunit.client.1.smithi123.stdout:8/38: truncate d0/fd 910803 0 2024-02-16T22:21:44.114 INFO:tasks.workunit.client.0.smithi114.stdout:8/54: rename d4/l13 to d4/d11/l14 0 2024-02-16T22:21:44.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.144+0000 7fb9bb0ee700 1 -- 172.21.15.114:0/3655624909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9b4100cb0 msgr2=0x7fb9b4101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:44.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.144+0000 7fb9bb0ee700 1 --2- 172.21.15.114:0/3655624909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9b4100cb0 0x7fb9b4101110 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0009910 tx=0x7fb9a0009c20 comp rx=0 tx=0).stop 2024-02-16T22:21:44.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.144+0000 7fb9bb0ee700 1 -- 172.21.15.114:0/3655624909 shutdown_connections 2024-02-16T22:21:44.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.144+0000 7fb9bb0ee700 1 --2- 172.21.15.114:0/3655624909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9b4100cb0 0x7fb9b4101110 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:44.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.144+0000 7fb9bb0ee700 1 --2- 172.21.15.114:0/3655624909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9b40ffab0 0x7fb9b40ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:44.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.144+0000 7fb9bb0ee700 1 -- 172.21.15.114:0/3655624909 >> 172.21.15.114:0/3655624909 conn(0x7fb9b40fb030 msgr2=0x7fb9b40fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:21:44.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.144+0000 7fb9bb0ee700 1 -- 172.21.15.114:0/3655624909 shutdown_connections 2024-02-16T22:21:44.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.144+0000 7fb9bb0ee700 1 -- 172.21.15.114:0/3655624909 wait complete. 2024-02-16T22:21:44.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.144+0000 7fb9bb0ee700 1 Processor -- start 2024-02-16T22:21:44.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.144+0000 7fb9bb0ee700 1 -- start start 2024-02-16T22:21:44.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.144+0000 7fb9bb0ee700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9b40ffab0 0x7fb9b4196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:44.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.144+0000 7fb9bb0ee700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9b4100cb0 0x7fb9b4196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:44.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.144+0000 7fb9bb0ee700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b4196ea0 con 0x7fb9b40ffab0 2024-02-16T22:21:44.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.144+0000 7fb9bb0ee700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b4196fe0 con 0x7fb9b4100cb0 2024-02-16T22:21:44.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.144+0000 7fb9b8e8a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9b40ffab0 0x7fb9b4196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:44.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.144+0000 7fb9b8e8a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9b40ffab0 0x7fb9b4196340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39434/0 (socket says 172.21.15.114:39434) 2024-02-16T22:21:44.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.145+0000 7fb9b8e8a700 1 -- 172.21.15.114:0/2505496517 learned_addr learned my addr 172.21.15.114:0/2505496517 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:21:44.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.145+0000 7fb9b8e8a700 1 -- 172.21.15.114:0/2505496517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9b4100cb0 msgr2=0x7fb9b4196880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:44.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.145+0000 7fb9b3fff700 1 --2- 172.21.15.114:0/2505496517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9b4100cb0 0x7fb9b4196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:44.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.145+0000 7fb9b8e8a700 1 --2- 172.21.15.114:0/2505496517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9b4100cb0 0x7fb9b4196880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:44.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.145+0000 7fb9b8e8a700 1 -- 172.21.15.114:0/2505496517 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a00095f0 con 0x7fb9b40ffab0 2024-02-16T22:21:44.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.145+0000 7fb9b8e8a700 1 --2- 172.21.15.114:0/2505496517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9b40ffab0 0x7fb9b4196340 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800d900 tx=0x7fb9a800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:21:44.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.145+0000 7fb9b1ffb700 1 -- 172.21.15.114:0/2505496517 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9a80041f0 con 0x7fb9b40ffab0 2024-02-16T22:21:44.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.145+0000 7fb9b1ffb700 1 -- 172.21.15.114:0/2505496517 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb9a8004350 con 0x7fb9b40ffab0 2024-02-16T22:21:44.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.145+0000 7fb9bb0ee700 1 -- 172.21.15.114:0/2505496517 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b419ba90 con 0x7fb9b40ffab0 2024-02-16T22:21:44.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.145+0000 7fb9b1ffb700 1 -- 172.21.15.114:0/2505496517 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9a8003da0 con 0x7fb9b40ffab0 2024-02-16T22:21:44.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.145+0000 7fb9bb0ee700 1 -- 172.21.15.114:0/2505496517 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b419bfe0 con 0x7fb9b40ffab0 2024-02-16T22:21:44.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.146+0000 7fb9bb0ee700 1 -- 172.21.15.114:0/2505496517 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9b4108610 con 0x7fb9b40ffab0 2024-02-16T22:21:44.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.147+0000 7fb9b1ffb700 1 -- 172.21.15.114:0/2505496517 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) v1 ==== 90933+0+0 (secure 0 0 0) 0x7fb9a8009730 con 0x7fb9b40ffab0 2024-02-16T22:21:44.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.147+0000 7fb9b1ffb700 1 --2- 172.21.15.114:0/2505496517 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fb9a406ce40 0x7fb9a406f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:44.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.147+0000 7fb9b1ffb700 1 -- 172.21.15.114:0/2505496517 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5726+0+0 (secure 0 0 0) 0x7fb9a808c770 con 0x7fb9b40ffab0 2024-02-16T22:21:44.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.147+0000 7fb9b3fff700 1 --2- 172.21.15.114:0/2505496517 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fb9a406ce40 0x7fb9a406f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:44.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.148+0000 7fb9b3fff700 1 --2- 172.21.15.114:0/2505496517 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fb9a406ce40 0x7fb9a406f300 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb9a000b5c0 tx=0x7fb9a0005960 comp rx=0 tx=0).ready entity=mgr.24441 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:21:44.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.152+0000 7fb9b1ffb700 1 -- 172.21.15.114:0/2505496517 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182869 (secure 0 0 0) 0x7fb9a8059f10 con 0x7fb9b40ffab0 2024-02-16T22:21:44.154 INFO:tasks.workunit.client.1.smithi123.stdout:5/17: sync 2024-02-16T22:21:44.154 INFO:tasks.workunit.client.1.smithi123.stdout:1/11: sync 2024-02-16T22:21:44.155 INFO:tasks.workunit.client.1.smithi123.stdout:1/12: dread - f0 zero size 2024-02-16T22:21:44.155 INFO:tasks.workunit.client.1.smithi123.stdout:1/13: dread - f0 zero size 2024-02-16T22:21:44.155 INFO:tasks.workunit.client.1.smithi123.stdout:1/14: rmdir - no directory 2024-02-16T22:21:44.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:43 smithi114 ceph-mon[29235]: from='client.24499 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:21:44.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:43 smithi114 ceph-mon[29235]: from='client.14744 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:21:44.203 INFO:tasks.workunit.client.0.smithi114.stdout:0/27: creat d1/fa x:0 0 0 2024-02-16T22:21:44.203 INFO:tasks.workunit.client.0.smithi114.stdout:0/28: truncate d1/f8 981510 0 2024-02-16T22:21:44.204 INFO:tasks.workunit.client.1.smithi123.stdout:4/57: creat d5/f12 x:0 0 0 2024-02-16T22:21:44.206 INFO:tasks.workunit.client.0.smithi114.stdout:3/36: unlink d4/l9 0 2024-02-16T22:21:44.207 INFO:tasks.workunit.client.1.smithi123.stdout:9/73: fdatasync d4/f16 0 2024-02-16T22:21:44.212 INFO:tasks.workunit.client.1.smithi123.stdout:8/39: fdatasync d0/fd 0 2024-02-16T22:21:44.216 INFO:tasks.workunit.client.1.smithi123.stdout:1/15: symlink l1 0 2024-02-16T22:21:44.216 INFO:tasks.workunit.client.1.smithi123.stdout:1/16: stat l1 0 2024-02-16T22:21:44.216 INFO:tasks.workunit.client.1.smithi123.stdout:7/33: truncate d3/d5/f9 460360 0 2024-02-16T22:21:44.216 INFO:tasks.workunit.client.0.smithi114.stdout:4/45: link c7 dd/ce 0 2024-02-16T22:21:44.217 INFO:tasks.workunit.client.0.smithi114.stdout:4/46: write f4 [5133248,9642] 0 2024-02-16T22:21:44.220 INFO:tasks.workunit.client.1.smithi123.stdout:0/38: getdents d1 0 2024-02-16T22:21:44.220 INFO:tasks.workunit.client.1.smithi123.stdout:0/39: chown d1/l2 281638 1 2024-02-16T22:21:44.221 INFO:tasks.workunit.client.1.smithi123.stdout:0/40: write - no filename 2024-02-16T22:21:44.221 INFO:tasks.workunit.client.1.smithi123.stdout:0/41: dread - no filename 2024-02-16T22:21:44.223 INFO:tasks.workunit.client.0.smithi114.stdout:4/47: dwrite f2 [0,4194304] 0 2024-02-16T22:21:44.224 INFO:tasks.workunit.client.0.smithi114.stdout:1/30: dread f1 [0,4194304] 0 2024-02-16T22:21:44.225 INFO:tasks.workunit.client.0.smithi114.stdout:1/31: write d4/fa [348620,39116] 0 2024-02-16T22:21:44.229 INFO:tasks.workunit.client.1.smithi123.stdout:1/17: mknod c2 0 2024-02-16T22:21:44.233 INFO:tasks.workunit.client.1.smithi123.stdout:7/34: symlink d3/d5/lc 0 2024-02-16T22:21:44.236 INFO:tasks.workunit.client.1.smithi123.stdout:0/42: creat d1/f6 x:0 0 0 2024-02-16T22:21:44.236 INFO:tasks.workunit.client.1.smithi123.stdout:6/32: fsync d0/d6/f8 0 2024-02-16T22:21:44.243 INFO:tasks.workunit.client.1.smithi123.stdout:1/18: mkdir d3 0 2024-02-16T22:21:44.247 INFO:tasks.workunit.client.1.smithi123.stdout:0/43: creat d1/f7 x:0 0 0 2024-02-16T22:21:44.247 INFO:tasks.workunit.client.1.smithi123.stdout:2/34: link l1 d4/la 0 2024-02-16T22:21:44.251 INFO:tasks.workunit.client.1.smithi123.stdout:5/18: getdents . 0 2024-02-16T22:21:44.256 INFO:tasks.workunit.client.1.smithi123.stdout:0/44: dwrite d1/f6 [0,4194304] 0 2024-02-16T22:21:44.256 INFO:tasks.workunit.client.1.smithi123.stdout:2/35: dwrite d4/d8/f9 [0,4194304] 0 2024-02-16T22:21:44.257 INFO:tasks.workunit.client.1.smithi123.stdout:1/19: symlink d3/l4 0 2024-02-16T22:21:44.258 INFO:tasks.workunit.client.1.smithi123.stdout:4/58: getdents d5 0 2024-02-16T22:21:44.259 INFO:tasks.workunit.client.1.smithi123.stdout:4/59: write d5/ff [325430,11305] 0 2024-02-16T22:21:44.262 INFO:tasks.workunit.client.1.smithi123.stdout:3/46: creat d0/fd x:0 0 0 2024-02-16T22:21:44.264 INFO:tasks.workunit.client.1.smithi123.stdout:5/19: rename l3 to l5 0 2024-02-16T22:21:44.264 INFO:tasks.workunit.client.1.smithi123.stdout:5/20: dwrite - no filename 2024-02-16T22:21:44.266 INFO:tasks.workunit.client.1.smithi123.stdout:0/45: mkdir d1/d8 0 2024-02-16T22:21:44.267 INFO:tasks.workunit.client.1.smithi123.stdout:2/36: symlink d4/d8/lb 0 2024-02-16T22:21:44.270 INFO:tasks.workunit.client.0.smithi114.stdout:8/55: rename d4/df to d4/d11/d15 0 2024-02-16T22:21:44.271 INFO:tasks.workunit.client.0.smithi114.stdout:0/29: rmdir d1 39 2024-02-16T22:21:44.272 INFO:tasks.workunit.client.1.smithi123.stdout:1/20: unlink f0 0 2024-02-16T22:21:44.272 INFO:tasks.workunit.client.1.smithi123.stdout:9/74: write d4/f10 [4062736,12609] 0 2024-02-16T22:21:44.273 INFO:tasks.workunit.client.1.smithi123.stdout:9/75: chown d4/f12 1422 1 2024-02-16T22:21:44.273 INFO:tasks.workunit.client.1.smithi123.stdout:9/76: fsync f2 0 2024-02-16T22:21:44.274 INFO:tasks.workunit.client.1.smithi123.stdout:9/77: dread - d4/f12 zero size 2024-02-16T22:21:44.274 INFO:tasks.workunit.client.1.smithi123.stdout:8/40: read d0/fd [131561,44482] 0 2024-02-16T22:21:44.277 INFO:tasks.workunit.client.1.smithi123.stdout:5/21: mkdir d6 0 2024-02-16T22:21:44.278 INFO:tasks.workunit.client.1.smithi123.stdout:5/22: fsync - no filename 2024-02-16T22:21:44.278 INFO:tasks.workunit.client.1.smithi123.stdout:8/41: dread d0/f3 [0,4194304] 0 2024-02-16T22:21:44.280 INFO:tasks.workunit.client.1.smithi123.stdout:0/46: symlink d1/l9 0 2024-02-16T22:21:44.281 INFO:tasks.workunit.client.1.smithi123.stdout:0/47: write d1/f7 [211164,114264] 0 2024-02-16T22:21:44.287 INFO:tasks.workunit.client.1.smithi123.stdout:2/37: rename d4/f7 to d4/fc 0 2024-02-16T22:21:44.288 INFO:tasks.workunit.client.1.smithi123.stdout:0/48: dwrite d1/f7 [0,4194304] 0 2024-02-16T22:21:44.289 INFO:tasks.workunit.client.1.smithi123.stdout:0/49: chown d1/l2 298436 1 2024-02-16T22:21:44.293 INFO:tasks.workunit.client.0.smithi114.stdout:8/56: unlink d4/d11/l14 0 2024-02-16T22:21:44.293 INFO:tasks.workunit.client.0.smithi114.stdout:8/57: dread - d4/f8 zero size 2024-02-16T22:21:44.294 INFO:tasks.workunit.client.1.smithi123.stdout:0/50: dread d1/f7 [0,4194304] 0 2024-02-16T22:21:44.294 INFO:tasks.workunit.client.1.smithi123.stdout:0/51: readlink d1/l3 0 2024-02-16T22:21:44.295 INFO:tasks.workunit.client.1.smithi123.stdout:0/52: write d1/f6 [1271987,81459] 0 2024-02-16T22:21:44.297 INFO:tasks.workunit.client.0.smithi114.stdout:4/48: write f5 [408498,87531] 0 2024-02-16T22:21:44.298 INFO:tasks.workunit.client.1.smithi123.stdout:1/21: creat d3/f5 x:0 0 0 2024-02-16T22:21:44.298 INFO:tasks.workunit.client.1.smithi123.stdout:1/22: dread - d3/f5 zero size 2024-02-16T22:21:44.299 INFO:tasks.workunit.client.1.smithi123.stdout:1/23: dread - d3/f5 zero size 2024-02-16T22:21:44.299 INFO:tasks.workunit.client.0.smithi114.stdout:4/49: dread f4 [4194304,4194304] 0 2024-02-16T22:21:44.301 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:43 smithi123 ceph-mon[37161]: from='client.24499 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:21:44.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:43 smithi123 ceph-mon[37161]: from='client.14744 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:21:44.306 INFO:tasks.workunit.client.1.smithi123.stdout:1/24: dwrite d3/f5 [0,4194304] 0 2024-02-16T22:21:44.308 INFO:tasks.workunit.client.1.smithi123.stdout:0/53: fdatasync d1/f7 0 2024-02-16T22:21:44.308 INFO:tasks.workunit.client.1.smithi123.stdout:7/35: write d3/f8 [199563,49043] 0 2024-02-16T22:21:44.309 INFO:tasks.workunit.client.1.smithi123.stdout:0/54: write d1/f6 [3622858,22078] 0 2024-02-16T22:21:44.311 INFO:tasks.workunit.client.1.smithi123.stdout:8/42: mkdir d0/df 0 2024-02-16T22:21:44.312 INFO:tasks.workunit.client.0.smithi114.stdout:1/32: dwrite f1 [0,4194304] 0 2024-02-16T22:21:44.313 INFO:tasks.workunit.client.0.smithi114.stdout:4/50: creat dd/ff x:0 0 0 2024-02-16T22:21:44.328 INFO:tasks.workunit.client.0.smithi114.stdout:1/33: creat d4/fb x:0 0 0 2024-02-16T22:21:44.328 INFO:tasks.workunit.client.0.smithi114.stdout:4/51: unlink f5 0 2024-02-16T22:21:44.329 INFO:tasks.workunit.client.0.smithi114.stdout:0/30: write d1/f7 [1303265,92290] 0 2024-02-16T22:21:44.330 INFO:tasks.workunit.client.1.smithi123.stdout:4/60: link d5/la d5/l13 0 2024-02-16T22:21:44.330 INFO:tasks.workunit.client.1.smithi123.stdout:4/61: stat d5 0 2024-02-16T22:21:44.331 INFO:tasks.workunit.client.1.smithi123.stdout:4/62: readlink d5/le 0 2024-02-16T22:21:44.336 INFO:tasks.workunit.client.0.smithi114.stdout:8/58: dwrite f0 [0,4194304] 0 2024-02-16T22:21:44.336 INFO:tasks.workunit.client.1.smithi123.stdout:4/63: dread f4 [0,4194304] 0 2024-02-16T22:21:44.337 INFO:tasks.workunit.client.1.smithi123.stdout:9/78: dwrite d4/f6 [4194304,4194304] 0 2024-02-16T22:21:44.337 INFO:tasks.workunit.client.1.smithi123.stdout:4/64: write f3 [1639121,56829] 0 2024-02-16T22:21:44.339 INFO:tasks.workunit.client.0.smithi114.stdout:0/31: dwrite d1/fa [0,4194304] 0 2024-02-16T22:21:44.340 INFO:tasks.workunit.client.0.smithi114.stdout:0/32: write d1/fa [277580,2023] 0 2024-02-16T22:21:44.341 INFO:tasks.workunit.client.0.smithi114.stdout:0/33: readlink d1/l6 0 2024-02-16T22:21:44.342 INFO:tasks.workunit.client.0.smithi114.stdout:0/34: write d1/f8 [1753882,47767] 0 2024-02-16T22:21:44.343 INFO:tasks.workunit.client.0.smithi114.stdout:0/35: truncate d1/f7 2618285 0 2024-02-16T22:21:44.350 INFO:tasks.workunit.client.1.smithi123.stdout:7/36: creat d3/fd x:0 0 0 2024-02-16T22:21:44.352 INFO:tasks.workunit.client.1.smithi123.stdout:1/25: write d3/f5 [5095903,39316] 0 2024-02-16T22:21:44.355 INFO:tasks.workunit.client.1.smithi123.stdout:6/33: mkdir d0/d9 0 2024-02-16T22:21:44.359 INFO:tasks.workunit.client.1.smithi123.stdout:1/26: dwrite d3/f5 [0,4194304] 0 2024-02-16T22:21:44.363 INFO:tasks.workunit.client.1.smithi123.stdout:9/79: creat d4/f1b x:0 0 0 2024-02-16T22:21:44.363 INFO:tasks.workunit.client.1.smithi123.stdout:4/65: rmdir d5 39 2024-02-16T22:21:44.365 INFO:tasks.workunit.client.1.smithi123.stdout:7/37: creat d3/fe x:0 0 0 2024-02-16T22:21:44.366 INFO:tasks.workunit.client.1.smithi123.stdout:0/55: getdents d1/d8 0 2024-02-16T22:21:44.368 INFO:tasks.workunit.client.1.smithi123.stdout:8/43: symlink d0/df/l10 0 2024-02-16T22:21:44.368 INFO:tasks.workunit.client.1.smithi123.stdout:9/80: fdatasync d4/f6 0 2024-02-16T22:21:44.371 INFO:tasks.workunit.client.1.smithi123.stdout:5/23: sync 2024-02-16T22:21:44.371 INFO:tasks.workunit.client.1.smithi123.stdout:5/24: dwrite - no filename 2024-02-16T22:21:44.371 INFO:tasks.workunit.client.1.smithi123.stdout:5/25: dwrite - no filename 2024-02-16T22:21:44.371 INFO:tasks.workunit.client.1.smithi123.stdout:1/27: rmdir d3 39 2024-02-16T22:21:44.373 INFO:tasks.workunit.client.1.smithi123.stdout:3/47: creat d0/fe x:0 0 0 2024-02-16T22:21:44.380 INFO:tasks.workunit.client.1.smithi123.stdout:2/38: truncate d4/d8/f9 333412 0 2024-02-16T22:21:44.380 INFO:tasks.workunit.client.1.smithi123.stdout:8/44: symlink d0/df/l11 0 2024-02-16T22:21:44.381 INFO:tasks.workunit.client.1.smithi123.stdout:8/45: dread - d0/f4 zero size 2024-02-16T22:21:44.382 INFO:tasks.workunit.client.1.smithi123.stdout:9/81: mknod d4/c1c 0 2024-02-16T22:21:44.387 INFO:tasks.workunit.client.1.smithi123.stdout:4/66: unlink d5/f7 0 2024-02-16T22:21:44.389 INFO:tasks.workunit.client.1.smithi123.stdout:1/28: dwrite d3/f5 [4194304,4194304] 0 2024-02-16T22:21:44.390 INFO:tasks.workunit.client.1.smithi123.stdout:1/29: write d3/f5 [1702327,47781] 0 2024-02-16T22:21:44.392 INFO:tasks.workunit.client.1.smithi123.stdout:0/56: creat d1/d8/fa x:0 0 0 2024-02-16T22:21:44.392 INFO:tasks.workunit.client.1.smithi123.stdout:1/30: write d3/f5 [7684745,77258] 0 2024-02-16T22:21:44.394 INFO:tasks.workunit.client.1.smithi123.stdout:2/39: chown l1 709 1 2024-02-16T22:21:44.394 INFO:tasks.workunit.client.1.smithi123.stdout:2/40: stat l1 0 2024-02-16T22:21:44.395 INFO:tasks.workunit.client.1.smithi123.stdout:8/46: mkdir d0/d12 0 2024-02-16T22:21:44.397 INFO:tasks.workunit.client.1.smithi123.stdout:0/57: dread d1/f7 [0,4194304] 0 2024-02-16T22:21:44.397 INFO:tasks.workunit.client.1.smithi123.stdout:8/47: write d0/f6 [3443560,112291] 0 2024-02-16T22:21:44.398 INFO:tasks.workunit.client.1.smithi123.stdout:8/48: chown d0/f6 714672 1 2024-02-16T22:21:44.398 INFO:tasks.workunit.client.1.smithi123.stdout:9/82: creat d4/d17/f1d x:0 0 0 2024-02-16T22:21:44.399 INFO:tasks.workunit.client.1.smithi123.stdout:5/26: mknod d6/c7 0 2024-02-16T22:21:44.399 INFO:tasks.workunit.client.1.smithi123.stdout:5/27: dread - no filename 2024-02-16T22:21:44.404 INFO:tasks.workunit.client.1.smithi123.stdout:9/83: dwrite d4/f16 [0,4194304] 0 2024-02-16T22:21:44.404 INFO:tasks.workunit.client.1.smithi123.stdout:4/67: creat d5/f14 x:0 0 0 2024-02-16T22:21:44.404 INFO:tasks.workunit.client.1.smithi123.stdout:9/84: readlink d4/l13 0 2024-02-16T22:21:44.406 INFO:tasks.workunit.client.1.smithi123.stdout:6/34: creat d0/d6/fa x:0 0 0 2024-02-16T22:21:44.412 INFO:tasks.workunit.client.1.smithi123.stdout:1/31: mkdir d3/d6 0 2024-02-16T22:21:44.414 INFO:tasks.workunit.client.1.smithi123.stdout:2/41: mknod d4/d8/cd 0 2024-02-16T22:21:44.419 INFO:tasks.workunit.client.1.smithi123.stdout:0/58: symlink d1/d4/lb 0 2024-02-16T22:21:44.422 INFO:tasks.workunit.client.1.smithi123.stdout:8/49: rename d0/f3 to d0/df/f13 0 2024-02-16T22:21:44.422 INFO:tasks.workunit.client.1.smithi123.stdout:8/50: truncate d0/f4 232977 0 2024-02-16T22:21:44.423 INFO:tasks.workunit.client.1.smithi123.stdout:5/28: creat d6/f8 x:0 0 0 2024-02-16T22:21:44.465 INFO:tasks.workunit.client.1.smithi123.stdout:3/48: creat d0/ff x:0 0 0 2024-02-16T22:21:44.467 INFO:tasks.workunit.client.1.smithi123.stdout:4/68: symlink d5/l15 0 2024-02-16T22:21:44.468 INFO:tasks.workunit.client.1.smithi123.stdout:4/69: stat d5 0 2024-02-16T22:21:44.472 INFO:tasks.workunit.client.1.smithi123.stdout:6/35: creat d0/d6/fb x:0 0 0 2024-02-16T22:21:44.472 INFO:tasks.workunit.client.1.smithi123.stdout:6/36: write d0/d6/f8 [747580,14961] 0 2024-02-16T22:21:44.474 INFO:tasks.workunit.client.1.smithi123.stdout:7/38: getdents d3/d5 0 2024-02-16T22:21:44.476 INFO:tasks.workunit.client.1.smithi123.stdout:1/32: creat d3/f7 x:0 0 0 2024-02-16T22:21:44.477 INFO:tasks.workunit.client.1.smithi123.stdout:1/33: readlink d3/l4 0 2024-02-16T22:21:44.480 INFO:tasks.workunit.client.1.smithi123.stdout:0/59: rmdir d1/d8 39 2024-02-16T22:21:44.483 INFO:tasks.workunit.client.1.smithi123.stdout:1/34: dwrite d3/f5 [0,4194304] 0 2024-02-16T22:21:44.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.483+0000 7fb9bb0ee700 1 -- 172.21.15.114:0/2505496517 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb9b419c6b0 con 0x7fb9b40ffab0 2024-02-16T22:21:44.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.484+0000 7fb9b1ffb700 1 -- 172.21.15.114:0/2505496517 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1825 (secure 0 0 0) 0x7fb9a8059aa0 con 0x7fb9b40ffab0 2024-02-16T22:21:44.486 INFO:tasks.workunit.client.1.smithi123.stdout:3/49: write d0/f7 [3792859,122514] 0 2024-02-16T22:21:44.486 INFO:teuthology.orchestra.run.smithi114.stdout:e14 2024-02-16T22:21:44.486 INFO:teuthology.orchestra.run.smithi114.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:21:44.487 INFO:teuthology.orchestra.run.smithi114.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:21:44.487 INFO:teuthology.orchestra.run.smithi114.stdout:legacy client fscid: 1 2024-02-16T22:21:44.487 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:21:44.487 INFO:teuthology.orchestra.run.smithi114.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:21:44.487 INFO:teuthology.orchestra.run.smithi114.stdout:fs_name cephfs 2024-02-16T22:21:44.487 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 12 2024-02-16T22:21:44.487 INFO:teuthology.orchestra.run.smithi114.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:21:44.487 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-02-16T22:18:02.108825+0000 2024-02-16T22:21:44.487 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-02-16T22:18:16.413371+0000 2024-02-16T22:21:44.487 INFO:teuthology.orchestra.run.smithi114.stdout:tableserver 0 2024-02-16T22:21:44.487 INFO:teuthology.orchestra.run.smithi114.stdout:root 0 2024-02-16T22:21:44.487 INFO:teuthology.orchestra.run.smithi114.stdout:session_timeout 60 2024-02-16T22:21:44.487 INFO:teuthology.orchestra.run.smithi114.stdout:session_autoclose 300 2024-02-16T22:21:44.487 INFO:teuthology.orchestra.run.smithi114.stdout:max_file_size 1099511627776 2024-02-16T22:21:44.488 INFO:teuthology.orchestra.run.smithi114.stdout:required_client_features {} 2024-02-16T22:21:44.488 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure 0 2024-02-16T22:21:44.488 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure_osd_epoch 0 2024-02-16T22:21:44.488 INFO:teuthology.orchestra.run.smithi114.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:21:44.488 INFO:teuthology.orchestra.run.smithi114.stdout:max_mds 1 2024-02-16T22:21:44.488 INFO:teuthology.orchestra.run.smithi114.stdout:in 0 2024-02-16T22:21:44.488 INFO:teuthology.orchestra.run.smithi114.stdout:up {0=14468} 2024-02-16T22:21:44.488 INFO:teuthology.orchestra.run.smithi114.stdout:failed 2024-02-16T22:21:44.488 INFO:teuthology.orchestra.run.smithi114.stdout:damaged 2024-02-16T22:21:44.488 INFO:teuthology.orchestra.run.smithi114.stdout:stopped 2024-02-16T22:21:44.488 INFO:teuthology.orchestra.run.smithi114.stdout:data_pools [3] 2024-02-16T22:21:44.488 INFO:teuthology.orchestra.run.smithi114.stdout:metadata_pool 2 2024-02-16T22:21:44.488 INFO:teuthology.orchestra.run.smithi114.stdout:inline_data enabled 2024-02-16T22:21:44.488 INFO:teuthology.orchestra.run.smithi114.stdout:balancer 2024-02-16T22:21:44.489 INFO:teuthology.orchestra.run.smithi114.stdout:bal_rank_mask -1 2024-02-16T22:21:44.489 INFO:teuthology.orchestra.run.smithi114.stdout:standby_count_wanted 1 2024-02-16T22:21:44.489 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.zzryke{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.114:6826/277953535,v1:172.21.15.114:6827/277953535] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:21:44.489 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:21:44.489 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:21:44.489 INFO:teuthology.orchestra.run.smithi114.stdout:Standby daemons: 2024-02-16T22:21:44.489 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:21:44.489 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.fdxpey{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:21:44.489 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.vhlqbe{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:21:44.489 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.gopcfa{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:21:44.490 INFO:tasks.workunit.client.1.smithi123.stdout:4/70: mknod d5/c16 0 2024-02-16T22:21:44.490 INFO:tasks.workunit.client.1.smithi123.stdout:4/71: truncate d5/ff 1076108 0 2024-02-16T22:21:44.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.488+0000 7fb9bb0ee700 1 -- 172.21.15.114:0/2505496517 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fb9a406ce40 msgr2=0x7fb9a406f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:44.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.488+0000 7fb9bb0ee700 1 --2- 172.21.15.114:0/2505496517 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fb9a406ce40 0x7fb9a406f300 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb9a000b5c0 tx=0x7fb9a0005960 comp rx=0 tx=0).stop 2024-02-16T22:21:44.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.488+0000 7fb9bb0ee700 1 -- 172.21.15.114:0/2505496517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9b40ffab0 msgr2=0x7fb9b4196340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:44.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.488+0000 7fb9bb0ee700 1 --2- 172.21.15.114:0/2505496517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9b40ffab0 0x7fb9b4196340 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800d900 tx=0x7fb9a800dcc0 comp rx=0 tx=0).stop 2024-02-16T22:21:44.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.488+0000 7fb9bb0ee700 1 -- 172.21.15.114:0/2505496517 shutdown_connections 2024-02-16T22:21:44.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.488+0000 7fb9bb0ee700 1 --2- 172.21.15.114:0/2505496517 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fb9a406ce40 0x7fb9a406f300 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:44.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.488+0000 7fb9bb0ee700 1 --2- 172.21.15.114:0/2505496517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9b40ffab0 0x7fb9b4196340 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:44.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.488+0000 7fb9bb0ee700 1 --2- 172.21.15.114:0/2505496517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9b4100cb0 0x7fb9b4196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:44.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.488+0000 7fb9bb0ee700 1 -- 172.21.15.114:0/2505496517 >> 172.21.15.114:0/2505496517 conn(0x7fb9b40fb030 msgr2=0x7fb9b4068410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:21:44.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.488+0000 7fb9bb0ee700 1 -- 172.21.15.114:0/2505496517 shutdown_connections 2024-02-16T22:21:44.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.488+0000 7fb9bb0ee700 1 -- 172.21.15.114:0/2505496517 wait complete. 2024-02-16T22:21:44.492 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 14 2024-02-16T22:21:44.494 INFO:tasks.workunit.client.1.smithi123.stdout:7/39: creat d3/d5/ff x:0 0 0 2024-02-16T22:21:44.495 INFO:tasks.workunit.client.1.smithi123.stdout:7/40: truncate d3/f8 509637 0 2024-02-16T22:21:44.504 INFO:tasks.workunit.client.1.smithi123.stdout:1/35: creat d3/f8 x:0 0 0 2024-02-16T22:21:44.505 INFO:tasks.workunit.client.1.smithi123.stdout:0/60: dwrite d1/d8/fa [0,4194304] 0 2024-02-16T22:21:44.512 INFO:tasks.workunit.client.1.smithi123.stdout:9/85: rmdir d4/d17/d18 0 2024-02-16T22:21:44.514 INFO:tasks.workunit.client.1.smithi123.stdout:6/37: mkdir d0/d9/dc 0 2024-02-16T22:21:44.516 INFO:tasks.workunit.client.1.smithi123.stdout:7/41: mknod d3/d5/c10 0 2024-02-16T22:21:44.520 INFO:tasks.workunit.client.1.smithi123.stdout:2/42: dread d4/d8/f9 [0,4194304] 0 2024-02-16T22:21:44.520 INFO:tasks.workunit.client.1.smithi123.stdout:6/38: dwrite d0/d6/f8 [0,4194304] 0 2024-02-16T22:21:44.521 INFO:tasks.workunit.client.1.smithi123.stdout:6/39: dread - d0/d6/fa zero size 2024-02-16T22:21:44.530 INFO:tasks.workunit.client.1.smithi123.stdout:4/72: link d5/f14 d5/f17 0 2024-02-16T22:21:44.531 INFO:tasks.workunit.client.1.smithi123.stdout:4/73: truncate d5/f14 31828 0 2024-02-16T22:21:44.532 INFO:tasks.workunit.client.1.smithi123.stdout:9/86: mkdir d4/d1e 0 2024-02-16T22:21:44.533 INFO:tasks.workunit.client.1.smithi123.stdout:9/87: chown d4/l15 0 1 2024-02-16T22:21:44.536 INFO:tasks.workunit.client.1.smithi123.stdout:7/42: mknod d3/d5/c11 0 2024-02-16T22:21:44.538 INFO:tasks.workunit.client.1.smithi123.stdout:8/51: getdents d0 0 2024-02-16T22:21:44.538 INFO:tasks.workunit.client.1.smithi123.stdout:9/88: dwrite d4/f6 [0,4194304] 0 2024-02-16T22:21:44.539 INFO:tasks.workunit.client.1.smithi123.stdout:8/52: chown d0/c9 6509 1 2024-02-16T22:21:44.539 INFO:tasks.workunit.client.1.smithi123.stdout:8/53: fdatasync d0/fd 0 2024-02-16T22:21:44.539 INFO:tasks.workunit.client.1.smithi123.stdout:9/89: write f2 [2550222,111354] 0 2024-02-16T22:21:44.543 INFO:tasks.workunit.client.1.smithi123.stdout:5/29: getdents d6 0 2024-02-16T22:21:44.544 INFO:tasks.workunit.client.1.smithi123.stdout:5/30: read - d6/f8 zero size 2024-02-16T22:21:44.544 INFO:tasks.workunit.client.1.smithi123.stdout:1/36: mknod d3/d6/c9 0 2024-02-16T22:21:44.546 INFO:tasks.workunit.client.1.smithi123.stdout:0/61: mknod d1/cc 0 2024-02-16T22:21:44.546 INFO:tasks.workunit.client.1.smithi123.stdout:3/50: truncate d0/f7 1957233 0 2024-02-16T22:21:44.548 INFO:tasks.workunit.client.1.smithi123.stdout:8/54: dwrite d0/f7 [0,4194304] 0 2024-02-16T22:21:44.549 INFO:tasks.workunit.client.1.smithi123.stdout:4/74: mknod d5/c18 0 2024-02-16T22:21:44.554 INFO:tasks.workunit.client.1.smithi123.stdout:9/90: rename f2 to d4/d17/f1f 0 2024-02-16T22:21:44.558 INFO:tasks.workunit.client.1.smithi123.stdout:5/31: creat d6/f9 x:0 0 0 2024-02-16T22:21:44.560 INFO:tasks.workunit.client.1.smithi123.stdout:1/37: mkdir d3/da 0 2024-02-16T22:21:44.566 INFO:tasks.workunit.client.1.smithi123.stdout:8/55: rename d0/f7 to d0/f14 0 2024-02-16T22:21:44.568 INFO:tasks.workunit.client.1.smithi123.stdout:2/43: link d4/c5 d4/ce 0 2024-02-16T22:21:44.571 INFO:tasks.workunit.client.1.smithi123.stdout:9/91: mkdir d4/d17/d20 0 2024-02-16T22:21:44.574 INFO:tasks.workunit.client.1.smithi123.stdout:6/40: link d0/f2 d0/d6/fd 0 2024-02-16T22:21:44.576 INFO:tasks.workunit.client.1.smithi123.stdout:5/32: mknod d6/ca 0 2024-02-16T22:21:44.586 INFO:tasks.workunit.client.1.smithi123.stdout:1/38: mkdir d3/d6/db 0 2024-02-16T22:21:44.589 INFO:tasks.workunit.client.1.smithi123.stdout:2/44: mknod d4/cf 0 2024-02-16T22:21:44.592 INFO:tasks.workunit.client.1.smithi123.stdout:4/75: write d5/f14 [46712,101745] 0 2024-02-16T22:21:44.594 INFO:tasks.workunit.client.1.smithi123.stdout:6/41: symlink d0/d9/le 0 2024-02-16T22:21:44.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.636+0000 7fca691d1700 1 -- 172.21.15.114:0/2171772802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fca64100cc0 msgr2=0x7fca64101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:44.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.636+0000 7fca691d1700 1 --2- 172.21.15.114:0/2171772802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fca64100cc0 0x7fca64101120 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fca58009910 tx=0x7fca58009c20 comp rx=0 tx=0).stop 2024-02-16T22:21:44.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.636+0000 7fca691d1700 1 -- 172.21.15.114:0/2171772802 shutdown_connections 2024-02-16T22:21:44.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.636+0000 7fca691d1700 1 --2- 172.21.15.114:0/2171772802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fca64100cc0 0x7fca64101120 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:44.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.636+0000 7fca691d1700 1 --2- 172.21.15.114:0/2171772802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca640ffac0 0x7fca640ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:44.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.636+0000 7fca691d1700 1 -- 172.21.15.114:0/2171772802 >> 172.21.15.114:0/2171772802 conn(0x7fca640fb060 msgr2=0x7fca640fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:21:44.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.636+0000 7fca691d1700 1 -- 172.21.15.114:0/2171772802 shutdown_connections 2024-02-16T22:21:44.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.636+0000 7fca691d1700 1 -- 172.21.15.114:0/2171772802 wait complete. 2024-02-16T22:21:44.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.637+0000 7fca691d1700 1 Processor -- start 2024-02-16T22:21:44.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.637+0000 7fca691d1700 1 -- start start 2024-02-16T22:21:44.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.637+0000 7fca691d1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca640ffac0 0x7fca64195270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:44.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.637+0000 7fca691d1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fca64100cc0 0x7fca641957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:44.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.637+0000 7fca691d1700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca64195dd0 con 0x7fca64100cc0 2024-02-16T22:21:44.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.637+0000 7fca691d1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca64195f10 con 0x7fca640ffac0 2024-02-16T22:21:44.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.637+0000 7fca6259c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fca64100cc0 0x7fca641957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:44.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.637+0000 7fca62d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca640ffac0 0x7fca64195270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:44.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.637+0000 7fca6259c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fca64100cc0 0x7fca641957b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39448/0 (socket says 172.21.15.114:39448) 2024-02-16T22:21:44.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.637+0000 7fca6259c700 1 -- 172.21.15.114:0/3270196989 learned_addr learned my addr 172.21.15.114:0/3270196989 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:21:44.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.638+0000 7fca62d9d700 1 -- 172.21.15.114:0/3270196989 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fca64100cc0 msgr2=0x7fca641957b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:44.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.638+0000 7fca62d9d700 1 --2- 172.21.15.114:0/3270196989 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fca64100cc0 0x7fca641957b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:44.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.638+0000 7fca62d9d700 1 -- 172.21.15.114:0/3270196989 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca580095f0 con 0x7fca640ffac0 2024-02-16T22:21:44.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.638+0000 7fca62d9d700 1 --2- 172.21.15.114:0/3270196989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca640ffac0 0x7fca64195270 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fca5400e940 tx=0x7fca5400ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:21:44.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.638+0000 7fca4bfff700 1 -- 172.21.15.114:0/3270196989 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca540041f0 con 0x7fca640ffac0 2024-02-16T22:21:44.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.638+0000 7fca4bfff700 1 -- 172.21.15.114:0/3270196989 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fca54004350 con 0x7fca640ffac0 2024-02-16T22:21:44.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.638+0000 7fca691d1700 1 -- 172.21.15.114:0/3270196989 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca64068db0 con 0x7fca640ffac0 2024-02-16T22:21:44.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.638+0000 7fca4bfff700 1 -- 172.21.15.114:0/3270196989 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca54003c80 con 0x7fca640ffac0 2024-02-16T22:21:44.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.638+0000 7fca691d1700 1 -- 172.21.15.114:0/3270196989 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca64069300 con 0x7fca640ffac0 2024-02-16T22:21:44.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.639+0000 7fca691d1700 1 -- 172.21.15.114:0/3270196989 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca64108620 con 0x7fca640ffac0 2024-02-16T22:21:44.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.639+0000 7fca4bfff700 1 -- 172.21.15.114:0/3270196989 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) v1 ==== 90933+0+0 (secure 0 0 0) 0x7fca54018430 con 0x7fca640ffac0 2024-02-16T22:21:44.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.640+0000 7fca4bfff700 1 --2- 172.21.15.114:0/3270196989 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fca4c071240 0x7fca4c073700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:44.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.640+0000 7fca4bfff700 1 -- 172.21.15.114:0/3270196989 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5726+0+0 (secure 0 0 0) 0x7fca54014070 con 0x7fca640ffac0 2024-02-16T22:21:44.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.640+0000 7fca6259c700 1 --2- 172.21.15.114:0/3270196989 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fca4c071240 0x7fca4c073700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:44.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.640+0000 7fca6259c700 1 --2- 172.21.15.114:0/3270196989 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fca4c071240 0x7fca4c073700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fca58009fd0 tx=0x7fca58005940 comp rx=0 tx=0).ready entity=mgr.24441 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:21:44.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.644+0000 7fca4bfff700 1 -- 172.21.15.114:0/3270196989 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182869 (secure 0 0 0) 0x7fca5405a2b0 con 0x7fca640ffac0 2024-02-16T22:21:44.655 INFO:tasks.workunit.client.1.smithi123.stdout:7/43: sync 2024-02-16T22:21:44.675 INFO:tasks.workunit.client.0.smithi114.stdout:4/52: symlink dd/l10 0 2024-02-16T22:21:44.676 INFO:tasks.workunit.client.0.smithi114.stdout:4/53: write dd/ff [569044,61019] 0 2024-02-16T22:21:44.680 INFO:tasks.workunit.client.0.smithi114.stdout:8/59: mkdir d4/d16 0 2024-02-16T22:21:44.683 INFO:tasks.workunit.client.0.smithi114.stdout:4/54: dwrite f2 [0,4194304] 0 2024-02-16T22:21:44.684 INFO:tasks.workunit.client.0.smithi114.stdout:4/55: write f2 [1815431,31478] 0 2024-02-16T22:21:44.686 INFO:tasks.workunit.client.0.smithi114.stdout:0/36: mkdir d1/db 0 2024-02-16T22:21:44.687 INFO:tasks.workunit.client.0.smithi114.stdout:0/37: write d1/fa [255816,112861] 0 2024-02-16T22:21:44.688 INFO:tasks.workunit.client.0.smithi114.stdout:0/38: truncate d1/f7 2941351 0 2024-02-16T22:21:44.688 INFO:tasks.workunit.client.0.smithi114.stdout:1/34: mknod d4/d9/cc 0 2024-02-16T22:21:44.699 INFO:tasks.workunit.client.0.smithi114.stdout:1/35: rmdir d4/d9 39 2024-02-16T22:21:44.798 INFO:tasks.workunit.client.0.smithi114.stdout:1/36: creat d4/fd x:0 0 0 2024-02-16T22:21:44.802 INFO:tasks.workunit.client.0.smithi114.stdout:1/37: rmdir d4/d9 39 2024-02-16T22:21:44.806 INFO:tasks.workunit.client.0.smithi114.stdout:1/38: mknod d4/d9/ce 0 2024-02-16T22:21:44.807 INFO:tasks.workunit.client.0.smithi114.stdout:1/39: dread - d4/fb zero size 2024-02-16T22:21:44.807 INFO:tasks.workunit.client.0.smithi114.stdout:1/40: dread - d4/fb zero size 2024-02-16T22:21:44.808 INFO:tasks.workunit.client.0.smithi114.stdout:1/41: readlink d4/l8 0 2024-02-16T22:21:44.808 INFO:tasks.workunit.client.0.smithi114.stdout:1/42: write d4/fd [773500,103805] 0 2024-02-16T22:21:44.809 INFO:tasks.workunit.client.0.smithi114.stdout:1/43: write d4/fa [840866,126115] 0 2024-02-16T22:21:44.811 INFO:tasks.workunit.client.0.smithi114.stdout:1/44: write d4/f5 [5139497,16875] 0 2024-02-16T22:21:44.811 INFO:tasks.workunit.client.0.smithi114.stdout:1/45: truncate d4/fb 768504 0 2024-02-16T22:21:44.812 INFO:tasks.workunit.client.0.smithi114.stdout:1/46: stat d4/c6 0 2024-02-16T22:21:44.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.881+0000 7fca691d1700 1 -- 172.21.15.114:0/3270196989 --> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fca64105610 con 0x7fca4c071240 2024-02-16T22:21:44.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.882+0000 7fca4bfff700 1 -- 172.21.15.114:0/3270196989 <== mgr.24441 v2:172.21.15.123:6828/1539029039 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7fca64105610 con 0x7fca4c071240 2024-02-16T22:21:44.885 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:21:44.885 INFO:teuthology.orchestra.run.smithi114.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:21:44.885 INFO:teuthology.orchestra.run.smithi114.stdout: "in_progress": true, 2024-02-16T22:21:44.885 INFO:teuthology.orchestra.run.smithi114.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:21:44.885 INFO:teuthology.orchestra.run.smithi114.stdout: "services_complete": [], 2024-02-16T22:21:44.885 INFO:teuthology.orchestra.run.smithi114.stdout: "progress": "1/23 daemons upgraded", 2024-02-16T22:21:44.885 INFO:teuthology.orchestra.run.smithi114.stdout: "message": "Currently upgrading mgr daemons", 2024-02-16T22:21:44.885 INFO:teuthology.orchestra.run.smithi114.stdout: "is_paused": false 2024-02-16T22:21:44.885 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:21:44.886 INFO:tasks.workunit.client.1.smithi123.stdout:0/62: link d1/cc d1/d8/cd 0 2024-02-16T22:21:44.886 INFO:tasks.workunit.client.0.smithi114.stdout:1/47: dread d4/fb [0,4194304] 0 2024-02-16T22:21:44.886 INFO:tasks.workunit.client.0.smithi114.stdout:1/48: fsync d4/fb 0 2024-02-16T22:21:44.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.886+0000 7fca691d1700 1 -- 172.21.15.114:0/3270196989 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fca4c071240 msgr2=0x7fca4c073700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:44.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.886+0000 7fca691d1700 1 --2- 172.21.15.114:0/3270196989 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fca4c071240 0x7fca4c073700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fca58009fd0 tx=0x7fca58005940 comp rx=0 tx=0).stop 2024-02-16T22:21:44.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.886+0000 7fca691d1700 1 -- 172.21.15.114:0/3270196989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca640ffac0 msgr2=0x7fca64195270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:44.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.886+0000 7fca691d1700 1 --2- 172.21.15.114:0/3270196989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca640ffac0 0x7fca64195270 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fca5400e940 tx=0x7fca5400ec50 comp rx=0 tx=0).stop 2024-02-16T22:21:44.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.886+0000 7fca691d1700 1 -- 172.21.15.114:0/3270196989 shutdown_connections 2024-02-16T22:21:44.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.886+0000 7fca691d1700 1 --2- 172.21.15.114:0/3270196989 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fca4c071240 0x7fca4c073700 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:44.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.886+0000 7fca691d1700 1 --2- 172.21.15.114:0/3270196989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca640ffac0 0x7fca64195270 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:44.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.886+0000 7fca691d1700 1 --2- 172.21.15.114:0/3270196989 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fca64100cc0 0x7fca641957b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:44.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.886+0000 7fca691d1700 1 -- 172.21.15.114:0/3270196989 >> 172.21.15.114:0/3270196989 conn(0x7fca640fb060 msgr2=0x7fca64103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:21:44.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.886+0000 7fca691d1700 1 -- 172.21.15.114:0/3270196989 shutdown_connections 2024-02-16T22:21:44.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:44.886+0000 7fca691d1700 1 -- 172.21.15.114:0/3270196989 wait complete. 2024-02-16T22:21:44.889 INFO:tasks.workunit.client.0.smithi114.stdout:1/49: read d4/fa [479730,34807] 0 2024-02-16T22:21:44.889 INFO:tasks.workunit.client.1.smithi123.stdout:1/39: creat d3/fc x:0 0 0 2024-02-16T22:21:44.890 INFO:tasks.workunit.client.1.smithi123.stdout:8/56: mknod d0/d12/c15 0 2024-02-16T22:21:44.891 INFO:tasks.workunit.client.1.smithi123.stdout:3/51: dwrite d0/f7 [0,4194304] 0 2024-02-16T22:21:44.891 INFO:tasks.workunit.client.1.smithi123.stdout:2/45: fdatasync d4/d8/f9 0 2024-02-16T22:21:44.892 INFO:tasks.workunit.client.0.smithi114.stdout:8/60: write d4/fa [908344,82500] 0 2024-02-16T22:21:44.893 INFO:tasks.workunit.client.0.smithi114.stdout:8/61: readlink d4/l5 0 2024-02-16T22:21:44.893 INFO:tasks.workunit.client.1.smithi123.stdout:2/46: read - d4/fc zero size 2024-02-16T22:21:44.894 INFO:tasks.workunit.client.1.smithi123.stdout:9/92: link d4/d17/f1d d4/f21 0 2024-02-16T22:21:44.896 INFO:tasks.workunit.client.1.smithi123.stdout:1/40: dwrite d3/f7 [0,4194304] 0 2024-02-16T22:21:44.898 INFO:tasks.workunit.client.0.smithi114.stdout:8/62: dread d4/fc [0,4194304] 0 2024-02-16T22:21:44.898 INFO:tasks.workunit.client.1.smithi123.stdout:8/57: dread d0/fd [0,4194304] 0 2024-02-16T22:21:44.900 INFO:tasks.workunit.client.0.smithi114.stdout:8/63: fdatasync d4/fa 0 2024-02-16T22:21:44.905 INFO:tasks.workunit.client.0.smithi114.stdout:4/56: dwrite f4 [4194304,4194304] 0 2024-02-16T22:21:44.906 INFO:tasks.workunit.client.1.smithi123.stdout:3/52: chown d0/fb 3595 1 2024-02-16T22:21:44.910 INFO:tasks.workunit.client.0.smithi114.stdout:8/64: write d4/fe [233825,108060] 0 2024-02-16T22:21:44.912 INFO:tasks.workunit.client.0.smithi114.stdout:0/39: truncate d1/fa 1095609 0 2024-02-16T22:21:44.914 INFO:tasks.workunit.client.1.smithi123.stdout:2/47: symlink d4/d8/l10 0 2024-02-16T22:21:44.914 INFO:tasks.workunit.client.1.smithi123.stdout:2/48: truncate d4/fc 908451 0 2024-02-16T22:21:44.915 INFO:tasks.workunit.client.0.smithi114.stdout:4/57: creat dd/f11 x:0 0 0 2024-02-16T22:21:44.915 INFO:tasks.workunit.client.0.smithi114.stdout:8/65: dread f0 [0,4194304] 0 2024-02-16T22:21:44.916 INFO:tasks.workunit.client.0.smithi114.stdout:8/66: truncate d4/f7 5133016 0 2024-02-16T22:21:44.918 INFO:tasks.workunit.client.0.smithi114.stdout:0/40: symlink d1/lc 0 2024-02-16T22:21:44.922 INFO:tasks.workunit.client.1.smithi123.stdout:4/76: dwrite f4 [0,4194304] 0 2024-02-16T22:21:44.922 INFO:tasks.workunit.client.0.smithi114.stdout:8/67: rename d4/fa to d4/d11/f17 0 2024-02-16T22:21:44.925 INFO:tasks.workunit.client.0.smithi114.stdout:0/41: mknod d1/cd 0 2024-02-16T22:21:44.930 INFO:tasks.workunit.client.0.smithi114.stdout:0/42: dread d1/f7 [0,4194304] 0 2024-02-16T22:21:44.933 INFO:tasks.workunit.client.1.smithi123.stdout:8/58: dread d0/df/f13 [0,4194304] 0 2024-02-16T22:21:44.933 INFO:tasks.workunit.client.1.smithi123.stdout:8/59: stat d0/fd 0 2024-02-16T22:21:44.934 INFO:tasks.workunit.client.1.smithi123.stdout:8/60: dread d0/f4 [0,4194304] 0 2024-02-16T22:21:44.935 INFO:tasks.workunit.client.1.smithi123.stdout:8/61: chown d0/d12 10 1 2024-02-16T22:21:44.937 INFO:tasks.workunit.client.0.smithi114.stdout:8/68: dwrite f0 [0,4194304] 0 2024-02-16T22:21:44.939 INFO:tasks.workunit.client.1.smithi123.stdout:6/42: creat d0/ff x:0 0 0 2024-02-16T22:21:44.939 INFO:tasks.workunit.client.1.smithi123.stdout:6/43: readlink d0/d9/le 0 2024-02-16T22:21:44.940 INFO:tasks.workunit.client.0.smithi114.stdout:0/43: dwrite d1/f7 [0,4194304] 0 2024-02-16T22:21:44.940 INFO:tasks.workunit.client.1.smithi123.stdout:6/44: chown d0 11 1 2024-02-16T22:21:44.940 INFO:tasks.workunit.client.1.smithi123.stdout:9/93: creat d4/d17/d20/f22 x:0 0 0 2024-02-16T22:21:44.941 INFO:tasks.workunit.client.1.smithi123.stdout:5/33: getdents d6 0 2024-02-16T22:21:44.943 INFO:tasks.workunit.client.1.smithi123.stdout:2/49: dwrite f2 [0,4194304] 0 2024-02-16T22:21:44.945 INFO:tasks.workunit.client.1.smithi123.stdout:4/77: creat d5/f19 x:0 0 0 2024-02-16T22:21:44.948 INFO:tasks.workunit.client.1.smithi123.stdout:1/41: mknod d3/da/cd 0 2024-02-16T22:21:44.950 INFO:tasks.workunit.client.1.smithi123.stdout:9/94: dwrite d4/f10 [0,4194304] 0 2024-02-16T22:21:44.958 INFO:tasks.workunit.client.0.smithi114.stdout:0/44: rmdir d1/db 0 2024-02-16T22:21:44.959 INFO:tasks.workunit.client.0.smithi114.stdout:0/45: chown d1/f8 676201 1 2024-02-16T22:21:44.959 INFO:tasks.workunit.client.0.smithi114.stdout:0/46: chown d1/cd 10 1 2024-02-16T22:21:44.961 INFO:tasks.workunit.client.1.smithi123.stdout:5/34: creat d6/fb x:0 0 0 2024-02-16T22:21:44.962 INFO:tasks.workunit.client.1.smithi123.stdout:2/50: mkdir d4/d11 0 2024-02-16T22:21:44.963 INFO:tasks.workunit.client.1.smithi123.stdout:2/51: dread d4/fc [0,4194304] 0 2024-02-16T22:21:44.963 INFO:tasks.workunit.client.0.smithi114.stdout:0/47: creat d1/fe x:0 0 0 2024-02-16T22:21:44.964 INFO:tasks.workunit.client.1.smithi123.stdout:2/52: truncate d4/f6 366368 0 2024-02-16T22:21:44.969 INFO:tasks.workunit.client.1.smithi123.stdout:8/62: truncate d0/fd 353820 0 2024-02-16T22:21:44.970 INFO:tasks.workunit.client.1.smithi123.stdout:5/35: mknod d6/cc 0 2024-02-16T22:21:44.971 INFO:tasks.workunit.client.1.smithi123.stdout:2/53: dwrite d4/d8/f9 [0,4194304] 0 2024-02-16T22:21:44.972 INFO:tasks.workunit.client.1.smithi123.stdout:5/36: write d6/f8 [1037272,99951] 0 2024-02-16T22:21:44.973 INFO:tasks.workunit.client.1.smithi123.stdout:2/54: dread d4/f6 [0,4194304] 0 2024-02-16T22:21:44.975 INFO:tasks.workunit.client.0.smithi114.stdout:0/48: fdatasync d1/f7 0 2024-02-16T22:21:44.976 INFO:tasks.workunit.client.1.smithi123.stdout:8/63: write d0/f4 [1266589,19560] 0 2024-02-16T22:21:44.979 INFO:tasks.workunit.client.1.smithi123.stdout:4/78: link d5/l13 d5/l1a 0 2024-02-16T22:21:44.981 INFO:tasks.workunit.client.1.smithi123.stdout:5/37: dwrite d6/f9 [0,4194304] 0 2024-02-16T22:21:44.982 INFO:tasks.workunit.client.1.smithi123.stdout:5/38: write d6/fb [1038152,112567] 0 2024-02-16T22:21:44.984 INFO:tasks.workunit.client.0.smithi114.stdout:0/49: dwrite d1/f7 [0,4194304] 0 2024-02-16T22:21:44.987 INFO:tasks.workunit.client.1.smithi123.stdout:1/42: getdents d3/d6 0 2024-02-16T22:21:44.987 INFO:tasks.workunit.client.1.smithi123.stdout:1/43: chown d3 37831 1 2024-02-16T22:21:44.987 INFO:tasks.workunit.client.1.smithi123.stdout:2/55: dwrite d4/fc [0,4194304] 0 2024-02-16T22:21:44.989 INFO:tasks.workunit.client.1.smithi123.stdout:4/79: creat d5/f1b x:0 0 0 2024-02-16T22:21:44.994 INFO:tasks.workunit.client.1.smithi123.stdout:5/39: mkdir d6/dd 0 2024-02-16T22:21:44.996 INFO:tasks.workunit.client.1.smithi123.stdout:2/56: dread f2 [0,4194304] 0 2024-02-16T22:21:44.997 INFO:tasks.workunit.client.1.smithi123.stdout:1/44: creat d3/fe x:0 0 0 2024-02-16T22:21:45.001 INFO:tasks.workunit.client.1.smithi123.stdout:4/80: dwrite d5/f17 [0,4194304] 0 2024-02-16T22:21:45.002 INFO:tasks.workunit.client.1.smithi123.stdout:5/40: dread d6/f8 [0,4194304] 0 2024-02-16T22:21:45.002 INFO:tasks.workunit.client.1.smithi123.stdout:8/64: getdents d0 0 2024-02-16T22:21:45.005 INFO:tasks.workunit.client.1.smithi123.stdout:1/45: getdents d3/d6/db 0 2024-02-16T22:21:45.006 INFO:tasks.workunit.client.1.smithi123.stdout:1/46: chown d3/fc 516743772 1 2024-02-16T22:21:45.010 INFO:tasks.workunit.client.1.smithi123.stdout:5/41: dwrite d6/f9 [0,4194304] 0 2024-02-16T22:21:45.012 INFO:tasks.workunit.client.1.smithi123.stdout:8/65: getdents d0 0 2024-02-16T22:21:45.012 INFO:tasks.workunit.client.1.smithi123.stdout:5/42: dread d6/f8 [0,4194304] 0 2024-02-16T22:21:45.012 INFO:tasks.workunit.client.1.smithi123.stdout:1/47: dread d3/f5 [0,4194304] 0 2024-02-16T22:21:45.013 INFO:tasks.workunit.client.1.smithi123.stdout:1/48: stat d3/f8 0 2024-02-16T22:21:45.015 INFO:tasks.workunit.client.1.smithi123.stdout:5/43: stat l0 0 2024-02-16T22:21:45.016 INFO:tasks.workunit.client.1.smithi123.stdout:1/49: creat d3/ff x:0 0 0 2024-02-16T22:21:45.017 INFO:tasks.workunit.client.1.smithi123.stdout:5/44: truncate d6/fb 1323367 0 2024-02-16T22:21:45.017 INFO:tasks.workunit.client.1.smithi123.stdout:1/50: chown d3/ff 726669 1 2024-02-16T22:21:45.020 INFO:tasks.workunit.client.1.smithi123.stdout:1/51: write d3/ff [501175,65694] 0 2024-02-16T22:21:45.020 INFO:tasks.workunit.client.1.smithi123.stdout:8/66: dwrite d0/fc [0,4194304] 0 2024-02-16T22:21:45.021 INFO:tasks.workunit.client.1.smithi123.stdout:8/67: chown d0/d12 23171 1 2024-02-16T22:21:45.028 INFO:tasks.workunit.client.1.smithi123.stdout:8/68: dwrite d0/f6 [0,4194304] 0 2024-02-16T22:21:45.029 INFO:tasks.workunit.client.1.smithi123.stdout:8/69: chown d0/df 6 1 2024-02-16T22:21:45.032 INFO:tasks.workunit.client.1.smithi123.stdout:1/52: dread d3/ff [0,4194304] 0 2024-02-16T22:21:45.036 INFO:tasks.workunit.client.1.smithi123.stdout:4/81: fdatasync d5/f17 0 2024-02-16T22:21:45.036 INFO:tasks.workunit.client.1.smithi123.stdout:4/82: stat d5/f17 0 2024-02-16T22:21:45.037 INFO:tasks.workunit.client.1.smithi123.stdout:1/53: mkdir d3/d6/db/d10 0 2024-02-16T22:21:45.037 INFO:tasks.workunit.client.1.smithi123.stdout:4/83: read d5/f17 [1136323,59296] 0 2024-02-16T22:21:45.042 INFO:tasks.workunit.client.1.smithi123.stdout:7/44: truncate d3/d5/f9 100960 0 2024-02-16T22:21:45.044 INFO:tasks.workunit.client.1.smithi123.stdout:7/45: mkdir d3/d12 0 2024-02-16T22:21:45.045 INFO:tasks.workunit.client.1.smithi123.stdout:0/63: truncate d1/f7 207296 0 2024-02-16T22:21:45.047 INFO:tasks.workunit.client.1.smithi123.stdout:0/64: write d1/f6 [4794158,26417] 0 2024-02-16T22:21:45.049 INFO:tasks.workunit.client.0.smithi114.stdout:8/69: truncate d4/f7 1605877 0 2024-02-16T22:21:45.049 INFO:tasks.workunit.client.0.smithi114.stdout:8/70: readlink d4/ld 0 2024-02-16T22:21:45.051 INFO:tasks.workunit.client.1.smithi123.stdout:3/53: truncate d0/fb 2004233 0 2024-02-16T22:21:45.053 INFO:tasks.workunit.client.1.smithi123.stdout:3/54: write d0/f6 [136674,89117] 0 2024-02-16T22:21:45.054 INFO:tasks.workunit.client.0.smithi114.stdout:0/50: rmdir d1 39 2024-02-16T22:21:45.054 INFO:tasks.workunit.client.1.smithi123.stdout:3/55: write d0/fd [709899,95349] 0 2024-02-16T22:21:45.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.055+0000 7fa465354700 1 -- 172.21.15.114:0/2315717747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa460100d00 msgr2=0x7fa460101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:45.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.055+0000 7fa465354700 1 --2- 172.21.15.114:0/2315717747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa460100d00 0x7fa460101180 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa454009910 tx=0x7fa454009c20 comp rx=0 tx=0).stop 2024-02-16T22:21:45.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.055+0000 7fa465354700 1 -- 172.21.15.114:0/2315717747 shutdown_connections 2024-02-16T22:21:45.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.055+0000 7fa465354700 1 --2- 172.21.15.114:0/2315717747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa460100d00 0x7fa460101180 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:45.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.055+0000 7fa465354700 1 --2- 172.21.15.114:0/2315717747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4600fffe0 0x7fa460100400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:45.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.055+0000 7fa465354700 1 -- 172.21.15.114:0/2315717747 >> 172.21.15.114:0/2315717747 conn(0x7fa4600fa7c0 msgr2=0x7fa4600fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:21:45.058 INFO:tasks.workunit.client.0.smithi114.stdout:0/51: write d1/f8 [237264,99109] 0 2024-02-16T22:21:45.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.055+0000 7fa465354700 1 -- 172.21.15.114:0/2315717747 shutdown_connections 2024-02-16T22:21:45.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.055+0000 7fa465354700 1 -- 172.21.15.114:0/2315717747 wait complete. 2024-02-16T22:21:45.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.056+0000 7fa465354700 1 Processor -- start 2024-02-16T22:21:45.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.056+0000 7fa465354700 1 -- start start 2024-02-16T22:21:45.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.057+0000 7fa465354700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4600fffe0 0x7fa4601963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:45.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.057+0000 7fa465354700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa460100d00 0x7fa4601968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:45.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.057+0000 7fa465354700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa460196f00 con 0x7fa4600fffe0 2024-02-16T22:21:45.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.057+0000 7fa465354700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa460197040 con 0x7fa460100d00 2024-02-16T22:21:45.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.057+0000 7fa45effd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4600fffe0 0x7fa4601963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:45.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.057+0000 7fa45effd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4600fffe0 0x7fa4601963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39460/0 (socket says 172.21.15.114:39460) 2024-02-16T22:21:45.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.057+0000 7fa45effd700 1 -- 172.21.15.114:0/2245095767 learned_addr learned my addr 172.21.15.114:0/2245095767 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:21:45.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.057+0000 7fa45e7fc700 1 --2- 172.21.15.114:0/2245095767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa460100d00 0x7fa4601968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:45.060 INFO:tasks.workunit.client.1.smithi123.stdout:3/56: symlink d0/l10 0 2024-02-16T22:21:45.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.057+0000 7fa45effd700 1 -- 172.21.15.114:0/2245095767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa460100d00 msgr2=0x7fa4601968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:45.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.057+0000 7fa45effd700 1 --2- 172.21.15.114:0/2245095767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa460100d00 0x7fa4601968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:45.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.057+0000 7fa45effd700 1 -- 172.21.15.114:0/2245095767 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4540095f0 con 0x7fa4600fffe0 2024-02-16T22:21:45.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.057+0000 7fa45effd700 1 --2- 172.21.15.114:0/2245095767 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4600fffe0 0x7fa4601963a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa45000e8f0 tx=0x7fa45000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:21:45.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.057+0000 7fa447fff700 1 -- 172.21.15.114:0/2245095767 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa450009790 con 0x7fa4600fffe0 2024-02-16T22:21:45.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.058+0000 7fa447fff700 1 -- 172.21.15.114:0/2245095767 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa45000c810 con 0x7fa4600fffe0 2024-02-16T22:21:45.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.058+0000 7fa465354700 1 -- 172.21.15.114:0/2245095767 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa46019baf0 con 0x7fa4600fffe0 2024-02-16T22:21:45.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.058+0000 7fa447fff700 1 -- 172.21.15.114:0/2245095767 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa450003d20 con 0x7fa4600fffe0 2024-02-16T22:21:45.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.058+0000 7fa465354700 1 -- 172.21.15.114:0/2245095767 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa46019c040 con 0x7fa4600fffe0 2024-02-16T22:21:45.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.059+0000 7fa465354700 1 -- 172.21.15.114:0/2245095767 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa460109750 con 0x7fa4600fffe0 2024-02-16T22:21:45.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.060+0000 7fa447fff700 1 -- 172.21.15.114:0/2245095767 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) v1 ==== 90933+0+0 (secure 0 0 0) 0x7fa4500041f0 con 0x7fa4600fffe0 2024-02-16T22:21:45.062 INFO:tasks.workunit.client.0.smithi114.stdout:0/52: mknod d1/cf 0 2024-02-16T22:21:45.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.060+0000 7fa447fff700 1 --2- 172.21.15.114:0/2245095767 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fa44806ce40 0x7fa44806f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:21:45.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.060+0000 7fa447fff700 1 -- 172.21.15.114:0/2245095767 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5726+0+0 (secure 0 0 0) 0x7fa450014070 con 0x7fa4600fffe0 2024-02-16T22:21:45.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.060+0000 7fa45e7fc700 1 --2- 172.21.15.114:0/2245095767 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fa44806ce40 0x7fa44806f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:21:45.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.061+0000 7fa45e7fc700 1 --2- 172.21.15.114:0/2245095767 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fa44806ce40 0x7fa44806f300 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa45400b6f0 tx=0x7fa454005fb0 comp rx=0 tx=0).ready entity=mgr.24441 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:21:45.063 INFO:tasks.workunit.client.1.smithi123.stdout:9/95: dwrite d4/d17/f1a [0,4194304] 0 2024-02-16T22:21:45.064 INFO:tasks.workunit.client.1.smithi123.stdout:9/96: chown d4/l14 85272 1 2024-02-16T22:21:45.067 INFO:tasks.workunit.client.0.smithi114.stdout:0/53: dread d1/f7 [0,4194304] 0 2024-02-16T22:21:45.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.066+0000 7fa447fff700 1 -- 172.21.15.114:0/2245095767 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182869 (secure 0 0 0) 0x7fa45005a000 con 0x7fa4600fffe0 2024-02-16T22:21:45.068 INFO:tasks.workunit.client.1.smithi123.stdout:9/97: creat d4/d17/f23 x:0 0 0 2024-02-16T22:21:45.068 INFO:tasks.workunit.client.1.smithi123.stdout:6/45: dwrite d0/f2 [0,4194304] 0 2024-02-16T22:21:45.070 INFO:tasks.workunit.client.1.smithi123.stdout:9/98: write d4/fa [277640,1041] 0 2024-02-16T22:21:45.070 INFO:tasks.workunit.client.1.smithi123.stdout:9/99: chown d4/f10 882 1 2024-02-16T22:21:45.078 INFO:tasks.workunit.client.1.smithi123.stdout:6/46: dread - d0/d6/fb zero size 2024-02-16T22:21:45.078 INFO:tasks.workunit.client.0.smithi114.stdout:0/54: link d1/cd d1/c10 0 2024-02-16T22:21:45.080 INFO:tasks.workunit.client.1.smithi123.stdout:6/47: symlink d0/d6/l10 0 2024-02-16T22:21:45.080 INFO:tasks.workunit.client.1.smithi123.stdout:4/84: truncate d5/f14 4074078 0 2024-02-16T22:21:45.081 INFO:tasks.workunit.client.1.smithi123.stdout:4/85: fdatasync d5/f19 0 2024-02-16T22:21:45.081 INFO:tasks.workunit.client.0.smithi114.stdout:0/55: symlink d1/l11 0 2024-02-16T22:21:45.082 INFO:tasks.workunit.client.1.smithi123.stdout:6/48: truncate d0/d6/fa 373977 0 2024-02-16T22:21:45.083 INFO:tasks.workunit.client.1.smithi123.stdout:6/49: write d0/d6/fb [312391,120592] 0 2024-02-16T22:21:45.083 INFO:tasks.workunit.client.0.smithi114.stdout:0/56: mknod d1/c12 0 2024-02-16T22:21:45.084 INFO:tasks.workunit.client.0.smithi114.stdout:0/57: write d1/fe [588901,50018] 0 2024-02-16T22:21:45.084 INFO:tasks.workunit.client.1.smithi123.stdout:1/54: getdents d3 0 2024-02-16T22:21:45.088 INFO:tasks.workunit.client.1.smithi123.stdout:4/86: getdents d5 0 2024-02-16T22:21:45.089 INFO:tasks.workunit.client.1.smithi123.stdout:4/87: write d5/f19 [886201,10963] 0 2024-02-16T22:21:45.091 INFO:tasks.workunit.client.1.smithi123.stdout:4/88: fdatasync d5/f1b 0 2024-02-16T22:21:45.092 INFO:tasks.workunit.client.1.smithi123.stdout:4/89: chown d5/le 2015833 1 2024-02-16T22:21:45.092 INFO:tasks.workunit.client.1.smithi123.stdout:1/55: dwrite d3/fc [0,4194304] 0 2024-02-16T22:21:45.096 INFO:tasks.workunit.client.1.smithi123.stdout:1/56: dread d3/f7 [0,4194304] 0 2024-02-16T22:21:45.115 INFO:tasks.workunit.client.1.smithi123.stdout:5/45: sync 2024-02-16T22:21:45.115 INFO:tasks.workunit.client.1.smithi123.stdout:2/57: sync 2024-02-16T22:21:45.116 INFO:tasks.workunit.client.1.smithi123.stdout:2/58: chown d4/d8/f9 14339084 1 2024-02-16T22:21:45.116 INFO:tasks.workunit.client.1.smithi123.stdout:2/59: read d4/fc [2786750,61004] 0 2024-02-16T22:21:45.141 INFO:tasks.workunit.client.1.smithi123.stdout:6/50: creat d0/d9/dc/f11 x:0 0 0 2024-02-16T22:21:45.143 INFO:tasks.workunit.client.1.smithi123.stdout:0/65: read d1/f7 [65978,113575] 0 2024-02-16T22:21:45.144 INFO:tasks.workunit.client.1.smithi123.stdout:4/90: mkdir d5/d1c 0 2024-02-16T22:21:45.145 INFO:tasks.workunit.client.1.smithi123.stdout:0/66: readlink d1/l9 0 2024-02-16T22:21:45.149 INFO:tasks.workunit.client.1.smithi123.stdout:6/51: dwrite d0/ff [0,4194304] 0 2024-02-16T22:21:45.152 INFO:tasks.workunit.client.1.smithi123.stdout:4/91: dwrite f3 [0,4194304] 0 2024-02-16T22:21:45.156 INFO:tasks.workunit.client.1.smithi123.stdout:6/52: dwrite d0/d6/fd [0,4194304] 0 2024-02-16T22:21:45.163 INFO:tasks.workunit.client.1.smithi123.stdout:1/57: rename d3/da to d3/d6/d11 0 2024-02-16T22:21:45.167 INFO:tasks.workunit.client.1.smithi123.stdout:3/57: getdents d0 0 2024-02-16T22:21:45.175 INFO:tasks.workunit.client.1.smithi123.stdout:9/100: fsync d4/d17/f23 0 2024-02-16T22:21:45.176 INFO:tasks.workunit.client.1.smithi123.stdout:8/70: dwrite d0/fd [0,4194304] 0 2024-02-16T22:21:45.179 INFO:tasks.workunit.client.1.smithi123.stdout:4/92: creat d5/f1d x:0 0 0 2024-02-16T22:21:45.184 INFO:tasks.workunit.client.1.smithi123.stdout:5/46: symlink d6/dd/le 0 2024-02-16T22:21:45.186 INFO:tasks.workunit.client.1.smithi123.stdout:3/58: mknod d0/c11 0 2024-02-16T22:21:45.188 INFO:tasks.workunit.client.1.smithi123.stdout:1/58: dwrite d3/f5 [8388608,4194304] 0 2024-02-16T22:21:45.189 INFO:tasks.workunit.client.0.smithi114.stdout:0/58: getdents d1 0 2024-02-16T22:21:45.190 INFO:tasks.workunit.client.0.smithi114.stdout:0/59: truncate d1/f8 2056517 0 2024-02-16T22:21:45.191 INFO:tasks.workunit.client.1.smithi123.stdout:9/101: chown d4/c11 3911 1 2024-02-16T22:21:45.193 INFO:tasks.workunit.client.1.smithi123.stdout:8/71: creat d0/df/f16 x:0 0 0 2024-02-16T22:21:45.195 INFO:tasks.workunit.client.1.smithi123.stdout:8/72: chown d0/df/f13 61182103 1 2024-02-16T22:21:45.197 INFO:tasks.workunit.client.1.smithi123.stdout:6/53: unlink d0/f5 0 2024-02-16T22:21:45.198 INFO:tasks.workunit.client.0.smithi114.stdout:0/60: getdents d1 0 2024-02-16T22:21:45.198 INFO:tasks.workunit.client.1.smithi123.stdout:8/73: chown d0/c9 1279836 1 2024-02-16T22:21:45.198 INFO:tasks.workunit.client.1.smithi123.stdout:6/54: stat d0/d6/c7 0 2024-02-16T22:21:45.200 INFO:tasks.workunit.client.1.smithi123.stdout:7/46: dwrite d3/f8 [0,4194304] 0 2024-02-16T22:21:45.200 INFO:tasks.workunit.client.1.smithi123.stdout:5/47: mkdir d6/dd/df 0 2024-02-16T22:21:45.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:44 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/3369193630' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:21:45.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:44 smithi114 ceph-mon[29235]: pgmap v20: 65 pgs: 65 active+clean; 221 MiB data, 883 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.0 MiB/s wr, 292 op/s 2024-02-16T22:21:45.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:44 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/2505496517' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:21:45.203 INFO:tasks.workunit.client.1.smithi123.stdout:0/67: dwrite d1/f7 [0,4194304] 0 2024-02-16T22:21:45.204 INFO:tasks.workunit.client.1.smithi123.stdout:7/47: write d3/d5/ff [733479,51126] 0 2024-02-16T22:21:45.204 INFO:tasks.workunit.client.1.smithi123.stdout:2/60: write f2 [2602167,6694] 0 2024-02-16T22:21:45.205 INFO:tasks.workunit.client.1.smithi123.stdout:7/48: dread - d3/fd zero size 2024-02-16T22:21:45.205 INFO:tasks.workunit.client.1.smithi123.stdout:0/68: chown d1/cc 42 1 2024-02-16T22:21:45.205 INFO:tasks.workunit.client.0.smithi114.stdout:0/61: fdatasync d1/fa 0 2024-02-16T22:21:45.207 INFO:tasks.workunit.client.0.smithi114.stdout:0/62: write d1/f8 [758235,113514] 0 2024-02-16T22:21:45.209 INFO:tasks.workunit.client.1.smithi123.stdout:7/49: write d3/d5/ff [1319075,7038] 0 2024-02-16T22:21:45.210 INFO:tasks.workunit.client.1.smithi123.stdout:7/50: write d3/d5/ff [829175,14181] 0 2024-02-16T22:21:45.212 INFO:tasks.workunit.client.1.smithi123.stdout:8/74: mkdir d0/d17 0 2024-02-16T22:21:45.213 INFO:tasks.workunit.client.0.smithi114.stdout:0/63: rmdir d1 39 2024-02-16T22:21:45.218 INFO:tasks.workunit.client.1.smithi123.stdout:6/55: rename d0/d6/c7 to d0/d9/dc/c12 0 2024-02-16T22:21:45.218 INFO:tasks.workunit.client.1.smithi123.stdout:6/56: fsync d0/d6/fb 0 2024-02-16T22:21:45.219 INFO:tasks.workunit.client.1.smithi123.stdout:6/57: fdatasync d0/d9/dc/f11 0 2024-02-16T22:21:45.220 INFO:tasks.workunit.client.1.smithi123.stdout:6/58: chown d0/d6/f8 23 1 2024-02-16T22:21:45.225 INFO:tasks.workunit.client.1.smithi123.stdout:6/59: dwrite d0/d6/fd [0,4194304] 0 2024-02-16T22:21:45.228 INFO:tasks.workunit.client.1.smithi123.stdout:2/61: fdatasync f2 0 2024-02-16T22:21:45.228 INFO:tasks.workunit.client.1.smithi123.stdout:7/51: mkdir d3/d5/d13 0 2024-02-16T22:21:45.229 INFO:tasks.workunit.client.1.smithi123.stdout:2/62: chown f2 990882186 1 2024-02-16T22:21:45.229 INFO:tasks.workunit.client.1.smithi123.stdout:2/63: fsync f2 0 2024-02-16T22:21:45.229 INFO:tasks.workunit.client.1.smithi123.stdout:7/52: write d3/d5/f9 [1188025,13383] 0 2024-02-16T22:21:45.234 INFO:tasks.workunit.client.1.smithi123.stdout:5/48: creat d6/dd/df/f10 x:0 0 0 2024-02-16T22:21:45.235 INFO:tasks.workunit.client.1.smithi123.stdout:1/59: link d3/d6/d11/cd d3/d6/db/c12 0 2024-02-16T22:21:45.239 INFO:tasks.workunit.client.1.smithi123.stdout:4/93: getdents d5 0 2024-02-16T22:21:45.240 INFO:tasks.workunit.client.1.smithi123.stdout:7/53: dwrite d3/fe [0,4194304] 0 2024-02-16T22:21:45.242 INFO:tasks.workunit.client.1.smithi123.stdout:3/59: rename d0/f3 to d0/f12 0 2024-02-16T22:21:45.244 INFO:tasks.workunit.client.1.smithi123.stdout:6/60: chown d0/d9/dc/c12 0 1 2024-02-16T22:21:45.245 INFO:tasks.workunit.client.1.smithi123.stdout:9/102: dwrite d4/f21 [0,4194304] 0 2024-02-16T22:21:45.245 INFO:tasks.workunit.client.1.smithi123.stdout:9/103: fdatasync d4/fa 0 2024-02-16T22:21:45.246 INFO:tasks.workunit.client.1.smithi123.stdout:5/49: creat d6/f11 x:0 0 0 2024-02-16T22:21:45.246 INFO:tasks.workunit.client.1.smithi123.stdout:9/104: readlink d4/l15 0 2024-02-16T22:21:45.248 INFO:tasks.workunit.client.1.smithi123.stdout:0/69: creat d1/fe x:0 0 0 2024-02-16T22:21:45.249 INFO:tasks.workunit.client.0.smithi114.stdout:0/64: truncate d1/f7 3232440 0 2024-02-16T22:21:45.250 INFO:tasks.workunit.client.1.smithi123.stdout:1/60: fsync d3/f7 0 2024-02-16T22:21:45.251 INFO:tasks.workunit.client.1.smithi123.stdout:2/64: mknod d4/d11/c12 0 2024-02-16T22:21:45.252 INFO:tasks.workunit.client.1.smithi123.stdout:2/65: chown d4/d11 2170764 1 2024-02-16T22:21:45.253 INFO:tasks.workunit.client.1.smithi123.stdout:2/66: fdatasync d4/d8/f9 0 2024-02-16T22:21:45.254 INFO:tasks.workunit.client.1.smithi123.stdout:2/67: write d4/d8/f9 [2507961,98975] 0 2024-02-16T22:21:45.254 INFO:tasks.workunit.client.1.smithi123.stdout:2/68: readlink d4/d8/l10 0 2024-02-16T22:21:45.255 INFO:tasks.workunit.client.0.smithi114.stdout:0/65: rename d1/l11 to d1/l13 0 2024-02-16T22:21:45.256 INFO:tasks.workunit.client.1.smithi123.stdout:8/75: write d0/df/f13 [1734198,63995] 0 2024-02-16T22:21:45.261 INFO:tasks.workunit.client.1.smithi123.stdout:6/61: unlink d0/ff 0 2024-02-16T22:21:45.262 INFO:tasks.workunit.client.0.smithi114.stdout:0/66: dread d1/f8 [0,4194304] 0 2024-02-16T22:21:45.264 INFO:tasks.workunit.client.1.smithi123.stdout:9/105: mkdir d4/d17/d24 0 2024-02-16T22:21:45.266 INFO:tasks.workunit.client.1.smithi123.stdout:3/60: dwrite d0/fe [0,4194304] 0 2024-02-16T22:21:45.267 INFO:tasks.workunit.client.0.smithi114.stdout:0/67: rename c0 to d1/c14 0 2024-02-16T22:21:45.271 INFO:tasks.workunit.client.0.smithi114.stdout:0/68: creat d1/f15 x:0 0 0 2024-02-16T22:21:45.272 INFO:tasks.workunit.client.1.smithi123.stdout:7/54: write f2 [1643440,50497] 0 2024-02-16T22:21:45.272 INFO:tasks.workunit.client.1.smithi123.stdout:9/106: dwrite d4/f16 [0,4194304] 0 2024-02-16T22:21:45.275 INFO:tasks.workunit.client.1.smithi123.stdout:7/55: truncate d3/f8 5180749 0 2024-02-16T22:21:45.275 INFO:tasks.workunit.client.1.smithi123.stdout:8/76: mknod d0/c18 0 2024-02-16T22:21:45.277 INFO:tasks.workunit.client.1.smithi123.stdout:2/69: dread d4/fc [0,4194304] 0 2024-02-16T22:21:45.278 INFO:tasks.workunit.client.1.smithi123.stdout:2/70: chown d4 444 1 2024-02-16T22:21:45.279 INFO:tasks.workunit.client.0.smithi114.stdout:0/69: dwrite d1/fe [0,4194304] 0 2024-02-16T22:21:45.279 INFO:tasks.workunit.client.1.smithi123.stdout:6/62: mkdir d0/d6/d13 0 2024-02-16T22:21:45.280 INFO:tasks.workunit.client.0.smithi114.stdout:0/70: read d1/fe [3303227,3027] 0 2024-02-16T22:21:45.289 INFO:tasks.workunit.client.0.smithi114.stdout:0/71: dwrite d1/fe [0,4194304] 0 2024-02-16T22:21:45.289 INFO:tasks.workunit.client.1.smithi123.stdout:4/94: dwrite d5/f17 [0,4194304] 0 2024-02-16T22:21:45.290 INFO:tasks.workunit.client.0.smithi114.stdout:0/72: read d1/f8 [1693871,48207] 0 2024-02-16T22:21:45.294 INFO:tasks.workunit.client.0.smithi114.stdout:0/73: readlink d1/l13 0 2024-02-16T22:21:45.300 INFO:tasks.workunit.client.1.smithi123.stdout:8/77: creat d0/d12/f19 x:0 0 0 2024-02-16T22:21:45.301 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:44 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/3369193630' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:21:45.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:44 smithi123 ceph-mon[37161]: pgmap v20: 65 pgs: 65 active+clean; 221 MiB data, 883 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.0 MiB/s wr, 292 op/s 2024-02-16T22:21:45.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:44 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/2505496517' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:21:45.303 INFO:tasks.workunit.client.0.smithi114.stdout:0/74: dwrite d1/f15 [0,4194304] 0 2024-02-16T22:21:45.304 INFO:tasks.workunit.client.1.smithi123.stdout:5/50: truncate d6/f9 3416346 0 2024-02-16T22:21:45.305 INFO:tasks.workunit.client.1.smithi123.stdout:6/63: mknod d0/d6/c14 0 2024-02-16T22:21:45.309 INFO:tasks.workunit.client.0.smithi114.stdout:0/75: mkdir d1/d16 0 2024-02-16T22:21:45.311 INFO:tasks.workunit.client.1.smithi123.stdout:4/95: creat d5/f1e x:0 0 0 2024-02-16T22:21:45.311 INFO:tasks.workunit.client.1.smithi123.stdout:4/96: stat d5/d1c 0 2024-02-16T22:21:45.312 INFO:tasks.workunit.client.1.smithi123.stdout:1/61: link d3/d6/c9 d3/d6/db/c13 0 2024-02-16T22:21:45.313 INFO:tasks.workunit.client.0.smithi114.stdout:0/76: fsync d1/f8 0 2024-02-16T22:21:45.313 INFO:tasks.workunit.client.1.smithi123.stdout:9/107: getdents d4/d1e 0 2024-02-16T22:21:45.314 INFO:tasks.workunit.client.1.smithi123.stdout:2/71: write d4/f6 [327940,52716] 0 2024-02-16T22:21:45.314 INFO:tasks.workunit.client.1.smithi123.stdout:2/72: fsync f2 0 2024-02-16T22:21:45.315 INFO:tasks.workunit.client.0.smithi114.stdout:0/77: dread d1/fa [0,4194304] 0 2024-02-16T22:21:45.316 INFO:tasks.workunit.client.0.smithi114.stdout:0/78: readlink d1/lc 0 2024-02-16T22:21:45.318 INFO:tasks.workunit.client.1.smithi123.stdout:2/73: dread d4/fc [0,4194304] 0 2024-02-16T22:21:45.318 INFO:tasks.workunit.client.1.smithi123.stdout:8/78: symlink d0/l1a 0 2024-02-16T22:21:45.320 INFO:tasks.workunit.client.0.smithi114.stdout:0/79: creat d1/d16/f17 x:0 0 0 2024-02-16T22:21:45.321 INFO:tasks.workunit.client.1.smithi123.stdout:6/64: creat d0/d6/f15 x:0 0 0 2024-02-16T22:21:45.321 INFO:tasks.workunit.client.1.smithi123.stdout:8/79: dread d0/df/f13 [0,4194304] 0 2024-02-16T22:21:45.322 INFO:tasks.workunit.client.1.smithi123.stdout:3/61: creat d0/f13 x:0 0 0 2024-02-16T22:21:45.322 INFO:tasks.workunit.client.0.smithi114.stdout:0/80: mkdir d1/d16/d18 0 2024-02-16T22:21:45.326 INFO:tasks.workunit.client.1.smithi123.stdout:7/56: rmdir d3/d5/d13 0 2024-02-16T22:21:45.327 INFO:tasks.workunit.client.0.smithi114.stdout:0/81: dread d1/f7 [0,4194304] 0 2024-02-16T22:21:45.327 INFO:tasks.workunit.client.1.smithi123.stdout:2/74: mkdir d4/d8/d13 0 2024-02-16T22:21:45.330 INFO:tasks.workunit.client.1.smithi123.stdout:2/75: dread f2 [0,4194304] 0 2024-02-16T22:21:45.330 INFO:tasks.workunit.client.1.smithi123.stdout:5/51: unlink d6/f9 0 2024-02-16T22:21:45.330 INFO:tasks.workunit.client.1.smithi123.stdout:2/76: truncate d4/d8/f9 4863971 0 2024-02-16T22:21:45.331 INFO:tasks.workunit.client.1.smithi123.stdout:2/77: write f2 [3181089,93644] 0 2024-02-16T22:21:45.333 INFO:tasks.workunit.client.0.smithi114.stdout:0/82: dread d1/f8 [0,4194304] 0 2024-02-16T22:21:45.334 INFO:tasks.workunit.client.1.smithi123.stdout:3/62: chown d0/la 67 1 2024-02-16T22:21:45.335 INFO:tasks.workunit.client.1.smithi123.stdout:4/97: link d5/f1d d5/f1f 0 2024-02-16T22:21:45.337 INFO:tasks.workunit.client.1.smithi123.stdout:9/108: creat d4/d1e/f25 x:0 0 0 2024-02-16T22:21:45.337 INFO:tasks.workunit.client.1.smithi123.stdout:8/80: dwrite d0/f14 [0,4194304] 0 2024-02-16T22:21:45.339 INFO:tasks.workunit.client.1.smithi123.stdout:5/52: creat d6/f12 x:0 0 0 2024-02-16T22:21:45.340 INFO:tasks.workunit.client.1.smithi123.stdout:6/65: creat d0/d6/d13/f16 x:0 0 0 2024-02-16T22:21:45.341 INFO:tasks.workunit.client.1.smithi123.stdout:2/78: symlink d4/d8/l14 0 2024-02-16T22:21:45.341 INFO:tasks.workunit.client.1.smithi123.stdout:2/79: stat d4/d11 0 2024-02-16T22:21:45.346 INFO:tasks.workunit.client.0.smithi114.stdout:0/83: creat d1/d16/f19 x:0 0 0 2024-02-16T22:21:45.346 INFO:tasks.workunit.client.0.smithi114.stdout:0/84: chown d1/f15 115410745 1 2024-02-16T22:21:45.347 INFO:tasks.workunit.client.1.smithi123.stdout:4/98: rename d5/c18 to d5/c20 0 2024-02-16T22:21:45.347 INFO:tasks.workunit.client.0.smithi114.stdout:0/85: readlink d1/l6 0 2024-02-16T22:21:45.348 INFO:tasks.workunit.client.1.smithi123.stdout:7/57: creat d3/d12/f14 x:0 0 0 2024-02-16T22:21:45.349 INFO:tasks.workunit.client.0.smithi114.stdout:0/86: mkdir d1/d1a 0 2024-02-16T22:21:45.351 INFO:tasks.workunit.client.1.smithi123.stdout:6/66: creat d0/d9/dc/f17 x:0 0 0 2024-02-16T22:21:45.351 INFO:tasks.workunit.client.1.smithi123.stdout:7/58: dread d3/d5/f9 [4194304,4194304] 0 2024-02-16T22:21:45.351 INFO:tasks.workunit.client.1.smithi123.stdout:6/67: dread - d0/d9/dc/f11 zero size 2024-02-16T22:21:45.351 INFO:tasks.workunit.client.1.smithi123.stdout:7/59: chown d3/d5/c10 8 1 2024-02-16T22:21:45.352 INFO:tasks.workunit.client.0.smithi114.stdout:0/87: mknod d1/d1a/c1b 0 2024-02-16T22:21:45.353 INFO:tasks.workunit.client.1.smithi123.stdout:7/60: stat d3/d5/c10 0 2024-02-16T22:21:45.353 INFO:tasks.workunit.client.1.smithi123.stdout:1/62: link d3/d6/db/c13 d3/d6/d11/c14 0 2024-02-16T22:21:45.353 INFO:tasks.workunit.client.1.smithi123.stdout:6/68: write d0/d6/f15 [118715,54156] 0 2024-02-16T22:21:45.354 INFO:tasks.workunit.client.1.smithi123.stdout:7/61: write d3/d5/ff [1678027,36053] 0 2024-02-16T22:21:45.355 INFO:tasks.workunit.client.1.smithi123.stdout:8/81: link d0/df/f13 d0/df/f1b 0 2024-02-16T22:21:45.356 INFO:tasks.workunit.client.0.smithi114.stdout:0/88: unlink d1/l6 0 2024-02-16T22:21:45.360 INFO:tasks.workunit.client.1.smithi123.stdout:2/80: mkdir d4/d8/d13/d15 0 2024-02-16T22:21:45.360 INFO:tasks.workunit.client.1.smithi123.stdout:6/69: dwrite d0/d6/f15 [0,4194304] 0 2024-02-16T22:21:45.361 INFO:tasks.workunit.client.1.smithi123.stdout:8/82: dread d0/fc [0,4194304] 0 2024-02-16T22:21:45.361 INFO:tasks.workunit.client.0.smithi114.stdout:0/89: creat d1/d16/d18/f1c x:0 0 0 2024-02-16T22:21:45.361 INFO:tasks.workunit.client.1.smithi123.stdout:8/83: chown d0/f6 786147 1 2024-02-16T22:21:45.362 INFO:tasks.workunit.client.1.smithi123.stdout:8/84: chown d0/fd 22921699 1 2024-02-16T22:21:45.363 INFO:tasks.workunit.client.1.smithi123.stdout:4/99: creat d5/d1c/f21 x:0 0 0 2024-02-16T22:21:45.364 INFO:tasks.workunit.client.1.smithi123.stdout:6/70: dread d0/d6/f8 [0,4194304] 0 2024-02-16T22:21:45.369 INFO:tasks.workunit.client.1.smithi123.stdout:7/62: mknod d3/d5/c15 0 2024-02-16T22:21:45.375 INFO:tasks.workunit.client.1.smithi123.stdout:8/85: creat d0/df/f1c x:0 0 0 2024-02-16T22:21:45.377 INFO:tasks.workunit.client.1.smithi123.stdout:1/63: rename d3/d6/d11/cd to d3/d6/db/d10/c15 0 2024-02-16T22:21:45.380 INFO:tasks.workunit.client.1.smithi123.stdout:2/81: link d4/f6 d4/f16 0 2024-02-16T22:21:45.382 INFO:tasks.workunit.client.1.smithi123.stdout:8/86: mkdir d0/df/d1d 0 2024-02-16T22:21:45.382 INFO:tasks.workunit.client.1.smithi123.stdout:5/53: getdents d6 0 2024-02-16T22:21:45.383 INFO:tasks.workunit.client.1.smithi123.stdout:8/87: write d0/d12/f19 [788918,106676] 0 2024-02-16T22:21:45.383 INFO:tasks.workunit.client.1.smithi123.stdout:5/54: fdatasync d6/f12 0 2024-02-16T22:21:45.384 INFO:tasks.workunit.client.1.smithi123.stdout:4/100: rename d5/f1b to d5/d1c/f22 0 2024-02-16T22:21:45.384 INFO:tasks.workunit.client.1.smithi123.stdout:8/88: fdatasync d0/f6 0 2024-02-16T22:21:45.386 INFO:tasks.workunit.client.1.smithi123.stdout:8/89: write d0/df/f1c [63794,112172] 0 2024-02-16T22:21:45.390 INFO:tasks.workunit.client.1.smithi123.stdout:4/101: unlink d5/fc 0 2024-02-16T22:21:45.390 INFO:tasks.workunit.client.1.smithi123.stdout:7/63: link l1 d3/d12/l16 0 2024-02-16T22:21:45.394 INFO:tasks.workunit.client.1.smithi123.stdout:6/71: rename d0/d6/d13/f16 to d0/d6/f18 0 2024-02-16T22:21:45.394 INFO:tasks.workunit.client.1.smithi123.stdout:5/55: dwrite d6/f11 [0,4194304] 0 2024-02-16T22:21:45.394 INFO:tasks.workunit.client.1.smithi123.stdout:7/64: write d3/d12/f14 [90936,8686] 0 2024-02-16T22:21:45.397 INFO:tasks.workunit.client.1.smithi123.stdout:3/63: write d0/fb [1016878,76498] 0 2024-02-16T22:21:45.398 INFO:tasks.workunit.client.1.smithi123.stdout:4/102: dread d5/ff [0,4194304] 0 2024-02-16T22:21:45.398 INFO:tasks.workunit.client.1.smithi123.stdout:7/65: fdatasync d3/fe 0 2024-02-16T22:21:45.400 INFO:tasks.workunit.client.1.smithi123.stdout:6/72: chown d0/d6/fb 12 1 2024-02-16T22:21:45.401 INFO:tasks.workunit.client.1.smithi123.stdout:8/90: dwrite d0/d12/f19 [0,4194304] 0 2024-02-16T22:21:45.405 INFO:tasks.workunit.client.1.smithi123.stdout:5/56: rmdir d6/dd/df 39 2024-02-16T22:21:45.407 INFO:tasks.workunit.client.1.smithi123.stdout:0/70: sync 2024-02-16T22:21:45.409 INFO:tasks.workunit.client.1.smithi123.stdout:4/103: stat d5/c20 0 2024-02-16T22:21:45.410 INFO:tasks.workunit.client.1.smithi123.stdout:4/104: dread - d5/f1d zero size 2024-02-16T22:21:45.411 INFO:tasks.workunit.client.1.smithi123.stdout:7/66: mknod d3/c17 0 2024-02-16T22:21:45.413 INFO:tasks.workunit.client.1.smithi123.stdout:6/73: creat d0/d9/f19 x:0 0 0 2024-02-16T22:21:45.414 INFO:tasks.workunit.client.1.smithi123.stdout:0/71: dwrite d1/f6 [0,4194304] 0 2024-02-16T22:21:45.416 INFO:tasks.workunit.client.1.smithi123.stdout:5/57: creat d6/f13 x:0 0 0 2024-02-16T22:21:45.419 INFO:tasks.workunit.client.1.smithi123.stdout:3/64: symlink d0/l14 0 2024-02-16T22:21:45.422 INFO:tasks.workunit.client.1.smithi123.stdout:7/67: dwrite d3/d12/f14 [0,4194304] 0 2024-02-16T22:21:45.424 INFO:tasks.workunit.client.1.smithi123.stdout:7/68: write d3/fd [797154,75722] 0 2024-02-16T22:21:45.424 INFO:tasks.workunit.client.1.smithi123.stdout:4/105: mkdir d5/d1c/d23 0 2024-02-16T22:21:45.425 INFO:tasks.workunit.client.1.smithi123.stdout:4/106: truncate d5/f12 437102 0 2024-02-16T22:21:45.426 INFO:tasks.workunit.client.1.smithi123.stdout:4/107: truncate d5/d1c/f21 202765 0 2024-02-16T22:21:45.428 INFO:tasks.workunit.client.1.smithi123.stdout:0/72: unlink d1/d4/lb 0 2024-02-16T22:21:45.431 INFO:tasks.workunit.client.1.smithi123.stdout:0/73: dread d1/f6 [0,4194304] 0 2024-02-16T22:21:45.433 INFO:tasks.workunit.client.1.smithi123.stdout:3/65: fdatasync d0/f7 0 2024-02-16T22:21:45.434 INFO:tasks.workunit.client.1.smithi123.stdout:7/69: rmdir d3/d12 39 2024-02-16T22:21:45.436 INFO:tasks.workunit.client.1.smithi123.stdout:4/108: creat d5/d1c/f24 x:0 0 0 2024-02-16T22:21:45.437 INFO:tasks.workunit.client.1.smithi123.stdout:1/64: truncate d3/ff 539799 0 2024-02-16T22:21:45.438 INFO:tasks.workunit.client.1.smithi123.stdout:5/58: rmdir d6/dd/df 39 2024-02-16T22:21:45.441 INFO:tasks.workunit.client.1.smithi123.stdout:2/82: dwrite d4/fc [0,4194304] 0 2024-02-16T22:21:45.441 INFO:tasks.workunit.client.1.smithi123.stdout:7/70: creat d3/d5/f18 x:0 0 0 2024-02-16T22:21:45.442 INFO:tasks.workunit.client.1.smithi123.stdout:0/74: dwrite d1/fe [0,4194304] 0 2024-02-16T22:21:45.446 INFO:tasks.workunit.client.1.smithi123.stdout:1/65: symlink d3/l16 0 2024-02-16T22:21:45.447 INFO:tasks.workunit.client.1.smithi123.stdout:8/91: write d0/df/f13 [4941720,45485] 0 2024-02-16T22:21:45.448 INFO:tasks.workunit.client.1.smithi123.stdout:5/59: symlink d6/l14 0 2024-02-16T22:21:45.450 INFO:tasks.workunit.client.1.smithi123.stdout:6/74: getdents d0/d6/d13 0 2024-02-16T22:21:45.452 INFO:tasks.workunit.client.1.smithi123.stdout:7/71: symlink d3/d5/l19 0 2024-02-16T22:21:45.454 INFO:tasks.workunit.client.1.smithi123.stdout:3/66: creat d0/f15 x:0 0 0 2024-02-16T22:21:45.455 INFO:tasks.workunit.client.1.smithi123.stdout:4/109: rmdir d5 39 2024-02-16T22:21:45.455 INFO:tasks.workunit.client.1.smithi123.stdout:5/60: dread d6/f8 [0,4194304] 0 2024-02-16T22:21:45.456 INFO:tasks.workunit.client.1.smithi123.stdout:2/83: dwrite d4/fc [0,4194304] 0 2024-02-16T22:21:45.456 INFO:tasks.workunit.client.1.smithi123.stdout:6/75: mkdir d0/d6/d1a 0 2024-02-16T22:21:45.458 INFO:tasks.workunit.client.1.smithi123.stdout:5/61: fsync d6/fb 0 2024-02-16T22:21:45.458 INFO:tasks.workunit.client.1.smithi123.stdout:0/75: mknod d1/cf 0 2024-02-16T22:21:45.468 INFO:tasks.workunit.client.1.smithi123.stdout:5/62: symlink d6/l15 0 2024-02-16T22:21:45.468 INFO:tasks.workunit.client.1.smithi123.stdout:1/66: dwrite d3/f8 [0,4194304] 0 2024-02-16T22:21:45.470 INFO:tasks.workunit.client.1.smithi123.stdout:7/72: dwrite d3/d5/f18 [0,4194304] 0 2024-02-16T22:21:45.470 INFO:tasks.workunit.client.1.smithi123.stdout:6/76: symlink d0/l1b 0 2024-02-16T22:21:45.472 INFO:tasks.workunit.client.1.smithi123.stdout:0/76: rename d1/d8/cd to d1/d8/c10 0 2024-02-16T22:21:45.472 INFO:tasks.workunit.client.1.smithi123.stdout:9/109: sync 2024-02-16T22:21:45.473 INFO:tasks.workunit.client.1.smithi123.stdout:4/110: creat d5/d1c/f25 x:0 0 0 2024-02-16T22:21:45.476 INFO:tasks.workunit.client.1.smithi123.stdout:4/111: stat d5/d1c 0 2024-02-16T22:21:45.477 INFO:tasks.workunit.client.1.smithi123.stdout:8/92: getdents d0/df 0 2024-02-16T22:21:45.479 INFO:tasks.workunit.client.1.smithi123.stdout:6/77: rmdir d0/d9/dc 39 2024-02-16T22:21:45.480 INFO:tasks.workunit.client.1.smithi123.stdout:3/67: rename d0/l8 to d0/l16 0 2024-02-16T22:21:45.481 INFO:tasks.workunit.client.1.smithi123.stdout:9/110: creat d4/f26 x:0 0 0 2024-02-16T22:21:45.481 INFO:tasks.workunit.client.1.smithi123.stdout:1/67: dwrite d3/f5 [4194304,4194304] 0 2024-02-16T22:21:45.481 INFO:tasks.workunit.client.1.smithi123.stdout:4/112: creat d5/f26 x:0 0 0 2024-02-16T22:21:45.482 INFO:tasks.workunit.client.1.smithi123.stdout:1/68: readlink l1 0 2024-02-16T22:21:45.482 INFO:tasks.workunit.client.1.smithi123.stdout:0/77: creat d1/d4/f11 x:0 0 0 2024-02-16T22:21:45.482 INFO:tasks.workunit.client.1.smithi123.stdout:9/111: dread - d4/f26 zero size 2024-02-16T22:21:45.483 INFO:tasks.workunit.client.1.smithi123.stdout:7/73: creat d3/d12/f1a x:0 0 0 2024-02-16T22:21:45.485 INFO:tasks.workunit.client.1.smithi123.stdout:5/63: mkdir d6/dd/df/d16 0 2024-02-16T22:21:45.485 INFO:tasks.workunit.client.1.smithi123.stdout:9/112: chown d4/d17/d20/f22 10 1 2024-02-16T22:21:45.485 INFO:tasks.workunit.client.1.smithi123.stdout:6/78: symlink d0/d6/d13/l1c 0 2024-02-16T22:21:45.486 INFO:tasks.workunit.client.1.smithi123.stdout:9/113: dread - d4/f26 zero size 2024-02-16T22:21:45.487 INFO:tasks.workunit.client.1.smithi123.stdout:7/74: truncate d3/fe 4955325 0 2024-02-16T22:21:45.488 INFO:tasks.workunit.client.1.smithi123.stdout:3/68: dread - d0/ff zero size 2024-02-16T22:21:45.488 INFO:tasks.workunit.client.1.smithi123.stdout:9/114: fsync d4/f10 0 2024-02-16T22:21:45.489 INFO:tasks.workunit.client.1.smithi123.stdout:2/84: truncate d4/fc 3832405 0 2024-02-16T22:21:45.490 INFO:tasks.workunit.client.1.smithi123.stdout:6/79: truncate d0/d6/f18 1014725 0 2024-02-16T22:21:45.490 INFO:tasks.workunit.client.1.smithi123.stdout:0/78: unlink d1/f7 0 2024-02-16T22:21:45.490 INFO:tasks.workunit.client.1.smithi123.stdout:8/93: dread d0/fc [0,4194304] 0 2024-02-16T22:21:45.491 INFO:tasks.workunit.client.1.smithi123.stdout:5/64: unlink d6/fb 0 2024-02-16T22:21:45.492 INFO:tasks.workunit.client.1.smithi123.stdout:0/79: read d1/d8/fa [4099064,103034] 0 2024-02-16T22:21:45.492 INFO:tasks.workunit.client.1.smithi123.stdout:6/80: write d0/d6/fb [246228,50944] 0 2024-02-16T22:21:45.492 INFO:tasks.workunit.client.1.smithi123.stdout:4/113: rename d5/la to d5/d1c/l27 0 2024-02-16T22:21:45.494 INFO:tasks.workunit.client.1.smithi123.stdout:2/85: creat d4/d8/f17 x:0 0 0 2024-02-16T22:21:45.496 INFO:tasks.workunit.client.1.smithi123.stdout:8/94: creat d0/d12/f1e x:0 0 0 2024-02-16T22:21:45.496 INFO:tasks.workunit.client.1.smithi123.stdout:5/65: fsync d6/f8 0 2024-02-16T22:21:45.497 INFO:tasks.workunit.client.1.smithi123.stdout:6/81: write d0/d6/f18 [1682924,65142] 0 2024-02-16T22:21:45.497 INFO:tasks.workunit.client.1.smithi123.stdout:9/115: rename d4/c11 to d4/d1e/c27 0 2024-02-16T22:21:45.498 INFO:tasks.workunit.client.1.smithi123.stdout:3/69: dread d0/f12 [0,4194304] 0 2024-02-16T22:21:45.499 INFO:tasks.workunit.client.1.smithi123.stdout:4/114: unlink d5/d1c/f22 0 2024-02-16T22:21:45.499 INFO:tasks.workunit.client.1.smithi123.stdout:6/82: dread - d0/d9/f19 zero size 2024-02-16T22:21:45.500 INFO:tasks.workunit.client.1.smithi123.stdout:2/86: creat d4/d11/f18 x:0 0 0 2024-02-16T22:21:45.502 INFO:tasks.workunit.client.1.smithi123.stdout:8/95: fdatasync d0/fc 0 2024-02-16T22:21:45.502 INFO:tasks.workunit.client.1.smithi123.stdout:6/83: readlink d0/d6/l10 0 2024-02-16T22:21:45.505 INFO:tasks.workunit.client.1.smithi123.stdout:0/80: creat d1/f12 x:0 0 0 2024-02-16T22:21:45.507 INFO:tasks.workunit.client.1.smithi123.stdout:9/116: creat d4/d17/d24/f28 x:0 0 0 2024-02-16T22:21:45.507 INFO:tasks.workunit.client.1.smithi123.stdout:5/66: dwrite d6/f11 [0,4194304] 0 2024-02-16T22:21:45.508 INFO:tasks.workunit.client.1.smithi123.stdout:3/70: creat d0/f17 x:0 0 0 2024-02-16T22:21:45.509 INFO:tasks.workunit.client.1.smithi123.stdout:7/75: fsync d3/d12/f1a 0 2024-02-16T22:21:45.509 INFO:tasks.workunit.client.1.smithi123.stdout:8/96: mknod d0/c1f 0 2024-02-16T22:21:45.510 INFO:tasks.workunit.client.1.smithi123.stdout:3/71: stat d0/fb 0 2024-02-16T22:21:45.511 INFO:tasks.workunit.client.1.smithi123.stdout:6/84: dread d0/d6/fb [0,4194304] 0 2024-02-16T22:21:45.511 INFO:tasks.workunit.client.1.smithi123.stdout:0/81: mkdir d1/d8/d13 0 2024-02-16T22:21:45.513 INFO:tasks.workunit.client.1.smithi123.stdout:5/67: unlink d6/ca 0 2024-02-16T22:21:45.513 INFO:tasks.workunit.client.1.smithi123.stdout:4/115: dread f4 [0,4194304] 0 2024-02-16T22:21:45.513 INFO:tasks.workunit.client.1.smithi123.stdout:8/97: fsync d0/df/f16 0 2024-02-16T22:21:45.515 INFO:tasks.workunit.client.1.smithi123.stdout:2/87: dread f2 [0,4194304] 0 2024-02-16T22:21:45.515 INFO:tasks.workunit.client.1.smithi123.stdout:5/68: dread - d6/f12 zero size 2024-02-16T22:21:45.516 INFO:tasks.workunit.client.1.smithi123.stdout:7/76: creat d3/d12/f1b x:0 0 0 2024-02-16T22:21:45.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.515+0000 7fa465354700 1 -- 172.21.15.114:0/2245095767 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa46019c710 con 0x7fa4600fffe0 2024-02-16T22:21:45.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.515+0000 7fa447fff700 1 -- 172.21.15.114:0/2245095767 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fa450059b90 con 0x7fa4600fffe0 2024-02-16T22:21:45.517 INFO:teuthology.orchestra.run.smithi114.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:21:45.518 INFO:teuthology.orchestra.run.smithi114.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:21:45.518 INFO:teuthology.orchestra.run.smithi114.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:21:45.518 INFO:tasks.workunit.client.1.smithi123.stdout:3/72: read d0/f6 [193770,66925] 0 2024-02-16T22:21:45.519 INFO:tasks.workunit.client.1.smithi123.stdout:6/85: symlink d0/d6/l1d 0 2024-02-16T22:21:45.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.518+0000 7fa465354700 1 -- 172.21.15.114:0/2245095767 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fa44806ce40 msgr2=0x7fa44806f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:45.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.518+0000 7fa465354700 1 --2- 172.21.15.114:0/2245095767 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fa44806ce40 0x7fa44806f300 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa45400b6f0 tx=0x7fa454005fb0 comp rx=0 tx=0).stop 2024-02-16T22:21:45.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.518+0000 7fa465354700 1 -- 172.21.15.114:0/2245095767 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4600fffe0 msgr2=0x7fa4601963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:21:45.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.518+0000 7fa465354700 1 --2- 172.21.15.114:0/2245095767 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4600fffe0 0x7fa4601963a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa45000e8f0 tx=0x7fa45000ec00 comp rx=0 tx=0).stop 2024-02-16T22:21:45.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.518+0000 7fa465354700 1 -- 172.21.15.114:0/2245095767 shutdown_connections 2024-02-16T22:21:45.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.518+0000 7fa465354700 1 --2- 172.21.15.114:0/2245095767 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fa44806ce40 0x7fa44806f300 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:45.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.518+0000 7fa465354700 1 --2- 172.21.15.114:0/2245095767 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4600fffe0 0x7fa4601963a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:45.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.518+0000 7fa465354700 1 --2- 172.21.15.114:0/2245095767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa460100d00 0x7fa4601968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:21:45.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.518+0000 7fa465354700 1 -- 172.21.15.114:0/2245095767 >> 172.21.15.114:0/2245095767 conn(0x7fa4600fa7c0 msgr2=0x7fa460103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:21:45.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.518+0000 7fa465354700 1 -- 172.21.15.114:0/2245095767 shutdown_connections 2024-02-16T22:21:45.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:21:45.518+0000 7fa465354700 1 -- 172.21.15.114:0/2245095767 wait complete. 2024-02-16T22:21:45.521 INFO:tasks.workunit.client.1.smithi123.stdout:3/73: write d0/f17 [83886,27327] 0 2024-02-16T22:21:45.522 INFO:tasks.workunit.client.1.smithi123.stdout:9/117: dwrite d4/f26 [0,4194304] 0 2024-02-16T22:21:45.522 INFO:tasks.workunit.client.1.smithi123.stdout:0/82: rename d1/fe to d1/d8/d13/f14 0 2024-02-16T22:21:45.523 INFO:tasks.workunit.client.1.smithi123.stdout:8/98: symlink d0/l20 0 2024-02-16T22:21:45.524 INFO:tasks.workunit.client.1.smithi123.stdout:8/99: fsync d0/d12/f1e 0 2024-02-16T22:21:45.524 INFO:tasks.workunit.client.1.smithi123.stdout:9/118: write d4/fe [4168714,99965] 0 2024-02-16T22:21:45.532 INFO:tasks.workunit.client.1.smithi123.stdout:4/116: symlink d5/d1c/d23/l28 0 2024-02-16T22:21:45.533 INFO:tasks.workunit.client.1.smithi123.stdout:2/88: symlink d4/d8/d13/d15/l19 0 2024-02-16T22:21:45.536 INFO:tasks.workunit.client.1.smithi123.stdout:5/69: creat d6/dd/df/d16/f17 x:0 0 0 2024-02-16T22:21:45.537 INFO:tasks.workunit.client.1.smithi123.stdout:9/119: creat d4/d17/f29 x:0 0 0 2024-02-16T22:21:45.539 INFO:tasks.workunit.client.1.smithi123.stdout:6/86: link d0/d6/fd d0/d6/f1e 0 2024-02-16T22:21:45.540 INFO:tasks.workunit.client.1.smithi123.stdout:3/74: symlink d0/l18 0 2024-02-16T22:21:45.541 INFO:tasks.workunit.client.1.smithi123.stdout:4/117: rename d5/ff to d5/f29 0 2024-02-16T22:21:45.544 INFO:tasks.workunit.client.1.smithi123.stdout:7/77: write d3/d5/f9 [5624208,28460] 0 2024-02-16T22:21:45.545 INFO:tasks.workunit.client.1.smithi123.stdout:5/70: dwrite d6/f8 [0,4194304] 0 2024-02-16T22:21:45.547 INFO:tasks.workunit.client.1.smithi123.stdout:0/83: write d1/d8/fa [2951594,72529] 0 2024-02-16T22:21:45.547 INFO:tasks.workunit.client.1.smithi123.stdout:8/100: rmdir d0/df/d1d 0 2024-02-16T22:21:45.550 INFO:tasks.workunit.client.1.smithi123.stdout:7/78: write f2 [1618240,113753] 0 2024-02-16T22:21:45.550 INFO:tasks.workunit.client.1.smithi123.stdout:3/75: truncate d0/fc 921156 0 2024-02-16T22:21:45.553 INFO:tasks.workunit.client.1.smithi123.stdout:0/84: read - d1/d4/f11 zero size 2024-02-16T22:21:45.559 INFO:tasks.workunit.client.1.smithi123.stdout:5/71: mknod d6/dd/c18 0 2024-02-16T22:21:45.562 INFO:tasks.workunit.client.1.smithi123.stdout:2/89: dwrite f2 [0,4194304] 0 2024-02-16T22:21:45.562 INFO:tasks.workunit.client.1.smithi123.stdout:0/85: creat d1/d8/d13/f15 x:0 0 0 2024-02-16T22:21:45.564 INFO:tasks.workunit.client.1.smithi123.stdout:9/120: dwrite d4/d17/f1f [0,4194304] 0 2024-02-16T22:21:45.566 INFO:tasks.workunit.client.1.smithi123.stdout:7/79: creat d3/f1c x:0 0 0 2024-02-16T22:21:45.566 INFO:tasks.workunit.client.1.smithi123.stdout:4/118: dwrite d5/f12 [0,4194304] 0 2024-02-16T22:21:45.568 INFO:tasks.workunit.client.1.smithi123.stdout:5/72: mknod d6/dd/df/c19 0 2024-02-16T22:21:45.571 INFO:tasks.workunit.client.1.smithi123.stdout:2/90: mkdir d4/d1a 0 2024-02-16T22:21:45.572 INFO:tasks.workunit.client.1.smithi123.stdout:2/91: chown d4 1 1 2024-02-16T22:21:45.572 INFO:tasks.workunit.client.1.smithi123.stdout:3/76: dread d0/f7 [0,4194304] 0 2024-02-16T22:21:45.572 INFO:tasks.workunit.client.1.smithi123.stdout:4/119: dread - d5/d1c/f24 zero size 2024-02-16T22:21:45.572 INFO:tasks.workunit.client.1.smithi123.stdout:9/121: dread - d4/f1b zero size 2024-02-16T22:21:45.574 INFO:tasks.workunit.client.1.smithi123.stdout:7/80: mkdir d3/d12/d1d 0 2024-02-16T22:21:45.575 INFO:tasks.workunit.client.1.smithi123.stdout:8/101: dwrite d0/f4 [0,4194304] 0 2024-02-16T22:21:45.575 INFO:tasks.workunit.client.1.smithi123.stdout:4/120: read d5/f14 [1773849,17092] 0 2024-02-16T22:21:45.576 INFO:tasks.workunit.client.1.smithi123.stdout:9/122: write d4/f16 [3642995,57223] 0 2024-02-16T22:21:45.577 INFO:tasks.workunit.client.1.smithi123.stdout:5/73: unlink d6/dd/df/f10 0 2024-02-16T22:21:45.577 INFO:tasks.workunit.client.1.smithi123.stdout:4/121: chown d5/ld 11 1 2024-02-16T22:21:45.581 INFO:tasks.workunit.client.1.smithi123.stdout:5/74: write d6/f12 [353416,11090] 0 2024-02-16T22:21:45.584 INFO:tasks.workunit.client.1.smithi123.stdout:5/75: fsync d6/f11 0 2024-02-16T22:21:45.584 INFO:tasks.workunit.client.1.smithi123.stdout:8/102: mknod d0/d17/c21 0 2024-02-16T22:21:45.585 INFO:tasks.workunit.client.1.smithi123.stdout:4/122: dread f3 [0,4194304] 0 2024-02-16T22:21:45.586 INFO:tasks.workunit.client.1.smithi123.stdout:7/81: mkdir d3/d12/d1d/d1e 0 2024-02-16T22:21:45.595 INFO:tasks.workunit.client.1.smithi123.stdout:3/77: creat d0/f19 x:0 0 0 2024-02-16T22:21:45.596 INFO:tasks.workunit.client.1.smithi123.stdout:4/123: write d5/f26 [353547,97834] 0 2024-02-16T22:21:45.596 INFO:tasks.workunit.client.1.smithi123.stdout:8/103: creat d0/d12/f22 x:0 0 0 2024-02-16T22:21:45.599 INFO:tasks.workunit.client.1.smithi123.stdout:8/104: stat d0/l20 0 2024-02-16T22:21:45.599 INFO:tasks.workunit.client.1.smithi123.stdout:2/92: write d4/f6 [945084,113312] 0 2024-02-16T22:21:45.599 INFO:tasks.workunit.client.1.smithi123.stdout:7/82: truncate f2 2162504 0 2024-02-16T22:21:45.599 INFO:tasks.workunit.client.1.smithi123.stdout:9/123: dwrite d4/f26 [0,4194304] 0 2024-02-16T22:21:45.601 INFO:tasks.workunit.client.1.smithi123.stdout:3/78: dread d0/f12 [0,4194304] 0 2024-02-16T22:21:45.601 INFO:tasks.workunit.client.1.smithi123.stdout:2/93: chown d4/d8 34223348 1 2024-02-16T22:21:45.601 INFO:tasks.workunit.client.1.smithi123.stdout:9/124: read d4/f21 [2596277,85118] 0 2024-02-16T22:21:45.602 INFO:tasks.workunit.client.1.smithi123.stdout:3/79: fsync d0/fd 0 2024-02-16T22:21:45.603 INFO:tasks.workunit.client.1.smithi123.stdout:5/76: dread d6/f11 [0,4194304] 0 2024-02-16T22:21:45.604 INFO:tasks.workunit.client.1.smithi123.stdout:3/80: write d0/ff [690266,17645] 0 2024-02-16T22:21:45.605 INFO:tasks.workunit.client.1.smithi123.stdout:3/81: chown d0/fe 2317801 1 2024-02-16T22:21:45.626 INFO:tasks.workunit.client.1.smithi123.stdout:1/69: sync 2024-02-16T22:21:45.680 INFO:tasks.workunit.client.1.smithi123.stdout:8/105: mknod d0/c23 0 2024-02-16T22:21:45.683 INFO:tasks.workunit.client.1.smithi123.stdout:2/94: mkdir d4/d8/d13/d15/d1b 0 2024-02-16T22:21:45.683 INFO:tasks.workunit.client.1.smithi123.stdout:5/77: rmdir d6/dd 39 2024-02-16T22:21:45.685 INFO:tasks.workunit.client.1.smithi123.stdout:3/82: unlink d0/f13 0 2024-02-16T22:21:45.685 INFO:tasks.workunit.client.1.smithi123.stdout:2/95: chown l3 63698 1 2024-02-16T22:21:45.686 INFO:tasks.workunit.client.1.smithi123.stdout:1/70: creat d3/d6/d11/f17 x:0 0 0 2024-02-16T22:21:45.686 INFO:tasks.workunit.client.1.smithi123.stdout:7/83: mknod d3/c1f 0 2024-02-16T22:21:45.686 INFO:tasks.workunit.client.1.smithi123.stdout:2/96: stat l3 0 2024-02-16T22:21:45.687 INFO:tasks.workunit.client.1.smithi123.stdout:4/124: dwrite f3 [4194304,4194304] 0 2024-02-16T22:21:45.687 INFO:tasks.workunit.client.1.smithi123.stdout:8/106: dread d0/fc [0,4194304] 0 2024-02-16T22:21:45.688 INFO:tasks.workunit.client.1.smithi123.stdout:2/97: chown d4/d8/d13/d15 474107575 1 2024-02-16T22:21:45.688 INFO:tasks.workunit.client.1.smithi123.stdout:8/107: chown d0/c1f 0 1 2024-02-16T22:21:45.691 INFO:tasks.workunit.client.1.smithi123.stdout:1/71: mkdir d3/d6/d18 0 2024-02-16T22:21:45.692 INFO:tasks.workunit.client.1.smithi123.stdout:7/84: dread d3/d12/f14 [0,4194304] 0 2024-02-16T22:21:45.692 INFO:tasks.workunit.client.1.smithi123.stdout:7/85: chown d3/cb 24992 1 2024-02-16T22:21:45.692 INFO:tasks.workunit.client.1.smithi123.stdout:9/125: link l1 d4/d17/d20/l2a 0 2024-02-16T22:21:45.696 INFO:tasks.workunit.client.1.smithi123.stdout:5/78: creat d6/dd/df/d16/f1a x:0 0 0 2024-02-16T22:21:45.697 INFO:tasks.workunit.client.1.smithi123.stdout:3/83: mknod d0/c1a 0 2024-02-16T22:21:45.697 INFO:tasks.workunit.client.1.smithi123.stdout:6/87: sync 2024-02-16T22:21:45.700 INFO:tasks.workunit.client.1.smithi123.stdout:4/125: fdatasync d5/f29 0 2024-02-16T22:21:45.701 INFO:tasks.workunit.client.1.smithi123.stdout:2/98: creat d4/d8/d13/d15/d1b/f1c x:0 0 0 2024-02-16T22:21:45.704 INFO:tasks.workunit.client.1.smithi123.stdout:7/86: dwrite d3/f1c [0,4194304] 0 2024-02-16T22:21:45.704 INFO:tasks.workunit.client.1.smithi123.stdout:1/72: getdents d3/d6/d18 0 2024-02-16T22:21:45.704 INFO:tasks.workunit.client.1.smithi123.stdout:3/84: stat d0/l16 0 2024-02-16T22:21:45.704 INFO:tasks.workunit.client.1.smithi123.stdout:4/126: chown d5/f29 748875208 1 2024-02-16T22:21:45.706 INFO:tasks.workunit.client.1.smithi123.stdout:9/126: dwrite d4/d17/f23 [0,4194304] 0 2024-02-16T22:21:45.706 INFO:tasks.workunit.client.1.smithi123.stdout:4/127: stat d5/f14 0 2024-02-16T22:21:45.707 INFO:tasks.workunit.client.1.smithi123.stdout:8/108: link d0/c9 d0/df/c24 0 2024-02-16T22:21:45.710 INFO:tasks.workunit.client.1.smithi123.stdout:7/87: rmdir d3/d12 39 2024-02-16T22:21:45.710 INFO:tasks.workunit.client.1.smithi123.stdout:6/88: dread d0/d6/fd [0,4194304] 0 2024-02-16T22:21:45.712 INFO:tasks.workunit.client.1.smithi123.stdout:1/73: dread d3/f5 [4194304,4194304] 0 2024-02-16T22:21:45.715 INFO:tasks.workunit.client.1.smithi123.stdout:9/127: mkdir d4/d17/d2b 0 2024-02-16T22:21:45.716 INFO:tasks.workunit.client.1.smithi123.stdout:9/128: chown d4/d17/f1d 899 1 2024-02-16T22:21:45.717 INFO:tasks.workunit.client.1.smithi123.stdout:8/109: mkdir d0/df/d25 0 2024-02-16T22:21:45.719 INFO:tasks.workunit.client.1.smithi123.stdout:3/85: mknod d0/c1b 0 2024-02-16T22:21:45.722 INFO:tasks.workunit.client.1.smithi123.stdout:1/74: mknod d3/c19 0 2024-02-16T22:21:45.724 INFO:tasks.workunit.client.1.smithi123.stdout:0/86: sync 2024-02-16T22:21:45.724 INFO:tasks.workunit.client.1.smithi123.stdout:9/129: mknod d4/d1e/c2c 0 2024-02-16T22:21:45.725 INFO:tasks.workunit.client.1.smithi123.stdout:5/79: getdents d6/dd/df 0 2024-02-16T22:21:45.725 INFO:tasks.workunit.client.1.smithi123.stdout:9/130: stat d4/c7 0 2024-02-16T22:21:45.725 INFO:tasks.workunit.client.1.smithi123.stdout:8/110: mkdir d0/d17/d26 0 2024-02-16T22:21:45.726 INFO:tasks.workunit.client.1.smithi123.stdout:0/87: truncate d1/d8/d13/f15 611358 0 2024-02-16T22:21:45.727 INFO:tasks.workunit.client.1.smithi123.stdout:8/111: fsync d0/df/f1b 0 2024-02-16T22:21:45.727 INFO:tasks.workunit.client.1.smithi123.stdout:9/131: truncate d4/d17/f29 1003913 0 2024-02-16T22:21:45.728 INFO:tasks.workunit.client.1.smithi123.stdout:8/112: fdatasync d0/df/f1b 0 2024-02-16T22:21:45.748 INFO:tasks.workunit.client.1.smithi123.stdout:7/88: mknod d3/c20 0 2024-02-16T22:21:45.750 INFO:tasks.workunit.client.1.smithi123.stdout:5/80: creat d6/dd/df/f1b x:0 0 0 2024-02-16T22:21:45.754 INFO:tasks.workunit.client.1.smithi123.stdout:9/132: write d4/f21 [4670463,7183] 0 2024-02-16T22:21:45.756 INFO:tasks.workunit.client.1.smithi123.stdout:8/113: creat d0/df/d25/f27 x:0 0 0 2024-02-16T22:21:45.758 INFO:tasks.workunit.client.1.smithi123.stdout:8/114: creat d0/df/d25/f28 x:0 0 0 2024-02-16T22:21:45.760 INFO:tasks.workunit.client.1.smithi123.stdout:9/133: dwrite d4/fa [0,4194304] 0 2024-02-16T22:21:45.761 INFO:tasks.workunit.client.1.smithi123.stdout:8/115: mkdir d0/d29 0 2024-02-16T22:21:45.762 INFO:tasks.workunit.client.1.smithi123.stdout:9/134: mknod d4/d1e/c2d 0 2024-02-16T22:21:45.769 INFO:tasks.workunit.client.1.smithi123.stdout:9/135: dwrite d4/f10 [0,4194304] 0 2024-02-16T22:21:45.772 INFO:tasks.workunit.client.1.smithi123.stdout:9/136: symlink d4/d17/d20/l2e 0 2024-02-16T22:21:45.772 INFO:tasks.workunit.client.1.smithi123.stdout:9/137: chown d4/d1e/c2c 48324 1 2024-02-16T22:21:45.775 INFO:tasks.workunit.client.1.smithi123.stdout:9/138: symlink d4/d17/d2b/l2f 0 2024-02-16T22:21:45.781 INFO:tasks.workunit.client.1.smithi123.stdout:9/139: dwrite d4/d17/f1f [0,4194304] 0 2024-02-16T22:21:45.783 INFO:tasks.workunit.client.1.smithi123.stdout:9/140: unlink d4/d17/f29 0 2024-02-16T22:21:45.791 INFO:tasks.workunit.client.1.smithi123.stdout:9/141: dwrite d4/fe [4194304,4194304] 0 2024-02-16T22:21:45.794 INFO:tasks.workunit.client.1.smithi123.stdout:9/142: symlink d4/d17/l30 0 2024-02-16T22:21:45.815 INFO:tasks.workunit.client.1.smithi123.stdout:2/99: rename d4/d8 to d4/d1a/d1d 0 2024-02-16T22:21:45.816 INFO:tasks.workunit.client.1.smithi123.stdout:6/89: rename d0 to d0/d6/d1f 22 2024-02-16T22:21:45.817 INFO:tasks.workunit.client.1.smithi123.stdout:6/90: mknod d0/d6/c20 0 2024-02-16T22:21:45.819 INFO:tasks.workunit.client.1.smithi123.stdout:5/81: rename d6/cc to d6/dd/c1c 0 2024-02-16T22:21:45.820 INFO:tasks.workunit.client.1.smithi123.stdout:6/91: truncate d0/d6/fd 2759929 0 2024-02-16T22:21:45.822 INFO:tasks.workunit.client.1.smithi123.stdout:7/89: rename d3/c1f to d3/d12/d1d/d1e/c21 0 2024-02-16T22:21:45.822 INFO:tasks.workunit.client.1.smithi123.stdout:4/128: truncate f3 4955765 0 2024-02-16T22:21:45.823 INFO:tasks.workunit.client.1.smithi123.stdout:2/100: dwrite d4/d1a/d1d/f17 [0,4194304] 0 2024-02-16T22:21:45.824 INFO:tasks.workunit.client.1.smithi123.stdout:7/90: write d3/d5/f18 [4067465,17908] 0 2024-02-16T22:21:45.824 INFO:tasks.workunit.client.1.smithi123.stdout:2/101: read - d4/d11/f18 zero size 2024-02-16T22:21:45.843 INFO:tasks.workunit.client.1.smithi123.stdout:8/116: truncate d0/d12/f19 2731585 0 2024-02-16T22:21:45.845 INFO:tasks.workunit.client.1.smithi123.stdout:9/143: getdents d4/d17/d2b 0 2024-02-16T22:21:45.847 INFO:tasks.workunit.client.1.smithi123.stdout:9/144: stat d4/l19 0 2024-02-16T22:21:45.851 INFO:tasks.workunit.client.1.smithi123.stdout:3/86: dwrite d0/f12 [0,4194304] 0 2024-02-16T22:21:45.852 INFO:tasks.workunit.client.1.smithi123.stdout:4/129: dwrite d5/f26 [0,4194304] 0 2024-02-16T22:21:45.855 INFO:tasks.workunit.client.1.smithi123.stdout:0/88: dwrite d1/f6 [0,4194304] 0 2024-02-16T22:21:45.855 INFO:tasks.workunit.client.1.smithi123.stdout:2/102: symlink d4/d1a/d1d/d13/d15/d1b/l1e 0 2024-02-16T22:21:45.855 INFO:tasks.workunit.client.1.smithi123.stdout:7/91: creat d3/d12/d1d/f22 x:0 0 0 2024-02-16T22:21:45.857 INFO:tasks.workunit.client.1.smithi123.stdout:6/92: creat d0/d9/f21 x:0 0 0 2024-02-16T22:21:45.859 INFO:tasks.workunit.client.1.smithi123.stdout:9/145: symlink d4/d17/d20/l31 0 2024-02-16T22:21:45.860 INFO:tasks.workunit.client.1.smithi123.stdout:0/89: dread - d1/f12 zero size 2024-02-16T22:21:45.861 INFO:tasks.workunit.client.1.smithi123.stdout:4/130: creat d5/d1c/d23/f2a x:0 0 0 2024-02-16T22:21:45.861 INFO:tasks.workunit.client.1.smithi123.stdout:6/93: dread - d0/d9/f19 zero size 2024-02-16T22:21:45.861 INFO:tasks.workunit.client.1.smithi123.stdout:7/92: chown d3/d12/d1d/d1e 473875449 1 2024-02-16T22:21:45.863 INFO:tasks.workunit.client.1.smithi123.stdout:2/103: symlink d4/d1a/l1f 0 2024-02-16T22:21:45.864 INFO:tasks.workunit.client.1.smithi123.stdout:3/87: write d0/f6 [126320,42047] 0 2024-02-16T22:21:45.865 INFO:tasks.workunit.client.1.smithi123.stdout:0/90: symlink d1/d8/l16 0 2024-02-16T22:21:45.865 INFO:tasks.workunit.client.1.smithi123.stdout:9/146: readlink d4/d17/d20/l2a 0 2024-02-16T22:21:45.866 INFO:tasks.workunit.client.1.smithi123.stdout:4/131: chown d5/d1c/d23/l28 106 1 2024-02-16T22:21:45.866 INFO:tasks.workunit.client.1.smithi123.stdout:8/117: dwrite d0/df/f16 [0,4194304] 0 2024-02-16T22:21:45.868 INFO:tasks.workunit.client.1.smithi123.stdout:7/93: symlink d3/d12/d1d/d1e/l23 0 2024-02-16T22:21:45.869 INFO:tasks.workunit.client.1.smithi123.stdout:0/91: mkdir d1/d4/d17 0 2024-02-16T22:21:45.870 INFO:tasks.workunit.client.1.smithi123.stdout:6/94: rename d0/d6/fb to d0/d6/d13/f22 0 2024-02-16T22:21:45.873 INFO:tasks.workunit.client.1.smithi123.stdout:2/104: dwrite d4/d1a/d1d/f17 [0,4194304] 0 2024-02-16T22:21:45.874 INFO:tasks.workunit.client.1.smithi123.stdout:7/94: mkdir d3/d12/d24 0 2024-02-16T22:21:45.874 INFO:tasks.workunit.client.1.smithi123.stdout:2/105: fdatasync d4/f16 0 2024-02-16T22:21:45.875 INFO:tasks.workunit.client.1.smithi123.stdout:7/95: dread - d3/d12/d1d/f22 zero size 2024-02-16T22:21:45.876 INFO:tasks.workunit.client.1.smithi123.stdout:8/118: mkdir d0/d17/d26/d2a 0 2024-02-16T22:21:45.876 INFO:tasks.workunit.client.1.smithi123.stdout:3/88: fdatasync d0/f6 0 2024-02-16T22:21:45.877 INFO:tasks.workunit.client.1.smithi123.stdout:0/92: symlink d1/d4/l18 0 2024-02-16T22:21:45.879 INFO:tasks.workunit.client.1.smithi123.stdout:0/93: dread d1/d8/d13/f15 [0,4194304] 0 2024-02-16T22:21:45.881 INFO:tasks.workunit.client.1.smithi123.stdout:1/75: sync 2024-02-16T22:21:45.881 INFO:tasks.workunit.client.1.smithi123.stdout:4/132: rename d5/l11 to d5/d1c/l2b 0 2024-02-16T22:21:45.884 INFO:tasks.workunit.client.1.smithi123.stdout:7/96: mknod d3/d12/d1d/c25 0 2024-02-16T22:21:45.884 INFO:tasks.workunit.client.1.smithi123.stdout:3/89: dwrite d0/f6 [0,4194304] 0 2024-02-16T22:21:45.885 INFO:tasks.workunit.client.1.smithi123.stdout:2/106: symlink d4/d1a/d1d/d13/d15/l20 0 2024-02-16T22:21:45.886 INFO:tasks.workunit.client.1.smithi123.stdout:8/119: mknod d0/df/c2b 0 2024-02-16T22:21:45.886 INFO:tasks.workunit.client.1.smithi123.stdout:2/107: write d4/f16 [1925415,75858] 0 2024-02-16T22:21:45.887 INFO:tasks.workunit.client.1.smithi123.stdout:6/95: creat d0/d9/dc/f23 x:0 0 0 2024-02-16T22:21:45.891 INFO:tasks.workunit.client.1.smithi123.stdout:9/147: getdents d4/d17/d24 0 2024-02-16T22:21:45.891 INFO:tasks.workunit.client.1.smithi123.stdout:4/133: write f4 [959883,49533] 0 2024-02-16T22:21:45.891 INFO:tasks.workunit.client.1.smithi123.stdout:1/76: symlink d3/d6/db/l1a 0 2024-02-16T22:21:45.892 INFO:tasks.workunit.client.1.smithi123.stdout:8/120: mknod d0/d12/c2c 0 2024-02-16T22:21:45.893 INFO:tasks.workunit.client.1.smithi123.stdout:2/108: symlink d4/d1a/d1d/d13/l21 0 2024-02-16T22:21:45.895 INFO:tasks.workunit.client.1.smithi123.stdout:6/96: rename d0/d9/le to d0/d9/dc/l24 0 2024-02-16T22:21:45.896 INFO:tasks.workunit.client.1.smithi123.stdout:7/97: mkdir d3/d26 0 2024-02-16T22:21:45.900 INFO:tasks.workunit.client.1.smithi123.stdout:4/134: rmdir d5/d1c/d23 39 2024-02-16T22:21:45.900 INFO:tasks.workunit.client.1.smithi123.stdout:4/135: chown d5/d1c 159933 1 2024-02-16T22:21:45.900 INFO:tasks.workunit.client.1.smithi123.stdout:9/148: mkdir d4/d17/d24/d32 0 2024-02-16T22:21:45.901 INFO:tasks.workunit.client.1.smithi123.stdout:3/90: symlink d0/l1c 0 2024-02-16T22:21:45.901 INFO:tasks.workunit.client.1.smithi123.stdout:9/149: write d4/f26 [511488,25541] 0 2024-02-16T22:21:45.902 INFO:tasks.workunit.client.1.smithi123.stdout:0/94: link d1/l9 d1/d4/d17/l19 0 2024-02-16T22:21:45.904 INFO:tasks.workunit.client.1.smithi123.stdout:3/91: dread d0/f17 [0,4194304] 0 2024-02-16T22:21:45.906 INFO:tasks.workunit.client.1.smithi123.stdout:9/150: mkdir d4/d33 0 2024-02-16T22:21:45.907 INFO:tasks.workunit.client.1.smithi123.stdout:1/77: rename d3/d6/d11/c14 to d3/d6/db/d10/c1b 0 2024-02-16T22:21:45.908 INFO:tasks.workunit.client.1.smithi123.stdout:0/95: readlink d1/d4/d17/l19 0 2024-02-16T22:21:45.910 INFO:tasks.workunit.client.1.smithi123.stdout:6/97: truncate d0/d6/fd 214668 0 2024-02-16T22:21:45.913 INFO:tasks.workunit.client.1.smithi123.stdout:9/151: dwrite d4/f26 [0,4194304] 0 2024-02-16T22:21:45.913 INFO:tasks.workunit.client.1.smithi123.stdout:6/98: truncate d0/d9/f21 443189 0 2024-02-16T22:21:45.915 INFO:tasks.workunit.client.1.smithi123.stdout:9/152: rename d4/d17 to d4/d17/d2b/d34 22 2024-02-16T22:21:45.915 INFO:tasks.workunit.client.1.smithi123.stdout:6/99: write d0/d6/f15 [3762514,114846] 0 2024-02-16T22:21:45.917 INFO:tasks.workunit.client.1.smithi123.stdout:0/96: dwrite d1/d8/fa [0,4194304] 0 2024-02-16T22:21:45.917 INFO:tasks.workunit.client.1.smithi123.stdout:9/153: fdatasync d4/f6 0 2024-02-16T22:21:45.919 INFO:tasks.workunit.client.1.smithi123.stdout:3/92: getdents d0 0 2024-02-16T22:21:45.919 INFO:tasks.workunit.client.1.smithi123.stdout:9/154: fdatasync d4/f21 0 2024-02-16T22:21:45.920 INFO:tasks.workunit.client.1.smithi123.stdout:3/93: readlink d0/l14 0 2024-02-16T22:21:45.920 INFO:tasks.workunit.client.1.smithi123.stdout:2/109: truncate d4/d1a/d1d/f9 3163999 0 2024-02-16T22:21:45.922 INFO:tasks.workunit.client.1.smithi123.stdout:3/94: stat d0/l16 0 2024-02-16T22:21:45.925 INFO:tasks.workunit.client.1.smithi123.stdout:6/100: mknod d0/d9/c25 0 2024-02-16T22:21:45.925 INFO:tasks.workunit.client.1.smithi123.stdout:3/95: write d0/ff [1710584,51404] 0 2024-02-16T22:21:45.927 INFO:tasks.workunit.client.1.smithi123.stdout:1/78: truncate d3/fc 1532454 0 2024-02-16T22:21:45.927 INFO:tasks.workunit.client.1.smithi123.stdout:3/96: dread - d0/f15 zero size 2024-02-16T22:21:45.928 INFO:tasks.workunit.client.1.smithi123.stdout:1/79: readlink l1 0 2024-02-16T22:21:45.930 INFO:tasks.workunit.client.1.smithi123.stdout:1/80: readlink l1 0 2024-02-16T22:21:45.930 INFO:tasks.workunit.client.1.smithi123.stdout:8/121: dwrite d0/fc [0,4194304] 0 2024-02-16T22:21:45.933 INFO:tasks.workunit.client.1.smithi123.stdout:6/101: mknod d0/d6/d1a/c26 0 2024-02-16T22:21:45.933 INFO:tasks.workunit.client.1.smithi123.stdout:8/122: dread - d0/df/d25/f28 zero size 2024-02-16T22:21:45.934 INFO:tasks.workunit.client.1.smithi123.stdout:9/155: dread d4/f21 [0,4194304] 0 2024-02-16T22:21:45.935 INFO:tasks.workunit.client.1.smithi123.stdout:2/110: dread f2 [0,4194304] 0 2024-02-16T22:21:45.938 INFO:tasks.workunit.client.1.smithi123.stdout:6/102: dread - d0/d9/dc/f17 zero size 2024-02-16T22:21:45.939 INFO:tasks.workunit.client.1.smithi123.stdout:9/156: fsync d4/d17/f23 0 2024-02-16T22:21:45.941 INFO:tasks.workunit.client.1.smithi123.stdout:2/111: chown d4/d1a/d1d/cd 1 1 2024-02-16T22:21:45.942 INFO:tasks.workunit.client.1.smithi123.stdout:3/97: dwrite d0/fd [0,4194304] 0 2024-02-16T22:21:45.943 INFO:tasks.workunit.client.1.smithi123.stdout:0/97: truncate d1/d8/fa 1424885 0 2024-02-16T22:21:45.944 INFO:tasks.workunit.client.1.smithi123.stdout:6/103: mknod d0/d6/c27 0 2024-02-16T22:21:45.945 INFO:tasks.workunit.client.1.smithi123.stdout:9/157: write d4/d17/d20/f22 [1027377,60542] 0 2024-02-16T22:21:45.945 INFO:tasks.workunit.client.1.smithi123.stdout:1/81: dread d3/f7 [0,4194304] 0 2024-02-16T22:21:45.945 INFO:tasks.workunit.client.1.smithi123.stdout:2/112: mknod d4/d1a/d1d/d13/c22 0 2024-02-16T22:21:45.946 INFO:tasks.workunit.client.1.smithi123.stdout:0/98: mknod d1/d4/c1a 0 2024-02-16T22:21:45.947 INFO:tasks.workunit.client.1.smithi123.stdout:8/123: dread d0/fc [0,4194304] 0 2024-02-16T22:21:45.947 INFO:tasks.workunit.client.1.smithi123.stdout:2/113: stat d4/d1a/d1d/lb 0 2024-02-16T22:21:45.948 INFO:tasks.workunit.client.1.smithi123.stdout:9/158: chown d4/f1b 1868021 1 2024-02-16T22:21:45.949 INFO:tasks.workunit.client.1.smithi123.stdout:3/98: rename d0/l14 to d0/l1d 0 2024-02-16T22:21:45.949 INFO:tasks.workunit.client.1.smithi123.stdout:2/114: fsync d4/d11/f18 0 2024-02-16T22:21:45.949 INFO:tasks.workunit.client.1.smithi123.stdout:0/99: read - d1/d4/f11 zero size 2024-02-16T22:21:45.949 INFO:tasks.workunit.client.1.smithi123.stdout:5/82: sync 2024-02-16T22:21:45.950 INFO:tasks.workunit.client.1.smithi123.stdout:1/82: mknod d3/d6/db/d10/c1c 0 2024-02-16T22:21:45.952 INFO:tasks.workunit.client.1.smithi123.stdout:0/100: read d1/d8/fa [1072044,9566] 0 2024-02-16T22:21:45.952 INFO:tasks.workunit.client.1.smithi123.stdout:5/83: read d6/f8 [627066,128851] 0 2024-02-16T22:21:45.952 INFO:tasks.workunit.client.1.smithi123.stdout:2/115: write d4/d11/f18 [516935,89386] 0 2024-02-16T22:21:45.954 INFO:tasks.workunit.client.1.smithi123.stdout:8/124: creat d0/d17/d26/f2d x:0 0 0 2024-02-16T22:21:45.956 INFO:tasks.workunit.client.1.smithi123.stdout:8/125: fdatasync d0/fd 0 2024-02-16T22:21:45.956 INFO:tasks.workunit.client.1.smithi123.stdout:1/83: mknod d3/d6/c1d 0 2024-02-16T22:21:45.957 INFO:tasks.workunit.client.1.smithi123.stdout:2/116: creat d4/f23 x:0 0 0 2024-02-16T22:21:45.958 INFO:tasks.workunit.client.1.smithi123.stdout:5/84: write d6/dd/df/d16/f17 [544361,73715] 0 2024-02-16T22:21:45.959 INFO:tasks.workunit.client.1.smithi123.stdout:2/117: readlink d4/d1a/l1f 0 2024-02-16T22:21:45.964 INFO:tasks.workunit.client.1.smithi123.stdout:5/85: write d6/dd/df/f1b [761245,84666] 0 2024-02-16T22:21:45.964 INFO:tasks.workunit.client.1.smithi123.stdout:9/159: dwrite d4/d17/f23 [0,4194304] 0 2024-02-16T22:21:45.969 INFO:tasks.workunit.client.1.smithi123.stdout:8/126: dwrite d0/df/f1c [0,4194304] 0 2024-02-16T22:21:45.974 INFO:tasks.workunit.client.1.smithi123.stdout:9/160: dread d4/d17/f23 [0,4194304] 0 2024-02-16T22:21:45.974 INFO:tasks.workunit.client.1.smithi123.stdout:8/127: dread d0/fc [0,4194304] 0 2024-02-16T22:21:45.996 INFO:tasks.workunit.client.1.smithi123.stdout:6/104: getdents d0/d6/d1a 0 2024-02-16T22:21:45.997 INFO:tasks.workunit.client.1.smithi123.stdout:6/105: chown d0/d6/f18 6170099 1 2024-02-16T22:21:45.998 INFO:tasks.workunit.client.1.smithi123.stdout:7/98: sync 2024-02-16T22:21:46.000 INFO:tasks.workunit.client.1.smithi123.stdout:2/118: rename d4/f6 to d4/d11/f24 0 2024-02-16T22:21:46.000 INFO:tasks.workunit.client.1.smithi123.stdout:7/99: stat d3/d12/d1d/f22 0 2024-02-16T22:21:46.000 INFO:tasks.workunit.client.1.smithi123.stdout:5/86: creat d6/dd/df/d16/f1d x:0 0 0 2024-02-16T22:21:46.001 INFO:tasks.workunit.client.1.smithi123.stdout:7/100: read d3/d5/f9 [2127075,100704] 0 2024-02-16T22:21:46.003 INFO:tasks.workunit.client.1.smithi123.stdout:5/87: read d6/f12 [177158,29688] 0 2024-02-16T22:21:46.003 INFO:tasks.workunit.client.1.smithi123.stdout:9/161: unlink d4/l15 0 2024-02-16T22:21:46.004 INFO:tasks.workunit.client.1.smithi123.stdout:9/162: chown d4/l13 15 1 2024-02-16T22:21:46.005 INFO:tasks.workunit.client.1.smithi123.stdout:8/128: mkdir d0/d17/d26/d2e 0 2024-02-16T22:21:46.008 INFO:tasks.workunit.client.1.smithi123.stdout:6/106: dread d0/f2 [0,4194304] 0 2024-02-16T22:21:46.009 INFO:tasks.workunit.client.1.smithi123.stdout:2/119: mkdir d4/d1a/d1d/d13/d25 0 2024-02-16T22:21:46.011 INFO:tasks.workunit.client.1.smithi123.stdout:6/107: chown d0/d6/d13/l1c 674684 1 2024-02-16T22:21:46.013 INFO:tasks.workunit.client.1.smithi123.stdout:7/101: symlink d3/d12/d1d/d1e/l27 0 2024-02-16T22:21:46.013 INFO:tasks.workunit.client.1.smithi123.stdout:5/88: dwrite d6/f12 [0,4194304] 0 2024-02-16T22:21:46.013 INFO:tasks.workunit.client.1.smithi123.stdout:6/108: chown d0/d6/c27 9042731 1 2024-02-16T22:21:46.015 INFO:tasks.workunit.client.1.smithi123.stdout:9/163: mknod d4/d17/d2b/c35 0 2024-02-16T22:21:46.017 INFO:tasks.workunit.client.1.smithi123.stdout:8/129: dwrite d0/d12/f22 [0,4194304] 0 2024-02-16T22:21:46.020 INFO:tasks.workunit.client.1.smithi123.stdout:4/136: sync 2024-02-16T22:21:46.021 INFO:tasks.workunit.client.1.smithi123.stdout:3/99: mknod d0/c1e 0 2024-02-16T22:21:46.024 INFO:tasks.workunit.client.1.smithi123.stdout:0/101: getdents d1/d4/d17 0 2024-02-16T22:21:46.025 INFO:tasks.workunit.client.1.smithi123.stdout:7/102: rmdir d3/d5 39 2024-02-16T22:21:46.027 INFO:tasks.workunit.client.1.smithi123.stdout:6/109: symlink d0/d6/d1a/l28 0 2024-02-16T22:21:46.028 INFO:tasks.workunit.client.1.smithi123.stdout:1/84: link d3/d6/db/c12 d3/d6/db/d10/c1e 0 2024-02-16T22:21:46.030 INFO:tasks.workunit.client.1.smithi123.stdout:9/164: dwrite d4/d17/f1d [0,4194304] 0 2024-02-16T22:21:46.030 INFO:tasks.workunit.client.1.smithi123.stdout:2/120: creat d4/d1a/d1d/d13/d25/f26 x:0 0 0 2024-02-16T22:21:46.033 INFO:tasks.workunit.client.1.smithi123.stdout:8/130: dwrite d0/df/f1c [0,4194304] 0 2024-02-16T22:21:46.034 INFO:tasks.workunit.client.1.smithi123.stdout:0/102: mknod d1/d4/c1b 0 2024-02-16T22:21:46.035 INFO:tasks.workunit.client.1.smithi123.stdout:2/121: dread - d4/d1a/d1d/d13/d15/d1b/f1c zero size 2024-02-16T22:21:46.035 INFO:tasks.workunit.client.1.smithi123.stdout:8/131: read - d0/d17/d26/f2d zero size 2024-02-16T22:21:46.037 INFO:tasks.workunit.client.1.smithi123.stdout:7/103: truncate d3/d5/f18 4877272 0 2024-02-16T22:21:46.037 INFO:tasks.workunit.client.1.smithi123.stdout:1/85: creat d3/d6/d11/f1f x:0 0 0 2024-02-16T22:21:46.037 INFO:tasks.workunit.client.1.smithi123.stdout:8/132: fdatasync d0/d12/f22 0 2024-02-16T22:21:46.038 INFO:tasks.workunit.client.1.smithi123.stdout:9/165: mkdir d4/d36 0 2024-02-16T22:21:46.039 INFO:tasks.workunit.client.1.smithi123.stdout:8/133: read - d0/d12/f1e zero size 2024-02-16T22:21:46.040 INFO:tasks.workunit.client.1.smithi123.stdout:8/134: write d0/d12/f22 [1849835,104814] 0 2024-02-16T22:21:46.041 INFO:tasks.workunit.client.1.smithi123.stdout:6/110: mkdir d0/d29 0 2024-02-16T22:21:46.042 INFO:tasks.workunit.client.1.smithi123.stdout:0/103: fsync d1/d8/d13/f14 0 2024-02-16T22:21:46.043 INFO:tasks.workunit.client.1.smithi123.stdout:0/104: write d1/d8/d13/f14 [593497,43823] 0 2024-02-16T22:21:46.046 INFO:tasks.workunit.client.1.smithi123.stdout:2/122: mkdir d4/d11/d27 0 2024-02-16T22:21:46.046 INFO:tasks.workunit.client.1.smithi123.stdout:7/104: rmdir d3/d5 39 2024-02-16T22:21:46.048 INFO:tasks.workunit.client.1.smithi123.stdout:7/105: chown d3/d12/d1d 2716974 1 2024-02-16T22:21:46.049 INFO:tasks.workunit.client.1.smithi123.stdout:3/100: creat d0/f1f x:0 0 0 2024-02-16T22:21:46.051 INFO:tasks.workunit.client.1.smithi123.stdout:8/135: fdatasync d0/d12/f22 0 2024-02-16T22:21:46.145 INFO:tasks.workunit.client.1.smithi123.stdout:0/105: symlink d1/d4/l1c 0 2024-02-16T22:21:46.146 INFO:tasks.workunit.client.1.smithi123.stdout:2/123: write d4/fc [4007137,47244] 0 2024-02-16T22:21:46.151 INFO:tasks.workunit.client.1.smithi123.stdout:7/106: dwrite d3/d5/ff [0,4194304] 0 2024-02-16T22:21:46.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:45 smithi114 ceph-mon[29235]: from='client.24513 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:21:46.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:45 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/2245095767' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:21:46.241 INFO:tasks.workunit.client.0.smithi114.stdout:7/33: sync 2024-02-16T22:21:46.241 INFO:tasks.workunit.client.0.smithi114.stdout:7/34: write d3/f8 [652456,106255] 0 2024-02-16T22:21:46.244 INFO:tasks.workunit.client.1.smithi123.stdout:9/166: rename c3 to d4/d1e/c37 0 2024-02-16T22:21:46.247 INFO:tasks.workunit.client.0.smithi114.stdout:7/35: mknod d3/c9 0 2024-02-16T22:21:46.249 INFO:tasks.workunit.client.1.smithi123.stdout:0/106: mkdir d1/d8/d13/d1d 0 2024-02-16T22:21:46.250 INFO:tasks.workunit.client.1.smithi123.stdout:9/167: dwrite d4/f26 [0,4194304] 0 2024-02-16T22:21:46.251 INFO:tasks.workunit.client.1.smithi123.stdout:3/101: dread d0/f6 [0,4194304] 0 2024-02-16T22:21:46.253 INFO:tasks.workunit.client.0.smithi114.stdout:7/36: dwrite d3/f8 [4194304,4194304] 0 2024-02-16T22:21:46.253 INFO:tasks.workunit.client.0.smithi114.stdout:7/37: chown d3 21035369 1 2024-02-16T22:21:46.254 INFO:tasks.workunit.client.1.smithi123.stdout:7/107: creat d3/d5/f28 x:0 0 0 2024-02-16T22:21:46.254 INFO:tasks.workunit.client.0.smithi114.stdout:7/38: chown d3 719862685 1 2024-02-16T22:21:46.257 INFO:tasks.workunit.client.1.smithi123.stdout:8/136: rename d0/c23 to d0/d17/d26/d2a/c2f 0 2024-02-16T22:21:46.259 INFO:tasks.workunit.client.0.smithi114.stdout:7/39: creat d3/fa x:0 0 0 2024-02-16T22:21:46.259 INFO:tasks.workunit.client.1.smithi123.stdout:8/137: write d0/df/f13 [6005639,87143] 0 2024-02-16T22:21:46.261 INFO:tasks.workunit.client.1.smithi123.stdout:9/168: dwrite d4/f26 [0,4194304] 0 2024-02-16T22:21:46.261 INFO:tasks.workunit.client.1.smithi123.stdout:0/107: dread d1/d8/d13/f15 [0,4194304] 0 2024-02-16T22:21:46.264 INFO:tasks.workunit.client.1.smithi123.stdout:0/108: write d1/f12 [528006,72990] 0 2024-02-16T22:21:46.269 INFO:tasks.workunit.client.1.smithi123.stdout:9/169: dread d4/f10 [0,4194304] 0 2024-02-16T22:21:46.269 INFO:tasks.workunit.client.1.smithi123.stdout:0/109: mknod d1/d4/d17/c1e 0 2024-02-16T22:21:46.272 INFO:tasks.workunit.client.1.smithi123.stdout:7/108: dread d3/d12/f14 [0,4194304] 0 2024-02-16T22:21:46.274 INFO:tasks.workunit.client.1.smithi123.stdout:7/109: write d3/d12/f1a [582442,106364] 0 2024-02-16T22:21:46.274 INFO:tasks.workunit.client.1.smithi123.stdout:2/124: link d4/d1a/d1d/f9 d4/d1a/d1d/d13/d15/d1b/f28 0 2024-02-16T22:21:46.275 INFO:tasks.workunit.client.1.smithi123.stdout:9/170: dread d4/d17/f1a [0,4194304] 0 2024-02-16T22:21:46.276 INFO:tasks.workunit.client.1.smithi123.stdout:9/171: fdatasync d4/d1e/f25 0 2024-02-16T22:21:46.277 INFO:tasks.workunit.client.1.smithi123.stdout:2/125: truncate d4/d11/f24 2359299 0 2024-02-16T22:21:46.277 INFO:tasks.workunit.client.1.smithi123.stdout:8/138: link d0/df/c24 d0/df/c30 0 2024-02-16T22:21:46.278 INFO:tasks.workunit.client.1.smithi123.stdout:9/172: write d4/d1e/f25 [111896,20042] 0 2024-02-16T22:21:46.279 INFO:tasks.workunit.client.1.smithi123.stdout:2/126: dread - d4/d1a/d1d/d13/d15/d1b/f1c zero size 2024-02-16T22:21:46.280 INFO:tasks.workunit.client.1.smithi123.stdout:9/173: write d4/f1b [892682,33930] 0 2024-02-16T22:21:46.280 INFO:tasks.workunit.client.1.smithi123.stdout:8/139: mknod d0/df/c31 0 2024-02-16T22:21:46.281 INFO:tasks.workunit.client.1.smithi123.stdout:8/140: readlink d0/l8 0 2024-02-16T22:21:46.282 INFO:tasks.workunit.client.1.smithi123.stdout:2/127: unlink l1 0 2024-02-16T22:21:46.282 INFO:tasks.workunit.client.1.smithi123.stdout:4/137: write d5/f17 [3738873,19621] 0 2024-02-16T22:21:46.283 INFO:tasks.workunit.client.1.smithi123.stdout:8/141: fdatasync d0/fc 0 2024-02-16T22:21:46.284 INFO:tasks.workunit.client.1.smithi123.stdout:2/128: chown d4/f16 7 1 2024-02-16T22:21:46.286 INFO:tasks.workunit.client.1.smithi123.stdout:5/89: dwrite d6/f8 [0,4194304] 0 2024-02-16T22:21:46.287 INFO:tasks.workunit.client.1.smithi123.stdout:9/174: link d4/f26 d4/d33/f38 0 2024-02-16T22:21:46.287 INFO:tasks.workunit.client.1.smithi123.stdout:5/90: dread - d6/f13 zero size 2024-02-16T22:21:46.287 INFO:tasks.workunit.client.1.smithi123.stdout:8/142: mkdir d0/d17/d26/d32 0 2024-02-16T22:21:46.289 INFO:tasks.workunit.client.1.smithi123.stdout:4/138: mkdir d5/d1c/d23/d2c 0 2024-02-16T22:21:46.289 INFO:tasks.workunit.client.1.smithi123.stdout:8/143: write d0/df/f16 [983316,51552] 0 2024-02-16T22:21:46.289 INFO:tasks.workunit.client.1.smithi123.stdout:4/139: readlink d5/l15 0 2024-02-16T22:21:46.290 INFO:tasks.workunit.client.1.smithi123.stdout:2/129: rename d4/d11/d27 to d4/d1a/d29 0 2024-02-16T22:21:46.291 INFO:tasks.workunit.client.1.smithi123.stdout:4/140: write d5/f1e [559183,129864] 0 2024-02-16T22:21:46.291 INFO:tasks.workunit.client.0.smithi114.stdout:5/23: sync 2024-02-16T22:21:46.291 INFO:tasks.workunit.client.0.smithi114.stdout:6/34: sync 2024-02-16T22:21:46.292 INFO:tasks.workunit.client.0.smithi114.stdout:5/24: write d0/f7 [448370,44739] 0 2024-02-16T22:21:46.293 INFO:tasks.workunit.client.1.smithi123.stdout:9/175: creat d4/d1e/f39 x:0 0 0 2024-02-16T22:21:46.293 INFO:tasks.workunit.client.0.smithi114.stdout:5/25: chown d0/c1 3 1 2024-02-16T22:21:46.294 INFO:tasks.workunit.client.1.smithi123.stdout:4/141: write d5/f14 [1058055,89615] 0 2024-02-16T22:21:46.295 INFO:tasks.workunit.client.0.smithi114.stdout:6/35: creat d6/fc x:0 0 0 2024-02-16T22:21:46.299 INFO:tasks.workunit.client.0.smithi114.stdout:6/36: dread f3 [0,4194304] 0 2024-02-16T22:21:46.300 INFO:tasks.workunit.client.1.smithi123.stdout:8/144: dwrite d0/d17/d26/f2d [0,4194304] 0 2024-02-16T22:21:46.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:45 smithi123 ceph-mon[37161]: from='client.24513 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:21:46.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:45 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/2245095767' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:21:46.302 INFO:tasks.workunit.client.1.smithi123.stdout:1/86: dwrite d3/f7 [0,4194304] 0 2024-02-16T22:21:46.303 INFO:tasks.workunit.client.1.smithi123.stdout:2/130: unlink d4/d1a/d1d/d13/d15/d1b/f1c 0 2024-02-16T22:21:46.306 INFO:tasks.workunit.client.1.smithi123.stdout:6/111: dwrite d0/d6/fd [0,4194304] 0 2024-02-16T22:21:46.307 INFO:tasks.workunit.client.0.smithi114.stdout:6/37: unlink f3 0 2024-02-16T22:21:46.308 INFO:tasks.workunit.client.0.smithi114.stdout:6/38: chown d6/l7 1994234 1 2024-02-16T22:21:46.313 INFO:tasks.workunit.client.1.smithi123.stdout:9/176: dwrite d4/f26 [0,4194304] 0 2024-02-16T22:21:46.314 INFO:tasks.workunit.client.0.smithi114.stdout:6/39: mkdir d6/dd 0 2024-02-16T22:21:46.317 INFO:tasks.workunit.client.1.smithi123.stdout:5/91: rename d6/dd/c1c to d6/dd/df/c1e 0 2024-02-16T22:21:46.317 INFO:tasks.workunit.client.1.smithi123.stdout:8/145: rename d0/d17/d26 to d0/d17/d26/d2a/d33 22 2024-02-16T22:21:46.318 INFO:tasks.workunit.client.0.smithi114.stdout:5/26: getdents d0 0 2024-02-16T22:21:46.319 INFO:tasks.workunit.client.1.smithi123.stdout:8/146: write d0/d12/f1e [769812,95574] 0 2024-02-16T22:21:46.320 INFO:tasks.workunit.client.1.smithi123.stdout:5/92: truncate d6/dd/df/d16/f1a 421718 0 2024-02-16T22:21:46.321 INFO:tasks.workunit.client.0.smithi114.stdout:6/40: symlink d6/le 0 2024-02-16T22:21:46.322 INFO:tasks.workunit.client.0.smithi114.stdout:5/27: dread d0/f7 [0,4194304] 0 2024-02-16T22:21:46.322 INFO:tasks.workunit.client.1.smithi123.stdout:5/93: dread d6/dd/df/d16/f1a [0,4194304] 0 2024-02-16T22:21:46.322 INFO:tasks.workunit.client.1.smithi123.stdout:5/94: stat d6 0 2024-02-16T22:21:46.323 INFO:tasks.workunit.client.0.smithi114.stdout:5/28: read d0/f7 [83063,23652] 0 2024-02-16T22:21:46.327 INFO:tasks.workunit.client.0.smithi114.stdout:5/29: creat d0/f8 x:0 0 0 2024-02-16T22:21:46.328 INFO:tasks.workunit.client.1.smithi123.stdout:2/131: write f2 [2065993,64560] 0 2024-02-16T22:21:46.328 INFO:tasks.workunit.client.0.smithi114.stdout:5/30: dread - d0/f8 zero size 2024-02-16T22:21:46.329 INFO:tasks.workunit.client.0.smithi114.stdout:6/41: dwrite d6/fc [0,4194304] 0 2024-02-16T22:21:46.329 INFO:tasks.workunit.client.0.smithi114.stdout:7/40: truncate d3/f8 2186444 0 2024-02-16T22:21:46.330 INFO:tasks.workunit.client.0.smithi114.stdout:7/41: chown d3/l7 1 1 2024-02-16T22:21:46.331 INFO:tasks.workunit.client.1.smithi123.stdout:6/112: creat d0/d6/d13/f2a x:0 0 0 2024-02-16T22:21:46.333 INFO:tasks.workunit.client.1.smithi123.stdout:4/142: creat d5/d1c/d23/d2c/f2d x:0 0 0 2024-02-16T22:21:46.334 INFO:tasks.workunit.client.0.smithi114.stdout:6/42: symlink d6/da/lf 0 2024-02-16T22:21:46.337 INFO:tasks.workunit.client.0.smithi114.stdout:5/31: dwrite d0/f8 [0,4194304] 0 2024-02-16T22:21:46.339 INFO:tasks.workunit.client.1.smithi123.stdout:8/147: mkdir d0/df/d25/d34 0 2024-02-16T22:21:46.343 INFO:tasks.workunit.client.1.smithi123.stdout:2/132: mknod d4/d1a/d1d/d13/d15/d1b/c2a 0 2024-02-16T22:21:46.343 INFO:tasks.workunit.client.1.smithi123.stdout:1/87: fdatasync d3/f7 0 2024-02-16T22:21:46.346 INFO:tasks.workunit.client.0.smithi114.stdout:5/32: write d0/f7 [1293859,87865] 0 2024-02-16T22:21:46.348 INFO:tasks.workunit.client.1.smithi123.stdout:6/113: creat d0/d6/d1a/f2b x:0 0 0 2024-02-16T22:21:46.349 INFO:tasks.workunit.client.1.smithi123.stdout:0/110: dwrite d1/d8/d13/f15 [0,4194304] 0 2024-02-16T22:21:46.349 INFO:tasks.workunit.client.1.smithi123.stdout:1/88: chown d3/d6/db/l1a 1650958 1 2024-02-16T22:21:46.350 INFO:tasks.workunit.client.1.smithi123.stdout:4/143: chown f3 13243 1 2024-02-16T22:21:46.352 INFO:tasks.workunit.client.0.smithi114.stdout:5/33: creat d0/f9 x:0 0 0 2024-02-16T22:21:46.352 INFO:tasks.workunit.client.1.smithi123.stdout:7/110: dwrite d3/d12/f14 [0,4194304] 0 2024-02-16T22:21:46.352 INFO:tasks.workunit.client.1.smithi123.stdout:1/89: truncate d3/fe 1037042 0 2024-02-16T22:21:46.352 INFO:tasks.workunit.client.1.smithi123.stdout:4/144: dread - d5/d1c/d23/f2a zero size 2024-02-16T22:21:46.353 INFO:tasks.workunit.client.1.smithi123.stdout:9/177: truncate d4/f6 7008576 0 2024-02-16T22:21:46.356 INFO:tasks.workunit.client.1.smithi123.stdout:4/145: rename d5 to d5/d1c/d23/d2e 22 2024-02-16T22:21:46.356 INFO:tasks.workunit.client.1.smithi123.stdout:4/146: fsync d5/f1e 0 2024-02-16T22:21:46.358 INFO:tasks.workunit.client.1.smithi123.stdout:4/147: rename d5/d1c to d5/d1c/d23/d2c/d2f 22 2024-02-16T22:21:46.360 INFO:tasks.workunit.client.0.smithi114.stdout:5/34: dwrite d0/f9 [0,4194304] 0 2024-02-16T22:21:46.362 INFO:tasks.workunit.client.1.smithi123.stdout:9/178: dwrite d4/d1e/f25 [0,4194304] 0 2024-02-16T22:21:46.375 INFO:tasks.workunit.client.1.smithi123.stdout:3/102: sync 2024-02-16T22:21:46.376 INFO:tasks.workunit.client.0.smithi114.stdout:7/42: fsync f2 0 2024-02-16T22:21:46.377 INFO:tasks.workunit.client.1.smithi123.stdout:1/90: rmdir d3/d6/db/d10 39 2024-02-16T22:21:46.378 INFO:tasks.workunit.client.1.smithi123.stdout:1/91: truncate d3/d6/d11/f1f 932402 0 2024-02-16T22:21:46.379 INFO:tasks.workunit.client.1.smithi123.stdout:8/148: link d0/fc d0/d17/f35 0 2024-02-16T22:21:46.380 INFO:tasks.workunit.client.0.smithi114.stdout:7/43: dread d3/f6 [0,4194304] 0 2024-02-16T22:21:46.383 INFO:tasks.workunit.client.0.smithi114.stdout:7/44: symlink d3/lb 0 2024-02-16T22:21:46.385 INFO:tasks.workunit.client.1.smithi123.stdout:9/179: write d4/d17/f1a [1664403,121952] 0 2024-02-16T22:21:46.386 INFO:tasks.workunit.client.1.smithi123.stdout:9/180: chown d4/f1b 28907 1 2024-02-16T22:21:46.386 INFO:tasks.workunit.client.1.smithi123.stdout:2/133: creat d4/d1a/d1d/f2b x:0 0 0 2024-02-16T22:21:46.387 INFO:tasks.workunit.client.1.smithi123.stdout:6/114: rmdir d0 39 2024-02-16T22:21:46.387 INFO:tasks.workunit.client.1.smithi123.stdout:9/181: write d4/fa [3670254,113652] 0 2024-02-16T22:21:46.389 INFO:tasks.workunit.client.1.smithi123.stdout:7/111: mknod d3/d12/d24/c29 0 2024-02-16T22:21:46.389 INFO:tasks.workunit.client.1.smithi123.stdout:7/112: write d3/d12/d1d/f22 [123884,69018] 0 2024-02-16T22:21:46.390 INFO:tasks.workunit.client.0.smithi114.stdout:5/35: getdents d0 0 2024-02-16T22:21:46.390 INFO:tasks.workunit.client.0.smithi114.stdout:7/45: dwrite d3/fa [0,4194304] 0 2024-02-16T22:21:46.391 INFO:tasks.workunit.client.0.smithi114.stdout:5/36: chown d0/c4 264 1 2024-02-16T22:21:46.393 INFO:tasks.workunit.client.1.smithi123.stdout:3/103: dread d0/fc [0,4194304] 0 2024-02-16T22:21:46.394 INFO:tasks.workunit.client.1.smithi123.stdout:5/95: truncate d6/dd/df/d16/f17 170592 0 2024-02-16T22:21:46.394 INFO:tasks.workunit.client.0.smithi114.stdout:7/46: dread f2 [0,4194304] 0 2024-02-16T22:21:46.398 INFO:tasks.workunit.client.1.smithi123.stdout:8/149: stat d0/d17/d26/d2a/c2f 0 2024-02-16T22:21:46.402 INFO:tasks.workunit.client.0.smithi114.stdout:7/47: rename d3/fa to d3/fc 0 2024-02-16T22:21:46.409 INFO:tasks.workunit.client.1.smithi123.stdout:0/111: creat d1/f1f x:0 0 0 2024-02-16T22:21:46.411 INFO:tasks.workunit.client.1.smithi123.stdout:9/182: unlink d4/d1e/f25 0 2024-02-16T22:21:46.412 INFO:tasks.workunit.client.0.smithi114.stdout:5/37: rmdir d0/d5 0 2024-02-16T22:21:46.412 INFO:tasks.workunit.client.0.smithi114.stdout:5/38: chown d0/c1 1 1 2024-02-16T22:21:46.415 INFO:tasks.workunit.client.0.smithi114.stdout:7/48: link d3/f5 d3/fd 0 2024-02-16T22:21:46.416 INFO:tasks.workunit.client.1.smithi123.stdout:9/183: dwrite d4/f12 [0,4194304] 0 2024-02-16T22:21:46.418 INFO:tasks.workunit.client.0.smithi114.stdout:5/39: dwrite d0/f8 [0,4194304] 0 2024-02-16T22:21:46.420 INFO:tasks.workunit.client.0.smithi114.stdout:7/49: unlink d3/f5 0 2024-02-16T22:21:46.421 INFO:tasks.workunit.client.1.smithi123.stdout:5/96: mkdir d6/dd/df/d1f 0 2024-02-16T22:21:46.421 INFO:tasks.workunit.client.1.smithi123.stdout:1/92: fdatasync d3/ff 0 2024-02-16T22:21:46.422 INFO:tasks.workunit.client.0.smithi114.stdout:5/40: rmdir d0 39 2024-02-16T22:21:46.423 INFO:tasks.workunit.client.1.smithi123.stdout:8/150: symlink d0/df/d25/l36 0 2024-02-16T22:21:46.425 INFO:tasks.workunit.client.0.smithi114.stdout:7/50: chown c1 10 1 2024-02-16T22:21:46.427 INFO:tasks.workunit.client.1.smithi123.stdout:4/148: link d5/d1c/f21 d5/d1c/f30 0 2024-02-16T22:21:46.428 INFO:tasks.workunit.client.0.smithi114.stdout:5/41: chown d0/l6 0 1 2024-02-16T22:21:46.428 INFO:tasks.workunit.client.0.smithi114.stdout:5/42: readlink d0/l6 0 2024-02-16T22:21:46.430 INFO:tasks.workunit.client.0.smithi114.stdout:7/51: fdatasync d3/fc 0 2024-02-16T22:21:46.432 INFO:tasks.workunit.client.1.smithi123.stdout:6/115: rename d0/f2 to d0/d29/f2c 0 2024-02-16T22:21:46.434 INFO:tasks.workunit.client.0.smithi114.stdout:5/43: rmdir d0 39 2024-02-16T22:21:46.436 INFO:tasks.workunit.client.1.smithi123.stdout:2/134: dwrite d4/f23 [0,4194304] 0 2024-02-16T22:21:46.437 INFO:tasks.workunit.client.0.smithi114.stdout:5/44: stat d0/l6 0 2024-02-16T22:21:46.440 INFO:tasks.workunit.client.1.smithi123.stdout:6/116: dwrite d0/d6/d13/f2a [0,4194304] 0 2024-02-16T22:21:46.441 INFO:tasks.workunit.client.0.smithi114.stdout:9/22: sync 2024-02-16T22:21:46.441 INFO:tasks.workunit.client.0.smithi114.stdout:2/42: sync 2024-02-16T22:21:46.443 INFO:tasks.workunit.client.0.smithi114.stdout:2/43: write fc [732180,102297] 0 2024-02-16T22:21:46.444 INFO:tasks.workunit.client.1.smithi123.stdout:9/184: mkdir d4/d1e/d3a 0 2024-02-16T22:21:46.445 INFO:tasks.workunit.client.0.smithi114.stdout:9/23: creat d2/f6 x:0 0 0 2024-02-16T22:21:46.446 INFO:tasks.workunit.client.0.smithi114.stdout:5/45: dwrite d0/f9 [4194304,4194304] 0 2024-02-16T22:21:46.448 INFO:tasks.workunit.client.1.smithi123.stdout:8/151: rmdir d0/d12 39 2024-02-16T22:21:46.449 INFO:tasks.workunit.client.1.smithi123.stdout:8/152: readlink d0/df/l11 0 2024-02-16T22:21:46.452 INFO:tasks.workunit.client.0.smithi114.stdout:2/44: dwrite f0 [0,4194304] 0 2024-02-16T22:21:46.452 INFO:tasks.workunit.client.1.smithi123.stdout:9/185: dwrite d4/d1e/f39 [0,4194304] 0 2024-02-16T22:21:46.455 INFO:tasks.workunit.client.0.smithi114.stdout:2/45: dread f4 [0,4194304] 0 2024-02-16T22:21:46.458 INFO:tasks.workunit.client.0.smithi114.stdout:5/46: creat d0/fa x:0 0 0 2024-02-16T22:21:46.458 INFO:tasks.workunit.client.1.smithi123.stdout:8/153: dwrite d0/d17/d26/f2d [0,4194304] 0 2024-02-16T22:21:46.459 INFO:tasks.workunit.client.0.smithi114.stdout:5/47: dread - d0/fa zero size 2024-02-16T22:21:46.462 INFO:tasks.workunit.client.0.smithi114.stdout:2/46: mkdir dd/df 0 2024-02-16T22:21:46.464 INFO:tasks.workunit.client.1.smithi123.stdout:0/112: unlink d1/d8/c10 0 2024-02-16T22:21:46.466 INFO:tasks.workunit.client.1.smithi123.stdout:2/135: creat d4/d1a/d1d/f2c x:0 0 0 2024-02-16T22:21:46.467 INFO:tasks.workunit.client.0.smithi114.stdout:2/47: rmdir dd 39 2024-02-16T22:21:46.469 INFO:tasks.workunit.client.1.smithi123.stdout:7/113: creat d3/f2a x:0 0 0 2024-02-16T22:21:46.470 INFO:tasks.workunit.client.0.smithi114.stdout:9/24: getdents d2 0 2024-02-16T22:21:46.471 INFO:tasks.workunit.client.0.smithi114.stdout:9/25: write d2/f6 [481060,126303] 0 2024-02-16T22:21:46.472 INFO:tasks.workunit.client.1.smithi123.stdout:7/114: dread d3/d12/f14 [0,4194304] 0 2024-02-16T22:21:46.472 INFO:tasks.workunit.client.0.smithi114.stdout:2/48: chown dd/df 51 1 2024-02-16T22:21:46.474 INFO:tasks.workunit.client.0.smithi114.stdout:9/26: symlink d2/l7 0 2024-02-16T22:21:46.477 INFO:tasks.workunit.client.0.smithi114.stdout:2/49: dread fc [0,4194304] 0 2024-02-16T22:21:46.478 INFO:tasks.workunit.client.0.smithi114.stdout:9/27: creat d2/f8 x:0 0 0 2024-02-16T22:21:46.478 INFO:tasks.workunit.client.0.smithi114.stdout:2/50: dread - fa zero size 2024-02-16T22:21:46.480 INFO:tasks.workunit.client.0.smithi114.stdout:9/28: symlink d2/l9 0 2024-02-16T22:21:46.482 INFO:tasks.workunit.client.0.smithi114.stdout:9/29: truncate d2/f8 359197 0 2024-02-16T22:21:46.482 INFO:tasks.workunit.client.1.smithi123.stdout:8/154: mknod d0/d17/c37 0 2024-02-16T22:21:46.483 INFO:tasks.workunit.client.0.smithi114.stdout:2/51: dread f8 [0,4194304] 0 2024-02-16T22:21:46.483 INFO:tasks.workunit.client.0.smithi114.stdout:2/52: chown f4 202264 1 2024-02-16T22:21:46.483 INFO:tasks.workunit.client.1.smithi123.stdout:0/113: creat d1/d8/d13/f20 x:0 0 0 2024-02-16T22:21:46.483 INFO:tasks.workunit.client.1.smithi123.stdout:2/136: symlink d4/d1a/l2d 0 2024-02-16T22:21:46.485 INFO:tasks.workunit.client.0.smithi114.stdout:2/53: fdatasync fc 0 2024-02-16T22:21:46.486 INFO:tasks.workunit.client.1.smithi123.stdout:6/117: link d0/d6/d13/f22 d0/d29/f2d 0 2024-02-16T22:21:46.487 INFO:tasks.workunit.client.1.smithi123.stdout:6/118: dread - d0/d9/dc/f17 zero size 2024-02-16T22:21:46.488 INFO:tasks.workunit.client.1.smithi123.stdout:7/115: rename d3/f1c to d3/d12/d1d/f2b 0 2024-02-16T22:21:46.488 INFO:tasks.workunit.client.1.smithi123.stdout:6/119: write d0/d9/dc/f17 [365826,121618] 0 2024-02-16T22:21:46.489 INFO:tasks.workunit.client.0.smithi114.stdout:5/48: fsync d0/fa 0 2024-02-16T22:21:46.489 INFO:tasks.workunit.client.0.smithi114.stdout:5/49: write d0/f8 [3916271,59384] 0 2024-02-16T22:21:46.489 INFO:tasks.workunit.client.0.smithi114.stdout:2/54: creat dd/f10 x:0 0 0 2024-02-16T22:21:46.490 INFO:tasks.workunit.client.0.smithi114.stdout:2/55: write fc [1542421,46954] 0 2024-02-16T22:21:46.492 INFO:tasks.workunit.client.1.smithi123.stdout:9/186: creat d4/d33/f3b x:0 0 0 2024-02-16T22:21:46.497 INFO:tasks.workunit.client.0.smithi114.stdout:2/56: creat dd/df/f11 x:0 0 0 2024-02-16T22:21:46.499 INFO:tasks.workunit.client.1.smithi123.stdout:2/137: rename l3 to d4/d1a/d1d/l2e 0 2024-02-16T22:21:46.500 INFO:tasks.workunit.client.0.smithi114.stdout:2/57: mknod dd/c12 0 2024-02-16T22:21:46.501 INFO:tasks.workunit.client.0.smithi114.stdout:2/58: fdatasync dd/df/f11 0 2024-02-16T22:21:46.501 INFO:tasks.workunit.client.0.smithi114.stdout:2/59: readlink l7 0 2024-02-16T22:21:46.501 INFO:tasks.workunit.client.1.smithi123.stdout:6/120: mkdir d0/d6/d1a/d2e 0 2024-02-16T22:21:46.502 INFO:tasks.workunit.client.1.smithi123.stdout:4/149: truncate f4 1530339 0 2024-02-16T22:21:46.502 INFO:tasks.workunit.client.1.smithi123.stdout:6/121: write d0/d9/dc/f23 [613933,89319] 0 2024-02-16T22:21:46.503 INFO:tasks.workunit.client.0.smithi114.stdout:9/30: getdents d2 0 2024-02-16T22:21:46.504 INFO:tasks.workunit.client.1.smithi123.stdout:1/93: getdents d3/d6 0 2024-02-16T22:21:46.506 INFO:tasks.workunit.client.0.smithi114.stdout:2/60: rename f8 to dd/f13 0 2024-02-16T22:21:46.506 INFO:tasks.workunit.client.0.smithi114.stdout:2/61: write f0 [1813302,46436] 0 2024-02-16T22:21:46.508 INFO:tasks.workunit.client.0.smithi114.stdout:9/31: rename d2/l7 to d2/la 0 2024-02-16T22:21:46.510 INFO:tasks.workunit.client.0.smithi114.stdout:2/62: dread f0 [0,4194304] 0 2024-02-16T22:21:46.511 INFO:tasks.workunit.client.0.smithi114.stdout:2/63: write f0 [4259296,20759] 0 2024-02-16T22:21:46.514 INFO:tasks.workunit.client.0.smithi114.stdout:9/32: creat d2/fb x:0 0 0 2024-02-16T22:21:46.514 INFO:tasks.workunit.client.0.smithi114.stdout:9/33: chown d2 160 1 2024-02-16T22:21:46.514 INFO:tasks.workunit.client.0.smithi114.stdout:9/34: write d2/f6 [28494,4863] 0 2024-02-16T22:21:46.516 INFO:tasks.workunit.client.1.smithi123.stdout:8/155: write d0/fc [1874029,57374] 0 2024-02-16T22:21:46.517 INFO:tasks.workunit.client.0.smithi114.stdout:2/64: creat dd/df/f14 x:0 0 0 2024-02-16T22:21:46.517 INFO:tasks.workunit.client.0.smithi114.stdout:2/65: chown l2 664192 1 2024-02-16T22:21:46.520 INFO:tasks.workunit.client.1.smithi123.stdout:4/150: rename d5/d1c/d23/d2c to d5/d1c/d23/d31 0 2024-02-16T22:21:46.520 INFO:tasks.workunit.client.1.smithi123.stdout:6/122: dread d0/d6/f8 [0,4194304] 0 2024-02-16T22:21:46.521 INFO:tasks.workunit.client.0.smithi114.stdout:9/35: creat d2/fc x:0 0 0 2024-02-16T22:21:46.522 INFO:tasks.workunit.client.0.smithi114.stdout:9/36: dread - d2/fc zero size 2024-02-16T22:21:46.523 INFO:tasks.workunit.client.1.smithi123.stdout:3/104: sync 2024-02-16T22:21:46.523 INFO:tasks.workunit.client.0.smithi114.stdout:2/66: mknod dd/c15 0 2024-02-16T22:21:46.524 INFO:tasks.workunit.client.1.smithi123.stdout:6/123: dread d0/d6/d13/f2a [0,4194304] 0 2024-02-16T22:21:46.526 INFO:tasks.workunit.client.0.smithi114.stdout:9/37: mkdir d2/dd 0 2024-02-16T22:21:46.528 INFO:tasks.workunit.client.0.smithi114.stdout:9/38: chown d2/l9 15 1 2024-02-16T22:21:46.529 INFO:tasks.workunit.client.1.smithi123.stdout:7/116: truncate d3/d12/f1a 159630 0 2024-02-16T22:21:46.530 INFO:tasks.workunit.client.1.smithi123.stdout:7/117: write d3/d5/ff [1944376,17501] 0 2024-02-16T22:21:46.531 INFO:tasks.workunit.client.0.smithi114.stdout:2/67: dwrite fc [0,4194304] 0 2024-02-16T22:21:46.534 INFO:tasks.workunit.client.0.smithi114.stdout:2/68: creat dd/f16 x:0 0 0 2024-02-16T22:21:46.536 INFO:tasks.workunit.client.1.smithi123.stdout:4/151: rename d5/c16 to d5/d1c/d23/c32 0 2024-02-16T22:21:46.537 INFO:tasks.workunit.client.0.smithi114.stdout:9/39: symlink d2/dd/le 0 2024-02-16T22:21:46.537 INFO:tasks.workunit.client.1.smithi123.stdout:4/152: write d5/f12 [4700133,18055] 0 2024-02-16T22:21:46.537 INFO:tasks.workunit.client.0.smithi114.stdout:9/40: chown d2/f6 7 1 2024-02-16T22:21:46.538 INFO:tasks.workunit.client.1.smithi123.stdout:4/153: read d5/d1c/f21 [185472,76896] 0 2024-02-16T22:21:46.538 INFO:tasks.workunit.client.1.smithi123.stdout:1/94: symlink d3/d6/db/l20 0 2024-02-16T22:21:46.539 INFO:tasks.workunit.client.1.smithi123.stdout:9/187: truncate d4/d17/f1f 2889748 0 2024-02-16T22:21:46.539 INFO:tasks.workunit.client.1.smithi123.stdout:4/154: chown d5/f26 1081481553 1 2024-02-16T22:21:46.540 INFO:tasks.workunit.client.0.smithi114.stdout:2/69: creat dd/f17 x:0 0 0 2024-02-16T22:21:46.541 INFO:tasks.workunit.client.1.smithi123.stdout:4/155: write d5/f17 [2589461,44789] 0 2024-02-16T22:21:46.541 INFO:tasks.workunit.client.1.smithi123.stdout:3/105: chown d0/f7 10243 1 2024-02-16T22:21:46.541 INFO:tasks.workunit.client.1.smithi123.stdout:5/97: sync 2024-02-16T22:21:46.542 INFO:tasks.workunit.client.0.smithi114.stdout:9/41: creat d2/dd/ff x:0 0 0 2024-02-16T22:21:46.542 INFO:tasks.workunit.client.1.smithi123.stdout:3/106: fdatasync d0/f1f 0 2024-02-16T22:21:46.542 INFO:tasks.workunit.client.1.smithi123.stdout:4/156: dread - d5/d1c/f24 zero size 2024-02-16T22:21:46.543 INFO:tasks.workunit.client.1.smithi123.stdout:6/124: rmdir d0/d29 39 2024-02-16T22:21:46.545 INFO:tasks.workunit.client.0.smithi114.stdout:2/70: symlink dd/l18 0 2024-02-16T22:21:46.546 INFO:tasks.workunit.client.1.smithi123.stdout:0/114: link d1/d4/d17/l19 d1/d4/l21 0 2024-02-16T22:21:46.546 INFO:tasks.workunit.client.1.smithi123.stdout:9/188: dwrite d4/d17/d20/f22 [0,4194304] 0 2024-02-16T22:21:46.547 INFO:tasks.workunit.client.0.smithi114.stdout:9/42: creat d2/dd/f10 x:0 0 0 2024-02-16T22:21:46.547 INFO:tasks.workunit.client.1.smithi123.stdout:0/115: write d1/d8/d13/f20 [436854,83412] 0 2024-02-16T22:21:46.548 INFO:tasks.workunit.client.1.smithi123.stdout:9/189: fdatasync d4/fe 0 2024-02-16T22:21:46.549 INFO:tasks.workunit.client.1.smithi123.stdout:0/116: chown d1/d8/d13/d1d 376069 1 2024-02-16T22:21:46.549 INFO:tasks.workunit.client.0.smithi114.stdout:9/43: mknod d2/c11 0 2024-02-16T22:21:46.549 INFO:tasks.workunit.client.0.smithi114.stdout:9/44: readlink d2/dd/le 0 2024-02-16T22:21:46.551 INFO:tasks.workunit.client.1.smithi123.stdout:8/156: creat d0/df/d25/d34/f38 x:0 0 0 2024-02-16T22:21:46.551 INFO:tasks.workunit.client.1.smithi123.stdout:0/117: write d1/d8/d13/f15 [2553147,180] 0 2024-02-16T22:21:46.551 INFO:tasks.workunit.client.1.smithi123.stdout:8/157: fsync d0/fd 0 2024-02-16T22:21:46.552 INFO:tasks.workunit.client.0.smithi114.stdout:9/45: creat d2/f12 x:0 0 0 2024-02-16T22:21:46.552 INFO:tasks.workunit.client.1.smithi123.stdout:0/118: dread - d1/f1f zero size 2024-02-16T22:21:46.553 INFO:tasks.workunit.client.0.smithi114.stdout:9/46: creat d2/f13 x:0 0 0 2024-02-16T22:21:46.554 INFO:tasks.workunit.client.1.smithi123.stdout:0/119: truncate d1/d8/d13/f20 1519046 0 2024-02-16T22:21:46.554 INFO:tasks.workunit.client.0.smithi114.stdout:9/47: write f1 [4296456,5341] 0 2024-02-16T22:21:46.555 INFO:tasks.workunit.client.0.smithi114.stdout:2/71: fsync fc 0 2024-02-16T22:21:46.555 INFO:tasks.workunit.client.0.smithi114.stdout:2/72: read f0 [1550212,49435] 0 2024-02-16T22:21:46.560 INFO:tasks.workunit.client.0.smithi114.stdout:9/48: mknod d2/c14 0 2024-02-16T22:21:46.561 INFO:tasks.workunit.client.0.smithi114.stdout:9/49: dread - d2/fc zero size 2024-02-16T22:21:46.561 INFO:tasks.workunit.client.0.smithi114.stdout:9/50: dread - d2/dd/ff zero size 2024-02-16T22:21:46.561 INFO:tasks.workunit.client.1.smithi123.stdout:4/157: readlink d5/l1a 0 2024-02-16T22:21:46.562 INFO:tasks.workunit.client.0.smithi114.stdout:2/73: mknod dd/df/c19 0 2024-02-16T22:21:46.563 INFO:tasks.workunit.client.1.smithi123.stdout:3/107: dread d0/ff [0,4194304] 0 2024-02-16T22:21:46.563 INFO:tasks.workunit.client.0.smithi114.stdout:2/74: truncate fa 76306 0 2024-02-16T22:21:46.564 INFO:tasks.workunit.client.1.smithi123.stdout:5/98: rename d6/dd/le to d6/dd/df/l20 0 2024-02-16T22:21:46.566 INFO:tasks.workunit.client.1.smithi123.stdout:6/125: rename d0/d6/d13 to d0/d6/d2f 0 2024-02-16T22:21:46.568 INFO:tasks.workunit.client.1.smithi123.stdout:2/138: fsync d4/d11/f24 0 2024-02-16T22:21:46.568 INFO:tasks.workunit.client.0.smithi114.stdout:2/75: rename dd/l18 to dd/l1a 0 2024-02-16T22:21:46.569 INFO:tasks.workunit.client.0.smithi114.stdout:9/51: dwrite d2/dd/f10 [0,4194304] 0 2024-02-16T22:21:46.569 INFO:tasks.workunit.client.1.smithi123.stdout:9/190: creat d4/d17/d20/f3c x:0 0 0 2024-02-16T22:21:46.570 INFO:tasks.workunit.client.0.smithi114.stdout:9/52: dread - d2/f12 zero size 2024-02-16T22:21:46.572 INFO:tasks.workunit.client.1.smithi123.stdout:7/118: symlink d3/d26/l2c 0 2024-02-16T22:21:46.574 INFO:tasks.workunit.client.0.smithi114.stdout:2/76: mknod dd/df/c1b 0 2024-02-16T22:21:46.578 INFO:tasks.workunit.client.0.smithi114.stdout:2/77: mkdir dd/df/d1c 0 2024-02-16T22:21:46.579 INFO:tasks.workunit.client.1.smithi123.stdout:1/95: creat d3/d6/d18/f21 x:0 0 0 2024-02-16T22:21:46.580 INFO:tasks.workunit.client.0.smithi114.stdout:9/53: dwrite d2/f8 [0,4194304] 0 2024-02-16T22:21:46.589 INFO:tasks.workunit.client.0.smithi114.stdout:2/78: dwrite dd/f13 [0,4194304] 0 2024-02-16T22:21:46.590 INFO:tasks.workunit.client.0.smithi114.stdout:2/79: truncate dd/f10 461729 0 2024-02-16T22:21:46.595 INFO:tasks.workunit.client.0.smithi114.stdout:2/80: creat dd/f1d x:0 0 0 2024-02-16T22:21:46.596 INFO:tasks.workunit.client.0.smithi114.stdout:2/81: stat dd/f16 0 2024-02-16T22:21:46.601 INFO:tasks.workunit.client.1.smithi123.stdout:7/119: rmdir d3/d12 39 2024-02-16T22:21:46.601 INFO:tasks.workunit.client.1.smithi123.stdout:8/158: creat d0/d29/f39 x:0 0 0 2024-02-16T22:21:46.603 INFO:tasks.workunit.client.1.smithi123.stdout:4/158: rename d5/d1c/l2b to d5/l33 0 2024-02-16T22:21:46.603 INFO:tasks.workunit.client.0.smithi114.stdout:9/54: truncate d2/f8 4043425 0 2024-02-16T22:21:46.604 INFO:tasks.workunit.client.0.smithi114.stdout:2/82: dwrite f0 [0,4194304] 0 2024-02-16T22:21:46.605 INFO:tasks.workunit.client.0.smithi114.stdout:9/55: truncate d2/dd/ff 946440 0 2024-02-16T22:21:46.607 INFO:tasks.workunit.client.1.smithi123.stdout:9/191: fdatasync d4/d17/f1f 0 2024-02-16T22:21:46.608 INFO:tasks.workunit.client.0.smithi114.stdout:2/83: creat dd/df/f1e x:0 0 0 2024-02-16T22:21:46.610 INFO:tasks.workunit.client.0.smithi114.stdout:2/84: mkdir dd/d1f 0 2024-02-16T22:21:46.612 INFO:tasks.workunit.client.0.smithi114.stdout:0/90: sync 2024-02-16T22:21:46.612 INFO:tasks.workunit.client.0.smithi114.stdout:1/50: sync 2024-02-16T22:21:46.613 INFO:tasks.workunit.client.0.smithi114.stdout:3/37: sync 2024-02-16T22:21:46.613 INFO:tasks.workunit.client.0.smithi114.stdout:8/71: sync 2024-02-16T22:21:46.613 INFO:tasks.workunit.client.0.smithi114.stdout:4/58: sync 2024-02-16T22:21:46.613 INFO:tasks.workunit.client.0.smithi114.stdout:4/59: fdatasync f9 0 2024-02-16T22:21:46.613 INFO:tasks.workunit.client.0.smithi114.stdout:8/72: readlink d4/l5 0 2024-02-16T22:21:46.614 INFO:tasks.workunit.client.0.smithi114.stdout:8/73: write d4/f9 [6225,65717] 0 2024-02-16T22:21:46.615 INFO:tasks.workunit.client.0.smithi114.stdout:8/74: stat d4/fe 0 2024-02-16T22:21:46.615 INFO:tasks.workunit.client.0.smithi114.stdout:8/75: rename d4 to d4/d18 22 2024-02-16T22:21:46.618 INFO:tasks.workunit.client.0.smithi114.stdout:1/51: rmdir d4 39 2024-02-16T22:21:46.619 INFO:tasks.workunit.client.0.smithi114.stdout:2/85: rmdir dd/df 39 2024-02-16T22:21:46.621 INFO:tasks.workunit.client.1.smithi123.stdout:6/126: dwrite d0/d29/f2d [0,4194304] 0 2024-02-16T22:21:46.622 INFO:tasks.workunit.client.0.smithi114.stdout:4/60: unlink l3 0 2024-02-16T22:21:46.625 INFO:tasks.workunit.client.0.smithi114.stdout:4/61: dread dd/ff [0,4194304] 0 2024-02-16T22:21:46.625 INFO:tasks.workunit.client.0.smithi114.stdout:4/62: readlink dd/l10 0 2024-02-16T22:21:46.626 INFO:tasks.workunit.client.0.smithi114.stdout:4/63: write dd/ff [13702,8071] 0 2024-02-16T22:21:46.627 INFO:tasks.workunit.client.0.smithi114.stdout:4/64: write f2 [166859,2165] 0 2024-02-16T22:21:46.628 INFO:tasks.workunit.client.0.smithi114.stdout:0/91: creat d1/d1a/f1d x:0 0 0 2024-02-16T22:21:46.630 INFO:tasks.workunit.client.0.smithi114.stdout:9/56: truncate d2/f8 3500415 0 2024-02-16T22:21:46.631 INFO:tasks.workunit.client.0.smithi114.stdout:9/57: read - d2/f12 zero size 2024-02-16T22:21:46.632 INFO:tasks.workunit.client.0.smithi114.stdout:8/76: creat d4/f19 x:0 0 0 2024-02-16T22:21:46.633 INFO:tasks.workunit.client.1.smithi123.stdout:6/127: fdatasync d0/d6/d2f/f22 0 2024-02-16T22:21:46.634 INFO:tasks.workunit.client.0.smithi114.stdout:8/77: write d4/d11/d15/f10 [289397,94944] 0 2024-02-16T22:21:46.636 INFO:tasks.workunit.client.0.smithi114.stdout:0/92: dwrite d1/fe [0,4194304] 0 2024-02-16T22:21:46.636 INFO:tasks.workunit.client.0.smithi114.stdout:8/78: chown d4/d11/d15/l12 117512 1 2024-02-16T22:21:46.637 INFO:tasks.workunit.client.0.smithi114.stdout:8/79: fdatasync d4/f19 0 2024-02-16T22:21:46.638 INFO:tasks.workunit.client.0.smithi114.stdout:8/80: stat d4/d11/d15 0 2024-02-16T22:21:46.645 INFO:tasks.workunit.client.0.smithi114.stdout:4/65: creat dd/f12 x:0 0 0 2024-02-16T22:21:46.646 INFO:tasks.workunit.client.0.smithi114.stdout:9/58: creat d2/f15 x:0 0 0 2024-02-16T22:21:46.646 INFO:tasks.workunit.client.0.smithi114.stdout:0/93: dread d1/fa [0,4194304] 0 2024-02-16T22:21:46.652 INFO:tasks.workunit.client.0.smithi114.stdout:8/81: rmdir d4/d11/d15 39 2024-02-16T22:21:46.652 INFO:tasks.workunit.client.0.smithi114.stdout:0/94: dwrite d1/f15 [0,4194304] 0 2024-02-16T22:21:46.653 INFO:tasks.workunit.client.0.smithi114.stdout:3/38: link f1 d4/fb 0 2024-02-16T22:21:46.653 INFO:tasks.workunit.client.0.smithi114.stdout:0/95: fsync d1/f8 0 2024-02-16T22:21:46.655 INFO:tasks.workunit.client.0.smithi114.stdout:4/66: creat dd/f13 x:0 0 0 2024-02-16T22:21:46.659 INFO:tasks.workunit.client.0.smithi114.stdout:1/52: mknod d4/cf 0 2024-02-16T22:21:46.661 INFO:tasks.workunit.client.0.smithi114.stdout:2/86: rename l7 to dd/l20 0 2024-02-16T22:21:46.662 INFO:tasks.workunit.client.0.smithi114.stdout:4/67: dwrite f4 [0,4194304] 0 2024-02-16T22:21:46.663 INFO:tasks.workunit.client.0.smithi114.stdout:4/68: readlink lc 0 2024-02-16T22:21:46.663 INFO:tasks.workunit.client.0.smithi114.stdout:2/87: write dd/f1d [565357,98818] 0 2024-02-16T22:21:46.666 INFO:tasks.workunit.client.0.smithi114.stdout:3/39: creat d4/fc x:0 0 0 2024-02-16T22:21:46.669 INFO:tasks.workunit.client.0.smithi114.stdout:1/53: rmdir d4 39 2024-02-16T22:21:46.670 INFO:tasks.workunit.client.0.smithi114.stdout:1/54: read f1 [2788484,16374] 0 2024-02-16T22:21:46.672 INFO:tasks.workunit.client.0.smithi114.stdout:0/96: rename d1/cf to d1/d16/c1e 0 2024-02-16T22:21:46.673 INFO:tasks.workunit.client.0.smithi114.stdout:4/69: mknod dd/c14 0 2024-02-16T22:21:46.681 INFO:tasks.workunit.client.0.smithi114.stdout:3/40: mkdir d4/dd 0 2024-02-16T22:21:46.683 INFO:tasks.workunit.client.0.smithi114.stdout:2/88: dwrite dd/df/f1e [0,4194304] 0 2024-02-16T22:21:46.683 INFO:tasks.workunit.client.0.smithi114.stdout:2/89: chown fc 2784 1 2024-02-16T22:21:46.688 INFO:tasks.workunit.client.0.smithi114.stdout:0/97: mkdir d1/d16/d1f 0 2024-02-16T22:21:46.692 INFO:tasks.workunit.client.1.smithi123.stdout:3/108: sync 2024-02-16T22:21:46.692 INFO:tasks.workunit.client.0.smithi114.stdout:2/90: dwrite dd/f10 [0,4194304] 0 2024-02-16T22:21:46.694 INFO:tasks.workunit.client.0.smithi114.stdout:8/82: link d4/f8 d4/d11/d15/f1a 0 2024-02-16T22:21:46.699 INFO:tasks.workunit.client.0.smithi114.stdout:3/41: chown d4/fb 50103 1 2024-02-16T22:21:46.699 INFO:tasks.workunit.client.0.smithi114.stdout:0/98: dwrite d1/fe [0,4194304] 0 2024-02-16T22:21:46.701 INFO:tasks.workunit.client.0.smithi114.stdout:4/70: dwrite dd/ff [0,4194304] 0 2024-02-16T22:21:46.705 INFO:tasks.workunit.client.0.smithi114.stdout:1/55: creat d4/d9/f10 x:0 0 0 2024-02-16T22:21:46.716 INFO:tasks.workunit.client.0.smithi114.stdout:8/83: write d4/f8 [89664,83046] 0 2024-02-16T22:21:46.718 INFO:tasks.workunit.client.0.smithi114.stdout:3/42: unlink f0 0 2024-02-16T22:21:46.720 INFO:tasks.workunit.client.0.smithi114.stdout:0/99: symlink d1/d16/d18/l20 0 2024-02-16T22:21:46.722 INFO:tasks.workunit.client.0.smithi114.stdout:4/71: rename f4 to dd/f15 0 2024-02-16T22:21:46.725 INFO:tasks.workunit.client.0.smithi114.stdout:1/56: write d4/f5 [363043,74958] 0 2024-02-16T22:21:46.732 INFO:tasks.workunit.client.0.smithi114.stdout:3/43: creat d4/fe x:0 0 0 2024-02-16T22:21:46.733 INFO:tasks.workunit.client.0.smithi114.stdout:0/100: symlink d1/d16/d18/l21 0 2024-02-16T22:21:46.734 INFO:tasks.workunit.client.0.smithi114.stdout:0/101: stat d1/fe 0 2024-02-16T22:21:46.740 INFO:tasks.workunit.client.1.smithi123.stdout:2/139: sync 2024-02-16T22:21:46.741 INFO:tasks.workunit.client.0.smithi114.stdout:2/91: rmdir dd/d1f 0 2024-02-16T22:21:46.741 INFO:tasks.workunit.client.0.smithi114.stdout:2/92: fsync fc 0 2024-02-16T22:21:46.742 INFO:tasks.workunit.client.0.smithi114.stdout:4/72: dwrite dd/f15 [4194304,4194304] 0 2024-02-16T22:21:46.742 INFO:tasks.workunit.client.0.smithi114.stdout:4/73: dread - dd/f11 zero size 2024-02-16T22:21:46.743 INFO:tasks.workunit.client.0.smithi114.stdout:8/84: mknod d4/d11/c1b 0 2024-02-16T22:21:46.743 INFO:tasks.workunit.client.1.smithi123.stdout:2/140: dread d4/d11/f18 [0,4194304] 0 2024-02-16T22:21:46.745 INFO:tasks.workunit.client.0.smithi114.stdout:8/85: write d4/d11/d15/f1a [327149,46695] 0 2024-02-16T22:21:46.746 INFO:tasks.workunit.client.0.smithi114.stdout:3/44: symlink d4/lf 0 2024-02-16T22:21:46.747 INFO:tasks.workunit.client.0.smithi114.stdout:3/45: read - d4/f6 zero size 2024-02-16T22:21:46.748 INFO:tasks.workunit.client.0.smithi114.stdout:2/93: dread dd/df/f1e [0,4194304] 0 2024-02-16T22:21:46.751 INFO:tasks.workunit.client.0.smithi114.stdout:0/102: creat d1/d16/d18/f22 x:0 0 0 2024-02-16T22:21:46.754 INFO:tasks.workunit.client.0.smithi114.stdout:0/103: dread d1/f7 [0,4194304] 0 2024-02-16T22:21:46.755 INFO:tasks.workunit.client.0.smithi114.stdout:0/104: write d1/d16/d18/f22 [734847,47138] 0 2024-02-16T22:21:46.756 INFO:tasks.workunit.client.0.smithi114.stdout:4/74: creat dd/f16 x:0 0 0 2024-02-16T22:21:46.756 INFO:tasks.workunit.client.0.smithi114.stdout:4/75: fsync f2 0 2024-02-16T22:21:46.757 INFO:tasks.workunit.client.0.smithi114.stdout:4/76: chown ca 0 1 2024-02-16T22:21:46.759 INFO:tasks.workunit.client.0.smithi114.stdout:0/105: dread d1/fe [0,4194304] 0 2024-02-16T22:21:46.760 INFO:tasks.workunit.client.0.smithi114.stdout:0/106: truncate d1/d16/d18/f1c 931888 0 2024-02-16T22:21:46.760 INFO:tasks.workunit.client.0.smithi114.stdout:0/107: chown d1/c12 223733 1 2024-02-16T22:21:46.761 INFO:tasks.workunit.client.0.smithi114.stdout:0/108: write d1/f15 [3562664,84283] 0 2024-02-16T22:21:46.762 INFO:tasks.workunit.client.0.smithi114.stdout:3/46: write d4/fa [3363563,86788] 0 2024-02-16T22:21:46.767 INFO:tasks.workunit.client.0.smithi114.stdout:0/109: dwrite d1/f15 [0,4194304] 0 2024-02-16T22:21:46.778 INFO:tasks.workunit.client.0.smithi114.stdout:3/47: symlink d4/l10 0 2024-02-16T22:21:46.778 INFO:tasks.workunit.client.0.smithi114.stdout:5/50: sync 2024-02-16T22:21:46.778 INFO:tasks.workunit.client.0.smithi114.stdout:6/43: sync 2024-02-16T22:21:46.778 INFO:tasks.workunit.client.0.smithi114.stdout:7/52: sync 2024-02-16T22:21:46.780 INFO:tasks.workunit.client.0.smithi114.stdout:2/94: mkdir dd/df/d1c/d21 0 2024-02-16T22:21:46.784 INFO:tasks.workunit.client.0.smithi114.stdout:0/110: mknod d1/d1a/c23 0 2024-02-16T22:21:46.786 INFO:tasks.workunit.client.0.smithi114.stdout:3/48: dwrite d4/fe [0,4194304] 0 2024-02-16T22:21:46.787 INFO:tasks.workunit.client.0.smithi114.stdout:0/111: write d1/d16/d18/f1c [765345,93235] 0 2024-02-16T22:21:46.792 INFO:tasks.workunit.client.0.smithi114.stdout:6/44: symlink d6/l10 0 2024-02-16T22:21:46.793 INFO:tasks.workunit.client.0.smithi114.stdout:1/57: dwrite d4/fa [0,4194304] 0 2024-02-16T22:21:46.793 INFO:tasks.workunit.client.0.smithi114.stdout:6/45: read d6/fc [322601,26401] 0 2024-02-16T22:21:46.794 INFO:tasks.workunit.client.0.smithi114.stdout:1/58: write d4/f5 [985411,26924] 0 2024-02-16T22:21:46.801 INFO:tasks.workunit.client.0.smithi114.stdout:3/49: unlink d4/lf 0 2024-02-16T22:21:46.802 INFO:tasks.workunit.client.0.smithi114.stdout:1/59: dwrite d4/fd [0,4194304] 0 2024-02-16T22:21:46.803 INFO:tasks.workunit.client.0.smithi114.stdout:1/60: chown f1 0 1 2024-02-16T22:21:46.804 INFO:tasks.workunit.client.0.smithi114.stdout:1/61: chown d4/c6 11580820 1 2024-02-16T22:21:46.805 INFO:tasks.workunit.client.0.smithi114.stdout:8/86: write d4/fc [4920,38551] 0 2024-02-16T22:21:46.807 INFO:tasks.workunit.client.0.smithi114.stdout:8/87: readlink d4/d11/d15/l12 0 2024-02-16T22:21:46.807 INFO:tasks.workunit.client.0.smithi114.stdout:3/50: dread d4/fe [0,4194304] 0 2024-02-16T22:21:46.815 INFO:tasks.workunit.client.0.smithi114.stdout:8/88: dwrite d4/fe [0,4194304] 0 2024-02-16T22:21:46.816 INFO:tasks.workunit.client.0.smithi114.stdout:8/89: chown d4/d11/d15/f1a 0 1 2024-02-16T22:21:46.818 INFO:tasks.workunit.client.0.smithi114.stdout:1/62: creat d4/d9/f11 x:0 0 0 2024-02-16T22:21:46.820 INFO:tasks.workunit.client.0.smithi114.stdout:1/63: fsync d4/d9/f10 0 2024-02-16T22:21:46.826 INFO:tasks.workunit.client.0.smithi114.stdout:3/51: symlink d4/l11 0 2024-02-16T22:21:46.826 INFO:tasks.workunit.client.0.smithi114.stdout:8/90: dwrite d4/f6 [0,4194304] 0 2024-02-16T22:21:46.828 INFO:tasks.workunit.client.0.smithi114.stdout:1/64: dwrite f1 [0,4194304] 0 2024-02-16T22:21:46.829 INFO:tasks.workunit.client.0.smithi114.stdout:1/65: stat l3 0 2024-02-16T22:21:46.829 INFO:tasks.workunit.client.0.smithi114.stdout:1/66: fdatasync d4/fa 0 2024-02-16T22:21:46.836 INFO:tasks.workunit.client.0.smithi114.stdout:4/77: truncate dd/ff 3364743 0 2024-02-16T22:21:46.836 INFO:tasks.workunit.client.0.smithi114.stdout:4/78: dread - dd/f16 zero size 2024-02-16T22:21:46.842 INFO:tasks.workunit.client.0.smithi114.stdout:7/53: creat d3/fe x:0 0 0 2024-02-16T22:21:46.845 INFO:tasks.workunit.client.0.smithi114.stdout:4/79: dwrite f9 [0,4194304] 0 2024-02-16T22:21:46.846 INFO:tasks.workunit.client.0.smithi114.stdout:8/91: rmdir d4/d11/d15 39 2024-02-16T22:21:46.853 INFO:tasks.workunit.client.0.smithi114.stdout:1/67: creat d4/d9/f12 x:0 0 0 2024-02-16T22:21:46.855 INFO:tasks.workunit.client.0.smithi114.stdout:5/51: getdents d0 0 2024-02-16T22:21:46.857 INFO:tasks.workunit.client.0.smithi114.stdout:5/52: write d0/f8 [4026648,66147] 0 2024-02-16T22:21:46.858 INFO:tasks.workunit.client.1.smithi123.stdout:1/96: symlink d3/d6/db/l22 0 2024-02-16T22:21:46.859 INFO:tasks.workunit.client.0.smithi114.stdout:0/112: dwrite d1/fa [0,4194304] 0 2024-02-16T22:21:46.860 INFO:tasks.workunit.client.1.smithi123.stdout:0/120: creat d1/f22 x:0 0 0 2024-02-16T22:21:46.861 INFO:tasks.workunit.client.1.smithi123.stdout:7/120: mknod d3/d5/c2d 0 2024-02-16T22:21:46.862 INFO:tasks.workunit.client.0.smithi114.stdout:4/80: mknod dd/c17 0 2024-02-16T22:21:46.863 INFO:tasks.workunit.client.1.smithi123.stdout:8/159: symlink d0/d29/l3a 0 2024-02-16T22:21:46.865 INFO:tasks.workunit.client.1.smithi123.stdout:4/159: symlink d5/d1c/d23/d31/l34 0 2024-02-16T22:21:46.867 INFO:tasks.workunit.client.0.smithi114.stdout:8/92: dread d4/d11/d15/f10 [0,4194304] 0 2024-02-16T22:21:46.868 INFO:tasks.workunit.client.1.smithi123.stdout:5/99: link d6/dd/df/f1b d6/f21 0 2024-02-16T22:21:46.868 INFO:tasks.workunit.client.1.smithi123.stdout:5/100: readlink l0 0 2024-02-16T22:21:46.868 INFO:tasks.workunit.client.0.smithi114.stdout:8/93: write d4/fc [2797684,89989] 0 2024-02-16T22:21:46.869 INFO:tasks.workunit.client.0.smithi114.stdout:8/94: fdatasync f0 0 2024-02-16T22:21:46.869 INFO:tasks.workunit.client.0.smithi114.stdout:8/95: readlink d4/d11/d15/l12 0 2024-02-16T22:21:46.871 INFO:tasks.workunit.client.0.smithi114.stdout:8/96: write d4/d11/d15/f10 [589100,96635] 0 2024-02-16T22:21:46.871 INFO:tasks.workunit.client.1.smithi123.stdout:9/192: write d4/d17/f1f [1436837,38345] 0 2024-02-16T22:21:46.873 INFO:tasks.workunit.client.0.smithi114.stdout:5/53: creat d0/fb x:0 0 0 2024-02-16T22:21:46.877 INFO:tasks.workunit.client.0.smithi114.stdout:4/81: creat dd/f18 x:0 0 0 2024-02-16T22:21:46.877 INFO:tasks.workunit.client.0.smithi114.stdout:0/113: fsync d1/fa 0 2024-02-16T22:21:46.878 INFO:tasks.workunit.client.0.smithi114.stdout:6/46: truncate d6/fc 2363271 0 2024-02-16T22:21:46.884 INFO:tasks.workunit.client.1.smithi123.stdout:1/97: stat d3/d6/db/d10/c1c 0 2024-02-16T22:21:46.887 INFO:tasks.workunit.client.0.smithi114.stdout:5/54: mknod d0/cc 0 2024-02-16T22:21:46.890 INFO:tasks.workunit.client.0.smithi114.stdout:4/82: rename dd/f18 to dd/f19 0 2024-02-16T22:21:46.892 INFO:tasks.workunit.client.0.smithi114.stdout:4/83: truncate dd/f16 993978 0 2024-02-16T22:21:46.892 INFO:tasks.workunit.client.1.smithi123.stdout:8/160: unlink d0/fc 0 2024-02-16T22:21:46.894 INFO:tasks.workunit.client.0.smithi114.stdout:0/114: creat d1/d16/f24 x:0 0 0 2024-02-16T22:21:46.894 INFO:tasks.workunit.client.0.smithi114.stdout:3/52: dwrite f1 [0,4194304] 0 2024-02-16T22:21:46.896 INFO:tasks.workunit.client.1.smithi123.stdout:5/101: creat d6/dd/df/f22 x:0 0 0 2024-02-16T22:21:46.898 INFO:tasks.workunit.client.1.smithi123.stdout:9/193: write d4/f10 [531417,72934] 0 2024-02-16T22:21:46.899 INFO:tasks.workunit.client.1.smithi123.stdout:2/141: dread d4/d1a/d1d/d13/d15/d1b/f28 [0,4194304] 0 2024-02-16T22:21:46.900 INFO:tasks.workunit.client.0.smithi114.stdout:1/68: creat d4/f13 x:0 0 0 2024-02-16T22:21:46.900 INFO:tasks.workunit.client.1.smithi123.stdout:9/194: stat d4/f16 0 2024-02-16T22:21:46.901 INFO:tasks.workunit.client.0.smithi114.stdout:4/84: dwrite dd/f12 [0,4194304] 0 2024-02-16T22:21:46.904 INFO:tasks.workunit.client.0.smithi114.stdout:5/55: creat d0/fd x:0 0 0 2024-02-16T22:21:46.906 INFO:tasks.workunit.client.0.smithi114.stdout:5/56: truncate d0/f9 8794435 0 2024-02-16T22:21:46.906 INFO:tasks.workunit.client.1.smithi123.stdout:1/98: creat d3/d6/d11/f23 x:0 0 0 2024-02-16T22:21:46.907 INFO:tasks.workunit.client.0.smithi114.stdout:9/59: sync 2024-02-16T22:21:46.908 INFO:tasks.workunit.client.1.smithi123.stdout:0/121: symlink d1/d8/d13/d1d/l23 0 2024-02-16T22:21:46.909 INFO:tasks.workunit.client.0.smithi114.stdout:9/60: chown d2/f15 4 1 2024-02-16T22:21:46.909 INFO:tasks.workunit.client.1.smithi123.stdout:0/122: truncate d1/f12 934223 0 2024-02-16T22:21:46.916 INFO:tasks.workunit.client.1.smithi123.stdout:3/109: write d0/f6 [2338966,123652] 0 2024-02-16T22:21:46.916 INFO:tasks.workunit.client.0.smithi114.stdout:6/47: dread d6/fc [0,4194304] 0 2024-02-16T22:21:46.917 INFO:tasks.workunit.client.0.smithi114.stdout:6/48: readlink d6/da/lf 0 2024-02-16T22:21:46.918 INFO:tasks.workunit.client.0.smithi114.stdout:8/97: creat d4/d11/f1c x:0 0 0 2024-02-16T22:21:46.919 INFO:tasks.workunit.client.0.smithi114.stdout:8/98: write d4/f19 [18959,122714] 0 2024-02-16T22:21:46.919 INFO:tasks.workunit.client.1.smithi123.stdout:6/128: creat d0/f30 x:0 0 0 2024-02-16T22:21:46.921 INFO:tasks.workunit.client.1.smithi123.stdout:2/142: symlink d4/d1a/l2f 0 2024-02-16T22:21:46.922 INFO:tasks.workunit.client.1.smithi123.stdout:7/121: write d3/f8 [4761858,25474] 0 2024-02-16T22:21:46.924 INFO:tasks.workunit.client.1.smithi123.stdout:9/195: creat d4/d17/d24/f3d x:0 0 0 2024-02-16T22:21:46.924 INFO:tasks.workunit.client.1.smithi123.stdout:9/196: dread - d4/d33/f3b zero size 2024-02-16T22:21:46.925 INFO:tasks.workunit.client.0.smithi114.stdout:3/53: creat d4/dd/f12 x:0 0 0 2024-02-16T22:21:46.926 INFO:tasks.workunit.client.1.smithi123.stdout:1/99: mkdir d3/d6/d24 0 2024-02-16T22:21:46.927 INFO:tasks.workunit.client.1.smithi123.stdout:1/100: fsync d3/d6/d11/f1f 0 2024-02-16T22:21:46.927 INFO:tasks.workunit.client.0.smithi114.stdout:8/99: dwrite d4/f6 [0,4194304] 0 2024-02-16T22:21:46.927 INFO:tasks.workunit.client.0.smithi114.stdout:8/100: stat d4/ld 0 2024-02-16T22:21:46.928 INFO:tasks.workunit.client.0.smithi114.stdout:8/101: fdatasync d4/d11/f1c 0 2024-02-16T22:21:46.928 INFO:tasks.workunit.client.0.smithi114.stdout:9/61: creat d2/f16 x:0 0 0 2024-02-16T22:21:46.928 INFO:tasks.workunit.client.0.smithi114.stdout:9/62: fdatasync d2/fb 0 2024-02-16T22:21:46.929 INFO:tasks.workunit.client.0.smithi114.stdout:8/102: read d4/d11/f17 [963696,112797] 0 2024-02-16T22:21:46.930 INFO:tasks.workunit.client.1.smithi123.stdout:0/123: unlink d1/d4/d17/c1e 0 2024-02-16T22:21:46.930 INFO:tasks.workunit.client.0.smithi114.stdout:9/63: write d2/f6 [755811,12784] 0 2024-02-16T22:21:46.930 INFO:tasks.workunit.client.0.smithi114.stdout:8/103: chown d4/f9 8 1 2024-02-16T22:21:46.931 INFO:tasks.workunit.client.0.smithi114.stdout:8/104: stat d4/fc 0 2024-02-16T22:21:46.931 INFO:tasks.workunit.client.0.smithi114.stdout:9/64: chown d2/fc 0 1 2024-02-16T22:21:46.933 INFO:tasks.workunit.client.1.smithi123.stdout:8/161: symlink d0/d17/d26/d2e/l3b 0 2024-02-16T22:21:46.934 INFO:tasks.workunit.client.1.smithi123.stdout:1/101: dwrite d3/d6/d18/f21 [0,4194304] 0 2024-02-16T22:21:46.935 INFO:tasks.workunit.client.1.smithi123.stdout:1/102: chown d3/d6/db/l20 14 1 2024-02-16T22:21:46.936 INFO:tasks.workunit.client.1.smithi123.stdout:1/103: write d3/f8 [2611105,84273] 0 2024-02-16T22:21:46.936 INFO:tasks.workunit.client.0.smithi114.stdout:1/69: mknod d4/c14 0 2024-02-16T22:21:46.936 INFO:tasks.workunit.client.1.smithi123.stdout:1/104: readlink d3/l16 0 2024-02-16T22:21:46.939 INFO:tasks.workunit.client.0.smithi114.stdout:6/49: symlink d6/da/l11 0 2024-02-16T22:21:46.941 INFO:tasks.workunit.client.1.smithi123.stdout:6/129: readlink d0/d9/dc/l24 0 2024-02-16T22:21:46.943 INFO:tasks.workunit.client.1.smithi123.stdout:2/143: write d4/f16 [3208454,19702] 0 2024-02-16T22:21:46.948 INFO:tasks.workunit.client.1.smithi123.stdout:9/197: symlink d4/d17/l3e 0 2024-02-16T22:21:46.950 INFO:tasks.workunit.client.0.smithi114.stdout:0/115: truncate d1/d16/d18/f1c 900488 0 2024-02-16T22:21:46.951 INFO:tasks.workunit.client.0.smithi114.stdout:3/54: creat d4/dd/f13 x:0 0 0 2024-02-16T22:21:46.953 INFO:tasks.workunit.client.1.smithi123.stdout:9/198: dwrite d4/f10 [0,4194304] 0 2024-02-16T22:21:46.958 INFO:tasks.workunit.client.0.smithi114.stdout:3/55: dwrite d4/fb [0,4194304] 0 2024-02-16T22:21:46.958 INFO:tasks.workunit.client.1.smithi123.stdout:0/124: rename d1/d4/l1c to d1/d8/d13/l24 0 2024-02-16T22:21:46.959 INFO:tasks.workunit.client.0.smithi114.stdout:3/56: write d4/fc [330716,104877] 0 2024-02-16T22:21:46.960 INFO:tasks.workunit.client.1.smithi123.stdout:0/125: truncate d1/f12 1658727 0 2024-02-16T22:21:46.961 INFO:tasks.workunit.client.1.smithi123.stdout:3/110: dread d0/fe [0,4194304] 0 2024-02-16T22:21:46.962 INFO:tasks.workunit.client.0.smithi114.stdout:9/65: mkdir d2/d17 0 2024-02-16T22:21:46.962 INFO:tasks.workunit.client.0.smithi114.stdout:3/57: write f1 [3032671,24374] 0 2024-02-16T22:21:46.962 INFO:tasks.workunit.client.0.smithi114.stdout:9/66: dread - d2/fb zero size 2024-02-16T22:21:46.964 INFO:tasks.workunit.client.1.smithi123.stdout:4/160: link d5/f29 d5/d1c/f35 0 2024-02-16T22:21:46.966 INFO:tasks.workunit.client.1.smithi123.stdout:0/126: dread d1/f12 [0,4194304] 0 2024-02-16T22:21:46.966 INFO:tasks.workunit.client.1.smithi123.stdout:3/111: dread d0/f6 [0,4194304] 0 2024-02-16T22:21:46.967 INFO:tasks.workunit.client.1.smithi123.stdout:0/127: read - d1/f1f zero size 2024-02-16T22:21:46.967 INFO:tasks.workunit.client.1.smithi123.stdout:4/161: write d5/d1c/d23/f2a [829609,6133] 0 2024-02-16T22:21:46.968 INFO:tasks.workunit.client.1.smithi123.stdout:4/162: dread - d5/f1f zero size 2024-02-16T22:21:46.969 INFO:tasks.workunit.client.1.smithi123.stdout:3/112: read - d0/f1f zero size 2024-02-16T22:21:46.969 INFO:tasks.workunit.client.0.smithi114.stdout:3/58: dwrite d4/fa [0,4194304] 0 2024-02-16T22:21:46.970 INFO:tasks.workunit.client.1.smithi123.stdout:0/128: write d1/d8/d13/f14 [2575143,86850] 0 2024-02-16T22:21:46.974 INFO:tasks.workunit.client.1.smithi123.stdout:4/163: dread d5/f1e [0,4194304] 0 2024-02-16T22:21:46.974 INFO:tasks.workunit.client.0.smithi114.stdout:0/116: creat d1/d16/f25 x:0 0 0 2024-02-16T22:21:46.975 INFO:tasks.workunit.client.0.smithi114.stdout:0/117: write d1/d16/f19 [993076,67479] 0 2024-02-16T22:21:46.976 INFO:tasks.workunit.client.1.smithi123.stdout:1/105: write d3/f5 [4105319,34796] 0 2024-02-16T22:21:46.977 INFO:tasks.workunit.client.0.smithi114.stdout:0/118: read d1/d16/d18/f22 [217248,75931] 0 2024-02-16T22:21:46.977 INFO:tasks.workunit.client.1.smithi123.stdout:1/106: fdatasync d3/d6/d11/f17 0 2024-02-16T22:21:46.980 INFO:tasks.workunit.client.0.smithi114.stdout:6/50: write d6/fc [2774332,95463] 0 2024-02-16T22:21:46.981 INFO:tasks.workunit.client.1.smithi123.stdout:6/130: write d0/d6/fd [2588594,2842] 0 2024-02-16T22:21:46.982 INFO:tasks.workunit.client.1.smithi123.stdout:7/122: write d3/d12/d1d/f2b [485900,84987] 0 2024-02-16T22:21:46.983 INFO:tasks.workunit.client.0.smithi114.stdout:4/85: truncate dd/ff 100202 0 2024-02-16T22:21:46.984 INFO:tasks.workunit.client.0.smithi114.stdout:0/119: dwrite d1/fa [0,4194304] 0 2024-02-16T22:21:46.986 INFO:tasks.workunit.client.0.smithi114.stdout:9/67: mknod d2/c18 0 2024-02-16T22:21:46.987 INFO:tasks.workunit.client.0.smithi114.stdout:9/68: stat d2/fc 0 2024-02-16T22:21:46.988 INFO:tasks.workunit.client.1.smithi123.stdout:6/131: dwrite d0/d29/f2d [0,4194304] 0 2024-02-16T22:21:46.990 INFO:tasks.workunit.client.0.smithi114.stdout:7/54: link d3/f6 d3/ff 0 2024-02-16T22:21:46.991 INFO:tasks.workunit.client.1.smithi123.stdout:2/144: rename d4/d1a/l2d to d4/d1a/d1d/d13/l30 0 2024-02-16T22:21:46.992 INFO:tasks.workunit.client.0.smithi114.stdout:3/59: symlink d4/l14 0 2024-02-16T22:21:46.993 INFO:tasks.workunit.client.0.smithi114.stdout:3/60: truncate d4/f6 17760 0 2024-02-16T22:21:46.994 INFO:tasks.workunit.client.0.smithi114.stdout:3/61: read - d4/dd/f12 zero size 2024-02-16T22:21:46.995 INFO:tasks.workunit.client.1.smithi123.stdout:5/102: getdents d6/dd/df 0 2024-02-16T22:21:46.997 INFO:tasks.workunit.client.0.smithi114.stdout:3/62: dread f1 [0,4194304] 0 2024-02-16T22:21:46.998 INFO:tasks.workunit.client.0.smithi114.stdout:3/63: dread - d4/dd/f12 zero size 2024-02-16T22:21:47.001 INFO:tasks.workunit.client.0.smithi114.stdout:4/86: mknod dd/c1a 0 2024-02-16T22:21:47.008 INFO:tasks.workunit.client.0.smithi114.stdout:8/105: dwrite d4/f7 [0,4194304] 0 2024-02-16T22:21:47.008 INFO:tasks.workunit.client.0.smithi114.stdout:9/69: mknod d2/c19 0 2024-02-16T22:21:47.008 INFO:tasks.workunit.client.0.smithi114.stdout:3/64: dread d4/fa [0,4194304] 0 2024-02-16T22:21:47.008 INFO:tasks.workunit.client.0.smithi114.stdout:1/70: link d4/f13 d4/f15 0 2024-02-16T22:21:47.008 INFO:tasks.workunit.client.0.smithi114.stdout:9/70: dread d2/dd/ff [0,4194304] 0 2024-02-16T22:21:47.009 INFO:tasks.workunit.client.1.smithi123.stdout:7/123: unlink d3/d12/d1d/d1e/l27 0 2024-02-16T22:21:47.009 INFO:tasks.workunit.client.1.smithi123.stdout:9/199: fsync d4/f6 0 2024-02-16T22:21:47.012 INFO:tasks.workunit.client.0.smithi114.stdout:6/51: mknod d6/dd/c12 0 2024-02-16T22:21:47.015 INFO:tasks.workunit.client.1.smithi123.stdout:3/113: fdatasync d0/f17 0 2024-02-16T22:21:47.016 INFO:tasks.workunit.client.0.smithi114.stdout:6/52: dread d6/fc [0,4194304] 0 2024-02-16T22:21:47.017 INFO:tasks.workunit.client.1.smithi123.stdout:5/103: mknod d6/dd/df/d16/c23 0 2024-02-16T22:21:47.019 INFO:tasks.workunit.client.0.smithi114.stdout:6/53: dread d6/fc [0,4194304] 0 2024-02-16T22:21:47.021 INFO:tasks.workunit.client.1.smithi123.stdout:1/107: creat d3/d6/d24/f25 x:0 0 0 2024-02-16T22:21:47.023 INFO:tasks.workunit.client.0.smithi114.stdout:8/106: write d4/d11/f17 [598785,16209] 0 2024-02-16T22:21:47.026 INFO:tasks.workunit.client.1.smithi123.stdout:7/124: creat d3/d12/f2e x:0 0 0 2024-02-16T22:21:47.027 INFO:tasks.workunit.client.1.smithi123.stdout:1/108: dwrite d3/d6/d18/f21 [0,4194304] 0 2024-02-16T22:21:47.027 INFO:tasks.workunit.client.1.smithi123.stdout:0/129: dread d1/d8/d13/f14 [0,4194304] 0 2024-02-16T22:21:47.028 INFO:tasks.workunit.client.0.smithi114.stdout:3/65: rename d4/dd/f12 to d4/dd/f15 0 2024-02-16T22:21:47.031 INFO:tasks.workunit.client.0.smithi114.stdout:9/71: mkdir d2/d1a 0 2024-02-16T22:21:47.031 INFO:tasks.workunit.client.0.smithi114.stdout:8/107: dwrite f0 [0,4194304] 0 2024-02-16T22:21:47.032 INFO:tasks.workunit.client.1.smithi123.stdout:6/132: symlink d0/d9/l31 0 2024-02-16T22:21:47.033 INFO:tasks.workunit.client.0.smithi114.stdout:8/108: chown d4/fe 52 1 2024-02-16T22:21:47.034 INFO:tasks.workunit.client.0.smithi114.stdout:1/71: dwrite d4/f13 [0,4194304] 0 2024-02-16T22:21:47.034 INFO:tasks.workunit.client.1.smithi123.stdout:7/125: dwrite d3/d5/f28 [0,4194304] 0 2024-02-16T22:21:47.034 INFO:tasks.workunit.client.1.smithi123.stdout:6/133: write d0/d29/f2c [4495582,131041] 0 2024-02-16T22:21:47.035 INFO:tasks.workunit.client.1.smithi123.stdout:7/126: write d3/fe [5650976,7464] 0 2024-02-16T22:21:47.036 INFO:tasks.workunit.client.1.smithi123.stdout:9/200: mkdir d4/d17/d3f 0 2024-02-16T22:21:47.038 INFO:tasks.workunit.client.1.smithi123.stdout:9/201: fdatasync d4/f16 0 2024-02-16T22:21:47.042 INFO:tasks.workunit.client.1.smithi123.stdout:3/114: write d0/fe [525466,2582] 0 2024-02-16T22:21:47.044 INFO:tasks.workunit.client.1.smithi123.stdout:5/104: unlink d6/f13 0 2024-02-16T22:21:47.047 INFO:tasks.workunit.client.1.smithi123.stdout:8/162: rename d0/d17/d26/d2e to d0/d17/d26/d2a/d3c 0 2024-02-16T22:21:47.048 INFO:tasks.workunit.client.1.smithi123.stdout:4/164: dwrite d5/f1d [0,4194304] 0 2024-02-16T22:21:47.049 INFO:tasks.workunit.client.0.smithi114.stdout:6/54: rename d6/da/lf to d6/da/l13 0 2024-02-16T22:21:47.053 INFO:tasks.workunit.client.1.smithi123.stdout:1/109: creat d3/d6/f26 x:0 0 0 2024-02-16T22:21:47.055 INFO:tasks.workunit.client.1.smithi123.stdout:0/130: rmdir d1/d8/d13 39 2024-02-16T22:21:47.056 INFO:tasks.workunit.client.1.smithi123.stdout:0/131: stat d1/d8/fa 0 2024-02-16T22:21:47.057 INFO:tasks.workunit.client.0.smithi114.stdout:3/66: rename d4/fe to d4/f16 0 2024-02-16T22:21:47.058 INFO:tasks.workunit.client.0.smithi114.stdout:0/120: truncate d1/f15 569865 0 2024-02-16T22:21:47.058 INFO:tasks.workunit.client.1.smithi123.stdout:8/163: dwrite d0/f14 [0,4194304] 0 2024-02-16T22:21:47.061 INFO:tasks.workunit.client.1.smithi123.stdout:4/165: dwrite d5/f14 [0,4194304] 0 2024-02-16T22:21:47.064 INFO:tasks.workunit.client.0.smithi114.stdout:7/55: truncate d3/f8 966101 0 2024-02-16T22:21:47.065 INFO:tasks.workunit.client.1.smithi123.stdout:1/110: dwrite d3/d6/d11/f23 [0,4194304] 0 2024-02-16T22:21:47.066 INFO:tasks.workunit.client.1.smithi123.stdout:6/134: symlink d0/d6/d2f/l32 0 2024-02-16T22:21:47.067 INFO:tasks.workunit.client.0.smithi114.stdout:8/109: creat d4/f1d x:0 0 0 2024-02-16T22:21:47.068 INFO:tasks.workunit.client.1.smithi123.stdout:7/127: creat d3/d12/d1d/f2f x:0 0 0 2024-02-16T22:21:47.071 INFO:tasks.workunit.client.0.smithi114.stdout:8/110: dread d4/f7 [0,4194304] 0 2024-02-16T22:21:47.073 INFO:tasks.workunit.client.0.smithi114.stdout:6/55: mknod d6/dd/c14 0 2024-02-16T22:21:47.074 INFO:tasks.workunit.client.0.smithi114.stdout:6/56: chown d6/l9 0 1 2024-02-16T22:21:47.074 INFO:tasks.workunit.client.0.smithi114.stdout:6/57: chown d6/l7 0 1 2024-02-16T22:21:47.075 INFO:tasks.workunit.client.1.smithi123.stdout:1/111: dwrite d3/d6/d18/f21 [4194304,4194304] 0 2024-02-16T22:21:47.079 INFO:tasks.workunit.client.0.smithi114.stdout:0/121: rename d1/d16/f25 to d1/d1a/f26 0 2024-02-16T22:21:47.082 INFO:tasks.workunit.client.1.smithi123.stdout:5/105: rmdir d6/dd/df 39 2024-02-16T22:21:47.089 INFO:tasks.workunit.client.1.smithi123.stdout:0/132: chown d1/d8/d13/f15 2083410 1 2024-02-16T22:21:47.089 INFO:tasks.workunit.client.1.smithi123.stdout:0/133: write d1/f6 [2312257,21101] 0 2024-02-16T22:21:47.090 INFO:tasks.workunit.client.1.smithi123.stdout:8/164: creat d0/df/d25/f3d x:0 0 0 2024-02-16T22:21:47.090 INFO:tasks.workunit.client.1.smithi123.stdout:8/165: chown d0/df/f13 17 1 2024-02-16T22:21:47.091 INFO:tasks.workunit.client.0.smithi114.stdout:4/87: getdents dd 0 2024-02-16T22:21:47.091 INFO:tasks.workunit.client.1.smithi123.stdout:8/166: truncate d0/d29/f39 120013 0 2024-02-16T22:21:47.092 INFO:tasks.workunit.client.1.smithi123.stdout:8/167: dread - d0/df/d25/f28 zero size 2024-02-16T22:21:47.094 INFO:tasks.workunit.client.1.smithi123.stdout:8/168: write d0/f14 [2661323,24003] 0 2024-02-16T22:21:47.094 INFO:tasks.workunit.client.1.smithi123.stdout:0/134: dread d1/d8/d13/f20 [0,4194304] 0 2024-02-16T22:21:47.095 INFO:tasks.workunit.client.1.smithi123.stdout:0/135: write d1/f1f [575718,95010] 0 2024-02-16T22:21:47.095 INFO:tasks.workunit.client.0.smithi114.stdout:6/58: creat d6/dd/f15 x:0 0 0 2024-02-16T22:21:47.096 INFO:tasks.workunit.client.0.smithi114.stdout:6/59: stat d6/le 0 2024-02-16T22:21:47.099 INFO:tasks.workunit.client.1.smithi123.stdout:6/135: symlink d0/d6/d2f/l33 0 2024-02-16T22:21:47.099 INFO:tasks.workunit.client.1.smithi123.stdout:6/136: fsync d0/d9/f21 0 2024-02-16T22:21:47.101 INFO:tasks.workunit.client.1.smithi123.stdout:2/145: getdents d4/d1a/d1d/d13 0 2024-02-16T22:21:47.102 INFO:tasks.workunit.client.0.smithi114.stdout:9/72: dwrite d2/f4 [0,4194304] 0 2024-02-16T22:21:47.103 INFO:tasks.workunit.client.1.smithi123.stdout:2/146: truncate d4/d1a/d1d/f2c 27359 0 2024-02-16T22:21:47.103 INFO:tasks.workunit.client.0.smithi114.stdout:7/56: truncate d3/ff 4405262 0 2024-02-16T22:21:47.103 INFO:tasks.workunit.client.1.smithi123.stdout:2/147: write d4/f23 [988411,44932] 0 2024-02-16T22:21:47.104 INFO:tasks.workunit.client.0.smithi114.stdout:9/73: chown d2/c19 297 1 2024-02-16T22:21:47.104 INFO:tasks.workunit.client.0.smithi114.stdout:9/74: chown d2/c14 456 1 2024-02-16T22:21:47.105 INFO:tasks.workunit.client.1.smithi123.stdout:6/137: dread d0/d9/dc/f23 [0,4194304] 0 2024-02-16T22:21:47.106 INFO:tasks.workunit.client.1.smithi123.stdout:7/128: rename d3/d5/lc to d3/d12/d24/l30 0 2024-02-16T22:21:47.106 INFO:tasks.workunit.client.0.smithi114.stdout:2/95: sync 2024-02-16T22:21:47.106 INFO:tasks.workunit.client.0.smithi114.stdout:5/57: sync 2024-02-16T22:21:47.107 INFO:tasks.workunit.client.1.smithi123.stdout:7/129: write d3/d5/ff [4067972,1007] 0 2024-02-16T22:21:47.108 INFO:tasks.workunit.client.0.smithi114.stdout:2/96: dread fa [0,4194304] 0 2024-02-16T22:21:47.109 INFO:tasks.workunit.client.0.smithi114.stdout:2/97: write dd/f17 [132368,17128] 0 2024-02-16T22:21:47.111 INFO:tasks.workunit.client.0.smithi114.stdout:1/72: creat d4/f16 x:0 0 0 2024-02-16T22:21:47.112 INFO:tasks.workunit.client.1.smithi123.stdout:7/130: dread d3/d5/ff [0,4194304] 0 2024-02-16T22:21:47.112 INFO:tasks.workunit.client.0.smithi114.stdout:4/88: rmdir dd 39 2024-02-16T22:21:47.114 INFO:tasks.workunit.client.1.smithi123.stdout:1/112: chown d3/d6/db/c13 132600 1 2024-02-16T22:21:47.116 INFO:tasks.workunit.client.1.smithi123.stdout:3/115: mkdir d0/d20 0 2024-02-16T22:21:47.117 INFO:tasks.workunit.client.0.smithi114.stdout:0/122: rmdir d1/d16 39 2024-02-16T22:21:47.117 INFO:tasks.workunit.client.0.smithi114.stdout:1/73: dwrite d4/d9/f10 [0,4194304] 0 2024-02-16T22:21:47.119 INFO:tasks.workunit.client.0.smithi114.stdout:0/123: write d1/d1a/f1d [683947,108149] 0 2024-02-16T22:21:47.124 INFO:tasks.workunit.client.0.smithi114.stdout:1/74: dwrite d4/fa [0,4194304] 0 2024-02-16T22:21:47.132 INFO:tasks.workunit.client.1.smithi123.stdout:8/169: unlink d0/d29/f39 0 2024-02-16T22:21:47.135 INFO:tasks.workunit.client.0.smithi114.stdout:5/58: creat d0/fe x:0 0 0 2024-02-16T22:21:47.135 INFO:tasks.workunit.client.0.smithi114.stdout:5/59: chown d0/fe 23 1 2024-02-16T22:21:47.136 INFO:tasks.workunit.client.0.smithi114.stdout:5/60: chown d0/fa 0 1 2024-02-16T22:21:47.136 INFO:tasks.workunit.client.1.smithi123.stdout:9/202: truncate d4/d17/f1a 677378 0 2024-02-16T22:21:47.137 INFO:tasks.workunit.client.0.smithi114.stdout:5/61: write d0/fb [974439,721] 0 2024-02-16T22:21:47.138 INFO:tasks.workunit.client.0.smithi114.stdout:5/62: chown d0/f9 9008104 1 2024-02-16T22:21:47.138 INFO:tasks.workunit.client.0.smithi114.stdout:8/111: truncate f0 2942922 0 2024-02-16T22:21:47.138 INFO:tasks.workunit.client.0.smithi114.stdout:5/63: write d0/f8 [5225237,19061] 0 2024-02-16T22:21:47.140 INFO:tasks.workunit.client.0.smithi114.stdout:8/112: chown d4/ld 1 1 2024-02-16T22:21:47.140 INFO:tasks.workunit.client.0.smithi114.stdout:5/64: chown d0/cc 58357905 1 2024-02-16T22:21:47.142 INFO:tasks.workunit.client.1.smithi123.stdout:2/148: unlink d4/d1a/d1d/d13/d15/l19 0 2024-02-16T22:21:47.143 INFO:tasks.workunit.client.0.smithi114.stdout:6/60: write d6/fc [864610,127729] 0 2024-02-16T22:21:47.143 INFO:tasks.workunit.client.1.smithi123.stdout:6/138: fdatasync d0/d6/d2f/f2a 0 2024-02-16T22:21:47.143 INFO:tasks.workunit.client.0.smithi114.stdout:6/61: read - d6/dd/f15 zero size 2024-02-16T22:21:47.144 INFO:tasks.workunit.client.1.smithi123.stdout:6/139: write d0/f30 [49647,78315] 0 2024-02-16T22:21:47.145 INFO:tasks.workunit.client.0.smithi114.stdout:3/67: truncate d4/f16 891634 0 2024-02-16T22:21:47.148 INFO:tasks.workunit.client.1.smithi123.stdout:1/113: rmdir d3 39 2024-02-16T22:21:47.149 INFO:tasks.workunit.client.1.smithi123.stdout:4/166: dwrite f4 [0,4194304] 0 2024-02-16T22:21:47.149 INFO:tasks.workunit.client.0.smithi114.stdout:1/75: symlink d4/d9/l17 0 2024-02-16T22:21:47.163 INFO:tasks.workunit.client.0.smithi114.stdout:2/98: truncate fc 2903123 0 2024-02-16T22:21:47.164 INFO:tasks.workunit.client.0.smithi114.stdout:2/99: write dd/f16 [811925,38714] 0 2024-02-16T22:21:47.165 INFO:tasks.workunit.client.0.smithi114.stdout:2/100: write dd/f10 [4283077,70606] 0 2024-02-16T22:21:47.166 INFO:tasks.workunit.client.1.smithi123.stdout:2/149: mknod d4/d11/c31 0 2024-02-16T22:21:47.167 INFO:tasks.workunit.client.0.smithi114.stdout:4/89: mkdir dd/d1b 0 2024-02-16T22:21:47.169 INFO:tasks.workunit.client.0.smithi114.stdout:5/65: mkdir d0/df 0 2024-02-16T22:21:47.172 INFO:tasks.workunit.client.1.smithi123.stdout:7/131: link d3/d12/d1d/f22 d3/d12/d24/f31 0 2024-02-16T22:21:47.175 INFO:tasks.workunit.client.0.smithi114.stdout:7/57: creat d3/f10 x:0 0 0 2024-02-16T22:21:47.177 INFO:tasks.workunit.client.0.smithi114.stdout:1/76: unlink d4/c7 0 2024-02-16T22:21:47.178 INFO:tasks.workunit.client.1.smithi123.stdout:3/116: creat d0/d20/f21 x:0 0 0 2024-02-16T22:21:47.178 INFO:tasks.workunit.client.0.smithi114.stdout:9/75: link d2/dd/le d2/dd/l1b 0 2024-02-16T22:21:47.181 INFO:tasks.workunit.client.1.smithi123.stdout:8/170: mkdir d0/d12/d3e 0 2024-02-16T22:21:47.186 INFO:tasks.workunit.client.0.smithi114.stdout:4/90: rename f2 to dd/d1b/f1c 0 2024-02-16T22:21:47.186 INFO:tasks.workunit.client.0.smithi114.stdout:3/68: write d4/fa [3678016,94423] 0 2024-02-16T22:21:47.187 INFO:tasks.workunit.client.1.smithi123.stdout:8/171: dwrite d0/d17/d26/f2d [0,4194304] 0 2024-02-16T22:21:47.187 INFO:tasks.workunit.client.1.smithi123.stdout:2/150: creat d4/d1a/f32 x:0 0 0 2024-02-16T22:21:47.188 INFO:tasks.workunit.client.1.smithi123.stdout:8/172: chown d0/df/f16 158 1 2024-02-16T22:21:47.189 INFO:tasks.workunit.client.1.smithi123.stdout:6/140: creat d0/d6/d1a/d2e/f34 x:0 0 0 2024-02-16T22:21:47.190 INFO:tasks.workunit.client.0.smithi114.stdout:5/66: unlink d0/f7 0 2024-02-16T22:21:47.191 INFO:tasks.workunit.client.1.smithi123.stdout:7/132: mknod d3/d12/d24/c32 0 2024-02-16T22:21:47.192 INFO:tasks.workunit.client.0.smithi114.stdout:8/113: creat d4/d16/f1e x:0 0 0 2024-02-16T22:21:47.193 INFO:tasks.workunit.client.0.smithi114.stdout:8/114: fdatasync d4/d11/f1c 0 2024-02-16T22:21:47.196 INFO:tasks.workunit.client.1.smithi123.stdout:4/167: dwrite d5/f29 [0,4194304] 0 2024-02-16T22:21:47.196 INFO:tasks.workunit.client.1.smithi123.stdout:3/117: chown d0/c1a 230 1 2024-02-16T22:21:47.197 INFO:tasks.workunit.client.1.smithi123.stdout:4/168: chown d5/d1c/f25 122342741 1 2024-02-16T22:21:47.198 INFO:tasks.workunit.client.0.smithi114.stdout:7/58: truncate d3/fd 951618 0 2024-02-16T22:21:47.198 INFO:tasks.workunit.client.1.smithi123.stdout:5/106: getdents d6 0 2024-02-16T22:21:47.200 INFO:tasks.workunit.client.0.smithi114.stdout:2/101: truncate fc 3561486 0 2024-02-16T22:21:47.201 INFO:tasks.workunit.client.0.smithi114.stdout:8/115: dwrite d4/d11/d15/f1a [0,4194304] 0 2024-02-16T22:21:47.203 INFO:tasks.workunit.client.1.smithi123.stdout:2/151: chown d4/la 4523822 1 2024-02-16T22:21:47.204 INFO:tasks.workunit.client.1.smithi123.stdout:4/169: dwrite d5/d1c/f25 [0,4194304] 0 2024-02-16T22:21:47.209 INFO:tasks.workunit.client.1.smithi123.stdout:0/136: sync 2024-02-16T22:21:47.213 INFO:tasks.workunit.client.1.smithi123.stdout:1/114: mknod d3/d6/db/c27 0 2024-02-16T22:21:47.215 INFO:tasks.workunit.client.0.smithi114.stdout:5/67: mkdir d0/d10 0 2024-02-16T22:21:47.220 INFO:tasks.workunit.client.1.smithi123.stdout:9/203: dwrite d4/f12 [0,4194304] 0 2024-02-16T22:21:47.220 INFO:tasks.workunit.client.1.smithi123.stdout:5/107: write d6/dd/df/f22 [33843,82343] 0 2024-02-16T22:21:47.223 INFO:tasks.workunit.client.1.smithi123.stdout:8/173: link d0/df/f1b d0/d12/f3f 0 2024-02-16T22:21:47.225 INFO:tasks.workunit.client.0.smithi114.stdout:3/69: dwrite f1 [4194304,4194304] 0 2024-02-16T22:21:47.226 INFO:tasks.workunit.client.0.smithi114.stdout:3/70: write d4/f6 [333860,390] 0 2024-02-16T22:21:47.227 INFO:tasks.workunit.client.1.smithi123.stdout:4/170: mkdir d5/d1c/d23/d36 0 2024-02-16T22:21:47.227 INFO:tasks.workunit.client.1.smithi123.stdout:1/115: dwrite d3/d6/d11/f23 [0,4194304] 0 2024-02-16T22:21:47.229 INFO:tasks.workunit.client.1.smithi123.stdout:8/174: dread - d0/df/d25/f3d zero size 2024-02-16T22:21:47.232 INFO:tasks.workunit.client.1.smithi123.stdout:0/137: creat d1/d8/d13/d1d/f25 x:0 0 0 2024-02-16T22:21:47.236 INFO:tasks.workunit.client.0.smithi114.stdout:6/62: getdents d6/da 0 2024-02-16T22:21:47.238 INFO:tasks.workunit.client.0.smithi114.stdout:7/59: symlink d3/l11 0 2024-02-16T22:21:47.238 INFO:tasks.workunit.client.1.smithi123.stdout:6/141: dwrite d0/d9/dc/f23 [0,4194304] 0 2024-02-16T22:21:47.239 INFO:tasks.workunit.client.1.smithi123.stdout:3/118: dwrite d0/fc [0,4194304] 0 2024-02-16T22:21:47.240 INFO:tasks.workunit.client.0.smithi114.stdout:9/76: rmdir d2/d17 0 2024-02-16T22:21:47.240 INFO:tasks.workunit.client.1.smithi123.stdout:5/108: dread d6/f8 [0,4194304] 0 2024-02-16T22:21:47.241 INFO:tasks.workunit.client.0.smithi114.stdout:2/102: dwrite dd/df/f1e [0,4194304] 0 2024-02-16T22:21:47.242 INFO:tasks.workunit.client.1.smithi123.stdout:3/119: read d0/f17 [16835,8983] 0 2024-02-16T22:21:47.242 INFO:tasks.workunit.client.0.smithi114.stdout:2/103: write dd/f10 [3979646,120699] 0 2024-02-16T22:21:47.243 INFO:tasks.workunit.client.1.smithi123.stdout:5/109: chown d6/l15 1 1 2024-02-16T22:21:47.244 INFO:tasks.workunit.client.0.smithi114.stdout:2/104: chown dd/f13 6753 1 2024-02-16T22:21:47.245 INFO:tasks.workunit.client.1.smithi123.stdout:3/120: dread - d0/f1f zero size 2024-02-16T22:21:47.246 INFO:tasks.workunit.client.0.smithi114.stdout:3/71: mknod d4/dd/c17 0 2024-02-16T22:21:47.247 INFO:tasks.workunit.client.0.smithi114.stdout:4/91: write dd/ff [425259,83786] 0 2024-02-16T22:21:47.247 INFO:tasks.workunit.client.1.smithi123.stdout:6/142: truncate d0/d9/dc/f11 262727 0 2024-02-16T22:21:47.248 INFO:tasks.workunit.client.1.smithi123.stdout:0/138: dwrite d1/f6 [4194304,4194304] 0 2024-02-16T22:21:47.248 INFO:tasks.workunit.client.0.smithi114.stdout:3/72: write d4/fc [901779,28570] 0 2024-02-16T22:21:47.249 INFO:tasks.workunit.client.1.smithi123.stdout:0/139: read d1/d8/d13/f14 [1833505,37460] 0 2024-02-16T22:21:47.250 INFO:tasks.workunit.client.1.smithi123.stdout:7/133: truncate d3/d5/f18 4707689 0 2024-02-16T22:21:47.262 INFO:tasks.workunit.client.0.smithi114.stdout:9/77: mkdir d2/dd/d1c 0 2024-02-16T22:21:47.263 INFO:tasks.workunit.client.0.smithi114.stdout:9/78: dread - d2/f16 zero size 2024-02-16T22:21:47.265 INFO:tasks.workunit.client.0.smithi114.stdout:5/68: truncate d0/f8 2319337 0 2024-02-16T22:21:47.265 INFO:tasks.workunit.client.0.smithi114.stdout:5/69: stat d0/c4 0 2024-02-16T22:21:47.266 INFO:tasks.workunit.client.0.smithi114.stdout:5/70: chown d0/fb 21018 1 2024-02-16T22:21:47.266 INFO:tasks.workunit.client.0.smithi114.stdout:5/71: readlink d0/l6 0 2024-02-16T22:21:47.267 INFO:tasks.workunit.client.1.smithi123.stdout:9/204: creat d4/d1e/d3a/f40 x:0 0 0 2024-02-16T22:21:47.270 INFO:tasks.workunit.client.0.smithi114.stdout:4/92: creat dd/d1b/f1d x:0 0 0 2024-02-16T22:21:47.274 INFO:tasks.workunit.client.0.smithi114.stdout:8/116: rename d4/f9 to d4/d11/f1f 0 2024-02-16T22:21:47.276 INFO:tasks.workunit.client.1.smithi123.stdout:0/140: unlink d1/d4/d17/l19 0 2024-02-16T22:21:47.277 INFO:tasks.workunit.client.0.smithi114.stdout:4/93: dwrite dd/f16 [0,4194304] 0 2024-02-16T22:21:47.279 INFO:tasks.workunit.client.0.smithi114.stdout:4/94: truncate dd/f13 70682 0 2024-02-16T22:21:47.281 INFO:tasks.workunit.client.1.smithi123.stdout:7/134: creat d3/d5/f33 x:0 0 0 2024-02-16T22:21:47.282 INFO:tasks.workunit.client.0.smithi114.stdout:5/72: rename d0/fd to d0/f11 0 2024-02-16T22:21:47.285 INFO:tasks.workunit.client.1.smithi123.stdout:5/110: write d6/f21 [1162365,52963] 0 2024-02-16T22:21:47.285 INFO:tasks.workunit.client.1.smithi123.stdout:4/171: symlink d5/d1c/d23/d36/l37 0 2024-02-16T22:21:47.285 INFO:tasks.workunit.client.1.smithi123.stdout:4/172: readlink d5/d1c/l27 0 2024-02-16T22:21:47.287 INFO:tasks.workunit.client.1.smithi123.stdout:9/205: mknod d4/d1e/c41 0 2024-02-16T22:21:47.289 INFO:tasks.workunit.client.0.smithi114.stdout:3/73: rename d4/fb to d4/f18 0 2024-02-16T22:21:47.291 INFO:tasks.workunit.client.1.smithi123.stdout:2/152: link d4/cf d4/d1a/d29/c33 0 2024-02-16T22:21:47.293 INFO:tasks.workunit.client.1.smithi123.stdout:3/121: rename d0/la to d0/d20/l22 0 2024-02-16T22:21:47.293 INFO:tasks.workunit.client.0.smithi114.stdout:7/60: truncate d3/fc 619826 0 2024-02-16T22:21:47.295 INFO:tasks.workunit.client.0.smithi114.stdout:8/117: fsync d4/d11/d15/f10 0 2024-02-16T22:21:47.295 INFO:tasks.workunit.client.0.smithi114.stdout:8/118: dread - d4/d11/f1c zero size 2024-02-16T22:21:47.296 INFO:tasks.workunit.client.0.smithi114.stdout:8/119: truncate d4/f19 932375 0 2024-02-16T22:21:47.297 INFO:tasks.workunit.client.0.smithi114.stdout:8/120: chown d4/f6 13140 1 2024-02-16T22:21:47.297 INFO:tasks.workunit.client.1.smithi123.stdout:0/141: mknod d1/d4/d17/c26 0 2024-02-16T22:21:47.300 INFO:tasks.workunit.client.0.smithi114.stdout:8/121: dread d4/fc [0,4194304] 0 2024-02-16T22:21:47.300 INFO:tasks.workunit.client.1.smithi123.stdout:1/116: creat d3/d6/db/f28 x:0 0 0 2024-02-16T22:21:47.301 INFO:tasks.workunit.client.0.smithi114.stdout:4/95: symlink dd/l1e 0 2024-02-16T22:21:47.301 INFO:tasks.workunit.client.1.smithi123.stdout:1/117: write d3/f7 [2831520,68362] 0 2024-02-16T22:21:47.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:46 smithi123 ceph-mon[37161]: pgmap v21: 65 pgs: 65 active+clean; 221 MiB data, 883 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.0 MiB/s wr, 292 op/s 2024-02-16T22:21:47.303 INFO:tasks.workunit.client.1.smithi123.stdout:1/118: write d3/d6/f26 [947876,57447] 0 2024-02-16T22:21:47.304 INFO:tasks.workunit.client.0.smithi114.stdout:8/122: dread d4/fe [0,4194304] 0 2024-02-16T22:21:47.304 INFO:tasks.workunit.client.1.smithi123.stdout:0/142: dwrite d1/d8/d13/d1d/f25 [0,4194304] 0 2024-02-16T22:21:47.305 INFO:tasks.workunit.client.0.smithi114.stdout:8/123: dread - d4/f1d zero size 2024-02-16T22:21:47.306 INFO:tasks.workunit.client.1.smithi123.stdout:5/111: rmdir d6 39 2024-02-16T22:21:47.308 INFO:tasks.workunit.client.0.smithi114.stdout:3/74: creat d4/dd/f19 x:0 0 0 2024-02-16T22:21:47.312 INFO:tasks.workunit.client.1.smithi123.stdout:3/122: dread - d0/f19 zero size 2024-02-16T22:21:47.312 INFO:tasks.workunit.client.1.smithi123.stdout:6/143: truncate d0/d6/d2f/f22 3240379 0 2024-02-16T22:21:47.316 INFO:tasks.workunit.client.1.smithi123.stdout:9/206: rename d4/fe to d4/d1e/f42 0 2024-02-16T22:21:47.316 INFO:tasks.workunit.client.0.smithi114.stdout:9/79: write d2/f8 [897554,102692] 0 2024-02-16T22:21:47.319 INFO:tasks.workunit.client.1.smithi123.stdout:8/175: getdents d0/d17/d26 0 2024-02-16T22:21:47.320 INFO:tasks.workunit.client.0.smithi114.stdout:9/80: dread f1 [0,4194304] 0 2024-02-16T22:21:47.321 INFO:tasks.workunit.client.1.smithi123.stdout:1/119: creat d3/d6/db/d10/f29 x:0 0 0 2024-02-16T22:21:47.323 INFO:tasks.workunit.client.0.smithi114.stdout:2/105: getdents dd/df 0 2024-02-16T22:21:47.326 INFO:tasks.workunit.client.1.smithi123.stdout:1/120: dread d3/d6/f26 [0,4194304] 0 2024-02-16T22:21:47.326 INFO:tasks.workunit.client.1.smithi123.stdout:7/135: unlink d3/d5/f18 0 2024-02-16T22:21:47.327 INFO:tasks.workunit.client.0.smithi114.stdout:4/96: rename dd/c17 to dd/d1b/c1f 0 2024-02-16T22:21:47.329 INFO:tasks.workunit.client.0.smithi114.stdout:5/73: mknod d0/d10/c12 0 2024-02-16T22:21:47.331 INFO:tasks.workunit.client.0.smithi114.stdout:3/75: unlink d4/fc 0 2024-02-16T22:21:47.332 INFO:tasks.workunit.client.1.smithi123.stdout:7/136: dwrite d3/d12/d1d/f2f [0,4194304] 0 2024-02-16T22:21:47.333 INFO:tasks.workunit.client.1.smithi123.stdout:3/123: mknod d0/d20/c23 0 2024-02-16T22:21:47.334 INFO:tasks.workunit.client.0.smithi114.stdout:3/76: write d4/dd/f19 [354873,51557] 0 2024-02-16T22:21:47.336 INFO:tasks.workunit.client.1.smithi123.stdout:2/153: rename d4/cf to d4/d1a/d29/c34 0 2024-02-16T22:21:47.340 INFO:tasks.workunit.client.1.smithi123.stdout:9/207: mknod d4/d33/c43 0 2024-02-16T22:21:47.341 INFO:tasks.workunit.client.1.smithi123.stdout:9/208: write d4/d17/d20/f22 [2935983,101212] 0 2024-02-16T22:21:47.341 INFO:tasks.workunit.client.1.smithi123.stdout:2/154: dwrite d4/f23 [0,4194304] 0 2024-02-16T22:21:47.343 INFO:tasks.workunit.client.1.smithi123.stdout:8/176: mkdir d0/df/d40 0 2024-02-16T22:21:47.343 INFO:tasks.workunit.client.1.smithi123.stdout:2/155: write d4/d1a/d1d/f17 [545588,46360] 0 2024-02-16T22:21:47.344 INFO:tasks.workunit.client.1.smithi123.stdout:8/177: write d0/fd [4745304,101856] 0 2024-02-16T22:21:47.347 INFO:tasks.workunit.client.1.smithi123.stdout:1/121: mkdir d3/d6/db/d10/d2a 0 2024-02-16T22:21:47.349 INFO:tasks.workunit.client.0.smithi114.stdout:4/97: creat dd/f20 x:0 0 0 2024-02-16T22:21:47.350 INFO:tasks.workunit.client.1.smithi123.stdout:5/112: dread d6/dd/df/d16/f1a [0,4194304] 0 2024-02-16T22:21:47.351 INFO:tasks.workunit.client.0.smithi114.stdout:5/74: chown d0/f8 2612871 1 2024-02-16T22:21:47.351 INFO:tasks.workunit.client.0.smithi114.stdout:7/61: dwrite f2 [0,4194304] 0 2024-02-16T22:21:47.416 INFO:tasks.workunit.client.0.smithi114.stdout:9/81: mkdir d2/dd/d1c/d1d 0 2024-02-16T22:21:47.418 INFO:tasks.workunit.client.0.smithi114.stdout:2/106: symlink dd/df/d1c/d21/l22 0 2024-02-16T22:21:47.420 INFO:tasks.workunit.client.0.smithi114.stdout:9/82: dread d2/f6 [0,4194304] 0 2024-02-16T22:21:47.424 INFO:tasks.workunit.client.1.smithi123.stdout:2/156: rmdir d4/d11 39 2024-02-16T22:21:47.427 INFO:tasks.workunit.client.1.smithi123.stdout:8/178: symlink d0/d12/l41 0 2024-02-16T22:21:47.430 INFO:tasks.workunit.client.0.smithi114.stdout:9/83: unlink d2/dd/le 0 2024-02-16T22:21:47.430 INFO:tasks.workunit.client.0.smithi114.stdout:9/84: chown d2/dd 111 1 2024-02-16T22:21:47.430 INFO:tasks.workunit.client.0.smithi114.stdout:9/85: chown d2 472 1 2024-02-16T22:21:47.431 INFO:tasks.workunit.client.0.smithi114.stdout:9/86: truncate d2/f16 935945 0 2024-02-16T22:21:47.431 INFO:tasks.workunit.client.0.smithi114.stdout:9/87: dread - d2/f13 zero size 2024-02-16T22:21:47.432 INFO:tasks.workunit.client.1.smithi123.stdout:1/122: rmdir d3/d6 39 2024-02-16T22:21:47.433 INFO:tasks.workunit.client.1.smithi123.stdout:8/179: dwrite d0/df/d25/f27 [0,4194304] 0 2024-02-16T22:21:47.434 INFO:tasks.workunit.client.0.smithi114.stdout:7/62: mknod d3/c12 0 2024-02-16T22:21:47.434 INFO:tasks.workunit.client.1.smithi123.stdout:8/180: write d0/df/f16 [585819,62852] 0 2024-02-16T22:21:47.435 INFO:tasks.workunit.client.0.smithi114.stdout:7/63: fdatasync d3/fe 0 2024-02-16T22:21:47.435 INFO:tasks.workunit.client.1.smithi123.stdout:4/173: getdents d5/d1c/d23 0 2024-02-16T22:21:47.436 INFO:tasks.workunit.client.0.smithi114.stdout:5/75: link d0/fa d0/d10/f13 0 2024-02-16T22:21:47.440 INFO:tasks.workunit.client.1.smithi123.stdout:6/144: link d0/d6/l1d d0/d6/d1a/l35 0 2024-02-16T22:21:47.441 INFO:tasks.workunit.client.1.smithi123.stdout:6/145: stat d0/d6/c27 0 2024-02-16T22:21:47.441 INFO:tasks.workunit.client.0.smithi114.stdout:2/107: symlink dd/l23 0 2024-02-16T22:21:47.442 INFO:tasks.workunit.client.0.smithi114.stdout:9/88: creat d2/dd/d1c/f1e x:0 0 0 2024-02-16T22:21:47.445 INFO:tasks.workunit.client.1.smithi123.stdout:7/137: truncate f2 1651902 0 2024-02-16T22:21:47.446 INFO:tasks.workunit.client.1.smithi123.stdout:7/138: chown d3/d12/d1d/f22 0 1 2024-02-16T22:21:47.447 INFO:tasks.workunit.client.0.smithi114.stdout:5/76: unlink d0/cc 0 2024-02-16T22:21:47.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:46 smithi114 ceph-mon[29235]: pgmap v21: 65 pgs: 65 active+clean; 221 MiB data, 883 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.0 MiB/s wr, 292 op/s 2024-02-16T22:21:47.451 INFO:tasks.workunit.client.1.smithi123.stdout:0/143: getdents d1/d8 0 2024-02-16T22:21:47.451 INFO:tasks.workunit.client.1.smithi123.stdout:9/209: truncate d4/f21 495504 0 2024-02-16T22:21:47.453 INFO:tasks.workunit.client.0.smithi114.stdout:2/108: rename lb to dd/df/l24 0 2024-02-16T22:21:47.459 INFO:tasks.workunit.client.0.smithi114.stdout:4/98: getdents dd 0 2024-02-16T22:21:47.459 INFO:tasks.workunit.client.0.smithi114.stdout:7/64: dwrite d3/fd [0,4194304] 0 2024-02-16T22:21:47.460 INFO:tasks.workunit.client.0.smithi114.stdout:7/65: chown d3/fe 7499377 1 2024-02-16T22:21:47.461 INFO:tasks.workunit.client.1.smithi123.stdout:5/113: creat d6/dd/df/d1f/f24 x:0 0 0 2024-02-16T22:21:47.464 INFO:tasks.workunit.client.0.smithi114.stdout:4/99: dread f9 [0,4194304] 0 2024-02-16T22:21:47.466 INFO:tasks.workunit.client.0.smithi114.stdout:5/77: rename d0/fe to d0/df/f14 0 2024-02-16T22:21:47.468 INFO:tasks.workunit.client.0.smithi114.stdout:2/109: mknod dd/df/d1c/d21/c25 0 2024-02-16T22:21:47.470 INFO:tasks.workunit.client.0.smithi114.stdout:2/110: write f0 [1937769,83828] 0 2024-02-16T22:21:47.471 INFO:tasks.workunit.client.0.smithi114.stdout:4/100: dwrite f9 [0,4194304] 0 2024-02-16T22:21:47.476 INFO:tasks.workunit.client.1.smithi123.stdout:2/157: mknod d4/c35 0 2024-02-16T22:21:47.477 INFO:tasks.workunit.client.1.smithi123.stdout:0/144: rmdir d1/d8/d13 39 2024-02-16T22:21:47.478 INFO:tasks.workunit.client.0.smithi114.stdout:5/78: creat d0/d10/f15 x:0 0 0 2024-02-16T22:21:47.479 INFO:tasks.workunit.client.1.smithi123.stdout:1/123: mknod d3/c2b 0 2024-02-16T22:21:47.480 INFO:tasks.workunit.client.0.smithi114.stdout:9/89: mkdir d2/dd/d1c/d1f 0 2024-02-16T22:21:47.481 INFO:tasks.workunit.client.0.smithi114.stdout:9/90: fsync d2/f8 0 2024-02-16T22:21:47.482 INFO:tasks.workunit.client.0.smithi114.stdout:9/91: write d2/f15 [611135,53972] 0 2024-02-16T22:21:47.482 INFO:tasks.workunit.client.1.smithi123.stdout:5/114: rename d6/dd/df/d16 to d6/dd/df/d1f/d25 0 2024-02-16T22:21:47.483 INFO:tasks.workunit.client.0.smithi114.stdout:9/92: write d2/f4 [2388297,32536] 0 2024-02-16T22:21:47.484 INFO:tasks.workunit.client.0.smithi114.stdout:2/111: creat dd/df/d1c/d21/f26 x:0 0 0 2024-02-16T22:21:47.485 INFO:tasks.workunit.client.0.smithi114.stdout:2/112: read dd/f17 [142691,88864] 0 2024-02-16T22:21:47.489 INFO:tasks.workunit.client.0.smithi114.stdout:2/113: dread f0 [0,4194304] 0 2024-02-16T22:21:47.489 INFO:tasks.workunit.client.0.smithi114.stdout:2/114: fsync dd/df/f1e 0 2024-02-16T22:21:47.490 INFO:tasks.workunit.client.1.smithi123.stdout:4/174: mknod d5/d1c/c38 0 2024-02-16T22:21:47.490 INFO:tasks.workunit.client.1.smithi123.stdout:6/146: write d0/d6/f8 [2449288,114662] 0 2024-02-16T22:21:47.491 INFO:tasks.workunit.client.0.smithi114.stdout:7/66: mknod d3/c13 0 2024-02-16T22:21:47.491 INFO:tasks.workunit.client.1.smithi123.stdout:6/147: chown d0/d6/d1a/d2e/f34 107 1 2024-02-16T22:21:47.492 INFO:tasks.workunit.client.0.smithi114.stdout:2/115: dread dd/f17 [0,4194304] 0 2024-02-16T22:21:47.492 INFO:tasks.workunit.client.1.smithi123.stdout:3/124: truncate d0/f12 1834342 0 2024-02-16T22:21:47.493 INFO:tasks.workunit.client.0.smithi114.stdout:5/79: mkdir d0/df/d16 0 2024-02-16T22:21:47.494 INFO:tasks.workunit.client.1.smithi123.stdout:7/139: write d3/d12/d24/f31 [182562,74065] 0 2024-02-16T22:21:47.504 INFO:tasks.workunit.client.1.smithi123.stdout:4/175: rmdir d5/d1c/d23/d31 39 2024-02-16T22:21:47.504 INFO:tasks.workunit.client.0.smithi114.stdout:7/67: chown d3/l7 65761048 1 2024-02-16T22:21:47.505 INFO:tasks.workunit.client.1.smithi123.stdout:6/148: creat d0/d6/d1a/f36 x:0 0 0 2024-02-16T22:21:47.506 INFO:tasks.workunit.client.0.smithi114.stdout:7/68: write d3/f8 [1373983,103393] 0 2024-02-16T22:21:47.506 INFO:tasks.workunit.client.0.smithi114.stdout:2/116: mknod dd/df/d1c/c27 0 2024-02-16T22:21:47.506 INFO:tasks.workunit.client.1.smithi123.stdout:8/181: dwrite d0/d17/f35 [0,4194304] 0 2024-02-16T22:21:47.507 INFO:tasks.workunit.client.0.smithi114.stdout:2/117: chown fa 0 1 2024-02-16T22:21:47.507 INFO:tasks.workunit.client.1.smithi123.stdout:2/158: dread d4/fc [0,4194304] 0 2024-02-16T22:21:47.508 INFO:tasks.workunit.client.1.smithi123.stdout:8/182: write d0/df/d25/f28 [99499,33851] 0 2024-02-16T22:21:47.509 INFO:tasks.workunit.client.0.smithi114.stdout:5/80: unlink d0/f11 0 2024-02-16T22:21:47.510 INFO:tasks.workunit.client.1.smithi123.stdout:7/140: mkdir d3/d12/d34 0 2024-02-16T22:21:47.517 INFO:tasks.workunit.client.1.smithi123.stdout:7/141: dwrite d3/f2a [0,4194304] 0 2024-02-16T22:21:47.517 INFO:tasks.workunit.client.1.smithi123.stdout:9/210: link d4/l9 d4/d17/d24/l44 0 2024-02-16T22:21:47.518 INFO:tasks.workunit.client.0.smithi114.stdout:4/101: getdents dd 0 2024-02-16T22:21:47.518 INFO:tasks.workunit.client.0.smithi114.stdout:4/102: read - dd/f20 zero size 2024-02-16T22:21:47.519 INFO:tasks.workunit.client.0.smithi114.stdout:4/103: dread - dd/f19 zero size 2024-02-16T22:21:47.525 INFO:tasks.workunit.client.1.smithi123.stdout:6/149: symlink d0/d6/l37 0 2024-02-16T22:21:47.527 INFO:tasks.workunit.client.0.smithi114.stdout:5/81: unlink d0/df/f14 0 2024-02-16T22:21:47.527 INFO:tasks.workunit.client.0.smithi114.stdout:5/82: dread - d0/d10/f15 zero size 2024-02-16T22:21:47.528 INFO:tasks.workunit.client.1.smithi123.stdout:5/115: dwrite d6/dd/df/d1f/d25/f1a [0,4194304] 0 2024-02-16T22:21:47.530 INFO:tasks.workunit.client.1.smithi123.stdout:8/183: mkdir d0/d29/d42 0 2024-02-16T22:21:47.530 INFO:tasks.workunit.client.0.smithi114.stdout:4/104: creat dd/f21 x:0 0 0 2024-02-16T22:21:47.530 INFO:tasks.workunit.client.1.smithi123.stdout:7/142: dwrite d3/d12/d24/f31 [0,4194304] 0 2024-02-16T22:21:47.535 INFO:tasks.workunit.client.1.smithi123.stdout:8/184: write d0/d12/f1e [901670,48615] 0 2024-02-16T22:21:47.536 INFO:tasks.workunit.client.1.smithi123.stdout:3/125: dwrite d0/f17 [0,4194304] 0 2024-02-16T22:21:47.538 INFO:tasks.workunit.client.0.smithi114.stdout:4/105: rename dd/f15 to dd/d1b/f22 0 2024-02-16T22:21:47.539 INFO:tasks.workunit.client.1.smithi123.stdout:0/145: write d1/d8/fa [118247,26464] 0 2024-02-16T22:21:47.542 INFO:tasks.workunit.client.0.smithi114.stdout:2/118: link dd/f10 dd/f28 0 2024-02-16T22:21:47.542 INFO:tasks.workunit.client.1.smithi123.stdout:9/211: creat d4/d17/d20/f45 x:0 0 0 2024-02-16T22:21:47.543 INFO:tasks.workunit.client.0.smithi114.stdout:2/119: write dd/df/f1e [1660556,60739] 0 2024-02-16T22:21:47.544 INFO:tasks.workunit.client.1.smithi123.stdout:1/124: mknod d3/d6/db/c2c 0 2024-02-16T22:21:47.545 INFO:tasks.workunit.client.0.smithi114.stdout:9/93: getdents d2/dd/d1c 0 2024-02-16T22:21:47.547 INFO:tasks.workunit.client.0.smithi114.stdout:7/69: getdents d3 0 2024-02-16T22:21:47.547 INFO:tasks.workunit.client.1.smithi123.stdout:9/212: chown d4/l14 18157 1 2024-02-16T22:21:47.547 INFO:tasks.workunit.client.1.smithi123.stdout:5/116: dread d6/f12 [0,4194304] 0 2024-02-16T22:21:47.548 INFO:tasks.workunit.client.0.smithi114.stdout:7/70: truncate d3/fe 371925 0 2024-02-16T22:21:47.548 INFO:tasks.workunit.client.0.smithi114.stdout:4/106: fsync f9 0 2024-02-16T22:21:47.548 INFO:tasks.workunit.client.1.smithi123.stdout:6/150: dwrite d0/d6/f15 [4194304,4194304] 0 2024-02-16T22:21:47.549 INFO:tasks.workunit.client.1.smithi123.stdout:5/117: readlink l2 0 2024-02-16T22:21:47.549 INFO:tasks.workunit.client.0.smithi114.stdout:9/94: mknod d2/dd/d1c/c20 0 2024-02-16T22:21:47.550 INFO:tasks.workunit.client.0.smithi114.stdout:0/124: sync 2024-02-16T22:21:47.551 INFO:tasks.workunit.client.1.smithi123.stdout:6/151: readlink d0/d6/d2f/l33 0 2024-02-16T22:21:47.551 INFO:tasks.workunit.client.1.smithi123.stdout:4/176: dwrite d5/d1c/f30 [0,4194304] 0 2024-02-16T22:21:47.551 INFO:tasks.workunit.client.1.smithi123.stdout:7/143: dread d3/d12/d24/f31 [0,4194304] 0 2024-02-16T22:21:47.551 INFO:tasks.workunit.client.0.smithi114.stdout:7/71: mkdir d3/d14 0 2024-02-16T22:21:47.552 INFO:tasks.workunit.client.1.smithi123.stdout:6/152: stat d0/d9/dc/f17 0 2024-02-16T22:21:47.552 INFO:tasks.workunit.client.0.smithi114.stdout:0/125: chown d1/fe 0 1 2024-02-16T22:21:47.554 INFO:tasks.workunit.client.1.smithi123.stdout:2/159: dread d4/d11/f18 [0,4194304] 0 2024-02-16T22:21:47.559 INFO:tasks.workunit.client.1.smithi123.stdout:3/126: symlink d0/d20/l24 0 2024-02-16T22:21:47.559 INFO:tasks.workunit.client.0.smithi114.stdout:4/107: dwrite dd/f13 [0,4194304] 0 2024-02-16T22:21:47.561 INFO:tasks.workunit.client.0.smithi114.stdout:9/95: dwrite d2/f4 [4194304,4194304] 0 2024-02-16T22:21:47.563 INFO:tasks.workunit.client.0.smithi114.stdout:4/108: creat dd/d1b/f23 x:0 0 0 2024-02-16T22:21:47.563 INFO:tasks.workunit.client.0.smithi114.stdout:4/109: readlink dd/l1e 0 2024-02-16T22:21:47.565 INFO:tasks.workunit.client.1.smithi123.stdout:5/118: symlink d6/l26 0 2024-02-16T22:21:47.565 INFO:tasks.workunit.client.0.smithi114.stdout:0/126: truncate d1/d16/d18/f1c 209536 0 2024-02-16T22:21:47.565 INFO:tasks.workunit.client.0.smithi114.stdout:0/127: readlink d1/l4 0 2024-02-16T22:21:47.568 INFO:tasks.workunit.client.0.smithi114.stdout:4/110: mkdir dd/d1b/d24 0 2024-02-16T22:21:47.569 INFO:tasks.workunit.client.0.smithi114.stdout:9/96: link f1 d2/d1a/f21 0 2024-02-16T22:21:47.569 INFO:tasks.workunit.client.0.smithi114.stdout:0/128: truncate d1/d16/d18/f22 1296030 0 2024-02-16T22:21:47.570 INFO:tasks.workunit.client.1.smithi123.stdout:2/160: chown d4/d1a/d1d/l2e 91 1 2024-02-16T22:21:47.571 INFO:tasks.workunit.client.0.smithi114.stdout:4/111: rename l6 to dd/d1b/d24/l25 0 2024-02-16T22:21:47.572 INFO:tasks.workunit.client.0.smithi114.stdout:4/112: chown f9 2 1 2024-02-16T22:21:47.574 INFO:tasks.workunit.client.0.smithi114.stdout:0/129: dread d1/d16/f19 [0,4194304] 0 2024-02-16T22:21:47.574 INFO:tasks.workunit.client.1.smithi123.stdout:0/146: mknod d1/c27 0 2024-02-16T22:21:47.575 INFO:tasks.workunit.client.0.smithi114.stdout:0/130: write d1/d1a/f1d [212957,11579] 0 2024-02-16T22:21:47.576 INFO:tasks.workunit.client.1.smithi123.stdout:9/213: mknod d4/d36/c46 0 2024-02-16T22:21:47.577 INFO:tasks.workunit.client.0.smithi114.stdout:4/113: dread dd/d1b/f1c [4194304,4194304] 0 2024-02-16T22:21:47.577 INFO:tasks.workunit.client.0.smithi114.stdout:0/131: dread d1/d16/f19 [0,4194304] 0 2024-02-16T22:21:47.577 INFO:tasks.workunit.client.0.smithi114.stdout:4/114: chown f9 682473 1 2024-02-16T22:21:47.578 INFO:tasks.workunit.client.1.smithi123.stdout:2/161: creat d4/d1a/d1d/d13/f36 x:0 0 0 2024-02-16T22:21:47.579 INFO:tasks.workunit.client.1.smithi123.stdout:3/127: symlink d0/l25 0 2024-02-16T22:21:47.580 INFO:tasks.workunit.client.0.smithi114.stdout:0/132: symlink d1/d16/d18/l27 0 2024-02-16T22:21:47.581 INFO:tasks.workunit.client.0.smithi114.stdout:0/133: write d1/fa [1921223,85330] 0 2024-02-16T22:21:47.581 INFO:tasks.workunit.client.0.smithi114.stdout:4/115: fdatasync dd/d1b/f22 0 2024-02-16T22:21:47.582 INFO:tasks.workunit.client.1.smithi123.stdout:8/185: rename d0/df/d25 to d0/d17/d43 0 2024-02-16T22:21:47.584 INFO:tasks.workunit.client.0.smithi114.stdout:4/116: symlink dd/d1b/l26 0 2024-02-16T22:21:47.584 INFO:tasks.workunit.client.0.smithi114.stdout:4/117: read dd/f13 [1487565,12604] 0 2024-02-16T22:21:47.585 INFO:tasks.workunit.client.1.smithi123.stdout:3/128: dwrite d0/f7 [0,4194304] 0 2024-02-16T22:21:47.585 INFO:tasks.workunit.client.1.smithi123.stdout:6/153: creat d0/f38 x:0 0 0 2024-02-16T22:21:47.586 INFO:tasks.workunit.client.1.smithi123.stdout:0/147: dread d1/d8/d13/f15 [0,4194304] 0 2024-02-16T22:21:47.586 INFO:tasks.workunit.client.0.smithi114.stdout:0/134: mkdir d1/d16/d1f/d28 0 2024-02-16T22:21:47.587 INFO:tasks.workunit.client.0.smithi114.stdout:0/135: readlink d1/lc 0 2024-02-16T22:21:47.588 INFO:tasks.workunit.client.0.smithi114.stdout:4/118: creat dd/d1b/f27 x:0 0 0 2024-02-16T22:21:47.589 INFO:tasks.workunit.client.1.smithi123.stdout:9/214: creat d4/d17/d3f/f47 x:0 0 0 2024-02-16T22:21:47.591 INFO:tasks.workunit.client.1.smithi123.stdout:9/215: write d4/d17/d3f/f47 [624116,28398] 0 2024-02-16T22:21:47.591 INFO:tasks.workunit.client.1.smithi123.stdout:1/125: rename d3/d6/d24 to d3/d6/db/d10/d2d 0 2024-02-16T22:21:47.593 INFO:tasks.workunit.client.1.smithi123.stdout:9/216: dread - d4/d17/d20/f3c zero size 2024-02-16T22:21:47.594 INFO:tasks.workunit.client.0.smithi114.stdout:4/119: unlink dd/d1b/f23 0 2024-02-16T22:21:47.596 INFO:tasks.workunit.client.0.smithi114.stdout:4/120: truncate dd/d1b/f1d 32604 0 2024-02-16T22:21:47.597 INFO:tasks.workunit.client.0.smithi114.stdout:4/121: chown ca 23543 1 2024-02-16T22:21:47.597 INFO:tasks.workunit.client.1.smithi123.stdout:7/144: getdents d3 0 2024-02-16T22:21:47.598 INFO:tasks.workunit.client.0.smithi114.stdout:4/122: write dd/d1b/f1d [1008728,100399] 0 2024-02-16T22:21:47.598 INFO:tasks.workunit.client.0.smithi114.stdout:4/123: stat dd/d1b 0 2024-02-16T22:21:47.599 INFO:tasks.workunit.client.1.smithi123.stdout:3/129: creat d0/d20/f26 x:0 0 0 2024-02-16T22:21:47.603 INFO:tasks.workunit.client.0.smithi114.stdout:4/124: symlink dd/l28 0 2024-02-16T22:21:47.604 INFO:tasks.workunit.client.0.smithi114.stdout:4/125: chown dd 0 1 2024-02-16T22:21:47.606 INFO:tasks.workunit.client.1.smithi123.stdout:6/154: mkdir d0/d9/dc/d39 0 2024-02-16T22:21:47.607 INFO:tasks.workunit.client.1.smithi123.stdout:0/148: chown d1/d4/l21 5 1 2024-02-16T22:21:47.609 INFO:tasks.workunit.client.1.smithi123.stdout:6/155: write d0/d6/fa [1156672,130693] 0 2024-02-16T22:21:47.610 INFO:tasks.workunit.client.1.smithi123.stdout:9/217: dwrite d4/f12 [4194304,4194304] 0 2024-02-16T22:21:47.610 INFO:tasks.workunit.client.1.smithi123.stdout:1/126: rename d3/d6/c1d to d3/d6/db/d10/c2e 0 2024-02-16T22:21:47.614 INFO:tasks.workunit.client.1.smithi123.stdout:9/218: chown d4/d17/d20/f45 7439 1 2024-02-16T22:21:47.614 INFO:tasks.workunit.client.1.smithi123.stdout:7/145: dwrite d3/f8 [0,4194304] 0 2024-02-16T22:21:47.616 INFO:tasks.workunit.client.1.smithi123.stdout:2/162: creat d4/f37 x:0 0 0 2024-02-16T22:21:47.617 INFO:tasks.workunit.client.1.smithi123.stdout:9/219: truncate d4/d17/d24/f28 638080 0 2024-02-16T22:21:47.619 INFO:tasks.workunit.client.1.smithi123.stdout:3/130: chown d0/f12 18239 1 2024-02-16T22:21:47.619 INFO:tasks.workunit.client.0.smithi114.stdout:7/72: unlink d3/fd 0 2024-02-16T22:21:47.621 INFO:tasks.workunit.client.1.smithi123.stdout:3/131: readlink d0/d20/l24 0 2024-02-16T22:21:47.623 INFO:tasks.workunit.client.1.smithi123.stdout:1/127: dwrite d3/d6/db/d10/f29 [0,4194304] 0 2024-02-16T22:21:47.623 INFO:tasks.workunit.client.0.smithi114.stdout:9/97: rmdir d2 39 2024-02-16T22:21:47.627 INFO:tasks.workunit.client.1.smithi123.stdout:0/149: rmdir d1/d4/d17 39 2024-02-16T22:21:47.628 INFO:tasks.workunit.client.1.smithi123.stdout:0/150: write d1/d8/fa [1245435,54374] 0 2024-02-16T22:21:47.634 INFO:tasks.workunit.client.0.smithi114.stdout:9/98: dread - d2/fc zero size 2024-02-16T22:21:47.634 INFO:tasks.workunit.client.1.smithi123.stdout:6/156: dread d0/d6/d2f/f22 [0,4194304] 0 2024-02-16T22:21:47.634 INFO:tasks.workunit.client.1.smithi123.stdout:7/146: mkdir d3/d26/d35 0 2024-02-16T22:21:47.643 INFO:tasks.workunit.client.0.smithi114.stdout:9/99: dwrite d2/f12 [0,4194304] 0 2024-02-16T22:21:47.643 INFO:tasks.workunit.client.1.smithi123.stdout:3/132: fsync d0/ff 0 2024-02-16T22:21:47.645 INFO:tasks.workunit.client.1.smithi123.stdout:1/128: symlink d3/d6/db/d10/l2f 0 2024-02-16T22:21:47.648 INFO:tasks.workunit.client.0.smithi114.stdout:9/100: rename d2/f16 to d2/dd/d1c/d1f/f22 0 2024-02-16T22:21:47.649 INFO:tasks.workunit.client.0.smithi114.stdout:9/101: write d2/fc [476926,97009] 0 2024-02-16T22:21:47.652 INFO:tasks.workunit.client.1.smithi123.stdout:6/157: mkdir d0/d9/dc/d3a 0 2024-02-16T22:21:47.654 INFO:tasks.workunit.client.1.smithi123.stdout:7/147: readlink d3/d12/d24/l30 0 2024-02-16T22:21:47.654 INFO:tasks.workunit.client.1.smithi123.stdout:0/151: dread d1/d8/d13/f15 [0,4194304] 0 2024-02-16T22:21:47.655 INFO:tasks.workunit.client.1.smithi123.stdout:8/186: write d0/d12/f3f [4649869,9724] 0 2024-02-16T22:21:47.655 INFO:tasks.workunit.client.0.smithi114.stdout:9/102: read f1 [3577112,49223] 0 2024-02-16T22:21:47.656 INFO:tasks.workunit.client.0.smithi114.stdout:9/103: chown d2/dd/d1c/d1f/f22 85 1 2024-02-16T22:21:47.656 INFO:tasks.workunit.client.1.smithi123.stdout:8/187: write d0/df/f16 [4867714,98106] 0 2024-02-16T22:21:47.657 INFO:tasks.workunit.client.0.smithi114.stdout:4/126: getdents dd/d1b 0 2024-02-16T22:21:47.657 INFO:tasks.workunit.client.1.smithi123.stdout:8/188: truncate d0/f14 4750734 0 2024-02-16T22:21:47.659 INFO:tasks.workunit.client.1.smithi123.stdout:7/148: dread d3/d5/f9 [0,4194304] 0 2024-02-16T22:21:47.659 INFO:tasks.workunit.client.1.smithi123.stdout:5/119: dread d6/f11 [0,4194304] 0 2024-02-16T22:21:47.660 INFO:tasks.workunit.client.1.smithi123.stdout:7/149: truncate d3/fd 1005239 0 2024-02-16T22:21:47.661 INFO:tasks.workunit.client.0.smithi114.stdout:4/127: write dd/f13 [825779,117378] 0 2024-02-16T22:21:47.663 INFO:tasks.workunit.client.1.smithi123.stdout:1/129: rename c2 to d3/d6/db/d10/c30 0 2024-02-16T22:21:47.665 INFO:tasks.workunit.client.0.smithi114.stdout:4/128: unlink dd/d1b/f27 0 2024-02-16T22:21:47.667 INFO:tasks.workunit.client.0.smithi114.stdout:7/73: rmdir d3 39 2024-02-16T22:21:47.673 INFO:tasks.workunit.client.1.smithi123.stdout:9/220: rmdir d4/d17/d24/d32 0 2024-02-16T22:21:47.674 INFO:tasks.workunit.client.1.smithi123.stdout:8/189: dread d0/f4 [0,4194304] 0 2024-02-16T22:21:47.674 INFO:tasks.workunit.client.0.smithi114.stdout:4/129: dwrite dd/d1b/f1c [0,4194304] 0 2024-02-16T22:21:47.676 INFO:tasks.workunit.client.0.smithi114.stdout:7/74: mknod d3/d14/c15 0 2024-02-16T22:21:47.677 INFO:tasks.workunit.client.1.smithi123.stdout:5/120: fsync d6/dd/df/f1b 0 2024-02-16T22:21:47.679 INFO:tasks.workunit.client.0.smithi114.stdout:4/130: rename lc to dd/d1b/d24/l29 0 2024-02-16T22:21:47.681 INFO:tasks.workunit.client.1.smithi123.stdout:8/190: dwrite d0/f14 [0,4194304] 0 2024-02-16T22:21:47.683 INFO:tasks.workunit.client.1.smithi123.stdout:8/191: stat d0/d29/d42 0 2024-02-16T22:21:47.683 INFO:tasks.workunit.client.0.smithi114.stdout:7/75: dread d3/fc [0,4194304] 0 2024-02-16T22:21:47.684 INFO:tasks.workunit.client.1.smithi123.stdout:2/163: dwrite d4/fc [0,4194304] 0 2024-02-16T22:21:47.684 INFO:tasks.workunit.client.1.smithi123.stdout:2/164: chown d4/d1a/d29 6683554 1 2024-02-16T22:21:47.685 INFO:tasks.workunit.client.1.smithi123.stdout:2/165: chown d4/d1a/l2f 0 1 2024-02-16T22:21:47.685 INFO:tasks.workunit.client.1.smithi123.stdout:6/158: mknod d0/d9/dc/d3a/c3b 0 2024-02-16T22:21:47.688 INFO:tasks.workunit.client.1.smithi123.stdout:6/159: write d0/d6/d1a/d2e/f34 [579150,51387] 0 2024-02-16T22:21:47.689 INFO:tasks.workunit.client.0.smithi114.stdout:9/104: dwrite d2/f12 [4194304,4194304] 0 2024-02-16T22:21:47.690 INFO:tasks.workunit.client.1.smithi123.stdout:0/152: creat d1/d4/d17/f28 x:0 0 0 2024-02-16T22:21:47.693 INFO:tasks.workunit.client.1.smithi123.stdout:8/192: dwrite d0/df/f16 [0,4194304] 0 2024-02-16T22:21:47.694 INFO:tasks.workunit.client.0.smithi114.stdout:7/76: unlink d3/c13 0 2024-02-16T22:21:47.695 INFO:tasks.workunit.client.1.smithi123.stdout:8/193: chown d0/d12/f22 3972199 1 2024-02-16T22:21:47.697 INFO:tasks.workunit.client.1.smithi123.stdout:6/160: dwrite d0/d6/d1a/f36 [0,4194304] 0 2024-02-16T22:21:47.698 INFO:tasks.workunit.client.0.smithi114.stdout:4/131: dwrite dd/f19 [0,4194304] 0 2024-02-16T22:21:47.699 INFO:tasks.workunit.client.0.smithi114.stdout:7/77: write d3/fe [1186119,101318] 0 2024-02-16T22:21:47.701 INFO:tasks.workunit.client.0.smithi114.stdout:4/132: write dd/d1b/f22 [8428361,34581] 0 2024-02-16T22:21:47.701 INFO:tasks.workunit.client.0.smithi114.stdout:9/105: dread d2/f5 [4194304,4194304] 0 2024-02-16T22:21:47.708 INFO:tasks.workunit.client.0.smithi114.stdout:7/78: chown d3/c9 88218993 1 2024-02-16T22:21:47.709 INFO:tasks.workunit.client.1.smithi123.stdout:4/177: sync 2024-02-16T22:21:47.713 INFO:tasks.workunit.client.1.smithi123.stdout:5/121: mkdir d6/dd/df/d1f/d27 0 2024-02-16T22:21:47.714 INFO:tasks.workunit.client.1.smithi123.stdout:7/150: write d3/d5/ff [2675901,41290] 0 2024-02-16T22:21:47.714 INFO:tasks.workunit.client.1.smithi123.stdout:1/130: mknod d3/c31 0 2024-02-16T22:21:47.716 INFO:tasks.workunit.client.1.smithi123.stdout:7/151: stat d3/cb 0 2024-02-16T22:21:47.716 INFO:tasks.workunit.client.1.smithi123.stdout:5/122: chown d6/dd/df/d1f/d25/c23 3865 1 2024-02-16T22:21:47.717 INFO:tasks.workunit.client.0.smithi114.stdout:9/106: dread d2/f4 [0,4194304] 0 2024-02-16T22:21:47.718 INFO:tasks.workunit.client.0.smithi114.stdout:7/79: readlink d3/lb 0 2024-02-16T22:21:47.718 INFO:tasks.workunit.client.1.smithi123.stdout:4/178: dwrite d5/f14 [4194304,4194304] 0 2024-02-16T22:21:47.719 INFO:tasks.workunit.client.1.smithi123.stdout:2/166: mkdir d4/d1a/d1d/d38 0 2024-02-16T22:21:47.724 INFO:tasks.workunit.client.1.smithi123.stdout:7/152: dread d3/d5/ff [0,4194304] 0 2024-02-16T22:21:47.724 INFO:tasks.workunit.client.1.smithi123.stdout:7/153: fdatasync d3/fd 0 2024-02-16T22:21:47.724 INFO:tasks.workunit.client.1.smithi123.stdout:9/221: truncate d4/d17/f1d 1341224 0 2024-02-16T22:21:47.724 INFO:tasks.workunit.client.1.smithi123.stdout:9/222: truncate d4/d17/d24/f3d 803410 0 2024-02-16T22:21:47.725 INFO:tasks.workunit.client.0.smithi114.stdout:7/80: truncate d3/f10 41336 0 2024-02-16T22:21:47.725 INFO:tasks.workunit.client.0.smithi114.stdout:7/81: truncate d3/fe 1662525 0 2024-02-16T22:21:47.725 INFO:tasks.workunit.client.0.smithi114.stdout:4/133: link dd/c1a dd/c2a 0 2024-02-16T22:21:47.727 INFO:tasks.workunit.client.1.smithi123.stdout:8/194: unlink d0/df/f13 0 2024-02-16T22:21:47.729 INFO:tasks.workunit.client.1.smithi123.stdout:3/133: getdents d0 0 2024-02-16T22:21:47.729 INFO:tasks.workunit.client.1.smithi123.stdout:3/134: chown d0/f17 191811 1 2024-02-16T22:21:47.731 INFO:tasks.workunit.client.0.smithi114.stdout:4/134: creat dd/f2b x:0 0 0 2024-02-16T22:21:47.731 INFO:tasks.workunit.client.0.smithi114.stdout:4/135: stat ca 0 2024-02-16T22:21:47.737 INFO:tasks.workunit.client.0.smithi114.stdout:4/136: creat dd/d1b/f2c x:0 0 0 2024-02-16T22:21:47.738 INFO:tasks.workunit.client.0.smithi114.stdout:4/137: truncate dd/f20 58539 0 2024-02-16T22:21:47.738 INFO:tasks.workunit.client.1.smithi123.stdout:2/167: creat d4/d1a/f39 x:0 0 0 2024-02-16T22:21:47.738 INFO:tasks.workunit.client.1.smithi123.stdout:2/168: readlink d4/d1a/l2f 0 2024-02-16T22:21:47.741 INFO:tasks.workunit.client.1.smithi123.stdout:7/154: fdatasync d3/d12/f14 0 2024-02-16T22:21:47.742 INFO:tasks.workunit.client.1.smithi123.stdout:9/223: rename d4/c7 to d4/d17/c48 0 2024-02-16T22:21:47.743 INFO:tasks.workunit.client.1.smithi123.stdout:0/153: dwrite d1/f12 [0,4194304] 0 2024-02-16T22:21:47.743 INFO:tasks.workunit.client.0.smithi114.stdout:9/107: truncate d2/dd/f10 4076732 0 2024-02-16T22:21:47.744 INFO:tasks.workunit.client.1.smithi123.stdout:0/154: readlink d1/d4/l21 0 2024-02-16T22:21:47.744 INFO:tasks.workunit.client.1.smithi123.stdout:0/155: fdatasync d1/f22 0 2024-02-16T22:21:47.745 INFO:tasks.workunit.client.1.smithi123.stdout:4/179: fdatasync d5/f14 0 2024-02-16T22:21:47.746 INFO:tasks.workunit.client.1.smithi123.stdout:0/156: write d1/f1f [178895,130816] 0 2024-02-16T22:21:47.747 INFO:tasks.workunit.client.1.smithi123.stdout:8/195: creat d0/d17/d26/f44 x:0 0 0 2024-02-16T22:21:47.747 INFO:tasks.workunit.client.0.smithi114.stdout:7/82: write d3/ff [1342231,23847] 0 2024-02-16T22:21:47.748 INFO:tasks.workunit.client.1.smithi123.stdout:3/135: rename d0/d20/c23 to d0/d20/c27 0 2024-02-16T22:21:47.749 INFO:tasks.workunit.client.1.smithi123.stdout:3/136: chown d0 4113 1 2024-02-16T22:21:47.749 INFO:tasks.workunit.client.0.smithi114.stdout:4/138: getdents dd/d1b/d24 0 2024-02-16T22:21:47.750 INFO:tasks.workunit.client.0.smithi114.stdout:4/139: write dd/d1b/f1c [5384148,23297] 0 2024-02-16T22:21:47.750 INFO:tasks.workunit.client.1.smithi123.stdout:0/157: chown d1/c27 903171 1 2024-02-16T22:21:47.752 INFO:tasks.workunit.client.1.smithi123.stdout:9/224: dwrite d4/d17/f1a [0,4194304] 0 2024-02-16T22:21:47.752 INFO:tasks.workunit.client.0.smithi114.stdout:4/140: write dd/f11 [743952,77834] 0 2024-02-16T22:21:47.752 INFO:tasks.workunit.client.0.smithi114.stdout:9/108: mkdir d2/d1a/d23 0 2024-02-16T22:21:47.753 INFO:tasks.workunit.client.0.smithi114.stdout:4/141: fsync dd/f13 0 2024-02-16T22:21:47.756 INFO:tasks.workunit.client.0.smithi114.stdout:9/109: mknod d2/dd/d1c/c24 0 2024-02-16T22:21:47.756 INFO:tasks.workunit.client.1.smithi123.stdout:7/155: fsync d3/d5/ff 0 2024-02-16T22:21:47.760 INFO:tasks.workunit.client.0.smithi114.stdout:4/142: symlink dd/l2d 0 2024-02-16T22:21:47.763 INFO:tasks.workunit.client.1.smithi123.stdout:4/180: rmdir d5 39 2024-02-16T22:21:47.764 INFO:tasks.workunit.client.0.smithi114.stdout:4/143: mkdir dd/d2e 0 2024-02-16T22:21:47.765 INFO:tasks.workunit.client.0.smithi114.stdout:4/144: write dd/d1b/f1c [8384198,109124] 0 2024-02-16T22:21:47.767 INFO:tasks.workunit.client.0.smithi114.stdout:4/145: write dd/d1b/f22 [6878526,120550] 0 2024-02-16T22:21:47.767 INFO:tasks.workunit.client.0.smithi114.stdout:4/146: read - dd/f2b zero size 2024-02-16T22:21:47.769 INFO:tasks.workunit.client.1.smithi123.stdout:3/137: creat d0/d20/f28 x:0 0 0 2024-02-16T22:21:47.771 INFO:tasks.workunit.client.0.smithi114.stdout:9/110: dread d2/dd/f10 [0,4194304] 0 2024-02-16T22:21:47.771 INFO:tasks.workunit.client.1.smithi123.stdout:0/158: rename d1/l9 to d1/d8/d13/l29 0 2024-02-16T22:21:47.772 INFO:tasks.workunit.client.1.smithi123.stdout:0/159: truncate d1/d4/d17/f28 510935 0 2024-02-16T22:21:47.774 INFO:tasks.workunit.client.0.smithi114.stdout:4/147: creat dd/d1b/d24/f2f x:0 0 0 2024-02-16T22:21:47.775 INFO:tasks.workunit.client.1.smithi123.stdout:2/169: unlink d4/d1a/d29/c34 0 2024-02-16T22:21:47.777 INFO:tasks.workunit.client.1.smithi123.stdout:9/225: dread d4/d17/f1d [0,4194304] 0 2024-02-16T22:21:47.780 INFO:tasks.workunit.client.0.smithi114.stdout:4/148: dwrite dd/d1b/f22 [4194304,4194304] 0 2024-02-16T22:21:47.780 INFO:tasks.workunit.client.0.smithi114.stdout:4/149: chown dd/d1b/f1d 182175534 1 2024-02-16T22:21:47.781 INFO:tasks.workunit.client.0.smithi114.stdout:4/150: stat dd/f19 0 2024-02-16T22:21:47.809 INFO:tasks.workunit.client.1.smithi123.stdout:6/161: sync 2024-02-16T22:21:47.809 INFO:tasks.workunit.client.1.smithi123.stdout:6/162: write d0/d6/f1e [60373,55778] 0 2024-02-16T22:21:47.823 INFO:tasks.workunit.client.0.smithi114.stdout:1/77: sync 2024-02-16T22:21:47.826 INFO:tasks.workunit.client.0.smithi114.stdout:1/78: dread d4/d9/f10 [0,4194304] 0 2024-02-16T22:21:47.854 INFO:tasks.workunit.client.0.smithi114.stdout:9/111: symlink d2/dd/d1c/l25 0 2024-02-16T22:21:47.854 INFO:tasks.workunit.client.0.smithi114.stdout:9/112: chown d2/fc 117678 1 2024-02-16T22:21:47.856 INFO:tasks.workunit.client.0.smithi114.stdout:7/83: truncate d3/f8 1215393 0 2024-02-16T22:21:47.856 INFO:tasks.workunit.client.0.smithi114.stdout:7/84: truncate d3/f10 159174 0 2024-02-16T22:21:47.859 INFO:tasks.workunit.client.1.smithi123.stdout:4/181: chown d5 5228816 1 2024-02-16T22:21:47.860 INFO:tasks.workunit.client.1.smithi123.stdout:1/131: getdents d3/d6/d11 0 2024-02-16T22:21:47.862 INFO:tasks.workunit.client.1.smithi123.stdout:8/196: mkdir d0/d17/d26/d32/d45 0 2024-02-16T22:21:47.864 INFO:tasks.workunit.client.1.smithi123.stdout:5/123: getdents d6/dd/df 0 2024-02-16T22:21:47.865 INFO:tasks.workunit.client.0.smithi114.stdout:9/113: creat d2/f26 x:0 0 0 2024-02-16T22:21:47.867 INFO:tasks.workunit.client.1.smithi123.stdout:3/138: mkdir d0/d20/d29 0 2024-02-16T22:21:47.869 INFO:tasks.workunit.client.1.smithi123.stdout:0/160: unlink d1/l2 0 2024-02-16T22:21:47.870 INFO:tasks.workunit.client.0.smithi114.stdout:4/151: getdents dd/d2e 0 2024-02-16T22:21:47.872 INFO:tasks.workunit.client.1.smithi123.stdout:2/170: rename d4/d11/f18 to d4/d1a/d29/f3a 0 2024-02-16T22:21:47.873 INFO:tasks.workunit.client.1.smithi123.stdout:2/171: truncate d4/d1a/d1d/d13/d25/f26 264115 0 2024-02-16T22:21:47.875 INFO:tasks.workunit.client.1.smithi123.stdout:9/226: creat d4/d17/d24/f49 x:0 0 0 2024-02-16T22:21:47.876 INFO:tasks.workunit.client.0.smithi114.stdout:4/152: dwrite dd/f12 [0,4194304] 0 2024-02-16T22:21:47.879 INFO:tasks.workunit.client.0.smithi114.stdout:9/114: creat d2/dd/d1c/f27 x:0 0 0 2024-02-16T22:21:47.880 INFO:tasks.workunit.client.1.smithi123.stdout:6/163: fsync d0/d29/f2d 0 2024-02-16T22:21:47.881 INFO:tasks.workunit.client.1.smithi123.stdout:6/164: chown d0/d9/c25 321361624 1 2024-02-16T22:21:47.883 INFO:tasks.workunit.client.0.smithi114.stdout:7/85: write d3/fc [513158,76368] 0 2024-02-16T22:21:47.884 INFO:tasks.workunit.client.1.smithi123.stdout:0/161: dread d1/f6 [0,4194304] 0 2024-02-16T22:21:47.885 INFO:tasks.workunit.client.1.smithi123.stdout:0/162: readlink d1/d8/d13/d1d/l23 0 2024-02-16T22:21:47.885 INFO:tasks.workunit.client.1.smithi123.stdout:4/182: write d5/d1c/d23/d31/f2d [872270,16714] 0 2024-02-16T22:21:47.888 INFO:tasks.workunit.client.1.smithi123.stdout:1/132: chown d3/d6/db/d10/c1e 37 1 2024-02-16T22:21:47.891 INFO:tasks.workunit.client.0.smithi114.stdout:7/86: symlink d3/d14/l16 0 2024-02-16T22:21:47.891 INFO:tasks.workunit.client.1.smithi123.stdout:1/133: dread d3/d6/d11/f1f [0,4194304] 0 2024-02-16T22:21:47.892 INFO:tasks.workunit.client.1.smithi123.stdout:0/163: dwrite d1/d8/fa [0,4194304] 0 2024-02-16T22:21:47.893 INFO:tasks.workunit.client.0.smithi114.stdout:1/79: getdents d4 0 2024-02-16T22:21:47.893 INFO:tasks.workunit.client.0.smithi114.stdout:1/80: fdatasync d4/f13 0 2024-02-16T22:21:47.894 INFO:tasks.workunit.client.1.smithi123.stdout:0/164: write d1/d4/f11 [498873,18145] 0 2024-02-16T22:21:47.895 INFO:tasks.workunit.client.0.smithi114.stdout:4/153: mknod dd/d2e/c30 0 2024-02-16T22:21:47.895 INFO:tasks.workunit.client.1.smithi123.stdout:0/165: write d1/f1f [1620088,74511] 0 2024-02-16T22:21:47.896 INFO:tasks.workunit.client.1.smithi123.stdout:0/166: truncate d1/f22 254143 0 2024-02-16T22:21:47.900 INFO:tasks.workunit.client.0.smithi114.stdout:9/115: creat d2/d1a/d23/f28 x:0 0 0 2024-02-16T22:21:47.900 INFO:tasks.workunit.client.1.smithi123.stdout:9/227: creat d4/d17/d24/f4a x:0 0 0 2024-02-16T22:21:47.901 INFO:tasks.workunit.client.0.smithi114.stdout:7/87: dwrite d3/fe [0,4194304] 0 2024-02-16T22:21:47.905 INFO:tasks.workunit.client.0.smithi114.stdout:4/154: mknod dd/c31 0 2024-02-16T22:21:47.908 INFO:tasks.workunit.client.0.smithi114.stdout:1/81: creat d4/d9/f18 x:0 0 0 2024-02-16T22:21:47.910 INFO:tasks.workunit.client.1.smithi123.stdout:4/183: creat d5/d1c/d23/d36/f39 x:0 0 0 2024-02-16T22:21:47.911 INFO:tasks.workunit.client.0.smithi114.stdout:7/88: readlink d3/l11 0 2024-02-16T22:21:47.911 INFO:tasks.workunit.client.0.smithi114.stdout:7/89: chown d3/l11 10811 1 2024-02-16T22:21:47.912 INFO:tasks.workunit.client.1.smithi123.stdout:8/197: chown d0/df/f1c 7098 1 2024-02-16T22:21:47.913 INFO:tasks.workunit.client.1.smithi123.stdout:8/198: write d0/f6 [521108,100541] 0 2024-02-16T22:21:47.913 INFO:tasks.workunit.client.0.smithi114.stdout:4/155: creat dd/f32 x:0 0 0 2024-02-16T22:21:47.914 INFO:tasks.workunit.client.1.smithi123.stdout:5/124: unlink d6/f12 0 2024-02-16T22:21:47.915 INFO:tasks.workunit.client.1.smithi123.stdout:3/139: mknod d0/d20/d29/c2a 0 2024-02-16T22:21:47.917 INFO:tasks.workunit.client.0.smithi114.stdout:7/90: symlink d3/d14/l17 0 2024-02-16T22:21:47.917 INFO:tasks.workunit.client.1.smithi123.stdout:1/134: dread d3/d6/f26 [0,4194304] 0 2024-02-16T22:21:47.918 INFO:tasks.workunit.client.1.smithi123.stdout:1/135: write d3/d6/db/d10/f29 [3822470,10769] 0 2024-02-16T22:21:47.918 INFO:tasks.workunit.client.0.smithi114.stdout:7/91: dread d3/f10 [0,4194304] 0 2024-02-16T22:21:47.922 INFO:tasks.workunit.client.1.smithi123.stdout:9/228: unlink d4/l14 0 2024-02-16T22:21:47.923 INFO:tasks.workunit.client.1.smithi123.stdout:9/229: chown d4/d17/d24 9863514 1 2024-02-16T22:21:47.923 INFO:tasks.workunit.client.0.smithi114.stdout:4/156: rmdir dd 39 2024-02-16T22:21:47.926 INFO:tasks.workunit.client.0.smithi114.stdout:9/116: write d2/dd/ff [1462557,115503] 0 2024-02-16T22:21:47.929 INFO:tasks.workunit.client.0.smithi114.stdout:1/82: truncate d4/fa 1291581 0 2024-02-16T22:21:47.929 INFO:tasks.workunit.client.0.smithi114.stdout:1/83: readlink d4/l8 0 2024-02-16T22:21:47.930 INFO:tasks.workunit.client.0.smithi114.stdout:1/84: dread - d4/d9/f11 zero size 2024-02-16T22:21:47.932 INFO:tasks.workunit.client.0.smithi114.stdout:7/92: link d3/fc d3/d14/f18 0 2024-02-16T22:21:47.932 INFO:tasks.workunit.client.0.smithi114.stdout:8/124: sync 2024-02-16T22:21:47.933 INFO:tasks.workunit.client.0.smithi114.stdout:6/63: sync 2024-02-16T22:21:47.933 INFO:tasks.workunit.client.0.smithi114.stdout:7/93: chown d3/d14/l17 3240311 1 2024-02-16T22:21:47.934 INFO:tasks.workunit.client.0.smithi114.stdout:9/117: unlink d2/l3 0 2024-02-16T22:21:47.934 INFO:tasks.workunit.client.1.smithi123.stdout:6/165: truncate d0/d9/dc/f11 100093 0 2024-02-16T22:21:47.935 INFO:tasks.workunit.client.0.smithi114.stdout:9/118: write d2/f26 [1005068,8396] 0 2024-02-16T22:21:47.935 INFO:tasks.workunit.client.1.smithi123.stdout:6/166: read - d0/d9/f19 zero size 2024-02-16T22:21:47.936 INFO:tasks.workunit.client.0.smithi114.stdout:9/119: write d2/f12 [2509753,31837] 0 2024-02-16T22:21:47.936 INFO:tasks.workunit.client.0.smithi114.stdout:4/157: mknod dd/d1b/d24/c33 0 2024-02-16T22:21:47.936 INFO:tasks.workunit.client.1.smithi123.stdout:8/199: creat d0/d12/f46 x:0 0 0 2024-02-16T22:21:47.936 INFO:tasks.workunit.client.1.smithi123.stdout:6/167: chown d0/d6/l1d 214067 1 2024-02-16T22:21:47.937 INFO:tasks.workunit.client.0.smithi114.stdout:4/158: chown dd/d1b/f22 2374836 1 2024-02-16T22:21:47.937 INFO:tasks.workunit.client.1.smithi123.stdout:6/168: dread - d0/d9/f19 zero size 2024-02-16T22:21:47.938 INFO:tasks.workunit.client.0.smithi114.stdout:4/159: write dd/f11 [1303473,128273] 0 2024-02-16T22:21:47.939 INFO:tasks.workunit.client.1.smithi123.stdout:3/140: unlink d0/l1c 0 2024-02-16T22:21:47.940 INFO:tasks.workunit.client.0.smithi114.stdout:6/64: rmdir d6/da 39 2024-02-16T22:21:47.940 INFO:tasks.workunit.client.1.smithi123.stdout:5/125: dread d6/dd/df/d1f/d25/f1a [0,4194304] 0 2024-02-16T22:21:47.942 INFO:tasks.workunit.client.1.smithi123.stdout:1/136: mknod d3/d6/db/d10/c32 0 2024-02-16T22:21:47.942 INFO:tasks.workunit.client.0.smithi114.stdout:4/160: dread f9 [0,4194304] 0 2024-02-16T22:21:47.943 INFO:tasks.workunit.client.1.smithi123.stdout:1/137: chown d3/d6/db/d10/l2f 3756194 1 2024-02-16T22:21:47.943 INFO:tasks.workunit.client.0.smithi114.stdout:4/161: fdatasync dd/d1b/f22 0 2024-02-16T22:21:47.945 INFO:tasks.workunit.client.0.smithi114.stdout:9/120: mknod d2/d1a/c29 0 2024-02-16T22:21:47.946 INFO:tasks.workunit.client.0.smithi114.stdout:9/121: stat d2/d1a/d23 0 2024-02-16T22:21:47.947 INFO:tasks.workunit.client.0.smithi114.stdout:6/65: dwrite d6/fc [0,4194304] 0 2024-02-16T22:21:47.948 INFO:tasks.workunit.client.0.smithi114.stdout:6/66: dread - d6/dd/f15 zero size 2024-02-16T22:21:47.949 INFO:tasks.workunit.client.0.smithi114.stdout:1/85: link d4/f16 d4/d9/f19 0 2024-02-16T22:21:47.950 INFO:tasks.workunit.client.0.smithi114.stdout:3/77: sync 2024-02-16T22:21:47.950 INFO:tasks.workunit.client.0.smithi114.stdout:5/83: sync 2024-02-16T22:21:47.950 INFO:tasks.workunit.client.0.smithi114.stdout:2/120: sync 2024-02-16T22:21:47.951 INFO:tasks.workunit.client.1.smithi123.stdout:9/230: unlink d4/d1e/c41 0 2024-02-16T22:21:47.951 INFO:tasks.workunit.client.0.smithi114.stdout:3/78: stat d4/dd/f19 0 2024-02-16T22:21:47.952 INFO:tasks.workunit.client.1.smithi123.stdout:9/231: chown d4/d33/f3b 1 1 2024-02-16T22:21:47.952 INFO:tasks.workunit.client.1.smithi123.stdout:9/232: readlink d4/d17/d20/l2a 0 2024-02-16T22:21:47.953 INFO:tasks.workunit.client.1.smithi123.stdout:7/156: getdents d3 0 2024-02-16T22:21:47.953 INFO:tasks.workunit.client.1.smithi123.stdout:9/233: readlink d4/d17/d20/l2e 0 2024-02-16T22:21:47.954 INFO:tasks.workunit.client.1.smithi123.stdout:7/157: stat d3/d12/d1d/f2b 0 2024-02-16T22:21:47.955 INFO:tasks.workunit.client.0.smithi114.stdout:6/67: dwrite d6/fc [0,4194304] 0 2024-02-16T22:21:47.959 INFO:tasks.workunit.client.0.smithi114.stdout:7/94: mknod d3/c19 0 2024-02-16T22:21:47.959 INFO:tasks.workunit.client.1.smithi123.stdout:7/158: dwrite d3/fe [0,4194304] 0 2024-02-16T22:21:47.960 INFO:tasks.workunit.client.1.smithi123.stdout:7/159: chown d3 5 1 2024-02-16T22:21:47.960 INFO:tasks.workunit.client.0.smithi114.stdout:6/68: dread d6/fc [0,4194304] 0 2024-02-16T22:21:47.963 INFO:tasks.workunit.client.1.smithi123.stdout:6/169: creat d0/d6/d1a/d2e/f3c x:0 0 0 2024-02-16T22:21:47.963 INFO:tasks.workunit.client.1.smithi123.stdout:6/170: readlink d0/d6/d1a/l28 0 2024-02-16T22:21:47.964 INFO:tasks.workunit.client.0.smithi114.stdout:4/162: symlink dd/d2e/l34 0 2024-02-16T22:21:47.967 INFO:tasks.workunit.client.0.smithi114.stdout:9/122: rmdir d2 39 2024-02-16T22:21:47.969 INFO:tasks.workunit.client.1.smithi123.stdout:5/126: creat d6/dd/df/f28 x:0 0 0 2024-02-16T22:21:47.970 INFO:tasks.workunit.client.1.smithi123.stdout:5/127: dread - d6/dd/df/f28 zero size 2024-02-16T22:21:47.971 INFO:tasks.workunit.client.1.smithi123.stdout:1/138: mknod d3/d6/d11/c33 0 2024-02-16T22:21:47.971 INFO:tasks.workunit.client.1.smithi123.stdout:5/128: dread - d6/dd/df/d1f/f24 zero size 2024-02-16T22:21:47.972 INFO:tasks.workunit.client.0.smithi114.stdout:1/86: symlink d4/d9/l1a 0 2024-02-16T22:21:47.972 INFO:tasks.workunit.client.1.smithi123.stdout:1/139: write d3/d6/db/d10/f29 [289610,51718] 0 2024-02-16T22:21:47.973 INFO:tasks.workunit.client.1.smithi123.stdout:0/167: creat d1/f2a x:0 0 0 2024-02-16T22:21:47.973 INFO:tasks.workunit.client.1.smithi123.stdout:0/168: dread - d1/f2a zero size 2024-02-16T22:21:47.974 INFO:tasks.workunit.client.0.smithi114.stdout:2/121: chown dd/l20 98 1 2024-02-16T22:21:47.974 INFO:tasks.workunit.client.0.smithi114.stdout:2/122: dread - dd/df/f11 zero size 2024-02-16T22:21:47.975 INFO:tasks.workunit.client.1.smithi123.stdout:2/172: getdents d4 0 2024-02-16T22:21:47.975 INFO:tasks.workunit.client.0.smithi114.stdout:2/123: stat l2 0 2024-02-16T22:21:47.976 INFO:tasks.workunit.client.1.smithi123.stdout:2/173: write d4/d1a/d1d/f2c [938447,25355] 0 2024-02-16T22:21:47.976 INFO:tasks.workunit.client.1.smithi123.stdout:9/234: symlink d4/d33/l4b 0 2024-02-16T22:21:47.978 INFO:tasks.workunit.client.0.smithi114.stdout:3/79: rename d4/f6 to d4/f1a 0 2024-02-16T22:21:47.980 INFO:tasks.workunit.client.0.smithi114.stdout:8/125: creat d4/d11/f20 x:0 0 0 2024-02-16T22:21:47.981 INFO:tasks.workunit.client.1.smithi123.stdout:4/184: rename d5/f26 to d5/d1c/d23/f3a 0 2024-02-16T22:21:47.982 INFO:tasks.workunit.client.1.smithi123.stdout:9/235: dwrite d4/d17/f1a [0,4194304] 0 2024-02-16T22:21:47.983 INFO:tasks.workunit.client.1.smithi123.stdout:9/236: chown d4/d17/d3f/f47 395 1 2024-02-16T22:21:47.983 INFO:tasks.workunit.client.1.smithi123.stdout:8/200: mkdir d0/d29/d42/d47 0 2024-02-16T22:21:47.986 INFO:tasks.workunit.client.1.smithi123.stdout:4/185: dread d5/d1c/d23/f2a [0,4194304] 0 2024-02-16T22:21:47.989 INFO:tasks.workunit.client.0.smithi114.stdout:4/163: unlink dd/d1b/f22 0 2024-02-16T22:21:47.990 INFO:tasks.workunit.client.0.smithi114.stdout:4/164: truncate dd/d1b/f1d 1887344 0 2024-02-16T22:21:47.990 INFO:tasks.workunit.client.0.smithi114.stdout:5/84: write d0/f8 [1297765,115138] 0 2024-02-16T22:21:47.991 INFO:tasks.workunit.client.0.smithi114.stdout:5/85: stat d0/c1 0 2024-02-16T22:21:47.991 INFO:tasks.workunit.client.0.smithi114.stdout:9/123: chown d2/dd/f10 7799406 1 2024-02-16T22:21:47.993 INFO:tasks.workunit.client.0.smithi114.stdout:1/87: creat d4/f1b x:0 0 0 2024-02-16T22:21:47.994 INFO:tasks.workunit.client.0.smithi114.stdout:5/86: dread d0/f9 [0,4194304] 0 2024-02-16T22:21:47.995 INFO:tasks.workunit.client.1.smithi123.stdout:5/129: symlink d6/dd/df/d1f/d25/l29 0 2024-02-16T22:21:47.995 INFO:tasks.workunit.client.0.smithi114.stdout:5/87: dread - d0/fa zero size 2024-02-16T22:21:47.996 INFO:tasks.workunit.client.0.smithi114.stdout:2/124: symlink dd/df/d1c/l29 0 2024-02-16T22:21:47.996 INFO:tasks.workunit.client.0.smithi114.stdout:5/88: dread - d0/d10/f15 zero size 2024-02-16T22:21:47.996 INFO:tasks.workunit.client.1.smithi123.stdout:5/130: write d6/dd/df/d1f/d25/f1d [64412,103867] 0 2024-02-16T22:21:47.997 INFO:tasks.workunit.client.1.smithi123.stdout:0/169: symlink d1/d8/d13/d1d/l2b 0 2024-02-16T22:21:47.997 INFO:tasks.workunit.client.1.smithi123.stdout:5/131: dread - d6/dd/df/f28 zero size 2024-02-16T22:21:47.998 INFO:tasks.workunit.client.1.smithi123.stdout:0/170: read d1/f22 [61678,52119] 0 2024-02-16T22:21:47.998 INFO:tasks.workunit.client.0.smithi114.stdout:3/80: rename d4/dd/c17 to d4/c1b 0 2024-02-16T22:21:47.998 INFO:tasks.workunit.client.0.smithi114.stdout:0/136: sync 2024-02-16T22:21:47.999 INFO:tasks.workunit.client.0.smithi114.stdout:0/137: read - d1/d1a/f26 zero size 2024-02-16T22:21:47.999 INFO:tasks.workunit.client.0.smithi114.stdout:0/138: chown d1/d16/d1f/d28 2 1 2024-02-16T22:21:48.001 INFO:tasks.workunit.client.1.smithi123.stdout:2/174: creat d4/d1a/d1d/d13/d25/f3b x:0 0 0 2024-02-16T22:21:48.001 INFO:tasks.workunit.client.0.smithi114.stdout:6/69: symlink d6/l16 0 2024-02-16T22:21:48.002 INFO:tasks.workunit.client.0.smithi114.stdout:7/95: truncate d3/d14/f18 383240 0 2024-02-16T22:21:48.004 INFO:tasks.workunit.client.1.smithi123.stdout:5/132: dwrite d6/dd/df/d1f/f24 [0,4194304] 0 2024-02-16T22:21:48.006 INFO:tasks.workunit.client.1.smithi123.stdout:8/201: rename d0/l1 to d0/d29/l48 0 2024-02-16T22:21:48.007 INFO:tasks.workunit.client.0.smithi114.stdout:4/165: mkdir dd/d1b/d24/d35 0 2024-02-16T22:21:48.007 INFO:tasks.workunit.client.1.smithi123.stdout:8/202: chown d0/c18 1 1 2024-02-16T22:21:48.008 INFO:tasks.workunit.client.1.smithi123.stdout:7/160: creat d3/d26/d35/f36 x:0 0 0 2024-02-16T22:21:48.009 INFO:tasks.workunit.client.1.smithi123.stdout:2/175: dwrite d4/f37 [0,4194304] 0 2024-02-16T22:21:48.009 INFO:tasks.workunit.client.1.smithi123.stdout:8/203: write d0/d17/d43/f28 [236934,128773] 0 2024-02-16T22:21:48.010 INFO:tasks.workunit.client.1.smithi123.stdout:9/237: truncate d4/d17/f1d 1665701 0 2024-02-16T22:21:48.011 INFO:tasks.workunit.client.1.smithi123.stdout:8/204: write d0/d17/d26/f2d [1590091,36677] 0 2024-02-16T22:21:48.016 INFO:tasks.workunit.client.1.smithi123.stdout:3/141: mknod d0/d20/d29/c2b 0 2024-02-16T22:21:48.018 INFO:tasks.workunit.client.1.smithi123.stdout:9/238: dwrite d4/d33/f38 [0,4194304] 0 2024-02-16T22:21:48.020 INFO:tasks.workunit.client.1.smithi123.stdout:1/140: unlink d3/fc 0 2024-02-16T22:21:48.021 INFO:tasks.workunit.client.0.smithi114.stdout:2/125: fdatasync f0 0 2024-02-16T22:21:48.023 INFO:tasks.workunit.client.0.smithi114.stdout:8/126: write d4/d11/d15/f10 [181273,109789] 0 2024-02-16T22:21:48.024 INFO:tasks.workunit.client.0.smithi114.stdout:8/127: write d4/d11/f20 [42236,47016] 0 2024-02-16T22:21:48.024 INFO:tasks.workunit.client.0.smithi114.stdout:5/89: mknod d0/c17 0 2024-02-16T22:21:48.025 INFO:tasks.workunit.client.0.smithi114.stdout:5/90: dread - d0/d10/f13 zero size 2024-02-16T22:21:48.026 INFO:tasks.workunit.client.0.smithi114.stdout:5/91: dread - d0/d10/f13 zero size 2024-02-16T22:21:48.028 INFO:tasks.workunit.client.0.smithi114.stdout:3/81: readlink l3 0 2024-02-16T22:21:48.029 INFO:tasks.workunit.client.0.smithi114.stdout:3/82: fsync d4/dd/f13 0 2024-02-16T22:21:48.031 INFO:tasks.workunit.client.0.smithi114.stdout:8/128: dread d4/d11/f17 [0,4194304] 0 2024-02-16T22:21:48.032 INFO:tasks.workunit.client.0.smithi114.stdout:0/139: fsync d1/d16/f19 0 2024-02-16T22:21:48.034 INFO:tasks.workunit.client.0.smithi114.stdout:6/70: creat d6/dd/f17 x:0 0 0 2024-02-16T22:21:48.037 INFO:tasks.workunit.client.1.smithi123.stdout:7/161: mknod d3/d26/c37 0 2024-02-16T22:21:48.040 INFO:tasks.workunit.client.0.smithi114.stdout:4/166: rename dd/d1b/f1c to dd/d1b/d24/f36 0 2024-02-16T22:21:48.044 INFO:tasks.workunit.client.1.smithi123.stdout:6/171: link d0/d6/l10 d0/d9/dc/l3d 0 2024-02-16T22:21:48.046 INFO:tasks.workunit.client.1.smithi123.stdout:2/176: truncate d4/d1a/d1d/d13/d15/d1b/f28 2395960 0 2024-02-16T22:21:48.047 INFO:tasks.workunit.client.0.smithi114.stdout:2/126: symlink dd/df/d1c/d21/l2a 0 2024-02-16T22:21:48.049 INFO:tasks.workunit.client.1.smithi123.stdout:3/142: creat d0/d20/f2c x:0 0 0 2024-02-16T22:21:48.050 INFO:tasks.workunit.client.0.smithi114.stdout:1/88: dwrite d4/d9/f11 [0,4194304] 0 2024-02-16T22:21:48.051 INFO:tasks.workunit.client.0.smithi114.stdout:5/92: mknod d0/d10/c18 0 2024-02-16T22:21:48.052 INFO:tasks.workunit.client.1.smithi123.stdout:1/141: mknod d3/d6/d11/c34 0 2024-02-16T22:21:48.053 INFO:tasks.workunit.client.1.smithi123.stdout:1/142: truncate d3/d6/d11/f1f 1171693 0 2024-02-16T22:21:48.053 INFO:tasks.workunit.client.0.smithi114.stdout:3/83: rename d4/fa to d4/f1c 0 2024-02-16T22:21:48.054 INFO:tasks.workunit.client.0.smithi114.stdout:8/129: mknod d4/c21 0 2024-02-16T22:21:48.054 INFO:tasks.workunit.client.0.smithi114.stdout:8/130: chown d4/f1d 1 1 2024-02-16T22:21:48.059 INFO:tasks.workunit.client.1.smithi123.stdout:8/205: creat d0/d12/d3e/f49 x:0 0 0 2024-02-16T22:21:48.060 INFO:tasks.workunit.client.1.smithi123.stdout:8/206: dread - d0/d17/d43/f3d zero size 2024-02-16T22:21:48.061 INFO:tasks.workunit.client.0.smithi114.stdout:6/71: dwrite d6/fc [0,4194304] 0 2024-02-16T22:21:48.062 INFO:tasks.workunit.client.0.smithi114.stdout:6/72: write d6/dd/f17 [843213,29172] 0 2024-02-16T22:21:48.063 INFO:tasks.workunit.client.0.smithi114.stdout:9/124: creat d2/dd/f2a x:0 0 0 2024-02-16T22:21:48.063 INFO:tasks.workunit.client.0.smithi114.stdout:9/125: chown d2/d1a 7 1 2024-02-16T22:21:48.065 INFO:tasks.workunit.client.1.smithi123.stdout:6/172: creat d0/d29/f3e x:0 0 0 2024-02-16T22:21:48.066 INFO:tasks.workunit.client.1.smithi123.stdout:6/173: read d0/d9/dc/f23 [984855,85219] 0 2024-02-16T22:21:48.067 INFO:tasks.workunit.client.1.smithi123.stdout:2/177: creat d4/d1a/d1d/d13/f3c x:0 0 0 2024-02-16T22:21:48.068 INFO:tasks.workunit.client.1.smithi123.stdout:2/178: fdatasync d4/f16 0 2024-02-16T22:21:48.069 INFO:tasks.workunit.client.1.smithi123.stdout:2/179: write d4/d1a/d1d/d13/f36 [866675,53404] 0 2024-02-16T22:21:48.070 INFO:tasks.workunit.client.1.smithi123.stdout:3/143: fsync d0/f7 0 2024-02-16T22:21:48.073 INFO:tasks.workunit.client.0.smithi114.stdout:2/127: symlink dd/df/d1c/d21/l2b 0 2024-02-16T22:21:48.077 INFO:tasks.workunit.client.1.smithi123.stdout:4/186: sync 2024-02-16T22:21:48.078 INFO:tasks.workunit.client.1.smithi123.stdout:4/187: write d5/f1f [1870003,132] 0 2024-02-16T22:21:48.079 INFO:tasks.workunit.client.1.smithi123.stdout:5/133: rmdir d6/dd/df/d1f/d27 0 2024-02-16T22:21:48.080 INFO:tasks.workunit.client.1.smithi123.stdout:7/162: creat d3/d12/d34/f38 x:0 0 0 2024-02-16T22:21:48.081 INFO:tasks.workunit.client.1.smithi123.stdout:7/163: chown d3/d12/f1b 75025 1 2024-02-16T22:21:48.083 INFO:tasks.workunit.client.0.smithi114.stdout:3/84: symlink d4/l1d 0 2024-02-16T22:21:48.083 INFO:tasks.workunit.client.0.smithi114.stdout:7/96: dwrite f2 [0,4194304] 0 2024-02-16T22:21:48.087 INFO:tasks.workunit.client.1.smithi123.stdout:6/174: creat d0/d9/dc/f3f x:0 0 0 2024-02-16T22:21:48.094 INFO:tasks.workunit.client.1.smithi123.stdout:9/239: dwrite d4/f21 [0,4194304] 0 2024-02-16T22:21:48.095 INFO:tasks.workunit.client.1.smithi123.stdout:2/180: rename d4/d1a/d1d/d13/c22 to d4/d1a/d1d/d13/d15/c3d 0 2024-02-16T22:21:48.096 INFO:tasks.workunit.client.1.smithi123.stdout:3/144: read d0/fe [186111,42854] 0 2024-02-16T22:21:48.097 INFO:tasks.workunit.client.0.smithi114.stdout:9/126: symlink d2/l2b 0 2024-02-16T22:21:48.097 INFO:tasks.workunit.client.0.smithi114.stdout:9/127: readlink d2/l9 0 2024-02-16T22:21:48.098 INFO:tasks.workunit.client.0.smithi114.stdout:9/128: fdatasync d2/dd/d1c/f1e 0 2024-02-16T22:21:48.099 INFO:tasks.workunit.client.1.smithi123.stdout:0/171: getdents d1/d4 0 2024-02-16T22:21:48.100 INFO:tasks.workunit.client.0.smithi114.stdout:1/89: truncate d4/fa 362316 0 2024-02-16T22:21:48.101 INFO:tasks.workunit.client.1.smithi123.stdout:3/145: dwrite d0/f7 [0,4194304] 0 2024-02-16T22:21:48.102 INFO:tasks.workunit.client.0.smithi114.stdout:5/93: mknod d0/df/d16/c19 0 2024-02-16T22:21:48.104 INFO:tasks.workunit.client.0.smithi114.stdout:9/129: dread d2/f15 [0,4194304] 0 2024-02-16T22:21:48.104 INFO:tasks.workunit.client.1.smithi123.stdout:5/134: symlink d6/dd/df/d1f/l2a 0 2024-02-16T22:21:48.104 INFO:tasks.workunit.client.0.smithi114.stdout:9/130: chown d2/l9 233 1 2024-02-16T22:21:48.106 INFO:tasks.workunit.client.1.smithi123.stdout:7/164: creat d3/d12/d24/f39 x:0 0 0 2024-02-16T22:21:48.106 INFO:tasks.workunit.client.0.smithi114.stdout:3/85: creat d4/dd/f1e x:0 0 0 2024-02-16T22:21:48.107 INFO:tasks.workunit.client.1.smithi123.stdout:8/207: symlink d0/d29/d42/d47/l4a 0 2024-02-16T22:21:48.108 INFO:tasks.workunit.client.0.smithi114.stdout:3/86: write d4/dd/f19 [273020,105429] 0 2024-02-16T22:21:48.109 INFO:tasks.workunit.client.0.smithi114.stdout:3/87: readlink d4/l10 0 2024-02-16T22:21:48.111 INFO:tasks.workunit.client.0.smithi114.stdout:0/140: link d1/lc d1/d1a/l29 0 2024-02-16T22:21:48.113 INFO:tasks.workunit.client.1.smithi123.stdout:6/175: fsync d0/d29/f2d 0 2024-02-16T22:21:48.114 INFO:tasks.workunit.client.0.smithi114.stdout:4/167: rmdir dd/d1b/d24/d35 0 2024-02-16T22:21:48.115 INFO:tasks.workunit.client.0.smithi114.stdout:4/168: stat dd/d1b/d24/f36 0 2024-02-16T22:21:48.116 INFO:tasks.workunit.client.0.smithi114.stdout:4/169: dread - dd/d1b/d24/f2f zero size 2024-02-16T22:21:48.116 INFO:tasks.workunit.client.0.smithi114.stdout:4/170: chown dd/f12 7171738 1 2024-02-16T22:21:48.117 INFO:tasks.workunit.client.1.smithi123.stdout:4/188: rename d5/d1c to d5/d1c/d3b 22 2024-02-16T22:21:48.118 INFO:tasks.workunit.client.0.smithi114.stdout:2/128: unlink dd/df/l24 0 2024-02-16T22:21:48.120 INFO:tasks.workunit.client.1.smithi123.stdout:6/176: dwrite d0/d6/f18 [0,4194304] 0 2024-02-16T22:21:48.120 INFO:tasks.workunit.client.1.smithi123.stdout:1/143: creat d3/f35 x:0 0 0 2024-02-16T22:21:48.121 INFO:tasks.workunit.client.0.smithi114.stdout:9/131: rmdir d2/d1a/d23 39 2024-02-16T22:21:48.123 INFO:tasks.workunit.client.1.smithi123.stdout:7/165: symlink d3/d26/l3a 0 2024-02-16T22:21:48.125 INFO:tasks.workunit.client.1.smithi123.stdout:5/135: rmdir d6 39 2024-02-16T22:21:48.125 INFO:tasks.workunit.client.1.smithi123.stdout:4/189: dwrite d5/f1f [0,4194304] 0 2024-02-16T22:21:48.126 INFO:tasks.workunit.client.0.smithi114.stdout:3/88: symlink d4/l1f 0 2024-02-16T22:21:48.132 INFO:tasks.workunit.client.0.smithi114.stdout:6/73: creat d6/f18 x:0 0 0 2024-02-16T22:21:48.132 INFO:tasks.workunit.client.1.smithi123.stdout:4/190: dwrite d5/f1d [0,4194304] 0 2024-02-16T22:21:48.134 INFO:tasks.workunit.client.0.smithi114.stdout:3/89: dwrite d4/dd/f19 [0,4194304] 0 2024-02-16T22:21:48.134 INFO:tasks.workunit.client.1.smithi123.stdout:4/191: truncate d5/d1c/f24 961768 0 2024-02-16T22:21:48.135 INFO:tasks.workunit.client.1.smithi123.stdout:4/192: truncate d5/f19 1591814 0 2024-02-16T22:21:48.137 INFO:tasks.workunit.client.0.smithi114.stdout:4/171: mkdir dd/d1b/d37 0 2024-02-16T22:21:48.137 INFO:tasks.workunit.client.0.smithi114.stdout:4/172: readlink dd/l28 0 2024-02-16T22:21:48.141 INFO:tasks.workunit.client.0.smithi114.stdout:2/129: mkdir dd/df/d2c 0 2024-02-16T22:21:48.143 INFO:tasks.workunit.client.1.smithi123.stdout:6/177: creat d0/d9/dc/d3a/f40 x:0 0 0 2024-02-16T22:21:48.146 INFO:tasks.workunit.client.1.smithi123.stdout:8/208: mkdir d0/d17/d4b 0 2024-02-16T22:21:48.147 INFO:tasks.workunit.client.1.smithi123.stdout:7/166: rename d3/d26/c37 to d3/d5/c3b 0 2024-02-16T22:21:48.148 INFO:tasks.workunit.client.1.smithi123.stdout:7/167: truncate d3/d12/d1d/f2f 5206594 0 2024-02-16T22:21:48.150 INFO:tasks.workunit.client.1.smithi123.stdout:7/168: write d3/d12/d1d/f2f [3830526,58431] 0 2024-02-16T22:21:48.151 INFO:tasks.workunit.client.0.smithi114.stdout:2/130: dwrite dd/df/f1e [4194304,4194304] 0 2024-02-16T22:21:48.153 INFO:tasks.workunit.client.0.smithi114.stdout:5/94: dwrite d0/f9 [4194304,4194304] 0 2024-02-16T22:21:48.154 INFO:tasks.workunit.client.1.smithi123.stdout:0/172: creat d1/f2c x:0 0 0 2024-02-16T22:21:48.155 INFO:tasks.workunit.client.0.smithi114.stdout:8/131: getdents d4/d11/d15 0 2024-02-16T22:21:48.161 INFO:tasks.workunit.client.0.smithi114.stdout:6/74: unlink d6/l7 0 2024-02-16T22:21:48.162 INFO:tasks.workunit.client.0.smithi114.stdout:6/75: read d6/dd/f17 [673178,29373] 0 2024-02-16T22:21:48.166 INFO:tasks.workunit.client.0.smithi114.stdout:2/131: dread dd/f1d [0,4194304] 0 2024-02-16T22:21:48.166 INFO:tasks.workunit.client.0.smithi114.stdout:3/90: symlink d4/dd/l20 0 2024-02-16T22:21:48.167 INFO:tasks.workunit.client.0.smithi114.stdout:3/91: read - d4/dd/f13 zero size 2024-02-16T22:21:48.174 INFO:tasks.workunit.client.1.smithi123.stdout:5/136: mkdir d6/dd/df/d2b 0 2024-02-16T22:21:48.174 INFO:tasks.workunit.client.1.smithi123.stdout:1/144: write d3/d6/f26 [422874,112411] 0 2024-02-16T22:21:48.178 INFO:tasks.workunit.client.1.smithi123.stdout:0/173: write d1/f6 [37463,124797] 0 2024-02-16T22:21:48.180 INFO:tasks.workunit.client.0.smithi114.stdout:7/97: rename f2 to d3/f1a 0 2024-02-16T22:21:48.182 INFO:tasks.workunit.client.0.smithi114.stdout:7/98: write d3/ff [5421107,37439] 0 2024-02-16T22:21:48.183 INFO:tasks.workunit.client.1.smithi123.stdout:8/209: truncate d0/df/f1b 3568058 0 2024-02-16T22:21:48.185 INFO:tasks.workunit.client.0.smithi114.stdout:0/141: creat d1/f2a x:0 0 0 2024-02-16T22:21:48.185 INFO:tasks.workunit.client.1.smithi123.stdout:2/181: dwrite d4/d1a/d1d/f9 [0,4194304] 0 2024-02-16T22:21:48.186 INFO:tasks.workunit.client.0.smithi114.stdout:0/142: dread - d1/f2a zero size 2024-02-16T22:21:48.186 INFO:tasks.workunit.client.0.smithi114.stdout:9/132: dwrite d2/d1a/f21 [0,4194304] 0 2024-02-16T22:21:48.186 INFO:tasks.workunit.client.1.smithi123.stdout:2/182: chown d4/d1a/d1d/d13/d15/d1b/f28 63 1 2024-02-16T22:21:48.187 INFO:tasks.workunit.client.1.smithi123.stdout:5/137: rename l0 to d6/dd/df/d1f/l2c 0 2024-02-16T22:21:48.190 INFO:tasks.workunit.client.0.smithi114.stdout:9/133: dread d2/dd/f10 [0,4194304] 0 2024-02-16T22:21:48.191 INFO:tasks.workunit.client.0.smithi114.stdout:8/132: symlink d4/l22 0 2024-02-16T22:21:48.196 INFO:tasks.workunit.client.1.smithi123.stdout:4/193: creat d5/d1c/d23/f3c x:0 0 0 2024-02-16T22:21:48.200 INFO:tasks.workunit.client.1.smithi123.stdout:0/174: dwrite d1/d8/d13/f15 [0,4194304] 0 2024-02-16T22:21:48.201 INFO:tasks.workunit.client.0.smithi114.stdout:2/132: symlink dd/df/d1c/l2d 0 2024-02-16T22:21:48.202 INFO:tasks.workunit.client.1.smithi123.stdout:0/175: write d1/f6 [3847734,124820] 0 2024-02-16T22:21:48.202 INFO:tasks.workunit.client.1.smithi123.stdout:8/210: mknod d0/d17/d26/c4c 0 2024-02-16T22:21:48.203 INFO:tasks.workunit.client.1.smithi123.stdout:3/146: sync 2024-02-16T22:21:48.203 INFO:tasks.workunit.client.1.smithi123.stdout:9/240: sync 2024-02-16T22:21:48.206 INFO:tasks.workunit.client.1.smithi123.stdout:4/194: dwrite d5/f1f [0,4194304] 0 2024-02-16T22:21:48.207 INFO:tasks.workunit.client.1.smithi123.stdout:2/183: rename d4/d1a/d1d/cd to d4/d11/c3e 0 2024-02-16T22:21:48.208 INFO:tasks.workunit.client.0.smithi114.stdout:4/173: mkdir dd/d1b/d37/d38 0 2024-02-16T22:21:48.211 INFO:tasks.workunit.client.0.smithi114.stdout:5/95: dwrite d0/d10/f13 [0,4194304] 0 2024-02-16T22:21:48.215 INFO:tasks.workunit.client.1.smithi123.stdout:9/241: dread d4/f12 [0,4194304] 0 2024-02-16T22:21:48.216 INFO:tasks.workunit.client.1.smithi123.stdout:8/211: dwrite d0/d17/d43/f28 [0,4194304] 0 2024-02-16T22:21:48.217 INFO:tasks.workunit.client.0.smithi114.stdout:3/92: write f1 [1934066,78794] 0 2024-02-16T22:21:48.218 INFO:tasks.workunit.client.1.smithi123.stdout:4/195: dwrite d5/f17 [0,4194304] 0 2024-02-16T22:21:48.218 INFO:tasks.workunit.client.0.smithi114.stdout:3/93: write d4/dd/f13 [961704,110551] 0 2024-02-16T22:21:48.219 INFO:tasks.workunit.client.0.smithi114.stdout:7/99: mkdir d3/d14/d1b 0 2024-02-16T22:21:48.219 INFO:tasks.workunit.client.0.smithi114.stdout:3/94: dread - d4/dd/f1e zero size 2024-02-16T22:21:48.221 INFO:tasks.workunit.client.1.smithi123.stdout:6/178: getdents d0/d9/dc 0 2024-02-16T22:21:48.225 INFO:tasks.workunit.client.0.smithi114.stdout:8/133: mknod d4/d16/c23 0 2024-02-16T22:21:48.227 INFO:tasks.workunit.client.1.smithi123.stdout:7/169: sync 2024-02-16T22:21:48.228 INFO:tasks.workunit.client.1.smithi123.stdout:2/184: symlink d4/d1a/d1d/d13/d15/l3f 0 2024-02-16T22:21:48.228 INFO:tasks.workunit.client.0.smithi114.stdout:6/76: link d6/dd/f17 d6/dd/f19 0 2024-02-16T22:21:48.229 INFO:tasks.workunit.client.0.smithi114.stdout:0/143: dwrite d1/d1a/f26 [0,4194304] 0 2024-02-16T22:21:48.230 INFO:tasks.workunit.client.0.smithi114.stdout:6/77: dread - d6/dd/f15 zero size 2024-02-16T22:21:48.231 INFO:tasks.workunit.client.1.smithi123.stdout:4/196: dwrite d5/f1e [0,4194304] 0 2024-02-16T22:21:48.233 INFO:tasks.workunit.client.1.smithi123.stdout:4/197: fdatasync d5/f29 0 2024-02-16T22:21:48.234 INFO:tasks.workunit.client.1.smithi123.stdout:6/179: dwrite d0/d6/d1a/f36 [4194304,4194304] 0 2024-02-16T22:21:48.239 INFO:tasks.workunit.client.1.smithi123.stdout:2/185: dwrite d4/fc [0,4194304] 0 2024-02-16T22:21:48.239 INFO:tasks.workunit.client.0.smithi114.stdout:2/133: creat dd/df/d1c/f2e x:0 0 0 2024-02-16T22:21:48.241 INFO:tasks.workunit.client.1.smithi123.stdout:2/186: readlink d4/la 0 2024-02-16T22:21:48.243 INFO:tasks.workunit.client.1.smithi123.stdout:1/145: truncate d3/d6/f26 770756 0 2024-02-16T22:21:48.243 INFO:tasks.workunit.client.1.smithi123.stdout:9/242: unlink d4/d17/d24/f3d 0 2024-02-16T22:21:48.245 INFO:tasks.workunit.client.0.smithi114.stdout:4/174: symlink dd/d2e/l39 0 2024-02-16T22:21:48.247 INFO:tasks.workunit.client.0.smithi114.stdout:1/90: dread d4/fa [0,4194304] 0 2024-02-16T22:21:48.247 INFO:tasks.workunit.client.1.smithi123.stdout:3/147: mknod d0/c2d 0 2024-02-16T22:21:48.248 INFO:tasks.workunit.client.0.smithi114.stdout:1/91: chown d4/fa 51462 1 2024-02-16T22:21:48.248 INFO:tasks.workunit.client.1.smithi123.stdout:1/146: dread - d3/d6/db/f28 zero size 2024-02-16T22:21:48.249 INFO:tasks.workunit.client.0.smithi114.stdout:5/96: creat d0/d10/f1a x:0 0 0 2024-02-16T22:21:48.251 INFO:tasks.workunit.client.1.smithi123.stdout:1/147: write d3/d6/db/f28 [193532,69955] 0 2024-02-16T22:21:48.251 INFO:tasks.workunit.client.1.smithi123.stdout:6/180: dwrite d0/d6/f18 [0,4194304] 0 2024-02-16T22:21:48.251 INFO:tasks.workunit.client.0.smithi114.stdout:3/95: mkdir d4/d21 0 2024-02-16T22:21:48.252 INFO:tasks.workunit.client.1.smithi123.stdout:1/148: chown d3/l16 1277135 1 2024-02-16T22:21:48.252 INFO:tasks.workunit.client.0.smithi114.stdout:3/96: stat d4/f18 0 2024-02-16T22:21:48.252 INFO:tasks.workunit.client.1.smithi123.stdout:6/181: write d0/d29/f2c [4149702,104189] 0 2024-02-16T22:21:48.254 INFO:tasks.workunit.client.1.smithi123.stdout:1/149: write d3/d6/d11/f23 [287131,77908] 0 2024-02-16T22:21:48.255 INFO:tasks.workunit.client.0.smithi114.stdout:9/134: creat d2/d1a/d23/f2c x:0 0 0 2024-02-16T22:21:48.257 INFO:tasks.workunit.client.0.smithi114.stdout:1/92: dwrite f1 [0,4194304] 0 2024-02-16T22:21:48.259 INFO:tasks.workunit.client.1.smithi123.stdout:6/182: dread d0/d29/f2d [0,4194304] 0 2024-02-16T22:21:48.259 INFO:tasks.workunit.client.0.smithi114.stdout:8/134: dread d4/d11/f17 [0,4194304] 0 2024-02-16T22:21:48.260 INFO:tasks.workunit.client.0.smithi114.stdout:0/144: rmdir d1/d16/d1f 39 2024-02-16T22:21:48.262 INFO:tasks.workunit.client.0.smithi114.stdout:6/78: creat d6/dd/f1a x:0 0 0 2024-02-16T22:21:48.263 INFO:tasks.workunit.client.0.smithi114.stdout:6/79: fsync d6/fc 0 2024-02-16T22:21:48.265 INFO:tasks.workunit.client.0.smithi114.stdout:2/134: creat dd/df/f2f x:0 0 0 2024-02-16T22:21:48.266 INFO:tasks.workunit.client.0.smithi114.stdout:2/135: readlink dd/df/d1c/d21/l2a 0 2024-02-16T22:21:48.267 INFO:tasks.workunit.client.0.smithi114.stdout:4/175: creat dd/d1b/d24/f3a x:0 0 0 2024-02-16T22:21:48.267 INFO:tasks.workunit.client.0.smithi114.stdout:2/136: dread - dd/df/f2f zero size 2024-02-16T22:21:48.268 INFO:tasks.workunit.client.1.smithi123.stdout:5/138: write d6/dd/df/d1f/d25/f17 [359054,26731] 0 2024-02-16T22:21:48.269 INFO:tasks.workunit.client.1.smithi123.stdout:2/187: symlink d4/d1a/d1d/d13/l40 0 2024-02-16T22:21:48.269 INFO:tasks.workunit.client.0.smithi114.stdout:4/176: read dd/d1b/f1d [1089836,81614] 0 2024-02-16T22:21:48.270 INFO:tasks.workunit.client.0.smithi114.stdout:4/177: dread - dd/d1b/d24/f3a zero size 2024-02-16T22:21:48.273 INFO:tasks.workunit.client.0.smithi114.stdout:7/100: link d3/fe d3/d14/f1c 0 2024-02-16T22:21:48.274 INFO:tasks.workunit.client.1.smithi123.stdout:2/188: dwrite f2 [0,4194304] 0 2024-02-16T22:21:48.275 INFO:tasks.workunit.client.1.smithi123.stdout:2/189: fsync d4/f23 0 2024-02-16T22:21:48.275 INFO:tasks.workunit.client.0.smithi114.stdout:9/135: rename d2/c11 to d2/d1a/d23/c2d 0 2024-02-16T22:21:48.276 INFO:tasks.workunit.client.1.smithi123.stdout:2/190: chown d4/d1a/d1d/f17 95 1 2024-02-16T22:21:48.276 INFO:tasks.workunit.client.1.smithi123.stdout:9/243: creat d4/d33/f4c x:0 0 0 2024-02-16T22:21:48.277 INFO:tasks.workunit.client.1.smithi123.stdout:9/244: fsync d4/d33/f38 0 2024-02-16T22:21:48.278 INFO:tasks.workunit.client.0.smithi114.stdout:8/135: mkdir d4/d24 0 2024-02-16T22:21:48.281 INFO:tasks.workunit.client.0.smithi114.stdout:6/80: readlink d6/da/lb 0 2024-02-16T22:21:48.284 INFO:tasks.workunit.client.1.smithi123.stdout:7/170: link d3/fd d3/d12/d34/f3c 0 2024-02-16T22:21:48.284 INFO:tasks.workunit.client.1.smithi123.stdout:2/191: dwrite d4/d1a/f39 [0,4194304] 0 2024-02-16T22:21:48.285 INFO:tasks.workunit.client.1.smithi123.stdout:7/171: fdatasync d3/d12/d1d/f2b 0 2024-02-16T22:21:48.285 INFO:tasks.workunit.client.0.smithi114.stdout:2/137: creat dd/df/d1c/d21/f30 x:0 0 0 2024-02-16T22:21:48.286 INFO:tasks.workunit.client.1.smithi123.stdout:7/172: read d3/f2a [403006,8168] 0 2024-02-16T22:21:48.286 INFO:tasks.workunit.client.1.smithi123.stdout:6/183: creat d0/d9/dc/f41 x:0 0 0 2024-02-16T22:21:48.287 INFO:tasks.workunit.client.1.smithi123.stdout:6/184: dread - d0/d9/f19 zero size 2024-02-16T22:21:48.287 INFO:tasks.workunit.client.0.smithi114.stdout:4/178: mknod dd/d2e/c3b 0 2024-02-16T22:21:48.294 INFO:tasks.workunit.client.0.smithi114.stdout:2/138: dwrite dd/df/f2f [0,4194304] 0 2024-02-16T22:21:48.294 INFO:tasks.workunit.client.1.smithi123.stdout:6/185: dwrite d0/d9/f19 [0,4194304] 0 2024-02-16T22:21:48.295 INFO:tasks.workunit.client.0.smithi114.stdout:9/136: creat d2/dd/d1c/d1f/f2e x:0 0 0 2024-02-16T22:21:48.295 INFO:tasks.workunit.client.0.smithi114.stdout:2/139: stat dd/c12 0 2024-02-16T22:21:48.297 INFO:tasks.workunit.client.1.smithi123.stdout:8/212: rmdir d0/d17/d4b 0 2024-02-16T22:21:48.300 INFO:tasks.workunit.client.0.smithi114.stdout:1/93: unlink d4/cf 0 2024-02-16T22:21:48.305 INFO:tasks.workunit.client.0.smithi114.stdout:6/81: creat d6/da/f1b x:0 0 0 2024-02-16T22:21:48.306 INFO:tasks.workunit.client.0.smithi114.stdout:6/82: write d6/f18 [559835,112474] 0 2024-02-16T22:21:48.306 INFO:tasks.workunit.client.1.smithi123.stdout:0/176: getdents d1 0 2024-02-16T22:21:48.308 INFO:tasks.workunit.client.1.smithi123.stdout:3/148: mkdir d0/d20/d2e 0 2024-02-16T22:21:48.310 INFO:tasks.workunit.client.0.smithi114.stdout:5/97: link d0/c1 d0/df/d16/c1b 0 2024-02-16T22:21:48.310 INFO:tasks.workunit.client.0.smithi114.stdout:6/83: dread d6/dd/f19 [0,4194304] 0 2024-02-16T22:21:48.310 INFO:tasks.workunit.client.0.smithi114.stdout:5/98: fsync d0/fa 0 2024-02-16T22:21:48.311 INFO:tasks.workunit.client.0.smithi114.stdout:6/84: read - d6/dd/f15 zero size 2024-02-16T22:21:48.313 INFO:tasks.workunit.client.0.smithi114.stdout:4/179: mknod dd/d1b/d24/c3c 0 2024-02-16T22:21:48.317 INFO:tasks.workunit.client.0.smithi114.stdout:5/99: dwrite d0/fa [0,4194304] 0 2024-02-16T22:21:48.320 INFO:tasks.workunit.client.0.smithi114.stdout:2/140: unlink dd/df/d1c/d21/l2b 0 2024-02-16T22:21:48.322 INFO:tasks.workunit.client.0.smithi114.stdout:2/141: write dd/df/d1c/d21/f26 [217468,29189] 0 2024-02-16T22:21:48.322 INFO:tasks.workunit.client.1.smithi123.stdout:2/192: unlink d4/d1a/d1d/d13/d15/l20 0 2024-02-16T22:21:48.329 INFO:tasks.workunit.client.0.smithi114.stdout:1/94: dwrite d4/f15 [0,4194304] 0 2024-02-16T22:21:48.330 INFO:tasks.workunit.client.1.smithi123.stdout:4/198: creat d5/d1c/f3d x:0 0 0 2024-02-16T22:21:48.336 INFO:tasks.workunit.client.0.smithi114.stdout:6/85: mkdir d6/da/d1c 0 2024-02-16T22:21:48.336 INFO:tasks.workunit.client.0.smithi114.stdout:6/86: readlink d6/l8 0 2024-02-16T22:21:48.339 INFO:tasks.workunit.client.1.smithi123.stdout:6/186: rename d0/d6/l10 to d0/d6/d2f/l42 0 2024-02-16T22:21:48.340 INFO:tasks.workunit.client.0.smithi114.stdout:4/180: creat dd/d1b/d24/f3d x:0 0 0 2024-02-16T22:21:48.341 INFO:tasks.workunit.client.1.smithi123.stdout:6/187: read d0/d6/fd [1889875,85858] 0 2024-02-16T22:21:48.342 INFO:tasks.workunit.client.1.smithi123.stdout:8/213: mkdir d0/d17/d26/d2a/d4d 0 2024-02-16T22:21:48.344 INFO:tasks.workunit.client.0.smithi114.stdout:0/145: rename d1/l13 to d1/d16/l2b 0 2024-02-16T22:21:48.345 INFO:tasks.workunit.client.1.smithi123.stdout:0/177: readlink d1/d8/l16 0 2024-02-16T22:21:48.346 INFO:tasks.workunit.client.0.smithi114.stdout:5/100: mknod d0/c1c 0 2024-02-16T22:21:48.347 INFO:tasks.workunit.client.0.smithi114.stdout:5/101: dread - d0/d10/f1a zero size 2024-02-16T22:21:48.347 INFO:tasks.workunit.client.0.smithi114.stdout:5/102: fdatasync d0/f8 0 2024-02-16T22:21:48.348 INFO:tasks.workunit.client.0.smithi114.stdout:5/103: readlink d0/l6 0 2024-02-16T22:21:48.349 INFO:tasks.workunit.client.0.smithi114.stdout:5/104: write d0/d10/f1a [1017628,62557] 0 2024-02-16T22:21:48.355 INFO:tasks.workunit.client.0.smithi114.stdout:1/95: readlink d4/d9/l1a 0 2024-02-16T22:21:48.356 INFO:tasks.workunit.client.0.smithi114.stdout:5/105: dwrite d0/f8 [0,4194304] 0 2024-02-16T22:21:48.357 INFO:tasks.workunit.client.0.smithi114.stdout:5/106: truncate d0/fb 2018483 0 2024-02-16T22:21:48.357 INFO:tasks.workunit.client.0.smithi114.stdout:5/107: read d0/f8 [528431,49022] 0 2024-02-16T22:21:48.359 INFO:tasks.workunit.client.1.smithi123.stdout:4/199: creat d5/d1c/d23/d31/f3e x:0 0 0 2024-02-16T22:21:48.359 INFO:tasks.workunit.client.1.smithi123.stdout:1/150: truncate d3/d6/d11/f23 593295 0 2024-02-16T22:21:48.360 INFO:tasks.workunit.client.0.smithi114.stdout:6/87: unlink d6/fc 0 2024-02-16T22:21:48.360 INFO:tasks.workunit.client.1.smithi123.stdout:1/151: write d3/f5 [4472693,89674] 0 2024-02-16T22:21:48.360 INFO:tasks.workunit.client.1.smithi123.stdout:9/245: dwrite d4/f6 [0,4194304] 0 2024-02-16T22:21:48.365 INFO:tasks.workunit.client.0.smithi114.stdout:2/142: rmdir dd/df/d1c 39 2024-02-16T22:21:48.366 INFO:tasks.workunit.client.0.smithi114.stdout:0/146: creat d1/d1a/f2c x:0 0 0 2024-02-16T22:21:48.369 INFO:tasks.workunit.client.1.smithi123.stdout:8/214: creat d0/d17/d43/d34/f4e x:0 0 0 2024-02-16T22:21:48.369 INFO:tasks.workunit.client.1.smithi123.stdout:8/215: fdatasync d0/d17/d26/f2d 0 2024-02-16T22:21:48.371 INFO:tasks.workunit.client.1.smithi123.stdout:0/178: symlink d1/l2d 0 2024-02-16T22:21:48.372 INFO:tasks.workunit.client.0.smithi114.stdout:8/136: dwrite f0 [0,4194304] 0 2024-02-16T22:21:48.374 INFO:tasks.workunit.client.0.smithi114.stdout:9/137: creat d2/dd/f2f x:0 0 0 2024-02-16T22:21:48.375 INFO:tasks.workunit.client.1.smithi123.stdout:3/149: creat d0/d20/d2e/f2f x:0 0 0 2024-02-16T22:21:48.376 INFO:tasks.workunit.client.0.smithi114.stdout:9/138: write d2/f12 [1107428,73446] 0 2024-02-16T22:21:48.377 INFO:tasks.workunit.client.0.smithi114.stdout:9/139: chown d2/dd/d1c/l25 13104 1 2024-02-16T22:21:48.380 INFO:tasks.workunit.client.0.smithi114.stdout:5/108: rmdir d0/df 39 2024-02-16T22:21:48.387 INFO:tasks.workunit.client.1.smithi123.stdout:4/200: rmdir d5/d1c/d23 39 2024-02-16T22:21:48.389 INFO:tasks.workunit.client.0.smithi114.stdout:4/181: link dd/f2b dd/d1b/f3e 0 2024-02-16T22:21:48.391 INFO:tasks.workunit.client.1.smithi123.stdout:1/152: unlink d3/ff 0 2024-02-16T22:21:48.393 INFO:tasks.workunit.client.1.smithi123.stdout:9/246: mknod d4/d1e/d3a/c4d 0 2024-02-16T22:21:48.395 INFO:tasks.workunit.client.1.smithi123.stdout:5/139: link d6/dd/df/d1f/l2c d6/dd/df/d2b/l2d 0 2024-02-16T22:21:48.397 INFO:tasks.workunit.client.0.smithi114.stdout:8/137: mkdir d4/d16/d25 0 2024-02-16T22:21:48.400 INFO:tasks.workunit.client.1.smithi123.stdout:6/188: symlink d0/d9/dc/d39/l43 0 2024-02-16T22:21:48.401 INFO:tasks.workunit.client.0.smithi114.stdout:1/96: mkdir d4/d1c 0 2024-02-16T22:21:48.401 INFO:tasks.workunit.client.1.smithi123.stdout:6/189: write d0/d9/dc/f41 [928560,97005] 0 2024-02-16T22:21:48.403 INFO:tasks.workunit.client.0.smithi114.stdout:9/140: symlink d2/l30 0 2024-02-16T22:21:48.407 INFO:tasks.workunit.client.1.smithi123.stdout:3/150: mknod d0/d20/d29/c30 0 2024-02-16T22:21:48.408 INFO:tasks.workunit.client.1.smithi123.stdout:2/193: symlink d4/d1a/d29/l41 0 2024-02-16T22:21:48.409 INFO:tasks.workunit.client.1.smithi123.stdout:7/173: getdents d3/d12/d24 0 2024-02-16T22:21:48.410 INFO:tasks.workunit.client.0.smithi114.stdout:4/182: mkdir dd/d2e/d3f 0 2024-02-16T22:21:48.411 INFO:tasks.workunit.client.0.smithi114.stdout:5/109: dwrite d0/f8 [0,4194304] 0 2024-02-16T22:21:48.412 INFO:tasks.workunit.client.0.smithi114.stdout:7/101: rename d3/d14/l17 to d3/l1d 0 2024-02-16T22:21:48.414 INFO:tasks.workunit.client.1.smithi123.stdout:9/247: mknod d4/d1e/d3a/c4e 0 2024-02-16T22:21:48.415 INFO:tasks.workunit.client.1.smithi123.stdout:5/140: creat d6/dd/df/d1f/f2e x:0 0 0 2024-02-16T22:21:48.417 INFO:tasks.workunit.client.0.smithi114.stdout:9/141: creat d2/dd/d1c/f31 x:0 0 0 2024-02-16T22:21:48.418 INFO:tasks.workunit.client.1.smithi123.stdout:8/216: mknod d0/d17/d26/d32/d45/c4f 0 2024-02-16T22:21:48.419 INFO:tasks.workunit.client.0.smithi114.stdout:6/88: rmdir d6/da/d1c 0 2024-02-16T22:21:48.419 INFO:tasks.workunit.client.1.smithi123.stdout:5/141: dread d6/dd/df/d1f/d25/f1d [0,4194304] 0 2024-02-16T22:21:48.421 INFO:tasks.workunit.client.0.smithi114.stdout:4/183: mknod dd/d1b/c40 0 2024-02-16T22:21:48.422 INFO:tasks.workunit.client.1.smithi123.stdout:2/194: creat d4/d1a/d1d/d13/d15/f42 x:0 0 0 2024-02-16T22:21:48.423 INFO:tasks.workunit.client.1.smithi123.stdout:8/217: dwrite d0/f14 [4194304,4194304] 0 2024-02-16T22:21:48.424 INFO:tasks.workunit.client.1.smithi123.stdout:8/218: chown d0/d17/d26/d32 63 1 2024-02-16T22:21:48.426 INFO:tasks.workunit.client.1.smithi123.stdout:8/219: truncate d0/d17/d26/f2d 5091323 0 2024-02-16T22:21:48.427 INFO:tasks.workunit.client.0.smithi114.stdout:7/102: fdatasync d3/f6 0 2024-02-16T22:21:48.428 INFO:tasks.workunit.client.0.smithi114.stdout:7/103: stat d3/d14 0 2024-02-16T22:21:48.429 INFO:tasks.workunit.client.1.smithi123.stdout:4/201: symlink d5/d1c/l3f 0 2024-02-16T22:21:48.429 INFO:tasks.workunit.client.1.smithi123.stdout:4/202: fdatasync d5/f1f 0 2024-02-16T22:21:48.430 INFO:tasks.workunit.client.0.smithi114.stdout:2/143: rename dd/l23 to dd/df/d2c/l31 0 2024-02-16T22:21:48.433 INFO:tasks.workunit.client.0.smithi114.stdout:0/147: link d1/d1a/f2c d1/d16/d1f/d28/f2d 0 2024-02-16T22:21:48.434 INFO:tasks.workunit.client.1.smithi123.stdout:9/248: symlink d4/d17/d20/l4f 0 2024-02-16T22:21:48.436 INFO:tasks.workunit.client.0.smithi114.stdout:8/138: mknod d4/d24/c26 0 2024-02-16T22:21:48.439 INFO:tasks.workunit.client.0.smithi114.stdout:9/142: readlink d2/la 0 2024-02-16T22:21:48.439 INFO:tasks.workunit.client.1.smithi123.stdout:9/249: dwrite d4/d17/d20/f45 [0,4194304] 0 2024-02-16T22:21:48.441 INFO:tasks.workunit.client.0.smithi114.stdout:0/148: dwrite d1/d16/f17 [0,4194304] 0 2024-02-16T22:21:48.441 INFO:tasks.workunit.client.1.smithi123.stdout:9/250: chown d4/d17/d24/f4a 114170 1 2024-02-16T22:21:48.447 INFO:tasks.workunit.client.0.smithi114.stdout:4/184: mkdir dd/d1b/d41 0 2024-02-16T22:21:48.449 INFO:tasks.workunit.client.1.smithi123.stdout:5/142: dwrite d6/f21 [0,4194304] 0 2024-02-16T22:21:48.450 INFO:tasks.workunit.client.0.smithi114.stdout:9/143: dwrite d2/dd/d1c/d1f/f2e [0,4194304] 0 2024-02-16T22:21:48.450 INFO:tasks.workunit.client.0.smithi114.stdout:5/110: creat d0/df/f1d x:0 0 0 2024-02-16T22:21:48.454 INFO:tasks.workunit.client.0.smithi114.stdout:9/144: write f1 [2819017,8972] 0 2024-02-16T22:21:48.454 INFO:tasks.workunit.client.1.smithi123.stdout:6/190: dwrite d0/d6/d2f/f22 [0,4194304] 0 2024-02-16T22:21:48.456 INFO:tasks.workunit.client.0.smithi114.stdout:0/149: dwrite d1/d1a/f1d [0,4194304] 0 2024-02-16T22:21:48.456 INFO:tasks.workunit.client.0.smithi114.stdout:9/145: read - d2/dd/f2f zero size 2024-02-16T22:21:48.458 INFO:tasks.workunit.client.0.smithi114.stdout:0/150: chown d1/fa 200100 1 2024-02-16T22:21:48.458 INFO:tasks.workunit.client.0.smithi114.stdout:7/104: readlink d3/l7 0 2024-02-16T22:21:48.459 INFO:tasks.workunit.client.0.smithi114.stdout:4/185: dread dd/f19 [0,4194304] 0 2024-02-16T22:21:48.462 INFO:tasks.workunit.client.1.smithi123.stdout:8/220: creat d0/d12/f50 x:0 0 0 2024-02-16T22:21:48.463 INFO:tasks.workunit.client.0.smithi114.stdout:2/144: truncate dd/f1d 559006 0 2024-02-16T22:21:48.464 INFO:tasks.workunit.client.1.smithi123.stdout:8/221: chown d0/d12/d3e 206537708 1 2024-02-16T22:21:48.464 INFO:tasks.workunit.client.0.smithi114.stdout:2/145: chown dd/c12 5728 1 2024-02-16T22:21:48.464 INFO:tasks.workunit.client.0.smithi114.stdout:2/146: readlink l2 0 2024-02-16T22:21:48.467 INFO:tasks.workunit.client.0.smithi114.stdout:8/139: symlink d4/d24/l27 0 2024-02-16T22:21:48.469 INFO:tasks.workunit.client.1.smithi123.stdout:7/174: dwrite d3/f8 [0,4194304] 0 2024-02-16T22:21:48.470 INFO:tasks.workunit.client.1.smithi123.stdout:1/153: link d3/l16 d3/d6/l36 0 2024-02-16T22:21:48.471 INFO:tasks.workunit.client.0.smithi114.stdout:1/97: rename d4/f1b to d4/f1d 0 2024-02-16T22:21:48.475 INFO:tasks.workunit.client.0.smithi114.stdout:6/89: write d6/dd/f17 [1379550,15332] 0 2024-02-16T22:21:48.476 INFO:tasks.workunit.client.1.smithi123.stdout:8/222: dwrite d0/d17/d43/d34/f4e [0,4194304] 0 2024-02-16T22:21:48.476 INFO:tasks.workunit.client.0.smithi114.stdout:1/98: dread d4/d9/f11 [0,4194304] 0 2024-02-16T22:21:48.479 INFO:tasks.workunit.client.1.smithi123.stdout:0/179: creat d1/f2e x:0 0 0 2024-02-16T22:21:48.479 INFO:tasks.workunit.client.1.smithi123.stdout:8/223: chown d0/d17/d26/d2a/d3c/l3b 67145041 1 2024-02-16T22:21:48.480 INFO:tasks.workunit.client.0.smithi114.stdout:5/111: rmdir d0/d10 39 2024-02-16T22:21:48.481 INFO:tasks.workunit.client.0.smithi114.stdout:5/112: chown d0/c4 2 1 2024-02-16T22:21:48.482 INFO:tasks.workunit.client.1.smithi123.stdout:1/154: dwrite d3/d6/d11/f17 [0,4194304] 0 2024-02-16T22:21:48.483 INFO:tasks.workunit.client.1.smithi123.stdout:6/191: rename d0/d9/f21 to d0/d9/dc/d39/f44 0 2024-02-16T22:21:48.484 INFO:tasks.workunit.client.1.smithi123.stdout:4/203: fdatasync d5/d1c/d23/f3a 0 2024-02-16T22:21:48.484 INFO:tasks.workunit.client.1.smithi123.stdout:1/155: read d3/f7 [3168536,63954] 0 2024-02-16T22:21:48.488 INFO:tasks.workunit.client.1.smithi123.stdout:7/175: mkdir d3/d26/d3d 0 2024-02-16T22:21:48.489 INFO:tasks.workunit.client.0.smithi114.stdout:4/186: creat dd/d1b/d24/f42 x:0 0 0 2024-02-16T22:21:48.492 INFO:tasks.workunit.client.1.smithi123.stdout:1/156: dwrite d3/d6/d11/f1f [0,4194304] 0 2024-02-16T22:21:48.492 INFO:tasks.workunit.client.1.smithi123.stdout:9/251: link d4/d17/d24/f4a d4/d1e/d3a/f50 0 2024-02-16T22:21:48.494 INFO:tasks.workunit.client.1.smithi123.stdout:1/157: chown d3/d6/db/d10/d2a 656 1 2024-02-16T22:21:48.494 INFO:tasks.workunit.client.1.smithi123.stdout:5/143: link d6/dd/df/d1f/f2e d6/dd/df/d1f/d25/f2f 0 2024-02-16T22:21:48.495 INFO:tasks.workunit.client.1.smithi123.stdout:1/158: chown d3/c2b 537 1 2024-02-16T22:21:48.496 INFO:tasks.workunit.client.0.smithi114.stdout:4/187: dwrite dd/f11 [0,4194304] 0 2024-02-16T22:21:48.497 INFO:tasks.workunit.client.0.smithi114.stdout:6/90: unlink d6/da/l11 0 2024-02-16T22:21:48.497 INFO:tasks.workunit.client.1.smithi123.stdout:8/224: stat d0/df/f16 0 2024-02-16T22:21:48.498 INFO:tasks.workunit.client.1.smithi123.stdout:1/159: chown d3/d6/db/d10/d2a 595 1 2024-02-16T22:21:48.498 INFO:tasks.workunit.client.0.smithi114.stdout:6/91: dread - d6/da/f1b zero size 2024-02-16T22:21:48.499 INFO:tasks.workunit.client.1.smithi123.stdout:8/225: chown d0/d12/f22 106690811 1 2024-02-16T22:21:48.501 INFO:tasks.workunit.client.1.smithi123.stdout:2/195: truncate d4/d1a/d29/f3a 760805 0 2024-02-16T22:21:48.502 INFO:tasks.workunit.client.1.smithi123.stdout:9/252: dwrite d4/f26 [0,4194304] 0 2024-02-16T22:21:48.503 INFO:tasks.workunit.client.1.smithi123.stdout:6/192: mknod d0/d6/c45 0 2024-02-16T22:21:48.504 INFO:tasks.workunit.client.0.smithi114.stdout:9/146: getdents d2/dd/d1c/d1d 0 2024-02-16T22:21:48.504 INFO:tasks.workunit.client.1.smithi123.stdout:8/226: truncate d0/d17/d26/f44 121697 0 2024-02-16T22:21:48.504 INFO:tasks.workunit.client.1.smithi123.stdout:3/151: sync 2024-02-16T22:21:48.505 INFO:tasks.workunit.client.1.smithi123.stdout:9/253: write d4/d17/f1d [3221481,103544] 0 2024-02-16T22:21:48.505 INFO:tasks.workunit.client.1.smithi123.stdout:5/144: dread d6/dd/df/d1f/d25/f1a [0,4194304] 0 2024-02-16T22:21:48.506 INFO:tasks.workunit.client.0.smithi114.stdout:9/147: readlink d2/l2b 0 2024-02-16T22:21:48.506 INFO:tasks.workunit.client.1.smithi123.stdout:4/204: truncate d5/d1c/d23/f2a 1320821 0 2024-02-16T22:21:48.507 INFO:tasks.workunit.client.0.smithi114.stdout:1/99: dwrite d4/f13 [0,4194304] 0 2024-02-16T22:21:48.508 INFO:tasks.workunit.client.1.smithi123.stdout:7/176: mkdir d3/d26/d3e 0 2024-02-16T22:21:48.508 INFO:tasks.workunit.client.1.smithi123.stdout:3/152: chown d0/d20/d2e 93008 1 2024-02-16T22:21:48.513 INFO:tasks.workunit.client.1.smithi123.stdout:1/160: creat d3/d6/d18/f37 x:0 0 0 2024-02-16T22:21:48.514 INFO:tasks.workunit.client.0.smithi114.stdout:5/113: dwrite d0/d10/f1a [0,4194304] 0 2024-02-16T22:21:48.516 INFO:tasks.workunit.client.0.smithi114.stdout:5/114: write d0/fb [2457550,72940] 0 2024-02-16T22:21:48.517 INFO:tasks.workunit.client.1.smithi123.stdout:8/227: dread d0/d12/f1e [0,4194304] 0 2024-02-16T22:21:48.518 INFO:tasks.workunit.client.1.smithi123.stdout:1/161: dread d3/d6/d11/f1f [0,4194304] 0 2024-02-16T22:21:48.520 INFO:tasks.workunit.client.1.smithi123.stdout:1/162: truncate d3/fe 1585351 0 2024-02-16T22:21:48.523 INFO:tasks.workunit.client.0.smithi114.stdout:6/92: rename d6/dd/f15 to d6/dd/f1d 0 2024-02-16T22:21:48.527 INFO:tasks.workunit.client.1.smithi123.stdout:9/254: fdatasync d4/d17/f23 0 2024-02-16T22:21:48.528 INFO:tasks.workunit.client.0.smithi114.stdout:0/151: link d1/d16/d1f/d28/f2d d1/d1a/f2e 0 2024-02-16T22:21:48.528 INFO:tasks.workunit.client.1.smithi123.stdout:9/255: fsync d4/d33/f4c 0 2024-02-16T22:21:48.532 INFO:tasks.workunit.client.0.smithi114.stdout:9/148: mkdir d2/d1a/d23/d32 0 2024-02-16T22:21:48.534 INFO:tasks.workunit.client.1.smithi123.stdout:7/177: read f2 [623669,63913] 0 2024-02-16T22:21:48.538 INFO:tasks.workunit.client.1.smithi123.stdout:5/145: dwrite d6/dd/df/d1f/d25/f1a [0,4194304] 0 2024-02-16T22:21:48.539 INFO:tasks.workunit.client.0.smithi114.stdout:9/149: dwrite d2/d1a/d23/f2c [0,4194304] 0 2024-02-16T22:21:48.540 INFO:tasks.workunit.client.1.smithi123.stdout:5/146: fdatasync d6/dd/df/f22 0 2024-02-16T22:21:48.540 INFO:tasks.workunit.client.0.smithi114.stdout:4/188: rmdir dd/d1b/d24 39 2024-02-16T22:21:48.542 INFO:tasks.workunit.client.1.smithi123.stdout:5/147: write d6/dd/df/f28 [430044,98558] 0 2024-02-16T22:21:48.542 INFO:tasks.workunit.client.0.smithi114.stdout:5/115: mkdir d0/df/d16/d1e 0 2024-02-16T22:21:48.549 INFO:tasks.workunit.client.1.smithi123.stdout:1/163: unlink d3/d6/db/d10/l2f 0 2024-02-16T22:21:48.552 INFO:tasks.workunit.client.1.smithi123.stdout:3/153: dread d0/f6 [0,4194304] 0 2024-02-16T22:21:48.553 INFO:tasks.workunit.client.1.smithi123.stdout:3/154: truncate d0/d20/f26 215360 0 2024-02-16T22:21:48.557 INFO:tasks.workunit.client.0.smithi114.stdout:8/140: getdents d4/d24 0 2024-02-16T22:21:48.558 INFO:tasks.workunit.client.0.smithi114.stdout:8/141: write d4/f19 [1943918,12077] 0 2024-02-16T22:21:48.559 INFO:tasks.workunit.client.1.smithi123.stdout:6/193: dread d0/d29/f2c [0,4194304] 0 2024-02-16T22:21:48.560 INFO:tasks.workunit.client.1.smithi123.stdout:5/148: dread d6/dd/df/f28 [0,4194304] 0 2024-02-16T22:21:48.560 INFO:tasks.workunit.client.1.smithi123.stdout:4/205: link d5/d1c/d23/d31/f3e d5/f40 0 2024-02-16T22:21:48.562 INFO:tasks.workunit.client.0.smithi114.stdout:9/150: mkdir d2/d33 0 2024-02-16T22:21:48.563 INFO:tasks.workunit.client.0.smithi114.stdout:9/151: dread - d2/dd/d1c/f1e zero size 2024-02-16T22:21:48.564 INFO:tasks.workunit.client.0.smithi114.stdout:7/105: dwrite d3/d14/f18 [0,4194304] 0 2024-02-16T22:21:48.564 INFO:tasks.workunit.client.1.smithi123.stdout:7/178: unlink d3/d12/d1d/c25 0 2024-02-16T22:21:48.564 INFO:tasks.workunit.client.1.smithi123.stdout:6/194: dread d0/d6/d1a/f36 [0,4194304] 0 2024-02-16T22:21:48.566 INFO:tasks.workunit.client.1.smithi123.stdout:6/195: dread d0/f30 [0,4194304] 0 2024-02-16T22:21:48.572 INFO:tasks.workunit.client.1.smithi123.stdout:8/228: unlink d0/d17/d43/d34/f4e 0 2024-02-16T22:21:48.575 INFO:tasks.workunit.client.1.smithi123.stdout:2/196: truncate d4/f37 3187167 0 2024-02-16T22:21:48.576 INFO:tasks.workunit.client.1.smithi123.stdout:1/164: rmdir d3/d6/db/d10 39 2024-02-16T22:21:48.578 INFO:tasks.workunit.client.1.smithi123.stdout:5/149: rmdir d6 39 2024-02-16T22:21:48.582 INFO:tasks.workunit.client.1.smithi123.stdout:2/197: dwrite d4/d1a/d1d/f2b [0,4194304] 0 2024-02-16T22:21:48.584 INFO:tasks.workunit.client.1.smithi123.stdout:8/229: symlink d0/d17/d26/d2a/d3c/l51 0 2024-02-16T22:21:48.587 INFO:tasks.workunit.client.1.smithi123.stdout:1/165: dwrite d3/d6/d18/f37 [0,4194304] 0 2024-02-16T22:21:48.590 INFO:tasks.workunit.client.1.smithi123.stdout:9/256: truncate d4/f6 3706484 0 2024-02-16T22:21:48.592 INFO:tasks.workunit.client.1.smithi123.stdout:7/179: truncate d3/fd 606667 0 2024-02-16T22:21:48.593 INFO:tasks.workunit.client.1.smithi123.stdout:7/180: readlink d3/l6 0 2024-02-16T22:21:48.593 INFO:tasks.workunit.client.1.smithi123.stdout:7/181: stat d3/d5/c11 0 2024-02-16T22:21:48.594 INFO:tasks.workunit.client.1.smithi123.stdout:6/196: mkdir d0/d9/d46 0 2024-02-16T22:21:48.595 INFO:tasks.workunit.client.1.smithi123.stdout:6/197: read d0/d6/f1e [1684450,96852] 0 2024-02-16T22:21:48.598 INFO:tasks.workunit.client.1.smithi123.stdout:3/155: creat d0/f31 x:0 0 0 2024-02-16T22:21:48.599 INFO:tasks.workunit.client.1.smithi123.stdout:0/180: creat d1/d8/f2f x:0 0 0 2024-02-16T22:21:48.599 INFO:tasks.workunit.client.0.smithi114.stdout:7/106: chown d3/l11 1943821 1 2024-02-16T22:21:48.599 INFO:tasks.workunit.client.1.smithi123.stdout:3/156: write d0/d20/f21 [854361,44442] 0 2024-02-16T22:21:48.600 INFO:tasks.workunit.client.0.smithi114.stdout:4/189: write dd/d1b/d24/f3a [964666,126615] 0 2024-02-16T22:21:48.600 INFO:tasks.workunit.client.1.smithi123.stdout:8/230: creat d0/d17/d26/d2a/f52 x:0 0 0 2024-02-16T22:21:48.601 INFO:tasks.workunit.client.0.smithi114.stdout:0/152: symlink d1/d16/d1f/d28/l2f 0 2024-02-16T22:21:48.602 INFO:tasks.workunit.client.1.smithi123.stdout:4/206: truncate d5/d1c/f25 2536401 0 2024-02-16T22:21:48.602 INFO:tasks.workunit.client.1.smithi123.stdout:4/207: stat d5/l15 0 2024-02-16T22:21:48.612 INFO:tasks.workunit.client.1.smithi123.stdout:6/198: mknod d0/d9/dc/d39/c47 0 2024-02-16T22:21:48.616 INFO:tasks.workunit.client.1.smithi123.stdout:5/150: mknod d6/dd/c30 0 2024-02-16T22:21:48.619 INFO:tasks.workunit.client.1.smithi123.stdout:0/181: write d1/f6 [9351510,3651] 0 2024-02-16T22:21:48.619 INFO:tasks.workunit.client.1.smithi123.stdout:0/182: fsync d1/f2e 0 2024-02-16T22:21:48.621 INFO:tasks.workunit.client.1.smithi123.stdout:8/231: rmdir d0/d17/d43/d34 39 2024-02-16T22:21:48.623 INFO:tasks.workunit.client.1.smithi123.stdout:9/257: dwrite d4/f12 [0,4194304] 0 2024-02-16T22:21:48.623 INFO:tasks.workunit.client.1.smithi123.stdout:1/166: creat d3/d6/db/d10/d2d/f38 x:0 0 0 2024-02-16T22:21:48.624 INFO:tasks.workunit.client.1.smithi123.stdout:9/258: fsync d4/d17/d24/f49 0 2024-02-16T22:21:48.630 INFO:tasks.workunit.client.1.smithi123.stdout:2/198: fsync d4/f37 0 2024-02-16T22:21:48.633 INFO:tasks.workunit.client.1.smithi123.stdout:3/157: mknod d0/c32 0 2024-02-16T22:21:48.635 INFO:tasks.workunit.client.1.smithi123.stdout:8/232: mknod d0/d12/d3e/c53 0 2024-02-16T22:21:48.636 INFO:tasks.workunit.client.1.smithi123.stdout:4/208: mknod d5/d1c/d23/c41 0 2024-02-16T22:21:48.637 INFO:tasks.workunit.client.1.smithi123.stdout:9/259: mknod d4/d17/d3f/c51 0 2024-02-16T22:21:48.638 INFO:tasks.workunit.client.1.smithi123.stdout:9/260: read - d4/d33/f3b zero size 2024-02-16T22:21:48.641 INFO:tasks.workunit.client.1.smithi123.stdout:2/199: creat d4/d1a/d1d/d13/d25/f43 x:0 0 0 2024-02-16T22:21:48.641 INFO:tasks.workunit.client.1.smithi123.stdout:2/200: chown d4/c35 4 1 2024-02-16T22:21:48.643 INFO:tasks.workunit.client.1.smithi123.stdout:3/158: fdatasync d0/f15 0 2024-02-16T22:21:48.643 INFO:tasks.workunit.client.1.smithi123.stdout:3/159: chown d0/d20 50502 1 2024-02-16T22:21:48.643 INFO:tasks.workunit.client.1.smithi123.stdout:8/233: chown d0/d17/d43/d34 23507455 1 2024-02-16T22:21:48.644 INFO:tasks.workunit.client.1.smithi123.stdout:8/234: fsync d0/d17/d43/d34/f38 0 2024-02-16T22:21:48.645 INFO:tasks.workunit.client.1.smithi123.stdout:1/167: link d3/d6/d11/f17 d3/d6/db/d10/d2d/f39 0 2024-02-16T22:21:48.646 INFO:tasks.workunit.client.1.smithi123.stdout:1/168: read - d3/f35 zero size 2024-02-16T22:21:48.646 INFO:tasks.workunit.client.1.smithi123.stdout:6/199: link d0/d9/dc/d39/c47 d0/d9/dc/d3a/c48 0 2024-02-16T22:21:48.649 INFO:tasks.workunit.client.1.smithi123.stdout:7/182: dread d3/d12/d34/f3c [0,4194304] 0 2024-02-16T22:21:48.649 INFO:tasks.workunit.client.1.smithi123.stdout:5/151: truncate d6/f11 3018787 0 2024-02-16T22:21:48.650 INFO:tasks.workunit.client.1.smithi123.stdout:2/201: mknod d4/d1a/d1d/d13/c44 0 2024-02-16T22:21:48.651 INFO:tasks.workunit.client.1.smithi123.stdout:0/183: creat d1/d8/f30 x:0 0 0 2024-02-16T22:21:48.652 INFO:tasks.workunit.client.1.smithi123.stdout:4/209: mkdir d5/d1c/d42 0 2024-02-16T22:21:48.654 INFO:tasks.workunit.client.1.smithi123.stdout:3/160: symlink d0/d20/l33 0 2024-02-16T22:21:48.655 INFO:tasks.workunit.client.1.smithi123.stdout:8/235: creat d0/d17/d26/d2a/f54 x:0 0 0 2024-02-16T22:21:48.658 INFO:tasks.workunit.client.1.smithi123.stdout:5/152: rmdir d6/dd/df 39 2024-02-16T22:21:48.660 INFO:tasks.workunit.client.1.smithi123.stdout:7/183: symlink d3/d12/d34/l3f 0 2024-02-16T22:21:48.662 INFO:tasks.workunit.client.1.smithi123.stdout:2/202: stat d4/ce 0 2024-02-16T22:21:48.662 INFO:tasks.workunit.client.1.smithi123.stdout:8/236: dwrite d0/d12/f46 [0,4194304] 0 2024-02-16T22:21:48.665 INFO:tasks.workunit.client.1.smithi123.stdout:3/161: mkdir d0/d20/d29/d34 0 2024-02-16T22:21:48.670 INFO:tasks.workunit.client.1.smithi123.stdout:8/237: creat d0/d17/d26/d2a/d3c/f55 x:0 0 0 2024-02-16T22:21:48.671 INFO:tasks.workunit.client.1.smithi123.stdout:0/184: dwrite d1/f2a [0,4194304] 0 2024-02-16T22:21:48.674 INFO:tasks.workunit.client.1.smithi123.stdout:3/162: rename d0/d20/f2c to d0/d20/d29/d34/f35 0 2024-02-16T22:21:48.677 INFO:tasks.workunit.client.1.smithi123.stdout:5/153: creat d6/dd/df/d1f/d25/f31 x:0 0 0 2024-02-16T22:21:48.678 INFO:tasks.workunit.client.1.smithi123.stdout:1/169: getdents d3/d6/db/d10/d2d 0 2024-02-16T22:21:48.680 INFO:tasks.workunit.client.1.smithi123.stdout:8/238: creat d0/d17/d26/d32/f56 x:0 0 0 2024-02-16T22:21:48.681 INFO:tasks.workunit.client.1.smithi123.stdout:6/200: dwrite d0/d29/f2c [0,4194304] 0 2024-02-16T22:21:48.683 INFO:tasks.workunit.client.1.smithi123.stdout:4/210: getdents d5/d1c/d23 0 2024-02-16T22:21:48.683 INFO:tasks.workunit.client.1.smithi123.stdout:7/184: dread d3/d12/d24/f31 [0,4194304] 0 2024-02-16T22:21:48.684 INFO:tasks.workunit.client.1.smithi123.stdout:5/154: mknod d6/dd/df/d1f/d25/c32 0 2024-02-16T22:21:48.685 INFO:tasks.workunit.client.1.smithi123.stdout:0/185: symlink d1/d4/l31 0 2024-02-16T22:21:48.686 INFO:tasks.workunit.client.1.smithi123.stdout:6/201: mknod d0/d9/dc/d39/c49 0 2024-02-16T22:21:48.689 INFO:tasks.workunit.client.1.smithi123.stdout:0/186: truncate d1/d8/d13/f15 4436733 0 2024-02-16T22:21:48.690 INFO:tasks.workunit.client.1.smithi123.stdout:0/187: write d1/d8/f30 [382421,3896] 0 2024-02-16T22:21:48.691 INFO:tasks.workunit.client.1.smithi123.stdout:3/163: dwrite d0/f1f [0,4194304] 0 2024-02-16T22:21:48.738 INFO:tasks.workunit.client.0.smithi114.stdout:3/97: sync 2024-02-16T22:21:48.740 INFO:tasks.workunit.client.1.smithi123.stdout:8/239: creat d0/d17/d26/d2a/d4d/f57 x:0 0 0 2024-02-16T22:21:48.742 INFO:tasks.workunit.client.1.smithi123.stdout:1/170: creat d3/d6/db/f3a x:0 0 0 2024-02-16T22:21:48.742 INFO:tasks.workunit.client.0.smithi114.stdout:6/93: rename d6/dd/f1a to d6/f1e 0 2024-02-16T22:21:48.742 INFO:tasks.workunit.client.1.smithi123.stdout:1/171: fsync d3/f5 0 2024-02-16T22:21:48.743 INFO:tasks.workunit.client.0.smithi114.stdout:1/100: link d4/c6 d4/d9/c1e 0 2024-02-16T22:21:48.743 INFO:tasks.workunit.client.0.smithi114.stdout:2/147: dwrite dd/f1d [0,4194304] 0 2024-02-16T22:21:48.744 INFO:tasks.workunit.client.0.smithi114.stdout:1/101: stat d4/d9/ce 0 2024-02-16T22:21:48.745 INFO:tasks.workunit.client.0.smithi114.stdout:8/142: mknod d4/d11/c28 0 2024-02-16T22:21:48.746 INFO:tasks.workunit.client.0.smithi114.stdout:3/98: symlink d4/l22 0 2024-02-16T22:21:48.746 INFO:tasks.workunit.client.1.smithi123.stdout:6/202: mknod d0/d9/dc/d39/c4a 0 2024-02-16T22:21:48.748 INFO:tasks.workunit.client.1.smithi123.stdout:7/185: write d3/d12/f14 [2769973,23431] 0 2024-02-16T22:21:48.753 INFO:tasks.workunit.client.0.smithi114.stdout:0/153: mkdir d1/d30 0 2024-02-16T22:21:48.753 INFO:tasks.workunit.client.0.smithi114.stdout:8/143: dwrite d4/d11/f1c [0,4194304] 0 2024-02-16T22:21:48.755 INFO:tasks.workunit.client.1.smithi123.stdout:5/155: symlink d6/l33 0 2024-02-16T22:21:48.756 INFO:tasks.workunit.client.0.smithi114.stdout:7/107: dread d3/ff [0,4194304] 0 2024-02-16T22:21:48.758 INFO:tasks.workunit.client.0.smithi114.stdout:4/190: rename dd/f11 to dd/d2e/d3f/f43 0 2024-02-16T22:21:48.760 INFO:tasks.workunit.client.0.smithi114.stdout:4/191: truncate dd/f21 264024 0 2024-02-16T22:21:48.763 INFO:tasks.workunit.client.0.smithi114.stdout:6/94: dwrite d6/f1e [0,4194304] 0 2024-02-16T22:21:48.763 INFO:tasks.workunit.client.0.smithi114.stdout:3/99: creat d4/dd/f23 x:0 0 0 2024-02-16T22:21:48.764 INFO:tasks.workunit.client.1.smithi123.stdout:6/203: symlink d0/d6/d1a/l4b 0 2024-02-16T22:21:48.765 INFO:tasks.workunit.client.0.smithi114.stdout:9/152: creat d2/dd/f34 x:0 0 0 2024-02-16T22:21:48.766 INFO:tasks.workunit.client.0.smithi114.stdout:9/153: chown d2/f26 718896 1 2024-02-16T22:21:48.767 INFO:tasks.workunit.client.1.smithi123.stdout:7/186: mknod d3/d12/d1d/d1e/c40 0 2024-02-16T22:21:48.769 INFO:tasks.workunit.client.1.smithi123.stdout:9/261: sync 2024-02-16T22:21:48.770 INFO:tasks.workunit.client.1.smithi123.stdout:0/188: unlink d1/cc 0 2024-02-16T22:21:48.772 INFO:tasks.workunit.client.1.smithi123.stdout:3/164: truncate d0/ff 1910301 0 2024-02-16T22:21:48.775 INFO:tasks.workunit.client.1.smithi123.stdout:5/156: symlink d6/dd/df/l34 0 2024-02-16T22:21:48.779 INFO:tasks.workunit.client.1.smithi123.stdout:1/172: mkdir d3/d6/db/d3b 0 2024-02-16T22:21:48.781 INFO:tasks.workunit.client.1.smithi123.stdout:6/204: mknod d0/d9/dc/d3a/c4c 0 2024-02-16T22:21:48.784 INFO:tasks.workunit.client.1.smithi123.stdout:7/187: creat d3/d12/d1d/f41 x:0 0 0 2024-02-16T22:21:48.784 INFO:tasks.workunit.client.1.smithi123.stdout:7/188: chown d3/d12/d24/c29 1479 1 2024-02-16T22:21:48.785 INFO:tasks.workunit.client.0.smithi114.stdout:2/148: unlink dd/df/d1c/d21/c25 0 2024-02-16T22:21:48.785 INFO:tasks.workunit.client.1.smithi123.stdout:7/189: chown d3/d12/d24/c32 62439706 1 2024-02-16T22:21:48.787 INFO:tasks.workunit.client.0.smithi114.stdout:4/192: mknod dd/d1b/d24/c44 0 2024-02-16T22:21:48.789 INFO:tasks.workunit.client.0.smithi114.stdout:1/102: symlink d4/l1f 0 2024-02-16T22:21:48.790 INFO:tasks.workunit.client.0.smithi114.stdout:8/144: dwrite d4/d11/f17 [0,4194304] 0 2024-02-16T22:21:48.793 INFO:tasks.workunit.client.0.smithi114.stdout:8/145: write d4/d11/f1c [2544881,104133] 0 2024-02-16T22:21:48.795 INFO:tasks.workunit.client.0.smithi114.stdout:6/95: symlink d6/da/l1f 0 2024-02-16T22:21:48.796 INFO:tasks.workunit.client.1.smithi123.stdout:1/173: fsync d3/d6/d11/f17 0 2024-02-16T22:21:48.797 INFO:tasks.workunit.client.0.smithi114.stdout:0/154: symlink d1/l31 0 2024-02-16T22:21:48.798 INFO:tasks.workunit.client.0.smithi114.stdout:0/155: read d1/d16/f19 [131909,52095] 0 2024-02-16T22:21:48.799 INFO:tasks.workunit.client.1.smithi123.stdout:6/205: fdatasync d0/d9/dc/f11 0 2024-02-16T22:21:48.799 INFO:tasks.workunit.client.1.smithi123.stdout:7/190: stat d3/d12/d1d/d1e/c21 0 2024-02-16T22:21:48.800 INFO:tasks.workunit.client.0.smithi114.stdout:2/149: readlink dd/l1a 0 2024-02-16T22:21:48.800 INFO:tasks.workunit.client.0.smithi114.stdout:7/108: dwrite d3/f8 [0,4194304] 0 2024-02-16T22:21:48.801 INFO:tasks.workunit.client.0.smithi114.stdout:7/109: readlink d3/lb 0 2024-02-16T22:21:48.803 INFO:tasks.workunit.client.0.smithi114.stdout:4/193: write dd/d2e/d3f/f43 [1784437,125802] 0 2024-02-16T22:21:48.807 INFO:tasks.workunit.client.1.smithi123.stdout:8/240: link d0/df/f1b d0/d12/f58 0 2024-02-16T22:21:48.808 INFO:tasks.workunit.client.1.smithi123.stdout:1/174: rmdir d3/d6/d11 39 2024-02-16T22:21:48.811 INFO:tasks.workunit.client.0.smithi114.stdout:8/146: creat d4/d24/f29 x:0 0 0 2024-02-16T22:21:48.812 INFO:tasks.workunit.client.0.smithi114.stdout:8/147: write d4/d11/f1c [5091201,54796] 0 2024-02-16T22:21:48.812 INFO:tasks.workunit.client.1.smithi123.stdout:6/206: mkdir d0/d6/d4d 0 2024-02-16T22:21:48.813 INFO:tasks.workunit.client.0.smithi114.stdout:6/96: write d6/dd/f1d [427712,80807] 0 2024-02-16T22:21:48.814 INFO:tasks.workunit.client.1.smithi123.stdout:0/189: rename d1/f2a to d1/d4/d17/f32 0 2024-02-16T22:21:48.820 INFO:tasks.workunit.client.1.smithi123.stdout:5/157: link d6/dd/df/d1f/f24 d6/dd/f35 0 2024-02-16T22:21:48.821 INFO:tasks.workunit.client.0.smithi114.stdout:8/148: dwrite d4/d11/f1c [0,4194304] 0 2024-02-16T22:21:48.822 INFO:tasks.workunit.client.0.smithi114.stdout:6/97: dwrite d6/dd/f1d [0,4194304] 0 2024-02-16T22:21:48.824 INFO:tasks.workunit.client.0.smithi114.stdout:6/98: write d6/f1e [4117132,105955] 0 2024-02-16T22:21:48.825 INFO:tasks.workunit.client.0.smithi114.stdout:6/99: read - d6/da/f1b zero size 2024-02-16T22:21:48.826 INFO:tasks.workunit.client.1.smithi123.stdout:1/175: symlink d3/d6/db/d10/l3c 0 2024-02-16T22:21:48.826 INFO:tasks.workunit.client.0.smithi114.stdout:1/103: truncate d4/fa 20858 0 2024-02-16T22:21:48.828 INFO:tasks.workunit.client.0.smithi114.stdout:3/100: rmdir d4/d21 0 2024-02-16T22:21:48.829 INFO:tasks.workunit.client.0.smithi114.stdout:3/101: chown d4/l1d 4 1 2024-02-16T22:21:48.834 INFO:tasks.workunit.client.1.smithi123.stdout:7/191: rename d3/d5/c11 to d3/d12/d34/c42 0 2024-02-16T22:21:48.835 INFO:tasks.workunit.client.1.smithi123.stdout:3/165: dwrite d0/ff [0,4194304] 0 2024-02-16T22:21:48.836 INFO:tasks.workunit.client.1.smithi123.stdout:8/241: dread d0/d12/f19 [0,4194304] 0 2024-02-16T22:21:48.837 INFO:tasks.workunit.client.0.smithi114.stdout:0/156: mknod d1/c32 0 2024-02-16T22:21:48.840 INFO:tasks.workunit.client.0.smithi114.stdout:2/150: truncate dd/f28 2343247 0 2024-02-16T22:21:48.840 INFO:tasks.workunit.client.0.smithi114.stdout:2/151: stat dd/df/f14 0 2024-02-16T22:21:48.841 INFO:tasks.workunit.client.0.smithi114.stdout:2/152: readlink dd/df/d1c/l29 0 2024-02-16T22:21:48.842 INFO:tasks.workunit.client.1.smithi123.stdout:6/207: mknod d0/c4e 0 2024-02-16T22:21:48.842 INFO:tasks.workunit.client.0.smithi114.stdout:2/153: chown dd/df/f1e 4724 1 2024-02-16T22:21:48.843 INFO:tasks.workunit.client.1.smithi123.stdout:8/242: dwrite d0/d17/d43/f27 [0,4194304] 0 2024-02-16T22:21:48.843 INFO:tasks.workunit.client.0.smithi114.stdout:7/110: symlink d3/d14/d1b/l1e 0 2024-02-16T22:21:48.844 INFO:tasks.workunit.client.1.smithi123.stdout:8/243: chown d0/d12/c2c 16023 1 2024-02-16T22:21:48.844 INFO:tasks.workunit.client.1.smithi123.stdout:6/208: dread d0/d6/fd [4194304,4194304] 0 2024-02-16T22:21:48.844 INFO:tasks.workunit.client.0.smithi114.stdout:7/111: readlink d3/d14/d1b/l1e 0 2024-02-16T22:21:48.852 INFO:tasks.workunit.client.0.smithi114.stdout:8/149: rename d4/d24/l27 to d4/d11/d15/l2a 0 2024-02-16T22:21:48.855 INFO:tasks.workunit.client.0.smithi114.stdout:4/194: mknod dd/d1b/d37/d38/c45 0 2024-02-16T22:21:48.858 INFO:tasks.workunit.client.0.smithi114.stdout:6/100: symlink d6/da/l20 0 2024-02-16T22:21:48.863 INFO:tasks.workunit.client.0.smithi114.stdout:3/102: mkdir d4/d24 0 2024-02-16T22:21:48.865 INFO:tasks.workunit.client.0.smithi114.stdout:3/103: write d4/dd/f13 [27466,30739] 0 2024-02-16T22:21:48.865 INFO:tasks.workunit.client.0.smithi114.stdout:0/157: chown d1/d1a/f2c 62821 1 2024-02-16T22:21:48.866 INFO:tasks.workunit.client.0.smithi114.stdout:0/158: stat d1/f7 0 2024-02-16T22:21:48.866 INFO:tasks.workunit.client.0.smithi114.stdout:2/154: creat dd/df/d2c/f32 x:0 0 0 2024-02-16T22:21:48.867 INFO:tasks.workunit.client.0.smithi114.stdout:0/159: read d1/f8 [1817864,102670] 0 2024-02-16T22:21:48.867 INFO:tasks.workunit.client.0.smithi114.stdout:7/112: mkdir d3/d14/d1b/d1f 0 2024-02-16T22:21:48.868 INFO:tasks.workunit.client.0.smithi114.stdout:0/160: chown d1/d16/d18 62910 1 2024-02-16T22:21:48.870 INFO:tasks.workunit.client.0.smithi114.stdout:4/195: creat dd/d1b/d37/d38/f46 x:0 0 0 2024-02-16T22:21:48.870 INFO:tasks.workunit.client.1.smithi123.stdout:4/211: sync 2024-02-16T22:21:48.870 INFO:tasks.workunit.client.1.smithi123.stdout:2/203: sync 2024-02-16T22:21:48.871 INFO:tasks.workunit.client.0.smithi114.stdout:7/113: dread d3/f8 [0,4194304] 0 2024-02-16T22:21:48.874 INFO:tasks.workunit.client.1.smithi123.stdout:6/209: chown d0/d6 1 1 2024-02-16T22:21:48.874 INFO:tasks.workunit.client.1.smithi123.stdout:0/190: mknod d1/d8/d13/c33 0 2024-02-16T22:21:48.875 INFO:tasks.workunit.client.0.smithi114.stdout:4/196: dread dd/f13 [0,4194304] 0 2024-02-16T22:21:48.875 INFO:tasks.workunit.client.1.smithi123.stdout:1/176: rename d3/d6/db/d10/c1e to d3/d6/db/d10/d2d/c3d 0 2024-02-16T22:21:48.875 INFO:tasks.workunit.client.1.smithi123.stdout:3/166: readlink d0/l1d 0 2024-02-16T22:21:48.877 INFO:tasks.workunit.client.1.smithi123.stdout:3/167: dread d0/d20/f26 [0,4194304] 0 2024-02-16T22:21:48.879 INFO:tasks.workunit.client.1.smithi123.stdout:8/244: mkdir d0/d17/d26/d2a/d59 0 2024-02-16T22:21:48.880 INFO:tasks.workunit.client.0.smithi114.stdout:4/197: dread dd/d1b/d24/f3a [0,4194304] 0 2024-02-16T22:21:48.880 INFO:tasks.workunit.client.1.smithi123.stdout:8/245: readlink d0/d29/d42/d47/l4a 0 2024-02-16T22:21:48.881 INFO:tasks.workunit.client.1.smithi123.stdout:4/212: dwrite d5/d1c/f24 [0,4194304] 0 2024-02-16T22:21:48.882 INFO:tasks.workunit.client.0.smithi114.stdout:8/150: dread d4/f6 [0,4194304] 0 2024-02-16T22:21:48.883 INFO:tasks.workunit.client.1.smithi123.stdout:8/246: write d0/df/f1c [4598325,113305] 0 2024-02-16T22:21:48.884 INFO:tasks.workunit.client.1.smithi123.stdout:5/158: dread d6/f11 [0,4194304] 0 2024-02-16T22:21:48.884 INFO:tasks.workunit.client.0.smithi114.stdout:8/151: rename d4 to d4/d16/d25/d2b 22 2024-02-16T22:21:48.886 INFO:tasks.workunit.client.1.smithi123.stdout:5/159: dread - d6/dd/df/d1f/f2e zero size 2024-02-16T22:21:48.888 INFO:tasks.workunit.client.1.smithi123.stdout:4/213: dwrite d5/f1e [0,4194304] 0 2024-02-16T22:21:48.890 INFO:tasks.workunit.client.1.smithi123.stdout:1/177: read d3/d6/d11/f1f [2551674,101282] 0 2024-02-16T22:21:48.891 INFO:tasks.workunit.client.0.smithi114.stdout:4/198: dwrite dd/d1b/d37/d38/f46 [0,4194304] 0 2024-02-16T22:21:48.891 INFO:tasks.workunit.client.0.smithi114.stdout:4/199: readlink dd/l10 0 2024-02-16T22:21:48.891 INFO:tasks.workunit.client.1.smithi123.stdout:1/178: stat d3/f8 0 2024-02-16T22:21:48.892 INFO:tasks.workunit.client.1.smithi123.stdout:1/179: read d3/f5 [8393280,34266] 0 2024-02-16T22:21:48.899 INFO:tasks.workunit.client.0.smithi114.stdout:3/104: creat d4/d24/f25 x:0 0 0 2024-02-16T22:21:48.901 INFO:tasks.workunit.client.1.smithi123.stdout:6/210: unlink d0/c3 0 2024-02-16T22:21:48.902 INFO:tasks.workunit.client.0.smithi114.stdout:0/161: truncate d1/f15 85313 0 2024-02-16T22:21:48.904 INFO:tasks.workunit.client.0.smithi114.stdout:8/152: mkdir d4/d24/d2c 0 2024-02-16T22:21:48.905 INFO:tasks.workunit.client.1.smithi123.stdout:8/247: mknod d0/d17/d26/d2a/d3c/c5a 0 2024-02-16T22:21:48.906 INFO:tasks.workunit.client.0.smithi114.stdout:6/101: creat d6/f21 x:0 0 0 2024-02-16T22:21:48.906 INFO:tasks.workunit.client.0.smithi114.stdout:6/102: stat d6/da/l13 0 2024-02-16T22:21:48.907 INFO:tasks.workunit.client.0.smithi114.stdout:6/103: readlink d6/le 0 2024-02-16T22:21:48.908 INFO:tasks.workunit.client.0.smithi114.stdout:6/104: stat d6/le 0 2024-02-16T22:21:48.911 INFO:tasks.workunit.client.0.smithi114.stdout:6/105: dread d6/dd/f17 [0,4194304] 0 2024-02-16T22:21:48.912 INFO:tasks.workunit.client.1.smithi123.stdout:4/214: creat d5/d1c/d23/f43 x:0 0 0 2024-02-16T22:21:48.913 INFO:tasks.workunit.client.0.smithi114.stdout:3/105: mkdir d4/d26 0 2024-02-16T22:21:48.914 INFO:tasks.workunit.client.1.smithi123.stdout:1/180: symlink d3/d6/d18/l3e 0 2024-02-16T22:21:48.915 INFO:tasks.workunit.client.1.smithi123.stdout:1/181: write d3/f5 [7371604,92575] 0 2024-02-16T22:21:48.916 INFO:tasks.workunit.client.0.smithi114.stdout:2/155: unlink dd/f10 0 2024-02-16T22:21:48.918 INFO:tasks.workunit.client.0.smithi114.stdout:8/153: mknod d4/d24/c2d 0 2024-02-16T22:21:48.919 INFO:tasks.workunit.client.1.smithi123.stdout:9/262: sync 2024-02-16T22:21:48.919 INFO:tasks.workunit.client.0.smithi114.stdout:8/154: write d4/f1d [123934,115174] 0 2024-02-16T22:21:48.919 INFO:tasks.workunit.client.1.smithi123.stdout:6/211: mknod d0/d6/d1a/d2e/c4f 0 2024-02-16T22:21:48.921 INFO:tasks.workunit.client.1.smithi123.stdout:9/263: read d4/d17/f1d [2253827,52895] 0 2024-02-16T22:21:48.921 INFO:tasks.workunit.client.1.smithi123.stdout:6/212: chown d0/d6/d2f 40 1 2024-02-16T22:21:48.922 INFO:tasks.workunit.client.1.smithi123.stdout:6/213: chown d0/d9/dc/d39 1573718 1 2024-02-16T22:21:48.922 INFO:tasks.workunit.client.1.smithi123.stdout:9/264: write d4/d17/d20/f3c [647750,53321] 0 2024-02-16T22:21:48.922 INFO:tasks.workunit.client.1.smithi123.stdout:8/248: creat d0/d17/d26/f5b x:0 0 0 2024-02-16T22:21:48.923 INFO:tasks.workunit.client.1.smithi123.stdout:8/249: write d0/d17/d43/f28 [3650793,110678] 0 2024-02-16T22:21:48.924 INFO:tasks.workunit.client.0.smithi114.stdout:7/114: link d3/d14/l16 d3/d14/d1b/l20 0 2024-02-16T22:21:48.925 INFO:tasks.workunit.client.1.smithi123.stdout:0/191: creat d1/d8/f34 x:0 0 0 2024-02-16T22:21:48.927 INFO:tasks.workunit.client.1.smithi123.stdout:5/160: creat d6/dd/df/d2b/f36 x:0 0 0 2024-02-16T22:21:48.928 INFO:tasks.workunit.client.0.smithi114.stdout:7/115: dread d3/d14/f1c [0,4194304] 0 2024-02-16T22:21:48.929 INFO:tasks.workunit.client.1.smithi123.stdout:2/204: rename d4/d11 to d4/d1a/d29/d45 0 2024-02-16T22:21:48.933 INFO:tasks.workunit.client.0.smithi114.stdout:2/156: rmdir dd/df/d1c 39 2024-02-16T22:21:48.933 INFO:tasks.workunit.client.0.smithi114.stdout:2/157: chown l2 55729711 1 2024-02-16T22:21:48.934 INFO:tasks.workunit.client.1.smithi123.stdout:2/205: dwrite d4/f37 [0,4194304] 0 2024-02-16T22:21:48.937 INFO:tasks.workunit.client.1.smithi123.stdout:3/168: link d0/f19 d0/d20/d29/d34/f36 0 2024-02-16T22:21:48.937 INFO:tasks.workunit.client.1.smithi123.stdout:3/169: fdatasync d0/f15 0 2024-02-16T22:21:48.942 INFO:tasks.workunit.client.0.smithi114.stdout:4/200: rename dd/d1b/d24/f42 to dd/d1b/f47 0 2024-02-16T22:21:48.943 INFO:tasks.workunit.client.1.smithi123.stdout:9/265: symlink d4/d36/l52 0 2024-02-16T22:21:48.945 INFO:tasks.workunit.client.1.smithi123.stdout:6/214: creat d0/d6/d1a/d2e/f50 x:0 0 0 2024-02-16T22:21:48.946 INFO:tasks.workunit.client.1.smithi123.stdout:6/215: chown d0/d9/dc/c12 64554227 1 2024-02-16T22:21:48.949 INFO:tasks.workunit.client.1.smithi123.stdout:1/182: rmdir d3/d6/d18 39 2024-02-16T22:21:48.950 INFO:tasks.workunit.client.1.smithi123.stdout:8/250: unlink d0/d17/d43/f3d 0 2024-02-16T22:21:48.951 INFO:tasks.workunit.client.0.smithi114.stdout:3/106: symlink d4/d26/l27 0 2024-02-16T22:21:48.952 INFO:tasks.workunit.client.0.smithi114.stdout:3/107: write d4/dd/f1e [116694,125077] 0 2024-02-16T22:21:48.953 INFO:tasks.workunit.client.1.smithi123.stdout:8/251: dread d0/d17/d43/f28 [0,4194304] 0 2024-02-16T22:21:48.954 INFO:tasks.workunit.client.1.smithi123.stdout:5/161: mkdir d6/dd/df/d1f/d25/d37 0 2024-02-16T22:21:48.957 INFO:tasks.workunit.client.1.smithi123.stdout:5/162: dread d6/f11 [0,4194304] 0 2024-02-16T22:21:48.957 INFO:tasks.workunit.client.1.smithi123.stdout:2/206: symlink d4/d1a/d1d/d13/d25/l46 0 2024-02-16T22:21:48.957 INFO:tasks.workunit.client.1.smithi123.stdout:8/252: dread d0/d17/d43/f27 [0,4194304] 0 2024-02-16T22:21:48.958 INFO:tasks.workunit.client.1.smithi123.stdout:5/163: write d6/dd/df/f22 [81439,47543] 0 2024-02-16T22:21:48.960 INFO:tasks.workunit.client.1.smithi123.stdout:3/170: write d0/f17 [5219275,31376] 0 2024-02-16T22:21:48.965 INFO:tasks.workunit.client.0.smithi114.stdout:1/104: dwrite d4/fa [0,4194304] 0 2024-02-16T22:21:48.966 INFO:tasks.workunit.client.0.smithi114.stdout:1/105: write d4/d9/f12 [134465,108334] 0 2024-02-16T22:21:48.967 INFO:tasks.workunit.client.0.smithi114.stdout:7/116: mknod d3/c21 0 2024-02-16T22:21:48.970 INFO:tasks.workunit.client.1.smithi123.stdout:6/216: unlink d0/d9/dc/d3a/c4c 0 2024-02-16T22:21:48.973 INFO:tasks.workunit.client.0.smithi114.stdout:3/108: creat d4/d26/f28 x:0 0 0 2024-02-16T22:21:48.976 INFO:tasks.workunit.client.0.smithi114.stdout:3/109: dread d4/f1a [0,4194304] 0 2024-02-16T22:21:48.984 INFO:tasks.workunit.client.0.smithi114.stdout:3/110: dwrite f1 [0,4194304] 0 2024-02-16T22:21:48.985 INFO:tasks.workunit.client.0.smithi114.stdout:4/201: write dd/f2b [452053,73379] 0 2024-02-16T22:21:48.986 INFO:tasks.workunit.client.1.smithi123.stdout:8/253: fdatasync d0/d12/f19 0 2024-02-16T22:21:48.988 INFO:tasks.workunit.client.1.smithi123.stdout:2/207: symlink d4/d1a/d1d/d13/l47 0 2024-02-16T22:21:48.989 INFO:tasks.workunit.client.0.smithi114.stdout:6/106: getdents d6/da 0 2024-02-16T22:21:48.989 INFO:tasks.workunit.client.0.smithi114.stdout:2/158: write fc [472741,60071] 0 2024-02-16T22:21:48.992 INFO:tasks.workunit.client.1.smithi123.stdout:3/171: mkdir d0/d20/d29/d34/d37 0 2024-02-16T22:21:48.993 INFO:tasks.workunit.client.1.smithi123.stdout:3/172: stat d0/c2d 0 2024-02-16T22:21:48.995 INFO:tasks.workunit.client.1.smithi123.stdout:4/215: fsync d5/d1c/d23/d31/f3e 0 2024-02-16T22:21:48.995 INFO:tasks.workunit.client.1.smithi123.stdout:0/192: write d1/f22 [94540,3550] 0 2024-02-16T22:21:48.997 INFO:tasks.workunit.client.1.smithi123.stdout:6/217: fsync d0/d6/f1e 0 2024-02-16T22:21:48.998 INFO:tasks.workunit.client.0.smithi114.stdout:0/162: rename d1/d16/d18/l21 to d1/d16/l33 0 2024-02-16T22:21:49.000 INFO:tasks.workunit.client.0.smithi114.stdout:3/111: write d4/f1c [3558413,34190] 0 2024-02-16T22:21:49.000 INFO:tasks.workunit.client.1.smithi123.stdout:4/216: dwrite d5/f14 [4194304,4194304] 0 2024-02-16T22:21:49.005 INFO:tasks.workunit.client.0.smithi114.stdout:6/107: mknod d6/da/c22 0 2024-02-16T22:21:49.006 INFO:tasks.workunit.client.0.smithi114.stdout:3/112: dwrite d4/f1c [0,4194304] 0 2024-02-16T22:21:49.006 INFO:tasks.workunit.client.1.smithi123.stdout:4/217: dwrite d5/f17 [4194304,4194304] 0 2024-02-16T22:21:49.009 INFO:tasks.workunit.client.1.smithi123.stdout:7/192: sync 2024-02-16T22:21:49.009 INFO:tasks.workunit.client.1.smithi123.stdout:2/208: symlink d4/d1a/l48 0 2024-02-16T22:21:49.010 INFO:tasks.workunit.client.0.smithi114.stdout:1/106: fdatasync d4/fb 0 2024-02-16T22:21:49.010 INFO:tasks.workunit.client.1.smithi123.stdout:2/209: write d4/f37 [2729088,124948] 0 2024-02-16T22:21:49.011 INFO:tasks.workunit.client.0.smithi114.stdout:1/107: write d4/fa [207568,126023] 0 2024-02-16T22:21:49.011 INFO:tasks.workunit.client.1.smithi123.stdout:2/210: chown d4/d1a/d1d/d13/l47 41911 1 2024-02-16T22:21:49.012 INFO:tasks.workunit.client.0.smithi114.stdout:1/108: read d4/f13 [656183,102731] 0 2024-02-16T22:21:49.012 INFO:tasks.workunit.client.1.smithi123.stdout:2/211: readlink d4/d1a/d1d/l10 0 2024-02-16T22:21:49.016 INFO:tasks.workunit.client.0.smithi114.stdout:1/109: chown d4/d9/c1e 133582 1 2024-02-16T22:21:49.016 INFO:tasks.workunit.client.0.smithi114.stdout:0/163: truncate d1/d16/d1f/d28/f2d 661048 0 2024-02-16T22:21:49.016 INFO:tasks.workunit.client.0.smithi114.stdout:4/202: mknod dd/d1b/d41/c48 0 2024-02-16T22:21:49.016 INFO:tasks.workunit.client.0.smithi114.stdout:0/164: chown d1/f7 0 1 2024-02-16T22:21:49.020 INFO:tasks.workunit.client.1.smithi123.stdout:0/193: dwrite d1/f1f [0,4194304] 0 2024-02-16T22:21:49.024 INFO:tasks.workunit.client.0.smithi114.stdout:2/159: dwrite dd/df/f2f [0,4194304] 0 2024-02-16T22:21:49.027 INFO:tasks.workunit.client.1.smithi123.stdout:8/254: dwrite d0/d12/f1e [0,4194304] 0 2024-02-16T22:21:49.029 INFO:tasks.workunit.client.1.smithi123.stdout:7/193: rename d3/c17 to d3/d12/d1d/c43 0 2024-02-16T22:21:49.030 INFO:tasks.workunit.client.1.smithi123.stdout:3/173: symlink d0/l38 0 2024-02-16T22:21:49.034 INFO:tasks.workunit.client.0.smithi114.stdout:7/117: rename d3/d14/d1b/d1f to d3/d22 0 2024-02-16T22:21:49.036 INFO:tasks.workunit.client.0.smithi114.stdout:4/203: dread dd/d1b/f1d [0,4194304] 0 2024-02-16T22:21:49.038 INFO:tasks.workunit.client.0.smithi114.stdout:1/110: getdents d4/d1c 0 2024-02-16T22:21:49.043 INFO:tasks.workunit.client.0.smithi114.stdout:6/108: link d6/f21 d6/f23 0 2024-02-16T22:21:49.043 INFO:tasks.workunit.client.0.smithi114.stdout:6/109: dread - d6/f23 zero size 2024-02-16T22:21:49.045 INFO:tasks.workunit.client.1.smithi123.stdout:1/183: getdents d3/d6 0 2024-02-16T22:21:49.046 INFO:tasks.workunit.client.1.smithi123.stdout:4/218: mknod d5/d1c/d23/d36/c44 0 2024-02-16T22:21:49.047 INFO:tasks.workunit.client.0.smithi114.stdout:7/118: fsync d3/d14/f1c 0 2024-02-16T22:21:49.050 INFO:tasks.workunit.client.0.smithi114.stdout:4/204: rename dd/f16 to dd/d2e/f49 0 2024-02-16T22:21:49.053 INFO:tasks.workunit.client.1.smithi123.stdout:9/266: sync 2024-02-16T22:21:49.053 INFO:tasks.workunit.client.0.smithi114.stdout:2/160: truncate dd/f17 73978 0 2024-02-16T22:21:49.054 INFO:tasks.workunit.client.1.smithi123.stdout:9/267: write d4/f1b [1717568,128106] 0 2024-02-16T22:21:49.055 INFO:tasks.workunit.client.1.smithi123.stdout:6/218: creat d0/d9/f51 x:0 0 0 2024-02-16T22:21:49.056 INFO:tasks.workunit.client.1.smithi123.stdout:1/184: truncate d3/f7 4041811 0 2024-02-16T22:21:49.058 INFO:tasks.workunit.client.0.smithi114.stdout:0/165: creat d1/d16/f34 x:0 0 0 2024-02-16T22:21:49.059 INFO:tasks.workunit.client.1.smithi123.stdout:4/219: creat d5/d1c/d23/d36/f45 x:0 0 0 2024-02-16T22:21:49.061 INFO:tasks.workunit.client.0.smithi114.stdout:6/110: symlink d6/dd/l24 0 2024-02-16T22:21:49.063 INFO:tasks.workunit.client.1.smithi123.stdout:2/212: link d4/d1a/d1d/d13/l40 d4/d1a/d1d/l49 0 2024-02-16T22:21:49.064 INFO:tasks.workunit.client.1.smithi123.stdout:2/213: truncate d4/d1a/d1d/d13/d15/f42 655218 0 2024-02-16T22:21:49.067 INFO:tasks.workunit.client.1.smithi123.stdout:2/214: dread d4/d1a/d1d/f2b [0,4194304] 0 2024-02-16T22:21:49.067 INFO:tasks.workunit.client.1.smithi123.stdout:9/268: creat d4/d17/d2b/f53 x:0 0 0 2024-02-16T22:21:49.068 INFO:tasks.workunit.client.0.smithi114.stdout:4/205: rmdir dd/d1b/d24 39 2024-02-16T22:21:49.069 INFO:tasks.workunit.client.1.smithi123.stdout:6/219: rename d0/d9/d46 to d0/d9/dc/d3a/d52 0 2024-02-16T22:21:49.072 INFO:tasks.workunit.client.1.smithi123.stdout:4/220: mknod d5/d1c/d23/d36/c46 0 2024-02-16T22:21:49.072 INFO:tasks.workunit.client.1.smithi123.stdout:4/221: chown d5/l10 0 1 2024-02-16T22:21:49.073 INFO:tasks.workunit.client.1.smithi123.stdout:4/222: fsync d5/f29 0 2024-02-16T22:21:49.073 INFO:tasks.workunit.client.1.smithi123.stdout:4/223: fdatasync d5/f17 0 2024-02-16T22:21:49.074 INFO:tasks.workunit.client.0.smithi114.stdout:2/161: fdatasync f4 0 2024-02-16T22:21:49.078 INFO:tasks.workunit.client.0.smithi114.stdout:0/166: rename d1/d1a/f1d to d1/d16/d1f/d28/f35 0 2024-02-16T22:21:49.082 INFO:tasks.workunit.client.1.smithi123.stdout:9/269: symlink d4/d36/l54 0 2024-02-16T22:21:49.083 INFO:tasks.workunit.client.0.smithi114.stdout:6/111: dread d6/dd/f17 [0,4194304] 0 2024-02-16T22:21:49.083 INFO:tasks.workunit.client.0.smithi114.stdout:7/119: mkdir d3/d23 0 2024-02-16T22:21:49.085 INFO:tasks.workunit.client.1.smithi123.stdout:5/164: sync 2024-02-16T22:21:49.086 INFO:tasks.workunit.client.1.smithi123.stdout:5/165: fsync d6/dd/df/d1f/d25/f1a 0 2024-02-16T22:21:49.089 INFO:tasks.workunit.client.1.smithi123.stdout:2/215: mkdir d4/d1a/d1d/d13/d15/d1b/d4a 0 2024-02-16T22:21:49.090 INFO:tasks.workunit.client.0.smithi114.stdout:6/112: dwrite d6/f18 [0,4194304] 0 2024-02-16T22:21:49.091 INFO:tasks.workunit.client.0.smithi114.stdout:7/120: dread d3/d14/f1c [0,4194304] 0 2024-02-16T22:21:49.091 INFO:tasks.workunit.client.1.smithi123.stdout:6/220: fdatasync d0/d9/dc/f23 0 2024-02-16T22:21:49.092 INFO:tasks.workunit.client.0.smithi114.stdout:1/111: rmdir d4/d1c 0 2024-02-16T22:21:49.092 INFO:tasks.workunit.client.1.smithi123.stdout:6/221: read - d0/d29/f3e zero size 2024-02-16T22:21:49.093 INFO:tasks.workunit.client.1.smithi123.stdout:1/185: mknod d3/d6/db/d10/d2a/c3f 0 2024-02-16T22:21:49.098 INFO:tasks.workunit.client.0.smithi114.stdout:0/167: creat d1/d1a/f36 x:0 0 0 2024-02-16T22:21:49.100 INFO:tasks.workunit.client.1.smithi123.stdout:8/255: rename d0/df/c31 to d0/df/c5c 0 2024-02-16T22:21:49.101 INFO:tasks.workunit.client.1.smithi123.stdout:7/194: getdents d3/d26 0 2024-02-16T22:21:49.102 INFO:tasks.workunit.client.1.smithi123.stdout:7/195: chown d3/d5 47 1 2024-02-16T22:21:49.109 INFO:tasks.workunit.client.0.smithi114.stdout:7/121: fsync d3/f10 0 2024-02-16T22:21:49.110 INFO:tasks.workunit.client.0.smithi114.stdout:7/122: stat d3/lb 0 2024-02-16T22:21:49.111 INFO:tasks.workunit.client.0.smithi114.stdout:4/206: dwrite dd/d1b/f1d [0,4194304] 0 2024-02-16T22:21:49.112 INFO:tasks.workunit.client.1.smithi123.stdout:2/216: rmdir d4/d1a/d1d 39 2024-02-16T22:21:49.113 INFO:tasks.workunit.client.0.smithi114.stdout:1/112: creat d4/d9/f20 x:0 0 0 2024-02-16T22:21:49.115 INFO:tasks.workunit.client.0.smithi114.stdout:2/162: rmdir dd/df/d1c 39 2024-02-16T22:21:49.116 INFO:tasks.workunit.client.1.smithi123.stdout:1/186: chown d3/d6/db/c12 16 1 2024-02-16T22:21:49.118 INFO:tasks.workunit.client.0.smithi114.stdout:0/168: symlink d1/d16/d1f/l37 0 2024-02-16T22:21:49.118 INFO:tasks.workunit.client.1.smithi123.stdout:4/224: mkdir d5/d1c/d23/d47 0 2024-02-16T22:21:49.120 INFO:tasks.workunit.client.1.smithi123.stdout:4/225: truncate d5/d1c/f3d 846029 0 2024-02-16T22:21:49.122 INFO:tasks.workunit.client.1.smithi123.stdout:7/196: write d3/d12/d34/f3c [1510351,16123] 0 2024-02-16T22:21:49.123 INFO:tasks.workunit.client.1.smithi123.stdout:9/270: dwrite d4/f21 [0,4194304] 0 2024-02-16T22:21:49.124 INFO:tasks.workunit.client.1.smithi123.stdout:9/271: chown d4/d17/l3e 225076 1 2024-02-16T22:21:49.124 INFO:tasks.workunit.client.0.smithi114.stdout:7/123: mknod d3/d14/c24 0 2024-02-16T22:21:49.125 INFO:tasks.workunit.client.0.smithi114.stdout:6/113: dwrite d6/f23 [0,4194304] 0 2024-02-16T22:21:49.125 INFO:tasks.workunit.client.0.smithi114.stdout:6/114: stat d6/dd/f19 0 2024-02-16T22:21:49.126 INFO:tasks.workunit.client.0.smithi114.stdout:4/207: readlink dd/d1b/d24/l25 0 2024-02-16T22:21:49.127 INFO:tasks.workunit.client.0.smithi114.stdout:4/208: write dd/f13 [867971,78262] 0 2024-02-16T22:21:49.128 INFO:tasks.workunit.client.1.smithi123.stdout:3/174: sync 2024-02-16T22:21:49.128 INFO:tasks.workunit.client.1.smithi123.stdout:0/194: sync 2024-02-16T22:21:49.128 INFO:tasks.workunit.client.1.smithi123.stdout:2/217: write d4/d1a/d1d/d13/d15/f42 [1160932,130060] 0 2024-02-16T22:21:49.129 INFO:tasks.workunit.client.0.smithi114.stdout:6/115: dread d6/dd/f19 [0,4194304] 0 2024-02-16T22:21:49.130 INFO:tasks.workunit.client.1.smithi123.stdout:3/175: chown d0/d20/d29/c30 0 1 2024-02-16T22:21:49.132 INFO:tasks.workunit.client.1.smithi123.stdout:6/222: getdents d0/d9/dc/d3a/d52 0 2024-02-16T22:21:49.133 INFO:tasks.workunit.client.1.smithi123.stdout:8/256: write d0/d12/f19 [2657370,27169] 0 2024-02-16T22:21:49.135 INFO:tasks.workunit.client.1.smithi123.stdout:8/257: stat d0/d29/d42/d47/l4a 0 2024-02-16T22:21:49.135 INFO:tasks.workunit.client.0.smithi114.stdout:0/169: creat d1/d1a/f38 x:0 0 0 2024-02-16T22:21:49.136 INFO:tasks.workunit.client.1.smithi123.stdout:2/218: dwrite d4/d1a/d1d/d13/f36 [0,4194304] 0 2024-02-16T22:21:49.136 INFO:tasks.workunit.client.1.smithi123.stdout:4/226: dread - d5/d1c/d23/d36/f45 zero size 2024-02-16T22:21:49.138 INFO:tasks.workunit.client.0.smithi114.stdout:4/209: dwrite dd/d1b/f2c [0,4194304] 0 2024-02-16T22:21:49.138 INFO:tasks.workunit.client.1.smithi123.stdout:7/197: mknod d3/d12/d1d/c44 0 2024-02-16T22:21:49.139 INFO:tasks.workunit.client.1.smithi123.stdout:4/227: truncate d5/d1c/d23/d36/f45 209577 0 2024-02-16T22:21:49.140 INFO:tasks.workunit.client.1.smithi123.stdout:9/272: rmdir d4/d17/d3f 39 2024-02-16T22:21:49.142 INFO:tasks.workunit.client.0.smithi114.stdout:7/124: creat d3/d14/d1b/f25 x:0 0 0 2024-02-16T22:21:49.145 INFO:tasks.workunit.client.1.smithi123.stdout:5/166: rename d6/dd/df/l34 to d6/l38 0 2024-02-16T22:21:49.146 INFO:tasks.workunit.client.0.smithi114.stdout:6/116: mkdir d6/dd/d25 0 2024-02-16T22:21:49.147 INFO:tasks.workunit.client.0.smithi114.stdout:6/117: stat d6/dd 0 2024-02-16T22:21:49.149 INFO:tasks.workunit.client.0.smithi114.stdout:2/163: fdatasync dd/f28 0 2024-02-16T22:21:49.152 INFO:tasks.workunit.client.1.smithi123.stdout:3/176: fdatasync d0/d20/d29/d34/f35 0 2024-02-16T22:21:49.154 INFO:tasks.workunit.client.0.smithi114.stdout:4/210: creat dd/d1b/d24/f4a x:0 0 0 2024-02-16T22:21:49.156 INFO:tasks.workunit.client.0.smithi114.stdout:1/113: creat d4/f21 x:0 0 0 2024-02-16T22:21:49.156 INFO:tasks.workunit.client.1.smithi123.stdout:7/198: creat d3/d5/f45 x:0 0 0 2024-02-16T22:21:49.160 INFO:tasks.workunit.client.1.smithi123.stdout:2/219: dwrite d4/d1a/d1d/f2b [0,4194304] 0 2024-02-16T22:21:49.160 INFO:tasks.workunit.client.1.smithi123.stdout:2/220: chown d4/d1a/l1f 0 1 2024-02-16T22:21:49.161 INFO:tasks.workunit.client.1.smithi123.stdout:5/167: truncate d6/dd/df/d1f/d25/f1d 381856 0 2024-02-16T22:21:49.164 INFO:tasks.workunit.client.1.smithi123.stdout:3/177: chown d0/f7 0 1 2024-02-16T22:21:49.165 INFO:tasks.workunit.client.0.smithi114.stdout:6/118: mkdir d6/dd/d25/d26 0 2024-02-16T22:21:49.167 INFO:tasks.workunit.client.1.smithi123.stdout:6/223: creat d0/d6/d4d/f53 x:0 0 0 2024-02-16T22:21:49.168 INFO:tasks.workunit.client.1.smithi123.stdout:6/224: write d0/d6/d1a/f2b [1006185,29292] 0 2024-02-16T22:21:49.168 INFO:tasks.workunit.client.1.smithi123.stdout:0/195: dwrite d1/d8/d13/f20 [0,4194304] 0 2024-02-16T22:21:49.169 INFO:tasks.workunit.client.1.smithi123.stdout:0/196: readlink d1/d4/l31 0 2024-02-16T22:21:49.169 INFO:tasks.workunit.client.1.smithi123.stdout:6/225: fdatasync d0/d6/d2f/f22 0 2024-02-16T22:21:49.170 INFO:tasks.workunit.client.1.smithi123.stdout:6/226: read d0/d6/f8 [534892,77070] 0 2024-02-16T22:21:49.177 INFO:tasks.workunit.client.0.smithi114.stdout:4/211: link dd/d1b/f47 dd/f4b 0 2024-02-16T22:21:49.177 INFO:tasks.workunit.client.1.smithi123.stdout:6/227: dread d0/d6/d1a/f2b [0,4194304] 0 2024-02-16T22:21:49.178 INFO:tasks.workunit.client.0.smithi114.stdout:0/170: dwrite d1/f15 [0,4194304] 0 2024-02-16T22:21:49.178 INFO:tasks.workunit.client.1.smithi123.stdout:4/228: truncate d5/f29 3008505 0 2024-02-16T22:21:49.178 INFO:tasks.workunit.client.1.smithi123.stdout:4/229: stat d5/f40 0 2024-02-16T22:21:49.181 INFO:tasks.workunit.client.1.smithi123.stdout:7/199: mknod d3/d12/d1d/d1e/c46 0 2024-02-16T22:21:49.181 INFO:tasks.workunit.client.1.smithi123.stdout:7/200: dread - d3/d12/d34/f38 zero size 2024-02-16T22:21:49.182 INFO:tasks.workunit.client.0.smithi114.stdout:1/114: write d4/d9/f11 [920830,85870] 0 2024-02-16T22:21:49.182 INFO:tasks.workunit.client.0.smithi114.stdout:5/116: sync 2024-02-16T22:21:49.184 INFO:tasks.workunit.client.0.smithi114.stdout:0/171: dread d1/d16/d1f/d28/f35 [0,4194304] 0 2024-02-16T22:21:49.185 INFO:tasks.workunit.client.0.smithi114.stdout:7/125: link d3/d14/f1c d3/f26 0 2024-02-16T22:21:49.186 INFO:tasks.workunit.client.1.smithi123.stdout:8/258: truncate d0/df/f16 1754910 0 2024-02-16T22:21:49.186 INFO:tasks.workunit.client.0.smithi114.stdout:0/172: read d1/fa [19828,67829] 0 2024-02-16T22:21:49.187 INFO:tasks.workunit.client.1.smithi123.stdout:3/178: creat d0/d20/d2e/f39 x:0 0 0 2024-02-16T22:21:49.191 INFO:tasks.workunit.client.0.smithi114.stdout:4/212: rename dd/d1b/f2c to dd/d1b/d24/f4c 0 2024-02-16T22:21:49.192 INFO:tasks.workunit.client.0.smithi114.stdout:4/213: write dd/d1b/d24/f3d [341670,6417] 0 2024-02-16T22:21:49.200 INFO:tasks.workunit.client.0.smithi114.stdout:5/117: creat d0/df/d16/f1f x:0 0 0 2024-02-16T22:21:49.200 INFO:tasks.workunit.client.0.smithi114.stdout:5/118: readlink d0/l6 0 2024-02-16T22:21:49.201 INFO:tasks.workunit.client.1.smithi123.stdout:9/273: creat d4/d17/f55 x:0 0 0 2024-02-16T22:21:49.201 INFO:tasks.workunit.client.0.smithi114.stdout:5/119: write d0/df/d16/f1f [949094,13055] 0 2024-02-16T22:21:49.202 INFO:tasks.workunit.client.0.smithi114.stdout:2/164: link f4 dd/df/f33 0 2024-02-16T22:21:49.203 INFO:tasks.workunit.client.0.smithi114.stdout:5/120: write d0/fa [1532148,59998] 0 2024-02-16T22:21:49.208 INFO:tasks.workunit.client.1.smithi123.stdout:8/259: dread d0/d17/d26/f2d [0,4194304] 0 2024-02-16T22:21:49.209 INFO:tasks.workunit.client.1.smithi123.stdout:8/260: write d0/f14 [7645202,49140] 0 2024-02-16T22:21:49.211 INFO:tasks.workunit.client.1.smithi123.stdout:1/187: rename d3/d6/db/d10/l3c to d3/d6/db/l40 0 2024-02-16T22:21:49.212 INFO:tasks.workunit.client.0.smithi114.stdout:6/119: truncate d6/f18 2184631 0 2024-02-16T22:21:49.212 INFO:tasks.workunit.client.0.smithi114.stdout:6/120: chown d6/dd/c12 3324 1 2024-02-16T22:21:49.213 INFO:tasks.workunit.client.1.smithi123.stdout:7/201: symlink d3/l47 0 2024-02-16T22:21:49.215 INFO:tasks.workunit.client.0.smithi114.stdout:4/214: symlink dd/d1b/d37/d38/l4d 0 2024-02-16T22:21:49.217 INFO:tasks.workunit.client.1.smithi123.stdout:2/221: link d4/d1a/d1d/f17 d4/d1a/d29/d45/f4b 0 2024-02-16T22:21:49.218 INFO:tasks.workunit.client.1.smithi123.stdout:0/197: dread d1/f6 [4194304,4194304] 0 2024-02-16T22:21:49.218 INFO:tasks.workunit.client.1.smithi123.stdout:2/222: stat d4/d1a/d1d/d13/d15/d1b/c2a 0 2024-02-16T22:21:49.219 INFO:tasks.workunit.client.0.smithi114.stdout:1/115: dwrite d4/d9/f19 [0,4194304] 0 2024-02-16T22:21:49.220 INFO:tasks.workunit.client.1.smithi123.stdout:0/198: read d1/d8/f30 [355643,45592] 0 2024-02-16T22:21:49.221 INFO:tasks.workunit.client.0.smithi114.stdout:2/165: creat dd/df/d2c/f34 x:0 0 0 2024-02-16T22:21:49.225 INFO:tasks.workunit.client.0.smithi114.stdout:0/173: mknod d1/d30/c39 0 2024-02-16T22:21:49.227 INFO:tasks.workunit.client.0.smithi114.stdout:6/121: creat d6/dd/d25/f27 x:0 0 0 2024-02-16T22:21:49.231 INFO:tasks.workunit.client.0.smithi114.stdout:1/116: read d4/d9/f10 [167273,24332] 0 2024-02-16T22:21:49.233 INFO:tasks.workunit.client.1.smithi123.stdout:5/168: rename d6/f11 to d6/dd/df/d1f/d25/f39 0 2024-02-16T22:21:49.236 INFO:tasks.workunit.client.1.smithi123.stdout:1/188: mknod d3/d6/db/d10/c41 0 2024-02-16T22:21:49.238 INFO:tasks.workunit.client.0.smithi114.stdout:1/117: dwrite d4/d9/f19 [0,4194304] 0 2024-02-16T22:21:49.239 INFO:tasks.workunit.client.0.smithi114.stdout:5/121: mkdir d0/df/d16/d1e/d20 0 2024-02-16T22:21:49.240 INFO:tasks.workunit.client.0.smithi114.stdout:5/122: write d0/df/f1d [758887,70824] 0 2024-02-16T22:21:49.241 INFO:tasks.workunit.client.0.smithi114.stdout:5/123: read d0/d10/f13 [1155720,46565] 0 2024-02-16T22:21:49.243 INFO:tasks.workunit.client.0.smithi114.stdout:0/174: mknod d1/d16/d18/c3a 0 2024-02-16T22:21:49.246 INFO:tasks.workunit.client.0.smithi114.stdout:6/122: truncate d6/dd/f19 577323 0 2024-02-16T22:21:49.250 INFO:tasks.workunit.client.1.smithi123.stdout:3/179: mknod d0/d20/d29/d34/d37/c3a 0 2024-02-16T22:21:49.251 INFO:tasks.workunit.client.0.smithi114.stdout:2/166: symlink dd/df/d1c/l35 0 2024-02-16T22:21:49.253 INFO:tasks.workunit.client.0.smithi114.stdout:1/118: creat d4/d9/f22 x:0 0 0 2024-02-16T22:21:49.256 INFO:tasks.workunit.client.1.smithi123.stdout:0/199: dwrite d1/d4/d17/f32 [0,4194304] 0 2024-02-16T22:21:49.257 INFO:tasks.workunit.client.0.smithi114.stdout:7/126: rename d3/d14/f1c to d3/f27 0 2024-02-16T22:21:49.257 INFO:tasks.workunit.client.1.smithi123.stdout:0/200: chown d1/d4/d17/f28 62 1 2024-02-16T22:21:49.261 INFO:tasks.workunit.client.0.smithi114.stdout:5/124: rmdir d0/df/d16/d1e 39 2024-02-16T22:21:49.262 INFO:tasks.workunit.client.1.smithi123.stdout:6/228: creat d0/f54 x:0 0 0 2024-02-16T22:21:49.263 INFO:tasks.workunit.client.0.smithi114.stdout:0/175: creat d1/d1a/f3b x:0 0 0 2024-02-16T22:21:49.264 INFO:tasks.workunit.client.1.smithi123.stdout:8/261: symlink d0/d17/d26/d2a/d59/l5d 0 2024-02-16T22:21:49.265 INFO:tasks.workunit.client.1.smithi123.stdout:8/262: dread - d0/d17/d26/d2a/f54 zero size 2024-02-16T22:21:49.265 INFO:tasks.workunit.client.0.smithi114.stdout:6/123: symlink d6/da/l28 0 2024-02-16T22:21:49.265 INFO:tasks.workunit.client.1.smithi123.stdout:8/263: dread - d0/d17/d43/d34/f38 zero size 2024-02-16T22:21:49.268 INFO:tasks.workunit.client.0.smithi114.stdout:0/176: dread d1/d16/f17 [0,4194304] 0 2024-02-16T22:21:49.268 INFO:tasks.workunit.client.0.smithi114.stdout:1/119: symlink d4/d9/l23 0 2024-02-16T22:21:49.269 INFO:tasks.workunit.client.0.smithi114.stdout:0/177: chown d1/d16/f24 1111671 1 2024-02-16T22:21:49.269 INFO:tasks.workunit.client.0.smithi114.stdout:4/215: truncate dd/d1b/d24/f4c 3382525 0 2024-02-16T22:21:49.270 INFO:tasks.workunit.client.1.smithi123.stdout:8/264: dwrite d0/d17/f35 [0,4194304] 0 2024-02-16T22:21:49.271 INFO:tasks.workunit.client.0.smithi114.stdout:4/216: truncate dd/d1b/f3e 788556 0 2024-02-16T22:21:49.271 INFO:tasks.workunit.client.1.smithi123.stdout:5/169: fsync d6/dd/df/d1f/f24 0 2024-02-16T22:21:49.272 INFO:tasks.workunit.client.1.smithi123.stdout:1/189: mkdir d3/d6/db/d10/d2d/d42 0 2024-02-16T22:21:49.274 INFO:tasks.workunit.client.0.smithi114.stdout:7/127: creat d3/d14/f28 x:0 0 0 2024-02-16T22:21:49.275 INFO:tasks.workunit.client.1.smithi123.stdout:7/202: getdents d3/d26/d3d 0 2024-02-16T22:21:49.275 INFO:tasks.workunit.client.0.smithi114.stdout:7/128: read d3/d14/f18 [1239265,107677] 0 2024-02-16T22:21:49.276 INFO:tasks.workunit.client.1.smithi123.stdout:7/203: write d3/d5/f33 [264452,100856] 0 2024-02-16T22:21:49.279 INFO:tasks.workunit.client.0.smithi114.stdout:4/217: dwrite dd/f20 [0,4194304] 0 2024-02-16T22:21:49.285 INFO:tasks.workunit.client.0.smithi114.stdout:7/129: dwrite d3/d14/f28 [0,4194304] 0 2024-02-16T22:21:49.288 INFO:tasks.workunit.client.0.smithi114.stdout:4/218: dwrite dd/f13 [0,4194304] 0 2024-02-16T22:21:49.290 INFO:tasks.workunit.client.0.smithi114.stdout:9/154: sync 2024-02-16T22:21:49.291 INFO:tasks.workunit.client.0.smithi114.stdout:9/155: fdatasync d2/dd/d1c/f31 0 2024-02-16T22:21:49.295 INFO:tasks.workunit.client.0.smithi114.stdout:0/178: mknod d1/d1a/c3c 0 2024-02-16T22:21:49.298 INFO:tasks.workunit.client.1.smithi123.stdout:4/230: rename d5/d1c/d23/l28 to d5/d1c/d23/d36/l48 0 2024-02-16T22:21:49.301 INFO:tasks.workunit.client.1.smithi123.stdout:8/265: readlink d0/l2 0 2024-02-16T22:21:49.302 INFO:tasks.workunit.client.1.smithi123.stdout:8/266: write d0/d17/d26/d2a/f54 [878390,93850] 0 2024-02-16T22:21:49.305 INFO:tasks.workunit.client.1.smithi123.stdout:5/170: creat d6/dd/df/d1f/d25/f3a x:0 0 0 2024-02-16T22:21:49.305 INFO:tasks.workunit.client.1.smithi123.stdout:5/171: chown d6/dd/df/d1f/d25/f1a 27319728 1 2024-02-16T22:21:49.306 INFO:tasks.workunit.client.1.smithi123.stdout:6/229: write d0/d6/f1e [5456442,38065] 0 2024-02-16T22:21:49.306 INFO:tasks.workunit.client.1.smithi123.stdout:1/190: symlink d3/d6/d18/l43 0 2024-02-16T22:21:49.309 INFO:tasks.workunit.client.1.smithi123.stdout:7/204: dread d3/d12/f1a [0,4194304] 0 2024-02-16T22:21:49.310 INFO:tasks.workunit.client.0.smithi114.stdout:5/125: creat d0/df/d16/d1e/f21 x:0 0 0 2024-02-16T22:21:49.313 INFO:tasks.workunit.client.0.smithi114.stdout:7/130: creat d3/d14/f29 x:0 0 0 2024-02-16T22:21:49.314 INFO:tasks.workunit.client.0.smithi114.stdout:5/126: dread d0/f8 [0,4194304] 0 2024-02-16T22:21:49.315 INFO:tasks.workunit.client.1.smithi123.stdout:9/274: getdents d4/d36 0 2024-02-16T22:21:49.317 INFO:tasks.workunit.client.0.smithi114.stdout:6/124: truncate d6/dd/f1d 1641122 0 2024-02-16T22:21:49.318 INFO:tasks.workunit.client.1.smithi123.stdout:4/231: write d5/d1c/d23/d31/f2d [1807758,76342] 0 2024-02-16T22:21:49.319 INFO:tasks.workunit.client.1.smithi123.stdout:4/232: stat d5/le 0 2024-02-16T22:21:49.319 INFO:tasks.workunit.client.0.smithi114.stdout:4/219: creat dd/d1b/d24/f4e x:0 0 0 2024-02-16T22:21:49.319 INFO:tasks.workunit.client.0.smithi114.stdout:4/220: fsync dd/f32 0 2024-02-16T22:21:49.320 INFO:tasks.workunit.client.0.smithi114.stdout:4/221: fdatasync dd/f2b 0 2024-02-16T22:21:49.323 INFO:tasks.workunit.client.1.smithi123.stdout:6/230: mknod d0/d6/d4d/c55 0 2024-02-16T22:21:49.324 INFO:tasks.workunit.client.0.smithi114.stdout:2/167: rename f4 to dd/f36 0 2024-02-16T22:21:49.324 INFO:tasks.workunit.client.1.smithi123.stdout:1/191: rename d3/d6/db to d3/d6/db/d44 22 2024-02-16T22:21:49.326 INFO:tasks.workunit.client.1.smithi123.stdout:1/192: write d3/d6/d18/f37 [4910402,98530] 0 2024-02-16T22:21:49.327 INFO:tasks.workunit.client.1.smithi123.stdout:5/172: dwrite d6/dd/f35 [4194304,4194304] 0 2024-02-16T22:21:49.329 INFO:tasks.workunit.client.1.smithi123.stdout:5/173: write d6/dd/df/d1f/d25/f3a [486928,29885] 0 2024-02-16T22:21:49.329 INFO:tasks.workunit.client.1.smithi123.stdout:1/193: fsync d3/d6/d18/f37 0 2024-02-16T22:21:49.330 INFO:tasks.workunit.client.1.smithi123.stdout:7/205: creat d3/d12/d1d/d1e/f48 x:0 0 0 2024-02-16T22:21:49.330 INFO:tasks.workunit.client.1.smithi123.stdout:7/206: chown d3/d5 85539 1 2024-02-16T22:21:49.331 INFO:tasks.workunit.client.0.smithi114.stdout:7/131: symlink d3/d14/l2a 0 2024-02-16T22:21:49.332 INFO:tasks.workunit.client.1.smithi123.stdout:7/207: dread - d3/d12/d1d/f41 zero size 2024-02-16T22:21:49.333 INFO:tasks.workunit.client.1.smithi123.stdout:2/223: getdents d4 0 2024-02-16T22:21:49.334 INFO:tasks.workunit.client.0.smithi114.stdout:6/125: creat d6/dd/f29 x:0 0 0 2024-02-16T22:21:49.336 INFO:tasks.workunit.client.1.smithi123.stdout:7/208: dread d3/d12/d34/f3c [0,4194304] 0 2024-02-16T22:21:49.336 INFO:tasks.workunit.client.1.smithi123.stdout:7/209: chown f2 22005 1 2024-02-16T22:21:49.337 INFO:tasks.workunit.client.1.smithi123.stdout:7/210: write d3/d26/d35/f36 [992687,11063] 0 2024-02-16T22:21:49.338 INFO:tasks.workunit.client.1.smithi123.stdout:7/211: truncate d3/d12/d34/f38 241400 0 2024-02-16T22:21:49.338 INFO:tasks.workunit.client.1.smithi123.stdout:7/212: readlink d3/d26/l3a 0 2024-02-16T22:21:49.339 INFO:tasks.workunit.client.1.smithi123.stdout:7/213: chown d3/d12/d24/l30 149 1 2024-02-16T22:21:49.339 INFO:tasks.workunit.client.1.smithi123.stdout:4/233: mkdir d5/d49 0 2024-02-16T22:21:49.340 INFO:tasks.workunit.client.1.smithi123.stdout:7/214: fdatasync d3/d5/f9 0 2024-02-16T22:21:49.340 INFO:tasks.workunit.client.0.smithi114.stdout:4/222: mknod dd/d1b/d41/c4f 0 2024-02-16T22:21:49.341 INFO:tasks.workunit.client.1.smithi123.stdout:8/267: mknod d0/c5e 0 2024-02-16T22:21:49.341 INFO:tasks.workunit.client.0.smithi114.stdout:4/223: write dd/d2e/d3f/f43 [2166860,23615] 0 2024-02-16T22:21:49.342 INFO:tasks.workunit.client.0.smithi114.stdout:4/224: fsync dd/d1b/d24/f3d 0 2024-02-16T22:21:49.345 INFO:tasks.workunit.client.1.smithi123.stdout:4/234: dwrite d5/d1c/d23/d31/f2d [0,4194304] 0 2024-02-16T22:21:49.351 INFO:tasks.workunit.client.0.smithi114.stdout:9/156: truncate d2/d1a/d23/f2c 2671009 0 2024-02-16T22:21:49.352 INFO:tasks.workunit.client.0.smithi114.stdout:7/132: creat d3/d22/f2b x:0 0 0 2024-02-16T22:21:49.355 INFO:tasks.workunit.client.1.smithi123.stdout:1/194: rename d3/d6/db/d10/c15 to d3/d6/c45 0 2024-02-16T22:21:49.357 INFO:tasks.workunit.client.0.smithi114.stdout:7/133: dread d3/ff [0,4194304] 0 2024-02-16T22:21:49.357 INFO:tasks.workunit.client.0.smithi114.stdout:9/157: dread d2/dd/ff [0,4194304] 0 2024-02-16T22:21:49.357 INFO:tasks.workunit.client.0.smithi114.stdout:7/134: chown d3/d14/d1b/l1e 16323396 1 2024-02-16T22:21:49.361 INFO:tasks.workunit.client.1.smithi123.stdout:3/180: link d0/c1a d0/d20/c3b 0 2024-02-16T22:21:49.361 INFO:tasks.workunit.client.0.smithi114.stdout:1/120: link d4/d9/c1e d4/c24 0 2024-02-16T22:21:49.362 INFO:tasks.workunit.client.1.smithi123.stdout:0/201: getdents d1/d4 0 2024-02-16T22:21:49.363 INFO:tasks.workunit.client.1.smithi123.stdout:9/275: fdatasync d4/f6 0 2024-02-16T22:21:49.364 INFO:tasks.workunit.client.0.smithi114.stdout:1/121: dread - d4/d9/f20 zero size 2024-02-16T22:21:49.364 INFO:tasks.workunit.client.1.smithi123.stdout:9/276: stat d4/d17/d2b 0 2024-02-16T22:21:49.366 INFO:tasks.workunit.client.0.smithi114.stdout:0/179: creat d1/f3d x:0 0 0 2024-02-16T22:21:49.366 INFO:tasks.workunit.client.0.smithi114.stdout:2/168: dwrite f0 [4194304,4194304] 0 2024-02-16T22:21:49.366 INFO:tasks.workunit.client.0.smithi114.stdout:9/158: dwrite d2/dd/f34 [0,4194304] 0 2024-02-16T22:21:49.368 INFO:tasks.workunit.client.0.smithi114.stdout:0/180: truncate d1/d1a/f36 741708 0 2024-02-16T22:21:49.369 INFO:tasks.workunit.client.0.smithi114.stdout:9/159: read d2/fc [50606,107871] 0 2024-02-16T22:21:49.370 INFO:tasks.workunit.client.0.smithi114.stdout:0/181: readlink d1/d16/d1f/d28/l2f 0 2024-02-16T22:21:49.375 INFO:tasks.workunit.client.1.smithi123.stdout:8/268: creat d0/d17/d26/d32/d45/f5f x:0 0 0 2024-02-16T22:21:49.378 INFO:tasks.workunit.client.0.smithi114.stdout:5/127: link d0/df/f1d d0/f22 0 2024-02-16T22:21:49.380 INFO:tasks.workunit.client.0.smithi114.stdout:7/135: readlink d3/d14/d1b/l20 0 2024-02-16T22:21:49.381 INFO:tasks.workunit.client.0.smithi114.stdout:7/136: stat d3/d22 0 2024-02-16T22:21:49.381 INFO:tasks.workunit.client.0.smithi114.stdout:7/137: fsync d3/f6 0 2024-02-16T22:21:49.381 INFO:tasks.workunit.client.1.smithi123.stdout:5/174: mkdir d6/d3b 0 2024-02-16T22:21:49.382 INFO:tasks.workunit.client.0.smithi114.stdout:7/138: truncate d3/d14/f29 411432 0 2024-02-16T22:21:49.382 INFO:tasks.workunit.client.1.smithi123.stdout:5/175: chown d6/f21 245 1 2024-02-16T22:21:49.383 INFO:tasks.workunit.client.0.smithi114.stdout:7/139: chown d3/c21 6287381 1 2024-02-16T22:21:49.384 INFO:tasks.workunit.client.1.smithi123.stdout:7/215: write d3/d12/f1a [300865,26974] 0 2024-02-16T22:21:49.385 INFO:tasks.workunit.client.1.smithi123.stdout:7/216: chown d3/d12/d34/l3f 361730 1 2024-02-16T22:21:49.388 INFO:tasks.workunit.client.1.smithi123.stdout:7/217: dread d3/d12/d24/f31 [0,4194304] 0 2024-02-16T22:21:49.388 INFO:tasks.workunit.client.1.smithi123.stdout:7/218: stat d3/c20 0 2024-02-16T22:21:49.391 INFO:tasks.workunit.client.1.smithi123.stdout:3/181: creat d0/d20/d29/f3c x:0 0 0 2024-02-16T22:21:49.394 INFO:tasks.workunit.client.1.smithi123.stdout:7/219: dread d3/fe [4194304,4194304] 0 2024-02-16T22:21:49.395 INFO:tasks.workunit.client.1.smithi123.stdout:7/220: truncate d3/d5/f45 540145 0 2024-02-16T22:21:49.395 INFO:tasks.workunit.client.0.smithi114.stdout:6/126: write d6/f18 [2992039,37624] 0 2024-02-16T22:21:49.396 INFO:tasks.workunit.client.1.smithi123.stdout:9/277: rmdir d4/d17/d2b 39 2024-02-16T22:21:49.397 INFO:tasks.workunit.client.0.smithi114.stdout:0/182: rmdir d1/d30 39 2024-02-16T22:21:49.398 INFO:tasks.workunit.client.0.smithi114.stdout:9/160: truncate d2/f15 520861 0 2024-02-16T22:21:49.399 INFO:tasks.workunit.client.0.smithi114.stdout:8/155: sync 2024-02-16T22:21:49.399 INFO:tasks.workunit.client.0.smithi114.stdout:3/113: sync 2024-02-16T22:21:49.400 INFO:tasks.workunit.client.1.smithi123.stdout:8/269: rename d0/d12/f22 to d0/d29/f60 0 2024-02-16T22:21:49.400 INFO:tasks.workunit.client.0.smithi114.stdout:4/225: truncate dd/f20 1632210 0 2024-02-16T22:21:49.401 INFO:tasks.workunit.client.1.smithi123.stdout:2/224: dwrite d4/d1a/d29/f3a [0,4194304] 0 2024-02-16T22:21:49.402 INFO:tasks.workunit.client.1.smithi123.stdout:6/231: link d0/d6/d2f/l32 d0/d6/l56 0 2024-02-16T22:21:49.403 INFO:tasks.workunit.client.0.smithi114.stdout:5/128: creat d0/d10/f23 x:0 0 0 2024-02-16T22:21:49.404 INFO:tasks.workunit.client.1.smithi123.stdout:4/235: creat d5/d1c/d42/f4a x:0 0 0 2024-02-16T22:21:49.405 INFO:tasks.workunit.client.0.smithi114.stdout:7/140: fdatasync d3/f10 0 2024-02-16T22:21:49.405 INFO:tasks.workunit.client.1.smithi123.stdout:2/225: truncate d4/d1a/d1d/f2c 1201958 0 2024-02-16T22:21:49.407 INFO:tasks.workunit.client.0.smithi114.stdout:8/156: dwrite d4/f19 [0,4194304] 0 2024-02-16T22:21:49.409 INFO:tasks.workunit.client.1.smithi123.stdout:8/270: dwrite d0/f14 [4194304,4194304] 0 2024-02-16T22:21:49.409 INFO:tasks.workunit.client.1.smithi123.stdout:2/226: chown d4/d1a/d29/d45/c12 2065 1 2024-02-16T22:21:49.412 INFO:tasks.workunit.client.1.smithi123.stdout:2/227: write d4/d1a/f39 [2115281,63526] 0 2024-02-16T22:21:49.416 INFO:tasks.workunit.client.1.smithi123.stdout:3/182: creat d0/d20/d29/f3d x:0 0 0 2024-02-16T22:21:49.416 INFO:tasks.workunit.client.1.smithi123.stdout:6/232: dwrite d0/d9/dc/d3a/f40 [0,4194304] 0 2024-02-16T22:21:49.416 INFO:tasks.workunit.client.1.smithi123.stdout:3/183: write d0/f1f [866262,38647] 0 2024-02-16T22:21:49.418 INFO:tasks.workunit.client.0.smithi114.stdout:6/127: dread d6/dd/f1d [0,4194304] 0 2024-02-16T22:21:49.418 INFO:tasks.workunit.client.1.smithi123.stdout:3/184: truncate d0/f19 734627 0 2024-02-16T22:21:49.418 INFO:tasks.workunit.client.1.smithi123.stdout:0/202: unlink d1/c27 0 2024-02-16T22:21:49.419 INFO:tasks.workunit.client.0.smithi114.stdout:0/183: mkdir d1/d1a/d3e 0 2024-02-16T22:21:49.421 INFO:tasks.workunit.client.1.smithi123.stdout:6/233: chown d0/d9/dc/d39/l43 31 1 2024-02-16T22:21:49.422 INFO:tasks.workunit.client.1.smithi123.stdout:7/221: creat d3/d12/d24/f49 x:0 0 0 2024-02-16T22:21:49.425 INFO:tasks.workunit.client.1.smithi123.stdout:2/228: dwrite d4/d1a/d1d/d13/d25/f43 [0,4194304] 0 2024-02-16T22:21:49.427 INFO:tasks.workunit.client.1.smithi123.stdout:9/278: rmdir d4 39 2024-02-16T22:21:49.429 INFO:tasks.workunit.client.0.smithi114.stdout:3/114: creat d4/dd/f29 x:0 0 0 2024-02-16T22:21:49.433 INFO:tasks.workunit.client.0.smithi114.stdout:5/129: unlink d0/c17 0 2024-02-16T22:21:49.434 INFO:tasks.workunit.client.1.smithi123.stdout:5/176: symlink d6/d3b/l3c 0 2024-02-16T22:21:49.435 INFO:tasks.workunit.client.0.smithi114.stdout:4/226: dread dd/f19 [0,4194304] 0 2024-02-16T22:21:49.436 INFO:tasks.workunit.client.0.smithi114.stdout:7/141: creat d3/d14/f2c x:0 0 0 2024-02-16T22:21:49.437 INFO:tasks.workunit.client.0.smithi114.stdout:4/227: write dd/d1b/f1d [811281,71851] 0 2024-02-16T22:21:49.437 INFO:tasks.workunit.client.1.smithi123.stdout:5/177: dread d6/f21 [0,4194304] 0 2024-02-16T22:21:49.438 INFO:tasks.workunit.client.0.smithi114.stdout:5/130: dread d0/df/f1d [0,4194304] 0 2024-02-16T22:21:49.438 INFO:tasks.workunit.client.0.smithi114.stdout:1/122: link d4/d9/f11 d4/f25 0 2024-02-16T22:21:49.440 INFO:tasks.workunit.client.0.smithi114.stdout:5/131: fdatasync d0/df/d16/f1f 0 2024-02-16T22:21:49.442 INFO:tasks.workunit.client.0.smithi114.stdout:2/169: creat dd/df/f37 x:0 0 0 2024-02-16T22:21:49.443 INFO:tasks.workunit.client.0.smithi114.stdout:7/142: dread d3/ff [0,4194304] 0 2024-02-16T22:21:49.444 INFO:tasks.workunit.client.0.smithi114.stdout:9/161: dread d2/fc [0,4194304] 0 2024-02-16T22:21:49.445 INFO:tasks.workunit.client.0.smithi114.stdout:6/128: creat d6/dd/d25/f2a x:0 0 0 2024-02-16T22:21:49.446 INFO:tasks.workunit.client.1.smithi123.stdout:3/185: symlink d0/d20/d2e/l3e 0 2024-02-16T22:21:49.447 INFO:tasks.workunit.client.0.smithi114.stdout:6/129: dread - d6/dd/d25/f2a zero size 2024-02-16T22:21:49.449 INFO:tasks.workunit.client.0.smithi114.stdout:1/123: dwrite d4/f21 [0,4194304] 0 2024-02-16T22:21:49.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:49 smithi114 ceph-mon[29235]: Upgrade: Updating mgr.smithi114.vcullz 2024-02-16T22:21:49.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:49 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:49.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:49 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi114.vcullz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:21:49.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:49 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi114.vcullz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:21:49.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:49 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T22:21:49.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:49 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:21:49.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:49 smithi114 ceph-mon[29235]: Deploying daemon mgr.smithi114.vcullz on smithi114 2024-02-16T22:21:49.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:49 smithi114 ceph-mon[29235]: pgmap v22: 65 pgs: 65 active+clean; 606 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 64 MiB/s wr, 375 op/s 2024-02-16T22:21:49.452 INFO:tasks.workunit.client.1.smithi123.stdout:7/222: creat d3/d12/d1d/d1e/f4a x:0 0 0 2024-02-16T22:21:49.453 INFO:tasks.workunit.client.0.smithi114.stdout:3/115: mknod d4/c2a 0 2024-02-16T22:21:49.453 INFO:tasks.workunit.client.0.smithi114.stdout:3/116: chown d4/dd 0 1 2024-02-16T22:21:49.454 INFO:tasks.workunit.client.0.smithi114.stdout:3/117: truncate d4/dd/f15 524495 0 2024-02-16T22:21:49.457 INFO:tasks.workunit.client.1.smithi123.stdout:1/195: creat d3/d6/f46 x:0 0 0 2024-02-16T22:21:49.463 INFO:tasks.workunit.client.1.smithi123.stdout:4/236: link d5/d1c/d23/d36/f39 d5/d49/f4b 0 2024-02-16T22:21:49.464 INFO:tasks.workunit.client.1.smithi123.stdout:4/237: write d5/d1c/d42/f4a [286829,54573] 0 2024-02-16T22:21:49.465 INFO:tasks.workunit.client.1.smithi123.stdout:8/271: mkdir d0/d61 0 2024-02-16T22:21:49.465 INFO:tasks.workunit.client.1.smithi123.stdout:8/272: read - d0/d17/d26/d32/f56 zero size 2024-02-16T22:21:49.466 INFO:tasks.workunit.client.1.smithi123.stdout:3/186: mknod d0/d20/d29/d34/d37/c3f 0 2024-02-16T22:21:49.468 INFO:tasks.workunit.client.1.smithi123.stdout:0/203: mknod d1/d8/c35 0 2024-02-16T22:21:49.471 INFO:tasks.workunit.client.0.smithi114.stdout:4/228: rename dd/d2e/d3f/f43 to dd/d2e/d3f/f50 0 2024-02-16T22:21:49.472 INFO:tasks.workunit.client.0.smithi114.stdout:2/170: mknod dd/df/d1c/d21/c38 0 2024-02-16T22:21:49.472 INFO:tasks.workunit.client.1.smithi123.stdout:7/223: dread d3/fd [0,4194304] 0 2024-02-16T22:21:49.475 INFO:tasks.workunit.client.1.smithi123.stdout:6/234: truncate d0/d6/d2f/f2a 52349 0 2024-02-16T22:21:49.478 INFO:tasks.workunit.client.0.smithi114.stdout:7/143: rename d3/c9 to d3/d14/c2d 0 2024-02-16T22:21:49.483 INFO:tasks.workunit.client.0.smithi114.stdout:2/171: dread fc [0,4194304] 0 2024-02-16T22:21:49.485 INFO:tasks.workunit.client.0.smithi114.stdout:0/184: truncate d1/d16/d1f/d28/f2d 1215745 0 2024-02-16T22:21:49.485 INFO:tasks.workunit.client.0.smithi114.stdout:2/172: read dd/df/f1e [3111669,120332] 0 2024-02-16T22:21:49.486 INFO:tasks.workunit.client.1.smithi123.stdout:8/273: dread d0/d17/d43/f27 [0,4194304] 0 2024-02-16T22:21:49.487 INFO:tasks.workunit.client.1.smithi123.stdout:0/204: mknod d1/d4/c36 0 2024-02-16T22:21:49.492 INFO:tasks.workunit.client.0.smithi114.stdout:9/162: dwrite d2/f5 [4194304,4194304] 0 2024-02-16T22:21:49.492 INFO:tasks.workunit.client.1.smithi123.stdout:4/238: dread d5/d1c/f35 [0,4194304] 0 2024-02-16T22:21:49.493 INFO:tasks.workunit.client.0.smithi114.stdout:9/163: write d2/f4 [8782704,43605] 0 2024-02-16T22:21:49.494 INFO:tasks.workunit.client.1.smithi123.stdout:6/235: mknod d0/d6/d2f/c57 0 2024-02-16T22:21:49.494 INFO:tasks.workunit.client.0.smithi114.stdout:3/118: write d4/dd/f19 [4724795,53697] 0 2024-02-16T22:21:49.494 INFO:tasks.workunit.client.0.smithi114.stdout:9/164: write f1 [4853805,122905] 0 2024-02-16T22:21:49.495 INFO:tasks.workunit.client.0.smithi114.stdout:3/119: chown d4/f1c 264231301 1 2024-02-16T22:21:49.496 INFO:tasks.workunit.client.0.smithi114.stdout:9/165: fsync d2/dd/d1c/f27 0 2024-02-16T22:21:49.496 INFO:tasks.workunit.client.0.smithi114.stdout:4/229: mknod dd/d1b/d37/c51 0 2024-02-16T22:21:49.497 INFO:tasks.workunit.client.0.smithi114.stdout:9/166: read d2/dd/f34 [2210521,21622] 0 2024-02-16T22:21:49.499 INFO:tasks.workunit.client.1.smithi123.stdout:3/187: mkdir d0/d40 0 2024-02-16T22:21:49.499 INFO:tasks.workunit.client.1.smithi123.stdout:6/236: dread d0/d6/f18 [0,4194304] 0 2024-02-16T22:21:49.501 INFO:tasks.workunit.client.1.smithi123.stdout:6/237: write d0/f38 [123852,52771] 0 2024-02-16T22:21:49.503 INFO:tasks.workunit.client.1.smithi123.stdout:0/205: dread d1/d8/f30 [0,4194304] 0 2024-02-16T22:21:49.505 INFO:tasks.workunit.client.0.smithi114.stdout:4/230: dread dd/d1b/f3e [0,4194304] 0 2024-02-16T22:21:49.506 INFO:tasks.workunit.client.1.smithi123.stdout:4/239: unlink d5/f19 0 2024-02-16T22:21:49.506 INFO:tasks.workunit.client.0.smithi114.stdout:9/167: dwrite d2/f8 [0,4194304] 0 2024-02-16T22:21:49.507 INFO:tasks.workunit.client.0.smithi114.stdout:0/185: mknod d1/d16/d1f/d28/c3f 0 2024-02-16T22:21:49.508 INFO:tasks.workunit.client.0.smithi114.stdout:0/186: chown d1/d1a/f38 0 1 2024-02-16T22:21:49.509 INFO:tasks.workunit.client.0.smithi114.stdout:0/187: read d1/d16/f19 [764900,12560] 0 2024-02-16T22:21:49.512 INFO:tasks.workunit.client.1.smithi123.stdout:5/178: link d6/l14 d6/dd/df/d1f/d25/d37/l3d 0 2024-02-16T22:21:49.515 INFO:tasks.workunit.client.0.smithi114.stdout:8/157: link d4/l5 d4/d11/l2e 0 2024-02-16T22:21:49.516 INFO:tasks.workunit.client.1.smithi123.stdout:3/188: mkdir d0/d20/d29/d34/d37/d41 0 2024-02-16T22:21:49.517 INFO:tasks.workunit.client.1.smithi123.stdout:7/224: truncate d3/d5/f28 2076434 0 2024-02-16T22:21:49.518 INFO:tasks.workunit.client.1.smithi123.stdout:7/225: write d3/d26/d35/f36 [699389,125511] 0 2024-02-16T22:21:49.519 INFO:tasks.workunit.client.1.smithi123.stdout:6/238: fsync d0/d6/f8 0 2024-02-16T22:21:49.521 INFO:tasks.workunit.client.0.smithi114.stdout:6/130: dwrite d6/dd/f19 [0,4194304] 0 2024-02-16T22:21:49.521 INFO:tasks.workunit.client.0.smithi114.stdout:6/131: fsync d6/f23 0 2024-02-16T22:21:49.524 INFO:tasks.workunit.client.0.smithi114.stdout:5/132: creat d0/f24 x:0 0 0 2024-02-16T22:21:49.526 INFO:tasks.workunit.client.1.smithi123.stdout:0/206: readlink d1/d8/d13/l29 0 2024-02-16T22:21:49.526 INFO:tasks.workunit.client.0.smithi114.stdout:1/124: truncate d4/fa 2216340 0 2024-02-16T22:21:49.528 INFO:tasks.workunit.client.0.smithi114.stdout:3/120: unlink d4/l14 0 2024-02-16T22:21:49.532 INFO:tasks.workunit.client.1.smithi123.stdout:4/240: readlink d5/l33 0 2024-02-16T22:21:49.536 INFO:tasks.workunit.client.1.smithi123.stdout:5/179: dread d6/dd/df/f28 [0,4194304] 0 2024-02-16T22:21:49.537 INFO:tasks.workunit.client.0.smithi114.stdout:2/173: mknod dd/df/c39 0 2024-02-16T22:21:49.537 INFO:tasks.workunit.client.1.smithi123.stdout:4/241: dwrite d5/f17 [0,4194304] 0 2024-02-16T22:21:49.538 INFO:tasks.workunit.client.0.smithi114.stdout:2/174: truncate dd/df/f14 1040377 0 2024-02-16T22:21:49.538 INFO:tasks.workunit.client.1.smithi123.stdout:4/242: stat d5/d1c/c38 0 2024-02-16T22:21:49.538 INFO:tasks.workunit.client.1.smithi123.stdout:3/189: creat d0/d20/f42 x:0 0 0 2024-02-16T22:21:49.545 INFO:tasks.workunit.client.0.smithi114.stdout:9/168: rename d2/dd/d1c to d2/d1a/d23/d35 0 2024-02-16T22:21:49.547 INFO:tasks.workunit.client.1.smithi123.stdout:7/226: rename d3/d12/d34/f38 to d3/d26/d35/f4b 0 2024-02-16T22:21:49.549 INFO:tasks.workunit.client.0.smithi114.stdout:0/188: mknod d1/d1a/c40 0 2024-02-16T22:21:49.551 INFO:tasks.workunit.client.1.smithi123.stdout:6/239: unlink d0/d9/dc/f41 0 2024-02-16T22:21:49.552 INFO:tasks.workunit.client.0.smithi114.stdout:7/144: write d3/fe [6446,59113] 0 2024-02-16T22:21:49.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:49 smithi123 ceph-mon[37161]: Upgrade: Updating mgr.smithi114.vcullz 2024-02-16T22:21:49.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:49 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:21:49.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:49 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi114.vcullz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:21:49.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:49 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi114.vcullz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:21:49.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:49 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T22:21:49.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:49 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:21:49.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:49 smithi123 ceph-mon[37161]: Deploying daemon mgr.smithi114.vcullz on smithi114 2024-02-16T22:21:49.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:49 smithi123 ceph-mon[37161]: pgmap v22: 65 pgs: 65 active+clean; 606 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 64 MiB/s wr, 375 op/s 2024-02-16T22:21:49.553 INFO:tasks.workunit.client.1.smithi123.stdout:7/227: dwrite d3/d12/d1d/f41 [0,4194304] 0 2024-02-16T22:21:49.554 INFO:tasks.workunit.client.1.smithi123.stdout:2/229: sync 2024-02-16T22:21:49.555 INFO:tasks.workunit.client.0.smithi114.stdout:7/145: dread d3/ff [0,4194304] 0 2024-02-16T22:21:49.556 INFO:tasks.workunit.client.0.smithi114.stdout:7/146: truncate d3/ff 6138809 0 2024-02-16T22:21:49.559 INFO:tasks.workunit.client.1.smithi123.stdout:8/274: rmdir d0/df/d40 0 2024-02-16T22:21:49.560 INFO:tasks.workunit.client.0.smithi114.stdout:1/125: write d4/fd [3626361,195] 0 2024-02-16T22:21:49.562 INFO:tasks.workunit.client.1.smithi123.stdout:2/230: dread d4/f16 [0,4194304] 0 2024-02-16T22:21:49.564 INFO:tasks.workunit.client.1.smithi123.stdout:4/243: creat d5/f4c x:0 0 0 2024-02-16T22:21:49.566 INFO:tasks.workunit.client.1.smithi123.stdout:3/190: rmdir d0/d20/d29/d34/d37 39 2024-02-16T22:21:49.568 INFO:tasks.workunit.client.1.smithi123.stdout:2/231: dwrite d4/d1a/d1d/d13/d25/f3b [0,4194304] 0 2024-02-16T22:21:49.576 INFO:tasks.workunit.client.1.smithi123.stdout:5/180: dwrite d6/dd/df/f28 [0,4194304] 0 2024-02-16T22:21:49.576 INFO:tasks.workunit.client.1.smithi123.stdout:2/232: dwrite d4/d1a/d1d/f2b [0,4194304] 0 2024-02-16T22:21:49.581 INFO:tasks.workunit.client.1.smithi123.stdout:6/240: creat d0/d29/f58 x:0 0 0 2024-02-16T22:21:49.582 INFO:tasks.workunit.client.1.smithi123.stdout:4/244: dread d5/d1c/f21 [0,4194304] 0 2024-02-16T22:21:49.583 INFO:tasks.workunit.client.0.smithi114.stdout:6/132: symlink d6/dd/d25/d26/l2b 0 2024-02-16T22:21:49.584 INFO:tasks.workunit.client.1.smithi123.stdout:4/245: truncate d5/d1c/d23/d36/f45 895673 0 2024-02-16T22:21:49.584 INFO:tasks.workunit.client.0.smithi114.stdout:4/231: dwrite dd/d1b/d24/f4c [0,4194304] 0 2024-02-16T22:21:49.585 INFO:tasks.workunit.client.1.smithi123.stdout:4/246: write d5/f1e [1087099,50541] 0 2024-02-16T22:21:49.589 INFO:tasks.workunit.client.0.smithi114.stdout:0/189: write d1/fe [3575720,12023] 0 2024-02-16T22:21:49.590 INFO:tasks.workunit.client.0.smithi114.stdout:0/190: truncate d1/fe 4935698 0 2024-02-16T22:21:49.591 INFO:tasks.workunit.client.1.smithi123.stdout:4/247: dwrite d5/f1e [0,4194304] 0 2024-02-16T22:21:49.591 INFO:tasks.workunit.client.0.smithi114.stdout:8/158: truncate d4/f6 735716 0 2024-02-16T22:21:49.592 INFO:tasks.workunit.client.0.smithi114.stdout:8/159: fsync d4/d11/d15/f10 0 2024-02-16T22:21:49.594 INFO:tasks.workunit.client.0.smithi114.stdout:8/160: write d4/d16/f1e [144752,677] 0 2024-02-16T22:21:49.598 INFO:tasks.workunit.client.1.smithi123.stdout:3/191: creat d0/d20/d29/f43 x:0 0 0 2024-02-16T22:21:49.599 INFO:tasks.workunit.client.1.smithi123.stdout:6/241: dread d0/d6/fa [0,4194304] 0 2024-02-16T22:21:49.600 INFO:tasks.workunit.client.1.smithi123.stdout:6/242: fdatasync d0/d6/f1e 0 2024-02-16T22:21:49.601 INFO:tasks.workunit.client.1.smithi123.stdout:2/233: unlink d4/d1a/d29/d45/f24 0 2024-02-16T22:21:49.602 INFO:tasks.workunit.client.1.smithi123.stdout:2/234: dread d4/d1a/d1d/d13/d25/f26 [0,4194304] 0 2024-02-16T22:21:49.603 INFO:tasks.workunit.client.0.smithi114.stdout:1/126: symlink d4/l26 0 2024-02-16T22:21:49.605 INFO:tasks.workunit.client.1.smithi123.stdout:0/207: truncate d1/d8/d13/f15 2277104 0 2024-02-16T22:21:49.609 INFO:tasks.workunit.client.0.smithi114.stdout:2/175: creat dd/f3a x:0 0 0 2024-02-16T22:21:49.610 INFO:tasks.workunit.client.1.smithi123.stdout:1/196: sync 2024-02-16T22:21:49.610 INFO:tasks.workunit.client.1.smithi123.stdout:9/279: sync 2024-02-16T22:21:49.611 INFO:tasks.workunit.client.0.smithi114.stdout:5/133: rename d0/df/d16/d1e/d20 to d0/df/d16/d25 0 2024-02-16T22:21:49.611 INFO:tasks.workunit.client.1.smithi123.stdout:4/248: creat d5/d1c/d42/f4d x:0 0 0 2024-02-16T22:21:49.612 INFO:tasks.workunit.client.0.smithi114.stdout:5/134: truncate d0/d10/f23 460400 0 2024-02-16T22:21:49.613 INFO:tasks.workunit.client.0.smithi114.stdout:5/135: readlink d0/l6 0 2024-02-16T22:21:49.613 INFO:tasks.workunit.client.0.smithi114.stdout:9/169: link d2/f6 d2/d1a/d23/d35/d1d/f36 0 2024-02-16T22:21:49.614 INFO:tasks.workunit.client.1.smithi123.stdout:5/181: creat d6/dd/df/d1f/d25/d37/f3e x:0 0 0 2024-02-16T22:21:49.615 INFO:tasks.workunit.client.0.smithi114.stdout:6/133: mkdir d6/d2c 0 2024-02-16T22:21:49.616 INFO:tasks.workunit.client.1.smithi123.stdout:3/192: creat d0/d20/d29/f44 x:0 0 0 2024-02-16T22:21:49.616 INFO:tasks.workunit.client.0.smithi114.stdout:6/134: fdatasync d6/dd/d25/f2a 0 2024-02-16T22:21:49.618 INFO:tasks.workunit.client.0.smithi114.stdout:7/147: rename d3/l11 to d3/l2e 0 2024-02-16T22:21:49.623 INFO:tasks.workunit.client.1.smithi123.stdout:6/243: mkdir d0/d6/d4d/d59 0 2024-02-16T22:21:49.625 INFO:tasks.workunit.client.1.smithi123.stdout:8/275: dread d0/df/f1b [0,4194304] 0 2024-02-16T22:21:49.626 INFO:tasks.workunit.client.1.smithi123.stdout:8/276: write d0/d17/d26/f44 [229060,58806] 0 2024-02-16T22:21:49.626 INFO:tasks.workunit.client.0.smithi114.stdout:8/161: dread d4/f8 [0,4194304] 0 2024-02-16T22:21:49.627 INFO:tasks.workunit.client.0.smithi114.stdout:8/162: chown d4/d24/d2c 383 1 2024-02-16T22:21:49.633 INFO:tasks.workunit.client.0.smithi114.stdout:0/191: write d1/fa [4712537,98801] 0 2024-02-16T22:21:49.635 INFO:tasks.workunit.client.0.smithi114.stdout:3/121: rename d4/c8 to d4/d24/c2b 0 2024-02-16T22:21:49.637 INFO:tasks.workunit.client.0.smithi114.stdout:0/192: dread d1/f15 [0,4194304] 0 2024-02-16T22:21:49.638 INFO:tasks.workunit.client.1.smithi123.stdout:0/208: creat d1/d8/d13/f37 x:0 0 0 2024-02-16T22:21:49.639 INFO:tasks.workunit.client.1.smithi123.stdout:0/209: stat d1/d8/d13/d1d 0 2024-02-16T22:21:49.642 INFO:tasks.workunit.client.0.smithi114.stdout:7/148: rmdir d3/d14 39 2024-02-16T22:21:49.644 INFO:tasks.workunit.client.0.smithi114.stdout:2/176: mkdir dd/d3b 0 2024-02-16T22:21:49.645 INFO:tasks.workunit.client.1.smithi123.stdout:9/280: stat d4/d33/l4b 0 2024-02-16T22:21:49.650 INFO:tasks.workunit.client.1.smithi123.stdout:9/281: dwrite d4/d17/d20/f3c [0,4194304] 0 2024-02-16T22:21:49.655 INFO:tasks.workunit.client.0.smithi114.stdout:4/232: rename dd/d1b/d24/f3a to dd/d1b/d37/f52 0 2024-02-16T22:21:49.657 INFO:tasks.workunit.client.0.smithi114.stdout:4/233: write dd/d1b/f1d [1362975,66810] 0 2024-02-16T22:21:49.660 INFO:tasks.workunit.client.1.smithi123.stdout:5/182: rmdir d6/d3b 39 2024-02-16T22:21:49.663 INFO:tasks.workunit.client.0.smithi114.stdout:3/122: mkdir d4/dd/d2c 0 2024-02-16T22:21:49.663 INFO:tasks.workunit.client.1.smithi123.stdout:5/183: chown d6/dd/df/f22 63615 1 2024-02-16T22:21:49.664 INFO:tasks.workunit.client.1.smithi123.stdout:5/184: write d6/dd/df/f28 [1537031,2729] 0 2024-02-16T22:21:49.665 INFO:tasks.workunit.client.1.smithi123.stdout:3/193: symlink d0/d20/d2e/l45 0 2024-02-16T22:21:49.665 INFO:tasks.workunit.client.1.smithi123.stdout:2/235: dread d4/f16 [0,4194304] 0 2024-02-16T22:21:49.667 INFO:tasks.workunit.client.1.smithi123.stdout:6/244: symlink d0/d6/d2f/l5a 0 2024-02-16T22:21:49.668 INFO:tasks.workunit.client.1.smithi123.stdout:9/282: dread d4/d1e/f42 [4194304,4194304] 0 2024-02-16T22:21:49.668 INFO:tasks.workunit.client.0.smithi114.stdout:4/234: dwrite dd/d1b/d24/f4e [0,4194304] 0 2024-02-16T22:21:49.669 INFO:tasks.workunit.client.1.smithi123.stdout:9/283: truncate d4/d33/f4c 151025 0 2024-02-16T22:21:49.672 INFO:tasks.workunit.client.1.smithi123.stdout:8/277: mkdir d0/d17/d26/d2a/d3c/d62 0 2024-02-16T22:21:49.673 INFO:tasks.workunit.client.1.smithi123.stdout:8/278: chown d0/d12/f58 5255 1 2024-02-16T22:21:49.674 INFO:tasks.workunit.client.0.smithi114.stdout:6/135: truncate d6/f21 2267299 0 2024-02-16T22:21:49.675 INFO:tasks.workunit.client.0.smithi114.stdout:4/235: dwrite dd/f32 [0,4194304] 0 2024-02-16T22:21:49.676 INFO:tasks.workunit.client.0.smithi114.stdout:6/136: dread - d6/da/f1b zero size 2024-02-16T22:21:49.676 INFO:tasks.workunit.client.0.smithi114.stdout:7/149: write d3/fc [3055531,6608] 0 2024-02-16T22:21:49.677 INFO:tasks.workunit.client.0.smithi114.stdout:4/236: readlink dd/l1e 0 2024-02-16T22:21:49.678 INFO:tasks.workunit.client.1.smithi123.stdout:7/228: getdents d3/d12/d24 0 2024-02-16T22:21:49.679 INFO:tasks.workunit.client.0.smithi114.stdout:7/150: write d3/f27 [4590180,35489] 0 2024-02-16T22:21:49.680 INFO:tasks.workunit.client.0.smithi114.stdout:4/237: dread dd/d1b/f3e [0,4194304] 0 2024-02-16T22:21:49.681 INFO:tasks.workunit.client.1.smithi123.stdout:8/279: dwrite d0/d12/f46 [4194304,4194304] 0 2024-02-16T22:21:49.683 INFO:tasks.workunit.client.1.smithi123.stdout:1/197: chown d3/d6/db/d10/d2d/f39 35301 1 2024-02-16T22:21:49.685 INFO:tasks.workunit.client.1.smithi123.stdout:1/198: chown d3/d6/db/d10/d2a 119146 1 2024-02-16T22:21:49.687 INFO:tasks.workunit.client.0.smithi114.stdout:8/163: write d4/fc [4503585,123297] 0 2024-02-16T22:21:49.687 INFO:tasks.workunit.client.0.smithi114.stdout:8/164: write d4/f1d [1235168,25757] 0 2024-02-16T22:21:49.689 INFO:tasks.workunit.client.0.smithi114.stdout:3/123: mkdir d4/d24/d2d 0 2024-02-16T22:21:49.689 INFO:tasks.workunit.client.0.smithi114.stdout:3/124: readlink l3 0 2024-02-16T22:21:49.692 INFO:tasks.workunit.client.0.smithi114.stdout:5/136: link d0/d10/c12 d0/df/c26 0 2024-02-16T22:21:49.692 INFO:tasks.workunit.client.0.smithi114.stdout:3/125: dread d4/dd/f1e [0,4194304] 0 2024-02-16T22:21:49.696 INFO:tasks.workunit.client.1.smithi123.stdout:6/245: rmdir d0/d6/d1a/d2e 39 2024-02-16T22:21:49.697 INFO:tasks.workunit.client.1.smithi123.stdout:4/249: dwrite d5/d49/f4b [0,4194304] 0 2024-02-16T22:21:49.697 INFO:tasks.workunit.client.1.smithi123.stdout:2/236: dread f2 [0,4194304] 0 2024-02-16T22:21:49.700 INFO:tasks.workunit.client.0.smithi114.stdout:5/137: dwrite d0/df/d16/d1e/f21 [0,4194304] 0 2024-02-16T22:21:49.700 INFO:tasks.workunit.client.1.smithi123.stdout:5/185: dwrite d6/dd/df/f1b [0,4194304] 0 2024-02-16T22:21:49.704 INFO:tasks.workunit.client.1.smithi123.stdout:9/284: dread d4/d1e/f42 [0,4194304] 0 2024-02-16T22:21:49.705 INFO:tasks.workunit.client.1.smithi123.stdout:4/250: dwrite d5/d1c/d23/d36/f45 [0,4194304] 0 2024-02-16T22:21:49.705 INFO:tasks.workunit.client.0.smithi114.stdout:2/177: mkdir dd/d3c 0 2024-02-16T22:21:49.711 INFO:tasks.workunit.client.0.smithi114.stdout:4/238: rename dd/l1e to dd/l53 0 2024-02-16T22:21:49.714 INFO:tasks.workunit.client.1.smithi123.stdout:8/280: unlink d0/l2 0 2024-02-16T22:21:49.716 INFO:tasks.workunit.client.0.smithi114.stdout:1/127: rename d4/f5 to d4/d9/f27 0 2024-02-16T22:21:49.717 INFO:tasks.workunit.client.1.smithi123.stdout:1/199: readlink d3/d6/db/l40 0 2024-02-16T22:21:49.719 INFO:tasks.workunit.client.1.smithi123.stdout:3/194: write d0/f19 [1086783,26124] 0 2024-02-16T22:21:49.723 INFO:tasks.workunit.client.1.smithi123.stdout:0/210: sync 2024-02-16T22:21:49.725 INFO:tasks.workunit.client.1.smithi123.stdout:0/211: stat d1/d4/l21 0 2024-02-16T22:21:49.725 INFO:tasks.workunit.client.1.smithi123.stdout:8/281: dwrite d0/d17/d26/d32/d45/f5f [0,4194304] 0 2024-02-16T22:21:49.725 INFO:tasks.workunit.client.1.smithi123.stdout:6/246: creat d0/d9/dc/d39/f5b x:0 0 0 2024-02-16T22:21:49.726 INFO:tasks.workunit.client.0.smithi114.stdout:5/138: chown d0/d10/c12 506821 1 2024-02-16T22:21:49.728 INFO:tasks.workunit.client.0.smithi114.stdout:2/178: symlink dd/df/d1c/d21/l3d 0 2024-02-16T22:21:49.729 INFO:tasks.workunit.client.1.smithi123.stdout:1/200: dwrite d3/d6/db/d10/f29 [0,4194304] 0 2024-02-16T22:21:49.729 INFO:tasks.workunit.client.1.smithi123.stdout:3/195: dwrite d0/ff [0,4194304] 0 2024-02-16T22:21:49.730 INFO:tasks.workunit.client.0.smithi114.stdout:7/151: symlink d3/d23/l2f 0 2024-02-16T22:21:49.732 INFO:tasks.workunit.client.0.smithi114.stdout:5/139: dread d0/d10/f1a [0,4194304] 0 2024-02-16T22:21:49.734 INFO:tasks.workunit.client.1.smithi123.stdout:9/285: mkdir d4/d56 0 2024-02-16T22:21:49.735 INFO:tasks.workunit.client.0.smithi114.stdout:4/239: unlink dd/c31 0 2024-02-16T22:21:49.743 INFO:tasks.workunit.client.0.smithi114.stdout:9/170: rename d2/la to d2/d1a/d23/d35/d1d/l37 0 2024-02-16T22:21:49.746 INFO:tasks.workunit.client.1.smithi123.stdout:9/286: dwrite d4/f16 [0,4194304] 0 2024-02-16T22:21:49.748 INFO:tasks.workunit.client.1.smithi123.stdout:8/282: symlink d0/d12/l63 0 2024-02-16T22:21:49.749 INFO:tasks.workunit.client.0.smithi114.stdout:0/193: getdents d1/d16/d1f 0 2024-02-16T22:21:49.750 INFO:tasks.workunit.client.0.smithi114.stdout:0/194: write d1/fe [4886371,121830] 0 2024-02-16T22:21:49.750 INFO:tasks.workunit.client.1.smithi123.stdout:6/247: creat d0/d6/f5c x:0 0 0 2024-02-16T22:21:49.754 INFO:tasks.workunit.client.1.smithi123.stdout:7/229: link d3/cb d3/d26/d35/c4c 0 2024-02-16T22:21:49.755 INFO:tasks.workunit.client.0.smithi114.stdout:5/140: creat d0/df/d16/d1e/f27 x:0 0 0 2024-02-16T22:21:49.756 INFO:tasks.workunit.client.0.smithi114.stdout:4/240: symlink dd/l54 0 2024-02-16T22:21:49.756 INFO:tasks.workunit.client.0.smithi114.stdout:5/141: stat d0/f24 0 2024-02-16T22:21:49.757 INFO:tasks.workunit.client.1.smithi123.stdout:9/287: mkdir d4/d36/d57 0 2024-02-16T22:21:49.758 INFO:tasks.workunit.client.0.smithi114.stdout:0/195: dwrite d1/d1a/f26 [0,4194304] 0 2024-02-16T22:21:49.758 INFO:tasks.workunit.client.0.smithi114.stdout:5/142: chown d0/d10/c12 125827 1 2024-02-16T22:21:49.758 INFO:tasks.workunit.client.1.smithi123.stdout:8/283: rmdir d0/d17 39 2024-02-16T22:21:49.760 INFO:tasks.workunit.client.0.smithi114.stdout:6/137: rename d6/dd/f19 to d6/dd/d25/d26/f2d 0 2024-02-16T22:21:49.761 INFO:tasks.workunit.client.0.smithi114.stdout:9/171: mknod d2/d1a/d23/c38 0 2024-02-16T22:21:49.763 INFO:tasks.workunit.client.1.smithi123.stdout:1/201: mkdir d3/d47 0 2024-02-16T22:21:49.766 INFO:tasks.workunit.client.1.smithi123.stdout:3/196: rename d0/l25 to d0/d40/l46 0 2024-02-16T22:21:49.770 INFO:tasks.workunit.client.1.smithi123.stdout:8/284: creat d0/d29/f64 x:0 0 0 2024-02-16T22:21:49.771 INFO:tasks.workunit.client.0.smithi114.stdout:2/179: symlink dd/df/l3e 0 2024-02-16T22:21:49.777 INFO:tasks.workunit.client.0.smithi114.stdout:1/128: creat d4/f28 x:0 0 0 2024-02-16T22:21:49.779 INFO:tasks.workunit.client.0.smithi114.stdout:0/196: fdatasync d1/f7 0 2024-02-16T22:21:49.780 INFO:tasks.workunit.client.0.smithi114.stdout:0/197: truncate d1/d1a/f26 4472359 0 2024-02-16T22:21:49.783 INFO:tasks.workunit.client.1.smithi123.stdout:3/197: symlink d0/d20/d2e/l47 0 2024-02-16T22:21:49.784 INFO:tasks.workunit.client.1.smithi123.stdout:3/198: chown d0/d20/d2e/f39 7 1 2024-02-16T22:21:49.784 INFO:tasks.workunit.client.1.smithi123.stdout:3/199: write d0/d20/d29/f44 [520798,70796] 0 2024-02-16T22:21:49.786 INFO:tasks.workunit.client.1.smithi123.stdout:4/251: write d5/d1c/d23/d31/f3e [887394,62843] 0 2024-02-16T22:21:49.787 INFO:tasks.workunit.client.0.smithi114.stdout:0/198: dwrite d1/d1a/f3b [0,4194304] 0 2024-02-16T22:21:49.788 INFO:tasks.workunit.client.0.smithi114.stdout:3/126: rename d4/d26/l27 to d4/d24/l2e 0 2024-02-16T22:21:49.791 INFO:tasks.workunit.client.1.smithi123.stdout:2/237: write d4/d1a/d1d/f9 [857757,54212] 0 2024-02-16T22:21:49.794 INFO:tasks.workunit.client.0.smithi114.stdout:6/138: creat d6/dd/d25/d26/f2e x:0 0 0 2024-02-16T22:21:49.796 INFO:tasks.workunit.client.1.smithi123.stdout:8/285: dwrite d0/d17/d26/f44 [0,4194304] 0 2024-02-16T22:21:49.798 INFO:tasks.workunit.client.1.smithi123.stdout:2/238: dread d4/d1a/d29/f3a [0,4194304] 0 2024-02-16T22:21:49.799 INFO:tasks.workunit.client.1.smithi123.stdout:5/186: getdents d6/d3b 0 2024-02-16T22:21:49.800 INFO:tasks.workunit.client.0.smithi114.stdout:8/165: truncate f0 570407 0 2024-02-16T22:21:49.802 INFO:tasks.workunit.client.1.smithi123.stdout:1/202: fsync d3/d6/d11/f1f 0 2024-02-16T22:21:49.803 INFO:tasks.workunit.client.0.smithi114.stdout:6/139: dwrite d6/dd/d25/f2a [0,4194304] 0 2024-02-16T22:21:49.806 INFO:tasks.workunit.client.1.smithi123.stdout:9/288: rename d4/d17/d3f to d4/d56/d58 0 2024-02-16T22:21:49.811 INFO:tasks.workunit.client.1.smithi123.stdout:3/200: creat d0/d20/d2e/f48 x:0 0 0 2024-02-16T22:21:49.822 INFO:tasks.workunit.client.1.smithi123.stdout:6/248: getdents d0/d6/d4d 0 2024-02-16T22:21:49.825 INFO:tasks.workunit.client.1.smithi123.stdout:8/286: symlink d0/d17/d43/d34/l65 0 2024-02-16T22:21:49.827 INFO:tasks.workunit.client.1.smithi123.stdout:8/287: truncate d0/d17/d26/f5b 318896 0 2024-02-16T22:21:49.827 INFO:tasks.workunit.client.1.smithi123.stdout:7/230: dwrite d3/d12/d34/f3c [0,4194304] 0 2024-02-16T22:21:49.828 INFO:tasks.workunit.client.0.smithi114.stdout:5/143: write d0/df/f1d [1164307,58357] 0 2024-02-16T22:21:49.829 INFO:tasks.workunit.client.1.smithi123.stdout:5/187: chown d6/dd/df/d1f/d25/f39 80 1 2024-02-16T22:21:49.829 INFO:tasks.workunit.client.0.smithi114.stdout:5/144: chown d0/df/d16/d1e/f21 128480281 1 2024-02-16T22:21:49.831 INFO:tasks.workunit.client.0.smithi114.stdout:5/145: write d0/df/f1d [2120408,122685] 0 2024-02-16T22:21:49.834 INFO:tasks.workunit.client.0.smithi114.stdout:6/140: unlink d6/da/l20 0 2024-02-16T22:21:49.838 INFO:tasks.workunit.client.1.smithi123.stdout:9/289: chown d4/d17/d2b/f53 747476 1 2024-02-16T22:21:49.838 INFO:tasks.workunit.client.0.smithi114.stdout:7/152: link d3/d14/d1b/l20 d3/d22/l30 0 2024-02-16T22:21:49.838 INFO:tasks.workunit.client.1.smithi123.stdout:9/290: chown d4/d33/l4b 11509 1 2024-02-16T22:21:49.840 INFO:tasks.workunit.client.1.smithi123.stdout:4/252: rename f3 to d5/d49/f4e 0 2024-02-16T22:21:49.841 INFO:tasks.workunit.client.0.smithi114.stdout:6/141: dwrite d6/dd/f29 [0,4194304] 0 2024-02-16T22:21:49.841 INFO:tasks.workunit.client.0.smithi114.stdout:4/241: creat dd/d1b/f55 x:0 0 0 2024-02-16T22:21:49.842 INFO:tasks.workunit.client.1.smithi123.stdout:4/253: write d5/f14 [1032788,113357] 0 2024-02-16T22:21:49.843 INFO:tasks.workunit.client.1.smithi123.stdout:4/254: chown d5/f4c 0 1 2024-02-16T22:21:49.844 INFO:tasks.workunit.client.1.smithi123.stdout:9/291: dwrite d4/d33/f4c [0,4194304] 0 2024-02-16T22:21:49.846 INFO:tasks.workunit.client.0.smithi114.stdout:0/199: mkdir d1/d1a/d3e/d41 0 2024-02-16T22:21:49.848 INFO:tasks.workunit.client.0.smithi114.stdout:3/127: mknod d4/d24/d2d/c2f 0 2024-02-16T22:21:49.850 INFO:tasks.workunit.client.1.smithi123.stdout:4/255: dwrite d5/f40 [0,4194304] 0 2024-02-16T22:21:49.852 INFO:tasks.workunit.client.0.smithi114.stdout:9/172: creat d2/d1a/f39 x:0 0 0 2024-02-16T22:21:49.852 INFO:tasks.workunit.client.1.smithi123.stdout:8/288: fdatasync d0/d29/f60 0 2024-02-16T22:21:49.853 INFO:tasks.workunit.client.0.smithi114.stdout:2/180: link dd/df/d2c/f34 dd/f3f 0 2024-02-16T22:21:49.854 INFO:tasks.workunit.client.1.smithi123.stdout:8/289: readlink d0/d12/l63 0 2024-02-16T22:21:49.854 INFO:tasks.workunit.client.1.smithi123.stdout:0/212: sync 2024-02-16T22:21:49.855 INFO:tasks.workunit.client.1.smithi123.stdout:7/231: mkdir d3/d12/d4d 0 2024-02-16T22:21:49.857 INFO:tasks.workunit.client.1.smithi123.stdout:5/188: mkdir d6/dd/df/d2b/d3f 0 2024-02-16T22:21:49.858 INFO:tasks.workunit.client.1.smithi123.stdout:2/239: dwrite d4/d1a/d29/f3a [0,4194304] 0 2024-02-16T22:21:49.858 INFO:tasks.workunit.client.1.smithi123.stdout:5/189: write d6/dd/df/d2b/f36 [288168,90801] 0 2024-02-16T22:21:49.859 INFO:tasks.workunit.client.1.smithi123.stdout:1/203: mkdir d3/d6/db/d10/d2d/d42/d48 0 2024-02-16T22:21:49.861 INFO:tasks.workunit.client.1.smithi123.stdout:1/204: dread - d3/d6/db/d10/d2d/f25 zero size 2024-02-16T22:21:49.862 INFO:tasks.workunit.client.0.smithi114.stdout:7/153: mknod d3/d23/c31 0 2024-02-16T22:21:49.863 INFO:tasks.workunit.client.0.smithi114.stdout:5/146: dread d0/d10/f13 [0,4194304] 0 2024-02-16T22:21:49.864 INFO:tasks.workunit.client.0.smithi114.stdout:5/147: fdatasync d0/df/f1d 0 2024-02-16T22:21:49.864 INFO:tasks.workunit.client.0.smithi114.stdout:9/173: dread f1 [4194304,4194304] 0 2024-02-16T22:21:49.865 INFO:tasks.workunit.client.0.smithi114.stdout:9/174: dread - d2/d1a/d23/d35/f27 zero size 2024-02-16T22:21:49.867 INFO:tasks.workunit.client.0.smithi114.stdout:6/142: creat d6/da/f2f x:0 0 0 2024-02-16T22:21:49.868 INFO:tasks.workunit.client.0.smithi114.stdout:6/143: chown d6/f1e 220079695 1 2024-02-16T22:21:49.869 INFO:tasks.workunit.client.1.smithi123.stdout:6/249: rename d0/d6/d4d/d59 to d0/d6/d2f/d5d 0 2024-02-16T22:21:49.870 INFO:tasks.workunit.client.1.smithi123.stdout:6/250: read - d0/d29/f58 zero size 2024-02-16T22:21:49.872 INFO:tasks.workunit.client.0.smithi114.stdout:4/242: mkdir dd/d2e/d56 0 2024-02-16T22:21:49.872 INFO:tasks.workunit.client.1.smithi123.stdout:3/201: mkdir d0/d20/d29/d34/d37/d49 0 2024-02-16T22:21:49.875 INFO:tasks.workunit.client.0.smithi114.stdout:3/128: rmdir d4/d24/d2d 39 2024-02-16T22:21:49.876 INFO:tasks.workunit.client.1.smithi123.stdout:9/292: mkdir d4/d56/d58/d59 0 2024-02-16T22:21:49.878 INFO:tasks.workunit.client.1.smithi123.stdout:3/202: dwrite d0/d20/d29/d34/f35 [0,4194304] 0 2024-02-16T22:21:49.882 INFO:tasks.workunit.client.0.smithi114.stdout:8/166: creat d4/d11/f2f x:0 0 0 2024-02-16T22:21:49.883 INFO:tasks.workunit.client.1.smithi123.stdout:3/203: dwrite d0/ff [0,4194304] 0 2024-02-16T22:21:49.885 INFO:tasks.workunit.client.1.smithi123.stdout:3/204: fsync d0/d20/d2e/f2f 0 2024-02-16T22:21:49.885 INFO:tasks.workunit.client.1.smithi123.stdout:3/205: fsync d0/f1f 0 2024-02-16T22:21:49.887 INFO:tasks.workunit.client.0.smithi114.stdout:7/154: fsync d3/f8 0 2024-02-16T22:21:49.888 INFO:tasks.workunit.client.0.smithi114.stdout:4/243: dread dd/f12 [0,4194304] 0 2024-02-16T22:21:49.889 INFO:tasks.workunit.client.1.smithi123.stdout:2/240: write d4/f23 [3585904,128001] 0 2024-02-16T22:21:49.894 INFO:tasks.workunit.client.0.smithi114.stdout:5/148: unlink d0/d10/c18 0 2024-02-16T22:21:49.895 INFO:tasks.workunit.client.1.smithi123.stdout:1/205: write d3/d6/d11/f23 [223388,78037] 0 2024-02-16T22:21:49.896 INFO:tasks.workunit.client.0.smithi114.stdout:9/175: creat d2/d1a/d23/d35/d1d/f3a x:0 0 0 2024-02-16T22:21:49.901 INFO:tasks.workunit.client.0.smithi114.stdout:1/129: getdents d4/d9 0 2024-02-16T22:21:49.901 INFO:tasks.workunit.client.0.smithi114.stdout:1/130: read d4/d9/f10 [3379665,29900] 0 2024-02-16T22:21:49.902 INFO:tasks.workunit.client.0.smithi114.stdout:1/131: stat d4/d9/f11 0 2024-02-16T22:21:49.905 INFO:tasks.workunit.client.0.smithi114.stdout:0/200: creat d1/d1a/d3e/d41/f42 x:0 0 0 2024-02-16T22:21:49.905 INFO:tasks.workunit.client.1.smithi123.stdout:5/190: write d6/dd/df/d1f/d25/f39 [912485,86221] 0 2024-02-16T22:21:49.906 INFO:tasks.workunit.client.1.smithi123.stdout:7/232: dwrite f2 [0,4194304] 0 2024-02-16T22:21:49.908 INFO:tasks.workunit.client.1.smithi123.stdout:8/290: mkdir d0/d61/d66 0 2024-02-16T22:21:49.908 INFO:tasks.workunit.client.1.smithi123.stdout:6/251: write d0/f30 [322106,92563] 0 2024-02-16T22:21:49.909 INFO:tasks.workunit.client.1.smithi123.stdout:5/191: dread d6/dd/df/d2b/f36 [0,4194304] 0 2024-02-16T22:21:49.910 INFO:tasks.workunit.client.1.smithi123.stdout:8/291: dread - d0/d17/d26/d32/f56 zero size 2024-02-16T22:21:49.910 INFO:tasks.workunit.client.1.smithi123.stdout:0/213: symlink d1/l38 0 2024-02-16T22:21:49.912 INFO:tasks.workunit.client.0.smithi114.stdout:2/181: rename dd/f1d to dd/df/d1c/f40 0 2024-02-16T22:21:49.912 INFO:tasks.workunit.client.0.smithi114.stdout:2/182: chown dd/f28 42 1 2024-02-16T22:21:49.913 INFO:tasks.workunit.client.1.smithi123.stdout:8/292: chown d0/df/f1c 1329 1 2024-02-16T22:21:49.913 INFO:tasks.workunit.client.1.smithi123.stdout:4/256: write d5/d49/f4e [480932,98496] 0 2024-02-16T22:21:49.913 INFO:tasks.workunit.client.0.smithi114.stdout:2/183: chown dd/df/f37 441079 1 2024-02-16T22:21:49.913 INFO:tasks.workunit.client.0.smithi114.stdout:8/167: symlink d4/d16/l30 0 2024-02-16T22:21:49.915 INFO:tasks.workunit.client.1.smithi123.stdout:6/252: dwrite d0/d9/dc/d3a/f40 [4194304,4194304] 0 2024-02-16T22:21:49.916 INFO:tasks.workunit.client.0.smithi114.stdout:7/155: symlink d3/d23/l32 0 2024-02-16T22:21:49.916 INFO:tasks.workunit.client.0.smithi114.stdout:7/156: chown d3/f8 1 1 2024-02-16T22:21:49.918 INFO:tasks.workunit.client.0.smithi114.stdout:4/244: mkdir dd/d1b/d24/d57 0 2024-02-16T22:21:49.926 INFO:tasks.workunit.client.0.smithi114.stdout:4/245: dwrite dd/d1b/d37/d38/f46 [4194304,4194304] 0 2024-02-16T22:21:49.926 INFO:tasks.workunit.client.1.smithi123.stdout:7/233: creat d3/f4e x:0 0 0 2024-02-16T22:21:49.930 INFO:tasks.workunit.client.0.smithi114.stdout:3/129: fdatasync d4/f16 0 2024-02-16T22:21:49.932 INFO:tasks.workunit.client.1.smithi123.stdout:5/192: mknod d6/d3b/c40 0 2024-02-16T22:21:49.938 INFO:tasks.workunit.client.1.smithi123.stdout:3/206: rename d0/d20/c27 to d0/d40/c4a 0 2024-02-16T22:21:49.939 INFO:tasks.workunit.client.0.smithi114.stdout:2/184: unlink dd/df/f37 0 2024-02-16T22:21:49.940 INFO:tasks.workunit.client.1.smithi123.stdout:3/207: write d0/d20/d2e/f48 [192796,27987] 0 2024-02-16T22:21:49.940 INFO:tasks.workunit.client.1.smithi123.stdout:2/241: dwrite d4/d1a/d29/d45/f4b [0,4194304] 0 2024-02-16T22:21:49.941 INFO:tasks.workunit.client.1.smithi123.stdout:3/208: dread - d0/d20/d29/f3c zero size 2024-02-16T22:21:49.941 INFO:tasks.workunit.client.0.smithi114.stdout:8/168: dwrite d4/f8 [0,4194304] 0 2024-02-16T22:21:49.945 INFO:tasks.workunit.client.0.smithi114.stdout:7/157: symlink d3/d22/l33 0 2024-02-16T22:21:49.947 INFO:tasks.workunit.client.1.smithi123.stdout:9/293: sync 2024-02-16T22:21:49.949 INFO:tasks.workunit.client.0.smithi114.stdout:7/158: chown d3/d14/c24 6419561 1 2024-02-16T22:21:49.951 INFO:tasks.workunit.client.0.smithi114.stdout:5/149: mkdir d0/d28 0 2024-02-16T22:21:49.951 INFO:tasks.workunit.client.0.smithi114.stdout:7/159: stat d3/d14/l16 0 2024-02-16T22:21:49.952 INFO:tasks.workunit.client.1.smithi123.stdout:7/234: fdatasync d3/d26/d35/f4b 0 2024-02-16T22:21:49.952 INFO:tasks.workunit.client.0.smithi114.stdout:1/132: dwrite d4/f13 [0,4194304] 0 2024-02-16T22:21:49.955 INFO:tasks.workunit.client.0.smithi114.stdout:8/169: dwrite d4/d11/f17 [0,4194304] 0 2024-02-16T22:21:49.955 INFO:tasks.workunit.client.1.smithi123.stdout:5/193: creat d6/dd/df/f41 x:0 0 0 2024-02-16T22:21:49.959 INFO:tasks.workunit.client.1.smithi123.stdout:6/253: symlink d0/d6/d1a/d2e/l5e 0 2024-02-16T22:21:49.960 INFO:tasks.workunit.client.1.smithi123.stdout:5/194: write d6/dd/df/d1f/f24 [3567120,12021] 0 2024-02-16T22:21:49.961 INFO:tasks.workunit.client.0.smithi114.stdout:0/201: truncate d1/d16/d1f/d28/f2d 1681648 0 2024-02-16T22:21:49.961 INFO:tasks.workunit.client.1.smithi123.stdout:2/242: symlink d4/d1a/d1d/d13/d25/l4c 0 2024-02-16T22:21:49.961 INFO:tasks.workunit.client.0.smithi114.stdout:7/160: dwrite d3/fe [0,4194304] 0 2024-02-16T22:21:49.962 INFO:tasks.workunit.client.1.smithi123.stdout:6/254: write d0/d6/f5c [6414,22893] 0 2024-02-16T22:21:49.962 INFO:tasks.workunit.client.1.smithi123.stdout:5/195: chown l4 0 1 2024-02-16T22:21:49.963 INFO:tasks.workunit.client.1.smithi123.stdout:8/293: dwrite d0/df/f1b [0,4194304] 0 2024-02-16T22:21:49.964 INFO:tasks.workunit.client.1.smithi123.stdout:9/294: rmdir d4/d17/d20 39 2024-02-16T22:21:49.965 INFO:tasks.workunit.client.0.smithi114.stdout:3/130: mknod d4/dd/c30 0 2024-02-16T22:21:49.966 INFO:tasks.workunit.client.1.smithi123.stdout:7/235: stat d3/d12/d34/c42 0 2024-02-16T22:21:49.966 INFO:tasks.workunit.client.1.smithi123.stdout:8/294: fsync d0/d17/d43/d34/f38 0 2024-02-16T22:21:49.967 INFO:tasks.workunit.client.1.smithi123.stdout:7/236: readlink d3/d12/d34/l3f 0 2024-02-16T22:21:49.968 INFO:tasks.workunit.client.0.smithi114.stdout:2/185: write dd/df/d1c/f40 [4096484,59465] 0 2024-02-16T22:21:49.968 INFO:tasks.workunit.client.1.smithi123.stdout:8/295: fdatasync d0/df/f1c 0 2024-02-16T22:21:49.969 INFO:tasks.workunit.client.1.smithi123.stdout:7/237: chown d3/d26/l3a 121 1 2024-02-16T22:21:49.969 INFO:tasks.workunit.client.1.smithi123.stdout:8/296: stat d0/d17/d26/d2a/d3c/l3b 0 2024-02-16T22:21:49.970 INFO:tasks.workunit.client.1.smithi123.stdout:2/243: rmdir d4/d1a/d29/d45 39 2024-02-16T22:21:49.974 INFO:tasks.workunit.client.0.smithi114.stdout:1/133: fdatasync d4/fb 0 2024-02-16T22:21:49.976 INFO:tasks.workunit.client.1.smithi123.stdout:2/244: dwrite d4/d1a/f32 [0,4194304] 0 2024-02-16T22:21:49.976 INFO:tasks.workunit.client.1.smithi123.stdout:5/196: symlink d6/dd/df/l42 0 2024-02-16T22:21:49.980 INFO:tasks.workunit.client.1.smithi123.stdout:9/295: creat d4/d56/d58/f5a x:0 0 0 2024-02-16T22:21:49.981 INFO:tasks.workunit.client.0.smithi114.stdout:8/170: mkdir d4/d11/d15/d31 0 2024-02-16T22:21:49.982 INFO:tasks.workunit.client.0.smithi114.stdout:8/171: write d4/d11/f20 [926551,91750] 0 2024-02-16T22:21:49.983 INFO:tasks.workunit.client.0.smithi114.stdout:9/176: creat d2/dd/f3b x:0 0 0 2024-02-16T22:21:49.985 INFO:tasks.workunit.client.0.smithi114.stdout:6/144: getdents d6/dd/d25 0 2024-02-16T22:21:49.985 INFO:tasks.workunit.client.1.smithi123.stdout:1/206: rename d3/d6/db/d10/d2d/d42 to d3/d6/d49 0 2024-02-16T22:21:49.986 INFO:tasks.workunit.client.0.smithi114.stdout:6/145: write d6/da/f2f [973575,25185] 0 2024-02-16T22:21:49.986 INFO:tasks.workunit.client.1.smithi123.stdout:7/238: creat d3/d12/f4f x:0 0 0 2024-02-16T22:21:49.986 INFO:tasks.workunit.client.0.smithi114.stdout:4/246: mknod dd/d1b/d24/d57/c58 0 2024-02-16T22:21:49.987 INFO:tasks.workunit.client.1.smithi123.stdout:8/297: mkdir d0/d17/d26/d32/d45/d67 0 2024-02-16T22:21:49.990 INFO:tasks.workunit.client.0.smithi114.stdout:7/161: creat d3/d22/f34 x:0 0 0 2024-02-16T22:21:49.992 INFO:tasks.workunit.client.1.smithi123.stdout:9/296: creat d4/d17/d2b/f5b x:0 0 0 2024-02-16T22:21:49.993 INFO:tasks.workunit.client.1.smithi123.stdout:1/207: stat d3/d6/db/f3a 0 2024-02-16T22:21:49.993 INFO:tasks.workunit.client.0.smithi114.stdout:1/134: write d4/d9/f27 [576138,3877] 0 2024-02-16T22:21:49.994 INFO:tasks.workunit.client.1.smithi123.stdout:1/208: dread - d3/d6/db/d10/d2d/f25 zero size 2024-02-16T22:21:49.995 INFO:tasks.workunit.client.1.smithi123.stdout:0/214: rename d1/d8/c35 to d1/d4/c39 0 2024-02-16T22:21:49.996 INFO:tasks.workunit.client.1.smithi123.stdout:4/257: rename d5 to d5/d1c/d23/d31/d4f 22 2024-02-16T22:21:49.996 INFO:tasks.workunit.client.1.smithi123.stdout:5/197: fsync d6/dd/df/f41 0 2024-02-16T22:21:49.997 INFO:tasks.workunit.client.0.smithi114.stdout:5/150: link d0/d10/f23 d0/df/d16/d1e/f29 0 2024-02-16T22:21:49.998 INFO:tasks.workunit.client.1.smithi123.stdout:0/215: write d1/d8/f34 [1002626,114911] 0 2024-02-16T22:21:49.998 INFO:tasks.workunit.client.1.smithi123.stdout:5/198: truncate d6/dd/df/f22 262291 0 2024-02-16T22:21:49.999 INFO:tasks.workunit.client.1.smithi123.stdout:7/239: rmdir d3/d12/d24 39 2024-02-16T22:21:50.000 INFO:tasks.workunit.client.1.smithi123.stdout:7/240: chown d3 1227 1 2024-02-16T22:21:50.000 INFO:tasks.workunit.client.1.smithi123.stdout:3/209: dread d0/fe [0,4194304] 0 2024-02-16T22:21:50.001 INFO:tasks.workunit.client.1.smithi123.stdout:8/298: mkdir d0/d17/d26/d68 0 2024-02-16T22:21:50.005 INFO:tasks.workunit.client.1.smithi123.stdout:4/258: dwrite f4 [0,4194304] 0 2024-02-16T22:21:50.006 INFO:tasks.workunit.client.1.smithi123.stdout:4/259: chown d5/le 17232274 1 2024-02-16T22:21:50.007 INFO:tasks.workunit.client.1.smithi123.stdout:4/260: fsync d5/d49/f4b 0 2024-02-16T22:21:50.007 INFO:tasks.workunit.client.1.smithi123.stdout:7/241: dread d3/d12/d34/f3c [0,4194304] 0 2024-02-16T22:21:50.008 INFO:tasks.workunit.client.0.smithi114.stdout:4/247: rmdir dd/d1b/d24/d57 39 2024-02-16T22:21:50.009 INFO:tasks.workunit.client.1.smithi123.stdout:7/242: write d3/f4e [427774,83070] 0 2024-02-16T22:21:50.010 INFO:tasks.workunit.client.1.smithi123.stdout:6/255: dread d0/d9/f19 [0,4194304] 0 2024-02-16T22:21:50.010 INFO:tasks.workunit.client.1.smithi123.stdout:7/243: read d3/d12/f1a [44726,16345] 0 2024-02-16T22:21:50.015 INFO:tasks.workunit.client.1.smithi123.stdout:1/209: dwrite d3/d6/d11/f1f [0,4194304] 0 2024-02-16T22:21:50.015 INFO:tasks.workunit.client.1.smithi123.stdout:4/261: dread d5/d49/f4b [0,4194304] 0 2024-02-16T22:21:50.016 INFO:tasks.workunit.client.0.smithi114.stdout:7/162: mknod d3/d22/c35 0 2024-02-16T22:21:50.017 INFO:tasks.workunit.client.0.smithi114.stdout:4/248: dwrite dd/d1b/d24/f4c [0,4194304] 0 2024-02-16T22:21:50.018 INFO:tasks.workunit.client.1.smithi123.stdout:7/244: dread d3/d12/d34/f3c [0,4194304] 0 2024-02-16T22:21:50.022 INFO:tasks.workunit.client.0.smithi114.stdout:8/172: dread d4/fc [0,4194304] 0 2024-02-16T22:21:50.022 INFO:tasks.workunit.client.1.smithi123.stdout:7/245: write d3/d12/d1d/d1e/f4a [1007756,4678] 0 2024-02-16T22:21:50.023 INFO:tasks.workunit.client.1.smithi123.stdout:2/245: dread d4/f37 [0,4194304] 0 2024-02-16T22:21:50.025 INFO:tasks.workunit.client.1.smithi123.stdout:1/210: dwrite d3/f8 [0,4194304] 0 2024-02-16T22:21:50.026 INFO:tasks.workunit.client.0.smithi114.stdout:3/131: truncate d4/f16 1000568 0 2024-02-16T22:21:50.028 INFO:tasks.workunit.client.0.smithi114.stdout:2/186: write fc [539060,78904] 0 2024-02-16T22:21:50.029 INFO:tasks.workunit.client.0.smithi114.stdout:2/187: chown dd/df/d2c/f34 15 1 2024-02-16T22:21:50.030 INFO:tasks.workunit.client.0.smithi114.stdout:2/188: chown l2 134970359 1 2024-02-16T22:21:50.031 INFO:tasks.workunit.client.1.smithi123.stdout:7/246: dwrite d3/f4e [0,4194304] 0 2024-02-16T22:21:50.031 INFO:tasks.workunit.client.0.smithi114.stdout:2/189: chown dd/df/d1c/d21/l2a 0 1 2024-02-16T22:21:50.031 INFO:tasks.workunit.client.0.smithi114.stdout:2/190: stat dd/df/d1c/d21/c38 0 2024-02-16T22:21:50.033 INFO:tasks.workunit.client.0.smithi114.stdout:2/191: chown dd/df/d1c/l35 14 1 2024-02-16T22:21:50.033 INFO:tasks.workunit.client.1.smithi123.stdout:9/297: creat d4/d56/d58/d59/f5c x:0 0 0 2024-02-16T22:21:50.036 INFO:tasks.workunit.client.0.smithi114.stdout:6/146: unlink d6/f23 0 2024-02-16T22:21:50.037 INFO:tasks.workunit.client.0.smithi114.stdout:6/147: readlink d6/l8 0 2024-02-16T22:21:50.038 INFO:tasks.workunit.client.0.smithi114.stdout:6/148: write d6/da/f2f [971526,24381] 0 2024-02-16T22:21:50.040 INFO:tasks.workunit.client.0.smithi114.stdout:9/177: write d2/fc [883187,17694] 0 2024-02-16T22:21:50.042 INFO:tasks.workunit.client.0.smithi114.stdout:0/202: rename d1/l4 to d1/l43 0 2024-02-16T22:21:50.044 INFO:tasks.workunit.client.1.smithi123.stdout:6/256: rename d0/d29 to d0/d9/dc/d5f 0 2024-02-16T22:21:50.051 INFO:tasks.workunit.client.0.smithi114.stdout:5/151: mknod d0/d28/c2a 0 2024-02-16T22:21:50.054 INFO:tasks.workunit.client.1.smithi123.stdout:2/246: creat d4/d1a/d1d/d13/d25/f4d x:0 0 0 2024-02-16T22:21:50.056 INFO:tasks.workunit.client.1.smithi123.stdout:1/211: creat d3/d6/db/d10/f4a x:0 0 0 2024-02-16T22:21:50.056 INFO:tasks.workunit.client.0.smithi114.stdout:5/152: dwrite d0/df/d16/f1f [0,4194304] 0 2024-02-16T22:21:50.058 INFO:tasks.workunit.client.1.smithi123.stdout:8/299: creat d0/d61/d66/f69 x:0 0 0 2024-02-16T22:21:50.058 INFO:tasks.workunit.client.0.smithi114.stdout:6/149: chown d6/f21 30780242 1 2024-02-16T22:21:50.061 INFO:tasks.workunit.client.1.smithi123.stdout:7/247: rmdir d3 39 2024-02-16T22:21:50.065 INFO:tasks.workunit.client.1.smithi123.stdout:9/298: mkdir d4/d1e/d5d 0 2024-02-16T22:21:50.065 INFO:tasks.workunit.client.0.smithi114.stdout:9/178: read d2/d1a/d23/d35/d1d/f36 [451626,69427] 0 2024-02-16T22:21:50.071 INFO:tasks.workunit.client.0.smithi114.stdout:4/249: truncate dd/d1b/f3e 27783 0 2024-02-16T22:21:50.072 INFO:tasks.workunit.client.1.smithi123.stdout:4/262: creat d5/d1c/d23/d47/f50 x:0 0 0 2024-02-16T22:21:50.073 INFO:tasks.workunit.client.0.smithi114.stdout:4/250: truncate dd/d1b/d24/f2f 49690 0 2024-02-16T22:21:50.073 INFO:tasks.workunit.client.1.smithi123.stdout:4/263: write f4 [1204630,117756] 0 2024-02-16T22:21:50.073 INFO:tasks.workunit.client.0.smithi114.stdout:4/251: stat dd/d2e/d56 0 2024-02-16T22:21:50.074 INFO:tasks.workunit.client.0.smithi114.stdout:8/173: rename d4/f6 to d4/d24/d2c/f32 0 2024-02-16T22:21:50.075 INFO:tasks.workunit.client.0.smithi114.stdout:8/174: fdatasync d4/d11/f2f 0 2024-02-16T22:21:50.078 INFO:tasks.workunit.client.0.smithi114.stdout:1/135: rename d4/fd to d4/f29 0 2024-02-16T22:21:50.078 INFO:tasks.workunit.client.0.smithi114.stdout:1/136: readlink d4/l26 0 2024-02-16T22:21:50.080 INFO:tasks.workunit.client.1.smithi123.stdout:3/210: creat d0/d20/d29/d34/f4b x:0 0 0 2024-02-16T22:21:50.081 INFO:tasks.workunit.client.1.smithi123.stdout:3/211: write d0/f7 [3681308,43492] 0 2024-02-16T22:21:50.082 INFO:tasks.workunit.client.1.smithi123.stdout:1/212: mkdir d3/d6/db/d10/d4b 0 2024-02-16T22:21:50.084 INFO:tasks.workunit.client.1.smithi123.stdout:8/300: creat d0/d29/d42/d47/f6a x:0 0 0 2024-02-16T22:21:50.084 INFO:tasks.workunit.client.1.smithi123.stdout:8/301: dread - d0/d12/f50 zero size 2024-02-16T22:21:50.085 INFO:tasks.workunit.client.1.smithi123.stdout:8/302: fdatasync d0/d12/f50 0 2024-02-16T22:21:50.085 INFO:tasks.workunit.client.1.smithi123.stdout:8/303: chown d0/f14 253813236 1 2024-02-16T22:21:50.087 INFO:tasks.workunit.client.0.smithi114.stdout:5/153: rmdir d0 39 2024-02-16T22:21:50.088 INFO:tasks.workunit.client.1.smithi123.stdout:0/216: link d1/d4/c1a d1/d8/c3a 0 2024-02-16T22:21:50.090 INFO:tasks.workunit.client.1.smithi123.stdout:8/304: dwrite d0/d12/f1e [0,4194304] 0 2024-02-16T22:21:50.091 INFO:tasks.workunit.client.1.smithi123.stdout:8/305: write d0/d12/f19 [1992598,88186] 0 2024-02-16T22:21:50.092 INFO:tasks.workunit.client.0.smithi114.stdout:9/179: readlink d2/d1a/d23/d35/d1d/l37 0 2024-02-16T22:21:50.092 INFO:tasks.workunit.client.1.smithi123.stdout:8/306: dread - d0/d61/d66/f69 zero size 2024-02-16T22:21:50.093 INFO:tasks.workunit.client.1.smithi123.stdout:0/217: dread d1/d4/d17/f32 [0,4194304] 0 2024-02-16T22:21:50.094 INFO:tasks.workunit.client.0.smithi114.stdout:7/163: link d3/d23/c31 d3/d22/c36 0 2024-02-16T22:21:50.094 INFO:tasks.workunit.client.1.smithi123.stdout:5/199: getdents d6/dd/df/d1f/d25 0 2024-02-16T22:21:50.095 INFO:tasks.workunit.client.1.smithi123.stdout:5/200: write d6/dd/df/d1f/d25/d37/f3e [422602,115141] 0 2024-02-16T22:21:50.096 INFO:tasks.workunit.client.0.smithi114.stdout:7/164: truncate d3/d14/d1b/f25 965325 0 2024-02-16T22:21:50.096 INFO:tasks.workunit.client.0.smithi114.stdout:4/252: rmdir dd/d1b/d24 39 2024-02-16T22:21:50.097 INFO:tasks.workunit.client.0.smithi114.stdout:7/165: write d3/fe [2004253,116148] 0 2024-02-16T22:21:50.098 INFO:tasks.workunit.client.0.smithi114.stdout:4/253: stat dd/d1b/f1d 0 2024-02-16T22:21:50.098 INFO:tasks.workunit.client.0.smithi114.stdout:7/166: stat d3/d23/c31 0 2024-02-16T22:21:50.100 INFO:tasks.workunit.client.1.smithi123.stdout:0/218: dwrite d1/d8/f34 [0,4194304] 0 2024-02-16T22:21:50.101 INFO:tasks.workunit.client.1.smithi123.stdout:9/299: symlink d4/d36/d57/l5e 0 2024-02-16T22:21:50.101 INFO:tasks.workunit.client.0.smithi114.stdout:1/137: fsync d4/f29 0 2024-02-16T22:21:50.102 INFO:tasks.workunit.client.1.smithi123.stdout:0/219: dread d1/d4/d17/f28 [0,4194304] 0 2024-02-16T22:21:50.103 INFO:tasks.workunit.client.0.smithi114.stdout:1/138: read d4/f15 [3487839,47523] 0 2024-02-16T22:21:50.103 INFO:tasks.workunit.client.0.smithi114.stdout:7/167: dread d3/fe [0,4194304] 0 2024-02-16T22:21:50.105 INFO:tasks.workunit.client.0.smithi114.stdout:3/132: rmdir d4/dd/d2c 0 2024-02-16T22:21:50.106 INFO:tasks.workunit.client.1.smithi123.stdout:6/257: creat d0/f60 x:0 0 0 2024-02-16T22:21:50.107 INFO:tasks.workunit.client.0.smithi114.stdout:2/192: creat dd/df/f41 x:0 0 0 2024-02-16T22:21:50.109 INFO:tasks.workunit.client.0.smithi114.stdout:5/154: chown d0/df 18359697 1 2024-02-16T22:21:50.110 INFO:tasks.workunit.client.0.smithi114.stdout:5/155: chown d0/l6 729802 1 2024-02-16T22:21:50.112 INFO:tasks.workunit.client.1.smithi123.stdout:6/258: dwrite d0/d9/dc/f17 [0,4194304] 0 2024-02-16T22:21:50.112 INFO:tasks.workunit.client.0.smithi114.stdout:7/168: dwrite d3/fe [4194304,4194304] 0 2024-02-16T22:21:50.113 INFO:tasks.workunit.client.1.smithi123.stdout:6/259: dread - d0/d6/d4d/f53 zero size 2024-02-16T22:21:50.113 INFO:tasks.workunit.client.1.smithi123.stdout:8/307: rmdir d0/d17/d26/d2a/d59 39 2024-02-16T22:21:50.117 INFO:tasks.workunit.client.0.smithi114.stdout:4/254: mkdir dd/d59 0 2024-02-16T22:21:50.118 INFO:tasks.workunit.client.1.smithi123.stdout:5/201: rmdir d6/dd/df/d1f/d25 39 2024-02-16T22:21:50.120 INFO:tasks.workunit.client.1.smithi123.stdout:3/212: dread d0/fc [0,4194304] 0 2024-02-16T22:21:50.122 INFO:tasks.workunit.client.1.smithi123.stdout:1/213: symlink d3/d6/db/l4c 0 2024-02-16T22:21:50.125 INFO:tasks.workunit.client.1.smithi123.stdout:7/248: creat d3/d12/d4d/f50 x:0 0 0 2024-02-16T22:21:50.127 INFO:tasks.workunit.client.1.smithi123.stdout:9/300: creat d4/d36/f5f x:0 0 0 2024-02-16T22:21:50.128 INFO:tasks.workunit.client.0.smithi114.stdout:1/139: mkdir d4/d9/d2a 0 2024-02-16T22:21:50.128 INFO:tasks.workunit.client.1.smithi123.stdout:9/301: chown d4/f10 114195471 1 2024-02-16T22:21:50.129 INFO:tasks.workunit.client.1.smithi123.stdout:9/302: dread - d4/d1e/d3a/f40 zero size 2024-02-16T22:21:50.130 INFO:tasks.workunit.client.0.smithi114.stdout:3/133: creat d4/d26/f31 x:0 0 0 2024-02-16T22:21:50.130 INFO:tasks.workunit.client.0.smithi114.stdout:9/180: dwrite d2/d1a/f21 [0,4194304] 0 2024-02-16T22:21:50.131 INFO:tasks.workunit.client.1.smithi123.stdout:4/264: dwrite d5/d1c/f25 [0,4194304] 0 2024-02-16T22:21:50.131 INFO:tasks.workunit.client.0.smithi114.stdout:3/134: write d4/dd/f29 [555126,130041] 0 2024-02-16T22:21:50.132 INFO:tasks.workunit.client.0.smithi114.stdout:9/181: dread - d2/d1a/d23/d35/f1e zero size 2024-02-16T22:21:50.132 INFO:tasks.workunit.client.1.smithi123.stdout:0/220: rename d1/d4/d17/c26 to d1/d8/d13/c3b 0 2024-02-16T22:21:50.133 INFO:tasks.workunit.client.0.smithi114.stdout:2/193: rmdir dd/df/d1c/d21 39 2024-02-16T22:21:50.136 INFO:tasks.workunit.client.0.smithi114.stdout:5/156: creat d0/df/f2b x:0 0 0 2024-02-16T22:21:50.138 INFO:tasks.workunit.client.0.smithi114.stdout:5/157: write d0/df/d16/d1e/f21 [1854272,78187] 0 2024-02-16T22:21:50.140 INFO:tasks.workunit.client.0.smithi114.stdout:0/203: getdents d1/d1a/d3e 0 2024-02-16T22:21:50.143 INFO:tasks.workunit.client.1.smithi123.stdout:8/308: read d0/d17/d26/f2d [4708677,83411] 0 2024-02-16T22:21:50.146 INFO:tasks.workunit.client.0.smithi114.stdout:7/169: symlink d3/d22/l37 0 2024-02-16T22:21:50.147 INFO:tasks.workunit.client.1.smithi123.stdout:2/247: getdents d4/d1a/d1d/d13/d25 0 2024-02-16T22:21:50.153 INFO:tasks.workunit.client.0.smithi114.stdout:6/150: rename d6/dd/d25/d26/f2e to d6/f30 0 2024-02-16T22:21:50.154 INFO:tasks.workunit.client.1.smithi123.stdout:5/202: dwrite d6/dd/df/d1f/d25/f17 [0,4194304] 0 2024-02-16T22:21:50.154 INFO:tasks.workunit.client.0.smithi114.stdout:6/151: read - d6/da/f1b zero size 2024-02-16T22:21:50.156 INFO:tasks.workunit.client.0.smithi114.stdout:8/175: link d4/d11/c1b d4/d16/d25/c33 0 2024-02-16T22:21:50.160 INFO:tasks.workunit.client.0.smithi114.stdout:3/135: dread f1 [4194304,4194304] 0 2024-02-16T22:21:50.162 INFO:tasks.workunit.client.0.smithi114.stdout:1/140: dread d4/d9/f27 [0,4194304] 0 2024-02-16T22:21:50.162 INFO:tasks.workunit.client.0.smithi114.stdout:6/152: dread d6/f18 [0,4194304] 0 2024-02-16T22:21:50.163 INFO:tasks.workunit.client.1.smithi123.stdout:9/303: mkdir d4/d56/d58/d59/d60 0 2024-02-16T22:21:50.164 INFO:tasks.workunit.client.1.smithi123.stdout:9/304: stat d4/d33/f38 0 2024-02-16T22:21:50.164 INFO:tasks.workunit.client.1.smithi123.stdout:9/305: chown d4/d1e/d3a/c4d 101 1 2024-02-16T22:21:50.168 INFO:tasks.workunit.client.1.smithi123.stdout:8/309: rmdir d0/d12 39 2024-02-16T22:21:50.170 INFO:tasks.workunit.client.0.smithi114.stdout:6/153: dwrite d6/da/f1b [0,4194304] 0 2024-02-16T22:21:50.171 INFO:tasks.workunit.client.0.smithi114.stdout:6/154: write d6/da/f2f [836562,28873] 0 2024-02-16T22:21:50.173 INFO:tasks.workunit.client.0.smithi114.stdout:7/170: read d3/f10 [74178,51006] 0 2024-02-16T22:21:50.175 INFO:tasks.workunit.client.0.smithi114.stdout:7/171: read d3/f27 [7302160,83490] 0 2024-02-16T22:21:50.177 INFO:tasks.workunit.client.1.smithi123.stdout:5/203: creat d6/dd/df/d2b/f43 x:0 0 0 2024-02-16T22:21:50.178 INFO:tasks.workunit.client.1.smithi123.stdout:5/204: fsync d6/dd/df/d1f/d25/f3a 0 2024-02-16T22:21:50.180 INFO:tasks.workunit.client.0.smithi114.stdout:8/176: rename d4/f19 to d4/f34 0 2024-02-16T22:21:50.181 INFO:tasks.workunit.client.0.smithi114.stdout:7/172: dwrite d3/f6 [0,4194304] 0 2024-02-16T22:21:50.182 INFO:tasks.workunit.client.0.smithi114.stdout:7/173: fsync d3/d14/f28 0 2024-02-16T22:21:50.182 INFO:tasks.workunit.client.1.smithi123.stdout:1/214: rename d3/d6/db/c27 to d3/d6/d18/c4d 0 2024-02-16T22:21:50.183 INFO:tasks.workunit.client.1.smithi123.stdout:1/215: dread - d3/d6/f46 zero size 2024-02-16T22:21:50.183 INFO:tasks.workunit.client.1.smithi123.stdout:0/221: dwrite d1/d8/f30 [0,4194304] 0 2024-02-16T22:21:50.184 INFO:tasks.workunit.client.1.smithi123.stdout:0/222: readlink d1/l2d 0 2024-02-16T22:21:50.185 INFO:tasks.workunit.client.0.smithi114.stdout:3/136: fdatasync d4/dd/f15 0 2024-02-16T22:21:50.185 INFO:tasks.workunit.client.1.smithi123.stdout:4/265: symlink d5/d1c/l51 0 2024-02-16T22:21:50.189 INFO:tasks.workunit.client.0.smithi114.stdout:7/174: dwrite d3/d14/f2c [0,4194304] 0 2024-02-16T22:21:50.192 INFO:tasks.workunit.client.0.smithi114.stdout:7/175: write d3/d22/f34 [142054,111300] 0 2024-02-16T22:21:50.194 INFO:tasks.workunit.client.1.smithi123.stdout:8/310: unlink d0/d17/d26/d2a/c2f 0 2024-02-16T22:21:50.197 INFO:tasks.workunit.client.1.smithi123.stdout:7/249: truncate f2 1721172 0 2024-02-16T22:21:50.198 INFO:tasks.workunit.client.0.smithi114.stdout:7/176: dread d3/d14/f18 [0,4194304] 0 2024-02-16T22:21:50.199 INFO:tasks.workunit.client.1.smithi123.stdout:7/250: write d3/d12/d1d/f2f [2439491,49320] 0 2024-02-16T22:21:50.199 INFO:tasks.workunit.client.1.smithi123.stdout:6/260: link d0/d6/d1a/l28 d0/d6/l61 0 2024-02-16T22:21:50.200 INFO:tasks.workunit.client.0.smithi114.stdout:9/182: truncate d2/d1a/f21 3084048 0 2024-02-16T22:21:50.201 INFO:tasks.workunit.client.0.smithi114.stdout:9/183: dread - d2/fb zero size 2024-02-16T22:21:50.201 INFO:tasks.workunit.client.0.smithi114.stdout:5/158: mknod d0/df/d16/c2c 0 2024-02-16T22:21:50.203 INFO:tasks.workunit.client.1.smithi123.stdout:3/213: creat d0/f4c x:0 0 0 2024-02-16T22:21:50.208 INFO:tasks.workunit.client.0.smithi114.stdout:5/159: dwrite d0/f9 [0,4194304] 0 2024-02-16T22:21:50.208 INFO:tasks.workunit.client.1.smithi123.stdout:9/306: link d4/d17/d2b/f5b d4/d56/f61 0 2024-02-16T22:21:50.210 INFO:tasks.workunit.client.0.smithi114.stdout:5/160: dread d0/df/d16/d1e/f29 [0,4194304] 0 2024-02-16T22:21:50.214 INFO:tasks.workunit.client.1.smithi123.stdout:8/311: write d0/d12/f19 [1643534,7494] 0 2024-02-16T22:21:50.220 INFO:tasks.workunit.client.1.smithi123.stdout:7/251: creat d3/d26/f51 x:0 0 0 2024-02-16T22:21:50.223 INFO:tasks.workunit.client.1.smithi123.stdout:5/205: mkdir d6/dd/d44 0 2024-02-16T22:21:50.227 INFO:tasks.workunit.client.0.smithi114.stdout:7/177: symlink d3/d22/l38 0 2024-02-16T22:21:50.228 INFO:tasks.workunit.client.1.smithi123.stdout:9/307: mknod d4/d56/d58/d59/c62 0 2024-02-16T22:21:50.228 INFO:tasks.workunit.client.0.smithi114.stdout:7/178: write d3/d14/f29 [1279303,104109] 0 2024-02-16T22:21:50.231 INFO:tasks.workunit.client.1.smithi123.stdout:8/312: creat d0/d17/d43/d34/f6b x:0 0 0 2024-02-16T22:21:50.232 INFO:tasks.workunit.client.0.smithi114.stdout:6/155: mknod d6/d2c/c31 0 2024-02-16T22:21:50.233 INFO:tasks.workunit.client.0.smithi114.stdout:4/255: getdents dd/d1b/d37/d38 0 2024-02-16T22:21:50.233 INFO:tasks.workunit.client.0.smithi114.stdout:6/156: truncate d6/dd/d25/f27 1008264 0 2024-02-16T22:21:50.235 INFO:tasks.workunit.client.1.smithi123.stdout:2/248: getdents d4/d1a/d1d 0 2024-02-16T22:21:50.235 INFO:tasks.workunit.client.1.smithi123.stdout:5/206: truncate d6/dd/df/d1f/d25/f2f 27525 0 2024-02-16T22:21:50.237 INFO:tasks.workunit.client.0.smithi114.stdout:8/177: unlink d4/d11/d15/l2a 0 2024-02-16T22:21:50.238 INFO:tasks.workunit.client.0.smithi114.stdout:8/178: readlink d4/l5 0 2024-02-16T22:21:50.238 INFO:tasks.workunit.client.1.smithi123.stdout:2/249: dread d4/d1a/d1d/d13/d25/f3b [0,4194304] 0 2024-02-16T22:21:50.238 INFO:tasks.workunit.client.1.smithi123.stdout:0/223: creat d1/f3c x:0 0 0 2024-02-16T22:21:50.239 INFO:tasks.workunit.client.1.smithi123.stdout:0/224: write d1/d4/f11 [1202217,123080] 0 2024-02-16T22:21:50.239 INFO:tasks.workunit.client.0.smithi114.stdout:4/256: dwrite dd/ff [0,4194304] 0 2024-02-16T22:21:50.240 INFO:tasks.workunit.client.1.smithi123.stdout:1/216: rename d3/d6/db/d10/d2d/f39 to d3/f4e 0 2024-02-16T22:21:50.243 INFO:tasks.workunit.client.0.smithi114.stdout:2/194: link c5 dd/df/d1c/c42 0 2024-02-16T22:21:50.244 INFO:tasks.workunit.client.1.smithi123.stdout:2/250: dwrite d4/d1a/d1d/f9 [0,4194304] 0 2024-02-16T22:21:50.244 INFO:tasks.workunit.client.1.smithi123.stdout:8/313: chown d0/d17/d26/d2a/d59 19205 1 2024-02-16T22:21:50.246 INFO:tasks.workunit.client.1.smithi123.stdout:6/261: creat d0/d9/f62 x:0 0 0 2024-02-16T22:21:50.247 INFO:tasks.workunit.client.1.smithi123.stdout:1/217: dwrite d3/d6/d11/f23 [0,4194304] 0 2024-02-16T22:21:50.247 INFO:tasks.workunit.client.0.smithi114.stdout:8/179: dwrite d4/d24/f29 [0,4194304] 0 2024-02-16T22:21:50.247 INFO:tasks.workunit.client.1.smithi123.stdout:7/252: mkdir d3/d12/d52 0 2024-02-16T22:21:50.248 INFO:tasks.workunit.client.1.smithi123.stdout:6/262: read - d0/d9/dc/d5f/f3e zero size 2024-02-16T22:21:50.248 INFO:tasks.workunit.client.0.smithi114.stdout:9/184: mkdir d2/dd/d3c 0 2024-02-16T22:21:50.255 INFO:tasks.workunit.client.0.smithi114.stdout:7/179: unlink d3/d14/d1b/f25 0 2024-02-16T22:21:50.255 INFO:tasks.workunit.client.0.smithi114.stdout:9/185: dread d2/f4 [4194304,4194304] 0 2024-02-16T22:21:50.256 INFO:tasks.workunit.client.1.smithi123.stdout:7/253: read d3/d12/f14 [1252014,19703] 0 2024-02-16T22:21:50.256 INFO:tasks.workunit.client.0.smithi114.stdout:9/186: fdatasync d2/dd/f2a 0 2024-02-16T22:21:50.258 INFO:tasks.workunit.client.0.smithi114.stdout:9/187: dread d2/d1a/d23/d35/d1f/f22 [0,4194304] 0 2024-02-16T22:21:50.260 INFO:tasks.workunit.client.0.smithi114.stdout:5/161: mkdir d0/d2d 0 2024-02-16T22:21:50.262 INFO:tasks.workunit.client.0.smithi114.stdout:1/141: getdents d4/d9 0 2024-02-16T22:21:50.265 INFO:tasks.workunit.client.1.smithi123.stdout:3/214: rename d0/d20/d29/d34/d37/c3f to d0/d20/d29/d34/d37/d49/c4d 0 2024-02-16T22:21:50.268 INFO:tasks.workunit.client.0.smithi114.stdout:3/137: rename d4/f1a to d4/d24/f32 0 2024-02-16T22:21:50.269 INFO:tasks.workunit.client.0.smithi114.stdout:9/188: dwrite d2/f13 [0,4194304] 0 2024-02-16T22:21:50.270 INFO:tasks.workunit.client.1.smithi123.stdout:3/215: dwrite d0/f4c [0,4194304] 0 2024-02-16T22:21:50.272 INFO:tasks.workunit.client.0.smithi114.stdout:1/142: dwrite d4/f21 [0,4194304] 0 2024-02-16T22:21:50.273 INFO:tasks.workunit.client.0.smithi114.stdout:9/189: write d2/d1a/d23/d35/f1e [881015,15163] 0 2024-02-16T22:21:50.273 INFO:tasks.workunit.client.1.smithi123.stdout:4/266: sync 2024-02-16T22:21:50.273 INFO:tasks.workunit.client.1.smithi123.stdout:9/308: link d4/f12 d4/d56/d58/d59/d60/f63 0 2024-02-16T22:21:50.276 INFO:tasks.workunit.client.0.smithi114.stdout:3/138: dread d4/dd/f29 [0,4194304] 0 2024-02-16T22:21:50.277 INFO:tasks.workunit.client.0.smithi114.stdout:2/195: stat dd/f36 0 2024-02-16T22:21:50.278 INFO:tasks.workunit.client.0.smithi114.stdout:3/139: write d4/d26/f28 [151340,112615] 0 2024-02-16T22:21:50.284 INFO:tasks.workunit.client.1.smithi123.stdout:1/218: mkdir d3/d6/db/d10/d4f 0 2024-02-16T22:21:50.287 INFO:tasks.workunit.client.1.smithi123.stdout:6/263: creat d0/d6/f63 x:0 0 0 2024-02-16T22:21:50.292 INFO:tasks.workunit.client.0.smithi114.stdout:0/204: link d1/d16/l2b d1/d16/l44 0 2024-02-16T22:21:50.297 INFO:tasks.workunit.client.1.smithi123.stdout:0/225: mkdir d1/d8/d3d 0 2024-02-16T22:21:50.305 INFO:tasks.workunit.client.1.smithi123.stdout:9/309: rename d4/d33/f4c to d4/d17/d2b/f64 0 2024-02-16T22:21:50.305 INFO:tasks.workunit.client.1.smithi123.stdout:9/310: chown d4/d1e/d3a/c4d 1463952 1 2024-02-16T22:21:50.306 INFO:tasks.workunit.client.0.smithi114.stdout:9/190: fdatasync d2/f6 0 2024-02-16T22:21:50.308 INFO:tasks.workunit.client.0.smithi114.stdout:3/140: mkdir d4/dd/d33 0 2024-02-16T22:21:50.310 INFO:tasks.workunit.client.1.smithi123.stdout:9/311: dread d4/d17/d2b/f64 [0,4194304] 0 2024-02-16T22:21:50.310 INFO:tasks.workunit.client.1.smithi123.stdout:8/314: dwrite d0/d17/d43/f27 [0,4194304] 0 2024-02-16T22:21:50.311 INFO:tasks.workunit.client.1.smithi123.stdout:9/312: dread - d4/d56/d58/f5a zero size 2024-02-16T22:21:50.313 INFO:tasks.workunit.client.0.smithi114.stdout:8/180: dread d4/f7 [0,4194304] 0 2024-02-16T22:21:50.320 INFO:tasks.workunit.client.0.smithi114.stdout:4/257: symlink dd/d1b/d24/l5a 0 2024-02-16T22:21:50.322 INFO:tasks.workunit.client.1.smithi123.stdout:1/219: dread d3/d6/d11/f17 [0,4194304] 0 2024-02-16T22:21:50.323 INFO:tasks.workunit.client.0.smithi114.stdout:1/143: symlink d4/d9/d2a/l2b 0 2024-02-16T22:21:50.325 INFO:tasks.workunit.client.1.smithi123.stdout:0/226: chown d1/d8/d13/l24 3714537 1 2024-02-16T22:21:50.326 INFO:tasks.workunit.client.0.smithi114.stdout:3/141: creat d4/d26/f34 x:0 0 0 2024-02-16T22:21:50.326 INFO:tasks.workunit.client.0.smithi114.stdout:3/142: chown f1 14 1 2024-02-16T22:21:50.327 INFO:tasks.workunit.client.1.smithi123.stdout:6/264: dwrite d0/d9/dc/f11 [0,4194304] 0 2024-02-16T22:21:50.333 INFO:tasks.workunit.client.1.smithi123.stdout:2/251: link d4/f37 d4/f4e 0 2024-02-16T22:21:50.335 INFO:tasks.workunit.client.0.smithi114.stdout:6/157: rename d6/dd/c14 to d6/c32 0 2024-02-16T22:21:50.336 INFO:tasks.workunit.client.0.smithi114.stdout:6/158: fsync d6/da/f2f 0 2024-02-16T22:21:50.336 INFO:tasks.workunit.client.1.smithi123.stdout:7/254: truncate d3/d12/d1d/f22 299102 0 2024-02-16T22:21:50.336 INFO:tasks.workunit.client.1.smithi123.stdout:8/315: mknod d0/d17/d26/d2a/d4d/c6c 0 2024-02-16T22:21:50.337 INFO:tasks.workunit.client.1.smithi123.stdout:8/316: chown d0/d29/d42/d47 1500778690 1 2024-02-16T22:21:50.338 INFO:tasks.workunit.client.1.smithi123.stdout:8/317: write d0/d17/d26/d2a/d3c/f55 [970960,10218] 0 2024-02-16T22:21:50.339 INFO:tasks.workunit.client.1.smithi123.stdout:9/313: chown d4/d17/d20/l31 0 1 2024-02-16T22:21:50.339 INFO:tasks.workunit.client.1.smithi123.stdout:8/318: stat d0/d17/d26/d32/d45 0 2024-02-16T22:21:50.340 INFO:tasks.workunit.client.0.smithi114.stdout:8/181: rename d4/d16/l30 to d4/d16/l35 0 2024-02-16T22:21:50.340 INFO:tasks.workunit.client.1.smithi123.stdout:9/314: write d4/d17/d24/f28 [496430,64012] 0 2024-02-16T22:21:50.341 INFO:tasks.workunit.client.0.smithi114.stdout:4/258: truncate dd/f19 3049800 0 2024-02-16T22:21:50.341 INFO:tasks.workunit.client.0.smithi114.stdout:4/259: chown dd/l28 93553 1 2024-02-16T22:21:50.342 INFO:tasks.workunit.client.1.smithi123.stdout:4/267: truncate d5/d49/f4e 2650390 0 2024-02-16T22:21:50.342 INFO:tasks.workunit.client.1.smithi123.stdout:4/268: dread - d5/d1c/d23/f43 zero size 2024-02-16T22:21:50.343 INFO:tasks.workunit.client.1.smithi123.stdout:4/269: fsync d5/f1e 0 2024-02-16T22:21:50.343 INFO:tasks.workunit.client.0.smithi114.stdout:6/159: dread d6/dd/f29 [0,4194304] 0 2024-02-16T22:21:50.343 INFO:tasks.workunit.client.0.smithi114.stdout:4/260: write dd/ff [2667692,122293] 0 2024-02-16T22:21:50.343 INFO:tasks.workunit.client.0.smithi114.stdout:0/205: truncate d1/d1a/f3b 2579124 0 2024-02-16T22:21:50.344 INFO:tasks.workunit.client.1.smithi123.stdout:5/207: link d6/l14 d6/dd/df/d2b/d3f/l45 0 2024-02-16T22:21:50.344 INFO:tasks.workunit.client.0.smithi114.stdout:6/160: chown d6/l10 1 1 2024-02-16T22:21:50.346 INFO:tasks.workunit.client.1.smithi123.stdout:1/220: creat d3/d6/d49/f50 x:0 0 0 2024-02-16T22:21:50.346 INFO:tasks.workunit.client.0.smithi114.stdout:9/191: creat d2/dd/d3c/f3d x:0 0 0 2024-02-16T22:21:50.349 INFO:tasks.workunit.client.1.smithi123.stdout:5/208: dwrite d6/dd/df/d1f/d25/f3a [0,4194304] 0 2024-02-16T22:21:50.350 INFO:tasks.workunit.client.0.smithi114.stdout:7/180: link d3/d22/l33 d3/l39 0 2024-02-16T22:21:50.352 INFO:tasks.workunit.client.0.smithi114.stdout:0/206: dread d1/f8 [0,4194304] 0 2024-02-16T22:21:50.353 INFO:tasks.workunit.client.1.smithi123.stdout:2/252: unlink d4/f4e 0 2024-02-16T22:21:50.360 INFO:tasks.workunit.client.1.smithi123.stdout:8/319: creat d0/d17/d26/f6d x:0 0 0 2024-02-16T22:21:50.361 INFO:tasks.workunit.client.0.smithi114.stdout:8/182: mkdir d4/d11/d15/d36 0 2024-02-16T22:21:50.362 INFO:tasks.workunit.client.1.smithi123.stdout:9/315: mknod d4/d36/c65 0 2024-02-16T22:21:50.363 INFO:tasks.workunit.client.1.smithi123.stdout:9/316: write d4/f21 [4181468,73563] 0 2024-02-16T22:21:50.367 INFO:tasks.workunit.client.1.smithi123.stdout:9/317: read d4/f1b [1126582,57716] 0 2024-02-16T22:21:50.368 INFO:tasks.workunit.client.1.smithi123.stdout:2/253: dread d4/d1a/f39 [0,4194304] 0 2024-02-16T22:21:50.370 INFO:tasks.workunit.client.1.smithi123.stdout:1/221: unlink d3/l4 0 2024-02-16T22:21:50.373 INFO:tasks.workunit.client.0.smithi114.stdout:9/192: mknod d2/d1a/d23/d35/d1f/c3e 0 2024-02-16T22:21:50.374 INFO:tasks.workunit.client.1.smithi123.stdout:3/216: link d0/f6 d0/d20/d29/d34/f4e 0 2024-02-16T22:21:50.375 INFO:tasks.workunit.client.0.smithi114.stdout:9/193: read - d2/dd/d3c/f3d zero size 2024-02-16T22:21:50.376 INFO:tasks.workunit.client.1.smithi123.stdout:5/209: readlink d6/l38 0 2024-02-16T22:21:50.376 INFO:tasks.workunit.client.1.smithi123.stdout:5/210: chown l5 1973055 1 2024-02-16T22:21:50.377 INFO:tasks.workunit.client.0.smithi114.stdout:9/194: fdatasync d2/dd/f3b 0 2024-02-16T22:21:50.377 INFO:tasks.workunit.client.0.smithi114.stdout:3/143: dwrite f1 [8388608,4194304] 0 2024-02-16T22:21:50.377 INFO:tasks.workunit.client.0.smithi114.stdout:3/144: dread - d4/d26/f34 zero size 2024-02-16T22:21:50.378 INFO:tasks.workunit.client.0.smithi114.stdout:7/181: rmdir d3 39 2024-02-16T22:21:50.380 INFO:tasks.workunit.client.1.smithi123.stdout:3/217: dwrite d0/d20/d29/f44 [0,4194304] 0 2024-02-16T22:21:50.381 INFO:tasks.workunit.client.0.smithi114.stdout:0/207: fsync d1/d16/d1f/d28/f35 0 2024-02-16T22:21:50.382 INFO:tasks.workunit.client.0.smithi114.stdout:0/208: truncate d1/f2a 1001877 0 2024-02-16T22:21:50.389 INFO:tasks.workunit.client.1.smithi123.stdout:9/318: symlink d4/d17/d24/l66 0 2024-02-16T22:21:50.395 INFO:tasks.workunit.client.0.smithi114.stdout:3/145: creat d4/dd/f35 x:0 0 0 2024-02-16T22:21:50.395 INFO:tasks.workunit.client.0.smithi114.stdout:3/146: chown d4/d24 5332105 1 2024-02-16T22:21:50.396 INFO:tasks.workunit.client.1.smithi123.stdout:5/211: truncate d6/dd/df/d1f/d25/f1d 1071997 0 2024-02-16T22:21:50.396 INFO:tasks.workunit.client.0.smithi114.stdout:2/196: rename dd/df/d1c/l2d to dd/df/l43 0 2024-02-16T22:21:50.398 INFO:tasks.workunit.client.1.smithi123.stdout:0/227: rename d1/d8/d13/d1d/l23 to d1/l3e 0 2024-02-16T22:21:50.398 INFO:tasks.workunit.client.0.smithi114.stdout:0/209: creat d1/d1a/f45 x:0 0 0 2024-02-16T22:21:50.399 INFO:tasks.workunit.client.1.smithi123.stdout:6/265: creat d0/d6/d1a/f64 x:0 0 0 2024-02-16T22:21:50.401 INFO:tasks.workunit.client.0.smithi114.stdout:8/183: creat d4/d11/d15/d31/f37 x:0 0 0 2024-02-16T22:21:50.401 INFO:tasks.workunit.client.1.smithi123.stdout:3/218: creat d0/d20/d2e/f4f x:0 0 0 2024-02-16T22:21:50.402 INFO:tasks.workunit.client.0.smithi114.stdout:1/144: link d4/c24 d4/c2c 0 2024-02-16T22:21:50.403 INFO:tasks.workunit.client.1.smithi123.stdout:8/320: getdents d0/d17/d26/d68 0 2024-02-16T22:21:50.403 INFO:tasks.workunit.client.0.smithi114.stdout:4/261: creat dd/d1b/f5b x:0 0 0 2024-02-16T22:21:50.404 INFO:tasks.workunit.client.0.smithi114.stdout:6/161: link d6/dd/d25/d26/l2b d6/dd/d25/l33 0 2024-02-16T22:21:50.404 INFO:tasks.workunit.client.0.smithi114.stdout:0/210: dwrite d1/fe [4194304,4194304] 0 2024-02-16T22:21:50.406 INFO:tasks.workunit.client.0.smithi114.stdout:9/195: mkdir d2/d33/d3f 0 2024-02-16T22:21:50.407 INFO:tasks.workunit.client.1.smithi123.stdout:9/319: write d4/f1b [1634970,32731] 0 2024-02-16T22:21:50.408 INFO:tasks.workunit.client.1.smithi123.stdout:9/320: write d4/f26 [3191799,23086] 0 2024-02-16T22:21:50.409 INFO:tasks.workunit.client.1.smithi123.stdout:4/270: creat d5/d1c/d23/f52 x:0 0 0 2024-02-16T22:21:50.411 INFO:tasks.workunit.client.0.smithi114.stdout:7/182: rename d3/d22/l33 to d3/d14/d1b/l3a 0 2024-02-16T22:21:50.411 INFO:tasks.workunit.client.1.smithi123.stdout:4/271: write d5/d1c/d23/d36/f45 [2558501,12877] 0 2024-02-16T22:21:50.412 INFO:tasks.workunit.client.1.smithi123.stdout:1/222: creat d3/d6/db/d10/d4f/f51 x:0 0 0 2024-02-16T22:21:50.419 INFO:tasks.workunit.client.0.smithi114.stdout:7/183: dwrite d3/d22/f2b [0,4194304] 0 2024-02-16T22:21:50.419 INFO:tasks.workunit.client.1.smithi123.stdout:7/255: rename d3/d26/l3a to d3/d12/d4d/l53 0 2024-02-16T22:21:50.420 INFO:tasks.workunit.client.0.smithi114.stdout:7/184: read d3/d22/f2b [879579,63775] 0 2024-02-16T22:21:50.426 INFO:tasks.workunit.client.0.smithi114.stdout:0/211: dread d1/fe [0,4194304] 0 2024-02-16T22:21:50.428 INFO:tasks.workunit.client.0.smithi114.stdout:6/162: mkdir d6/dd/d25/d34 0 2024-02-16T22:21:50.428 INFO:tasks.workunit.client.0.smithi114.stdout:6/163: chown d6/dd 25916 1 2024-02-16T22:21:50.429 INFO:tasks.workunit.client.1.smithi123.stdout:8/321: symlink d0/d17/d26/d2a/d3c/l6e 0 2024-02-16T22:21:50.429 INFO:tasks.workunit.client.0.smithi114.stdout:9/196: symlink d2/d1a/d23/d35/d1f/l40 0 2024-02-16T22:21:50.431 INFO:tasks.workunit.client.0.smithi114.stdout:3/147: symlink d4/dd/d33/l36 0 2024-02-16T22:21:50.434 INFO:tasks.workunit.client.0.smithi114.stdout:8/184: rename d4/f34 to d4/d24/f38 0 2024-02-16T22:21:50.443 INFO:tasks.workunit.client.0.smithi114.stdout:7/185: creat d3/d23/f3b x:0 0 0 2024-02-16T22:21:50.446 INFO:tasks.workunit.client.0.smithi114.stdout:4/262: creat dd/d59/f5c x:0 0 0 2024-02-16T22:21:50.446 INFO:tasks.workunit.client.1.smithi123.stdout:0/228: mknod d1/d8/d3d/c3f 0 2024-02-16T22:21:50.449 INFO:tasks.workunit.client.0.smithi114.stdout:0/212: fdatasync d1/f7 0 2024-02-16T22:21:50.450 INFO:tasks.workunit.client.0.smithi114.stdout:7/186: dwrite d3/d22/f34 [0,4194304] 0 2024-02-16T22:21:50.450 INFO:tasks.workunit.client.1.smithi123.stdout:7/256: dwrite d3/d12/d24/f49 [0,4194304] 0 2024-02-16T22:21:50.453 INFO:tasks.workunit.client.0.smithi114.stdout:6/164: write d6/dd/f29 [128226,129992] 0 2024-02-16T22:21:50.456 INFO:tasks.workunit.client.1.smithi123.stdout:2/254: sync 2024-02-16T22:21:50.456 INFO:tasks.workunit.client.1.smithi123.stdout:5/212: truncate d6/dd/df/d1f/f24 5113692 0 2024-02-16T22:21:50.458 INFO:tasks.workunit.client.0.smithi114.stdout:3/148: truncate d4/dd/f1e 658941 0 2024-02-16T22:21:50.460 INFO:tasks.workunit.client.1.smithi123.stdout:8/322: rename d0/d17/d26/d2a/f52 to d0/d17/d26/d32/f6f 0 2024-02-16T22:21:50.460 INFO:tasks.workunit.client.1.smithi123.stdout:4/272: dread d5/d1c/f30 [0,4194304] 0 2024-02-16T22:21:50.462 INFO:tasks.workunit.client.0.smithi114.stdout:2/197: creat dd/df/f44 x:0 0 0 2024-02-16T22:21:50.463 INFO:tasks.workunit.client.1.smithi123.stdout:1/223: mkdir d3/d52 0 2024-02-16T22:21:50.463 INFO:tasks.workunit.client.1.smithi123.stdout:2/255: dwrite d4/d1a/d1d/f2b [0,4194304] 0 2024-02-16T22:21:50.465 INFO:tasks.workunit.client.0.smithi114.stdout:1/145: rename d4/f29 to d4/f2d 0 2024-02-16T22:21:50.465 INFO:tasks.workunit.client.1.smithi123.stdout:0/229: mknod d1/d4/d17/c40 0 2024-02-16T22:21:50.468 INFO:tasks.workunit.client.1.smithi123.stdout:6/266: creat d0/f65 x:0 0 0 2024-02-16T22:21:50.469 INFO:tasks.workunit.client.0.smithi114.stdout:4/263: creat dd/d1b/d37/f5d x:0 0 0 2024-02-16T22:21:50.469 INFO:tasks.workunit.client.0.smithi114.stdout:4/264: readlink dd/l2d 0 2024-02-16T22:21:50.470 INFO:tasks.workunit.client.1.smithi123.stdout:3/219: truncate d0/fd 58401 0 2024-02-16T22:21:50.470 INFO:tasks.workunit.client.1.smithi123.stdout:6/267: write d0/d9/f62 [181690,101437] 0 2024-02-16T22:21:50.472 INFO:tasks.workunit.client.1.smithi123.stdout:0/230: dwrite d1/f3c [0,4194304] 0 2024-02-16T22:21:50.473 INFO:tasks.workunit.client.1.smithi123.stdout:0/231: dread - d1/d8/f2f zero size 2024-02-16T22:21:50.473 INFO:tasks.workunit.client.0.smithi114.stdout:0/213: symlink d1/d1a/d3e/d41/l46 0 2024-02-16T22:21:50.477 INFO:tasks.workunit.client.0.smithi114.stdout:7/187: write d3/fe [1205345,122961] 0 2024-02-16T22:21:50.481 INFO:tasks.workunit.client.1.smithi123.stdout:5/213: rename l5 to d6/dd/df/l46 0 2024-02-16T22:21:50.483 INFO:tasks.workunit.client.0.smithi114.stdout:0/214: dwrite d1/d1a/f36 [0,4194304] 0 2024-02-16T22:21:50.484 INFO:tasks.workunit.client.0.smithi114.stdout:0/215: chown d1/d1a/d3e 19 1 2024-02-16T22:21:50.487 INFO:tasks.workunit.client.0.smithi114.stdout:9/197: symlink d2/d1a/d23/d32/l41 0 2024-02-16T22:21:50.489 INFO:tasks.workunit.client.0.smithi114.stdout:3/149: dread d4/dd/f19 [0,4194304] 0 2024-02-16T22:21:50.489 INFO:tasks.workunit.client.1.smithi123.stdout:8/323: unlink d0/d17/d26/f44 0 2024-02-16T22:21:50.491 INFO:tasks.workunit.client.0.smithi114.stdout:3/150: truncate d4/d26/f34 786952 0 2024-02-16T22:21:50.493 INFO:tasks.workunit.client.0.smithi114.stdout:8/185: mkdir d4/d16/d25/d39 0 2024-02-16T22:21:50.494 INFO:tasks.workunit.client.1.smithi123.stdout:1/224: mkdir d3/d6/db/d10/d2a/d53 0 2024-02-16T22:21:50.494 INFO:tasks.workunit.client.0.smithi114.stdout:9/198: dwrite d2/d1a/d23/d35/f31 [0,4194304] 0 2024-02-16T22:21:50.496 INFO:tasks.workunit.client.0.smithi114.stdout:2/198: truncate dd/df/f33 4323667 0 2024-02-16T22:21:50.500 INFO:tasks.workunit.client.1.smithi123.stdout:7/257: symlink d3/d26/d3d/l54 0 2024-02-16T22:21:50.503 INFO:tasks.workunit.client.1.smithi123.stdout:3/220: truncate d0/fe 1380074 0 2024-02-16T22:21:50.504 INFO:tasks.workunit.client.1.smithi123.stdout:7/258: dread d3/fd [0,4194304] 0 2024-02-16T22:21:50.505 INFO:tasks.workunit.client.0.smithi114.stdout:4/265: creat dd/d1b/d37/d38/f5e x:0 0 0 2024-02-16T22:21:50.505 INFO:tasks.workunit.client.1.smithi123.stdout:0/232: readlink d1/l3e 0 2024-02-16T22:21:50.506 INFO:tasks.workunit.client.0.smithi114.stdout:1/146: dwrite d4/d9/f27 [0,4194304] 0 2024-02-16T22:21:50.509 INFO:tasks.workunit.client.1.smithi123.stdout:6/268: rename d0/d6/c20 to d0/d9/dc/d5f/c66 0 2024-02-16T22:21:50.510 INFO:tasks.workunit.client.1.smithi123.stdout:6/269: write d0/d6/f63 [77756,23857] 0 2024-02-16T22:21:50.513 INFO:tasks.workunit.client.1.smithi123.stdout:9/321: getdents d4/d56/d58 0 2024-02-16T22:21:50.515 INFO:tasks.workunit.client.0.smithi114.stdout:7/188: symlink d3/d23/l3c 0 2024-02-16T22:21:50.516 INFO:tasks.workunit.client.0.smithi114.stdout:1/147: dwrite d4/f21 [0,4194304] 0 2024-02-16T22:21:50.516 INFO:tasks.workunit.client.1.smithi123.stdout:9/322: chown d4/d17/d24/f49 264219887 1 2024-02-16T22:21:50.516 INFO:tasks.workunit.client.0.smithi114.stdout:4/266: dwrite dd/d1b/f5b [0,4194304] 0 2024-02-16T22:21:50.517 INFO:tasks.workunit.client.0.smithi114.stdout:1/148: chown d4/f28 170 1 2024-02-16T22:21:50.518 INFO:tasks.workunit.client.1.smithi123.stdout:6/270: dwrite d0/f54 [0,4194304] 0 2024-02-16T22:21:50.522 INFO:tasks.workunit.client.1.smithi123.stdout:1/225: unlink d3/d6/db/c13 0 2024-02-16T22:21:50.522 INFO:tasks.workunit.client.0.smithi114.stdout:7/189: write d3/f27 [6994908,75819] 0 2024-02-16T22:21:50.522 INFO:tasks.workunit.client.0.smithi114.stdout:0/216: mkdir d1/d1a/d47 0 2024-02-16T22:21:50.522 INFO:tasks.workunit.client.0.smithi114.stdout:7/190: chown d3/d14/f18 14022 1 2024-02-16T22:21:50.523 INFO:tasks.workunit.client.0.smithi114.stdout:0/217: chown d1/d16/d1f/d28/f35 212 1 2024-02-16T22:21:50.523 INFO:tasks.workunit.client.1.smithi123.stdout:1/226: fsync d3/d6/f46 0 2024-02-16T22:21:50.528 INFO:tasks.workunit.client.1.smithi123.stdout:0/233: chown d1/d8/d13/f15 13 1 2024-02-16T22:21:50.529 INFO:tasks.workunit.client.1.smithi123.stdout:0/234: fdatasync d1/d4/f11 0 2024-02-16T22:21:50.530 INFO:tasks.workunit.client.1.smithi123.stdout:5/214: creat d6/dd/d44/f47 x:0 0 0 2024-02-16T22:21:50.531 INFO:tasks.workunit.client.1.smithi123.stdout:0/235: chown d1/f22 24 1 2024-02-16T22:21:50.532 INFO:tasks.workunit.client.0.smithi114.stdout:2/199: readlink dd/df/d2c/l31 0 2024-02-16T22:21:50.533 INFO:tasks.workunit.client.1.smithi123.stdout:7/259: dread d3/d12/d1d/f2f [0,4194304] 0 2024-02-16T22:21:50.538 INFO:tasks.workunit.client.1.smithi123.stdout:9/323: creat d4/d36/d57/f67 x:0 0 0 2024-02-16T22:21:50.540 INFO:tasks.workunit.client.1.smithi123.stdout:8/324: symlink d0/d17/l70 0 2024-02-16T22:21:50.540 INFO:tasks.workunit.client.1.smithi123.stdout:8/325: fdatasync d0/f6 0 2024-02-16T22:21:50.542 INFO:tasks.workunit.client.1.smithi123.stdout:6/271: rename d0/d9/dc/f11 to d0/d9/dc/f67 0 2024-02-16T22:21:50.542 INFO:tasks.workunit.client.1.smithi123.stdout:6/272: chown d0/d6/d2f/f22 283324 1 2024-02-16T22:21:50.544 INFO:tasks.workunit.client.1.smithi123.stdout:2/256: creat d4/f4f x:0 0 0 2024-02-16T22:21:50.545 INFO:tasks.workunit.client.0.smithi114.stdout:1/149: symlink d4/d9/d2a/l2e 0 2024-02-16T22:21:50.546 INFO:tasks.workunit.client.1.smithi123.stdout:3/221: mkdir d0/d50 0 2024-02-16T22:21:50.547 INFO:tasks.workunit.client.0.smithi114.stdout:6/165: creat d6/f35 x:0 0 0 2024-02-16T22:21:50.550 INFO:tasks.workunit.client.1.smithi123.stdout:3/222: read d0/d20/d29/d34/f4e [2571103,24661] 0 2024-02-16T22:21:50.550 INFO:tasks.workunit.client.0.smithi114.stdout:0/218: chown d1/d1a/f2c 3525131 1 2024-02-16T22:21:50.551 INFO:tasks.workunit.client.0.smithi114.stdout:8/186: getdents d4/d11/d15/d36 0 2024-02-16T22:21:50.551 INFO:tasks.workunit.client.1.smithi123.stdout:3/223: chown d0/d20/d2e/f39 300287 1 2024-02-16T22:21:50.552 INFO:tasks.workunit.client.0.smithi114.stdout:9/199: truncate d2/f15 1254898 0 2024-02-16T22:21:50.553 INFO:tasks.workunit.client.1.smithi123.stdout:5/215: dread d6/dd/df/d1f/d25/f39 [0,4194304] 0 2024-02-16T22:21:50.553 INFO:tasks.workunit.client.1.smithi123.stdout:7/260: creat d3/d12/d24/f55 x:0 0 0 2024-02-16T22:21:50.554 INFO:tasks.workunit.client.1.smithi123.stdout:9/324: symlink d4/d17/d24/l68 0 2024-02-16T22:21:50.555 INFO:tasks.workunit.client.0.smithi114.stdout:4/267: mknod dd/d2e/d56/c5f 0 2024-02-16T22:21:50.556 INFO:tasks.workunit.client.0.smithi114.stdout:1/150: creat d4/d9/f2f x:0 0 0 2024-02-16T22:21:50.557 INFO:tasks.workunit.client.1.smithi123.stdout:8/326: chown d0/d17/c21 75851447 1 2024-02-16T22:21:50.557 INFO:tasks.workunit.client.0.smithi114.stdout:1/151: fsync d4/d9/f19 0 2024-02-16T22:21:50.561 INFO:tasks.workunit.client.1.smithi123.stdout:6/273: rename d0/d6/d4d/f53 to d0/d6/d2f/d5d/f68 0 2024-02-16T22:21:50.562 INFO:tasks.workunit.client.0.smithi114.stdout:0/219: read d1/d1a/f3b [1041713,42912] 0 2024-02-16T22:21:50.562 INFO:tasks.workunit.client.0.smithi114.stdout:4/268: dwrite dd/d1b/d24/f4a [0,4194304] 0 2024-02-16T22:21:50.563 INFO:tasks.workunit.client.1.smithi123.stdout:2/257: mknod d4/d1a/d1d/c50 0 2024-02-16T22:21:50.563 INFO:tasks.workunit.client.1.smithi123.stdout:3/224: dread d0/d20/d29/d34/f4e [0,4194304] 0 2024-02-16T22:21:50.563 INFO:tasks.workunit.client.1.smithi123.stdout:3/225: dread - d0/d20/d2e/f2f zero size 2024-02-16T22:21:50.566 INFO:tasks.workunit.client.0.smithi114.stdout:2/200: rmdir dd/df/d1c/d21 39 2024-02-16T22:21:50.567 INFO:tasks.workunit.client.0.smithi114.stdout:2/201: truncate dd/f3a 728311 0 2024-02-16T22:21:50.568 INFO:tasks.workunit.client.0.smithi114.stdout:2/202: dread - dd/df/d1c/f2e zero size 2024-02-16T22:21:50.572 INFO:tasks.workunit.client.1.smithi123.stdout:7/261: creat d3/d12/d24/f56 x:0 0 0 2024-02-16T22:21:50.573 INFO:tasks.workunit.client.1.smithi123.stdout:4/273: sync 2024-02-16T22:21:50.574 INFO:tasks.workunit.client.0.smithi114.stdout:7/191: rename d3/d14/d1b to d3/d3d 0 2024-02-16T22:21:50.575 INFO:tasks.workunit.client.0.smithi114.stdout:7/192: chown d3/d14/f28 38 1 2024-02-16T22:21:50.575 INFO:tasks.workunit.client.0.smithi114.stdout:1/152: dwrite d4/f16 [0,4194304] 0 2024-02-16T22:21:50.576 INFO:tasks.workunit.client.1.smithi123.stdout:9/325: creat d4/d17/d24/f69 x:0 0 0 2024-02-16T22:21:50.578 INFO:tasks.workunit.client.1.smithi123.stdout:8/327: rmdir d0/d17/d26/d32 39 2024-02-16T22:21:50.580 INFO:tasks.workunit.client.0.smithi114.stdout:6/166: write d6/f21 [1062125,94932] 0 2024-02-16T22:21:50.582 INFO:tasks.workunit.client.1.smithi123.stdout:6/274: truncate d0/d6/f8 122947 0 2024-02-16T22:21:50.584 INFO:tasks.workunit.client.1.smithi123.stdout:6/275: write d0/d6/f63 [668075,98086] 0 2024-02-16T22:21:50.584 INFO:tasks.workunit.client.0.smithi114.stdout:9/200: fsync d2/d1a/d23/f2c 0 2024-02-16T22:21:50.585 INFO:tasks.workunit.client.1.smithi123.stdout:8/328: dread d0/d12/f19 [0,4194304] 0 2024-02-16T22:21:50.587 INFO:tasks.workunit.client.0.smithi114.stdout:2/203: chown dd/c12 24874 1 2024-02-16T22:21:50.591 INFO:tasks.workunit.client.0.smithi114.stdout:3/151: rename d4/l11 to d4/d24/d2d/l37 0 2024-02-16T22:21:50.592 INFO:tasks.workunit.client.1.smithi123.stdout:8/329: dwrite d0/d12/f1e [0,4194304] 0 2024-02-16T22:21:50.593 INFO:tasks.workunit.client.1.smithi123.stdout:8/330: write d0/d12/f50 [1029065,108097] 0 2024-02-16T22:21:50.593 INFO:tasks.workunit.client.1.smithi123.stdout:5/216: symlink d6/dd/l48 0 2024-02-16T22:21:50.596 INFO:tasks.workunit.client.0.smithi114.stdout:7/193: symlink d3/d23/l3e 0 2024-02-16T22:21:50.597 INFO:tasks.workunit.client.1.smithi123.stdout:7/262: creat d3/d12/d34/f57 x:0 0 0 2024-02-16T22:21:50.598 INFO:tasks.workunit.client.1.smithi123.stdout:2/258: write d4/d1a/d1d/d13/d25/f26 [102409,37271] 0 2024-02-16T22:21:50.599 INFO:tasks.workunit.client.0.smithi114.stdout:0/220: mkdir d1/d16/d48 0 2024-02-16T22:21:50.600 INFO:tasks.workunit.client.0.smithi114.stdout:0/221: chown d1/d16/d48 721715 1 2024-02-16T22:21:50.601 INFO:tasks.workunit.client.0.smithi114.stdout:3/152: dwrite d4/f18 [0,4194304] 0 2024-02-16T22:21:50.605 INFO:tasks.workunit.client.0.smithi114.stdout:3/153: read d4/f1c [4101142,52375] 0 2024-02-16T22:21:50.607 INFO:tasks.workunit.client.1.smithi123.stdout:6/276: rename d0/d9/dc/d39/c4a to d0/d9/dc/d3a/c69 0 2024-02-16T22:21:50.611 INFO:tasks.workunit.client.0.smithi114.stdout:9/201: dread d2/d1a/d23/d35/d1f/f2e [0,4194304] 0 2024-02-16T22:21:50.612 INFO:tasks.workunit.client.0.smithi114.stdout:6/167: dwrite d6/dd/f17 [4194304,4194304] 0 2024-02-16T22:21:50.613 INFO:tasks.workunit.client.0.smithi114.stdout:9/202: write d2/d1a/f39 [1048458,119906] 0 2024-02-16T22:21:50.614 INFO:tasks.workunit.client.0.smithi114.stdout:8/187: rename d4/d24/f29 to d4/d11/d15/f3a 0 2024-02-16T22:21:50.616 INFO:tasks.workunit.client.1.smithi123.stdout:3/226: mknod d0/d50/c51 0 2024-02-16T22:21:50.619 INFO:tasks.workunit.client.1.smithi123.stdout:0/236: getdents d1 0 2024-02-16T22:21:50.621 INFO:tasks.workunit.client.1.smithi123.stdout:8/331: rmdir d0/d17/d26/d2a/d4d 39 2024-02-16T22:21:50.623 INFO:tasks.workunit.client.0.smithi114.stdout:0/222: mknod d1/d16/d18/c49 0 2024-02-16T22:21:50.623 INFO:tasks.workunit.client.0.smithi114.stdout:9/203: dread d2/f13 [0,4194304] 0 2024-02-16T22:21:50.623 INFO:tasks.workunit.client.0.smithi114.stdout:6/168: dwrite d6/da/f2f [0,4194304] 0 2024-02-16T22:21:50.626 INFO:tasks.workunit.client.1.smithi123.stdout:2/259: write d4/d1a/d29/d45/f4b [2978144,118700] 0 2024-02-16T22:21:50.629 INFO:tasks.workunit.client.0.smithi114.stdout:4/269: dwrite dd/f19 [0,4194304] 0 2024-02-16T22:21:50.630 INFO:tasks.workunit.client.1.smithi123.stdout:1/227: sync 2024-02-16T22:21:50.631 INFO:tasks.workunit.client.1.smithi123.stdout:4/274: fdatasync d5/d1c/d23/f2a 0 2024-02-16T22:21:50.632 INFO:tasks.workunit.client.1.smithi123.stdout:1/228: stat d3/d6/db/d10/d2d/f38 0 2024-02-16T22:21:50.636 INFO:tasks.workunit.client.0.smithi114.stdout:0/223: dwrite d1/d16/f24 [0,4194304] 0 2024-02-16T22:21:50.638 INFO:tasks.workunit.client.1.smithi123.stdout:4/275: dwrite d5/d1c/d23/d36/f39 [4194304,4194304] 0 2024-02-16T22:21:50.639 INFO:tasks.workunit.client.0.smithi114.stdout:0/224: read d1/d1a/f26 [1666796,103546] 0 2024-02-16T22:21:50.641 INFO:tasks.workunit.client.0.smithi114.stdout:2/204: symlink dd/df/d1c/l45 0 2024-02-16T22:21:50.644 INFO:tasks.workunit.client.0.smithi114.stdout:3/154: dwrite d4/d24/f32 [0,4194304] 0 2024-02-16T22:21:50.645 INFO:tasks.workunit.client.0.smithi114.stdout:2/205: write dd/df/f41 [846370,106732] 0 2024-02-16T22:21:50.647 INFO:tasks.workunit.client.0.smithi114.stdout:0/225: truncate d1/d1a/d3e/d41/f42 1000478 0 2024-02-16T22:21:50.648 INFO:tasks.workunit.client.1.smithi123.stdout:7/263: dwrite d3/d12/f14 [4194304,4194304] 0 2024-02-16T22:21:50.648 INFO:tasks.workunit.client.0.smithi114.stdout:7/194: dwrite d3/fc [0,4194304] 0 2024-02-16T22:21:50.655 INFO:tasks.workunit.client.1.smithi123.stdout:0/237: read d1/d4/d17/f28 [271855,94192] 0 2024-02-16T22:21:50.661 INFO:tasks.workunit.client.0.smithi114.stdout:6/169: mknod d6/dd/c36 0 2024-02-16T22:21:50.662 INFO:tasks.workunit.client.0.smithi114.stdout:6/170: write d6/dd/d25/f2a [2729368,9978] 0 2024-02-16T22:21:50.666 INFO:tasks.workunit.client.1.smithi123.stdout:2/260: mknod d4/d1a/d1d/d13/d15/c51 0 2024-02-16T22:21:50.675 INFO:tasks.workunit.client.0.smithi114.stdout:3/155: symlink d4/dd/l38 0 2024-02-16T22:21:50.676 INFO:tasks.workunit.client.0.smithi114.stdout:3/156: write d4/f18 [9305017,93934] 0 2024-02-16T22:21:50.676 INFO:tasks.workunit.client.1.smithi123.stdout:4/276: unlink d5/d1c/d23/d36/f39 0 2024-02-16T22:21:50.679 INFO:tasks.workunit.client.0.smithi114.stdout:0/226: readlink d1/d16/l44 0 2024-02-16T22:21:50.680 INFO:tasks.workunit.client.1.smithi123.stdout:2/261: fdatasync d4/d1a/d1d/f17 0 2024-02-16T22:21:50.680 INFO:tasks.workunit.client.1.smithi123.stdout:7/264: symlink d3/d5/l58 0 2024-02-16T22:21:50.681 INFO:tasks.workunit.client.0.smithi114.stdout:7/195: rmdir d3/d3d 39 2024-02-16T22:21:50.682 INFO:tasks.workunit.client.1.smithi123.stdout:2/262: write d4/d1a/d1d/f17 [2683680,56093] 0 2024-02-16T22:21:50.682 INFO:tasks.workunit.client.0.smithi114.stdout:7/196: chown d3/lb 628 1 2024-02-16T22:21:50.683 INFO:tasks.workunit.client.1.smithi123.stdout:4/277: dwrite d5/d1c/f25 [0,4194304] 0 2024-02-16T22:21:50.683 INFO:tasks.workunit.client.0.smithi114.stdout:3/157: dwrite d4/f18 [4194304,4194304] 0 2024-02-16T22:21:50.684 INFO:tasks.workunit.client.1.smithi123.stdout:4/278: dread - d5/d1c/d23/f52 zero size 2024-02-16T22:21:50.688 INFO:tasks.workunit.client.0.smithi114.stdout:8/188: mkdir d4/d11/d15/d36/d3b 0 2024-02-16T22:21:50.690 INFO:tasks.workunit.client.1.smithi123.stdout:4/279: dwrite d5/d1c/d42/f4a [0,4194304] 0 2024-02-16T22:21:50.692 INFO:tasks.workunit.client.0.smithi114.stdout:8/189: dread d4/d11/f1f [0,4194304] 0 2024-02-16T22:21:50.695 INFO:tasks.workunit.client.0.smithi114.stdout:4/270: rmdir dd/d1b/d24/d57 39 2024-02-16T22:21:50.695 INFO:tasks.workunit.client.0.smithi114.stdout:4/271: chown dd/f21 0 1 2024-02-16T22:21:50.698 INFO:tasks.workunit.client.1.smithi123.stdout:5/217: rename d6/dd/df/d1f/d25/l29 to d6/dd/df/d1f/l49 0 2024-02-16T22:21:50.701 INFO:tasks.workunit.client.1.smithi123.stdout:6/277: creat d0/d9/f6a x:0 0 0 2024-02-16T22:21:50.701 INFO:tasks.workunit.client.1.smithi123.stdout:9/326: sync 2024-02-16T22:21:50.702 INFO:tasks.workunit.client.1.smithi123.stdout:6/278: stat d0/d9/l31 0 2024-02-16T22:21:50.705 INFO:tasks.workunit.client.0.smithi114.stdout:2/206: symlink dd/l46 0 2024-02-16T22:21:50.706 INFO:tasks.workunit.client.1.smithi123.stdout:7/265: mkdir d3/d59 0 2024-02-16T22:21:50.709 INFO:tasks.workunit.client.0.smithi114.stdout:1/153: rename f1 to d4/f30 0 2024-02-16T22:21:50.710 INFO:tasks.workunit.client.1.smithi123.stdout:8/332: mknod d0/d17/d26/d32/d45/d67/c71 0 2024-02-16T22:21:50.713 INFO:tasks.workunit.client.1.smithi123.stdout:7/266: dwrite d3/d12/d4d/f50 [0,4194304] 0 2024-02-16T22:21:50.713 INFO:tasks.workunit.client.0.smithi114.stdout:9/204: truncate d2/f13 552411 0 2024-02-16T22:21:50.718 INFO:tasks.workunit.client.1.smithi123.stdout:4/280: chown d5/d49/f4b 7874 1 2024-02-16T22:21:50.719 INFO:tasks.workunit.client.1.smithi123.stdout:4/281: fsync d5/d1c/f24 0 2024-02-16T22:21:50.720 INFO:tasks.workunit.client.1.smithi123.stdout:1/229: truncate d3/d6/d18/f21 8261243 0 2024-02-16T22:21:50.720 INFO:tasks.workunit.client.1.smithi123.stdout:7/267: dwrite d3/d26/f51 [0,4194304] 0 2024-02-16T22:21:50.724 INFO:tasks.workunit.client.1.smithi123.stdout:0/238: sync 2024-02-16T22:21:50.725 INFO:tasks.workunit.client.1.smithi123.stdout:6/279: symlink d0/d9/dc/d3a/d52/l6b 0 2024-02-16T22:21:50.726 INFO:tasks.workunit.client.0.smithi114.stdout:6/171: rename d6/l10 to d6/d2c/l37 0 2024-02-16T22:21:50.727 INFO:tasks.workunit.client.1.smithi123.stdout:3/227: getdents d0/d50 0 2024-02-16T22:21:50.731 INFO:tasks.workunit.client.0.smithi114.stdout:8/190: truncate f0 124983 0 2024-02-16T22:21:50.732 INFO:tasks.workunit.client.1.smithi123.stdout:8/333: unlink d0/d12/l41 0 2024-02-16T22:21:50.733 INFO:tasks.workunit.client.0.smithi114.stdout:1/154: fsync d4/d9/f27 0 2024-02-16T22:21:50.733 INFO:tasks.workunit.client.0.smithi114.stdout:1/155: stat d4/d9/d2a/l2b 0 2024-02-16T22:21:50.736 INFO:tasks.workunit.client.0.smithi114.stdout:4/272: symlink dd/d1b/d24/l60 0 2024-02-16T22:21:50.739 INFO:tasks.workunit.client.0.smithi114.stdout:2/207: mknod dd/df/c47 0 2024-02-16T22:21:50.741 INFO:tasks.workunit.client.0.smithi114.stdout:2/208: dread dd/f3a [0,4194304] 0 2024-02-16T22:21:50.744 INFO:tasks.workunit.client.0.smithi114.stdout:0/227: rename d1/d16/d1f/d28/f2d to d1/d1a/d3e/f4a 0 2024-02-16T22:21:50.745 INFO:tasks.workunit.client.0.smithi114.stdout:3/158: dwrite d4/dd/f1e [0,4194304] 0 2024-02-16T22:21:50.745 INFO:tasks.workunit.client.1.smithi123.stdout:5/218: symlink d6/dd/df/d1f/l4a 0 2024-02-16T22:21:50.747 INFO:tasks.workunit.client.0.smithi114.stdout:0/228: truncate d1/d16/f34 533773 0 2024-02-16T22:21:50.748 INFO:tasks.workunit.client.0.smithi114.stdout:6/172: mkdir d6/d2c/d38 0 2024-02-16T22:21:50.748 INFO:tasks.workunit.client.0.smithi114.stdout:0/229: fdatasync d1/d16/f24 0 2024-02-16T22:21:50.750 INFO:tasks.workunit.client.0.smithi114.stdout:2/209: dwrite dd/df/d1c/f40 [0,4194304] 0 2024-02-16T22:21:50.753 INFO:tasks.workunit.client.0.smithi114.stdout:8/191: creat d4/f3c x:0 0 0 2024-02-16T22:21:50.759 INFO:tasks.workunit.client.1.smithi123.stdout:0/239: creat d1/d4/d17/f41 x:0 0 0 2024-02-16T22:21:50.760 INFO:tasks.workunit.client.1.smithi123.stdout:0/240: stat d1/d4/l21 0 2024-02-16T22:21:50.760 INFO:tasks.workunit.client.0.smithi114.stdout:8/192: dwrite d4/f1d [0,4194304] 0 2024-02-16T22:21:50.762 INFO:tasks.workunit.client.1.smithi123.stdout:6/280: rename d0/d9/dc/f17 to d0/d9/dc/d5f/f6c 0 2024-02-16T22:21:50.763 INFO:tasks.workunit.client.1.smithi123.stdout:6/281: dread - d0/d9/f6a zero size 2024-02-16T22:21:50.764 INFO:tasks.workunit.client.0.smithi114.stdout:4/273: mkdir dd/d1b/d37/d61 0 2024-02-16T22:21:50.765 INFO:tasks.workunit.client.0.smithi114.stdout:4/274: read dd/d1b/d24/f4c [3430864,73854] 0 2024-02-16T22:21:50.766 INFO:tasks.workunit.client.1.smithi123.stdout:3/228: creat d0/d20/f52 x:0 0 0 2024-02-16T22:21:50.768 INFO:tasks.workunit.client.1.smithi123.stdout:8/334: mkdir d0/d17/d26/d2a/d3c/d72 0 2024-02-16T22:21:50.771 INFO:tasks.workunit.client.1.smithi123.stdout:2/263: link d4/d1a/d1d/d13/f36 d4/f52 0 2024-02-16T22:21:50.772 INFO:tasks.workunit.client.0.smithi114.stdout:9/205: write d2/d1a/f21 [2608627,118637] 0 2024-02-16T22:21:50.774 INFO:tasks.workunit.client.0.smithi114.stdout:3/159: creat d4/d26/f39 x:0 0 0 2024-02-16T22:21:50.775 INFO:tasks.workunit.client.0.smithi114.stdout:3/160: write d4/d24/f25 [224409,101323] 0 2024-02-16T22:21:50.777 INFO:tasks.workunit.client.1.smithi123.stdout:4/282: symlink d5/d1c/l53 0 2024-02-16T22:21:50.778 INFO:tasks.workunit.client.1.smithi123.stdout:7/268: getdents d3/d26/d3e 0 2024-02-16T22:21:50.780 INFO:tasks.workunit.client.0.smithi114.stdout:2/210: chown dd/l1a 15268 1 2024-02-16T22:21:50.784 INFO:tasks.workunit.client.1.smithi123.stdout:9/327: link d4/d17/d24/f4a d4/d17/d20/f6a 0 2024-02-16T22:21:50.788 INFO:tasks.workunit.client.1.smithi123.stdout:0/241: rmdir d1/d8 39 2024-02-16T22:21:50.794 INFO:tasks.workunit.client.0.smithi114.stdout:4/275: creat dd/d1b/d41/f62 x:0 0 0 2024-02-16T22:21:50.796 INFO:tasks.workunit.client.0.smithi114.stdout:1/156: dwrite d4/fa [0,4194304] 0 2024-02-16T22:21:50.797 INFO:tasks.workunit.client.1.smithi123.stdout:5/219: truncate d6/dd/df/d1f/d25/f1d 1331499 0 2024-02-16T22:21:50.797 INFO:tasks.workunit.client.0.smithi114.stdout:1/157: readlink d4/d9/l1a 0 2024-02-16T22:21:50.799 INFO:tasks.workunit.client.1.smithi123.stdout:4/283: mknod d5/c54 0 2024-02-16T22:21:50.799 INFO:tasks.workunit.client.1.smithi123.stdout:5/220: dread d6/dd/df/f22 [0,4194304] 0 2024-02-16T22:21:50.800 INFO:tasks.workunit.client.0.smithi114.stdout:3/161: stat d4/f16 0 2024-02-16T22:21:50.804 INFO:tasks.workunit.client.0.smithi114.stdout:6/173: rename l5 to d6/dd/l39 0 2024-02-16T22:21:50.805 INFO:tasks.workunit.client.0.smithi114.stdout:1/158: dwrite d4/f16 [0,4194304] 0 2024-02-16T22:21:50.805 INFO:tasks.workunit.client.0.smithi114.stdout:6/174: fsync d6/dd/d25/d26/f2d 0 2024-02-16T22:21:50.806 INFO:tasks.workunit.client.0.smithi114.stdout:1/159: read - d4/d9/f2f zero size 2024-02-16T22:21:50.807 INFO:tasks.workunit.client.1.smithi123.stdout:7/269: dwrite d3/d12/d1d/f2f [0,4194304] 0 2024-02-16T22:21:50.808 INFO:tasks.workunit.client.1.smithi123.stdout:9/328: write d4/d1e/f42 [4234770,129563] 0 2024-02-16T22:21:50.812 INFO:tasks.workunit.client.0.smithi114.stdout:0/230: link d1/d16/f19 d1/d16/d18/f4b 0 2024-02-16T22:21:50.820 INFO:tasks.workunit.client.0.smithi114.stdout:8/193: link d4/f8 d4/d24/f3d 0 2024-02-16T22:21:50.820 INFO:tasks.workunit.client.1.smithi123.stdout:8/335: getdents d0/d17/d26/d2a/d3c/d72 0 2024-02-16T22:21:50.821 INFO:tasks.workunit.client.1.smithi123.stdout:8/336: chown d0/d17/d26/d32/d45/d67/c71 1481 1 2024-02-16T22:21:50.822 INFO:tasks.workunit.client.1.smithi123.stdout:2/264: mkdir d4/d1a/d29/d53 0 2024-02-16T22:21:50.826 INFO:tasks.workunit.client.1.smithi123.stdout:4/284: creat d5/d1c/d23/d36/f55 x:0 0 0 2024-02-16T22:21:50.826 INFO:tasks.workunit.client.1.smithi123.stdout:4/285: stat d5/d1c/c38 0 2024-02-16T22:21:50.826 INFO:tasks.workunit.client.0.smithi114.stdout:9/206: symlink d2/d33/d3f/l42 0 2024-02-16T22:21:50.827 INFO:tasks.workunit.client.1.smithi123.stdout:1/230: link d3/d6/d18/l43 d3/d6/db/l54 0 2024-02-16T22:21:50.828 INFO:tasks.workunit.client.0.smithi114.stdout:3/162: symlink d4/l3a 0 2024-02-16T22:21:50.828 INFO:tasks.workunit.client.0.smithi114.stdout:4/276: dwrite dd/d2e/f49 [0,4194304] 0 2024-02-16T22:21:50.829 INFO:tasks.workunit.client.1.smithi123.stdout:7/270: creat d3/d12/d4d/f5a x:0 0 0 2024-02-16T22:21:50.830 INFO:tasks.workunit.client.1.smithi123.stdout:4/286: dwrite d5/f1f [0,4194304] 0 2024-02-16T22:21:50.831 INFO:tasks.workunit.client.1.smithi123.stdout:9/329: creat d4/f6b x:0 0 0 2024-02-16T22:21:50.831 INFO:tasks.workunit.client.0.smithi114.stdout:6/175: chown d6/dd/d25/l33 67253 1 2024-02-16T22:21:50.833 INFO:tasks.workunit.client.0.smithi114.stdout:1/160: mknod d4/d9/c31 0 2024-02-16T22:21:50.835 INFO:tasks.workunit.client.1.smithi123.stdout:0/242: read - d1/d4/d17/f41 zero size 2024-02-16T22:21:50.836 INFO:tasks.workunit.client.0.smithi114.stdout:8/194: creat d4/d24/f3e x:0 0 0 2024-02-16T22:21:50.837 INFO:tasks.workunit.client.1.smithi123.stdout:4/287: read d5/f29 [93792,118316] 0 2024-02-16T22:21:50.837 INFO:tasks.workunit.client.1.smithi123.stdout:6/282: creat d0/f6d x:0 0 0 2024-02-16T22:21:50.838 INFO:tasks.workunit.client.0.smithi114.stdout:9/207: symlink d2/d1a/d23/d35/l43 0 2024-02-16T22:21:50.838 INFO:tasks.workunit.client.0.smithi114.stdout:9/208: stat d2/l9 0 2024-02-16T22:21:50.838 INFO:tasks.workunit.client.1.smithi123.stdout:6/283: chown d0/d9 25562438 1 2024-02-16T22:21:50.843 INFO:tasks.workunit.client.0.smithi114.stdout:6/176: rmdir d6 39 2024-02-16T22:21:50.844 INFO:tasks.workunit.client.1.smithi123.stdout:6/284: dwrite d0/d9/dc/d5f/f58 [0,4194304] 0 2024-02-16T22:21:50.845 INFO:tasks.workunit.client.0.smithi114.stdout:1/161: fdatasync d4/f15 0 2024-02-16T22:21:50.845 INFO:tasks.workunit.client.1.smithi123.stdout:6/285: truncate d0/d6/d1a/f64 324092 0 2024-02-16T22:21:50.845 INFO:tasks.workunit.client.0.smithi114.stdout:1/162: chown d4/d9/ce 653443 1 2024-02-16T22:21:50.847 INFO:tasks.workunit.client.0.smithi114.stdout:0/231: truncate d1/d16/d18/f1c 1003107 0 2024-02-16T22:21:50.848 INFO:tasks.workunit.client.1.smithi123.stdout:9/330: mkdir d4/d17/d2b/d6c 0 2024-02-16T22:21:50.849 INFO:tasks.workunit.client.1.smithi123.stdout:0/243: fdatasync d1/d8/f30 0 2024-02-16T22:21:50.849 INFO:tasks.workunit.client.0.smithi114.stdout:9/209: creat d2/f44 x:0 0 0 2024-02-16T22:21:50.851 INFO:tasks.workunit.client.1.smithi123.stdout:3/229: link d0/l16 d0/d20/d29/d34/d37/d41/l53 0 2024-02-16T22:21:50.852 INFO:tasks.workunit.client.0.smithi114.stdout:4/277: mknod dd/d1b/d24/c63 0 2024-02-16T22:21:50.852 INFO:tasks.workunit.client.0.smithi114.stdout:2/211: link dd/df/c1b dd/df/d1c/c48 0 2024-02-16T22:21:50.853 INFO:tasks.workunit.client.0.smithi114.stdout:4/278: fdatasync dd/d1b/d24/f2f 0 2024-02-16T22:21:50.855 INFO:tasks.workunit.client.0.smithi114.stdout:6/177: readlink d6/da/l13 0 2024-02-16T22:21:50.858 INFO:tasks.workunit.client.1.smithi123.stdout:1/231: mkdir d3/d6/db/d55 0 2024-02-16T22:21:50.859 INFO:tasks.workunit.client.1.smithi123.stdout:1/232: stat d3/d6/db/d10/d2a/c3f 0 2024-02-16T22:21:50.863 INFO:tasks.workunit.client.0.smithi114.stdout:8/195: mknod d4/d11/d15/d36/d3b/c3f 0 2024-02-16T22:21:50.865 INFO:tasks.workunit.client.0.smithi114.stdout:9/210: symlink d2/d1a/d23/d35/d1f/l45 0 2024-02-16T22:21:50.868 INFO:tasks.workunit.client.0.smithi114.stdout:9/211: dread d2/d1a/f21 [0,4194304] 0 2024-02-16T22:21:50.869 INFO:tasks.workunit.client.1.smithi123.stdout:0/244: dwrite d1/f22 [0,4194304] 0 2024-02-16T22:21:50.869 INFO:tasks.workunit.client.1.smithi123.stdout:3/230: chown d0/f12 207509356 1 2024-02-16T22:21:50.870 INFO:tasks.workunit.client.1.smithi123.stdout:0/245: truncate d1/d8/f2f 457630 0 2024-02-16T22:21:50.870 INFO:tasks.workunit.client.1.smithi123.stdout:2/265: creat d4/f54 x:0 0 0 2024-02-16T22:21:50.873 INFO:tasks.workunit.client.1.smithi123.stdout:5/221: link d6/dd/df/d1f/l2a d6/dd/l4b 0 2024-02-16T22:21:50.874 INFO:tasks.workunit.client.0.smithi114.stdout:6/178: symlink d6/d2c/l3a 0 2024-02-16T22:21:50.874 INFO:tasks.workunit.client.0.smithi114.stdout:9/212: dwrite d2/f44 [0,4194304] 0 2024-02-16T22:21:50.875 INFO:tasks.workunit.client.1.smithi123.stdout:1/233: symlink d3/d6/d11/l56 0 2024-02-16T22:21:50.876 INFO:tasks.workunit.client.0.smithi114.stdout:9/213: dread - d2/d1a/d23/d35/f27 zero size 2024-02-16T22:21:50.880 INFO:tasks.workunit.client.1.smithi123.stdout:7/271: dwrite d3/d12/f1a [0,4194304] 0 2024-02-16T22:21:50.881 INFO:tasks.workunit.client.0.smithi114.stdout:0/232: fsync d1/d1a/f2c 0 2024-02-16T22:21:50.888 INFO:tasks.workunit.client.0.smithi114.stdout:3/163: getdents d4/dd/d33 0 2024-02-16T22:21:50.894 INFO:tasks.workunit.client.0.smithi114.stdout:4/279: symlink dd/d1b/l64 0 2024-02-16T22:21:50.895 INFO:tasks.workunit.client.0.smithi114.stdout:3/164: dwrite d4/f18 [4194304,4194304] 0 2024-02-16T22:21:50.896 INFO:tasks.workunit.client.0.smithi114.stdout:1/163: truncate d4/f16 1349470 0 2024-02-16T22:21:50.897 INFO:tasks.workunit.client.0.smithi114.stdout:3/165: write d4/f18 [3161554,87948] 0 2024-02-16T22:21:50.898 INFO:tasks.workunit.client.1.smithi123.stdout:1/234: symlink d3/d6/d18/l57 0 2024-02-16T22:21:50.898 INFO:tasks.workunit.client.1.smithi123.stdout:1/235: readlink d3/d6/d18/l57 0 2024-02-16T22:21:50.899 INFO:tasks.workunit.client.1.smithi123.stdout:1/236: chown d3/d47 102 1 2024-02-16T22:21:50.899 INFO:tasks.workunit.client.1.smithi123.stdout:4/288: write d5/d49/f4e [2719521,94589] 0 2024-02-16T22:21:50.901 INFO:tasks.workunit.client.0.smithi114.stdout:6/179: unlink d6/dd/l24 0 2024-02-16T22:21:50.902 INFO:tasks.workunit.client.0.smithi114.stdout:6/180: write d6/f21 [3183298,51997] 0 2024-02-16T22:21:50.903 INFO:tasks.workunit.client.0.smithi114.stdout:6/181: dread - d6/f35 zero size 2024-02-16T22:21:50.904 INFO:tasks.workunit.client.0.smithi114.stdout:6/182: write d6/dd/f29 [1670529,110542] 0 2024-02-16T22:21:50.904 INFO:tasks.workunit.client.1.smithi123.stdout:9/331: truncate d4/d17/d20/f22 3687386 0 2024-02-16T22:21:50.905 INFO:tasks.workunit.client.1.smithi123.stdout:4/289: dwrite d5/d1c/d23/d31/f3e [0,4194304] 0 2024-02-16T22:21:50.907 INFO:tasks.workunit.client.1.smithi123.stdout:3/231: truncate d0/fd 933605 0 2024-02-16T22:21:50.910 INFO:tasks.workunit.client.1.smithi123.stdout:0/246: unlink d1/f3c 0 2024-02-16T22:21:50.910 INFO:tasks.workunit.client.0.smithi114.stdout:2/212: dwrite dd/f36 [0,4194304] 0 2024-02-16T22:21:50.911 INFO:tasks.workunit.client.0.smithi114.stdout:8/196: truncate d4/d11/d15/f10 364587 0 2024-02-16T22:21:50.912 INFO:tasks.workunit.client.1.smithi123.stdout:2/266: getdents d4/d1a/d29/d53 0 2024-02-16T22:21:50.912 INFO:tasks.workunit.client.0.smithi114.stdout:3/166: fdatasync f1 0 2024-02-16T22:21:50.922 INFO:tasks.workunit.client.0.smithi114.stdout:4/280: rename dd/f4b to dd/d59/f65 0 2024-02-16T22:21:50.925 INFO:tasks.workunit.client.1.smithi123.stdout:9/332: rename d4/d1e/d5d to d4/d56/d6d 0 2024-02-16T22:21:50.925 INFO:tasks.workunit.client.1.smithi123.stdout:9/333: dread - d4/d17/f55 zero size 2024-02-16T22:21:50.926 INFO:tasks.workunit.client.1.smithi123.stdout:9/334: fsync d4/d17/d2b/f53 0 2024-02-16T22:21:50.927 INFO:tasks.workunit.client.1.smithi123.stdout:7/272: creat d3/d26/d3e/f5b x:0 0 0 2024-02-16T22:21:50.928 INFO:tasks.workunit.client.1.smithi123.stdout:3/232: mknod d0/d20/c54 0 2024-02-16T22:21:50.930 INFO:tasks.workunit.client.1.smithi123.stdout:8/337: sync 2024-02-16T22:21:50.930 INFO:tasks.workunit.client.1.smithi123.stdout:0/247: mkdir d1/d4/d42 0 2024-02-16T22:21:50.930 INFO:tasks.workunit.client.1.smithi123.stdout:8/338: write d0/d29/f60 [972331,77299] 0 2024-02-16T22:21:50.931 INFO:tasks.workunit.client.1.smithi123.stdout:5/222: creat d6/f4c x:0 0 0 2024-02-16T22:21:50.932 INFO:tasks.workunit.client.1.smithi123.stdout:1/237: mknod d3/d52/c58 0 2024-02-16T22:21:50.938 INFO:tasks.workunit.client.1.smithi123.stdout:9/335: mkdir d4/d56/d58/d6e 0 2024-02-16T22:21:50.939 INFO:tasks.workunit.client.1.smithi123.stdout:9/336: write d4/d17/d24/f49 [105770,51889] 0 2024-02-16T22:21:50.940 INFO:tasks.workunit.client.0.smithi114.stdout:3/167: symlink d4/dd/l3b 0 2024-02-16T22:21:50.941 INFO:tasks.workunit.client.0.smithi114.stdout:4/281: stat dd/c1a 0 2024-02-16T22:21:50.942 INFO:tasks.workunit.client.0.smithi114.stdout:1/164: symlink d4/l32 0 2024-02-16T22:21:50.943 INFO:tasks.workunit.client.1.smithi123.stdout:0/248: creat d1/d4/f43 x:0 0 0 2024-02-16T22:21:50.945 INFO:tasks.workunit.client.0.smithi114.stdout:6/183: symlink d6/l3b 0 2024-02-16T22:21:50.946 INFO:tasks.workunit.client.0.smithi114.stdout:6/184: read - d6/f35 zero size 2024-02-16T22:21:50.948 INFO:tasks.workunit.client.1.smithi123.stdout:6/286: sync 2024-02-16T22:21:50.948 INFO:tasks.workunit.client.1.smithi123.stdout:4/290: dread d5/d1c/d23/f3a [0,4194304] 0 2024-02-16T22:21:50.948 INFO:tasks.workunit.client.1.smithi123.stdout:2/267: rename d4/d1a/d1d/d13/d15/c3d to d4/d1a/d29/d53/c55 0 2024-02-16T22:21:50.949 INFO:tasks.workunit.client.1.smithi123.stdout:3/233: dread d0/fb [0,4194304] 0 2024-02-16T22:21:50.950 INFO:tasks.workunit.client.0.smithi114.stdout:2/213: symlink dd/d3c/l49 0 2024-02-16T22:21:50.950 INFO:tasks.workunit.client.1.smithi123.stdout:3/234: read d0/f19 [43111,52644] 0 2024-02-16T22:21:50.951 INFO:tasks.workunit.client.1.smithi123.stdout:3/235: chown d0/d20/d29/d34/d37/c3a 573 1 2024-02-16T22:21:50.951 INFO:tasks.workunit.client.0.smithi114.stdout:8/197: mknod d4/d11/c40 0 2024-02-16T22:21:50.951 INFO:tasks.workunit.client.0.smithi114.stdout:2/214: read dd/f28 [818688,35583] 0 2024-02-16T22:21:50.953 INFO:tasks.workunit.client.0.smithi114.stdout:3/168: creat d4/d26/f3c x:0 0 0 2024-02-16T22:21:50.956 INFO:tasks.workunit.client.1.smithi123.stdout:0/249: readlink d1/l3 0 2024-02-16T22:21:50.957 INFO:tasks.workunit.client.0.smithi114.stdout:9/214: getdents d2/d1a 0 2024-02-16T22:21:50.978 INFO:tasks.workunit.client.1.smithi123.stdout:1/238: creat d3/d6/db/d3b/f59 x:0 0 0 2024-02-16T22:21:50.978 INFO:tasks.workunit.client.1.smithi123.stdout:1/239: dread - d3/d6/d49/f50 zero size 2024-02-16T22:21:50.979 INFO:tasks.workunit.client.1.smithi123.stdout:1/240: chown d3/d6/d11/c34 2 1 2024-02-16T22:21:51.008 INFO:tasks.workunit.client.0.smithi114.stdout:6/185: fdatasync d6/dd/f1d 0 2024-02-16T22:21:51.009 INFO:tasks.workunit.client.0.smithi114.stdout:6/186: chown d6/da/f2f 4412696 1 2024-02-16T22:21:51.013 INFO:tasks.workunit.client.0.smithi114.stdout:8/198: symlink d4/d11/d15/d36/d3b/l41 0 2024-02-16T22:21:51.023 INFO:tasks.workunit.client.1.smithi123.stdout:7/273: truncate d3/d12/d24/f49 1186575 0 2024-02-16T22:21:51.024 INFO:tasks.workunit.client.1.smithi123.stdout:7/274: truncate d3/d12/d24/f39 113003 0 2024-02-16T22:21:51.031 INFO:tasks.workunit.client.0.smithi114.stdout:9/215: dread f1 [0,4194304] 0 2024-02-16T22:21:51.033 INFO:tasks.workunit.client.1.smithi123.stdout:0/250: mkdir d1/d8/d3d/d44 0 2024-02-16T22:21:51.034 INFO:tasks.workunit.client.0.smithi114.stdout:6/187: fdatasync d6/f18 0 2024-02-16T22:21:51.040 INFO:tasks.workunit.client.1.smithi123.stdout:1/241: creat d3/d6/db/d10/d2d/f5a x:0 0 0 2024-02-16T22:21:51.041 INFO:tasks.workunit.client.0.smithi114.stdout:6/188: dwrite d6/dd/f29 [0,4194304] 0 2024-02-16T22:21:51.042 INFO:tasks.workunit.client.1.smithi123.stdout:5/223: rename d6/dd/df/d1f/d25/f1d to d6/dd/d44/f4d 0 2024-02-16T22:21:51.046 INFO:tasks.workunit.client.1.smithi123.stdout:1/242: dwrite d3/d6/db/d10/d2d/f38 [0,4194304] 0 2024-02-16T22:21:51.047 INFO:tasks.workunit.client.1.smithi123.stdout:1/243: stat d3/d6/db/d10/d4f/f51 0 2024-02-16T22:21:51.047 INFO:tasks.workunit.client.1.smithi123.stdout:1/244: stat d3/f4e 0 2024-02-16T22:21:51.048 INFO:tasks.workunit.client.1.smithi123.stdout:4/291: mkdir d5/d1c/d23/d56 0 2024-02-16T22:21:51.049 INFO:tasks.workunit.client.1.smithi123.stdout:4/292: chown d5/d1c/d23/d31/l34 28826225 1 2024-02-16T22:21:51.049 INFO:tasks.workunit.client.0.smithi114.stdout:9/216: dread d2/f4 [4194304,4194304] 0 2024-02-16T22:21:51.053 INFO:tasks.workunit.client.1.smithi123.stdout:7/275: creat d3/d26/d35/f5c x:0 0 0 2024-02-16T22:21:51.059 INFO:tasks.workunit.client.0.smithi114.stdout:6/189: mknod d6/d2c/c3c 0 2024-02-16T22:21:51.065 INFO:tasks.workunit.client.0.smithi114.stdout:3/169: getdents d4/dd 0 2024-02-16T22:21:51.067 INFO:tasks.workunit.client.1.smithi123.stdout:8/339: getdents d0/d29/d42 0 2024-02-16T22:21:51.067 INFO:tasks.workunit.client.1.smithi123.stdout:8/340: readlink d0/d17/d26/d2a/d3c/l3b 0 2024-02-16T22:21:51.069 INFO:tasks.workunit.client.0.smithi114.stdout:6/190: rename d6/da/f2f to d6/dd/d25/d26/f3d 0 2024-02-16T22:21:51.071 INFO:tasks.workunit.client.1.smithi123.stdout:5/224: readlink d6/dd/df/d1f/l49 0 2024-02-16T22:21:51.073 INFO:tasks.workunit.client.1.smithi123.stdout:5/225: write d6/dd/df/d1f/d25/f3a [4948619,111007] 0 2024-02-16T22:21:51.077 INFO:tasks.workunit.client.0.smithi114.stdout:6/191: readlink d6/dd/l39 0 2024-02-16T22:21:51.077 INFO:tasks.workunit.client.0.smithi114.stdout:6/192: fdatasync d6/dd/f29 0 2024-02-16T22:21:51.080 INFO:tasks.workunit.client.0.smithi114.stdout:3/170: symlink d4/d24/l3d 0 2024-02-16T22:21:51.080 INFO:tasks.workunit.client.0.smithi114.stdout:3/171: dread - d4/dd/f35 zero size 2024-02-16T22:21:51.081 INFO:tasks.workunit.client.1.smithi123.stdout:7/276: creat d3/d12/d24/f5d x:0 0 0 2024-02-16T22:21:51.084 INFO:tasks.workunit.client.1.smithi123.stdout:9/337: getdents d4/d17/d2b 0 2024-02-16T22:21:51.086 INFO:tasks.workunit.client.1.smithi123.stdout:3/236: creat d0/f55 x:0 0 0 2024-02-16T22:21:51.089 INFO:tasks.workunit.client.0.smithi114.stdout:3/172: rename d4/l1f to d4/d26/l3e 0 2024-02-16T22:21:51.091 INFO:tasks.workunit.client.1.smithi123.stdout:8/341: creat d0/d29/d42/d47/f73 x:0 0 0 2024-02-16T22:21:51.092 INFO:tasks.workunit.client.1.smithi123.stdout:7/277: dread d3/d12/d1d/f41 [0,4194304] 0 2024-02-16T22:21:51.093 INFO:tasks.workunit.client.1.smithi123.stdout:7/278: dread - d3/d26/d35/f5c zero size 2024-02-16T22:21:51.094 INFO:tasks.workunit.client.1.smithi123.stdout:7/279: write d3/d12/f2e [77167,3136] 0 2024-02-16T22:21:51.095 INFO:tasks.workunit.client.1.smithi123.stdout:6/287: rename d0/d6/d2f/l42 to d0/d9/dc/d5f/l6e 0 2024-02-16T22:21:51.096 INFO:tasks.workunit.client.1.smithi123.stdout:7/280: write d3/d12/d24/f5d [350586,48622] 0 2024-02-16T22:21:51.098 INFO:tasks.workunit.client.1.smithi123.stdout:7/281: chown d3/d12/d1d/f41 244622903 1 2024-02-16T22:21:51.098 INFO:tasks.workunit.client.1.smithi123.stdout:8/342: dwrite d0/d17/d26/d2a/f54 [0,4194304] 0 2024-02-16T22:21:51.098 INFO:tasks.workunit.client.1.smithi123.stdout:7/282: chown d3/d12/d34 2096412 1 2024-02-16T22:21:51.099 INFO:tasks.workunit.client.0.smithi114.stdout:3/173: symlink d4/d26/l3f 0 2024-02-16T22:21:51.105 INFO:tasks.workunit.client.0.smithi114.stdout:3/174: rename d4/dd/f23 to d4/d26/f40 0 2024-02-16T22:21:51.107 INFO:tasks.workunit.client.0.smithi114.stdout:3/175: write d4/d24/f32 [382440,100733] 0 2024-02-16T22:21:51.107 INFO:tasks.workunit.client.1.smithi123.stdout:9/338: mkdir d4/d56/d6f 0 2024-02-16T22:21:51.108 INFO:tasks.workunit.client.1.smithi123.stdout:9/339: write d4/d17/d2b/f53 [417675,87628] 0 2024-02-16T22:21:51.110 INFO:tasks.workunit.client.1.smithi123.stdout:3/237: creat d0/d40/f56 x:0 0 0 2024-02-16T22:21:51.111 INFO:tasks.workunit.client.1.smithi123.stdout:3/238: chown d0/d20/l24 0 1 2024-02-16T22:21:51.114 INFO:tasks.workunit.client.1.smithi123.stdout:3/239: truncate d0/d20/d2e/f48 994441 0 2024-02-16T22:21:51.114 INFO:tasks.workunit.client.1.smithi123.stdout:9/340: dwrite d4/d56/d58/f5a [0,4194304] 0 2024-02-16T22:21:51.114 INFO:tasks.workunit.client.1.smithi123.stdout:3/240: stat d0/d20/d29/d34/f4e 0 2024-02-16T22:21:51.115 INFO:tasks.workunit.client.0.smithi114.stdout:3/176: chown d4/d24/d2d/l37 63889671 1 2024-02-16T22:21:51.115 INFO:tasks.workunit.client.1.smithi123.stdout:9/341: read - d4/d17/f55 zero size 2024-02-16T22:21:51.116 INFO:tasks.workunit.client.1.smithi123.stdout:3/241: truncate d0/d20/d29/f3c 54233 0 2024-02-16T22:21:51.117 INFO:tasks.workunit.client.1.smithi123.stdout:9/342: read d4/d1e/f42 [6005555,22419] 0 2024-02-16T22:21:51.117 INFO:tasks.workunit.client.1.smithi123.stdout:3/242: read d0/f6 [3887698,121802] 0 2024-02-16T22:21:51.118 INFO:tasks.workunit.client.1.smithi123.stdout:8/343: creat d0/d12/f74 x:0 0 0 2024-02-16T22:21:51.119 INFO:tasks.workunit.client.1.smithi123.stdout:0/251: symlink d1/d4/l45 0 2024-02-16T22:21:51.122 INFO:tasks.workunit.client.1.smithi123.stdout:9/343: dwrite d4/d56/d58/d59/f5c [0,4194304] 0 2024-02-16T22:21:51.123 INFO:tasks.workunit.client.1.smithi123.stdout:9/344: dread - d4/d36/d57/f67 zero size 2024-02-16T22:21:51.124 INFO:tasks.workunit.client.1.smithi123.stdout:5/226: mknod d6/dd/df/d2b/d3f/c4e 0 2024-02-16T22:21:51.125 INFO:tasks.workunit.client.0.smithi114.stdout:3/177: mknod d4/d24/c41 0 2024-02-16T22:21:51.128 INFO:tasks.workunit.client.1.smithi123.stdout:4/293: creat d5/d1c/d23/f57 x:0 0 0 2024-02-16T22:21:51.131 INFO:tasks.workunit.client.0.smithi114.stdout:1/165: mkdir d4/d33 0 2024-02-16T22:21:51.132 INFO:tasks.workunit.client.1.smithi123.stdout:9/345: dwrite d4/d1e/d3a/f40 [0,4194304] 0 2024-02-16T22:21:51.133 INFO:tasks.workunit.client.0.smithi114.stdout:4/282: write dd/f20 [1502403,57094] 0 2024-02-16T22:21:51.135 INFO:tasks.workunit.client.1.smithi123.stdout:5/227: dwrite d6/dd/df/d1f/d25/f1a [0,4194304] 0 2024-02-16T22:21:51.135 INFO:tasks.workunit.client.0.smithi114.stdout:8/199: unlink d4/f7 0 2024-02-16T22:21:51.140 INFO:tasks.workunit.client.1.smithi123.stdout:4/294: dwrite d5/d49/f4e [0,4194304] 0 2024-02-16T22:21:51.142 INFO:tasks.workunit.client.1.smithi123.stdout:3/243: mknod d0/d20/c57 0 2024-02-16T22:21:51.147 INFO:tasks.workunit.client.0.smithi114.stdout:8/200: rename d4/d16/d25/c33 to d4/d24/d2c/c42 0 2024-02-16T22:21:51.148 INFO:tasks.workunit.client.0.smithi114.stdout:8/201: chown d4/d11/f2f 770649 1 2024-02-16T22:21:51.149 INFO:tasks.workunit.client.1.smithi123.stdout:3/244: dwrite d0/d20/d2e/f39 [0,4194304] 0 2024-02-16T22:21:51.153 INFO:tasks.workunit.client.0.smithi114.stdout:4/283: getdents dd/d1b/d37/d61 0 2024-02-16T22:21:51.153 INFO:tasks.workunit.client.0.smithi114.stdout:4/284: chown dd/d2e/d3f 1077479 1 2024-02-16T22:21:51.155 INFO:tasks.workunit.client.1.smithi123.stdout:0/252: mknod d1/d4/c46 0 2024-02-16T22:21:51.159 INFO:tasks.workunit.client.0.smithi114.stdout:4/285: rmdir dd/d1b/d37/d38 39 2024-02-16T22:21:51.162 INFO:tasks.workunit.client.1.smithi123.stdout:9/346: mknod d4/d1e/c70 0 2024-02-16T22:21:51.163 INFO:tasks.workunit.client.1.smithi123.stdout:7/283: dread d3/fe [0,4194304] 0 2024-02-16T22:21:51.164 INFO:tasks.workunit.client.1.smithi123.stdout:5/228: creat d6/dd/df/d2b/f4f x:0 0 0 2024-02-16T22:21:51.168 INFO:tasks.workunit.client.0.smithi114.stdout:4/286: truncate dd/d1b/d37/f52 892939 0 2024-02-16T22:21:51.169 INFO:tasks.workunit.client.1.smithi123.stdout:6/288: creat d0/d6/d1a/f6f x:0 0 0 2024-02-16T22:21:51.171 INFO:tasks.workunit.client.0.smithi114.stdout:8/202: getdents d4/d11/d15 0 2024-02-16T22:21:51.172 INFO:tasks.workunit.client.1.smithi123.stdout:2/268: dwrite d4/f37 [0,4194304] 0 2024-02-16T22:21:51.176 INFO:tasks.workunit.client.0.smithi114.stdout:8/203: symlink d4/d16/l43 0 2024-02-16T22:21:51.178 INFO:tasks.workunit.client.1.smithi123.stdout:2/269: dread d4/d1a/d1d/d13/d15/f42 [0,4194304] 0 2024-02-16T22:21:51.179 INFO:tasks.workunit.client.0.smithi114.stdout:8/204: truncate d4/d24/f38 4595594 0 2024-02-16T22:21:51.180 INFO:tasks.workunit.client.0.smithi114.stdout:4/287: dwrite dd/d1b/d37/d38/f5e [0,4194304] 0 2024-02-16T22:21:51.180 INFO:tasks.workunit.client.1.smithi123.stdout:9/347: mkdir d4/d56/d6d/d71 0 2024-02-16T22:21:51.181 INFO:tasks.workunit.client.0.smithi114.stdout:8/205: write d4/d11/f17 [3685479,114867] 0 2024-02-16T22:21:51.182 INFO:tasks.workunit.client.0.smithi114.stdout:8/206: chown d4/d24 26 1 2024-02-16T22:21:51.183 INFO:tasks.workunit.client.1.smithi123.stdout:7/284: creat d3/d12/d1d/d1e/f5e x:0 0 0 2024-02-16T22:21:51.186 INFO:tasks.workunit.client.1.smithi123.stdout:5/229: read d6/dd/d44/f4d [1214804,129899] 0 2024-02-16T22:21:51.188 INFO:tasks.workunit.client.0.smithi114.stdout:9/217: write d2/d1a/d23/d35/d1d/f36 [1799692,74031] 0 2024-02-16T22:21:51.190 INFO:tasks.workunit.client.0.smithi114.stdout:4/288: dwrite dd/d1b/f5b [4194304,4194304] 0 2024-02-16T22:21:51.191 INFO:tasks.workunit.client.1.smithi123.stdout:6/289: mkdir d0/d9/dc/d3a/d70 0 2024-02-16T22:21:51.191 INFO:tasks.workunit.client.1.smithi123.stdout:1/245: rename d3/d6/d18/f37 to d3/d6/f5b 0 2024-02-16T22:21:51.193 INFO:tasks.workunit.client.0.smithi114.stdout:8/207: creat d4/d11/d15/d36/d3b/f44 x:0 0 0 2024-02-16T22:21:51.195 INFO:tasks.workunit.client.0.smithi114.stdout:8/208: truncate d4/f3c 685011 0 2024-02-16T22:21:51.197 INFO:tasks.workunit.client.0.smithi114.stdout:9/218: dwrite d2/fb [0,4194304] 0 2024-02-16T22:21:51.197 INFO:tasks.workunit.client.0.smithi114.stdout:8/209: chown d4/d24/f3e 40 1 2024-02-16T22:21:51.198 INFO:tasks.workunit.client.0.smithi114.stdout:8/210: fdatasync d4/d11/f17 0 2024-02-16T22:21:51.198 INFO:tasks.workunit.client.0.smithi114.stdout:9/219: truncate d2/d1a/f39 1691374 0 2024-02-16T22:21:51.202 INFO:tasks.workunit.client.0.smithi114.stdout:4/289: creat dd/d1b/d37/d61/f66 x:0 0 0 2024-02-16T22:21:51.202 INFO:tasks.workunit.client.0.smithi114.stdout:4/290: dread - dd/d1b/d37/f5d zero size 2024-02-16T22:21:51.205 INFO:tasks.workunit.client.1.smithi123.stdout:9/348: dread d4/f16 [0,4194304] 0 2024-02-16T22:21:51.206 INFO:tasks.workunit.client.0.smithi114.stdout:8/211: symlink d4/d11/d15/l45 0 2024-02-16T22:21:51.207 INFO:tasks.workunit.client.1.smithi123.stdout:9/349: truncate d4/d17/d24/f49 695667 0 2024-02-16T22:21:51.207 INFO:tasks.workunit.client.0.smithi114.stdout:9/220: mkdir d2/d1a/d23/d32/d46 0 2024-02-16T22:21:51.209 INFO:tasks.workunit.client.1.smithi123.stdout:5/230: dread d6/dd/df/d1f/f2e [0,4194304] 0 2024-02-16T22:21:51.210 INFO:tasks.workunit.client.0.smithi114.stdout:4/291: dwrite dd/d1b/d24/f3d [0,4194304] 0 2024-02-16T22:21:51.210 INFO:tasks.workunit.client.0.smithi114.stdout:9/221: fsync d2/d1a/d23/d35/d1d/f36 0 2024-02-16T22:21:51.211 INFO:tasks.workunit.client.1.smithi123.stdout:6/290: symlink d0/d9/dc/l71 0 2024-02-16T22:21:51.211 INFO:tasks.workunit.client.0.smithi114.stdout:9/222: stat d2/f4 0 2024-02-16T22:21:51.211 INFO:tasks.workunit.client.1.smithi123.stdout:6/291: write d0/f38 [1011271,117373] 0 2024-02-16T22:21:51.216 INFO:tasks.workunit.client.0.smithi114.stdout:8/212: dread d4/d11/d15/f10 [0,4194304] 0 2024-02-16T22:21:51.216 INFO:tasks.workunit.client.0.smithi114.stdout:8/213: stat d4/l5 0 2024-02-16T22:21:51.219 INFO:tasks.workunit.client.0.smithi114.stdout:6/193: truncate d6/f21 273312 0 2024-02-16T22:21:51.222 INFO:tasks.workunit.client.0.smithi114.stdout:8/214: mkdir d4/d16/d25/d46 0 2024-02-16T22:21:51.224 INFO:tasks.workunit.client.1.smithi123.stdout:9/350: write d4/d1e/f42 [7919221,76344] 0 2024-02-16T22:21:51.226 INFO:tasks.workunit.client.0.smithi114.stdout:4/292: rename dd/c1a to dd/d1b/d24/c67 0 2024-02-16T22:21:51.228 INFO:tasks.workunit.client.0.smithi114.stdout:8/215: symlink d4/d24/d2c/l47 0 2024-02-16T22:21:51.229 INFO:tasks.workunit.client.0.smithi114.stdout:8/216: truncate d4/d11/d15/d31/f37 776956 0 2024-02-16T22:21:51.231 INFO:tasks.workunit.client.0.smithi114.stdout:3/178: rmdir d4/d26 39 2024-02-16T22:21:51.232 INFO:tasks.workunit.client.1.smithi123.stdout:3/245: sync 2024-02-16T22:21:51.233 INFO:tasks.workunit.client.1.smithi123.stdout:9/351: mknod d4/c72 0 2024-02-16T22:21:51.234 INFO:tasks.workunit.client.0.smithi114.stdout:9/223: getdents d2/d33 0 2024-02-16T22:21:51.235 INFO:tasks.workunit.client.1.smithi123.stdout:9/352: truncate d4/d17/d24/f49 993591 0 2024-02-16T22:21:51.236 INFO:tasks.workunit.client.0.smithi114.stdout:8/217: dwrite d4/d11/f17 [4194304,4194304] 0 2024-02-16T22:21:51.238 INFO:tasks.workunit.client.1.smithi123.stdout:8/344: link d0/fd d0/f75 0 2024-02-16T22:21:51.239 INFO:tasks.workunit.client.1.smithi123.stdout:8/345: chown d0/d17/d26/f5b 68734 1 2024-02-16T22:21:51.240 INFO:tasks.workunit.client.1.smithi123.stdout:6/292: truncate d0/d6/d2f/f2a 528495 0 2024-02-16T22:21:51.242 INFO:tasks.workunit.client.0.smithi114.stdout:4/293: mkdir dd/d1b/d68 0 2024-02-16T22:21:51.243 INFO:tasks.workunit.client.1.smithi123.stdout:6/293: read d0/d9/dc/d5f/f2d [2437667,129862] 0 2024-02-16T22:21:51.243 INFO:tasks.workunit.client.1.smithi123.stdout:4/295: rmdir d5/d1c/d42 39 2024-02-16T22:21:51.243 INFO:tasks.workunit.client.1.smithi123.stdout:6/294: fsync d0/f60 0 2024-02-16T22:21:51.244 INFO:tasks.workunit.client.1.smithi123.stdout:1/246: creat d3/d6/db/f5c x:0 0 0 2024-02-16T22:21:51.247 INFO:tasks.workunit.client.1.smithi123.stdout:2/270: getdents d4 0 2024-02-16T22:21:51.248 INFO:tasks.workunit.client.0.smithi114.stdout:6/194: rename d6/dd/d25/f2a to d6/f3e 0 2024-02-16T22:21:51.248 INFO:tasks.workunit.client.0.smithi114.stdout:9/224: mkdir d2/d1a/d23/d35/d47 0 2024-02-16T22:21:51.249 INFO:tasks.workunit.client.1.smithi123.stdout:1/247: dwrite d3/d6/f46 [0,4194304] 0 2024-02-16T22:21:51.249 INFO:tasks.workunit.client.0.smithi114.stdout:6/195: write d6/da/f1b [4951177,10959] 0 2024-02-16T22:21:51.251 INFO:tasks.workunit.client.0.smithi114.stdout:3/179: creat d4/d26/f42 x:0 0 0 2024-02-16T22:21:51.254 INFO:tasks.workunit.client.0.smithi114.stdout:4/294: creat dd/d2e/d56/f69 x:0 0 0 2024-02-16T22:21:51.255 INFO:tasks.workunit.client.1.smithi123.stdout:5/231: creat d6/dd/f50 x:0 0 0 2024-02-16T22:21:51.260 INFO:tasks.workunit.client.0.smithi114.stdout:8/218: rename d4/c21 to d4/d16/d25/c48 0 2024-02-16T22:21:51.261 INFO:tasks.workunit.client.0.smithi114.stdout:4/295: dwrite dd/f19 [0,4194304] 0 2024-02-16T22:21:51.262 INFO:tasks.workunit.client.0.smithi114.stdout:4/296: chown dd/l10 14778 1 2024-02-16T22:21:51.265 INFO:tasks.workunit.client.0.smithi114.stdout:6/196: creat d6/da/f3f x:0 0 0 2024-02-16T22:21:51.265 INFO:tasks.workunit.client.0.smithi114.stdout:1/166: write d4/d9/f19 [1856420,112123] 0 2024-02-16T22:21:51.265 INFO:tasks.workunit.client.0.smithi114.stdout:8/219: read d4/d11/f17 [1708075,1921] 0 2024-02-16T22:21:51.267 INFO:tasks.workunit.client.0.smithi114.stdout:8/220: write d4/f1d [4159516,9077] 0 2024-02-16T22:21:51.270 INFO:tasks.workunit.client.0.smithi114.stdout:8/221: readlink d4/l5 0 2024-02-16T22:21:51.270 INFO:tasks.workunit.client.0.smithi114.stdout:9/225: write d2/d1a/d23/d35/d1f/f2e [2490464,106578] 0 2024-02-16T22:21:51.272 INFO:tasks.workunit.client.1.smithi123.stdout:1/248: symlink d3/d6/d49/l5d 0 2024-02-16T22:21:51.276 INFO:tasks.workunit.client.1.smithi123.stdout:5/232: creat d6/dd/d44/f51 x:0 0 0 2024-02-16T22:21:51.278 INFO:tasks.workunit.client.0.smithi114.stdout:4/297: creat dd/d2e/f6a x:0 0 0 2024-02-16T22:21:51.278 INFO:tasks.workunit.client.1.smithi123.stdout:1/249: dwrite d3/d6/db/d10/f4a [0,4194304] 0 2024-02-16T22:21:51.279 INFO:tasks.workunit.client.1.smithi123.stdout:8/346: creat d0/d17/d26/d68/f76 x:0 0 0 2024-02-16T22:21:51.279 INFO:tasks.workunit.client.1.smithi123.stdout:1/250: write d3/d6/d49/f50 [173332,46527] 0 2024-02-16T22:21:51.280 INFO:tasks.workunit.client.1.smithi123.stdout:4/296: getdents d5/d1c/d23/d56 0 2024-02-16T22:21:51.281 INFO:tasks.workunit.client.1.smithi123.stdout:1/251: write d3/d6/f46 [1557536,93455] 0 2024-02-16T22:21:51.283 INFO:tasks.workunit.client.1.smithi123.stdout:1/252: fsync d3/d6/db/d10/d2d/f5a 0 2024-02-16T22:21:51.283 INFO:tasks.workunit.client.1.smithi123.stdout:6/295: creat d0/d9/dc/d3a/d70/f72 x:0 0 0 2024-02-16T22:21:51.288 INFO:tasks.workunit.client.1.smithi123.stdout:3/246: creat d0/f58 x:0 0 0 2024-02-16T22:21:51.288 INFO:tasks.workunit.client.1.smithi123.stdout:3/247: chown d0/f19 2144592271 1 2024-02-16T22:21:51.288 INFO:tasks.workunit.client.1.smithi123.stdout:3/248: write d0/f4c [2629695,10130] 0 2024-02-16T22:21:51.297 INFO:tasks.workunit.client.0.smithi114.stdout:6/197: creat d6/dd/d25/d34/f40 x:0 0 0 2024-02-16T22:21:51.314 INFO:tasks.workunit.client.1.smithi123.stdout:0/253: sync 2024-02-16T22:21:51.317 INFO:tasks.workunit.client.1.smithi123.stdout:6/296: mknod d0/d9/dc/d3a/d70/c73 0 2024-02-16T22:21:51.318 INFO:tasks.workunit.client.1.smithi123.stdout:1/253: truncate d3/d6/d11/f17 1842183 0 2024-02-16T22:21:51.318 INFO:tasks.workunit.client.1.smithi123.stdout:0/254: rename d1/d8 to d1/d8/d13/d47 22 2024-02-16T22:21:51.319 INFO:tasks.workunit.client.1.smithi123.stdout:6/297: chown d0/d9/dc/f3f 6937 1 2024-02-16T22:21:51.319 INFO:tasks.workunit.client.1.smithi123.stdout:0/255: chown d1/d8/d13/c33 340 1 2024-02-16T22:21:51.320 INFO:tasks.workunit.client.1.smithi123.stdout:6/298: chown d0/d6/d1a/d2e/c4f 173303102 1 2024-02-16T22:21:51.322 INFO:tasks.workunit.client.1.smithi123.stdout:1/254: dread d3/d6/db/d10/f4a [0,4194304] 0 2024-02-16T22:21:51.326 INFO:tasks.workunit.client.1.smithi123.stdout:4/297: dread f4 [0,4194304] 0 2024-02-16T22:21:51.368 INFO:tasks.workunit.client.1.smithi123.stdout:2/271: sync 2024-02-16T22:21:51.371 INFO:tasks.workunit.client.1.smithi123.stdout:2/272: dread d4/d1a/d1d/f2b [0,4194304] 0 2024-02-16T22:21:51.371 INFO:tasks.workunit.client.1.smithi123.stdout:2/273: chown d4/d1a/d1d/d13/c44 5205 1 2024-02-16T22:21:51.407 INFO:tasks.workunit.client.1.smithi123.stdout:8/347: sync 2024-02-16T22:21:51.408 INFO:tasks.workunit.client.1.smithi123.stdout:8/348: fdatasync d0/d17/d43/d34/f38 0 2024-02-16T22:21:51.409 INFO:tasks.workunit.client.1.smithi123.stdout:8/349: read d0/d12/f50 [781093,75900] 0 2024-02-16T22:21:51.409 INFO:tasks.workunit.client.1.smithi123.stdout:8/350: fdatasync d0/d17/d26/d2a/f54 0 2024-02-16T22:21:51.487 INFO:tasks.workunit.client.0.smithi114.stdout:8/222: mknod d4/d16/d25/d39/c49 0 2024-02-16T22:21:51.493 INFO:tasks.workunit.client.0.smithi114.stdout:8/223: dread d4/d11/f20 [0,4194304] 0 2024-02-16T22:21:51.496 INFO:tasks.workunit.client.0.smithi114.stdout:1/167: rename d4/d9/f19 to d4/f34 0 2024-02-16T22:21:51.497 INFO:tasks.workunit.client.1.smithi123.stdout:3/249: truncate d0/f12 2845713 0 2024-02-16T22:21:51.498 INFO:tasks.workunit.client.1.smithi123.stdout:3/250: read - d0/d20/d29/d34/f4b zero size 2024-02-16T22:21:51.502 INFO:tasks.workunit.client.1.smithi123.stdout:7/285: truncate d3/d12/d34/f3c 84097 0 2024-02-16T22:21:51.504 INFO:tasks.workunit.client.1.smithi123.stdout:6/299: write d0/d9/f19 [4876909,51326] 0 2024-02-16T22:21:51.504 INFO:tasks.workunit.client.1.smithi123.stdout:6/300: fsync d0/d9/f51 0 2024-02-16T22:21:51.506 INFO:tasks.workunit.client.1.smithi123.stdout:1/255: mkdir d3/d6/d11/d5e 0 2024-02-16T22:21:51.508 INFO:tasks.workunit.client.1.smithi123.stdout:4/298: truncate d5/d1c/d42/f4d 504791 0 2024-02-16T22:21:51.509 INFO:tasks.workunit.client.1.smithi123.stdout:4/299: write d5/f1e [3380828,60510] 0 2024-02-16T22:21:51.510 INFO:tasks.workunit.client.1.smithi123.stdout:4/300: dread d5/d1c/f3d [0,4194304] 0 2024-02-16T22:21:51.515 INFO:tasks.workunit.client.1.smithi123.stdout:8/351: write d0/d17/d43/f28 [3730025,113252] 0 2024-02-16T22:21:51.515 INFO:tasks.workunit.client.1.smithi123.stdout:5/233: link d6/dd/d44/f4d d6/dd/df/d1f/d25/d37/f52 0 2024-02-16T22:21:51.517 INFO:tasks.workunit.client.1.smithi123.stdout:3/251: creat d0/d40/f59 x:0 0 0 2024-02-16T22:21:51.518 INFO:tasks.workunit.client.1.smithi123.stdout:9/353: write d4/f12 [7418226,28992] 0 2024-02-16T22:21:51.523 INFO:tasks.workunit.client.1.smithi123.stdout:4/301: mknod d5/c58 0 2024-02-16T22:21:51.524 INFO:tasks.workunit.client.1.smithi123.stdout:2/274: unlink d4/f16 0 2024-02-16T22:21:51.528 INFO:tasks.workunit.client.1.smithi123.stdout:4/302: dwrite d5/f17 [4194304,4194304] 0 2024-02-16T22:21:51.528 INFO:tasks.workunit.client.1.smithi123.stdout:3/252: dread d0/f17 [0,4194304] 0 2024-02-16T22:21:51.531 INFO:tasks.workunit.client.1.smithi123.stdout:6/301: mkdir d0/d74 0 2024-02-16T22:21:51.531 INFO:tasks.workunit.client.1.smithi123.stdout:9/354: truncate d4/d1e/d3a/f50 519105 0 2024-02-16T22:21:51.532 INFO:tasks.workunit.client.1.smithi123.stdout:6/302: chown d0/d9/dc/d39/c49 6214 1 2024-02-16T22:21:51.532 INFO:tasks.workunit.client.1.smithi123.stdout:9/355: chown d4/c72 0 1 2024-02-16T22:21:51.532 INFO:tasks.workunit.client.1.smithi123.stdout:9/356: chown d4/d17/d20/f3c 492 1 2024-02-16T22:21:51.535 INFO:tasks.workunit.client.1.smithi123.stdout:2/275: creat d4/d1a/d1d/d13/d15/f56 x:0 0 0 2024-02-16T22:21:51.536 INFO:tasks.workunit.client.1.smithi123.stdout:8/352: mkdir d0/d17/d26/d2a/d77 0 2024-02-16T22:21:51.537 INFO:tasks.workunit.client.1.smithi123.stdout:0/256: link d1/cf d1/d4/c48 0 2024-02-16T22:21:51.538 INFO:tasks.workunit.client.1.smithi123.stdout:0/257: chown d1/d4/f43 35123564 1 2024-02-16T22:21:51.539 INFO:tasks.workunit.client.1.smithi123.stdout:4/303: chown d5/d1c/d23/d36/l48 195 1 2024-02-16T22:21:51.539 INFO:tasks.workunit.client.1.smithi123.stdout:3/253: mkdir d0/d20/d5a 0 2024-02-16T22:21:51.540 INFO:tasks.workunit.client.1.smithi123.stdout:6/303: write d0/d9/dc/f67 [3867081,118434] 0 2024-02-16T22:21:51.543 INFO:tasks.workunit.client.1.smithi123.stdout:9/357: creat d4/d33/f73 x:0 0 0 2024-02-16T22:21:51.544 INFO:tasks.workunit.client.1.smithi123.stdout:5/234: creat d6/f53 x:0 0 0 2024-02-16T22:21:51.545 INFO:tasks.workunit.client.1.smithi123.stdout:2/276: rmdir d4/d1a/d29/d53 39 2024-02-16T22:21:51.549 INFO:tasks.workunit.client.1.smithi123.stdout:3/254: mknod d0/d20/d2e/c5b 0 2024-02-16T22:21:51.551 INFO:tasks.workunit.client.1.smithi123.stdout:9/358: rmdir d4/d17/d20 39 2024-02-16T22:21:51.551 INFO:tasks.workunit.client.1.smithi123.stdout:9/359: chown d4/d1e/d3a 0 1 2024-02-16T22:21:51.551 INFO:tasks.workunit.client.1.smithi123.stdout:5/235: mknod d6/dd/df/d2b/c54 0 2024-02-16T22:21:51.551 INFO:tasks.workunit.client.1.smithi123.stdout:9/360: chown d4/d17/d2b/d6c 52663412 1 2024-02-16T22:21:51.552 INFO:tasks.workunit.client.1.smithi123.stdout:2/277: symlink d4/l57 0 2024-02-16T22:21:51.552 INFO:tasks.workunit.client.1.smithi123.stdout:3/255: dread d0/f4c [0,4194304] 0 2024-02-16T22:21:51.553 INFO:tasks.workunit.client.1.smithi123.stdout:9/361: stat d4/d17/d24/l68 0 2024-02-16T22:21:51.553 INFO:tasks.workunit.client.1.smithi123.stdout:4/304: link d5/f40 d5/f59 0 2024-02-16T22:21:51.554 INFO:tasks.workunit.client.1.smithi123.stdout:4/305: truncate d5/d1c/d23/f52 703613 0 2024-02-16T22:21:51.555 INFO:tasks.workunit.client.1.smithi123.stdout:5/236: unlink l4 0 2024-02-16T22:21:51.557 INFO:tasks.workunit.client.1.smithi123.stdout:3/256: creat d0/d50/f5c x:0 0 0 2024-02-16T22:21:51.559 INFO:tasks.workunit.client.1.smithi123.stdout:4/306: mknod d5/d1c/d23/d36/c5a 0 2024-02-16T22:21:51.559 INFO:tasks.workunit.client.1.smithi123.stdout:5/237: dwrite d6/dd/df/f28 [0,4194304] 0 2024-02-16T22:21:51.560 INFO:tasks.workunit.client.1.smithi123.stdout:2/278: getdents d4/d1a/d1d/d13/d15/d1b/d4a 0 2024-02-16T22:21:51.562 INFO:tasks.workunit.client.1.smithi123.stdout:5/238: unlink l2 0 2024-02-16T22:21:51.563 INFO:tasks.workunit.client.1.smithi123.stdout:5/239: chown d6/dd/df/d1f/d25/f31 0 1 2024-02-16T22:21:51.563 INFO:tasks.workunit.client.1.smithi123.stdout:9/362: getdents d4/d17/d20 0 2024-02-16T22:21:51.564 INFO:tasks.workunit.client.1.smithi123.stdout:9/363: chown d4/d33/f3b 109936708 1 2024-02-16T22:21:51.614 INFO:tasks.workunit.client.0.smithi114.stdout:6/198: mkdir d6/d2c/d38/d41 0 2024-02-16T22:21:51.618 INFO:tasks.workunit.client.0.smithi114.stdout:1/168: dwrite d4/fa [0,4194304] 0 2024-02-16T22:21:51.620 INFO:tasks.workunit.client.1.smithi123.stdout:5/240: symlink d6/dd/df/l55 0 2024-02-16T22:21:51.622 INFO:tasks.workunit.client.1.smithi123.stdout:5/241: truncate d6/dd/df/d1f/d25/f31 30444 0 2024-02-16T22:21:51.622 INFO:tasks.workunit.client.1.smithi123.stdout:7/286: dread d3/d12/d34/f3c [0,4194304] 0 2024-02-16T22:21:51.623 INFO:tasks.workunit.client.1.smithi123.stdout:7/287: chown d3/c20 9750 1 2024-02-16T22:21:51.624 INFO:tasks.workunit.client.1.smithi123.stdout:7/288: dread - d3/d12/d24/f55 zero size 2024-02-16T22:21:51.625 INFO:tasks.workunit.client.1.smithi123.stdout:7/289: write d3/d12/d1d/d1e/f4a [948404,66051] 0 2024-02-16T22:21:51.625 INFO:tasks.workunit.client.0.smithi114.stdout:4/298: getdents dd/d2e 0 2024-02-16T22:21:51.628 INFO:tasks.workunit.client.1.smithi123.stdout:4/307: dread d5/d1c/d23/d36/f45 [0,4194304] 0 2024-02-16T22:21:51.629 INFO:tasks.workunit.client.0.smithi114.stdout:6/199: creat d6/dd/d25/d34/f42 x:0 0 0 2024-02-16T22:21:51.631 INFO:tasks.workunit.client.0.smithi114.stdout:3/180: dwrite d4/dd/f19 [0,4194304] 0 2024-02-16T22:21:51.632 INFO:tasks.workunit.client.1.smithi123.stdout:9/364: rename d4/d36/d57/f67 to d4/d56/f74 0 2024-02-16T22:21:51.633 INFO:tasks.workunit.client.0.smithi114.stdout:9/226: dwrite d2/f4 [0,4194304] 0 2024-02-16T22:21:51.634 INFO:tasks.workunit.client.0.smithi114.stdout:9/227: stat f1 0 2024-02-16T22:21:51.634 INFO:tasks.workunit.client.0.smithi114.stdout:6/200: write d6/dd/f17 [3557866,90610] 0 2024-02-16T22:21:51.636 INFO:tasks.workunit.client.0.smithi114.stdout:6/201: write d6/dd/d25/d34/f40 [366882,70434] 0 2024-02-16T22:21:51.637 INFO:tasks.workunit.client.0.smithi114.stdout:1/169: fdatasync d4/f13 0 2024-02-16T22:21:51.640 INFO:tasks.workunit.client.0.smithi114.stdout:8/224: dwrite d4/d11/f17 [4194304,4194304] 0 2024-02-16T22:21:51.648 INFO:tasks.workunit.client.1.smithi123.stdout:7/290: unlink d3/fe 0 2024-02-16T22:21:51.650 INFO:tasks.workunit.client.0.smithi114.stdout:3/181: truncate d4/f1c 3137132 0 2024-02-16T22:21:51.651 INFO:tasks.workunit.client.0.smithi114.stdout:3/182: write d4/d24/f32 [3996639,14550] 0 2024-02-16T22:21:51.652 INFO:tasks.workunit.client.0.smithi114.stdout:3/183: fsync d4/d26/f42 0 2024-02-16T22:21:51.653 INFO:tasks.workunit.client.0.smithi114.stdout:6/202: mknod d6/d2c/c43 0 2024-02-16T22:21:51.656 INFO:tasks.workunit.client.1.smithi123.stdout:6/304: getdents d0 0 2024-02-16T22:21:51.656 INFO:tasks.workunit.client.1.smithi123.stdout:7/291: creat d3/d12/d1d/d1e/f5f x:0 0 0 2024-02-16T22:21:51.659 INFO:tasks.workunit.client.1.smithi123.stdout:4/308: mknod d5/d1c/c5b 0 2024-02-16T22:21:51.660 INFO:tasks.workunit.client.0.smithi114.stdout:8/225: mknod d4/d24/c4a 0 2024-02-16T22:21:51.662 INFO:tasks.workunit.client.1.smithi123.stdout:6/305: dwrite d0/d9/dc/d5f/f2d [0,4194304] 0 2024-02-16T22:21:51.666 INFO:tasks.workunit.client.1.smithi123.stdout:1/256: write d3/d6/d11/f17 [646905,96678] 0 2024-02-16T22:21:51.667 INFO:tasks.workunit.client.1.smithi123.stdout:1/257: chown d3/d52/c58 22377 1 2024-02-16T22:21:51.668 INFO:tasks.workunit.client.1.smithi123.stdout:7/292: rmdir d3/d5 39 2024-02-16T22:21:51.671 INFO:tasks.workunit.client.0.smithi114.stdout:9/228: rename d2/dd/l1b to d2/d33/l48 0 2024-02-16T22:21:51.672 INFO:tasks.workunit.client.1.smithi123.stdout:4/309: unlink d5/d1c/d23/d36/l48 0 2024-02-16T22:21:51.676 INFO:tasks.workunit.client.1.smithi123.stdout:6/306: mknod d0/d6/d2f/c75 0 2024-02-16T22:21:51.677 INFO:tasks.workunit.client.1.smithi123.stdout:5/242: getdents d6/dd/df/d2b 0 2024-02-16T22:21:51.678 INFO:tasks.workunit.client.1.smithi123.stdout:5/243: chown d6/dd/d44 4782637 1 2024-02-16T22:21:51.678 INFO:tasks.workunit.client.1.smithi123.stdout:0/258: truncate d1/f1f 3771038 0 2024-02-16T22:21:51.679 INFO:tasks.workunit.client.0.smithi114.stdout:8/226: unlink d4/d24/c2d 0 2024-02-16T22:21:51.681 INFO:tasks.workunit.client.0.smithi114.stdout:9/229: creat d2/d1a/f49 x:0 0 0 2024-02-16T22:21:51.682 INFO:tasks.workunit.client.0.smithi114.stdout:3/184: mkdir d4/d43 0 2024-02-16T22:21:51.682 INFO:tasks.workunit.client.1.smithi123.stdout:0/259: dwrite d1/f22 [0,4194304] 0 2024-02-16T22:21:51.685 INFO:tasks.workunit.client.1.smithi123.stdout:9/365: getdents d4/d1e/d3a 0 2024-02-16T22:21:51.685 INFO:tasks.workunit.client.1.smithi123.stdout:8/353: sync 2024-02-16T22:21:51.687 INFO:tasks.workunit.client.1.smithi123.stdout:6/307: fsync d0/d6/f18 0 2024-02-16T22:21:51.687 INFO:tasks.workunit.client.1.smithi123.stdout:9/366: read - d4/d36/f5f zero size 2024-02-16T22:21:51.688 INFO:tasks.workunit.client.0.smithi114.stdout:8/227: rename d4/d11/f2f to d4/d16/f4b 0 2024-02-16T22:21:51.689 INFO:tasks.workunit.client.0.smithi114.stdout:9/230: rename d2 to d2/d4a 22 2024-02-16T22:21:51.693 INFO:tasks.workunit.client.0.smithi114.stdout:3/185: write d4/d26/f40 [570308,24848] 0 2024-02-16T22:21:51.694 INFO:tasks.workunit.client.1.smithi123.stdout:5/244: mkdir d6/dd/df/d2b/d56 0 2024-02-16T22:21:51.695 INFO:tasks.workunit.client.0.smithi114.stdout:3/186: readlink d4/dd/l3b 0 2024-02-16T22:21:51.696 INFO:tasks.workunit.client.0.smithi114.stdout:3/187: rename d4/d43 to d4/d43/d44 22 2024-02-16T22:21:51.697 INFO:tasks.workunit.client.1.smithi123.stdout:7/293: symlink d3/d12/d52/l60 0 2024-02-16T22:21:51.697 INFO:tasks.workunit.client.0.smithi114.stdout:9/231: dwrite d2/dd/f2a [0,4194304] 0 2024-02-16T22:21:51.701 INFO:tasks.workunit.client.1.smithi123.stdout:4/310: symlink d5/d1c/l5c 0 2024-02-16T22:21:51.701 INFO:tasks.workunit.client.1.smithi123.stdout:4/311: read d5/d1c/f21 [921230,109263] 0 2024-02-16T22:21:51.702 INFO:tasks.workunit.client.1.smithi123.stdout:4/312: readlink d5/ld 0 2024-02-16T22:21:51.703 INFO:tasks.workunit.client.1.smithi123.stdout:0/260: write d1/d8/d13/f14 [2761700,33378] 0 2024-02-16T22:21:51.704 INFO:tasks.workunit.client.0.smithi114.stdout:1/170: dwrite d4/f15 [0,4194304] 0 2024-02-16T22:21:51.706 INFO:tasks.workunit.client.0.smithi114.stdout:6/203: getdents d6/dd 0 2024-02-16T22:21:51.709 INFO:tasks.workunit.client.0.smithi114.stdout:3/188: mkdir d4/d24/d2d/d45 0 2024-02-16T22:21:51.711 INFO:tasks.workunit.client.0.smithi114.stdout:6/204: truncate d6/dd/d25/d26/f2d 8623156 0 2024-02-16T22:21:51.711 INFO:tasks.workunit.client.0.smithi114.stdout:3/189: chown d4/d24/d2d/d45 479321 1 2024-02-16T22:21:51.712 INFO:tasks.workunit.client.1.smithi123.stdout:6/308: fdatasync d0/d6/d1a/f36 0 2024-02-16T22:21:51.713 INFO:tasks.workunit.client.0.smithi114.stdout:6/205: chown d6/dd/d25/f27 56 1 2024-02-16T22:21:51.715 INFO:tasks.workunit.client.0.smithi114.stdout:1/171: dwrite d4/d9/f2f [0,4194304] 0 2024-02-16T22:21:51.715 INFO:tasks.workunit.client.1.smithi123.stdout:9/367: dread d4/d17/d20/f22 [0,4194304] 0 2024-02-16T22:21:51.716 INFO:tasks.workunit.client.0.smithi114.stdout:6/206: write d6/da/f3f [1025234,119068] 0 2024-02-16T22:21:51.716 INFO:tasks.workunit.client.0.smithi114.stdout:1/172: stat d4/f30 0 2024-02-16T22:21:51.717 INFO:tasks.workunit.client.0.smithi114.stdout:3/190: dread d4/d26/f40 [0,4194304] 0 2024-02-16T22:21:51.719 INFO:tasks.workunit.client.1.smithi123.stdout:7/294: readlink l1 0 2024-02-16T22:21:51.720 INFO:tasks.workunit.client.1.smithi123.stdout:7/295: dread d3/fd [0,4194304] 0 2024-02-16T22:21:51.721 INFO:tasks.workunit.client.1.smithi123.stdout:7/296: truncate d3/d12/d24/f56 325255 0 2024-02-16T22:21:51.721 INFO:tasks.workunit.client.1.smithi123.stdout:4/313: mknod d5/d1c/d23/d47/c5d 0 2024-02-16T22:21:51.723 INFO:tasks.workunit.client.1.smithi123.stdout:7/297: fsync d3/d12/d24/f5d 0 2024-02-16T22:21:51.723 INFO:tasks.workunit.client.0.smithi114.stdout:4/299: truncate dd/d1b/d37/d38/f46 1481937 0 2024-02-16T22:21:51.728 INFO:tasks.workunit.client.0.smithi114.stdout:6/207: symlink d6/da/l44 0 2024-02-16T22:21:51.728 INFO:tasks.workunit.client.1.smithi123.stdout:4/314: dwrite d5/d1c/d42/f4a [0,4194304] 0 2024-02-16T22:21:51.729 INFO:tasks.workunit.client.1.smithi123.stdout:4/315: write d5/d1c/d23/d36/f55 [692653,42707] 0 2024-02-16T22:21:51.730 INFO:tasks.workunit.client.1.smithi123.stdout:9/368: mknod d4/d1e/d3a/c75 0 2024-02-16T22:21:51.730 INFO:tasks.workunit.client.1.smithi123.stdout:4/316: chown d5/d1c/l3f 263069 1 2024-02-16T22:21:51.731 INFO:tasks.workunit.client.1.smithi123.stdout:4/317: read d5/d1c/f25 [946941,29433] 0 2024-02-16T22:21:51.733 INFO:tasks.workunit.client.1.smithi123.stdout:3/257: sync 2024-02-16T22:21:51.734 INFO:tasks.workunit.client.1.smithi123.stdout:3/258: chown d0/d20/d29/c30 311 1 2024-02-16T22:21:51.734 INFO:tasks.workunit.client.1.smithi123.stdout:3/259: fdatasync d0/d40/f56 0 2024-02-16T22:21:51.736 INFO:tasks.workunit.client.0.smithi114.stdout:6/208: dwrite d6/dd/d25/d34/f40 [0,4194304] 0 2024-02-16T22:21:51.737 INFO:tasks.workunit.client.0.smithi114.stdout:1/173: mknod d4/d33/c35 0 2024-02-16T22:21:51.738 INFO:tasks.workunit.client.0.smithi114.stdout:1/174: readlink d4/d9/d2a/l2b 0 2024-02-16T22:21:51.740 INFO:tasks.workunit.client.1.smithi123.stdout:0/261: unlink d1/l3e 0 2024-02-16T22:21:51.741 INFO:tasks.workunit.client.0.smithi114.stdout:3/191: mkdir d4/d46 0 2024-02-16T22:21:51.742 INFO:tasks.workunit.client.0.smithi114.stdout:3/192: fsync d4/d24/f32 0 2024-02-16T22:21:51.743 INFO:tasks.workunit.client.1.smithi123.stdout:8/354: creat d0/d17/d26/d2a/f78 x:0 0 0 2024-02-16T22:21:51.748 INFO:tasks.workunit.client.0.smithi114.stdout:3/193: dwrite d4/dd/f19 [0,4194304] 0 2024-02-16T22:21:51.750 INFO:tasks.workunit.client.0.smithi114.stdout:1/175: unlink d4/d9/d2a/l2e 0 2024-02-16T22:21:51.751 INFO:tasks.workunit.client.1.smithi123.stdout:9/369: creat d4/d17/d24/f76 x:0 0 0 2024-02-16T22:21:51.752 INFO:tasks.workunit.client.1.smithi123.stdout:9/370: chown d4/f10 0 1 2024-02-16T22:21:51.753 INFO:tasks.workunit.client.1.smithi123.stdout:4/318: rmdir d5 39 2024-02-16T22:21:51.757 INFO:tasks.workunit.client.0.smithi114.stdout:6/209: symlink d6/d2c/d38/d41/l45 0 2024-02-16T22:21:51.758 INFO:tasks.workunit.client.1.smithi123.stdout:5/245: rmdir d6/dd 39 2024-02-16T22:21:51.759 INFO:tasks.workunit.client.1.smithi123.stdout:2/279: sync 2024-02-16T22:21:51.759 INFO:tasks.workunit.client.1.smithi123.stdout:1/258: sync 2024-02-16T22:21:51.760 INFO:tasks.workunit.client.1.smithi123.stdout:1/259: fdatasync d3/d6/db/d10/d2d/f38 0 2024-02-16T22:21:51.761 INFO:tasks.workunit.client.1.smithi123.stdout:8/355: unlink d0/d12/f46 0 2024-02-16T22:21:51.761 INFO:tasks.workunit.client.0.smithi114.stdout:8/228: write d4/d24/f3d [37290,45699] 0 2024-02-16T22:21:51.763 INFO:tasks.workunit.client.1.smithi123.stdout:6/309: link d0/d9/dc/d39/c49 d0/d9/dc/d3a/d52/c76 0 2024-02-16T22:21:51.764 INFO:tasks.workunit.client.0.smithi114.stdout:6/210: dwrite d6/f1e [0,4194304] 0 2024-02-16T22:21:51.765 INFO:tasks.workunit.client.1.smithi123.stdout:1/260: dwrite d3/f35 [0,4194304] 0 2024-02-16T22:21:51.766 INFO:tasks.workunit.client.0.smithi114.stdout:6/211: dread - d6/f30 zero size 2024-02-16T22:21:51.769 INFO:tasks.workunit.client.0.smithi114.stdout:9/232: write d2/dd/f10 [1666421,109233] 0 2024-02-16T22:21:51.770 INFO:tasks.workunit.client.0.smithi114.stdout:3/194: mknod d4/d43/c47 0 2024-02-16T22:21:51.771 INFO:tasks.workunit.client.0.smithi114.stdout:3/195: readlink d4/d24/d2d/l37 0 2024-02-16T22:21:51.773 INFO:tasks.workunit.client.0.smithi114.stdout:1/176: unlink d4/d9/c1e 0 2024-02-16T22:21:51.774 INFO:tasks.workunit.client.1.smithi123.stdout:4/319: dread d5/d1c/d42/f4a [0,4194304] 0 2024-02-16T22:21:51.776 INFO:tasks.workunit.client.1.smithi123.stdout:3/260: mkdir d0/d20/d29/d34/d5d 0 2024-02-16T22:21:51.776 INFO:tasks.workunit.client.1.smithi123.stdout:4/320: write d5/f1d [2094850,112994] 0 2024-02-16T22:21:51.783 INFO:tasks.workunit.client.1.smithi123.stdout:2/280: unlink d4/d1a/d1d/f17 0 2024-02-16T22:21:51.785 INFO:tasks.workunit.client.1.smithi123.stdout:5/246: dwrite d6/dd/df/f28 [0,4194304] 0 2024-02-16T22:21:51.786 INFO:tasks.workunit.client.0.smithi114.stdout:3/196: mkdir d4/d46/d48 0 2024-02-16T22:21:51.787 INFO:tasks.workunit.client.0.smithi114.stdout:3/197: dread - d4/d26/f42 zero size 2024-02-16T22:21:51.794 INFO:tasks.workunit.client.0.smithi114.stdout:9/233: rename d2/d1a/d23/d35/c24 to d2/d1a/d23/d35/d1f/c4b 0 2024-02-16T22:21:51.796 INFO:tasks.workunit.client.0.smithi114.stdout:1/177: creat d4/f36 x:0 0 0 2024-02-16T22:21:51.797 INFO:tasks.workunit.client.0.smithi114.stdout:1/178: chown d4/f2d 7534 1 2024-02-16T22:21:51.798 INFO:tasks.workunit.client.1.smithi123.stdout:6/310: creat d0/d6/d1a/d2e/f77 x:0 0 0 2024-02-16T22:21:51.799 INFO:tasks.workunit.client.0.smithi114.stdout:9/234: unlink d2/d1a/d23/d32/l41 0 2024-02-16T22:21:51.801 INFO:tasks.workunit.client.1.smithi123.stdout:1/261: rename d3/d6/d11/d5e to d3/d6/db/d10/d2a/d5f 0 2024-02-16T22:21:51.801 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:51 smithi123 ceph-mon[37161]: pgmap v23: 65 pgs: 65 active+clean; 606 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 62 MiB/s wr, 182 op/s 2024-02-16T22:21:51.804 INFO:tasks.workunit.client.0.smithi114.stdout:9/235: mknod d2/d33/d3f/c4c 0 2024-02-16T22:21:51.810 INFO:tasks.workunit.client.0.smithi114.stdout:1/179: creat d4/f37 x:0 0 0 2024-02-16T22:21:51.810 INFO:tasks.workunit.client.1.smithi123.stdout:7/298: getdents d3/d26/d3e 0 2024-02-16T22:21:51.811 INFO:tasks.workunit.client.0.smithi114.stdout:4/300: truncate dd/d1b/d37/d38/f46 2428786 0 2024-02-16T22:21:51.812 INFO:tasks.workunit.client.0.smithi114.stdout:4/301: chown dd/d1b/d41 26 1 2024-02-16T22:21:51.812 INFO:tasks.workunit.client.0.smithi114.stdout:9/236: creat d2/d1a/d23/d35/d47/f4d x:0 0 0 2024-02-16T22:21:51.815 INFO:tasks.workunit.client.0.smithi114.stdout:1/180: unlink d4/d9/f2f 0 2024-02-16T22:21:51.821 INFO:tasks.workunit.client.1.smithi123.stdout:5/247: creat d6/dd/df/f57 x:0 0 0 2024-02-16T22:21:51.821 INFO:tasks.workunit.client.1.smithi123.stdout:5/248: readlink d6/dd/df/l55 0 2024-02-16T22:21:51.822 INFO:tasks.workunit.client.1.smithi123.stdout:5/249: chown d6/dd/df/d1f/l4a 50279312 1 2024-02-16T22:21:51.823 INFO:tasks.workunit.client.1.smithi123.stdout:5/250: read d6/dd/df/d1f/d25/f2f [6476,46742] 0 2024-02-16T22:21:51.823 INFO:tasks.workunit.client.1.smithi123.stdout:5/251: chown d6 0 1 2024-02-16T22:21:51.823 INFO:tasks.workunit.client.1.smithi123.stdout:5/252: dread - d6/dd/df/f41 zero size 2024-02-16T22:21:51.826 INFO:tasks.workunit.client.1.smithi123.stdout:9/371: write d4/d17/d20/f6a [1532321,4896] 0 2024-02-16T22:21:51.830 INFO:tasks.workunit.client.1.smithi123.stdout:6/311: unlink d0/f38 0 2024-02-16T22:21:51.831 INFO:tasks.workunit.client.0.smithi114.stdout:4/302: creat dd/d1b/d24/f6b x:0 0 0 2024-02-16T22:21:51.832 INFO:tasks.workunit.client.0.smithi114.stdout:9/237: rmdir d2/d1a/d23 39 2024-02-16T22:21:51.835 INFO:tasks.workunit.client.1.smithi123.stdout:3/261: rename d0/d20/d29/d34/f4b to d0/d20/d29/d34/d37/f5e 0 2024-02-16T22:21:51.836 INFO:tasks.workunit.client.0.smithi114.stdout:4/303: rename dd/d59/f65 to dd/d2e/d3f/f6c 0 2024-02-16T22:21:51.838 INFO:tasks.workunit.client.1.smithi123.stdout:1/262: symlink d3/d6/db/d10/l60 0 2024-02-16T22:21:51.838 INFO:tasks.workunit.client.1.smithi123.stdout:1/263: stat d3/f35 0 2024-02-16T22:21:51.838 INFO:tasks.workunit.client.1.smithi123.stdout:1/264: truncate d3/f8 4419975 0 2024-02-16T22:21:51.839 INFO:tasks.workunit.client.0.smithi114.stdout:1/181: write d4/d9/f11 [3799829,55512] 0 2024-02-16T22:21:51.839 INFO:tasks.workunit.client.1.smithi123.stdout:0/262: truncate d1/f1f 1208975 0 2024-02-16T22:21:51.840 INFO:tasks.workunit.client.1.smithi123.stdout:0/263: dread - d1/d8/d13/f37 zero size 2024-02-16T22:21:51.841 INFO:tasks.workunit.client.0.smithi114.stdout:9/238: chown d2/d1a/d23/d35/d47 81313022 1 2024-02-16T22:21:51.842 INFO:tasks.workunit.client.1.smithi123.stdout:4/321: mkdir d5/d1c/d5e 0 2024-02-16T22:21:51.842 INFO:tasks.workunit.client.0.smithi114.stdout:4/304: mknod dd/d1b/d68/c6d 0 2024-02-16T22:21:51.844 INFO:tasks.workunit.client.0.smithi114.stdout:1/182: unlink d4/f13 0 2024-02-16T22:21:51.846 INFO:tasks.workunit.client.1.smithi123.stdout:2/281: creat d4/d1a/d1d/d38/f58 x:0 0 0 2024-02-16T22:21:51.848 INFO:tasks.workunit.client.1.smithi123.stdout:8/356: link d0/d17/d26/d2a/f78 d0/d17/d26/f79 0 2024-02-16T22:21:51.849 INFO:tasks.workunit.client.1.smithi123.stdout:8/357: dread - d0/d29/d42/d47/f6a zero size 2024-02-16T22:21:51.849 INFO:tasks.workunit.client.0.smithi114.stdout:1/183: creat d4/d33/f38 x:0 0 0 2024-02-16T22:21:51.850 INFO:tasks.workunit.client.0.smithi114.stdout:1/184: truncate d4/d9/f12 556272 0 2024-02-16T22:21:51.851 INFO:tasks.workunit.client.1.smithi123.stdout:5/253: rename d6/dd/df/d2b to d6/dd/df/d58 0 2024-02-16T22:21:51.851 INFO:tasks.workunit.client.0.smithi114.stdout:1/185: write d4/d9/f18 [403268,61137] 0 2024-02-16T22:21:51.852 INFO:tasks.workunit.client.0.smithi114.stdout:9/239: getdents d2/d1a/d23/d32/d46 0 2024-02-16T22:21:51.852 INFO:tasks.workunit.client.1.smithi123.stdout:9/372: write d4/d17/d2b/f64 [131571,5823] 0 2024-02-16T22:21:51.855 INFO:tasks.workunit.client.1.smithi123.stdout:6/312: stat d0/d6/d1a/d2e/f77 0 2024-02-16T22:21:51.856 INFO:tasks.workunit.client.0.smithi114.stdout:9/240: mknod d2/d33/c4e 0 2024-02-16T22:21:51.860 INFO:tasks.workunit.client.1.smithi123.stdout:3/262: mknod d0/d40/c5f 0 2024-02-16T22:21:51.861 INFO:tasks.workunit.client.1.smithi123.stdout:1/265: fdatasync d3/d6/db/d10/f4a 0 2024-02-16T22:21:51.863 INFO:tasks.workunit.client.0.smithi114.stdout:1/186: link d4/f21 d4/d9/d2a/f39 0 2024-02-16T22:21:51.866 INFO:tasks.workunit.client.1.smithi123.stdout:0/264: symlink d1/d8/d3d/l49 0 2024-02-16T22:21:51.869 INFO:tasks.workunit.client.0.smithi114.stdout:1/187: dwrite d4/f25 [0,4194304] 0 2024-02-16T22:21:51.869 INFO:tasks.workunit.client.1.smithi123.stdout:4/322: creat d5/d1c/d23/d36/f5f x:0 0 0 2024-02-16T22:21:51.874 INFO:tasks.workunit.client.1.smithi123.stdout:7/299: creat d3/d5/f61 x:0 0 0 2024-02-16T22:21:51.877 INFO:tasks.workunit.client.1.smithi123.stdout:2/282: rename d4/d1a/d29 to d4/d1a/d1d/d59 0 2024-02-16T22:21:51.878 INFO:tasks.workunit.client.1.smithi123.stdout:8/358: readlink d0/l8 0 2024-02-16T22:21:51.885 INFO:tasks.workunit.client.1.smithi123.stdout:4/323: unlink d5/l15 0 2024-02-16T22:21:51.886 INFO:tasks.workunit.client.1.smithi123.stdout:7/300: mknod d3/d26/d3e/c62 0 2024-02-16T22:21:51.887 INFO:tasks.workunit.client.1.smithi123.stdout:7/301: chown d3/d12/d1d/d1e/l23 901345 1 2024-02-16T22:21:51.888 INFO:tasks.workunit.client.1.smithi123.stdout:1/266: dread d3/d6/f26 [0,4194304] 0 2024-02-16T22:21:51.890 INFO:tasks.workunit.client.1.smithi123.stdout:5/254: unlink d6/dd/d44/f4d 0 2024-02-16T22:21:51.894 INFO:tasks.workunit.client.1.smithi123.stdout:9/373: creat d4/d56/d6f/f77 x:0 0 0 2024-02-16T22:21:51.894 INFO:tasks.workunit.client.1.smithi123.stdout:9/374: stat d4/d17/d24/f69 0 2024-02-16T22:21:51.897 INFO:tasks.workunit.client.1.smithi123.stdout:3/263: mkdir d0/d20/d29/d34/d5d/d60 0 2024-02-16T22:21:51.898 INFO:tasks.workunit.client.1.smithi123.stdout:0/265: creat d1/d4/d42/f4a x:0 0 0 2024-02-16T22:21:51.899 INFO:tasks.workunit.client.1.smithi123.stdout:0/266: stat d1/d8/d13/f15 0 2024-02-16T22:21:51.899 INFO:tasks.workunit.client.0.smithi114.stdout:1/188: write d4/fb [1177257,58732] 0 2024-02-16T22:21:51.900 INFO:tasks.workunit.client.1.smithi123.stdout:7/302: creat d3/d26/d3e/f63 x:0 0 0 2024-02-16T22:21:51.903 INFO:tasks.workunit.client.1.smithi123.stdout:9/375: rename d4/d1e/f39 to d4/d36/f78 0 2024-02-16T22:21:51.904 INFO:tasks.workunit.client.1.smithi123.stdout:3/264: creat d0/d20/d29/d34/d5d/f61 x:0 0 0 2024-02-16T22:21:51.905 INFO:tasks.workunit.client.1.smithi123.stdout:4/324: mknod d5/d1c/c60 0 2024-02-16T22:21:51.907 INFO:tasks.workunit.client.1.smithi123.stdout:1/267: creat d3/d47/f61 x:0 0 0 2024-02-16T22:21:51.908 INFO:tasks.workunit.client.1.smithi123.stdout:7/303: rmdir d3/d12/d24 39 2024-02-16T22:21:51.909 INFO:tasks.workunit.client.1.smithi123.stdout:7/304: truncate d3/d12/d4d/f5a 301104 0 2024-02-16T22:21:51.909 INFO:tasks.workunit.client.1.smithi123.stdout:5/255: fdatasync d6/dd/f35 0 2024-02-16T22:21:51.911 INFO:tasks.workunit.client.1.smithi123.stdout:8/359: creat d0/d17/d26/f7a x:0 0 0 2024-02-16T22:21:51.911 INFO:tasks.workunit.client.1.smithi123.stdout:8/360: dread - d0/d17/d26/d68/f76 zero size 2024-02-16T22:21:51.912 INFO:tasks.workunit.client.1.smithi123.stdout:2/283: link d4/d1a/d1d/d13/d15/l3f d4/d1a/d1d/d13/d15/l5a 0 2024-02-16T22:21:51.914 INFO:tasks.workunit.client.1.smithi123.stdout:6/313: rename d0/l1b to d0/l78 0 2024-02-16T22:21:51.914 INFO:tasks.workunit.client.1.smithi123.stdout:6/314: chown d0/d6/f63 30589 1 2024-02-16T22:21:51.914 INFO:tasks.workunit.client.1.smithi123.stdout:6/315: readlink d0/d9/l31 0 2024-02-16T22:21:51.915 INFO:tasks.workunit.client.1.smithi123.stdout:6/316: truncate d0/d6/f5c 942339 0 2024-02-16T22:21:51.917 INFO:tasks.workunit.client.1.smithi123.stdout:0/267: truncate d1/f1f 1377096 0 2024-02-16T22:21:51.919 INFO:tasks.workunit.client.1.smithi123.stdout:6/317: dread d0/d6/f5c [0,4194304] 0 2024-02-16T22:21:51.920 INFO:tasks.workunit.client.1.smithi123.stdout:1/268: creat d3/d6/d11/f62 x:0 0 0 2024-02-16T22:21:51.921 INFO:tasks.workunit.client.1.smithi123.stdout:7/305: creat d3/d12/d1d/f64 x:0 0 0 2024-02-16T22:21:51.923 INFO:tasks.workunit.client.1.smithi123.stdout:4/325: dwrite d5/d1c/d23/d36/f45 [0,4194304] 0 2024-02-16T22:21:51.923 INFO:tasks.workunit.client.1.smithi123.stdout:3/265: symlink d0/l62 0 2024-02-16T22:21:51.924 INFO:tasks.workunit.client.1.smithi123.stdout:1/269: fsync d3/d6/d49/f50 0 2024-02-16T22:21:51.926 INFO:tasks.workunit.client.1.smithi123.stdout:0/268: dwrite d1/d8/f30 [0,4194304] 0 2024-02-16T22:21:51.927 INFO:tasks.workunit.client.1.smithi123.stdout:6/318: creat d0/d9/dc/d5f/f79 x:0 0 0 2024-02-16T22:21:51.927 INFO:tasks.workunit.client.1.smithi123.stdout:4/326: chown d5/d1c/f3d 0 1 2024-02-16T22:21:51.929 INFO:tasks.workunit.client.1.smithi123.stdout:4/327: dread - d5/d1c/d23/d47/f50 zero size 2024-02-16T22:21:51.929 INFO:tasks.workunit.client.1.smithi123.stdout:6/319: dread - d0/d9/dc/f3f zero size 2024-02-16T22:21:51.931 INFO:tasks.workunit.client.1.smithi123.stdout:5/256: symlink d6/dd/df/d1f/d25/d37/l59 0 2024-02-16T22:21:51.931 INFO:tasks.workunit.client.1.smithi123.stdout:6/320: stat d0/d9/dc/d3a/c3b 0 2024-02-16T22:21:51.931 INFO:tasks.workunit.client.1.smithi123.stdout:9/376: getdents d4/d56/d6f 0 2024-02-16T22:21:51.934 INFO:tasks.workunit.client.1.smithi123.stdout:9/377: truncate d4/d56/f61 500937 0 2024-02-16T22:21:51.934 INFO:tasks.workunit.client.1.smithi123.stdout:1/270: rename d3/d6/db/d10/c1c to d3/d6/db/d55/c63 0 2024-02-16T22:21:51.935 INFO:tasks.workunit.client.1.smithi123.stdout:2/284: dwrite d4/f52 [0,4194304] 0 2024-02-16T22:21:51.936 INFO:tasks.workunit.client.1.smithi123.stdout:5/257: truncate d6/dd/df/d58/f4f 837938 0 2024-02-16T22:21:51.937 INFO:tasks.workunit.client.1.smithi123.stdout:1/271: readlink d3/d6/db/l4c 0 2024-02-16T22:21:51.937 INFO:tasks.workunit.client.1.smithi123.stdout:7/306: link d3/d12/d1d/d1e/f5e d3/d12/d1d/d1e/f65 0 2024-02-16T22:21:51.938 INFO:tasks.workunit.client.1.smithi123.stdout:1/272: chown d3/d6/d49 82 1 2024-02-16T22:21:51.938 INFO:tasks.workunit.client.1.smithi123.stdout:2/285: dread - d4/d1a/d1d/d13/d15/f56 zero size 2024-02-16T22:21:51.939 INFO:tasks.workunit.client.1.smithi123.stdout:5/258: dread - d6/dd/d44/f47 zero size 2024-02-16T22:21:51.941 INFO:tasks.workunit.client.1.smithi123.stdout:5/259: dread - d6/f53 zero size 2024-02-16T22:21:51.941 INFO:tasks.workunit.client.1.smithi123.stdout:6/321: dwrite d0/d6/d2f/f22 [0,4194304] 0 2024-02-16T22:21:51.941 INFO:tasks.workunit.client.1.smithi123.stdout:5/260: chown d6/dd/df/d1f 3 1 2024-02-16T22:21:51.943 INFO:tasks.workunit.client.0.smithi114.stdout:5/162: sync 2024-02-16T22:21:51.944 INFO:tasks.workunit.client.1.smithi123.stdout:6/322: write d0/d6/d1a/f64 [711205,67301] 0 2024-02-16T22:21:51.946 INFO:tasks.workunit.client.1.smithi123.stdout:1/273: symlink d3/d6/db/d3b/l64 0 2024-02-16T22:21:51.948 INFO:tasks.workunit.client.0.smithi114.stdout:5/163: mkdir d0/df/d16/d25/d2e 0 2024-02-16T22:21:51.949 INFO:tasks.workunit.client.1.smithi123.stdout:2/286: dwrite d4/d1a/d1d/d13/d25/f4d [0,4194304] 0 2024-02-16T22:21:51.949 INFO:tasks.workunit.client.0.smithi114.stdout:5/164: readlink d0/l6 0 2024-02-16T22:21:51.950 INFO:tasks.workunit.client.0.smithi114.stdout:5/165: dread - d0/df/d16/d1e/f27 zero size 2024-02-16T22:21:51.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:51 smithi114 ceph-mon[29235]: pgmap v23: 65 pgs: 65 active+clean; 606 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 62 MiB/s wr, 182 op/s 2024-02-16T22:21:51.952 INFO:tasks.workunit.client.1.smithi123.stdout:7/307: rename d3/d12/d34/c42 to d3/d12/d4d/c66 0 2024-02-16T22:21:51.952 INFO:tasks.workunit.client.1.smithi123.stdout:7/308: chown d3/d26 69327662 1 2024-02-16T22:21:51.987 INFO:tasks.workunit.client.1.smithi123.stdout:7/309: dread d3/d12/d1d/f2b [0,4194304] 0 2024-02-16T22:21:51.992 INFO:tasks.workunit.client.1.smithi123.stdout:7/310: dwrite d3/d26/d3e/f5b [0,4194304] 0 2024-02-16T22:21:52.001 INFO:tasks.workunit.client.1.smithi123.stdout:8/361: sync 2024-02-16T22:21:52.021 INFO:tasks.workunit.client.1.smithi123.stdout:8/362: dread d0/d17/d43/f27 [0,4194304] 0 2024-02-16T22:21:52.039 INFO:tasks.workunit.client.1.smithi123.stdout:6/323: write d0/f60 [126877,93031] 0 2024-02-16T22:21:52.039 INFO:tasks.workunit.client.0.smithi114.stdout:5/166: mknod d0/df/d16/d25/d2e/c2f 0 2024-02-16T22:21:52.040 INFO:tasks.workunit.client.0.smithi114.stdout:5/167: stat d0/fb 0 2024-02-16T22:21:52.042 INFO:tasks.workunit.client.1.smithi123.stdout:0/269: write d1/d8/f34 [4571855,1923] 0 2024-02-16T22:21:52.043 INFO:tasks.workunit.client.1.smithi123.stdout:0/270: stat d1/d8/d13/d1d/l2b 0 2024-02-16T22:21:52.045 INFO:tasks.workunit.client.0.smithi114.stdout:5/168: mkdir d0/d2d/d30 0 2024-02-16T22:21:52.047 INFO:tasks.workunit.client.1.smithi123.stdout:3/266: dwrite d0/fe [0,4194304] 0 2024-02-16T22:21:52.050 INFO:tasks.workunit.client.1.smithi123.stdout:8/363: mkdir d0/d29/d42/d47/d7b 0 2024-02-16T22:21:52.050 INFO:tasks.workunit.client.1.smithi123.stdout:8/364: fsync d0/f6 0 2024-02-16T22:21:52.051 INFO:tasks.workunit.client.1.smithi123.stdout:4/328: getdents d5/d49 0 2024-02-16T22:21:52.052 INFO:tasks.workunit.client.0.smithi114.stdout:5/169: dwrite d0/df/f1d [0,4194304] 0 2024-02-16T22:21:52.052 INFO:tasks.workunit.client.1.smithi123.stdout:8/365: chown d0/d29/d42/d47/f73 459951940 1 2024-02-16T22:21:52.052 INFO:tasks.workunit.client.1.smithi123.stdout:4/329: rename d5/d1c to d5/d1c/d5e/d61 22 2024-02-16T22:21:52.052 INFO:tasks.workunit.client.1.smithi123.stdout:8/366: chown d0/d17/d26/d68 5 1 2024-02-16T22:21:52.053 INFO:tasks.workunit.client.1.smithi123.stdout:8/367: fdatasync d0/df/f1b 0 2024-02-16T22:21:52.053 INFO:tasks.workunit.client.0.smithi114.stdout:5/170: write d0/d10/f15 [698150,86333] 0 2024-02-16T22:21:52.056 INFO:tasks.workunit.client.1.smithi123.stdout:9/378: link d4/d17/d20/l2e d4/l79 0 2024-02-16T22:21:52.056 INFO:tasks.workunit.client.1.smithi123.stdout:9/379: stat d4/d17/d24/l68 0 2024-02-16T22:21:52.057 INFO:tasks.workunit.client.1.smithi123.stdout:5/261: dread d6/dd/df/d1f/d25/d37/f3e [0,4194304] 0 2024-02-16T22:21:52.058 INFO:tasks.workunit.client.1.smithi123.stdout:1/274: mkdir d3/d6/d65 0 2024-02-16T22:21:52.059 INFO:tasks.workunit.client.1.smithi123.stdout:0/271: fdatasync d1/f12 0 2024-02-16T22:21:52.060 INFO:tasks.workunit.client.0.smithi114.stdout:5/171: creat d0/d2d/d30/f31 x:0 0 0 2024-02-16T22:21:52.061 INFO:tasks.workunit.client.0.smithi114.stdout:5/172: chown d0/df/d16/d1e/f21 568 1 2024-02-16T22:21:52.062 INFO:tasks.workunit.client.0.smithi114.stdout:5/173: chown d0/fa 5648 1 2024-02-16T22:21:52.062 INFO:tasks.workunit.client.1.smithi123.stdout:7/311: unlink d3/d12/d24/c29 0 2024-02-16T22:21:52.065 INFO:tasks.workunit.client.0.smithi114.stdout:5/174: symlink d0/df/d16/d25/l32 0 2024-02-16T22:21:52.065 INFO:tasks.workunit.client.1.smithi123.stdout:9/380: dwrite d4/d17/f55 [0,4194304] 0 2024-02-16T22:21:52.066 INFO:tasks.workunit.client.1.smithi123.stdout:4/330: fdatasync d5/f29 0 2024-02-16T22:21:52.068 INFO:tasks.workunit.client.0.smithi114.stdout:5/175: stat d0/c1 0 2024-02-16T22:21:52.069 INFO:tasks.workunit.client.1.smithi123.stdout:9/381: write d4/d17/d2b/f64 [3612163,109351] 0 2024-02-16T22:21:52.070 INFO:tasks.workunit.client.0.smithi114.stdout:5/176: write d0/df/d16/d1e/f27 [118772,62831] 0 2024-02-16T22:21:52.070 INFO:tasks.workunit.client.1.smithi123.stdout:7/312: dwrite d3/f8 [0,4194304] 0 2024-02-16T22:21:52.072 INFO:tasks.workunit.client.1.smithi123.stdout:5/262: truncate d6/dd/df/f22 594736 0 2024-02-16T22:21:52.075 INFO:tasks.workunit.client.1.smithi123.stdout:1/275: creat d3/d6/d49/f66 x:0 0 0 2024-02-16T22:21:52.076 INFO:tasks.workunit.client.1.smithi123.stdout:0/272: mkdir d1/d4/d42/d4b 0 2024-02-16T22:21:52.079 INFO:tasks.workunit.client.1.smithi123.stdout:9/382: dwrite d4/f10 [0,4194304] 0 2024-02-16T22:21:52.079 INFO:tasks.workunit.client.1.smithi123.stdout:3/267: symlink d0/d20/d29/d34/l63 0 2024-02-16T22:21:52.082 INFO:tasks.workunit.client.0.smithi114.stdout:5/177: dread d0/fb [0,4194304] 0 2024-02-16T22:21:52.083 INFO:tasks.workunit.client.0.smithi114.stdout:5/178: stat d0/d10/f15 0 2024-02-16T22:21:52.084 INFO:tasks.workunit.client.0.smithi114.stdout:7/197: sync 2024-02-16T22:21:52.085 INFO:tasks.workunit.client.0.smithi114.stdout:7/198: write d3/fe [701887,66382] 0 2024-02-16T22:21:52.085 INFO:tasks.workunit.client.0.smithi114.stdout:5/179: rmdir d0/d2d 39 2024-02-16T22:21:52.087 INFO:tasks.workunit.client.1.smithi123.stdout:9/383: write d4/d17/d24/f69 [253614,40250] 0 2024-02-16T22:21:52.087 INFO:tasks.workunit.client.1.smithi123.stdout:4/331: rename d5/f12 to d5/d1c/d23/d47/f62 0 2024-02-16T22:21:52.087 INFO:tasks.workunit.client.1.smithi123.stdout:4/332: truncate d5/f59 4527652 0 2024-02-16T22:21:52.087 INFO:tasks.workunit.client.1.smithi123.stdout:8/368: dread d0/df/f1b [0,4194304] 0 2024-02-16T22:21:52.088 INFO:tasks.workunit.client.1.smithi123.stdout:7/313: creat d3/d12/d52/f67 x:0 0 0 2024-02-16T22:21:52.089 INFO:tasks.workunit.client.1.smithi123.stdout:0/273: dread d1/d8/f30 [0,4194304] 0 2024-02-16T22:21:52.089 INFO:tasks.workunit.client.1.smithi123.stdout:5/263: symlink d6/dd/df/d58/d3f/l5a 0 2024-02-16T22:21:52.090 INFO:tasks.workunit.client.0.smithi114.stdout:7/199: chown d3/l1d 14 1 2024-02-16T22:21:52.090 INFO:tasks.workunit.client.1.smithi123.stdout:1/276: dwrite d3/f35 [0,4194304] 0 2024-02-16T22:21:52.090 INFO:tasks.workunit.client.1.smithi123.stdout:2/287: getdents d4/d1a 0 2024-02-16T22:21:52.092 INFO:tasks.workunit.client.1.smithi123.stdout:8/369: write d0/d61/d66/f69 [1010821,78191] 0 2024-02-16T22:21:52.092 INFO:tasks.workunit.client.0.smithi114.stdout:5/180: mknod d0/d2d/c33 0 2024-02-16T22:21:52.093 INFO:tasks.workunit.client.1.smithi123.stdout:6/324: getdents d0/d6/d1a/d2e 0 2024-02-16T22:21:52.094 INFO:tasks.workunit.client.1.smithi123.stdout:8/370: chown d0/d17/d26/d2a/d3c/d72 6 1 2024-02-16T22:21:52.096 INFO:tasks.workunit.client.1.smithi123.stdout:7/314: mknod d3/d12/d24/c68 0 2024-02-16T22:21:52.097 INFO:tasks.workunit.client.0.smithi114.stdout:7/200: dwrite d3/d22/f34 [0,4194304] 0 2024-02-16T22:21:52.097 INFO:tasks.workunit.client.0.smithi114.stdout:5/181: fsync d0/fa 0 2024-02-16T22:21:52.099 INFO:tasks.workunit.client.0.smithi114.stdout:5/182: readlink d0/df/d16/d25/l32 0 2024-02-16T22:21:52.099 INFO:tasks.workunit.client.1.smithi123.stdout:2/288: mknod d4/d1a/d1d/c5b 0 2024-02-16T22:21:52.100 INFO:tasks.workunit.client.1.smithi123.stdout:0/274: symlink d1/d8/d13/l4c 0 2024-02-16T22:21:52.101 INFO:tasks.workunit.client.1.smithi123.stdout:1/277: dwrite d3/d6/db/f3a [0,4194304] 0 2024-02-16T22:21:52.101 INFO:tasks.workunit.client.1.smithi123.stdout:2/289: write d4/f4f [19638,46363] 0 2024-02-16T22:21:52.102 INFO:tasks.workunit.client.1.smithi123.stdout:4/333: dwrite d5/d1c/d23/d36/f5f [0,4194304] 0 2024-02-16T22:21:52.103 INFO:tasks.workunit.client.1.smithi123.stdout:4/334: chown d5/f1d 324956 1 2024-02-16T22:21:52.107 INFO:tasks.workunit.client.0.smithi114.stdout:5/183: truncate d0/d10/f1a 4761830 0 2024-02-16T22:21:52.107 INFO:tasks.workunit.client.0.smithi114.stdout:7/201: dwrite d3/fc [0,4194304] 0 2024-02-16T22:21:52.110 INFO:tasks.workunit.client.1.smithi123.stdout:6/325: rename d0/d9/dc/d39/l43 to d0/d9/dc/d3a/d52/l7a 0 2024-02-16T22:21:52.110 INFO:tasks.workunit.client.1.smithi123.stdout:6/326: readlink d0/d6/d2f/l1c 0 2024-02-16T22:21:52.111 INFO:tasks.workunit.client.1.smithi123.stdout:5/264: fdatasync d6/f8 0 2024-02-16T22:21:52.113 INFO:tasks.workunit.client.1.smithi123.stdout:5/265: write d6/dd/df/d1f/d25/f3a [5952661,21924] 0 2024-02-16T22:21:52.114 INFO:tasks.workunit.client.1.smithi123.stdout:5/266: chown d6/dd/df/d58/f43 0 1 2024-02-16T22:21:52.114 INFO:tasks.workunit.client.1.smithi123.stdout:0/275: unlink d1/d8/c3a 0 2024-02-16T22:21:52.116 INFO:tasks.workunit.client.0.smithi114.stdout:7/202: symlink d3/d3d/l3f 0 2024-02-16T22:21:52.121 INFO:tasks.workunit.client.1.smithi123.stdout:7/315: mknod d3/d59/c69 0 2024-02-16T22:21:52.123 INFO:tasks.workunit.client.0.smithi114.stdout:7/203: dwrite d3/d14/f18 [0,4194304] 0 2024-02-16T22:21:52.123 INFO:tasks.workunit.client.1.smithi123.stdout:6/327: dread d0/d9/f62 [0,4194304] 0 2024-02-16T22:21:52.124 INFO:tasks.workunit.client.1.smithi123.stdout:2/290: creat d4/d1a/d1d/d13/d15/d1b/d4a/f5c x:0 0 0 2024-02-16T22:21:52.126 INFO:tasks.workunit.client.0.smithi114.stdout:7/204: symlink d3/d23/l40 0 2024-02-16T22:21:52.127 INFO:tasks.workunit.client.1.smithi123.stdout:5/267: getdents d6/dd/df/d58/d56 0 2024-02-16T22:21:52.130 INFO:tasks.workunit.client.1.smithi123.stdout:1/278: getdents d3/d6/db/d10/d2d 0 2024-02-16T22:21:52.132 INFO:tasks.workunit.client.1.smithi123.stdout:0/276: getdents d1/d4/d17 0 2024-02-16T22:21:52.133 INFO:tasks.workunit.client.1.smithi123.stdout:0/277: chown d1/d4/l21 56368 1 2024-02-16T22:21:52.134 INFO:tasks.workunit.client.1.smithi123.stdout:5/268: creat d6/dd/f5b x:0 0 0 2024-02-16T22:21:52.135 INFO:tasks.workunit.client.1.smithi123.stdout:6/328: read d0/f54 [734368,114357] 0 2024-02-16T22:21:52.135 INFO:tasks.workunit.client.1.smithi123.stdout:5/269: chown d6/dd/df/d58/d56 2146998 1 2024-02-16T22:21:52.136 INFO:tasks.workunit.client.1.smithi123.stdout:0/278: creat d1/d8/d13/f4d x:0 0 0 2024-02-16T22:21:52.136 INFO:tasks.workunit.client.1.smithi123.stdout:5/270: write d6/f53 [222531,100570] 0 2024-02-16T22:21:52.138 INFO:tasks.workunit.client.1.smithi123.stdout:0/279: mknod d1/d4/d42/d4b/c4e 0 2024-02-16T22:21:52.139 INFO:tasks.workunit.client.1.smithi123.stdout:0/280: mknod d1/d4/c4f 0 2024-02-16T22:21:52.142 INFO:tasks.workunit.client.1.smithi123.stdout:5/271: dwrite d6/f53 [0,4194304] 0 2024-02-16T22:21:52.143 INFO:tasks.workunit.client.1.smithi123.stdout:5/272: read d6/dd/df/d1f/f2e [16485,18831] 0 2024-02-16T22:21:52.145 INFO:tasks.workunit.client.1.smithi123.stdout:5/273: write d6/f21 [2159324,50562] 0 2024-02-16T22:21:52.146 INFO:tasks.workunit.client.1.smithi123.stdout:0/281: dwrite d1/d8/fa [0,4194304] 0 2024-02-16T22:21:52.150 INFO:tasks.workunit.client.1.smithi123.stdout:5/274: rename d6/dd/df/d1f/f24 to d6/dd/df/d1f/f5c 0 2024-02-16T22:21:52.154 INFO:tasks.workunit.client.1.smithi123.stdout:5/275: creat d6/dd/df/d1f/d25/f5d x:0 0 0 2024-02-16T22:21:52.154 INFO:tasks.workunit.client.1.smithi123.stdout:0/282: dwrite d1/d8/d13/f4d [0,4194304] 0 2024-02-16T22:21:52.157 INFO:tasks.workunit.client.1.smithi123.stdout:5/276: creat d6/dd/df/f5e x:0 0 0 2024-02-16T22:21:52.158 INFO:tasks.workunit.client.1.smithi123.stdout:0/283: creat d1/d4/d42/d4b/f50 x:0 0 0 2024-02-16T22:21:52.159 INFO:tasks.workunit.client.1.smithi123.stdout:5/277: write d6/dd/df/d1f/d25/f5d [733683,43866] 0 2024-02-16T22:21:52.162 INFO:tasks.workunit.client.1.smithi123.stdout:5/278: mkdir d6/dd/d5f 0 2024-02-16T22:21:52.163 INFO:tasks.workunit.client.1.smithi123.stdout:5/279: readlink d6/dd/df/l46 0 2024-02-16T22:21:52.165 INFO:tasks.workunit.client.1.smithi123.stdout:5/280: creat d6/dd/d5f/f60 x:0 0 0 2024-02-16T22:21:52.183 INFO:tasks.workunit.client.1.smithi123.stdout:9/384: sync 2024-02-16T22:21:52.185 INFO:tasks.workunit.client.1.smithi123.stdout:9/385: write d4/f16 [1486704,100310] 0 2024-02-16T22:21:52.189 INFO:tasks.workunit.client.1.smithi123.stdout:9/386: dread d4/d1e/d3a/f40 [0,4194304] 0 2024-02-16T22:21:52.191 INFO:tasks.workunit.client.1.smithi123.stdout:9/387: write d4/d17/d24/f28 [1675738,110789] 0 2024-02-16T22:21:52.195 INFO:tasks.workunit.client.1.smithi123.stdout:9/388: link d4/d1e/d3a/c4e d4/d17/d20/c7a 0 2024-02-16T22:21:52.195 INFO:tasks.workunit.client.1.smithi123.stdout:9/389: dread - d4/d56/d6f/f77 zero size 2024-02-16T22:21:52.196 INFO:tasks.workunit.client.1.smithi123.stdout:9/390: write d4/d33/f73 [579921,98541] 0 2024-02-16T22:21:52.201 INFO:tasks.workunit.client.1.smithi123.stdout:9/391: rename d4/d56/d58/d59/d60/f63 to d4/d17/d20/f7b 0 2024-02-16T22:21:52.201 INFO:tasks.workunit.client.1.smithi123.stdout:3/268: write d0/d20/d29/d34/f36 [2142564,52665] 0 2024-02-16T22:21:52.202 INFO:tasks.workunit.client.1.smithi123.stdout:7/316: getdents d3/d12/d24 0 2024-02-16T22:21:52.204 INFO:tasks.workunit.client.1.smithi123.stdout:3/269: rename d0/d20/f28 to d0/d20/d29/d34/d37/f64 0 2024-02-16T22:21:52.206 INFO:tasks.workunit.client.1.smithi123.stdout:8/371: write d0/fd [2445935,118145] 0 2024-02-16T22:21:52.208 INFO:tasks.workunit.client.1.smithi123.stdout:7/317: dwrite d3/d12/d52/f67 [0,4194304] 0 2024-02-16T22:21:52.209 INFO:tasks.workunit.client.1.smithi123.stdout:7/318: write d3/d12/d1d/d1e/f5f [197904,66785] 0 2024-02-16T22:21:52.210 INFO:tasks.workunit.client.1.smithi123.stdout:3/270: link d0/c32 d0/d20/d2e/c65 0 2024-02-16T22:21:52.212 INFO:tasks.workunit.client.1.smithi123.stdout:9/392: fdatasync d4/d17/d24/f28 0 2024-02-16T22:21:52.213 INFO:tasks.workunit.client.0.smithi114.stdout:5/184: truncate d0/fb 1700549 0 2024-02-16T22:21:52.214 INFO:tasks.workunit.client.1.smithi123.stdout:7/319: creat d3/d12/d52/f6a x:0 0 0 2024-02-16T22:21:52.214 INFO:tasks.workunit.client.1.smithi123.stdout:4/335: write d5/d1c/d23/f2a [869965,33379] 0 2024-02-16T22:21:52.215 INFO:tasks.workunit.client.1.smithi123.stdout:8/372: dwrite d0/f6 [0,4194304] 0 2024-02-16T22:21:52.215 INFO:tasks.workunit.client.1.smithi123.stdout:3/271: unlink d0/d20/f21 0 2024-02-16T22:21:52.221 INFO:tasks.workunit.client.1.smithi123.stdout:2/291: rmdir d4/d1a/d1d 39 2024-02-16T22:21:52.223 INFO:tasks.workunit.client.1.smithi123.stdout:1/279: write d3/d6/f5b [4009445,115501] 0 2024-02-16T22:21:52.224 INFO:tasks.workunit.client.1.smithi123.stdout:3/272: symlink d0/d20/d29/d34/d37/l66 0 2024-02-16T22:21:52.224 INFO:tasks.workunit.client.1.smithi123.stdout:1/280: chown d3/d6/db/d10/c41 15955314 1 2024-02-16T22:21:52.226 INFO:tasks.workunit.client.1.smithi123.stdout:1/281: write d3/d6/db/f3a [1335771,122439] 0 2024-02-16T22:21:52.226 INFO:tasks.workunit.client.1.smithi123.stdout:6/329: write d0/d6/fa [2250098,95633] 0 2024-02-16T22:21:52.228 INFO:tasks.workunit.client.1.smithi123.stdout:6/330: chown d0/d6/l1d 1 1 2024-02-16T22:21:52.229 INFO:tasks.workunit.client.1.smithi123.stdout:3/273: write d0/d20/d29/d34/d37/f5e [378104,14111] 0 2024-02-16T22:21:52.229 INFO:tasks.workunit.client.1.smithi123.stdout:6/331: write d0/d9/f19 [4883166,14960] 0 2024-02-16T22:21:52.231 INFO:tasks.workunit.client.1.smithi123.stdout:3/274: truncate d0/d20/f42 645240 0 2024-02-16T22:21:52.231 INFO:tasks.workunit.client.1.smithi123.stdout:2/292: fdatasync d4/d1a/d1d/f9 0 2024-02-16T22:21:52.232 INFO:tasks.workunit.client.1.smithi123.stdout:2/293: stat d4/d1a/d1d/d13/d25/l4c 0 2024-02-16T22:21:52.233 INFO:tasks.workunit.client.1.smithi123.stdout:2/294: dread - d4/d1a/d1d/d13/f3c zero size 2024-02-16T22:21:52.237 INFO:tasks.workunit.client.1.smithi123.stdout:0/284: write d1/d4/d17/f28 [633416,88586] 0 2024-02-16T22:21:52.237 INFO:tasks.workunit.client.1.smithi123.stdout:8/373: dread d0/f75 [0,4194304] 0 2024-02-16T22:21:52.238 INFO:tasks.workunit.client.1.smithi123.stdout:7/320: rename d3/d12/d4d/l53 to d3/d12/l6b 0 2024-02-16T22:21:52.239 INFO:tasks.workunit.client.1.smithi123.stdout:5/281: truncate d6/dd/df/d1f/f5c 5689451 0 2024-02-16T22:21:52.239 INFO:tasks.workunit.client.1.smithi123.stdout:7/321: stat d3/d5/l19 0 2024-02-16T22:21:52.239 INFO:tasks.workunit.client.1.smithi123.stdout:5/282: dread - d6/dd/df/f5e zero size 2024-02-16T22:21:52.240 INFO:tasks.workunit.client.1.smithi123.stdout:7/322: write d3/d12/d24/f5d [384278,115635] 0 2024-02-16T22:21:52.241 INFO:tasks.workunit.client.1.smithi123.stdout:5/283: fsync d6/dd/df/d1f/d25/f1a 0 2024-02-16T22:21:52.241 INFO:tasks.workunit.client.1.smithi123.stdout:7/323: write d3/d12/d4d/f50 [2149310,529] 0 2024-02-16T22:21:52.243 INFO:tasks.workunit.client.1.smithi123.stdout:8/374: read d0/d12/f19 [2715926,44308] 0 2024-02-16T22:21:52.244 INFO:tasks.workunit.client.1.smithi123.stdout:8/375: chown d0/d17/d26/d32/f56 91 1 2024-02-16T22:21:52.246 INFO:tasks.workunit.client.1.smithi123.stdout:3/275: rename d0/d20/d29/f44 to d0/d20/d29/d34/d37/d49/f67 0 2024-02-16T22:21:52.246 INFO:tasks.workunit.client.1.smithi123.stdout:3/276: fdatasync d0/f15 0 2024-02-16T22:21:52.250 INFO:tasks.workunit.client.1.smithi123.stdout:6/332: creat d0/d74/f7b x:0 0 0 2024-02-16T22:21:52.250 INFO:tasks.workunit.client.1.smithi123.stdout:6/333: stat d0/d74/f7b 0 2024-02-16T22:21:52.253 INFO:tasks.workunit.client.0.smithi114.stdout:5/185: dread d0/fb [0,4194304] 0 2024-02-16T22:21:52.253 INFO:tasks.workunit.client.1.smithi123.stdout:5/284: creat d6/dd/d44/f61 x:0 0 0 2024-02-16T22:21:52.253 INFO:tasks.workunit.client.0.smithi114.stdout:5/186: chown d0 3 1 2024-02-16T22:21:52.254 INFO:tasks.workunit.client.1.smithi123.stdout:5/285: chown d6/dd/c30 60349139 1 2024-02-16T22:21:52.254 INFO:tasks.workunit.client.1.smithi123.stdout:5/286: dread d6/dd/df/d1f/f2e [0,4194304] 0 2024-02-16T22:21:52.255 INFO:tasks.workunit.client.1.smithi123.stdout:7/324: mkdir d3/d5/d6c 0 2024-02-16T22:21:52.256 INFO:tasks.workunit.client.1.smithi123.stdout:9/393: write d4/d17/d2b/f5b [893338,64718] 0 2024-02-16T22:21:52.257 INFO:tasks.workunit.client.1.smithi123.stdout:9/394: fdatasync d4/d17/f1f 0 2024-02-16T22:21:52.258 INFO:tasks.workunit.client.1.smithi123.stdout:8/376: chown d0/df/c24 607033 1 2024-02-16T22:21:52.258 INFO:tasks.workunit.client.0.smithi114.stdout:5/187: symlink d0/d28/l34 0 2024-02-16T22:21:52.260 INFO:tasks.workunit.client.1.smithi123.stdout:0/285: rename d1/d4/d17/f32 to d1/d4/f51 0 2024-02-16T22:21:52.261 INFO:tasks.workunit.client.1.smithi123.stdout:3/277: creat d0/d20/f68 x:0 0 0 2024-02-16T22:21:52.261 INFO:tasks.workunit.client.0.smithi114.stdout:5/188: rmdir d0/df/d16/d25 39 2024-02-16T22:21:52.261 INFO:tasks.workunit.client.1.smithi123.stdout:4/336: write d5/d1c/d23/d47/f62 [1263048,102754] 0 2024-02-16T22:21:52.262 INFO:tasks.workunit.client.1.smithi123.stdout:1/282: link d3/d6/db/l1a d3/d6/db/l67 0 2024-02-16T22:21:52.264 INFO:tasks.workunit.client.1.smithi123.stdout:8/377: dwrite d0/d17/d26/d32/d45/f5f [0,4194304] 0 2024-02-16T22:21:52.265 INFO:tasks.workunit.client.1.smithi123.stdout:5/287: creat d6/dd/df/f62 x:0 0 0 2024-02-16T22:21:52.266 INFO:tasks.workunit.client.1.smithi123.stdout:5/288: fsync d6/dd/df/f28 0 2024-02-16T22:21:52.268 INFO:tasks.workunit.client.1.smithi123.stdout:7/325: creat d3/d12/d24/f6d x:0 0 0 2024-02-16T22:21:52.268 INFO:tasks.workunit.client.1.smithi123.stdout:2/295: write d4/d1a/d1d/f2b [3383807,113743] 0 2024-02-16T22:21:52.271 INFO:tasks.workunit.client.1.smithi123.stdout:9/395: stat d4/d36/f78 0 2024-02-16T22:21:52.272 INFO:tasks.workunit.client.1.smithi123.stdout:9/396: chown d4/d17/d2b/f53 1493625 1 2024-02-16T22:21:52.274 INFO:tasks.workunit.client.1.smithi123.stdout:0/286: creat d1/d4/d42/d4b/f52 x:0 0 0 2024-02-16T22:21:52.274 INFO:tasks.workunit.client.1.smithi123.stdout:0/287: dread - d1/d4/d42/f4a zero size 2024-02-16T22:21:52.276 INFO:tasks.workunit.client.1.smithi123.stdout:4/337: creat d5/f63 x:0 0 0 2024-02-16T22:21:52.276 INFO:tasks.workunit.client.1.smithi123.stdout:6/334: getdents d0/d6/d1a/d2e 0 2024-02-16T22:21:52.277 INFO:tasks.workunit.client.1.smithi123.stdout:1/283: readlink d3/d6/db/l67 0 2024-02-16T22:21:52.278 INFO:tasks.workunit.client.1.smithi123.stdout:8/378: rmdir d0/d17/d26/d32/d45/d67 39 2024-02-16T22:21:52.280 INFO:tasks.workunit.client.1.smithi123.stdout:5/289: creat d6/dd/df/d58/f63 x:0 0 0 2024-02-16T22:21:52.282 INFO:tasks.workunit.client.1.smithi123.stdout:5/290: write d6/dd/df/d58/f43 [933674,129211] 0 2024-02-16T22:21:52.282 INFO:tasks.workunit.client.1.smithi123.stdout:5/291: fsync d6/dd/df/f5e 0 2024-02-16T22:21:52.283 INFO:tasks.workunit.client.1.smithi123.stdout:2/296: symlink d4/d1a/d1d/d13/d15/l5d 0 2024-02-16T22:21:52.285 INFO:tasks.workunit.client.1.smithi123.stdout:9/397: rename d4/d17/d20/l2a to d4/d17/d20/l7c 0 2024-02-16T22:21:52.287 INFO:tasks.workunit.client.1.smithi123.stdout:9/398: truncate d4/d17/d20/f3c 4383222 0 2024-02-16T22:21:52.287 INFO:tasks.workunit.client.1.smithi123.stdout:9/399: chown d4/cf 44 1 2024-02-16T22:21:52.287 INFO:tasks.workunit.client.0.smithi114.stdout:5/189: dwrite d0/fa [0,4194304] 0 2024-02-16T22:21:52.289 INFO:tasks.workunit.client.1.smithi123.stdout:3/278: creat d0/d20/d29/d34/d37/d41/f69 x:0 0 0 2024-02-16T22:21:52.289 INFO:tasks.workunit.client.1.smithi123.stdout:6/335: symlink d0/d6/d4d/l7c 0 2024-02-16T22:21:52.290 INFO:tasks.workunit.client.1.smithi123.stdout:6/336: dread - d0/d6/d1a/d2e/f3c zero size 2024-02-16T22:21:52.292 INFO:tasks.workunit.client.1.smithi123.stdout:8/379: mkdir d0/d12/d3e/d7c 0 2024-02-16T22:21:52.295 INFO:tasks.workunit.client.1.smithi123.stdout:0/288: unlink d1/f2c 0 2024-02-16T22:21:52.296 INFO:tasks.workunit.client.1.smithi123.stdout:4/338: mkdir d5/d1c/d64 0 2024-02-16T22:21:52.297 INFO:tasks.workunit.client.1.smithi123.stdout:9/400: write d4/d1e/d3a/f40 [1712791,118837] 0 2024-02-16T22:21:52.298 INFO:tasks.workunit.client.1.smithi123.stdout:9/401: write d4/d17/d24/f28 [1631068,99893] 0 2024-02-16T22:21:52.299 INFO:tasks.workunit.client.1.smithi123.stdout:1/284: mkdir d3/d6/d49/d48/d68 0 2024-02-16T22:21:52.300 INFO:tasks.workunit.client.0.smithi114.stdout:5/190: write d0/d10/f23 [542602,84970] 0 2024-02-16T22:21:52.301 INFO:tasks.workunit.client.1.smithi123.stdout:6/337: creat d0/d9/dc/d3a/f7d x:0 0 0 2024-02-16T22:21:52.301 INFO:tasks.workunit.client.0.smithi114.stdout:5/191: dread - d0/d2d/d30/f31 zero size 2024-02-16T22:21:52.303 INFO:tasks.workunit.client.1.smithi123.stdout:6/338: write d0/d9/f19 [4180579,13976] 0 2024-02-16T22:21:52.304 INFO:tasks.workunit.client.1.smithi123.stdout:7/326: rename d3/d12 to d3/d6e 0 2024-02-16T22:21:52.304 INFO:tasks.workunit.client.1.smithi123.stdout:8/380: mknod d0/d17/d26/d68/c7d 0 2024-02-16T22:21:52.307 INFO:tasks.workunit.client.1.smithi123.stdout:9/402: dwrite d4/d56/d58/f5a [0,4194304] 0 2024-02-16T22:21:52.310 INFO:tasks.workunit.client.1.smithi123.stdout:0/289: creat d1/d4/d17/f53 x:0 0 0 2024-02-16T22:21:52.313 INFO:tasks.workunit.client.1.smithi123.stdout:6/339: dwrite d0/d9/f6a [0,4194304] 0 2024-02-16T22:21:52.313 INFO:tasks.workunit.client.1.smithi123.stdout:3/279: link d0/f15 d0/d20/d29/d34/d37/d49/f6a 0 2024-02-16T22:21:52.315 INFO:tasks.workunit.client.0.smithi114.stdout:5/192: write d0/fb [2289855,114117] 0 2024-02-16T22:21:52.320 INFO:tasks.workunit.client.1.smithi123.stdout:7/327: chown d3/d6e/d1d/f22 0 1 2024-02-16T22:21:52.321 INFO:tasks.workunit.client.1.smithi123.stdout:5/292: creat d6/dd/df/d1f/f64 x:0 0 0 2024-02-16T22:21:52.321 INFO:tasks.workunit.client.1.smithi123.stdout:7/328: chown d3/f4e 27904 1 2024-02-16T22:21:52.322 INFO:tasks.workunit.client.0.smithi114.stdout:5/193: dread d0/d10/f15 [0,4194304] 0 2024-02-16T22:21:52.323 INFO:tasks.workunit.client.1.smithi123.stdout:6/340: dwrite d0/d6/d2f/f22 [0,4194304] 0 2024-02-16T22:21:52.323 INFO:tasks.workunit.client.1.smithi123.stdout:6/341: readlink d0/d6/d2f/l33 0 2024-02-16T22:21:52.324 INFO:tasks.workunit.client.0.smithi114.stdout:5/194: write d0/df/f2b [1006182,121344] 0 2024-02-16T22:21:52.324 INFO:tasks.workunit.client.1.smithi123.stdout:4/339: mknod d5/d1c/d5e/c65 0 2024-02-16T22:21:52.326 INFO:tasks.workunit.client.1.smithi123.stdout:0/290: mkdir d1/d4/d17/d54 0 2024-02-16T22:21:52.328 INFO:tasks.workunit.client.1.smithi123.stdout:3/280: rmdir d0/d20/d29 39 2024-02-16T22:21:52.331 INFO:tasks.workunit.client.0.smithi114.stdout:2/215: sync 2024-02-16T22:21:52.331 INFO:tasks.workunit.client.0.smithi114.stdout:8/229: sync 2024-02-16T22:21:52.332 INFO:tasks.workunit.client.1.smithi123.stdout:1/285: rename d3/d6/d18/c4d to d3/d6/db/d55/c69 0 2024-02-16T22:21:52.332 INFO:tasks.workunit.client.0.smithi114.stdout:8/230: readlink d4/l5 0 2024-02-16T22:21:52.333 INFO:tasks.workunit.client.0.smithi114.stdout:0/233: sync 2024-02-16T22:21:52.336 INFO:tasks.workunit.client.1.smithi123.stdout:5/293: unlink d6/l26 0 2024-02-16T22:21:52.339 INFO:tasks.workunit.client.0.smithi114.stdout:8/231: creat d4/d24/d2c/f4c x:0 0 0 2024-02-16T22:21:52.339 INFO:tasks.workunit.client.0.smithi114.stdout:2/216: read f0 [3873434,51407] 0 2024-02-16T22:21:52.340 INFO:tasks.workunit.client.1.smithi123.stdout:6/342: creat d0/d6/d2f/d5d/f7e x:0 0 0 2024-02-16T22:21:52.340 INFO:tasks.workunit.client.0.smithi114.stdout:0/234: dwrite d1/d1a/f45 [0,4194304] 0 2024-02-16T22:21:52.341 INFO:tasks.workunit.client.1.smithi123.stdout:9/403: dread d4/d33/f38 [0,4194304] 0 2024-02-16T22:21:52.342 INFO:tasks.workunit.client.1.smithi123.stdout:4/340: rmdir d5/d1c/d23/d31 39 2024-02-16T22:21:52.343 INFO:tasks.workunit.client.0.smithi114.stdout:0/235: stat d1/d16/d18/c49 0 2024-02-16T22:21:52.348 INFO:tasks.workunit.client.1.smithi123.stdout:8/381: rename d0/d29/d42/d47/d7b to d0/d61/d66/d7e 0 2024-02-16T22:21:52.350 INFO:tasks.workunit.client.0.smithi114.stdout:0/236: dwrite d1/d16/d1f/d28/f35 [0,4194304] 0 2024-02-16T22:21:52.353 INFO:tasks.workunit.client.0.smithi114.stdout:5/195: getdents d0/df/d16/d25/d2e 0 2024-02-16T22:21:52.357 INFO:tasks.workunit.client.0.smithi114.stdout:8/232: creat d4/d11/d15/d31/f4d x:0 0 0 2024-02-16T22:21:52.358 INFO:tasks.workunit.client.0.smithi114.stdout:0/237: dwrite d1/f2a [0,4194304] 0 2024-02-16T22:21:52.359 INFO:tasks.workunit.client.0.smithi114.stdout:8/233: write d4/d24/f3e [1046078,125310] 0 2024-02-16T22:21:52.359 INFO:tasks.workunit.client.1.smithi123.stdout:6/343: read d0/d6/d1a/f2b [565407,113043] 0 2024-02-16T22:21:52.359 INFO:tasks.workunit.client.0.smithi114.stdout:8/234: chown d4/d16/d25 421997 1 2024-02-16T22:21:52.362 INFO:tasks.workunit.client.1.smithi123.stdout:4/341: fsync d5/d1c/d23/f3c 0 2024-02-16T22:21:52.363 INFO:tasks.workunit.client.0.smithi114.stdout:7/205: sync 2024-02-16T22:21:52.363 INFO:tasks.workunit.client.0.smithi114.stdout:6/212: sync 2024-02-16T22:21:52.363 INFO:tasks.workunit.client.0.smithi114.stdout:9/241: sync 2024-02-16T22:21:52.363 INFO:tasks.workunit.client.0.smithi114.stdout:3/198: sync 2024-02-16T22:21:52.363 INFO:tasks.workunit.client.0.smithi114.stdout:4/305: sync 2024-02-16T22:21:52.363 INFO:tasks.workunit.client.0.smithi114.stdout:1/189: sync 2024-02-16T22:21:52.364 INFO:tasks.workunit.client.0.smithi114.stdout:5/196: mknod d0/df/d16/d25/d2e/c35 0 2024-02-16T22:21:52.365 INFO:tasks.workunit.client.0.smithi114.stdout:9/242: fsync d2/d1a/f49 0 2024-02-16T22:21:52.366 INFO:tasks.workunit.client.1.smithi123.stdout:9/404: dwrite d4/d17/f1a [0,4194304] 0 2024-02-16T22:21:52.367 INFO:tasks.workunit.client.0.smithi114.stdout:7/206: dread d3/f10 [0,4194304] 0 2024-02-16T22:21:52.368 INFO:tasks.workunit.client.0.smithi114.stdout:5/197: write d0/d10/f13 [4005729,42384] 0 2024-02-16T22:21:52.368 INFO:tasks.workunit.client.1.smithi123.stdout:7/329: write d3/d5/f28 [403837,72841] 0 2024-02-16T22:21:52.371 INFO:tasks.workunit.client.0.smithi114.stdout:8/235: dwrite d4/d24/f3d [0,4194304] 0 2024-02-16T22:21:52.378 INFO:tasks.workunit.client.1.smithi123.stdout:2/297: sync 2024-02-16T22:21:52.379 INFO:tasks.workunit.client.0.smithi114.stdout:3/199: unlink d4/d26/f40 0 2024-02-16T22:21:52.380 INFO:tasks.workunit.client.0.smithi114.stdout:3/200: read d4/dd/f1e [996584,38458] 0 2024-02-16T22:21:52.381 INFO:tasks.workunit.client.1.smithi123.stdout:0/291: truncate d1/d8/d13/f14 1645176 0 2024-02-16T22:21:52.383 INFO:tasks.workunit.client.0.smithi114.stdout:4/306: mknod dd/d59/c6e 0 2024-02-16T22:21:52.386 INFO:tasks.workunit.client.1.smithi123.stdout:6/344: rename d0/d6/fa to d0/d6/d1a/d2e/f7f 0 2024-02-16T22:21:52.386 INFO:tasks.workunit.client.0.smithi114.stdout:9/243: creat d2/dd/d3c/f4f x:0 0 0 2024-02-16T22:21:52.387 INFO:tasks.workunit.client.1.smithi123.stdout:6/345: truncate d0/d6/d1a/d2e/f7f 2481628 0 2024-02-16T22:21:52.387 INFO:tasks.workunit.client.0.smithi114.stdout:0/238: symlink d1/d16/l4c 0 2024-02-16T22:21:52.388 INFO:tasks.workunit.client.1.smithi123.stdout:9/405: stat d4/d17/d20/l7c 0 2024-02-16T22:21:52.388 INFO:tasks.workunit.client.0.smithi114.stdout:0/239: write d1/d1a/d3e/d41/f42 [234787,49925] 0 2024-02-16T22:21:52.389 INFO:tasks.workunit.client.1.smithi123.stdout:9/406: truncate d4/d1e/d3a/f50 2298365 0 2024-02-16T22:21:52.390 INFO:tasks.workunit.client.1.smithi123.stdout:3/281: symlink d0/d20/d29/d34/l6b 0 2024-02-16T22:21:52.392 INFO:tasks.workunit.client.0.smithi114.stdout:8/236: symlink d4/d11/d15/d36/d3b/l4e 0 2024-02-16T22:21:52.394 INFO:tasks.workunit.client.0.smithi114.stdout:8/237: chown d4/d24/d2c/f4c 16624 1 2024-02-16T22:21:52.394 INFO:tasks.workunit.client.1.smithi123.stdout:2/298: mkdir d4/d1a/d1d/d38/d5e 0 2024-02-16T22:21:52.395 INFO:tasks.workunit.client.1.smithi123.stdout:3/282: dwrite d0/ff [0,4194304] 0 2024-02-16T22:21:52.400 INFO:tasks.workunit.client.0.smithi114.stdout:6/213: link d6/dd/d25/d26/f3d d6/d2c/f46 0 2024-02-16T22:21:52.400 INFO:tasks.workunit.client.1.smithi123.stdout:1/286: write d3/d6/d18/f21 [3595518,79256] 0 2024-02-16T22:21:52.403 INFO:tasks.workunit.client.1.smithi123.stdout:5/294: write d6/dd/f35 [1316240,98943] 0 2024-02-16T22:21:52.404 INFO:tasks.workunit.client.0.smithi114.stdout:9/244: chown d2/d1a/d23/c2d 31549957 1 2024-02-16T22:21:52.404 INFO:tasks.workunit.client.1.smithi123.stdout:5/295: read - d6/dd/df/f41 zero size 2024-02-16T22:21:52.405 INFO:tasks.workunit.client.1.smithi123.stdout:5/296: dread - d6/f4c zero size 2024-02-16T22:21:52.405 INFO:tasks.workunit.client.1.smithi123.stdout:6/346: unlink d0/d9/f51 0 2024-02-16T22:21:52.408 INFO:tasks.workunit.client.1.smithi123.stdout:5/297: dread d6/dd/df/f28 [0,4194304] 0 2024-02-16T22:21:52.410 INFO:tasks.workunit.client.0.smithi114.stdout:4/307: dread dd/ff [0,4194304] 0 2024-02-16T22:21:52.411 INFO:tasks.workunit.client.0.smithi114.stdout:4/308: dread - dd/d59/f5c zero size 2024-02-16T22:21:52.412 INFO:tasks.workunit.client.0.smithi114.stdout:2/217: getdents dd/df/d1c 0 2024-02-16T22:21:52.412 INFO:tasks.workunit.client.1.smithi123.stdout:7/330: truncate d3/d5/ff 2685864 0 2024-02-16T22:21:52.414 INFO:tasks.workunit.client.1.smithi123.stdout:7/331: chown d3/d6e/d1d/f41 62375939 1 2024-02-16T22:21:52.414 INFO:tasks.workunit.client.1.smithi123.stdout:5/298: dwrite d6/dd/df/d1f/d25/f5d [0,4194304] 0 2024-02-16T22:21:52.414 INFO:tasks.workunit.client.0.smithi114.stdout:8/238: dread d4/fe [0,4194304] 0 2024-02-16T22:21:52.416 INFO:tasks.workunit.client.1.smithi123.stdout:9/407: rename d4/f6 to d4/d36/d57/f7d 0 2024-02-16T22:21:52.419 INFO:tasks.workunit.client.0.smithi114.stdout:0/240: rename d1/d16/f17 to d1/d16/d1f/f4d 0 2024-02-16T22:21:52.420 INFO:tasks.workunit.client.0.smithi114.stdout:8/239: dread d4/d11/d15/f1a [0,4194304] 0 2024-02-16T22:21:52.423 INFO:tasks.workunit.client.0.smithi114.stdout:7/207: link d3/f27 d3/d3d/f41 0 2024-02-16T22:21:52.424 INFO:tasks.workunit.client.1.smithi123.stdout:8/382: dwrite d0/df/f16 [0,4194304] 0 2024-02-16T22:21:52.424 INFO:tasks.workunit.client.0.smithi114.stdout:1/190: write d4/f2d [327190,47066] 0 2024-02-16T22:21:52.427 INFO:tasks.workunit.client.1.smithi123.stdout:0/292: mknod d1/d4/d17/d54/c55 0 2024-02-16T22:21:52.427 INFO:tasks.workunit.client.1.smithi123.stdout:0/293: truncate d1/f2e 789042 0 2024-02-16T22:21:52.427 INFO:tasks.workunit.client.0.smithi114.stdout:3/201: mknod d4/d24/d2d/d45/c49 0 2024-02-16T22:21:52.432 INFO:tasks.workunit.client.0.smithi114.stdout:4/309: dread dd/d1b/d24/f4e [0,4194304] 0 2024-02-16T22:21:52.432 INFO:tasks.workunit.client.1.smithi123.stdout:0/294: dwrite d1/d8/fa [0,4194304] 0 2024-02-16T22:21:52.435 INFO:tasks.workunit.client.0.smithi114.stdout:9/245: readlink d2/d33/l48 0 2024-02-16T22:21:52.435 INFO:tasks.workunit.client.0.smithi114.stdout:9/246: fdatasync d2/f12 0 2024-02-16T22:21:52.439 INFO:tasks.workunit.client.0.smithi114.stdout:5/198: write d0/d10/f1a [2563983,113075] 0 2024-02-16T22:21:52.440 INFO:tasks.workunit.client.0.smithi114.stdout:2/218: creat dd/d3c/f4a x:0 0 0 2024-02-16T22:21:52.443 INFO:tasks.workunit.client.1.smithi123.stdout:6/347: unlink d0/d6/d1a/d2e/f34 0 2024-02-16T22:21:52.444 INFO:tasks.workunit.client.0.smithi114.stdout:9/247: dwrite d2/fb [0,4194304] 0 2024-02-16T22:21:52.445 INFO:tasks.workunit.client.1.smithi123.stdout:6/348: write d0/d9/dc/f23 [4610992,57039] 0 2024-02-16T22:21:52.449 INFO:tasks.workunit.client.0.smithi114.stdout:6/214: rename d6/da/l28 to d6/dd/d25/d34/l47 0 2024-02-16T22:21:52.451 INFO:tasks.workunit.client.0.smithi114.stdout:6/215: read d6/dd/d25/d34/f40 [184022,78038] 0 2024-02-16T22:21:52.451 INFO:tasks.workunit.client.1.smithi123.stdout:5/299: unlink d6/dd/f35 0 2024-02-16T22:21:52.451 INFO:tasks.workunit.client.1.smithi123.stdout:5/300: stat d6/dd 0 2024-02-16T22:21:52.452 INFO:tasks.workunit.client.0.smithi114.stdout:6/216: truncate d6/f30 870793 0 2024-02-16T22:21:52.452 INFO:tasks.workunit.client.0.smithi114.stdout:0/241: creat d1/d1a/f4e x:0 0 0 2024-02-16T22:21:52.454 INFO:tasks.workunit.client.1.smithi123.stdout:9/408: mkdir d4/d56/d6f/d7e 0 2024-02-16T22:21:52.455 INFO:tasks.workunit.client.1.smithi123.stdout:9/409: chown d4/d17/f1f 241973 1 2024-02-16T22:21:52.456 INFO:tasks.workunit.client.0.smithi114.stdout:8/240: readlink d4/d16/l35 0 2024-02-16T22:21:52.456 INFO:tasks.workunit.client.1.smithi123.stdout:3/283: dwrite d0/f4c [4194304,4194304] 0 2024-02-16T22:21:52.458 INFO:tasks.workunit.client.1.smithi123.stdout:7/332: rename d3/d6e/d24/f5d to d3/d6e/d1d/d1e/f6f 0 2024-02-16T22:21:52.459 INFO:tasks.workunit.client.0.smithi114.stdout:0/242: dwrite d1/d1a/f4e [0,4194304] 0 2024-02-16T22:21:52.460 INFO:tasks.workunit.client.0.smithi114.stdout:7/208: mknod d3/d3d/c42 0 2024-02-16T22:21:52.461 INFO:tasks.workunit.client.1.smithi123.stdout:9/410: dread d4/d1e/d3a/f40 [0,4194304] 0 2024-02-16T22:21:52.464 INFO:tasks.workunit.client.0.smithi114.stdout:3/202: mkdir d4/d26/d4a 0 2024-02-16T22:21:52.464 INFO:tasks.workunit.client.1.smithi123.stdout:2/299: symlink d4/d1a/d1d/d59/d53/l5f 0 2024-02-16T22:21:52.465 INFO:tasks.workunit.client.0.smithi114.stdout:3/203: read d4/dd/f1e [1985434,10181] 0 2024-02-16T22:21:52.467 INFO:tasks.workunit.client.0.smithi114.stdout:4/310: write dd/d1b/d24/f36 [228843,39774] 0 2024-02-16T22:21:52.468 INFO:tasks.workunit.client.1.smithi123.stdout:7/333: dread d3/d26/d35/f36 [0,4194304] 0 2024-02-16T22:21:52.469 INFO:tasks.workunit.client.1.smithi123.stdout:0/295: creat d1/d4/f56 x:0 0 0 2024-02-16T22:21:52.470 INFO:tasks.workunit.client.1.smithi123.stdout:4/342: link d5/d1c/f30 d5/d1c/d23/d31/f66 0 2024-02-16T22:21:52.470 INFO:tasks.workunit.client.0.smithi114.stdout:5/199: mknod d0/df/d16/d25/c36 0 2024-02-16T22:21:52.471 INFO:tasks.workunit.client.0.smithi114.stdout:2/219: symlink dd/df/d1c/l4b 0 2024-02-16T22:21:52.471 INFO:tasks.workunit.client.0.smithi114.stdout:5/200: read - d0/f24 zero size 2024-02-16T22:21:52.472 INFO:tasks.workunit.client.0.smithi114.stdout:5/201: fdatasync d0/fb 0 2024-02-16T22:21:52.472 INFO:tasks.workunit.client.1.smithi123.stdout:1/287: symlink d3/d6/d65/l6a 0 2024-02-16T22:21:52.472 INFO:tasks.workunit.client.1.smithi123.stdout:1/288: readlink d3/d6/d18/l3e 0 2024-02-16T22:21:52.477 INFO:tasks.workunit.client.0.smithi114.stdout:8/241: rmdir d4/d16 39 2024-02-16T22:21:52.478 INFO:tasks.workunit.client.1.smithi123.stdout:5/301: chown d6/c7 404 1 2024-02-16T22:21:52.478 INFO:tasks.workunit.client.1.smithi123.stdout:5/302: readlink d6/dd/df/d58/d3f/l5a 0 2024-02-16T22:21:52.480 INFO:tasks.workunit.client.0.smithi114.stdout:0/243: write d1/fe [3208226,41355] 0 2024-02-16T22:21:52.481 INFO:tasks.workunit.client.0.smithi114.stdout:1/191: mknod d4/d9/c3a 0 2024-02-16T22:21:52.484 INFO:tasks.workunit.client.0.smithi114.stdout:7/209: fsync d3/f10 0 2024-02-16T22:21:52.485 INFO:tasks.workunit.client.1.smithi123.stdout:3/284: mknod d0/d40/c6c 0 2024-02-16T22:21:52.487 INFO:tasks.workunit.client.0.smithi114.stdout:4/311: rmdir dd/d1b/d24 39 2024-02-16T22:21:52.489 INFO:tasks.workunit.client.0.smithi114.stdout:2/220: stat dd/c15 0 2024-02-16T22:21:52.493 INFO:tasks.workunit.client.0.smithi114.stdout:5/202: creat d0/d28/f37 x:0 0 0 2024-02-16T22:21:52.494 INFO:tasks.workunit.client.0.smithi114.stdout:5/203: dread - d0/f24 zero size 2024-02-16T22:21:52.494 INFO:tasks.workunit.client.1.smithi123.stdout:7/334: mknod d3/d6e/d1d/c70 0 2024-02-16T22:21:52.494 INFO:tasks.workunit.client.1.smithi123.stdout:7/335: chown d3/d6e/d1d/d1e/f5f 15 1 2024-02-16T22:21:52.496 INFO:tasks.workunit.client.1.smithi123.stdout:0/296: mkdir d1/d4/d42/d4b/d57 0 2024-02-16T22:21:52.497 INFO:tasks.workunit.client.1.smithi123.stdout:0/297: fdatasync d1/d4/d42/d4b/f50 0 2024-02-16T22:21:52.498 INFO:tasks.workunit.client.1.smithi123.stdout:4/343: mknod d5/d1c/d23/d36/c67 0 2024-02-16T22:21:52.500 INFO:tasks.workunit.client.0.smithi114.stdout:5/204: dwrite d0/d28/f37 [0,4194304] 0 2024-02-16T22:21:52.502 INFO:tasks.workunit.client.1.smithi123.stdout:1/289: mknod d3/d6/d11/c6b 0 2024-02-16T22:21:52.505 INFO:tasks.workunit.client.0.smithi114.stdout:0/244: rmdir d1/d1a/d3e 39 2024-02-16T22:21:52.507 INFO:tasks.workunit.client.0.smithi114.stdout:6/217: write d6/d2c/f46 [2827086,78980] 0 2024-02-16T22:21:52.511 INFO:tasks.workunit.client.0.smithi114.stdout:7/210: creat d3/d14/f43 x:0 0 0 2024-02-16T22:21:52.514 INFO:tasks.workunit.client.0.smithi114.stdout:4/312: dread dd/d1b/d24/f2f [0,4194304] 0 2024-02-16T22:21:52.514 INFO:tasks.workunit.client.0.smithi114.stdout:2/221: mkdir dd/d3c/d4c 0 2024-02-16T22:21:52.514 INFO:tasks.workunit.client.0.smithi114.stdout:4/313: chown dd/d1b/d24/f6b 50 1 2024-02-16T22:21:52.515 INFO:tasks.workunit.client.0.smithi114.stdout:2/222: chown dd/d3c/d4c 0 1 2024-02-16T22:21:52.515 INFO:tasks.workunit.client.0.smithi114.stdout:2/223: dread - dd/f3f zero size 2024-02-16T22:21:52.517 INFO:tasks.workunit.client.0.smithi114.stdout:6/218: fsync d6/dd/d25/d26/f3d 0 2024-02-16T22:21:52.518 INFO:tasks.workunit.client.1.smithi123.stdout:7/336: creat d3/d6e/d52/f71 x:0 0 0 2024-02-16T22:21:52.519 INFO:tasks.workunit.client.0.smithi114.stdout:5/205: creat d0/d28/f38 x:0 0 0 2024-02-16T22:21:52.520 INFO:tasks.workunit.client.1.smithi123.stdout:4/344: symlink d5/l68 0 2024-02-16T22:21:52.521 INFO:tasks.workunit.client.0.smithi114.stdout:9/248: truncate d2/fb 1886663 0 2024-02-16T22:21:52.522 INFO:tasks.workunit.client.0.smithi114.stdout:8/242: creat d4/d16/d25/d39/f4f x:0 0 0 2024-02-16T22:21:52.523 INFO:tasks.workunit.client.1.smithi123.stdout:1/290: symlink d3/d6/db/d10/d2a/d5f/l6c 0 2024-02-16T22:21:52.528 INFO:tasks.workunit.client.1.smithi123.stdout:3/285: rename d0/c1e to d0/d20/d29/d34/d37/c6d 0 2024-02-16T22:21:52.533 INFO:tasks.workunit.client.1.smithi123.stdout:7/337: fdatasync d3/d6e/d24/f31 0 2024-02-16T22:21:52.534 INFO:tasks.workunit.client.1.smithi123.stdout:7/338: write d3/d6e/f1a [270843,27223] 0 2024-02-16T22:21:52.537 INFO:tasks.workunit.client.1.smithi123.stdout:4/345: write f4 [1118966,1269] 0 2024-02-16T22:21:52.541 INFO:tasks.workunit.client.1.smithi123.stdout:6/349: getdents d0/d9/dc/d3a/d52 0 2024-02-16T22:21:52.542 INFO:tasks.workunit.client.1.smithi123.stdout:6/350: dread - d0/d9/dc/d39/f5b zero size 2024-02-16T22:21:52.543 INFO:tasks.workunit.client.0.smithi114.stdout:8/243: unlink d4/fe 0 2024-02-16T22:21:52.543 INFO:tasks.workunit.client.1.smithi123.stdout:5/303: dwrite d6/dd/df/f22 [0,4194304] 0 2024-02-16T22:21:52.543 INFO:tasks.workunit.client.0.smithi114.stdout:0/245: fdatasync d1/d16/d18/f1c 0 2024-02-16T22:21:52.544 INFO:tasks.workunit.client.0.smithi114.stdout:0/246: read d1/f2a [1718153,84432] 0 2024-02-16T22:21:52.547 INFO:tasks.workunit.client.1.smithi123.stdout:0/298: rename d1/d4/d17/f28 to d1/d4/d42/d4b/f58 0 2024-02-16T22:21:52.547 INFO:tasks.workunit.client.1.smithi123.stdout:0/299: dread - d1/d8/d13/f37 zero size 2024-02-16T22:21:52.548 INFO:tasks.workunit.client.0.smithi114.stdout:6/219: write d6/dd/d25/d34/f40 [4006403,86065] 0 2024-02-16T22:21:52.551 INFO:tasks.workunit.client.0.smithi114.stdout:4/314: rename c7 to dd/d1b/d41/c6f 0 2024-02-16T22:21:52.551 INFO:tasks.workunit.client.0.smithi114.stdout:9/249: rename d2/d1a/d23 to d2/d1a/d23/d50 22 2024-02-16T22:21:52.552 INFO:tasks.workunit.client.1.smithi123.stdout:8/383: getdents d0/d17/d26/d2a/d4d 0 2024-02-16T22:21:52.553 INFO:tasks.workunit.client.0.smithi114.stdout:2/224: dwrite dd/f28 [0,4194304] 0 2024-02-16T22:21:52.555 INFO:tasks.workunit.client.0.smithi114.stdout:5/206: mknod d0/c39 0 2024-02-16T22:21:52.556 INFO:tasks.workunit.client.0.smithi114.stdout:5/207: read d0/f22 [3687955,39416] 0 2024-02-16T22:21:52.556 INFO:tasks.workunit.client.0.smithi114.stdout:8/244: creat d4/d24/f50 x:0 0 0 2024-02-16T22:21:52.558 INFO:tasks.workunit.client.1.smithi123.stdout:1/291: mknod d3/d6/d49/d48/d68/c6d 0 2024-02-16T22:21:52.559 INFO:tasks.workunit.client.1.smithi123.stdout:1/292: fsync d3/d6/db/d10/d2d/f5a 0 2024-02-16T22:21:52.565 INFO:tasks.workunit.client.1.smithi123.stdout:5/304: creat d6/d3b/f65 x:0 0 0 2024-02-16T22:21:52.566 INFO:tasks.workunit.client.0.smithi114.stdout:1/192: link l3 d4/l3b 0 2024-02-16T22:21:52.568 INFO:tasks.workunit.client.0.smithi114.stdout:6/220: chown d6/c32 16 1 2024-02-16T22:21:52.568 INFO:tasks.workunit.client.1.smithi123.stdout:3/286: symlink d0/d20/d5a/l6e 0 2024-02-16T22:21:52.570 INFO:tasks.workunit.client.1.smithi123.stdout:0/300: creat d1/d8/d3d/f59 x:0 0 0 2024-02-16T22:21:52.570 INFO:tasks.workunit.client.0.smithi114.stdout:9/250: mknod d2/d33/c51 0 2024-02-16T22:21:52.571 INFO:tasks.workunit.client.1.smithi123.stdout:3/287: read d0/fc [205821,9061] 0 2024-02-16T22:21:52.571 INFO:tasks.workunit.client.1.smithi123.stdout:0/301: dread d1/d8/f2f [0,4194304] 0 2024-02-16T22:21:52.571 INFO:tasks.workunit.client.0.smithi114.stdout:2/225: creat dd/df/d2c/f4d x:0 0 0 2024-02-16T22:21:52.572 INFO:tasks.workunit.client.1.smithi123.stdout:0/302: readlink d1/l2d 0 2024-02-16T22:21:52.572 INFO:tasks.workunit.client.0.smithi114.stdout:4/315: read dd/f12 [1603782,5245] 0 2024-02-16T22:21:52.574 INFO:tasks.workunit.client.1.smithi123.stdout:4/346: mkdir d5/d1c/d23/d69 0 2024-02-16T22:21:52.577 INFO:tasks.workunit.client.1.smithi123.stdout:1/293: rename d3/d6/db/d10/d2a/c3f to d3/d6/db/d10/d2a/c6e 0 2024-02-16T22:21:52.578 INFO:tasks.workunit.client.1.smithi123.stdout:9/411: sync 2024-02-16T22:21:52.579 INFO:tasks.workunit.client.0.smithi114.stdout:5/208: rename d0/c2 to d0/d2d/c3a 0 2024-02-16T22:21:52.580 INFO:tasks.workunit.client.1.smithi123.stdout:9/412: write d4/f1b [1465859,2120] 0 2024-02-16T22:21:52.582 INFO:tasks.workunit.client.1.smithi123.stdout:5/305: creat d6/dd/df/d1f/d25/f66 x:0 0 0 2024-02-16T22:21:52.584 INFO:tasks.workunit.client.1.smithi123.stdout:1/294: dwrite d3/d6/db/d10/d2d/f5a [0,4194304] 0 2024-02-16T22:21:52.586 INFO:tasks.workunit.client.1.smithi123.stdout:8/384: symlink d0/d17/d26/d2a/l7f 0 2024-02-16T22:21:52.586 INFO:tasks.workunit.client.0.smithi114.stdout:8/245: write d4/d11/f20 [444810,32499] 0 2024-02-16T22:21:52.588 INFO:tasks.workunit.client.0.smithi114.stdout:9/251: creat d2/d1a/d23/f52 x:0 0 0 2024-02-16T22:21:52.589 INFO:tasks.workunit.client.1.smithi123.stdout:0/303: creat d1/d8/d13/f5a x:0 0 0 2024-02-16T22:21:52.591 INFO:tasks.workunit.client.1.smithi123.stdout:2/300: link d4/d1a/f39 d4/d1a/d1d/d59/d45/f60 0 2024-02-16T22:21:52.591 INFO:tasks.workunit.client.1.smithi123.stdout:4/347: dread d5/f1f [0,4194304] 0 2024-02-16T22:21:52.591 INFO:tasks.workunit.client.0.smithi114.stdout:0/247: write d1/d16/d18/f22 [1323484,115096] 0 2024-02-16T22:21:52.593 INFO:tasks.workunit.client.0.smithi114.stdout:0/248: write d1/fe [2300879,70964] 0 2024-02-16T22:21:52.593 INFO:tasks.workunit.client.0.smithi114.stdout:4/316: unlink dd/d1b/f1d 0 2024-02-16T22:21:52.594 INFO:tasks.workunit.client.1.smithi123.stdout:8/385: dwrite d0/d29/f60 [0,4194304] 0 2024-02-16T22:21:52.594 INFO:tasks.workunit.client.0.smithi114.stdout:9/252: dread d2/d1a/d23/d35/f1e [0,4194304] 0 2024-02-16T22:21:52.594 INFO:tasks.workunit.client.0.smithi114.stdout:4/317: readlink dd/d1b/l26 0 2024-02-16T22:21:52.595 INFO:tasks.workunit.client.0.smithi114.stdout:4/318: stat dd/d1b/f47 0 2024-02-16T22:21:52.595 INFO:tasks.workunit.client.0.smithi114.stdout:9/253: write d2/f26 [743417,24547] 0 2024-02-16T22:21:52.597 INFO:tasks.workunit.client.1.smithi123.stdout:9/413: fsync d4/d1e/d3a/f40 0 2024-02-16T22:21:52.598 INFO:tasks.workunit.client.1.smithi123.stdout:9/414: fdatasync d4/d1e/f42 0 2024-02-16T22:21:52.599 INFO:tasks.workunit.client.1.smithi123.stdout:5/306: creat d6/dd/d5f/f67 x:0 0 0 2024-02-16T22:21:52.602 INFO:tasks.workunit.client.1.smithi123.stdout:1/295: creat d3/d6/db/d10/d2a/f6f x:0 0 0 2024-02-16T22:21:52.602 INFO:tasks.workunit.client.1.smithi123.stdout:1/296: stat d3/d6/d11/f23 0 2024-02-16T22:21:52.603 INFO:tasks.workunit.client.0.smithi114.stdout:4/319: dwrite dd/d1b/d37/d38/f5e [0,4194304] 0 2024-02-16T22:21:52.604 INFO:tasks.workunit.client.0.smithi114.stdout:4/320: dread - dd/d1b/f47 zero size 2024-02-16T22:21:52.605 INFO:tasks.workunit.client.0.smithi114.stdout:2/226: mkdir dd/d4e 0 2024-02-16T22:21:52.608 INFO:tasks.workunit.client.1.smithi123.stdout:1/297: dwrite d3/d6/db/d10/f29 [0,4194304] 0 2024-02-16T22:21:52.609 INFO:tasks.workunit.client.0.smithi114.stdout:0/249: read d1/d1a/f26 [3694197,51318] 0 2024-02-16T22:21:52.611 INFO:tasks.workunit.client.1.smithi123.stdout:7/339: sync 2024-02-16T22:21:52.611 INFO:tasks.workunit.client.1.smithi123.stdout:4/348: rmdir d5/d1c 39 2024-02-16T22:21:52.611 INFO:tasks.workunit.client.1.smithi123.stdout:7/340: read - d3/d6e/f4f zero size 2024-02-16T22:21:52.613 INFO:tasks.workunit.client.1.smithi123.stdout:8/386: rename d0/d17/d26/d68/f76 to d0/d17/d26/d2a/d3c/f80 0 2024-02-16T22:21:52.613 INFO:tasks.workunit.client.1.smithi123.stdout:6/351: getdents d0/d6/d4d 0 2024-02-16T22:21:52.614 INFO:tasks.workunit.client.1.smithi123.stdout:8/387: readlink d0/d17/d43/d34/l65 0 2024-02-16T22:21:52.614 INFO:tasks.workunit.client.0.smithi114.stdout:9/254: chown d2/d1a/f39 95383 1 2024-02-16T22:21:52.616 INFO:tasks.workunit.client.0.smithi114.stdout:9/255: write d2/d1a/d23/d35/d1d/f3a [759092,7870] 0 2024-02-16T22:21:52.619 INFO:tasks.workunit.client.0.smithi114.stdout:1/193: creat d4/f3c x:0 0 0 2024-02-16T22:21:52.619 INFO:tasks.workunit.client.1.smithi123.stdout:8/388: dwrite d0/d17/f35 [0,4194304] 0 2024-02-16T22:21:52.624 INFO:tasks.workunit.client.0.smithi114.stdout:5/209: truncate d0/d28/f37 1947833 0 2024-02-16T22:21:52.625 INFO:tasks.workunit.client.1.smithi123.stdout:9/415: rmdir d4/d17/d20 39 2024-02-16T22:21:52.626 INFO:tasks.workunit.client.1.smithi123.stdout:3/288: creat d0/d20/f6f x:0 0 0 2024-02-16T22:21:52.628 INFO:tasks.workunit.client.1.smithi123.stdout:1/298: fdatasync d3/d6/f26 0 2024-02-16T22:21:52.630 INFO:tasks.workunit.client.1.smithi123.stdout:1/299: chown d3/d6/db/d10/c32 1491 1 2024-02-16T22:21:52.632 INFO:tasks.workunit.client.1.smithi123.stdout:1/300: fdatasync d3/d6/db/d10/d4f/f51 0 2024-02-16T22:21:52.632 INFO:tasks.workunit.client.1.smithi123.stdout:8/389: dwrite d0/d17/d26/d32/f56 [0,4194304] 0 2024-02-16T22:21:52.633 INFO:tasks.workunit.client.1.smithi123.stdout:1/301: write d3/d6/d49/f50 [237922,96110] 0 2024-02-16T22:21:52.634 INFO:tasks.workunit.client.0.smithi114.stdout:9/256: symlink d2/d1a/d23/d35/d1d/l53 0 2024-02-16T22:21:52.635 INFO:tasks.workunit.client.1.smithi123.stdout:1/302: dread - d3/d6/db/f5c zero size 2024-02-16T22:21:52.635 INFO:tasks.workunit.client.1.smithi123.stdout:7/341: creat d3/d6e/d52/f72 x:0 0 0 2024-02-16T22:21:52.635 INFO:tasks.workunit.client.1.smithi123.stdout:8/390: truncate d0/d17/d26/f6d 626315 0 2024-02-16T22:21:52.635 INFO:tasks.workunit.client.1.smithi123.stdout:1/303: readlink d3/d6/d11/l56 0 2024-02-16T22:21:52.636 INFO:tasks.workunit.client.1.smithi123.stdout:7/342: write d3/d6e/d4d/f50 [1691545,20327] 0 2024-02-16T22:21:52.637 INFO:tasks.workunit.client.1.smithi123.stdout:5/307: dread d6/f8 [0,4194304] 0 2024-02-16T22:21:52.638 INFO:tasks.workunit.client.0.smithi114.stdout:1/194: rename d4/d9/d2a to d4/d9/d3d 0 2024-02-16T22:21:52.638 INFO:tasks.workunit.client.1.smithi123.stdout:5/308: chown d6/dd/df/f5e 0 1 2024-02-16T22:21:52.640 INFO:tasks.workunit.client.1.smithi123.stdout:5/309: truncate d6/dd/df/d1f/d25/f66 203308 0 2024-02-16T22:21:52.641 INFO:tasks.workunit.client.0.smithi114.stdout:6/221: getdents d6 0 2024-02-16T22:21:52.641 INFO:tasks.workunit.client.1.smithi123.stdout:5/310: write d6/dd/df/d58/f63 [592024,21391] 0 2024-02-16T22:21:52.642 INFO:tasks.workunit.client.0.smithi114.stdout:8/246: creat d4/d11/f51 x:0 0 0 2024-02-16T22:21:52.643 INFO:tasks.workunit.client.0.smithi114.stdout:8/247: readlink d4/l22 0 2024-02-16T22:21:52.644 INFO:tasks.workunit.client.0.smithi114.stdout:5/210: write d0/d10/f15 [943375,66220] 0 2024-02-16T22:21:52.645 INFO:tasks.workunit.client.1.smithi123.stdout:7/343: read d3/d6e/d1d/f2f [8034,22328] 0 2024-02-16T22:21:52.652 INFO:tasks.workunit.client.0.smithi114.stdout:5/211: dwrite d0/df/d16/d1e/f29 [0,4194304] 0 2024-02-16T22:21:52.653 INFO:tasks.workunit.client.0.smithi114.stdout:9/257: creat d2/d1a/d23/d35/d1d/f54 x:0 0 0 2024-02-16T22:21:52.653 INFO:tasks.workunit.client.1.smithi123.stdout:0/304: dwrite d1/d4/f51 [0,4194304] 0 2024-02-16T22:21:52.653 INFO:tasks.workunit.client.1.smithi123.stdout:4/349: write d5/f40 [2849838,31981] 0 2024-02-16T22:21:52.654 INFO:tasks.workunit.client.1.smithi123.stdout:3/289: mknod d0/d40/c70 0 2024-02-16T22:21:52.660 INFO:tasks.workunit.client.0.smithi114.stdout:0/250: dwrite d1/d16/d18/f1c [0,4194304] 0 2024-02-16T22:21:52.662 INFO:tasks.workunit.client.0.smithi114.stdout:4/321: creat dd/d1b/d24/f70 x:0 0 0 2024-02-16T22:21:52.671 INFO:tasks.workunit.client.1.smithi123.stdout:8/391: rename d0/d29/d42 to d0/d17/d26/d68/d81 0 2024-02-16T22:21:52.672 INFO:tasks.workunit.client.1.smithi123.stdout:8/392: stat d0/d12/f74 0 2024-02-16T22:21:52.673 INFO:tasks.workunit.client.0.smithi114.stdout:1/195: rename d4/d9/f27 to d4/d33/f3e 0 2024-02-16T22:21:52.675 INFO:tasks.workunit.client.1.smithi123.stdout:6/352: link d0/d6/d1a/d2e/f7f d0/d9/dc/d3a/f80 0 2024-02-16T22:21:52.676 INFO:tasks.workunit.client.1.smithi123.stdout:5/311: rmdir d6/dd/d5f 39 2024-02-16T22:21:52.678 INFO:tasks.workunit.client.1.smithi123.stdout:7/344: rmdir d3/d26/d3e 39 2024-02-16T22:21:52.679 INFO:tasks.workunit.client.0.smithi114.stdout:8/248: mkdir d4/d11/d52 0 2024-02-16T22:21:52.679 INFO:tasks.workunit.client.1.smithi123.stdout:9/416: mkdir d4/d17/d7f 0 2024-02-16T22:21:52.681 INFO:tasks.workunit.client.0.smithi114.stdout:0/251: mknod d1/d16/d1f/d28/c4f 0 2024-02-16T22:21:52.681 INFO:tasks.workunit.client.1.smithi123.stdout:4/350: chown d5/d1c/d64 43 1 2024-02-16T22:21:52.683 INFO:tasks.workunit.client.0.smithi114.stdout:4/322: creat dd/d59/f71 x:0 0 0 2024-02-16T22:21:52.684 INFO:tasks.workunit.client.1.smithi123.stdout:0/305: creat d1/d4/d17/d54/f5b x:0 0 0 2024-02-16T22:21:52.685 INFO:tasks.workunit.client.0.smithi114.stdout:8/249: dwrite d4/f3c [0,4194304] 0 2024-02-16T22:21:52.686 INFO:tasks.workunit.client.0.smithi114.stdout:8/250: readlink d4/d16/l35 0 2024-02-16T22:21:52.690 INFO:tasks.workunit.client.1.smithi123.stdout:2/301: link d4/d1a/d1d/d59/d53/c55 d4/d1a/d1d/d13/d25/c61 0 2024-02-16T22:21:52.691 INFO:tasks.workunit.client.0.smithi114.stdout:5/212: fsync d0/d28/f37 0 2024-02-16T22:21:52.693 INFO:tasks.workunit.client.1.smithi123.stdout:1/304: mkdir d3/d6/db/d10/d4b/d70 0 2024-02-16T22:21:52.696 INFO:tasks.workunit.client.0.smithi114.stdout:6/222: truncate d6/dd/d25/d26/f2d 7286426 0 2024-02-16T22:21:52.697 INFO:tasks.workunit.client.0.smithi114.stdout:1/196: mkdir d4/d9/d3d/d3f 0 2024-02-16T22:21:52.698 INFO:tasks.workunit.client.0.smithi114.stdout:5/213: dwrite d0/d28/f38 [0,4194304] 0 2024-02-16T22:21:52.700 INFO:tasks.workunit.client.0.smithi114.stdout:5/214: read d0/df/d16/d1e/f29 [2189106,64472] 0 2024-02-16T22:21:52.701 INFO:tasks.workunit.client.1.smithi123.stdout:6/353: creat d0/d9/dc/d3a/f81 x:0 0 0 2024-02-16T22:21:52.701 INFO:tasks.workunit.client.0.smithi114.stdout:8/251: fdatasync d4/f3c 0 2024-02-16T22:21:52.703 INFO:tasks.workunit.client.0.smithi114.stdout:0/252: chown d1/d30/c39 63 1 2024-02-16T22:21:52.705 INFO:tasks.workunit.client.1.smithi123.stdout:5/312: unlink d6/dd/df/d1f/d25/f2f 0 2024-02-16T22:21:52.707 INFO:tasks.workunit.client.0.smithi114.stdout:4/323: dread dd/d1b/d24/f36 [4194304,4194304] 0 2024-02-16T22:21:52.708 INFO:tasks.workunit.client.1.smithi123.stdout:7/345: truncate d3/d26/d3e/f63 362924 0 2024-02-16T22:21:52.708 INFO:tasks.workunit.client.1.smithi123.stdout:7/346: dread - d3/d5/f61 zero size 2024-02-16T22:21:52.709 INFO:tasks.workunit.client.1.smithi123.stdout:7/347: fdatasync d3/f4e 0 2024-02-16T22:21:52.710 INFO:tasks.workunit.client.0.smithi114.stdout:9/258: write d2/fb [1248127,37176] 0 2024-02-16T22:21:52.712 INFO:tasks.workunit.client.1.smithi123.stdout:9/417: read d4/d17/d20/f3c [1663220,114148] 0 2024-02-16T22:21:52.718 INFO:tasks.workunit.client.1.smithi123.stdout:1/305: rename d3/d6/db/d10/d2a/d5f/l6c to d3/d6/db/d10/d2a/d53/l71 0 2024-02-16T22:21:52.718 INFO:tasks.workunit.client.1.smithi123.stdout:1/306: stat d3/d6/db/d10 0 2024-02-16T22:21:52.720 INFO:tasks.workunit.client.1.smithi123.stdout:8/393: link d0/d17/d26/d2a/d3c/f55 d0/d17/d26/d68/d81/d47/f82 0 2024-02-16T22:21:52.720 INFO:tasks.workunit.client.1.smithi123.stdout:8/394: chown d0/d17/d26/d2a/l7f 3 1 2024-02-16T22:21:52.721 INFO:tasks.workunit.client.1.smithi123.stdout:8/395: fsync d0/f14 0 2024-02-16T22:21:52.721 INFO:tasks.workunit.client.1.smithi123.stdout:1/307: read d3/d6/f5b [1176446,18967] 0 2024-02-16T22:21:52.722 INFO:tasks.workunit.client.1.smithi123.stdout:6/354: mknod d0/d6/d2f/c82 0 2024-02-16T22:21:52.722 INFO:tasks.workunit.client.1.smithi123.stdout:1/308: chown d3/d6/db/d10/d2a 98534648 1 2024-02-16T22:21:52.723 INFO:tasks.workunit.client.1.smithi123.stdout:8/396: truncate d0/d17/d26/d68/d81/d47/f6a 792866 0 2024-02-16T22:21:52.726 INFO:tasks.workunit.client.1.smithi123.stdout:7/348: symlink d3/d26/d35/l73 0 2024-02-16T22:21:52.729 INFO:tasks.workunit.client.0.smithi114.stdout:2/227: getdents dd/df/d1c/d21 0 2024-02-16T22:21:52.730 INFO:tasks.workunit.client.1.smithi123.stdout:4/351: fsync d5/d49/f4b 0 2024-02-16T22:21:52.732 INFO:tasks.workunit.client.1.smithi123.stdout:0/306: creat d1/d8/d3d/d44/f5c x:0 0 0 2024-02-16T22:21:52.740 INFO:tasks.workunit.client.0.smithi114.stdout:2/228: mknod dd/d3c/c4f 0 2024-02-16T22:21:52.744 INFO:tasks.workunit.client.1.smithi123.stdout:9/418: rename d4/l8 to d4/d1e/l80 0 2024-02-16T22:21:52.744 INFO:tasks.workunit.client.0.smithi114.stdout:6/223: getdents d6/dd 0 2024-02-16T22:21:52.744 INFO:tasks.workunit.client.1.smithi123.stdout:8/397: rmdir d0/d17/d26/d2a 39 2024-02-16T22:21:52.745 INFO:tasks.workunit.client.0.smithi114.stdout:1/197: link d4/d9/c3a d4/d33/c40 0 2024-02-16T22:21:52.745 INFO:tasks.workunit.client.1.smithi123.stdout:4/352: dwrite d5/d1c/d23/f2a [0,4194304] 0 2024-02-16T22:21:52.746 INFO:tasks.workunit.client.0.smithi114.stdout:1/198: stat d4/d9/f11 0 2024-02-16T22:21:52.746 INFO:tasks.workunit.client.1.smithi123.stdout:8/398: chown d0/d12/f19 137616226 1 2024-02-16T22:21:52.747 INFO:tasks.workunit.client.1.smithi123.stdout:3/290: link d0/d20/d2e/c65 d0/d20/d29/d34/d5d/d60/c71 0 2024-02-16T22:21:52.751 INFO:tasks.workunit.client.1.smithi123.stdout:5/313: creat d6/dd/df/d1f/f68 x:0 0 0 2024-02-16T22:21:52.752 INFO:tasks.workunit.client.1.smithi123.stdout:9/419: mkdir d4/d81 0 2024-02-16T22:21:52.752 INFO:tasks.workunit.client.1.smithi123.stdout:5/314: readlink d6/dd/df/l46 0 2024-02-16T22:21:52.753 INFO:tasks.workunit.client.0.smithi114.stdout:4/324: write dd/d1b/d37/d38/f46 [3183876,41310] 0 2024-02-16T22:21:52.753 INFO:tasks.workunit.client.0.smithi114.stdout:8/252: link d4/d16/l35 d4/d11/d52/l53 0 2024-02-16T22:21:52.754 INFO:tasks.workunit.client.1.smithi123.stdout:3/291: dwrite d0/d20/f52 [0,4194304] 0 2024-02-16T22:21:52.754 INFO:tasks.workunit.client.0.smithi114.stdout:9/259: creat d2/dd/f55 x:0 0 0 2024-02-16T22:21:52.754 INFO:tasks.workunit.client.0.smithi114.stdout:4/325: fsync dd/d1b/d37/d61/f66 0 2024-02-16T22:21:52.755 INFO:tasks.workunit.client.0.smithi114.stdout:6/224: creat d6/dd/d25/d34/f48 x:0 0 0 2024-02-16T22:21:52.755 INFO:tasks.workunit.client.0.smithi114.stdout:1/199: creat d4/d9/d3d/f41 x:0 0 0 2024-02-16T22:21:52.755 INFO:tasks.workunit.client.1.smithi123.stdout:8/399: truncate d0/d12/f50 2100635 0 2024-02-16T22:21:52.756 INFO:tasks.workunit.client.0.smithi114.stdout:5/215: getdents d0/df 0 2024-02-16T22:21:52.757 INFO:tasks.workunit.client.0.smithi114.stdout:2/229: mknod dd/df/c50 0 2024-02-16T22:21:52.758 INFO:tasks.workunit.client.0.smithi114.stdout:5/216: stat d0/d28/f38 0 2024-02-16T22:21:52.758 INFO:tasks.workunit.client.0.smithi114.stdout:2/230: readlink dd/df/l3e 0 2024-02-16T22:21:52.759 INFO:tasks.workunit.client.1.smithi123.stdout:3/292: dwrite d0/f4c [0,4194304] 0 2024-02-16T22:21:52.759 INFO:tasks.workunit.client.0.smithi114.stdout:8/253: creat d4/d16/d25/f54 x:0 0 0 2024-02-16T22:21:52.760 INFO:tasks.workunit.client.1.smithi123.stdout:3/293: write d0/f1f [2413441,41020] 0 2024-02-16T22:21:52.760 INFO:tasks.workunit.client.1.smithi123.stdout:0/307: truncate d1/d4/f11 905137 0 2024-02-16T22:21:52.762 INFO:tasks.workunit.client.0.smithi114.stdout:1/200: mknod d4/d33/c42 0 2024-02-16T22:21:52.764 INFO:tasks.workunit.client.0.smithi114.stdout:5/217: dread d0/f22 [0,4194304] 0 2024-02-16T22:21:52.765 INFO:tasks.workunit.client.0.smithi114.stdout:9/260: creat d2/d1a/d23/f56 x:0 0 0 2024-02-16T22:21:52.767 INFO:tasks.workunit.client.0.smithi114.stdout:6/225: symlink d6/dd/l49 0 2024-02-16T22:21:52.768 INFO:tasks.workunit.client.0.smithi114.stdout:6/226: truncate d6/dd/d25/f27 1069781 0 2024-02-16T22:21:52.769 INFO:tasks.workunit.client.0.smithi114.stdout:6/227: chown d6/dd/d25/d34/f40 56 1 2024-02-16T22:21:52.770 INFO:tasks.workunit.client.0.smithi114.stdout:4/326: mknod dd/d1b/d37/d38/c72 0 2024-02-16T22:21:52.775 INFO:tasks.workunit.client.0.smithi114.stdout:1/201: creat d4/d33/f43 x:0 0 0 2024-02-16T22:21:52.776 INFO:tasks.workunit.client.0.smithi114.stdout:9/261: creat d2/d33/f57 x:0 0 0 2024-02-16T22:21:52.780 INFO:tasks.workunit.client.0.smithi114.stdout:8/254: creat d4/d16/d25/d46/f55 x:0 0 0 2024-02-16T22:21:52.781 INFO:tasks.workunit.client.0.smithi114.stdout:1/202: symlink d4/d9/d3d/l44 0 2024-02-16T22:21:52.782 INFO:tasks.workunit.client.0.smithi114.stdout:9/262: symlink d2/d33/l58 0 2024-02-16T22:21:52.783 INFO:tasks.workunit.client.0.smithi114.stdout:9/263: chown d2/f12 931481260 1 2024-02-16T22:21:52.788 INFO:tasks.workunit.client.0.smithi114.stdout:9/264: creat d2/dd/d3c/f59 x:0 0 0 2024-02-16T22:21:52.789 INFO:tasks.workunit.client.0.smithi114.stdout:9/265: write d2/f6 [2836963,72717] 0 2024-02-16T22:21:52.791 INFO:tasks.workunit.client.0.smithi114.stdout:4/327: rename dd/d1b/d37/d38/f46 to dd/d1b/d37/f73 0 2024-02-16T22:21:52.793 INFO:tasks.workunit.client.0.smithi114.stdout:2/231: rename dd/f16 to dd/f51 0 2024-02-16T22:21:52.795 INFO:tasks.workunit.client.0.smithi114.stdout:6/228: getdents d6/d2c/d38 0 2024-02-16T22:21:52.797 INFO:tasks.workunit.client.0.smithi114.stdout:5/218: truncate d0/df/d16/d1e/f21 2880248 0 2024-02-16T22:21:52.799 INFO:tasks.workunit.client.0.smithi114.stdout:8/255: rename d4/fc to d4/d24/d2c/f56 0 2024-02-16T22:21:52.800 INFO:tasks.workunit.client.1.smithi123.stdout:6/355: truncate d0/d9/f62 281012 0 2024-02-16T22:21:52.801 INFO:tasks.workunit.client.0.smithi114.stdout:9/266: dwrite d2/f12 [0,4194304] 0 2024-02-16T22:21:52.801 INFO:tasks.workunit.client.1.smithi123.stdout:6/356: stat d0 0 2024-02-16T22:21:52.803 INFO:tasks.workunit.client.1.smithi123.stdout:9/420: write d4/d36/f78 [2358470,54441] 0 2024-02-16T22:21:52.804 INFO:tasks.workunit.client.1.smithi123.stdout:2/302: rename d4/d1a/d1d/d59/l41 to d4/d1a/d1d/d59/d45/l62 0 2024-02-16T22:21:52.805 INFO:tasks.workunit.client.0.smithi114.stdout:4/328: dwrite dd/f21 [0,4194304] 0 2024-02-16T22:21:52.805 INFO:tasks.workunit.client.0.smithi114.stdout:9/267: rename d2 to d2/d1a/d23/d5a 22 2024-02-16T22:21:52.807 INFO:tasks.workunit.client.1.smithi123.stdout:3/294: dwrite d0/d20/d2e/f48 [0,4194304] 0 2024-02-16T22:21:52.808 INFO:tasks.workunit.client.1.smithi123.stdout:3/295: fsync d0/d20/f68 0 2024-02-16T22:21:52.809 INFO:tasks.workunit.client.0.smithi114.stdout:9/268: readlink d2/d33/l58 0 2024-02-16T22:21:52.810 INFO:tasks.workunit.client.0.smithi114.stdout:1/203: dwrite d4/f16 [0,4194304] 0 2024-02-16T22:21:52.810 INFO:tasks.workunit.client.0.smithi114.stdout:2/232: dwrite dd/f28 [4194304,4194304] 0 2024-02-16T22:21:52.814 INFO:tasks.workunit.client.0.smithi114.stdout:2/233: truncate dd/f28 8543200 0 2024-02-16T22:21:52.816 INFO:tasks.workunit.client.0.smithi114.stdout:8/256: mknod d4/d11/d15/d36/c57 0 2024-02-16T22:21:52.822 INFO:tasks.workunit.client.0.smithi114.stdout:5/219: dwrite d0/df/d16/d1e/f29 [4194304,4194304] 0 2024-02-16T22:21:52.824 INFO:tasks.workunit.client.1.smithi123.stdout:8/400: mknod d0/d17/d43/c83 0 2024-02-16T22:21:52.825 INFO:tasks.workunit.client.0.smithi114.stdout:4/329: write dd/d1b/d24/f36 [7276824,65552] 0 2024-02-16T22:21:52.828 INFO:tasks.workunit.client.1.smithi123.stdout:8/401: dread d0/df/f1b [0,4194304] 0 2024-02-16T22:21:52.829 INFO:tasks.workunit.client.1.smithi123.stdout:1/309: sync 2024-02-16T22:21:52.829 INFO:tasks.workunit.client.0.smithi114.stdout:9/269: symlink d2/d1a/d23/d35/d1d/l5b 0 2024-02-16T22:21:52.833 INFO:tasks.workunit.client.1.smithi123.stdout:4/353: getdents d5/d1c/d5e 0 2024-02-16T22:21:52.835 INFO:tasks.workunit.client.1.smithi123.stdout:9/421: mkdir d4/d17/d2b/d82 0 2024-02-16T22:21:52.835 INFO:tasks.workunit.client.1.smithi123.stdout:9/422: readlink d4/d17/l30 0 2024-02-16T22:21:52.836 INFO:tasks.workunit.client.1.smithi123.stdout:2/303: mkdir d4/d1a/d1d/d38/d63 0 2024-02-16T22:21:52.840 INFO:tasks.workunit.client.1.smithi123.stdout:9/423: dread d4/d1e/d3a/f50 [0,4194304] 0 2024-02-16T22:21:52.840 INFO:tasks.workunit.client.1.smithi123.stdout:9/424: chown d4/d36/d57 118310359 1 2024-02-16T22:21:52.842 INFO:tasks.workunit.client.1.smithi123.stdout:1/310: creat d3/d6/db/d10/d4b/f72 x:0 0 0 2024-02-16T22:21:52.842 INFO:tasks.workunit.client.1.smithi123.stdout:1/311: fdatasync d3/d6/db/f5c 0 2024-02-16T22:21:52.843 INFO:tasks.workunit.client.0.smithi114.stdout:1/204: dwrite d4/f30 [0,4194304] 0 2024-02-16T22:21:52.844 INFO:tasks.workunit.client.0.smithi114.stdout:2/234: rename fc to dd/df/f52 0 2024-02-16T22:21:52.846 INFO:tasks.workunit.client.1.smithi123.stdout:4/354: creat d5/d1c/d42/f6a x:0 0 0 2024-02-16T22:21:52.847 INFO:tasks.workunit.client.0.smithi114.stdout:9/270: truncate d2/d1a/f21 1484895 0 2024-02-16T22:21:52.848 INFO:tasks.workunit.client.1.smithi123.stdout:2/304: mknod d4/d1a/d1d/c64 0 2024-02-16T22:21:52.856 INFO:tasks.workunit.client.0.smithi114.stdout:1/205: dread d4/fa [0,4194304] 0 2024-02-16T22:21:52.856 INFO:tasks.workunit.client.0.smithi114.stdout:8/257: rename d4/d16/l35 to d4/d11/d52/l58 0 2024-02-16T22:21:52.857 INFO:tasks.workunit.client.1.smithi123.stdout:0/308: write d1/f6 [1548251,87903] 0 2024-02-16T22:21:52.859 INFO:tasks.workunit.client.0.smithi114.stdout:2/235: rmdir dd/df/d2c 39 2024-02-16T22:21:52.861 INFO:tasks.workunit.client.1.smithi123.stdout:1/312: symlink d3/d6/db/d10/d4b/l73 0 2024-02-16T22:21:52.861 INFO:tasks.workunit.client.0.smithi114.stdout:5/220: getdents d0/df/d16/d1e 0 2024-02-16T22:21:52.866 INFO:tasks.workunit.client.0.smithi114.stdout:4/330: truncate dd/ff 1097090 0 2024-02-16T22:21:52.869 INFO:tasks.workunit.client.1.smithi123.stdout:8/402: creat d0/d17/d26/d2a/d77/f84 x:0 0 0 2024-02-16T22:21:52.869 INFO:tasks.workunit.client.0.smithi114.stdout:1/206: creat d4/d33/f45 x:0 0 0 2024-02-16T22:21:52.870 INFO:tasks.workunit.client.1.smithi123.stdout:8/403: write d0/d17/f35 [4789817,34059] 0 2024-02-16T22:21:52.870 INFO:tasks.workunit.client.1.smithi123.stdout:8/404: read d0/d17/f35 [3123840,43842] 0 2024-02-16T22:21:52.873 INFO:tasks.workunit.client.0.smithi114.stdout:8/258: mknod d4/d16/d25/d39/c59 0 2024-02-16T22:21:52.874 INFO:tasks.workunit.client.0.smithi114.stdout:2/236: read - dd/df/d2c/f32 zero size 2024-02-16T22:21:52.875 INFO:tasks.workunit.client.0.smithi114.stdout:2/237: chown dd/l1a 1472455606 1 2024-02-16T22:21:52.875 INFO:tasks.workunit.client.1.smithi123.stdout:6/357: truncate d0/d9/dc/d5f/f6c 779085 0 2024-02-16T22:21:52.876 INFO:tasks.workunit.client.1.smithi123.stdout:8/405: dwrite d0/d12/f58 [0,4194304] 0 2024-02-16T22:21:52.877 INFO:tasks.workunit.client.1.smithi123.stdout:6/358: write d0/d9/dc/d5f/f3e [330639,22568] 0 2024-02-16T22:21:52.877 INFO:tasks.workunit.client.0.smithi114.stdout:5/221: mkdir d0/df/d16/d25/d2e/d3b 0 2024-02-16T22:21:52.878 INFO:tasks.workunit.client.1.smithi123.stdout:1/313: readlink d3/d6/db/d10/d2a/d53/l71 0 2024-02-16T22:21:52.879 INFO:tasks.workunit.client.1.smithi123.stdout:1/314: stat d3/d6/db/d10/d2a 0 2024-02-16T22:21:52.880 INFO:tasks.workunit.client.1.smithi123.stdout:4/355: write d5/f1d [1514451,48365] 0 2024-02-16T22:21:52.880 INFO:tasks.workunit.client.0.smithi114.stdout:8/259: dwrite d4/d11/d15/d31/f37 [0,4194304] 0 2024-02-16T22:21:52.881 INFO:tasks.workunit.client.0.smithi114.stdout:4/331: mkdir dd/d1b/d41/d74 0 2024-02-16T22:21:52.882 INFO:tasks.workunit.client.0.smithi114.stdout:8/260: write d4/d11/f51 [902763,63842] 0 2024-02-16T22:21:52.884 INFO:tasks.workunit.client.0.smithi114.stdout:8/261: write d4/d11/f51 [1998505,121529] 0 2024-02-16T22:21:52.885 INFO:tasks.workunit.client.0.smithi114.stdout:8/262: stat d4/d11/f20 0 2024-02-16T22:21:52.886 INFO:tasks.workunit.client.0.smithi114.stdout:1/207: mkdir d4/d9/d46 0 2024-02-16T22:21:52.886 INFO:tasks.workunit.client.0.smithi114.stdout:8/263: readlink d4/d11/d15/l12 0 2024-02-16T22:21:52.889 INFO:tasks.workunit.client.0.smithi114.stdout:1/208: dread d4/d9/f18 [0,4194304] 0 2024-02-16T22:21:52.890 INFO:tasks.workunit.client.1.smithi123.stdout:8/406: read d0/f14 [2465983,16268] 0 2024-02-16T22:21:52.891 INFO:tasks.workunit.client.1.smithi123.stdout:9/425: truncate d4/f12 6113933 0 2024-02-16T22:21:52.892 INFO:tasks.workunit.client.0.smithi114.stdout:2/238: unlink dd/df/d2c/l31 0 2024-02-16T22:21:52.895 INFO:tasks.workunit.client.0.smithi114.stdout:5/222: rmdir d0/d28 39 2024-02-16T22:21:52.897 INFO:tasks.workunit.client.1.smithi123.stdout:4/356: symlink d5/d1c/d23/d47/l6b 0 2024-02-16T22:21:52.898 INFO:tasks.workunit.client.0.smithi114.stdout:4/332: creat dd/d2e/d3f/f75 x:0 0 0 2024-02-16T22:21:52.901 INFO:tasks.workunit.client.0.smithi114.stdout:8/264: truncate d4/d24/f3d 4560411 0 2024-02-16T22:21:52.903 INFO:tasks.workunit.client.1.smithi123.stdout:8/407: creat d0/d17/d26/d68/d81/f85 x:0 0 0 2024-02-16T22:21:52.903 INFO:tasks.workunit.client.1.smithi123.stdout:4/357: dwrite d5/f1e [0,4194304] 0 2024-02-16T22:21:52.904 INFO:tasks.workunit.client.1.smithi123.stdout:0/309: creat d1/d8/f5d x:0 0 0 2024-02-16T22:21:52.908 INFO:tasks.workunit.client.0.smithi114.stdout:5/223: chown d0/d28/f37 1590998 1 2024-02-16T22:21:52.913 INFO:tasks.workunit.client.1.smithi123.stdout:9/426: mknod d4/d56/d6f/d7e/c83 0 2024-02-16T22:21:52.914 INFO:tasks.workunit.client.0.smithi114.stdout:4/333: dread dd/f20 [0,4194304] 0 2024-02-16T22:21:52.915 INFO:tasks.workunit.client.1.smithi123.stdout:4/358: creat d5/d1c/d42/f6c x:0 0 0 2024-02-16T22:21:52.916 INFO:tasks.workunit.client.0.smithi114.stdout:1/209: mkdir d4/d9/d46/d47 0 2024-02-16T22:21:52.917 INFO:tasks.workunit.client.1.smithi123.stdout:4/359: write d5/f4c [157798,75105] 0 2024-02-16T22:21:52.918 INFO:tasks.workunit.client.1.smithi123.stdout:8/408: creat d0/d61/d66/f86 x:0 0 0 2024-02-16T22:21:52.919 INFO:tasks.workunit.client.0.smithi114.stdout:4/334: dwrite dd/d1b/d24/f6b [0,4194304] 0 2024-02-16T22:21:52.920 INFO:tasks.workunit.client.0.smithi114.stdout:2/239: rename dd/df/d1c/c42 to dd/d3b/c53 0 2024-02-16T22:21:52.923 INFO:tasks.workunit.client.1.smithi123.stdout:8/409: dread d0/f6 [0,4194304] 0 2024-02-16T22:21:52.924 INFO:tasks.workunit.client.1.smithi123.stdout:8/410: readlink d0/d17/d26/d2a/d3c/l6e 0 2024-02-16T22:21:52.924 INFO:tasks.workunit.client.1.smithi123.stdout:9/427: mkdir d4/d56/d84 0 2024-02-16T22:21:52.928 INFO:tasks.workunit.client.1.smithi123.stdout:7/349: sync 2024-02-16T22:21:52.928 INFO:tasks.workunit.client.1.smithi123.stdout:0/310: truncate d1/f1f 280558 0 2024-02-16T22:21:52.932 INFO:tasks.workunit.client.1.smithi123.stdout:6/359: truncate d0/d9/f6a 2558966 0 2024-02-16T22:21:52.933 INFO:tasks.workunit.client.1.smithi123.stdout:2/305: dwrite d4/d1a/d1d/d59/d45/f4b [0,4194304] 0 2024-02-16T22:21:52.934 INFO:tasks.workunit.client.1.smithi123.stdout:2/306: chown d4/d1a/d1d/l10 23507248 1 2024-02-16T22:21:52.934 INFO:tasks.workunit.client.1.smithi123.stdout:7/350: symlink d3/d6e/d24/l74 0 2024-02-16T22:21:52.936 INFO:tasks.workunit.client.1.smithi123.stdout:8/411: rename d0/d17/d26/d68/d81/f85 to d0/d17/d26/d68/d81/f87 0 2024-02-16T22:21:52.936 INFO:tasks.workunit.client.1.smithi123.stdout:8/412: chown d0 7596 1 2024-02-16T22:21:52.939 INFO:tasks.workunit.client.1.smithi123.stdout:7/351: rename d3/d6e/d1d/d1e/f48 to d3/d6e/d34/f75 0 2024-02-16T22:21:52.939 INFO:tasks.workunit.client.1.smithi123.stdout:9/428: getdents d4/d17/d20 0 2024-02-16T22:21:52.940 INFO:tasks.workunit.client.1.smithi123.stdout:8/413: symlink d0/d61/d66/d7e/l88 0 2024-02-16T22:21:52.942 INFO:tasks.workunit.client.1.smithi123.stdout:7/352: mknod d3/c76 0 2024-02-16T22:21:52.944 INFO:tasks.workunit.client.1.smithi123.stdout:8/414: creat d0/d61/d66/d7e/f89 x:0 0 0 2024-02-16T22:21:52.945 INFO:tasks.workunit.client.1.smithi123.stdout:7/353: rename d3/d6e/f1a to d3/d26/d35/f77 0 2024-02-16T22:21:52.950 INFO:tasks.workunit.client.1.smithi123.stdout:8/415: rename d0/d61/d66/f69 to d0/d17/d26/d32/d45/d67/f8a 0 2024-02-16T22:21:52.952 INFO:tasks.workunit.client.1.smithi123.stdout:8/416: write d0/f6 [2995318,56380] 0 2024-02-16T22:21:52.954 INFO:tasks.workunit.client.1.smithi123.stdout:8/417: unlink d0/d12/f19 0 2024-02-16T22:21:52.958 INFO:tasks.workunit.client.1.smithi123.stdout:4/360: dread d5/d1c/d23/d31/f3e [0,4194304] 0 2024-02-16T22:21:52.960 INFO:tasks.workunit.client.1.smithi123.stdout:4/361: mkdir d5/d1c/d23/d6d 0 2024-02-16T22:21:52.961 INFO:tasks.workunit.client.1.smithi123.stdout:5/315: sync 2024-02-16T22:21:52.961 INFO:tasks.workunit.client.1.smithi123.stdout:3/296: sync 2024-02-16T22:21:52.961 INFO:tasks.workunit.client.0.smithi114.stdout:5/224: symlink d0/d10/l3c 0 2024-02-16T22:21:52.961 INFO:tasks.workunit.client.1.smithi123.stdout:7/354: dread f2 [0,4194304] 0 2024-02-16T22:21:52.963 INFO:tasks.workunit.client.0.smithi114.stdout:5/225: write d0/d10/f1a [136670,52178] 0 2024-02-16T22:21:52.963 INFO:tasks.workunit.client.1.smithi123.stdout:7/355: fdatasync d3/d6e/d52/f67 0 2024-02-16T22:21:52.964 INFO:tasks.workunit.client.1.smithi123.stdout:2/307: dread d4/d1a/d1d/d13/d25/f43 [0,4194304] 0 2024-02-16T22:21:52.966 INFO:tasks.workunit.client.1.smithi123.stdout:6/360: dread d0/f54 [0,4194304] 0 2024-02-16T22:21:52.967 INFO:tasks.workunit.client.1.smithi123.stdout:3/297: symlink d0/d50/l72 0 2024-02-16T22:21:52.968 INFO:tasks.workunit.client.0.smithi114.stdout:2/240: creat dd/d4e/f54 x:0 0 0 2024-02-16T22:21:52.970 INFO:tasks.workunit.client.0.smithi114.stdout:2/241: write dd/df/d1c/d21/f30 [962771,82267] 0 2024-02-16T22:21:52.971 INFO:tasks.workunit.client.0.smithi114.stdout:8/265: getdents d4/d24/d2c 0 2024-02-16T22:21:52.972 INFO:tasks.workunit.client.1.smithi123.stdout:8/418: dread d0/d12/f50 [0,4194304] 0 2024-02-16T22:21:52.974 INFO:tasks.workunit.client.1.smithi123.stdout:6/361: read - d0/d9/dc/d3a/f81 zero size 2024-02-16T22:21:52.979 INFO:tasks.workunit.client.0.smithi114.stdout:2/242: dwrite dd/df/f33 [0,4194304] 0 2024-02-16T22:21:52.979 INFO:tasks.workunit.client.0.smithi114.stdout:8/266: dwrite d4/d16/f1e [0,4194304] 0 2024-02-16T22:21:52.982 INFO:tasks.workunit.client.1.smithi123.stdout:4/362: dwrite d5/d1c/d23/f52 [0,4194304] 0 2024-02-16T22:21:52.982 INFO:tasks.workunit.client.1.smithi123.stdout:6/362: chown d0/d6/d1a/l4b 16182930 1 2024-02-16T22:21:52.986 INFO:tasks.workunit.client.1.smithi123.stdout:5/316: dwrite d6/dd/df/f62 [0,4194304] 0 2024-02-16T22:21:52.987 INFO:tasks.workunit.client.1.smithi123.stdout:7/356: dwrite d3/d6e/d34/f57 [0,4194304] 0 2024-02-16T22:21:52.987 INFO:tasks.workunit.client.1.smithi123.stdout:4/363: read - d5/d1c/d42/f6a zero size 2024-02-16T22:21:52.992 INFO:tasks.workunit.client.1.smithi123.stdout:3/298: dwrite d0/f19 [0,4194304] 0 2024-02-16T22:21:52.992 INFO:tasks.workunit.client.1.smithi123.stdout:5/317: chown d6/dd/df/d1f/l49 0 1 2024-02-16T22:21:52.994 INFO:tasks.workunit.client.1.smithi123.stdout:4/364: stat d5/d1c/d42/f4a 0 2024-02-16T22:21:52.995 INFO:tasks.workunit.client.0.smithi114.stdout:5/226: link d0/d10/c12 d0/df/c3d 0 2024-02-16T22:21:52.996 INFO:tasks.workunit.client.1.smithi123.stdout:7/357: dread d3/d5/f45 [0,4194304] 0 2024-02-16T22:21:53.002 INFO:tasks.workunit.client.0.smithi114.stdout:5/227: dread d0/fa [0,4194304] 0 2024-02-16T22:21:53.002 INFO:tasks.workunit.client.1.smithi123.stdout:1/315: sync 2024-02-16T22:21:53.003 INFO:tasks.workunit.client.0.smithi114.stdout:1/210: dwrite d4/d9/d3d/f39 [4194304,4194304] 0 2024-02-16T22:21:53.012 INFO:tasks.workunit.client.1.smithi123.stdout:0/311: truncate d1/f6 5653361 0 2024-02-16T22:21:53.013 INFO:tasks.workunit.client.1.smithi123.stdout:5/318: rmdir d6/d3b 39 2024-02-16T22:21:53.018 INFO:tasks.workunit.client.0.smithi114.stdout:1/211: write d4/f25 [4076805,44758] 0 2024-02-16T22:21:53.019 INFO:tasks.workunit.client.1.smithi123.stdout:9/429: truncate d4/d17/f55 2449905 0 2024-02-16T22:21:53.020 INFO:tasks.workunit.client.0.smithi114.stdout:1/212: truncate d4/d9/f20 945067 0 2024-02-16T22:21:53.020 INFO:tasks.workunit.client.1.smithi123.stdout:5/319: dread d6/dd/df/d58/f43 [0,4194304] 0 2024-02-16T22:21:53.021 INFO:tasks.workunit.client.0.smithi114.stdout:5/228: mkdir d0/df/d16/d25/d3e 0 2024-02-16T22:21:53.027 INFO:tasks.workunit.client.0.smithi114.stdout:1/213: dwrite d4/fb [0,4194304] 0 2024-02-16T22:21:53.027 INFO:tasks.workunit.client.1.smithi123.stdout:6/363: dread d0/d9/dc/f23 [0,4194304] 0 2024-02-16T22:21:53.028 INFO:tasks.workunit.client.0.smithi114.stdout:5/229: dread d0/df/d16/d1e/f29 [4194304,4194304] 0 2024-02-16T22:21:53.028 INFO:tasks.workunit.client.0.smithi114.stdout:2/243: rename dd/f51 to dd/f55 0 2024-02-16T22:21:53.028 INFO:tasks.workunit.client.0.smithi114.stdout:1/214: dread - d4/d9/d3d/f41 zero size 2024-02-16T22:21:53.029 INFO:tasks.workunit.client.0.smithi114.stdout:2/244: chown dd/df/f33 6468992 1 2024-02-16T22:21:53.030 INFO:tasks.workunit.client.1.smithi123.stdout:1/316: chown d3/d6/l36 1117 1 2024-02-16T22:21:53.030 INFO:tasks.workunit.client.0.smithi114.stdout:5/230: read d0/fb [1813178,36369] 0 2024-02-16T22:21:53.039 INFO:tasks.workunit.client.1.smithi123.stdout:0/312: symlink d1/d4/d42/d4b/l5e 0 2024-02-16T22:21:53.040 INFO:tasks.workunit.client.1.smithi123.stdout:0/313: truncate d1/d8/d13/f37 746546 0 2024-02-16T22:21:53.040 INFO:tasks.workunit.client.0.smithi114.stdout:1/215: dwrite d4/f25 [0,4194304] 0 2024-02-16T22:21:53.040 INFO:tasks.workunit.client.0.smithi114.stdout:4/335: truncate dd/d1b/d37/d38/f5e 1514007 0 2024-02-16T22:21:53.043 INFO:tasks.workunit.client.1.smithi123.stdout:3/299: symlink d0/d20/d29/d34/d5d/d60/l73 0 2024-02-16T22:21:53.045 INFO:tasks.workunit.client.0.smithi114.stdout:5/231: mkdir d0/df/d16/d1e/d3f 0 2024-02-16T22:21:53.046 INFO:tasks.workunit.client.1.smithi123.stdout:4/365: mkdir d5/d1c/d23/d56/d6e 0 2024-02-16T22:21:53.047 INFO:tasks.workunit.client.1.smithi123.stdout:4/366: chown d5/d1c/d42/f4a 2285979 1 2024-02-16T22:21:53.048 INFO:tasks.workunit.client.1.smithi123.stdout:9/430: creat d4/f85 x:0 0 0 2024-02-16T22:21:53.048 INFO:tasks.workunit.client.1.smithi123.stdout:2/308: write d4/d1a/d1d/d13/d25/f3b [2432879,10991] 0 2024-02-16T22:21:53.049 INFO:tasks.workunit.client.0.smithi114.stdout:1/216: creat d4/d9/d46/f48 x:0 0 0 2024-02-16T22:21:53.050 INFO:tasks.workunit.client.1.smithi123.stdout:3/300: dwrite d0/d20/d29/d34/d5d/f61 [0,4194304] 0 2024-02-16T22:21:53.051 INFO:tasks.workunit.client.1.smithi123.stdout:5/320: rename d6/dd/c30 to d6/dd/df/d1f/d25/c69 0 2024-02-16T22:21:53.052 INFO:tasks.workunit.client.1.smithi123.stdout:5/321: dread - d6/dd/df/d1f/f64 zero size 2024-02-16T22:21:53.053 INFO:tasks.workunit.client.1.smithi123.stdout:8/419: link d0/c9 d0/d17/d26/d68/c8b 0 2024-02-16T22:21:53.054 INFO:tasks.workunit.client.1.smithi123.stdout:3/301: dread d0/ff [0,4194304] 0 2024-02-16T22:21:53.055 INFO:tasks.workunit.client.0.smithi114.stdout:8/267: write d4/d24/d2c/f32 [47960,27250] 0 2024-02-16T22:21:53.058 INFO:tasks.workunit.client.0.smithi114.stdout:1/217: dwrite d4/d9/f11 [4194304,4194304] 0 2024-02-16T22:21:53.059 INFO:tasks.workunit.client.0.smithi114.stdout:1/218: stat d4/d33/c42 0 2024-02-16T22:21:53.060 INFO:tasks.workunit.client.0.smithi114.stdout:1/219: readlink d4/d9/d3d/l2b 0 2024-02-16T22:21:53.060 INFO:tasks.workunit.client.1.smithi123.stdout:6/364: mkdir d0/d9/dc/d83 0 2024-02-16T22:21:53.065 INFO:tasks.workunit.client.0.smithi114.stdout:8/268: dread d4/d11/d15/d31/f37 [0,4194304] 0 2024-02-16T22:21:53.065 INFO:tasks.workunit.client.1.smithi123.stdout:0/314: truncate d1/d8/d13/f15 3269293 0 2024-02-16T22:21:53.066 INFO:tasks.workunit.client.1.smithi123.stdout:0/315: write d1/d8/d3d/f59 [1028719,95093] 0 2024-02-16T22:21:53.068 INFO:tasks.workunit.client.0.smithi114.stdout:5/232: dwrite d0/d10/f23 [4194304,4194304] 0 2024-02-16T22:21:53.071 INFO:tasks.workunit.client.0.smithi114.stdout:5/233: write d0/df/d16/d1e/f27 [908738,89269] 0 2024-02-16T22:21:53.072 INFO:tasks.workunit.client.1.smithi123.stdout:2/309: symlink d4/l65 0 2024-02-16T22:21:53.073 INFO:tasks.workunit.client.0.smithi114.stdout:4/336: creat dd/d1b/d24/f76 x:0 0 0 2024-02-16T22:21:53.074 INFO:tasks.workunit.client.0.smithi114.stdout:8/269: mknod d4/d11/d15/d36/d3b/c5a 0 2024-02-16T22:21:53.078 INFO:tasks.workunit.client.0.smithi114.stdout:8/270: dread d4/d16/f1e [0,4194304] 0 2024-02-16T22:21:53.079 INFO:tasks.workunit.client.0.smithi114.stdout:2/245: rename c5 to dd/df/d1c/c56 0 2024-02-16T22:21:53.081 INFO:tasks.workunit.client.1.smithi123.stdout:1/317: rename d3/d6/d18/l57 to d3/d6/db/d55/l74 0 2024-02-16T22:21:53.082 INFO:tasks.workunit.client.1.smithi123.stdout:1/318: write d3/f4e [1038356,85733] 0 2024-02-16T22:21:53.086 INFO:tasks.workunit.client.1.smithi123.stdout:3/302: rmdir d0/d20/d29/d34 39 2024-02-16T22:21:53.089 INFO:tasks.workunit.client.0.smithi114.stdout:4/337: mkdir dd/d1b/d41/d77 0 2024-02-16T22:21:53.093 INFO:tasks.workunit.client.0.smithi114.stdout:8/271: rmdir d4/d11/d15 39 2024-02-16T22:21:53.095 INFO:tasks.workunit.client.0.smithi114.stdout:2/246: rmdir dd/df/d1c/d21 39 2024-02-16T22:21:53.096 INFO:tasks.workunit.client.1.smithi123.stdout:4/367: mknod d5/d1c/d64/c6f 0 2024-02-16T22:21:53.098 INFO:tasks.workunit.client.1.smithi123.stdout:9/431: symlink d4/d17/d2b/d6c/l86 0 2024-02-16T22:21:53.100 INFO:tasks.workunit.client.0.smithi114.stdout:4/338: rmdir dd/d2e/d56 39 2024-02-16T22:21:53.104 INFO:tasks.workunit.client.1.smithi123.stdout:2/310: rename d4/d1a/f39 to d4/d1a/d1d/d13/d25/f66 0 2024-02-16T22:21:53.105 INFO:tasks.workunit.client.0.smithi114.stdout:2/247: mknod dd/d4e/c57 0 2024-02-16T22:21:53.106 INFO:tasks.workunit.client.0.smithi114.stdout:1/220: link d4/l8 d4/d9/d46/l49 0 2024-02-16T22:21:53.109 INFO:tasks.workunit.client.0.smithi114.stdout:4/339: creat dd/f78 x:0 0 0 2024-02-16T22:21:53.109 INFO:tasks.workunit.client.0.smithi114.stdout:2/248: dread dd/f36 [0,4194304] 0 2024-02-16T22:21:53.110 INFO:tasks.workunit.client.0.smithi114.stdout:2/249: chown dd/d3c 26504 1 2024-02-16T22:21:53.110 INFO:tasks.workunit.client.0.smithi114.stdout:4/340: dread - dd/d2e/d3f/f75 zero size 2024-02-16T22:21:53.110 INFO:tasks.workunit.client.0.smithi114.stdout:8/272: mkdir d4/d11/d15/d31/d5b 0 2024-02-16T22:21:53.111 INFO:tasks.workunit.client.0.smithi114.stdout:8/273: chown d4/d11/d15/d36/d3b/l41 5 1 2024-02-16T22:21:53.113 INFO:tasks.workunit.client.1.smithi123.stdout:7/358: getdents d3/d6e/d4d 0 2024-02-16T22:21:53.114 INFO:tasks.workunit.client.1.smithi123.stdout:7/359: read d3/d6e/d1d/d1e/f5f [170446,1584] 0 2024-02-16T22:21:53.116 INFO:tasks.workunit.client.0.smithi114.stdout:2/250: creat dd/d4e/f58 x:0 0 0 2024-02-16T22:21:53.116 INFO:tasks.workunit.client.1.smithi123.stdout:5/322: write d6/dd/df/d1f/d25/d37/f3e [690635,44142] 0 2024-02-16T22:21:53.117 INFO:tasks.workunit.client.1.smithi123.stdout:5/323: write d6/dd/df/d1f/d25/f1a [3416096,58384] 0 2024-02-16T22:21:53.121 INFO:tasks.workunit.client.0.smithi114.stdout:5/234: dwrite d0/f8 [0,4194304] 0 2024-02-16T22:21:53.123 INFO:tasks.workunit.client.0.smithi114.stdout:8/274: rename d4/l22 to d4/d24/d2c/l5c 0 2024-02-16T22:21:53.124 INFO:tasks.workunit.client.0.smithi114.stdout:8/275: write d4/f1d [2703970,115311] 0 2024-02-16T22:21:53.125 INFO:tasks.workunit.client.0.smithi114.stdout:1/221: mkdir d4/d4a 0 2024-02-16T22:21:53.125 INFO:tasks.workunit.client.0.smithi114.stdout:8/276: write d4/d24/f3e [156261,112946] 0 2024-02-16T22:21:53.127 INFO:tasks.workunit.client.0.smithi114.stdout:8/277: fdatasync d4/d24/f3e 0 2024-02-16T22:21:53.129 INFO:tasks.workunit.client.0.smithi114.stdout:4/341: creat dd/d1b/d41/d74/f79 x:0 0 0 2024-02-16T22:21:53.133 INFO:tasks.workunit.client.0.smithi114.stdout:5/235: creat d0/d2d/d30/f40 x:0 0 0 2024-02-16T22:21:53.133 INFO:tasks.workunit.client.0.smithi114.stdout:2/251: dread dd/df/f2f [0,4194304] 0 2024-02-16T22:21:53.134 INFO:tasks.workunit.client.1.smithi123.stdout:8/420: creat d0/d17/d26/f8c x:0 0 0 2024-02-16T22:21:53.134 INFO:tasks.workunit.client.1.smithi123.stdout:8/421: chown d0/df 53593 1 2024-02-16T22:21:53.135 INFO:tasks.workunit.client.1.smithi123.stdout:8/422: fsync d0/d29/f64 0 2024-02-16T22:21:53.137 INFO:tasks.workunit.client.1.smithi123.stdout:3/303: mknod d0/d20/c74 0 2024-02-16T22:21:53.139 INFO:tasks.workunit.client.1.smithi123.stdout:0/316: truncate d1/f22 2739158 0 2024-02-16T22:21:53.139 INFO:tasks.workunit.client.0.smithi114.stdout:1/222: write d4/f34 [4948739,23002] 0 2024-02-16T22:21:53.140 INFO:tasks.workunit.client.1.smithi123.stdout:8/423: dread d0/d17/d26/d32/d45/d67/f8a [0,4194304] 0 2024-02-16T22:21:53.140 INFO:tasks.workunit.client.1.smithi123.stdout:8/424: readlink d0/l1a 0 2024-02-16T22:21:53.141 INFO:tasks.workunit.client.1.smithi123.stdout:8/425: write d0/d17/d26/f7a [661902,101571] 0 2024-02-16T22:21:53.142 INFO:tasks.workunit.client.1.smithi123.stdout:1/319: unlink d3/d6/db/d55/l74 0 2024-02-16T22:21:53.144 INFO:tasks.workunit.client.1.smithi123.stdout:6/365: rmdir d0/d9/dc/d83 0 2024-02-16T22:21:53.145 INFO:tasks.workunit.client.0.smithi114.stdout:8/278: rmdir d4/d16/d25/d46 39 2024-02-16T22:21:53.146 INFO:tasks.workunit.client.1.smithi123.stdout:5/324: creat d6/dd/df/d1f/d25/f6a x:0 0 0 2024-02-16T22:21:53.147 INFO:tasks.workunit.client.1.smithi123.stdout:4/368: mknod d5/d1c/c70 0 2024-02-16T22:21:53.149 INFO:tasks.workunit.client.1.smithi123.stdout:9/432: mkdir d4/d17/d87 0 2024-02-16T22:21:53.150 INFO:tasks.workunit.client.1.smithi123.stdout:9/433: fsync d4/d33/f73 0 2024-02-16T22:21:53.151 INFO:tasks.workunit.client.1.smithi123.stdout:2/311: dwrite d4/d1a/d1d/d13/d15/f42 [0,4194304] 0 2024-02-16T22:21:53.155 INFO:tasks.workunit.client.1.smithi123.stdout:0/317: symlink d1/d8/d13/d1d/l5f 0 2024-02-16T22:21:53.155 INFO:tasks.workunit.client.0.smithi114.stdout:8/279: creat d4/d11/d15/d36/f5d x:0 0 0 2024-02-16T22:21:53.156 INFO:tasks.workunit.client.1.smithi123.stdout:0/318: read - d1/d4/d17/f53 zero size 2024-02-16T22:21:53.159 INFO:tasks.workunit.client.0.smithi114.stdout:3/204: sync 2024-02-16T22:21:53.159 INFO:tasks.workunit.client.0.smithi114.stdout:8/280: dread d4/d24/f3e [0,4194304] 0 2024-02-16T22:21:53.161 INFO:tasks.workunit.client.0.smithi114.stdout:3/205: truncate d4/d26/f3c 722666 0 2024-02-16T22:21:53.161 INFO:tasks.workunit.client.0.smithi114.stdout:8/281: stat d4/d24/f50 0 2024-02-16T22:21:53.161 INFO:tasks.workunit.client.0.smithi114.stdout:8/282: chown d4/d16/d25/d39 121908597 1 2024-02-16T22:21:53.162 INFO:tasks.workunit.client.1.smithi123.stdout:1/320: mknod d3/d6/d11/c75 0 2024-02-16T22:21:53.164 INFO:tasks.workunit.client.1.smithi123.stdout:4/369: unlink d5/d1c/d23/d47/c5d 0 2024-02-16T22:21:53.165 INFO:tasks.workunit.client.0.smithi114.stdout:1/223: fdatasync d4/f1d 0 2024-02-16T22:21:53.166 INFO:tasks.workunit.client.0.smithi114.stdout:4/342: creat dd/d1b/d24/f7a x:0 0 0 2024-02-16T22:21:53.167 INFO:tasks.workunit.client.0.smithi114.stdout:4/343: readlink dd/d1b/d24/l25 0 2024-02-16T22:21:53.167 INFO:tasks.workunit.client.0.smithi114.stdout:2/252: creat dd/df/f59 x:0 0 0 2024-02-16T22:21:53.168 INFO:tasks.workunit.client.0.smithi114.stdout:4/344: dread - dd/d1b/d24/f70 zero size 2024-02-16T22:21:53.168 INFO:tasks.workunit.client.0.smithi114.stdout:2/253: chown dd/f28 15759 1 2024-02-16T22:21:53.171 INFO:tasks.workunit.client.0.smithi114.stdout:3/206: rename d4/d24/d2d/l37 to d4/d24/d2d/d45/l4b 0 2024-02-16T22:21:53.176 INFO:tasks.workunit.client.0.smithi114.stdout:5/236: rmdir d0/df/d16/d1e/d3f 0 2024-02-16T22:21:53.180 INFO:tasks.workunit.client.1.smithi123.stdout:8/426: rename d0/df/c30 to d0/d17/d26/d68/d81/d47/c8d 0 2024-02-16T22:21:53.181 INFO:tasks.workunit.client.1.smithi123.stdout:8/427: write d0/df/f16 [3436667,26688] 0 2024-02-16T22:21:53.184 INFO:tasks.workunit.client.0.smithi114.stdout:2/254: creat dd/df/d2c/f5a x:0 0 0 2024-02-16T22:21:53.185 INFO:tasks.workunit.client.1.smithi123.stdout:9/434: dwrite d4/d17/d24/f4a [0,4194304] 0 2024-02-16T22:21:53.186 INFO:tasks.workunit.client.1.smithi123.stdout:9/435: stat d4/f85 0 2024-02-16T22:21:53.186 INFO:tasks.workunit.client.0.smithi114.stdout:3/207: unlink d4/d26/f28 0 2024-02-16T22:21:53.187 INFO:tasks.workunit.client.1.smithi123.stdout:9/436: chown d4/d1e/c2c 1 1 2024-02-16T22:21:53.188 INFO:tasks.workunit.client.0.smithi114.stdout:8/283: dwrite d4/d11/d15/d31/f37 [0,4194304] 0 2024-02-16T22:21:53.188 INFO:tasks.workunit.client.1.smithi123.stdout:4/370: mkdir d5/d49/d71 0 2024-02-16T22:21:53.189 INFO:tasks.workunit.client.1.smithi123.stdout:3/304: dwrite d0/d20/d29/d34/f4e [0,4194304] 0 2024-02-16T22:21:53.189 INFO:tasks.workunit.client.0.smithi114.stdout:8/284: readlink d4/d11/d15/l12 0 2024-02-16T22:21:53.189 INFO:tasks.workunit.client.0.smithi114.stdout:5/237: mknod d0/df/d16/d25/d2e/c41 0 2024-02-16T22:21:53.191 INFO:tasks.workunit.client.0.smithi114.stdout:1/224: creat d4/d9/d46/d47/f4b x:0 0 0 2024-02-16T22:21:53.191 INFO:tasks.workunit.client.0.smithi114.stdout:1/225: stat d4/d9/d3d/d3f 0 2024-02-16T22:21:53.192 INFO:tasks.workunit.client.0.smithi114.stdout:1/226: write d4/d9/f20 [94274,124842] 0 2024-02-16T22:21:53.193 INFO:tasks.workunit.client.0.smithi114.stdout:4/345: symlink dd/d1b/d24/l7b 0 2024-02-16T22:21:53.193 INFO:tasks.workunit.client.0.smithi114.stdout:1/227: chown d4/d9/c31 1977 1 2024-02-16T22:21:53.196 INFO:tasks.workunit.client.0.smithi114.stdout:2/255: fdatasync dd/df/d2c/f34 0 2024-02-16T22:21:53.197 INFO:tasks.workunit.client.1.smithi123.stdout:2/312: truncate d4/d1a/d1d/f9 4153517 0 2024-02-16T22:21:53.198 INFO:tasks.workunit.client.1.smithi123.stdout:3/305: dwrite d0/f58 [0,4194304] 0 2024-02-16T22:21:53.198 INFO:tasks.workunit.client.0.smithi114.stdout:1/228: dread d4/f16 [0,4194304] 0 2024-02-16T22:21:53.200 INFO:tasks.workunit.client.1.smithi123.stdout:2/313: chown d4/d1a/d1d/d59/d45/f4b 375819 1 2024-02-16T22:21:53.200 INFO:tasks.workunit.client.0.smithi114.stdout:1/229: chown d4/d33/f38 6937 1 2024-02-16T22:21:53.200 INFO:tasks.workunit.client.1.smithi123.stdout:0/319: mknod d1/c60 0 2024-02-16T22:21:53.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:52 smithi114 ceph-mon[29235]: pgmap v24: 65 pgs: 65 active+clean; 1.3 GiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 20 MiB/s rd, 180 MiB/s wr, 367 op/s 2024-02-16T22:21:53.203 INFO:tasks.workunit.client.0.smithi114.stdout:5/238: creat d0/d10/f42 x:0 0 0 2024-02-16T22:21:53.205 INFO:tasks.workunit.client.0.smithi114.stdout:2/256: dwrite dd/df/f41 [0,4194304] 0 2024-02-16T22:21:53.208 INFO:tasks.workunit.client.0.smithi114.stdout:4/346: write dd/d1b/d37/f73 [2982113,122990] 0 2024-02-16T22:21:53.208 INFO:tasks.workunit.client.1.smithi123.stdout:0/320: dwrite d1/d4/f43 [0,4194304] 0 2024-02-16T22:21:53.209 INFO:tasks.workunit.client.1.smithi123.stdout:5/325: creat d6/dd/df/d1f/f6b x:0 0 0 2024-02-16T22:21:53.209 INFO:tasks.workunit.client.1.smithi123.stdout:5/326: chown d6/dd/l48 135235024 1 2024-02-16T22:21:53.209 INFO:tasks.workunit.client.0.smithi114.stdout:1/230: dread d4/f25 [4194304,4194304] 0 2024-02-16T22:21:53.210 INFO:tasks.workunit.client.1.smithi123.stdout:9/437: rmdir d4/d1e/d3a 39 2024-02-16T22:21:53.212 INFO:tasks.workunit.client.0.smithi114.stdout:3/208: symlink d4/d24/l4c 0 2024-02-16T22:21:53.217 INFO:tasks.workunit.client.0.smithi114.stdout:5/239: unlink d0/f24 0 2024-02-16T22:21:53.218 INFO:tasks.workunit.client.0.smithi114.stdout:5/240: write d0/f8 [459317,106862] 0 2024-02-16T22:21:53.219 INFO:tasks.workunit.client.1.smithi123.stdout:6/366: rename d0/l78 to d0/d9/dc/d5f/l84 0 2024-02-16T22:21:53.220 INFO:tasks.workunit.client.0.smithi114.stdout:5/241: write d0/f9 [741972,115149] 0 2024-02-16T22:21:53.221 INFO:tasks.workunit.client.0.smithi114.stdout:5/242: fdatasync d0/d2d/d30/f40 0 2024-02-16T22:21:53.227 INFO:tasks.workunit.client.1.smithi123.stdout:0/321: mknod d1/d4/d17/c61 0 2024-02-16T22:21:53.227 INFO:tasks.workunit.client.1.smithi123.stdout:0/322: chown d1/d8/d13/d1d 198 1 2024-02-16T22:21:53.228 INFO:tasks.workunit.client.0.smithi114.stdout:5/243: dwrite d0/d10/f1a [0,4194304] 0 2024-02-16T22:21:53.228 INFO:tasks.workunit.client.0.smithi114.stdout:3/209: readlink d4/d24/d2d/d45/l4b 0 2024-02-16T22:21:53.228 INFO:tasks.workunit.client.1.smithi123.stdout:0/323: chown d1/d4/d17/c40 0 1 2024-02-16T22:21:53.231 INFO:tasks.workunit.client.1.smithi123.stdout:2/314: creat d4/d1a/d1d/d38/d63/f67 x:0 0 0 2024-02-16T22:21:53.232 INFO:tasks.workunit.client.1.smithi123.stdout:3/306: fsync d0/d20/d29/d34/f4e 0 2024-02-16T22:21:53.236 INFO:tasks.workunit.client.1.smithi123.stdout:8/428: rename d0/d17/d26/d2a/d3c/l6e to d0/d17/d26/d32/d45/d67/l8e 0 2024-02-16T22:21:53.237 INFO:tasks.workunit.client.1.smithi123.stdout:1/321: getdents d3/d6/db/d10/d4f 0 2024-02-16T22:21:53.239 INFO:tasks.workunit.client.0.smithi114.stdout:2/257: mknod dd/df/c5b 0 2024-02-16T22:21:53.240 INFO:tasks.workunit.client.1.smithi123.stdout:6/367: dwrite d0/f6d [0,4194304] 0 2024-02-16T22:21:53.248 INFO:tasks.workunit.client.1.smithi123.stdout:2/315: chown d4/d1a/d1d/d59/d45/l62 4021877 1 2024-02-16T22:21:53.249 INFO:tasks.workunit.client.1.smithi123.stdout:2/316: write d4/d1a/d1d/d59/f3a [2527066,54434] 0 2024-02-16T22:21:53.252 INFO:tasks.workunit.client.1.smithi123.stdout:3/307: mkdir d0/d50/d75 0 2024-02-16T22:21:53.252 INFO:tasks.workunit.client.0.smithi114.stdout:2/258: dread dd/f17 [0,4194304] 0 2024-02-16T22:21:53.252 INFO:tasks.workunit.client.0.smithi114.stdout:1/231: dread d4/d9/f10 [0,4194304] 0 2024-02-16T22:21:53.252 INFO:tasks.workunit.client.0.smithi114.stdout:1/232: readlink l3 0 2024-02-16T22:21:53.253 INFO:tasks.workunit.client.1.smithi123.stdout:3/308: dread d0/d20/d29/f3c [0,4194304] 0 2024-02-16T22:21:53.255 INFO:tasks.workunit.client.1.smithi123.stdout:7/360: sync 2024-02-16T22:21:53.256 INFO:tasks.workunit.client.1.smithi123.stdout:8/429: mkdir d0/d17/d26/d68/d81/d8f 0 2024-02-16T22:21:53.259 INFO:tasks.workunit.client.1.smithi123.stdout:6/368: chown d0/d9/dc/l3d 0 1 2024-02-16T22:21:53.261 INFO:tasks.workunit.client.0.smithi114.stdout:1/233: rmdir d4/d9 39 2024-02-16T22:21:53.267 INFO:tasks.workunit.client.1.smithi123.stdout:5/327: truncate d6/dd/df/f62 3722318 0 2024-02-16T22:21:53.268 INFO:tasks.workunit.client.0.smithi114.stdout:1/234: stat d4/d9/d46/l49 0 2024-02-16T22:21:53.269 INFO:tasks.workunit.client.1.smithi123.stdout:0/324: write d1/d8/f2f [744099,50033] 0 2024-02-16T22:21:53.269 INFO:tasks.workunit.client.0.smithi114.stdout:4/347: write dd/d1b/d37/d38/f5e [1596284,2393] 0 2024-02-16T22:21:53.269 INFO:tasks.workunit.client.0.smithi114.stdout:2/259: link dd/df/f33 dd/d3b/f5c 0 2024-02-16T22:21:53.270 INFO:tasks.workunit.client.0.smithi114.stdout:4/348: chown dd/d1b/d41/d74 1702 1 2024-02-16T22:21:53.272 INFO:tasks.workunit.client.0.smithi114.stdout:4/349: truncate dd/d2e/f6a 237836 0 2024-02-16T22:21:53.272 INFO:tasks.workunit.client.0.smithi114.stdout:1/235: rmdir d4/d9/d3d 39 2024-02-16T22:21:53.273 INFO:tasks.workunit.client.0.smithi114.stdout:3/210: truncate d4/d24/f25 314326 0 2024-02-16T22:21:53.274 INFO:tasks.workunit.client.0.smithi114.stdout:3/211: chown d4/d26/f34 1051 1 2024-02-16T22:21:53.274 INFO:tasks.workunit.client.1.smithi123.stdout:8/430: mknod d0/d61/d66/d7e/c90 0 2024-02-16T22:21:53.274 INFO:tasks.workunit.client.0.smithi114.stdout:3/212: write d4/d26/f3c [135538,117758] 0 2024-02-16T22:21:53.275 INFO:tasks.workunit.client.1.smithi123.stdout:8/431: write d0/d17/d43/d34/f38 [10084,49104] 0 2024-02-16T22:21:53.278 INFO:tasks.workunit.client.1.smithi123.stdout:6/369: write d0/d9/dc/d39/f44 [1083535,9013] 0 2024-02-16T22:21:53.278 INFO:tasks.workunit.client.1.smithi123.stdout:6/370: readlink d0/d9/l31 0 2024-02-16T22:21:53.281 INFO:tasks.workunit.client.1.smithi123.stdout:2/317: rename d4/d1a/l1f to d4/d1a/d1d/d13/l68 0 2024-02-16T22:21:53.282 INFO:tasks.workunit.client.1.smithi123.stdout:7/361: write d3/d6e/d1d/f22 [764781,56148] 0 2024-02-16T22:21:53.283 INFO:tasks.workunit.client.1.smithi123.stdout:7/362: chown d3/d6e/d1d/d1e/f65 29424 1 2024-02-16T22:21:53.284 INFO:tasks.workunit.client.0.smithi114.stdout:3/213: creat d4/d26/f4d x:0 0 0 2024-02-16T22:21:53.286 INFO:tasks.workunit.client.0.smithi114.stdout:1/236: creat d4/d4a/f4c x:0 0 0 2024-02-16T22:21:53.287 INFO:tasks.workunit.client.0.smithi114.stdout:1/237: dread - d4/d33/f45 zero size 2024-02-16T22:21:53.287 INFO:tasks.workunit.client.1.smithi123.stdout:0/325: mkdir d1/d8/d13/d1d/d62 0 2024-02-16T22:21:53.289 INFO:tasks.workunit.client.0.smithi114.stdout:3/214: rename d4/dd/f15 to d4/d24/d2d/f4e 0 2024-02-16T22:21:53.290 INFO:tasks.workunit.client.0.smithi114.stdout:2/260: truncate dd/df/f52 1343375 0 2024-02-16T22:21:53.302 INFO:tasks.workunit.client.0.smithi114.stdout:3/215: symlink d4/d24/l4f 0 2024-02-16T22:21:53.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:52 smithi123 ceph-mon[37161]: pgmap v24: 65 pgs: 65 active+clean; 1.3 GiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 20 MiB/s rd, 180 MiB/s wr, 367 op/s 2024-02-16T22:21:53.303 INFO:tasks.workunit.client.1.smithi123.stdout:5/328: fsync d6/dd/df/d1f/f2e 0 2024-02-16T22:21:53.304 INFO:tasks.workunit.client.1.smithi123.stdout:5/329: dread - d6/dd/d44/f61 zero size 2024-02-16T22:21:53.304 INFO:tasks.workunit.client.1.smithi123.stdout:6/371: write d0/d6/d1a/f36 [1015425,88868] 0 2024-02-16T22:21:53.305 INFO:tasks.workunit.client.0.smithi114.stdout:1/238: dwrite d4/f16 [0,4194304] 0 2024-02-16T22:21:53.306 INFO:tasks.workunit.client.1.smithi123.stdout:7/363: rename d3/d6e/d1d/d1e/f5f to d3/d6e/d1d/d1e/f78 0 2024-02-16T22:21:53.307 INFO:tasks.workunit.client.1.smithi123.stdout:7/364: stat d3/d5/c7 0 2024-02-16T22:21:53.309 INFO:tasks.workunit.client.0.smithi114.stdout:3/216: fsync d4/d24/d2d/f4e 0 2024-02-16T22:21:53.310 INFO:tasks.workunit.client.1.smithi123.stdout:6/372: dwrite d0/d9/dc/d5f/f2c [0,4194304] 0 2024-02-16T22:21:53.310 INFO:tasks.workunit.client.1.smithi123.stdout:0/326: symlink d1/d8/d3d/l63 0 2024-02-16T22:21:53.312 INFO:tasks.workunit.client.0.smithi114.stdout:2/261: creat dd/f5d x:0 0 0 2024-02-16T22:21:53.312 INFO:tasks.workunit.client.1.smithi123.stdout:1/322: getdents d3/d6/db/d10 0 2024-02-16T22:21:53.313 INFO:tasks.workunit.client.1.smithi123.stdout:6/373: stat d0/d9/dc/d5f/f2c 0 2024-02-16T22:21:53.313 INFO:tasks.workunit.client.0.smithi114.stdout:2/262: truncate dd/df/d2c/f32 971236 0 2024-02-16T22:21:53.313 INFO:tasks.workunit.client.0.smithi114.stdout:3/217: creat d4/dd/d33/f50 x:0 0 0 2024-02-16T22:21:53.314 INFO:tasks.workunit.client.0.smithi114.stdout:3/218: readlink d4/dd/l20 0 2024-02-16T22:21:53.314 INFO:tasks.workunit.client.1.smithi123.stdout:6/374: dread - d0/d6/d2f/d5d/f7e zero size 2024-02-16T22:21:53.315 INFO:tasks.workunit.client.0.smithi114.stdout:2/263: write dd/df/d2c/f5a [359401,120937] 0 2024-02-16T22:21:53.315 INFO:tasks.workunit.client.1.smithi123.stdout:2/318: fdatasync d4/d1a/d1d/d13/d25/f66 0 2024-02-16T22:21:53.316 INFO:tasks.workunit.client.1.smithi123.stdout:0/327: dread d1/d8/fa [0,4194304] 0 2024-02-16T22:21:53.319 INFO:tasks.workunit.client.1.smithi123.stdout:4/371: sync 2024-02-16T22:21:53.319 INFO:tasks.workunit.client.1.smithi123.stdout:0/328: truncate d1/d8/d3d/d44/f5c 359304 0 2024-02-16T22:21:53.319 INFO:tasks.workunit.client.1.smithi123.stdout:3/309: rename d0/d20/d29/d34 to d0/d20/d76 0 2024-02-16T22:21:53.321 INFO:tasks.workunit.client.0.smithi114.stdout:1/239: getdents d4/d9/d46/d47 0 2024-02-16T22:21:53.322 INFO:tasks.workunit.client.0.smithi114.stdout:1/240: read d4/f34 [339113,18036] 0 2024-02-16T22:21:53.324 INFO:tasks.workunit.client.1.smithi123.stdout:4/372: dwrite d5/d1c/d23/d36/f55 [0,4194304] 0 2024-02-16T22:21:53.326 INFO:tasks.workunit.client.1.smithi123.stdout:7/365: mknod d3/d26/c79 0 2024-02-16T22:21:53.329 INFO:tasks.workunit.client.1.smithi123.stdout:2/319: mknod d4/d1a/d1d/d13/d15/d1b/d4a/c69 0 2024-02-16T22:21:53.330 INFO:tasks.workunit.client.1.smithi123.stdout:2/320: chown d4/d1a/d1d/l10 370419 1 2024-02-16T22:21:53.331 INFO:tasks.workunit.client.0.smithi114.stdout:1/241: dwrite d4/d4a/f4c [0,4194304] 0 2024-02-16T22:21:53.334 INFO:tasks.workunit.client.1.smithi123.stdout:0/329: mkdir d1/d8/d13/d1d/d64 0 2024-02-16T22:21:53.337 INFO:tasks.workunit.client.0.smithi114.stdout:1/242: rename d4/d4a to d4/d9/d46/d4d 0 2024-02-16T22:21:53.339 INFO:tasks.workunit.client.1.smithi123.stdout:8/432: rename d0/d17/d26/d2a/d3c/f80 to d0/d17/d26/d68/d81/d47/f91 0 2024-02-16T22:21:53.339 INFO:tasks.workunit.client.0.smithi114.stdout:3/219: dread d4/f18 [0,4194304] 0 2024-02-16T22:21:53.340 INFO:tasks.workunit.client.1.smithi123.stdout:6/375: symlink d0/d9/l85 0 2024-02-16T22:21:53.341 INFO:tasks.workunit.client.1.smithi123.stdout:3/310: mknod d0/d20/d76/d5d/d60/c77 0 2024-02-16T22:21:53.343 INFO:tasks.workunit.client.1.smithi123.stdout:7/366: creat d3/d59/f7a x:0 0 0 2024-02-16T22:21:53.345 INFO:tasks.workunit.client.1.smithi123.stdout:7/367: dread d3/d6e/d24/f56 [0,4194304] 0 2024-02-16T22:21:53.347 INFO:tasks.workunit.client.1.smithi123.stdout:5/330: rename d6/dd/df/d1f/d25/f6a to d6/dd/df/f6c 0 2024-02-16T22:21:53.349 INFO:tasks.workunit.client.1.smithi123.stdout:6/376: unlink d0/d9/dc/d5f/l6e 0 2024-02-16T22:21:53.350 INFO:tasks.workunit.client.1.smithi123.stdout:4/373: getdents d5/d49/d71 0 2024-02-16T22:21:53.350 INFO:tasks.workunit.client.1.smithi123.stdout:4/374: stat d5/f59 0 2024-02-16T22:21:53.352 INFO:tasks.workunit.client.1.smithi123.stdout:3/311: mknod d0/d40/c78 0 2024-02-16T22:21:53.354 INFO:tasks.workunit.client.1.smithi123.stdout:0/330: link d1/d8/f2f d1/d4/d42/f65 0 2024-02-16T22:21:53.354 INFO:tasks.workunit.client.1.smithi123.stdout:9/438: sync 2024-02-16T22:21:53.355 INFO:tasks.workunit.client.0.smithi114.stdout:3/220: getdents d4/d24/d2d/d45 0 2024-02-16T22:21:53.359 INFO:tasks.workunit.client.0.smithi114.stdout:3/221: creat d4/d24/d2d/d45/f51 x:0 0 0 2024-02-16T22:21:53.361 INFO:tasks.workunit.client.1.smithi123.stdout:1/323: rename d3/d6/db/d3b to d3/d6/d11/d76 0 2024-02-16T22:21:53.362 INFO:tasks.workunit.client.1.smithi123.stdout:6/377: symlink d0/d74/l86 0 2024-02-16T22:21:53.362 INFO:tasks.workunit.client.0.smithi114.stdout:3/222: mknod d4/d43/c52 0 2024-02-16T22:21:53.364 INFO:tasks.workunit.client.1.smithi123.stdout:2/321: creat d4/d1a/f6a x:0 0 0 2024-02-16T22:21:53.364 INFO:tasks.workunit.client.1.smithi123.stdout:3/312: dread d0/fb [0,4194304] 0 2024-02-16T22:21:53.367 INFO:tasks.workunit.client.1.smithi123.stdout:5/331: mknod d6/dd/c6d 0 2024-02-16T22:21:53.368 INFO:tasks.workunit.client.1.smithi123.stdout:4/375: dread d5/d1c/f24 [0,4194304] 0 2024-02-16T22:21:53.369 INFO:tasks.workunit.client.0.smithi114.stdout:3/223: dwrite d4/d26/f42 [0,4194304] 0 2024-02-16T22:21:53.369 INFO:tasks.workunit.client.0.smithi114.stdout:3/224: chown d4/d43 984335751 1 2024-02-16T22:21:53.371 INFO:tasks.workunit.client.1.smithi123.stdout:4/376: dread d5/d1c/f24 [0,4194304] 0 2024-02-16T22:21:53.374 INFO:tasks.workunit.client.0.smithi114.stdout:3/225: mkdir d4/d43/d53 0 2024-02-16T22:21:53.375 INFO:tasks.workunit.client.1.smithi123.stdout:3/313: mknod d0/d50/c79 0 2024-02-16T22:21:53.376 INFO:tasks.workunit.client.1.smithi123.stdout:9/439: fdatasync d4/d17/f1a 0 2024-02-16T22:21:53.377 INFO:tasks.workunit.client.1.smithi123.stdout:4/377: dwrite d5/d1c/d23/d47/f62 [4194304,4194304] 0 2024-02-16T22:21:53.377 INFO:tasks.workunit.client.0.smithi114.stdout:3/226: dread d4/d26/f42 [0,4194304] 0 2024-02-16T22:21:53.381 INFO:tasks.workunit.client.1.smithi123.stdout:9/440: dread d4/d17/d24/f4a [0,4194304] 0 2024-02-16T22:21:53.381 INFO:tasks.workunit.client.1.smithi123.stdout:9/441: stat d4/d17/d24/f76 0 2024-02-16T22:21:53.382 INFO:tasks.workunit.client.1.smithi123.stdout:9/442: chown d4/d36/d57 13429754 1 2024-02-16T22:21:53.410 INFO:tasks.workunit.client.0.smithi114.stdout:3/227: mknod d4/d43/c54 0 2024-02-16T22:21:53.412 INFO:tasks.workunit.client.1.smithi123.stdout:6/378: symlink d0/d9/l87 0 2024-02-16T22:21:53.413 INFO:tasks.workunit.client.1.smithi123.stdout:3/314: creat d0/d40/f7a x:0 0 0 2024-02-16T22:21:53.414 INFO:tasks.workunit.client.0.smithi114.stdout:3/228: symlink d4/dd/d33/l55 0 2024-02-16T22:21:53.414 INFO:tasks.workunit.client.1.smithi123.stdout:6/379: read d0/d6/d2f/f22 [2106876,19945] 0 2024-02-16T22:21:53.418 INFO:tasks.workunit.client.1.smithi123.stdout:7/368: dwrite d3/f2a [4194304,4194304] 0 2024-02-16T22:21:53.419 INFO:tasks.workunit.client.1.smithi123.stdout:8/433: rename d0/d17/d26/f2d to d0/d17/d26/d2a/f92 0 2024-02-16T22:21:53.420 INFO:tasks.workunit.client.1.smithi123.stdout:7/369: write d3/d6e/d4d/f5a [522349,43613] 0 2024-02-16T22:21:53.421 INFO:tasks.workunit.client.1.smithi123.stdout:4/378: creat d5/d1c/d5e/f72 x:0 0 0 2024-02-16T22:21:53.421 INFO:tasks.workunit.client.1.smithi123.stdout:7/370: chown d3/d59/f7a 431 1 2024-02-16T22:21:53.423 INFO:tasks.workunit.client.1.smithi123.stdout:4/379: chown d5/d1c/d23/d47/f50 47788 1 2024-02-16T22:21:53.424 INFO:tasks.workunit.client.1.smithi123.stdout:5/332: dread d6/dd/df/d1f/f5c [0,4194304] 0 2024-02-16T22:21:53.427 INFO:tasks.workunit.client.1.smithi123.stdout:5/333: write d6/dd/df/d1f/d25/d37/f3e [179084,48306] 0 2024-02-16T22:21:53.428 INFO:tasks.workunit.client.1.smithi123.stdout:3/315: dread d0/fc [0,4194304] 0 2024-02-16T22:21:53.430 INFO:tasks.workunit.client.1.smithi123.stdout:6/380: chown d0/d9/f6a 5970 1 2024-02-16T22:21:53.430 INFO:tasks.workunit.client.1.smithi123.stdout:3/316: chown d0/c32 26 1 2024-02-16T22:21:53.432 INFO:tasks.workunit.client.0.smithi114.stdout:3/229: dread d4/f1c [0,4194304] 0 2024-02-16T22:21:53.434 INFO:tasks.workunit.client.1.smithi123.stdout:0/331: rename d1/c60 to d1/d8/d13/d1d/c66 0 2024-02-16T22:21:53.435 INFO:tasks.workunit.client.1.smithi123.stdout:8/434: mknod d0/d17/d26/d2a/d4d/c93 0 2024-02-16T22:21:53.436 INFO:tasks.workunit.client.1.smithi123.stdout:9/443: symlink d4/d1e/l88 0 2024-02-16T22:21:53.437 INFO:tasks.workunit.client.1.smithi123.stdout:9/444: chown d4/d36/c46 1 1 2024-02-16T22:21:53.437 INFO:tasks.workunit.client.1.smithi123.stdout:4/380: fsync d5/d1c/d5e/f72 0 2024-02-16T22:21:53.442 INFO:tasks.workunit.client.1.smithi123.stdout:9/445: dwrite d4/d33/f3b [0,4194304] 0 2024-02-16T22:21:53.443 INFO:tasks.workunit.client.1.smithi123.stdout:1/324: getdents d3/d52 0 2024-02-16T22:21:53.447 INFO:tasks.workunit.client.0.smithi114.stdout:3/230: getdents d4/d24/d2d/d45 0 2024-02-16T22:21:53.448 INFO:tasks.workunit.client.1.smithi123.stdout:1/325: stat d3/d6/db/d10/d4f 0 2024-02-16T22:21:53.450 INFO:tasks.workunit.client.1.smithi123.stdout:3/317: write d0/d20/d76/d37/f64 [38660,12460] 0 2024-02-16T22:21:53.450 INFO:tasks.workunit.client.1.smithi123.stdout:0/332: fsync d1/d4/d42/f65 0 2024-02-16T22:21:53.451 INFO:tasks.workunit.client.1.smithi123.stdout:3/318: chown d0/d50/c79 1 1 2024-02-16T22:21:53.451 INFO:tasks.workunit.client.0.smithi114.stdout:3/231: dwrite d4/dd/f13 [0,4194304] 0 2024-02-16T22:21:53.453 INFO:tasks.workunit.client.1.smithi123.stdout:4/381: truncate d5/f29 3837260 0 2024-02-16T22:21:53.456 INFO:tasks.workunit.client.1.smithi123.stdout:1/326: mkdir d3/d6/d49/d48/d68/d77 0 2024-02-16T22:21:53.456 INFO:tasks.workunit.client.0.smithi114.stdout:3/232: creat d4/d43/f56 x:0 0 0 2024-02-16T22:21:53.459 INFO:tasks.workunit.client.1.smithi123.stdout:6/381: mkdir d0/d6/d1a/d88 0 2024-02-16T22:21:53.461 INFO:tasks.workunit.client.1.smithi123.stdout:2/322: sync 2024-02-16T22:21:53.461 INFO:tasks.workunit.client.1.smithi123.stdout:2/323: stat d4/f52 0 2024-02-16T22:21:53.462 INFO:tasks.workunit.client.1.smithi123.stdout:0/333: unlink d1/d4/d42/d4b/f50 0 2024-02-16T22:21:53.463 INFO:tasks.workunit.client.0.smithi114.stdout:3/233: creat d4/d46/d48/f57 x:0 0 0 2024-02-16T22:21:53.465 INFO:tasks.workunit.client.1.smithi123.stdout:3/319: symlink d0/d50/l7b 0 2024-02-16T22:21:53.467 INFO:tasks.workunit.client.1.smithi123.stdout:9/446: dwrite d4/d17/d20/f6a [4194304,4194304] 0 2024-02-16T22:21:53.472 INFO:tasks.workunit.client.1.smithi123.stdout:7/371: link d3/d26/d3e/c62 d3/c7b 0 2024-02-16T22:21:53.473 INFO:tasks.workunit.client.1.smithi123.stdout:4/382: mkdir d5/d1c/d23/d56/d6e/d73 0 2024-02-16T22:21:53.473 INFO:tasks.workunit.client.1.smithi123.stdout:8/435: dread d0/d17/d43/f27 [0,4194304] 0 2024-02-16T22:21:53.474 INFO:tasks.workunit.client.1.smithi123.stdout:3/320: creat d0/d50/f7c x:0 0 0 2024-02-16T22:21:53.476 INFO:tasks.workunit.client.1.smithi123.stdout:3/321: chown d0/d20/d5a 442 1 2024-02-16T22:21:53.476 INFO:tasks.workunit.client.1.smithi123.stdout:8/436: chown d0/d12/c2c 3 1 2024-02-16T22:21:53.477 INFO:tasks.workunit.client.1.smithi123.stdout:2/324: dwrite d4/d1a/f32 [0,4194304] 0 2024-02-16T22:21:53.478 INFO:tasks.workunit.client.1.smithi123.stdout:5/334: getdents d6/dd/df/d1f 0 2024-02-16T22:21:53.480 INFO:tasks.workunit.client.1.smithi123.stdout:0/334: mknod d1/d8/d13/d1d/d64/c67 0 2024-02-16T22:21:53.481 INFO:tasks.workunit.client.1.smithi123.stdout:7/372: truncate d3/d5/f45 1587675 0 2024-02-16T22:21:53.482 INFO:tasks.workunit.client.1.smithi123.stdout:1/327: creat d3/d6/f78 x:0 0 0 2024-02-16T22:21:53.485 INFO:tasks.workunit.client.1.smithi123.stdout:3/322: truncate d0/d20/d2e/f4f 395067 0 2024-02-16T22:21:53.486 INFO:tasks.workunit.client.1.smithi123.stdout:9/447: link d4/d1e/c70 d4/d17/d2b/d82/c89 0 2024-02-16T22:21:53.488 INFO:tasks.workunit.client.1.smithi123.stdout:4/383: dwrite d5/f1e [4194304,4194304] 0 2024-02-16T22:21:53.488 INFO:tasks.workunit.client.1.smithi123.stdout:6/382: rename d0/d9/dc/d3a/d70/c73 to d0/c89 0 2024-02-16T22:21:53.488 INFO:tasks.workunit.client.1.smithi123.stdout:3/323: chown d0/d20/d2e 6750 1 2024-02-16T22:21:53.491 INFO:tasks.workunit.client.1.smithi123.stdout:7/373: creat d3/d6e/f7c x:0 0 0 2024-02-16T22:21:53.494 INFO:tasks.workunit.client.1.smithi123.stdout:8/437: dread d0/d12/f3f [0,4194304] 0 2024-02-16T22:21:53.497 INFO:tasks.workunit.client.1.smithi123.stdout:2/325: dwrite d4/d1a/d1d/d13/d25/f26 [0,4194304] 0 2024-02-16T22:21:53.497 INFO:tasks.workunit.client.1.smithi123.stdout:0/335: truncate d1/f12 3784120 0 2024-02-16T22:21:53.502 INFO:tasks.workunit.client.1.smithi123.stdout:9/448: stat d4/d1e/d3a/c4e 0 2024-02-16T22:21:53.502 INFO:tasks.workunit.client.1.smithi123.stdout:1/328: dwrite d3/d6/d11/f17 [0,4194304] 0 2024-02-16T22:21:53.502 INFO:tasks.workunit.client.1.smithi123.stdout:6/383: mkdir d0/d8a 0 2024-02-16T22:21:53.503 INFO:tasks.workunit.client.1.smithi123.stdout:0/336: write d1/d4/d42/f4a [826079,17380] 0 2024-02-16T22:21:53.507 INFO:tasks.workunit.client.1.smithi123.stdout:3/324: dwrite d0/d20/d2e/f39 [0,4194304] 0 2024-02-16T22:21:53.509 INFO:tasks.workunit.client.1.smithi123.stdout:5/335: rename d6/f53 to d6/dd/df/d1f/d25/d37/f6e 0 2024-02-16T22:21:53.512 INFO:tasks.workunit.client.1.smithi123.stdout:2/326: creat d4/d1a/d1d/d59/d53/f6b x:0 0 0 2024-02-16T22:21:53.513 INFO:tasks.workunit.client.1.smithi123.stdout:9/449: unlink d4/d36/c46 0 2024-02-16T22:21:53.514 INFO:tasks.workunit.client.1.smithi123.stdout:8/438: getdents d0/d17/d26/d2a/d3c/d62 0 2024-02-16T22:21:53.516 INFO:tasks.workunit.client.1.smithi123.stdout:8/439: truncate d0/d17/d26/f8c 418788 0 2024-02-16T22:21:53.518 INFO:tasks.workunit.client.1.smithi123.stdout:3/325: fsync d0/d20/d76/d37/d49/f6a 0 2024-02-16T22:21:53.519 INFO:tasks.workunit.client.1.smithi123.stdout:6/384: mkdir d0/d6/d1a/d88/d8b 0 2024-02-16T22:21:53.520 INFO:tasks.workunit.client.1.smithi123.stdout:9/450: symlink d4/d33/l8a 0 2024-02-16T22:21:53.520 INFO:tasks.workunit.client.1.smithi123.stdout:5/336: mkdir d6/dd/df/d58/d3f/d6f 0 2024-02-16T22:21:53.522 INFO:tasks.workunit.client.1.smithi123.stdout:8/440: dread d0/d17/d26/d32/f56 [0,4194304] 0 2024-02-16T22:21:53.522 INFO:tasks.workunit.client.1.smithi123.stdout:2/327: creat d4/d1a/d1d/d13/d15/d1b/f6c x:0 0 0 2024-02-16T22:21:53.522 INFO:tasks.workunit.client.1.smithi123.stdout:1/329: creat d3/d6/db/d10/d4b/d70/f79 x:0 0 0 2024-02-16T22:21:53.523 INFO:tasks.workunit.client.1.smithi123.stdout:0/337: mknod d1/d4/d42/d4b/d57/c68 0 2024-02-16T22:21:53.523 INFO:tasks.workunit.client.1.smithi123.stdout:2/328: fsync d4/f37 0 2024-02-16T22:21:53.525 INFO:tasks.workunit.client.1.smithi123.stdout:6/385: mkdir d0/d6/d1a/d88/d8c 0 2024-02-16T22:21:53.525 INFO:tasks.workunit.client.1.smithi123.stdout:7/374: dwrite d3/d6e/d1d/f41 [4194304,4194304] 0 2024-02-16T22:21:53.527 INFO:tasks.workunit.client.1.smithi123.stdout:3/326: creat d0/d50/d75/f7d x:0 0 0 2024-02-16T22:21:53.529 INFO:tasks.workunit.client.1.smithi123.stdout:8/441: rmdir d0/d17/d26/d68 39 2024-02-16T22:21:53.531 INFO:tasks.workunit.client.1.smithi123.stdout:9/451: symlink d4/d56/d84/l8b 0 2024-02-16T22:21:53.531 INFO:tasks.workunit.client.1.smithi123.stdout:8/442: truncate d0/d61/d66/d7e/f89 240368 0 2024-02-16T22:21:53.532 INFO:tasks.workunit.client.1.smithi123.stdout:0/338: mknod d1/d8/d13/d1d/c69 0 2024-02-16T22:21:53.534 INFO:tasks.workunit.client.1.smithi123.stdout:8/443: write d0/df/f16 [854567,93568] 0 2024-02-16T22:21:53.537 INFO:tasks.workunit.client.1.smithi123.stdout:3/327: dwrite d0/d50/f5c [0,4194304] 0 2024-02-16T22:21:53.538 INFO:tasks.workunit.client.1.smithi123.stdout:8/444: dread d0/d17/d26/f5b [0,4194304] 0 2024-02-16T22:21:53.538 INFO:tasks.workunit.client.1.smithi123.stdout:3/328: fsync d0/d40/f56 0 2024-02-16T22:21:53.543 INFO:tasks.workunit.client.1.smithi123.stdout:5/337: rename d6/dd/df/d1f/d25/d37/f3e to d6/dd/df/d58/f70 0 2024-02-16T22:21:53.549 INFO:tasks.workunit.client.1.smithi123.stdout:6/386: dread d0/d9/dc/d5f/f2d [0,4194304] 0 2024-02-16T22:21:53.551 INFO:tasks.workunit.client.1.smithi123.stdout:7/375: link d3/d26/d35/f36 d3/d6e/d52/f7d 0 2024-02-16T22:21:53.552 INFO:tasks.workunit.client.1.smithi123.stdout:1/330: dread d3/d6/d18/f21 [4194304,4194304] 0 2024-02-16T22:21:53.552 INFO:tasks.workunit.client.1.smithi123.stdout:5/338: dread d6/dd/df/d58/f70 [0,4194304] 0 2024-02-16T22:21:53.553 INFO:tasks.workunit.client.1.smithi123.stdout:9/452: write d4/d17/f1a [1165574,119611] 0 2024-02-16T22:21:53.554 INFO:tasks.workunit.client.1.smithi123.stdout:5/339: chown d6/dd/df/d1f/d25/f3a 589 1 2024-02-16T22:21:53.555 INFO:tasks.workunit.client.1.smithi123.stdout:6/387: unlink d0/d6/d2f/f22 0 2024-02-16T22:21:53.555 INFO:tasks.workunit.client.1.smithi123.stdout:1/331: dread d3/fe [0,4194304] 0 2024-02-16T22:21:53.558 INFO:tasks.workunit.client.0.smithi114.stdout:7/211: sync 2024-02-16T22:21:53.559 INFO:tasks.workunit.client.1.smithi123.stdout:2/329: dread d4/d1a/d1d/f9 [0,4194304] 0 2024-02-16T22:21:53.561 INFO:tasks.workunit.client.1.smithi123.stdout:1/332: dwrite d3/fe [0,4194304] 0 2024-02-16T22:21:53.561 INFO:tasks.workunit.client.1.smithi123.stdout:4/384: sync 2024-02-16T22:21:53.561 INFO:tasks.workunit.client.1.smithi123.stdout:1/333: chown l1 5208279 1 2024-02-16T22:21:53.564 INFO:tasks.workunit.client.1.smithi123.stdout:0/339: getdents d1/d4/d42 0 2024-02-16T22:21:53.566 INFO:tasks.workunit.client.1.smithi123.stdout:7/376: truncate d3/d6e/f1b 721870 0 2024-02-16T22:21:53.569 INFO:tasks.workunit.client.1.smithi123.stdout:3/329: truncate d0/d20/d76/d37/d49/f67 1767301 0 2024-02-16T22:21:53.570 INFO:tasks.workunit.client.0.smithi114.stdout:7/212: rename d3/c4 to d3/d22/c44 0 2024-02-16T22:21:53.571 INFO:tasks.workunit.client.1.smithi123.stdout:8/445: link d0/df/c5c d0/d61/c94 0 2024-02-16T22:21:53.573 INFO:tasks.workunit.client.0.smithi114.stdout:7/213: creat d3/d3d/f45 x:0 0 0 2024-02-16T22:21:53.574 INFO:tasks.workunit.client.1.smithi123.stdout:1/334: dread d3/d6/d11/f23 [0,4194304] 0 2024-02-16T22:21:53.575 INFO:tasks.workunit.client.1.smithi123.stdout:9/453: write d4/d56/d58/f47 [1213097,106474] 0 2024-02-16T22:21:53.578 INFO:tasks.workunit.client.1.smithi123.stdout:2/330: rename d4/d1a/d1d/d13/d15/d1b to d4/d1a/d1d/d13/d15/d6d 0 2024-02-16T22:21:53.578 INFO:tasks.workunit.client.1.smithi123.stdout:2/331: fsync d4/fc 0 2024-02-16T22:21:53.581 INFO:tasks.workunit.client.0.smithi114.stdout:7/214: mkdir d3/d46 0 2024-02-16T22:21:53.584 INFO:tasks.workunit.client.0.smithi114.stdout:7/215: rmdir d3/d14 39 2024-02-16T22:21:53.584 INFO:tasks.workunit.client.1.smithi123.stdout:4/385: read d5/f29 [310357,15056] 0 2024-02-16T22:21:53.587 INFO:tasks.workunit.client.1.smithi123.stdout:3/330: mkdir d0/d50/d7e 0 2024-02-16T22:21:53.588 INFO:tasks.workunit.client.0.smithi114.stdout:7/216: rmdir d3/d14 39 2024-02-16T22:21:53.588 INFO:tasks.workunit.client.1.smithi123.stdout:8/446: symlink d0/d61/d66/l95 0 2024-02-16T22:21:53.589 INFO:tasks.workunit.client.1.smithi123.stdout:6/388: mkdir d0/d9/dc/d39/d8d 0 2024-02-16T22:21:53.590 INFO:tasks.workunit.client.1.smithi123.stdout:6/389: write d0/f6d [2824406,91224] 0 2024-02-16T22:21:53.590 INFO:tasks.workunit.client.1.smithi123.stdout:5/340: link d6/dd/df/d1f/f68 d6/dd/df/d58/d56/f71 0 2024-02-16T22:21:53.590 INFO:tasks.workunit.client.1.smithi123.stdout:8/447: read d0/d17/d43/f28 [1017745,125370] 0 2024-02-16T22:21:53.591 INFO:tasks.workunit.client.0.smithi114.stdout:7/217: rename d3/f8 to d3/d46/f47 0 2024-02-16T22:21:53.591 INFO:tasks.workunit.client.0.smithi114.stdout:7/218: readlink d3/d3d/l3f 0 2024-02-16T22:21:53.594 INFO:tasks.workunit.client.1.smithi123.stdout:9/454: dread d4/d33/f38 [0,4194304] 0 2024-02-16T22:21:53.598 INFO:tasks.workunit.client.1.smithi123.stdout:2/332: symlink d4/d1a/d1d/l6e 0 2024-02-16T22:21:53.601 INFO:tasks.workunit.client.1.smithi123.stdout:2/333: dread d4/d1a/f32 [0,4194304] 0 2024-02-16T22:21:53.604 INFO:tasks.workunit.client.1.smithi123.stdout:4/386: unlink d5/d1c/c70 0 2024-02-16T22:21:53.605 INFO:tasks.workunit.client.1.smithi123.stdout:3/331: fdatasync d0/f17 0 2024-02-16T22:21:53.607 INFO:tasks.workunit.client.0.smithi114.stdout:7/219: truncate d3/d14/f2c 3008857 0 2024-02-16T22:21:53.614 INFO:tasks.workunit.client.1.smithi123.stdout:1/335: symlink d3/d6/l7a 0 2024-02-16T22:21:53.614 INFO:tasks.workunit.client.1.smithi123.stdout:1/336: chown d3/f4e 230871 1 2024-02-16T22:21:53.615 INFO:tasks.workunit.client.1.smithi123.stdout:9/455: mkdir d4/d17/d24/d8c 0 2024-02-16T22:21:53.615 INFO:tasks.workunit.client.1.smithi123.stdout:6/390: write d0/d6/f5c [1359858,130578] 0 2024-02-16T22:21:53.616 INFO:tasks.workunit.client.1.smithi123.stdout:1/337: write d3/d6/d11/d76/f59 [377326,17827] 0 2024-02-16T22:21:53.616 INFO:tasks.workunit.client.0.smithi114.stdout:7/220: write d3/f1a [3809385,61420] 0 2024-02-16T22:21:53.617 INFO:tasks.workunit.client.1.smithi123.stdout:6/391: read - d0/d6/d1a/d2e/f3c zero size 2024-02-16T22:21:53.618 INFO:tasks.workunit.client.1.smithi123.stdout:0/340: creat d1/f6a x:0 0 0 2024-02-16T22:21:53.619 INFO:tasks.workunit.client.1.smithi123.stdout:7/377: creat d3/d26/f7e x:0 0 0 2024-02-16T22:21:53.619 INFO:tasks.workunit.client.1.smithi123.stdout:0/341: truncate d1/d8/f5d 307983 0 2024-02-16T22:21:53.619 INFO:tasks.workunit.client.1.smithi123.stdout:7/378: chown d3/d6e/f2e 32619680 1 2024-02-16T22:21:53.620 INFO:tasks.workunit.client.1.smithi123.stdout:7/379: chown d3/d6e/d24/f55 11 1 2024-02-16T22:21:53.620 INFO:tasks.workunit.client.0.smithi114.stdout:7/221: unlink d3/d3d/f45 0 2024-02-16T22:21:53.621 INFO:tasks.workunit.client.1.smithi123.stdout:7/380: truncate d3/d5/f33 1070725 0 2024-02-16T22:21:53.622 INFO:tasks.workunit.client.1.smithi123.stdout:3/332: rename d0/d20/d2e/f39 to d0/d20/d76/d37/f7f 0 2024-02-16T22:21:53.623 INFO:tasks.workunit.client.0.smithi114.stdout:7/222: creat d3/d22/f48 x:0 0 0 2024-02-16T22:21:53.624 INFO:tasks.workunit.client.0.smithi114.stdout:7/223: write d3/d22/f48 [307995,52878] 0 2024-02-16T22:21:53.626 INFO:tasks.workunit.client.0.smithi114.stdout:7/224: truncate d3/d46/f47 4482963 0 2024-02-16T22:21:53.626 INFO:tasks.workunit.client.0.smithi114.stdout:7/225: chown d3/d14/f29 8113 1 2024-02-16T22:21:53.626 INFO:tasks.workunit.client.0.smithi114.stdout:7/226: chown d3/d23 12 1 2024-02-16T22:21:53.628 INFO:tasks.workunit.client.1.smithi123.stdout:7/381: creat d3/d5/f7f x:0 0 0 2024-02-16T22:21:53.628 INFO:tasks.workunit.client.1.smithi123.stdout:5/341: link d6/dd/d5f/f60 d6/dd/df/d1f/d25/f72 0 2024-02-16T22:21:53.629 INFO:tasks.workunit.client.1.smithi123.stdout:3/333: rename d0/c1a to d0/d50/d7e/c80 0 2024-02-16T22:21:53.630 INFO:tasks.workunit.client.0.smithi114.stdout:7/227: mknod d3/d3d/c49 0 2024-02-16T22:21:53.630 INFO:tasks.workunit.client.1.smithi123.stdout:3/334: write d0/d50/f7c [489127,17291] 0 2024-02-16T22:21:53.631 INFO:tasks.workunit.client.1.smithi123.stdout:8/448: creat d0/df/f96 x:0 0 0 2024-02-16T22:21:53.631 INFO:tasks.workunit.client.1.smithi123.stdout:9/456: symlink d4/d1e/l8d 0 2024-02-16T22:21:53.632 INFO:tasks.workunit.client.0.smithi114.stdout:7/228: creat d3/d3d/f4a x:0 0 0 2024-02-16T22:21:53.632 INFO:tasks.workunit.client.1.smithi123.stdout:9/457: chown d4/cd 4163 1 2024-02-16T22:21:53.633 INFO:tasks.workunit.client.0.smithi114.stdout:7/229: write d3/ff [4554794,43504] 0 2024-02-16T22:21:53.633 INFO:tasks.workunit.client.1.smithi123.stdout:2/334: creat d4/d1a/d1d/d59/f6f x:0 0 0 2024-02-16T22:21:53.636 INFO:tasks.workunit.client.1.smithi123.stdout:5/342: dwrite d6/dd/df/f22 [0,4194304] 0 2024-02-16T22:21:53.638 INFO:tasks.workunit.client.1.smithi123.stdout:4/387: rename d5/d1c/d23/d36 to d5/d1c/d23/d74 0 2024-02-16T22:21:53.641 INFO:tasks.workunit.client.0.smithi114.stdout:7/230: getdents d3/d23 0 2024-02-16T22:21:53.643 INFO:tasks.workunit.client.1.smithi123.stdout:7/382: write d3/d26/d35/f77 [2952106,4583] 0 2024-02-16T22:21:53.646 INFO:tasks.workunit.client.1.smithi123.stdout:3/335: truncate d0/ff 4367321 0 2024-02-16T22:21:53.647 INFO:tasks.workunit.client.1.smithi123.stdout:3/336: write d0/f6 [3674438,53558] 0 2024-02-16T22:21:53.649 INFO:tasks.workunit.client.1.smithi123.stdout:9/458: rename d4/d56 to d4/d36/d8e 0 2024-02-16T22:21:53.652 INFO:tasks.workunit.client.0.smithi114.stdout:7/231: read d3/d14/f2c [992203,51887] 0 2024-02-16T22:21:53.653 INFO:tasks.workunit.client.1.smithi123.stdout:0/342: creat d1/d8/f6b x:0 0 0 2024-02-16T22:21:53.654 INFO:tasks.workunit.client.0.smithi114.stdout:7/232: read d3/ff [447454,31686] 0 2024-02-16T22:21:53.657 INFO:tasks.workunit.client.1.smithi123.stdout:3/337: mknod d0/d20/d76/d5d/d60/c81 0 2024-02-16T22:21:53.658 INFO:tasks.workunit.client.0.smithi114.stdout:7/233: creat d3/d3d/f4b x:0 0 0 2024-02-16T22:21:53.662 INFO:tasks.workunit.client.0.smithi114.stdout:7/234: creat d3/d14/f4c x:0 0 0 2024-02-16T22:21:53.664 INFO:tasks.workunit.client.1.smithi123.stdout:8/449: truncate d0/d17/d26/d2a/f54 2268395 0 2024-02-16T22:21:53.664 INFO:tasks.workunit.client.1.smithi123.stdout:8/450: fsync d0/d17/d26/d32/d45/f5f 0 2024-02-16T22:21:53.665 INFO:tasks.workunit.client.1.smithi123.stdout:9/459: symlink d4/d36/d8e/d58/l8f 0 2024-02-16T22:21:53.665 INFO:tasks.workunit.client.1.smithi123.stdout:9/460: dread - d4/f6b zero size 2024-02-16T22:21:53.667 INFO:tasks.workunit.client.0.smithi114.stdout:7/235: creat d3/d14/f4d x:0 0 0 2024-02-16T22:21:53.669 INFO:tasks.workunit.client.1.smithi123.stdout:9/461: fsync d4/d17/f1a 0 2024-02-16T22:21:53.670 INFO:tasks.workunit.client.1.smithi123.stdout:0/343: mknod d1/d8/d13/c6c 0 2024-02-16T22:21:53.673 INFO:tasks.workunit.client.0.smithi114.stdout:7/236: mkdir d3/d23/d4e 0 2024-02-16T22:21:53.674 INFO:tasks.workunit.client.1.smithi123.stdout:3/338: creat d0/d20/d76/d5d/f82 x:0 0 0 2024-02-16T22:21:53.675 INFO:tasks.workunit.client.0.smithi114.stdout:7/237: symlink d3/d22/l4f 0 2024-02-16T22:21:53.679 INFO:tasks.workunit.client.0.smithi114.stdout:7/238: dread d3/d14/f18 [0,4194304] 0 2024-02-16T22:21:53.680 INFO:tasks.workunit.client.0.smithi114.stdout:7/239: chown d3/d23/d4e 1757915 1 2024-02-16T22:21:53.680 INFO:tasks.workunit.client.0.smithi114.stdout:7/240: dread - d3/d14/f4d zero size 2024-02-16T22:21:53.681 INFO:tasks.workunit.client.1.smithi123.stdout:2/335: creat d4/d1a/d1d/d59/d45/f70 x:0 0 0 2024-02-16T22:21:53.683 INFO:tasks.workunit.client.0.smithi114.stdout:7/241: dread d3/d46/f47 [4194304,4194304] 0 2024-02-16T22:21:53.686 INFO:tasks.workunit.client.0.smithi114.stdout:7/242: rmdir d3 39 2024-02-16T22:21:53.689 INFO:tasks.workunit.client.1.smithi123.stdout:5/343: rename d6/dd/df/d1f/d25/f72 to d6/dd/f73 0 2024-02-16T22:21:53.690 INFO:tasks.workunit.client.0.smithi114.stdout:7/243: chown d3/d22/l30 123 1 2024-02-16T22:21:53.690 INFO:tasks.workunit.client.1.smithi123.stdout:3/339: rmdir d0/d20/d76/d37 39 2024-02-16T22:21:53.691 INFO:tasks.workunit.client.1.smithi123.stdout:6/392: truncate d0/d9/dc/f23 293332 0 2024-02-16T22:21:53.691 INFO:tasks.workunit.client.0.smithi114.stdout:7/244: write d3/d14/f43 [839832,85213] 0 2024-02-16T22:21:53.694 INFO:tasks.workunit.client.1.smithi123.stdout:7/383: getdents d3/d6e/d52 0 2024-02-16T22:21:53.697 INFO:tasks.workunit.client.0.smithi114.stdout:7/245: truncate d3/d14/f28 2736689 0 2024-02-16T22:21:53.698 INFO:tasks.workunit.client.1.smithi123.stdout:3/340: symlink d0/d50/d75/l83 0 2024-02-16T22:21:53.699 INFO:tasks.workunit.client.1.smithi123.stdout:3/341: write d0/f7 [4645507,16536] 0 2024-02-16T22:21:53.699 INFO:tasks.workunit.client.1.smithi123.stdout:6/393: mkdir d0/d6/d8e 0 2024-02-16T22:21:53.701 INFO:tasks.workunit.client.0.smithi114.stdout:7/246: symlink d3/d14/l50 0 2024-02-16T22:21:53.701 INFO:tasks.workunit.client.0.smithi114.stdout:7/247: write d3/d3d/f4b [91683,39876] 0 2024-02-16T22:21:53.704 INFO:tasks.workunit.client.1.smithi123.stdout:2/336: rename d4/d1a/d1d/d59/c33 to d4/d1a/d1d/d13/d15/d6d/c71 0 2024-02-16T22:21:53.710 INFO:tasks.workunit.client.1.smithi123.stdout:6/394: creat d0/d6/d4d/f8f x:0 0 0 2024-02-16T22:21:53.710 INFO:tasks.workunit.client.1.smithi123.stdout:6/395: chown d0/d9/dc/d3a/d70 13194 1 2024-02-16T22:21:53.712 INFO:tasks.workunit.client.1.smithi123.stdout:7/384: link d3/d6e/d1d/f22 d3/d6e/d1d/f80 0 2024-02-16T22:21:53.713 INFO:tasks.workunit.client.1.smithi123.stdout:2/337: mkdir d4/d1a/d1d/d13/d25/d72 0 2024-02-16T22:21:53.715 INFO:tasks.workunit.client.1.smithi123.stdout:6/396: creat d0/d74/f90 x:0 0 0 2024-02-16T22:21:53.716 INFO:tasks.workunit.client.1.smithi123.stdout:7/385: symlink d3/d6e/d1d/d1e/l81 0 2024-02-16T22:21:53.716 INFO:tasks.workunit.client.1.smithi123.stdout:9/462: write d4/d17/d20/f3c [1466165,45480] 0 2024-02-16T22:21:53.717 INFO:tasks.workunit.client.1.smithi123.stdout:3/342: creat d0/d20/d29/f84 x:0 0 0 2024-02-16T22:21:53.718 INFO:tasks.workunit.client.1.smithi123.stdout:1/338: sync 2024-02-16T22:21:53.718 INFO:tasks.workunit.client.1.smithi123.stdout:2/338: fsync d4/d1a/d1d/f9 0 2024-02-16T22:21:53.720 INFO:tasks.workunit.client.1.smithi123.stdout:8/451: dwrite d0/d17/d26/d2a/f78 [0,4194304] 0 2024-02-16T22:21:53.728 INFO:tasks.workunit.client.1.smithi123.stdout:6/397: dwrite d0/d9/dc/d3a/f7d [0,4194304] 0 2024-02-16T22:21:53.732 INFO:tasks.workunit.client.1.smithi123.stdout:0/344: dwrite d1/f12 [0,4194304] 0 2024-02-16T22:21:53.733 INFO:tasks.workunit.client.1.smithi123.stdout:8/452: dwrite d0/d17/d26/d2a/f78 [0,4194304] 0 2024-02-16T22:21:53.733 INFO:tasks.workunit.client.0.smithi114.stdout:7/248: write d3/ff [1376656,13703] 0 2024-02-16T22:21:53.734 INFO:tasks.workunit.client.1.smithi123.stdout:6/398: truncate d0/d9/dc/d39/f5b 120394 0 2024-02-16T22:21:53.738 INFO:tasks.workunit.client.1.smithi123.stdout:3/343: mkdir d0/d85 0 2024-02-16T22:21:53.738 INFO:tasks.workunit.client.1.smithi123.stdout:4/388: sync 2024-02-16T22:21:53.738 INFO:tasks.workunit.client.1.smithi123.stdout:9/463: mknod d4/d17/d7f/c90 0 2024-02-16T22:21:53.740 INFO:tasks.workunit.client.0.smithi114.stdout:7/249: creat d3/d46/f51 x:0 0 0 2024-02-16T22:21:53.740 INFO:tasks.workunit.client.1.smithi123.stdout:3/344: readlink d0/d20/d2e/l3e 0 2024-02-16T22:21:53.740 INFO:tasks.workunit.client.1.smithi123.stdout:4/389: chown d5/d1c/d23/d56 876392 1 2024-02-16T22:21:53.743 INFO:tasks.workunit.client.1.smithi123.stdout:2/339: creat d4/d1a/d1d/d13/d25/d72/f73 x:0 0 0 2024-02-16T22:21:53.744 INFO:tasks.workunit.client.1.smithi123.stdout:5/344: dwrite d6/dd/df/d1f/d25/d37/f52 [0,4194304] 0 2024-02-16T22:21:53.746 INFO:tasks.workunit.client.1.smithi123.stdout:7/386: dread d3/d26/d3e/f5b [0,4194304] 0 2024-02-16T22:21:53.746 INFO:tasks.workunit.client.0.smithi114.stdout:7/250: creat d3/d23/f52 x:0 0 0 2024-02-16T22:21:53.747 INFO:tasks.workunit.client.1.smithi123.stdout:7/387: fdatasync d3/d5/f61 0 2024-02-16T22:21:53.749 INFO:tasks.workunit.client.1.smithi123.stdout:9/464: creat d4/d36/d8e/d58/f91 x:0 0 0 2024-02-16T22:21:53.751 INFO:tasks.workunit.client.1.smithi123.stdout:3/345: creat d0/d20/d76/d5d/f86 x:0 0 0 2024-02-16T22:21:53.753 INFO:tasks.workunit.client.1.smithi123.stdout:3/346: read - d0/d40/f59 zero size 2024-02-16T22:21:53.753 INFO:tasks.workunit.client.1.smithi123.stdout:4/390: unlink d5/d1c/d23/d74/f45 0 2024-02-16T22:21:53.754 INFO:tasks.workunit.client.1.smithi123.stdout:0/345: truncate d1/f1f 290403 0 2024-02-16T22:21:53.754 INFO:tasks.workunit.client.1.smithi123.stdout:2/340: mknod d4/d1a/d1d/d13/d25/d72/c74 0 2024-02-16T22:21:53.755 INFO:tasks.workunit.client.1.smithi123.stdout:6/399: dwrite d0/f65 [0,4194304] 0 2024-02-16T22:21:53.755 INFO:tasks.workunit.client.1.smithi123.stdout:8/453: mknod d0/d17/d26/d2a/d3c/d72/c97 0 2024-02-16T22:21:53.756 INFO:tasks.workunit.client.1.smithi123.stdout:6/400: chown d0/d8a 0 1 2024-02-16T22:21:53.756 INFO:tasks.workunit.client.1.smithi123.stdout:7/388: symlink d3/d26/d35/l82 0 2024-02-16T22:21:53.761 INFO:tasks.workunit.client.1.smithi123.stdout:5/345: mknod d6/dd/df/d58/d3f/d6f/c74 0 2024-02-16T22:21:53.761 INFO:tasks.workunit.client.1.smithi123.stdout:4/391: creat d5/d1c/d42/f75 x:0 0 0 2024-02-16T22:21:53.763 INFO:tasks.workunit.client.1.smithi123.stdout:4/392: chown d5/d1c/d42 7735 1 2024-02-16T22:21:53.763 INFO:tasks.workunit.client.1.smithi123.stdout:9/465: truncate d4/d33/f73 528530 0 2024-02-16T22:21:53.764 INFO:tasks.workunit.client.1.smithi123.stdout:3/347: mknod d0/d20/d76/d37/d49/c87 0 2024-02-16T22:21:53.765 INFO:tasks.workunit.client.1.smithi123.stdout:0/346: mkdir d1/d8/d13/d1d/d62/d6d 0 2024-02-16T22:21:53.767 INFO:tasks.workunit.client.1.smithi123.stdout:5/346: rmdir d6/dd/df/d1f/d25 39 2024-02-16T22:21:53.768 INFO:tasks.workunit.client.1.smithi123.stdout:2/341: dwrite d4/d1a/d1d/d13/d25/f3b [4194304,4194304] 0 2024-02-16T22:21:53.768 INFO:tasks.workunit.client.1.smithi123.stdout:6/401: rmdir d0/d6/d1a/d88/d8c 0 2024-02-16T22:21:53.769 INFO:tasks.workunit.client.1.smithi123.stdout:7/389: dwrite d3/d6e/d34/f57 [0,4194304] 0 2024-02-16T22:21:53.771 INFO:tasks.workunit.client.1.smithi123.stdout:3/348: mknod d0/d50/d7e/c88 0 2024-02-16T22:21:53.771 INFO:tasks.workunit.client.1.smithi123.stdout:4/393: mkdir d5/d1c/d23/d76 0 2024-02-16T22:21:53.772 INFO:tasks.workunit.client.1.smithi123.stdout:9/466: rename d4/d17/d2b/d82 to d4/d36/d8e/d6d/d71/d92 0 2024-02-16T22:21:53.773 INFO:tasks.workunit.client.1.smithi123.stdout:6/402: mkdir d0/d6/d91 0 2024-02-16T22:21:53.774 INFO:tasks.workunit.client.1.smithi123.stdout:2/342: creat d4/d1a/d1d/f75 x:0 0 0 2024-02-16T22:21:53.776 INFO:tasks.workunit.client.1.smithi123.stdout:3/349: symlink d0/d50/d75/l89 0 2024-02-16T22:21:53.777 INFO:tasks.workunit.client.1.smithi123.stdout:4/394: creat d5/d1c/d23/d56/d6e/f77 x:0 0 0 2024-02-16T22:21:53.778 INFO:tasks.workunit.client.1.smithi123.stdout:9/467: mkdir d4/d36/d8e/d84/d93 0 2024-02-16T22:21:53.780 INFO:tasks.workunit.client.1.smithi123.stdout:9/468: write d4/d1e/d3a/f50 [6641647,16142] 0 2024-02-16T22:21:53.781 INFO:tasks.workunit.client.1.smithi123.stdout:5/347: dread d6/dd/df/d1f/d25/d37/f52 [0,4194304] 0 2024-02-16T22:21:53.784 INFO:tasks.workunit.client.1.smithi123.stdout:4/395: truncate d5/d1c/d23/d31/f66 110795 0 2024-02-16T22:21:53.784 INFO:tasks.workunit.client.1.smithi123.stdout:7/390: link d3/d6e/d1d/d1e/c40 d3/d6e/d1d/c83 0 2024-02-16T22:21:53.786 INFO:tasks.workunit.client.1.smithi123.stdout:2/343: read d4/d1a/d1d/d13/d25/f4d [1401610,98320] 0 2024-02-16T22:21:53.788 INFO:tasks.workunit.client.1.smithi123.stdout:6/403: link d0/f6d d0/d6/d8e/f92 0 2024-02-16T22:21:53.788 INFO:tasks.workunit.client.1.smithi123.stdout:9/469: rmdir d4/d17/d24 39 2024-02-16T22:21:53.789 INFO:tasks.workunit.client.1.smithi123.stdout:9/470: chown d4 0 1 2024-02-16T22:21:53.792 INFO:tasks.workunit.client.1.smithi123.stdout:6/404: dread d0/d9/dc/d39/f5b [0,4194304] 0 2024-02-16T22:21:53.792 INFO:tasks.workunit.client.1.smithi123.stdout:4/396: dwrite d5/d1c/d23/d47/f50 [0,4194304] 0 2024-02-16T22:21:53.793 INFO:tasks.workunit.client.1.smithi123.stdout:4/397: dread - d5/f63 zero size 2024-02-16T22:21:53.794 INFO:tasks.workunit.client.1.smithi123.stdout:5/348: rename d6/l33 to d6/dd/df/d1f/d25/l75 0 2024-02-16T22:21:53.794 INFO:tasks.workunit.client.1.smithi123.stdout:6/405: write d0/d6/d1a/d2e/f50 [488536,102226] 0 2024-02-16T22:21:53.796 INFO:tasks.workunit.client.1.smithi123.stdout:1/339: dread d3/f35 [0,4194304] 0 2024-02-16T22:21:53.797 INFO:tasks.workunit.client.1.smithi123.stdout:5/349: truncate d6/dd/d44/f51 525457 0 2024-02-16T22:21:53.797 INFO:tasks.workunit.client.1.smithi123.stdout:9/471: fdatasync d4/d17/d24/f69 0 2024-02-16T22:21:53.798 INFO:tasks.workunit.client.1.smithi123.stdout:5/350: stat d6/dd/l48 0 2024-02-16T22:21:53.798 INFO:tasks.workunit.client.1.smithi123.stdout:4/398: getdents d5/d1c/d23/d69 0 2024-02-16T22:21:53.799 INFO:tasks.workunit.client.1.smithi123.stdout:9/472: write d4/d36/d8e/d6f/f77 [226161,52319] 0 2024-02-16T22:21:53.799 INFO:tasks.workunit.client.1.smithi123.stdout:6/406: mkdir d0/d6/d91/d93 0 2024-02-16T22:21:53.802 INFO:tasks.workunit.client.1.smithi123.stdout:5/351: readlink d6/dd/df/d1f/l49 0 2024-02-16T22:21:53.804 INFO:tasks.workunit.client.1.smithi123.stdout:4/399: rename d5/f59 to d5/d1c/d23/d6d/f78 0 2024-02-16T22:21:53.804 INFO:tasks.workunit.client.1.smithi123.stdout:5/352: truncate d6/dd/f73 753164 0 2024-02-16T22:21:53.807 INFO:tasks.workunit.client.1.smithi123.stdout:1/340: dwrite d3/d6/db/d10/d4b/f72 [0,4194304] 0 2024-02-16T22:21:53.807 INFO:tasks.workunit.client.1.smithi123.stdout:9/473: rename d4/d33/l8a to d4/d17/d24/d8c/l94 0 2024-02-16T22:21:53.808 INFO:tasks.workunit.client.1.smithi123.stdout:5/353: fdatasync d6/dd/df/d58/f43 0 2024-02-16T22:21:53.811 INFO:tasks.workunit.client.1.smithi123.stdout:9/474: write d4/d36/d8e/f61 [1831174,55079] 0 2024-02-16T22:21:53.811 INFO:tasks.workunit.client.1.smithi123.stdout:4/400: creat d5/d1c/d23/d69/f79 x:0 0 0 2024-02-16T22:21:53.812 INFO:tasks.workunit.client.1.smithi123.stdout:6/407: dwrite d0/d6/d2f/d5d/f68 [0,4194304] 0 2024-02-16T22:21:53.814 INFO:tasks.workunit.client.1.smithi123.stdout:5/354: mkdir d6/dd/df/d76 0 2024-02-16T22:21:53.816 INFO:tasks.workunit.client.1.smithi123.stdout:9/475: creat d4/d36/d8e/f95 x:0 0 0 2024-02-16T22:21:53.817 INFO:tasks.workunit.client.1.smithi123.stdout:5/355: creat d6/dd/d44/f77 x:0 0 0 2024-02-16T22:21:53.819 INFO:tasks.workunit.client.1.smithi123.stdout:9/476: stat d4/d17/d24/f76 0 2024-02-16T22:21:53.819 INFO:tasks.workunit.client.1.smithi123.stdout:4/401: dwrite d5/d49/f4e [0,4194304] 0 2024-02-16T22:21:53.822 INFO:tasks.workunit.client.1.smithi123.stdout:6/408: dwrite d0/d74/f7b [0,4194304] 0 2024-02-16T22:21:53.824 INFO:tasks.workunit.client.1.smithi123.stdout:5/356: mkdir d6/dd/d78 0 2024-02-16T22:21:53.831 INFO:tasks.workunit.client.0.smithi114.stdout:7/251: truncate d3/d22/f34 3320093 0 2024-02-16T22:21:53.831 INFO:tasks.workunit.client.1.smithi123.stdout:9/477: dwrite d4/d36/d8e/d6f/f77 [0,4194304] 0 2024-02-16T22:21:53.834 INFO:tasks.workunit.client.0.smithi114.stdout:7/252: symlink d3/d14/l53 0 2024-02-16T22:21:53.835 INFO:tasks.workunit.client.0.smithi114.stdout:7/253: dread d3/f10 [0,4194304] 0 2024-02-16T22:21:53.836 INFO:tasks.workunit.client.0.smithi114.stdout:7/254: write d3/d23/f52 [468658,46497] 0 2024-02-16T22:21:53.837 INFO:tasks.workunit.client.1.smithi123.stdout:9/478: rename d4/d81 to d4/d36/d8e/d84/d93/d96 0 2024-02-16T22:21:53.840 INFO:tasks.workunit.client.0.smithi114.stdout:7/255: mknod d3/d14/c54 0 2024-02-16T22:21:53.846 INFO:tasks.workunit.client.0.smithi114.stdout:7/256: truncate d3/d3d/f41 5108971 0 2024-02-16T22:21:53.847 INFO:tasks.workunit.client.1.smithi123.stdout:7/391: fsync d3/d6e/d34/f57 0 2024-02-16T22:21:53.847 INFO:tasks.workunit.client.1.smithi123.stdout:9/479: link d4/d36/l52 d4/d36/d8e/d58/l97 0 2024-02-16T22:21:53.847 INFO:tasks.workunit.client.1.smithi123.stdout:7/392: dread - d3/d6e/d24/f55 zero size 2024-02-16T22:21:53.847 INFO:tasks.workunit.client.1.smithi123.stdout:7/393: fdatasync d3/f4e 0 2024-02-16T22:21:53.848 INFO:tasks.workunit.client.1.smithi123.stdout:9/480: mkdir d4/d36/d8e/d98 0 2024-02-16T22:21:53.849 INFO:tasks.workunit.client.1.smithi123.stdout:8/454: write d0/d17/d26/d68/d81/d47/f82 [1221319,46020] 0 2024-02-16T22:21:53.852 INFO:tasks.workunit.client.0.smithi114.stdout:7/257: link d3/d3d/c42 d3/d3d/c55 0 2024-02-16T22:21:53.854 INFO:tasks.workunit.client.1.smithi123.stdout:0/347: sync 2024-02-16T22:21:53.862 INFO:tasks.workunit.client.0.smithi114.stdout:7/258: rename d3/d23/c31 to d3/c56 0 2024-02-16T22:21:53.864 INFO:tasks.workunit.client.1.smithi123.stdout:9/481: link d4/d1e/l8d d4/d36/d8e/d98/l99 0 2024-02-16T22:21:53.865 INFO:tasks.workunit.client.1.smithi123.stdout:9/482: read d4/d33/f3b [123234,9960] 0 2024-02-16T22:21:53.865 INFO:tasks.workunit.client.1.smithi123.stdout:7/394: getdents d3/d26/d3e 0 2024-02-16T22:21:53.866 INFO:tasks.workunit.client.1.smithi123.stdout:4/402: read d5/d1c/d23/d31/f66 [82740,54628] 0 2024-02-16T22:21:53.867 INFO:tasks.workunit.client.1.smithi123.stdout:4/403: write d5/f4c [1249585,367] 0 2024-02-16T22:21:53.871 INFO:tasks.workunit.client.1.smithi123.stdout:9/483: creat d4/d17/d2b/f9a x:0 0 0 2024-02-16T22:21:53.873 INFO:tasks.workunit.client.0.smithi114.stdout:7/259: link d3/d3d/c55 d3/d22/c57 0 2024-02-16T22:21:53.873 INFO:tasks.workunit.client.1.smithi123.stdout:3/350: dwrite d0/f12 [0,4194304] 0 2024-02-16T22:21:53.874 INFO:tasks.workunit.client.1.smithi123.stdout:3/351: readlink d0/d20/d76/l6b 0 2024-02-16T22:21:53.874 INFO:tasks.workunit.client.1.smithi123.stdout:3/352: chown d0/d50/d75/l83 40 1 2024-02-16T22:21:53.878 INFO:tasks.workunit.client.1.smithi123.stdout:4/404: symlink d5/d1c/d23/d56/d6e/l7a 0 2024-02-16T22:21:53.880 INFO:tasks.workunit.client.1.smithi123.stdout:2/344: dwrite d4/d1a/d1d/d13/d25/f66 [0,4194304] 0 2024-02-16T22:21:53.883 INFO:tasks.workunit.client.1.smithi123.stdout:3/353: rmdir d0/d20/d76/d5d 39 2024-02-16T22:21:53.885 INFO:tasks.workunit.client.1.smithi123.stdout:4/405: mkdir d5/d1c/d64/d7b 0 2024-02-16T22:21:53.885 INFO:tasks.workunit.client.1.smithi123.stdout:9/484: creat d4/d36/d8e/d58/d6e/f9b x:0 0 0 2024-02-16T22:21:53.886 INFO:tasks.workunit.client.1.smithi123.stdout:1/341: write d3/f4e [4257229,103732] 0 2024-02-16T22:21:53.887 INFO:tasks.workunit.client.1.smithi123.stdout:2/345: chown d4/d1a/d1d/d59/d45/c31 30 1 2024-02-16T22:21:53.890 INFO:tasks.workunit.client.1.smithi123.stdout:5/357: getdents d6/dd/df 0 2024-02-16T22:21:53.891 INFO:tasks.workunit.client.1.smithi123.stdout:9/485: creat d4/d36/d57/f9c x:0 0 0 2024-02-16T22:21:53.891 INFO:tasks.workunit.client.1.smithi123.stdout:2/346: fsync d4/d1a/d1d/d13/d25/f43 0 2024-02-16T22:21:53.892 INFO:tasks.workunit.client.1.smithi123.stdout:2/347: dread - d4/d1a/d1d/f75 zero size 2024-02-16T22:21:53.893 INFO:tasks.workunit.client.1.smithi123.stdout:5/358: creat d6/dd/df/f79 x:0 0 0 2024-02-16T22:21:53.894 INFO:tasks.workunit.client.1.smithi123.stdout:5/359: chown d6/dd/d44 0 1 2024-02-16T22:21:53.894 INFO:tasks.workunit.client.1.smithi123.stdout:9/486: mkdir d4/d36/d8e/d58/d59/d60/d9d 0 2024-02-16T22:21:53.895 INFO:tasks.workunit.client.1.smithi123.stdout:4/406: getdents d5/d1c/d23/d31 0 2024-02-16T22:21:53.896 INFO:tasks.workunit.client.1.smithi123.stdout:6/409: write d0/d6/d1a/f2b [1730877,51650] 0 2024-02-16T22:21:53.896 INFO:tasks.workunit.client.1.smithi123.stdout:2/348: creat d4/d1a/d1d/d38/d63/f76 x:0 0 0 2024-02-16T22:21:53.896 INFO:tasks.workunit.client.1.smithi123.stdout:9/487: chown d4/d17/d24/f4a 2511418 1 2024-02-16T22:21:53.898 INFO:tasks.workunit.client.1.smithi123.stdout:5/360: dread - d6/dd/df/d1f/f64 zero size 2024-02-16T22:21:53.901 INFO:tasks.workunit.client.1.smithi123.stdout:6/410: creat d0/d74/f94 x:0 0 0 2024-02-16T22:21:53.901 INFO:tasks.workunit.client.1.smithi123.stdout:5/361: chown d6/dd/df/f41 26 1 2024-02-16T22:21:53.903 INFO:tasks.workunit.client.1.smithi123.stdout:4/407: dwrite d5/f14 [0,4194304] 0 2024-02-16T22:21:53.905 INFO:tasks.workunit.client.1.smithi123.stdout:4/408: readlink d5/d1c/l5c 0 2024-02-16T22:21:53.905 INFO:tasks.workunit.client.1.smithi123.stdout:9/488: link d4/d17/d20/f7b d4/d36/d8e/d6d/f9e 0 2024-02-16T22:21:53.910 INFO:tasks.workunit.client.1.smithi123.stdout:8/455: truncate d0/d12/f58 654496 0 2024-02-16T22:21:53.911 INFO:tasks.workunit.client.1.smithi123.stdout:6/411: dwrite d0/d6/f5c [0,4194304] 0 2024-02-16T22:21:53.911 INFO:tasks.workunit.client.1.smithi123.stdout:5/362: creat d6/d3b/f7a x:0 0 0 2024-02-16T22:21:53.913 INFO:tasks.workunit.client.1.smithi123.stdout:1/342: dread d3/d6/f5b [0,4194304] 0 2024-02-16T22:21:53.913 INFO:tasks.workunit.client.1.smithi123.stdout:9/489: unlink d4/d1e/d3a/c4e 0 2024-02-16T22:21:53.913 INFO:tasks.workunit.client.1.smithi123.stdout:2/349: dread f2 [0,4194304] 0 2024-02-16T22:21:53.915 INFO:tasks.workunit.client.1.smithi123.stdout:4/409: mknod d5/d1c/d23/c7c 0 2024-02-16T22:21:53.916 INFO:tasks.workunit.client.1.smithi123.stdout:1/343: write d3/d6/db/d10/d2d/f25 [803143,26330] 0 2024-02-16T22:21:53.917 INFO:tasks.workunit.client.1.smithi123.stdout:8/456: creat d0/d61/d66/f98 x:0 0 0 2024-02-16T22:21:53.919 INFO:tasks.workunit.client.1.smithi123.stdout:2/350: write f2 [3496368,76145] 0 2024-02-16T22:21:53.919 INFO:tasks.workunit.client.1.smithi123.stdout:7/395: write f2 [2378437,68095] 0 2024-02-16T22:21:53.920 INFO:tasks.workunit.client.1.smithi123.stdout:5/363: symlink d6/dd/df/d1f/d25/d37/l7b 0 2024-02-16T22:21:53.922 INFO:tasks.workunit.client.1.smithi123.stdout:2/351: stat d4/d1a/d1d/d13/d25/l4c 0 2024-02-16T22:21:53.924 INFO:tasks.workunit.client.1.smithi123.stdout:9/490: dwrite d4/d36/f5f [0,4194304] 0 2024-02-16T22:21:53.924 INFO:tasks.workunit.client.1.smithi123.stdout:6/412: link d0/d6/d1a/d2e/f77 d0/d6/d91/d93/f95 0 2024-02-16T22:21:53.925 INFO:tasks.workunit.client.1.smithi123.stdout:2/352: dread - d4/d1a/d1d/d13/d15/d6d/f6c zero size 2024-02-16T22:21:53.926 INFO:tasks.workunit.client.1.smithi123.stdout:6/413: dread - d0/d6/d91/d93/f95 zero size 2024-02-16T22:21:53.928 INFO:tasks.workunit.client.1.smithi123.stdout:7/396: mknod d3/d6e/d1d/c84 0 2024-02-16T22:21:53.930 INFO:tasks.workunit.client.1.smithi123.stdout:5/364: readlink d6/dd/df/d1f/d25/d37/l3d 0 2024-02-16T22:21:53.932 INFO:tasks.workunit.client.1.smithi123.stdout:5/365: truncate d6/dd/f50 708402 0 2024-02-16T22:21:53.935 INFO:tasks.workunit.client.1.smithi123.stdout:3/354: dwrite d0/f31 [0,4194304] 0 2024-02-16T22:21:53.936 INFO:tasks.workunit.client.1.smithi123.stdout:3/355: readlink d0/d20/d76/d37/l66 0 2024-02-16T22:21:53.937 INFO:tasks.workunit.client.1.smithi123.stdout:9/491: rename d4/d1e/d3a/f50 to d4/d1e/d3a/f9f 0 2024-02-16T22:21:53.941 INFO:tasks.workunit.client.1.smithi123.stdout:5/366: dwrite d6/dd/d44/f51 [0,4194304] 0 2024-02-16T22:21:53.941 INFO:tasks.workunit.client.1.smithi123.stdout:7/397: rename d3/d26/l2c to d3/d26/d35/l85 0 2024-02-16T22:21:53.942 INFO:tasks.workunit.client.1.smithi123.stdout:7/398: chown d3/l47 9878 1 2024-02-16T22:21:53.942 INFO:tasks.workunit.client.1.smithi123.stdout:3/356: dread d0/d50/f5c [0,4194304] 0 2024-02-16T22:21:53.945 INFO:tasks.workunit.client.1.smithi123.stdout:6/414: mkdir d0/d96 0 2024-02-16T22:21:53.947 INFO:tasks.workunit.client.0.smithi114.stdout:7/260: dwrite d3/f27 [0,4194304] 0 2024-02-16T22:21:53.949 INFO:tasks.workunit.client.1.smithi123.stdout:7/399: dwrite d3/d26/d35/f4b [0,4194304] 0 2024-02-16T22:21:53.952 INFO:tasks.workunit.client.1.smithi123.stdout:7/400: truncate d3/d6e/d1d/d1e/f65 454741 0 2024-02-16T22:21:53.954 INFO:tasks.workunit.client.1.smithi123.stdout:2/353: link d4/fc d4/d1a/d1d/d13/d25/f77 0 2024-02-16T22:21:53.957 INFO:tasks.workunit.client.0.smithi114.stdout:7/261: rename d3/f10 to d3/d3d/f58 0 2024-02-16T22:21:53.961 INFO:tasks.workunit.client.1.smithi123.stdout:5/367: mknod d6/dd/d44/c7c 0 2024-02-16T22:21:53.962 INFO:tasks.workunit.client.1.smithi123.stdout:0/348: sync 2024-02-16T22:21:53.964 INFO:tasks.workunit.client.1.smithi123.stdout:1/344: dwrite d3/d6/f26 [0,4194304] 0 2024-02-16T22:21:53.964 INFO:tasks.workunit.client.1.smithi123.stdout:5/368: write d6/dd/df/d1f/d25/f1a [4647391,92861] 0 2024-02-16T22:21:53.965 INFO:tasks.workunit.client.1.smithi123.stdout:4/410: truncate d5/f1f 3924566 0 2024-02-16T22:21:53.966 INFO:tasks.workunit.client.1.smithi123.stdout:4/411: fsync d5/d1c/d23/d69/f79 0 2024-02-16T22:21:53.968 INFO:tasks.workunit.client.1.smithi123.stdout:6/415: readlink d0/d9/dc/l3d 0 2024-02-16T22:21:53.969 INFO:tasks.workunit.client.1.smithi123.stdout:8/457: truncate d0/d12/f1e 2978274 0 2024-02-16T22:21:53.970 INFO:tasks.workunit.client.1.smithi123.stdout:6/416: dread - d0/d6/d1a/d2e/f77 zero size 2024-02-16T22:21:53.971 INFO:tasks.workunit.client.1.smithi123.stdout:7/401: chown d3/d6e/f1b 1026477506 1 2024-02-16T22:21:53.972 INFO:tasks.workunit.client.1.smithi123.stdout:6/417: readlink d0/d6/d2f/l1c 0 2024-02-16T22:21:53.974 INFO:tasks.workunit.client.1.smithi123.stdout:4/412: dwrite d5/f1e [8388608,4194304] 0 2024-02-16T22:21:53.976 INFO:tasks.workunit.client.1.smithi123.stdout:2/354: symlink d4/d1a/d1d/d13/l78 0 2024-02-16T22:21:53.978 INFO:tasks.workunit.client.1.smithi123.stdout:5/369: dread d6/dd/df/d58/f43 [0,4194304] 0 2024-02-16T22:21:53.978 INFO:tasks.workunit.client.1.smithi123.stdout:9/492: truncate d4/d17/f1f 1577194 0 2024-02-16T22:21:53.979 INFO:tasks.workunit.client.1.smithi123.stdout:4/413: read d5/d1c/d23/f3a [2017550,27303] 0 2024-02-16T22:21:53.980 INFO:tasks.workunit.client.1.smithi123.stdout:9/493: chown d4/d36/d57/f9c 270 1 2024-02-16T22:21:53.982 INFO:tasks.workunit.client.1.smithi123.stdout:3/357: unlink d0/d50/f5c 0 2024-02-16T22:21:53.983 INFO:tasks.workunit.client.1.smithi123.stdout:3/358: chown d0/d20/d29/c2b 3119 1 2024-02-16T22:21:53.983 INFO:tasks.workunit.client.1.smithi123.stdout:0/349: rename d1/l2d to d1/d8/d13/d1d/d62/d6d/l6e 0 2024-02-16T22:21:53.984 INFO:tasks.workunit.client.1.smithi123.stdout:3/359: stat d0/d20/d29/f3c 0 2024-02-16T22:21:53.986 INFO:tasks.workunit.client.1.smithi123.stdout:4/414: dwrite d5/d1c/d5e/f72 [0,4194304] 0 2024-02-16T22:21:53.988 INFO:tasks.workunit.client.1.smithi123.stdout:6/418: mkdir d0/d6/d91/d97 0 2024-02-16T22:21:53.993 INFO:tasks.workunit.client.1.smithi123.stdout:9/494: mknod d4/d17/d7f/ca0 0 2024-02-16T22:21:53.996 INFO:tasks.workunit.client.0.smithi114.stdout:8/285: sync 2024-02-16T22:21:53.996 INFO:tasks.workunit.client.0.smithi114.stdout:6/229: sync 2024-02-16T22:21:53.997 INFO:tasks.workunit.client.0.smithi114.stdout:9/271: sync 2024-02-16T22:21:53.997 INFO:tasks.workunit.client.0.smithi114.stdout:0/253: sync 2024-02-16T22:21:53.999 INFO:tasks.workunit.client.0.smithi114.stdout:9/272: chown d2/c14 1190050 1 2024-02-16T22:21:53.999 INFO:tasks.workunit.client.0.smithi114.stdout:9/273: fdatasync d2/f44 0 2024-02-16T22:21:54.000 INFO:tasks.workunit.client.0.smithi114.stdout:7/262: dwrite d3/d14/f28 [0,4194304] 0 2024-02-16T22:21:54.004 INFO:tasks.workunit.client.1.smithi123.stdout:0/350: creat d1/d8/d13/d1d/f6f x:0 0 0 2024-02-16T22:21:54.009 INFO:tasks.workunit.client.0.smithi114.stdout:8/286: symlink d4/d11/d15/d36/d3b/l5e 0 2024-02-16T22:21:54.011 INFO:tasks.workunit.client.1.smithi123.stdout:3/360: dread d0/fe [0,4194304] 0 2024-02-16T22:21:54.012 INFO:tasks.workunit.client.1.smithi123.stdout:3/361: chown d0/d20/d2e/c5b 3112 1 2024-02-16T22:21:54.012 INFO:tasks.workunit.client.1.smithi123.stdout:2/355: dread d4/d1a/d1d/d13/d25/f4d [0,4194304] 0 2024-02-16T22:21:54.014 INFO:tasks.workunit.client.1.smithi123.stdout:2/356: write d4/d1a/d1d/d59/d45/f70 [444240,113518] 0 2024-02-16T22:21:54.014 INFO:tasks.workunit.client.1.smithi123.stdout:2/357: readlink d4/d1a/d1d/l2e 0 2024-02-16T22:21:54.015 INFO:tasks.workunit.client.1.smithi123.stdout:2/358: dread - d4/d1a/d1d/d38/f58 zero size 2024-02-16T22:21:54.021 INFO:tasks.workunit.client.1.smithi123.stdout:1/345: truncate d3/d6/d18/f21 1755234 0 2024-02-16T22:21:54.023 INFO:tasks.workunit.client.1.smithi123.stdout:0/351: mknod d1/d8/d3d/c70 0 2024-02-16T22:21:54.025 INFO:tasks.workunit.client.1.smithi123.stdout:7/402: creat d3/f86 x:0 0 0 2024-02-16T22:21:54.027 INFO:tasks.workunit.client.1.smithi123.stdout:7/403: read d3/d26/d3e/f63 [267140,44575] 0 2024-02-16T22:21:54.027 INFO:tasks.workunit.client.1.smithi123.stdout:4/415: mknod d5/d1c/d23/d76/c7d 0 2024-02-16T22:21:54.028 INFO:tasks.workunit.client.1.smithi123.stdout:7/404: readlink d3/d6e/d24/l30 0 2024-02-16T22:21:54.028 INFO:tasks.workunit.client.0.smithi114.stdout:6/230: write d6/f21 [1087790,2566] 0 2024-02-16T22:21:54.032 INFO:tasks.workunit.client.0.smithi114.stdout:0/254: rename d1/cd to d1/d16/c50 0 2024-02-16T22:21:54.033 INFO:tasks.workunit.client.0.smithi114.stdout:9/274: truncate d2/f4 356859 0 2024-02-16T22:21:54.033 INFO:tasks.workunit.client.0.smithi114.stdout:0/255: read - d1/f3d zero size 2024-02-16T22:21:54.034 INFO:tasks.workunit.client.1.smithi123.stdout:3/362: dread d0/f4c [0,4194304] 0 2024-02-16T22:21:54.034 INFO:tasks.workunit.client.1.smithi123.stdout:6/419: creat d0/d6/d1a/d88/d8b/f98 x:0 0 0 2024-02-16T22:21:54.035 INFO:tasks.workunit.client.1.smithi123.stdout:8/458: dwrite d0/d12/f3f [0,4194304] 0 2024-02-16T22:21:54.035 INFO:tasks.workunit.client.1.smithi123.stdout:2/359: dread d4/d1a/d1d/d13/d25/f77 [0,4194304] 0 2024-02-16T22:21:54.036 INFO:tasks.workunit.client.0.smithi114.stdout:7/263: getdents d3/d14 0 2024-02-16T22:21:54.036 INFO:tasks.workunit.client.0.smithi114.stdout:7/264: fsync d3/f26 0 2024-02-16T22:21:54.037 INFO:tasks.workunit.client.1.smithi123.stdout:2/360: read d4/fc [669706,16288] 0 2024-02-16T22:21:54.038 INFO:tasks.workunit.client.1.smithi123.stdout:3/363: write d0/d20/f6f [137225,98240] 0 2024-02-16T22:21:54.039 INFO:tasks.workunit.client.1.smithi123.stdout:7/405: dwrite d3/d6e/d4d/f50 [0,4194304] 0 2024-02-16T22:21:54.040 INFO:tasks.workunit.client.0.smithi114.stdout:6/231: creat d6/d2c/d38/d41/f4a x:0 0 0 2024-02-16T22:21:54.040 INFO:tasks.workunit.client.1.smithi123.stdout:2/361: write d4/d1a/d1d/d13/d25/f43 [143349,71971] 0 2024-02-16T22:21:54.043 INFO:tasks.workunit.client.0.smithi114.stdout:8/287: rename d4/d11/d15/d31/d5b to d4/d24/d2c/d5f 0 2024-02-16T22:21:54.046 INFO:tasks.workunit.client.1.smithi123.stdout:8/459: dread d0/d17/d26/d2a/d3c/f55 [0,4194304] 0 2024-02-16T22:21:54.048 INFO:tasks.workunit.client.0.smithi114.stdout:9/275: mkdir d2/d5c 0 2024-02-16T22:21:54.048 INFO:tasks.workunit.client.0.smithi114.stdout:8/288: dread d4/d11/d15/d31/f37 [0,4194304] 0 2024-02-16T22:21:54.050 INFO:tasks.workunit.client.1.smithi123.stdout:2/362: dwrite d4/d1a/d1d/d13/d15/f42 [0,4194304] 0 2024-02-16T22:21:54.055 INFO:tasks.workunit.client.0.smithi114.stdout:7/265: symlink d3/d22/l59 0 2024-02-16T22:21:54.056 INFO:tasks.workunit.client.0.smithi114.stdout:5/244: sync 2024-02-16T22:21:54.056 INFO:tasks.workunit.client.0.smithi114.stdout:1/243: sync 2024-02-16T22:21:54.056 INFO:tasks.workunit.client.0.smithi114.stdout:4/350: sync 2024-02-16T22:21:54.056 INFO:tasks.workunit.client.0.smithi114.stdout:3/234: sync 2024-02-16T22:21:54.056 INFO:tasks.workunit.client.0.smithi114.stdout:2/264: sync 2024-02-16T22:21:54.063 INFO:tasks.workunit.client.0.smithi114.stdout:0/256: dwrite d1/d1a/d3e/d41/f42 [0,4194304] 0 2024-02-16T22:21:54.067 INFO:tasks.workunit.client.0.smithi114.stdout:3/235: dwrite f1 [8388608,4194304] 0 2024-02-16T22:21:54.067 INFO:tasks.workunit.client.0.smithi114.stdout:2/265: creat dd/d4e/f5e x:0 0 0 2024-02-16T22:21:54.068 INFO:tasks.workunit.client.0.smithi114.stdout:2/266: stat dd/df/f59 0 2024-02-16T22:21:54.068 INFO:tasks.workunit.client.1.smithi123.stdout:9/495: truncate d4/d36/d8e/d6f/f77 1399642 0 2024-02-16T22:21:54.070 INFO:tasks.workunit.client.0.smithi114.stdout:4/351: mkdir dd/d1b/d37/d38/d7c 0 2024-02-16T22:21:54.072 INFO:tasks.workunit.client.1.smithi123.stdout:6/420: symlink d0/d6/d2f/d5d/l99 0 2024-02-16T22:21:54.073 INFO:tasks.workunit.client.1.smithi123.stdout:5/370: getdents d6/dd/df/d58 0 2024-02-16T22:21:54.078 INFO:tasks.workunit.client.1.smithi123.stdout:3/364: symlink d0/d50/l8a 0 2024-02-16T22:21:54.081 INFO:tasks.workunit.client.0.smithi114.stdout:7/266: dwrite d3/d22/f2b [4194304,4194304] 0 2024-02-16T22:21:54.081 INFO:tasks.workunit.client.1.smithi123.stdout:1/346: dwrite d3/d6/db/d10/f4a [0,4194304] 0 2024-02-16T22:21:54.084 INFO:tasks.workunit.client.0.smithi114.stdout:6/232: write d6/dd/f17 [2151645,35301] 0 2024-02-16T22:21:54.087 INFO:tasks.workunit.client.0.smithi114.stdout:0/257: mkdir d1/d16/d1f/d28/d51 0 2024-02-16T22:21:54.088 INFO:tasks.workunit.client.0.smithi114.stdout:0/258: readlink d1/d16/d18/l27 0 2024-02-16T22:21:54.089 INFO:tasks.workunit.client.0.smithi114.stdout:6/233: fdatasync d6/dd/d25/d26/f2d 0 2024-02-16T22:21:54.091 INFO:tasks.workunit.client.0.smithi114.stdout:2/267: rmdir dd/df/d2c 39 2024-02-16T22:21:54.093 INFO:tasks.workunit.client.1.smithi123.stdout:0/352: rename d1/d8/d13/d1d/c66 to d1/d4/d42/d4b/d57/c71 0 2024-02-16T22:21:54.098 INFO:tasks.workunit.client.1.smithi123.stdout:6/421: creat d0/d6/d1a/d88/f9a x:0 0 0 2024-02-16T22:21:54.103 INFO:tasks.workunit.client.0.smithi114.stdout:7/267: rmdir d3/d23 39 2024-02-16T22:21:54.105 INFO:tasks.workunit.client.0.smithi114.stdout:9/276: link d2/d33/c51 d2/d1a/d23/d32/c5d 0 2024-02-16T22:21:54.106 INFO:tasks.workunit.client.0.smithi114.stdout:8/289: dwrite d4/f8 [0,4194304] 0 2024-02-16T22:21:54.106 INFO:tasks.workunit.client.0.smithi114.stdout:9/277: chown d2/d1a/d23/d35/d1f 15187640 1 2024-02-16T22:21:54.111 INFO:tasks.workunit.client.1.smithi123.stdout:8/460: link d0/df/f16 d0/d17/d26/d2a/d77/f99 0 2024-02-16T22:21:54.112 INFO:tasks.workunit.client.1.smithi123.stdout:8/461: write d0/f14 [3671120,42460] 0 2024-02-16T22:21:54.114 INFO:tasks.workunit.client.1.smithi123.stdout:2/363: mknod d4/d1a/d1d/d38/d5e/c79 0 2024-02-16T22:21:54.116 INFO:tasks.workunit.client.1.smithi123.stdout:2/364: truncate d4/d1a/d1d/d59/f6f 998414 0 2024-02-16T22:21:54.117 INFO:tasks.workunit.client.0.smithi114.stdout:9/278: dwrite d2/d1a/d23/d35/d1d/f54 [0,4194304] 0 2024-02-16T22:21:54.118 INFO:tasks.workunit.client.1.smithi123.stdout:4/416: rename d5/d1c/d23/d56/d6e/f77 to d5/d1c/d23/d47/f7e 0 2024-02-16T22:21:54.119 INFO:tasks.workunit.client.0.smithi114.stdout:9/279: chown d2/l2b 1 1 2024-02-16T22:21:54.119 INFO:tasks.workunit.client.0.smithi114.stdout:3/236: dwrite d4/f1c [0,4194304] 0 2024-02-16T22:21:54.119 INFO:tasks.workunit.client.0.smithi114.stdout:9/280: stat d2/f6 0 2024-02-16T22:21:54.121 INFO:tasks.workunit.client.0.smithi114.stdout:0/259: mkdir d1/d30/d52 0 2024-02-16T22:21:54.123 INFO:tasks.workunit.client.1.smithi123.stdout:0/353: creat d1/d4/d42/d4b/d57/f72 x:0 0 0 2024-02-16T22:21:54.125 INFO:tasks.workunit.client.1.smithi123.stdout:6/422: fsync d0/d6/d8e/f92 0 2024-02-16T22:21:54.126 INFO:tasks.workunit.client.0.smithi114.stdout:6/234: symlink d6/da/l4b 0 2024-02-16T22:21:54.128 INFO:tasks.workunit.client.0.smithi114.stdout:2/268: chown dd/df/d2c/f34 178 1 2024-02-16T22:21:54.128 INFO:tasks.workunit.client.0.smithi114.stdout:2/269: fdatasync dd/df/f59 0 2024-02-16T22:21:54.129 INFO:tasks.workunit.client.0.smithi114.stdout:5/245: truncate d0/df/d16/d1e/f29 8059806 0 2024-02-16T22:21:54.133 INFO:tasks.workunit.client.1.smithi123.stdout:4/417: dwrite d5/d1c/d23/d74/f5f [4194304,4194304] 0 2024-02-16T22:21:54.134 INFO:tasks.workunit.client.1.smithi123.stdout:5/371: dwrite d6/dd/d5f/f60 [0,4194304] 0 2024-02-16T22:21:54.135 INFO:tasks.workunit.client.0.smithi114.stdout:2/270: dwrite dd/d4e/f58 [0,4194304] 0 2024-02-16T22:21:54.136 INFO:tasks.workunit.client.1.smithi123.stdout:4/418: fsync d5/f17 0 2024-02-16T22:21:54.137 INFO:tasks.workunit.client.1.smithi123.stdout:4/419: stat d5/f4c 0 2024-02-16T22:21:54.139 INFO:tasks.workunit.client.1.smithi123.stdout:7/406: dwrite d3/d6e/d52/f7d [0,4194304] 0 2024-02-16T22:21:54.140 INFO:tasks.workunit.client.1.smithi123.stdout:9/496: rename d4/d17/f1d to d4/fa1 0 2024-02-16T22:21:54.141 INFO:tasks.workunit.client.0.smithi114.stdout:3/237: creat d4/d26/f58 x:0 0 0 2024-02-16T22:21:54.141 INFO:tasks.workunit.client.1.smithi123.stdout:2/365: write d4/d1a/d1d/d13/d25/f4d [1569131,83632] 0 2024-02-16T22:21:54.142 INFO:tasks.workunit.client.0.smithi114.stdout:3/238: stat d4/f16 0 2024-02-16T22:21:54.142 INFO:tasks.workunit.client.0.smithi114.stdout:3/239: chown d4/d26/d4a 14 1 2024-02-16T22:21:54.143 INFO:tasks.workunit.client.0.smithi114.stdout:9/281: chown d2/f4 17033959 1 2024-02-16T22:21:54.143 INFO:tasks.workunit.client.1.smithi123.stdout:4/420: chown d5/d1c/d23/f57 111272 1 2024-02-16T22:21:54.146 INFO:tasks.workunit.client.1.smithi123.stdout:0/354: creat d1/d4/d42/d4b/d57/f73 x:0 0 0 2024-02-16T22:21:54.147 INFO:tasks.workunit.client.0.smithi114.stdout:6/235: fdatasync d6/f30 0 2024-02-16T22:21:54.150 INFO:tasks.workunit.client.0.smithi114.stdout:5/246: creat d0/d10/f43 x:0 0 0 2024-02-16T22:21:54.150 INFO:tasks.workunit.client.1.smithi123.stdout:3/365: creat d0/d20/f8b x:0 0 0 2024-02-16T22:21:54.153 INFO:tasks.workunit.client.0.smithi114.stdout:4/352: link dd/d2e/f6a dd/d2e/d56/f7d 0 2024-02-16T22:21:54.155 INFO:tasks.workunit.client.1.smithi123.stdout:4/421: dread d5/d1c/f24 [0,4194304] 0 2024-02-16T22:21:54.156 INFO:tasks.workunit.client.0.smithi114.stdout:1/244: getdents d4/d9 0 2024-02-16T22:21:54.157 INFO:tasks.workunit.client.1.smithi123.stdout:4/422: write d5/d1c/d5e/f72 [3648304,83145] 0 2024-02-16T22:21:54.157 INFO:tasks.workunit.client.1.smithi123.stdout:4/423: chown d5/d1c/d23/d74/c46 1 1 2024-02-16T22:21:54.159 INFO:tasks.workunit.client.0.smithi114.stdout:6/236: dwrite d6/f30 [0,4194304] 0 2024-02-16T22:21:54.163 INFO:tasks.workunit.client.0.smithi114.stdout:7/268: mkdir d3/d5a 0 2024-02-16T22:21:54.164 INFO:tasks.workunit.client.0.smithi114.stdout:2/271: creat dd/df/d2c/f5f x:0 0 0 2024-02-16T22:21:54.164 INFO:tasks.workunit.client.0.smithi114.stdout:7/269: fsync d3/d14/f28 0 2024-02-16T22:21:54.166 INFO:tasks.workunit.client.1.smithi123.stdout:1/347: rename d3/d6/d18/l43 to d3/d6/db/d10/d2d/l7b 0 2024-02-16T22:21:54.167 INFO:tasks.workunit.client.1.smithi123.stdout:9/497: creat d4/d36/d8e/d58/fa2 x:0 0 0 2024-02-16T22:21:54.168 INFO:tasks.workunit.client.0.smithi114.stdout:3/240: mknod d4/dd/c59 0 2024-02-16T22:21:54.169 INFO:tasks.workunit.client.0.smithi114.stdout:3/241: dread - d4/d26/f39 zero size 2024-02-16T22:21:54.170 INFO:tasks.workunit.client.1.smithi123.stdout:2/366: symlink d4/d1a/d1d/d38/d63/l7a 0 2024-02-16T22:21:54.171 INFO:tasks.workunit.client.0.smithi114.stdout:3/242: dread - d4/d46/d48/f57 zero size 2024-02-16T22:21:54.171 INFO:tasks.workunit.client.1.smithi123.stdout:1/348: dread d3/d6/db/d10/f4a [0,4194304] 0 2024-02-16T22:21:54.175 INFO:tasks.workunit.client.1.smithi123.stdout:0/355: dwrite d1/d4/d42/d4b/f58 [0,4194304] 0 2024-02-16T22:21:54.176 INFO:tasks.workunit.client.0.smithi114.stdout:5/247: rename d0/d10/f43 to d0/df/d16/d25/d3e/f44 0 2024-02-16T22:21:54.177 INFO:tasks.workunit.client.1.smithi123.stdout:4/424: symlink d5/d1c/d23/d76/l7f 0 2024-02-16T22:21:54.177 INFO:tasks.workunit.client.0.smithi114.stdout:3/243: dread d4/d26/f3c [0,4194304] 0 2024-02-16T22:21:54.179 INFO:tasks.workunit.client.0.smithi114.stdout:3/244: fsync d4/dd/f13 0 2024-02-16T22:21:54.180 INFO:tasks.workunit.client.1.smithi123.stdout:6/423: link d0/d9/dc/d3a/d70/f72 d0/d6/d4d/f9b 0 2024-02-16T22:21:54.184 INFO:tasks.workunit.client.0.smithi114.stdout:1/245: fsync d4/f21 0 2024-02-16T22:21:54.185 INFO:tasks.workunit.client.0.smithi114.stdout:1/246: chown d4/d33/c42 377 1 2024-02-16T22:21:54.186 INFO:tasks.workunit.client.1.smithi123.stdout:9/498: creat d4/d36/d8e/d6d/d71/fa3 x:0 0 0 2024-02-16T22:21:54.190 INFO:tasks.workunit.client.0.smithi114.stdout:2/272: mknod dd/df/d2c/c60 0 2024-02-16T22:21:54.190 INFO:tasks.workunit.client.0.smithi114.stdout:7/270: dread d3/d14/f18 [0,4194304] 0 2024-02-16T22:21:54.191 INFO:tasks.workunit.client.0.smithi114.stdout:7/271: dread - d3/d3d/f4a zero size 2024-02-16T22:21:54.193 INFO:tasks.workunit.client.0.smithi114.stdout:9/282: symlink d2/d5c/l5e 0 2024-02-16T22:21:54.201 INFO:tasks.workunit.client.0.smithi114.stdout:5/248: rename d0/df/d16/c19 to d0/d2d/d30/c45 0 2024-02-16T22:21:54.208 INFO:tasks.workunit.client.1.smithi123.stdout:2/367: mkdir d4/d1a/d1d/d38/d7b 0 2024-02-16T22:21:54.209 INFO:tasks.workunit.client.0.smithi114.stdout:0/260: dread d1/d1a/f36 [0,4194304] 0 2024-02-16T22:21:54.209 INFO:tasks.workunit.client.0.smithi114.stdout:5/249: dwrite d0/d10/f1a [0,4194304] 0 2024-02-16T22:21:54.210 INFO:tasks.workunit.client.1.smithi123.stdout:9/499: read d4/d36/f78 [1983428,3767] 0 2024-02-16T22:21:54.211 INFO:tasks.workunit.client.0.smithi114.stdout:5/250: fdatasync d0/d2d/d30/f40 0 2024-02-16T22:21:54.211 INFO:tasks.workunit.client.1.smithi123.stdout:2/368: dread d4/d1a/d1d/d59/f6f [0,4194304] 0 2024-02-16T22:21:54.214 INFO:tasks.workunit.client.1.smithi123.stdout:5/372: write d6/f8 [3155999,98810] 0 2024-02-16T22:21:54.215 INFO:tasks.workunit.client.1.smithi123.stdout:4/425: symlink d5/d1c/d64/d7b/l80 0 2024-02-16T22:21:54.215 INFO:tasks.workunit.client.1.smithi123.stdout:2/369: dread d4/d1a/d1d/d13/d25/f66 [0,4194304] 0 2024-02-16T22:21:54.215 INFO:tasks.workunit.client.1.smithi123.stdout:5/373: chown d6/dd/df 1408 1 2024-02-16T22:21:54.217 INFO:tasks.workunit.client.1.smithi123.stdout:2/370: write d4/f54 [46742,88359] 0 2024-02-16T22:21:54.217 INFO:tasks.workunit.client.0.smithi114.stdout:8/290: truncate d4/d11/f17 4092002 0 2024-02-16T22:21:54.218 INFO:tasks.workunit.client.1.smithi123.stdout:2/371: write d4/d1a/d1d/d38/d63/f67 [712688,57666] 0 2024-02-16T22:21:54.220 INFO:tasks.workunit.client.1.smithi123.stdout:2/372: stat d4/d1a/d1d/d38/f58 0 2024-02-16T22:21:54.223 INFO:tasks.workunit.client.1.smithi123.stdout:4/426: dwrite d5/d1c/d23/f57 [0,4194304] 0 2024-02-16T22:21:54.223 INFO:tasks.workunit.client.1.smithi123.stdout:7/407: rename d3/d6e/d1d/d1e to d3/d87 0 2024-02-16T22:21:54.227 INFO:tasks.workunit.client.1.smithi123.stdout:7/408: dread d3/d87/f4a [0,4194304] 0 2024-02-16T22:21:54.232 INFO:tasks.workunit.client.0.smithi114.stdout:4/353: link dd/d1b/d24/c44 dd/d1b/d37/d38/c7e 0 2024-02-16T22:21:54.233 INFO:tasks.workunit.client.1.smithi123.stdout:5/374: creat d6/dd/d44/f7d x:0 0 0 2024-02-16T22:21:54.238 INFO:tasks.workunit.client.0.smithi114.stdout:2/273: truncate dd/f13 2841261 0 2024-02-16T22:21:54.241 INFO:tasks.workunit.client.1.smithi123.stdout:2/373: rmdir d4/d1a/d1d/d59 39 2024-02-16T22:21:54.243 INFO:tasks.workunit.client.1.smithi123.stdout:9/500: dread d4/d17/f23 [0,4194304] 0 2024-02-16T22:21:54.244 INFO:tasks.workunit.client.1.smithi123.stdout:4/427: mknod d5/d49/c81 0 2024-02-16T22:21:54.247 INFO:tasks.workunit.client.0.smithi114.stdout:6/237: truncate d6/f30 2996679 0 2024-02-16T22:21:54.248 INFO:tasks.workunit.client.1.smithi123.stdout:8/462: sync 2024-02-16T22:21:54.248 INFO:tasks.workunit.client.1.smithi123.stdout:9/501: dwrite d4/f6b [0,4194304] 0 2024-02-16T22:21:54.248 INFO:tasks.workunit.client.1.smithi123.stdout:8/463: stat d0/l8 0 2024-02-16T22:21:54.249 INFO:tasks.workunit.client.0.smithi114.stdout:6/238: truncate d6/dd/d25/f27 1379491 0 2024-02-16T22:21:54.250 INFO:tasks.workunit.client.0.smithi114.stdout:5/251: link d0/d28/f38 d0/d28/f46 0 2024-02-16T22:21:54.253 INFO:tasks.workunit.client.0.smithi114.stdout:5/252: truncate d0/d2d/d30/f40 228421 0 2024-02-16T22:21:54.254 INFO:tasks.workunit.client.0.smithi114.stdout:5/253: fdatasync d0/fb 0 2024-02-16T22:21:54.255 INFO:tasks.workunit.client.1.smithi123.stdout:6/424: getdents d0/d6/d8e 0 2024-02-16T22:21:54.257 INFO:tasks.workunit.client.0.smithi114.stdout:1/247: dwrite d4/d9/f10 [4194304,4194304] 0 2024-02-16T22:21:54.259 INFO:tasks.workunit.client.0.smithi114.stdout:4/354: dread dd/f13 [0,4194304] 0 2024-02-16T22:21:54.261 INFO:tasks.workunit.client.0.smithi114.stdout:7/272: mkdir d3/d23/d4e/d5b 0 2024-02-16T22:21:54.262 INFO:tasks.workunit.client.1.smithi123.stdout:2/374: stat d4/d1a/l2f 0 2024-02-16T22:21:54.267 INFO:tasks.workunit.client.0.smithi114.stdout:9/283: dwrite d2/dd/ff [0,4194304] 0 2024-02-16T22:21:54.268 INFO:tasks.workunit.client.1.smithi123.stdout:5/375: rmdir d6/dd/d44 39 2024-02-16T22:21:54.268 INFO:tasks.workunit.client.0.smithi114.stdout:6/239: creat d6/dd/d25/f4c x:0 0 0 2024-02-16T22:21:54.273 INFO:tasks.workunit.client.0.smithi114.stdout:4/355: dwrite dd/d2e/d56/f69 [0,4194304] 0 2024-02-16T22:21:54.274 INFO:tasks.workunit.client.0.smithi114.stdout:3/245: dwrite d4/d24/f25 [0,4194304] 0 2024-02-16T22:21:54.276 INFO:tasks.workunit.client.0.smithi114.stdout:5/254: unlink d0/d28/c2a 0 2024-02-16T22:21:54.280 INFO:tasks.workunit.client.1.smithi123.stdout:8/464: write d0/fd [3056339,109500] 0 2024-02-16T22:21:54.282 INFO:tasks.workunit.client.1.smithi123.stdout:8/465: truncate d0/d17/d43/d34/f38 918190 0 2024-02-16T22:21:54.283 INFO:tasks.workunit.client.1.smithi123.stdout:8/466: write d0/d17/d26/f7a [1309706,124398] 0 2024-02-16T22:21:54.286 INFO:tasks.workunit.client.0.smithi114.stdout:5/255: dwrite d0/f9 [4194304,4194304] 0 2024-02-16T22:21:54.287 INFO:tasks.workunit.client.1.smithi123.stdout:7/409: unlink d3/d87/c40 0 2024-02-16T22:21:54.292 INFO:tasks.workunit.client.0.smithi114.stdout:1/248: dwrite d4/f16 [4194304,4194304] 0 2024-02-16T22:21:54.292 INFO:tasks.workunit.client.1.smithi123.stdout:3/366: rename d0/d50/l72 to d0/l8c 0 2024-02-16T22:21:54.295 INFO:tasks.workunit.client.0.smithi114.stdout:8/291: rename d4/d24/f3e to d4/d11/f60 0 2024-02-16T22:21:54.296 INFO:tasks.workunit.client.0.smithi114.stdout:8/292: chown d4/d11/d15/f1a 44017 1 2024-02-16T22:21:54.300 INFO:tasks.workunit.client.0.smithi114.stdout:9/284: rmdir d2 39 2024-02-16T22:21:54.307 INFO:tasks.workunit.client.0.smithi114.stdout:6/240: mkdir d6/dd/d25/d4d 0 2024-02-16T22:21:54.307 INFO:tasks.workunit.client.0.smithi114.stdout:2/274: dwrite dd/df/f33 [0,4194304] 0 2024-02-16T22:21:54.308 INFO:tasks.workunit.client.1.smithi123.stdout:9/502: read d4/d17/d20/f7b [5613729,71438] 0 2024-02-16T22:21:54.310 INFO:tasks.workunit.client.1.smithi123.stdout:8/467: rmdir d0/d61/d66/d7e 39 2024-02-16T22:21:54.311 INFO:tasks.workunit.client.0.smithi114.stdout:4/356: unlink dd/d2e/d56/f7d 0 2024-02-16T22:21:54.312 INFO:tasks.workunit.client.0.smithi114.stdout:4/357: chown dd/d1b/l64 6498713 1 2024-02-16T22:21:54.315 INFO:tasks.workunit.client.1.smithi123.stdout:6/425: fsync d0/d9/dc/f23 0 2024-02-16T22:21:54.319 INFO:tasks.workunit.client.1.smithi123.stdout:0/356: rename d1/d4/d42/d4b to d1/d4/d17/d74 0 2024-02-16T22:21:54.321 INFO:tasks.workunit.client.0.smithi114.stdout:7/273: dread d3/d22/f34 [0,4194304] 0 2024-02-16T22:21:54.321 INFO:tasks.workunit.client.1.smithi123.stdout:1/349: sync 2024-02-16T22:21:54.325 INFO:tasks.workunit.client.1.smithi123.stdout:4/428: truncate d5/f1e 3745784 0 2024-02-16T22:21:54.327 INFO:tasks.workunit.client.0.smithi114.stdout:1/249: creat d4/d9/f4e x:0 0 0 2024-02-16T22:21:54.328 INFO:tasks.workunit.client.0.smithi114.stdout:1/250: write d4/f30 [1774645,125640] 0 2024-02-16T22:21:54.333 INFO:tasks.workunit.client.0.smithi114.stdout:6/241: mknod d6/dd/d25/c4e 0 2024-02-16T22:21:54.336 INFO:tasks.workunit.client.0.smithi114.stdout:6/242: dread d6/f30 [0,4194304] 0 2024-02-16T22:21:54.337 INFO:tasks.workunit.client.0.smithi114.stdout:6/243: stat d6/dd/f1d 0 2024-02-16T22:21:54.337 INFO:tasks.workunit.client.1.smithi123.stdout:2/375: sync 2024-02-16T22:21:54.338 INFO:tasks.workunit.client.1.smithi123.stdout:9/503: dread d4/d36/d8e/d58/f5a [0,4194304] 0 2024-02-16T22:21:54.471 INFO:tasks.workunit.client.1.smithi123.stdout:6/426: unlink d0/d6/d1a/d88/d8b/f98 0 2024-02-16T22:21:54.473 INFO:tasks.workunit.client.0.smithi114.stdout:3/246: symlink d4/l5a 0 2024-02-16T22:21:54.475 INFO:tasks.workunit.client.1.smithi123.stdout:3/367: rename d0/f1f to d0/d50/d7e/f8d 0 2024-02-16T22:21:54.476 INFO:tasks.workunit.client.1.smithi123.stdout:5/376: link d6/dd/df/d1f/f64 d6/dd/d44/f7e 0 2024-02-16T22:21:54.477 INFO:tasks.workunit.client.0.smithi114.stdout:5/256: symlink d0/df/d16/d25/d2e/d3b/l47 0 2024-02-16T22:21:54.477 INFO:tasks.workunit.client.1.smithi123.stdout:5/377: write d6/dd/df/d1f/d25/f17 [3078662,12956] 0 2024-02-16T22:21:54.483 INFO:tasks.workunit.client.0.smithi114.stdout:1/251: truncate d4/f28 704594 0 2024-02-16T22:21:54.483 INFO:tasks.workunit.client.0.smithi114.stdout:8/293: mknod d4/d11/c61 0 2024-02-16T22:21:54.484 INFO:tasks.workunit.client.0.smithi114.stdout:5/257: dwrite d0/df/f2b [0,4194304] 0 2024-02-16T22:21:54.489 INFO:tasks.workunit.client.0.smithi114.stdout:6/244: rmdir d6/dd/d25/d34 39 2024-02-16T22:21:54.493 INFO:tasks.workunit.client.0.smithi114.stdout:9/285: dread d2/f15 [0,4194304] 0 2024-02-16T22:21:54.494 INFO:tasks.workunit.client.1.smithi123.stdout:2/376: stat d4/d1a/l48 0 2024-02-16T22:21:54.496 INFO:tasks.workunit.client.0.smithi114.stdout:6/245: dwrite d6/dd/f29 [0,4194304] 0 2024-02-16T22:21:54.497 INFO:tasks.workunit.client.1.smithi123.stdout:4/429: dread d5/d49/f4b [4194304,4194304] 0 2024-02-16T22:21:54.497 INFO:tasks.workunit.client.1.smithi123.stdout:9/504: mknod d4/d17/d2b/d6c/ca4 0 2024-02-16T22:21:54.497 INFO:tasks.workunit.client.1.smithi123.stdout:4/430: dread d5/d1c/d42/f4d [0,4194304] 0 2024-02-16T22:21:54.498 INFO:tasks.workunit.client.1.smithi123.stdout:9/505: write d4/d36/d8e/d58/fa2 [599535,13436] 0 2024-02-16T22:21:54.506 INFO:tasks.workunit.client.0.smithi114.stdout:2/275: mkdir dd/df/d1c/d21/d61 0 2024-02-16T22:21:54.507 INFO:tasks.workunit.client.1.smithi123.stdout:3/368: mknod d0/d20/d76/d37/c8e 0 2024-02-16T22:21:54.508 INFO:tasks.workunit.client.1.smithi123.stdout:3/369: chown d0/d50/c51 4631 1 2024-02-16T22:21:54.509 INFO:tasks.workunit.client.1.smithi123.stdout:3/370: readlink d0/d20/d76/d37/l66 0 2024-02-16T22:21:54.510 INFO:tasks.workunit.client.1.smithi123.stdout:3/371: stat d0/d50/d7e 0 2024-02-16T22:21:54.510 INFO:tasks.workunit.client.1.smithi123.stdout:1/350: truncate d3/d6/d18/f21 546860 0 2024-02-16T22:21:54.511 INFO:tasks.workunit.client.1.smithi123.stdout:8/468: dwrite d0/d17/d26/d2a/f92 [0,4194304] 0 2024-02-16T22:21:54.513 INFO:tasks.workunit.client.1.smithi123.stdout:8/469: chown d0/d29/f60 36693514 1 2024-02-16T22:21:54.515 INFO:tasks.workunit.client.1.smithi123.stdout:7/410: getdents d3/d5 0 2024-02-16T22:21:54.516 INFO:tasks.workunit.client.1.smithi123.stdout:8/470: chown d0/d17/d43 430561 1 2024-02-16T22:21:54.516 INFO:tasks.workunit.client.0.smithi114.stdout:8/294: symlink d4/d24/l62 0 2024-02-16T22:21:54.518 INFO:tasks.workunit.client.0.smithi114.stdout:6/246: dread d6/dd/d25/d26/f2d [4194304,4194304] 0 2024-02-16T22:21:54.519 INFO:tasks.workunit.client.1.smithi123.stdout:3/372: dwrite d0/f55 [0,4194304] 0 2024-02-16T22:21:54.520 INFO:tasks.workunit.client.1.smithi123.stdout:0/357: dwrite d1/f1f [0,4194304] 0 2024-02-16T22:21:54.524 INFO:tasks.workunit.client.1.smithi123.stdout:4/431: mknod d5/d1c/d64/d7b/c82 0 2024-02-16T22:21:54.525 INFO:tasks.workunit.client.1.smithi123.stdout:9/506: mknod d4/d36/d8e/d6d/ca5 0 2024-02-16T22:21:54.526 INFO:tasks.workunit.client.0.smithi114.stdout:9/286: mkdir d2/d5c/d5f 0 2024-02-16T22:21:54.527 INFO:tasks.workunit.client.1.smithi123.stdout:2/377: dwrite d4/fc [4194304,4194304] 0 2024-02-16T22:21:54.530 INFO:tasks.workunit.client.1.smithi123.stdout:5/378: mkdir d6/dd/df/d76/d7f 0 2024-02-16T22:21:54.531 INFO:tasks.workunit.client.0.smithi114.stdout:5/258: write d0/d10/f13 [4226964,78938] 0 2024-02-16T22:21:54.539 INFO:tasks.workunit.client.0.smithi114.stdout:2/276: mknod dd/df/c62 0 2024-02-16T22:21:54.539 INFO:tasks.workunit.client.1.smithi123.stdout:3/373: readlink d0/l16 0 2024-02-16T22:21:54.540 INFO:tasks.workunit.client.1.smithi123.stdout:6/427: sync 2024-02-16T22:21:54.540 INFO:tasks.workunit.client.1.smithi123.stdout:8/471: readlink d0/d61/d66/d7e/l88 0 2024-02-16T22:21:54.540 INFO:tasks.workunit.client.1.smithi123.stdout:1/351: dwrite d3/fe [0,4194304] 0 2024-02-16T22:21:54.540 INFO:tasks.workunit.client.1.smithi123.stdout:2/378: dwrite f2 [4194304,4194304] 0 2024-02-16T22:21:54.540 INFO:tasks.workunit.client.0.smithi114.stdout:2/277: read dd/f28 [5961981,20318] 0 2024-02-16T22:21:54.544 INFO:tasks.workunit.client.0.smithi114.stdout:4/358: rename dd/ce to dd/d2e/c7f 0 2024-02-16T22:21:54.545 INFO:tasks.workunit.client.0.smithi114.stdout:0/261: sync 2024-02-16T22:21:54.549 INFO:tasks.workunit.client.0.smithi114.stdout:7/274: getdents d3/d23 0 2024-02-16T22:21:54.551 INFO:tasks.workunit.client.0.smithi114.stdout:1/252: symlink d4/d9/d46/l4f 0 2024-02-16T22:21:54.552 INFO:tasks.workunit.client.1.smithi123.stdout:8/472: stat d0/d17/d26/d68/d81/d47/l4a 0 2024-02-16T22:21:54.552 INFO:tasks.workunit.client.1.smithi123.stdout:6/428: fdatasync d0/d9/dc/d3a/f7d 0 2024-02-16T22:21:54.554 INFO:tasks.workunit.client.0.smithi114.stdout:6/247: dread d6/da/f1b [0,4194304] 0 2024-02-16T22:21:54.555 INFO:tasks.workunit.client.1.smithi123.stdout:6/429: write d0/d9/dc/d3a/f81 [287834,96431] 0 2024-02-16T22:21:54.555 INFO:tasks.workunit.client.0.smithi114.stdout:4/359: dwrite dd/d1b/d24/f7a [0,4194304] 0 2024-02-16T22:21:54.556 INFO:tasks.workunit.client.1.smithi123.stdout:3/374: rename d0/f31 to d0/d20/d76/d37/d49/f8f 0 2024-02-16T22:21:54.557 INFO:tasks.workunit.client.0.smithi114.stdout:5/259: read d0/df/d16/d1e/f21 [660294,101764] 0 2024-02-16T22:21:54.558 INFO:tasks.workunit.client.0.smithi114.stdout:3/247: getdents d4/d26 0 2024-02-16T22:21:54.558 INFO:tasks.workunit.client.1.smithi123.stdout:2/379: mknod d4/d1a/d1d/d13/d15/c7c 0 2024-02-16T22:21:54.560 INFO:tasks.workunit.client.1.smithi123.stdout:2/380: write d4/d1a/d1d/f75 [547060,37116] 0 2024-02-16T22:21:54.561 INFO:tasks.workunit.client.0.smithi114.stdout:0/262: mknod d1/d16/d1f/c53 0 2024-02-16T22:21:54.561 INFO:tasks.workunit.client.0.smithi114.stdout:6/248: dread d6/da/f1b [0,4194304] 0 2024-02-16T22:21:54.561 INFO:tasks.workunit.client.1.smithi123.stdout:8/473: dwrite d0/d17/d43/d34/f38 [0,4194304] 0 2024-02-16T22:21:54.562 INFO:tasks.workunit.client.0.smithi114.stdout:6/249: stat d6/da/f1b 0 2024-02-16T22:21:54.566 INFO:tasks.workunit.client.0.smithi114.stdout:1/253: chown d4/d9/d3d/l2b 75265 1 2024-02-16T22:21:54.569 INFO:tasks.workunit.client.1.smithi123.stdout:8/474: dwrite d0/d12/f74 [0,4194304] 0 2024-02-16T22:21:54.571 INFO:tasks.workunit.client.0.smithi114.stdout:4/360: creat dd/d59/f80 x:0 0 0 2024-02-16T22:21:54.572 INFO:tasks.workunit.client.0.smithi114.stdout:2/278: mkdir dd/df/d1c/d21/d61/d63 0 2024-02-16T22:21:54.573 INFO:tasks.workunit.client.1.smithi123.stdout:3/375: fsync d0/d20/d76/d5d/f86 0 2024-02-16T22:21:54.574 INFO:tasks.workunit.client.1.smithi123.stdout:3/376: stat d0/d20/d29/f84 0 2024-02-16T22:21:54.574 INFO:tasks.workunit.client.0.smithi114.stdout:3/248: mkdir d4/d43/d5b 0 2024-02-16T22:21:54.576 INFO:tasks.workunit.client.1.smithi123.stdout:3/377: dread - d0/d20/d76/d37/d49/f6a zero size 2024-02-16T22:21:54.576 INFO:tasks.workunit.client.1.smithi123.stdout:0/358: getdents d1/d8/d3d/d44 0 2024-02-16T22:21:54.582 INFO:tasks.workunit.client.1.smithi123.stdout:0/359: dwrite d1/d8/f5d [0,4194304] 0 2024-02-16T22:21:54.583 INFO:tasks.workunit.client.0.smithi114.stdout:8/295: getdents d4/d11/d15 0 2024-02-16T22:21:54.583 INFO:tasks.workunit.client.0.smithi114.stdout:8/296: dread - d4/d16/d25/d39/f4f zero size 2024-02-16T22:21:54.584 INFO:tasks.workunit.client.0.smithi114.stdout:1/254: creat d4/d33/f50 x:0 0 0 2024-02-16T22:21:54.588 INFO:tasks.workunit.client.0.smithi114.stdout:9/287: getdents d2 0 2024-02-16T22:21:54.590 INFO:tasks.workunit.client.0.smithi114.stdout:9/288: fdatasync d2/d1a/d23/d35/d47/f4d 0 2024-02-16T22:21:54.590 INFO:tasks.workunit.client.0.smithi114.stdout:4/361: rmdir dd/d1b/d24 39 2024-02-16T22:21:54.591 INFO:tasks.workunit.client.1.smithi123.stdout:7/411: dwrite d3/d87/f65 [0,4194304] 0 2024-02-16T22:21:54.593 INFO:tasks.workunit.client.1.smithi123.stdout:7/412: write d3/d6e/d24/f39 [868450,42700] 0 2024-02-16T22:21:54.593 INFO:tasks.workunit.client.1.smithi123.stdout:0/360: dread d1/d8/d3d/f59 [0,4194304] 0 2024-02-16T22:21:54.594 INFO:tasks.workunit.client.0.smithi114.stdout:6/250: dread d6/d2c/f46 [0,4194304] 0 2024-02-16T22:21:54.594 INFO:tasks.workunit.client.1.smithi123.stdout:7/413: fsync d3/d5/f9 0 2024-02-16T22:21:54.594 INFO:tasks.workunit.client.1.smithi123.stdout:3/378: creat d0/d20/d76/d5d/d60/f90 x:0 0 0 2024-02-16T22:21:54.595 INFO:tasks.workunit.client.0.smithi114.stdout:3/249: creat d4/d46/d48/f5c x:0 0 0 2024-02-16T22:21:54.595 INFO:tasks.workunit.client.1.smithi123.stdout:4/432: dwrite d5/f29 [0,4194304] 0 2024-02-16T22:21:54.596 INFO:tasks.workunit.client.1.smithi123.stdout:9/507: write d4/f12 [2510078,50772] 0 2024-02-16T22:21:54.601 INFO:tasks.workunit.client.0.smithi114.stdout:8/297: symlink d4/d16/l63 0 2024-02-16T22:21:54.603 INFO:tasks.workunit.client.0.smithi114.stdout:3/250: dwrite d4/dd/f13 [0,4194304] 0 2024-02-16T22:21:54.604 INFO:tasks.workunit.client.0.smithi114.stdout:3/251: chown d4/c2a 106079265 1 2024-02-16T22:21:54.606 INFO:tasks.workunit.client.1.smithi123.stdout:8/475: dread d0/d17/d26/d32/d45/f5f [0,4194304] 0 2024-02-16T22:21:54.607 INFO:tasks.workunit.client.1.smithi123.stdout:8/476: chown d0/d17/d26/d68/d81/d47 409125 1 2024-02-16T22:21:54.607 INFO:tasks.workunit.client.0.smithi114.stdout:4/362: mknod dd/d1b/d41/c81 0 2024-02-16T22:21:54.608 INFO:tasks.workunit.client.0.smithi114.stdout:4/363: dread - dd/d59/f71 zero size 2024-02-16T22:21:54.609 INFO:tasks.workunit.client.0.smithi114.stdout:7/275: write d3/fc [2983018,89551] 0 2024-02-16T22:21:54.610 INFO:tasks.workunit.client.0.smithi114.stdout:9/289: mknod d2/d33/d3f/c60 0 2024-02-16T22:21:54.614 INFO:tasks.workunit.client.1.smithi123.stdout:5/379: truncate d6/dd/df/d1f/d25/f17 713705 0 2024-02-16T22:21:54.614 INFO:tasks.workunit.client.0.smithi114.stdout:7/276: dread d3/d22/f48 [0,4194304] 0 2024-02-16T22:21:54.615 INFO:tasks.workunit.client.0.smithi114.stdout:0/263: dread d1/f2a [0,4194304] 0 2024-02-16T22:21:54.616 INFO:tasks.workunit.client.0.smithi114.stdout:6/251: creat d6/d2c/d38/f4f x:0 0 0 2024-02-16T22:21:54.617 INFO:tasks.workunit.client.1.smithi123.stdout:3/379: symlink d0/d40/l91 0 2024-02-16T22:21:54.617 INFO:tasks.workunit.client.0.smithi114.stdout:6/252: write d6/f35 [606663,21666] 0 2024-02-16T22:21:54.619 INFO:tasks.workunit.client.0.smithi114.stdout:6/253: write d6/dd/d25/f27 [303439,3454] 0 2024-02-16T22:21:54.620 INFO:tasks.workunit.client.1.smithi123.stdout:5/380: dwrite d6/dd/df/f79 [0,4194304] 0 2024-02-16T22:21:54.621 INFO:tasks.workunit.client.0.smithi114.stdout:8/298: creat d4/d24/f64 x:0 0 0 2024-02-16T22:21:54.623 INFO:tasks.workunit.client.0.smithi114.stdout:0/264: dread d1/d1a/d3e/d41/f42 [0,4194304] 0 2024-02-16T22:21:54.623 INFO:tasks.workunit.client.1.smithi123.stdout:5/381: dread d6/dd/df/d58/f4f [0,4194304] 0 2024-02-16T22:21:54.624 INFO:tasks.workunit.client.1.smithi123.stdout:3/380: dwrite d0/d20/d2e/f48 [0,4194304] 0 2024-02-16T22:21:54.624 INFO:tasks.workunit.client.1.smithi123.stdout:5/382: stat d6/dd/df/d1f/d25/d37/l3d 0 2024-02-16T22:21:54.624 INFO:tasks.workunit.client.1.smithi123.stdout:0/361: rename d1/d8/fa to d1/d4/d17/d74/d57/f75 0 2024-02-16T22:21:54.625 INFO:tasks.workunit.client.1.smithi123.stdout:0/362: fsync d1/f2e 0 2024-02-16T22:21:54.627 INFO:tasks.workunit.client.0.smithi114.stdout:1/255: symlink d4/d9/d3d/d3f/l51 0 2024-02-16T22:21:54.630 INFO:tasks.workunit.client.1.smithi123.stdout:6/430: sync 2024-02-16T22:21:54.630 INFO:tasks.workunit.client.0.smithi114.stdout:6/254: dwrite d6/f35 [0,4194304] 0 2024-02-16T22:21:54.631 INFO:tasks.workunit.client.0.smithi114.stdout:6/255: read d6/f35 [4174979,54043] 0 2024-02-16T22:21:54.632 INFO:tasks.workunit.client.0.smithi114.stdout:3/252: symlink d4/d46/d48/l5d 0 2024-02-16T22:21:54.633 INFO:tasks.workunit.client.1.smithi123.stdout:7/414: creat d3/d6e/d52/f88 x:0 0 0 2024-02-16T22:21:54.634 INFO:tasks.workunit.client.1.smithi123.stdout:1/352: truncate d3/d6/f26 641392 0 2024-02-16T22:21:54.635 INFO:tasks.workunit.client.1.smithi123.stdout:9/508: mkdir d4/d36/d8e/d6d/d71/da6 0 2024-02-16T22:21:54.637 INFO:tasks.workunit.client.1.smithi123.stdout:4/433: creat d5/d1c/d23/d31/f83 x:0 0 0 2024-02-16T22:21:54.639 INFO:tasks.workunit.client.1.smithi123.stdout:3/381: dwrite d0/d20/d76/f35 [0,4194304] 0 2024-02-16T22:21:54.640 INFO:tasks.workunit.client.1.smithi123.stdout:1/353: write d3/d6/d11/d76/f59 [309248,97008] 0 2024-02-16T22:21:54.641 INFO:tasks.workunit.client.1.smithi123.stdout:0/363: dwrite d1/d4/d17/d74/f52 [0,4194304] 0 2024-02-16T22:21:54.645 INFO:tasks.workunit.client.0.smithi114.stdout:3/253: dwrite d4/d46/d48/f57 [0,4194304] 0 2024-02-16T22:21:54.647 INFO:tasks.workunit.client.0.smithi114.stdout:4/364: dwrite dd/d1b/d24/f4a [0,4194304] 0 2024-02-16T22:21:54.647 INFO:tasks.workunit.client.1.smithi123.stdout:4/434: dread d5/d1c/f30 [0,4194304] 0 2024-02-16T22:21:54.648 INFO:tasks.workunit.client.1.smithi123.stdout:7/415: dwrite d3/d26/d35/f77 [0,4194304] 0 2024-02-16T22:21:54.649 INFO:tasks.workunit.client.0.smithi114.stdout:9/290: dread d2/d1a/d23/d35/f31 [0,4194304] 0 2024-02-16T22:21:54.653 INFO:tasks.workunit.client.0.smithi114.stdout:9/291: stat d2/d1a/d23/f56 0 2024-02-16T22:21:54.659 INFO:tasks.workunit.client.0.smithi114.stdout:5/260: dwrite d0/d10/f23 [0,4194304] 0 2024-02-16T22:21:54.666 INFO:tasks.workunit.client.0.smithi114.stdout:8/299: creat d4/d11/d15/d36/f65 x:0 0 0 2024-02-16T22:21:54.667 INFO:tasks.workunit.client.0.smithi114.stdout:8/300: truncate d4/d16/f4b 864810 0 2024-02-16T22:21:54.670 INFO:tasks.workunit.client.0.smithi114.stdout:1/256: unlink d4/d9/d46/d47/f4b 0 2024-02-16T22:21:54.672 INFO:tasks.workunit.client.0.smithi114.stdout:1/257: write d4/d9/d3d/f39 [4587518,106777] 0 2024-02-16T22:21:54.686 INFO:tasks.workunit.client.0.smithi114.stdout:9/292: mknod d2/d1a/d23/d32/c61 0 2024-02-16T22:21:54.692 INFO:tasks.workunit.client.1.smithi123.stdout:0/364: dread d1/d8/f34 [0,4194304] 0 2024-02-16T22:21:54.693 INFO:tasks.workunit.client.0.smithi114.stdout:9/293: dwrite d2/dd/f3b [0,4194304] 0 2024-02-16T22:21:54.695 INFO:tasks.workunit.client.0.smithi114.stdout:8/301: symlink d4/d11/d15/l66 0 2024-02-16T22:21:54.697 INFO:tasks.workunit.client.1.smithi123.stdout:6/431: dwrite d0/d9/dc/f67 [0,4194304] 0 2024-02-16T22:21:54.699 INFO:tasks.workunit.client.1.smithi123.stdout:9/509: write d4/fa1 [221358,14820] 0 2024-02-16T22:21:54.701 INFO:tasks.workunit.client.0.smithi114.stdout:9/294: dwrite d2/d33/f57 [0,4194304] 0 2024-02-16T22:21:54.709 INFO:tasks.workunit.client.1.smithi123.stdout:8/477: symlink d0/d12/d3e/d7c/l9a 0 2024-02-16T22:21:54.710 INFO:tasks.workunit.client.0.smithi114.stdout:6/256: dread d6/dd/d25/d34/f40 [0,4194304] 0 2024-02-16T22:21:54.714 INFO:tasks.workunit.client.1.smithi123.stdout:7/416: symlink d3/d26/d3d/l89 0 2024-02-16T22:21:54.715 INFO:tasks.workunit.client.1.smithi123.stdout:7/417: chown d3/d6e/d24/l74 20 1 2024-02-16T22:21:54.716 INFO:tasks.workunit.client.1.smithi123.stdout:2/381: link d4/d1a/d1d/c5b d4/d1a/d1d/d59/c7d 0 2024-02-16T22:21:54.718 INFO:tasks.workunit.client.0.smithi114.stdout:5/261: symlink d0/df/d16/l48 0 2024-02-16T22:21:54.719 INFO:tasks.workunit.client.1.smithi123.stdout:5/383: symlink d6/dd/df/d1f/l80 0 2024-02-16T22:21:54.720 INFO:tasks.workunit.client.0.smithi114.stdout:8/302: mkdir d4/d24/d2c/d67 0 2024-02-16T22:21:54.721 INFO:tasks.workunit.client.0.smithi114.stdout:8/303: write d4/d11/d15/d36/f5d [571508,17916] 0 2024-02-16T22:21:54.723 INFO:tasks.workunit.client.1.smithi123.stdout:3/382: sync 2024-02-16T22:21:54.723 INFO:tasks.workunit.client.1.smithi123.stdout:0/365: mknod d1/d4/d17/d74/c76 0 2024-02-16T22:21:54.723 INFO:tasks.workunit.client.1.smithi123.stdout:3/383: chown d0/d20/c57 21524579 1 2024-02-16T22:21:54.725 INFO:tasks.workunit.client.1.smithi123.stdout:6/432: mknod d0/d6/d1a/d2e/c9c 0 2024-02-16T22:21:54.726 INFO:tasks.workunit.client.0.smithi114.stdout:9/295: creat d2/d1a/d23/d35/d1d/f62 x:0 0 0 2024-02-16T22:21:54.727 INFO:tasks.workunit.client.0.smithi114.stdout:9/296: write d2/fb [1184952,121652] 0 2024-02-16T22:21:54.728 INFO:tasks.workunit.client.0.smithi114.stdout:9/297: stat d2/d5c/d5f 0 2024-02-16T22:21:54.733 INFO:tasks.workunit.client.0.smithi114.stdout:6/257: symlink d6/d2c/d38/l50 0 2024-02-16T22:21:54.734 INFO:tasks.workunit.client.0.smithi114.stdout:4/365: link f9 dd/d1b/d37/d38/d7c/f82 0 2024-02-16T22:21:54.735 INFO:tasks.workunit.client.0.smithi114.stdout:6/258: write d6/dd/d25/d34/f42 [869976,15408] 0 2024-02-16T22:21:54.736 INFO:tasks.workunit.client.0.smithi114.stdout:9/298: dwrite d2/dd/d3c/f4f [0,4194304] 0 2024-02-16T22:21:54.736 INFO:tasks.workunit.client.0.smithi114.stdout:7/277: getdents d3/d23/d4e 0 2024-02-16T22:21:54.737 INFO:tasks.workunit.client.0.smithi114.stdout:6/259: readlink d6/d2c/d38/l50 0 2024-02-16T22:21:54.738 INFO:tasks.workunit.client.0.smithi114.stdout:4/366: write dd/d1b/d41/f62 [570064,50326] 0 2024-02-16T22:21:54.741 INFO:tasks.workunit.client.1.smithi123.stdout:7/418: mknod d3/d6e/d52/c8a 0 2024-02-16T22:21:54.743 INFO:tasks.workunit.client.0.smithi114.stdout:0/265: getdents d1 0 2024-02-16T22:21:54.744 INFO:tasks.workunit.client.1.smithi123.stdout:2/382: rename d4/d1a/d1d/d13/l78 to d4/d1a/d1d/d38/l7e 0 2024-02-16T22:21:54.748 INFO:tasks.workunit.client.1.smithi123.stdout:3/384: stat d0/d40/l46 0 2024-02-16T22:21:54.750 INFO:tasks.workunit.client.1.smithi123.stdout:2/383: dwrite d4/fc [4194304,4194304] 0 2024-02-16T22:21:54.751 INFO:tasks.workunit.client.0.smithi114.stdout:9/299: unlink d2/d1a/d23/d35/d1f/f2e 0 2024-02-16T22:21:54.753 INFO:tasks.workunit.client.0.smithi114.stdout:6/260: symlink d6/da/l51 0 2024-02-16T22:21:54.758 INFO:tasks.workunit.client.0.smithi114.stdout:7/278: dread d3/d14/f2c [0,4194304] 0 2024-02-16T22:21:54.759 INFO:tasks.workunit.client.0.smithi114.stdout:4/367: creat dd/d1b/d41/f83 x:0 0 0 2024-02-16T22:21:54.760 INFO:tasks.workunit.client.0.smithi114.stdout:6/261: dread d6/da/f3f [0,4194304] 0 2024-02-16T22:21:54.761 INFO:tasks.workunit.client.0.smithi114.stdout:0/266: creat d1/d16/d1f/d28/f54 x:0 0 0 2024-02-16T22:21:54.761 INFO:tasks.workunit.client.0.smithi114.stdout:1/258: write d4/f15 [5008471,113699] 0 2024-02-16T22:21:54.762 INFO:tasks.workunit.client.1.smithi123.stdout:1/354: creat d3/f7c x:0 0 0 2024-02-16T22:21:54.763 INFO:tasks.workunit.client.1.smithi123.stdout:1/355: write d3/f8 [5064037,7640] 0 2024-02-16T22:21:54.764 INFO:tasks.workunit.client.0.smithi114.stdout:8/304: symlink d4/d16/d25/l68 0 2024-02-16T22:21:54.764 INFO:tasks.workunit.client.1.smithi123.stdout:1/356: truncate d3/d6/d49/f66 885932 0 2024-02-16T22:21:54.766 INFO:tasks.workunit.client.1.smithi123.stdout:8/478: unlink d0/d17/d26/d32/d45/d67/l8e 0 2024-02-16T22:21:54.766 INFO:tasks.workunit.client.0.smithi114.stdout:8/305: dread d4/d11/f1f [0,4194304] 0 2024-02-16T22:21:54.768 INFO:tasks.workunit.client.1.smithi123.stdout:4/435: write d5/f1f [1512272,53911] 0 2024-02-16T22:21:54.768 INFO:tasks.workunit.client.0.smithi114.stdout:9/300: mkdir d2/d1a/d23/d35/d47/d63 0 2024-02-16T22:21:54.770 INFO:tasks.workunit.client.1.smithi123.stdout:8/479: dread d0/d12/f74 [0,4194304] 0 2024-02-16T22:21:54.773 INFO:tasks.workunit.client.0.smithi114.stdout:9/301: dread d2/dd/d3c/f4f [0,4194304] 0 2024-02-16T22:21:54.773 INFO:tasks.workunit.client.0.smithi114.stdout:5/262: creat d0/df/d16/f49 x:0 0 0 2024-02-16T22:21:54.774 INFO:tasks.workunit.client.1.smithi123.stdout:8/480: dwrite d0/df/f1b [0,4194304] 0 2024-02-16T22:21:54.774 INFO:tasks.workunit.client.0.smithi114.stdout:5/263: write d0/f8 [3635217,26593] 0 2024-02-16T22:21:54.775 INFO:tasks.workunit.client.0.smithi114.stdout:5/264: chown d0/l6 1 1 2024-02-16T22:21:54.776 INFO:tasks.workunit.client.1.smithi123.stdout:5/384: link d6/dd/df/f41 d6/dd/df/d58/d3f/d6f/f81 0 2024-02-16T22:21:54.781 INFO:tasks.workunit.client.1.smithi123.stdout:3/385: dread - d0/d20/d29/f43 zero size 2024-02-16T22:21:54.782 INFO:tasks.workunit.client.0.smithi114.stdout:4/368: rename ca to dd/d1b/d37/d61/c84 0 2024-02-16T22:21:54.782 INFO:tasks.workunit.client.0.smithi114.stdout:9/302: dwrite d2/d1a/d23/d35/d47/f4d [0,4194304] 0 2024-02-16T22:21:54.783 INFO:tasks.workunit.client.1.smithi123.stdout:9/510: truncate d4/d17/d2b/f64 1748436 0 2024-02-16T22:21:54.785 INFO:tasks.workunit.client.0.smithi114.stdout:6/262: unlink d6/dd/d25/d34/f48 0 2024-02-16T22:21:54.785 INFO:tasks.workunit.client.0.smithi114.stdout:0/267: dread d1/d16/d1f/f4d [0,4194304] 0 2024-02-16T22:21:54.786 INFO:tasks.workunit.client.0.smithi114.stdout:6/263: fsync d6/f21 0 2024-02-16T22:21:54.786 INFO:tasks.workunit.client.0.smithi114.stdout:1/259: mknod d4/d9/d46/c52 0 2024-02-16T22:21:54.787 INFO:tasks.workunit.client.1.smithi123.stdout:3/386: dwrite d0/d50/d75/f7d [0,4194304] 0 2024-02-16T22:21:54.787 INFO:tasks.workunit.client.1.smithi123.stdout:2/384: fdatasync d4/d1a/d1d/d13/d25/f66 0 2024-02-16T22:21:54.789 INFO:tasks.workunit.client.1.smithi123.stdout:3/387: fdatasync d0/d20/f42 0 2024-02-16T22:21:54.789 INFO:tasks.workunit.client.1.smithi123.stdout:6/433: truncate d0/d6/d1a/d2e/f7f 1973003 0 2024-02-16T22:21:54.790 INFO:tasks.workunit.client.1.smithi123.stdout:3/388: fdatasync d0/d20/f68 0 2024-02-16T22:21:54.791 INFO:tasks.workunit.client.1.smithi123.stdout:2/385: write d4/d1a/d1d/d59/d45/f4b [1325201,125905] 0 2024-02-16T22:21:54.792 INFO:tasks.workunit.client.1.smithi123.stdout:1/357: creat d3/d47/f7d x:0 0 0 2024-02-16T22:21:54.794 INFO:tasks.workunit.client.0.smithi114.stdout:5/265: unlink d0/fb 0 2024-02-16T22:21:54.794 INFO:tasks.workunit.client.1.smithi123.stdout:2/386: fsync d4/d1a/d1d/d13/d15/d6d/d4a/f5c 0 2024-02-16T22:21:54.795 INFO:tasks.workunit.client.1.smithi123.stdout:4/436: mkdir d5/d1c/d64/d7b/d84 0 2024-02-16T22:21:54.798 INFO:tasks.workunit.client.1.smithi123.stdout:6/434: dwrite d0/d9/dc/f67 [0,4194304] 0 2024-02-16T22:21:54.799 INFO:tasks.workunit.client.1.smithi123.stdout:7/419: rename d3/d87/f78 to d3/d6e/d24/f8b 0 2024-02-16T22:21:54.799 INFO:tasks.workunit.client.1.smithi123.stdout:6/435: chown d0/d6/d1a/d88 9935 1 2024-02-16T22:21:54.799 INFO:tasks.workunit.client.0.smithi114.stdout:7/279: rename d3/d22/f34 to d3/d14/f5c 0 2024-02-16T22:21:54.799 INFO:tasks.workunit.client.1.smithi123.stdout:7/420: stat d3/d6e/f1b 0 2024-02-16T22:21:54.800 INFO:tasks.workunit.client.1.smithi123.stdout:7/421: write d3/f8 [5997015,2483] 0 2024-02-16T22:21:54.801 INFO:tasks.workunit.client.1.smithi123.stdout:8/481: mkdir d0/d17/d26/d68/d81/d47/d9b 0 2024-02-16T22:21:54.802 INFO:tasks.workunit.client.1.smithi123.stdout:8/482: chown d0/d17/d43/f27 6 1 2024-02-16T22:21:54.810 INFO:tasks.workunit.client.1.smithi123.stdout:9/511: mkdir d4/d36/d8e/d58/d6e/da7 0 2024-02-16T22:21:54.818 INFO:tasks.workunit.client.0.smithi114.stdout:5/266: dread d0/d28/f46 [0,4194304] 0 2024-02-16T22:21:54.820 INFO:tasks.workunit.client.0.smithi114.stdout:6/264: dread d6/f35 [0,4194304] 0 2024-02-16T22:21:54.822 INFO:tasks.workunit.client.0.smithi114.stdout:6/265: write d6/d2c/d38/f4f [973805,23460] 0 2024-02-16T22:21:54.822 INFO:tasks.workunit.client.0.smithi114.stdout:8/306: symlink d4/d16/d25/l69 0 2024-02-16T22:21:54.822 INFO:tasks.workunit.client.1.smithi123.stdout:3/389: dread d0/d20/d76/f36 [0,4194304] 0 2024-02-16T22:21:54.824 INFO:tasks.workunit.client.0.smithi114.stdout:8/307: write d4/d11/d15/d36/d3b/f44 [670167,49591] 0 2024-02-16T22:21:54.829 INFO:tasks.workunit.client.0.smithi114.stdout:1/260: rmdir d4 39 2024-02-16T22:21:54.832 INFO:tasks.workunit.client.1.smithi123.stdout:1/358: unlink d3/c19 0 2024-02-16T22:21:54.834 INFO:tasks.workunit.client.0.smithi114.stdout:7/280: truncate d3/d46/f47 3882999 0 2024-02-16T22:21:54.835 INFO:tasks.workunit.client.0.smithi114.stdout:7/281: write d3/d3d/f4a [20080,77170] 0 2024-02-16T22:21:54.836 INFO:tasks.workunit.client.0.smithi114.stdout:7/282: chown d3/d14/f4d 32920 1 2024-02-16T22:21:54.836 INFO:tasks.workunit.client.1.smithi123.stdout:2/387: unlink d4/d1a/d1d/d13/d25/d72/c74 0 2024-02-16T22:21:54.837 INFO:tasks.workunit.client.1.smithi123.stdout:2/388: write d4/fc [2956060,116690] 0 2024-02-16T22:21:54.837 INFO:tasks.workunit.client.1.smithi123.stdout:4/437: symlink d5/d1c/d23/d47/l85 0 2024-02-16T22:21:54.838 INFO:tasks.workunit.client.1.smithi123.stdout:2/389: truncate d4/d1a/f6a 119618 0 2024-02-16T22:21:54.838 INFO:tasks.workunit.client.1.smithi123.stdout:4/438: read d5/d1c/f24 [1410972,65163] 0 2024-02-16T22:21:54.840 INFO:tasks.workunit.client.1.smithi123.stdout:6/436: unlink d0/d9/dc/d3a/c3b 0 2024-02-16T22:21:54.841 INFO:tasks.workunit.client.1.smithi123.stdout:6/437: fsync d0/d6/d2f/d5d/f68 0 2024-02-16T22:21:54.841 INFO:tasks.workunit.client.0.smithi114.stdout:0/268: mkdir d1/d16/d55 0 2024-02-16T22:21:54.842 INFO:tasks.workunit.client.0.smithi114.stdout:0/269: stat d1/d1a/f36 0 2024-02-16T22:21:54.846 INFO:tasks.workunit.client.1.smithi123.stdout:9/512: creat d4/d36/fa8 x:0 0 0 2024-02-16T22:21:54.847 INFO:tasks.workunit.client.1.smithi123.stdout:8/483: dread d0/d12/f74 [0,4194304] 0 2024-02-16T22:21:54.847 INFO:tasks.workunit.client.1.smithi123.stdout:0/366: link d1/d8/d3d/l49 d1/d8/l77 0 2024-02-16T22:21:54.848 INFO:tasks.workunit.client.0.smithi114.stdout:6/266: rename d6/da/l1f to d6/da/l52 0 2024-02-16T22:21:54.849 INFO:tasks.workunit.client.0.smithi114.stdout:4/369: write dd/d1b/d24/f4c [1984015,24596] 0 2024-02-16T22:21:54.850 INFO:tasks.workunit.client.1.smithi123.stdout:3/390: mknod d0/d50/d75/c92 0 2024-02-16T22:21:54.850 INFO:tasks.workunit.client.0.smithi114.stdout:6/267: fsync d6/d2c/d38/d41/f4a 0 2024-02-16T22:21:54.850 INFO:tasks.workunit.client.0.smithi114.stdout:6/268: fsync d6/d2c/f46 0 2024-02-16T22:21:54.852 INFO:tasks.workunit.client.0.smithi114.stdout:6/269: write d6/dd/d25/f27 [477696,75135] 0 2024-02-16T22:21:54.852 INFO:tasks.workunit.client.0.smithi114.stdout:8/308: rename d4/d11/d15/d36/d3b/c3f to d4/d16/d25/d39/c6a 0 2024-02-16T22:21:54.855 INFO:tasks.workunit.client.0.smithi114.stdout:7/283: chown d3/d22/c36 31 1 2024-02-16T22:21:54.857 INFO:tasks.workunit.client.1.smithi123.stdout:1/359: dread d3/d6/db/d10/f4a [0,4194304] 0 2024-02-16T22:21:54.857 INFO:tasks.workunit.client.1.smithi123.stdout:5/385: sync 2024-02-16T22:21:54.858 INFO:tasks.workunit.client.1.smithi123.stdout:1/360: write d3/d6/d11/f1f [2419693,25955] 0 2024-02-16T22:21:54.859 INFO:tasks.workunit.client.1.smithi123.stdout:4/439: rename d5/f14 to d5/d1c/d64/d7b/d84/f86 0 2024-02-16T22:21:54.859 INFO:tasks.workunit.client.1.smithi123.stdout:1/361: write d3/d6/d11/f1f [4242618,38398] 0 2024-02-16T22:21:54.860 INFO:tasks.workunit.client.0.smithi114.stdout:5/267: mknod d0/c4a 0 2024-02-16T22:21:54.860 INFO:tasks.workunit.client.0.smithi114.stdout:5/268: chown d0/c4a 1356028 1 2024-02-16T22:21:54.860 INFO:tasks.workunit.client.1.smithi123.stdout:4/440: read - d5/d1c/d42/f6c zero size 2024-02-16T22:21:54.861 INFO:tasks.workunit.client.1.smithi123.stdout:1/362: chown d3/d6/db/l22 601685 1 2024-02-16T22:21:54.866 INFO:tasks.workunit.client.1.smithi123.stdout:1/363: dwrite d3/d6/db/d10/d2d/f5a [0,4194304] 0 2024-02-16T22:21:54.868 INFO:tasks.workunit.client.1.smithi123.stdout:9/513: mknod d4/d36/ca9 0 2024-02-16T22:21:54.868 INFO:tasks.workunit.client.0.smithi114.stdout:1/261: mknod d4/d9/d46/c53 0 2024-02-16T22:21:54.871 INFO:tasks.workunit.client.1.smithi123.stdout:7/422: dwrite d3/d6e/d1d/f64 [0,4194304] 0 2024-02-16T22:21:54.872 INFO:tasks.workunit.client.0.smithi114.stdout:8/309: dwrite d4/d11/f1f [0,4194304] 0 2024-02-16T22:21:54.873 INFO:tasks.workunit.client.0.smithi114.stdout:7/284: symlink d3/d22/l5d 0 2024-02-16T22:21:54.875 INFO:tasks.workunit.client.0.smithi114.stdout:7/285: chown d3/ff 1798363 1 2024-02-16T22:21:54.875 INFO:tasks.workunit.client.1.smithi123.stdout:0/367: creat d1/d8/d13/d1d/d62/d6d/f78 x:0 0 0 2024-02-16T22:21:54.875 INFO:tasks.workunit.client.0.smithi114.stdout:7/286: truncate d3/d46/f51 438576 0 2024-02-16T22:21:54.877 INFO:tasks.workunit.client.0.smithi114.stdout:9/303: getdents d2/d1a/d23/d35/d1d 0 2024-02-16T22:21:54.877 INFO:tasks.workunit.client.0.smithi114.stdout:9/304: readlink d2/d33/d3f/l42 0 2024-02-16T22:21:54.879 INFO:tasks.workunit.client.0.smithi114.stdout:9/305: write d2/d1a/d23/d35/d1d/f62 [840665,21439] 0 2024-02-16T22:21:54.883 INFO:tasks.workunit.client.0.smithi114.stdout:4/370: creat dd/d1b/d24/d57/f85 x:0 0 0 2024-02-16T22:21:54.883 INFO:tasks.workunit.client.0.smithi114.stdout:4/371: chown dd/d1b 407 1 2024-02-16T22:21:54.891 INFO:tasks.workunit.client.1.smithi123.stdout:5/386: fsync d6/dd/df/d58/d3f/d6f/f81 0 2024-02-16T22:21:54.892 INFO:tasks.workunit.client.0.smithi114.stdout:8/310: creat d4/d11/d15/d36/f6b x:0 0 0 2024-02-16T22:21:54.894 INFO:tasks.workunit.client.0.smithi114.stdout:7/287: creat d3/d23/f5e x:0 0 0 2024-02-16T22:21:54.894 INFO:tasks.workunit.client.0.smithi114.stdout:8/311: read d4/d24/d2c/f56 [1489421,106804] 0 2024-02-16T22:21:54.895 INFO:tasks.workunit.client.1.smithi123.stdout:7/423: dread d3/d5/f28 [0,4194304] 0 2024-02-16T22:21:54.895 INFO:tasks.workunit.client.1.smithi123.stdout:2/390: rename d4/d1a/d1d/c5b to d4/d1a/d1d/d13/d15/d6d/c7f 0 2024-02-16T22:21:54.896 INFO:tasks.workunit.client.0.smithi114.stdout:9/306: creat d2/d1a/d23/d35/d1f/f64 x:0 0 0 2024-02-16T22:21:54.899 INFO:tasks.workunit.client.0.smithi114.stdout:6/270: creat d6/dd/d25/f53 x:0 0 0 2024-02-16T22:21:54.900 INFO:tasks.workunit.client.1.smithi123.stdout:2/391: dwrite d4/d1a/d1d/f2b [0,4194304] 0 2024-02-16T22:21:54.902 INFO:tasks.workunit.client.1.smithi123.stdout:2/392: truncate d4/d1a/d1d/d13/d25/f43 4247903 0 2024-02-16T22:21:54.902 INFO:tasks.workunit.client.0.smithi114.stdout:1/262: link d4/d9/d3d/f41 d4/d9/d3d/f54 0 2024-02-16T22:21:54.904 INFO:tasks.workunit.client.0.smithi114.stdout:7/288: chown d3/d22/c57 595 1 2024-02-16T22:21:54.905 INFO:tasks.workunit.client.1.smithi123.stdout:1/364: creat d3/d6/d18/f7e x:0 0 0 2024-02-16T22:21:54.908 INFO:tasks.workunit.client.1.smithi123.stdout:2/393: dwrite d4/d1a/d1d/d13/d25/f4d [0,4194304] 0 2024-02-16T22:21:54.909 INFO:tasks.workunit.client.0.smithi114.stdout:4/372: symlink dd/d1b/d37/l86 0 2024-02-16T22:21:54.909 INFO:tasks.workunit.client.1.smithi123.stdout:2/394: readlink d4/d1a/d1d/l6e 0 2024-02-16T22:21:54.910 INFO:tasks.workunit.client.1.smithi123.stdout:2/395: truncate d4/d1a/d1d/d13/f3c 1008614 0 2024-02-16T22:21:54.911 INFO:tasks.workunit.client.1.smithi123.stdout:2/396: chown d4/d1a/d1d/d13/d25/l4c 30179074 1 2024-02-16T22:21:54.913 INFO:tasks.workunit.client.1.smithi123.stdout:9/514: chown d4/d17/f55 2593090 1 2024-02-16T22:21:54.913 INFO:tasks.workunit.client.1.smithi123.stdout:9/515: chown d4/fa 24695253 1 2024-02-16T22:21:54.913 INFO:tasks.workunit.client.1.smithi123.stdout:9/516: fsync d4/f12 0 2024-02-16T22:21:54.916 INFO:tasks.workunit.client.1.smithi123.stdout:8/484: mkdir d0/d9c 0 2024-02-16T22:21:54.916 INFO:tasks.workunit.client.1.smithi123.stdout:8/485: chown d0/d12/f74 258329 1 2024-02-16T22:21:54.918 INFO:tasks.workunit.client.0.smithi114.stdout:7/289: symlink d3/d14/l5f 0 2024-02-16T22:21:54.920 INFO:tasks.workunit.client.0.smithi114.stdout:9/307: mkdir d2/d1a/d65 0 2024-02-16T22:21:54.921 INFO:tasks.workunit.client.0.smithi114.stdout:6/271: dread d6/f18 [0,4194304] 0 2024-02-16T22:21:54.923 INFO:tasks.workunit.client.0.smithi114.stdout:4/373: mkdir dd/d1b/d37/d38/d87 0 2024-02-16T22:21:54.925 INFO:tasks.workunit.client.0.smithi114.stdout:4/374: write dd/d1b/d24/f2f [1025645,22146] 0 2024-02-16T22:21:54.926 INFO:tasks.workunit.client.0.smithi114.stdout:1/263: mknod d4/d9/c55 0 2024-02-16T22:21:54.928 INFO:tasks.workunit.client.0.smithi114.stdout:7/290: rename d3/d22/l5d to d3/d23/l60 0 2024-02-16T22:21:54.928 INFO:tasks.workunit.client.1.smithi123.stdout:5/387: rmdir d6/dd/df 39 2024-02-16T22:21:54.929 INFO:tasks.workunit.client.0.smithi114.stdout:7/291: fsync d3/d14/f43 0 2024-02-16T22:21:54.929 INFO:tasks.workunit.client.0.smithi114.stdout:4/375: read dd/f32 [2548478,81891] 0 2024-02-16T22:21:54.930 INFO:tasks.workunit.client.0.smithi114.stdout:9/308: mknod d2/d5c/c66 0 2024-02-16T22:21:54.932 INFO:tasks.workunit.client.0.smithi114.stdout:4/376: write dd/d1b/d37/d61/f66 [648405,12929] 0 2024-02-16T22:21:54.934 INFO:tasks.workunit.client.1.smithi123.stdout:4/441: mkdir d5/d1c/d23/d56/d6e/d73/d87 0 2024-02-16T22:21:54.936 INFO:tasks.workunit.client.0.smithi114.stdout:6/272: rename d6/dd/d25/d26 to d6/d2c/d38/d54 0 2024-02-16T22:21:54.940 INFO:tasks.workunit.client.0.smithi114.stdout:7/292: mkdir d3/d22/d61 0 2024-02-16T22:21:54.941 INFO:tasks.workunit.client.1.smithi123.stdout:1/365: mknod d3/d6/d49/d48/c7f 0 2024-02-16T22:21:54.942 INFO:tasks.workunit.client.0.smithi114.stdout:6/273: dwrite d6/dd/f17 [4194304,4194304] 0 2024-02-16T22:21:54.946 INFO:tasks.workunit.client.1.smithi123.stdout:3/391: write d0/fd [313273,54636] 0 2024-02-16T22:21:54.946 INFO:tasks.workunit.client.0.smithi114.stdout:6/274: dread d6/dd/f29 [0,4194304] 0 2024-02-16T22:21:54.948 INFO:tasks.workunit.client.0.smithi114.stdout:4/377: creat dd/d1b/d68/f88 x:0 0 0 2024-02-16T22:21:54.952 INFO:tasks.workunit.client.1.smithi123.stdout:3/392: dwrite d0/d20/f8b [0,4194304] 0 2024-02-16T22:21:54.952 INFO:tasks.workunit.client.1.smithi123.stdout:9/517: rmdir d4 39 2024-02-16T22:21:54.953 INFO:tasks.workunit.client.1.smithi123.stdout:3/393: chown d0/f15 1 1 2024-02-16T22:21:54.954 INFO:tasks.workunit.client.0.smithi114.stdout:1/264: mknod d4/d9/d3d/c56 0 2024-02-16T22:21:54.956 INFO:tasks.workunit.client.1.smithi123.stdout:8/486: creat d0/d17/d26/d68/d81/d47/f9d x:0 0 0 2024-02-16T22:21:54.960 INFO:tasks.workunit.client.1.smithi123.stdout:3/394: dread d0/d20/d2e/f48 [0,4194304] 0 2024-02-16T22:21:54.962 INFO:tasks.workunit.client.1.smithi123.stdout:7/424: dwrite d3/fd [0,4194304] 0 2024-02-16T22:21:54.962 INFO:tasks.workunit.client.0.smithi114.stdout:9/309: link d2/d1a/d23/d35/f31 d2/d1a/d23/d35/f67 0 2024-02-16T22:21:54.964 INFO:tasks.workunit.client.0.smithi114.stdout:6/275: symlink d6/dd/l55 0 2024-02-16T22:21:54.965 INFO:tasks.workunit.client.1.smithi123.stdout:8/487: dwrite d0/d61/d66/d7e/f89 [0,4194304] 0 2024-02-16T22:21:54.969 INFO:tasks.workunit.client.1.smithi123.stdout:3/395: read d0/d20/d76/d37/f7f [2173589,10961] 0 2024-02-16T22:21:54.969 INFO:tasks.workunit.client.1.smithi123.stdout:5/388: fdatasync d6/dd/df/d1f/d25/f5d 0 2024-02-16T22:21:54.969 INFO:tasks.workunit.client.0.smithi114.stdout:1/265: truncate d4/d9/f20 1506420 0 2024-02-16T22:21:54.972 INFO:tasks.workunit.client.1.smithi123.stdout:6/438: getdents d0/d9 0 2024-02-16T22:21:54.973 INFO:tasks.workunit.client.1.smithi123.stdout:2/397: dwrite d4/d1a/d1d/d59/f6f [0,4194304] 0 2024-02-16T22:21:54.974 INFO:tasks.workunit.client.1.smithi123.stdout:1/366: fsync d3/d6/d11/f23 0 2024-02-16T22:21:54.975 INFO:tasks.workunit.client.0.smithi114.stdout:9/310: mknod d2/d1a/d23/d35/d47/c68 0 2024-02-16T22:21:54.976 INFO:tasks.workunit.client.0.smithi114.stdout:9/311: chown d2/d1a/d23/d32/c61 110886356 1 2024-02-16T22:21:54.976 INFO:tasks.workunit.client.0.smithi114.stdout:6/276: creat d6/da/f56 x:0 0 0 2024-02-16T22:21:54.980 INFO:tasks.workunit.client.0.smithi114.stdout:9/312: creat d2/d1a/d23/d35/d1d/f69 x:0 0 0 2024-02-16T22:21:54.982 INFO:tasks.workunit.client.0.smithi114.stdout:9/313: write d2/d33/f57 [2010678,121032] 0 2024-02-16T22:21:54.983 INFO:tasks.workunit.client.0.smithi114.stdout:4/378: dread dd/d2e/f49 [0,4194304] 0 2024-02-16T22:21:54.985 INFO:tasks.workunit.client.0.smithi114.stdout:9/314: truncate d2/dd/d3c/f59 579345 0 2024-02-16T22:21:54.985 INFO:tasks.workunit.client.0.smithi114.stdout:1/266: unlink d4/d9/f18 0 2024-02-16T22:21:54.985 INFO:tasks.workunit.client.1.smithi123.stdout:0/368: link d1/d4/c36 d1/d4/c79 0 2024-02-16T22:21:54.986 INFO:tasks.workunit.client.0.smithi114.stdout:1/267: dread - d4/d33/f38 zero size 2024-02-16T22:21:54.986 INFO:tasks.workunit.client.0.smithi114.stdout:6/277: dread d6/dd/f29 [0,4194304] 0 2024-02-16T22:21:54.988 INFO:tasks.workunit.client.0.smithi114.stdout:4/379: creat dd/f89 x:0 0 0 2024-02-16T22:21:54.989 INFO:tasks.workunit.client.0.smithi114.stdout:4/380: fsync dd/d1b/d37/f5d 0 2024-02-16T22:21:54.991 INFO:tasks.workunit.client.0.smithi114.stdout:1/268: chown d4/d9/d46 15438203 1 2024-02-16T22:21:54.993 INFO:tasks.workunit.client.1.smithi123.stdout:8/488: unlink d0/d17/d26/d2a/d59/l5d 0 2024-02-16T22:21:54.993 INFO:tasks.workunit.client.0.smithi114.stdout:9/315: truncate d2/dd/d3c/f4f 1896433 0 2024-02-16T22:21:54.997 INFO:tasks.workunit.client.1.smithi123.stdout:3/396: mkdir d0/d20/d76/d5d/d60/d93 0 2024-02-16T22:21:54.997 INFO:tasks.workunit.client.1.smithi123.stdout:9/518: dread d4/f26 [0,4194304] 0 2024-02-16T22:21:54.998 INFO:tasks.workunit.client.1.smithi123.stdout:9/519: chown d4/d1e/c2d 224628389 1 2024-02-16T22:21:54.998 INFO:tasks.workunit.client.0.smithi114.stdout:4/381: chown dd/d1b/d41/c6f 940075852 1 2024-02-16T22:21:55.000 INFO:tasks.workunit.client.1.smithi123.stdout:4/442: link d5/d1c/d42/f6a d5/d1c/d23/d56/d6e/d73/f88 0 2024-02-16T22:21:55.002 INFO:tasks.workunit.client.1.smithi123.stdout:1/367: dread d3/f8 [0,4194304] 0 2024-02-16T22:21:55.003 INFO:tasks.workunit.client.0.smithi114.stdout:1/269: rename d4/f34 to d4/d33/f57 0 2024-02-16T22:21:55.004 INFO:tasks.workunit.client.0.smithi114.stdout:7/293: truncate d3/d22/f2b 7037767 0 2024-02-16T22:21:55.005 INFO:tasks.workunit.client.1.smithi123.stdout:5/389: mkdir d6/dd/df/d58/d3f/d82 0 2024-02-16T22:21:55.008 INFO:tasks.workunit.client.0.smithi114.stdout:4/382: rmdir dd/d1b/d37/d61 39 2024-02-16T22:21:55.010 INFO:tasks.workunit.client.1.smithi123.stdout:6/439: creat d0/d6/d8e/f9d x:0 0 0 2024-02-16T22:21:55.012 INFO:tasks.workunit.client.1.smithi123.stdout:2/398: unlink d4/d1a/d1d/l14 0 2024-02-16T22:21:55.014 INFO:tasks.workunit.client.0.smithi114.stdout:9/316: rename d2/dd/f55 to d2/d1a/d23/d35/d47/d63/f6a 0 2024-02-16T22:21:55.015 INFO:tasks.workunit.client.0.smithi114.stdout:2/279: sync 2024-02-16T22:21:55.016 INFO:tasks.workunit.client.0.smithi114.stdout:1/270: unlink d4/d33/f50 0 2024-02-16T22:21:55.018 INFO:tasks.workunit.client.1.smithi123.stdout:4/443: dread d5/d1c/d64/d7b/d84/f86 [0,4194304] 0 2024-02-16T22:21:55.019 INFO:tasks.workunit.client.1.smithi123.stdout:4/444: readlink d5/d1c/d23/d74/l37 0 2024-02-16T22:21:55.019 INFO:tasks.workunit.client.0.smithi114.stdout:7/294: creat d3/d23/f62 x:0 0 0 2024-02-16T22:21:55.019 INFO:tasks.workunit.client.1.smithi123.stdout:7/425: write d3/d6e/d24/f31 [1317672,70678] 0 2024-02-16T22:21:55.021 INFO:tasks.workunit.client.0.smithi114.stdout:4/383: mkdir dd/d59/d8a 0 2024-02-16T22:21:55.022 INFO:tasks.workunit.client.1.smithi123.stdout:0/369: mkdir d1/d8/d13/d1d/d62/d7a 0 2024-02-16T22:21:55.030 INFO:tasks.workunit.client.0.smithi114.stdout:9/317: dread d2/d1a/d23/d35/f1e [0,4194304] 0 2024-02-16T22:21:55.031 INFO:tasks.workunit.client.0.smithi114.stdout:9/318: read - d2/d1a/d23/f52 zero size 2024-02-16T22:21:55.031 INFO:tasks.workunit.client.1.smithi123.stdout:9/520: fdatasync d4/d17/f55 0 2024-02-16T22:21:55.033 INFO:tasks.workunit.client.0.smithi114.stdout:6/278: truncate d6/f30 1275861 0 2024-02-16T22:21:55.034 INFO:tasks.workunit.client.0.smithi114.stdout:1/271: symlink d4/d9/d46/l58 0 2024-02-16T22:21:55.037 INFO:tasks.workunit.client.1.smithi123.stdout:2/399: creat d4/d1a/d1d/d13/d25/f80 x:0 0 0 2024-02-16T22:21:55.040 INFO:tasks.workunit.client.1.smithi123.stdout:4/445: creat d5/d49/f89 x:0 0 0 2024-02-16T22:21:55.041 INFO:tasks.workunit.client.1.smithi123.stdout:4/446: stat d5/d1c/d23/d76/l7f 0 2024-02-16T22:21:55.042 INFO:tasks.workunit.client.1.smithi123.stdout:3/397: mkdir d0/d20/d94 0 2024-02-16T22:21:55.042 INFO:tasks.workunit.client.1.smithi123.stdout:2/400: dwrite d4/d1a/d1d/d13/d25/f4d [0,4194304] 0 2024-02-16T22:21:55.043 INFO:tasks.workunit.client.1.smithi123.stdout:4/447: dread - d5/d1c/d23/f3c zero size 2024-02-16T22:21:55.044 INFO:tasks.workunit.client.1.smithi123.stdout:2/401: chown d4/d1a/d1d/d13/d25 22604 1 2024-02-16T22:21:55.044 INFO:tasks.workunit.client.0.smithi114.stdout:6/279: dread d6/f18 [0,4194304] 0 2024-02-16T22:21:55.045 INFO:tasks.workunit.client.0.smithi114.stdout:6/280: chown d6/da 25 1 2024-02-16T22:21:55.045 INFO:tasks.workunit.client.1.smithi123.stdout:2/402: write d4/f23 [1074403,116814] 0 2024-02-16T22:21:55.045 INFO:tasks.workunit.client.1.smithi123.stdout:6/440: write d0/d9/dc/d5f/f3e [58069,130653] 0 2024-02-16T22:21:55.046 INFO:tasks.workunit.client.1.smithi123.stdout:6/441: dread - d0/d6/d1a/f6f zero size 2024-02-16T22:21:55.047 INFO:tasks.workunit.client.0.smithi114.stdout:4/384: symlink dd/d1b/d24/l8b 0 2024-02-16T22:21:55.049 INFO:tasks.workunit.client.1.smithi123.stdout:7/426: dread d3/d26/d35/f4b [0,4194304] 0 2024-02-16T22:21:55.056 INFO:tasks.workunit.client.0.smithi114.stdout:6/281: creat d6/dd/d25/d34/f57 x:0 0 0 2024-02-16T22:21:55.059 INFO:tasks.workunit.client.0.smithi114.stdout:7/295: fsync d3/d23/f62 0 2024-02-16T22:21:55.059 INFO:tasks.workunit.client.1.smithi123.stdout:4/448: creat d5/d1c/d64/f8a x:0 0 0 2024-02-16T22:21:55.060 INFO:tasks.workunit.client.0.smithi114.stdout:7/296: dread - d3/d23/f5e zero size 2024-02-16T22:21:55.061 INFO:tasks.workunit.client.0.smithi114.stdout:4/385: dread dd/d2e/f6a [0,4194304] 0 2024-02-16T22:21:55.062 INFO:tasks.workunit.client.0.smithi114.stdout:4/386: write dd/d1b/d68/f88 [1037296,71083] 0 2024-02-16T22:21:55.064 INFO:tasks.workunit.client.0.smithi114.stdout:4/387: write dd/d1b/d24/f7a [149578,34942] 0 2024-02-16T22:21:55.064 INFO:tasks.workunit.client.0.smithi114.stdout:6/282: mkdir d6/dd/d25/d34/d58 0 2024-02-16T22:21:55.067 INFO:tasks.workunit.client.1.smithi123.stdout:2/403: mknod d4/d1a/d1d/d59/d53/c81 0 2024-02-16T22:21:55.068 INFO:tasks.workunit.client.1.smithi123.stdout:8/489: dwrite d0/f4 [4194304,4194304] 0 2024-02-16T22:21:55.070 INFO:tasks.workunit.client.1.smithi123.stdout:6/442: mknod d0/d9/dc/d39/c9e 0 2024-02-16T22:21:55.071 INFO:tasks.workunit.client.1.smithi123.stdout:5/390: write d6/dd/df/d58/f43 [1777860,115345] 0 2024-02-16T22:21:55.072 INFO:tasks.workunit.client.1.smithi123.stdout:7/427: write d3/d6e/d34/f75 [912580,23556] 0 2024-02-16T22:21:55.073 INFO:tasks.workunit.client.1.smithi123.stdout:7/428: chown d3/d26/d35/f36 62 1 2024-02-16T22:21:55.074 INFO:tasks.workunit.client.1.smithi123.stdout:7/429: chown d3/d87/f5e 44225 1 2024-02-16T22:21:55.077 INFO:tasks.workunit.client.0.smithi114.stdout:2/280: write dd/f55 [1007844,40250] 0 2024-02-16T22:21:55.080 INFO:tasks.workunit.client.0.smithi114.stdout:9/319: dwrite d2/d1a/d23/d35/f1e [0,4194304] 0 2024-02-16T22:21:55.083 INFO:tasks.workunit.client.0.smithi114.stdout:9/320: write d2/d1a/d23/d35/d47/f4d [322325,98163] 0 2024-02-16T22:21:55.084 INFO:tasks.workunit.client.1.smithi123.stdout:3/398: rmdir d0 39 2024-02-16T22:21:55.086 INFO:tasks.workunit.client.0.smithi114.stdout:9/321: dread - d2/d1a/d23/d35/d47/d63/f6a zero size 2024-02-16T22:21:55.087 INFO:tasks.workunit.client.0.smithi114.stdout:2/281: dwrite dd/df/f14 [0,4194304] 0 2024-02-16T22:21:55.088 INFO:tasks.workunit.client.0.smithi114.stdout:1/272: getdents d4/d9/d46 0 2024-02-16T22:21:55.089 INFO:tasks.workunit.client.1.smithi123.stdout:1/368: write d3/f7 [310098,108019] 0 2024-02-16T22:21:55.089 INFO:tasks.workunit.client.1.smithi123.stdout:0/370: dwrite d1/d4/d42/f65 [0,4194304] 0 2024-02-16T22:21:55.090 INFO:tasks.workunit.client.0.smithi114.stdout:9/322: stat d2/d1a/d23/d35/f27 0 2024-02-16T22:21:55.090 INFO:tasks.workunit.client.1.smithi123.stdout:0/371: fsync d1/d4/d17/d74/f58 0 2024-02-16T22:21:55.091 INFO:tasks.workunit.client.0.smithi114.stdout:2/282: truncate dd/df/d2c/f4d 123808 0 2024-02-16T22:21:55.091 INFO:tasks.workunit.client.0.smithi114.stdout:4/388: dread dd/ff [0,4194304] 0 2024-02-16T22:21:55.092 INFO:tasks.workunit.client.0.smithi114.stdout:9/323: fsync d2/d33/f57 0 2024-02-16T22:21:55.094 INFO:tasks.workunit.client.0.smithi114.stdout:9/324: truncate d2/d1a/d23/d35/d1d/f69 396411 0 2024-02-16T22:21:55.096 INFO:tasks.workunit.client.1.smithi123.stdout:9/521: getdents d4/d36/d8e/d6d 0 2024-02-16T22:21:55.102 INFO:tasks.workunit.client.1.smithi123.stdout:4/449: link d5/d1c/d23/d74/f55 d5/d1c/d64/f8b 0 2024-02-16T22:21:55.104 INFO:tasks.workunit.client.1.smithi123.stdout:2/404: symlink d4/d1a/d1d/d59/d45/l82 0 2024-02-16T22:21:55.108 INFO:tasks.workunit.client.0.smithi114.stdout:7/297: write d3/f1a [4330420,8147] 0 2024-02-16T22:21:55.109 INFO:tasks.workunit.client.0.smithi114.stdout:1/273: rmdir d4/d9/d3d 39 2024-02-16T22:21:55.111 INFO:tasks.workunit.client.1.smithi123.stdout:3/399: dwrite d0/d20/d2e/f2f [0,4194304] 0 2024-02-16T22:21:55.113 INFO:tasks.workunit.client.1.smithi123.stdout:1/369: mkdir d3/d6/db/d10/d2a/d80 0 2024-02-16T22:21:55.113 INFO:tasks.workunit.client.1.smithi123.stdout:3/400: write d0/d50/d75/f7d [4909343,44689] 0 2024-02-16T22:21:55.113 INFO:tasks.workunit.client.0.smithi114.stdout:4/389: creat dd/d2e/d56/f8c x:0 0 0 2024-02-16T22:21:55.117 INFO:tasks.workunit.client.1.smithi123.stdout:8/490: dwrite d0/d17/d26/d2a/d3c/f55 [0,4194304] 0 2024-02-16T22:21:55.117 INFO:tasks.workunit.client.1.smithi123.stdout:0/372: unlink d1/f2e 0 2024-02-16T22:21:55.119 INFO:tasks.workunit.client.0.smithi114.stdout:6/283: rmdir d6/dd/d25/d4d 0 2024-02-16T22:21:55.125 INFO:tasks.workunit.client.1.smithi123.stdout:9/522: rename d4/d17 to d4/d36/daa 0 2024-02-16T22:21:55.126 INFO:tasks.workunit.client.1.smithi123.stdout:9/523: dread - d4/d36/daa/d2b/f9a zero size 2024-02-16T22:21:55.127 INFO:tasks.workunit.client.0.smithi114.stdout:1/274: stat d4/d9/d3d/d3f/l51 0 2024-02-16T22:21:55.127 INFO:tasks.workunit.client.1.smithi123.stdout:9/524: dread - d4/d36/fa8 zero size 2024-02-16T22:21:55.133 INFO:tasks.workunit.client.1.smithi123.stdout:2/405: dread d4/d1a/d1d/d13/d25/f3b [0,4194304] 0 2024-02-16T22:21:55.138 INFO:tasks.workunit.client.0.smithi114.stdout:9/325: dwrite d2/d1a/d23/d35/f67 [0,4194304] 0 2024-02-16T22:21:55.138 INFO:tasks.workunit.client.0.smithi114.stdout:6/284: symlink d6/dd/l59 0 2024-02-16T22:21:55.139 INFO:tasks.workunit.client.1.smithi123.stdout:5/391: write d6/dd/df/d1f/f5c [1138783,96814] 0 2024-02-16T22:21:55.139 INFO:tasks.workunit.client.0.smithi114.stdout:9/326: write d2/d1a/f39 [634118,76264] 0 2024-02-16T22:21:55.140 INFO:tasks.workunit.client.1.smithi123.stdout:5/392: write d6/dd/df/f1b [960144,66603] 0 2024-02-16T22:21:55.140 INFO:tasks.workunit.client.1.smithi123.stdout:5/393: chown d6/d3b/c40 709604241 1 2024-02-16T22:21:55.143 INFO:tasks.workunit.client.0.smithi114.stdout:2/283: write dd/f13 [2315967,30144] 0 2024-02-16T22:21:55.144 INFO:tasks.workunit.client.0.smithi114.stdout:2/284: chown dd/d4e 370185511 1 2024-02-16T22:21:55.145 INFO:tasks.workunit.client.1.smithi123.stdout:3/401: truncate d0/f15 630940 0 2024-02-16T22:21:55.145 INFO:tasks.workunit.client.1.smithi123.stdout:3/402: chown d0/d20/d76/d5d 641475684 1 2024-02-16T22:21:55.147 INFO:tasks.workunit.client.1.smithi123.stdout:8/491: creat d0/d17/d26/d68/d81/f9e x:0 0 0 2024-02-16T22:21:55.150 INFO:tasks.workunit.client.1.smithi123.stdout:4/450: rename d5/d1c/d64/c6f to d5/d1c/d23/d6d/c8c 0 2024-02-16T22:21:55.151 INFO:tasks.workunit.client.0.smithi114.stdout:1/275: symlink d4/d9/l59 0 2024-02-16T22:21:55.152 INFO:tasks.workunit.client.1.smithi123.stdout:7/430: creat d3/f8c x:0 0 0 2024-02-16T22:21:55.153 INFO:tasks.workunit.client.1.smithi123.stdout:7/431: chown d3/d6e/d52/f72 2593 1 2024-02-16T22:21:55.153 INFO:tasks.workunit.client.1.smithi123.stdout:9/525: creat d4/d36/d8e/d6d/d71/fab x:0 0 0 2024-02-16T22:21:55.155 INFO:tasks.workunit.client.1.smithi123.stdout:7/432: write d3/d6e/d1d/f64 [1193374,21333] 0 2024-02-16T22:21:55.155 INFO:tasks.workunit.client.0.smithi114.stdout:7/298: rmdir d3/d23/d4e/d5b 0 2024-02-16T22:21:55.155 INFO:tasks.workunit.client.0.smithi114.stdout:6/285: dwrite d6/da/f1b [0,4194304] 0 2024-02-16T22:21:55.155 INFO:tasks.workunit.client.1.smithi123.stdout:9/526: chown d4/d36/d8e/d6d/d71/da6 30765216 1 2024-02-16T22:21:55.156 INFO:tasks.workunit.client.1.smithi123.stdout:4/451: dwrite f4 [0,4194304] 0 2024-02-16T22:21:55.156 INFO:tasks.workunit.client.1.smithi123.stdout:4/452: stat d5/d1c/d23/d31/f83 0 2024-02-16T22:21:55.157 INFO:tasks.workunit.client.1.smithi123.stdout:4/453: chown d5/d1c/l5c 53173 1 2024-02-16T22:21:55.159 INFO:tasks.workunit.client.1.smithi123.stdout:9/527: dread d4/f26 [0,4194304] 0 2024-02-16T22:21:55.159 INFO:tasks.workunit.client.1.smithi123.stdout:9/528: chown d4/d1e 31091 1 2024-02-16T22:21:55.161 INFO:tasks.workunit.client.1.smithi123.stdout:1/370: symlink d3/d6/l81 0 2024-02-16T22:21:55.165 INFO:tasks.workunit.client.0.smithi114.stdout:2/285: dwrite dd/df/f1e [0,4194304] 0 2024-02-16T22:21:55.165 INFO:tasks.workunit.client.1.smithi123.stdout:6/443: getdents d0/d9/dc 0 2024-02-16T22:21:55.165 INFO:tasks.workunit.client.1.smithi123.stdout:6/444: readlink d0/d9/dc/l3d 0 2024-02-16T22:21:55.166 INFO:tasks.workunit.client.1.smithi123.stdout:6/445: dread - d0/d6/d1a/d2e/f3c zero size 2024-02-16T22:21:55.167 INFO:tasks.workunit.client.1.smithi123.stdout:6/446: write d0/f65 [3982198,79113] 0 2024-02-16T22:21:55.170 INFO:tasks.workunit.client.1.smithi123.stdout:3/403: rmdir d0/d20/d76/d37/d49 39 2024-02-16T22:21:55.171 INFO:tasks.workunit.client.1.smithi123.stdout:5/394: truncate d6/dd/df/d1f/f68 711084 0 2024-02-16T22:21:55.172 INFO:tasks.workunit.client.0.smithi114.stdout:6/286: unlink d6/da/l51 0 2024-02-16T22:21:55.173 INFO:tasks.workunit.client.1.smithi123.stdout:0/373: symlink d1/d8/d13/d1d/d62/d7a/l7b 0 2024-02-16T22:21:55.173 INFO:tasks.workunit.client.0.smithi114.stdout:1/276: dwrite d4/f16 [4194304,4194304] 0 2024-02-16T22:21:55.174 INFO:tasks.workunit.client.1.smithi123.stdout:0/374: readlink d1/d8/d13/l29 0 2024-02-16T22:21:55.174 INFO:tasks.workunit.client.0.smithi114.stdout:4/390: getdents dd/d59 0 2024-02-16T22:21:55.175 INFO:tasks.workunit.client.0.smithi114.stdout:6/287: readlink d6/d2c/d38/d54/l2b 0 2024-02-16T22:21:55.175 INFO:tasks.workunit.client.0.smithi114.stdout:4/391: dread - dd/f89 zero size 2024-02-16T22:21:55.178 INFO:tasks.workunit.client.1.smithi123.stdout:2/406: mknod d4/d1a/d1d/d38/d7b/c83 0 2024-02-16T22:21:55.178 INFO:tasks.workunit.client.0.smithi114.stdout:7/299: mknod d3/d22/d61/c63 0 2024-02-16T22:21:55.179 INFO:tasks.workunit.client.0.smithi114.stdout:2/286: dwrite dd/df/f14 [0,4194304] 0 2024-02-16T22:21:55.180 INFO:tasks.workunit.client.1.smithi123.stdout:9/529: mkdir d4/d1e/d3a/dac 0 2024-02-16T22:21:55.182 INFO:tasks.workunit.client.0.smithi114.stdout:4/392: mknod dd/d2e/d3f/c8d 0 2024-02-16T22:21:55.183 INFO:tasks.workunit.client.1.smithi123.stdout:2/407: dwrite d4/fc [4194304,4194304] 0 2024-02-16T22:21:55.184 INFO:tasks.workunit.client.1.smithi123.stdout:2/408: truncate d4/f52 4811225 0 2024-02-16T22:21:55.185 INFO:tasks.workunit.client.1.smithi123.stdout:2/409: chown d4/d1a/d1d/l10 8909 1 2024-02-16T22:21:55.186 INFO:tasks.workunit.client.0.smithi114.stdout:1/277: rename d4/f28 to d4/d9/d3d/d3f/f5a 0 2024-02-16T22:21:55.187 INFO:tasks.workunit.client.0.smithi114.stdout:7/300: mknod d3/d14/c64 0 2024-02-16T22:21:55.187 INFO:tasks.workunit.client.0.smithi114.stdout:6/288: dwrite d6/f1e [4194304,4194304] 0 2024-02-16T22:21:55.188 INFO:tasks.workunit.client.0.smithi114.stdout:4/393: write dd/d1b/f55 [566272,40687] 0 2024-02-16T22:21:55.188 INFO:tasks.workunit.client.0.smithi114.stdout:1/278: truncate d4/d9/f4e 713446 0 2024-02-16T22:21:55.189 INFO:tasks.workunit.client.1.smithi123.stdout:3/404: unlink d0/d20/l33 0 2024-02-16T22:21:55.190 INFO:tasks.workunit.client.0.smithi114.stdout:6/289: readlink d6/dd/l49 0 2024-02-16T22:21:55.193 INFO:tasks.workunit.client.0.smithi114.stdout:2/287: unlink dd/df/d2c/c60 0 2024-02-16T22:21:55.197 INFO:tasks.workunit.client.1.smithi123.stdout:0/375: mknod d1/d4/d17/d74/c7c 0 2024-02-16T22:21:55.198 INFO:tasks.workunit.client.0.smithi114.stdout:4/394: dwrite dd/d1b/d24/f4c [0,4194304] 0 2024-02-16T22:21:55.200 INFO:tasks.workunit.client.1.smithi123.stdout:4/454: creat d5/d49/d71/f8d x:0 0 0 2024-02-16T22:21:55.201 INFO:tasks.workunit.client.0.smithi114.stdout:1/279: creat d4/d9/d46/d4d/f5b x:0 0 0 2024-02-16T22:21:55.201 INFO:tasks.workunit.client.1.smithi123.stdout:4/455: chown d5/d1c/d23/d74/c44 24723 1 2024-02-16T22:21:55.202 INFO:tasks.workunit.client.1.smithi123.stdout:3/405: dread d0/d20/d76/d37/f64 [0,4194304] 0 2024-02-16T22:21:55.203 INFO:tasks.workunit.client.1.smithi123.stdout:0/376: dwrite d1/d8/d13/f5a [0,4194304] 0 2024-02-16T22:21:55.204 INFO:tasks.workunit.client.0.smithi114.stdout:9/327: truncate d2/d1a/f39 373138 0 2024-02-16T22:21:55.204 INFO:tasks.workunit.client.1.smithi123.stdout:3/406: truncate d0/d20/d76/d5d/f82 383533 0 2024-02-16T22:21:55.206 INFO:tasks.workunit.client.1.smithi123.stdout:9/530: dread d4/d36/daa/d20/f3c [4194304,4194304] 0 2024-02-16T22:21:55.207 INFO:tasks.workunit.client.0.smithi114.stdout:2/288: chown dd/df/d1c/l29 202087 1 2024-02-16T22:21:55.208 INFO:tasks.workunit.client.0.smithi114.stdout:1/280: dwrite d4/d9/d3d/f39 [0,4194304] 0 2024-02-16T22:21:55.209 INFO:tasks.workunit.client.0.smithi114.stdout:7/301: truncate d3/d14/f5c 3083943 0 2024-02-16T22:21:55.210 INFO:tasks.workunit.client.0.smithi114.stdout:1/281: write d4/f1d [540316,51533] 0 2024-02-16T22:21:55.211 INFO:tasks.workunit.client.1.smithi123.stdout:6/447: mknod d0/d6/d91/d93/c9f 0 2024-02-16T22:21:55.213 INFO:tasks.workunit.client.0.smithi114.stdout:3/254: sync 2024-02-16T22:21:55.214 INFO:tasks.workunit.client.0.smithi114.stdout:3/255: write d4/d24/d2d/d45/f51 [65143,39588] 0 2024-02-16T22:21:55.216 INFO:tasks.workunit.client.1.smithi123.stdout:2/410: symlink d4/l84 0 2024-02-16T22:21:55.217 INFO:tasks.workunit.client.1.smithi123.stdout:2/411: dread d4/f54 [0,4194304] 0 2024-02-16T22:21:55.219 INFO:tasks.workunit.client.1.smithi123.stdout:2/412: dread - d4/d1a/d1d/d38/d63/f76 zero size 2024-02-16T22:21:55.219 INFO:tasks.workunit.client.1.smithi123.stdout:5/395: link d6/dd/df/d1f/d25/f31 d6/dd/df/d58/d3f/d82/f83 0 2024-02-16T22:21:55.220 INFO:tasks.workunit.client.1.smithi123.stdout:5/396: stat d6/dd/df/d1f/d25/d37 0 2024-02-16T22:21:55.221 INFO:tasks.workunit.client.1.smithi123.stdout:5/397: chown d6/dd/df/d1f/f64 1 1 2024-02-16T22:21:55.222 INFO:tasks.workunit.client.0.smithi114.stdout:3/256: dwrite d4/dd/d33/f50 [0,4194304] 0 2024-02-16T22:21:55.222 INFO:tasks.workunit.client.0.smithi114.stdout:3/257: chown d4/l10 278 1 2024-02-16T22:21:55.223 INFO:tasks.workunit.client.1.smithi123.stdout:5/398: write d6/dd/f50 [1706054,71254] 0 2024-02-16T22:21:55.223 INFO:tasks.workunit.client.0.smithi114.stdout:3/258: write d4/dd/f19 [618283,62428] 0 2024-02-16T22:21:55.224 INFO:tasks.workunit.client.1.smithi123.stdout:2/413: dwrite d4/fc [0,4194304] 0 2024-02-16T22:21:55.225 INFO:tasks.workunit.client.0.smithi114.stdout:7/302: symlink d3/d46/l65 0 2024-02-16T22:21:55.227 INFO:tasks.workunit.client.0.smithi114.stdout:1/282: creat d4/d9/d46/d4d/f5c x:0 0 0 2024-02-16T22:21:55.231 INFO:tasks.workunit.client.1.smithi123.stdout:0/377: unlink d1/d8/f30 0 2024-02-16T22:21:55.237 INFO:tasks.workunit.client.1.smithi123.stdout:9/531: chown d4/d36/d8e/d6d/d71/d92/c89 349814 1 2024-02-16T22:21:55.238 INFO:tasks.workunit.client.1.smithi123.stdout:9/532: fsync d4/f6b 0 2024-02-16T22:21:55.240 INFO:tasks.workunit.client.1.smithi123.stdout:1/371: link d3/d6/db/d10/l60 d3/d6/db/d10/d2a/d80/l82 0 2024-02-16T22:21:55.244 INFO:tasks.workunit.client.1.smithi123.stdout:6/448: rename d0/d9/dc/f67 to d0/d6/d8e/fa0 0 2024-02-16T22:21:55.250 INFO:tasks.workunit.client.0.smithi114.stdout:6/290: dwrite d6/f30 [0,4194304] 0 2024-02-16T22:21:55.253 INFO:tasks.workunit.client.0.smithi114.stdout:2/289: read dd/df/d1c/d21/f26 [187178,128472] 0 2024-02-16T22:21:55.254 INFO:tasks.workunit.client.0.smithi114.stdout:7/303: mknod d3/c66 0 2024-02-16T22:21:55.255 INFO:tasks.workunit.client.1.smithi123.stdout:8/492: link d0/d17/d26/d2a/d3c/l51 d0/d17/d26/l9f 0 2024-02-16T22:21:55.259 INFO:tasks.workunit.client.0.smithi114.stdout:6/291: dwrite d6/da/f56 [0,4194304] 0 2024-02-16T22:21:55.259 INFO:tasks.workunit.client.1.smithi123.stdout:5/399: creat d6/dd/df/d58/d3f/d6f/f84 x:0 0 0 2024-02-16T22:21:55.261 INFO:tasks.workunit.client.0.smithi114.stdout:9/328: truncate d2/dd/d3c/f59 308085 0 2024-02-16T22:21:55.264 INFO:tasks.workunit.client.0.smithi114.stdout:9/329: write d2/d1a/d23/d35/f27 [130073,24890] 0 2024-02-16T22:21:55.265 INFO:tasks.workunit.client.1.smithi123.stdout:4/456: symlink d5/d1c/d23/l8e 0 2024-02-16T22:21:55.267 INFO:tasks.workunit.client.1.smithi123.stdout:7/433: sync 2024-02-16T22:21:55.267 INFO:tasks.workunit.client.0.smithi114.stdout:9/330: write d2/d33/f57 [1532092,81066] 0 2024-02-16T22:21:55.267 INFO:tasks.workunit.client.0.smithi114.stdout:7/304: dwrite d3/d3d/f4a [0,4194304] 0 2024-02-16T22:21:55.267 INFO:tasks.workunit.client.0.smithi114.stdout:3/259: rmdir d4/d43/d5b 0 2024-02-16T22:21:55.268 INFO:tasks.workunit.client.0.smithi114.stdout:7/305: chown d3/d23/l40 193137825 1 2024-02-16T22:21:55.269 INFO:tasks.workunit.client.1.smithi123.stdout:0/378: symlink d1/d4/d17/l7d 0 2024-02-16T22:21:55.276 INFO:tasks.workunit.client.0.smithi114.stdout:3/260: dwrite d4/d26/f4d [0,4194304] 0 2024-02-16T22:21:55.281 INFO:tasks.workunit.client.0.smithi114.stdout:7/306: dwrite d3/fc [0,4194304] 0 2024-02-16T22:21:55.283 INFO:tasks.workunit.client.0.smithi114.stdout:6/292: rename d6/da/c22 to d6/d2c/c5a 0 2024-02-16T22:21:55.290 INFO:tasks.workunit.client.1.smithi123.stdout:2/414: rename d4/d1a/d1d/d13/d15/d6d/l1e to d4/d1a/d1d/d13/l85 0 2024-02-16T22:21:55.291 INFO:tasks.workunit.client.1.smithi123.stdout:1/372: dread d3/d6/d18/f21 [0,4194304] 0 2024-02-16T22:21:55.294 INFO:tasks.workunit.client.0.smithi114.stdout:2/290: creat dd/d3c/d4c/f64 x:0 0 0 2024-02-16T22:21:55.296 INFO:tasks.workunit.client.1.smithi123.stdout:4/457: creat d5/d1c/d23/d31/f8f x:0 0 0 2024-02-16T22:21:55.297 INFO:tasks.workunit.client.0.smithi114.stdout:3/261: symlink d4/d24/d2d/d45/l5e 0 2024-02-16T22:21:55.297 INFO:tasks.workunit.client.1.smithi123.stdout:7/434: chown d3/d6e/d1d/c70 1709 1 2024-02-16T22:21:55.298 INFO:tasks.workunit.client.0.smithi114.stdout:7/307: mknod d3/d22/c67 0 2024-02-16T22:21:55.299 INFO:tasks.workunit.client.0.smithi114.stdout:3/262: dread d4/d26/f34 [0,4194304] 0 2024-02-16T22:21:55.299 INFO:tasks.workunit.client.1.smithi123.stdout:3/407: unlink d0/d20/d76/d37/d49/f6a 0 2024-02-16T22:21:55.300 INFO:tasks.workunit.client.1.smithi123.stdout:3/408: write d0/f12 [1847046,17856] 0 2024-02-16T22:21:55.300 INFO:tasks.workunit.client.0.smithi114.stdout:3/263: write d4/d46/d48/f5c [600583,17378] 0 2024-02-16T22:21:55.300 INFO:tasks.workunit.client.0.smithi114.stdout:6/293: symlink d6/dd/d25/d34/l5b 0 2024-02-16T22:21:55.300 INFO:tasks.workunit.client.1.smithi123.stdout:3/409: dread - d0/d20/d76/d5d/f86 zero size 2024-02-16T22:21:55.301 INFO:tasks.workunit.client.1.smithi123.stdout:3/410: fsync d0/d20/d76/d37/f5e 0 2024-02-16T22:21:55.302 INFO:tasks.workunit.client.0.smithi114.stdout:3/264: chown d4/d24/d2d/d45/f51 175228 1 2024-02-16T22:21:55.305 INFO:tasks.workunit.client.1.smithi123.stdout:0/379: dread d1/d4/d17/d74/f58 [0,4194304] 0 2024-02-16T22:21:55.305 INFO:tasks.workunit.client.1.smithi123.stdout:1/373: mknod d3/d6/d18/c83 0 2024-02-16T22:21:55.306 INFO:tasks.workunit.client.1.smithi123.stdout:0/380: read - d1/d4/d17/d74/d57/f72 zero size 2024-02-16T22:21:55.309 INFO:tasks.workunit.client.0.smithi114.stdout:9/331: creat d2/d1a/f6b x:0 0 0 2024-02-16T22:21:55.310 INFO:tasks.workunit.client.0.smithi114.stdout:6/294: fdatasync d6/f3e 0 2024-02-16T22:21:55.313 INFO:tasks.workunit.client.0.smithi114.stdout:3/265: creat d4/dd/d33/f5f x:0 0 0 2024-02-16T22:21:55.314 INFO:tasks.workunit.client.0.smithi114.stdout:2/291: getdents dd/df/d1c/d21/d61/d63 0 2024-02-16T22:21:55.315 INFO:tasks.workunit.client.0.smithi114.stdout:2/292: dread - dd/d3c/d4c/f64 zero size 2024-02-16T22:21:55.319 INFO:tasks.workunit.client.0.smithi114.stdout:7/308: link d3/d23/f5e d3/d23/f68 0 2024-02-16T22:21:55.319 INFO:tasks.workunit.client.0.smithi114.stdout:1/283: truncate d4/d9/f12 231573 0 2024-02-16T22:21:55.321 INFO:tasks.workunit.client.1.smithi123.stdout:6/449: creat d0/d9/dc/d3a/fa1 x:0 0 0 2024-02-16T22:21:55.322 INFO:tasks.workunit.client.0.smithi114.stdout:6/295: rmdir d6/d2c 39 2024-02-16T22:21:55.322 INFO:tasks.workunit.client.1.smithi123.stdout:6/450: readlink d0/d6/d2f/l33 0 2024-02-16T22:21:55.322 INFO:tasks.workunit.client.0.smithi114.stdout:6/296: dread - d6/dd/d25/f4c zero size 2024-02-16T22:21:55.324 INFO:tasks.workunit.client.1.smithi123.stdout:0/381: creat d1/d4/d42/f7e x:0 0 0 2024-02-16T22:21:55.324 INFO:tasks.workunit.client.0.smithi114.stdout:2/293: mknod dd/df/d1c/d21/c65 0 2024-02-16T22:21:55.325 INFO:tasks.workunit.client.1.smithi123.stdout:1/374: truncate d3/d6/db/f28 23523 0 2024-02-16T22:21:55.326 INFO:tasks.workunit.client.1.smithi123.stdout:1/375: stat d3/d6/d11/c33 0 2024-02-16T22:21:55.326 INFO:tasks.workunit.client.0.smithi114.stdout:7/309: mknod d3/d46/c69 0 2024-02-16T22:21:55.326 INFO:tasks.workunit.client.0.smithi114.stdout:7/310: fsync d3/f1a 0 2024-02-16T22:21:55.327 INFO:tasks.workunit.client.0.smithi114.stdout:7/311: read d3/d3d/f4a [1770718,18113] 0 2024-02-16T22:21:55.328 INFO:tasks.workunit.client.0.smithi114.stdout:1/284: creat d4/d9/d46/d4d/f5d x:0 0 0 2024-02-16T22:21:55.328 INFO:tasks.workunit.client.0.smithi114.stdout:1/285: fsync d4/d9/f20 0 2024-02-16T22:21:55.330 INFO:tasks.workunit.client.1.smithi123.stdout:9/533: write d4/d36/daa/d20/f22 [3405544,33982] 0 2024-02-16T22:21:55.331 INFO:tasks.workunit.client.1.smithi123.stdout:9/534: chown d4/d36/daa/d2b/d6c/l86 5 1 2024-02-16T22:21:55.331 INFO:tasks.workunit.client.0.smithi114.stdout:3/266: symlink d4/d24/l60 0 2024-02-16T22:21:55.333 INFO:tasks.workunit.client.1.smithi123.stdout:7/435: creat d3/d5/d6c/f8d x:0 0 0 2024-02-16T22:21:55.333 INFO:tasks.workunit.client.0.smithi114.stdout:3/267: fsync d4/dd/f19 0 2024-02-16T22:21:55.334 INFO:tasks.workunit.client.0.smithi114.stdout:7/312: dwrite d3/d23/f52 [0,4194304] 0 2024-02-16T22:21:55.335 INFO:tasks.workunit.client.1.smithi123.stdout:4/458: write d5/d1c/f25 [4769057,95900] 0 2024-02-16T22:21:55.335 INFO:tasks.workunit.client.1.smithi123.stdout:4/459: chown d5/d1c/d23/f43 20421 1 2024-02-16T22:21:55.337 INFO:tasks.workunit.client.1.smithi123.stdout:2/415: rename d4/f4f to d4/d1a/f86 0 2024-02-16T22:21:55.347 INFO:tasks.workunit.client.1.smithi123.stdout:5/400: getdents d6/dd/df/d58/d3f/d82 0 2024-02-16T22:21:55.349 INFO:tasks.workunit.client.1.smithi123.stdout:0/382: rmdir d1/d4 39 2024-02-16T22:21:55.350 INFO:tasks.workunit.client.1.smithi123.stdout:9/535: rmdir d4/d36/d8e/d58/d6e 39 2024-02-16T22:21:55.351 INFO:tasks.workunit.client.0.smithi114.stdout:1/286: mknod d4/d9/d46/c5e 0 2024-02-16T22:21:55.354 INFO:tasks.workunit.client.1.smithi123.stdout:9/536: dread d4/d36/daa/d2b/f53 [0,4194304] 0 2024-02-16T22:21:55.355 INFO:tasks.workunit.client.1.smithi123.stdout:6/451: fsync d0/d9/dc/d5f/f79 0 2024-02-16T22:21:55.358 INFO:tasks.workunit.client.0.smithi114.stdout:6/297: mknod d6/dd/d25/c5c 0 2024-02-16T22:21:55.360 INFO:tasks.workunit.client.0.smithi114.stdout:3/268: mknod d4/d24/d2d/d45/c61 0 2024-02-16T22:21:55.363 INFO:tasks.workunit.client.1.smithi123.stdout:3/411: rename d0/d50/d7e/f8d to d0/d20/d2e/f95 0 2024-02-16T22:21:55.363 INFO:tasks.workunit.client.0.smithi114.stdout:6/298: dwrite d6/dd/f17 [0,4194304] 0 2024-02-16T22:21:55.364 INFO:tasks.workunit.client.0.smithi114.stdout:6/299: readlink d6/dd/d25/l33 0 2024-02-16T22:21:55.367 INFO:tasks.workunit.client.0.smithi114.stdout:7/313: dread d3/d14/f5c [0,4194304] 0 2024-02-16T22:21:55.367 INFO:tasks.workunit.client.1.smithi123.stdout:8/493: link d0/df/l11 d0/df/la0 0 2024-02-16T22:21:55.371 INFO:tasks.workunit.client.1.smithi123.stdout:5/401: write d6/dd/df/d58/f70 [1780794,80544] 0 2024-02-16T22:21:55.372 INFO:tasks.workunit.client.1.smithi123.stdout:0/383: creat d1/d8/d13/d1d/d62/d7a/f7f x:0 0 0 2024-02-16T22:21:55.375 INFO:tasks.workunit.client.1.smithi123.stdout:7/436: fdatasync d3/d6e/d1d/f2f 0 2024-02-16T22:21:55.376 INFO:tasks.workunit.client.1.smithi123.stdout:6/452: dread d0/f65 [0,4194304] 0 2024-02-16T22:21:55.378 INFO:tasks.workunit.client.1.smithi123.stdout:9/537: creat d4/d36/fad x:0 0 0 2024-02-16T22:21:55.382 INFO:tasks.workunit.client.0.smithi114.stdout:2/294: truncate dd/d4e/f58 3152163 0 2024-02-16T22:21:55.382 INFO:tasks.workunit.client.1.smithi123.stdout:3/412: rmdir d0/d20/d76/d5d/d60 39 2024-02-16T22:21:55.383 INFO:tasks.workunit.client.0.smithi114.stdout:2/295: write dd/df/d2c/f4d [182805,120145] 0 2024-02-16T22:21:55.383 INFO:tasks.workunit.client.1.smithi123.stdout:1/376: truncate d3/d6/db/d10/d2d/f5a 730177 0 2024-02-16T22:21:55.385 INFO:tasks.workunit.client.0.smithi114.stdout:1/287: creat d4/d9/d46/f5f x:0 0 0 2024-02-16T22:21:55.387 INFO:tasks.workunit.client.0.smithi114.stdout:9/332: getdents d2/d1a/d23/d35/d1d 0 2024-02-16T22:21:55.390 INFO:tasks.workunit.client.0.smithi114.stdout:0/270: sync 2024-02-16T22:21:55.395 INFO:tasks.workunit.client.0.smithi114.stdout:9/333: dwrite d2/d33/f57 [0,4194304] 0 2024-02-16T22:21:55.396 INFO:tasks.workunit.client.0.smithi114.stdout:9/334: write d2/dd/f2a [4438328,90547] 0 2024-02-16T22:21:55.398 INFO:tasks.workunit.client.1.smithi123.stdout:8/494: dread d0/d17/d26/d2a/f78 [0,4194304] 0 2024-02-16T22:21:55.401 INFO:tasks.workunit.client.0.smithi114.stdout:2/296: mknod dd/df/d2c/c66 0 2024-02-16T22:21:55.402 INFO:tasks.workunit.client.0.smithi114.stdout:2/297: chown dd/f13 46605495 1 2024-02-16T22:21:55.402 INFO:tasks.workunit.client.0.smithi114.stdout:2/298: chown dd/f55 4050 1 2024-02-16T22:21:55.404 INFO:tasks.workunit.client.1.smithi123.stdout:7/437: rename d3/d6e/d1d/f64 to d3/d6e/d52/f8e 0 2024-02-16T22:21:55.404 INFO:tasks.workunit.client.1.smithi123.stdout:2/416: truncate d4/d1a/d1d/d59/d45/f70 220626 0 2024-02-16T22:21:55.406 INFO:tasks.workunit.client.1.smithi123.stdout:7/438: write d3/d6e/f4f [705432,89156] 0 2024-02-16T22:21:55.406 INFO:tasks.workunit.client.1.smithi123.stdout:8/495: dwrite d0/d17/d43/d34/f38 [0,4194304] 0 2024-02-16T22:21:55.409 INFO:tasks.workunit.client.1.smithi123.stdout:9/538: symlink d4/d36/d8e/lae 0 2024-02-16T22:21:55.409 INFO:tasks.workunit.client.1.smithi123.stdout:1/377: dread d3/d6/db/d10/d4b/f72 [0,4194304] 0 2024-02-16T22:21:55.411 INFO:tasks.workunit.client.0.smithi114.stdout:6/300: mknod d6/d2c/d38/d54/c5d 0 2024-02-16T22:21:55.412 INFO:tasks.workunit.client.0.smithi114.stdout:6/301: chown d6/dd/l49 2797549 1 2024-02-16T22:21:55.412 INFO:tasks.workunit.client.1.smithi123.stdout:4/460: creat d5/f90 x:0 0 0 2024-02-16T22:21:55.415 INFO:tasks.workunit.client.1.smithi123.stdout:3/413: truncate d0/d20/d76/d37/f64 515857 0 2024-02-16T22:21:55.415 INFO:tasks.workunit.client.1.smithi123.stdout:9/539: dwrite d4/d1e/f42 [8388608,4194304] 0 2024-02-16T22:21:55.418 INFO:tasks.workunit.client.0.smithi114.stdout:9/335: mknod d2/d1a/d23/d32/c6c 0 2024-02-16T22:21:55.419 INFO:tasks.workunit.client.1.smithi123.stdout:5/402: mknod d6/dd/df/d1f/c85 0 2024-02-16T22:21:55.420 INFO:tasks.workunit.client.0.smithi114.stdout:9/336: dread - d2/d1a/d23/f56 zero size 2024-02-16T22:21:55.420 INFO:tasks.workunit.client.0.smithi114.stdout:9/337: dread - d2/d1a/d23/d35/d47/d63/f6a zero size 2024-02-16T22:21:55.421 INFO:tasks.workunit.client.1.smithi123.stdout:0/384: creat d1/d4/d17/d54/f80 x:0 0 0 2024-02-16T22:21:55.421 INFO:tasks.workunit.client.0.smithi114.stdout:0/271: dread d1/d1a/f3b [0,4194304] 0 2024-02-16T22:21:55.423 INFO:tasks.workunit.client.0.smithi114.stdout:0/272: dread - d1/d16/d1f/d28/f54 zero size 2024-02-16T22:21:55.424 INFO:tasks.workunit.client.0.smithi114.stdout:3/269: link d4/d43/f56 d4/d24/f62 0 2024-02-16T22:21:55.424 INFO:tasks.workunit.client.1.smithi123.stdout:5/403: dread d6/dd/df/f79 [0,4194304] 0 2024-02-16T22:21:55.432 INFO:tasks.workunit.client.0.smithi114.stdout:4/395: sync 2024-02-16T22:21:55.432 INFO:tasks.workunit.client.0.smithi114.stdout:8/312: sync 2024-02-16T22:21:55.432 INFO:tasks.workunit.client.0.smithi114.stdout:5/269: sync 2024-02-16T22:21:55.435 INFO:tasks.workunit.client.0.smithi114.stdout:9/338: rename d2/d1a/d23/d35/f27 to d2/d1a/d23/d35/d47/f6d 0 2024-02-16T22:21:55.437 INFO:tasks.workunit.client.0.smithi114.stdout:7/314: truncate d3/fc 3462732 0 2024-02-16T22:21:55.438 INFO:tasks.workunit.client.0.smithi114.stdout:0/273: creat d1/d16/d1f/d28/f56 x:0 0 0 2024-02-16T22:21:55.441 INFO:tasks.workunit.client.0.smithi114.stdout:3/270: unlink d4/dd/d33/l36 0 2024-02-16T22:21:55.442 INFO:tasks.workunit.client.1.smithi123.stdout:3/414: unlink d0/d50/d75/c92 0 2024-02-16T22:21:55.444 INFO:tasks.workunit.client.0.smithi114.stdout:3/271: dread d4/dd/d33/f50 [0,4194304] 0 2024-02-16T22:21:55.445 INFO:tasks.workunit.client.0.smithi114.stdout:2/299: creat dd/df/d1c/f67 x:0 0 0 2024-02-16T22:21:55.447 INFO:tasks.workunit.client.0.smithi114.stdout:1/288: getdents d4/d33 0 2024-02-16T22:21:55.448 INFO:tasks.workunit.client.0.smithi114.stdout:5/270: mknod d0/df/c4b 0 2024-02-16T22:21:55.452 INFO:tasks.workunit.client.0.smithi114.stdout:4/396: symlink dd/d1b/d41/l8e 0 2024-02-16T22:21:55.455 INFO:tasks.workunit.client.1.smithi123.stdout:5/404: creat d6/dd/df/d58/d3f/d82/f86 x:0 0 0 2024-02-16T22:21:55.456 INFO:tasks.workunit.client.1.smithi123.stdout:2/417: write d4/d1a/d1d/f9 [1334920,71664] 0 2024-02-16T22:21:55.457 INFO:tasks.workunit.client.1.smithi123.stdout:6/453: write d0/d6/d1a/d2e/f7f [2532522,65074] 0 2024-02-16T22:21:55.458 INFO:tasks.workunit.client.0.smithi114.stdout:5/271: dwrite d0/d2d/d30/f31 [0,4194304] 0 2024-02-16T22:21:55.459 INFO:tasks.workunit.client.0.smithi114.stdout:5/272: read d0/d10/f23 [389205,38856] 0 2024-02-16T22:21:55.460 INFO:tasks.workunit.client.0.smithi114.stdout:7/315: mkdir d3/d22/d61/d6a 0 2024-02-16T22:21:55.460 INFO:tasks.workunit.client.0.smithi114.stdout:5/273: chown d0/df/d16/f1f 5 1 2024-02-16T22:21:55.462 INFO:tasks.workunit.client.1.smithi123.stdout:2/418: dwrite d4/d1a/d1d/d13/d25/f43 [0,4194304] 0 2024-02-16T22:21:55.463 INFO:tasks.workunit.client.1.smithi123.stdout:2/419: read d4/d1a/d1d/d13/d25/f77 [4803120,66712] 0 2024-02-16T22:21:55.467 INFO:tasks.workunit.client.0.smithi114.stdout:1/289: dread d4/f30 [0,4194304] 0 2024-02-16T22:21:55.467 INFO:tasks.workunit.client.0.smithi114.stdout:0/274: chown d1/d16/l33 357347131 1 2024-02-16T22:21:55.469 INFO:tasks.workunit.client.1.smithi123.stdout:7/439: truncate d3/f4e 1623166 0 2024-02-16T22:21:55.480 INFO:tasks.workunit.client.1.smithi123.stdout:0/385: dwrite d1/d8/d3d/f59 [0,4194304] 0 2024-02-16T22:21:55.481 INFO:tasks.workunit.client.0.smithi114.stdout:2/300: symlink dd/df/d2c/l68 0 2024-02-16T22:21:55.481 INFO:tasks.workunit.client.1.smithi123.stdout:0/386: chown d1/d8/d13/d1d/f6f 8561 1 2024-02-16T22:21:55.481 INFO:tasks.workunit.client.1.smithi123.stdout:0/387: dread - d1/d4/d42/f7e zero size 2024-02-16T22:21:55.491 INFO:tasks.workunit.client.0.smithi114.stdout:8/313: link d4/d11/d15/d36/f6b d4/d11/d15/f6c 0 2024-02-16T22:21:55.491 INFO:tasks.workunit.client.0.smithi114.stdout:8/314: chown d4/d24/d2c 196582 1 2024-02-16T22:21:55.496 INFO:tasks.workunit.client.0.smithi114.stdout:4/397: truncate dd/d2e/d3f/f6c 104076 0 2024-02-16T22:21:55.498 INFO:tasks.workunit.client.1.smithi123.stdout:1/378: sync 2024-02-16T22:21:55.499 INFO:tasks.workunit.client.1.smithi123.stdout:4/461: truncate d5/d1c/d64/d7b/d84/f86 3350655 0 2024-02-16T22:21:55.502 INFO:tasks.workunit.client.0.smithi114.stdout:5/274: rmdir d0/df/d16/d25 39 2024-02-16T22:21:55.502 INFO:tasks.workunit.client.1.smithi123.stdout:6/454: mkdir d0/d6/d91/da2 0 2024-02-16T22:21:55.503 INFO:tasks.workunit.client.1.smithi123.stdout:6/455: chown d0/d6/d1a/d2e/f3c 163561337 1 2024-02-16T22:21:55.504 INFO:tasks.workunit.client.0.smithi114.stdout:0/275: rmdir d1/d1a 39 2024-02-16T22:21:55.506 INFO:tasks.workunit.client.1.smithi123.stdout:2/420: symlink d4/d1a/d1d/d13/d15/d6d/l87 0 2024-02-16T22:21:55.507 INFO:tasks.workunit.client.1.smithi123.stdout:7/440: mknod d3/d6e/d52/c8f 0 2024-02-16T22:21:55.508 INFO:tasks.workunit.client.0.smithi114.stdout:6/302: getdents d6/d2c/d38 0 2024-02-16T22:21:55.508 INFO:tasks.workunit.client.0.smithi114.stdout:6/303: readlink d6/l3b 0 2024-02-16T22:21:55.509 INFO:tasks.workunit.client.1.smithi123.stdout:8/496: link d0/d17/d43/f27 d0/d17/d26/d68/d81/fa1 0 2024-02-16T22:21:55.511 INFO:tasks.workunit.client.1.smithi123.stdout:3/415: creat d0/d20/d76/d5d/d60/f96 x:0 0 0 2024-02-16T22:21:55.516 INFO:tasks.workunit.client.1.smithi123.stdout:0/388: chown d1/d4/c79 102495967 1 2024-02-16T22:21:55.516 INFO:tasks.workunit.client.1.smithi123.stdout:0/389: dread - d1/f6a zero size 2024-02-16T22:21:55.517 INFO:tasks.workunit.client.0.smithi114.stdout:8/315: mkdir d4/d24/d6d 0 2024-02-16T22:21:55.518 INFO:tasks.workunit.client.0.smithi114.stdout:9/339: truncate d2/d1a/d23/d35/d1d/f3a 683654 0 2024-02-16T22:21:55.519 INFO:tasks.workunit.client.1.smithi123.stdout:9/540: link d4/d36/daa/d24/l44 d4/d1e/d3a/laf 0 2024-02-16T22:21:55.519 INFO:tasks.workunit.client.1.smithi123.stdout:5/405: dwrite d6/dd/df/d1f/f68 [0,4194304] 0 2024-02-16T22:21:55.520 INFO:tasks.workunit.client.1.smithi123.stdout:9/541: read - d4/d36/d8e/d6d/d71/fab zero size 2024-02-16T22:21:55.521 INFO:tasks.workunit.client.0.smithi114.stdout:4/398: rename dd/d1b/d68 to dd/d59/d8f 0 2024-02-16T22:21:55.525 INFO:tasks.workunit.client.0.smithi114.stdout:6/304: dread d6/f1e [0,4194304] 0 2024-02-16T22:21:55.527 INFO:tasks.workunit.client.1.smithi123.stdout:1/379: rmdir d3/d6 39 2024-02-16T22:21:55.529 INFO:tasks.workunit.client.0.smithi114.stdout:6/305: dread d6/f35 [0,4194304] 0 2024-02-16T22:21:55.530 INFO:tasks.workunit.client.1.smithi123.stdout:4/462: creat d5/d49/f91 x:0 0 0 2024-02-16T22:21:55.535 INFO:tasks.workunit.client.0.smithi114.stdout:6/306: dwrite d6/f3e [0,4194304] 0 2024-02-16T22:21:55.536 INFO:tasks.workunit.client.0.smithi114.stdout:6/307: stat d6/dd/l49 0 2024-02-16T22:21:55.536 INFO:tasks.workunit.client.0.smithi114.stdout:6/308: chown d6/dd 1 1 2024-02-16T22:21:55.538 INFO:tasks.workunit.client.0.smithi114.stdout:6/309: write d6/dd/d25/f4c [165420,28443] 0 2024-02-16T22:21:55.545 INFO:tasks.workunit.client.0.smithi114.stdout:9/340: unlink d2/f6 0 2024-02-16T22:21:55.546 INFO:tasks.workunit.client.0.smithi114.stdout:9/341: dread d2/d1a/d23/d35/d1d/f69 [0,4194304] 0 2024-02-16T22:21:55.548 INFO:tasks.workunit.client.1.smithi123.stdout:2/421: write d4/d1a/d1d/d59/d45/f60 [2375734,80691] 0 2024-02-16T22:21:55.549 INFO:tasks.workunit.client.1.smithi123.stdout:2/422: chown d4/d1a/d1d/l2e 0 1 2024-02-16T22:21:55.551 INFO:tasks.workunit.client.1.smithi123.stdout:9/542: unlink d4/d36/d8e/d58/c51 0 2024-02-16T22:21:55.555 INFO:tasks.workunit.client.1.smithi123.stdout:9/543: read d4/d36/daa/d24/f4a [5615281,82321] 0 2024-02-16T22:21:55.555 INFO:tasks.workunit.client.1.smithi123.stdout:4/463: rename d5/d1c/d64/d7b/d84 to d5/d1c/d23/d74/d92 0 2024-02-16T22:21:55.557 INFO:tasks.workunit.client.0.smithi114.stdout:5/275: truncate d0/df/f2b 2276151 0 2024-02-16T22:21:55.557 INFO:tasks.workunit.client.0.smithi114.stdout:5/276: write d0/fa [1657505,94557] 0 2024-02-16T22:21:55.559 INFO:tasks.workunit.client.0.smithi114.stdout:6/310: mkdir d6/d2c/d5e 0 2024-02-16T22:21:55.560 INFO:tasks.workunit.client.0.smithi114.stdout:3/272: getdents d4/d24/d2d 0 2024-02-16T22:21:55.562 INFO:tasks.workunit.client.1.smithi123.stdout:8/497: mkdir d0/d17/d26/d68/d81/d47/d9b/da2 0 2024-02-16T22:21:55.562 INFO:tasks.workunit.client.1.smithi123.stdout:8/498: fdatasync d0/d61/d66/f98 0 2024-02-16T22:21:55.567 INFO:tasks.workunit.client.1.smithi123.stdout:0/390: mknod d1/d8/c81 0 2024-02-16T22:21:55.567 INFO:tasks.workunit.client.0.smithi114.stdout:4/399: creat dd/d1b/d41/d77/f90 x:0 0 0 2024-02-16T22:21:55.569 INFO:tasks.workunit.client.1.smithi123.stdout:5/406: unlink d6/dd/df/d1f/l2c 0 2024-02-16T22:21:55.570 INFO:tasks.workunit.client.1.smithi123.stdout:6/456: dwrite d0/d9/dc/d5f/f6c [0,4194304] 0 2024-02-16T22:21:55.575 INFO:tasks.workunit.client.0.smithi114.stdout:0/276: symlink d1/d1a/d3e/d41/l57 0 2024-02-16T22:21:55.577 INFO:tasks.workunit.client.0.smithi114.stdout:1/290: getdents d4/d9/d46/d47 0 2024-02-16T22:21:55.577 INFO:tasks.workunit.client.0.smithi114.stdout:0/277: readlink d1/d1a/d3e/d41/l46 0 2024-02-16T22:21:55.577 INFO:tasks.workunit.client.0.smithi114.stdout:4/400: dread dd/d1b/d37/d38/f5e [0,4194304] 0 2024-02-16T22:21:55.577 INFO:tasks.workunit.client.0.smithi114.stdout:1/291: fdatasync d4/f21 0 2024-02-16T22:21:55.583 INFO:tasks.workunit.client.0.smithi114.stdout:6/311: creat d6/d2c/d38/d54/f5f x:0 0 0 2024-02-16T22:21:55.583 INFO:tasks.workunit.client.0.smithi114.stdout:6/312: stat d6/d2c/d38/d54/f5f 0 2024-02-16T22:21:55.584 INFO:tasks.workunit.client.0.smithi114.stdout:6/313: write d6/f21 [1365011,63278] 0 2024-02-16T22:21:55.584 INFO:tasks.workunit.client.0.smithi114.stdout:8/316: link d4/d11/f51 d4/d16/d25/f6e 0 2024-02-16T22:21:55.587 INFO:tasks.workunit.client.0.smithi114.stdout:9/342: symlink d2/d1a/d23/d32/d46/l6e 0 2024-02-16T22:21:55.588 INFO:tasks.workunit.client.1.smithi123.stdout:5/407: dread - d6/dd/d44/f7d zero size 2024-02-16T22:21:55.588 INFO:tasks.workunit.client.1.smithi123.stdout:3/416: write d0/d20/d76/d37/d49/f67 [1030246,128049] 0 2024-02-16T22:21:55.588 INFO:tasks.workunit.client.1.smithi123.stdout:9/544: symlink d4/d36/daa/d7f/lb0 0 2024-02-16T22:21:55.588 INFO:tasks.workunit.client.0.smithi114.stdout:0/278: chown d1/d30/c39 65847053 1 2024-02-16T22:21:55.589 INFO:tasks.workunit.client.1.smithi123.stdout:1/380: write d3/d6/db/d10/d4b/f72 [1640161,31566] 0 2024-02-16T22:21:55.590 INFO:tasks.workunit.client.1.smithi123.stdout:8/499: chown d0/d17/d26/d2a/f54 162156 1 2024-02-16T22:21:55.592 INFO:tasks.workunit.client.1.smithi123.stdout:8/500: write d0/d17/d26/d2a/d77/f84 [1010687,36614] 0 2024-02-16T22:21:55.595 INFO:tasks.workunit.client.1.smithi123.stdout:6/457: creat d0/d6/d2f/d5d/fa3 x:0 0 0 2024-02-16T22:21:55.598 INFO:tasks.workunit.client.0.smithi114.stdout:3/273: link f1 d4/d26/f63 0 2024-02-16T22:21:55.599 INFO:tasks.workunit.client.1.smithi123.stdout:0/391: mkdir d1/d82 0 2024-02-16T22:21:55.599 INFO:tasks.workunit.client.1.smithi123.stdout:0/392: chown d1/d8/d13/f20 55592906 1 2024-02-16T22:21:55.600 INFO:tasks.workunit.client.0.smithi114.stdout:5/277: dread d0/df/d16/f1f [0,4194304] 0 2024-02-16T22:21:55.602 INFO:tasks.workunit.client.1.smithi123.stdout:0/393: dread d1/d8/f5d [0,4194304] 0 2024-02-16T22:21:55.606 INFO:tasks.workunit.client.0.smithi114.stdout:6/314: mkdir d6/d2c/d38/d41/d60 0 2024-02-16T22:21:55.608 INFO:tasks.workunit.client.0.smithi114.stdout:9/343: unlink d2/d33/l58 0 2024-02-16T22:21:55.608 INFO:tasks.workunit.client.1.smithi123.stdout:1/381: rmdir d3/d6/db/d10/d2a 39 2024-02-16T22:21:55.610 INFO:tasks.workunit.client.1.smithi123.stdout:8/501: symlink d0/d17/d26/d68/d81/d47/la3 0 2024-02-16T22:21:55.611 INFO:tasks.workunit.client.1.smithi123.stdout:2/423: truncate d4/d1a/d1d/d13/d25/f26 3811394 0 2024-02-16T22:21:55.612 INFO:tasks.workunit.client.0.smithi114.stdout:4/401: truncate dd/f2b 766616 0 2024-02-16T22:21:55.613 INFO:tasks.workunit.client.1.smithi123.stdout:6/458: chown d0/d9/dc/d5f/l84 25 1 2024-02-16T22:21:55.614 INFO:tasks.workunit.client.0.smithi114.stdout:1/292: symlink d4/l60 0 2024-02-16T22:21:55.614 INFO:tasks.workunit.client.1.smithi123.stdout:8/502: dread d0/d17/d26/d32/d45/f5f [0,4194304] 0 2024-02-16T22:21:55.614 INFO:tasks.workunit.client.1.smithi123.stdout:6/459: dread - d0/d9/dc/d3a/d70/f72 zero size 2024-02-16T22:21:55.619 INFO:tasks.workunit.client.0.smithi114.stdout:8/317: symlink d4/d24/d6d/l6f 0 2024-02-16T22:21:55.620 INFO:tasks.workunit.client.1.smithi123.stdout:3/417: symlink d0/d20/d76/l97 0 2024-02-16T22:21:55.623 INFO:tasks.workunit.client.1.smithi123.stdout:7/441: sync 2024-02-16T22:21:55.628 INFO:tasks.workunit.client.0.smithi114.stdout:0/279: unlink d1/d16/l4c 0 2024-02-16T22:21:55.633 INFO:tasks.workunit.client.0.smithi114.stdout:1/293: creat d4/d9/d3d/d3f/f61 x:0 0 0 2024-02-16T22:21:55.636 INFO:tasks.workunit.client.0.smithi114.stdout:8/318: mknod d4/d24/d2c/c70 0 2024-02-16T22:21:55.637 INFO:tasks.workunit.client.1.smithi123.stdout:2/424: mknod d4/d1a/d1d/d38/d5e/c88 0 2024-02-16T22:21:55.638 INFO:tasks.workunit.client.0.smithi114.stdout:3/274: dwrite d4/d26/f34 [0,4194304] 0 2024-02-16T22:21:55.638 INFO:tasks.workunit.client.1.smithi123.stdout:9/545: truncate d4/f1b 947182 0 2024-02-16T22:21:55.638 INFO:tasks.workunit.client.1.smithi123.stdout:2/425: chown d4/d1a/d1d/l6e 316347 1 2024-02-16T22:21:55.638 INFO:tasks.workunit.client.1.smithi123.stdout:3/418: dread d0/d20/f52 [0,4194304] 0 2024-02-16T22:21:55.638 INFO:tasks.workunit.client.1.smithi123.stdout:9/546: fsync d4/d36/d57/f9c 0 2024-02-16T22:21:55.639 INFO:tasks.workunit.client.0.smithi114.stdout:9/344: write d2/d1a/d23/f2c [1604633,54284] 0 2024-02-16T22:21:55.643 INFO:tasks.workunit.client.1.smithi123.stdout:1/382: dread d3/d6/f26 [0,4194304] 0 2024-02-16T22:21:55.646 INFO:tasks.workunit.client.1.smithi123.stdout:5/408: truncate d6/f8 564575 0 2024-02-16T22:21:55.651 INFO:tasks.workunit.client.1.smithi123.stdout:5/409: dwrite d6/dd/f50 [0,4194304] 0 2024-02-16T22:21:55.654 INFO:tasks.workunit.client.0.smithi114.stdout:6/315: truncate d6/d2c/d38/d54/f2d 3315839 0 2024-02-16T22:21:55.657 INFO:tasks.workunit.client.1.smithi123.stdout:7/442: unlink d3/d5/f7f 0 2024-02-16T22:21:55.661 INFO:tasks.workunit.client.0.smithi114.stdout:1/294: mknod d4/d9/d3d/c62 0 2024-02-16T22:21:55.665 INFO:tasks.workunit.client.1.smithi123.stdout:8/503: dwrite d0/d17/f35 [0,4194304] 0 2024-02-16T22:21:55.665 INFO:tasks.workunit.client.0.smithi114.stdout:3/275: creat d4/dd/f64 x:0 0 0 2024-02-16T22:21:55.666 INFO:tasks.workunit.client.1.smithi123.stdout:8/504: chown d0/d61/d66/f86 642892 1 2024-02-16T22:21:55.668 INFO:tasks.workunit.client.0.smithi114.stdout:9/345: write d2/d1a/d23/d35/d47/d63/f6a [677510,55976] 0 2024-02-16T22:21:55.669 INFO:tasks.workunit.client.0.smithi114.stdout:0/280: mknod d1/d1a/d47/c58 0 2024-02-16T22:21:55.671 INFO:tasks.workunit.client.0.smithi114.stdout:4/402: creat dd/d1b/f91 x:0 0 0 2024-02-16T22:21:55.673 INFO:tasks.workunit.client.1.smithi123.stdout:8/505: dwrite d0/f6 [0,4194304] 0 2024-02-16T22:21:55.673 INFO:tasks.workunit.client.0.smithi114.stdout:4/403: dread - dd/d1b/d24/d57/f85 zero size 2024-02-16T22:21:55.674 INFO:tasks.workunit.client.0.smithi114.stdout:4/404: chown dd/d1b/f5b 39040216 1 2024-02-16T22:21:55.675 INFO:tasks.workunit.client.0.smithi114.stdout:6/316: fsync d6/f18 0 2024-02-16T22:21:55.678 INFO:tasks.workunit.client.1.smithi123.stdout:3/419: symlink d0/d20/d76/d5d/d60/l98 0 2024-02-16T22:21:55.680 INFO:tasks.workunit.client.0.smithi114.stdout:8/319: fsync d4/d11/f17 0 2024-02-16T22:21:55.680 INFO:tasks.workunit.client.0.smithi114.stdout:0/281: dread d1/d16/d18/f22 [0,4194304] 0 2024-02-16T22:21:55.683 INFO:tasks.workunit.client.0.smithi114.stdout:3/276: rename d4/dd/d33/f50 to d4/d26/d4a/f65 0 2024-02-16T22:21:55.685 INFO:tasks.workunit.client.0.smithi114.stdout:3/277: dread - d4/dd/d33/f5f zero size 2024-02-16T22:21:55.686 INFO:tasks.workunit.client.0.smithi114.stdout:9/346: symlink d2/dd/d3c/l6f 0 2024-02-16T22:21:55.687 INFO:tasks.workunit.client.1.smithi123.stdout:4/464: sync 2024-02-16T22:21:55.688 INFO:tasks.workunit.client.0.smithi114.stdout:9/347: write d2/d33/f57 [913839,21114] 0 2024-02-16T22:21:55.688 INFO:tasks.workunit.client.1.smithi123.stdout:4/465: chown d5/d1c/d23/l8e 21068 1 2024-02-16T22:21:55.688 INFO:tasks.workunit.client.1.smithi123.stdout:1/383: mkdir d3/d6/d49/d48/d84 0 2024-02-16T22:21:55.689 INFO:tasks.workunit.client.0.smithi114.stdout:9/348: readlink d2/l9 0 2024-02-16T22:21:55.689 INFO:tasks.workunit.client.0.smithi114.stdout:8/320: dwrite d4/d24/d2c/f32 [0,4194304] 0 2024-02-16T22:21:55.689 INFO:tasks.workunit.client.1.smithi123.stdout:4/466: write d5/f1d [4451231,110078] 0 2024-02-16T22:21:55.689 INFO:tasks.workunit.client.0.smithi114.stdout:9/349: stat d2/d5c/d5f 0 2024-02-16T22:21:55.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:55 smithi114 ceph-mon[29235]: pgmap v25: 65 pgs: 65 active+clean; 1.3 GiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 20 MiB/s rd, 178 MiB/s wr, 267 op/s 2024-02-16T22:21:55.706 INFO:tasks.workunit.client.1.smithi123.stdout:9/547: dread d4/d36/daa/d20/f3c [0,4194304] 0 2024-02-16T22:21:55.706 INFO:tasks.workunit.client.1.smithi123.stdout:0/394: sync 2024-02-16T22:21:55.710 INFO:tasks.workunit.client.1.smithi123.stdout:9/548: dwrite d4/d36/daa/d20/f7b [4194304,4194304] 0 2024-02-16T22:21:55.723 INFO:tasks.workunit.client.0.smithi114.stdout:6/317: mkdir d6/dd/d25/d34/d61 0 2024-02-16T22:21:55.723 INFO:tasks.workunit.client.0.smithi114.stdout:6/318: fdatasync d6/da/f56 0 2024-02-16T22:21:55.724 INFO:tasks.workunit.client.0.smithi114.stdout:1/295: mkdir d4/d63 0 2024-02-16T22:21:55.730 INFO:tasks.workunit.client.1.smithi123.stdout:7/443: creat d3/d6e/d24/f90 x:0 0 0 2024-02-16T22:21:55.731 INFO:tasks.workunit.client.1.smithi123.stdout:7/444: read d3/d5/f33 [683553,24367] 0 2024-02-16T22:21:55.734 INFO:tasks.workunit.client.0.smithi114.stdout:3/278: creat d4/d46/d48/f66 x:0 0 0 2024-02-16T22:21:55.735 INFO:tasks.workunit.client.0.smithi114.stdout:3/279: chown d4/d24/l60 0 1 2024-02-16T22:21:55.737 INFO:tasks.workunit.client.0.smithi114.stdout:4/405: truncate dd/d1b/d37/f52 108488 0 2024-02-16T22:21:55.738 INFO:tasks.workunit.client.0.smithi114.stdout:4/406: chown dd/d2e/c7f 86908 1 2024-02-16T22:21:55.740 INFO:tasks.workunit.client.0.smithi114.stdout:4/407: write dd/d59/d8f/f88 [183675,21323] 0 2024-02-16T22:21:55.741 INFO:tasks.workunit.client.1.smithi123.stdout:1/384: stat d3/d6/db/d10/d2a/d53/l71 0 2024-02-16T22:21:55.742 INFO:tasks.workunit.client.0.smithi114.stdout:3/280: dwrite d4/d24/d2d/d45/f51 [0,4194304] 0 2024-02-16T22:21:55.743 INFO:tasks.workunit.client.1.smithi123.stdout:1/385: read d3/f4e [1561368,23565] 0 2024-02-16T22:21:55.743 INFO:tasks.workunit.client.1.smithi123.stdout:0/395: rmdir d1/d4 39 2024-02-16T22:21:55.743 INFO:tasks.workunit.client.1.smithi123.stdout:1/386: chown d3/d6/db/d10/d4b 99 1 2024-02-16T22:21:55.743 INFO:tasks.workunit.client.1.smithi123.stdout:0/396: chown d1/d8/d13/c33 26 1 2024-02-16T22:21:55.744 INFO:tasks.workunit.client.0.smithi114.stdout:0/282: fsync d1/f2a 0 2024-02-16T22:21:55.745 INFO:tasks.workunit.client.0.smithi114.stdout:0/283: stat d1/d1a/f4e 0 2024-02-16T22:21:55.753 INFO:tasks.workunit.client.0.smithi114.stdout:6/319: mkdir d6/dd/d25/d34/d61/d62 0 2024-02-16T22:21:55.754 INFO:tasks.workunit.client.0.smithi114.stdout:6/320: stat d6/d2c/d38/f4f 0 2024-02-16T22:21:55.758 INFO:tasks.workunit.client.0.smithi114.stdout:1/296: mknod d4/d9/c64 0 2024-02-16T22:21:55.760 INFO:tasks.workunit.client.0.smithi114.stdout:3/281: chown d4/d24/c2b 0 1 2024-02-16T22:21:55.761 INFO:tasks.workunit.client.0.smithi114.stdout:3/282: stat d4/d43/f56 0 2024-02-16T22:21:55.762 INFO:tasks.workunit.client.0.smithi114.stdout:3/283: truncate d4/d24/f62 350413 0 2024-02-16T22:21:55.762 INFO:tasks.workunit.client.0.smithi114.stdout:0/284: rename d1/d1a/f2c to d1/d16/d18/f59 0 2024-02-16T22:21:55.767 INFO:tasks.workunit.client.0.smithi114.stdout:8/321: link d4/d24/f3d d4/d11/f71 0 2024-02-16T22:21:55.768 INFO:tasks.workunit.client.1.smithi123.stdout:0/397: mknod d1/d8/d13/d1d/c83 0 2024-02-16T22:21:55.771 INFO:tasks.workunit.client.1.smithi123.stdout:9/549: rename d4/d36/daa/d24/d8c/l94 to d4/d36/d8e/d6f/lb1 0 2024-02-16T22:21:55.773 INFO:tasks.workunit.client.1.smithi123.stdout:6/460: getdents d0/d74 0 2024-02-16T22:21:55.773 INFO:tasks.workunit.client.1.smithi123.stdout:9/550: write d4/d36/daa/d20/f7b [3519862,110929] 0 2024-02-16T22:21:55.773 INFO:tasks.workunit.client.1.smithi123.stdout:3/420: write d0/f15 [547054,53681] 0 2024-02-16T22:21:55.774 INFO:tasks.workunit.client.1.smithi123.stdout:5/410: symlink d6/dd/df/d58/d3f/d6f/l87 0 2024-02-16T22:21:55.774 INFO:tasks.workunit.client.1.smithi123.stdout:6/461: dread - d0/d6/d4d/f9b zero size 2024-02-16T22:21:55.777 INFO:tasks.workunit.client.1.smithi123.stdout:3/421: write d0/d40/f59 [1027417,112726] 0 2024-02-16T22:21:55.777 INFO:tasks.workunit.client.1.smithi123.stdout:6/462: fdatasync d0/d9/dc/d3a/f7d 0 2024-02-16T22:21:55.778 INFO:tasks.workunit.client.1.smithi123.stdout:3/422: chown d0/d20/d5a/l6e 24 1 2024-02-16T22:21:55.779 INFO:tasks.workunit.client.1.smithi123.stdout:1/387: dread d3/f5 [0,4194304] 0 2024-02-16T22:21:55.780 INFO:tasks.workunit.client.1.smithi123.stdout:3/423: dread d0/d20/f42 [0,4194304] 0 2024-02-16T22:21:55.780 INFO:tasks.workunit.client.1.smithi123.stdout:3/424: chown d0/d40/c5f 4 1 2024-02-16T22:21:55.781 INFO:tasks.workunit.client.1.smithi123.stdout:8/506: creat d0/d17/d26/fa4 x:0 0 0 2024-02-16T22:21:55.782 INFO:tasks.workunit.client.1.smithi123.stdout:8/507: write d0/d17/d26/f7a [316992,95139] 0 2024-02-16T22:21:55.784 INFO:tasks.workunit.client.0.smithi114.stdout:8/322: rmdir d4 39 2024-02-16T22:21:55.785 INFO:tasks.workunit.client.1.smithi123.stdout:2/426: link d4/d1a/d1d/d59/d45/l82 d4/d1a/l89 0 2024-02-16T22:21:55.791 INFO:tasks.workunit.client.1.smithi123.stdout:5/411: dread d6/f21 [0,4194304] 0 2024-02-16T22:21:55.792 INFO:tasks.workunit.client.0.smithi114.stdout:0/285: rename d1/l31 to d1/d1a/d47/l5a 0 2024-02-16T22:21:55.792 INFO:tasks.workunit.client.0.smithi114.stdout:6/321: truncate d6/f1e 3759187 0 2024-02-16T22:21:55.794 INFO:tasks.workunit.client.0.smithi114.stdout:1/297: write d4/d9/f12 [1095755,52203] 0 2024-02-16T22:21:55.794 INFO:tasks.workunit.client.1.smithi123.stdout:0/398: truncate d1/d4/d17/f53 527294 0 2024-02-16T22:21:55.796 INFO:tasks.workunit.client.1.smithi123.stdout:7/445: rename d3/d6e/f1b to d3/d26/d3d/f91 0 2024-02-16T22:21:55.797 INFO:tasks.workunit.client.1.smithi123.stdout:7/446: chown d3/d87/c46 379 1 2024-02-16T22:21:55.800 INFO:tasks.workunit.client.0.smithi114.stdout:8/323: write d4/d11/d15/d36/f65 [603315,87234] 0 2024-02-16T22:21:55.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:55 smithi123 ceph-mon[37161]: pgmap v25: 65 pgs: 65 active+clean; 1.3 GiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 20 MiB/s rd, 178 MiB/s wr, 267 op/s 2024-02-16T22:21:55.803 INFO:tasks.workunit.client.0.smithi114.stdout:0/286: symlink d1/d1a/d47/l5b 0 2024-02-16T22:21:55.805 INFO:tasks.workunit.client.0.smithi114.stdout:6/322: stat d6/d2c/c5a 0 2024-02-16T22:21:55.811 INFO:tasks.workunit.client.0.smithi114.stdout:4/408: link dd/d1b/f47 dd/d1b/d37/f92 0 2024-02-16T22:21:55.813 INFO:tasks.workunit.client.0.smithi114.stdout:6/323: dwrite d6/d2c/d38/d41/f4a [0,4194304] 0 2024-02-16T22:21:55.813 INFO:tasks.workunit.client.0.smithi114.stdout:8/324: fsync d4/d11/f60 0 2024-02-16T22:21:55.819 INFO:tasks.workunit.client.0.smithi114.stdout:3/284: getdents d4/d46/d48 0 2024-02-16T22:21:55.820 INFO:tasks.workunit.client.1.smithi123.stdout:4/467: getdents d5/d1c/d23/d74/d92 0 2024-02-16T22:21:55.821 INFO:tasks.workunit.client.0.smithi114.stdout:3/285: chown d4/c2a 98038 1 2024-02-16T22:21:55.821 INFO:tasks.workunit.client.1.smithi123.stdout:4/468: chown d5/d1c/d23/f57 46 1 2024-02-16T22:21:55.823 INFO:tasks.workunit.client.0.smithi114.stdout:6/324: dwrite d6/f3e [0,4194304] 0 2024-02-16T22:21:55.824 INFO:tasks.workunit.client.1.smithi123.stdout:2/427: dread d4/d1a/f32 [0,4194304] 0 2024-02-16T22:21:55.824 INFO:tasks.workunit.client.0.smithi114.stdout:6/325: write d6/f30 [1630034,56603] 0 2024-02-16T22:21:55.825 INFO:tasks.workunit.client.0.smithi114.stdout:1/298: mkdir d4/d9/d65 0 2024-02-16T22:21:55.826 INFO:tasks.workunit.client.1.smithi123.stdout:5/412: symlink d6/dd/df/d58/d3f/l88 0 2024-02-16T22:21:55.827 INFO:tasks.workunit.client.1.smithi123.stdout:0/399: chown d1/d8/d13/f14 33409 1 2024-02-16T22:21:55.828 INFO:tasks.workunit.client.1.smithi123.stdout:5/413: truncate d6/d3b/f65 174177 0 2024-02-16T22:21:55.829 INFO:tasks.workunit.client.1.smithi123.stdout:5/414: chown d6/dd/d44/f7d 163448 1 2024-02-16T22:21:55.829 INFO:tasks.workunit.client.1.smithi123.stdout:2/428: dwrite d4/d1a/d1d/d13/d15/f42 [0,4194304] 0 2024-02-16T22:21:55.830 INFO:tasks.workunit.client.1.smithi123.stdout:3/425: rename d0/d20/d2e/f4f to d0/d85/f99 0 2024-02-16T22:21:55.832 INFO:tasks.workunit.client.0.smithi114.stdout:0/287: dread d1/d16/d18/f1c [0,4194304] 0 2024-02-16T22:21:55.832 INFO:tasks.workunit.client.0.smithi114.stdout:4/409: symlink dd/d59/l93 0 2024-02-16T22:21:55.832 INFO:tasks.workunit.client.1.smithi123.stdout:3/426: write d0/d20/f6f [684469,108809] 0 2024-02-16T22:21:55.832 INFO:tasks.workunit.client.1.smithi123.stdout:7/447: mkdir d3/d6e/d4d/d92 0 2024-02-16T22:21:55.833 INFO:tasks.workunit.client.0.smithi114.stdout:4/410: read dd/f12 [3632316,35814] 0 2024-02-16T22:21:55.833 INFO:tasks.workunit.client.1.smithi123.stdout:3/427: write d0/d20/d76/f4e [3511345,25730] 0 2024-02-16T22:21:55.835 INFO:tasks.workunit.client.1.smithi123.stdout:3/428: readlink d0/d20/d2e/l3e 0 2024-02-16T22:21:55.835 INFO:tasks.workunit.client.0.smithi114.stdout:4/411: write dd/d1b/f91 [730781,63470] 0 2024-02-16T22:21:55.837 INFO:tasks.workunit.client.1.smithi123.stdout:3/429: write d0/f12 [3871450,118174] 0 2024-02-16T22:21:55.838 INFO:tasks.workunit.client.1.smithi123.stdout:6/463: symlink d0/d6/d1a/la4 0 2024-02-16T22:21:55.839 INFO:tasks.workunit.client.1.smithi123.stdout:8/508: mkdir d0/d61/d66/da5 0 2024-02-16T22:21:55.841 INFO:tasks.workunit.client.0.smithi114.stdout:4/412: dread dd/d1b/d24/f2f [0,4194304] 0 2024-02-16T22:21:55.841 INFO:tasks.workunit.client.1.smithi123.stdout:7/448: dwrite d3/d59/f7a [0,4194304] 0 2024-02-16T22:21:55.845 INFO:tasks.workunit.client.0.smithi114.stdout:0/288: dread d1/d1a/f2e [0,4194304] 0 2024-02-16T22:21:55.847 INFO:tasks.workunit.client.0.smithi114.stdout:8/325: symlink d4/d16/d25/l72 0 2024-02-16T22:21:55.852 INFO:tasks.workunit.client.1.smithi123.stdout:2/429: unlink d4/f37 0 2024-02-16T22:21:55.854 INFO:tasks.workunit.client.0.smithi114.stdout:0/289: dwrite d1/d16/f24 [0,4194304] 0 2024-02-16T22:21:55.855 INFO:tasks.workunit.client.1.smithi123.stdout:0/400: dread d1/d4/f11 [0,4194304] 0 2024-02-16T22:21:55.856 INFO:tasks.workunit.client.0.smithi114.stdout:0/290: chown d1/d1a/f45 1634382 1 2024-02-16T22:21:55.858 INFO:tasks.workunit.client.0.smithi114.stdout:3/286: link d4/d26/f39 d4/d24/f67 0 2024-02-16T22:21:55.859 INFO:tasks.workunit.client.0.smithi114.stdout:4/413: dwrite dd/d2e/d3f/f50 [0,4194304] 0 2024-02-16T22:21:55.860 INFO:tasks.workunit.client.0.smithi114.stdout:3/287: write d4/d46/d48/f5c [278248,40382] 0 2024-02-16T22:21:55.861 INFO:tasks.workunit.client.0.smithi114.stdout:0/291: truncate d1/d1a/f36 4481140 0 2024-02-16T22:21:55.864 INFO:tasks.workunit.client.1.smithi123.stdout:9/551: truncate d4/d36/daa/d24/f28 80961 0 2024-02-16T22:21:55.868 INFO:tasks.workunit.client.0.smithi114.stdout:6/326: write d6/da/f3f [2136948,46848] 0 2024-02-16T22:21:55.871 INFO:tasks.workunit.client.0.smithi114.stdout:4/414: symlink dd/d2e/d3f/l94 0 2024-02-16T22:21:55.872 INFO:tasks.workunit.client.1.smithi123.stdout:1/388: creat d3/d6/db/d10/f85 x:0 0 0 2024-02-16T22:21:55.874 INFO:tasks.workunit.client.1.smithi123.stdout:6/464: read d0/d6/d2f/f2a [522607,27383] 0 2024-02-16T22:21:55.876 INFO:tasks.workunit.client.0.smithi114.stdout:8/326: creat d4/d16/d25/d46/f73 x:0 0 0 2024-02-16T22:21:55.876 INFO:tasks.workunit.client.1.smithi123.stdout:8/509: creat d0/d17/d26/d68/d81/d47/fa6 x:0 0 0 2024-02-16T22:21:55.876 INFO:tasks.workunit.client.0.smithi114.stdout:8/327: read - d4/d24/f50 zero size 2024-02-16T22:21:55.878 INFO:tasks.workunit.client.1.smithi123.stdout:4/469: mknod d5/c93 0 2024-02-16T22:21:55.879 INFO:tasks.workunit.client.0.smithi114.stdout:4/415: symlink dd/d2e/d3f/l95 0 2024-02-16T22:21:55.880 INFO:tasks.workunit.client.1.smithi123.stdout:7/449: chown d3/d6e/d1d/c44 719 1 2024-02-16T22:21:55.881 INFO:tasks.workunit.client.0.smithi114.stdout:0/292: mkdir d1/d16/d5c 0 2024-02-16T22:21:55.883 INFO:tasks.workunit.client.0.smithi114.stdout:4/416: dread dd/d1b/d41/f62 [0,4194304] 0 2024-02-16T22:21:55.883 INFO:tasks.workunit.client.0.smithi114.stdout:0/293: write d1/d16/d1f/d28/f54 [48583,117162] 0 2024-02-16T22:21:55.884 INFO:tasks.workunit.client.1.smithi123.stdout:5/415: mkdir d6/dd/df/d58/d89 0 2024-02-16T22:21:55.888 INFO:tasks.workunit.client.0.smithi114.stdout:8/328: truncate d4/d11/f1c 3075459 0 2024-02-16T22:21:55.889 INFO:tasks.workunit.client.1.smithi123.stdout:5/416: dwrite d6/dd/f73 [0,4194304] 0 2024-02-16T22:21:55.894 INFO:tasks.workunit.client.1.smithi123.stdout:0/401: creat d1/d8/d13/d1d/d64/f84 x:0 0 0 2024-02-16T22:21:55.895 INFO:tasks.workunit.client.1.smithi123.stdout:0/402: read - d1/d4/d17/d54/f5b zero size 2024-02-16T22:21:55.896 INFO:tasks.workunit.client.0.smithi114.stdout:4/417: creat dd/d1b/d37/d38/f96 x:0 0 0 2024-02-16T22:21:55.897 INFO:tasks.workunit.client.1.smithi123.stdout:9/552: chown d4/d36/daa/d20/f45 540832 1 2024-02-16T22:21:55.897 INFO:tasks.workunit.client.0.smithi114.stdout:6/327: write d6/d2c/d38/d54/f2d [3989870,75555] 0 2024-02-16T22:21:55.898 INFO:tasks.workunit.client.0.smithi114.stdout:0/294: rmdir d1/d1a/d3e 39 2024-02-16T22:21:55.901 INFO:tasks.workunit.client.1.smithi123.stdout:1/389: unlink d3/d6/db/f5c 0 2024-02-16T22:21:55.902 INFO:tasks.workunit.client.0.smithi114.stdout:8/329: symlink d4/d24/d2c/d5f/l74 0 2024-02-16T22:21:55.903 INFO:tasks.workunit.client.1.smithi123.stdout:6/465: mkdir d0/d6/d2f/da5 0 2024-02-16T22:21:55.906 INFO:tasks.workunit.client.0.smithi114.stdout:3/288: getdents d4/d26/d4a 0 2024-02-16T22:21:55.907 INFO:tasks.workunit.client.1.smithi123.stdout:4/470: creat d5/d1c/d23/d6d/f94 x:0 0 0 2024-02-16T22:21:55.907 INFO:tasks.workunit.client.1.smithi123.stdout:4/471: chown d5/l33 81 1 2024-02-16T22:21:55.909 INFO:tasks.workunit.client.1.smithi123.stdout:4/472: dread d5/d1c/f30 [0,4194304] 0 2024-02-16T22:21:55.910 INFO:tasks.workunit.client.0.smithi114.stdout:6/328: chown d6/da/l52 4035006 1 2024-02-16T22:21:55.912 INFO:tasks.workunit.client.1.smithi123.stdout:2/430: unlink d4/d1a/d1d/d13/l40 0 2024-02-16T22:21:55.914 INFO:tasks.workunit.client.1.smithi123.stdout:2/431: chown d4/d1a/d1d/d13/l47 97050 1 2024-02-16T22:21:55.914 INFO:tasks.workunit.client.0.smithi114.stdout:3/289: dwrite d4/d24/d2d/f4e [0,4194304] 0 2024-02-16T22:21:55.916 INFO:tasks.workunit.client.1.smithi123.stdout:9/553: truncate d4/d36/daa/d20/f3c 4906792 0 2024-02-16T22:21:55.916 INFO:tasks.workunit.client.0.smithi114.stdout:3/290: write d4/d26/f34 [2967678,41772] 0 2024-02-16T22:21:55.918 INFO:tasks.workunit.client.0.smithi114.stdout:3/291: stat d4/d24/d2d/d45/f51 0 2024-02-16T22:21:55.920 INFO:tasks.workunit.client.0.smithi114.stdout:6/329: dwrite d6/d2c/d38/d54/f2d [0,4194304] 0 2024-02-16T22:21:55.922 INFO:tasks.workunit.client.0.smithi114.stdout:4/418: creat dd/d1b/f97 x:0 0 0 2024-02-16T22:21:55.926 INFO:tasks.workunit.client.1.smithi123.stdout:1/390: unlink d3/d6/d18/f21 0 2024-02-16T22:21:55.926 INFO:tasks.workunit.client.1.smithi123.stdout:1/391: chown d3/d6/d49/f50 1895 1 2024-02-16T22:21:55.927 INFO:tasks.workunit.client.1.smithi123.stdout:1/392: dread - d3/d47/f61 zero size 2024-02-16T22:21:55.927 INFO:tasks.workunit.client.1.smithi123.stdout:1/393: fsync d3/d6/db/d10/d2d/f38 0 2024-02-16T22:21:55.928 INFO:tasks.workunit.client.0.smithi114.stdout:8/330: creat d4/d11/f75 x:0 0 0 2024-02-16T22:21:55.929 INFO:tasks.workunit.client.0.smithi114.stdout:8/331: write d4/d11/d15/d36/d3b/f44 [1359878,78820] 0 2024-02-16T22:21:55.932 INFO:tasks.workunit.client.1.smithi123.stdout:1/394: dread d3/d6/db/d10/d4b/f72 [0,4194304] 0 2024-02-16T22:21:55.933 INFO:tasks.workunit.client.1.smithi123.stdout:6/466: stat d0/d9/dc/d5f/f2c 0 2024-02-16T22:21:55.937 INFO:tasks.workunit.client.0.smithi114.stdout:6/330: symlink d6/d2c/l63 0 2024-02-16T22:21:55.938 INFO:tasks.workunit.client.0.smithi114.stdout:6/331: read d6/f35 [2522819,99240] 0 2024-02-16T22:21:55.938 INFO:tasks.workunit.client.0.smithi114.stdout:4/419: fsync dd/f13 0 2024-02-16T22:21:55.939 INFO:tasks.workunit.client.0.smithi114.stdout:4/420: fsync dd/f19 0 2024-02-16T22:21:55.939 INFO:tasks.workunit.client.0.smithi114.stdout:6/332: truncate d6/d2c/d38/d41/f4a 4521538 0 2024-02-16T22:21:55.940 INFO:tasks.workunit.client.1.smithi123.stdout:4/473: rename d5/d49/d71/f8d to d5/d1c/d23/d69/f95 0 2024-02-16T22:21:55.941 INFO:tasks.workunit.client.1.smithi123.stdout:4/474: stat d5/d1c/l5c 0 2024-02-16T22:21:55.942 INFO:tasks.workunit.client.1.smithi123.stdout:5/417: creat d6/dd/df/d58/d89/f8a x:0 0 0 2024-02-16T22:21:55.946 INFO:tasks.workunit.client.1.smithi123.stdout:3/430: link d0/d20/d2e/l45 d0/d20/d76/d37/l9a 0 2024-02-16T22:21:55.946 INFO:tasks.workunit.client.1.smithi123.stdout:3/431: chown d0/d20/d76/d37/d41/l53 44928356 1 2024-02-16T22:21:55.954 INFO:tasks.workunit.client.1.smithi123.stdout:8/510: write d0/d12/f1e [3929592,113449] 0 2024-02-16T22:21:55.956 INFO:tasks.workunit.client.1.smithi123.stdout:3/432: dread d0/f12 [0,4194304] 0 2024-02-16T22:21:55.957 INFO:tasks.workunit.client.0.smithi114.stdout:8/332: symlink d4/d11/l76 0 2024-02-16T22:21:55.959 INFO:tasks.workunit.client.0.smithi114.stdout:3/292: dwrite d4/d26/f31 [0,4194304] 0 2024-02-16T22:21:55.966 INFO:tasks.workunit.client.1.smithi123.stdout:0/403: sync 2024-02-16T22:21:55.966 INFO:tasks.workunit.client.0.smithi114.stdout:3/293: dwrite d4/d26/f31 [0,4194304] 0 2024-02-16T22:21:55.970 INFO:tasks.workunit.client.1.smithi123.stdout:0/404: dread d1/d8/f2f [0,4194304] 0 2024-02-16T22:21:56.026 INFO:tasks.workunit.client.1.smithi123.stdout:9/554: rename d4/d36/daa/d24 to d4/d36/d57/db2 0 2024-02-16T22:21:56.029 INFO:tasks.workunit.client.1.smithi123.stdout:4/475: rmdir d5/d1c/d5e 39 2024-02-16T22:21:56.032 INFO:tasks.workunit.client.1.smithi123.stdout:4/476: read d5/d1c/d23/d74/f5f [580018,117152] 0 2024-02-16T22:21:56.033 INFO:tasks.workunit.client.0.smithi114.stdout:3/294: mkdir d4/d24/d2d/d45/d68 0 2024-02-16T22:21:56.034 INFO:tasks.workunit.client.0.smithi114.stdout:3/295: dread - d4/dd/f64 zero size 2024-02-16T22:21:56.038 INFO:tasks.workunit.client.0.smithi114.stdout:8/333: symlink d4/l77 0 2024-02-16T22:21:56.046 INFO:tasks.workunit.client.1.smithi123.stdout:1/395: link d3/d6/db/d10/d2d/f25 d3/d6/d49/d48/d68/d77/f86 0 2024-02-16T22:21:56.048 INFO:tasks.workunit.client.1.smithi123.stdout:8/511: chown d0/df/c24 337410630 1 2024-02-16T22:21:56.050 INFO:tasks.workunit.client.1.smithi123.stdout:7/450: link d3/d6e/l6b d3/d26/d3e/l93 0 2024-02-16T22:21:56.051 INFO:tasks.workunit.client.1.smithi123.stdout:7/451: write d3/d87/f65 [762636,71069] 0 2024-02-16T22:21:56.062 INFO:tasks.workunit.client.1.smithi123.stdout:2/432: link d4/d1a/f32 d4/d1a/d1d/d13/d15/d6d/d4a/f8a 0 2024-02-16T22:21:56.064 INFO:tasks.workunit.client.0.smithi114.stdout:4/421: mkdir dd/d1b/d24/d98 0 2024-02-16T22:21:56.064 INFO:tasks.workunit.client.0.smithi114.stdout:4/422: stat dd/d1b/d41/c6f 0 2024-02-16T22:21:56.066 INFO:tasks.workunit.client.1.smithi123.stdout:1/396: mkdir d3/d6/db/d10/d4b/d70/d87 0 2024-02-16T22:21:56.067 INFO:tasks.workunit.client.1.smithi123.stdout:3/433: mkdir d0/d20/d29/d9b 0 2024-02-16T22:21:56.068 INFO:tasks.workunit.client.1.smithi123.stdout:3/434: write d0/d50/f7c [304621,57609] 0 2024-02-16T22:21:56.069 INFO:tasks.workunit.client.1.smithi123.stdout:7/452: dread d3/d26/d3e/f5b [0,4194304] 0 2024-02-16T22:21:56.081 INFO:tasks.workunit.client.1.smithi123.stdout:4/477: link d5/d1c/d23/d31/f3e d5/d1c/d23/d47/f96 0 2024-02-16T22:21:56.084 INFO:tasks.workunit.client.1.smithi123.stdout:1/397: mknod d3/d6/d65/c88 0 2024-02-16T22:21:56.085 INFO:tasks.workunit.client.1.smithi123.stdout:1/398: truncate d3/d6/d49/f50 993869 0 2024-02-16T22:21:56.087 INFO:tasks.workunit.client.1.smithi123.stdout:3/435: creat d0/d20/d76/d37/d49/f9c x:0 0 0 2024-02-16T22:21:56.089 INFO:tasks.workunit.client.1.smithi123.stdout:0/405: creat d1/d8/f85 x:0 0 0 2024-02-16T22:21:56.090 INFO:tasks.workunit.client.1.smithi123.stdout:0/406: readlink d1/d4/l31 0 2024-02-16T22:21:56.090 INFO:tasks.workunit.client.1.smithi123.stdout:5/418: getdents d6/dd/df/d58/d89 0 2024-02-16T22:21:56.091 INFO:tasks.workunit.client.1.smithi123.stdout:0/407: dread - d1/d4/d42/f7e zero size 2024-02-16T22:21:56.093 INFO:tasks.workunit.client.1.smithi123.stdout:3/436: dwrite d0/d20/f8b [4194304,4194304] 0 2024-02-16T22:21:56.096 INFO:tasks.workunit.client.1.smithi123.stdout:3/437: write d0/d20/d76/d37/d49/f67 [2150344,27393] 0 2024-02-16T22:21:56.096 INFO:tasks.workunit.client.1.smithi123.stdout:1/399: symlink d3/d52/l89 0 2024-02-16T22:21:56.099 INFO:tasks.workunit.client.1.smithi123.stdout:3/438: fsync d0/d50/d75/f7d 0 2024-02-16T22:21:56.099 INFO:tasks.workunit.client.1.smithi123.stdout:4/478: dwrite d5/d1c/d23/d47/f7e [0,4194304] 0 2024-02-16T22:21:56.099 INFO:tasks.workunit.client.1.smithi123.stdout:3/439: stat d0/f15 0 2024-02-16T22:21:56.101 INFO:tasks.workunit.client.1.smithi123.stdout:0/408: unlink d1/d4/c4f 0 2024-02-16T22:21:56.108 INFO:tasks.workunit.client.1.smithi123.stdout:2/433: dread d4/d1a/d1d/f2b [0,4194304] 0 2024-02-16T22:21:56.111 INFO:tasks.workunit.client.1.smithi123.stdout:3/440: creat d0/d20/d5a/f9d x:0 0 0 2024-02-16T22:21:56.113 INFO:tasks.workunit.client.1.smithi123.stdout:0/409: symlink d1/d8/d13/l86 0 2024-02-16T22:21:56.113 INFO:tasks.workunit.client.1.smithi123.stdout:1/400: dwrite d3/d6/db/d10/d2d/f25 [0,4194304] 0 2024-02-16T22:21:56.114 INFO:tasks.workunit.client.1.smithi123.stdout:3/441: write d0/d20/d76/d5d/f61 [1721000,119917] 0 2024-02-16T22:21:56.115 INFO:tasks.workunit.client.1.smithi123.stdout:2/434: chown d4/d1a/d1d/d13/d25/d72/f73 67051113 1 2024-02-16T22:21:56.117 INFO:tasks.workunit.client.1.smithi123.stdout:0/410: truncate d1/d4/d17/d54/f80 279387 0 2024-02-16T22:21:56.117 INFO:tasks.workunit.client.1.smithi123.stdout:1/401: truncate d3/d6/db/d10/f85 234822 0 2024-02-16T22:21:56.119 INFO:tasks.workunit.client.1.smithi123.stdout:4/479: creat d5/d1c/f97 x:0 0 0 2024-02-16T22:21:56.121 INFO:tasks.workunit.client.1.smithi123.stdout:3/442: dwrite d0/d20/d5a/f9d [0,4194304] 0 2024-02-16T22:21:56.122 INFO:tasks.workunit.client.1.smithi123.stdout:2/435: chown d4/d1a/d1d/l2e 173 1 2024-02-16T22:21:56.124 INFO:tasks.workunit.client.1.smithi123.stdout:0/411: creat d1/d8/d3d/f87 x:0 0 0 2024-02-16T22:21:56.127 INFO:tasks.workunit.client.1.smithi123.stdout:1/402: dwrite d3/d6/db/d10/d4b/d70/f79 [0,4194304] 0 2024-02-16T22:21:56.134 INFO:tasks.workunit.client.1.smithi123.stdout:0/412: dwrite d1/f12 [0,4194304] 0 2024-02-16T22:21:56.136 INFO:tasks.workunit.client.1.smithi123.stdout:0/413: write d1/d8/d13/d1d/d64/f84 [249805,38918] 0 2024-02-16T22:21:56.137 INFO:tasks.workunit.client.1.smithi123.stdout:1/403: chown d3/d6/db/d10/d2d/c3d 464 1 2024-02-16T22:21:56.138 INFO:tasks.workunit.client.1.smithi123.stdout:0/414: stat d1/d8/d13/c6c 0 2024-02-16T22:21:56.138 INFO:tasks.workunit.client.1.smithi123.stdout:2/436: dwrite d4/d1a/d1d/d38/d63/f67 [0,4194304] 0 2024-02-16T22:21:56.145 INFO:tasks.workunit.client.1.smithi123.stdout:9/555: rmdir d4/d36/d8e/d6d 39 2024-02-16T22:21:56.146 INFO:tasks.workunit.client.1.smithi123.stdout:1/404: symlink d3/d6/d65/l8a 0 2024-02-16T22:21:56.147 INFO:tasks.workunit.client.1.smithi123.stdout:1/405: fdatasync d3/d6/db/f3a 0 2024-02-16T22:21:56.150 INFO:tasks.workunit.client.0.smithi114.stdout:4/423: write dd/d2e/d3f/f6c [775666,3793] 0 2024-02-16T22:21:56.151 INFO:tasks.workunit.client.1.smithi123.stdout:1/406: mkdir d3/d6/d11/d8b 0 2024-02-16T22:21:56.151 INFO:tasks.workunit.client.0.smithi114.stdout:4/424: readlink dd/d1b/d37/l86 0 2024-02-16T22:21:56.151 INFO:tasks.workunit.client.1.smithi123.stdout:4/480: dread d5/d1c/d23/f57 [0,4194304] 0 2024-02-16T22:21:56.153 INFO:tasks.workunit.client.1.smithi123.stdout:4/481: truncate d5/d1c/d42/f6a 582638 0 2024-02-16T22:21:56.153 INFO:tasks.workunit.client.1.smithi123.stdout:4/482: read d5/d1c/d23/d56/d6e/d73/f88 [561252,16473] 0 2024-02-16T22:21:56.159 INFO:tasks.workunit.client.1.smithi123.stdout:4/483: mkdir d5/d1c/d23/d31/d98 0 2024-02-16T22:21:56.159 INFO:tasks.workunit.client.1.smithi123.stdout:4/484: chown d5/d1c/d23 6163177 1 2024-02-16T22:21:56.160 INFO:tasks.workunit.client.0.smithi114.stdout:3/296: mknod d4/c69 0 2024-02-16T22:21:56.161 INFO:tasks.workunit.client.1.smithi123.stdout:6/467: rename d0/d6/d1a/f2b to d0/d6/d91/d97/fa6 0 2024-02-16T22:21:56.163 INFO:tasks.workunit.client.1.smithi123.stdout:4/485: mknod d5/d1c/d23/d47/c99 0 2024-02-16T22:21:56.163 INFO:tasks.workunit.client.1.smithi123.stdout:4/486: chown d5/l10 16936 1 2024-02-16T22:21:56.163 INFO:tasks.workunit.client.0.smithi114.stdout:3/297: mkdir d4/d6a 0 2024-02-16T22:21:56.167 INFO:tasks.workunit.client.1.smithi123.stdout:6/468: dwrite d0/d6/d2f/d5d/f68 [0,4194304] 0 2024-02-16T22:21:56.167 INFO:tasks.workunit.client.1.smithi123.stdout:8/512: rename d0/d17 to d0/d29/da7 0 2024-02-16T22:21:56.169 INFO:tasks.workunit.client.0.smithi114.stdout:3/298: truncate d4/d26/f3c 1714258 0 2024-02-16T22:21:56.169 INFO:tasks.workunit.client.0.smithi114.stdout:3/299: chown d4/l22 1822364 1 2024-02-16T22:21:56.172 INFO:tasks.workunit.client.1.smithi123.stdout:4/487: mkdir d5/d1c/d9a 0 2024-02-16T22:21:56.173 INFO:tasks.workunit.client.1.smithi123.stdout:8/513: creat d0/d29/da7/d26/d68/d81/d47/d9b/fa8 x:0 0 0 2024-02-16T22:21:56.175 INFO:tasks.workunit.client.0.smithi114.stdout:3/300: rmdir d4/d24 39 2024-02-16T22:21:56.176 INFO:tasks.workunit.client.1.smithi123.stdout:9/556: dread d4/d36/d57/db2/f4a [0,4194304] 0 2024-02-16T22:21:56.177 INFO:tasks.workunit.client.1.smithi123.stdout:7/453: rename d3/d6e/d1d/c83 to d3/d5/d6c/c94 0 2024-02-16T22:21:56.178 INFO:tasks.workunit.client.1.smithi123.stdout:4/488: unlink d5/l33 0 2024-02-16T22:21:56.179 INFO:tasks.workunit.client.0.smithi114.stdout:3/301: mknod d4/d46/d48/c6b 0 2024-02-16T22:21:56.180 INFO:tasks.workunit.client.0.smithi114.stdout:3/302: write d4/d26/f31 [2733001,73091] 0 2024-02-16T22:21:56.182 INFO:tasks.workunit.client.0.smithi114.stdout:3/303: dread d4/d43/f56 [0,4194304] 0 2024-02-16T22:21:56.184 INFO:tasks.workunit.client.1.smithi123.stdout:7/454: dwrite d3/d6e/d24/f31 [0,4194304] 0 2024-02-16T22:21:56.185 INFO:tasks.workunit.client.1.smithi123.stdout:5/419: rename d6/dd/df/d58/d3f/d6f/f81 to d6/dd/d44/f8b 0 2024-02-16T22:21:56.185 INFO:tasks.workunit.client.1.smithi123.stdout:7/455: stat d3/d59/f7a 0 2024-02-16T22:21:56.186 INFO:tasks.workunit.client.1.smithi123.stdout:1/407: dwrite d3/d6/d49/d48/d68/d77/f86 [0,4194304] 0 2024-02-16T22:21:56.187 INFO:tasks.workunit.client.0.smithi114.stdout:7/316: sync 2024-02-16T22:21:56.187 INFO:tasks.workunit.client.0.smithi114.stdout:2/301: sync 2024-02-16T22:21:56.187 INFO:tasks.workunit.client.0.smithi114.stdout:9/350: sync 2024-02-16T22:21:56.187 INFO:tasks.workunit.client.0.smithi114.stdout:5/278: sync 2024-02-16T22:21:56.188 INFO:tasks.workunit.client.1.smithi123.stdout:7/456: chown d3/d5/l19 282904 1 2024-02-16T22:21:56.188 INFO:tasks.workunit.client.0.smithi114.stdout:7/317: readlink d3/d22/l38 0 2024-02-16T22:21:56.190 INFO:tasks.workunit.client.1.smithi123.stdout:1/408: write d3/d6/db/d10/d2a/f6f [379174,6062] 0 2024-02-16T22:21:56.190 INFO:tasks.workunit.client.1.smithi123.stdout:4/489: mkdir d5/d1c/d23/d74/d92/d9b 0 2024-02-16T22:21:56.191 INFO:tasks.workunit.client.1.smithi123.stdout:1/409: chown d3/d6/d49 1021267 1 2024-02-16T22:21:56.191 INFO:tasks.workunit.client.0.smithi114.stdout:5/279: write d0/d10/f1a [1162024,11378] 0 2024-02-16T22:21:56.191 INFO:tasks.workunit.client.0.smithi114.stdout:7/318: write d3/ff [6985049,84529] 0 2024-02-16T22:21:56.192 INFO:tasks.workunit.client.0.smithi114.stdout:8/334: mkdir d4/d11/d78 0 2024-02-16T22:21:56.195 INFO:tasks.workunit.client.1.smithi123.stdout:5/420: dwrite d6/dd/df/d1f/d25/f66 [0,4194304] 0 2024-02-16T22:21:56.195 INFO:tasks.workunit.client.0.smithi114.stdout:3/304: truncate d4/d24/f67 401230 0 2024-02-16T22:21:56.196 INFO:tasks.workunit.client.1.smithi123.stdout:5/421: stat d6/dd/df/d76/d7f 0 2024-02-16T22:21:56.199 INFO:tasks.workunit.client.1.smithi123.stdout:9/557: mkdir d4/d36/d8e/d6d/db3 0 2024-02-16T22:21:56.201 INFO:tasks.workunit.client.0.smithi114.stdout:2/302: mkdir dd/df/d1c/d21/d61/d69 0 2024-02-16T22:21:56.201 INFO:tasks.workunit.client.0.smithi114.stdout:3/305: dread d4/d24/d2d/f4e [0,4194304] 0 2024-02-16T22:21:56.204 INFO:tasks.workunit.client.0.smithi114.stdout:9/351: mknod d2/d1a/d23/d32/c70 0 2024-02-16T22:21:56.205 INFO:tasks.workunit.client.1.smithi123.stdout:7/457: dread d3/d26/f51 [0,4194304] 0 2024-02-16T22:21:56.208 INFO:tasks.workunit.client.1.smithi123.stdout:3/443: dwrite d0/ff [0,4194304] 0 2024-02-16T22:21:56.210 INFO:tasks.workunit.client.0.smithi114.stdout:2/303: dwrite dd/f55 [0,4194304] 0 2024-02-16T22:21:56.210 INFO:tasks.workunit.client.0.smithi114.stdout:7/319: rmdir d3/d23 39 2024-02-16T22:21:56.211 INFO:tasks.workunit.client.1.smithi123.stdout:0/415: write d1/f6 [5436838,7625] 0 2024-02-16T22:21:56.212 INFO:tasks.workunit.client.0.smithi114.stdout:3/306: dwrite d4/dd/f64 [0,4194304] 0 2024-02-16T22:21:56.212 INFO:tasks.workunit.client.1.smithi123.stdout:7/458: dwrite d3/d6e/d24/f6d [0,4194304] 0 2024-02-16T22:21:56.213 INFO:tasks.workunit.client.0.smithi114.stdout:3/307: stat l3 0 2024-02-16T22:21:56.219 INFO:tasks.workunit.client.0.smithi114.stdout:8/335: rename d4/ld to d4/d11/d15/d31/l79 0 2024-02-16T22:21:56.220 INFO:tasks.workunit.client.1.smithi123.stdout:2/437: dwrite d4/d1a/f32 [4194304,4194304] 0 2024-02-16T22:21:56.221 INFO:tasks.workunit.client.0.smithi114.stdout:8/336: read - d4/d11/f75 zero size 2024-02-16T22:21:56.221 INFO:tasks.workunit.client.0.smithi114.stdout:4/425: write dd/d1b/d37/f52 [950087,52759] 0 2024-02-16T22:21:56.222 INFO:tasks.workunit.client.1.smithi123.stdout:1/410: mkdir d3/d6/db/d10/d2d/d8c 0 2024-02-16T22:21:56.227 INFO:tasks.workunit.client.1.smithi123.stdout:5/422: creat d6/dd/df/f8c x:0 0 0 2024-02-16T22:21:56.231 INFO:tasks.workunit.client.1.smithi123.stdout:6/469: rename d0/f30 to d0/d9/dc/d3a/d52/fa7 0 2024-02-16T22:21:56.235 INFO:tasks.workunit.client.0.smithi114.stdout:2/304: readlink dd/df/d1c/l45 0 2024-02-16T22:21:56.240 INFO:tasks.workunit.client.0.smithi114.stdout:3/308: write d4/d26/f42 [3120234,123705] 0 2024-02-16T22:21:56.241 INFO:tasks.workunit.client.1.smithi123.stdout:0/416: creat d1/d4/d17/d54/f88 x:0 0 0 2024-02-16T22:21:56.242 INFO:tasks.workunit.client.1.smithi123.stdout:0/417: chown d1/d8/d13/l24 7952033 1 2024-02-16T22:21:56.243 INFO:tasks.workunit.client.0.smithi114.stdout:5/280: rename d0/df/d16/d25/c36 to d0/df/d16/d25/d2e/d3b/c4c 0 2024-02-16T22:21:56.244 INFO:tasks.workunit.client.1.smithi123.stdout:7/459: symlink d3/d59/l95 0 2024-02-16T22:21:56.245 INFO:tasks.workunit.client.0.smithi114.stdout:5/281: truncate d0/df/d16/f49 461905 0 2024-02-16T22:21:56.245 INFO:tasks.workunit.client.1.smithi123.stdout:7/460: dread - d3/f8c zero size 2024-02-16T22:21:56.247 INFO:tasks.workunit.client.0.smithi114.stdout:4/426: unlink dd/d1b/d24/f4c 0 2024-02-16T22:21:56.247 INFO:tasks.workunit.client.1.smithi123.stdout:8/514: dwrite d0/d29/da7/d26/d2a/d77/f99 [0,4194304] 0 2024-02-16T22:21:56.248 INFO:tasks.workunit.client.1.smithi123.stdout:8/515: truncate d0/d29/da7/d26/d68/d81/d47/f73 664745 0 2024-02-16T22:21:56.252 INFO:tasks.workunit.client.0.smithi114.stdout:9/352: creat d2/d1a/d65/f71 x:0 0 0 2024-02-16T22:21:56.252 INFO:tasks.workunit.client.1.smithi123.stdout:2/438: stat d4/d1a/d1d/f75 0 2024-02-16T22:21:56.256 INFO:tasks.workunit.client.0.smithi114.stdout:2/305: unlink dd/df/l3e 0 2024-02-16T22:21:56.258 INFO:tasks.workunit.client.1.smithi123.stdout:5/423: rmdir d6/dd/df/d1f/d25 39 2024-02-16T22:21:56.258 INFO:tasks.workunit.client.1.smithi123.stdout:5/424: stat d6/d3b/f65 0 2024-02-16T22:21:56.259 INFO:tasks.workunit.client.0.smithi114.stdout:7/320: dread d3/d14/f29 [0,4194304] 0 2024-02-16T22:21:56.263 INFO:tasks.workunit.client.0.smithi114.stdout:2/306: dwrite dd/f5d [0,4194304] 0 2024-02-16T22:21:56.263 INFO:tasks.workunit.client.1.smithi123.stdout:6/470: mknod d0/d6/d91/ca8 0 2024-02-16T22:21:56.265 INFO:tasks.workunit.client.1.smithi123.stdout:3/444: dread d0/f6 [0,4194304] 0 2024-02-16T22:21:56.266 INFO:tasks.workunit.client.0.smithi114.stdout:8/337: rename d4/d11/d15 to d4/d11/d78/d7a 0 2024-02-16T22:21:56.269 INFO:tasks.workunit.client.1.smithi123.stdout:0/418: symlink d1/d4/l89 0 2024-02-16T22:21:56.270 INFO:tasks.workunit.client.0.smithi114.stdout:4/427: unlink dd/d1b/d41/c48 0 2024-02-16T22:21:56.272 INFO:tasks.workunit.client.1.smithi123.stdout:7/461: rename d3/l6 to d3/d6e/d34/l96 0 2024-02-16T22:21:56.273 INFO:tasks.workunit.client.1.smithi123.stdout:7/462: read d3/d6e/f4f [448079,10795] 0 2024-02-16T22:21:56.274 INFO:tasks.workunit.client.1.smithi123.stdout:4/490: creat d5/d1c/d23/f9c x:0 0 0 2024-02-16T22:21:56.275 INFO:tasks.workunit.client.1.smithi123.stdout:2/439: creat d4/d1a/d1d/d13/d15/d6d/d4a/f8b x:0 0 0 2024-02-16T22:21:56.277 INFO:tasks.workunit.client.1.smithi123.stdout:1/411: symlink d3/d6/d49/d48/d68/d77/l8d 0 2024-02-16T22:21:56.277 INFO:tasks.workunit.client.1.smithi123.stdout:1/412: fdatasync d3/d6/d11/d76/f59 0 2024-02-16T22:21:56.278 INFO:tasks.workunit.client.1.smithi123.stdout:1/413: write d3/d6/d11/f62 [913887,52794] 0 2024-02-16T22:21:56.282 INFO:tasks.workunit.client.0.smithi114.stdout:0/295: sync 2024-02-16T22:21:56.283 INFO:tasks.workunit.client.0.smithi114.stdout:1/299: sync 2024-02-16T22:21:56.283 INFO:tasks.workunit.client.0.smithi114.stdout:6/333: sync 2024-02-16T22:21:56.283 INFO:tasks.workunit.client.0.smithi114.stdout:3/309: write d4/d24/f67 [1174152,55054] 0 2024-02-16T22:21:56.284 INFO:tasks.workunit.client.1.smithi123.stdout:3/445: symlink d0/d50/l9e 0 2024-02-16T22:21:56.285 INFO:tasks.workunit.client.0.smithi114.stdout:8/338: mkdir d4/d16/d25/d39/d7b 0 2024-02-16T22:21:56.288 INFO:tasks.workunit.client.1.smithi123.stdout:9/558: dwrite d4/d33/f73 [0,4194304] 0 2024-02-16T22:21:56.290 INFO:tasks.workunit.client.0.smithi114.stdout:9/353: mkdir d2/d5c/d5f/d72 0 2024-02-16T22:21:56.292 INFO:tasks.workunit.client.0.smithi114.stdout:9/354: read d2/dd/ff [1670469,93579] 0 2024-02-16T22:21:56.293 INFO:tasks.workunit.client.0.smithi114.stdout:7/321: creat d3/d23/f6b x:0 0 0 2024-02-16T22:21:56.294 INFO:tasks.workunit.client.1.smithi123.stdout:2/440: mknod d4/d1a/d1d/d38/d5e/c8c 0 2024-02-16T22:21:56.295 INFO:tasks.workunit.client.0.smithi114.stdout:2/307: mknod dd/df/d1c/d21/d61/d63/c6a 0 2024-02-16T22:21:56.295 INFO:tasks.workunit.client.0.smithi114.stdout:2/308: truncate dd/df/f33 5163115 0 2024-02-16T22:21:56.296 INFO:tasks.workunit.client.0.smithi114.stdout:1/300: dread - d4/d9/f22 zero size 2024-02-16T22:21:56.301 INFO:tasks.workunit.client.0.smithi114.stdout:6/334: symlink d6/dd/d25/d34/d61/l64 0 2024-02-16T22:21:56.302 INFO:tasks.workunit.client.0.smithi114.stdout:2/309: dwrite dd/f36 [0,4194304] 0 2024-02-16T22:21:56.303 INFO:tasks.workunit.client.1.smithi123.stdout:6/471: creat d0/d9/dc/d39/fa9 x:0 0 0 2024-02-16T22:21:56.306 INFO:tasks.workunit.client.1.smithi123.stdout:3/446: creat d0/d85/f9f x:0 0 0 2024-02-16T22:21:56.307 INFO:tasks.workunit.client.1.smithi123.stdout:1/414: write d3/f4e [2219119,55798] 0 2024-02-16T22:21:56.308 INFO:tasks.workunit.client.1.smithi123.stdout:5/425: write d6/f8 [1543177,17796] 0 2024-02-16T22:21:56.308 INFO:tasks.workunit.client.0.smithi114.stdout:4/428: mkdir dd/d1b/d37/d99 0 2024-02-16T22:21:56.309 INFO:tasks.workunit.client.1.smithi123.stdout:5/426: stat d6/dd/df/d58/d3f/l88 0 2024-02-16T22:21:56.313 INFO:tasks.workunit.client.0.smithi114.stdout:9/355: creat d2/d1a/d23/d35/d47/d63/f73 x:0 0 0 2024-02-16T22:21:56.313 INFO:tasks.workunit.client.1.smithi123.stdout:9/559: chown d4/d36/d8e/d58/d6e 48774 1 2024-02-16T22:21:56.316 INFO:tasks.workunit.client.0.smithi114.stdout:1/301: symlink d4/d9/d46/l66 0 2024-02-16T22:21:56.316 INFO:tasks.workunit.client.1.smithi123.stdout:2/441: readlink d4/d1a/d1d/d13/d15/l3f 0 2024-02-16T22:21:56.317 INFO:tasks.workunit.client.0.smithi114.stdout:4/429: dwrite dd/d1b/d41/d77/f90 [0,4194304] 0 2024-02-16T22:21:56.317 INFO:tasks.workunit.client.0.smithi114.stdout:7/322: dread d3/d14/f18 [0,4194304] 0 2024-02-16T22:21:56.319 INFO:tasks.workunit.client.0.smithi114.stdout:7/323: dread - d3/d23/f6b zero size 2024-02-16T22:21:56.321 INFO:tasks.workunit.client.0.smithi114.stdout:1/302: truncate d4/d9/d3d/d3f/f61 447915 0 2024-02-16T22:21:56.324 INFO:tasks.workunit.client.0.smithi114.stdout:5/282: getdents d0/df/d16/d25/d2e/d3b 0 2024-02-16T22:21:56.326 INFO:tasks.workunit.client.0.smithi114.stdout:7/324: dread d3/d3d/f4a [0,4194304] 0 2024-02-16T22:21:56.326 INFO:tasks.workunit.client.0.smithi114.stdout:7/325: chown d3/f6 1916234 1 2024-02-16T22:21:56.327 INFO:tasks.workunit.client.1.smithi123.stdout:8/516: rename d0/d12/d3e/d7c/l9a to d0/d29/da7/d26/d2a/la9 0 2024-02-16T22:21:56.327 INFO:tasks.workunit.client.1.smithi123.stdout:8/517: chown d0/d29 3072462 1 2024-02-16T22:21:56.328 INFO:tasks.workunit.client.1.smithi123.stdout:1/415: stat d3/d6/db/d10/d2d/f5a 0 2024-02-16T22:21:56.330 INFO:tasks.workunit.client.1.smithi123.stdout:5/427: mknod d6/d3b/c8d 0 2024-02-16T22:21:56.330 INFO:tasks.workunit.client.0.smithi114.stdout:0/296: write d1/d1a/d3e/f4a [2164473,36311] 0 2024-02-16T22:21:56.331 INFO:tasks.workunit.client.0.smithi114.stdout:0/297: readlink d1/d1a/d3e/d41/l46 0 2024-02-16T22:21:56.332 INFO:tasks.workunit.client.1.smithi123.stdout:9/560: write d4/d36/d8e/f74 [291785,97434] 0 2024-02-16T22:21:56.333 INFO:tasks.workunit.client.0.smithi114.stdout:4/430: symlink dd/d1b/d41/d77/l9a 0 2024-02-16T22:21:56.334 INFO:tasks.workunit.client.1.smithi123.stdout:4/491: rmdir d5/d49/d71 0 2024-02-16T22:21:56.335 INFO:tasks.workunit.client.1.smithi123.stdout:4/492: chown d5/d1c/d64 433 1 2024-02-16T22:21:56.336 INFO:tasks.workunit.client.1.smithi123.stdout:8/518: dread d0/d12/f50 [0,4194304] 0 2024-02-16T22:21:56.336 INFO:tasks.workunit.client.1.smithi123.stdout:4/493: write d5/d1c/d23/d6d/f94 [220460,8440] 0 2024-02-16T22:21:56.337 INFO:tasks.workunit.client.0.smithi114.stdout:2/310: mkdir dd/d6b 0 2024-02-16T22:21:56.337 INFO:tasks.workunit.client.0.smithi114.stdout:0/298: dread d1/d16/f19 [0,4194304] 0 2024-02-16T22:21:56.337 INFO:tasks.workunit.client.0.smithi114.stdout:2/311: fsync dd/df/d1c/d21/f30 0 2024-02-16T22:21:56.339 INFO:tasks.workunit.client.1.smithi123.stdout:6/472: creat d0/d6/d2f/da5/faa x:0 0 0 2024-02-16T22:21:56.339 INFO:tasks.workunit.client.0.smithi114.stdout:1/303: mkdir d4/d9/d46/d67 0 2024-02-16T22:21:56.340 INFO:tasks.workunit.client.1.smithi123.stdout:3/447: mkdir d0/da0 0 2024-02-16T22:21:56.342 INFO:tasks.workunit.client.1.smithi123.stdout:0/419: rename d1/d8/d13/d1d/d62/d7a/l7b to d1/d8/d13/l8a 0 2024-02-16T22:21:56.345 INFO:tasks.workunit.client.1.smithi123.stdout:5/428: fsync d6/dd/df/d58/d56/f71 0 2024-02-16T22:21:56.347 INFO:tasks.workunit.client.1.smithi123.stdout:7/463: getdents d3/d6e 0 2024-02-16T22:21:56.349 INFO:tasks.workunit.client.1.smithi123.stdout:9/561: fdatasync d4/d36/daa/d20/f45 0 2024-02-16T22:21:56.350 INFO:tasks.workunit.client.1.smithi123.stdout:9/562: write d4/f85 [246813,45094] 0 2024-02-16T22:21:56.350 INFO:tasks.workunit.client.1.smithi123.stdout:9/563: chown d4 1587732564 1 2024-02-16T22:21:56.350 INFO:tasks.workunit.client.1.smithi123.stdout:9/564: stat d4/d36/d8e/d58/d6e/da7 0 2024-02-16T22:21:56.354 INFO:tasks.workunit.client.1.smithi123.stdout:2/442: mknod d4/d1a/d1d/c8d 0 2024-02-16T22:21:56.354 INFO:tasks.workunit.client.1.smithi123.stdout:1/416: write d3/f5 [9750109,66836] 0 2024-02-16T22:21:56.355 INFO:tasks.workunit.client.1.smithi123.stdout:2/443: dread - d4/d1a/d1d/d13/d25/f80 zero size 2024-02-16T22:21:56.356 INFO:tasks.workunit.client.1.smithi123.stdout:8/519: dread - d0/d29/da7/d26/d2a/d4d/f57 zero size 2024-02-16T22:21:56.356 INFO:tasks.workunit.client.1.smithi123.stdout:2/444: dread d4/d1a/f86 [0,4194304] 0 2024-02-16T22:21:56.357 INFO:tasks.workunit.client.0.smithi114.stdout:0/299: fsync d1/d1a/f3b 0 2024-02-16T22:21:56.358 INFO:tasks.workunit.client.0.smithi114.stdout:0/300: fsync d1/d1a/f4e 0 2024-02-16T22:21:56.359 INFO:tasks.workunit.client.0.smithi114.stdout:8/339: getdents d4/d24/d6d 0 2024-02-16T22:21:56.360 INFO:tasks.workunit.client.0.smithi114.stdout:8/340: fsync d4/d11/d78/d7a/d36/f65 0 2024-02-16T22:21:56.361 INFO:tasks.workunit.client.1.smithi123.stdout:3/448: truncate d0/f12 940215 0 2024-02-16T22:21:56.363 INFO:tasks.workunit.client.1.smithi123.stdout:0/420: mknod d1/d4/d17/d54/c8b 0 2024-02-16T22:21:56.364 INFO:tasks.workunit.client.0.smithi114.stdout:4/431: dread f9 [0,4194304] 0 2024-02-16T22:21:56.365 INFO:tasks.workunit.client.1.smithi123.stdout:8/520: dwrite d0/d29/da7/d26/d68/d81/d47/f73 [0,4194304] 0 2024-02-16T22:21:56.365 INFO:tasks.workunit.client.0.smithi114.stdout:7/326: link d3/f1a d3/d22/f6c 0 2024-02-16T22:21:56.366 INFO:tasks.workunit.client.0.smithi114.stdout:9/356: link d2/d1a/d23/d32/c70 d2/c74 0 2024-02-16T22:21:56.366 INFO:tasks.workunit.client.1.smithi123.stdout:2/445: dwrite d4/d1a/d1d/d13/d25/f43 [0,4194304] 0 2024-02-16T22:21:56.366 INFO:tasks.workunit.client.1.smithi123.stdout:0/421: write d1/f6a [420999,39076] 0 2024-02-16T22:21:56.371 INFO:tasks.workunit.client.1.smithi123.stdout:5/429: creat d6/dd/df/d58/d3f/d6f/f8e x:0 0 0 2024-02-16T22:21:56.375 INFO:tasks.workunit.client.1.smithi123.stdout:9/565: symlink d4/d36/d8e/d58/d6e/lb4 0 2024-02-16T22:21:56.376 INFO:tasks.workunit.client.1.smithi123.stdout:6/473: dwrite d0/d9/dc/d5f/f58 [0,4194304] 0 2024-02-16T22:21:56.379 INFO:tasks.workunit.client.1.smithi123.stdout:9/566: read d4/d36/d8e/f61 [266112,45442] 0 2024-02-16T22:21:56.380 INFO:tasks.workunit.client.1.smithi123.stdout:5/430: write d6/dd/df/d58/d3f/d6f/f8e [848016,64214] 0 2024-02-16T22:21:56.380 INFO:tasks.workunit.client.1.smithi123.stdout:3/449: dwrite d0/d20/d2e/f2f [0,4194304] 0 2024-02-16T22:21:56.382 INFO:tasks.workunit.client.1.smithi123.stdout:4/494: symlink d5/d1c/d23/d74/d92/d9b/l9d 0 2024-02-16T22:21:56.382 INFO:tasks.workunit.client.1.smithi123.stdout:6/474: fdatasync d0/d9/dc/d39/fa9 0 2024-02-16T22:21:56.386 INFO:tasks.workunit.client.1.smithi123.stdout:4/495: chown d5/f1d 3065907 1 2024-02-16T22:21:56.387 INFO:tasks.workunit.client.1.smithi123.stdout:8/521: dwrite d0/d29/da7/d26/d2a/d77/f99 [0,4194304] 0 2024-02-16T22:21:56.389 INFO:tasks.workunit.client.0.smithi114.stdout:6/335: link d6/f1e d6/d2c/d38/d41/d60/f65 0 2024-02-16T22:21:56.390 INFO:tasks.workunit.client.1.smithi123.stdout:3/450: read d0/f7 [3325941,98845] 0 2024-02-16T22:21:56.392 INFO:tasks.workunit.client.1.smithi123.stdout:2/446: mknod d4/d1a/d1d/d13/d25/c8e 0 2024-02-16T22:21:56.394 INFO:tasks.workunit.client.1.smithi123.stdout:1/417: dwrite d3/d6/f26 [0,4194304] 0 2024-02-16T22:21:56.395 INFO:tasks.workunit.client.1.smithi123.stdout:4/496: dwrite d5/f1f [0,4194304] 0 2024-02-16T22:21:56.396 INFO:tasks.workunit.client.1.smithi123.stdout:1/418: rename d3/d6 to d3/d6/d49/d8e 22 2024-02-16T22:21:56.397 INFO:tasks.workunit.client.1.smithi123.stdout:7/464: mkdir d3/d6e/d1d/d97 0 2024-02-16T22:21:56.398 INFO:tasks.workunit.client.1.smithi123.stdout:7/465: write d3/d26/d35/f77 [921532,113736] 0 2024-02-16T22:21:56.401 INFO:tasks.workunit.client.0.smithi114.stdout:2/312: link dd/d3c/f4a dd/d6b/f6c 0 2024-02-16T22:21:56.402 INFO:tasks.workunit.client.0.smithi114.stdout:2/313: dread - dd/d4e/f54 zero size 2024-02-16T22:21:56.405 INFO:tasks.workunit.client.0.smithi114.stdout:2/314: dread dd/df/d2c/f32 [0,4194304] 0 2024-02-16T22:21:56.408 INFO:tasks.workunit.client.0.smithi114.stdout:5/283: rename d0/f9 to d0/df/d16/f4d 0 2024-02-16T22:21:56.412 INFO:tasks.workunit.client.1.smithi123.stdout:0/422: write d1/d8/f34 [2147598,37354] 0 2024-02-16T22:21:56.412 INFO:tasks.workunit.client.1.smithi123.stdout:9/567: fdatasync d4/d33/f38 0 2024-02-16T22:21:56.414 INFO:tasks.workunit.client.0.smithi114.stdout:4/432: mkdir dd/d1b/d41/d74/d9b 0 2024-02-16T22:21:56.415 INFO:tasks.workunit.client.0.smithi114.stdout:7/327: symlink d3/d14/l6d 0 2024-02-16T22:21:56.416 INFO:tasks.workunit.client.1.smithi123.stdout:6/475: creat d0/d6/d91/d93/fab x:0 0 0 2024-02-16T22:21:56.417 INFO:tasks.workunit.client.1.smithi123.stdout:5/431: readlink d6/dd/df/d1f/l2a 0 2024-02-16T22:21:56.418 INFO:tasks.workunit.client.1.smithi123.stdout:5/432: read - d6/dd/d44/f77 zero size 2024-02-16T22:21:56.419 INFO:tasks.workunit.client.0.smithi114.stdout:6/336: unlink d6/d2c/l3a 0 2024-02-16T22:21:56.424 INFO:tasks.workunit.client.1.smithi123.stdout:2/447: truncate d4/d1a/d1d/f9 4263182 0 2024-02-16T22:21:56.425 INFO:tasks.workunit.client.0.smithi114.stdout:8/341: mkdir d4/d16/d25/d7c 0 2024-02-16T22:21:56.427 INFO:tasks.workunit.client.0.smithi114.stdout:8/342: chown d4/d16/d25/d39/f4f 20184 1 2024-02-16T22:21:56.428 INFO:tasks.workunit.client.0.smithi114.stdout:4/433: rename dd/d1b/d24/l25 to dd/d1b/d37/d99/l9c 0 2024-02-16T22:21:56.430 INFO:tasks.workunit.client.0.smithi114.stdout:7/328: creat d3/d23/d4e/f6e x:0 0 0 2024-02-16T22:21:56.430 INFO:tasks.workunit.client.0.smithi114.stdout:5/284: dwrite d0/f22 [0,4194304] 0 2024-02-16T22:21:56.430 INFO:tasks.workunit.client.1.smithi123.stdout:1/419: mknod d3/d6/db/d10/d2a/d53/c8f 0 2024-02-16T22:21:56.432 INFO:tasks.workunit.client.1.smithi123.stdout:7/466: symlink d3/d26/d3d/l98 0 2024-02-16T22:21:56.433 INFO:tasks.workunit.client.1.smithi123.stdout:7/467: chown d3/d6e/d24/c32 61 1 2024-02-16T22:21:56.433 INFO:tasks.workunit.client.0.smithi114.stdout:6/337: mkdir d6/d2c/d66 0 2024-02-16T22:21:56.434 INFO:tasks.workunit.client.0.smithi114.stdout:5/285: chown d0/df/d16/f4d 263619344 1 2024-02-16T22:21:56.437 INFO:tasks.workunit.client.0.smithi114.stdout:2/315: getdents dd/df/d1c/d21/d61/d69 0 2024-02-16T22:21:56.437 INFO:tasks.workunit.client.0.smithi114.stdout:6/338: fsync d6/d2c/d38/d41/f4a 0 2024-02-16T22:21:56.439 INFO:tasks.workunit.client.0.smithi114.stdout:0/301: dread d1/f15 [0,4194304] 0 2024-02-16T22:21:56.440 INFO:tasks.workunit.client.0.smithi114.stdout:4/434: dwrite dd/d1b/d37/f73 [0,4194304] 0 2024-02-16T22:21:56.440 INFO:tasks.workunit.client.1.smithi123.stdout:9/568: unlink d4/d36/daa/d2b/f9a 0 2024-02-16T22:21:56.441 INFO:tasks.workunit.client.1.smithi123.stdout:9/569: write d4/d36/d57/db2/f69 [320839,53425] 0 2024-02-16T22:21:56.442 INFO:tasks.workunit.client.1.smithi123.stdout:9/570: chown d4/d36/daa/d2b/f5b 3 1 2024-02-16T22:21:56.442 INFO:tasks.workunit.client.0.smithi114.stdout:0/302: dread d1/d16/f19 [0,4194304] 0 2024-02-16T22:21:56.447 INFO:tasks.workunit.client.0.smithi114.stdout:8/343: creat d4/d24/f7d x:0 0 0 2024-02-16T22:21:56.450 INFO:tasks.workunit.client.1.smithi123.stdout:5/433: chown d6/dd/d44/f8b 258 1 2024-02-16T22:21:56.452 INFO:tasks.workunit.client.0.smithi114.stdout:7/329: mkdir d3/d23/d6f 0 2024-02-16T22:21:56.454 INFO:tasks.workunit.client.0.smithi114.stdout:8/344: dwrite d4/f1d [0,4194304] 0 2024-02-16T22:21:56.460 INFO:tasks.workunit.client.0.smithi114.stdout:9/357: dwrite d2/f13 [0,4194304] 0 2024-02-16T22:21:56.461 INFO:tasks.workunit.client.0.smithi114.stdout:6/339: unlink d6/dd/d25/d34/f57 0 2024-02-16T22:21:56.465 INFO:tasks.workunit.client.1.smithi123.stdout:1/420: rmdir d3/d6/d65 39 2024-02-16T22:21:56.466 INFO:tasks.workunit.client.1.smithi123.stdout:1/421: chown d3/d6/d11/f1f 95268869 1 2024-02-16T22:21:56.467 INFO:tasks.workunit.client.0.smithi114.stdout:4/435: dread dd/d1b/d24/f4a [0,4194304] 0 2024-02-16T22:21:56.468 INFO:tasks.workunit.client.1.smithi123.stdout:0/423: mknod d1/c8c 0 2024-02-16T22:21:56.468 INFO:tasks.workunit.client.1.smithi123.stdout:1/422: stat d3/d6/db/d10/d2a/d53/c8f 0 2024-02-16T22:21:56.472 INFO:tasks.workunit.client.1.smithi123.stdout:4/497: dwrite d5/d1c/d23/d74/f5f [0,4194304] 0 2024-02-16T22:21:56.473 INFO:tasks.workunit.client.0.smithi114.stdout:7/330: fdatasync d3/d22/f6c 0 2024-02-16T22:21:56.475 INFO:tasks.workunit.client.1.smithi123.stdout:9/571: dread d4/d36/daa/d20/f6a [0,4194304] 0 2024-02-16T22:21:56.476 INFO:tasks.workunit.client.1.smithi123.stdout:9/572: chown d4/d36/d8e/d84 6 1 2024-02-16T22:21:56.476 INFO:tasks.workunit.client.1.smithi123.stdout:9/573: chown d4/d1e/d3a/laf 2 1 2024-02-16T22:21:56.478 INFO:tasks.workunit.client.1.smithi123.stdout:8/522: creat d0/faa x:0 0 0 2024-02-16T22:21:56.480 INFO:tasks.workunit.client.0.smithi114.stdout:8/345: mkdir d4/d11/d78/d7a/d7e 0 2024-02-16T22:21:56.481 INFO:tasks.workunit.client.0.smithi114.stdout:8/346: truncate d4/d11/f75 659731 0 2024-02-16T22:21:56.482 INFO:tasks.workunit.client.1.smithi123.stdout:9/574: dwrite d4/f26 [0,4194304] 0 2024-02-16T22:21:56.482 INFO:tasks.workunit.client.1.smithi123.stdout:1/423: dread d3/d6/d11/f1f [4194304,4194304] 0 2024-02-16T22:21:56.482 INFO:tasks.workunit.client.1.smithi123.stdout:3/451: creat d0/d20/d76/d37/fa1 x:0 0 0 2024-02-16T22:21:56.484 INFO:tasks.workunit.client.1.smithi123.stdout:1/424: stat d3/d6/d18/l3e 0 2024-02-16T22:21:56.484 INFO:tasks.workunit.client.1.smithi123.stdout:3/452: stat d0/d20/d76/d37/l66 0 2024-02-16T22:21:56.485 INFO:tasks.workunit.client.0.smithi114.stdout:5/286: write d0/d28/f46 [3364894,80123] 0 2024-02-16T22:21:56.487 INFO:tasks.workunit.client.1.smithi123.stdout:3/453: chown d0/fb 265596731 1 2024-02-16T22:21:56.489 INFO:tasks.workunit.client.1.smithi123.stdout:0/424: creat d1/d8/d13/d1d/f8d x:0 0 0 2024-02-16T22:21:56.490 INFO:tasks.workunit.client.0.smithi114.stdout:9/358: read f1 [134169,78611] 0 2024-02-16T22:21:56.491 INFO:tasks.workunit.client.1.smithi123.stdout:9/575: dwrite d4/d36/d8e/f74 [0,4194304] 0 2024-02-16T22:21:56.493 INFO:tasks.workunit.client.0.smithi114.stdout:2/316: rename dd/df/c1b to dd/df/d1c/d21/d61/c6d 0 2024-02-16T22:21:56.494 INFO:tasks.workunit.client.1.smithi123.stdout:6/476: rename d0/d6/d2f/l32 to d0/d6/d1a/lac 0 2024-02-16T22:21:56.495 INFO:tasks.workunit.client.0.smithi114.stdout:0/303: write d1/d16/f19 [1685688,82710] 0 2024-02-16T22:21:56.496 INFO:tasks.workunit.client.0.smithi114.stdout:9/359: dread d2/d1a/d23/d35/f67 [0,4194304] 0 2024-02-16T22:21:56.500 INFO:tasks.workunit.client.0.smithi114.stdout:4/436: dread dd/d1b/d24/f7a [0,4194304] 0 2024-02-16T22:21:56.501 INFO:tasks.workunit.client.0.smithi114.stdout:4/437: chown cb 49650869 1 2024-02-16T22:21:56.504 INFO:tasks.workunit.client.1.smithi123.stdout:4/498: creat d5/d1c/d23/d76/f9e x:0 0 0 2024-02-16T22:21:56.505 INFO:tasks.workunit.client.0.smithi114.stdout:4/438: write dd/d1b/d24/f7a [1894241,9508] 0 2024-02-16T22:21:56.505 INFO:tasks.workunit.client.1.smithi123.stdout:4/499: write d5/d1c/d23/d69/f79 [690876,3595] 0 2024-02-16T22:21:56.507 INFO:tasks.workunit.client.1.smithi123.stdout:5/434: rmdir d6/dd/df/d1f/d25 39 2024-02-16T22:21:56.507 INFO:tasks.workunit.client.0.smithi114.stdout:9/360: dwrite d2/d1a/d23/d35/f31 [4194304,4194304] 0 2024-02-16T22:21:56.514 INFO:tasks.workunit.client.1.smithi123.stdout:2/448: creat d4/d1a/d1d/f8f x:0 0 0 2024-02-16T22:21:56.515 INFO:tasks.workunit.client.1.smithi123.stdout:7/468: creat d3/d26/f99 x:0 0 0 2024-02-16T22:21:56.516 INFO:tasks.workunit.client.0.smithi114.stdout:6/340: creat d6/d2c/d5e/f67 x:0 0 0 2024-02-16T22:21:56.517 INFO:tasks.workunit.client.0.smithi114.stdout:6/341: chown d6/d2c/d38/d54/c5d 1740784 1 2024-02-16T22:21:56.518 INFO:tasks.workunit.client.1.smithi123.stdout:0/425: chown d1/d4/c1a 31699 1 2024-02-16T22:21:56.519 INFO:tasks.workunit.client.0.smithi114.stdout:6/342: write d6/dd/f17 [2847893,107504] 0 2024-02-16T22:21:56.519 INFO:tasks.workunit.client.0.smithi114.stdout:7/331: dwrite d3/d3d/f4a [0,4194304] 0 2024-02-16T22:21:56.522 INFO:tasks.workunit.client.0.smithi114.stdout:7/332: write d3/fe [3162241,71255] 0 2024-02-16T22:21:56.522 INFO:tasks.workunit.client.0.smithi114.stdout:6/343: write d6/dd/d25/d34/f42 [790047,25303] 0 2024-02-16T22:21:56.526 INFO:tasks.workunit.client.0.smithi114.stdout:9/361: creat d2/d1a/d23/d35/f75 x:0 0 0 2024-02-16T22:21:56.529 INFO:tasks.workunit.client.1.smithi123.stdout:6/477: creat d0/d6/d2f/fad x:0 0 0 2024-02-16T22:21:56.530 INFO:tasks.workunit.client.0.smithi114.stdout:7/333: dwrite d3/d3d/f41 [0,4194304] 0 2024-02-16T22:21:56.531 INFO:tasks.workunit.client.1.smithi123.stdout:4/500: mkdir d5/d1c/d23/d56/d6e/d9f 0 2024-02-16T22:21:56.531 INFO:tasks.workunit.client.1.smithi123.stdout:8/523: write d0/d29/da7/d26/d68/d81/fa1 [2666038,43289] 0 2024-02-16T22:21:56.532 INFO:tasks.workunit.client.1.smithi123.stdout:8/524: fdatasync d0/d29/f64 0 2024-02-16T22:21:56.535 INFO:tasks.workunit.client.1.smithi123.stdout:8/525: read d0/d12/f58 [1427104,23602] 0 2024-02-16T22:21:56.536 INFO:tasks.workunit.client.1.smithi123.stdout:8/526: write d0/f6 [4476640,59721] 0 2024-02-16T22:21:56.540 INFO:tasks.workunit.client.0.smithi114.stdout:4/439: link dd/d1b/d41/d77/f90 dd/d1b/d37/d38/d7c/f9d 0 2024-02-16T22:21:56.540 INFO:tasks.workunit.client.0.smithi114.stdout:0/304: dread d1/d1a/f4e [0,4194304] 0 2024-02-16T22:21:56.541 INFO:tasks.workunit.client.1.smithi123.stdout:3/454: getdents d0/d20/d29/d9b 0 2024-02-16T22:21:56.542 INFO:tasks.workunit.client.0.smithi114.stdout:6/344: dread d6/dd/d25/d34/f42 [0,4194304] 0 2024-02-16T22:21:56.542 INFO:tasks.workunit.client.0.smithi114.stdout:0/305: chown d1/d1a/c40 2771 1 2024-02-16T22:21:56.542 INFO:tasks.workunit.client.1.smithi123.stdout:3/455: read d0/d50/f7c [90192,62463] 0 2024-02-16T22:21:56.544 INFO:tasks.workunit.client.1.smithi123.stdout:0/426: rename d1/d8/d13/d1d/d62/d6d to d1/d4/d17/d54/d8e 0 2024-02-16T22:21:56.550 INFO:tasks.workunit.client.0.smithi114.stdout:7/334: dread d3/f6 [0,4194304] 0 2024-02-16T22:21:56.550 INFO:tasks.workunit.client.1.smithi123.stdout:6/478: creat d0/d6/d91/d93/fae x:0 0 0 2024-02-16T22:21:56.550 INFO:tasks.workunit.client.0.smithi114.stdout:7/335: chown d3/d3d/c42 0 1 2024-02-16T22:21:56.553 INFO:tasks.workunit.client.0.smithi114.stdout:5/287: rename d0/df/d16/d25/d2e/c41 to d0/df/d16/c4e 0 2024-02-16T22:21:56.556 INFO:tasks.workunit.client.0.smithi114.stdout:7/336: dread d3/d14/f43 [0,4194304] 0 2024-02-16T22:21:56.560 INFO:tasks.workunit.client.0.smithi114.stdout:0/306: chown d1/c10 1984510890 1 2024-02-16T22:21:56.564 INFO:tasks.workunit.client.0.smithi114.stdout:8/347: dwrite f0 [0,4194304] 0 2024-02-16T22:21:56.564 INFO:tasks.workunit.client.0.smithi114.stdout:2/317: rename dd/c12 to dd/df/d1c/d21/d61/d69/c6e 0 2024-02-16T22:21:56.567 INFO:tasks.workunit.client.0.smithi114.stdout:4/440: mknod dd/d1b/d24/d98/c9e 0 2024-02-16T22:21:56.568 INFO:tasks.workunit.client.0.smithi114.stdout:2/318: dread dd/df/f14 [0,4194304] 0 2024-02-16T22:21:56.572 INFO:tasks.workunit.client.1.smithi123.stdout:2/449: rename d4/d1a/d1d/d59/d45/f60 to d4/d1a/d1d/d38/f90 0 2024-02-16T22:21:56.573 INFO:tasks.workunit.client.1.smithi123.stdout:4/501: write d5/d1c/d23/d6d/f78 [3520292,30910] 0 2024-02-16T22:21:56.575 INFO:tasks.workunit.client.1.smithi123.stdout:9/576: link d4/c72 d4/d36/d8e/d84/d93/cb5 0 2024-02-16T22:21:56.577 INFO:tasks.workunit.client.0.smithi114.stdout:0/307: chown d1/d16/c1e 4047811 1 2024-02-16T22:21:56.579 INFO:tasks.workunit.client.1.smithi123.stdout:4/502: dwrite d5/f4c [0,4194304] 0 2024-02-16T22:21:56.580 INFO:tasks.workunit.client.1.smithi123.stdout:5/435: creat d6/f8f x:0 0 0 2024-02-16T22:21:56.581 INFO:tasks.workunit.client.1.smithi123.stdout:5/436: truncate d6/f8f 541677 0 2024-02-16T22:21:56.581 INFO:tasks.workunit.client.0.smithi114.stdout:6/345: dwrite d6/f35 [4194304,4194304] 0 2024-02-16T22:21:56.584 INFO:tasks.workunit.client.0.smithi114.stdout:9/362: rename d2/l9 to d2/d5c/d5f/d72/l76 0 2024-02-16T22:21:56.586 INFO:tasks.workunit.client.1.smithi123.stdout:1/425: getdents d3/d6/db/d55 0 2024-02-16T22:21:56.591 INFO:tasks.workunit.client.0.smithi114.stdout:4/441: symlink dd/d2e/d3f/l9f 0 2024-02-16T22:21:56.592 INFO:tasks.workunit.client.0.smithi114.stdout:4/442: chown f9 32 1 2024-02-16T22:21:56.592 INFO:tasks.workunit.client.0.smithi114.stdout:9/363: dwrite d2/d1a/f49 [0,4194304] 0 2024-02-16T22:21:56.592 INFO:tasks.workunit.client.1.smithi123.stdout:3/456: mknod d0/d20/d29/d9b/ca2 0 2024-02-16T22:21:56.593 INFO:tasks.workunit.client.0.smithi114.stdout:4/443: dread - dd/d1b/d24/f70 zero size 2024-02-16T22:21:56.597 INFO:tasks.workunit.client.0.smithi114.stdout:5/288: link d0/d2d/d30/f31 d0/df/d16/d25/d2e/f4f 0 2024-02-16T22:21:56.599 INFO:tasks.workunit.client.1.smithi123.stdout:3/457: dwrite d0/d40/f7a [0,4194304] 0 2024-02-16T22:21:56.600 INFO:tasks.workunit.client.0.smithi114.stdout:7/337: link d3/d14/f43 d3/d14/f70 0 2024-02-16T22:21:56.600 INFO:tasks.workunit.client.1.smithi123.stdout:2/450: symlink d4/d1a/d1d/d59/d53/l91 0 2024-02-16T22:21:56.602 INFO:tasks.workunit.client.0.smithi114.stdout:4/444: dwrite dd/d59/d8f/f88 [0,4194304] 0 2024-02-16T22:21:56.604 INFO:tasks.workunit.client.1.smithi123.stdout:3/458: dread d0/f6 [0,4194304] 0 2024-02-16T22:21:56.604 INFO:tasks.workunit.client.1.smithi123.stdout:6/479: mkdir d0/d6/d1a/daf 0 2024-02-16T22:21:56.606 INFO:tasks.workunit.client.1.smithi123.stdout:9/577: mknod d4/d36/daa/d2b/d6c/cb6 0 2024-02-16T22:21:56.607 INFO:tasks.workunit.client.0.smithi114.stdout:8/348: mknod d4/d16/d25/d7c/c7f 0 2024-02-16T22:21:56.608 INFO:tasks.workunit.client.1.smithi123.stdout:3/459: dread d0/f15 [0,4194304] 0 2024-02-16T22:21:56.611 INFO:tasks.workunit.client.1.smithi123.stdout:7/469: getdents d3 0 2024-02-16T22:21:56.612 INFO:tasks.workunit.client.0.smithi114.stdout:2/319: mknod dd/c6f 0 2024-02-16T22:21:56.612 INFO:tasks.workunit.client.0.smithi114.stdout:5/289: dread d0/df/d16/d1e/f27 [0,4194304] 0 2024-02-16T22:21:56.613 INFO:tasks.workunit.client.0.smithi114.stdout:9/364: mknod d2/d33/d3f/c77 0 2024-02-16T22:21:56.613 INFO:tasks.workunit.client.1.smithi123.stdout:5/437: mkdir d6/dd/df/d90 0 2024-02-16T22:21:56.615 INFO:tasks.workunit.client.0.smithi114.stdout:4/445: dread dd/ff [0,4194304] 0 2024-02-16T22:21:56.616 INFO:tasks.workunit.client.1.smithi123.stdout:2/451: creat d4/d1a/d1d/d38/f92 x:0 0 0 2024-02-16T22:21:56.616 INFO:tasks.workunit.client.0.smithi114.stdout:7/338: fsync d3/f6 0 2024-02-16T22:21:56.617 INFO:tasks.workunit.client.0.smithi114.stdout:7/339: fdatasync d3/d3d/f4b 0 2024-02-16T22:21:56.619 INFO:tasks.workunit.client.0.smithi114.stdout:7/340: write d3/d14/f28 [1603429,126595] 0 2024-02-16T22:21:56.620 INFO:tasks.workunit.client.0.smithi114.stdout:6/346: creat d6/d2c/d66/f68 x:0 0 0 2024-02-16T22:21:56.622 INFO:tasks.workunit.client.0.smithi114.stdout:8/349: readlink d4/d11/d52/l53 0 2024-02-16T22:21:56.624 INFO:tasks.workunit.client.1.smithi123.stdout:6/480: mkdir d0/d6/d4d/db0 0 2024-02-16T22:21:56.625 INFO:tasks.workunit.client.0.smithi114.stdout:6/347: dread d6/da/f3f [0,4194304] 0 2024-02-16T22:21:56.628 INFO:tasks.workunit.client.0.smithi114.stdout:9/365: fsync d2/f5 0 2024-02-16T22:21:56.629 INFO:tasks.workunit.client.0.smithi114.stdout:2/320: dread dd/df/f14 [0,4194304] 0 2024-02-16T22:21:56.629 INFO:tasks.workunit.client.1.smithi123.stdout:3/460: rmdir d0/d20/d29/d9b 39 2024-02-16T22:21:56.630 INFO:tasks.workunit.client.0.smithi114.stdout:6/348: dread d6/dd/f17 [0,4194304] 0 2024-02-16T22:21:56.631 INFO:tasks.workunit.client.0.smithi114.stdout:5/290: symlink d0/df/d16/d25/d3e/l50 0 2024-02-16T22:21:56.631 INFO:tasks.workunit.client.0.smithi114.stdout:6/349: fdatasync d6/da/f56 0 2024-02-16T22:21:56.635 INFO:tasks.workunit.client.0.smithi114.stdout:4/446: creat dd/d59/d8f/fa0 x:0 0 0 2024-02-16T22:21:56.641 INFO:tasks.workunit.client.1.smithi123.stdout:8/527: rename d0/d29/da7/d26/d68/d81/d47/l4a to d0/lab 0 2024-02-16T22:21:56.641 INFO:tasks.workunit.client.1.smithi123.stdout:5/438: fsync d6/dd/df/d1f/f6b 0 2024-02-16T22:21:56.642 INFO:tasks.workunit.client.0.smithi114.stdout:7/341: fdatasync d3/d14/f5c 0 2024-02-16T22:21:56.642 INFO:tasks.workunit.client.0.smithi114.stdout:9/366: dread d2/dd/f10 [0,4194304] 0 2024-02-16T22:21:56.642 INFO:tasks.workunit.client.1.smithi123.stdout:2/452: rmdir d4/d1a/d1d/d59/d45 39 2024-02-16T22:21:56.644 INFO:tasks.workunit.client.1.smithi123.stdout:9/578: dread d4/d36/daa/f1f [0,4194304] 0 2024-02-16T22:21:56.645 INFO:tasks.workunit.client.1.smithi123.stdout:2/453: write d4/d1a/d1d/d13/f36 [5128664,83782] 0 2024-02-16T22:21:56.646 INFO:tasks.workunit.client.1.smithi123.stdout:0/427: link d1/d4/d17/d74/d57/f75 d1/d8/d13/d1d/d62/d7a/f8f 0 2024-02-16T22:21:56.646 INFO:tasks.workunit.client.1.smithi123.stdout:4/503: dwrite d5/d1c/d23/d56/d6e/d73/f88 [0,4194304] 0 2024-02-16T22:21:56.648 INFO:tasks.workunit.client.1.smithi123.stdout:4/504: chown d5/d1c/d42/f6c 5 1 2024-02-16T22:21:56.648 INFO:tasks.workunit.client.0.smithi114.stdout:3/310: sync 2024-02-16T22:21:56.651 INFO:tasks.workunit.client.1.smithi123.stdout:6/481: truncate d0/d6/f18 1581563 0 2024-02-16T22:21:56.662 INFO:tasks.workunit.client.0.smithi114.stdout:6/350: rename d6/dd/d25/l33 to d6/d2c/d38/l69 0 2024-02-16T22:21:56.666 INFO:tasks.workunit.client.0.smithi114.stdout:7/342: mknod d3/d22/d61/c71 0 2024-02-16T22:21:56.668 INFO:tasks.workunit.client.0.smithi114.stdout:9/367: mkdir d2/d1a/d23/d32/d46/d78 0 2024-02-16T22:21:56.671 INFO:tasks.workunit.client.1.smithi123.stdout:1/426: rename d3/f4e to d3/d6/db/d10/d2a/d80/f90 0 2024-02-16T22:21:56.672 INFO:tasks.workunit.client.0.smithi114.stdout:8/350: creat d4/d11/d78/d7a/d7e/f80 x:0 0 0 2024-02-16T22:21:56.673 INFO:tasks.workunit.client.0.smithi114.stdout:8/351: write f0 [2661296,22310] 0 2024-02-16T22:21:56.674 INFO:tasks.workunit.client.0.smithi114.stdout:3/311: unlink l3 0 2024-02-16T22:21:56.676 INFO:tasks.workunit.client.1.smithi123.stdout:3/461: dwrite d0/d20/d76/f36 [4194304,4194304] 0 2024-02-16T22:21:56.677 INFO:tasks.workunit.client.1.smithi123.stdout:5/439: dread d6/dd/df/d1f/d25/f66 [0,4194304] 0 2024-02-16T22:21:56.678 INFO:tasks.workunit.client.1.smithi123.stdout:9/579: creat d4/d1e/d3a/fb7 x:0 0 0 2024-02-16T22:21:56.679 INFO:tasks.workunit.client.1.smithi123.stdout:0/428: truncate d1/d8/d13/f14 696659 0 2024-02-16T22:21:56.681 INFO:tasks.workunit.client.1.smithi123.stdout:5/440: dread d6/dd/f50 [0,4194304] 0 2024-02-16T22:21:56.687 INFO:tasks.workunit.client.1.smithi123.stdout:4/505: mknod d5/d1c/d23/d56/d6e/d73/d87/ca0 0 2024-02-16T22:21:56.688 INFO:tasks.workunit.client.1.smithi123.stdout:0/429: creat d1/d8/d13/d1d/d62/d7a/f90 x:0 0 0 2024-02-16T22:21:56.689 INFO:tasks.workunit.client.1.smithi123.stdout:7/470: getdents d3/d87 0 2024-02-16T22:21:56.690 INFO:tasks.workunit.client.1.smithi123.stdout:4/506: dread - d5/d1c/d23/f3c zero size 2024-02-16T22:21:56.692 INFO:tasks.workunit.client.0.smithi114.stdout:6/351: creat d6/dd/d25/d34/d61/f6a x:0 0 0 2024-02-16T22:21:56.692 INFO:tasks.workunit.client.1.smithi123.stdout:2/454: dread d4/fc [0,4194304] 0 2024-02-16T22:21:56.693 INFO:tasks.workunit.client.0.smithi114.stdout:7/343: mknod d3/d23/c72 0 2024-02-16T22:21:56.693 INFO:tasks.workunit.client.1.smithi123.stdout:3/462: dwrite d0/fe [4194304,4194304] 0 2024-02-16T22:21:56.693 INFO:tasks.workunit.client.0.smithi114.stdout:8/352: symlink d4/d11/d78/d7a/d7e/l81 0 2024-02-16T22:21:56.694 INFO:tasks.workunit.client.0.smithi114.stdout:7/344: fdatasync d3/d46/f51 0 2024-02-16T22:21:56.694 INFO:tasks.workunit.client.0.smithi114.stdout:2/321: creat dd/f70 x:0 0 0 2024-02-16T22:21:56.694 INFO:tasks.workunit.client.1.smithi123.stdout:0/430: creat d1/d4/d42/f91 x:0 0 0 2024-02-16T22:21:56.697 INFO:tasks.workunit.client.1.smithi123.stdout:9/580: link d4/d1e/d3a/c4d d4/d36/d8e/d58/d6e/da7/cb8 0 2024-02-16T22:21:56.702 INFO:tasks.workunit.client.0.smithi114.stdout:6/352: dwrite d6/d2c/d5e/f67 [0,4194304] 0 2024-02-16T22:21:56.702 INFO:tasks.workunit.client.1.smithi123.stdout:7/471: creat d3/d26/d35/f9a x:0 0 0 2024-02-16T22:21:56.703 INFO:tasks.workunit.client.0.smithi114.stdout:9/368: link d2/dd/f10 d2/d5c/d5f/d72/f79 0 2024-02-16T22:21:56.704 INFO:tasks.workunit.client.0.smithi114.stdout:9/369: dread - d2/d1a/d23/d35/f75 zero size 2024-02-16T22:21:56.705 INFO:tasks.workunit.client.0.smithi114.stdout:5/291: getdents d0/df/d16/d1e 0 2024-02-16T22:21:56.707 INFO:tasks.workunit.client.0.smithi114.stdout:5/292: readlink d0/df/d16/l48 0 2024-02-16T22:21:56.708 INFO:tasks.workunit.client.0.smithi114.stdout:5/293: chown d0/df/d16/l48 3 1 2024-02-16T22:21:56.708 INFO:tasks.workunit.client.0.smithi114.stdout:7/345: fsync d3/d46/f47 0 2024-02-16T22:21:56.709 INFO:tasks.workunit.client.1.smithi123.stdout:3/463: fdatasync d0/d20/f52 0 2024-02-16T22:21:56.709 INFO:tasks.workunit.client.1.smithi123.stdout:3/464: fdatasync d0/f17 0 2024-02-16T22:21:56.710 INFO:tasks.workunit.client.0.smithi114.stdout:4/447: link dd/d2e/l39 dd/d1b/d37/d61/la1 0 2024-02-16T22:21:56.710 INFO:tasks.workunit.client.1.smithi123.stdout:7/472: dwrite d3/f86 [0,4194304] 0 2024-02-16T22:21:56.715 INFO:tasks.workunit.client.0.smithi114.stdout:8/353: creat d4/d16/d25/d39/f82 x:0 0 0 2024-02-16T22:21:56.716 INFO:tasks.workunit.client.1.smithi123.stdout:9/581: symlink d4/d36/d8e/d6f/d7e/lb9 0 2024-02-16T22:21:56.716 INFO:tasks.workunit.client.0.smithi114.stdout:8/354: truncate d4/d11/f60 1828531 0 2024-02-16T22:21:56.717 INFO:tasks.workunit.client.0.smithi114.stdout:8/355: write d4/d24/f7d [93289,8115] 0 2024-02-16T22:21:56.722 INFO:tasks.workunit.client.0.smithi114.stdout:6/353: truncate d6/dd/d25/d34/f40 1741635 0 2024-02-16T22:21:56.725 INFO:tasks.workunit.client.0.smithi114.stdout:9/370: mknod d2/d33/d3f/c7a 0 2024-02-16T22:21:56.727 INFO:tasks.workunit.client.1.smithi123.stdout:3/465: rename d0/l62 to d0/d20/d76/d5d/d60/la3 0 2024-02-16T22:21:56.729 INFO:tasks.workunit.client.0.smithi114.stdout:5/294: rmdir d0/d2d 39 2024-02-16T22:21:56.737 INFO:tasks.workunit.client.0.smithi114.stdout:6/354: creat d6/d2c/d38/d54/f6b x:0 0 0 2024-02-16T22:21:56.740 INFO:tasks.workunit.client.1.smithi123.stdout:3/466: dread d0/d50/f7c [0,4194304] 0 2024-02-16T22:21:56.741 INFO:tasks.workunit.client.1.smithi123.stdout:3/467: fdatasync d0/d20/d5a/f9d 0 2024-02-16T22:21:56.742 INFO:tasks.workunit.client.1.smithi123.stdout:9/582: link d4/d36/daa/d7f/ca0 d4/d36/d8e/d6d/cba 0 2024-02-16T22:21:56.745 INFO:tasks.workunit.client.1.smithi123.stdout:8/528: truncate d0/f6 3045770 0 2024-02-16T22:21:56.746 INFO:tasks.workunit.client.1.smithi123.stdout:8/529: stat d0/d29/da7/d43/d34/f6b 0 2024-02-16T22:21:56.750 INFO:tasks.workunit.client.0.smithi114.stdout:5/295: creat d0/d2d/f51 x:0 0 0 2024-02-16T22:21:56.750 INFO:tasks.workunit.client.1.smithi123.stdout:5/441: write d6/dd/df/f41 [120434,104718] 0 2024-02-16T22:21:56.752 INFO:tasks.workunit.client.0.smithi114.stdout:2/322: getdents dd/d3c 0 2024-02-16T22:21:56.753 INFO:tasks.workunit.client.1.smithi123.stdout:1/427: truncate d3/d6/db/d10/d2d/f25 4122890 0 2024-02-16T22:21:56.754 INFO:tasks.workunit.client.1.smithi123.stdout:5/442: fdatasync d6/dd/df/d1f/d25/f39 0 2024-02-16T22:21:56.757 INFO:tasks.workunit.client.1.smithi123.stdout:1/428: unlink d3/d6/db/c12 0 2024-02-16T22:21:56.758 INFO:tasks.workunit.client.0.smithi114.stdout:4/448: getdents dd/d1b/d41 0 2024-02-16T22:21:56.759 INFO:tasks.workunit.client.1.smithi123.stdout:3/468: dread d0/f4c [4194304,4194304] 0 2024-02-16T22:21:56.759 INFO:tasks.workunit.client.1.smithi123.stdout:3/469: fsync d0/f58 0 2024-02-16T22:21:56.760 INFO:tasks.workunit.client.1.smithi123.stdout:3/470: write d0/fe [510299,21504] 0 2024-02-16T22:21:56.761 INFO:tasks.workunit.client.0.smithi114.stdout:5/296: unlink d0/df/d16/d25/d2e/c35 0 2024-02-16T22:21:56.762 INFO:tasks.workunit.client.0.smithi114.stdout:5/297: write d0/d10/f13 [2106490,66585] 0 2024-02-16T22:21:56.762 INFO:tasks.workunit.client.1.smithi123.stdout:5/443: truncate d6/dd/df/d58/d3f/d82/f83 680802 0 2024-02-16T22:21:56.763 INFO:tasks.workunit.client.0.smithi114.stdout:5/298: read - d0/df/d16/d25/d3e/f44 zero size 2024-02-16T22:21:56.764 INFO:tasks.workunit.client.0.smithi114.stdout:2/323: mknod dd/df/d1c/d21/d61/c71 0 2024-02-16T22:21:56.764 INFO:tasks.workunit.client.0.smithi114.stdout:2/324: chown dd/d6b 376 1 2024-02-16T22:21:56.764 INFO:tasks.workunit.client.0.smithi114.stdout:2/325: readlink dd/df/d1c/l45 0 2024-02-16T22:21:56.766 INFO:tasks.workunit.client.1.smithi123.stdout:1/429: rename d3/d6/d49/d48/d68/d77 to d3/d6/db/d10/d2a/d5f/d91 0 2024-02-16T22:21:56.766 INFO:tasks.workunit.client.1.smithi123.stdout:1/430: dread - d3/d6/f78 zero size 2024-02-16T22:21:56.769 INFO:tasks.workunit.client.0.smithi114.stdout:9/371: getdents d2 0 2024-02-16T22:21:56.770 INFO:tasks.workunit.client.0.smithi114.stdout:4/449: rmdir dd/d1b/d37/d99 39 2024-02-16T22:21:56.770 INFO:tasks.workunit.client.1.smithi123.stdout:2/455: write d4/d1a/d1d/d13/d25/f26 [3699740,737] 0 2024-02-16T22:21:56.771 INFO:tasks.workunit.client.1.smithi123.stdout:0/431: dwrite d1/d4/f11 [0,4194304] 0 2024-02-16T22:21:56.772 INFO:tasks.workunit.client.1.smithi123.stdout:0/432: stat d1/d8/d13/c6c 0 2024-02-16T22:21:56.772 INFO:tasks.workunit.client.1.smithi123.stdout:5/444: creat d6/dd/df/d58/d3f/d6f/f91 x:0 0 0 2024-02-16T22:21:56.775 INFO:tasks.workunit.client.0.smithi114.stdout:5/299: mknod d0/df/d16/d1e/c52 0 2024-02-16T22:21:56.776 INFO:tasks.workunit.client.0.smithi114.stdout:5/300: stat d0/df/d16/f1f 0 2024-02-16T22:21:56.777 INFO:tasks.workunit.client.0.smithi114.stdout:2/326: rmdir dd/df/d1c/d21/d61/d63 39 2024-02-16T22:21:56.779 INFO:tasks.workunit.client.1.smithi123.stdout:7/473: dwrite d3/d6e/f4f [0,4194304] 0 2024-02-16T22:21:56.782 INFO:tasks.workunit.client.0.smithi114.stdout:6/355: getdents d6/d2c/d38/d41/d60 0 2024-02-16T22:21:56.782 INFO:tasks.workunit.client.0.smithi114.stdout:7/346: write d3/d14/f43 [493385,68238] 0 2024-02-16T22:21:56.783 INFO:tasks.workunit.client.1.smithi123.stdout:3/471: mknod d0/da0/ca4 0 2024-02-16T22:21:56.783 INFO:tasks.workunit.client.0.smithi114.stdout:6/356: stat d6/d2c/d38/d54/c5d 0 2024-02-16T22:21:56.786 INFO:tasks.workunit.client.0.smithi114.stdout:9/372: fdatasync d2/d1a/d23/d35/f31 0 2024-02-16T22:21:56.787 INFO:tasks.workunit.client.0.smithi114.stdout:5/301: dwrite d0/d28/f37 [0,4194304] 0 2024-02-16T22:21:56.788 INFO:tasks.workunit.client.1.smithi123.stdout:3/472: dwrite d0/d20/d76/f36 [0,4194304] 0 2024-02-16T22:21:56.792 INFO:tasks.workunit.client.0.smithi114.stdout:7/347: dwrite d3/d14/f4d [0,4194304] 0 2024-02-16T22:21:56.792 INFO:tasks.workunit.client.1.smithi123.stdout:0/433: creat d1/d8/d3d/d44/f92 x:0 0 0 2024-02-16T22:21:56.795 INFO:tasks.workunit.client.1.smithi123.stdout:5/445: readlink d6/l15 0 2024-02-16T22:21:56.801 INFO:tasks.workunit.client.0.smithi114.stdout:6/357: mknod d6/d2c/d38/d54/c6c 0 2024-02-16T22:21:56.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:56 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:21:56.804 INFO:tasks.workunit.client.1.smithi123.stdout:5/446: dwrite d6/dd/df/d1f/d25/f5d [4194304,4194304] 0 2024-02-16T22:21:56.806 INFO:tasks.workunit.client.1.smithi123.stdout:7/474: mknod d3/d6e/d52/c9b 0 2024-02-16T22:21:56.808 INFO:tasks.workunit.client.1.smithi123.stdout:7/475: dread d3/d5/f45 [0,4194304] 0 2024-02-16T22:21:56.809 INFO:tasks.workunit.client.1.smithi123.stdout:7/476: write d3/d87/f65 [5054499,103219] 0 2024-02-16T22:21:56.810 INFO:tasks.workunit.client.1.smithi123.stdout:7/477: dread - d3/d5/f61 zero size 2024-02-16T22:21:56.811 INFO:tasks.workunit.client.1.smithi123.stdout:2/456: symlink d4/d1a/d1d/l93 0 2024-02-16T22:21:56.814 INFO:tasks.workunit.client.0.smithi114.stdout:5/302: mkdir d0/df/d53 0 2024-02-16T22:21:56.816 INFO:tasks.workunit.client.1.smithi123.stdout:3/473: creat d0/d20/d5a/fa5 x:0 0 0 2024-02-16T22:21:56.816 INFO:tasks.workunit.client.1.smithi123.stdout:3/474: chown d0/d20/f26 5796954 1 2024-02-16T22:21:56.817 INFO:tasks.workunit.client.0.smithi114.stdout:7/348: rmdir d3/d22 39 2024-02-16T22:21:56.820 INFO:tasks.workunit.client.1.smithi123.stdout:0/434: mknod d1/d4/d42/c93 0 2024-02-16T22:21:56.826 INFO:tasks.workunit.client.1.smithi123.stdout:5/447: creat d6/dd/df/d58/d3f/f92 x:0 0 0 2024-02-16T22:21:56.827 INFO:tasks.workunit.client.0.smithi114.stdout:5/303: fdatasync d0/df/d16/d25/d3e/f44 0 2024-02-16T22:21:56.829 INFO:tasks.workunit.client.0.smithi114.stdout:6/358: rmdir d6/d2c 39 2024-02-16T22:21:56.829 INFO:tasks.workunit.client.0.smithi114.stdout:6/359: chown d6/dd/d25/d34/d58 6 1 2024-02-16T22:21:56.832 INFO:tasks.workunit.client.1.smithi123.stdout:7/478: write d3/d26/d3e/f5b [3763783,45088] 0 2024-02-16T22:21:56.834 INFO:tasks.workunit.client.1.smithi123.stdout:1/431: dwrite d3/d6/db/d10/d2d/f25 [0,4194304] 0 2024-02-16T22:21:56.835 INFO:tasks.workunit.client.0.smithi114.stdout:4/450: getdents dd/d1b/d24 0 2024-02-16T22:21:56.837 INFO:tasks.workunit.client.0.smithi114.stdout:4/451: chown dd/d1b/l26 15863 1 2024-02-16T22:21:56.839 INFO:tasks.workunit.client.0.smithi114.stdout:5/304: mknod d0/df/d16/d25/c54 0 2024-02-16T22:21:56.839 INFO:tasks.workunit.client.0.smithi114.stdout:4/452: write dd/d1b/d24/f70 [1027254,121869] 0 2024-02-16T22:21:56.839 INFO:tasks.workunit.client.1.smithi123.stdout:2/457: dread d4/d1a/d1d/d13/d25/f4d [0,4194304] 0 2024-02-16T22:21:56.840 INFO:tasks.workunit.client.1.smithi123.stdout:1/432: dwrite d3/fe [4194304,4194304] 0 2024-02-16T22:21:56.840 INFO:tasks.workunit.client.0.smithi114.stdout:9/373: dwrite d2/f8 [4194304,4194304] 0 2024-02-16T22:21:56.841 INFO:tasks.workunit.client.0.smithi114.stdout:4/453: read - dd/d1b/d37/d38/f96 zero size 2024-02-16T22:21:56.843 INFO:tasks.workunit.client.0.smithi114.stdout:7/349: mkdir d3/d22/d73 0 2024-02-16T22:21:56.851 INFO:tasks.workunit.client.0.smithi114.stdout:4/454: fsync dd/f12 0 2024-02-16T22:21:56.852 INFO:tasks.workunit.client.0.smithi114.stdout:4/455: dread - dd/d1b/f97 zero size 2024-02-16T22:21:56.855 INFO:tasks.workunit.client.0.smithi114.stdout:9/374: rename d2/d1a/d23/d35/d1f to d2/d5c/d5f/d7b 0 2024-02-16T22:21:56.856 INFO:tasks.workunit.client.1.smithi123.stdout:3/475: write d0/d20/f26 [1222617,64196] 0 2024-02-16T22:21:56.859 INFO:tasks.workunit.client.0.smithi114.stdout:6/360: creat d6/d2c/f6d x:0 0 0 2024-02-16T22:21:56.861 INFO:tasks.workunit.client.0.smithi114.stdout:2/327: getdents dd/df/d1c/d21/d61/d63 0 2024-02-16T22:21:56.861 INFO:tasks.workunit.client.0.smithi114.stdout:2/328: readlink l2 0 2024-02-16T22:21:56.862 INFO:tasks.workunit.client.1.smithi123.stdout:2/458: unlink d4/d1a/d1d/d13/d25/f4d 0 2024-02-16T22:21:56.864 INFO:tasks.workunit.client.1.smithi123.stdout:1/433: rename d3/d6/db/d10/d2a/d5f to d3/d6/d49/d48/d84/d92 0 2024-02-16T22:21:56.865 INFO:tasks.workunit.client.0.smithi114.stdout:4/456: fsync dd/d2e/f6a 0 2024-02-16T22:21:56.866 INFO:tasks.workunit.client.0.smithi114.stdout:4/457: fsync dd/d1b/d24/f36 0 2024-02-16T22:21:56.867 INFO:tasks.workunit.client.0.smithi114.stdout:7/350: write d3/d23/f3b [252690,20805] 0 2024-02-16T22:21:56.868 INFO:tasks.workunit.client.1.smithi123.stdout:7/479: symlink d3/d6e/d4d/d92/l9c 0 2024-02-16T22:21:56.869 INFO:tasks.workunit.client.0.smithi114.stdout:7/351: write d3/d3d/f4b [1113981,125492] 0 2024-02-16T22:21:56.871 INFO:tasks.workunit.client.0.smithi114.stdout:7/352: dread - d3/d23/f6b zero size 2024-02-16T22:21:56.871 INFO:tasks.workunit.client.1.smithi123.stdout:3/476: rmdir d0/d20/d2e 39 2024-02-16T22:21:56.873 INFO:tasks.workunit.client.0.smithi114.stdout:6/361: rename d6/da/l4b to d6/dd/l6e 0 2024-02-16T22:21:56.874 INFO:tasks.workunit.client.0.smithi114.stdout:4/458: dwrite dd/d2e/d3f/f75 [0,4194304] 0 2024-02-16T22:21:56.874 INFO:tasks.workunit.client.1.smithi123.stdout:4/507: sync 2024-02-16T22:21:56.874 INFO:tasks.workunit.client.1.smithi123.stdout:6/482: sync 2024-02-16T22:21:56.875 INFO:tasks.workunit.client.1.smithi123.stdout:4/508: chown d5/d1c/c60 6785 1 2024-02-16T22:21:56.875 INFO:tasks.workunit.client.1.smithi123.stdout:0/435: rmdir d1/d82 0 2024-02-16T22:21:56.876 INFO:tasks.workunit.client.1.smithi123.stdout:4/509: write d5/d1c/f24 [4853406,79252] 0 2024-02-16T22:21:56.877 INFO:tasks.workunit.client.1.smithi123.stdout:0/436: write d1/d4/d17/d74/f58 [3250116,123169] 0 2024-02-16T22:21:56.878 INFO:tasks.workunit.client.0.smithi114.stdout:2/329: dread - dd/df/d1c/f2e zero size 2024-02-16T22:21:56.878 INFO:tasks.workunit.client.0.smithi114.stdout:6/362: dread d6/f35 [0,4194304] 0 2024-02-16T22:21:56.878 INFO:tasks.workunit.client.1.smithi123.stdout:2/459: chown d4/d1a/d1d/d59/d45/f4b 3221 1 2024-02-16T22:21:56.879 INFO:tasks.workunit.client.1.smithi123.stdout:0/437: write d1/d4/d17/d74/f52 [1610551,20320] 0 2024-02-16T22:21:56.879 INFO:tasks.workunit.client.1.smithi123.stdout:1/434: creat d3/d6/d49/d48/d84/f93 x:0 0 0 2024-02-16T22:21:56.880 INFO:tasks.workunit.client.0.smithi114.stdout:9/375: mknod d2/d1a/d23/d32/d46/d78/c7c 0 2024-02-16T22:21:56.881 INFO:tasks.workunit.client.1.smithi123.stdout:2/460: dread - d4/d1a/d1d/d38/f92 zero size 2024-02-16T22:21:56.882 INFO:tasks.workunit.client.1.smithi123.stdout:2/461: read - d4/d1a/d1d/d13/d25/d72/f73 zero size 2024-02-16T22:21:56.882 INFO:tasks.workunit.client.1.smithi123.stdout:5/448: creat d6/dd/df/d1f/d25/d37/f93 x:0 0 0 2024-02-16T22:21:56.884 INFO:tasks.workunit.client.0.smithi114.stdout:7/353: unlink d3/d14/f28 0 2024-02-16T22:21:56.884 INFO:tasks.workunit.client.1.smithi123.stdout:2/462: stat d4/d1a/d1d/d13/d15/c7c 0 2024-02-16T22:21:56.885 INFO:tasks.workunit.client.1.smithi123.stdout:3/477: mkdir d0/d20/d5a/da6 0 2024-02-16T22:21:56.885 INFO:tasks.workunit.client.1.smithi123.stdout:2/463: write d4/d1a/d1d/d13/d15/d6d/d4a/f8b [487527,46312] 0 2024-02-16T22:21:56.886 INFO:tasks.workunit.client.1.smithi123.stdout:2/464: chown d4/d1a/f32 0 1 2024-02-16T22:21:56.887 INFO:tasks.workunit.client.1.smithi123.stdout:2/465: chown d4/d1a/d1d/d38/d5e/c88 18376 1 2024-02-16T22:21:56.889 INFO:tasks.workunit.client.0.smithi114.stdout:4/459: symlink dd/d1b/d41/la2 0 2024-02-16T22:21:56.893 INFO:tasks.workunit.client.0.smithi114.stdout:2/330: chown dd/df/d1c/c27 1519348 1 2024-02-16T22:21:56.894 INFO:tasks.workunit.client.1.smithi123.stdout:7/480: unlink d3/d26/d3d/f91 0 2024-02-16T22:21:56.894 INFO:tasks.workunit.client.1.smithi123.stdout:6/483: dread d0/d9/dc/d5f/f2c [0,4194304] 0 2024-02-16T22:21:56.895 INFO:tasks.workunit.client.0.smithi114.stdout:6/363: creat d6/dd/d25/d34/f6f x:0 0 0 2024-02-16T22:21:56.895 INFO:tasks.workunit.client.0.smithi114.stdout:6/364: stat d6/l16 0 2024-02-16T22:21:56.898 INFO:tasks.workunit.client.1.smithi123.stdout:1/435: creat d3/d6/db/d10/d4b/d70/f94 x:0 0 0 2024-02-16T22:21:56.899 INFO:tasks.workunit.client.1.smithi123.stdout:1/436: write d3/d6/f78 [621758,76350] 0 2024-02-16T22:21:56.900 INFO:tasks.workunit.client.1.smithi123.stdout:5/449: symlink d6/dd/d5f/l94 0 2024-02-16T22:21:56.900 INFO:tasks.workunit.client.1.smithi123.stdout:9/583: sync 2024-02-16T22:21:56.901 INFO:tasks.workunit.client.1.smithi123.stdout:9/584: fdatasync d4/f16 0 2024-02-16T22:21:56.904 INFO:tasks.workunit.client.1.smithi123.stdout:2/466: unlink d4/d1a/d1d/d13/d15/d6d/d4a/f8a 0 2024-02-16T22:21:56.905 INFO:tasks.workunit.client.0.smithi114.stdout:4/460: creat dd/d1b/d37/d38/fa3 x:0 0 0 2024-02-16T22:21:56.906 INFO:tasks.workunit.client.0.smithi114.stdout:2/331: rename fa to dd/df/d1c/d21/d61/d69/f72 0 2024-02-16T22:21:56.907 INFO:tasks.workunit.client.0.smithi114.stdout:2/332: truncate dd/d4e/f54 311824 0 2024-02-16T22:21:56.907 INFO:tasks.workunit.client.1.smithi123.stdout:4/510: mkdir d5/d1c/d23/d56/d6e/d9f/da1 0 2024-02-16T22:21:56.909 INFO:tasks.workunit.client.1.smithi123.stdout:7/481: dread d3/d26/d35/f36 [0,4194304] 0 2024-02-16T22:21:56.910 INFO:tasks.workunit.client.1.smithi123.stdout:6/484: symlink d0/d9/dc/d3a/d52/lb1 0 2024-02-16T22:21:56.911 INFO:tasks.workunit.client.0.smithi114.stdout:7/354: mkdir d3/d74 0 2024-02-16T22:21:56.912 INFO:tasks.workunit.client.1.smithi123.stdout:0/438: dwrite d1/d8/d13/d1d/f25 [4194304,4194304] 0 2024-02-16T22:21:56.915 INFO:tasks.workunit.client.1.smithi123.stdout:2/467: dwrite d4/d1a/d1d/d59/d53/f6b [0,4194304] 0 2024-02-16T22:21:56.916 INFO:tasks.workunit.client.1.smithi123.stdout:2/468: write d4/d1a/d1d/d38/f58 [13760,114366] 0 2024-02-16T22:21:56.917 INFO:tasks.workunit.client.0.smithi114.stdout:6/365: dwrite d6/da/f3f [0,4194304] 0 2024-02-16T22:21:56.919 INFO:tasks.workunit.client.1.smithi123.stdout:1/437: dread d3/d6/db/d10/d2a/d80/f90 [0,4194304] 0 2024-02-16T22:21:56.920 INFO:tasks.workunit.client.0.smithi114.stdout:6/366: truncate d6/dd/d25/f4c 335187 0 2024-02-16T22:21:56.921 INFO:tasks.workunit.client.0.smithi114.stdout:2/333: dwrite dd/df/d2c/f5a [0,4194304] 0 2024-02-16T22:21:56.932 INFO:tasks.workunit.client.0.smithi114.stdout:7/355: readlink d3/l2e 0 2024-02-16T22:21:56.933 INFO:tasks.workunit.client.0.smithi114.stdout:4/461: dread dd/d1b/d24/f6b [0,4194304] 0 2024-02-16T22:21:56.935 INFO:tasks.workunit.client.0.smithi114.stdout:9/376: getdents d2/d1a/d23/d35/d47/d63 0 2024-02-16T22:21:56.935 INFO:tasks.workunit.client.0.smithi114.stdout:4/462: write dd/d59/f80 [112543,95625] 0 2024-02-16T22:21:56.935 INFO:tasks.workunit.client.0.smithi114.stdout:6/367: read d6/d2c/d38/d54/f2d [861615,30680] 0 2024-02-16T22:21:56.936 INFO:tasks.workunit.client.0.smithi114.stdout:6/368: stat d6/d2c 0 2024-02-16T22:21:56.937 INFO:tasks.workunit.client.1.smithi123.stdout:4/511: mkdir d5/d1c/d23/d47/da2 0 2024-02-16T22:21:56.937 INFO:tasks.workunit.client.0.smithi114.stdout:2/334: creat dd/df/d1c/d21/d61/d63/f73 x:0 0 0 2024-02-16T22:21:56.938 INFO:tasks.workunit.client.1.smithi123.stdout:4/512: dread - d5/d49/f89 zero size 2024-02-16T22:21:56.939 INFO:tasks.workunit.client.1.smithi123.stdout:4/513: write d5/d1c/d23/d47/f96 [5083549,87008] 0 2024-02-16T22:21:56.940 INFO:tasks.workunit.client.1.smithi123.stdout:8/530: sync 2024-02-16T22:21:56.941 INFO:tasks.workunit.client.1.smithi123.stdout:4/514: chown d5/d1c/d23/f43 3933534 1 2024-02-16T22:21:56.942 INFO:tasks.workunit.client.0.smithi114.stdout:4/463: read dd/f20 [1338453,126860] 0 2024-02-16T22:21:56.945 INFO:tasks.workunit.client.1.smithi123.stdout:8/531: dwrite d0/f14 [0,4194304] 0 2024-02-16T22:21:56.945 INFO:tasks.workunit.client.0.smithi114.stdout:4/464: dread f9 [0,4194304] 0 2024-02-16T22:21:56.950 INFO:tasks.workunit.client.1.smithi123.stdout:8/532: dwrite d0/d29/da7/d26/d68/d81/d47/f6a [0,4194304] 0 2024-02-16T22:21:56.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:56 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:21:57.009 INFO:tasks.workunit.client.1.smithi123.stdout:7/482: mknod d3/d5/d6c/c9d 0 2024-02-16T22:21:57.010 INFO:tasks.workunit.client.1.smithi123.stdout:7/483: write d3/d6e/d24/f55 [152876,57053] 0 2024-02-16T22:21:57.010 INFO:tasks.workunit.client.1.smithi123.stdout:7/484: chown d3/d26/f51 25285033 1 2024-02-16T22:21:57.011 INFO:tasks.workunit.client.0.smithi114.stdout:9/377: creat d2/d33/f7d x:0 0 0 2024-02-16T22:21:57.012 INFO:tasks.workunit.client.0.smithi114.stdout:9/378: dread - d2/d33/f7d zero size 2024-02-16T22:21:57.012 INFO:tasks.workunit.client.1.smithi123.stdout:3/478: write d0/d20/d2e/f48 [897561,12324] 0 2024-02-16T22:21:57.012 INFO:tasks.workunit.client.0.smithi114.stdout:9/379: dread - d2/d33/f7d zero size 2024-02-16T22:21:57.014 INFO:tasks.workunit.client.1.smithi123.stdout:9/585: write d4/d36/daa/d20/f45 [4273829,78275] 0 2024-02-16T22:21:57.015 INFO:tasks.workunit.client.1.smithi123.stdout:9/586: write d4/d36/d57/db2/f76 [14693,51234] 0 2024-02-16T22:21:57.016 INFO:tasks.workunit.client.0.smithi114.stdout:2/335: read dd/d4e/f58 [19482,66600] 0 2024-02-16T22:21:57.017 INFO:tasks.workunit.client.1.smithi123.stdout:0/439: symlink d1/d4/d17/l94 0 2024-02-16T22:21:57.018 INFO:tasks.workunit.client.1.smithi123.stdout:0/440: truncate d1/d8/d13/f5a 4870483 0 2024-02-16T22:21:57.018 INFO:tasks.workunit.client.1.smithi123.stdout:2/469: creat d4/d1a/d1d/d38/d7b/f94 x:0 0 0 2024-02-16T22:21:57.019 INFO:tasks.workunit.client.0.smithi114.stdout:7/356: write d3/d23/f68 [621903,91752] 0 2024-02-16T22:21:57.019 INFO:tasks.workunit.client.0.smithi114.stdout:4/465: creat dd/d59/fa4 x:0 0 0 2024-02-16T22:21:57.020 INFO:tasks.workunit.client.1.smithi123.stdout:2/470: truncate d4/d1a/d1d/d38/f90 4589459 0 2024-02-16T22:21:57.020 INFO:tasks.workunit.client.0.smithi114.stdout:6/369: mknod d6/dd/d25/d34/d61/d62/c70 0 2024-02-16T22:21:57.021 INFO:tasks.workunit.client.1.smithi123.stdout:2/471: write d4/d1a/d1d/d59/f6f [2070094,42491] 0 2024-02-16T22:21:57.022 INFO:tasks.workunit.client.0.smithi114.stdout:6/370: read d6/f30 [1427354,17341] 0 2024-02-16T22:21:57.024 INFO:tasks.workunit.client.0.smithi114.stdout:9/380: creat d2/d33/d3f/f7e x:0 0 0 2024-02-16T22:21:57.024 INFO:tasks.workunit.client.0.smithi114.stdout:2/336: rmdir dd/d3c/d4c 39 2024-02-16T22:21:57.025 INFO:tasks.workunit.client.0.smithi114.stdout:4/466: dread dd/d1b/d24/f6b [0,4194304] 0 2024-02-16T22:21:57.026 INFO:tasks.workunit.client.0.smithi114.stdout:2/337: chown dd/df/f33 32301 1 2024-02-16T22:21:57.027 INFO:tasks.workunit.client.0.smithi114.stdout:4/467: truncate dd/d2e/f49 4946735 0 2024-02-16T22:21:57.029 INFO:tasks.workunit.client.1.smithi123.stdout:6/485: write d0/d9/f6a [1257454,101110] 0 2024-02-16T22:21:57.031 INFO:tasks.workunit.client.0.smithi114.stdout:7/357: mknod d3/d74/c75 0 2024-02-16T22:21:57.031 INFO:tasks.workunit.client.0.smithi114.stdout:7/358: chown d3/d14/f5c 14136 1 2024-02-16T22:21:57.033 INFO:tasks.workunit.client.0.smithi114.stdout:9/381: rename d2/d1a/d23/f2c to d2/dd/d3c/f7f 0 2024-02-16T22:21:57.035 INFO:tasks.workunit.client.1.smithi123.stdout:1/438: write d3/d6/f46 [163976,34739] 0 2024-02-16T22:21:57.037 INFO:tasks.workunit.client.1.smithi123.stdout:8/533: mkdir d0/d61/dac 0 2024-02-16T22:21:57.038 INFO:tasks.workunit.client.1.smithi123.stdout:1/439: dread d3/d6/f78 [0,4194304] 0 2024-02-16T22:21:57.042 INFO:tasks.workunit.client.0.smithi114.stdout:6/371: symlink d6/dd/d25/l71 0 2024-02-16T22:21:57.043 INFO:tasks.workunit.client.1.smithi123.stdout:8/534: dwrite d0/d29/da7/d26/d68/d81/d47/f6a [0,4194304] 0 2024-02-16T22:21:57.044 INFO:tasks.workunit.client.1.smithi123.stdout:0/441: dread d1/d4/d17/d74/f58 [0,4194304] 0 2024-02-16T22:21:57.045 INFO:tasks.workunit.client.0.smithi114.stdout:2/338: write dd/df/d1c/d21/f26 [176998,25027] 0 2024-02-16T22:21:57.046 INFO:tasks.workunit.client.1.smithi123.stdout:4/515: dread d5/d1c/d5e/f72 [0,4194304] 0 2024-02-16T22:21:57.046 INFO:tasks.workunit.client.1.smithi123.stdout:0/442: write d1/d8/f6b [469612,99405] 0 2024-02-16T22:21:57.048 INFO:tasks.workunit.client.1.smithi123.stdout:4/516: fdatasync d5/d1c/d23/d56/d6e/d73/f88 0 2024-02-16T22:21:57.048 INFO:tasks.workunit.client.0.smithi114.stdout:7/359: write d3/d14/f29 [1875130,81228] 0 2024-02-16T22:21:57.050 INFO:tasks.workunit.client.1.smithi123.stdout:5/450: creat d6/dd/df/d58/f95 x:0 0 0 2024-02-16T22:21:57.051 INFO:tasks.workunit.client.0.smithi114.stdout:7/360: read d3/d22/f6c [1335663,54992] 0 2024-02-16T22:21:57.051 INFO:tasks.workunit.client.0.smithi114.stdout:6/372: symlink d6/d2c/d5e/l72 0 2024-02-16T22:21:57.052 INFO:tasks.workunit.client.0.smithi114.stdout:7/361: truncate d3/d23/f3b 664143 0 2024-02-16T22:21:57.053 INFO:tasks.workunit.client.1.smithi123.stdout:2/472: mknod d4/d1a/d1d/d38/c95 0 2024-02-16T22:21:57.053 INFO:tasks.workunit.client.1.smithi123.stdout:2/473: fdatasync d4/d1a/d1d/d38/f92 0 2024-02-16T22:21:57.058 INFO:tasks.workunit.client.0.smithi114.stdout:9/382: read d2/d1a/d23/d35/d1d/f36 [1273848,127677] 0 2024-02-16T22:21:57.060 INFO:tasks.workunit.client.0.smithi114.stdout:6/373: dread d6/dd/d25/d34/f42 [0,4194304] 0 2024-02-16T22:21:57.061 INFO:tasks.workunit.client.0.smithi114.stdout:7/362: mkdir d3/d22/d76 0 2024-02-16T22:21:57.063 INFO:tasks.workunit.client.0.smithi114.stdout:7/363: readlink d3/d14/l5f 0 2024-02-16T22:21:57.064 INFO:tasks.workunit.client.1.smithi123.stdout:9/587: symlink d4/d36/d8e/d58/lbb 0 2024-02-16T22:21:57.065 INFO:tasks.workunit.client.0.smithi114.stdout:9/383: truncate d2/f5 656352 0 2024-02-16T22:21:57.066 INFO:tasks.workunit.client.1.smithi123.stdout:8/535: symlink d0/d29/lad 0 2024-02-16T22:21:57.067 INFO:tasks.workunit.client.0.smithi114.stdout:2/339: dwrite dd/df/f44 [0,4194304] 0 2024-02-16T22:21:57.071 INFO:tasks.workunit.client.0.smithi114.stdout:4/468: dread dd/d1b/f5b [4194304,4194304] 0 2024-02-16T22:21:57.071 INFO:tasks.workunit.client.1.smithi123.stdout:0/443: rmdir d1/d8/d13/d1d/d64 39 2024-02-16T22:21:57.076 INFO:tasks.workunit.client.0.smithi114.stdout:4/469: dread dd/d1b/d37/d61/f66 [0,4194304] 0 2024-02-16T22:21:57.078 INFO:tasks.workunit.client.0.smithi114.stdout:9/384: rename d2/d33/l48 to d2/d1a/d23/d32/l80 0 2024-02-16T22:21:57.081 INFO:tasks.workunit.client.0.smithi114.stdout:2/340: mkdir dd/df/d1c/d21/d74 0 2024-02-16T22:21:57.083 INFO:tasks.workunit.client.0.smithi114.stdout:6/374: write d6/dd/f29 [2539332,80475] 0 2024-02-16T22:21:57.087 INFO:tasks.workunit.client.1.smithi123.stdout:7/485: link d3/d6e/f2e d3/d6e/d34/f9e 0 2024-02-16T22:21:57.088 INFO:tasks.workunit.client.1.smithi123.stdout:3/479: link d0/fe d0/d20/d76/fa7 0 2024-02-16T22:21:57.089 INFO:tasks.workunit.client.1.smithi123.stdout:9/588: mknod d4/d36/d8e/d84/cbc 0 2024-02-16T22:21:57.090 INFO:tasks.workunit.client.1.smithi123.stdout:9/589: write d4/d36/d57/db2/f76 [436627,113893] 0 2024-02-16T22:21:57.090 INFO:tasks.workunit.client.0.smithi114.stdout:2/341: rename dd/d4e/f54 to dd/df/d1c/d21/d74/f75 0 2024-02-16T22:21:57.091 INFO:tasks.workunit.client.1.smithi123.stdout:5/451: symlink d6/dd/df/d76/d7f/l96 0 2024-02-16T22:21:57.092 INFO:tasks.workunit.client.1.smithi123.stdout:5/452: fdatasync d6/dd/df/d58/d3f/d6f/f8e 0 2024-02-16T22:21:57.096 INFO:tasks.workunit.client.1.smithi123.stdout:4/517: dread d5/d1c/d42/f4a [0,4194304] 0 2024-02-16T22:21:57.096 INFO:tasks.workunit.client.0.smithi114.stdout:9/385: link d2/dd/f3b d2/d1a/d23/d35/d47/d63/f81 0 2024-02-16T22:21:57.096 INFO:tasks.workunit.client.0.smithi114.stdout:9/386: chown d2/d1a/d23/d35/l43 882 1 2024-02-16T22:21:57.097 INFO:tasks.workunit.client.1.smithi123.stdout:4/518: dread - d5/d1c/d42/f6c zero size 2024-02-16T22:21:57.099 INFO:tasks.workunit.client.1.smithi123.stdout:7/486: dwrite d3/d87/f6f [0,4194304] 0 2024-02-16T22:21:57.109 INFO:tasks.workunit.client.0.smithi114.stdout:7/364: getdents d3/d46 0 2024-02-16T22:21:57.109 INFO:tasks.workunit.client.0.smithi114.stdout:7/365: dread - d3/d23/f62 zero size 2024-02-16T22:21:57.110 INFO:tasks.workunit.client.1.smithi123.stdout:5/453: creat d6/dd/df/d1f/d25/f97 x:0 0 0 2024-02-16T22:21:57.110 INFO:tasks.workunit.client.1.smithi123.stdout:5/454: dread - d6/dd/d44/f7d zero size 2024-02-16T22:21:57.111 INFO:tasks.workunit.client.0.smithi114.stdout:4/470: rename dd/d1b/d37/d61/c84 to dd/d1b/d37/d99/ca5 0 2024-02-16T22:21:57.112 INFO:tasks.workunit.client.1.smithi123.stdout:1/440: rename d3/d6/db/d10/d2d/l7b to d3/d6/db/d10/l95 0 2024-02-16T22:21:57.112 INFO:tasks.workunit.client.1.smithi123.stdout:9/590: rename d4/d36/d8e to d4/d36/d8e/d84/dbd 22 2024-02-16T22:21:57.113 INFO:tasks.workunit.client.0.smithi114.stdout:6/375: creat d6/dd/d25/f73 x:0 0 0 2024-02-16T22:21:57.117 INFO:tasks.workunit.client.0.smithi114.stdout:7/366: creat d3/d14/f77 x:0 0 0 2024-02-16T22:21:57.118 INFO:tasks.workunit.client.0.smithi114.stdout:7/367: write d3/d14/f4c [934809,73722] 0 2024-02-16T22:21:57.118 INFO:tasks.workunit.client.1.smithi123.stdout:3/480: creat d0/d20/d29/d9b/fa8 x:0 0 0 2024-02-16T22:21:57.120 INFO:tasks.workunit.client.0.smithi114.stdout:2/342: write dd/d6b/f6c [946081,73873] 0 2024-02-16T22:21:57.120 INFO:tasks.workunit.client.1.smithi123.stdout:4/519: dwrite d5/d1c/d23/f57 [0,4194304] 0 2024-02-16T22:21:57.121 INFO:tasks.workunit.client.0.smithi114.stdout:6/376: symlink d6/d2c/d38/l74 0 2024-02-16T22:21:57.122 INFO:tasks.workunit.client.1.smithi123.stdout:4/520: write d5/f1f [5240526,70158] 0 2024-02-16T22:21:57.124 INFO:tasks.workunit.client.0.smithi114.stdout:7/368: rename d3/d14/l2a to d3/d23/l78 0 2024-02-16T22:21:57.125 INFO:tasks.workunit.client.1.smithi123.stdout:2/474: link d4/d1a/d1d/d59/d45/f70 d4/d1a/d1d/d38/d63/f96 0 2024-02-16T22:21:57.126 INFO:tasks.workunit.client.1.smithi123.stdout:2/475: readlink d4/d1a/d1d/d59/d45/l62 0 2024-02-16T22:21:57.127 INFO:tasks.workunit.client.1.smithi123.stdout:2/476: write d4/d1a/d1d/d13/d15/d6d/f28 [2611811,39782] 0 2024-02-16T22:21:57.127 INFO:tasks.workunit.client.1.smithi123.stdout:2/477: chown d4/f54 982 1 2024-02-16T22:21:57.128 INFO:tasks.workunit.client.1.smithi123.stdout:5/455: mkdir d6/dd/df/d58/d89/d98 0 2024-02-16T22:21:57.128 INFO:tasks.workunit.client.0.smithi114.stdout:2/343: dwrite dd/df/f1e [0,4194304] 0 2024-02-16T22:21:57.133 INFO:tasks.workunit.client.1.smithi123.stdout:1/441: mknod d3/d6/d18/c96 0 2024-02-16T22:21:57.134 INFO:tasks.workunit.client.1.smithi123.stdout:1/442: read d3/d6/db/d10/f85 [198235,96376] 0 2024-02-16T22:21:57.135 INFO:tasks.workunit.client.1.smithi123.stdout:1/443: fdatasync d3/d6/db/d10/d2d/f38 0 2024-02-16T22:21:57.136 INFO:tasks.workunit.client.0.smithi114.stdout:6/377: mkdir d6/d2c/d75 0 2024-02-16T22:21:57.140 INFO:tasks.workunit.client.0.smithi114.stdout:2/344: unlink dd/df/d1c/d21/l3d 0 2024-02-16T22:21:57.142 INFO:tasks.workunit.client.0.smithi114.stdout:2/345: fdatasync dd/df/f33 0 2024-02-16T22:21:57.144 INFO:tasks.workunit.client.0.smithi114.stdout:6/378: truncate d6/f35 7200523 0 2024-02-16T22:21:57.144 INFO:tasks.workunit.client.0.smithi114.stdout:4/471: dwrite dd/d1b/d24/f3d [4194304,4194304] 0 2024-02-16T22:21:57.145 INFO:tasks.workunit.client.1.smithi123.stdout:8/536: getdents d0 0 2024-02-16T22:21:57.147 INFO:tasks.workunit.client.0.smithi114.stdout:2/346: rmdir dd/df/d1c 39 2024-02-16T22:21:57.151 INFO:tasks.workunit.client.0.smithi114.stdout:6/379: rmdir d6 39 2024-02-16T22:21:57.153 INFO:tasks.workunit.client.0.smithi114.stdout:4/472: creat dd/d2e/d3f/fa6 x:0 0 0 2024-02-16T22:21:57.153 INFO:tasks.workunit.client.1.smithi123.stdout:7/487: dread d3/d6e/d1d/f2b [0,4194304] 0 2024-02-16T22:21:57.154 INFO:tasks.workunit.client.0.smithi114.stdout:4/473: truncate dd/d59/f80 246821 0 2024-02-16T22:21:57.154 INFO:tasks.workunit.client.1.smithi123.stdout:7/488: dread - d3/d5/d6c/f8d zero size 2024-02-16T22:21:57.155 INFO:tasks.workunit.client.0.smithi114.stdout:7/369: link d3/d22/l38 d3/d23/d4e/l79 0 2024-02-16T22:21:57.155 INFO:tasks.workunit.client.1.smithi123.stdout:1/444: dread d3/d6/f78 [0,4194304] 0 2024-02-16T22:21:57.156 INFO:tasks.workunit.client.1.smithi123.stdout:9/591: mkdir d4/d36/d8e/dbe 0 2024-02-16T22:21:57.159 INFO:tasks.workunit.client.1.smithi123.stdout:6/486: sync 2024-02-16T22:21:57.163 INFO:tasks.workunit.client.1.smithi123.stdout:8/537: mkdir d0/d29/da7/d43/d34/dae 0 2024-02-16T22:21:57.164 INFO:tasks.workunit.client.0.smithi114.stdout:4/474: symlink dd/d1b/d37/d38/la7 0 2024-02-16T22:21:57.166 INFO:tasks.workunit.client.0.smithi114.stdout:7/370: rename d3/d46/f51 to d3/d22/d61/d6a/f7a 0 2024-02-16T22:21:57.167 INFO:tasks.workunit.client.1.smithi123.stdout:5/456: symlink d6/dd/df/d1f/l99 0 2024-02-16T22:21:57.168 INFO:tasks.workunit.client.1.smithi123.stdout:5/457: dread - d6/dd/df/d58/d3f/d6f/f91 zero size 2024-02-16T22:21:57.169 INFO:tasks.workunit.client.0.smithi114.stdout:2/347: mknod dd/df/d1c/d21/d61/d63/c76 0 2024-02-16T22:21:57.171 INFO:tasks.workunit.client.1.smithi123.stdout:3/481: write d0/f12 [1645970,119206] 0 2024-02-16T22:21:57.172 INFO:tasks.workunit.client.1.smithi123.stdout:3/482: chown d0/d20/d76/d37/d41 79 1 2024-02-16T22:21:57.172 INFO:tasks.workunit.client.1.smithi123.stdout:7/489: truncate d3/d5/f28 1714494 0 2024-02-16T22:21:57.174 INFO:tasks.workunit.client.1.smithi123.stdout:1/445: symlink d3/d6/db/d10/d2a/d80/l97 0 2024-02-16T22:21:57.176 INFO:tasks.workunit.client.0.smithi114.stdout:6/380: symlink d6/dd/d25/d34/d58/l76 0 2024-02-16T22:21:57.177 INFO:tasks.workunit.client.0.smithi114.stdout:4/475: dwrite dd/d1b/d24/f6b [0,4194304] 0 2024-02-16T22:21:57.177 INFO:tasks.workunit.client.1.smithi123.stdout:2/478: dread d4/d1a/f32 [0,4194304] 0 2024-02-16T22:21:57.178 INFO:tasks.workunit.client.1.smithi123.stdout:2/479: dread d4/d1a/f86 [0,4194304] 0 2024-02-16T22:21:57.179 INFO:tasks.workunit.client.1.smithi123.stdout:2/480: chown d4/d1a/d1d/f8f 0 1 2024-02-16T22:21:57.181 INFO:tasks.workunit.client.0.smithi114.stdout:6/381: rmdir d6/d2c/d38/d54 39 2024-02-16T22:21:57.183 INFO:tasks.workunit.client.0.smithi114.stdout:4/476: symlink dd/d1b/d41/la8 0 2024-02-16T22:21:57.184 INFO:tasks.workunit.client.0.smithi114.stdout:4/477: write dd/d1b/d24/f36 [902624,64066] 0 2024-02-16T22:21:57.185 INFO:tasks.workunit.client.1.smithi123.stdout:3/483: creat d0/d50/fa9 x:0 0 0 2024-02-16T22:21:57.185 INFO:tasks.workunit.client.0.smithi114.stdout:7/371: link d3/d22/c35 d3/d22/d76/c7b 0 2024-02-16T22:21:57.187 INFO:tasks.workunit.client.1.smithi123.stdout:7/490: creat d3/d26/d35/f9f x:0 0 0 2024-02-16T22:21:57.192 INFO:tasks.workunit.client.0.smithi114.stdout:7/372: dwrite d3/d14/f43 [0,4194304] 0 2024-02-16T22:21:57.196 INFO:tasks.workunit.client.1.smithi123.stdout:8/538: dwrite d0/f6 [0,4194304] 0 2024-02-16T22:21:57.196 INFO:tasks.workunit.client.1.smithi123.stdout:4/521: getdents d5/d1c/d23/d56/d6e/d9f 0 2024-02-16T22:21:57.199 INFO:tasks.workunit.client.1.smithi123.stdout:2/481: read d4/d1a/d1d/d59/d45/f70 [83236,86843] 0 2024-02-16T22:21:57.200 INFO:tasks.workunit.client.0.smithi114.stdout:6/382: link d6/d2c/f6d d6/d2c/d38/d54/f77 0 2024-02-16T22:21:57.202 INFO:tasks.workunit.client.1.smithi123.stdout:3/484: rename d0/fb to d0/da0/faa 0 2024-02-16T22:21:57.203 INFO:tasks.workunit.client.1.smithi123.stdout:1/446: write d3/f8 [3898153,127915] 0 2024-02-16T22:21:57.204 INFO:tasks.workunit.client.0.smithi114.stdout:4/478: creat dd/d2e/fa9 x:0 0 0 2024-02-16T22:21:57.205 INFO:tasks.workunit.client.1.smithi123.stdout:7/491: mknod d3/d5/d6c/ca0 0 2024-02-16T22:21:57.206 INFO:tasks.workunit.client.0.smithi114.stdout:7/373: creat d3/d22/d73/f7c x:0 0 0 2024-02-16T22:21:57.208 INFO:tasks.workunit.client.1.smithi123.stdout:3/485: dwrite d0/f55 [0,4194304] 0 2024-02-16T22:21:57.208 INFO:tasks.workunit.client.0.smithi114.stdout:6/383: rename d6/da/f3f to d6/dd/d25/d34/d58/f78 0 2024-02-16T22:21:57.211 INFO:tasks.workunit.client.0.smithi114.stdout:4/479: rename dd/d2e/d3f/c8d to dd/d2e/d3f/caa 0 2024-02-16T22:21:57.212 INFO:tasks.workunit.client.1.smithi123.stdout:8/539: fdatasync d0/d29/da7/d26/d2a/f54 0 2024-02-16T22:21:57.213 INFO:tasks.workunit.client.0.smithi114.stdout:7/374: dread d3/d14/f4c [0,4194304] 0 2024-02-16T22:21:57.213 INFO:tasks.workunit.client.0.smithi114.stdout:4/480: write dd/d1b/f55 [440860,99107] 0 2024-02-16T22:21:57.216 INFO:tasks.workunit.client.1.smithi123.stdout:5/458: creat d6/dd/df/d58/f9a x:0 0 0 2024-02-16T22:21:57.219 INFO:tasks.workunit.client.0.smithi114.stdout:6/384: dwrite d6/d2c/d38/d41/f4a [4194304,4194304] 0 2024-02-16T22:21:57.220 INFO:tasks.workunit.client.1.smithi123.stdout:2/482: symlink d4/d1a/d1d/d38/d7b/l97 0 2024-02-16T22:21:57.223 INFO:tasks.workunit.client.0.smithi114.stdout:7/375: symlink d3/d22/d76/l7d 0 2024-02-16T22:21:57.224 INFO:tasks.workunit.client.1.smithi123.stdout:0/444: sync 2024-02-16T22:21:57.226 INFO:tasks.workunit.client.1.smithi123.stdout:7/492: symlink d3/d6e/d4d/la1 0 2024-02-16T22:21:57.226 INFO:tasks.workunit.client.0.smithi114.stdout:7/376: creat d3/d46/f7e x:0 0 0 2024-02-16T22:21:57.228 INFO:tasks.workunit.client.0.smithi114.stdout:7/377: write d3/d23/f3b [1056858,4234] 0 2024-02-16T22:21:57.228 INFO:tasks.workunit.client.1.smithi123.stdout:6/487: getdents d0/d9/dc/d5f 0 2024-02-16T22:21:57.229 INFO:tasks.workunit.client.0.smithi114.stdout:6/385: creat d6/d2c/d75/f79 x:0 0 0 2024-02-16T22:21:57.231 INFO:tasks.workunit.client.0.smithi114.stdout:4/481: rename dd/f21 to dd/d1b/d37/fab 0 2024-02-16T22:21:57.235 INFO:tasks.workunit.client.1.smithi123.stdout:8/540: creat d0/d29/da7/d26/d32/faf x:0 0 0 2024-02-16T22:21:57.236 INFO:tasks.workunit.client.0.smithi114.stdout:7/378: dwrite d3/d23/f62 [0,4194304] 0 2024-02-16T22:21:57.238 INFO:tasks.workunit.client.0.smithi114.stdout:7/379: chown d3/d14/c2d 210858 1 2024-02-16T22:21:57.241 INFO:tasks.workunit.client.0.smithi114.stdout:4/482: dwrite dd/d1b/d24/f3d [0,4194304] 0 2024-02-16T22:21:57.242 INFO:tasks.workunit.client.1.smithi123.stdout:0/445: creat d1/d4/d42/f95 x:0 0 0 2024-02-16T22:21:57.242 INFO:tasks.workunit.client.1.smithi123.stdout:0/446: chown d1/d8/d13/d1d/f8d 6679 1 2024-02-16T22:21:57.244 INFO:tasks.workunit.client.1.smithi123.stdout:1/447: truncate d3/d6/db/f28 816295 0 2024-02-16T22:21:57.245 INFO:tasks.workunit.client.1.smithi123.stdout:1/448: truncate d3/d47/f7d 506047 0 2024-02-16T22:21:57.250 INFO:tasks.workunit.client.0.smithi114.stdout:4/483: dwrite dd/d1b/f47 [0,4194304] 0 2024-02-16T22:21:57.252 INFO:tasks.workunit.client.0.smithi114.stdout:4/484: truncate dd/d1b/d37/d38/d7c/f9d 4259340 0 2024-02-16T22:21:57.252 INFO:tasks.workunit.client.0.smithi114.stdout:4/485: chown dd 0 1 2024-02-16T22:21:57.253 INFO:tasks.workunit.client.0.smithi114.stdout:4/486: truncate dd/d2e/fa9 228276 0 2024-02-16T22:21:57.254 INFO:tasks.workunit.client.0.smithi114.stdout:4/487: fsync dd/d1b/d24/d57/f85 0 2024-02-16T22:21:57.259 INFO:tasks.workunit.client.1.smithi123.stdout:1/449: dread d3/f5 [8388608,4194304] 0 2024-02-16T22:21:57.272 INFO:tasks.workunit.client.1.smithi123.stdout:8/541: dread d0/f75 [0,4194304] 0 2024-02-16T22:21:57.298 INFO:tasks.workunit.client.1.smithi123.stdout:9/592: sync 2024-02-16T22:21:57.299 INFO:tasks.workunit.client.0.smithi114.stdout:7/380: creat d3/d23/d4e/f7f x:0 0 0 2024-02-16T22:21:57.299 INFO:tasks.workunit.client.1.smithi123.stdout:9/593: readlink d4/d36/d8e/d84/l8b 0 2024-02-16T22:21:57.299 INFO:tasks.workunit.client.0.smithi114.stdout:7/381: truncate d3/ff 7136877 0 2024-02-16T22:21:57.299 INFO:tasks.workunit.client.0.smithi114.stdout:0/308: sync 2024-02-16T22:21:57.300 INFO:tasks.workunit.client.0.smithi114.stdout:1/304: sync 2024-02-16T22:21:57.300 INFO:tasks.workunit.client.1.smithi123.stdout:3/486: getdents d0/d20/d94 0 2024-02-16T22:21:57.302 INFO:tasks.workunit.client.0.smithi114.stdout:4/488: rmdir dd/d59 39 2024-02-16T22:21:57.303 INFO:tasks.workunit.client.1.smithi123.stdout:9/594: dwrite d4/d36/d8e/f95 [0,4194304] 0 2024-02-16T22:21:57.304 INFO:tasks.workunit.client.1.smithi123.stdout:4/522: rename d5/d1c/l3f to d5/d1c/d23/la3 0 2024-02-16T22:21:57.306 INFO:tasks.workunit.client.0.smithi114.stdout:6/386: rename d6/dd/l49 to d6/d2c/d38/l7a 0 2024-02-16T22:21:57.308 INFO:tasks.workunit.client.0.smithi114.stdout:1/305: rmdir d4/d33 39 2024-02-16T22:21:57.309 INFO:tasks.workunit.client.0.smithi114.stdout:1/306: stat d4/d9/f20 0 2024-02-16T22:21:57.310 INFO:tasks.workunit.client.1.smithi123.stdout:7/493: mknod d3/d26/ca2 0 2024-02-16T22:21:57.310 INFO:tasks.workunit.client.1.smithi123.stdout:7/494: readlink d3/d6e/d4d/d92/l9c 0 2024-02-16T22:21:57.313 INFO:tasks.workunit.client.0.smithi114.stdout:7/382: symlink d3/d74/l80 0 2024-02-16T22:21:57.313 INFO:tasks.workunit.client.0.smithi114.stdout:7/383: fsync d3/f27 0 2024-02-16T22:21:57.316 INFO:tasks.workunit.client.1.smithi123.stdout:6/488: symlink d0/d96/lb2 0 2024-02-16T22:21:57.316 INFO:tasks.workunit.client.1.smithi123.stdout:0/447: sync 2024-02-16T22:21:57.318 INFO:tasks.workunit.client.0.smithi114.stdout:4/489: read - dd/d59/fa4 zero size 2024-02-16T22:21:57.318 INFO:tasks.workunit.client.1.smithi123.stdout:3/487: mknod d0/d20/d2e/cab 0 2024-02-16T22:21:57.320 INFO:tasks.workunit.client.0.smithi114.stdout:7/384: dwrite d3/d46/f7e [0,4194304] 0 2024-02-16T22:21:57.321 INFO:tasks.workunit.client.0.smithi114.stdout:1/307: rename d4/d9/f20 to d4/d9/d46/d67/f68 0 2024-02-16T22:21:57.322 INFO:tasks.workunit.client.0.smithi114.stdout:7/385: stat d3/d3d/f4a 0 2024-02-16T22:21:57.322 INFO:tasks.workunit.client.0.smithi114.stdout:1/308: write d4/f21 [4091105,35792] 0 2024-02-16T22:21:57.326 INFO:tasks.workunit.client.0.smithi114.stdout:7/386: dread d3/d14/f29 [0,4194304] 0 2024-02-16T22:21:57.327 INFO:tasks.workunit.client.0.smithi114.stdout:6/387: dread d6/d2c/f46 [0,4194304] 0 2024-02-16T22:21:57.328 INFO:tasks.workunit.client.1.smithi123.stdout:9/595: unlink d4/d36/d57/db2/l66 0 2024-02-16T22:21:57.328 INFO:tasks.workunit.client.0.smithi114.stdout:7/387: write d3/d23/f5e [577461,130098] 0 2024-02-16T22:21:57.332 INFO:tasks.workunit.client.1.smithi123.stdout:9/596: dread d4/d36/d8e/f95 [0,4194304] 0 2024-02-16T22:21:57.332 INFO:tasks.workunit.client.1.smithi123.stdout:5/459: rename d6/dd/l4b to d6/d3b/l9b 0 2024-02-16T22:21:57.333 INFO:tasks.workunit.client.1.smithi123.stdout:5/460: fsync d6/d3b/f65 0 2024-02-16T22:21:57.334 INFO:tasks.workunit.client.1.smithi123.stdout:7/495: dread d3/d6e/d24/f31 [0,4194304] 0 2024-02-16T22:21:57.335 INFO:tasks.workunit.client.1.smithi123.stdout:7/496: chown d3/c20 2 1 2024-02-16T22:21:57.336 INFO:tasks.workunit.client.1.smithi123.stdout:7/497: write d3/d6e/d24/f90 [283185,75807] 0 2024-02-16T22:21:57.336 INFO:tasks.workunit.client.0.smithi114.stdout:4/490: symlink dd/d1b/d37/d99/lac 0 2024-02-16T22:21:57.339 INFO:tasks.workunit.client.1.smithi123.stdout:7/498: dread d3/d6e/d24/f90 [0,4194304] 0 2024-02-16T22:21:57.342 INFO:tasks.workunit.client.0.smithi114.stdout:0/309: creat d1/d16/f5d x:0 0 0 2024-02-16T22:21:57.346 INFO:tasks.workunit.client.0.smithi114.stdout:6/388: readlink d6/dd/d25/d34/l47 0 2024-02-16T22:21:57.348 INFO:tasks.workunit.client.1.smithi123.stdout:3/488: dwrite d0/fc [0,4194304] 0 2024-02-16T22:21:57.349 INFO:tasks.workunit.client.0.smithi114.stdout:7/388: symlink d3/d22/d61/d6a/l81 0 2024-02-16T22:21:57.350 INFO:tasks.workunit.client.0.smithi114.stdout:6/389: dread d6/d2c/d38/d54/f3d [0,4194304] 0 2024-02-16T22:21:57.354 INFO:tasks.workunit.client.1.smithi123.stdout:2/483: rename d4/d1a/d1d/d38/d63 to d4/d1a/d1d/d13/d25/d98 0 2024-02-16T22:21:57.356 INFO:tasks.workunit.client.1.smithi123.stdout:0/448: write d1/d4/d17/d74/f58 [4041706,21590] 0 2024-02-16T22:21:57.358 INFO:tasks.workunit.client.1.smithi123.stdout:0/449: read - d1/d4/d17/d74/d57/f72 zero size 2024-02-16T22:21:57.359 INFO:tasks.workunit.client.0.smithi114.stdout:0/310: rmdir d1/d1a/d3e 39 2024-02-16T22:21:57.359 INFO:tasks.workunit.client.1.smithi123.stdout:6/489: dwrite d0/d9/dc/d39/f5b [0,4194304] 0 2024-02-16T22:21:57.360 INFO:tasks.workunit.client.1.smithi123.stdout:9/597: truncate d4/d36/daa/d2b/f5b 1744781 0 2024-02-16T22:21:57.363 INFO:tasks.workunit.client.0.smithi114.stdout:1/309: dwrite d4/d9/d3d/f41 [0,4194304] 0 2024-02-16T22:21:57.363 INFO:tasks.workunit.client.1.smithi123.stdout:5/461: creat d6/dd/df/d58/d89/f9c x:0 0 0 2024-02-16T22:21:57.371 INFO:tasks.workunit.client.0.smithi114.stdout:0/311: creat d1/d30/f5e x:0 0 0 2024-02-16T22:21:57.373 INFO:tasks.workunit.client.0.smithi114.stdout:0/312: truncate d1/d16/d1f/d28/f56 393382 0 2024-02-16T22:21:57.373 INFO:tasks.workunit.client.1.smithi123.stdout:8/542: rename d0/d29/da7/d26/f79 to d0/d29/da7/d26/d68/d81/d47/d9b/fb0 0 2024-02-16T22:21:57.374 INFO:tasks.workunit.client.0.smithi114.stdout:7/389: rename d3/d22/f2b to d3/d14/f82 0 2024-02-16T22:21:57.375 INFO:tasks.workunit.client.1.smithi123.stdout:8/543: truncate d0/d29/da7/d26/d68/d81/d47/f9d 743420 0 2024-02-16T22:21:57.376 INFO:tasks.workunit.client.1.smithi123.stdout:4/523: link d5/d1c/f3d d5/d1c/d23/d56/d6e/d9f/da1/fa4 0 2024-02-16T22:21:57.377 INFO:tasks.workunit.client.1.smithi123.stdout:4/524: dread - d5/d1c/d42/f75 zero size 2024-02-16T22:21:57.378 INFO:tasks.workunit.client.0.smithi114.stdout:7/390: rmdir d3/d22/d61 39 2024-02-16T22:21:57.379 INFO:tasks.workunit.client.1.smithi123.stdout:0/450: mkdir d1/d4/d17/d54/d8e/d96 0 2024-02-16T22:21:57.379 INFO:tasks.workunit.client.0.smithi114.stdout:7/391: chown d3/f27 0 1 2024-02-16T22:21:57.380 INFO:tasks.workunit.client.1.smithi123.stdout:6/490: symlink d0/d9/dc/d3a/d52/lb3 0 2024-02-16T22:21:57.380 INFO:tasks.workunit.client.0.smithi114.stdout:7/392: readlink d3/d14/l16 0 2024-02-16T22:21:57.380 INFO:tasks.workunit.client.0.smithi114.stdout:0/313: dwrite d1/d30/f5e [0,4194304] 0 2024-02-16T22:21:57.383 INFO:tasks.workunit.client.1.smithi123.stdout:1/450: getdents d3/d6/db/d10/d2a/d80 0 2024-02-16T22:21:57.386 INFO:tasks.workunit.client.0.smithi114.stdout:7/393: fsync d3/d14/f4c 0 2024-02-16T22:21:57.389 INFO:tasks.workunit.client.1.smithi123.stdout:1/451: dwrite d3/d6/f26 [0,4194304] 0 2024-02-16T22:21:57.391 INFO:tasks.workunit.client.1.smithi123.stdout:1/452: stat d3/d6/db/d10/d2a/d53/l71 0 2024-02-16T22:21:57.392 INFO:tasks.workunit.client.0.smithi114.stdout:7/394: dwrite d3/f6 [4194304,4194304] 0 2024-02-16T22:21:57.394 INFO:tasks.workunit.client.0.smithi114.stdout:7/395: write d3/d3d/f41 [1907945,35176] 0 2024-02-16T22:21:57.399 INFO:tasks.workunit.client.0.smithi114.stdout:7/396: dread d3/f27 [0,4194304] 0 2024-02-16T22:21:57.400 INFO:tasks.workunit.client.1.smithi123.stdout:4/525: dread d5/d1c/d23/d47/f62 [0,4194304] 0 2024-02-16T22:21:57.401 INFO:tasks.workunit.client.1.smithi123.stdout:8/544: creat d0/d29/da7/d26/d2a/d3c/fb1 x:0 0 0 2024-02-16T22:21:57.409 INFO:tasks.workunit.client.1.smithi123.stdout:6/491: creat d0/d9/dc/d39/fb4 x:0 0 0 2024-02-16T22:21:57.410 INFO:tasks.workunit.client.1.smithi123.stdout:6/492: write d0/d9/f19 [737153,7735] 0 2024-02-16T22:21:57.411 INFO:tasks.workunit.client.1.smithi123.stdout:9/598: creat d4/d36/d8e/d98/fbf x:0 0 0 2024-02-16T22:21:57.418 INFO:tasks.workunit.client.1.smithi123.stdout:1/453: mknod d3/d47/c98 0 2024-02-16T22:21:57.419 INFO:tasks.workunit.client.0.smithi114.stdout:7/397: link d3/d14/l16 d3/d22/d61/d6a/l83 0 2024-02-16T22:21:57.420 INFO:tasks.workunit.client.1.smithi123.stdout:2/484: truncate d4/d1a/d1d/d13/d25/f43 68623 0 2024-02-16T22:21:57.420 INFO:tasks.workunit.client.0.smithi114.stdout:0/314: dwrite d1/d1a/f26 [0,4194304] 0 2024-02-16T22:21:57.421 INFO:tasks.workunit.client.0.smithi114.stdout:0/315: write d1/f2a [3271095,34032] 0 2024-02-16T22:21:57.422 INFO:tasks.workunit.client.1.smithi123.stdout:4/526: unlink d5/d1c/f21 0 2024-02-16T22:21:57.425 INFO:tasks.workunit.client.0.smithi114.stdout:7/398: creat d3/d22/d61/f84 x:0 0 0 2024-02-16T22:21:57.426 INFO:tasks.workunit.client.0.smithi114.stdout:0/316: readlink d1/l43 0 2024-02-16T22:21:57.426 INFO:tasks.workunit.client.0.smithi114.stdout:1/310: write d4/f36 [389186,45404] 0 2024-02-16T22:21:57.426 INFO:tasks.workunit.client.1.smithi123.stdout:8/545: dread d0/d29/da7/d26/d68/d81/fa1 [0,4194304] 0 2024-02-16T22:21:57.427 INFO:tasks.workunit.client.0.smithi114.stdout:1/311: stat d4/d9/f22 0 2024-02-16T22:21:57.428 INFO:tasks.workunit.client.1.smithi123.stdout:4/527: dwrite d5/d49/f89 [0,4194304] 0 2024-02-16T22:21:57.428 INFO:tasks.workunit.client.1.smithi123.stdout:0/451: unlink d1/l38 0 2024-02-16T22:21:57.430 INFO:tasks.workunit.client.0.smithi114.stdout:7/399: rmdir d3/d46 39 2024-02-16T22:21:57.431 INFO:tasks.workunit.client.1.smithi123.stdout:0/452: write d1/d8/d3d/f59 [669891,111013] 0 2024-02-16T22:21:57.431 INFO:tasks.workunit.client.1.smithi123.stdout:4/528: write d5/d1c/f97 [485012,125528] 0 2024-02-16T22:21:57.431 INFO:tasks.workunit.client.0.smithi114.stdout:7/400: write d3/d23/f5e [1543649,112353] 0 2024-02-16T22:21:57.432 INFO:tasks.workunit.client.0.smithi114.stdout:7/401: readlink d3/d22/d76/l7d 0 2024-02-16T22:21:57.434 INFO:tasks.workunit.client.0.smithi114.stdout:0/317: read d1/d1a/d3e/f4a [1004214,33066] 0 2024-02-16T22:21:57.434 INFO:tasks.workunit.client.0.smithi114.stdout:7/402: write d3/d3d/f4b [1998742,28386] 0 2024-02-16T22:21:57.436 INFO:tasks.workunit.client.1.smithi123.stdout:6/493: creat d0/d9/fb5 x:0 0 0 2024-02-16T22:21:57.444 INFO:tasks.workunit.client.1.smithi123.stdout:7/499: getdents d3/d6e/d52 0 2024-02-16T22:21:57.445 INFO:tasks.workunit.client.1.smithi123.stdout:3/489: getdents d0/d20/d2e 0 2024-02-16T22:21:57.447 INFO:tasks.workunit.client.0.smithi114.stdout:1/312: mkdir d4/d63/d69 0 2024-02-16T22:21:57.450 INFO:tasks.workunit.client.1.smithi123.stdout:5/462: truncate d6/dd/df/d58/f43 1208516 0 2024-02-16T22:21:57.453 INFO:tasks.workunit.client.1.smithi123.stdout:2/485: rmdir d4/d1a/d1d/d13/d25/d72 39 2024-02-16T22:21:57.454 INFO:tasks.workunit.client.1.smithi123.stdout:2/486: write d4/d1a/d1d/f8f [628671,70165] 0 2024-02-16T22:21:57.455 INFO:tasks.workunit.client.0.smithi114.stdout:7/403: dread d3/d3d/f41 [0,4194304] 0 2024-02-16T22:21:57.457 INFO:tasks.workunit.client.0.smithi114.stdout:0/318: write d1/f15 [601598,93780] 0 2024-02-16T22:21:57.458 INFO:tasks.workunit.client.0.smithi114.stdout:1/313: mknod d4/d63/d69/c6a 0 2024-02-16T22:21:57.458 INFO:tasks.workunit.client.0.smithi114.stdout:0/319: chown d1/d16/f5d 55 1 2024-02-16T22:21:57.458 INFO:tasks.workunit.client.0.smithi114.stdout:1/314: chown d4/d9/d3d 513070507 1 2024-02-16T22:21:57.459 INFO:tasks.workunit.client.0.smithi114.stdout:1/315: chown d4/d9/d46/d4d/f4c 1 1 2024-02-16T22:21:57.463 INFO:tasks.workunit.client.1.smithi123.stdout:4/529: creat d5/d1c/d23/d56/d6e/fa5 x:0 0 0 2024-02-16T22:21:57.466 INFO:tasks.workunit.client.1.smithi123.stdout:6/494: truncate d0/d6/d91/d93/f95 105238 0 2024-02-16T22:21:57.468 INFO:tasks.workunit.client.1.smithi123.stdout:7/500: unlink d3/d5/ca 0 2024-02-16T22:21:57.469 INFO:tasks.workunit.client.0.smithi114.stdout:0/320: mknod d1/d1a/c5f 0 2024-02-16T22:21:57.469 INFO:tasks.workunit.client.0.smithi114.stdout:7/404: write d3/d14/f29 [1327919,6299] 0 2024-02-16T22:21:57.474 INFO:tasks.workunit.client.1.smithi123.stdout:1/454: mknod d3/d6/d11/d8b/c99 0 2024-02-16T22:21:57.477 INFO:tasks.workunit.client.1.smithi123.stdout:9/599: sync 2024-02-16T22:21:57.477 INFO:tasks.workunit.client.1.smithi123.stdout:5/463: rename d6/dd/df/d1f/d25/f39 to d6/dd/df/d90/f9d 0 2024-02-16T22:21:57.481 INFO:tasks.workunit.client.0.smithi114.stdout:0/321: truncate d1/fa 3247986 0 2024-02-16T22:21:57.481 INFO:tasks.workunit.client.1.smithi123.stdout:9/600: dwrite d4/d36/fad [0,4194304] 0 2024-02-16T22:21:57.482 INFO:tasks.workunit.client.0.smithi114.stdout:1/316: getdents d4/d9/d3d 0 2024-02-16T22:21:57.486 INFO:tasks.workunit.client.0.smithi114.stdout:0/322: rename d1/d30 to d1/d16/d1f/d28/d51/d60 0 2024-02-16T22:21:57.491 INFO:tasks.workunit.client.0.smithi114.stdout:7/405: link d3/d74/l80 d3/d3d/l85 0 2024-02-16T22:21:57.494 INFO:tasks.workunit.client.1.smithi123.stdout:7/501: symlink d3/d59/la3 0 2024-02-16T22:21:57.495 INFO:tasks.workunit.client.1.smithi123.stdout:7/502: fsync d3/d6e/d34/f57 0 2024-02-16T22:21:57.495 INFO:tasks.workunit.client.1.smithi123.stdout:7/503: dread - d3/d5/d6c/f8d zero size 2024-02-16T22:21:57.496 INFO:tasks.workunit.client.1.smithi123.stdout:7/504: readlink d3/d6e/d24/l30 0 2024-02-16T22:21:57.496 INFO:tasks.workunit.client.1.smithi123.stdout:6/495: dread d0/d9/dc/d3a/d52/fa7 [0,4194304] 0 2024-02-16T22:21:57.497 INFO:tasks.workunit.client.1.smithi123.stdout:3/490: mkdir d0/d20/d94/dac 0 2024-02-16T22:21:57.497 INFO:tasks.workunit.client.1.smithi123.stdout:3/491: chown d0/d20/f42 19364347 1 2024-02-16T22:21:57.498 INFO:tasks.workunit.client.1.smithi123.stdout:3/492: chown d0/fd 200 1 2024-02-16T22:21:57.498 INFO:tasks.workunit.client.0.smithi114.stdout:7/406: creat d3/d3d/f86 x:0 0 0 2024-02-16T22:21:57.499 INFO:tasks.workunit.client.1.smithi123.stdout:3/493: chown d0/d20/d5a/f9d 3597 1 2024-02-16T22:21:57.500 INFO:tasks.workunit.client.1.smithi123.stdout:1/455: creat d3/d52/f9a x:0 0 0 2024-02-16T22:21:57.500 INFO:tasks.workunit.client.0.smithi114.stdout:0/323: write d1/d1a/f38 [754714,98042] 0 2024-02-16T22:21:57.501 INFO:tasks.workunit.client.0.smithi114.stdout:1/317: dwrite d4/f25 [0,4194304] 0 2024-02-16T22:21:57.502 INFO:tasks.workunit.client.0.smithi114.stdout:0/324: fdatasync d1/d16/d1f/d28/f56 0 2024-02-16T22:21:57.503 INFO:tasks.workunit.client.1.smithi123.stdout:2/487: symlink d4/d1a/l99 0 2024-02-16T22:21:57.504 INFO:tasks.workunit.client.1.smithi123.stdout:9/601: mknod d4/d36/daa/d2b/d6c/cc0 0 2024-02-16T22:21:57.505 INFO:tasks.workunit.client.0.smithi114.stdout:1/318: mkdir d4/d63/d6b 0 2024-02-16T22:21:57.507 INFO:tasks.workunit.client.1.smithi123.stdout:8/546: creat d0/d29/da7/d26/d2a/fb2 x:0 0 0 2024-02-16T22:21:57.508 INFO:tasks.workunit.client.1.smithi123.stdout:8/547: chown d0/d29/da7/d26/f7a 39 1 2024-02-16T22:21:57.509 INFO:tasks.workunit.client.0.smithi114.stdout:7/407: mkdir d3/d87 0 2024-02-16T22:21:57.510 INFO:tasks.workunit.client.1.smithi123.stdout:0/453: creat d1/d8/d13/d1d/f97 x:0 0 0 2024-02-16T22:21:57.511 INFO:tasks.workunit.client.0.smithi114.stdout:0/325: mkdir d1/d16/d1f/d28/d51/d61 0 2024-02-16T22:21:57.513 INFO:tasks.workunit.client.1.smithi123.stdout:7/505: mknod d3/d26/d35/ca4 0 2024-02-16T22:21:57.513 INFO:tasks.workunit.client.0.smithi114.stdout:1/319: dread - d4/f37 zero size 2024-02-16T22:21:57.514 INFO:tasks.workunit.client.1.smithi123.stdout:3/494: creat d0/d20/d76/d5d/d60/fad x:0 0 0 2024-02-16T22:21:57.515 INFO:tasks.workunit.client.0.smithi114.stdout:0/326: mkdir d1/d1a/d47/d62 0 2024-02-16T22:21:57.516 INFO:tasks.workunit.client.1.smithi123.stdout:1/456: symlink d3/d6/d49/d48/d84/d92/d91/l9b 0 2024-02-16T22:21:57.517 INFO:tasks.workunit.client.0.smithi114.stdout:7/408: dwrite d3/d23/d4e/f7f [0,4194304] 0 2024-02-16T22:21:57.519 INFO:tasks.workunit.client.0.smithi114.stdout:1/320: mknod d4/d9/d46/d47/c6c 0 2024-02-16T22:21:57.520 INFO:tasks.workunit.client.0.smithi114.stdout:0/327: chown d1/c9 0 1 2024-02-16T22:21:57.520 INFO:tasks.workunit.client.1.smithi123.stdout:9/602: mkdir d4/d36/d8e/d58/d59/d60/dc1 0 2024-02-16T22:21:57.521 INFO:tasks.workunit.client.0.smithi114.stdout:7/409: readlink d3/d22/l30 0 2024-02-16T22:21:57.524 INFO:tasks.workunit.client.0.smithi114.stdout:1/321: unlink d4/f1d 0 2024-02-16T22:21:57.526 INFO:tasks.workunit.client.0.smithi114.stdout:0/328: write d1/fe [8754705,104172] 0 2024-02-16T22:21:57.529 INFO:tasks.workunit.client.0.smithi114.stdout:7/410: rename d3/l7 to d3/d22/d76/l88 0 2024-02-16T22:21:57.530 INFO:tasks.workunit.client.1.smithi123.stdout:6/496: link d0/d74/f90 d0/d6/d8e/fb6 0 2024-02-16T22:21:57.531 INFO:tasks.workunit.client.1.smithi123.stdout:6/497: fsync d0/d9/dc/d3a/fa1 0 2024-02-16T22:21:57.532 INFO:tasks.workunit.client.1.smithi123.stdout:7/506: mknod d3/d6e/d24/ca5 0 2024-02-16T22:21:57.535 INFO:tasks.workunit.client.1.smithi123.stdout:1/457: readlink d3/d6/db/l54 0 2024-02-16T22:21:57.536 INFO:tasks.workunit.client.0.smithi114.stdout:0/329: creat d1/d16/d1f/f63 x:0 0 0 2024-02-16T22:21:57.536 INFO:tasks.workunit.client.1.smithi123.stdout:1/458: fsync d3/d6/db/d10/d2d/f25 0 2024-02-16T22:21:57.536 INFO:tasks.workunit.client.1.smithi123.stdout:0/454: dread d1/d4/f51 [0,4194304] 0 2024-02-16T22:21:57.537 INFO:tasks.workunit.client.1.smithi123.stdout:1/459: write d3/f7c [480866,115334] 0 2024-02-16T22:21:57.537 INFO:tasks.workunit.client.0.smithi114.stdout:1/322: mknod d4/d9/d65/c6d 0 2024-02-16T22:21:57.538 INFO:tasks.workunit.client.1.smithi123.stdout:0/455: truncate d1/d8/d13/d1d/f8d 368524 0 2024-02-16T22:21:57.539 INFO:tasks.workunit.client.1.smithi123.stdout:4/530: getdents d5 0 2024-02-16T22:21:57.540 INFO:tasks.workunit.client.1.smithi123.stdout:9/603: creat d4/d36/d8e/d58/d6e/fc2 x:0 0 0 2024-02-16T22:21:57.541 INFO:tasks.workunit.client.1.smithi123.stdout:9/604: read - d4/d36/d8e/d58/d6e/f9b zero size 2024-02-16T22:21:57.542 INFO:tasks.workunit.client.1.smithi123.stdout:7/507: mkdir d3/d5/d6c/da6 0 2024-02-16T22:21:57.543 INFO:tasks.workunit.client.1.smithi123.stdout:3/495: mkdir d0/d20/d76/d5d/dae 0 2024-02-16T22:21:57.544 INFO:tasks.workunit.client.1.smithi123.stdout:5/464: getdents d6/dd/df/d76/d7f 0 2024-02-16T22:21:57.545 INFO:tasks.workunit.client.1.smithi123.stdout:1/460: rmdir d3/d6/db 39 2024-02-16T22:21:57.546 INFO:tasks.workunit.client.0.smithi114.stdout:3/312: sync 2024-02-16T22:21:57.547 INFO:tasks.workunit.client.1.smithi123.stdout:7/508: dwrite d3/d6e/f7c [0,4194304] 0 2024-02-16T22:21:57.548 INFO:tasks.workunit.client.1.smithi123.stdout:2/488: rename d4/d1a/d1d/d59/d45/f70 to d4/d1a/d1d/d13/d25/f9a 0 2024-02-16T22:21:57.548 INFO:tasks.workunit.client.1.smithi123.stdout:0/456: symlink d1/d4/d17/l98 0 2024-02-16T22:21:57.549 INFO:tasks.workunit.client.1.smithi123.stdout:0/457: chown d1/d4/d17/d74/d57 783722 1 2024-02-16T22:21:57.552 INFO:tasks.workunit.client.1.smithi123.stdout:8/548: creat d0/d29/da7/fb3 x:0 0 0 2024-02-16T22:21:57.552 INFO:tasks.workunit.client.0.smithi114.stdout:1/323: dwrite d4/f2d [4194304,4194304] 0 2024-02-16T22:21:57.554 INFO:tasks.workunit.client.0.smithi114.stdout:0/330: dwrite d1/d16/d18/f1c [4194304,4194304] 0 2024-02-16T22:21:57.555 INFO:tasks.workunit.client.1.smithi123.stdout:9/605: fsync d4/d36/d8e/d58/f5a 0 2024-02-16T22:21:57.556 INFO:tasks.workunit.client.1.smithi123.stdout:0/458: dwrite d1/d8/f6b [0,4194304] 0 2024-02-16T22:21:57.560 INFO:tasks.workunit.client.0.smithi114.stdout:1/324: dread d4/f36 [0,4194304] 0 2024-02-16T22:21:57.562 INFO:tasks.workunit.client.0.smithi114.stdout:1/325: dread - d4/d9/d46/d4d/f5d zero size 2024-02-16T22:21:57.564 INFO:tasks.workunit.client.0.smithi114.stdout:3/313: dread d4/d46/d48/f5c [0,4194304] 0 2024-02-16T22:21:57.617 INFO:tasks.workunit.client.0.smithi114.stdout:8/356: sync 2024-02-16T22:21:57.617 INFO:tasks.workunit.client.0.smithi114.stdout:5/305: sync 2024-02-16T22:21:57.617 INFO:tasks.workunit.client.0.smithi114.stdout:9/387: sync 2024-02-16T22:21:57.617 INFO:tasks.workunit.client.0.smithi114.stdout:2/348: sync 2024-02-16T22:21:57.618 INFO:tasks.workunit.client.0.smithi114.stdout:8/357: fdatasync d4/d24/d2c/f32 0 2024-02-16T22:21:57.618 INFO:tasks.workunit.client.0.smithi114.stdout:5/306: truncate d0/d2d/f51 562516 0 2024-02-16T22:21:57.618 INFO:tasks.workunit.client.0.smithi114.stdout:8/358: chown d4/d16/d25/d39 827720 1 2024-02-16T22:21:57.619 INFO:tasks.workunit.client.0.smithi114.stdout:8/359: chown d4/d24/f50 1 1 2024-02-16T22:21:57.626 INFO:tasks.workunit.client.0.smithi114.stdout:8/360: dread d4/d16/d25/f6e [0,4194304] 0 2024-02-16T22:21:57.644 INFO:tasks.workunit.client.1.smithi123.stdout:5/465: rmdir d6/dd/df/d76 39 2024-02-16T22:21:57.645 INFO:tasks.workunit.client.1.smithi123.stdout:5/466: dread - d6/dd/df/d58/d3f/d6f/f84 zero size 2024-02-16T22:21:57.645 INFO:tasks.workunit.client.0.smithi114.stdout:0/331: symlink d1/d1a/d47/l64 0 2024-02-16T22:21:57.648 INFO:tasks.workunit.client.1.smithi123.stdout:1/461: mknod d3/d6/d11/c9c 0 2024-02-16T22:21:57.651 INFO:tasks.workunit.client.1.smithi123.stdout:7/509: unlink d3/d6e/d34/l3f 0 2024-02-16T22:21:57.652 INFO:tasks.workunit.client.1.smithi123.stdout:2/489: truncate d4/d1a/d1d/d13/d25/f77 9000422 0 2024-02-16T22:21:57.653 INFO:tasks.workunit.client.1.smithi123.stdout:6/498: truncate d0/f54 2707382 0 2024-02-16T22:21:57.654 INFO:tasks.workunit.client.1.smithi123.stdout:4/531: fdatasync d5/f17 0 2024-02-16T22:21:57.655 INFO:tasks.workunit.client.1.smithi123.stdout:3/496: write d0/d20/d29/f3d [493144,22326] 0 2024-02-16T22:21:57.659 INFO:tasks.workunit.client.0.smithi114.stdout:9/388: rename d2/d1a/d23/d32/c70 to d2/d1a/d23/d32/d46/d78/c82 0 2024-02-16T22:21:57.659 INFO:tasks.workunit.client.0.smithi114.stdout:9/389: readlink d2/l30 0 2024-02-16T22:21:57.660 INFO:tasks.workunit.client.0.smithi114.stdout:5/307: rmdir d0/df/d16/d25/d3e 39 2024-02-16T22:21:57.667 INFO:tasks.workunit.client.0.smithi114.stdout:8/361: creat d4/d11/d52/f83 x:0 0 0 2024-02-16T22:21:57.668 INFO:tasks.workunit.client.0.smithi114.stdout:8/362: write d4/d24/d2c/f32 [1702709,26338] 0 2024-02-16T22:21:57.670 INFO:tasks.workunit.client.0.smithi114.stdout:8/363: write d4/f1d [2649478,63528] 0 2024-02-16T22:21:57.673 INFO:tasks.workunit.client.0.smithi114.stdout:0/332: dread d1/fe [0,4194304] 0 2024-02-16T22:21:57.676 INFO:tasks.workunit.client.0.smithi114.stdout:3/314: symlink d4/d24/d2d/d45/d68/l6c 0 2024-02-16T22:21:57.681 INFO:tasks.workunit.client.0.smithi114.stdout:5/308: dread d0/f8 [0,4194304] 0 2024-02-16T22:21:57.689 INFO:tasks.workunit.client.0.smithi114.stdout:0/333: truncate d1/d1a/f3b 1339636 0 2024-02-16T22:21:57.689 INFO:tasks.workunit.client.0.smithi114.stdout:0/334: readlink d1/d1a/d47/l64 0 2024-02-16T22:21:57.692 INFO:tasks.workunit.client.0.smithi114.stdout:2/349: creat dd/d3c/f77 x:0 0 0 2024-02-16T22:21:57.701 INFO:tasks.workunit.client.0.smithi114.stdout:2/350: mknod dd/df/d1c/d21/c78 0 2024-02-16T22:21:57.702 INFO:tasks.workunit.client.0.smithi114.stdout:1/326: truncate d4/d9/d46/d67/f68 1168595 0 2024-02-16T22:21:57.702 INFO:tasks.workunit.client.0.smithi114.stdout:1/327: chown d4/fa 3605268 1 2024-02-16T22:21:57.703 INFO:tasks.workunit.client.0.smithi114.stdout:9/390: link d2/d33/d3f/c7a d2/d1a/d65/c83 0 2024-02-16T22:21:57.703 INFO:tasks.workunit.client.0.smithi114.stdout:1/328: dread - d4/d9/d46/d4d/f5c zero size 2024-02-16T22:21:57.704 INFO:tasks.workunit.client.0.smithi114.stdout:9/391: chown d2/d33/d3f/c60 1965892584 1 2024-02-16T22:21:57.704 INFO:tasks.workunit.client.0.smithi114.stdout:9/392: fsync d2/f12 0 2024-02-16T22:21:57.705 INFO:tasks.workunit.client.0.smithi114.stdout:0/335: truncate d1/fa 88503 0 2024-02-16T22:21:57.707 INFO:tasks.workunit.client.0.smithi114.stdout:3/315: creat d4/d24/f6d x:0 0 0 2024-02-16T22:21:57.707 INFO:tasks.workunit.client.0.smithi114.stdout:9/393: read d2/dd/f34 [2894930,40745] 0 2024-02-16T22:21:57.708 INFO:tasks.workunit.client.0.smithi114.stdout:5/309: dread d0/d10/f23 [4194304,4194304] 0 2024-02-16T22:21:57.709 INFO:tasks.workunit.client.0.smithi114.stdout:3/316: write d4/d24/f67 [1685807,31618] 0 2024-02-16T22:21:57.712 INFO:tasks.workunit.client.1.smithi123.stdout:6/499: creat d0/d6/d1a/d88/fb7 x:0 0 0 2024-02-16T22:21:57.713 INFO:tasks.workunit.client.0.smithi114.stdout:0/336: mknod d1/d16/d1f/d28/c65 0 2024-02-16T22:21:57.716 INFO:tasks.workunit.client.0.smithi114.stdout:2/351: creat dd/d3c/d4c/f79 x:0 0 0 2024-02-16T22:21:57.716 INFO:tasks.workunit.client.1.smithi123.stdout:3/497: stat d0/d20/d2e/l45 0 2024-02-16T22:21:57.717 INFO:tasks.workunit.client.0.smithi114.stdout:3/317: creat d4/d43/f6e x:0 0 0 2024-02-16T22:21:57.718 INFO:tasks.workunit.client.0.smithi114.stdout:8/364: dwrite d4/d11/f71 [4194304,4194304] 0 2024-02-16T22:21:57.719 INFO:tasks.workunit.client.0.smithi114.stdout:3/318: chown d4/d24/l4c 1 1 2024-02-16T22:21:57.719 INFO:tasks.workunit.client.1.smithi123.stdout:6/500: dread d0/d6/d91/d97/fa6 [0,4194304] 0 2024-02-16T22:21:57.719 INFO:tasks.workunit.client.1.smithi123.stdout:6/501: readlink d0/d9/l87 0 2024-02-16T22:21:57.720 INFO:tasks.workunit.client.0.smithi114.stdout:9/394: rename d2/d1a/d23/d35/c20 to d2/d5c/d5f/d72/c84 0 2024-02-16T22:21:57.721 INFO:tasks.workunit.client.1.smithi123.stdout:9/606: rename d4/d36/daa/d20/f3c to d4/d1e/d3a/fc3 0 2024-02-16T22:21:57.722 INFO:tasks.workunit.client.0.smithi114.stdout:9/395: truncate d2/d1a/d23/d35/d47/d63/f81 5007147 0 2024-02-16T22:21:57.722 INFO:tasks.workunit.client.0.smithi114.stdout:1/329: link d4/f2d d4/d9/d46/d67/f6e 0 2024-02-16T22:21:57.724 INFO:tasks.workunit.client.0.smithi114.stdout:0/337: mkdir d1/d16/d1f/d28/d66 0 2024-02-16T22:21:57.730 INFO:tasks.workunit.client.1.smithi123.stdout:4/532: creat d5/d1c/d23/d76/fa6 x:0 0 0 2024-02-16T22:21:57.731 INFO:tasks.workunit.client.1.smithi123.stdout:4/533: chown d5/d1c/d23/f9c 158 1 2024-02-16T22:21:57.731 INFO:tasks.workunit.client.1.smithi123.stdout:3/498: dread d0/d20/f8b [0,4194304] 0 2024-02-16T22:21:57.732 INFO:tasks.workunit.client.0.smithi114.stdout:0/338: dwrite d1/d16/d1f/d28/d51/d60/f5e [0,4194304] 0 2024-02-16T22:21:57.735 INFO:tasks.workunit.client.0.smithi114.stdout:1/330: symlink d4/d63/d69/l6f 0 2024-02-16T22:21:57.736 INFO:tasks.workunit.client.0.smithi114.stdout:1/331: fdatasync d4/d9/d3d/d3f/f61 0 2024-02-16T22:21:57.740 INFO:tasks.workunit.client.1.smithi123.stdout:6/502: creat d0/d6/d91/d97/fb8 x:0 0 0 2024-02-16T22:21:57.742 INFO:tasks.workunit.client.1.smithi123.stdout:0/459: rename d1/d4/f56 to d1/d4/d17/d54/f99 0 2024-02-16T22:21:57.744 INFO:tasks.workunit.client.1.smithi123.stdout:9/607: mknod d4/cc4 0 2024-02-16T22:21:57.748 INFO:tasks.workunit.client.1.smithi123.stdout:1/462: creat d3/d6/f9d x:0 0 0 2024-02-16T22:21:57.749 INFO:tasks.workunit.client.1.smithi123.stdout:1/463: write d3/d6/f9d [928939,92778] 0 2024-02-16T22:21:57.750 INFO:tasks.workunit.client.0.smithi114.stdout:0/339: mkdir d1/d16/d1f/d28/d67 0 2024-02-16T22:21:57.751 INFO:tasks.workunit.client.1.smithi123.stdout:2/490: creat d4/d1a/f9b x:0 0 0 2024-02-16T22:21:57.753 INFO:tasks.workunit.client.0.smithi114.stdout:1/332: fdatasync d4/f36 0 2024-02-16T22:21:57.754 INFO:tasks.workunit.client.0.smithi114.stdout:1/333: write d4/fb [2019080,102827] 0 2024-02-16T22:21:57.757 INFO:tasks.workunit.client.1.smithi123.stdout:4/534: read d5/d1c/f3d [96883,2273] 0 2024-02-16T22:21:57.757 INFO:tasks.workunit.client.1.smithi123.stdout:4/535: chown d5/d1c/d23/d69 1 1 2024-02-16T22:21:57.759 INFO:tasks.workunit.client.1.smithi123.stdout:8/549: getdents d0/d29/da7/d43/d34 0 2024-02-16T22:21:57.761 INFO:tasks.workunit.client.1.smithi123.stdout:6/503: creat d0/d96/fb9 x:0 0 0 2024-02-16T22:21:57.763 INFO:tasks.workunit.client.0.smithi114.stdout:4/491: sync 2024-02-16T22:21:57.764 INFO:tasks.workunit.client.0.smithi114.stdout:6/390: sync 2024-02-16T22:21:57.765 INFO:tasks.workunit.client.1.smithi123.stdout:0/460: mknod d1/d8/d13/d1d/d62/d7a/c9a 0 2024-02-16T22:21:57.767 INFO:tasks.workunit.client.1.smithi123.stdout:5/467: getdents d6/dd/df/d58/d56 0 2024-02-16T22:21:57.768 INFO:tasks.workunit.client.0.smithi114.stdout:1/334: mkdir d4/d9/d65/d70 0 2024-02-16T22:21:57.769 INFO:tasks.workunit.client.1.smithi123.stdout:5/468: read d6/dd/df/d1f/d25/f1a [3365643,55314] 0 2024-02-16T22:21:57.769 INFO:tasks.workunit.client.1.smithi123.stdout:7/510: getdents d3/d26/d35 0 2024-02-16T22:21:57.770 INFO:tasks.workunit.client.1.smithi123.stdout:5/469: fdatasync d6/f8 0 2024-02-16T22:21:57.770 INFO:tasks.workunit.client.0.smithi114.stdout:3/319: getdents d4/d24/d2d/d45 0 2024-02-16T22:21:57.770 INFO:tasks.workunit.client.1.smithi123.stdout:5/470: chown d6/dd/df/d1f/d25/d37/f93 361 1 2024-02-16T22:21:57.771 INFO:tasks.workunit.client.0.smithi114.stdout:5/310: truncate d0/fa 3867556 0 2024-02-16T22:21:57.772 INFO:tasks.workunit.client.1.smithi123.stdout:1/464: rename d3/d6/d49/d48/d84/d92/d91 to d3/d6/d18/d9e 0 2024-02-16T22:21:57.773 INFO:tasks.workunit.client.1.smithi123.stdout:1/465: truncate d3/d47/f7d 631734 0 2024-02-16T22:21:57.774 INFO:tasks.workunit.client.0.smithi114.stdout:8/365: dwrite d4/d24/d2c/f4c [0,4194304] 0 2024-02-16T22:21:57.775 INFO:tasks.workunit.client.1.smithi123.stdout:2/491: mknod d4/d1a/d1d/d38/d5e/c9c 0 2024-02-16T22:21:57.777 INFO:tasks.workunit.client.1.smithi123.stdout:4/536: mknod d5/d49/ca7 0 2024-02-16T22:21:57.780 INFO:tasks.workunit.client.0.smithi114.stdout:1/335: chown d4/d33/c40 7 1 2024-02-16T22:21:57.783 INFO:tasks.workunit.client.1.smithi123.stdout:4/537: dwrite d5/d1c/d23/d69/f79 [0,4194304] 0 2024-02-16T22:21:57.787 INFO:tasks.workunit.client.0.smithi114.stdout:2/352: dwrite dd/df/d1c/d21/d61/d69/f72 [0,4194304] 0 2024-02-16T22:21:57.789 INFO:tasks.workunit.client.0.smithi114.stdout:9/396: write d2/d1a/f39 [368151,14387] 0 2024-02-16T22:21:57.796 INFO:tasks.workunit.client.0.smithi114.stdout:9/397: dread d2/f8 [4194304,4194304] 0 2024-02-16T22:21:57.798 INFO:tasks.workunit.client.1.smithi123.stdout:5/471: creat d6/dd/df/d58/d3f/f9e x:0 0 0 2024-02-16T22:21:57.800 INFO:tasks.workunit.client.1.smithi123.stdout:8/550: rename d0/d29/da7/d26/f8c to d0/d29/da7/d26/d2a/d4d/fb4 0 2024-02-16T22:21:57.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:57 smithi123 ceph-mon[37161]: pgmap v26: 65 pgs: 65 active+clean; 1.3 GiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 20 MiB/s rd, 178 MiB/s wr, 267 op/s 2024-02-16T22:21:57.802 INFO:tasks.workunit.client.0.smithi114.stdout:1/336: mkdir d4/d9/d46/d71 0 2024-02-16T22:21:57.802 INFO:tasks.workunit.client.1.smithi123.stdout:0/461: rmdir d1/d8 39 2024-02-16T22:21:57.803 INFO:tasks.workunit.client.0.smithi114.stdout:7/411: sync 2024-02-16T22:21:57.804 INFO:tasks.workunit.client.0.smithi114.stdout:3/320: mknod d4/d6a/c6f 0 2024-02-16T22:21:57.805 INFO:tasks.workunit.client.0.smithi114.stdout:3/321: write d4/d24/f67 [981779,54897] 0 2024-02-16T22:21:57.805 INFO:tasks.workunit.client.0.smithi114.stdout:3/322: chown d4/d26/l3e 5423775 1 2024-02-16T22:21:57.809 INFO:tasks.workunit.client.1.smithi123.stdout:3/499: link d0/d20/d76/d37/f7f d0/d20/faf 0 2024-02-16T22:21:57.811 INFO:tasks.workunit.client.0.smithi114.stdout:0/340: truncate d1/d1a/f36 4324094 0 2024-02-16T22:21:57.812 INFO:tasks.workunit.client.1.smithi123.stdout:6/504: symlink d0/d9/dc/d39/d8d/lba 0 2024-02-16T22:21:57.813 INFO:tasks.workunit.client.0.smithi114.stdout:2/353: fdatasync dd/df/d1c/d21/d74/f75 0 2024-02-16T22:21:57.817 INFO:tasks.workunit.client.0.smithi114.stdout:9/398: chown d2/dd/d3c/f59 82229154 1 2024-02-16T22:21:57.819 INFO:tasks.workunit.client.0.smithi114.stdout:8/366: write d4/d11/d78/d7a/f10 [665827,4763] 0 2024-02-16T22:21:57.821 INFO:tasks.workunit.client.1.smithi123.stdout:5/472: readlink d6/dd/df/d1f/d25/l75 0 2024-02-16T22:21:57.821 INFO:tasks.workunit.client.0.smithi114.stdout:4/492: dwrite dd/d1b/f5b [0,4194304] 0 2024-02-16T22:21:57.821 INFO:tasks.workunit.client.0.smithi114.stdout:6/391: write d6/f35 [266538,123728] 0 2024-02-16T22:21:57.822 INFO:tasks.workunit.client.1.smithi123.stdout:7/511: dread d3/d6e/d1d/f41 [0,4194304] 0 2024-02-16T22:21:57.823 INFO:tasks.workunit.client.1.smithi123.stdout:1/466: mkdir d3/d6/db/d10/d4f/d9f 0 2024-02-16T22:21:57.824 INFO:tasks.workunit.client.1.smithi123.stdout:1/467: chown d3/d6/db/l40 104353875 1 2024-02-16T22:21:57.824 INFO:tasks.workunit.client.1.smithi123.stdout:8/551: rmdir d0/d61 39 2024-02-16T22:21:57.826 INFO:tasks.workunit.client.0.smithi114.stdout:1/337: rename d4/d9/d46/d71 to d4/d63/d69/d72 0 2024-02-16T22:21:57.829 INFO:tasks.workunit.client.1.smithi123.stdout:2/492: symlink d4/d1a/d1d/d59/d45/l9d 0 2024-02-16T22:21:57.830 INFO:tasks.workunit.client.0.smithi114.stdout:3/323: read d4/d26/f3c [1705369,122745] 0 2024-02-16T22:21:57.831 INFO:tasks.workunit.client.0.smithi114.stdout:5/311: truncate d0/df/d16/f4d 2325690 0 2024-02-16T22:21:57.832 INFO:tasks.workunit.client.0.smithi114.stdout:5/312: chown d0/d2d/d30 16 1 2024-02-16T22:21:57.832 INFO:tasks.workunit.client.0.smithi114.stdout:0/341: fdatasync d1/d1a/f2e 0 2024-02-16T22:21:57.833 INFO:tasks.workunit.client.1.smithi123.stdout:6/505: chown d0/d9/dc/d39/c47 8182686 1 2024-02-16T22:21:57.834 INFO:tasks.workunit.client.1.smithi123.stdout:9/608: sync 2024-02-16T22:21:57.835 INFO:tasks.workunit.client.0.smithi114.stdout:2/354: readlink dd/df/d1c/l35 0 2024-02-16T22:21:57.837 INFO:tasks.workunit.client.1.smithi123.stdout:4/538: creat d5/d1c/d42/fa8 x:0 0 0 2024-02-16T22:21:57.839 INFO:tasks.workunit.client.0.smithi114.stdout:4/493: chown dd/c2a 3846995 1 2024-02-16T22:21:57.839 INFO:tasks.workunit.client.1.smithi123.stdout:6/506: dread d0/d9/f62 [0,4194304] 0 2024-02-16T22:21:57.839 INFO:tasks.workunit.client.1.smithi123.stdout:6/507: write d0/d6/d8e/f9d [993648,24518] 0 2024-02-16T22:21:57.840 INFO:tasks.workunit.client.0.smithi114.stdout:4/494: write dd/d1b/d37/f5d [942545,96055] 0 2024-02-16T22:21:57.844 INFO:tasks.workunit.client.0.smithi114.stdout:6/392: mknod d6/dd/d25/d34/d58/c7b 0 2024-02-16T22:21:57.844 INFO:tasks.workunit.client.1.smithi123.stdout:6/508: dwrite d0/d74/f7b [0,4194304] 0 2024-02-16T22:21:57.845 INFO:tasks.workunit.client.1.smithi123.stdout:1/468: mknod d3/d6/d18/ca0 0 2024-02-16T22:21:57.849 INFO:tasks.workunit.client.0.smithi114.stdout:7/412: rename d3/d74 to d3/d23/d6f/d89 0 2024-02-16T22:21:57.849 INFO:tasks.workunit.client.0.smithi114.stdout:8/367: rename d4/d11/d78 to d4/d11/d78/d7a/d31/d84 22 2024-02-16T22:21:57.850 INFO:tasks.workunit.client.0.smithi114.stdout:7/413: chown d3/d14/f70 362416 1 2024-02-16T22:21:57.851 INFO:tasks.workunit.client.0.smithi114.stdout:7/414: chown d3/d3d/l20 756 1 2024-02-16T22:21:57.856 INFO:tasks.workunit.client.1.smithi123.stdout:3/500: mkdir d0/d20/d76/d5d/dae/db0 0 2024-02-16T22:21:57.858 INFO:tasks.workunit.client.0.smithi114.stdout:8/368: dwrite d4/d11/d78/d7a/d36/f5d [0,4194304] 0 2024-02-16T22:21:57.858 INFO:tasks.workunit.client.1.smithi123.stdout:7/512: write d3/d26/f51 [5081449,54952] 0 2024-02-16T22:21:57.859 INFO:tasks.workunit.client.0.smithi114.stdout:7/415: dread d3/d14/f4d [0,4194304] 0 2024-02-16T22:21:57.860 INFO:tasks.workunit.client.0.smithi114.stdout:0/342: write d1/f3d [489331,115532] 0 2024-02-16T22:21:57.862 INFO:tasks.workunit.client.1.smithi123.stdout:9/609: unlink d4/f21 0 2024-02-16T22:21:57.865 INFO:tasks.workunit.client.1.smithi123.stdout:4/539: mkdir d5/d1c/d23/d74/d92/da9 0 2024-02-16T22:21:57.865 INFO:tasks.workunit.client.0.smithi114.stdout:2/355: rmdir dd/df/d1c 39 2024-02-16T22:21:57.867 INFO:tasks.workunit.client.0.smithi114.stdout:0/343: dread d1/d1a/f38 [0,4194304] 0 2024-02-16T22:21:57.870 INFO:tasks.workunit.client.1.smithi123.stdout:5/473: mkdir d6/dd/df/d9f 0 2024-02-16T22:21:57.870 INFO:tasks.workunit.client.1.smithi123.stdout:5/474: stat d6/dd/df/d1f/d25/d37/f6e 0 2024-02-16T22:21:57.870 INFO:tasks.workunit.client.0.smithi114.stdout:3/324: write d4/d26/f3c [834022,32531] 0 2024-02-16T22:21:57.871 INFO:tasks.workunit.client.0.smithi114.stdout:4/495: unlink dd/d1b/d37/d38/d7c/f9d 0 2024-02-16T22:21:57.874 INFO:tasks.workunit.client.0.smithi114.stdout:6/393: rmdir d6/dd/d25/d34/d61/d62 39 2024-02-16T22:21:57.875 INFO:tasks.workunit.client.0.smithi114.stdout:9/399: truncate d2/dd/ff 2193056 0 2024-02-16T22:21:57.876 INFO:tasks.workunit.client.1.smithi123.stdout:5/475: dwrite d6/dd/df/d58/d3f/d82/f86 [0,4194304] 0 2024-02-16T22:21:57.877 INFO:tasks.workunit.client.0.smithi114.stdout:4/496: dread dd/d1b/d37/f92 [0,4194304] 0 2024-02-16T22:21:57.879 INFO:tasks.workunit.client.1.smithi123.stdout:2/493: dread d4/fc [0,4194304] 0 2024-02-16T22:21:57.882 INFO:tasks.workunit.client.1.smithi123.stdout:6/509: dread d0/d6/f63 [0,4194304] 0 2024-02-16T22:21:57.883 INFO:tasks.workunit.client.1.smithi123.stdout:6/510: fdatasync d0/d9/dc/d3a/fa1 0 2024-02-16T22:21:57.885 INFO:tasks.workunit.client.1.smithi123.stdout:1/469: dwrite d3/d6/d11/f17 [4194304,4194304] 0 2024-02-16T22:21:57.885 INFO:tasks.workunit.client.0.smithi114.stdout:2/356: dread dd/d3b/f5c [0,4194304] 0 2024-02-16T22:21:57.890 INFO:tasks.workunit.client.0.smithi114.stdout:8/369: creat d4/d24/d2c/f85 x:0 0 0 2024-02-16T22:21:57.892 INFO:tasks.workunit.client.1.smithi123.stdout:1/470: dwrite d3/f7c [0,4194304] 0 2024-02-16T22:21:57.894 INFO:tasks.workunit.client.1.smithi123.stdout:1/471: chown d3/d6/db/d10/d2d/f25 125502 1 2024-02-16T22:21:57.894 INFO:tasks.workunit.client.1.smithi123.stdout:1/472: readlink l1 0 2024-02-16T22:21:57.894 INFO:tasks.workunit.client.0.smithi114.stdout:5/313: symlink d0/df/d16/d25/d3e/l55 0 2024-02-16T22:21:57.895 INFO:tasks.workunit.client.1.smithi123.stdout:0/462: truncate d1/d8/d13/d1d/f25 6882462 0 2024-02-16T22:21:57.896 INFO:tasks.workunit.client.0.smithi114.stdout:5/314: read d0/df/d16/f49 [419915,48979] 0 2024-02-16T22:21:57.898 INFO:tasks.workunit.client.0.smithi114.stdout:0/344: mknod d1/d1a/d47/c68 0 2024-02-16T22:21:57.899 INFO:tasks.workunit.client.0.smithi114.stdout:0/345: chown d1/d16/d18/f1c 362 1 2024-02-16T22:21:57.900 INFO:tasks.workunit.client.1.smithi123.stdout:2/494: symlink d4/d1a/d1d/d38/l9e 0 2024-02-16T22:21:57.901 INFO:tasks.workunit.client.1.smithi123.stdout:8/552: creat d0/d12/fb5 x:0 0 0 2024-02-16T22:21:57.902 INFO:tasks.workunit.client.0.smithi114.stdout:6/394: creat d6/d2c/f7c x:0 0 0 2024-02-16T22:21:57.903 INFO:tasks.workunit.client.1.smithi123.stdout:1/473: dwrite d3/d6/d18/f7e [0,4194304] 0 2024-02-16T22:21:57.904 INFO:tasks.workunit.client.0.smithi114.stdout:6/395: readlink d6/da/l44 0 2024-02-16T22:21:57.904 INFO:tasks.workunit.client.0.smithi114.stdout:9/400: write d2/dd/d3c/f7f [2181577,42122] 0 2024-02-16T22:21:57.904 INFO:tasks.workunit.client.0.smithi114.stdout:6/396: readlink d6/da/lb 0 2024-02-16T22:21:57.904 INFO:tasks.workunit.client.0.smithi114.stdout:6/397: stat l4 0 2024-02-16T22:21:57.905 INFO:tasks.workunit.client.0.smithi114.stdout:6/398: dread - d6/d2c/f7c zero size 2024-02-16T22:21:57.905 INFO:tasks.workunit.client.1.smithi123.stdout:4/540: dread d5/d49/f4b [0,4194304] 0 2024-02-16T22:21:57.905 INFO:tasks.workunit.client.1.smithi123.stdout:3/501: creat d0/d20/d94/dac/fb1 x:0 0 0 2024-02-16T22:21:57.905 INFO:tasks.workunit.client.1.smithi123.stdout:9/610: getdents d4/d36/d8e/d84/d93/d96 0 2024-02-16T22:21:57.906 INFO:tasks.workunit.client.1.smithi123.stdout:0/463: creat d1/d4/d17/d54/f9b x:0 0 0 2024-02-16T22:21:57.908 INFO:tasks.workunit.client.1.smithi123.stdout:5/476: creat d6/dd/d78/fa0 x:0 0 0 2024-02-16T22:21:57.909 INFO:tasks.workunit.client.1.smithi123.stdout:1/474: dread d3/f7c [0,4194304] 0 2024-02-16T22:21:57.910 INFO:tasks.workunit.client.0.smithi114.stdout:8/370: creat d4/d11/d78/f86 x:0 0 0 2024-02-16T22:21:57.911 INFO:tasks.workunit.client.1.smithi123.stdout:8/553: fdatasync d0/d29/da7/d26/d68/d81/d47/f73 0 2024-02-16T22:21:57.917 INFO:tasks.workunit.client.0.smithi114.stdout:8/371: dwrite d4/d24/f50 [0,4194304] 0 2024-02-16T22:21:57.918 INFO:tasks.workunit.client.1.smithi123.stdout:7/513: rename d3/d5/l58 to d3/d6e/d1d/la7 0 2024-02-16T22:21:57.918 INFO:tasks.workunit.client.1.smithi123.stdout:7/514: fsync d3/d87/f65 0 2024-02-16T22:21:57.919 INFO:tasks.workunit.client.0.smithi114.stdout:8/372: fdatasync d4/f1d 0 2024-02-16T22:21:57.922 INFO:tasks.workunit.client.1.smithi123.stdout:3/502: dread d0/fe [0,4194304] 0 2024-02-16T22:21:57.922 INFO:tasks.workunit.client.0.smithi114.stdout:3/325: symlink d4/d43/d53/l70 0 2024-02-16T22:21:57.929 INFO:tasks.workunit.client.0.smithi114.stdout:4/497: mkdir dd/d1b/d37/d38/d87/dad 0 2024-02-16T22:21:57.931 INFO:tasks.workunit.client.1.smithi123.stdout:0/464: creat d1/d8/d13/d1d/d62/d7a/f9c x:0 0 0 2024-02-16T22:21:57.931 INFO:tasks.workunit.client.0.smithi114.stdout:9/401: truncate d2/d1a/d23/f28 860988 0 2024-02-16T22:21:57.932 INFO:tasks.workunit.client.0.smithi114.stdout:9/402: write d2/d33/f7d [324580,95131] 0 2024-02-16T22:21:57.935 INFO:tasks.workunit.client.0.smithi114.stdout:1/338: getdents d4/d63/d69 0 2024-02-16T22:21:57.940 INFO:tasks.workunit.client.1.smithi123.stdout:2/495: dread d4/d1a/d1d/d13/d25/d98/f67 [0,4194304] 0 2024-02-16T22:21:57.941 INFO:tasks.workunit.client.0.smithi114.stdout:7/416: link d3/d22/f48 d3/d5a/f8a 0 2024-02-16T22:21:57.941 INFO:tasks.workunit.client.1.smithi123.stdout:2/496: readlink d4/d1a/d1d/d59/d45/l62 0 2024-02-16T22:21:57.941 INFO:tasks.workunit.client.0.smithi114.stdout:1/339: dwrite d4/fb [0,4194304] 0 2024-02-16T22:21:57.943 INFO:tasks.workunit.client.1.smithi123.stdout:6/511: write d0/d9/dc/d5f/f2d [1209079,26970] 0 2024-02-16T22:21:57.944 INFO:tasks.workunit.client.1.smithi123.stdout:6/512: write d0/d74/f7b [5130666,67865] 0 2024-02-16T22:21:57.948 INFO:tasks.workunit.client.0.smithi114.stdout:0/346: creat d1/d16/d48/f69 x:0 0 0 2024-02-16T22:21:57.949 INFO:tasks.workunit.client.1.smithi123.stdout:3/503: dread d0/d20/d76/f4e [0,4194304] 0 2024-02-16T22:21:57.950 INFO:tasks.workunit.client.1.smithi123.stdout:3/504: write d0/d40/f56 [663334,84036] 0 2024-02-16T22:21:57.950 INFO:tasks.workunit.client.1.smithi123.stdout:7/515: mkdir d3/d87/da8 0 2024-02-16T22:21:57.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:57 smithi114 ceph-mon[29235]: pgmap v26: 65 pgs: 65 active+clean; 1.3 GiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 20 MiB/s rd, 178 MiB/s wr, 267 op/s 2024-02-16T22:21:57.953 INFO:tasks.workunit.client.1.smithi123.stdout:7/516: write d3/fd [4246466,72365] 0 2024-02-16T22:21:57.954 INFO:tasks.workunit.client.0.smithi114.stdout:8/373: creat d4/d16/d25/d7c/f87 x:0 0 0 2024-02-16T22:21:57.955 INFO:tasks.workunit.client.0.smithi114.stdout:3/326: dread d4/d26/f34 [0,4194304] 0 2024-02-16T22:21:57.957 INFO:tasks.workunit.client.1.smithi123.stdout:9/611: mkdir d4/d1e/d3a/dac/dc5 0 2024-02-16T22:21:57.957 INFO:tasks.workunit.client.1.smithi123.stdout:4/541: dwrite d5/f17 [4194304,4194304] 0 2024-02-16T22:21:57.958 INFO:tasks.workunit.client.0.smithi114.stdout:4/498: mknod dd/d1b/d24/d98/cae 0 2024-02-16T22:21:57.960 INFO:tasks.workunit.client.1.smithi123.stdout:0/465: mkdir d1/d4/d9d 0 2024-02-16T22:21:57.962 INFO:tasks.workunit.client.0.smithi114.stdout:6/399: mknod d6/dd/d25/d34/d61/c7d 0 2024-02-16T22:21:57.962 INFO:tasks.workunit.client.1.smithi123.stdout:2/497: dread d4/d1a/d1d/d13/d25/f66 [0,4194304] 0 2024-02-16T22:21:57.964 INFO:tasks.workunit.client.1.smithi123.stdout:5/477: symlink d6/dd/df/d1f/la1 0 2024-02-16T22:21:57.964 INFO:tasks.workunit.client.1.smithi123.stdout:1/475: dwrite d3/d6/d11/f23 [4194304,4194304] 0 2024-02-16T22:21:57.966 INFO:tasks.workunit.client.1.smithi123.stdout:5/478: write d6/dd/d5f/f60 [4899343,118709] 0 2024-02-16T22:21:57.971 INFO:tasks.workunit.client.0.smithi114.stdout:1/340: rename d4/d9/f4e to d4/d9/d46/d47/f73 0 2024-02-16T22:21:57.974 INFO:tasks.workunit.client.0.smithi114.stdout:5/315: link d0/df/d16/d1e/f27 d0/df/d16/d25/d2e/f56 0 2024-02-16T22:21:57.974 INFO:tasks.workunit.client.0.smithi114.stdout:0/347: creat d1/d16/d48/f6a x:0 0 0 2024-02-16T22:21:57.976 INFO:tasks.workunit.client.0.smithi114.stdout:8/374: symlink d4/d16/d25/d7c/l88 0 2024-02-16T22:21:57.977 INFO:tasks.workunit.client.0.smithi114.stdout:8/375: fdatasync d4/d11/f75 0 2024-02-16T22:21:57.977 INFO:tasks.workunit.client.0.smithi114.stdout:3/327: symlink d4/dd/l71 0 2024-02-16T22:21:57.980 INFO:tasks.workunit.client.0.smithi114.stdout:4/499: creat dd/d1b/d37/d38/faf x:0 0 0 2024-02-16T22:21:57.984 INFO:tasks.workunit.client.0.smithi114.stdout:7/417: mknod d3/d87/c8b 0 2024-02-16T22:21:57.987 INFO:tasks.workunit.client.0.smithi114.stdout:4/500: dwrite dd/d1b/d24/d57/f85 [0,4194304] 0 2024-02-16T22:21:57.987 INFO:tasks.workunit.client.1.smithi123.stdout:7/517: symlink d3/d5/la9 0 2024-02-16T22:21:57.987 INFO:tasks.workunit.client.1.smithi123.stdout:9/612: mkdir d4/d36/daa/d2b/dc6 0 2024-02-16T22:21:57.988 INFO:tasks.workunit.client.0.smithi114.stdout:4/501: chown dd/d1b/d37/d61 13772 1 2024-02-16T22:21:57.988 INFO:tasks.workunit.client.1.smithi123.stdout:8/554: dwrite d0/d29/da7/d43/f28 [4194304,4194304] 0 2024-02-16T22:21:57.988 INFO:tasks.workunit.client.0.smithi114.stdout:4/502: write dd/d1b/d41/d74/f79 [133566,72888] 0 2024-02-16T22:21:57.990 INFO:tasks.workunit.client.1.smithi123.stdout:0/466: write d1/d8/f5d [1178083,69840] 0 2024-02-16T22:21:57.995 INFO:tasks.workunit.client.1.smithi123.stdout:9/613: write d4/d36/d8e/d58/f47 [1893218,128465] 0 2024-02-16T22:21:57.996 INFO:tasks.workunit.client.1.smithi123.stdout:1/476: mknod d3/d6/d49/d48/d84/d92/ca1 0 2024-02-16T22:21:57.997 INFO:tasks.workunit.client.1.smithi123.stdout:5/479: dwrite d6/dd/df/d58/f9a [0,4194304] 0 2024-02-16T22:21:57.998 INFO:tasks.workunit.client.1.smithi123.stdout:6/513: creat d0/d6/d91/da2/fbb x:0 0 0 2024-02-16T22:21:58.000 INFO:tasks.workunit.client.1.smithi123.stdout:6/514: dread - d0/d74/f94 zero size 2024-02-16T22:21:58.000 INFO:tasks.workunit.client.0.smithi114.stdout:1/341: mkdir d4/d33/d74 0 2024-02-16T22:21:58.003 INFO:tasks.workunit.client.0.smithi114.stdout:0/348: symlink d1/d16/d1f/d28/l6b 0 2024-02-16T22:21:58.007 INFO:tasks.workunit.client.1.smithi123.stdout:4/542: mkdir d5/d1c/d23/d47/da2/daa 0 2024-02-16T22:21:58.008 INFO:tasks.workunit.client.1.smithi123.stdout:4/543: truncate d5/d1c/d23/f57 4386827 0 2024-02-16T22:21:58.009 INFO:tasks.workunit.client.0.smithi114.stdout:3/328: mkdir d4/dd/d33/d72 0 2024-02-16T22:21:58.010 INFO:tasks.workunit.client.1.smithi123.stdout:7/518: rmdir d3/d6e/d4d 39 2024-02-16T22:21:58.014 INFO:tasks.workunit.client.0.smithi114.stdout:2/357: getdents dd/df/d1c/d21/d61 0 2024-02-16T22:21:58.017 INFO:tasks.workunit.client.1.smithi123.stdout:8/555: write d0/d29/da7/d26/d68/d81/d47/d9b/fb0 [4732576,83320] 0 2024-02-16T22:21:58.018 INFO:tasks.workunit.client.0.smithi114.stdout:7/418: fsync d3/d22/f6c 0 2024-02-16T22:21:58.021 INFO:tasks.workunit.client.1.smithi123.stdout:9/614: mkdir d4/d36/d8e/d58/d59/dc7 0 2024-02-16T22:21:58.022 INFO:tasks.workunit.client.0.smithi114.stdout:7/419: dread d3/d14/f70 [0,4194304] 0 2024-02-16T22:21:58.024 INFO:tasks.workunit.client.1.smithi123.stdout:1/477: mknod d3/d6/d49/d48/d68/ca2 0 2024-02-16T22:21:58.025 INFO:tasks.workunit.client.1.smithi123.stdout:1/478: dread d3/d6/d49/f66 [0,4194304] 0 2024-02-16T22:21:58.028 INFO:tasks.workunit.client.0.smithi114.stdout:3/329: mknod d4/d46/d48/c73 0 2024-02-16T22:21:58.030 INFO:tasks.workunit.client.1.smithi123.stdout:6/515: rename d0/d6/c14 to d0/d6/d2f/d5d/cbc 0 2024-02-16T22:21:58.031 INFO:tasks.workunit.client.1.smithi123.stdout:5/480: dwrite d6/dd/df/f79 [0,4194304] 0 2024-02-16T22:21:58.031 INFO:tasks.workunit.client.0.smithi114.stdout:6/400: link d6/dd/l59 d6/d2c/d38/d41/l7e 0 2024-02-16T22:21:58.032 INFO:tasks.workunit.client.0.smithi114.stdout:9/403: truncate d2/dd/f3b 3092528 0 2024-02-16T22:21:58.032 INFO:tasks.workunit.client.1.smithi123.stdout:6/516: write d0/d6/d91/d97/fb8 [123628,104580] 0 2024-02-16T22:21:58.032 INFO:tasks.workunit.client.1.smithi123.stdout:5/481: dread - d6/d3b/f7a zero size 2024-02-16T22:21:58.033 INFO:tasks.workunit.client.1.smithi123.stdout:3/505: link d0/d20/d2e/l45 d0/d20/d29/d9b/lb2 0 2024-02-16T22:21:58.034 INFO:tasks.workunit.client.0.smithi114.stdout:3/330: dread d4/f16 [0,4194304] 0 2024-02-16T22:21:58.040 INFO:tasks.workunit.client.0.smithi114.stdout:1/342: dread d4/d33/f3e [0,4194304] 0 2024-02-16T22:21:58.041 INFO:tasks.workunit.client.1.smithi123.stdout:8/556: rmdir d0/d29/da7/d26/d68 39 2024-02-16T22:21:58.042 INFO:tasks.workunit.client.0.smithi114.stdout:1/343: fdatasync d4/d9/f12 0 2024-02-16T22:21:58.042 INFO:tasks.workunit.client.0.smithi114.stdout:1/344: dread - d4/d9/d46/f48 zero size 2024-02-16T22:21:58.042 INFO:tasks.workunit.client.1.smithi123.stdout:8/557: chown d0/d29/da7/d26/d2a/d3c/l3b 6208549 1 2024-02-16T22:21:58.044 INFO:tasks.workunit.client.0.smithi114.stdout:4/503: mkdir dd/d1b/db0 0 2024-02-16T22:21:58.051 INFO:tasks.workunit.client.1.smithi123.stdout:2/498: sync 2024-02-16T22:21:58.051 INFO:tasks.workunit.client.1.smithi123.stdout:1/479: dread - d3/d6/db/d10/d4f/f51 zero size 2024-02-16T22:21:58.055 INFO:tasks.workunit.client.1.smithi123.stdout:6/517: read d0/d9/dc/f23 [219711,92618] 0 2024-02-16T22:21:58.056 INFO:tasks.workunit.client.0.smithi114.stdout:0/349: write d1/d1a/d3e/d41/f42 [1016881,108035] 0 2024-02-16T22:21:58.057 INFO:tasks.workunit.client.0.smithi114.stdout:0/350: chown d1/d16/d1f/d28/d51/d60/c39 1508 1 2024-02-16T22:21:58.058 INFO:tasks.workunit.client.1.smithi123.stdout:3/506: truncate d0/d20/d29/f3c 190298 0 2024-02-16T22:21:58.063 INFO:tasks.workunit.client.1.smithi123.stdout:7/519: getdents d3/d87/da8 0 2024-02-16T22:21:58.065 INFO:tasks.workunit.client.0.smithi114.stdout:0/351: dwrite d1/d1a/f2e [0,4194304] 0 2024-02-16T22:21:58.065 INFO:tasks.workunit.client.1.smithi123.stdout:3/507: dwrite d0/d20/d76/d5d/f86 [0,4194304] 0 2024-02-16T22:21:58.065 INFO:tasks.workunit.client.1.smithi123.stdout:0/467: truncate d1/d8/f6b 3166237 0 2024-02-16T22:21:58.065 INFO:tasks.workunit.client.1.smithi123.stdout:7/520: stat d3/d6e/d24/f56 0 2024-02-16T22:21:58.066 INFO:tasks.workunit.client.0.smithi114.stdout:7/420: write d3/d3d/f41 [5880741,22875] 0 2024-02-16T22:21:58.067 INFO:tasks.workunit.client.0.smithi114.stdout:0/352: write d1/d16/d1f/d28/d51/d60/f5e [1868224,82737] 0 2024-02-16T22:21:58.068 INFO:tasks.workunit.client.1.smithi123.stdout:4/544: dwrite d5/d1c/d42/f4a [0,4194304] 0 2024-02-16T22:21:58.068 INFO:tasks.workunit.client.0.smithi114.stdout:4/504: creat dd/d1b/d41/d77/fb1 x:0 0 0 2024-02-16T22:21:58.070 INFO:tasks.workunit.client.0.smithi114.stdout:9/404: write d2/d1a/f21 [2195190,102317] 0 2024-02-16T22:21:58.070 INFO:tasks.workunit.client.1.smithi123.stdout:4/545: fdatasync d5/d1c/d23/d6d/f94 0 2024-02-16T22:21:58.070 INFO:tasks.workunit.client.0.smithi114.stdout:9/405: stat d2/l30 0 2024-02-16T22:21:58.071 INFO:tasks.workunit.client.0.smithi114.stdout:3/331: write d4/d43/f56 [772276,68297] 0 2024-02-16T22:21:58.072 INFO:tasks.workunit.client.0.smithi114.stdout:9/406: write d2/f26 [1705480,21566] 0 2024-02-16T22:21:58.073 INFO:tasks.workunit.client.1.smithi123.stdout:2/499: readlink d4/d1a/d1d/l10 0 2024-02-16T22:21:58.073 INFO:tasks.workunit.client.1.smithi123.stdout:3/508: dwrite d0/ff [0,4194304] 0 2024-02-16T22:21:58.076 INFO:tasks.workunit.client.1.smithi123.stdout:6/518: mkdir d0/d6/d2f/dbd 0 2024-02-16T22:21:58.078 INFO:tasks.workunit.client.1.smithi123.stdout:1/480: dread d3/f35 [0,4194304] 0 2024-02-16T22:21:58.081 INFO:tasks.workunit.client.0.smithi114.stdout:2/358: creat dd/f7a x:0 0 0 2024-02-16T22:21:58.081 INFO:tasks.workunit.client.1.smithi123.stdout:6/519: write d0/d6/d8e/f9d [595891,38128] 0 2024-02-16T22:21:58.082 INFO:tasks.workunit.client.1.smithi123.stdout:3/509: dread d0/d20/f8b [4194304,4194304] 0 2024-02-16T22:21:58.082 INFO:tasks.workunit.client.1.smithi123.stdout:8/558: symlink d0/lb6 0 2024-02-16T22:21:58.087 INFO:tasks.workunit.client.1.smithi123.stdout:7/521: symlink d3/d6e/d24/laa 0 2024-02-16T22:21:58.087 INFO:tasks.workunit.client.0.smithi114.stdout:7/421: rename d3/d23/d6f/d89/c75 to d3/d3d/c8c 0 2024-02-16T22:21:58.090 INFO:tasks.workunit.client.1.smithi123.stdout:8/559: dwrite d0/d29/da7/d26/d32/d45/f5f [4194304,4194304] 0 2024-02-16T22:21:58.092 INFO:tasks.workunit.client.1.smithi123.stdout:4/546: mknod d5/d1c/d23/d74/d92/d9b/cab 0 2024-02-16T22:21:58.094 INFO:tasks.workunit.client.0.smithi114.stdout:9/407: symlink d2/d1a/d23/d35/d47/l85 0 2024-02-16T22:21:58.095 INFO:tasks.workunit.client.0.smithi114.stdout:9/408: stat d2/d1a/d23/d35/d47/l85 0 2024-02-16T22:21:58.096 INFO:tasks.workunit.client.1.smithi123.stdout:1/481: mknod d3/d6/db/d10/d4b/ca3 0 2024-02-16T22:21:58.096 INFO:tasks.workunit.client.0.smithi114.stdout:2/359: mkdir dd/d3b/d7b 0 2024-02-16T22:21:58.097 INFO:tasks.workunit.client.1.smithi123.stdout:1/482: chown d3/d6/d11/d76/l64 28246 1 2024-02-16T22:21:58.097 INFO:tasks.workunit.client.1.smithi123.stdout:1/483: read d3/d6/d18/f7e [3667482,9720] 0 2024-02-16T22:21:58.099 INFO:tasks.workunit.client.0.smithi114.stdout:2/360: write dd/df/d2c/f5a [1162329,8570] 0 2024-02-16T22:21:58.099 INFO:tasks.workunit.client.1.smithi123.stdout:6/520: truncate d0/d6/d1a/f64 166413 0 2024-02-16T22:21:58.100 INFO:tasks.workunit.client.0.smithi114.stdout:7/422: symlink d3/d3d/l8d 0 2024-02-16T22:21:58.100 INFO:tasks.workunit.client.0.smithi114.stdout:4/505: mknod dd/d1b/cb2 0 2024-02-16T22:21:58.102 INFO:tasks.workunit.client.1.smithi123.stdout:0/468: fsync d1/d8/d13/f15 0 2024-02-16T22:21:58.103 INFO:tasks.workunit.client.1.smithi123.stdout:1/484: dwrite d3/d6/d11/d76/f59 [0,4194304] 0 2024-02-16T22:21:58.103 INFO:tasks.workunit.client.1.smithi123.stdout:0/469: stat d1/d8/d13/d1d/d62/d7a/f7f 0 2024-02-16T22:21:58.104 INFO:tasks.workunit.client.0.smithi114.stdout:6/401: getdents d6/dd/d25/d34/d61 0 2024-02-16T22:21:58.105 INFO:tasks.workunit.client.1.smithi123.stdout:9/615: link d4/d36/d8e/d58/d59/f5c d4/d1e/fc8 0 2024-02-16T22:21:58.108 INFO:tasks.workunit.client.0.smithi114.stdout:2/361: unlink dd/d3c/f4a 0 2024-02-16T22:21:58.109 INFO:tasks.workunit.client.0.smithi114.stdout:7/423: creat d3/d87/f8e x:0 0 0 2024-02-16T22:21:58.111 INFO:tasks.workunit.client.0.smithi114.stdout:7/424: write d3/d14/f29 [1648746,5308] 0 2024-02-16T22:21:58.111 INFO:tasks.workunit.client.1.smithi123.stdout:9/616: read d4/d36/daa/f23 [1342864,125946] 0 2024-02-16T22:21:58.112 INFO:tasks.workunit.client.0.smithi114.stdout:7/425: fsync d3/d23/f5e 0 2024-02-16T22:21:58.113 INFO:tasks.workunit.client.0.smithi114.stdout:7/426: chown d3/d3d/f86 1405 1 2024-02-16T22:21:58.114 INFO:tasks.workunit.client.0.smithi114.stdout:4/506: creat dd/d2e/d56/fb3 x:0 0 0 2024-02-16T22:21:58.114 INFO:tasks.workunit.client.0.smithi114.stdout:7/427: read d3/d14/f4c [769880,113922] 0 2024-02-16T22:21:58.115 INFO:tasks.workunit.client.0.smithi114.stdout:7/428: stat d3/d23/d4e 0 2024-02-16T22:21:58.119 INFO:tasks.workunit.client.1.smithi123.stdout:6/521: mknod d0/d9/dc/d3a/d70/cbe 0 2024-02-16T22:21:58.121 INFO:tasks.workunit.client.0.smithi114.stdout:0/353: rename d1/d16/d1f/d28/f56 to d1/f6c 0 2024-02-16T22:21:58.121 INFO:tasks.workunit.client.1.smithi123.stdout:5/482: dwrite d6/dd/df/d58/f43 [0,4194304] 0 2024-02-16T22:21:58.122 INFO:tasks.workunit.client.1.smithi123.stdout:6/522: write d0/d9/dc/d3a/f7d [2590989,45042] 0 2024-02-16T22:21:58.124 INFO:tasks.workunit.client.0.smithi114.stdout:1/345: truncate d4/d9/d3d/f41 2067743 0 2024-02-16T22:21:58.127 INFO:tasks.workunit.client.1.smithi123.stdout:1/485: creat d3/d47/fa4 x:0 0 0 2024-02-16T22:21:58.132 INFO:tasks.workunit.client.1.smithi123.stdout:1/486: dwrite d3/f8 [0,4194304] 0 2024-02-16T22:21:58.135 INFO:tasks.workunit.client.1.smithi123.stdout:2/500: rename d4/d1a/d1d/d13/d15/d6d/f28 to d4/d1a/d1d/f9f 0 2024-02-16T22:21:58.143 INFO:tasks.workunit.client.0.smithi114.stdout:9/409: dread d2/d1a/d23/d35/f31 [4194304,4194304] 0 2024-02-16T22:21:58.143 INFO:tasks.workunit.client.0.smithi114.stdout:4/507: symlink dd/d1b/d37/d38/lb4 0 2024-02-16T22:21:58.144 INFO:tasks.workunit.client.0.smithi114.stdout:4/508: chown dd/d1b/d24/f4e 353955626 1 2024-02-16T22:21:58.144 INFO:tasks.workunit.client.1.smithi123.stdout:5/483: mkdir d6/dd/df/d1f/d25/da2 0 2024-02-16T22:21:58.145 INFO:tasks.workunit.client.1.smithi123.stdout:5/484: stat d6/c7 0 2024-02-16T22:21:58.145 INFO:tasks.workunit.client.1.smithi123.stdout:5/485: stat d6/dd/df/f57 0 2024-02-16T22:21:58.146 INFO:tasks.workunit.client.1.smithi123.stdout:7/522: creat d3/d6e/d1d/fab x:0 0 0 2024-02-16T22:21:58.147 INFO:tasks.workunit.client.1.smithi123.stdout:5/486: dread - d6/dd/df/d58/d89/f9c zero size 2024-02-16T22:21:58.149 INFO:tasks.workunit.client.1.smithi123.stdout:8/560: dread d0/d12/f3f [0,4194304] 0 2024-02-16T22:21:58.149 INFO:tasks.workunit.client.0.smithi114.stdout:0/354: rename d1/d16/d1f/d28/d66 to d1/d16/d1f/d28/d51/d61/d6d 0 2024-02-16T22:21:58.150 INFO:tasks.workunit.client.0.smithi114.stdout:3/332: truncate d4/d24/f25 2230390 0 2024-02-16T22:21:58.150 INFO:tasks.workunit.client.0.smithi114.stdout:0/355: chown d1/d16/d18/c3a 7 1 2024-02-16T22:21:58.152 INFO:tasks.workunit.client.1.smithi123.stdout:4/547: creat d5/d1c/fac x:0 0 0 2024-02-16T22:21:58.152 INFO:tasks.workunit.client.1.smithi123.stdout:5/487: dwrite d6/dd/df/d1f/d25/f97 [0,4194304] 0 2024-02-16T22:21:58.153 INFO:tasks.workunit.client.1.smithi123.stdout:4/548: fsync d5/d1c/d42/f4a 0 2024-02-16T22:21:58.153 INFO:tasks.workunit.client.0.smithi114.stdout:1/346: rmdir d4/d9/d46 39 2024-02-16T22:21:58.161 INFO:tasks.workunit.client.1.smithi123.stdout:3/510: rename d0/ff to d0/d20/d76/d5d/dae/db0/fb3 0 2024-02-16T22:21:58.163 INFO:tasks.workunit.client.0.smithi114.stdout:9/410: write d2/d1a/d23/d35/d1d/f54 [677605,61648] 0 2024-02-16T22:21:58.163 INFO:tasks.workunit.client.1.smithi123.stdout:2/501: creat d4/d1a/d1d/d13/d15/d6d/fa0 x:0 0 0 2024-02-16T22:21:58.164 INFO:tasks.workunit.client.1.smithi123.stdout:2/502: write d4/d1a/d1d/d38/f58 [8222,118440] 0 2024-02-16T22:21:58.165 INFO:tasks.workunit.client.0.smithi114.stdout:4/509: creat dd/d1b/d37/d38/d7c/fb5 x:0 0 0 2024-02-16T22:21:58.168 INFO:tasks.workunit.client.0.smithi114.stdout:6/402: dwrite d6/d2c/f6d [0,4194304] 0 2024-02-16T22:21:58.169 INFO:tasks.workunit.client.0.smithi114.stdout:6/403: dread - d6/d2c/d38/d54/f6b zero size 2024-02-16T22:21:58.171 INFO:tasks.workunit.client.0.smithi114.stdout:2/362: write dd/df/f11 [792914,23885] 0 2024-02-16T22:21:58.172 INFO:tasks.workunit.client.1.smithi123.stdout:7/523: creat d3/d5/d6c/fac x:0 0 0 2024-02-16T22:21:58.173 INFO:tasks.workunit.client.1.smithi123.stdout:7/524: truncate d3/d6e/d24/f6d 5162292 0 2024-02-16T22:21:58.174 INFO:tasks.workunit.client.1.smithi123.stdout:7/525: chown d3/d6e/d24/l74 102525 1 2024-02-16T22:21:58.175 INFO:tasks.workunit.client.1.smithi123.stdout:7/526: write d3/f2a [3298513,84474] 0 2024-02-16T22:21:58.176 INFO:tasks.workunit.client.0.smithi114.stdout:0/356: mkdir d1/d16/d1f/d28/d51/d61/d6d/d6e 0 2024-02-16T22:21:58.179 INFO:tasks.workunit.client.0.smithi114.stdout:3/333: dread d4/d26/f31 [0,4194304] 0 2024-02-16T22:21:58.184 INFO:tasks.workunit.client.1.smithi123.stdout:5/488: creat d6/dd/d44/fa3 x:0 0 0 2024-02-16T22:21:58.185 INFO:tasks.workunit.client.0.smithi114.stdout:7/429: creat d3/d22/f8f x:0 0 0 2024-02-16T22:21:58.186 INFO:tasks.workunit.client.0.smithi114.stdout:7/430: fdatasync d3/d3d/f4a 0 2024-02-16T22:21:58.186 INFO:tasks.workunit.client.1.smithi123.stdout:4/549: symlink d5/d1c/d23/d6d/lad 0 2024-02-16T22:21:58.189 INFO:tasks.workunit.client.0.smithi114.stdout:9/411: creat d2/d1a/d23/d35/f86 x:0 0 0 2024-02-16T22:21:58.194 INFO:tasks.workunit.client.0.smithi114.stdout:4/510: unlink dd/d1b/d37/f92 0 2024-02-16T22:21:58.196 INFO:tasks.workunit.client.0.smithi114.stdout:6/404: rename d6/dd/d25/c4e to d6/dd/d25/d34/c7f 0 2024-02-16T22:21:58.196 INFO:tasks.workunit.client.1.smithi123.stdout:1/487: dwrite d3/d6/db/d10/d4f/f51 [0,4194304] 0 2024-02-16T22:21:58.198 INFO:tasks.workunit.client.1.smithi123.stdout:3/511: creat d0/d20/d76/d37/d49/fb4 x:0 0 0 2024-02-16T22:21:58.198 INFO:tasks.workunit.client.1.smithi123.stdout:9/617: truncate d4/d36/daa/f1a 3489746 0 2024-02-16T22:21:58.199 INFO:tasks.workunit.client.1.smithi123.stdout:3/512: write d0/d40/f59 [225840,25038] 0 2024-02-16T22:21:58.203 INFO:tasks.workunit.client.1.smithi123.stdout:8/561: symlink d0/d61/lb7 0 2024-02-16T22:21:58.207 INFO:tasks.workunit.client.1.smithi123.stdout:5/489: dread - d6/dd/d44/f47 zero size 2024-02-16T22:21:58.211 INFO:tasks.workunit.client.1.smithi123.stdout:4/550: symlink d5/d1c/d23/d56/lae 0 2024-02-16T22:21:58.215 INFO:tasks.workunit.client.1.smithi123.stdout:1/488: symlink d3/d47/la5 0 2024-02-16T22:21:58.216 INFO:tasks.workunit.client.1.smithi123.stdout:4/551: dwrite d5/d49/f91 [0,4194304] 0 2024-02-16T22:21:58.217 INFO:tasks.workunit.client.1.smithi123.stdout:1/489: chown d3/d6/d49/f50 3371 1 2024-02-16T22:21:58.217 INFO:tasks.workunit.client.1.smithi123.stdout:3/513: symlink d0/d50/lb5 0 2024-02-16T22:21:58.218 INFO:tasks.workunit.client.1.smithi123.stdout:0/470: sync 2024-02-16T22:21:58.219 INFO:tasks.workunit.client.1.smithi123.stdout:6/523: sync 2024-02-16T22:21:58.219 INFO:tasks.workunit.client.1.smithi123.stdout:2/503: sync 2024-02-16T22:21:58.221 INFO:tasks.workunit.client.1.smithi123.stdout:5/490: truncate d6/dd/df/f62 50342 0 2024-02-16T22:21:58.224 INFO:tasks.workunit.client.1.smithi123.stdout:3/514: creat d0/d20/d76/d5d/dae/fb6 x:0 0 0 2024-02-16T22:21:58.226 INFO:tasks.workunit.client.1.smithi123.stdout:3/515: chown d0/d20/d76/d37/d49/f9c 78905 1 2024-02-16T22:21:58.228 INFO:tasks.workunit.client.1.smithi123.stdout:5/491: dwrite d6/dd/d5f/f60 [4194304,4194304] 0 2024-02-16T22:21:58.246 INFO:tasks.workunit.client.0.smithi114.stdout:1/347: unlink d4/d9/cc 0 2024-02-16T22:21:58.249 INFO:tasks.workunit.client.0.smithi114.stdout:7/431: creat d3/d23/d6f/f90 x:0 0 0 2024-02-16T22:21:58.249 INFO:tasks.workunit.client.0.smithi114.stdout:6/405: dread d6/dd/d25/d34/d58/f78 [0,4194304] 0 2024-02-16T22:21:58.249 INFO:tasks.workunit.client.0.smithi114.stdout:1/348: write d4/d9/d3d/f39 [7407038,89784] 0 2024-02-16T22:21:58.251 INFO:tasks.workunit.client.0.smithi114.stdout:2/363: rename dd/f13 to dd/df/d1c/d21/d74/f7c 0 2024-02-16T22:21:58.252 INFO:tasks.workunit.client.0.smithi114.stdout:3/334: write d4/d26/f63 [7557968,33422] 0 2024-02-16T22:21:58.252 INFO:tasks.workunit.client.1.smithi123.stdout:2/504: unlink d4/c5 0 2024-02-16T22:21:58.258 INFO:tasks.workunit.client.0.smithi114.stdout:4/511: mknod dd/d1b/d37/d38/d87/dad/cb6 0 2024-02-16T22:21:58.259 INFO:tasks.workunit.client.0.smithi114.stdout:9/412: symlink d2/d1a/d23/d35/d1d/l87 0 2024-02-16T22:21:58.263 INFO:tasks.workunit.client.1.smithi123.stdout:9/618: write d4/fa1 [5026591,88176] 0 2024-02-16T22:21:58.264 INFO:tasks.workunit.client.1.smithi123.stdout:8/562: symlink d0/df/lb8 0 2024-02-16T22:21:58.267 INFO:tasks.workunit.client.0.smithi114.stdout:1/349: creat d4/d9/d3d/d3f/f75 x:0 0 0 2024-02-16T22:21:58.268 INFO:tasks.workunit.client.1.smithi123.stdout:5/492: mknod d6/dd/d5f/ca4 0 2024-02-16T22:21:58.270 INFO:tasks.workunit.client.0.smithi114.stdout:2/364: unlink dd/df/d2c/f32 0 2024-02-16T22:21:58.272 INFO:tasks.workunit.client.0.smithi114.stdout:6/406: rename d6/d2c/d38/d54/f6b to d6/d2c/d5e/f80 0 2024-02-16T22:21:58.272 INFO:tasks.workunit.client.0.smithi114.stdout:6/407: chown d6/d2c/d75 265697131 1 2024-02-16T22:21:58.273 INFO:tasks.workunit.client.1.smithi123.stdout:1/490: truncate d3/fe 6388054 0 2024-02-16T22:21:58.274 INFO:tasks.workunit.client.1.smithi123.stdout:1/491: dread - d3/d6/db/d10/d4b/d70/f94 zero size 2024-02-16T22:21:58.274 INFO:tasks.workunit.client.0.smithi114.stdout:3/335: mkdir d4/d24/d2d/d45/d74 0 2024-02-16T22:21:58.275 INFO:tasks.workunit.client.1.smithi123.stdout:6/524: fdatasync d0/d6/f8 0 2024-02-16T22:21:58.276 INFO:tasks.workunit.client.0.smithi114.stdout:0/357: link d1/d16/d18/f22 d1/d16/d1f/d28/d51/d61/d6d/d6e/f6f 0 2024-02-16T22:21:58.278 INFO:tasks.workunit.client.1.smithi123.stdout:0/471: creat d1/d4/d17/d54/d8e/d96/f9e x:0 0 0 2024-02-16T22:21:58.279 INFO:tasks.workunit.client.0.smithi114.stdout:4/512: symlink dd/d2e/d56/lb7 0 2024-02-16T22:21:58.281 INFO:tasks.workunit.client.0.smithi114.stdout:9/413: unlink d2/c18 0 2024-02-16T22:21:58.283 INFO:tasks.workunit.client.1.smithi123.stdout:9/619: symlink d4/d36/d8e/d84/lc9 0 2024-02-16T22:21:58.283 INFO:tasks.workunit.client.1.smithi123.stdout:9/620: chown d4/d36/d8e/d84/d93/cb5 0 1 2024-02-16T22:21:58.288 INFO:tasks.workunit.client.0.smithi114.stdout:0/358: dread d1/d16/f24 [0,4194304] 0 2024-02-16T22:21:58.290 INFO:tasks.workunit.client.0.smithi114.stdout:2/365: rmdir dd/d4e 39 2024-02-16T22:21:58.292 INFO:tasks.workunit.client.1.smithi123.stdout:4/552: sync 2024-02-16T22:21:58.293 INFO:tasks.workunit.client.1.smithi123.stdout:4/553: fsync d5/d1c/d42/f4a 0 2024-02-16T22:21:58.294 INFO:tasks.workunit.client.1.smithi123.stdout:2/505: creat d4/fa1 x:0 0 0 2024-02-16T22:21:58.296 INFO:tasks.workunit.client.0.smithi114.stdout:6/408: creat d6/dd/d25/d34/d58/f81 x:0 0 0 2024-02-16T22:21:58.297 INFO:tasks.workunit.client.0.smithi114.stdout:6/409: dread - d6/d2c/d66/f68 zero size 2024-02-16T22:21:58.297 INFO:tasks.workunit.client.1.smithi123.stdout:5/493: mkdir d6/dd/d78/da5 0 2024-02-16T22:21:58.300 INFO:tasks.workunit.client.0.smithi114.stdout:3/336: mkdir d4/d46/d48/d75 0 2024-02-16T22:21:58.301 INFO:tasks.workunit.client.1.smithi123.stdout:7/527: getdents d3/d6e/d4d 0 2024-02-16T22:21:58.301 INFO:tasks.workunit.client.0.smithi114.stdout:3/337: write d4/dd/d33/f5f [662097,20062] 0 2024-02-16T22:21:58.301 INFO:tasks.workunit.client.0.smithi114.stdout:1/350: write d4/f36 [1175208,102157] 0 2024-02-16T22:21:58.302 INFO:tasks.workunit.client.0.smithi114.stdout:6/410: dread d6/dd/f17 [0,4194304] 0 2024-02-16T22:21:58.303 INFO:tasks.workunit.client.1.smithi123.stdout:3/516: truncate d0/d40/f56 487151 0 2024-02-16T22:21:58.304 INFO:tasks.workunit.client.1.smithi123.stdout:3/517: dread d0/d20/f42 [0,4194304] 0 2024-02-16T22:21:58.306 INFO:tasks.workunit.client.1.smithi123.stdout:1/492: mknod d3/d6/db/d10/d2a/ca6 0 2024-02-16T22:21:58.308 INFO:tasks.workunit.client.1.smithi123.stdout:1/493: write d3/d6/db/d10/d4b/d70/f79 [2383912,84230] 0 2024-02-16T22:21:58.310 INFO:tasks.workunit.client.1.smithi123.stdout:3/518: dwrite d0/d20/d76/d5d/d60/f96 [0,4194304] 0 2024-02-16T22:21:58.312 INFO:tasks.workunit.client.0.smithi114.stdout:2/366: symlink dd/df/d1c/d21/l7d 0 2024-02-16T22:21:58.314 INFO:tasks.workunit.client.1.smithi123.stdout:9/621: creat d4/d36/d8e/d6d/d71/d92/fca x:0 0 0 2024-02-16T22:21:58.315 INFO:tasks.workunit.client.1.smithi123.stdout:9/622: write d4/f85 [312745,48588] 0 2024-02-16T22:21:58.318 INFO:tasks.workunit.client.1.smithi123.stdout:4/554: symlink d5/d1c/d23/d56/d6e/d73/d87/laf 0 2024-02-16T22:21:58.318 INFO:tasks.workunit.client.1.smithi123.stdout:3/519: dwrite d0/d20/f26 [0,4194304] 0 2024-02-16T22:21:58.320 INFO:tasks.workunit.client.0.smithi114.stdout:6/411: write d6/d2c/d5e/f80 [967917,91445] 0 2024-02-16T22:21:58.321 INFO:tasks.workunit.client.1.smithi123.stdout:5/494: symlink d6/dd/df/d90/la6 0 2024-02-16T22:21:58.322 INFO:tasks.workunit.client.1.smithi123.stdout:5/495: dread - d6/dd/df/f6c zero size 2024-02-16T22:21:58.322 INFO:tasks.workunit.client.0.smithi114.stdout:4/513: creat dd/d1b/db0/fb8 x:0 0 0 2024-02-16T22:21:58.323 INFO:tasks.workunit.client.0.smithi114.stdout:4/514: chown dd/d1b/d37/d99/lac 31758248 1 2024-02-16T22:21:58.325 INFO:tasks.workunit.client.1.smithi123.stdout:7/528: mknod d3/d6e/d4d/cad 0 2024-02-16T22:21:58.325 INFO:tasks.workunit.client.0.smithi114.stdout:7/432: link d3/d22/d76/l7d d3/d22/l91 0 2024-02-16T22:21:58.328 INFO:tasks.workunit.client.0.smithi114.stdout:6/412: dread d6/dd/d25/f27 [0,4194304] 0 2024-02-16T22:21:58.329 INFO:tasks.workunit.client.0.smithi114.stdout:4/515: dread dd/d1b/d24/f3d [0,4194304] 0 2024-02-16T22:21:58.330 INFO:tasks.workunit.client.1.smithi123.stdout:7/529: dwrite f2 [0,4194304] 0 2024-02-16T22:21:58.330 INFO:tasks.workunit.client.0.smithi114.stdout:3/338: truncate d4/d24/f25 654960 0 2024-02-16T22:21:58.332 INFO:tasks.workunit.client.1.smithi123.stdout:1/494: symlink d3/d6/db/d10/d2a/d53/la7 0 2024-02-16T22:21:58.335 INFO:tasks.workunit.client.1.smithi123.stdout:0/472: rename d1/d8/d13/d1d/c83 to d1/d8/d13/d1d/d64/c9f 0 2024-02-16T22:21:58.335 INFO:tasks.workunit.client.0.smithi114.stdout:6/413: dwrite d6/da/f1b [0,4194304] 0 2024-02-16T22:21:58.338 INFO:tasks.workunit.client.0.smithi114.stdout:2/367: symlink dd/d4e/l7e 0 2024-02-16T22:21:58.340 INFO:tasks.workunit.client.1.smithi123.stdout:8/563: rmdir d0/d29/da7/d26/d2a/d59 0 2024-02-16T22:21:58.343 INFO:tasks.workunit.client.0.smithi114.stdout:1/351: link d4/d9/d46/d4d/f5c d4/d9/d3d/d3f/f76 0 2024-02-16T22:21:58.347 INFO:tasks.workunit.client.0.smithi114.stdout:9/414: getdents d2/d1a/d23/d35/d1d 0 2024-02-16T22:21:58.348 INFO:tasks.workunit.client.1.smithi123.stdout:4/555: truncate d5/f1e 4598796 0 2024-02-16T22:21:58.350 INFO:tasks.workunit.client.0.smithi114.stdout:6/414: creat d6/d2c/d75/f82 x:0 0 0 2024-02-16T22:21:58.350 INFO:tasks.workunit.client.1.smithi123.stdout:5/496: creat d6/d3b/fa7 x:0 0 0 2024-02-16T22:21:58.352 INFO:tasks.workunit.client.0.smithi114.stdout:7/433: mkdir d3/d92 0 2024-02-16T22:21:58.353 INFO:tasks.workunit.client.1.smithi123.stdout:7/530: truncate d3/d6e/f14 1222147 0 2024-02-16T22:21:58.357 INFO:tasks.workunit.client.1.smithi123.stdout:1/495: mknod d3/d6/d11/ca8 0 2024-02-16T22:21:58.358 INFO:tasks.workunit.client.1.smithi123.stdout:9/623: dread d4/d36/f78 [0,4194304] 0 2024-02-16T22:21:58.367 INFO:tasks.workunit.client.0.smithi114.stdout:2/368: dread dd/f5d [0,4194304] 0 2024-02-16T22:21:58.368 INFO:tasks.workunit.client.0.smithi114.stdout:1/352: creat d4/d33/d74/f77 x:0 0 0 2024-02-16T22:21:58.369 INFO:tasks.workunit.client.1.smithi123.stdout:8/564: rmdir d0/d29/da7 39 2024-02-16T22:21:58.370 INFO:tasks.workunit.client.0.smithi114.stdout:9/415: dread d2/d5c/d5f/d72/f79 [0,4194304] 0 2024-02-16T22:21:58.371 INFO:tasks.workunit.client.0.smithi114.stdout:9/416: write d2/d1a/f39 [1154851,10650] 0 2024-02-16T22:21:58.371 INFO:tasks.workunit.client.0.smithi114.stdout:9/417: chown d2/f44 661 1 2024-02-16T22:21:58.375 INFO:tasks.workunit.client.1.smithi123.stdout:6/525: getdents d0/d6/d2f/da5 0 2024-02-16T22:21:58.378 INFO:tasks.workunit.client.1.smithi123.stdout:3/520: rename d0/d20/d76/d37/c6d to d0/da0/cb7 0 2024-02-16T22:21:58.380 INFO:tasks.workunit.client.1.smithi123.stdout:0/473: mkdir d1/da0 0 2024-02-16T22:21:58.381 INFO:tasks.workunit.client.1.smithi123.stdout:0/474: read - d1/d8/d13/d1d/d62/d7a/f9c zero size 2024-02-16T22:21:58.381 INFO:tasks.workunit.client.1.smithi123.stdout:0/475: dread - d1/d8/d13/d1d/f97 zero size 2024-02-16T22:21:58.385 INFO:tasks.workunit.client.1.smithi123.stdout:8/565: write d0/f6 [1890573,117680] 0 2024-02-16T22:21:58.386 INFO:tasks.workunit.client.1.smithi123.stdout:4/556: write d5/d1c/d23/d31/f2d [2914478,115111] 0 2024-02-16T22:21:58.387 INFO:tasks.workunit.client.1.smithi123.stdout:5/497: symlink d6/dd/df/d9f/la8 0 2024-02-16T22:21:58.387 INFO:tasks.workunit.client.1.smithi123.stdout:2/506: write d4/d1a/d1d/d13/d25/f9a [645889,12799] 0 2024-02-16T22:21:58.389 INFO:tasks.workunit.client.1.smithi123.stdout:7/531: unlink d3/d6e/d1d/f80 0 2024-02-16T22:21:58.389 INFO:tasks.workunit.client.1.smithi123.stdout:5/498: write d6/dd/df/d58/d3f/d6f/f91 [121586,92167] 0 2024-02-16T22:21:58.392 INFO:tasks.workunit.client.1.smithi123.stdout:9/624: mkdir d4/d36/daa/d20/dcb 0 2024-02-16T22:21:58.394 INFO:tasks.workunit.client.1.smithi123.stdout:1/496: rename d3/d6/db/l54 to d3/d6/db/d55/la9 0 2024-02-16T22:21:58.395 INFO:tasks.workunit.client.1.smithi123.stdout:3/521: creat d0/d20/d2e/fb8 x:0 0 0 2024-02-16T22:21:58.396 INFO:tasks.workunit.client.1.smithi123.stdout:3/522: write d0/d20/d76/f36 [4724901,89327] 0 2024-02-16T22:21:58.399 INFO:tasks.workunit.client.0.smithi114.stdout:7/434: mknod d3/d92/c93 0 2024-02-16T22:21:58.399 INFO:tasks.workunit.client.0.smithi114.stdout:7/435: fsync d3/d23/d4e/f6e 0 2024-02-16T22:21:58.401 INFO:tasks.workunit.client.0.smithi114.stdout:6/415: mkdir d6/d83 0 2024-02-16T22:21:58.404 INFO:tasks.workunit.client.1.smithi123.stdout:5/499: symlink d6/dd/df/d9f/la9 0 2024-02-16T22:21:58.405 INFO:tasks.workunit.client.1.smithi123.stdout:9/625: stat d4/d36/d8e/d6d/cba 0 2024-02-16T22:21:58.405 INFO:tasks.workunit.client.1.smithi123.stdout:9/626: read - d4/d1e/d3a/fb7 zero size 2024-02-16T22:21:58.406 INFO:tasks.workunit.client.0.smithi114.stdout:7/436: dwrite d3/d23/f6b [0,4194304] 0 2024-02-16T22:21:58.408 INFO:tasks.workunit.client.0.smithi114.stdout:7/437: read d3/d23/d4e/f7f [761644,28111] 0 2024-02-16T22:21:58.409 INFO:tasks.workunit.client.0.smithi114.stdout:7/438: chown d3/d23/l32 3438 1 2024-02-16T22:21:58.410 INFO:tasks.workunit.client.1.smithi123.stdout:6/526: rename d0/d6/d1a/d88/fb7 to d0/d96/fbf 0 2024-02-16T22:21:58.410 INFO:tasks.workunit.client.0.smithi114.stdout:9/418: mknod d2/d1a/d23/d32/c88 0 2024-02-16T22:21:58.411 INFO:tasks.workunit.client.0.smithi114.stdout:1/353: dwrite d4/d9/d3d/d3f/f5a [0,4194304] 0 2024-02-16T22:21:58.413 INFO:tasks.workunit.client.1.smithi123.stdout:1/497: dread d3/d6/d49/f66 [0,4194304] 0 2024-02-16T22:21:58.413 INFO:tasks.workunit.client.0.smithi114.stdout:1/354: dread d4/d9/d46/d47/f73 [0,4194304] 0 2024-02-16T22:21:58.543 INFO:tasks.workunit.client.0.smithi114.stdout:6/416: rmdir d6/d2c/d38/d41/d60 39 2024-02-16T22:21:58.551 INFO:tasks.workunit.client.0.smithi114.stdout:9/419: mkdir d2/dd/d3c/d89 0 2024-02-16T22:21:58.552 INFO:tasks.workunit.client.1.smithi123.stdout:5/500: read d6/dd/df/d58/f63 [33422,67297] 0 2024-02-16T22:21:58.552 INFO:tasks.workunit.client.0.smithi114.stdout:1/355: creat d4/d9/d46/d4d/f78 x:0 0 0 2024-02-16T22:21:58.557 INFO:tasks.workunit.client.0.smithi114.stdout:2/369: rename dd/d3b/d7b to dd/d3c/d7f 0 2024-02-16T22:21:58.560 INFO:tasks.workunit.client.1.smithi123.stdout:7/532: rename d3/d6e/d1d/c44 to d3/cae 0 2024-02-16T22:21:58.561 INFO:tasks.workunit.client.1.smithi123.stdout:7/533: dread d3/d5/f45 [0,4194304] 0 2024-02-16T22:21:58.563 INFO:tasks.workunit.client.0.smithi114.stdout:1/356: mkdir d4/d9/d3d/d3f/d79 0 2024-02-16T22:21:58.564 INFO:tasks.workunit.client.1.smithi123.stdout:0/476: link d1/d4/d17/d74/l5e d1/d8/d3d/la1 0 2024-02-16T22:21:58.565 INFO:tasks.workunit.client.0.smithi114.stdout:6/417: mkdir d6/d2c/d38/d41/d60/d84 0 2024-02-16T22:21:58.566 INFO:tasks.workunit.client.1.smithi123.stdout:3/523: creat d0/d20/d76/d5d/d60/d93/fb9 x:0 0 0 2024-02-16T22:21:58.566 INFO:tasks.workunit.client.0.smithi114.stdout:6/418: truncate d6/dd/d25/f4c 798065 0 2024-02-16T22:21:58.567 INFO:tasks.workunit.client.1.smithi123.stdout:2/507: truncate d4/d1a/d1d/d13/d25/f3b 834283 0 2024-02-16T22:21:58.567 INFO:tasks.workunit.client.1.smithi123.stdout:8/566: symlink d0/d29/da7/d26/d2a/d3c/d62/lb9 0 2024-02-16T22:21:58.568 INFO:tasks.workunit.client.1.smithi123.stdout:2/508: dread d4/d1a/f86 [0,4194304] 0 2024-02-16T22:21:58.569 INFO:tasks.workunit.client.0.smithi114.stdout:7/439: rename d3/d14/f4c to d3/d22/d61/d6a/f94 0 2024-02-16T22:21:58.569 INFO:tasks.workunit.client.0.smithi114.stdout:9/420: rename d2/d33 to d2/d33/d8a 22 2024-02-16T22:21:58.570 INFO:tasks.workunit.client.0.smithi114.stdout:9/421: stat d2/d33/d3f/c4c 0 2024-02-16T22:21:58.571 INFO:tasks.workunit.client.1.smithi123.stdout:4/557: link d5/d1c/d64/f8a d5/d49/fb0 0 2024-02-16T22:21:58.571 INFO:tasks.workunit.client.1.smithi123.stdout:9/627: write d4/f10 [3055402,48730] 0 2024-02-16T22:21:58.575 INFO:tasks.workunit.client.1.smithi123.stdout:8/567: dread d0/d29/da7/d26/f7a [0,4194304] 0 2024-02-16T22:21:58.575 INFO:tasks.workunit.client.1.smithi123.stdout:5/501: symlink d6/dd/d44/laa 0 2024-02-16T22:21:58.576 INFO:tasks.workunit.client.1.smithi123.stdout:8/568: fdatasync d0/d29/da7/d26/d32/faf 0 2024-02-16T22:21:58.577 INFO:tasks.workunit.client.0.smithi114.stdout:2/370: rename dd/d4e/f5e to dd/d3c/d4c/f80 0 2024-02-16T22:21:58.578 INFO:tasks.workunit.client.1.smithi123.stdout:4/558: dwrite d5/f90 [0,4194304] 0 2024-02-16T22:21:58.578 INFO:tasks.workunit.client.1.smithi123.stdout:4/559: truncate d5/d1c/d42/f75 25582 0 2024-02-16T22:21:58.580 INFO:tasks.workunit.client.1.smithi123.stdout:6/527: rename d0/f65 to d0/d6/d8e/fc0 0 2024-02-16T22:21:58.582 INFO:tasks.workunit.client.1.smithi123.stdout:6/528: dread d0/d9/f62 [0,4194304] 0 2024-02-16T22:21:58.582 INFO:tasks.workunit.client.1.smithi123.stdout:6/529: readlink d0/d6/d2f/d5d/l99 0 2024-02-16T22:21:58.587 INFO:tasks.workunit.client.0.smithi114.stdout:7/440: dread d3/d14/f5c [0,4194304] 0 2024-02-16T22:21:58.588 INFO:tasks.workunit.client.1.smithi123.stdout:3/524: truncate d0/d20/d29/f43 454635 0 2024-02-16T22:21:58.588 INFO:tasks.workunit.client.0.smithi114.stdout:7/441: write d3/d23/f5e [636829,103335] 0 2024-02-16T22:21:58.590 INFO:tasks.workunit.client.0.smithi114.stdout:2/371: mknod dd/df/d1c/d21/d61/d63/c81 0 2024-02-16T22:21:58.592 INFO:tasks.workunit.client.0.smithi114.stdout:9/422: mkdir d2/dd/d3c/d89/d8b 0 2024-02-16T22:21:58.593 INFO:tasks.workunit.client.0.smithi114.stdout:9/423: write d2/f44 [3006924,40726] 0 2024-02-16T22:21:58.595 INFO:tasks.workunit.client.1.smithi123.stdout:9/628: chown d4/d36/daa/d2b/f64 413 1 2024-02-16T22:21:58.598 INFO:tasks.workunit.client.0.smithi114.stdout:9/424: write d2/d1a/d23/d35/d47/f4d [1917547,8284] 0 2024-02-16T22:21:58.598 INFO:tasks.workunit.client.0.smithi114.stdout:2/372: chown dd/ce 35855753 1 2024-02-16T22:21:58.600 INFO:tasks.workunit.client.0.smithi114.stdout:1/357: dwrite d4/f2d [0,4194304] 0 2024-02-16T22:21:58.601 INFO:tasks.workunit.client.0.smithi114.stdout:6/419: dwrite d6/f18 [0,4194304] 0 2024-02-16T22:21:58.602 INFO:tasks.workunit.client.1.smithi123.stdout:0/477: write d1/d8/f2f [5065689,68124] 0 2024-02-16T22:21:58.610 INFO:tasks.workunit.client.0.smithi114.stdout:2/373: dwrite dd/df/d1c/d21/d61/d63/f73 [0,4194304] 0 2024-02-16T22:21:58.615 INFO:tasks.workunit.client.0.smithi114.stdout:1/358: mknod d4/d9/d46/d47/c7a 0 2024-02-16T22:21:58.619 INFO:tasks.workunit.client.0.smithi114.stdout:2/374: dwrite dd/df/f1e [0,4194304] 0 2024-02-16T22:21:58.620 INFO:tasks.workunit.client.0.smithi114.stdout:2/375: dread - dd/f7a zero size 2024-02-16T22:21:58.621 INFO:tasks.workunit.client.0.smithi114.stdout:7/442: getdents d3/d23/d4e 0 2024-02-16T22:21:58.623 INFO:tasks.workunit.client.1.smithi123.stdout:4/560: write d5/d1c/d23/d74/f55 [4694294,75502] 0 2024-02-16T22:21:58.627 INFO:tasks.workunit.client.1.smithi123.stdout:3/525: creat d0/da0/fba x:0 0 0 2024-02-16T22:21:58.628 INFO:tasks.workunit.client.0.smithi114.stdout:2/376: symlink dd/d3b/l82 0 2024-02-16T22:21:58.629 INFO:tasks.workunit.client.1.smithi123.stdout:0/478: dread d1/d8/f34 [0,4194304] 0 2024-02-16T22:21:58.634 INFO:tasks.workunit.client.1.smithi123.stdout:5/502: mknod d6/dd/df/cab 0 2024-02-16T22:21:58.634 INFO:tasks.workunit.client.1.smithi123.stdout:6/530: dread d0/d6/f18 [0,4194304] 0 2024-02-16T22:21:58.635 INFO:tasks.workunit.client.1.smithi123.stdout:8/569: unlink d0/d29/da7/d26/d68/d81/f9e 0 2024-02-16T22:21:58.636 INFO:tasks.workunit.client.1.smithi123.stdout:1/498: getdents d3/d6/d18 0 2024-02-16T22:21:58.636 INFO:tasks.workunit.client.1.smithi123.stdout:5/503: readlink d6/dd/d5f/l94 0 2024-02-16T22:21:58.636 INFO:tasks.workunit.client.1.smithi123.stdout:3/526: dwrite d0/f12 [0,4194304] 0 2024-02-16T22:21:58.637 INFO:tasks.workunit.client.1.smithi123.stdout:7/534: link d3/d5/f45 d3/d87/da8/faf 0 2024-02-16T22:21:58.637 INFO:tasks.workunit.client.1.smithi123.stdout:8/570: chown d0/d12/d3e/f49 11714 1 2024-02-16T22:21:58.637 INFO:tasks.workunit.client.0.smithi114.stdout:9/425: write d2/d1a/d23/d35/d1d/f36 [3242563,25945] 0 2024-02-16T22:21:58.639 INFO:tasks.workunit.client.1.smithi123.stdout:9/629: mknod d4/d36/d8e/dbe/ccc 0 2024-02-16T22:21:58.640 INFO:tasks.workunit.client.1.smithi123.stdout:4/561: dread d5/d1c/d23/d31/f2d [0,4194304] 0 2024-02-16T22:21:58.640 INFO:tasks.workunit.client.1.smithi123.stdout:3/527: read d0/d20/d76/d37/f5e [169182,113988] 0 2024-02-16T22:21:58.642 INFO:tasks.workunit.client.1.smithi123.stdout:7/535: creat d3/d6e/d4d/d92/fb0 x:0 0 0 2024-02-16T22:21:58.643 INFO:tasks.workunit.client.0.smithi114.stdout:9/426: unlink d2/d1a/d23/d32/l80 0 2024-02-16T22:21:58.647 INFO:tasks.workunit.client.1.smithi123.stdout:5/504: unlink d6/dd/df/d58/d3f/l45 0 2024-02-16T22:21:58.648 INFO:tasks.workunit.client.0.smithi114.stdout:9/427: dread - d2/dd/f2f zero size 2024-02-16T22:21:58.652 INFO:tasks.workunit.client.1.smithi123.stdout:6/531: creat d0/d9/dc/d5f/fc1 x:0 0 0 2024-02-16T22:21:58.653 INFO:tasks.workunit.client.0.smithi114.stdout:1/359: dread d4/f16 [4194304,4194304] 0 2024-02-16T22:21:58.653 INFO:tasks.workunit.client.1.smithi123.stdout:0/479: symlink d1/la2 0 2024-02-16T22:21:58.655 INFO:tasks.workunit.client.0.smithi114.stdout:9/428: creat d2/d5c/d5f/f8c x:0 0 0 2024-02-16T22:21:58.657 INFO:tasks.workunit.client.1.smithi123.stdout:8/571: mkdir d0/d29/dba 0 2024-02-16T22:21:58.658 INFO:tasks.workunit.client.0.smithi114.stdout:2/377: dread dd/df/f44 [0,4194304] 0 2024-02-16T22:21:58.658 INFO:tasks.workunit.client.1.smithi123.stdout:9/630: creat d4/d36/d8e/d6d/d71/d92/fcd x:0 0 0 2024-02-16T22:21:58.660 INFO:tasks.workunit.client.1.smithi123.stdout:9/631: write d4/f26 [3263285,92419] 0 2024-02-16T22:21:58.661 INFO:tasks.workunit.client.1.smithi123.stdout:4/562: creat d5/d1c/d64/d7b/fb1 x:0 0 0 2024-02-16T22:21:58.662 INFO:tasks.workunit.client.1.smithi123.stdout:9/632: chown d4/f10 1057731098 1 2024-02-16T22:21:58.662 INFO:tasks.workunit.client.0.smithi114.stdout:1/360: mkdir d4/d9/d3d/d3f/d79/d7b 0 2024-02-16T22:21:58.664 INFO:tasks.workunit.client.1.smithi123.stdout:2/509: dwrite d4/f54 [0,4194304] 0 2024-02-16T22:21:58.665 INFO:tasks.workunit.client.0.smithi114.stdout:2/378: dwrite dd/d3b/f5c [0,4194304] 0 2024-02-16T22:21:58.668 INFO:tasks.workunit.client.1.smithi123.stdout:2/510: dread d4/d1a/d1d/d38/f90 [0,4194304] 0 2024-02-16T22:21:58.671 INFO:tasks.workunit.client.1.smithi123.stdout:8/572: dread d0/d29/da7/f35 [0,4194304] 0 2024-02-16T22:21:58.672 INFO:tasks.workunit.client.1.smithi123.stdout:2/511: dread d4/d1a/d1d/d38/f58 [0,4194304] 0 2024-02-16T22:21:58.674 INFO:tasks.workunit.client.1.smithi123.stdout:8/573: truncate d0/d29/da7/d26/d2a/d3c/fb1 949642 0 2024-02-16T22:21:58.674 INFO:tasks.workunit.client.0.smithi114.stdout:1/361: truncate d4/d9/d3d/f54 66026 0 2024-02-16T22:21:58.676 INFO:tasks.workunit.client.0.smithi114.stdout:9/429: dread d2/d1a/d23/d35/d1d/f54 [0,4194304] 0 2024-02-16T22:21:58.676 INFO:tasks.workunit.client.1.smithi123.stdout:8/574: write d0/d29/da7/d26/d68/d81/d47/f9d [371035,118266] 0 2024-02-16T22:21:58.678 INFO:tasks.workunit.client.1.smithi123.stdout:8/575: chown d0/d29/da7/d26/d2a/d77/f84 122 1 2024-02-16T22:21:58.683 INFO:tasks.workunit.client.0.smithi114.stdout:9/430: rmdir d2/d5c/d5f/d72 39 2024-02-16T22:21:58.684 INFO:tasks.workunit.client.1.smithi123.stdout:8/576: dwrite d0/d29/da7/d43/f28 [4194304,4194304] 0 2024-02-16T22:21:58.684 INFO:tasks.workunit.client.0.smithi114.stdout:9/431: write d2/d33/f7d [430453,84828] 0 2024-02-16T22:21:58.687 INFO:tasks.workunit.client.0.smithi114.stdout:1/362: unlink d4/c6 0 2024-02-16T22:21:58.689 INFO:tasks.workunit.client.0.smithi114.stdout:5/316: sync 2024-02-16T22:21:58.697 INFO:tasks.workunit.client.0.smithi114.stdout:1/363: unlink d4/d9/l59 0 2024-02-16T22:21:58.699 INFO:tasks.workunit.client.0.smithi114.stdout:2/379: dread dd/df/f2f [0,4194304] 0 2024-02-16T22:21:58.700 INFO:tasks.workunit.client.0.smithi114.stdout:5/317: creat d0/df/f57 x:0 0 0 2024-02-16T22:21:58.703 INFO:tasks.workunit.client.0.smithi114.stdout:1/364: mkdir d4/d63/d69/d7c 0 2024-02-16T22:21:58.704 INFO:tasks.workunit.client.1.smithi123.stdout:7/536: dwrite d3/d6e/d52/f6a [0,4194304] 0 2024-02-16T22:21:58.705 INFO:tasks.workunit.client.0.smithi114.stdout:2/380: mkdir dd/df/d1c/d21/d61/d63/d83 0 2024-02-16T22:21:58.711 INFO:tasks.workunit.client.1.smithi123.stdout:6/532: creat d0/d9/dc/d39/fc2 x:0 0 0 2024-02-16T22:21:58.715 INFO:tasks.workunit.client.1.smithi123.stdout:0/480: mkdir d1/d4/d17/d74/d57/da3 0 2024-02-16T22:21:58.719 INFO:tasks.workunit.client.0.smithi114.stdout:5/318: dread d0/f22 [0,4194304] 0 2024-02-16T22:21:58.720 INFO:tasks.workunit.client.1.smithi123.stdout:4/563: creat d5/d1c/d23/d47/da2/fb2 x:0 0 0 2024-02-16T22:21:58.720 INFO:tasks.workunit.client.1.smithi123.stdout:4/564: dread - d5/f63 zero size 2024-02-16T22:21:58.722 INFO:tasks.workunit.client.1.smithi123.stdout:9/633: symlink d4/d36/d8e/d6f/lce 0 2024-02-16T22:21:58.723 INFO:tasks.workunit.client.1.smithi123.stdout:9/634: write d4/d36/d57/db2/f69 [1406012,2226] 0 2024-02-16T22:21:58.727 INFO:tasks.workunit.client.1.smithi123.stdout:3/528: sync 2024-02-16T22:21:58.727 INFO:tasks.workunit.client.0.smithi114.stdout:9/432: dwrite d2/d1a/d23/d35/d1d/f3a [0,4194304] 0 2024-02-16T22:21:58.727 INFO:tasks.workunit.client.0.smithi114.stdout:9/433: fdatasync d2/dd/f2a 0 2024-02-16T22:21:58.728 INFO:tasks.workunit.client.0.smithi114.stdout:9/434: fsync f1 0 2024-02-16T22:21:58.728 INFO:tasks.workunit.client.0.smithi114.stdout:5/319: symlink d0/df/d16/d25/l58 0 2024-02-16T22:21:58.729 INFO:tasks.workunit.client.0.smithi114.stdout:5/320: read - d0/df/d16/d25/d3e/f44 zero size 2024-02-16T22:21:58.736 INFO:tasks.workunit.client.0.smithi114.stdout:9/435: creat d2/d5c/f8d x:0 0 0 2024-02-16T22:21:58.737 INFO:tasks.workunit.client.0.smithi114.stdout:2/381: write dd/df/d2c/f34 [241221,101699] 0 2024-02-16T22:21:58.738 INFO:tasks.workunit.client.0.smithi114.stdout:5/321: creat d0/df/d16/d25/d2e/d3b/f59 x:0 0 0 2024-02-16T22:21:58.740 INFO:tasks.workunit.client.0.smithi114.stdout:9/436: mknod d2/dd/d3c/c8e 0 2024-02-16T22:21:58.741 INFO:tasks.workunit.client.0.smithi114.stdout:1/365: truncate d4/f21 7909222 0 2024-02-16T22:21:58.742 INFO:tasks.workunit.client.0.smithi114.stdout:9/437: write d2/d5c/d5f/f8c [477471,18543] 0 2024-02-16T22:21:58.742 INFO:tasks.workunit.client.1.smithi123.stdout:7/537: mkdir d3/d26/d3d/db1 0 2024-02-16T22:21:58.747 INFO:tasks.workunit.client.1.smithi123.stdout:5/505: mkdir d6/dac 0 2024-02-16T22:21:58.748 INFO:tasks.workunit.client.1.smithi123.stdout:6/533: creat d0/d9/fc3 x:0 0 0 2024-02-16T22:21:58.752 INFO:tasks.workunit.client.0.smithi114.stdout:9/438: creat d2/d1a/d65/f8f x:0 0 0 2024-02-16T22:21:58.755 INFO:tasks.workunit.client.0.smithi114.stdout:5/322: mknod d0/df/d16/c5a 0 2024-02-16T22:21:58.755 INFO:tasks.workunit.client.1.smithi123.stdout:4/565: symlink d5/d1c/d23/d74/lb3 0 2024-02-16T22:21:58.755 INFO:tasks.workunit.client.1.smithi123.stdout:4/566: write d5/d49/f91 [2656718,45704] 0 2024-02-16T22:21:58.757 INFO:tasks.workunit.client.0.smithi114.stdout:1/366: chown d4/d9/d46/d4d/f5b 106 1 2024-02-16T22:21:58.758 INFO:tasks.workunit.client.1.smithi123.stdout:9/635: creat d4/d1e/d3a/fcf x:0 0 0 2024-02-16T22:21:58.760 INFO:tasks.workunit.client.1.smithi123.stdout:3/529: creat d0/d20/d76/d5d/d60/d93/fbb x:0 0 0 2024-02-16T22:21:58.760 INFO:tasks.workunit.client.1.smithi123.stdout:3/530: readlink d0/l1d 0 2024-02-16T22:21:58.762 INFO:tasks.workunit.client.0.smithi114.stdout:9/439: creat d2/d1a/d23/d32/d46/f90 x:0 0 0 2024-02-16T22:21:58.765 INFO:tasks.workunit.client.0.smithi114.stdout:5/323: fsync d0/f8 0 2024-02-16T22:21:58.772 INFO:tasks.workunit.client.0.smithi114.stdout:9/440: dwrite d2/d5c/d5f/d7b/f64 [0,4194304] 0 2024-02-16T22:21:58.773 INFO:tasks.workunit.client.1.smithi123.stdout:1/499: getdents d3/d6/d11/d8b 0 2024-02-16T22:21:58.773 INFO:tasks.workunit.client.1.smithi123.stdout:9/636: dread d4/d36/d8e/f95 [0,4194304] 0 2024-02-16T22:21:58.774 INFO:tasks.workunit.client.0.smithi114.stdout:2/382: dwrite dd/f17 [0,4194304] 0 2024-02-16T22:21:58.781 INFO:tasks.workunit.client.0.smithi114.stdout:1/367: creat d4/d9/d46/d4d/f7d x:0 0 0 2024-02-16T22:21:58.783 INFO:tasks.workunit.client.0.smithi114.stdout:9/441: dread d2/d1a/d23/d35/d47/d63/f6a [0,4194304] 0 2024-02-16T22:21:58.784 INFO:tasks.workunit.client.1.smithi123.stdout:6/534: unlink d0/d6/d4d/f9b 0 2024-02-16T22:21:58.784 INFO:tasks.workunit.client.0.smithi114.stdout:9/442: chown d2/dd/d3c/l6f 67 1 2024-02-16T22:21:58.785 INFO:tasks.workunit.client.0.smithi114.stdout:9/443: stat d2/d1a/d23/d35/d1d/f54 0 2024-02-16T22:21:58.789 INFO:tasks.workunit.client.0.smithi114.stdout:1/368: mknod d4/d9/d46/d67/c7e 0 2024-02-16T22:21:58.791 INFO:tasks.workunit.client.0.smithi114.stdout:5/324: dread d0/df/d16/d1e/f21 [0,4194304] 0 2024-02-16T22:21:58.792 INFO:tasks.workunit.client.1.smithi123.stdout:3/531: creat d0/d20/d94/fbc x:0 0 0 2024-02-16T22:21:58.794 INFO:tasks.workunit.client.0.smithi114.stdout:9/444: rmdir d2/d1a 39 2024-02-16T22:21:58.795 INFO:tasks.workunit.client.1.smithi123.stdout:7/538: dwrite d3/d6e/d24/f56 [0,4194304] 0 2024-02-16T22:21:58.801 INFO:tasks.workunit.client.0.smithi114.stdout:2/383: dread dd/df/d2c/f5a [0,4194304] 0 2024-02-16T22:21:58.801 INFO:tasks.workunit.client.0.smithi114.stdout:9/445: mkdir d2/d5c/d5f/d91 0 2024-02-16T22:21:58.802 INFO:tasks.workunit.client.0.smithi114.stdout:9/446: chown d2/dd/f2f 32834 1 2024-02-16T22:21:58.803 INFO:tasks.workunit.client.0.smithi114.stdout:2/384: chown dd/df/d2c/f5a 2664 1 2024-02-16T22:21:58.804 INFO:tasks.workunit.client.1.smithi123.stdout:1/500: dread d3/d6/f26 [0,4194304] 0 2024-02-16T22:21:58.805 INFO:tasks.workunit.client.0.smithi114.stdout:5/325: creat d0/df/d16/f5b x:0 0 0 2024-02-16T22:21:58.806 INFO:tasks.workunit.client.1.smithi123.stdout:0/481: write d1/d4/d17/d74/d57/f75 [2162050,121827] 0 2024-02-16T22:21:58.806 INFO:tasks.workunit.client.1.smithi123.stdout:5/506: dwrite d6/dd/d44/f61 [0,4194304] 0 2024-02-16T22:21:58.808 INFO:tasks.workunit.client.0.smithi114.stdout:9/447: rmdir d2/dd/d3c 39 2024-02-16T22:21:58.808 INFO:tasks.workunit.client.1.smithi123.stdout:3/532: mkdir d0/d20/d29/d9b/dbd 0 2024-02-16T22:21:58.811 INFO:tasks.workunit.client.0.smithi114.stdout:2/385: creat dd/d3c/d4c/f84 x:0 0 0 2024-02-16T22:21:58.812 INFO:tasks.workunit.client.1.smithi123.stdout:2/512: getdents d4/d1a 0 2024-02-16T22:21:58.814 INFO:tasks.workunit.client.0.smithi114.stdout:5/326: creat d0/df/d16/d25/d3e/f5c x:0 0 0 2024-02-16T22:21:58.816 INFO:tasks.workunit.client.0.smithi114.stdout:1/369: write d4/d33/f43 [211280,125945] 0 2024-02-16T22:21:58.819 INFO:tasks.workunit.client.0.smithi114.stdout:2/386: unlink dd/d3c/c4f 0 2024-02-16T22:21:58.820 INFO:tasks.workunit.client.1.smithi123.stdout:4/567: truncate d5/d49/f89 3233068 0 2024-02-16T22:21:58.820 INFO:tasks.workunit.client.0.smithi114.stdout:5/327: dwrite d0/d10/f1a [0,4194304] 0 2024-02-16T22:21:58.821 INFO:tasks.workunit.client.1.smithi123.stdout:5/507: dread d6/dd/df/d1f/f68 [0,4194304] 0 2024-02-16T22:21:58.822 INFO:tasks.workunit.client.1.smithi123.stdout:9/637: truncate d4/d1e/fc8 2246445 0 2024-02-16T22:21:58.824 INFO:tasks.workunit.client.0.smithi114.stdout:9/448: creat d2/d1a/d23/d35/d47/f92 x:0 0 0 2024-02-16T22:21:58.825 INFO:tasks.workunit.client.1.smithi123.stdout:6/535: symlink d0/d6/d4d/db0/lc4 0 2024-02-16T22:21:58.825 INFO:tasks.workunit.client.0.smithi114.stdout:9/449: stat d2/d1a/d23/d35 0 2024-02-16T22:21:58.826 INFO:tasks.workunit.client.1.smithi123.stdout:6/536: write d0/d9/dc/d39/f5b [4676347,49770] 0 2024-02-16T22:21:58.827 INFO:tasks.workunit.client.0.smithi114.stdout:2/387: write dd/df/f44 [986803,109464] 0 2024-02-16T22:21:58.828 INFO:tasks.workunit.client.0.smithi114.stdout:2/388: fdatasync dd/d3c/d4c/f84 0 2024-02-16T22:21:58.828 INFO:tasks.workunit.client.1.smithi123.stdout:0/482: creat d1/d4/d17/d54/d8e/fa4 x:0 0 0 2024-02-16T22:21:58.828 INFO:tasks.workunit.client.0.smithi114.stdout:5/328: creat d0/df/f5d x:0 0 0 2024-02-16T22:21:58.829 INFO:tasks.workunit.client.1.smithi123.stdout:0/483: dread - d1/d4/d17/d54/d8e/f78 zero size 2024-02-16T22:21:58.829 INFO:tasks.workunit.client.0.smithi114.stdout:5/329: dread - d0/df/f5d zero size 2024-02-16T22:21:58.830 INFO:tasks.workunit.client.1.smithi123.stdout:3/533: creat d0/d20/d76/d5d/dae/db0/fbe x:0 0 0 2024-02-16T22:21:58.830 INFO:tasks.workunit.client.1.smithi123.stdout:0/484: read - d1/d4/d17/d54/d8e/d96/f9e zero size 2024-02-16T22:21:58.831 INFO:tasks.workunit.client.1.smithi123.stdout:0/485: read - d1/d4/d17/d54/f88 zero size 2024-02-16T22:21:58.831 INFO:tasks.workunit.client.0.smithi114.stdout:9/450: creat d2/d1a/d23/d35/d47/f93 x:0 0 0 2024-02-16T22:21:58.832 INFO:tasks.workunit.client.1.smithi123.stdout:8/577: link d0/la d0/d29/lbb 0 2024-02-16T22:21:58.833 INFO:tasks.workunit.client.0.smithi114.stdout:9/451: dread - d2/dd/f2f zero size 2024-02-16T22:21:58.833 INFO:tasks.workunit.client.0.smithi114.stdout:3/339: sync 2024-02-16T22:21:58.833 INFO:tasks.workunit.client.0.smithi114.stdout:8/376: sync 2024-02-16T22:21:58.833 INFO:tasks.workunit.client.0.smithi114.stdout:7/443: sync 2024-02-16T22:21:58.833 INFO:tasks.workunit.client.0.smithi114.stdout:4/516: sync 2024-02-16T22:21:58.833 INFO:tasks.workunit.client.0.smithi114.stdout:6/420: sync 2024-02-16T22:21:58.834 INFO:tasks.workunit.client.0.smithi114.stdout:0/359: sync 2024-02-16T22:21:58.835 INFO:tasks.workunit.client.1.smithi123.stdout:4/568: mknod d5/d1c/d23/d76/cb4 0 2024-02-16T22:21:58.835 INFO:tasks.workunit.client.1.smithi123.stdout:2/513: dwrite d4/d1a/d1d/f9 [0,4194304] 0 2024-02-16T22:21:58.837 INFO:tasks.workunit.client.0.smithi114.stdout:1/370: rename d4/f15 to d4/d63/d69/f7f 0 2024-02-16T22:21:58.839 INFO:tasks.workunit.client.1.smithi123.stdout:5/508: dread d6/dd/df/d58/d3f/d82/f83 [0,4194304] 0 2024-02-16T22:21:58.840 INFO:tasks.workunit.client.1.smithi123.stdout:5/509: chown d6/dd/df/d58/d3f/d6f/f84 0 1 2024-02-16T22:21:58.840 INFO:tasks.workunit.client.0.smithi114.stdout:9/452: dwrite f1 [0,4194304] 0 2024-02-16T22:21:58.841 INFO:tasks.workunit.client.1.smithi123.stdout:9/638: symlink d4/d36/d8e/dbe/ld0 0 2024-02-16T22:21:58.841 INFO:tasks.workunit.client.1.smithi123.stdout:9/639: stat d4/d36/ca9 0 2024-02-16T22:21:58.846 INFO:tasks.workunit.client.0.smithi114.stdout:9/453: read - d2/d1a/d23/d35/d47/f93 zero size 2024-02-16T22:21:58.848 INFO:tasks.workunit.client.0.smithi114.stdout:7/444: symlink d3/d22/d73/l95 0 2024-02-16T22:21:58.848 INFO:tasks.workunit.client.0.smithi114.stdout:8/377: dwrite d4/d11/d78/d7a/d36/f5d [0,4194304] 0 2024-02-16T22:21:58.848 INFO:tasks.workunit.client.1.smithi123.stdout:9/640: dwrite d4/d36/f5f [4194304,4194304] 0 2024-02-16T22:21:58.852 INFO:tasks.workunit.client.0.smithi114.stdout:4/517: rmdir dd/d1b/d37/d38 39 2024-02-16T22:21:58.856 INFO:tasks.workunit.client.0.smithi114.stdout:3/340: mkdir d4/dd/d33/d76 0 2024-02-16T22:21:58.856 INFO:tasks.workunit.client.0.smithi114.stdout:8/378: dwrite d4/d24/f64 [0,4194304] 0 2024-02-16T22:21:58.859 INFO:tasks.workunit.client.1.smithi123.stdout:0/486: mkdir d1/d4/d17/d54/d8e/da5 0 2024-02-16T22:21:58.866 INFO:tasks.workunit.client.1.smithi123.stdout:7/539: rename d3/d26 to d3/d6e/d1d/db2 0 2024-02-16T22:21:58.866 INFO:tasks.workunit.client.1.smithi123.stdout:0/487: dwrite d1/f1f [4194304,4194304] 0 2024-02-16T22:21:58.867 INFO:tasks.workunit.client.0.smithi114.stdout:7/445: dread d3/d14/f29 [0,4194304] 0 2024-02-16T22:21:58.872 INFO:tasks.workunit.client.1.smithi123.stdout:4/569: chown d5/d1c/l5c 2 1 2024-02-16T22:21:58.873 INFO:tasks.workunit.client.1.smithi123.stdout:4/570: read d5/d1c/d23/d74/f55 [4720917,118705] 0 2024-02-16T22:21:58.873 INFO:tasks.workunit.client.0.smithi114.stdout:6/421: creat d6/d2c/d75/f85 x:0 0 0 2024-02-16T22:21:58.875 INFO:tasks.workunit.client.1.smithi123.stdout:2/514: mkdir d4/d1a/d1d/d13/d15/d6d/d4a/da2 0 2024-02-16T22:21:58.875 INFO:tasks.workunit.client.0.smithi114.stdout:6/422: write d6/d2c/f46 [4668252,53884] 0 2024-02-16T22:21:58.876 INFO:tasks.workunit.client.0.smithi114.stdout:6/423: stat d6/da/l52 0 2024-02-16T22:21:58.876 INFO:tasks.workunit.client.0.smithi114.stdout:6/424: stat d6/f1e 0 2024-02-16T22:21:58.878 INFO:tasks.workunit.client.0.smithi114.stdout:1/371: fsync d4/d63/d69/f7f 0 2024-02-16T22:21:58.879 INFO:tasks.workunit.client.0.smithi114.stdout:2/389: creat dd/df/d1c/f85 x:0 0 0 2024-02-16T22:21:58.880 INFO:tasks.workunit.client.0.smithi114.stdout:1/372: write d4/d9/d3d/d3f/f61 [1470388,50897] 0 2024-02-16T22:21:58.883 INFO:tasks.workunit.client.0.smithi114.stdout:9/454: rmdir d2/d1a/d23/d35/d47 39 2024-02-16T22:21:58.883 INFO:tasks.workunit.client.1.smithi123.stdout:6/537: dwrite d0/d6/d2f/f2a [0,4194304] 0 2024-02-16T22:21:58.883 INFO:tasks.workunit.client.1.smithi123.stdout:6/538: fdatasync d0/d96/fb9 0 2024-02-16T22:21:58.884 INFO:tasks.workunit.client.1.smithi123.stdout:6/539: dread - d0/d96/fbf zero size 2024-02-16T22:21:58.885 INFO:tasks.workunit.client.1.smithi123.stdout:6/540: chown d0/d6/fd 161 1 2024-02-16T22:21:58.887 INFO:tasks.workunit.client.1.smithi123.stdout:1/501: rename d3/d6/db/d10/d4b/ca3 to d3/d6/db/d10/d2a/d80/caa 0 2024-02-16T22:21:58.888 INFO:tasks.workunit.client.1.smithi123.stdout:7/540: symlink d3/lb3 0 2024-02-16T22:21:58.889 INFO:tasks.workunit.client.1.smithi123.stdout:1/502: write d3/d52/f9a [796988,75294] 0 2024-02-16T22:21:58.889 INFO:tasks.workunit.client.0.smithi114.stdout:9/455: dread d2/f15 [0,4194304] 0 2024-02-16T22:21:58.890 INFO:tasks.workunit.client.0.smithi114.stdout:3/341: mkdir d4/d24/d2d/d45/d68/d77 0 2024-02-16T22:21:58.891 INFO:tasks.workunit.client.0.smithi114.stdout:8/379: rename d4/d11/d78/d7a/l66 to d4/d24/d6d/l89 0 2024-02-16T22:21:58.894 INFO:tasks.workunit.client.0.smithi114.stdout:7/446: rmdir d3/d87 39 2024-02-16T22:21:58.895 INFO:tasks.workunit.client.0.smithi114.stdout:6/425: symlink d6/dd/d25/d34/d61/d62/l86 0 2024-02-16T22:21:58.896 INFO:tasks.workunit.client.1.smithi123.stdout:2/515: fsync d4/d1a/d1d/d38/f58 0 2024-02-16T22:21:58.897 INFO:tasks.workunit.client.1.smithi123.stdout:2/516: dread - d4/d1a/d1d/d38/d7b/f94 zero size 2024-02-16T22:21:58.897 INFO:tasks.workunit.client.0.smithi114.stdout:2/390: creat dd/d3c/d4c/f86 x:0 0 0 2024-02-16T22:21:58.899 INFO:tasks.workunit.client.0.smithi114.stdout:0/360: creat d1/f70 x:0 0 0 2024-02-16T22:21:58.899 INFO:tasks.workunit.client.0.smithi114.stdout:2/391: chown dd/df/d1c/d21/c65 322028 1 2024-02-16T22:21:58.900 INFO:tasks.workunit.client.0.smithi114.stdout:1/373: mkdir d4/d63/d69/d80 0 2024-02-16T22:21:58.903 INFO:tasks.workunit.client.0.smithi114.stdout:6/426: dwrite d6/d2c/d38/f4f [0,4194304] 0 2024-02-16T22:21:58.904 INFO:tasks.workunit.client.1.smithi123.stdout:3/534: creat d0/d20/d29/fbf x:0 0 0 2024-02-16T22:21:58.905 INFO:tasks.workunit.client.0.smithi114.stdout:1/374: truncate d4/d33/d74/f77 90719 0 2024-02-16T22:21:58.905 INFO:tasks.workunit.client.0.smithi114.stdout:6/427: chown d6/d2c/d38/f4f 5791716 1 2024-02-16T22:21:58.906 INFO:tasks.workunit.client.1.smithi123.stdout:3/535: dread d0/d20/d29/f3d [0,4194304] 0 2024-02-16T22:21:58.909 INFO:tasks.workunit.client.0.smithi114.stdout:6/428: fdatasync d6/d2c/d5e/f67 0 2024-02-16T22:21:58.909 INFO:tasks.workunit.client.1.smithi123.stdout:9/641: rename d4/d36/daa/d87 to d4/d36/d8e/d58/d59/dc7/dd1 0 2024-02-16T22:21:58.910 INFO:tasks.workunit.client.0.smithi114.stdout:6/429: write d6/d2c/d75/f79 [197008,101293] 0 2024-02-16T22:21:58.911 INFO:tasks.workunit.client.0.smithi114.stdout:4/518: symlink dd/d1b/d37/d38/lb9 0 2024-02-16T22:21:58.911 INFO:tasks.workunit.client.0.smithi114.stdout:5/330: dwrite d0/d10/f13 [0,4194304] 0 2024-02-16T22:21:58.912 INFO:tasks.workunit.client.1.smithi123.stdout:7/541: creat d3/d6e/d52/fb4 x:0 0 0 2024-02-16T22:21:58.912 INFO:tasks.workunit.client.0.smithi114.stdout:6/430: write d6/d2c/f7c [502418,107676] 0 2024-02-16T22:21:58.914 INFO:tasks.workunit.client.1.smithi123.stdout:1/503: rmdir d3/d6/db/d10 39 2024-02-16T22:21:58.915 INFO:tasks.workunit.client.0.smithi114.stdout:9/456: creat d2/d1a/d65/f94 x:0 0 0 2024-02-16T22:21:58.920 INFO:tasks.workunit.client.0.smithi114.stdout:8/380: symlink d4/d24/l8a 0 2024-02-16T22:21:58.921 INFO:tasks.workunit.client.0.smithi114.stdout:5/331: dwrite d0/df/d16/d25/d3e/f5c [0,4194304] 0 2024-02-16T22:21:58.921 INFO:tasks.workunit.client.0.smithi114.stdout:8/381: chown d4/d24/d6d/l6f 56350764 1 2024-02-16T22:21:58.925 INFO:tasks.workunit.client.0.smithi114.stdout:5/332: dread d0/fa [0,4194304] 0 2024-02-16T22:21:58.941 INFO:tasks.workunit.client.0.smithi114.stdout:7/447: creat d3/d23/d6f/d89/f96 x:0 0 0 2024-02-16T22:21:58.944 INFO:tasks.workunit.client.1.smithi123.stdout:2/517: symlink d4/d1a/d1d/d13/d15/d6d/d4a/la3 0 2024-02-16T22:21:58.945 INFO:tasks.workunit.client.0.smithi114.stdout:0/361: creat d1/d16/d48/f71 x:0 0 0 2024-02-16T22:21:58.945 INFO:tasks.workunit.client.1.smithi123.stdout:2/518: chown d4/d1a/d1d/d13/d25/d98/f96 1708723 1 2024-02-16T22:21:58.946 INFO:tasks.workunit.client.1.smithi123.stdout:5/510: creat d6/dd/df/d58/fad x:0 0 0 2024-02-16T22:21:58.947 INFO:tasks.workunit.client.1.smithi123.stdout:5/511: chown d6/dd/df/d1f/l49 1893 1 2024-02-16T22:21:58.951 INFO:tasks.workunit.client.1.smithi123.stdout:3/536: write d0/fe [6704206,55281] 0 2024-02-16T22:21:58.955 INFO:tasks.workunit.client.1.smithi123.stdout:9/642: creat d4/d36/d8e/d58/d59/d60/fd2 x:0 0 0 2024-02-16T22:21:58.960 INFO:tasks.workunit.client.0.smithi114.stdout:3/342: write d4/dd/f1e [2968827,37367] 0 2024-02-16T22:21:58.960 INFO:tasks.workunit.client.1.smithi123.stdout:9/643: write d4/d36/f5f [9243050,29048] 0 2024-02-16T22:21:58.960 INFO:tasks.workunit.client.1.smithi123.stdout:3/537: dwrite d0/d20/d2e/fb8 [0,4194304] 0 2024-02-16T22:21:58.961 INFO:tasks.workunit.client.0.smithi114.stdout:5/333: mkdir d0/df/d5e 0 2024-02-16T22:21:58.962 INFO:tasks.workunit.client.0.smithi114.stdout:5/334: stat d0/df/d16/d25/d3e/l50 0 2024-02-16T22:21:58.963 INFO:tasks.workunit.client.1.smithi123.stdout:8/578: getdents d0/d29/da7/d26/d2a/d77 0 2024-02-16T22:21:58.964 INFO:tasks.workunit.client.1.smithi123.stdout:2/519: chown d4/d1a/d1d/d38/l7e 27006631 1 2024-02-16T22:21:58.965 INFO:tasks.workunit.client.1.smithi123.stdout:5/512: rmdir d6/dd/df/d1f 39 2024-02-16T22:21:58.969 INFO:tasks.workunit.client.1.smithi123.stdout:4/571: rename d5/d1c/d23/d6d/c8c to d5/d1c/d23/d31/cb5 0 2024-02-16T22:21:58.970 INFO:tasks.workunit.client.0.smithi114.stdout:1/375: rename d4/d9/d3d/f54 to d4/d9/d3d/d3f/f81 0 2024-02-16T22:21:58.972 INFO:tasks.workunit.client.0.smithi114.stdout:1/376: write d4/d9/d46/f48 [843457,56251] 0 2024-02-16T22:21:58.973 INFO:tasks.workunit.client.0.smithi114.stdout:6/431: unlink d6/dd/l6e 0 2024-02-16T22:21:58.974 INFO:tasks.workunit.client.0.smithi114.stdout:2/392: truncate dd/df/d1c/d21/d61/d63/f73 942073 0 2024-02-16T22:21:58.974 INFO:tasks.workunit.client.0.smithi114.stdout:6/432: write d6/da/f56 [985957,30280] 0 2024-02-16T22:21:58.977 INFO:tasks.workunit.client.1.smithi123.stdout:6/541: sync 2024-02-16T22:21:58.978 INFO:tasks.workunit.client.0.smithi114.stdout:3/343: mknod d4/d43/c78 0 2024-02-16T22:21:58.982 INFO:tasks.workunit.client.1.smithi123.stdout:3/538: rmdir d0/d20/d76/d5d/dae/db0 39 2024-02-16T22:21:58.986 INFO:tasks.workunit.client.1.smithi123.stdout:0/488: getdents d1/d4/d17 0 2024-02-16T22:21:58.987 INFO:tasks.workunit.client.1.smithi123.stdout:0/489: dread - d1/d4/d17/d54/f9b zero size 2024-02-16T22:21:58.988 INFO:tasks.workunit.client.0.smithi114.stdout:0/362: creat d1/d1a/d47/d62/f72 x:0 0 0 2024-02-16T22:21:58.988 INFO:tasks.workunit.client.0.smithi114.stdout:0/363: chown d1/d16/d18/f1c 16 1 2024-02-16T22:21:58.989 INFO:tasks.workunit.client.1.smithi123.stdout:9/644: dwrite d4/d36/daa/d2b/f53 [0,4194304] 0 2024-02-16T22:21:58.990 INFO:tasks.workunit.client.0.smithi114.stdout:2/393: rmdir dd/df/d2c 39 2024-02-16T22:21:58.992 INFO:tasks.workunit.client.1.smithi123.stdout:7/542: creat d3/d6e/fb5 x:0 0 0 2024-02-16T22:21:58.993 INFO:tasks.workunit.client.1.smithi123.stdout:6/542: fdatasync d0/d6/d2f/d5d/f7e 0 2024-02-16T22:21:58.993 INFO:tasks.workunit.client.1.smithi123.stdout:6/543: dread - d0/d9/fb5 zero size 2024-02-16T22:21:58.994 INFO:tasks.workunit.client.0.smithi114.stdout:0/364: dwrite d1/f2a [0,4194304] 0 2024-02-16T22:21:58.996 INFO:tasks.workunit.client.1.smithi123.stdout:1/504: link d3/d6/d49/f66 d3/d6/d49/d48/d68/fab 0 2024-02-16T22:21:58.998 INFO:tasks.workunit.client.0.smithi114.stdout:9/457: creat d2/dd/d3c/d89/d8b/f95 x:0 0 0 2024-02-16T22:21:58.999 INFO:tasks.workunit.client.0.smithi114.stdout:8/382: link d4/d11/d78/d7a/d7e/l81 d4/d16/d25/d46/l8b 0 2024-02-16T22:21:59.000 INFO:tasks.workunit.client.0.smithi114.stdout:7/448: rename d3/f26 to d3/f97 0 2024-02-16T22:21:59.000 INFO:tasks.workunit.client.0.smithi114.stdout:7/449: write d3/d23/f6b [3822541,25443] 0 2024-02-16T22:21:59.001 INFO:tasks.workunit.client.0.smithi114.stdout:7/450: read d3/d23/d4e/f7f [2710485,66708] 0 2024-02-16T22:21:59.002 INFO:tasks.workunit.client.0.smithi114.stdout:7/451: fsync d3/d23/f5e 0 2024-02-16T22:21:59.011 INFO:tasks.workunit.client.1.smithi123.stdout:9/645: rmdir d4/d36/daa/d2b/d6c 39 2024-02-16T22:21:59.012 INFO:tasks.workunit.client.0.smithi114.stdout:3/344: write d4/d24/d2d/f4e [65364,71679] 0 2024-02-16T22:21:59.015 INFO:tasks.workunit.client.0.smithi114.stdout:2/394: dread dd/f3a [0,4194304] 0 2024-02-16T22:21:59.015 INFO:tasks.workunit.client.0.smithi114.stdout:5/335: truncate d0/df/f1d 3657358 0 2024-02-16T22:21:59.016 INFO:tasks.workunit.client.0.smithi114.stdout:6/433: creat d6/d83/f87 x:0 0 0 2024-02-16T22:21:59.016 INFO:tasks.workunit.client.1.smithi123.stdout:4/572: write d5/d49/f89 [3291142,125400] 0 2024-02-16T22:21:59.017 INFO:tasks.workunit.client.1.smithi123.stdout:6/544: mknod d0/d6/d2f/d5d/cc5 0 2024-02-16T22:21:59.020 INFO:tasks.workunit.client.1.smithi123.stdout:8/579: creat d0/d29/da7/d43/fbc x:0 0 0 2024-02-16T22:21:59.020 INFO:tasks.workunit.client.1.smithi123.stdout:8/580: dread - d0/d29/da7/fb3 zero size 2024-02-16T22:21:59.022 INFO:tasks.workunit.client.0.smithi114.stdout:1/377: dwrite d4/d9/d46/d67/f68 [0,4194304] 0 2024-02-16T22:21:59.022 INFO:tasks.workunit.client.0.smithi114.stdout:8/383: symlink d4/d16/d25/d39/l8c 0 2024-02-16T22:21:59.027 INFO:tasks.workunit.client.0.smithi114.stdout:7/452: symlink d3/d23/d6f/d89/l98 0 2024-02-16T22:21:59.030 INFO:tasks.workunit.client.1.smithi123.stdout:6/545: mkdir d0/d6/d2f/da5/dc6 0 2024-02-16T22:21:59.032 INFO:tasks.workunit.client.1.smithi123.stdout:1/505: mknod d3/d6/db/d10/d2a/d80/cac 0 2024-02-16T22:21:59.035 INFO:tasks.workunit.client.1.smithi123.stdout:2/520: rename d4/l57 to d4/d1a/d1d/la4 0 2024-02-16T22:21:59.036 INFO:tasks.workunit.client.1.smithi123.stdout:2/521: truncate d4/d1a/d1d/f9 4466674 0 2024-02-16T22:21:59.039 INFO:tasks.workunit.client.1.smithi123.stdout:8/581: unlink d0/d29/da7/d26/d32/d45/d67/f8a 0 2024-02-16T22:21:59.041 INFO:tasks.workunit.client.0.smithi114.stdout:6/434: dread d6/f35 [0,4194304] 0 2024-02-16T22:21:59.042 INFO:tasks.workunit.client.1.smithi123.stdout:4/573: write d5/d1c/d23/d74/f55 [4667536,100924] 0 2024-02-16T22:21:59.043 INFO:tasks.workunit.client.1.smithi123.stdout:7/543: creat d3/d6e/d1d/fb6 x:0 0 0 2024-02-16T22:21:59.043 INFO:tasks.workunit.client.1.smithi123.stdout:4/574: chown d5/d1c/d23/d69/f79 3267 1 2024-02-16T22:21:59.044 INFO:tasks.workunit.client.0.smithi114.stdout:9/458: read d2/d1a/d23/d35/d47/d63/f6a [451196,92230] 0 2024-02-16T22:21:59.045 INFO:tasks.workunit.client.1.smithi123.stdout:6/546: fdatasync d0/f60 0 2024-02-16T22:21:59.045 INFO:tasks.workunit.client.0.smithi114.stdout:9/459: fdatasync d2/f26 0 2024-02-16T22:21:59.047 INFO:tasks.workunit.client.0.smithi114.stdout:3/345: mknod d4/c79 0 2024-02-16T22:21:59.050 INFO:tasks.workunit.client.1.smithi123.stdout:5/513: rename d6/dd/d44/f7d to d6/dd/df/d90/fae 0 2024-02-16T22:21:59.050 INFO:tasks.workunit.client.1.smithi123.stdout:5/514: dread - d6/dd/df/d58/d89/f8a zero size 2024-02-16T22:21:59.053 INFO:tasks.workunit.client.1.smithi123.stdout:2/522: unlink d4/d1a/d1d/d13/d25/f80 0 2024-02-16T22:21:59.053 INFO:tasks.workunit.client.0.smithi114.stdout:9/460: dwrite d2/d33/f7d [0,4194304] 0 2024-02-16T22:21:59.056 INFO:tasks.workunit.client.0.smithi114.stdout:2/395: rename dd/df/d2c to dd/df/d1c/d21/d87 0 2024-02-16T22:21:59.061 INFO:tasks.workunit.client.0.smithi114.stdout:0/365: creat d1/d16/f73 x:0 0 0 2024-02-16T22:21:59.062 INFO:tasks.workunit.client.1.smithi123.stdout:9/646: write d4/d36/daa/f1a [3518318,112818] 0 2024-02-16T22:21:59.063 INFO:tasks.workunit.client.1.smithi123.stdout:8/582: dwrite d0/d12/f74 [0,4194304] 0 2024-02-16T22:21:59.063 INFO:tasks.workunit.client.1.smithi123.stdout:7/544: mkdir d3/d6e/d1d/db2/d35/db7 0 2024-02-16T22:21:59.064 INFO:tasks.workunit.client.1.smithi123.stdout:9/647: write d4/d36/daa/d20/f22 [228217,48547] 0 2024-02-16T22:21:59.068 INFO:tasks.workunit.client.0.smithi114.stdout:5/336: write d0/df/f2b [391340,123609] 0 2024-02-16T22:21:59.070 INFO:tasks.workunit.client.0.smithi114.stdout:3/346: dread d4/dd/f19 [0,4194304] 0 2024-02-16T22:21:59.070 INFO:tasks.workunit.client.1.smithi123.stdout:1/506: mknod d3/d6/cad 0 2024-02-16T22:21:59.071 INFO:tasks.workunit.client.0.smithi114.stdout:1/378: fsync d4/f37 0 2024-02-16T22:21:59.071 INFO:tasks.workunit.client.1.smithi123.stdout:7/545: dwrite d3/d6e/d1d/fab [0,4194304] 0 2024-02-16T22:21:59.073 INFO:tasks.workunit.client.1.smithi123.stdout:4/575: dwrite d5/d1c/f25 [4194304,4194304] 0 2024-02-16T22:21:59.073 INFO:tasks.workunit.client.0.smithi114.stdout:9/461: symlink d2/d1a/d23/d35/d47/l96 0 2024-02-16T22:21:59.074 INFO:tasks.workunit.client.1.smithi123.stdout:1/507: fsync d3/d6/db/d10/d2a/f6f 0 2024-02-16T22:21:59.076 INFO:tasks.workunit.client.1.smithi123.stdout:4/576: stat d5/d1c/d64/d7b/fb1 0 2024-02-16T22:21:59.076 INFO:tasks.workunit.client.1.smithi123.stdout:1/508: write d3/d6/f9d [1167988,30311] 0 2024-02-16T22:21:59.077 INFO:tasks.workunit.client.1.smithi123.stdout:2/523: mknod d4/d1a/d1d/d13/d25/d98/ca5 0 2024-02-16T22:21:59.078 INFO:tasks.workunit.client.1.smithi123.stdout:1/509: write d3/f8 [3358267,54178] 0 2024-02-16T22:21:59.078 INFO:tasks.workunit.client.1.smithi123.stdout:4/577: write d5/d1c/d23/d74/f55 [4964491,88386] 0 2024-02-16T22:21:59.079 INFO:tasks.workunit.client.0.smithi114.stdout:3/347: unlink d4/dd/d33/l55 0 2024-02-16T22:21:59.080 INFO:tasks.workunit.client.0.smithi114.stdout:3/348: chown d4/dd/l71 124792 1 2024-02-16T22:21:59.081 INFO:tasks.workunit.client.1.smithi123.stdout:7/546: dwrite d3/d6e/d1d/fab [0,4194304] 0 2024-02-16T22:21:59.082 INFO:tasks.workunit.client.0.smithi114.stdout:1/379: mkdir d4/d63/d69/d72/d82 0 2024-02-16T22:21:59.083 INFO:tasks.workunit.client.1.smithi123.stdout:7/547: chown d3/d5/d6c/f8d 0 1 2024-02-16T22:21:59.084 INFO:tasks.workunit.client.1.smithi123.stdout:2/524: dread d4/d1a/d1d/d13/d25/d98/f96 [0,4194304] 0 2024-02-16T22:21:59.085 INFO:tasks.workunit.client.0.smithi114.stdout:8/384: rename d4/d11/f1c to d4/d16/d25/d7c/f8d 0 2024-02-16T22:21:59.087 INFO:tasks.workunit.client.0.smithi114.stdout:2/396: mkdir dd/d3c/d88 0 2024-02-16T22:21:59.089 INFO:tasks.workunit.client.0.smithi114.stdout:1/380: readlink d4/d9/l23 0 2024-02-16T22:21:59.091 INFO:tasks.workunit.client.0.smithi114.stdout:1/381: write d4/d63/d69/f7f [5514523,105363] 0 2024-02-16T22:21:59.093 INFO:tasks.workunit.client.0.smithi114.stdout:5/337: rename d0/c39 to d0/df/d16/d1e/c5f 0 2024-02-16T22:21:59.095 INFO:tasks.workunit.client.0.smithi114.stdout:5/338: stat d0/df/f5d 0 2024-02-16T22:21:59.096 INFO:tasks.workunit.client.0.smithi114.stdout:2/397: fsync dd/f3a 0 2024-02-16T22:21:59.098 INFO:tasks.workunit.client.0.smithi114.stdout:5/339: readlink d0/df/d16/d25/d2e/d3b/l47 0 2024-02-16T22:21:59.099 INFO:tasks.workunit.client.0.smithi114.stdout:1/382: dwrite d4/d9/d46/d67/f68 [4194304,4194304] 0 2024-02-16T22:21:59.110 INFO:tasks.workunit.client.0.smithi114.stdout:3/349: link d4/d43/c47 d4/d24/d2d/d45/d74/c7a 0 2024-02-16T22:21:59.111 INFO:tasks.workunit.client.1.smithi123.stdout:4/578: unlink d5/d1c/d23/d74/c67 0 2024-02-16T22:21:59.118 INFO:tasks.workunit.client.1.smithi123.stdout:8/583: dwrite d0/d29/da7/d26/f7a [0,4194304] 0 2024-02-16T22:21:59.119 INFO:tasks.workunit.client.0.smithi114.stdout:3/350: rmdir d4/d24/d2d/d45/d68 39 2024-02-16T22:21:59.123 INFO:tasks.workunit.client.0.smithi114.stdout:5/340: rename d0/df/d16/f4d to d0/df/d16/d1e/f60 0 2024-02-16T22:21:59.125 INFO:tasks.workunit.client.1.smithi123.stdout:3/539: rename d0/d20/d5a/fa5 to d0/fc0 0 2024-02-16T22:21:59.127 INFO:tasks.workunit.client.1.smithi123.stdout:5/515: creat d6/dd/df/d1f/d25/da2/faf x:0 0 0 2024-02-16T22:21:59.128 INFO:tasks.workunit.client.0.smithi114.stdout:8/385: getdents d4/d24 0 2024-02-16T22:21:59.129 INFO:tasks.workunit.client.0.smithi114.stdout:8/386: chown d4/d24/d2c/d67 26814 1 2024-02-16T22:21:59.133 INFO:tasks.workunit.client.1.smithi123.stdout:9/648: write d4/d1e/fc8 [449693,128147] 0 2024-02-16T22:21:59.134 INFO:tasks.workunit.client.1.smithi123.stdout:9/649: dread - d4/d36/d8e/d6d/d71/fab zero size 2024-02-16T22:21:59.135 INFO:tasks.workunit.client.1.smithi123.stdout:7/548: fdatasync d3/d87/da8/faf 0 2024-02-16T22:21:59.136 INFO:tasks.workunit.client.0.smithi114.stdout:2/398: creat dd/d6b/f89 x:0 0 0 2024-02-16T22:21:59.137 INFO:tasks.workunit.client.0.smithi114.stdout:8/387: stat d4/d24/d6d/l89 0 2024-02-16T22:21:59.138 INFO:tasks.workunit.client.1.smithi123.stdout:4/579: dread d5/d49/f89 [0,4194304] 0 2024-02-16T22:21:59.138 INFO:tasks.workunit.client.1.smithi123.stdout:6/547: link d0/d6/d91/d93/c9f d0/d9/dc/cc7 0 2024-02-16T22:21:59.139 INFO:tasks.workunit.client.0.smithi114.stdout:2/399: dread dd/f3f [0,4194304] 0 2024-02-16T22:21:59.140 INFO:tasks.workunit.client.0.smithi114.stdout:5/341: dread d0/df/f2b [0,4194304] 0 2024-02-16T22:21:59.140 INFO:tasks.workunit.client.0.smithi114.stdout:1/383: getdents d4/d9/d3d/d3f 0 2024-02-16T22:21:59.141 INFO:tasks.workunit.client.1.smithi123.stdout:4/580: stat d5/d1c/d23/d74/d92/f86 0 2024-02-16T22:21:59.142 INFO:tasks.workunit.client.1.smithi123.stdout:9/650: dwrite d4/d36/fa8 [0,4194304] 0 2024-02-16T22:21:59.143 INFO:tasks.workunit.client.1.smithi123.stdout:4/581: write d5/d1c/d23/d6d/f94 [187417,123027] 0 2024-02-16T22:21:59.143 INFO:tasks.workunit.client.1.smithi123.stdout:9/651: fsync d4/d1e/d3a/fb7 0 2024-02-16T22:21:59.146 INFO:tasks.workunit.client.1.smithi123.stdout:3/540: unlink d0/d40/c78 0 2024-02-16T22:21:59.147 INFO:tasks.workunit.client.0.smithi114.stdout:8/388: creat d4/d11/d78/d7a/d31/f8e x:0 0 0 2024-02-16T22:21:59.148 INFO:tasks.workunit.client.1.smithi123.stdout:4/582: write d5/d1c/d23/d47/da2/fb2 [64251,87600] 0 2024-02-16T22:21:59.148 INFO:tasks.workunit.client.1.smithi123.stdout:7/549: dread d3/d6e/d1d/f41 [0,4194304] 0 2024-02-16T22:21:59.150 INFO:tasks.workunit.client.0.smithi114.stdout:2/400: dread dd/df/d1c/d21/d61/d63/f73 [0,4194304] 0 2024-02-16T22:21:59.153 INFO:tasks.workunit.client.0.smithi114.stdout:1/384: rename d4/d9/d3d/l44 to d4/d63/l83 0 2024-02-16T22:21:59.154 INFO:tasks.workunit.client.0.smithi114.stdout:3/351: truncate d4/dd/f13 4078323 0 2024-02-16T22:21:59.154 INFO:tasks.workunit.client.1.smithi123.stdout:8/584: symlink d0/d29/da7/d43/d34/dae/lbd 0 2024-02-16T22:21:59.156 INFO:tasks.workunit.client.0.smithi114.stdout:8/389: fdatasync d4/d11/f51 0 2024-02-16T22:21:59.158 INFO:tasks.workunit.client.0.smithi114.stdout:5/342: unlink d0/df/c26 0 2024-02-16T22:21:59.159 INFO:tasks.workunit.client.1.smithi123.stdout:6/548: creat d0/d9/dc/d3a/fc8 x:0 0 0 2024-02-16T22:21:59.159 INFO:tasks.workunit.client.1.smithi123.stdout:6/549: read - d0/d74/f94 zero size 2024-02-16T22:21:59.160 INFO:tasks.workunit.client.0.smithi114.stdout:2/401: rename dd/df/d1c/d21/d74 to dd/d3b/d8a 0 2024-02-16T22:21:59.161 INFO:tasks.workunit.client.1.smithi123.stdout:5/516: getdents d6/dd/df/d58/d89/d98 0 2024-02-16T22:21:59.161 INFO:tasks.workunit.client.1.smithi123.stdout:1/510: sync 2024-02-16T22:21:59.164 INFO:tasks.workunit.client.1.smithi123.stdout:1/511: dread d3/d52/f9a [0,4194304] 0 2024-02-16T22:21:59.164 INFO:tasks.workunit.client.0.smithi114.stdout:3/352: symlink d4/d26/l7b 0 2024-02-16T22:21:59.165 INFO:tasks.workunit.client.1.smithi123.stdout:1/512: readlink d3/d6/db/d10/d2a/d53/l71 0 2024-02-16T22:21:59.165 INFO:tasks.workunit.client.1.smithi123.stdout:3/541: fdatasync d0/fc0 0 2024-02-16T22:21:59.166 INFO:tasks.workunit.client.1.smithi123.stdout:1/513: fdatasync d3/d6/db/d10/d4b/d70/f79 0 2024-02-16T22:21:59.207 INFO:tasks.workunit.client.1.smithi123.stdout:7/550: unlink d3/d87/f65 0 2024-02-16T22:21:59.208 INFO:tasks.workunit.client.0.smithi114.stdout:5/343: mknod d0/d28/c61 0 2024-02-16T22:21:59.209 INFO:tasks.workunit.client.1.smithi123.stdout:2/525: getdents d4/d1a/d1d/d13/d15 0 2024-02-16T22:21:59.211 INFO:tasks.workunit.client.1.smithi123.stdout:8/585: fdatasync d0/d12/f50 0 2024-02-16T22:21:59.213 INFO:tasks.workunit.client.1.smithi123.stdout:8/586: truncate d0/d29/da7/d26/d2a/d4d/fb4 915538 0 2024-02-16T22:21:59.213 INFO:tasks.workunit.client.1.smithi123.stdout:0/490: rename d1/d4/d17/d54/c55 to d1/d8/ca6 0 2024-02-16T22:21:59.215 INFO:tasks.workunit.client.0.smithi114.stdout:3/353: creat d4/d6a/f7c x:0 0 0 2024-02-16T22:21:59.215 INFO:tasks.workunit.client.0.smithi114.stdout:3/354: chown f1 7964942 1 2024-02-16T22:21:59.216 INFO:tasks.workunit.client.0.smithi114.stdout:1/385: dwrite d4/d33/f3e [4194304,4194304] 0 2024-02-16T22:21:59.217 INFO:tasks.workunit.client.1.smithi123.stdout:2/526: dread d4/d1a/d1d/d13/d25/f77 [4194304,4194304] 0 2024-02-16T22:21:59.219 INFO:tasks.workunit.client.0.smithi114.stdout:5/344: fdatasync d0/d2d/d30/f31 0 2024-02-16T22:21:59.222 INFO:tasks.workunit.client.0.smithi114.stdout:2/402: mkdir dd/d3c/d8b 0 2024-02-16T22:21:59.222 INFO:tasks.workunit.client.1.smithi123.stdout:9/652: mknod d4/d36/d8e/cd3 0 2024-02-16T22:21:59.225 INFO:tasks.workunit.client.0.smithi114.stdout:8/390: creat d4/d11/f8f x:0 0 0 2024-02-16T22:21:59.227 INFO:tasks.workunit.client.1.smithi123.stdout:1/514: write d3/d6/f26 [1699787,114773] 0 2024-02-16T22:21:59.227 INFO:tasks.workunit.client.1.smithi123.stdout:1/515: chown d3/f5 5 1 2024-02-16T22:21:59.227 INFO:tasks.workunit.client.0.smithi114.stdout:1/386: symlink d4/d9/d46/l84 0 2024-02-16T22:21:59.229 INFO:tasks.workunit.client.0.smithi114.stdout:1/387: fdatasync d4/d63/d69/f7f 0 2024-02-16T22:21:59.230 INFO:tasks.workunit.client.0.smithi114.stdout:5/345: rmdir d0/df/d16/d25/d2e 39 2024-02-16T22:21:59.231 INFO:tasks.workunit.client.0.smithi114.stdout:8/391: dwrite d4/f8 [0,4194304] 0 2024-02-16T22:21:59.234 INFO:tasks.workunit.client.0.smithi114.stdout:8/392: read d4/d24/f38 [2041631,88766] 0 2024-02-16T22:21:59.237 INFO:tasks.workunit.client.1.smithi123.stdout:8/587: mkdir d0/d29/da7/d26/d68/dbe 0 2024-02-16T22:21:59.240 INFO:tasks.workunit.client.1.smithi123.stdout:6/550: symlink d0/d9/dc/lc9 0 2024-02-16T22:21:59.244 INFO:tasks.workunit.client.1.smithi123.stdout:5/517: mkdir d6/dd/d78/da5/db0 0 2024-02-16T22:21:59.246 INFO:tasks.workunit.client.1.smithi123.stdout:7/551: dwrite d3/d6e/d1d/db2/f7e [0,4194304] 0 2024-02-16T22:21:59.248 INFO:tasks.workunit.client.1.smithi123.stdout:7/552: write d3/d6e/f7c [2606834,79891] 0 2024-02-16T22:21:59.248 INFO:tasks.workunit.client.0.smithi114.stdout:2/403: rename dd/df/d1c/c56 to dd/df/d1c/d21/d61/d63/c8c 0 2024-02-16T22:21:59.250 INFO:tasks.workunit.client.1.smithi123.stdout:3/542: symlink d0/lc1 0 2024-02-16T22:21:59.251 INFO:tasks.workunit.client.0.smithi114.stdout:8/393: unlink d4/d24/d2c/f4c 0 2024-02-16T22:21:59.251 INFO:tasks.workunit.client.1.smithi123.stdout:3/543: truncate d0/d20/d76/d5d/d60/f90 930192 0 2024-02-16T22:21:59.253 INFO:tasks.workunit.client.0.smithi114.stdout:3/355: link d4/l5a d4/d26/d4a/l7d 0 2024-02-16T22:21:59.256 INFO:tasks.workunit.client.1.smithi123.stdout:8/588: dread d0/d29/da7/d26/d2a/d3c/f55 [0,4194304] 0 2024-02-16T22:21:59.257 INFO:tasks.workunit.client.1.smithi123.stdout:8/589: chown d0/df/la0 19637884 1 2024-02-16T22:21:59.258 INFO:tasks.workunit.client.1.smithi123.stdout:6/551: mkdir d0/d96/dca 0 2024-02-16T22:21:59.258 INFO:tasks.workunit.client.1.smithi123.stdout:8/590: write d0/d29/da7/d26/d32/d45/f5f [2916562,116242] 0 2024-02-16T22:21:59.259 INFO:tasks.workunit.client.0.smithi114.stdout:8/394: creat d4/d11/d78/f90 x:0 0 0 2024-02-16T22:21:59.259 INFO:tasks.workunit.client.1.smithi123.stdout:8/591: fsync d0/d29/da7/d43/fbc 0 2024-02-16T22:21:59.260 INFO:tasks.workunit.client.0.smithi114.stdout:8/395: readlink d4/d16/d25/d7c/l88 0 2024-02-16T22:21:59.260 INFO:tasks.workunit.client.1.smithi123.stdout:2/527: symlink d4/d1a/d1d/d59/la6 0 2024-02-16T22:21:59.263 INFO:tasks.workunit.client.1.smithi123.stdout:9/653: getdents d4/d36/d8e/d58/d59/d60/dc1 0 2024-02-16T22:21:59.263 INFO:tasks.workunit.client.1.smithi123.stdout:5/518: creat d6/dd/df/d58/d3f/d82/fb1 x:0 0 0 2024-02-16T22:21:59.264 INFO:tasks.workunit.client.1.smithi123.stdout:5/519: chown d6/d3b/c8d 870 1 2024-02-16T22:21:59.264 INFO:tasks.workunit.client.0.smithi114.stdout:2/404: symlink dd/df/d1c/d21/d61/d63/d83/l8d 0 2024-02-16T22:21:59.265 INFO:tasks.workunit.client.1.smithi123.stdout:5/520: chown d6/dd/df/d1f/d25/c32 703530392 1 2024-02-16T22:21:59.265 INFO:tasks.workunit.client.0.smithi114.stdout:8/396: dread d4/d11/d78/d7a/d36/d3b/f44 [0,4194304] 0 2024-02-16T22:21:59.267 INFO:tasks.workunit.client.0.smithi114.stdout:8/397: stat d4/d16/d25/f54 0 2024-02-16T22:21:59.267 INFO:tasks.workunit.client.0.smithi114.stdout:5/346: truncate d0/df/d16/d25/d2e/f56 92147 0 2024-02-16T22:21:59.268 INFO:tasks.workunit.client.1.smithi123.stdout:7/553: creat d3/d5/fb8 x:0 0 0 2024-02-16T22:21:59.270 INFO:tasks.workunit.client.0.smithi114.stdout:3/356: mkdir d4/d7e 0 2024-02-16T22:21:59.271 INFO:tasks.workunit.client.1.smithi123.stdout:3/544: write d0/f4c [8658727,1599] 0 2024-02-16T22:21:59.272 INFO:tasks.workunit.client.1.smithi123.stdout:3/545: dread - d0/d20/d76/d37/d49/fb4 zero size 2024-02-16T22:21:59.272 INFO:tasks.workunit.client.0.smithi114.stdout:2/405: dwrite dd/f70 [0,4194304] 0 2024-02-16T22:21:59.274 INFO:tasks.workunit.client.0.smithi114.stdout:2/406: write dd/f36 [4129730,62413] 0 2024-02-16T22:21:59.274 INFO:tasks.workunit.client.1.smithi123.stdout:4/583: link d5/d1c/d23/d31/f66 d5/d1c/d64/fb6 0 2024-02-16T22:21:59.275 INFO:tasks.workunit.client.1.smithi123.stdout:4/584: chown d5/d1c/d23/d31/f3e 476409 1 2024-02-16T22:21:59.277 INFO:tasks.workunit.client.1.smithi123.stdout:0/491: sync 2024-02-16T22:21:59.279 INFO:tasks.workunit.client.1.smithi123.stdout:0/492: read d1/f6a [127687,75161] 0 2024-02-16T22:21:59.283 INFO:tasks.workunit.client.1.smithi123.stdout:5/521: dread d6/dd/df/d1f/d25/d37/f6e [0,4194304] 0 2024-02-16T22:21:59.283 INFO:tasks.workunit.client.1.smithi123.stdout:1/516: dwrite d3/d6/d18/f7e [0,4194304] 0 2024-02-16T22:21:59.284 INFO:tasks.workunit.client.1.smithi123.stdout:6/552: fdatasync d0/d6/d8e/fa0 0 2024-02-16T22:21:59.285 INFO:tasks.workunit.client.1.smithi123.stdout:2/528: symlink d4/d1a/d1d/d38/d7b/la7 0 2024-02-16T22:21:59.287 INFO:tasks.workunit.client.0.smithi114.stdout:1/388: truncate d4/d9/f10 215548 0 2024-02-16T22:21:59.287 INFO:tasks.workunit.client.1.smithi123.stdout:2/529: write d4/d1a/d1d/f9 [1609366,35171] 0 2024-02-16T22:21:59.289 INFO:tasks.workunit.client.0.smithi114.stdout:2/407: rename dd/d4e/l7e to dd/df/d1c/d21/d61/d63/d83/l8e 0 2024-02-16T22:21:59.290 INFO:tasks.workunit.client.1.smithi123.stdout:5/522: dread d6/dd/df/d58/d3f/d6f/f8e [0,4194304] 0 2024-02-16T22:21:59.291 INFO:tasks.workunit.client.1.smithi123.stdout:2/530: read d4/d1a/d1d/d59/d45/f4b [1408216,81813] 0 2024-02-16T22:21:59.294 INFO:tasks.workunit.client.0.smithi114.stdout:5/347: link d0/d2d/d30/f40 d0/df/d53/f62 0 2024-02-16T22:21:59.296 INFO:tasks.workunit.client.1.smithi123.stdout:4/585: rename d5/d1c/d42/f6c to d5/d49/fb7 0 2024-02-16T22:21:59.297 INFO:tasks.workunit.client.1.smithi123.stdout:4/586: chown d5/d1c/d23/d76 14 1 2024-02-16T22:21:59.297 INFO:tasks.workunit.client.0.smithi114.stdout:3/357: creat d4/dd/d33/d76/f7f x:0 0 0 2024-02-16T22:21:59.299 INFO:tasks.workunit.client.0.smithi114.stdout:8/398: write d4/d11/d78/d7a/d31/f4d [961536,19560] 0 2024-02-16T22:21:59.299 INFO:tasks.workunit.client.0.smithi114.stdout:3/358: write d4/d26/f58 [678768,25193] 0 2024-02-16T22:21:59.300 INFO:tasks.workunit.client.0.smithi114.stdout:8/399: fdatasync d4/d11/d78/d7a/f10 0 2024-02-16T22:21:59.300 INFO:tasks.workunit.client.1.smithi123.stdout:3/546: dread d0/d20/f52 [0,4194304] 0 2024-02-16T22:21:59.305 INFO:tasks.workunit.client.1.smithi123.stdout:6/553: rmdir d0 39 2024-02-16T22:21:59.307 INFO:tasks.workunit.client.1.smithi123.stdout:1/517: truncate d3/d52/f9a 1650771 0 2024-02-16T22:21:59.312 INFO:tasks.workunit.client.1.smithi123.stdout:8/592: mkdir d0/d29/da7/d26/d68/dbe/dbf 0 2024-02-16T22:21:59.314 INFO:tasks.workunit.client.0.smithi114.stdout:4/519: sync 2024-02-16T22:21:59.316 INFO:tasks.workunit.client.1.smithi123.stdout:5/523: creat d6/dd/d5f/fb2 x:0 0 0 2024-02-16T22:21:59.317 INFO:tasks.workunit.client.1.smithi123.stdout:2/531: chown d4/d1a/d1d/d13/d25/f3b 798 1 2024-02-16T22:21:59.318 INFO:tasks.workunit.client.1.smithi123.stdout:2/532: dread - d4/d1a/d1d/d13/d15/d6d/d4a/f5c zero size 2024-02-16T22:21:59.324 INFO:tasks.workunit.client.1.smithi123.stdout:4/587: mkdir d5/db8 0 2024-02-16T22:21:59.326 INFO:tasks.workunit.client.1.smithi123.stdout:0/493: mkdir d1/d8/da7 0 2024-02-16T22:21:59.328 INFO:tasks.workunit.client.0.smithi114.stdout:8/400: stat d4/d16/d25/d46 0 2024-02-16T22:21:59.328 INFO:tasks.workunit.client.0.smithi114.stdout:2/408: stat dd/df/d1c/d21/d61/c6d 0 2024-02-16T22:21:59.328 INFO:tasks.workunit.client.0.smithi114.stdout:5/348: fdatasync d0/df/f2b 0 2024-02-16T22:21:59.329 INFO:tasks.workunit.client.0.smithi114.stdout:5/349: readlink d0/d28/l34 0 2024-02-16T22:21:59.329 INFO:tasks.workunit.client.0.smithi114.stdout:8/401: chown d4/d16/d25/l68 29088 1 2024-02-16T22:21:59.331 INFO:tasks.workunit.client.0.smithi114.stdout:1/389: link d4/d9/c64 d4/d9/d65/c85 0 2024-02-16T22:21:59.332 INFO:tasks.workunit.client.0.smithi114.stdout:8/402: dread d4/d24/f7d [0,4194304] 0 2024-02-16T22:21:59.335 INFO:tasks.workunit.client.1.smithi123.stdout:5/524: fsync d6/dd/d5f/fb2 0 2024-02-16T22:21:59.336 INFO:tasks.workunit.client.0.smithi114.stdout:5/350: unlink d0/d10/f15 0 2024-02-16T22:21:59.337 INFO:tasks.workunit.client.0.smithi114.stdout:5/351: write d0/df/d16/d25/d2e/d3b/f59 [132468,114650] 0 2024-02-16T22:21:59.339 INFO:tasks.workunit.client.0.smithi114.stdout:2/409: rmdir dd/df/d1c/d21/d61 39 2024-02-16T22:21:59.344 INFO:tasks.workunit.client.1.smithi123.stdout:1/518: link d3/d6/d11/f62 d3/d6/d18/d9e/fae 0 2024-02-16T22:21:59.344 INFO:tasks.workunit.client.1.smithi123.stdout:8/593: dwrite d0/d29/da7/d43/d34/f6b [0,4194304] 0 2024-02-16T22:21:59.345 INFO:tasks.workunit.client.0.smithi114.stdout:1/390: symlink d4/d9/d46/d67/l86 0 2024-02-16T22:21:59.346 INFO:tasks.workunit.client.0.smithi114.stdout:3/359: dread d4/d24/f32 [0,4194304] 0 2024-02-16T22:21:59.347 INFO:tasks.workunit.client.1.smithi123.stdout:3/547: write d0/d40/f56 [878891,12186] 0 2024-02-16T22:21:59.348 INFO:tasks.workunit.client.0.smithi114.stdout:3/360: write d4/dd/f64 [1539781,100547] 0 2024-02-16T22:21:59.348 INFO:tasks.workunit.client.1.smithi123.stdout:3/548: stat d0/d20/d76/d5d/d60/l98 0 2024-02-16T22:21:59.349 INFO:tasks.workunit.client.0.smithi114.stdout:4/520: fdatasync dd/d1b/d41/d77/f90 0 2024-02-16T22:21:59.349 INFO:tasks.workunit.client.0.smithi114.stdout:3/361: write d4/dd/f64 [5221073,58933] 0 2024-02-16T22:21:59.349 INFO:tasks.workunit.client.0.smithi114.stdout:4/521: stat dd/d1b/d24/c33 0 2024-02-16T22:21:59.350 INFO:tasks.workunit.client.0.smithi114.stdout:8/403: rename d4/d16/d25/d46/f55 to d4/d16/f91 0 2024-02-16T22:21:59.351 INFO:tasks.workunit.client.0.smithi114.stdout:3/362: chown d4/c79 3 1 2024-02-16T22:21:59.351 INFO:tasks.workunit.client.1.smithi123.stdout:9/654: getdents d4/d36/d57 0 2024-02-16T22:21:59.353 INFO:tasks.workunit.client.1.smithi123.stdout:9/655: write d4/d36/d8e/d6d/d71/fab [856373,28904] 0 2024-02-16T22:21:59.353 INFO:tasks.workunit.client.1.smithi123.stdout:7/554: getdents d3/d6e/d24 0 2024-02-16T22:21:59.356 INFO:tasks.workunit.client.1.smithi123.stdout:5/525: rename d6/dd/df/d58/d3f/d82/f86 to d6/d3b/fb3 0 2024-02-16T22:21:59.359 INFO:tasks.workunit.client.1.smithi123.stdout:4/588: fdatasync d5/d1c/d23/d31/f66 0 2024-02-16T22:21:59.360 INFO:tasks.workunit.client.1.smithi123.stdout:6/554: write d0/d6/d91/d97/fa6 [1182321,16321] 0 2024-02-16T22:21:59.368 INFO:tasks.workunit.client.1.smithi123.stdout:8/594: fdatasync d0/df/f96 0 2024-02-16T22:21:59.369 INFO:tasks.workunit.client.0.smithi114.stdout:3/363: mkdir d4/d46/d48/d80 0 2024-02-16T22:21:59.370 INFO:tasks.workunit.client.0.smithi114.stdout:3/364: write d4/d26/f4d [2505080,49311] 0 2024-02-16T22:21:59.370 INFO:tasks.workunit.client.0.smithi114.stdout:4/522: dread dd/d1b/d24/f4a [0,4194304] 0 2024-02-16T22:21:59.372 INFO:tasks.workunit.client.0.smithi114.stdout:4/523: read dd/ff [615677,23195] 0 2024-02-16T22:21:59.373 INFO:tasks.workunit.client.0.smithi114.stdout:1/391: truncate d4/f3c 388415 0 2024-02-16T22:21:59.375 INFO:tasks.workunit.client.1.smithi123.stdout:7/555: write d3/d6e/d1d/db2/d35/f4b [4974074,6212] 0 2024-02-16T22:21:59.376 INFO:tasks.workunit.client.1.smithi123.stdout:7/556: chown d3/d6e/l16 9509 1 2024-02-16T22:21:59.377 INFO:tasks.workunit.client.1.smithi123.stdout:7/557: write d3/d6e/d24/f55 [1059893,55646] 0 2024-02-16T22:21:59.378 INFO:tasks.workunit.client.1.smithi123.stdout:7/558: truncate d3/d6e/d34/f75 1176000 0 2024-02-16T22:21:59.379 INFO:tasks.workunit.client.1.smithi123.stdout:7/559: readlink d3/d5/l19 0 2024-02-16T22:21:59.382 INFO:tasks.workunit.client.1.smithi123.stdout:6/555: creat d0/d6/d1a/d88/fcb x:0 0 0 2024-02-16T22:21:59.383 INFO:tasks.workunit.client.1.smithi123.stdout:6/556: stat d0/d6/d8e/f92 0 2024-02-16T22:21:59.383 INFO:tasks.workunit.client.1.smithi123.stdout:7/560: dwrite d3/d87/f6f [0,4194304] 0 2024-02-16T22:21:59.386 INFO:tasks.workunit.client.1.smithi123.stdout:1/519: getdents d3/d6/d11/d8b 0 2024-02-16T22:21:59.389 INFO:tasks.workunit.client.1.smithi123.stdout:3/549: mknod d0/d20/d76/d5d/dae/db0/cc2 0 2024-02-16T22:21:59.390 INFO:tasks.workunit.client.1.smithi123.stdout:7/561: dwrite d3/d6e/d1d/db2/d35/f9f [0,4194304] 0 2024-02-16T22:21:59.392 INFO:tasks.workunit.client.1.smithi123.stdout:9/656: mknod d4/d36/daa/d2b/dc6/cd4 0 2024-02-16T22:21:59.394 INFO:tasks.workunit.client.0.smithi114.stdout:5/352: getdents d0/df 0 2024-02-16T22:21:59.395 INFO:tasks.workunit.client.1.smithi123.stdout:5/526: creat d6/dac/fb4 x:0 0 0 2024-02-16T22:21:59.396 INFO:tasks.workunit.client.1.smithi123.stdout:0/494: sync 2024-02-16T22:21:59.396 INFO:tasks.workunit.client.0.smithi114.stdout:3/365: dwrite d4/d26/f34 [0,4194304] 0 2024-02-16T22:21:59.396 INFO:tasks.workunit.client.0.smithi114.stdout:3/366: chown d4/d46 11742 1 2024-02-16T22:21:59.399 INFO:tasks.workunit.client.1.smithi123.stdout:2/533: getdents d4/d1a/d1d/d13/d15/d6d 0 2024-02-16T22:21:59.400 INFO:tasks.workunit.client.1.smithi123.stdout:3/550: dwrite d0/d20/d5a/f9d [0,4194304] 0 2024-02-16T22:21:59.401 INFO:tasks.workunit.client.1.smithi123.stdout:7/562: dwrite d3/d6e/d4d/f50 [0,4194304] 0 2024-02-16T22:21:59.402 INFO:tasks.workunit.client.0.smithi114.stdout:5/353: mkdir d0/df/d16/d1e/d63 0 2024-02-16T22:21:59.403 INFO:tasks.workunit.client.0.smithi114.stdout:2/410: link dd/df/d1c/d21/d61/d63/d83/l8e dd/df/d1c/d21/d61/d63/l8f 0 2024-02-16T22:21:59.404 INFO:tasks.workunit.client.0.smithi114.stdout:1/392: dwrite d4/d9/f22 [0,4194304] 0 2024-02-16T22:21:59.405 INFO:tasks.workunit.client.0.smithi114.stdout:5/354: write d0/d28/f37 [1400362,108204] 0 2024-02-16T22:21:59.406 INFO:tasks.workunit.client.0.smithi114.stdout:3/367: symlink d4/d26/d4a/l81 0 2024-02-16T22:21:59.411 INFO:tasks.workunit.client.1.smithi123.stdout:6/557: mkdir d0/d9/dc/d5f/dcc 0 2024-02-16T22:21:59.411 INFO:tasks.workunit.client.1.smithi123.stdout:6/558: chown d0/d6/d2f/d5d 1518452 1 2024-02-16T22:21:59.412 INFO:tasks.workunit.client.1.smithi123.stdout:8/595: mkdir d0/d61/d66/dc0 0 2024-02-16T22:21:59.413 INFO:tasks.workunit.client.0.smithi114.stdout:5/355: dwrite d0/d28/f46 [0,4194304] 0 2024-02-16T22:21:59.415 INFO:tasks.workunit.client.1.smithi123.stdout:1/520: mknod d3/d6/d49/caf 0 2024-02-16T22:21:59.417 INFO:tasks.workunit.client.0.smithi114.stdout:2/411: truncate f0 7320723 0 2024-02-16T22:21:59.417 INFO:tasks.workunit.client.0.smithi114.stdout:1/393: read - d4/d33/f45 zero size 2024-02-16T22:21:59.420 INFO:tasks.workunit.client.1.smithi123.stdout:9/657: creat d4/d36/d8e/d58/d59/fd5 x:0 0 0 2024-02-16T22:21:59.421 INFO:tasks.workunit.client.1.smithi123.stdout:9/658: dread - d4/d36/d8e/d58/d59/fd5 zero size 2024-02-16T22:21:59.422 INFO:tasks.workunit.client.1.smithi123.stdout:9/659: chown d4/d36/d8e/d58/f91 22 1 2024-02-16T22:21:59.422 INFO:tasks.workunit.client.1.smithi123.stdout:5/527: creat d6/d3b/fb5 x:0 0 0 2024-02-16T22:21:59.424 INFO:tasks.workunit.client.1.smithi123.stdout:5/528: chown d6/dd/df/d58/d3f/f9e 70074 1 2024-02-16T22:21:59.424 INFO:tasks.workunit.client.1.smithi123.stdout:0/495: symlink d1/d4/d17/d54/d8e/la8 0 2024-02-16T22:21:59.425 INFO:tasks.workunit.client.1.smithi123.stdout:5/529: read d6/dd/df/f41 [39427,72896] 0 2024-02-16T22:21:59.426 INFO:tasks.workunit.client.1.smithi123.stdout:2/534: write d4/d1a/d1d/d13/d25/d98/f67 [4928547,126578] 0 2024-02-16T22:21:59.430 INFO:tasks.workunit.client.1.smithi123.stdout:2/535: dread d4/d1a/d1d/f9f [0,4194304] 0 2024-02-16T22:21:59.430 INFO:tasks.workunit.client.1.smithi123.stdout:7/563: creat d3/d6e/d24/fb9 x:0 0 0 2024-02-16T22:21:59.432 INFO:tasks.workunit.client.1.smithi123.stdout:7/564: write d3/d6e/d52/fb4 [598709,107781] 0 2024-02-16T22:21:59.432 INFO:tasks.workunit.client.0.smithi114.stdout:1/394: creat d4/d33/d74/f87 x:0 0 0 2024-02-16T22:21:59.432 INFO:tasks.workunit.client.1.smithi123.stdout:6/559: unlink d0/d6/c45 0 2024-02-16T22:21:59.433 INFO:tasks.workunit.client.0.smithi114.stdout:4/524: link dd/f2b dd/d1b/d37/fba 0 2024-02-16T22:21:59.435 INFO:tasks.workunit.client.1.smithi123.stdout:3/551: dread d0/d20/d2e/f48 [0,4194304] 0 2024-02-16T22:21:59.437 INFO:tasks.workunit.client.1.smithi123.stdout:2/536: dwrite d4/d1a/d1d/f9 [0,4194304] 0 2024-02-16T22:21:59.441 INFO:tasks.workunit.client.1.smithi123.stdout:6/560: dwrite d0/d6/d2f/d5d/f68 [0,4194304] 0 2024-02-16T22:21:59.442 INFO:tasks.workunit.client.0.smithi114.stdout:1/395: creat d4/d9/d3d/f88 x:0 0 0 2024-02-16T22:21:59.445 INFO:tasks.workunit.client.0.smithi114.stdout:4/525: rename dd/d2e/d56/f69 to dd/d1b/d41/fbb 0 2024-02-16T22:21:59.448 INFO:tasks.workunit.client.0.smithi114.stdout:2/412: link dd/d6b/f89 dd/d4e/f90 0 2024-02-16T22:21:59.450 INFO:tasks.workunit.client.1.smithi123.stdout:4/589: truncate d5/d49/f91 794071 0 2024-02-16T22:21:59.450 INFO:tasks.workunit.client.0.smithi114.stdout:5/356: dread d0/df/d16/d1e/f60 [0,4194304] 0 2024-02-16T22:21:59.452 INFO:tasks.workunit.client.1.smithi123.stdout:4/590: write d5/d1c/d23/d56/d6e/d73/f88 [5219469,88869] 0 2024-02-16T22:21:59.452 INFO:tasks.workunit.client.0.smithi114.stdout:4/526: fsync dd/d1b/d37/d61/f66 0 2024-02-16T22:21:59.455 INFO:tasks.workunit.client.0.smithi114.stdout:5/357: dread d0/d28/f38 [0,4194304] 0 2024-02-16T22:21:59.456 INFO:tasks.workunit.client.1.smithi123.stdout:7/565: mkdir d3/d6e/d1d/db2/d35/dba 0 2024-02-16T22:21:59.457 INFO:tasks.workunit.client.0.smithi114.stdout:2/413: creat dd/d3b/d8a/f91 x:0 0 0 2024-02-16T22:21:59.463 INFO:tasks.workunit.client.0.smithi114.stdout:4/527: chown dd/d1b/d37/d99/l9c 6422402 1 2024-02-16T22:21:59.465 INFO:tasks.workunit.client.1.smithi123.stdout:8/596: dread d0/d29/da7/d26/d2a/f92 [0,4194304] 0 2024-02-16T22:21:59.467 INFO:tasks.workunit.client.1.smithi123.stdout:3/552: dwrite d0/d20/d76/d37/d49/f8f [4194304,4194304] 0 2024-02-16T22:21:59.469 INFO:tasks.workunit.client.1.smithi123.stdout:2/537: rmdir d4/d1a/d1d/d59/d45 39 2024-02-16T22:21:59.472 INFO:tasks.workunit.client.1.smithi123.stdout:1/521: dread d3/d6/db/f3a [0,4194304] 0 2024-02-16T22:21:59.473 INFO:tasks.workunit.client.1.smithi123.stdout:1/522: readlink d3/d6/db/d10/d2a/d80/l97 0 2024-02-16T22:21:59.475 INFO:tasks.workunit.client.1.smithi123.stdout:4/591: unlink d5/d1c/d23/d6d/lad 0 2024-02-16T22:21:59.477 INFO:tasks.workunit.client.1.smithi123.stdout:7/566: rmdir d3/d6e/d52 39 2024-02-16T22:21:59.477 INFO:tasks.workunit.client.1.smithi123.stdout:5/530: dread d6/dd/f73 [0,4194304] 0 2024-02-16T22:21:59.477 INFO:tasks.workunit.client.1.smithi123.stdout:4/592: write d5/d1c/d42/fa8 [631629,85438] 0 2024-02-16T22:21:59.478 INFO:tasks.workunit.client.0.smithi114.stdout:4/528: link dd/ff dd/d1b/d41/d74/d9b/fbc 0 2024-02-16T22:21:59.479 INFO:tasks.workunit.client.1.smithi123.stdout:1/523: dwrite d3/d6/d11/f23 [0,4194304] 0 2024-02-16T22:21:59.480 INFO:tasks.workunit.client.1.smithi123.stdout:5/531: fdatasync d6/dd/df/d58/d3f/f9e 0 2024-02-16T22:21:59.480 INFO:tasks.workunit.client.1.smithi123.stdout:1/524: chown d3/d6/d11 430004 1 2024-02-16T22:21:59.481 INFO:tasks.workunit.client.0.smithi114.stdout:4/529: write dd/d2e/d3f/f75 [1787039,43176] 0 2024-02-16T22:21:59.481 INFO:tasks.workunit.client.0.smithi114.stdout:3/368: truncate d4/d24/d2d/f4e 914801 0 2024-02-16T22:21:59.482 INFO:tasks.workunit.client.1.smithi123.stdout:7/567: truncate d3/d6e/d1d/db2/d35/f4b 5238741 0 2024-02-16T22:21:59.484 INFO:tasks.workunit.client.1.smithi123.stdout:8/597: rename d0/d29/da7/d26/d32/faf to d0/d29/da7/d43/d34/dae/fc1 0 2024-02-16T22:21:59.487 INFO:tasks.workunit.client.1.smithi123.stdout:0/496: sync 2024-02-16T22:21:59.489 INFO:tasks.workunit.client.1.smithi123.stdout:2/538: creat d4/d1a/d1d/d59/d53/fa8 x:0 0 0 2024-02-16T22:21:59.490 INFO:tasks.workunit.client.1.smithi123.stdout:6/561: creat d0/d8a/fcd x:0 0 0 2024-02-16T22:21:59.493 INFO:tasks.workunit.client.1.smithi123.stdout:8/598: dwrite d0/d12/fb5 [0,4194304] 0 2024-02-16T22:21:59.494 INFO:tasks.workunit.client.1.smithi123.stdout:3/553: dwrite d0/d20/d2e/f95 [0,4194304] 0 2024-02-16T22:21:59.496 INFO:tasks.workunit.client.0.smithi114.stdout:4/530: link dd/l54 dd/d2e/d3f/lbd 0 2024-02-16T22:21:59.497 INFO:tasks.workunit.client.1.smithi123.stdout:4/593: dread d5/d1c/d64/fb6 [0,4194304] 0 2024-02-16T22:21:59.499 INFO:tasks.workunit.client.1.smithi123.stdout:4/594: write d5/d1c/d42/fa8 [1107420,41319] 0 2024-02-16T22:21:59.502 INFO:tasks.workunit.client.1.smithi123.stdout:0/497: mknod d1/d4/d17/ca9 0 2024-02-16T22:21:59.505 INFO:tasks.workunit.client.0.smithi114.stdout:3/369: dread d4/d26/f31 [0,4194304] 0 2024-02-16T22:21:59.506 INFO:tasks.workunit.client.1.smithi123.stdout:9/660: dwrite d4/d36/d8e/d6f/f77 [0,4194304] 0 2024-02-16T22:21:59.508 INFO:tasks.workunit.client.0.smithi114.stdout:5/358: dwrite d0/df/d16/f49 [0,4194304] 0 2024-02-16T22:21:59.511 INFO:tasks.workunit.client.1.smithi123.stdout:5/532: symlink d6/dd/df/d1f/lb6 0 2024-02-16T22:21:59.513 INFO:tasks.workunit.client.1.smithi123.stdout:9/661: dwrite d4/d36/d8e/d6d/d71/d92/fcd [0,4194304] 0 2024-02-16T22:21:59.516 INFO:tasks.workunit.client.0.smithi114.stdout:3/370: dread d4/d24/f62 [0,4194304] 0 2024-02-16T22:21:59.533 INFO:tasks.workunit.client.0.smithi114.stdout:4/531: dread dd/d1b/f47 [0,4194304] 0 2024-02-16T22:21:59.534 INFO:tasks.workunit.client.1.smithi123.stdout:9/662: read d4/d1e/d3a/f9f [7497319,8185] 0 2024-02-16T22:21:59.534 INFO:tasks.workunit.client.0.smithi114.stdout:5/359: symlink d0/d28/l64 0 2024-02-16T22:21:59.538 INFO:tasks.workunit.client.0.smithi114.stdout:3/371: rmdir d4/d43/d53 39 2024-02-16T22:21:59.538 INFO:tasks.workunit.client.0.smithi114.stdout:3/372: stat d4/d24/f6d 0 2024-02-16T22:21:59.543 INFO:tasks.workunit.client.0.smithi114.stdout:3/373: dread - d4/dd/f35 zero size 2024-02-16T22:21:59.546 INFO:tasks.workunit.client.0.smithi114.stdout:3/374: symlink d4/dd/d33/l82 0 2024-02-16T22:21:59.548 INFO:tasks.workunit.client.1.smithi123.stdout:3/554: dread d0/f6 [0,4194304] 0 2024-02-16T22:21:59.548 INFO:tasks.workunit.client.1.smithi123.stdout:6/562: unlink d0/d9/dc/l3d 0 2024-02-16T22:21:59.550 INFO:tasks.workunit.client.1.smithi123.stdout:3/555: truncate d0/d20/d76/d5d/d60/d93/fb9 587943 0 2024-02-16T22:21:59.550 INFO:tasks.workunit.client.0.smithi114.stdout:3/375: mkdir d4/d26/d83 0 2024-02-16T22:21:59.551 INFO:tasks.workunit.client.0.smithi114.stdout:3/376: fdatasync d4/f18 0 2024-02-16T22:21:59.551 INFO:tasks.workunit.client.1.smithi123.stdout:3/556: fsync d0/d20/d76/f36 0 2024-02-16T22:21:59.551 INFO:tasks.workunit.client.1.smithi123.stdout:1/525: truncate d3/d6/d18/f7e 3702310 0 2024-02-16T22:21:59.551 INFO:tasks.workunit.client.1.smithi123.stdout:3/557: write d0/d85/f9f [705710,19508] 0 2024-02-16T22:21:59.552 INFO:tasks.workunit.client.1.smithi123.stdout:7/568: truncate d3/d6e/d24/f6d 2114487 0 2024-02-16T22:21:59.552 INFO:tasks.workunit.client.1.smithi123.stdout:7/569: chown d3 1002144895 1 2024-02-16T22:21:59.553 INFO:tasks.workunit.client.0.smithi114.stdout:5/360: write d0/d2d/d30/f31 [4712197,51923] 0 2024-02-16T22:21:59.553 INFO:tasks.workunit.client.1.smithi123.stdout:3/558: dread - d0/d20/d94/dac/fb1 zero size 2024-02-16T22:21:59.553 INFO:tasks.workunit.client.1.smithi123.stdout:7/570: chown d3/d6e/d1d/fb6 104 1 2024-02-16T22:21:59.555 INFO:tasks.workunit.client.0.smithi114.stdout:5/361: truncate d0/df/d16/d25/d3e/f44 1010721 0 2024-02-16T22:21:59.555 INFO:tasks.workunit.client.1.smithi123.stdout:3/559: dread - d0/d20/d76/d37/d49/f9c zero size 2024-02-16T22:21:59.555 INFO:tasks.workunit.client.0.smithi114.stdout:3/377: creat d4/dd/d33/f84 x:0 0 0 2024-02-16T22:21:59.557 INFO:tasks.workunit.client.1.smithi123.stdout:4/595: fsync d5/d1c/f30 0 2024-02-16T22:21:59.557 INFO:tasks.workunit.client.1.smithi123.stdout:1/526: dwrite d3/d6/d11/f23 [4194304,4194304] 0 2024-02-16T22:21:59.559 INFO:tasks.workunit.client.1.smithi123.stdout:1/527: stat d3/d6/db/l67 0 2024-02-16T22:21:59.560 INFO:tasks.workunit.client.1.smithi123.stdout:0/498: creat d1/da0/faa x:0 0 0 2024-02-16T22:21:59.561 INFO:tasks.workunit.client.1.smithi123.stdout:1/528: chown d3/f35 237 1 2024-02-16T22:21:59.563 INFO:tasks.workunit.client.1.smithi123.stdout:6/563: creat d0/d74/fce x:0 0 0 2024-02-16T22:21:59.565 INFO:tasks.workunit.client.1.smithi123.stdout:3/560: dwrite d0/d20/d76/d5d/dae/db0/fbe [0,4194304] 0 2024-02-16T22:21:59.565 INFO:tasks.workunit.client.1.smithi123.stdout:2/539: getdents d4/d1a/d1d/d13/d15 0 2024-02-16T22:21:59.566 INFO:tasks.workunit.client.1.smithi123.stdout:3/561: chown d0/d20/d94/dac/fb1 7507 1 2024-02-16T22:21:59.567 INFO:tasks.workunit.client.1.smithi123.stdout:3/562: chown d0/d50/lb5 18 1 2024-02-16T22:21:59.567 INFO:tasks.workunit.client.1.smithi123.stdout:3/563: fsync d0/d40/f56 0 2024-02-16T22:21:59.571 INFO:tasks.workunit.client.1.smithi123.stdout:8/599: rename d0/d29/da7/d43/d34/f6b to d0/d29/da7/d43/fc2 0 2024-02-16T22:21:59.572 INFO:tasks.workunit.client.0.smithi114.stdout:5/362: write d0/d28/f46 [5021418,106524] 0 2024-02-16T22:21:59.573 INFO:tasks.workunit.client.1.smithi123.stdout:9/663: mkdir d4/d36/daa/d2b/dd6 0 2024-02-16T22:21:59.574 INFO:tasks.workunit.client.0.smithi114.stdout:3/378: link d4/c2a d4/d24/d2d/d45/d68/d77/c85 0 2024-02-16T22:21:59.577 INFO:tasks.workunit.client.0.smithi114.stdout:3/379: creat d4/d24/d2d/d45/d68/f86 x:0 0 0 2024-02-16T22:21:59.580 INFO:tasks.workunit.client.0.smithi114.stdout:5/363: creat d0/f65 x:0 0 0 2024-02-16T22:21:59.581 INFO:tasks.workunit.client.0.smithi114.stdout:5/364: stat d0/df/d16/d25/l58 0 2024-02-16T22:21:59.581 INFO:tasks.workunit.client.0.smithi114.stdout:5/365: readlink d0/l6 0 2024-02-16T22:21:59.582 INFO:tasks.workunit.client.0.smithi114.stdout:5/366: write d0/df/f5d [88581,35924] 0 2024-02-16T22:21:59.583 INFO:tasks.workunit.client.0.smithi114.stdout:5/367: chown d0/df/d16/d25/d2e/d3b 66279 1 2024-02-16T22:21:59.590 INFO:tasks.workunit.client.0.smithi114.stdout:5/368: dwrite d0/df/d16/f49 [0,4194304] 0 2024-02-16T22:21:59.591 INFO:tasks.workunit.client.1.smithi123.stdout:4/596: rename d5/d1c/d23/d74/d92/d9b/cab to d5/d1c/d23/d6d/cb9 0 2024-02-16T22:21:59.593 INFO:tasks.workunit.client.1.smithi123.stdout:7/571: creat d3/d6e/d1d/db2/d35/db7/fbb x:0 0 0 2024-02-16T22:21:59.594 INFO:tasks.workunit.client.0.smithi114.stdout:5/369: fdatasync d0/d10/f13 0 2024-02-16T22:21:59.594 INFO:tasks.workunit.client.0.smithi114.stdout:3/380: write d4/d24/f25 [708746,86223] 0 2024-02-16T22:21:59.594 INFO:tasks.workunit.client.1.smithi123.stdout:5/533: sync 2024-02-16T22:21:59.595 INFO:tasks.workunit.client.0.smithi114.stdout:5/370: chown d0/df/d16/l48 11984918 1 2024-02-16T22:21:59.600 INFO:tasks.workunit.client.1.smithi123.stdout:6/564: symlink d0/d6/d1a/lcf 0 2024-02-16T22:21:59.601 INFO:tasks.workunit.client.0.smithi114.stdout:3/381: dwrite d4/d26/f58 [0,4194304] 0 2024-02-16T22:21:59.602 INFO:tasks.workunit.client.1.smithi123.stdout:9/664: dwrite d4/d36/d57/f7d [0,4194304] 0 2024-02-16T22:21:59.603 INFO:tasks.workunit.client.1.smithi123.stdout:3/564: link d0/d20/d94/dac/fb1 d0/d20/d76/d5d/dae/db0/fc3 0 2024-02-16T22:21:59.605 INFO:tasks.workunit.client.0.smithi114.stdout:5/371: creat d0/df/d16/d25/f66 x:0 0 0 2024-02-16T22:21:59.606 INFO:tasks.workunit.client.1.smithi123.stdout:4/597: mkdir d5/dba 0 2024-02-16T22:21:59.608 INFO:tasks.workunit.client.0.smithi114.stdout:3/382: unlink d4/d24/d2d/d45/f51 0 2024-02-16T22:21:59.610 INFO:tasks.workunit.client.1.smithi123.stdout:1/529: rename d3/d6/db/d10/c41 to d3/d6/d49/d48/cb0 0 2024-02-16T22:21:59.612 INFO:tasks.workunit.client.1.smithi123.stdout:1/530: read d3/d6/f9d [788296,68805] 0 2024-02-16T22:21:59.612 INFO:tasks.workunit.client.1.smithi123.stdout:5/534: symlink d6/dd/df/d58/d89/lb7 0 2024-02-16T22:21:59.613 INFO:tasks.workunit.client.0.smithi114.stdout:3/383: mkdir d4/d87 0 2024-02-16T22:21:59.615 INFO:tasks.workunit.client.1.smithi123.stdout:8/600: creat d0/d29/fc3 x:0 0 0 2024-02-16T22:21:59.618 INFO:tasks.workunit.client.1.smithi123.stdout:1/531: unlink d3/d6/db/d10/d2a/d53/c8f 0 2024-02-16T22:21:59.618 INFO:tasks.workunit.client.1.smithi123.stdout:0/499: getdents d1/d8/d13/d1d/d62 0 2024-02-16T22:21:59.618 INFO:tasks.workunit.client.0.smithi114.stdout:7/453: sync 2024-02-16T22:21:59.619 INFO:tasks.workunit.client.0.smithi114.stdout:6/435: sync 2024-02-16T22:21:59.619 INFO:tasks.workunit.client.1.smithi123.stdout:0/500: chown d1/d4/d17/c61 49 1 2024-02-16T22:21:59.619 INFO:tasks.workunit.client.1.smithi123.stdout:2/540: creat d4/d1a/d1d/d59/d45/fa9 x:0 0 0 2024-02-16T22:21:59.620 INFO:tasks.workunit.client.0.smithi114.stdout:6/436: readlink d6/dd/l39 0 2024-02-16T22:21:59.620 INFO:tasks.workunit.client.1.smithi123.stdout:6/565: symlink d0/d6/d2f/da5/dc6/ld0 0 2024-02-16T22:21:59.621 INFO:tasks.workunit.client.1.smithi123.stdout:2/541: stat d4/d1a/d1d/d13/d15 0 2024-02-16T22:21:59.621 INFO:tasks.workunit.client.1.smithi123.stdout:4/598: symlink d5/d1c/d23/d31/d98/lbb 0 2024-02-16T22:21:59.622 INFO:tasks.workunit.client.1.smithi123.stdout:2/542: write d4/d1a/f6a [590265,79120] 0 2024-02-16T22:21:59.623 INFO:tasks.workunit.client.1.smithi123.stdout:8/601: symlink d0/d29/da7/d26/d2a/d77/lc4 0 2024-02-16T22:21:59.623 INFO:tasks.workunit.client.0.smithi114.stdout:3/384: dread d4/d46/d48/f5c [0,4194304] 0 2024-02-16T22:21:59.623 INFO:tasks.workunit.client.0.smithi114.stdout:6/437: symlink d6/l88 0 2024-02-16T22:21:59.625 INFO:tasks.workunit.client.0.smithi114.stdout:7/454: creat d3/f99 x:0 0 0 2024-02-16T22:21:59.626 INFO:tasks.workunit.client.1.smithi123.stdout:1/532: rmdir d3/d6/d49/d48 39 2024-02-16T22:21:59.629 INFO:tasks.workunit.client.0.smithi114.stdout:7/455: mknod d3/d23/d6f/d89/c9a 0 2024-02-16T22:21:59.631 INFO:tasks.workunit.client.0.smithi114.stdout:5/372: dread d0/d10/f23 [0,4194304] 0 2024-02-16T22:21:59.631 INFO:tasks.workunit.client.1.smithi123.stdout:4/599: creat d5/d1c/d23/d56/d6e/fbc x:0 0 0 2024-02-16T22:21:59.632 INFO:tasks.workunit.client.1.smithi123.stdout:5/535: creat d6/fb8 x:0 0 0 2024-02-16T22:21:59.633 INFO:tasks.workunit.client.1.smithi123.stdout:8/602: dwrite d0/d29/da7/d26/d68/d81/d47/f9d [0,4194304] 0 2024-02-16T22:21:59.636 INFO:tasks.workunit.client.0.smithi114.stdout:5/373: creat d0/df/d16/d25/d2e/f67 x:0 0 0 2024-02-16T22:21:59.637 INFO:tasks.workunit.client.1.smithi123.stdout:7/572: dread d3/d6e/d52/f67 [0,4194304] 0 2024-02-16T22:21:59.639 INFO:tasks.workunit.client.0.smithi114.stdout:6/438: rename d6/dd/d25/d34/l47 to d6/d2c/d38/l89 0 2024-02-16T22:21:59.640 INFO:tasks.workunit.client.1.smithi123.stdout:4/600: mkdir d5/d1c/d23/d74/d92/d9b/dbd 0 2024-02-16T22:21:59.640 INFO:tasks.workunit.client.0.smithi114.stdout:6/439: write d6/d83/f87 [126716,47343] 0 2024-02-16T22:21:59.641 INFO:tasks.workunit.client.0.smithi114.stdout:6/440: chown d6/d2c/d38/d54/c6c 286878 1 2024-02-16T22:21:59.642 INFO:tasks.workunit.client.1.smithi123.stdout:6/566: dread d0/d6/d8e/fa0 [0,4194304] 0 2024-02-16T22:21:59.642 INFO:tasks.workunit.client.0.smithi114.stdout:5/374: creat d0/df/d16/d1e/d63/f68 x:0 0 0 2024-02-16T22:21:59.643 INFO:tasks.workunit.client.0.smithi114.stdout:3/385: dread d4/dd/f1e [0,4194304] 0 2024-02-16T22:21:59.644 INFO:tasks.workunit.client.1.smithi123.stdout:8/603: chown d0/la 61 1 2024-02-16T22:21:59.644 INFO:tasks.workunit.client.1.smithi123.stdout:0/501: dread d1/d8/d13/f5a [0,4194304] 0 2024-02-16T22:21:59.644 INFO:tasks.workunit.client.0.smithi114.stdout:5/375: dread d0/df/d16/d25/d3e/f44 [0,4194304] 0 2024-02-16T22:21:59.645 INFO:tasks.workunit.client.0.smithi114.stdout:5/376: read d0/d2d/d30/f40 [58516,119014] 0 2024-02-16T22:21:59.665 INFO:tasks.workunit.client.1.smithi123.stdout:7/573: write d3/d6e/d1d/f41 [5052756,84312] 0 2024-02-16T22:21:59.666 INFO:tasks.workunit.client.1.smithi123.stdout:7/574: chown d3/d5/c15 112 1 2024-02-16T22:21:59.672 INFO:tasks.workunit.client.1.smithi123.stdout:6/567: mknod d0/d6/cd1 0 2024-02-16T22:21:59.674 INFO:tasks.workunit.client.1.smithi123.stdout:6/568: fsync d0/d9/dc/d3a/fa1 0 2024-02-16T22:21:59.674 INFO:tasks.workunit.client.0.smithi114.stdout:3/386: mknod d4/d24/d2d/d45/d68/d77/c88 0 2024-02-16T22:21:59.674 INFO:tasks.workunit.client.1.smithi123.stdout:4/601: dwrite d5/d1c/d23/d31/f2d [0,4194304] 0 2024-02-16T22:21:59.677 INFO:tasks.workunit.client.1.smithi123.stdout:8/604: truncate d0/d61/d66/f86 284048 0 2024-02-16T22:21:59.678 INFO:tasks.workunit.client.0.smithi114.stdout:3/387: write d4/d43/f56 [332174,124845] 0 2024-02-16T22:21:59.684 INFO:tasks.workunit.client.0.smithi114.stdout:3/388: dread d4/dd/f64 [4194304,4194304] 0 2024-02-16T22:21:59.697 INFO:tasks.workunit.client.1.smithi123.stdout:1/533: write d3/d6/d11/f62 [506351,43808] 0 2024-02-16T22:21:59.698 INFO:tasks.workunit.client.1.smithi123.stdout:1/534: dread - d3/d47/f61 zero size 2024-02-16T22:21:59.699 INFO:tasks.workunit.client.0.smithi114.stdout:7/456: dwrite d3/d14/f18 [0,4194304] 0 2024-02-16T22:21:59.704 INFO:tasks.workunit.client.0.smithi114.stdout:7/457: mknod d3/d23/d4e/c9b 0 2024-02-16T22:21:59.708 INFO:tasks.workunit.client.1.smithi123.stdout:5/536: mknod d6/dd/df/d58/cb9 0 2024-02-16T22:21:59.709 INFO:tasks.workunit.client.1.smithi123.stdout:2/543: link d4/d1a/d1d/d13/d25/d98/l7a d4/laa 0 2024-02-16T22:21:59.710 INFO:tasks.workunit.client.1.smithi123.stdout:0/502: mknod d1/d4/d17/d54/d8e/da5/cab 0 2024-02-16T22:21:59.712 INFO:tasks.workunit.client.0.smithi114.stdout:5/377: dwrite d0/d2d/d30/f40 [0,4194304] 0 2024-02-16T22:21:59.712 INFO:tasks.workunit.client.1.smithi123.stdout:3/565: sync 2024-02-16T22:21:59.713 INFO:tasks.workunit.client.1.smithi123.stdout:9/665: sync 2024-02-16T22:21:59.713 INFO:tasks.workunit.client.1.smithi123.stdout:6/569: creat d0/d6/d1a/d88/fd2 x:0 0 0 2024-02-16T22:21:59.714 INFO:tasks.workunit.client.1.smithi123.stdout:3/566: write d0/fc [427662,28804] 0 2024-02-16T22:21:59.715 INFO:tasks.workunit.client.1.smithi123.stdout:8/605: creat d0/d29/da7/d26/d68/d81/d47/d9b/fc5 x:0 0 0 2024-02-16T22:21:59.715 INFO:tasks.workunit.client.1.smithi123.stdout:6/570: read - d0/d6/d2f/d5d/fa3 zero size 2024-02-16T22:21:59.716 INFO:tasks.workunit.client.1.smithi123.stdout:3/567: read d0/d20/d76/d5d/d60/d93/fb9 [316325,3272] 0 2024-02-16T22:21:59.717 INFO:tasks.workunit.client.1.smithi123.stdout:3/568: chown d0/d40/c6c 341934 1 2024-02-16T22:21:59.719 INFO:tasks.workunit.client.1.smithi123.stdout:2/544: symlink d4/d1a/d1d/d13/d15/lab 0 2024-02-16T22:21:59.722 INFO:tasks.workunit.client.1.smithi123.stdout:0/503: creat d1/d8/d3d/d44/fac x:0 0 0 2024-02-16T22:21:59.726 INFO:tasks.workunit.client.1.smithi123.stdout:9/666: mknod d4/d33/cd7 0 2024-02-16T22:21:59.726 INFO:tasks.workunit.client.1.smithi123.stdout:9/667: readlink d4/d1e/l88 0 2024-02-16T22:21:59.727 INFO:tasks.workunit.client.0.smithi114.stdout:5/378: getdents d0 0 2024-02-16T22:21:59.729 INFO:tasks.workunit.client.0.smithi114.stdout:3/389: dwrite d4/d24/f32 [0,4194304] 0 2024-02-16T22:21:59.730 INFO:tasks.workunit.client.1.smithi123.stdout:8/606: creat d0/d29/da7/d26/d2a/d4d/fc6 x:0 0 0 2024-02-16T22:21:59.731 INFO:tasks.workunit.client.1.smithi123.stdout:8/607: readlink d0/d29/lad 0 2024-02-16T22:21:59.731 INFO:tasks.workunit.client.0.smithi114.stdout:5/379: creat d0/df/d16/d25/d2e/f69 x:0 0 0 2024-02-16T22:21:59.733 INFO:tasks.workunit.client.0.smithi114.stdout:3/390: rename d4/l22 to d4/dd/d33/d72/l89 0 2024-02-16T22:21:59.733 INFO:tasks.workunit.client.0.smithi114.stdout:3/391: write d4/d6a/f7c [285782,122587] 0 2024-02-16T22:21:59.734 INFO:tasks.workunit.client.1.smithi123.stdout:3/569: mkdir d0/d20/d2e/dc4 0 2024-02-16T22:21:59.734 INFO:tasks.workunit.client.0.smithi114.stdout:5/380: chown d0/df/d16/d1e/c5f 368038460 1 2024-02-16T22:21:59.735 INFO:tasks.workunit.client.0.smithi114.stdout:3/392: write d4/d46/d48/f57 [3098121,51977] 0 2024-02-16T22:21:59.735 INFO:tasks.workunit.client.1.smithi123.stdout:3/570: fdatasync d0/d20/f26 0 2024-02-16T22:21:59.736 INFO:tasks.workunit.client.0.smithi114.stdout:5/381: truncate d0/df/f57 836852 0 2024-02-16T22:21:59.736 INFO:tasks.workunit.client.0.smithi114.stdout:3/393: write d4/dd/d33/f84 [337097,3450] 0 2024-02-16T22:21:59.737 INFO:tasks.workunit.client.1.smithi123.stdout:2/545: write d4/d1a/d1d/d13/d25/f66 [4912733,118290] 0 2024-02-16T22:21:59.738 INFO:tasks.workunit.client.1.smithi123.stdout:7/575: dread d3/d87/f5e [0,4194304] 0 2024-02-16T22:21:59.740 INFO:tasks.workunit.client.1.smithi123.stdout:7/576: chown d3/d87/da8 1 1 2024-02-16T22:21:59.740 INFO:tasks.workunit.client.1.smithi123.stdout:6/571: dread d0/d6/d1a/d2e/f50 [0,4194304] 0 2024-02-16T22:21:59.743 INFO:tasks.workunit.client.0.smithi114.stdout:7/458: write d3/d23/f52 [5236701,128969] 0 2024-02-16T22:21:59.744 INFO:tasks.workunit.client.0.smithi114.stdout:5/382: symlink d0/df/d16/d25/l6a 0 2024-02-16T22:21:59.745 INFO:tasks.workunit.client.1.smithi123.stdout:2/546: dread d4/d1a/d1d/d13/d25/f66 [4194304,4194304] 0 2024-02-16T22:21:59.746 INFO:tasks.workunit.client.1.smithi123.stdout:1/535: creat d3/d6/d49/fb1 x:0 0 0 2024-02-16T22:21:59.748 INFO:tasks.workunit.client.0.smithi114.stdout:7/459: mkdir d3/d23/d6f/d9c 0 2024-02-16T22:21:59.751 INFO:tasks.workunit.client.0.smithi114.stdout:3/394: creat d4/d43/f8a x:0 0 0 2024-02-16T22:21:59.753 INFO:tasks.workunit.client.1.smithi123.stdout:5/537: dwrite d6/dd/df/d1f/d25/f1a [0,4194304] 0 2024-02-16T22:21:59.753 INFO:tasks.workunit.client.0.smithi114.stdout:7/460: creat d3/d22/d61/f9d x:0 0 0 2024-02-16T22:21:59.756 INFO:tasks.workunit.client.1.smithi123.stdout:4/602: dwrite d5/d1c/d5e/f72 [0,4194304] 0 2024-02-16T22:21:59.756 INFO:tasks.workunit.client.1.smithi123.stdout:7/577: fdatasync d3/d6e/d24/f49 0 2024-02-16T22:21:59.757 INFO:tasks.workunit.client.1.smithi123.stdout:7/578: fsync d3/f8c 0 2024-02-16T22:21:59.757 INFO:tasks.workunit.client.1.smithi123.stdout:4/603: write d5/d1c/d23/d56/d6e/d73/f88 [2800427,42649] 0 2024-02-16T22:21:59.758 INFO:tasks.workunit.client.1.smithi123.stdout:6/572: symlink d0/d9/dc/d39/d8d/ld3 0 2024-02-16T22:21:59.761 INFO:tasks.workunit.client.0.smithi114.stdout:3/395: rename d4/dd/f29 to d4/d24/d2d/d45/d74/f8b 0 2024-02-16T22:21:59.762 INFO:tasks.workunit.client.0.smithi114.stdout:3/396: read - d4/dd/d33/d76/f7f zero size 2024-02-16T22:21:59.763 INFO:tasks.workunit.client.1.smithi123.stdout:0/504: write d1/d4/d17/f41 [359271,97472] 0 2024-02-16T22:21:59.764 INFO:tasks.workunit.client.0.smithi114.stdout:3/397: write d4/d46/d48/f57 [3253301,120051] 0 2024-02-16T22:21:59.766 INFO:tasks.workunit.client.1.smithi123.stdout:7/579: dwrite d3/d6e/d24/f55 [0,4194304] 0 2024-02-16T22:21:59.767 INFO:tasks.workunit.client.1.smithi123.stdout:2/547: truncate d4/d1a/d1d/d38/f58 131662 0 2024-02-16T22:21:59.770 INFO:tasks.workunit.client.1.smithi123.stdout:1/536: symlink d3/d6/db/d10/d2a/d53/lb2 0 2024-02-16T22:21:59.773 INFO:tasks.workunit.client.0.smithi114.stdout:7/461: dwrite d3/d87/f8e [0,4194304] 0 2024-02-16T22:21:59.776 INFO:tasks.workunit.client.0.smithi114.stdout:3/398: dwrite d4/d46/d48/f57 [0,4194304] 0 2024-02-16T22:21:59.778 INFO:tasks.workunit.client.0.smithi114.stdout:5/383: truncate d0/d28/f38 1392070 0 2024-02-16T22:21:59.781 INFO:tasks.workunit.client.0.smithi114.stdout:7/462: dread d3/d14/f5c [0,4194304] 0 2024-02-16T22:21:59.782 INFO:tasks.workunit.client.0.smithi114.stdout:7/463: chown d3/d22/d61/f9d 25948 1 2024-02-16T22:21:59.783 INFO:tasks.workunit.client.1.smithi123.stdout:5/538: readlink d6/dd/df/d1f/l2a 0 2024-02-16T22:21:59.786 INFO:tasks.workunit.client.0.smithi114.stdout:3/399: dread d4/dd/d33/f5f [0,4194304] 0 2024-02-16T22:21:59.786 INFO:tasks.workunit.client.1.smithi123.stdout:9/668: truncate d4/d36/d57/f7d 2790749 0 2024-02-16T22:21:59.787 INFO:tasks.workunit.client.1.smithi123.stdout:9/669: write d4/d33/f38 [5053799,90160] 0 2024-02-16T22:21:59.788 INFO:tasks.workunit.client.1.smithi123.stdout:9/670: chown d4/d36/d57/db2/l44 1060962410 1 2024-02-16T22:21:59.791 INFO:tasks.workunit.client.0.smithi114.stdout:3/400: mknod d4/d6a/c8c 0 2024-02-16T22:21:59.792 INFO:tasks.workunit.client.1.smithi123.stdout:6/573: fdatasync d0/d9/dc/d3a/f40 0 2024-02-16T22:21:59.792 INFO:tasks.workunit.client.0.smithi114.stdout:3/401: fdatasync d4/d24/f6d 0 2024-02-16T22:21:59.794 INFO:tasks.workunit.client.0.smithi114.stdout:5/384: rename d0/d10/c12 to d0/df/d16/d25/d3e/c6b 0 2024-02-16T22:21:59.794 INFO:tasks.workunit.client.1.smithi123.stdout:0/505: mkdir d1/d4/d17/dad 0 2024-02-16T22:21:59.796 INFO:tasks.workunit.client.1.smithi123.stdout:2/548: chown d4/d1a/d1d/l49 7462 1 2024-02-16T22:21:59.797 INFO:tasks.workunit.client.0.smithi114.stdout:5/385: truncate d0/df/d16/f1f 751949 0 2024-02-16T22:21:59.799 INFO:tasks.workunit.client.1.smithi123.stdout:3/571: rename d0/d40/f59 to d0/d20/d76/fc5 0 2024-02-16T22:21:59.800 INFO:tasks.workunit.client.0.smithi114.stdout:3/402: dread d4/d24/f67 [0,4194304] 0 2024-02-16T22:21:59.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:21:59 smithi123 ceph-mon[37161]: pgmap v27: 65 pgs: 65 active+clean; 1.8 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 54 MiB/s rd, 282 MiB/s wr, 465 op/s 2024-02-16T22:21:59.804 INFO:tasks.workunit.client.0.smithi114.stdout:3/403: chown d4/d24/d2d/d45/d68/d77/c85 620927525 1 2024-02-16T22:21:59.808 INFO:tasks.workunit.client.0.smithi114.stdout:5/386: dread d0/d28/f46 [0,4194304] 0 2024-02-16T22:21:59.809 INFO:tasks.workunit.client.0.smithi114.stdout:3/404: unlink d4/l10 0 2024-02-16T22:21:59.812 INFO:tasks.workunit.client.1.smithi123.stdout:8/608: sync 2024-02-16T22:21:59.813 INFO:tasks.workunit.client.0.smithi114.stdout:5/387: symlink d0/d2d/d30/l6c 0 2024-02-16T22:21:59.814 INFO:tasks.workunit.client.1.smithi123.stdout:7/580: mknod d3/d6e/d1d/db2/d35/dba/cbc 0 2024-02-16T22:21:59.817 INFO:tasks.workunit.client.0.smithi114.stdout:7/464: dwrite d3/d22/f48 [0,4194304] 0 2024-02-16T22:21:59.817 INFO:tasks.workunit.client.1.smithi123.stdout:4/604: dwrite d5/d1c/d23/f2a [0,4194304] 0 2024-02-16T22:21:59.820 INFO:tasks.workunit.client.0.smithi114.stdout:3/405: symlink d4/d43/l8d 0 2024-02-16T22:21:59.825 INFO:tasks.workunit.client.1.smithi123.stdout:5/539: write d6/dd/df/d1f/f64 [590327,49611] 0 2024-02-16T22:21:59.825 INFO:tasks.workunit.client.1.smithi123.stdout:1/537: dwrite d3/d6/db/d10/f29 [0,4194304] 0 2024-02-16T22:21:59.826 INFO:tasks.workunit.client.1.smithi123.stdout:1/538: readlink d3/d6/db/l20 0 2024-02-16T22:21:59.827 INFO:tasks.workunit.client.1.smithi123.stdout:5/540: dread - d6/dd/df/d90/fae zero size 2024-02-16T22:21:59.828 INFO:tasks.workunit.client.0.smithi114.stdout:7/465: unlink d3/l2e 0 2024-02-16T22:21:59.828 INFO:tasks.workunit.client.1.smithi123.stdout:1/539: readlink d3/d47/la5 0 2024-02-16T22:21:59.830 INFO:tasks.workunit.client.1.smithi123.stdout:3/572: read d0/d20/d29/f43 [304448,45050] 0 2024-02-16T22:21:59.831 INFO:tasks.workunit.client.1.smithi123.stdout:3/573: chown d0/f19 1726116 1 2024-02-16T22:21:59.834 INFO:tasks.workunit.client.0.smithi114.stdout:7/466: creat d3/d22/d61/d6a/f9e x:0 0 0 2024-02-16T22:21:59.837 INFO:tasks.workunit.client.0.smithi114.stdout:7/467: fdatasync d3/f27 0 2024-02-16T22:21:59.841 INFO:tasks.workunit.client.1.smithi123.stdout:0/506: creat d1/d8/da7/fae x:0 0 0 2024-02-16T22:21:59.842 INFO:tasks.workunit.client.1.smithi123.stdout:0/507: write d1/d4/d17/d54/f80 [1248585,25557] 0 2024-02-16T22:21:59.842 INFO:tasks.workunit.client.0.smithi114.stdout:3/406: getdents d4/dd/d33 0 2024-02-16T22:21:59.844 INFO:tasks.workunit.client.1.smithi123.stdout:5/541: mkdir d6/dd/d44/dba 0 2024-02-16T22:21:59.845 INFO:tasks.workunit.client.0.smithi114.stdout:1/396: sync 2024-02-16T22:21:59.845 INFO:tasks.workunit.client.0.smithi114.stdout:0/366: sync 2024-02-16T22:21:59.845 INFO:tasks.workunit.client.0.smithi114.stdout:8/404: sync 2024-02-16T22:21:59.845 INFO:tasks.workunit.client.0.smithi114.stdout:2/414: sync 2024-02-16T22:21:59.845 INFO:tasks.workunit.client.0.smithi114.stdout:9/462: sync 2024-02-16T22:21:59.845 INFO:tasks.workunit.client.0.smithi114.stdout:4/532: sync 2024-02-16T22:21:59.845 INFO:tasks.workunit.client.0.smithi114.stdout:6/441: sync 2024-02-16T22:21:59.846 INFO:tasks.workunit.client.0.smithi114.stdout:6/442: readlink d6/l3b 0 2024-02-16T22:21:59.846 INFO:tasks.workunit.client.0.smithi114.stdout:9/463: chown f1 1160 1 2024-02-16T22:21:59.846 INFO:tasks.workunit.client.0.smithi114.stdout:0/367: readlink d1/d16/d1f/l37 0 2024-02-16T22:21:59.847 INFO:tasks.workunit.client.1.smithi123.stdout:3/574: mkdir d0/d20/d76/d5d/d60/dc6 0 2024-02-16T22:21:59.851 INFO:tasks.workunit.client.0.smithi114.stdout:7/468: getdents d3/d14 0 2024-02-16T22:21:59.852 INFO:tasks.workunit.client.1.smithi123.stdout:6/574: creat d0/d6/d1a/fd4 x:0 0 0 2024-02-16T22:21:59.854 INFO:tasks.workunit.client.0.smithi114.stdout:8/405: symlink d4/d16/d25/d46/l92 0 2024-02-16T22:21:59.856 INFO:tasks.workunit.client.0.smithi114.stdout:8/406: readlink d4/d11/d52/l58 0 2024-02-16T22:21:59.858 INFO:tasks.workunit.client.0.smithi114.stdout:4/533: fdatasync dd/d1b/d24/f3d 0 2024-02-16T22:21:59.859 INFO:tasks.workunit.client.1.smithi123.stdout:8/609: dread d0/d29/da7/d26/d32/f56 [0,4194304] 0 2024-02-16T22:21:59.861 INFO:tasks.workunit.client.0.smithi114.stdout:5/388: dwrite d0/df/f1d [0,4194304] 0 2024-02-16T22:21:59.862 INFO:tasks.workunit.client.0.smithi114.stdout:8/407: fdatasync d4/d16/d25/d46/f73 0 2024-02-16T22:21:59.865 INFO:tasks.workunit.client.0.smithi114.stdout:7/469: dwrite d3/d23/d6f/f90 [0,4194304] 0 2024-02-16T22:21:59.866 INFO:tasks.workunit.client.0.smithi114.stdout:1/397: rename d4/d9/d46/d47/f73 to d4/d9/d65/d70/f89 0 2024-02-16T22:21:59.868 INFO:tasks.workunit.client.0.smithi114.stdout:6/443: dread d6/dd/d25/d34/d58/f78 [0,4194304] 0 2024-02-16T22:21:59.869 INFO:tasks.workunit.client.1.smithi123.stdout:9/671: rename d4/d36/d8e/d58/d59/f5c to d4/d36/daa/d20/fd8 0 2024-02-16T22:21:59.869 INFO:tasks.workunit.client.0.smithi114.stdout:3/407: dread d4/f1c [0,4194304] 0 2024-02-16T22:21:59.870 INFO:tasks.workunit.client.0.smithi114.stdout:2/415: fdatasync f0 0 2024-02-16T22:21:59.873 INFO:tasks.workunit.client.1.smithi123.stdout:1/540: creat d3/d6/db/d10/d4b/d70/d87/fb3 x:0 0 0 2024-02-16T22:21:59.874 INFO:tasks.workunit.client.1.smithi123.stdout:3/575: rmdir d0/d50/d7e 39 2024-02-16T22:21:59.878 INFO:tasks.workunit.client.1.smithi123.stdout:2/549: getdents d4/d1a/d1d/d13/d25/d98 0 2024-02-16T22:21:59.879 INFO:tasks.workunit.client.0.smithi114.stdout:2/416: dwrite dd/df/f11 [0,4194304] 0 2024-02-16T22:21:59.880 INFO:tasks.workunit.client.1.smithi123.stdout:8/610: creat d0/d61/d66/d7e/fc7 x:0 0 0 2024-02-16T22:21:59.884 INFO:tasks.workunit.client.0.smithi114.stdout:5/389: mkdir d0/df/d6d 0 2024-02-16T22:21:59.886 INFO:tasks.workunit.client.0.smithi114.stdout:8/408: creat d4/d24/d2c/d5f/f93 x:0 0 0 2024-02-16T22:21:59.886 INFO:tasks.workunit.client.1.smithi123.stdout:9/672: unlink d4/d33/cd7 0 2024-02-16T22:21:59.887 INFO:tasks.workunit.client.1.smithi123.stdout:0/508: write d1/f6a [1440754,47321] 0 2024-02-16T22:21:59.888 INFO:tasks.workunit.client.1.smithi123.stdout:0/509: chown d1/d8/d13/f37 30 1 2024-02-16T22:21:59.889 INFO:tasks.workunit.client.1.smithi123.stdout:5/542: fdatasync d6/dd/df/d1f/d25/f17 0 2024-02-16T22:21:59.892 INFO:tasks.workunit.client.0.smithi114.stdout:2/417: dread dd/d3b/f5c [0,4194304] 0 2024-02-16T22:21:59.892 INFO:tasks.workunit.client.0.smithi114.stdout:1/398: rmdir d4/d9/d3d 39 2024-02-16T22:21:59.894 INFO:tasks.workunit.client.0.smithi114.stdout:9/464: dwrite d2/d5c/d5f/d72/f79 [4194304,4194304] 0 2024-02-16T22:21:59.898 INFO:tasks.workunit.client.0.smithi114.stdout:5/390: dwrite d0/d2d/d30/f40 [0,4194304] 0 2024-02-16T22:21:59.901 INFO:tasks.workunit.client.0.smithi114.stdout:3/408: creat d4/d46/d48/f8e x:0 0 0 2024-02-16T22:21:59.903 INFO:tasks.workunit.client.1.smithi123.stdout:2/550: creat d4/d1a/d1d/d13/d15/fac x:0 0 0 2024-02-16T22:21:59.904 INFO:tasks.workunit.client.0.smithi114.stdout:0/368: creat d1/f74 x:0 0 0 2024-02-16T22:21:59.905 INFO:tasks.workunit.client.1.smithi123.stdout:8/611: fdatasync d0/d29/da7/d26/d32/f6f 0 2024-02-16T22:21:59.911 INFO:tasks.workunit.client.1.smithi123.stdout:4/605: sync 2024-02-16T22:21:59.911 INFO:tasks.workunit.client.1.smithi123.stdout:1/541: write d3/d6/db/d10/d2d/f5a [403349,25494] 0 2024-02-16T22:21:59.912 INFO:tasks.workunit.client.0.smithi114.stdout:0/369: dwrite d1/d16/d18/f59 [4194304,4194304] 0 2024-02-16T22:21:59.912 INFO:tasks.workunit.client.0.smithi114.stdout:6/444: write d6/f30 [2066829,109332] 0 2024-02-16T22:21:59.914 INFO:tasks.workunit.client.1.smithi123.stdout:5/543: rmdir d6/dd/df/d1f/d25/da2 39 2024-02-16T22:21:59.917 INFO:tasks.workunit.client.1.smithi123.stdout:6/575: truncate d0/d9/dc/d39/f5b 1996235 0 2024-02-16T22:21:59.918 INFO:tasks.workunit.client.1.smithi123.stdout:3/576: dwrite d0/f7 [0,4194304] 0 2024-02-16T22:21:59.919 INFO:tasks.workunit.client.0.smithi114.stdout:7/470: symlink d3/d22/l9f 0 2024-02-16T22:21:59.919 INFO:tasks.workunit.client.0.smithi114.stdout:7/471: write d3/d23/f52 [3224276,128467] 0 2024-02-16T22:21:59.920 INFO:tasks.workunit.client.0.smithi114.stdout:7/472: chown d3/fe 11 1 2024-02-16T22:21:59.920 INFO:tasks.workunit.client.1.smithi123.stdout:7/581: link d3/d6e/d1d/db2/d3e/c62 d3/d6e/d1d/db2/cbd 0 2024-02-16T22:21:59.925 INFO:tasks.workunit.client.1.smithi123.stdout:3/577: dwrite d0/d85/f9f [0,4194304] 0 2024-02-16T22:21:59.931 INFO:tasks.workunit.client.0.smithi114.stdout:8/409: write d4/d11/d78/d7a/f6c [526493,27605] 0 2024-02-16T22:21:59.932 INFO:tasks.workunit.client.0.smithi114.stdout:3/409: mkdir d4/dd/d33/d8f 0 2024-02-16T22:21:59.933 INFO:tasks.workunit.client.0.smithi114.stdout:3/410: stat d4/d24/c41 0 2024-02-16T22:21:59.934 INFO:tasks.workunit.client.0.smithi114.stdout:3/411: write d4/d26/f34 [2172673,96571] 0 2024-02-16T22:21:59.938 INFO:tasks.workunit.client.1.smithi123.stdout:8/612: write d0/d29/da7/f35 [1519854,1192] 0 2024-02-16T22:21:59.943 INFO:tasks.workunit.client.0.smithi114.stdout:0/370: unlink d1/d16/d48/f71 0 2024-02-16T22:21:59.944 INFO:tasks.workunit.client.0.smithi114.stdout:0/371: read - d1/d16/f5d zero size 2024-02-16T22:21:59.946 INFO:tasks.workunit.client.1.smithi123.stdout:9/673: link d4/d36/d8e/d6d/ca5 d4/d36/d8e/dbe/cd9 0 2024-02-16T22:21:59.948 INFO:tasks.workunit.client.0.smithi114.stdout:7/473: creat d3/d92/fa0 x:0 0 0 2024-02-16T22:21:59.950 INFO:tasks.workunit.client.1.smithi123.stdout:0/510: creat d1/d8/faf x:0 0 0 2024-02-16T22:21:59.950 INFO:tasks.workunit.client.0.smithi114.stdout:0/372: dwrite d1/d16/f5d [0,4194304] 0 2024-02-16T22:21:59.950 INFO:tasks.workunit.client.0.smithi114.stdout:9/465: truncate d2/f4 1533841 0 2024-02-16T22:21:59.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:21:59 smithi114 ceph-mon[29235]: pgmap v27: 65 pgs: 65 active+clean; 1.8 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 54 MiB/s rd, 282 MiB/s wr, 465 op/s 2024-02-16T22:21:59.952 INFO:tasks.workunit.client.0.smithi114.stdout:5/391: truncate d0/df/d16/d1e/f27 779036 0 2024-02-16T22:21:59.953 INFO:tasks.workunit.client.1.smithi123.stdout:1/542: mkdir d3/d6/db/d10/d4f/d9f/db4 0 2024-02-16T22:21:59.954 INFO:tasks.workunit.client.1.smithi123.stdout:5/544: mkdir d6/dd/dbb 0 2024-02-16T22:21:59.956 INFO:tasks.workunit.client.0.smithi114.stdout:5/392: dread d0/d10/f23 [4194304,4194304] 0 2024-02-16T22:21:59.958 INFO:tasks.workunit.client.0.smithi114.stdout:4/534: link dd/d1b/d41/d74/d9b/fbc dd/d1b/d41/d74/fbe 0 2024-02-16T22:21:59.958 INFO:tasks.workunit.client.0.smithi114.stdout:5/393: chown d0/df/d16/d25/l32 60610970 1 2024-02-16T22:21:59.959 INFO:tasks.workunit.client.0.smithi114.stdout:4/535: write dd/d59/f71 [113829,50930] 0 2024-02-16T22:21:59.960 INFO:tasks.workunit.client.1.smithi123.stdout:2/551: rename d4/d1a/d1d/d59/d45/c31 to d4/cad 0 2024-02-16T22:21:59.962 INFO:tasks.workunit.client.1.smithi123.stdout:2/552: truncate d4/d1a/d1d/d13/d15/d6d/fa0 492280 0 2024-02-16T22:21:59.964 INFO:tasks.workunit.client.1.smithi123.stdout:6/576: dread d0/f6d [0,4194304] 0 2024-02-16T22:21:59.966 INFO:tasks.workunit.client.0.smithi114.stdout:2/418: rename dd/f3f to dd/f92 0 2024-02-16T22:21:59.967 INFO:tasks.workunit.client.1.smithi123.stdout:3/578: mkdir d0/d20/d76/d5d/dae/dc7 0 2024-02-16T22:21:59.968 INFO:tasks.workunit.client.0.smithi114.stdout:1/399: creat d4/d9/f8a x:0 0 0 2024-02-16T22:21:59.970 INFO:tasks.workunit.client.1.smithi123.stdout:9/674: creat d4/d36/d8e/d6f/d7e/fda x:0 0 0 2024-02-16T22:21:59.973 INFO:tasks.workunit.client.1.smithi123.stdout:0/511: creat d1/d4/d17/d54/d8e/da5/fb0 x:0 0 0 2024-02-16T22:21:59.975 INFO:tasks.workunit.client.1.smithi123.stdout:4/606: mknod d5/d1c/d9a/cbe 0 2024-02-16T22:21:59.975 INFO:tasks.workunit.client.1.smithi123.stdout:4/607: write d5/f63 [1019549,26370] 0 2024-02-16T22:21:59.979 INFO:tasks.workunit.client.0.smithi114.stdout:9/466: rename d2/d1a/d23/d35/d47/f93 to d2/d5c/d5f/d91/f97 0 2024-02-16T22:21:59.980 INFO:tasks.workunit.client.1.smithi123.stdout:8/613: write d0/d29/da7/d26/d68/d81/d47/f82 [518572,13623] 0 2024-02-16T22:21:59.980 INFO:tasks.workunit.client.1.smithi123.stdout:1/543: chown d3/d6/d49/d48/d68/c6d 291042 1 2024-02-16T22:21:59.980 INFO:tasks.workunit.client.0.smithi114.stdout:9/467: dread d2/d1a/d23/d35/d47/d63/f6a [0,4194304] 0 2024-02-16T22:21:59.982 INFO:tasks.workunit.client.1.smithi123.stdout:5/545: rmdir d6/dd/d44 39 2024-02-16T22:21:59.983 INFO:tasks.workunit.client.1.smithi123.stdout:7/582: truncate d3/d6e/d34/f75 1063573 0 2024-02-16T22:21:59.987 INFO:tasks.workunit.client.0.smithi114.stdout:5/394: mkdir d0/d28/d6e 0 2024-02-16T22:21:59.988 INFO:tasks.workunit.client.0.smithi114.stdout:5/395: chown d0/d10/f42 38182 1 2024-02-16T22:21:59.988 INFO:tasks.workunit.client.1.smithi123.stdout:7/583: dwrite d3/d6e/d1d/fab [0,4194304] 0 2024-02-16T22:21:59.988 INFO:tasks.workunit.client.1.smithi123.stdout:6/577: mkdir d0/d9/dc/d3a/d70/dd5 0 2024-02-16T22:21:59.989 INFO:tasks.workunit.client.0.smithi114.stdout:5/396: write d0/df/d16/f5b [704192,957] 0 2024-02-16T22:21:59.990 INFO:tasks.workunit.client.1.smithi123.stdout:7/584: write d3/d6e/d1d/fb6 [556237,99093] 0 2024-02-16T22:21:59.990 INFO:tasks.workunit.client.1.smithi123.stdout:2/553: read d4/d1a/d1d/d13/d25/f66 [2760443,34375] 0 2024-02-16T22:21:59.994 INFO:tasks.workunit.client.0.smithi114.stdout:8/410: truncate d4/d24/f64 939842 0 2024-02-16T22:21:59.996 INFO:tasks.workunit.client.0.smithi114.stdout:8/411: chown d4/d11/d78/d7a/d36/f6b 1106 1 2024-02-16T22:21:59.997 INFO:tasks.workunit.client.0.smithi114.stdout:7/474: dwrite d3/d14/f70 [0,4194304] 0 2024-02-16T22:22:00.000 INFO:tasks.workunit.client.0.smithi114.stdout:3/412: write d4/d24/d2d/d45/d74/f8b [974801,71478] 0 2024-02-16T22:22:00.001 INFO:tasks.workunit.client.1.smithi123.stdout:4/608: rename d5/d1c/d23/d56/d6e/fbc to d5/dba/fbf 0 2024-02-16T22:22:00.005 INFO:tasks.workunit.client.0.smithi114.stdout:7/475: dwrite d3/d3d/f4a [0,4194304] 0 2024-02-16T22:22:00.011 INFO:tasks.workunit.client.1.smithi123.stdout:6/578: unlink d0/d9/dc/d3a/d52/fa7 0 2024-02-16T22:22:00.012 INFO:tasks.workunit.client.1.smithi123.stdout:5/546: dread d6/dd/df/f62 [0,4194304] 0 2024-02-16T22:22:00.013 INFO:tasks.workunit.client.0.smithi114.stdout:0/373: rename d1/d16/d1f/d28/l6b to d1/d16/d1f/d28/d51/d61/l75 0 2024-02-16T22:22:00.019 INFO:tasks.workunit.client.1.smithi123.stdout:9/675: mknod d4/d1e/cdb 0 2024-02-16T22:22:00.019 INFO:tasks.workunit.client.0.smithi114.stdout:2/419: dwrite dd/df/f33 [0,4194304] 0 2024-02-16T22:22:00.020 INFO:tasks.workunit.client.1.smithi123.stdout:9/676: fdatasync d4/d33/f73 0 2024-02-16T22:22:00.021 INFO:tasks.workunit.client.1.smithi123.stdout:7/585: mkdir d3/d6e/d1d/db2/d35/dbe 0 2024-02-16T22:22:00.021 INFO:tasks.workunit.client.1.smithi123.stdout:8/614: write d0/d12/f3f [2305308,43724] 0 2024-02-16T22:22:00.022 INFO:tasks.workunit.client.1.smithi123.stdout:8/615: stat d0/d29/da7/d43/f28 0 2024-02-16T22:22:00.023 INFO:tasks.workunit.client.0.smithi114.stdout:9/468: rmdir d2/d5c/d5f 39 2024-02-16T22:22:00.027 INFO:tasks.workunit.client.0.smithi114.stdout:4/536: link dd/d59/f71 dd/d1b/d37/d38/d87/dad/fbf 0 2024-02-16T22:22:00.028 INFO:tasks.workunit.client.1.smithi123.stdout:7/586: dwrite d3/d6e/d1d/fb6 [0,4194304] 0 2024-02-16T22:22:00.029 INFO:tasks.workunit.client.1.smithi123.stdout:2/554: dread d4/d1a/d1d/f75 [0,4194304] 0 2024-02-16T22:22:00.030 INFO:tasks.workunit.client.0.smithi114.stdout:6/445: getdents d6 0 2024-02-16T22:22:00.031 INFO:tasks.workunit.client.1.smithi123.stdout:2/555: read d4/d1a/d1d/f75 [482630,129205] 0 2024-02-16T22:22:00.031 INFO:tasks.workunit.client.1.smithi123.stdout:2/556: stat d4/d1a/d1d/d59/d53 0 2024-02-16T22:22:00.033 INFO:tasks.workunit.client.1.smithi123.stdout:1/544: dread d3/f5 [4194304,4194304] 0 2024-02-16T22:22:00.034 INFO:tasks.workunit.client.0.smithi114.stdout:1/400: creat d4/d63/d69/d7c/f8b x:0 0 0 2024-02-16T22:22:00.035 INFO:tasks.workunit.client.0.smithi114.stdout:1/401: read - d4/d33/f38 zero size 2024-02-16T22:22:00.036 INFO:tasks.workunit.client.0.smithi114.stdout:7/476: rmdir d3/d22/d76 39 2024-02-16T22:22:00.037 INFO:tasks.workunit.client.1.smithi123.stdout:3/579: creat d0/fc8 x:0 0 0 2024-02-16T22:22:00.038 INFO:tasks.workunit.client.0.smithi114.stdout:8/412: dwrite d4/d24/d2c/f56 [0,4194304] 0 2024-02-16T22:22:00.044 INFO:tasks.workunit.client.0.smithi114.stdout:4/537: rename dd/d1b/d24/d57/c58 to dd/d1b/d37/d99/cc0 0 2024-02-16T22:22:00.045 INFO:tasks.workunit.client.1.smithi123.stdout:8/616: write d0/d29/da7/d26/d2a/f92 [3125804,1064] 0 2024-02-16T22:22:00.047 INFO:tasks.workunit.client.0.smithi114.stdout:6/446: mkdir d6/d2c/d38/d54/d8a 0 2024-02-16T22:22:00.049 INFO:tasks.workunit.client.1.smithi123.stdout:0/512: rmdir d1/d4/d9d 0 2024-02-16T22:22:00.053 INFO:tasks.workunit.client.1.smithi123.stdout:7/587: rename d3/d5/f45 to d3/d6e/d1d/db2/d3e/fbf 0 2024-02-16T22:22:00.054 INFO:tasks.workunit.client.0.smithi114.stdout:1/402: creat d4/d9/d46/d47/f8c x:0 0 0 2024-02-16T22:22:00.055 INFO:tasks.workunit.client.1.smithi123.stdout:7/588: dread - d3/d6e/d1d/db2/d35/f5c zero size 2024-02-16T22:22:00.055 INFO:tasks.workunit.client.0.smithi114.stdout:8/413: mknod d4/d16/d25/d46/c94 0 2024-02-16T22:22:00.060 INFO:tasks.workunit.client.1.smithi123.stdout:1/545: dread d3/f35 [0,4194304] 0 2024-02-16T22:22:00.060 INFO:tasks.workunit.client.1.smithi123.stdout:4/609: mkdir d5/d1c/d23/d56/d6e/d9f/da1/dc0 0 2024-02-16T22:22:00.062 INFO:tasks.workunit.client.0.smithi114.stdout:8/414: dwrite d4/d24/d2c/f85 [0,4194304] 0 2024-02-16T22:22:00.064 INFO:tasks.workunit.client.1.smithi123.stdout:6/579: symlink d0/d9/dc/d5f/dcc/ld6 0 2024-02-16T22:22:00.065 INFO:tasks.workunit.client.1.smithi123.stdout:6/580: readlink d0/d6/d4d/db0/lc4 0 2024-02-16T22:22:00.066 INFO:tasks.workunit.client.1.smithi123.stdout:5/547: symlink d6/dd/df/d76/d7f/lbc 0 2024-02-16T22:22:00.067 INFO:tasks.workunit.client.1.smithi123.stdout:3/580: mknod d0/d20/d76/d5d/d60/d93/cc9 0 2024-02-16T22:22:00.067 INFO:tasks.workunit.client.1.smithi123.stdout:3/581: fdatasync d0/f19 0 2024-02-16T22:22:00.069 INFO:tasks.workunit.client.0.smithi114.stdout:6/447: mkdir d6/dd/d25/d34/d61/d62/d8b 0 2024-02-16T22:22:00.070 INFO:tasks.workunit.client.0.smithi114.stdout:9/469: dwrite d2/fc [0,4194304] 0 2024-02-16T22:22:00.070 INFO:tasks.workunit.client.0.smithi114.stdout:6/448: readlink d6/da/l44 0 2024-02-16T22:22:00.071 INFO:tasks.workunit.client.0.smithi114.stdout:9/470: stat d2/d1a/d23/d35/d1d/l53 0 2024-02-16T22:22:00.074 INFO:tasks.workunit.client.1.smithi123.stdout:8/617: stat d0/c1f 0 2024-02-16T22:22:00.077 INFO:tasks.workunit.client.0.smithi114.stdout:7/477: symlink d3/d22/d76/la1 0 2024-02-16T22:22:00.078 INFO:tasks.workunit.client.1.smithi123.stdout:2/557: write d4/d1a/d1d/d13/d25/f9a [1036214,24426] 0 2024-02-16T22:22:00.083 INFO:tasks.workunit.client.0.smithi114.stdout:0/374: creat d1/d1a/f76 x:0 0 0 2024-02-16T22:22:00.084 INFO:tasks.workunit.client.1.smithi123.stdout:9/677: write d4/d36/d8e/f61 [965430,101408] 0 2024-02-16T22:22:00.088 INFO:tasks.workunit.client.1.smithi123.stdout:4/610: write d5/d49/fb7 [3884,58892] 0 2024-02-16T22:22:00.088 INFO:tasks.workunit.client.0.smithi114.stdout:5/397: getdents d0/df/d16/d25/d2e/d3b 0 2024-02-16T22:22:00.090 INFO:tasks.workunit.client.1.smithi123.stdout:1/546: write d3/d6/db/d10/f85 [1138383,72984] 0 2024-02-16T22:22:00.090 INFO:tasks.workunit.client.0.smithi114.stdout:5/398: dread - d0/df/d16/d25/d2e/f69 zero size 2024-02-16T22:22:00.092 INFO:tasks.workunit.client.1.smithi123.stdout:8/618: fdatasync d0/d29/da7/d26/d2a/f54 0 2024-02-16T22:22:00.094 INFO:tasks.workunit.client.0.smithi114.stdout:6/449: mkdir d6/d2c/d38/d54/d8c 0 2024-02-16T22:22:00.094 INFO:tasks.workunit.client.1.smithi123.stdout:0/513: mknod d1/cb1 0 2024-02-16T22:22:00.097 INFO:tasks.workunit.client.1.smithi123.stdout:7/589: mknod d3/d6e/d1d/cc0 0 2024-02-16T22:22:00.097 INFO:tasks.workunit.client.0.smithi114.stdout:3/413: getdents d4/d24/d2d/d45/d74 0 2024-02-16T22:22:00.098 INFO:tasks.workunit.client.0.smithi114.stdout:3/414: readlink d4/d26/l3f 0 2024-02-16T22:22:00.099 INFO:tasks.workunit.client.0.smithi114.stdout:7/478: rmdir d3/d3d 39 2024-02-16T22:22:00.100 INFO:tasks.workunit.client.0.smithi114.stdout:1/403: mknod d4/d63/d69/d72/d82/c8d 0 2024-02-16T22:22:00.102 INFO:tasks.workunit.client.1.smithi123.stdout:9/678: unlink d4/d36/daa/l3e 0 2024-02-16T22:22:00.103 INFO:tasks.workunit.client.0.smithi114.stdout:0/375: truncate d1/d16/f24 1602246 0 2024-02-16T22:22:00.104 INFO:tasks.workunit.client.1.smithi123.stdout:6/581: truncate d0/d6/d8e/fc0 4409426 0 2024-02-16T22:22:00.105 INFO:tasks.workunit.client.0.smithi114.stdout:2/420: getdents dd/d3b 0 2024-02-16T22:22:00.106 INFO:tasks.workunit.client.1.smithi123.stdout:5/548: creat d6/dd/d44/fbd x:0 0 0 2024-02-16T22:22:00.107 INFO:tasks.workunit.client.0.smithi114.stdout:8/415: symlink d4/d16/d25/l95 0 2024-02-16T22:22:00.107 INFO:tasks.workunit.client.1.smithi123.stdout:4/611: creat d5/d1c/d23/d31/fc1 x:0 0 0 2024-02-16T22:22:00.109 INFO:tasks.workunit.client.0.smithi114.stdout:4/538: rename dd/d2e/d56/lb7 to dd/d1b/d37/lc1 0 2024-02-16T22:22:00.109 INFO:tasks.workunit.client.1.smithi123.stdout:3/582: truncate d0/d20/d76/d37/f7f 3325401 0 2024-02-16T22:22:00.111 INFO:tasks.workunit.client.0.smithi114.stdout:6/450: dread d6/da/f1b [0,4194304] 0 2024-02-16T22:22:00.112 INFO:tasks.workunit.client.1.smithi123.stdout:1/547: unlink d3/d6/l36 0 2024-02-16T22:22:00.112 INFO:tasks.workunit.client.0.smithi114.stdout:4/539: chown dd/d1b/d24/f36 0 1 2024-02-16T22:22:00.112 INFO:tasks.workunit.client.0.smithi114.stdout:0/376: dwrite d1/d16/d48/f69 [0,4194304] 0 2024-02-16T22:22:00.113 INFO:tasks.workunit.client.0.smithi114.stdout:0/377: stat d1/d16 0 2024-02-16T22:22:00.113 INFO:tasks.workunit.client.0.smithi114.stdout:6/451: chown d6/dd/l55 6955461 1 2024-02-16T22:22:00.113 INFO:tasks.workunit.client.1.smithi123.stdout:4/612: dwrite d5/d1c/d64/d7b/fb1 [0,4194304] 0 2024-02-16T22:22:00.115 INFO:tasks.workunit.client.0.smithi114.stdout:0/378: write d1/d16/d18/f59 [7019380,87664] 0 2024-02-16T22:22:00.116 INFO:tasks.workunit.client.1.smithi123.stdout:8/619: rename d0/d12/d3e/f49 to d0/d29/da7/d26/d2a/d4d/fc8 0 2024-02-16T22:22:00.116 INFO:tasks.workunit.client.1.smithi123.stdout:4/613: read d5/d1c/d23/f57 [3946745,90323] 0 2024-02-16T22:22:00.116 INFO:tasks.workunit.client.1.smithi123.stdout:8/620: chown d0/d29 393619519 1 2024-02-16T22:22:00.117 INFO:tasks.workunit.client.1.smithi123.stdout:4/614: truncate d5/f1d 5319370 0 2024-02-16T22:22:00.120 INFO:tasks.workunit.client.1.smithi123.stdout:7/590: unlink d3/d6e/d1d/db2/d35/f77 0 2024-02-16T22:22:00.124 INFO:tasks.workunit.client.0.smithi114.stdout:9/471: dwrite d2/d5c/d5f/d91/f97 [0,4194304] 0 2024-02-16T22:22:00.125 INFO:tasks.workunit.client.0.smithi114.stdout:8/416: truncate d4/f3c 1228770 0 2024-02-16T22:22:00.126 INFO:tasks.workunit.client.0.smithi114.stdout:6/452: dwrite d6/d2c/f7c [0,4194304] 0 2024-02-16T22:22:00.128 INFO:tasks.workunit.client.1.smithi123.stdout:9/679: write d4/d1e/fc8 [1987747,11382] 0 2024-02-16T22:22:00.128 INFO:tasks.workunit.client.0.smithi114.stdout:2/421: rename dd/d4e/f58 to dd/d3b/d8a/f93 0 2024-02-16T22:22:00.128 INFO:tasks.workunit.client.0.smithi114.stdout:6/453: truncate d6/f18 4264690 0 2024-02-16T22:22:00.129 INFO:tasks.workunit.client.1.smithi123.stdout:9/680: write d4/d36/d8e/d58/d6e/f9b [91171,79670] 0 2024-02-16T22:22:00.131 INFO:tasks.workunit.client.0.smithi114.stdout:5/399: link d0/df/d16/d1e/d63/f68 d0/df/d53/f6f 0 2024-02-16T22:22:00.132 INFO:tasks.workunit.client.0.smithi114.stdout:2/422: write dd/df/d1c/d21/d87/f4d [234504,57354] 0 2024-02-16T22:22:00.135 INFO:tasks.workunit.client.0.smithi114.stdout:5/400: chown d0/df/d16/d25/d3e/f44 43329395 1 2024-02-16T22:22:00.139 INFO:tasks.workunit.client.1.smithi123.stdout:5/549: dwrite d6/dd/f73 [4194304,4194304] 0 2024-02-16T22:22:00.151 INFO:tasks.workunit.client.0.smithi114.stdout:4/540: creat dd/d1b/d37/d38/d87/fc2 x:0 0 0 2024-02-16T22:22:00.152 INFO:tasks.workunit.client.0.smithi114.stdout:4/541: write dd/d1b/d37/f73 [4554744,40912] 0 2024-02-16T22:22:00.157 INFO:tasks.workunit.client.1.smithi123.stdout:1/548: unlink d3/d6/db/l40 0 2024-02-16T22:22:00.164 INFO:tasks.workunit.client.0.smithi114.stdout:0/379: rmdir d1/d16/d18 39 2024-02-16T22:22:00.166 INFO:tasks.workunit.client.1.smithi123.stdout:0/514: mkdir d1/d4/d17/dad/db2 0 2024-02-16T22:22:00.166 INFO:tasks.workunit.client.1.smithi123.stdout:2/558: truncate d4/d1a/d1d/f8f 4725 0 2024-02-16T22:22:00.173 INFO:tasks.workunit.client.1.smithi123.stdout:8/621: fdatasync d0/df/f96 0 2024-02-16T22:22:00.175 INFO:tasks.workunit.client.0.smithi114.stdout:7/479: mkdir d3/d46/da2 0 2024-02-16T22:22:00.176 INFO:tasks.workunit.client.0.smithi114.stdout:7/480: dread - d3/d14/f77 zero size 2024-02-16T22:22:00.177 INFO:tasks.workunit.client.0.smithi114.stdout:1/404: truncate d4/fa 948361 0 2024-02-16T22:22:00.177 INFO:tasks.workunit.client.0.smithi114.stdout:7/481: write d3/d14/f77 [665556,124004] 0 2024-02-16T22:22:00.178 INFO:tasks.workunit.client.1.smithi123.stdout:4/615: creat d5/d1c/d23/d74/d92/d9b/fc2 x:0 0 0 2024-02-16T22:22:00.180 INFO:tasks.workunit.client.0.smithi114.stdout:8/417: mknod d4/d24/d2c/d5f/c96 0 2024-02-16T22:22:00.183 INFO:tasks.workunit.client.0.smithi114.stdout:8/418: fdatasync d4/d11/d78/f90 0 2024-02-16T22:22:00.184 INFO:tasks.workunit.client.1.smithi123.stdout:6/582: mknod d0/d9/dc/cd7 0 2024-02-16T22:22:00.184 INFO:tasks.workunit.client.0.smithi114.stdout:7/482: dwrite d3/d14/f70 [0,4194304] 0 2024-02-16T22:22:00.185 INFO:tasks.workunit.client.0.smithi114.stdout:7/483: truncate d3/f6 9226455 0 2024-02-16T22:22:00.187 INFO:tasks.workunit.client.0.smithi114.stdout:6/454: creat d6/dd/d25/d34/d61/d62/f8d x:0 0 0 2024-02-16T22:22:00.190 INFO:tasks.workunit.client.1.smithi123.stdout:5/550: rename d6/dd/df/d1f/d25 to d6/dd/df/d58/d3f/dbe 0 2024-02-16T22:22:00.190 INFO:tasks.workunit.client.0.smithi114.stdout:6/455: read d6/f30 [167541,18738] 0 2024-02-16T22:22:00.191 INFO:tasks.workunit.client.1.smithi123.stdout:3/583: symlink d0/d20/d5a/da6/lca 0 2024-02-16T22:22:00.192 INFO:tasks.workunit.client.0.smithi114.stdout:2/423: read f0 [1104662,102246] 0 2024-02-16T22:22:00.194 INFO:tasks.workunit.client.0.smithi114.stdout:2/424: chown dd/df/d1c/d21/d87/l68 3016954 1 2024-02-16T22:22:00.195 INFO:tasks.workunit.client.0.smithi114.stdout:2/425: chown dd/df/c19 972 1 2024-02-16T22:22:00.196 INFO:tasks.workunit.client.0.smithi114.stdout:5/401: dread d0/df/d16/d25/d2e/f56 [0,4194304] 0 2024-02-16T22:22:00.197 INFO:tasks.workunit.client.1.smithi123.stdout:1/549: mkdir d3/d6/d49/d48/d84/db5 0 2024-02-16T22:22:00.197 INFO:tasks.workunit.client.1.smithi123.stdout:3/584: dwrite d0/fe [4194304,4194304] 0 2024-02-16T22:22:00.198 INFO:tasks.workunit.client.1.smithi123.stdout:3/585: fsync d0/d20/d76/d37/f5e 0 2024-02-16T22:22:00.198 INFO:tasks.workunit.client.0.smithi114.stdout:4/542: rmdir dd/d1b/d41/d74 39 2024-02-16T22:22:00.202 INFO:tasks.workunit.client.1.smithi123.stdout:1/550: dwrite d3/d6/d49/fb1 [0,4194304] 0 2024-02-16T22:22:00.203 INFO:tasks.workunit.client.1.smithi123.stdout:1/551: chown d3/d6/d49 2 1 2024-02-16T22:22:00.203 INFO:tasks.workunit.client.0.smithi114.stdout:2/426: dwrite dd/f17 [0,4194304] 0 2024-02-16T22:22:00.204 INFO:tasks.workunit.client.1.smithi123.stdout:0/515: creat d1/d8/d3d/d44/fb3 x:0 0 0 2024-02-16T22:22:00.209 INFO:tasks.workunit.client.0.smithi114.stdout:0/380: chown d1/d16/d18/f59 24937 1 2024-02-16T22:22:00.210 INFO:tasks.workunit.client.0.smithi114.stdout:3/415: dwrite d4/d24/d2d/f4e [0,4194304] 0 2024-02-16T22:22:00.211 INFO:tasks.workunit.client.0.smithi114.stdout:0/381: chown d1/d16/d18/l27 31289 1 2024-02-16T22:22:00.212 INFO:tasks.workunit.client.1.smithi123.stdout:7/591: sync 2024-02-16T22:22:00.212 INFO:tasks.workunit.client.0.smithi114.stdout:3/416: read - d4/dd/d33/d76/f7f zero size 2024-02-16T22:22:00.213 INFO:tasks.workunit.client.1.smithi123.stdout:7/592: read - d3/d6e/d24/fb9 zero size 2024-02-16T22:22:00.217 INFO:tasks.workunit.client.0.smithi114.stdout:8/419: mknod d4/d11/d78/d7a/d36/d3b/c97 0 2024-02-16T22:22:00.217 INFO:tasks.workunit.client.0.smithi114.stdout:8/420: readlink d4/d16/d25/l68 0 2024-02-16T22:22:00.218 INFO:tasks.workunit.client.1.smithi123.stdout:7/593: dread d3/d6e/d1d/db2/f7e [0,4194304] 0 2024-02-16T22:22:00.219 INFO:tasks.workunit.client.1.smithi123.stdout:4/616: unlink d5/d1c/d42/fa8 0 2024-02-16T22:22:00.226 INFO:tasks.workunit.client.1.smithi123.stdout:4/617: dwrite d5/d1c/d23/d31/fc1 [0,4194304] 0 2024-02-16T22:22:00.227 INFO:tasks.workunit.client.1.smithi123.stdout:5/551: symlink d6/dd/df/d58/d89/lbf 0 2024-02-16T22:22:00.227 INFO:tasks.workunit.client.1.smithi123.stdout:9/681: write d4/d36/daa/f1f [263171,123241] 0 2024-02-16T22:22:00.229 INFO:tasks.workunit.client.1.smithi123.stdout:9/682: write d4/d36/daa/d2b/f5b [1887178,62835] 0 2024-02-16T22:22:00.229 INFO:tasks.workunit.client.0.smithi114.stdout:6/456: rename d6/d2c/d5e/f80 to d6/dd/d25/d34/f8e 0 2024-02-16T22:22:00.230 INFO:tasks.workunit.client.1.smithi123.stdout:9/683: write d4/d36/fa8 [763637,86952] 0 2024-02-16T22:22:00.231 INFO:tasks.workunit.client.1.smithi123.stdout:9/684: fdatasync d4/d36/daa/d20/f7b 0 2024-02-16T22:22:00.237 INFO:tasks.workunit.client.0.smithi114.stdout:6/457: dwrite d6/dd/f29 [0,4194304] 0 2024-02-16T22:22:00.238 INFO:tasks.workunit.client.0.smithi114.stdout:5/402: dread d0/f8 [0,4194304] 0 2024-02-16T22:22:00.239 INFO:tasks.workunit.client.1.smithi123.stdout:3/586: dwrite d0/d20/f8b [8388608,4194304] 0 2024-02-16T22:22:00.246 INFO:tasks.workunit.client.1.smithi123.stdout:1/552: rename d3/d6/db/d10/d4b/d70/f79 to d3/d6/db/d10/d4f/fb6 0 2024-02-16T22:22:00.249 INFO:tasks.workunit.client.0.smithi114.stdout:6/458: dwrite d6/d83/f87 [0,4194304] 0 2024-02-16T22:22:00.251 INFO:tasks.workunit.client.0.smithi114.stdout:5/403: dwrite d0/d28/f37 [4194304,4194304] 0 2024-02-16T22:22:00.255 INFO:tasks.workunit.client.0.smithi114.stdout:1/405: fsync d4/fa 0 2024-02-16T22:22:00.257 INFO:tasks.workunit.client.0.smithi114.stdout:3/417: creat d4/d26/d4a/f90 x:0 0 0 2024-02-16T22:22:00.261 INFO:tasks.workunit.client.0.smithi114.stdout:7/484: write d3/d14/f82 [7039690,28990] 0 2024-02-16T22:22:00.262 INFO:tasks.workunit.client.1.smithi123.stdout:4/618: dread d5/d1c/d23/d56/d6e/d73/f88 [0,4194304] 0 2024-02-16T22:22:00.264 INFO:tasks.workunit.client.1.smithi123.stdout:7/594: mknod d3/d6e/d34/cc1 0 2024-02-16T22:22:00.268 INFO:tasks.workunit.client.1.smithi123.stdout:6/583: creat d0/d9/dc/d3a/d70/dd5/fd8 x:0 0 0 2024-02-16T22:22:00.268 INFO:tasks.workunit.client.1.smithi123.stdout:6/584: read - d0/d9/dc/d3a/fc8 zero size 2024-02-16T22:22:00.274 INFO:tasks.workunit.client.0.smithi114.stdout:0/382: rename d1/d16/d48/f69 to d1/d16/d18/f77 0 2024-02-16T22:22:00.276 INFO:tasks.workunit.client.1.smithi123.stdout:5/552: write d6/dd/df/f6c [288365,126170] 0 2024-02-16T22:22:00.276 INFO:tasks.workunit.client.0.smithi114.stdout:2/427: write dd/df/f59 [249362,82106] 0 2024-02-16T22:22:00.277 INFO:tasks.workunit.client.0.smithi114.stdout:2/428: fsync dd/d3b/d8a/f75 0 2024-02-16T22:22:00.278 INFO:tasks.workunit.client.1.smithi123.stdout:2/559: write d4/fc [4376749,45496] 0 2024-02-16T22:22:00.280 INFO:tasks.workunit.client.1.smithi123.stdout:2/560: truncate d4/d1a/d1d/d13/d15/d6d/d4a/f5c 13343 0 2024-02-16T22:22:00.282 INFO:tasks.workunit.client.0.smithi114.stdout:4/543: mkdir dd/d1b/d41/dc3 0 2024-02-16T22:22:00.285 INFO:tasks.workunit.client.0.smithi114.stdout:6/459: truncate d6/f1e 4141614 0 2024-02-16T22:22:00.287 INFO:tasks.workunit.client.0.smithi114.stdout:5/404: creat d0/df/d16/d1e/f70 x:0 0 0 2024-02-16T22:22:00.288 INFO:tasks.workunit.client.0.smithi114.stdout:3/418: dread d4/d26/f63 [0,4194304] 0 2024-02-16T22:22:00.295 INFO:tasks.workunit.client.0.smithi114.stdout:7/485: write d3/d14/f4d [3097402,69649] 0 2024-02-16T22:22:00.295 INFO:tasks.workunit.client.0.smithi114.stdout:7/486: fsync d3/fe 0 2024-02-16T22:22:00.296 INFO:tasks.workunit.client.0.smithi114.stdout:7/487: fsync d3/d23/f52 0 2024-02-16T22:22:00.297 INFO:tasks.workunit.client.1.smithi123.stdout:8/622: creat d0/d29/da7/fc9 x:0 0 0 2024-02-16T22:22:00.297 INFO:tasks.workunit.client.1.smithi123.stdout:9/685: write d4/d33/f3b [885326,79220] 0 2024-02-16T22:22:00.298 INFO:tasks.workunit.client.1.smithi123.stdout:9/686: dread - d4/d36/d8e/d6f/d7e/fda zero size 2024-02-16T22:22:00.299 INFO:tasks.workunit.client.0.smithi114.stdout:4/544: unlink dd/c2a 0 2024-02-16T22:22:00.300 INFO:tasks.workunit.client.1.smithi123.stdout:9/687: truncate d4/d36/d57/db2/f4a 8883592 0 2024-02-16T22:22:00.301 INFO:tasks.workunit.client.1.smithi123.stdout:9/688: write d4/d36/daa/d20/f7b [3960528,57473] 0 2024-02-16T22:22:00.302 INFO:tasks.workunit.client.1.smithi123.stdout:4/619: rename d5/d1c/d42 to d5/d1c/d5e/dc3 0 2024-02-16T22:22:00.304 INFO:tasks.workunit.client.1.smithi123.stdout:7/595: fsync d3/d6e/d24/f90 0 2024-02-16T22:22:00.306 INFO:tasks.workunit.client.0.smithi114.stdout:3/419: mkdir d4/d26/d91 0 2024-02-16T22:22:00.306 INFO:tasks.workunit.client.0.smithi114.stdout:3/420: stat d4/dd 0 2024-02-16T22:22:00.307 INFO:tasks.workunit.client.1.smithi123.stdout:5/553: creat d6/dd/df/d76/d7f/fc0 x:0 0 0 2024-02-16T22:22:00.307 INFO:tasks.workunit.client.0.smithi114.stdout:3/421: write d4/dd/d33/f84 [198197,61294] 0 2024-02-16T22:22:00.308 INFO:tasks.workunit.client.0.smithi114.stdout:1/406: creat d4/d9/d3d/f8e x:0 0 0 2024-02-16T22:22:00.309 INFO:tasks.workunit.client.0.smithi114.stdout:1/407: dread - d4/d33/d74/f87 zero size 2024-02-16T22:22:00.310 INFO:tasks.workunit.client.0.smithi114.stdout:8/421: link d4/d24/d2c/d5f/l74 d4/d24/d2c/l98 0 2024-02-16T22:22:00.312 INFO:tasks.workunit.client.1.smithi123.stdout:3/587: mknod d0/d20/d76/d5d/dae/dc7/ccb 0 2024-02-16T22:22:00.312 INFO:tasks.workunit.client.0.smithi114.stdout:0/383: mkdir d1/d78 0 2024-02-16T22:22:00.313 INFO:tasks.workunit.client.0.smithi114.stdout:0/384: readlink d1/d1a/d47/l5b 0 2024-02-16T22:22:00.314 INFO:tasks.workunit.client.0.smithi114.stdout:7/488: truncate d3/d14/f5c 431312 0 2024-02-16T22:22:00.315 INFO:tasks.workunit.client.0.smithi114.stdout:1/408: dwrite d4/d9/d46/d67/f68 [4194304,4194304] 0 2024-02-16T22:22:00.317 INFO:tasks.workunit.client.0.smithi114.stdout:6/460: link d6/dd/d25/d34/d58/f78 d6/d83/f8f 0 2024-02-16T22:22:00.318 INFO:tasks.workunit.client.0.smithi114.stdout:6/461: chown d6/d2c/d66 6509744 1 2024-02-16T22:22:00.319 INFO:tasks.workunit.client.0.smithi114.stdout:6/462: dread - d6/dd/d25/d34/d61/f6a zero size 2024-02-16T22:22:00.323 INFO:tasks.workunit.client.1.smithi123.stdout:6/585: rename d0/d6/d2f/d5d/l99 to d0/d6/d1a/d88/d8b/ld9 0 2024-02-16T22:22:00.325 INFO:tasks.workunit.client.0.smithi114.stdout:3/422: mknod d4/dd/d33/c92 0 2024-02-16T22:22:00.326 INFO:tasks.workunit.client.1.smithi123.stdout:7/596: symlink d3/d6e/d1d/db2/d35/lc2 0 2024-02-16T22:22:00.330 INFO:tasks.workunit.client.1.smithi123.stdout:1/553: dwrite d3/d6/d49/f66 [0,4194304] 0 2024-02-16T22:22:00.333 INFO:tasks.workunit.client.1.smithi123.stdout:2/561: mkdir d4/dae 0 2024-02-16T22:22:00.333 INFO:tasks.workunit.client.0.smithi114.stdout:5/405: write d0/df/d16/f1f [344874,108959] 0 2024-02-16T22:22:00.334 INFO:tasks.workunit.client.0.smithi114.stdout:7/489: unlink d3/d46/c69 0 2024-02-16T22:22:00.334 INFO:tasks.workunit.client.0.smithi114.stdout:7/490: fdatasync d3/fc 0 2024-02-16T22:22:00.336 INFO:tasks.workunit.client.1.smithi123.stdout:3/588: creat d0/d20/d76/d5d/dae/db0/fcc x:0 0 0 2024-02-16T22:22:00.336 INFO:tasks.workunit.client.1.smithi123.stdout:8/623: dwrite d0/df/f96 [0,4194304] 0 2024-02-16T22:22:00.337 INFO:tasks.workunit.client.0.smithi114.stdout:6/463: creat d6/dd/d25/d34/d58/f90 x:0 0 0 2024-02-16T22:22:00.339 INFO:tasks.workunit.client.0.smithi114.stdout:6/464: read d6/da/f56 [1148132,18010] 0 2024-02-16T22:22:00.340 INFO:tasks.workunit.client.1.smithi123.stdout:0/516: getdents d1/d8/d13/d1d/d64 0 2024-02-16T22:22:00.340 INFO:tasks.workunit.client.0.smithi114.stdout:6/465: chown d6/dd/c12 2431746 1 2024-02-16T22:22:00.344 INFO:tasks.workunit.client.1.smithi123.stdout:9/689: unlink d4/d36/daa/d2b/d6c/l86 0 2024-02-16T22:22:00.345 INFO:tasks.workunit.client.1.smithi123.stdout:8/624: dwrite d0/d29/da7/d26/d2a/d77/f84 [0,4194304] 0 2024-02-16T22:22:00.346 INFO:tasks.workunit.client.0.smithi114.stdout:5/406: creat d0/d28/f71 x:0 0 0 2024-02-16T22:22:00.346 INFO:tasks.workunit.client.0.smithi114.stdout:5/407: stat d0/df/d16/c2c 0 2024-02-16T22:22:00.346 INFO:tasks.workunit.client.1.smithi123.stdout:6/586: mkdir d0/d6/d91/dda 0 2024-02-16T22:22:00.348 INFO:tasks.workunit.client.1.smithi123.stdout:5/554: dwrite d6/dd/d44/f47 [0,4194304] 0 2024-02-16T22:22:00.348 INFO:tasks.workunit.client.1.smithi123.stdout:8/625: write d0/d29/f64 [934583,37290] 0 2024-02-16T22:22:00.349 INFO:tasks.workunit.client.1.smithi123.stdout:6/587: write d0/d74/f7b [3915467,103253] 0 2024-02-16T22:22:00.351 INFO:tasks.workunit.client.1.smithi123.stdout:8/626: fsync d0/d29/da7/d26/f7a 0 2024-02-16T22:22:00.355 INFO:tasks.workunit.client.1.smithi123.stdout:4/620: rename d5/d1c/f3d to d5/d1c/d23/d47/da2/daa/fc4 0 2024-02-16T22:22:00.355 INFO:tasks.workunit.client.1.smithi123.stdout:7/597: creat d3/d6e/d1d/db2/d35/dba/fc3 x:0 0 0 2024-02-16T22:22:00.355 INFO:tasks.workunit.client.0.smithi114.stdout:1/409: dread d4/d63/d69/f7f [0,4194304] 0 2024-02-16T22:22:00.356 INFO:tasks.workunit.client.0.smithi114.stdout:3/423: mkdir d4/d24/d93 0 2024-02-16T22:22:00.356 INFO:tasks.workunit.client.1.smithi123.stdout:7/598: write d3/d6e/d1d/db2/d35/f4b [3202729,128397] 0 2024-02-16T22:22:00.358 INFO:tasks.workunit.client.1.smithi123.stdout:7/599: dread - d3/d5/d6c/fac zero size 2024-02-16T22:22:00.360 INFO:tasks.workunit.client.1.smithi123.stdout:3/589: chown d0/d20/d29/f3c 266 1 2024-02-16T22:22:00.362 INFO:tasks.workunit.client.1.smithi123.stdout:0/517: mkdir d1/d4/d17/d54/db4 0 2024-02-16T22:22:00.364 INFO:tasks.workunit.client.0.smithi114.stdout:6/466: dwrite d6/dd/d25/d34/f8e [0,4194304] 0 2024-02-16T22:22:00.364 INFO:tasks.workunit.client.0.smithi114.stdout:3/424: creat d4/dd/d33/d72/f94 x:0 0 0 2024-02-16T22:22:00.365 INFO:tasks.workunit.client.0.smithi114.stdout:8/422: link d4/d11/f20 d4/d11/f99 0 2024-02-16T22:22:00.412 INFO:tasks.workunit.client.1.smithi123.stdout:1/554: sync 2024-02-16T22:22:00.468 INFO:tasks.workunit.client.0.smithi114.stdout:8/423: dread - d4/d16/d25/d46/f73 zero size 2024-02-16T22:22:00.469 INFO:tasks.workunit.client.1.smithi123.stdout:3/590: dread d0/d20/f52 [0,4194304] 0 2024-02-16T22:22:00.473 INFO:tasks.workunit.client.0.smithi114.stdout:6/467: dread d6/dd/d25/d34/f42 [0,4194304] 0 2024-02-16T22:22:00.476 INFO:tasks.workunit.client.1.smithi123.stdout:8/627: truncate d0/d29/da7/d26/d32/f56 3466011 0 2024-02-16T22:22:00.485 INFO:tasks.workunit.client.1.smithi123.stdout:0/518: rename d1/d4/c1b to d1/d8/da7/cb5 0 2024-02-16T22:22:00.490 INFO:tasks.workunit.client.1.smithi123.stdout:5/555: mkdir d6/dc1 0 2024-02-16T22:22:00.493 INFO:tasks.workunit.client.1.smithi123.stdout:4/621: write d5/d1c/d23/f43 [834591,10595] 0 2024-02-16T22:22:00.496 INFO:tasks.workunit.client.1.smithi123.stdout:5/556: dwrite d6/dd/df/d58/d3f/f92 [0,4194304] 0 2024-02-16T22:22:00.496 INFO:tasks.workunit.client.1.smithi123.stdout:4/622: read d5/d1c/d23/d47/f96 [4980635,119256] 0 2024-02-16T22:22:00.502 INFO:tasks.workunit.client.1.smithi123.stdout:7/600: creat d3/d6e/d1d/d97/fc4 x:0 0 0 2024-02-16T22:22:00.503 INFO:tasks.workunit.client.1.smithi123.stdout:7/601: chown d3/d5/d6c/c9d 7185 1 2024-02-16T22:22:00.503 INFO:tasks.workunit.client.0.smithi114.stdout:3/425: dwrite d4/dd/f1e [4194304,4194304] 0 2024-02-16T22:22:00.504 INFO:tasks.workunit.client.1.smithi123.stdout:2/562: write d4/d1a/d1d/f2c [646842,35313] 0 2024-02-16T22:22:00.509 INFO:tasks.workunit.client.1.smithi123.stdout:7/602: dwrite d3/f8c [0,4194304] 0 2024-02-16T22:22:00.511 INFO:tasks.workunit.client.1.smithi123.stdout:7/603: write d3/d6e/d24/fb9 [730107,69308] 0 2024-02-16T22:22:00.593 INFO:tasks.workunit.client.1.smithi123.stdout:9/690: creat d4/d36/d8e/d58/fdc x:0 0 0 2024-02-16T22:22:00.594 INFO:tasks.workunit.client.1.smithi123.stdout:9/691: chown d4/d36/d8e/d58/d59/c62 3478368 1 2024-02-16T22:22:00.594 INFO:tasks.workunit.client.0.smithi114.stdout:5/408: rename d0/df/d16/d25/l58 to d0/df/d16/l72 0 2024-02-16T22:22:00.595 INFO:tasks.workunit.client.1.smithi123.stdout:9/692: fsync d4/d36/d8e/d6d/f9e 0 2024-02-16T22:22:00.595 INFO:tasks.workunit.client.1.smithi123.stdout:1/555: mknod d3/d6/db/d10/d2d/d8c/cb7 0 2024-02-16T22:22:00.596 INFO:tasks.workunit.client.0.smithi114.stdout:5/409: dread - d0/df/d16/d25/d2e/f67 zero size 2024-02-16T22:22:00.596 INFO:tasks.workunit.client.1.smithi123.stdout:3/591: symlink d0/d20/d76/d5d/d60/dc6/lcd 0 2024-02-16T22:22:00.597 INFO:tasks.workunit.client.0.smithi114.stdout:7/491: getdents d3/d22/d61/d6a 0 2024-02-16T22:22:00.598 INFO:tasks.workunit.client.1.smithi123.stdout:6/588: truncate d0/d6/d1a/d2e/f7f 2117633 0 2024-02-16T22:22:00.599 INFO:tasks.workunit.client.1.smithi123.stdout:6/589: write d0/d9/dc/d3a/fa1 [576372,120249] 0 2024-02-16T22:22:00.601 INFO:tasks.workunit.client.0.smithi114.stdout:1/410: creat d4/d9/f8f x:0 0 0 2024-02-16T22:22:00.602 INFO:tasks.workunit.client.1.smithi123.stdout:8/628: mkdir d0/df/dca 0 2024-02-16T22:22:00.603 INFO:tasks.workunit.client.1.smithi123.stdout:8/629: write d0/d29/fc3 [379132,57479] 0 2024-02-16T22:22:00.604 INFO:tasks.workunit.client.1.smithi123.stdout:5/557: read d6/dd/df/d58/f36 [327790,20698] 0 2024-02-16T22:22:00.604 INFO:tasks.workunit.client.1.smithi123.stdout:5/558: chown d6/dac 26450746 1 2024-02-16T22:22:00.605 INFO:tasks.workunit.client.1.smithi123.stdout:4/623: fdatasync d5/f29 0 2024-02-16T22:22:00.609 INFO:tasks.workunit.client.0.smithi114.stdout:3/426: rename d4/d24/c2b to d4/d46/d48/d75/c95 0 2024-02-16T22:22:00.610 INFO:tasks.workunit.client.1.smithi123.stdout:3/592: dread d0/d20/d2e/fb8 [0,4194304] 0 2024-02-16T22:22:00.610 INFO:tasks.workunit.client.1.smithi123.stdout:3/593: stat d0/d20/d29/fbf 0 2024-02-16T22:22:00.611 INFO:tasks.workunit.client.0.smithi114.stdout:7/492: stat d3/l39 0 2024-02-16T22:22:00.614 INFO:tasks.workunit.client.0.smithi114.stdout:3/427: dread d4/f16 [0,4194304] 0 2024-02-16T22:22:00.614 INFO:tasks.workunit.client.0.smithi114.stdout:8/424: fdatasync d4/d11/f20 0 2024-02-16T22:22:00.617 INFO:tasks.workunit.client.0.smithi114.stdout:3/428: read d4/d26/d4a/f65 [3749124,42589] 0 2024-02-16T22:22:00.617 INFO:tasks.workunit.client.0.smithi114.stdout:3/429: readlink d4/d43/l8d 0 2024-02-16T22:22:00.618 INFO:tasks.workunit.client.0.smithi114.stdout:3/430: readlink d4/l3a 0 2024-02-16T22:22:00.619 INFO:tasks.workunit.client.0.smithi114.stdout:6/468: mknod d6/d2c/d38/d41/c91 0 2024-02-16T22:22:00.621 INFO:tasks.workunit.client.1.smithi123.stdout:2/563: write d4/d1a/d1d/d59/f3a [3790117,97179] 0 2024-02-16T22:22:00.623 INFO:tasks.workunit.client.1.smithi123.stdout:2/564: dread d4/d1a/d1d/f75 [0,4194304] 0 2024-02-16T22:22:00.624 INFO:tasks.workunit.client.1.smithi123.stdout:6/590: write d0/d9/dc/d5f/f2c [4168584,125377] 0 2024-02-16T22:22:00.625 INFO:tasks.workunit.client.0.smithi114.stdout:6/469: dwrite d6/d2c/d75/f82 [0,4194304] 0 2024-02-16T22:22:00.631 INFO:tasks.workunit.client.0.smithi114.stdout:8/425: creat d4/d11/d78/d7a/d7e/f9a x:0 0 0 2024-02-16T22:22:00.631 INFO:tasks.workunit.client.0.smithi114.stdout:7/493: dwrite d3/d3d/f4a [0,4194304] 0 2024-02-16T22:22:00.642 INFO:tasks.workunit.client.1.smithi123.stdout:3/594: rename d0/d20/d76/d37/d49/f9c to d0/d20/d76/d5d/d60/d93/fce 0 2024-02-16T22:22:00.643 INFO:tasks.workunit.client.1.smithi123.stdout:3/595: write d0/f12 [3875079,13294] 0 2024-02-16T22:22:00.645 INFO:tasks.workunit.client.0.smithi114.stdout:3/431: unlink d4/d43/d53/l70 0 2024-02-16T22:22:00.647 INFO:tasks.workunit.client.0.smithi114.stdout:5/410: rename d0/df/d16/d1e/f70 to d0/df/d16/d25/f73 0 2024-02-16T22:22:00.648 INFO:tasks.workunit.client.0.smithi114.stdout:5/411: fdatasync d0/df/f5d 0 2024-02-16T22:22:00.649 INFO:tasks.workunit.client.0.smithi114.stdout:5/412: chown d0/df/f2b 56 1 2024-02-16T22:22:00.649 INFO:tasks.workunit.client.1.smithi123.stdout:6/591: dread d0/d6/fd [0,4194304] 0 2024-02-16T22:22:00.651 INFO:tasks.workunit.client.1.smithi123.stdout:8/630: mknod d0/d29/da7/d26/d32/d45/ccb 0 2024-02-16T22:22:00.653 INFO:tasks.workunit.client.0.smithi114.stdout:8/426: link d4/d11/c61 d4/d24/d2c/d5f/c9b 0 2024-02-16T22:22:00.655 INFO:tasks.workunit.client.1.smithi123.stdout:5/559: symlink d6/lc2 0 2024-02-16T22:22:00.655 INFO:tasks.workunit.client.0.smithi114.stdout:5/413: creat d0/d10/f74 x:0 0 0 2024-02-16T22:22:00.656 INFO:tasks.workunit.client.1.smithi123.stdout:5/560: stat d6/dd/d44/f77 0 2024-02-16T22:22:00.656 INFO:tasks.workunit.client.1.smithi123.stdout:4/624: mkdir d5/d1c/d23/d56/d6e/d9f/da1/dc0/dc5 0 2024-02-16T22:22:00.657 INFO:tasks.workunit.client.1.smithi123.stdout:8/631: dread - d0/d29/da7/d26/d68/d81/d47/fa6 zero size 2024-02-16T22:22:00.657 INFO:tasks.workunit.client.1.smithi123.stdout:5/561: dread - d6/d3b/fb5 zero size 2024-02-16T22:22:00.658 INFO:tasks.workunit.client.1.smithi123.stdout:0/519: link d1/d8/d13/c6c d1/d8/cb6 0 2024-02-16T22:22:00.659 INFO:tasks.workunit.client.1.smithi123.stdout:6/592: dwrite d0/d8a/fcd [0,4194304] 0 2024-02-16T22:22:00.660 INFO:tasks.workunit.client.0.smithi114.stdout:1/411: truncate d4/f30 1490283 0 2024-02-16T22:22:00.664 INFO:tasks.workunit.client.1.smithi123.stdout:7/604: creat d3/d6e/d1d/db2/fc5 x:0 0 0 2024-02-16T22:22:00.665 INFO:tasks.workunit.client.1.smithi123.stdout:7/605: chown d3/d6e/d24/l74 12521573 1 2024-02-16T22:22:00.668 INFO:tasks.workunit.client.0.smithi114.stdout:7/494: dwrite d3/d14/f2c [0,4194304] 0 2024-02-16T22:22:00.670 INFO:tasks.workunit.client.0.smithi114.stdout:7/495: chown d3/d23/f68 5 1 2024-02-16T22:22:00.672 INFO:tasks.workunit.client.0.smithi114.stdout:7/496: write d3/d46/f7e [2383007,117243] 0 2024-02-16T22:22:00.679 INFO:tasks.workunit.client.0.smithi114.stdout:3/432: write d4/dd/f13 [3639048,129761] 0 2024-02-16T22:22:00.680 INFO:tasks.workunit.client.1.smithi123.stdout:8/632: dread d0/df/f1b [0,4194304] 0 2024-02-16T22:22:00.681 INFO:tasks.workunit.client.0.smithi114.stdout:7/497: creat d3/d23/d6f/fa3 x:0 0 0 2024-02-16T22:22:00.681 INFO:tasks.workunit.client.1.smithi123.stdout:8/633: write d0/d61/d66/d7e/fc7 [218514,51026] 0 2024-02-16T22:22:00.682 INFO:tasks.workunit.client.0.smithi114.stdout:7/498: stat d3/d3d/c49 0 2024-02-16T22:22:00.690 INFO:tasks.workunit.client.0.smithi114.stdout:5/414: dwrite d0/df/d53/f6f [0,4194304] 0 2024-02-16T22:22:00.693 INFO:tasks.workunit.client.0.smithi114.stdout:5/415: chown d0/df/f1d 61464 1 2024-02-16T22:22:00.695 INFO:tasks.workunit.client.0.smithi114.stdout:3/433: link d4/dd/f19 d4/d43/d53/f96 0 2024-02-16T22:22:00.696 INFO:tasks.workunit.client.0.smithi114.stdout:8/427: dwrite d4/f3c [0,4194304] 0 2024-02-16T22:22:00.700 INFO:tasks.workunit.client.0.smithi114.stdout:7/499: dwrite d3/d22/d61/d6a/f7a [0,4194304] 0 2024-02-16T22:22:00.702 INFO:tasks.workunit.client.0.smithi114.stdout:1/412: truncate d4/d33/f57 4868052 0 2024-02-16T22:22:00.703 INFO:tasks.workunit.client.0.smithi114.stdout:1/413: chown d4/d9/d46 94 1 2024-02-16T22:22:00.707 INFO:tasks.workunit.client.0.smithi114.stdout:3/434: fsync d4/d24/f67 0 2024-02-16T22:22:00.711 INFO:tasks.workunit.client.0.smithi114.stdout:7/500: creat d3/d14/fa4 x:0 0 0 2024-02-16T22:22:00.712 INFO:tasks.workunit.client.0.smithi114.stdout:7/501: truncate d3/d87/f8e 4694375 0 2024-02-16T22:22:00.713 INFO:tasks.workunit.client.1.smithi123.stdout:2/565: sync 2024-02-16T22:22:00.715 INFO:tasks.workunit.client.0.smithi114.stdout:1/414: fdatasync d4/f36 0 2024-02-16T22:22:00.722 INFO:tasks.workunit.client.0.smithi114.stdout:1/415: dwrite d4/d33/d74/f77 [0,4194304] 0 2024-02-16T22:22:00.724 INFO:tasks.workunit.client.0.smithi114.stdout:7/502: symlink d3/d23/d6f/la5 0 2024-02-16T22:22:00.730 INFO:tasks.workunit.client.0.smithi114.stdout:3/435: symlink d4/d87/l97 0 2024-02-16T22:22:00.732 INFO:tasks.workunit.client.0.smithi114.stdout:8/428: dread d4/d11/d78/d7a/d36/f5d [0,4194304] 0 2024-02-16T22:22:00.733 INFO:tasks.workunit.client.0.smithi114.stdout:8/429: dread - d4/d11/d52/f83 zero size 2024-02-16T22:22:00.733 INFO:tasks.workunit.client.0.smithi114.stdout:5/416: getdents d0/df/d16/d25/d3e 0 2024-02-16T22:22:00.737 INFO:tasks.workunit.client.0.smithi114.stdout:7/503: mkdir d3/d23/d6f/d89/da6 0 2024-02-16T22:22:00.737 INFO:tasks.workunit.client.0.smithi114.stdout:7/504: chown d3/d14/c64 1062738 1 2024-02-16T22:22:00.740 INFO:tasks.workunit.client.0.smithi114.stdout:8/430: unlink d4/d24/d2c/f85 0 2024-02-16T22:22:00.741 INFO:tasks.workunit.client.1.smithi123.stdout:5/562: mkdir d6/dd/df/d58/d89/dc3 0 2024-02-16T22:22:00.742 INFO:tasks.workunit.client.1.smithi123.stdout:5/563: write d6/dd/df/d58/d3f/d6f/f91 [513733,89551] 0 2024-02-16T22:22:00.743 INFO:tasks.workunit.client.1.smithi123.stdout:0/520: creat d1/d4/d42/fb7 x:0 0 0 2024-02-16T22:22:00.744 INFO:tasks.workunit.client.0.smithi114.stdout:1/416: mknod d4/c90 0 2024-02-16T22:22:00.745 INFO:tasks.workunit.client.1.smithi123.stdout:6/593: mknod d0/d6/d1a/d2e/cdb 0 2024-02-16T22:22:00.747 INFO:tasks.workunit.client.0.smithi114.stdout:8/431: creat d4/d11/d78/f9c x:0 0 0 2024-02-16T22:22:00.748 INFO:tasks.workunit.client.1.smithi123.stdout:3/596: symlink d0/d50/d7e/lcf 0 2024-02-16T22:22:00.749 INFO:tasks.workunit.client.1.smithi123.stdout:3/597: dread - d0/d20/d29/d9b/fa8 zero size 2024-02-16T22:22:00.749 INFO:tasks.workunit.client.1.smithi123.stdout:9/693: link d4/d36/d8e/d98/l99 d4/d36/d8e/d58/d59/dc7/dd1/ldd 0 2024-02-16T22:22:00.750 INFO:tasks.workunit.client.1.smithi123.stdout:9/694: read d4/d36/d57/db2/f69 [1303105,77667] 0 2024-02-16T22:22:00.751 INFO:tasks.workunit.client.1.smithi123.stdout:1/556: getdents d3/d6/db/d10/d2a 0 2024-02-16T22:22:00.752 INFO:tasks.workunit.client.0.smithi114.stdout:8/432: truncate d4/d16/d25/f54 114084 0 2024-02-16T22:22:00.752 INFO:tasks.workunit.client.1.smithi123.stdout:1/557: fdatasync d3/d6/d11/f62 0 2024-02-16T22:22:00.753 INFO:tasks.workunit.client.1.smithi123.stdout:9/695: write d4/d36/d8e/d6d/d71/d92/fcd [3433106,59688] 0 2024-02-16T22:22:00.753 INFO:tasks.workunit.client.1.smithi123.stdout:2/566: creat d4/d1a/d1d/d13/d15/d6d/d4a/faf x:0 0 0 2024-02-16T22:22:00.753 INFO:tasks.workunit.client.1.smithi123.stdout:1/558: write d3/d6/db/f3a [95770,9059] 0 2024-02-16T22:22:00.754 INFO:tasks.workunit.client.0.smithi114.stdout:5/417: creat d0/df/d16/d25/f75 x:0 0 0 2024-02-16T22:22:00.754 INFO:tasks.workunit.client.1.smithi123.stdout:1/559: chown d3/d6/d11 2 1 2024-02-16T22:22:00.754 INFO:tasks.workunit.client.1.smithi123.stdout:9/696: truncate d4/d33/f73 4339258 0 2024-02-16T22:22:00.755 INFO:tasks.workunit.client.0.smithi114.stdout:8/433: chown d4/d16/d25/c48 277 1 2024-02-16T22:22:00.757 INFO:tasks.workunit.client.0.smithi114.stdout:8/434: truncate d4/d11/d78/d7a/f1a 8568005 0 2024-02-16T22:22:00.757 INFO:tasks.workunit.client.0.smithi114.stdout:1/417: rename d4/d9/d46/d4d/f5b to d4/f91 0 2024-02-16T22:22:00.758 INFO:tasks.workunit.client.1.smithi123.stdout:5/564: chown d6/dd/df/d58/d3f/dbe/da2/faf 6228679 1 2024-02-16T22:22:00.758 INFO:tasks.workunit.client.1.smithi123.stdout:0/521: creat d1/d4/d17/d74/fb8 x:0 0 0 2024-02-16T22:22:00.760 INFO:tasks.workunit.client.1.smithi123.stdout:9/697: dwrite d4/d36/daa/f1a [0,4194304] 0 2024-02-16T22:22:00.760 INFO:tasks.workunit.client.0.smithi114.stdout:5/418: rename d0/df/d16/d1e/c52 to d0/d28/c76 0 2024-02-16T22:22:00.761 INFO:tasks.workunit.client.1.smithi123.stdout:9/698: readlink d4/d36/d8e/d58/l8f 0 2024-02-16T22:22:00.761 INFO:tasks.workunit.client.1.smithi123.stdout:7/606: symlink d3/d6e/d1d/db2/lc6 0 2024-02-16T22:22:00.762 INFO:tasks.workunit.client.1.smithi123.stdout:7/607: write d3/d6e/f4f [2020333,122683] 0 2024-02-16T22:22:00.763 INFO:tasks.workunit.client.0.smithi114.stdout:9/472: sync 2024-02-16T22:22:00.764 INFO:tasks.workunit.client.0.smithi114.stdout:9/473: fdatasync d2/d33/f7d 0 2024-02-16T22:22:00.764 INFO:tasks.workunit.client.0.smithi114.stdout:8/435: mkdir d4/d11/d9d 0 2024-02-16T22:22:00.766 INFO:tasks.workunit.client.0.smithi114.stdout:8/436: write d4/f8 [2311911,15202] 0 2024-02-16T22:22:00.768 INFO:tasks.workunit.client.0.smithi114.stdout:5/419: creat d0/df/d16/d25/d2e/d3b/f77 x:0 0 0 2024-02-16T22:22:00.769 INFO:tasks.workunit.client.0.smithi114.stdout:5/420: dread - d0/d28/f71 zero size 2024-02-16T22:22:00.770 INFO:tasks.workunit.client.1.smithi123.stdout:8/634: rename d0/f14 to d0/d12/fcc 0 2024-02-16T22:22:00.775 INFO:tasks.workunit.client.1.smithi123.stdout:4/625: link d5/d1c/d23/f3a d5/d1c/d64/fc6 0 2024-02-16T22:22:00.776 INFO:tasks.workunit.client.1.smithi123.stdout:4/626: chown d5/d1c/d64/d7b/c82 854363640 1 2024-02-16T22:22:00.777 INFO:tasks.workunit.client.0.smithi114.stdout:8/437: rename d4/d24/c26 to d4/d11/d78/d7a/d36/c9e 0 2024-02-16T22:22:00.779 INFO:tasks.workunit.client.0.smithi114.stdout:5/421: creat d0/df/d16/d25/d2e/d3b/f78 x:0 0 0 2024-02-16T22:22:00.780 INFO:tasks.workunit.client.1.smithi123.stdout:0/522: creat d1/da0/fb9 x:0 0 0 2024-02-16T22:22:00.781 INFO:tasks.workunit.client.1.smithi123.stdout:0/523: dread - d1/d4/d42/f95 zero size 2024-02-16T22:22:00.782 INFO:tasks.workunit.client.1.smithi123.stdout:0/524: truncate d1/d4/d17/d74/d57/f72 783484 0 2024-02-16T22:22:00.785 INFO:tasks.workunit.client.1.smithi123.stdout:9/699: unlink d4/d36/d57/db2/f4a 0 2024-02-16T22:22:00.787 INFO:tasks.workunit.client.1.smithi123.stdout:7/608: creat d3/fc7 x:0 0 0 2024-02-16T22:22:00.787 INFO:tasks.workunit.client.1.smithi123.stdout:6/594: dwrite d0/d6/f18 [0,4194304] 0 2024-02-16T22:22:00.790 INFO:tasks.workunit.client.1.smithi123.stdout:4/627: fdatasync d5/d1c/f97 0 2024-02-16T22:22:00.792 INFO:tasks.workunit.client.1.smithi123.stdout:5/565: symlink d6/dd/df/d58/d89/d98/lc4 0 2024-02-16T22:22:00.816 INFO:tasks.workunit.client.1.smithi123.stdout:6/595: unlink d0/d6/d1a/d2e/l5e 0 2024-02-16T22:22:00.816 INFO:tasks.workunit.client.1.smithi123.stdout:8/635: link d0/d29/da7/d26/d32/f6f d0/d61/d66/dc0/fcd 0 2024-02-16T22:22:00.816 INFO:tasks.workunit.client.1.smithi123.stdout:5/566: write d6/d3b/fb5 [86404,52312] 0 2024-02-16T22:22:00.816 INFO:tasks.workunit.client.1.smithi123.stdout:1/560: write d3/d6/db/d10/f4a [16534,7101] 0 2024-02-16T22:22:00.818 INFO:tasks.workunit.client.1.smithi123.stdout:8/636: chown d0/d61/d66 2127261487 1 2024-02-16T22:22:00.851 INFO:tasks.workunit.client.1.smithi123.stdout:3/598: sync 2024-02-16T22:22:00.852 INFO:tasks.workunit.client.1.smithi123.stdout:5/567: rmdir d6/dd/d44 39 2024-02-16T22:22:00.853 INFO:tasks.workunit.client.1.smithi123.stdout:6/596: mknod d0/d6/d2f/da5/dc6/cdc 0 2024-02-16T22:22:00.856 INFO:tasks.workunit.client.1.smithi123.stdout:1/561: creat d3/d6/db/d10/d4b/d70/fb8 x:0 0 0 2024-02-16T22:22:00.856 INFO:tasks.workunit.client.1.smithi123.stdout:4/628: truncate d5/d1c/d23/f3a 684882 0 2024-02-16T22:22:00.858 INFO:tasks.workunit.client.1.smithi123.stdout:2/567: dwrite d4/d1a/d1d/d38/f58 [0,4194304] 0 2024-02-16T22:22:00.859 INFO:tasks.workunit.client.1.smithi123.stdout:6/597: mkdir d0/d6/d2f/da5/dc6/ddd 0 2024-02-16T22:22:00.860 INFO:tasks.workunit.client.1.smithi123.stdout:4/629: stat d5/d1c/d23/d74/c46 0 2024-02-16T22:22:00.860 INFO:tasks.workunit.client.1.smithi123.stdout:9/700: dwrite d4/d36/daa/f23 [0,4194304] 0 2024-02-16T22:22:00.862 INFO:tasks.workunit.client.1.smithi123.stdout:8/637: creat d0/d29/da7/d26/d32/d45/fce x:0 0 0 2024-02-16T22:22:00.863 INFO:tasks.workunit.client.1.smithi123.stdout:9/701: chown d4/fa1 993212 1 2024-02-16T22:22:00.863 INFO:tasks.workunit.client.0.smithi114.stdout:1/418: write d4/d9/d46/d4d/f5c [1046304,19999] 0 2024-02-16T22:22:00.863 INFO:tasks.workunit.client.1.smithi123.stdout:4/630: chown d5/d1c/d9a/cbe 3578 1 2024-02-16T22:22:00.864 INFO:tasks.workunit.client.1.smithi123.stdout:1/562: creat d3/d6/db/d10/d4f/d9f/fb9 x:0 0 0 2024-02-16T22:22:00.865 INFO:tasks.workunit.client.1.smithi123.stdout:3/599: dwrite d0/d20/d94/fbc [0,4194304] 0 2024-02-16T22:22:00.866 INFO:tasks.workunit.client.1.smithi123.stdout:8/638: write d0/d29/da7/d26/d2a/d4d/fc6 [771162,111042] 0 2024-02-16T22:22:00.867 INFO:tasks.workunit.client.1.smithi123.stdout:2/568: mknod d4/d1a/d1d/d13/cb0 0 2024-02-16T22:22:00.868 INFO:tasks.workunit.client.1.smithi123.stdout:8/639: chown d0/d12/fcc 60997 1 2024-02-16T22:22:00.869 INFO:tasks.workunit.client.0.smithi114.stdout:1/419: unlink d4/f36 0 2024-02-16T22:22:00.870 INFO:tasks.workunit.client.1.smithi123.stdout:8/640: fdatasync d0/d29/da7/d26/d68/d81/d47/d9b/fa8 0 2024-02-16T22:22:00.870 INFO:tasks.workunit.client.1.smithi123.stdout:6/598: creat d0/d9/dc/d5f/dcc/fde x:0 0 0 2024-02-16T22:22:00.876 INFO:tasks.workunit.client.0.smithi114.stdout:9/474: dread d2/f12 [0,4194304] 0 2024-02-16T22:22:00.876 INFO:tasks.workunit.client.0.smithi114.stdout:8/438: link d4/l5 d4/d16/d25/d46/l9f 0 2024-02-16T22:22:00.876 INFO:tasks.workunit.client.0.smithi114.stdout:9/475: fdatasync d2/dd/f2a 0 2024-02-16T22:22:00.881 INFO:tasks.workunit.client.1.smithi123.stdout:1/563: unlink d3/d6/l7a 0 2024-02-16T22:22:00.881 INFO:tasks.workunit.client.1.smithi123.stdout:1/564: chown d3/d6/db 122 1 2024-02-16T22:22:00.882 INFO:tasks.workunit.client.1.smithi123.stdout:3/600: creat d0/d40/fd0 x:0 0 0 2024-02-16T22:22:00.884 INFO:tasks.workunit.client.0.smithi114.stdout:1/420: mkdir d4/d9/d46/d67/d92 0 2024-02-16T22:22:00.885 INFO:tasks.workunit.client.1.smithi123.stdout:9/702: truncate d4/d36/d57/db2/f69 48099 0 2024-02-16T22:22:00.888 INFO:tasks.workunit.client.1.smithi123.stdout:2/569: fdatasync d4/d1a/d1d/f75 0 2024-02-16T22:22:00.890 INFO:tasks.workunit.client.1.smithi123.stdout:1/565: dread d3/d6/d18/d9e/fae [0,4194304] 0 2024-02-16T22:22:00.891 INFO:tasks.workunit.client.0.smithi114.stdout:8/439: rename d4/d16/d25/l69 to d4/d16/d25/d46/la0 0 2024-02-16T22:22:00.891 INFO:tasks.workunit.client.1.smithi123.stdout:8/641: mknod d0/d29/da7/d26/d32/d45/d67/ccf 0 2024-02-16T22:22:00.892 INFO:tasks.workunit.client.0.smithi114.stdout:8/440: chown d4/d11/d78/d7a/d36/d3b/c5a 19 1 2024-02-16T22:22:00.892 INFO:tasks.workunit.client.0.smithi114.stdout:1/421: symlink d4/d9/d46/l93 0 2024-02-16T22:22:00.894 INFO:tasks.workunit.client.1.smithi123.stdout:4/631: dread d5/d1c/d64/fc6 [0,4194304] 0 2024-02-16T22:22:00.894 INFO:tasks.workunit.client.1.smithi123.stdout:3/601: rename d0/fd to d0/d20/d2e/fd1 0 2024-02-16T22:22:00.896 INFO:tasks.workunit.client.0.smithi114.stdout:5/422: link d0/df/d16/d25/d2e/f4f d0/df/d16/f79 0 2024-02-16T22:22:00.900 INFO:tasks.workunit.client.1.smithi123.stdout:6/599: mknod d0/d6/d91/dda/cdf 0 2024-02-16T22:22:00.901 INFO:tasks.workunit.client.1.smithi123.stdout:0/525: sync 2024-02-16T22:22:00.904 INFO:tasks.workunit.client.0.smithi114.stdout:5/423: mkdir d0/df/d7a 0 2024-02-16T22:22:00.905 INFO:tasks.workunit.client.1.smithi123.stdout:8/642: fdatasync d0/d12/fcc 0 2024-02-16T22:22:00.908 INFO:tasks.workunit.client.0.smithi114.stdout:8/441: mknod d4/d24/d2c/d67/ca1 0 2024-02-16T22:22:00.910 INFO:tasks.workunit.client.1.smithi123.stdout:9/703: rename d4/d36/d8e/d58/d59/dc7/dd1 to d4/d36/d8e/d58/d6e/da7/dde 0 2024-02-16T22:22:00.911 INFO:tasks.workunit.client.0.smithi114.stdout:1/422: truncate d4/d9/d3d/d3f/f81 215772 0 2024-02-16T22:22:00.912 INFO:tasks.workunit.client.1.smithi123.stdout:3/602: mkdir d0/d20/d76/d5d/d60/d93/dd2 0 2024-02-16T22:22:00.913 INFO:tasks.workunit.client.0.smithi114.stdout:5/424: creat d0/df/d16/d25/d2e/f7b x:0 0 0 2024-02-16T22:22:00.913 INFO:tasks.workunit.client.1.smithi123.stdout:2/570: dread d4/f52 [0,4194304] 0 2024-02-16T22:22:00.914 INFO:tasks.workunit.client.1.smithi123.stdout:4/632: write d5/d1c/d64/fc6 [784874,95183] 0 2024-02-16T22:22:00.915 INFO:tasks.workunit.client.0.smithi114.stdout:9/476: getdents d2/d1a 0 2024-02-16T22:22:00.916 INFO:tasks.workunit.client.0.smithi114.stdout:8/442: creat d4/d24/d2c/d5f/fa2 x:0 0 0 2024-02-16T22:22:00.917 INFO:tasks.workunit.client.0.smithi114.stdout:1/423: creat d4/d9/d46/d47/f94 x:0 0 0 2024-02-16T22:22:00.919 INFO:tasks.workunit.client.0.smithi114.stdout:9/477: mkdir d2/d1a/d23/d35/d47/d63/d98 0 2024-02-16T22:22:00.920 INFO:tasks.workunit.client.0.smithi114.stdout:8/443: symlink d4/d11/d78/la3 0 2024-02-16T22:22:00.927 INFO:tasks.workunit.client.0.smithi114.stdout:9/478: rename d2/d33/c4e to d2/d1a/d23/d35/d47/d63/d98/c99 0 2024-02-16T22:22:00.927 INFO:tasks.workunit.client.0.smithi114.stdout:4/545: sync 2024-02-16T22:22:00.927 INFO:tasks.workunit.client.0.smithi114.stdout:2/429: sync 2024-02-16T22:22:00.928 INFO:tasks.workunit.client.0.smithi114.stdout:2/430: chown dd/f7a 39 1 2024-02-16T22:22:00.929 INFO:tasks.workunit.client.1.smithi123.stdout:2/571: mkdir d4/d1a/d1d/d13/d15/d6d/db1 0 2024-02-16T22:22:00.929 INFO:tasks.workunit.client.1.smithi123.stdout:2/572: readlink d4/d1a/d1d/d13/d25/l4c 0 2024-02-16T22:22:00.931 INFO:tasks.workunit.client.0.smithi114.stdout:1/424: link d4/d9/d3d/f8e d4/d63/d69/d80/f95 0 2024-02-16T22:22:00.934 INFO:tasks.workunit.client.0.smithi114.stdout:5/425: dwrite d0/d10/f23 [8388608,4194304] 0 2024-02-16T22:22:00.937 INFO:tasks.workunit.client.1.smithi123.stdout:4/633: rename d5/d1c/d23/d31/l34 to d5/dba/lc7 0 2024-02-16T22:22:00.938 INFO:tasks.workunit.client.1.smithi123.stdout:7/609: sync 2024-02-16T22:22:00.938 INFO:tasks.workunit.client.1.smithi123.stdout:3/603: dwrite d0/d20/d94/dac/fb1 [0,4194304] 0 2024-02-16T22:22:00.940 INFO:tasks.workunit.client.1.smithi123.stdout:0/526: link d1/d8/d3d/d44/fac d1/d4/d17/d54/fba 0 2024-02-16T22:22:00.940 INFO:tasks.workunit.client.1.smithi123.stdout:4/634: dread d5/d49/fb7 [0,4194304] 0 2024-02-16T22:22:00.942 INFO:tasks.workunit.client.0.smithi114.stdout:5/426: dwrite d0/df/d16/f5b [0,4194304] 0 2024-02-16T22:22:00.943 INFO:tasks.workunit.client.0.smithi114.stdout:4/546: rename dd/f13 to dd/d1b/d37/d38/d7c/fc4 0 2024-02-16T22:22:00.945 INFO:tasks.workunit.client.1.smithi123.stdout:1/566: creat d3/d6/db/d10/fba x:0 0 0 2024-02-16T22:22:00.949 INFO:tasks.workunit.client.0.smithi114.stdout:2/431: creat dd/df/d1c/d21/d87/f94 x:0 0 0 2024-02-16T22:22:00.951 INFO:tasks.workunit.client.1.smithi123.stdout:8/643: mkdir d0/d61/dac/dd0 0 2024-02-16T22:22:00.952 INFO:tasks.workunit.client.0.smithi114.stdout:2/432: truncate dd/df/d1c/d21/f26 1267031 0 2024-02-16T22:22:00.954 INFO:tasks.workunit.client.0.smithi114.stdout:8/444: dwrite d4/d16/d25/f54 [0,4194304] 0 2024-02-16T22:22:00.955 INFO:tasks.workunit.client.0.smithi114.stdout:8/445: dread - d4/d11/f8f zero size 2024-02-16T22:22:00.970 INFO:tasks.workunit.client.1.smithi123.stdout:5/568: sync 2024-02-16T22:22:01.032 INFO:tasks.workunit.client.0.smithi114.stdout:4/547: creat dd/d1b/d24/d98/fc5 x:0 0 0 2024-02-16T22:22:01.037 INFO:tasks.workunit.client.1.smithi123.stdout:3/604: fdatasync d0/d20/d76/d5d/d60/d93/fce 0 2024-02-16T22:22:01.041 INFO:tasks.workunit.client.0.smithi114.stdout:8/446: unlink d4/d11/d78/d7a/d31/f37 0 2024-02-16T22:22:01.043 INFO:tasks.workunit.client.1.smithi123.stdout:1/567: mkdir d3/d6/d11/d8b/dbb 0 2024-02-16T22:22:01.045 INFO:tasks.workunit.client.0.smithi114.stdout:5/427: creat d0/d28/d6e/f7c x:0 0 0 2024-02-16T22:22:01.046 INFO:tasks.workunit.client.0.smithi114.stdout:5/428: write d0/df/d16/d25/d2e/f67 [579767,3265] 0 2024-02-16T22:22:01.047 INFO:tasks.workunit.client.1.smithi123.stdout:8/644: symlink d0/d29/da7/d26/d68/dbe/ld1 0 2024-02-16T22:22:01.047 INFO:tasks.workunit.client.0.smithi114.stdout:5/429: chown d0/d10/f74 144351869 1 2024-02-16T22:22:01.050 INFO:tasks.workunit.client.1.smithi123.stdout:9/704: unlink d4/d36/daa/d20/f45 0 2024-02-16T22:22:01.053 INFO:tasks.workunit.client.1.smithi123.stdout:6/600: getdents d0/d6/d4d 0 2024-02-16T22:22:01.055 INFO:tasks.workunit.client.1.smithi123.stdout:6/601: dread d0/d6/fd [4194304,4194304] 0 2024-02-16T22:22:01.056 INFO:tasks.workunit.client.0.smithi114.stdout:5/430: creat d0/df/d16/d25/d2e/f7d x:0 0 0 2024-02-16T22:22:01.057 INFO:tasks.workunit.client.0.smithi114.stdout:5/431: write d0/df/d16/d25/d3e/f5c [525282,60665] 0 2024-02-16T22:22:01.061 INFO:tasks.workunit.client.0.smithi114.stdout:5/432: dread d0/d2d/d30/f40 [0,4194304] 0 2024-02-16T22:22:01.062 INFO:tasks.workunit.client.0.smithi114.stdout:5/433: fdatasync d0/df/d16/d25/d2e/d3b/f78 0 2024-02-16T22:22:01.063 INFO:tasks.workunit.client.0.smithi114.stdout:8/447: link d4/d11/d78/d7a/d31/f8e d4/d24/d2c/d67/fa4 0 2024-02-16T22:22:01.068 INFO:tasks.workunit.client.0.smithi114.stdout:4/548: creat dd/d1b/d41/d74/d9b/fc6 x:0 0 0 2024-02-16T22:22:01.073 INFO:tasks.workunit.client.0.smithi114.stdout:2/433: getdents dd/df/d1c 0 2024-02-16T22:22:01.074 INFO:tasks.workunit.client.0.smithi114.stdout:5/434: creat d0/d28/d6e/f7e x:0 0 0 2024-02-16T22:22:01.079 INFO:tasks.workunit.client.0.smithi114.stdout:4/549: creat dd/d1b/d24/d57/fc7 x:0 0 0 2024-02-16T22:22:01.083 INFO:tasks.workunit.client.0.smithi114.stdout:5/435: mkdir d0/df/d16/d25/d3e/d7f 0 2024-02-16T22:22:01.087 INFO:tasks.workunit.client.0.smithi114.stdout:4/550: creat dd/d1b/d24/fc8 x:0 0 0 2024-02-16T22:22:01.090 INFO:tasks.workunit.client.0.smithi114.stdout:5/436: creat d0/df/f80 x:0 0 0 2024-02-16T22:22:01.093 INFO:tasks.workunit.client.0.smithi114.stdout:5/437: creat d0/f81 x:0 0 0 2024-02-16T22:22:01.095 INFO:tasks.workunit.client.1.smithi123.stdout:9/705: unlink d4/d36/d8e/d84/l8b 0 2024-02-16T22:22:01.097 INFO:tasks.workunit.client.1.smithi123.stdout:9/706: read d4/d36/d8e/d58/f5a [3865042,88608] 0 2024-02-16T22:22:01.098 INFO:tasks.workunit.client.1.smithi123.stdout:9/707: stat d4/d36/d8e/d58/d59/c62 0 2024-02-16T22:22:01.101 INFO:tasks.workunit.client.1.smithi123.stdout:9/708: read d4/d1e/d3a/fc3 [2377643,47604] 0 2024-02-16T22:22:01.104 INFO:tasks.workunit.client.1.smithi123.stdout:9/709: creat d4/d36/daa/d2b/d6c/fdf x:0 0 0 2024-02-16T22:22:01.106 INFO:tasks.workunit.client.0.smithi114.stdout:0/385: sync 2024-02-16T22:22:01.106 INFO:tasks.workunit.client.0.smithi114.stdout:6/470: sync 2024-02-16T22:22:01.110 INFO:tasks.workunit.client.0.smithi114.stdout:0/386: mkdir d1/d16/d1f/d28/d51/d61/d6d/d79 0 2024-02-16T22:22:01.111 INFO:tasks.workunit.client.0.smithi114.stdout:0/387: fdatasync d1/d16/f73 0 2024-02-16T22:22:01.112 INFO:tasks.workunit.client.0.smithi114.stdout:0/388: write d1/d16/d1f/d28/f54 [172371,31036] 0 2024-02-16T22:22:01.113 INFO:tasks.workunit.client.0.smithi114.stdout:0/389: dread - d1/d16/d48/f6a zero size 2024-02-16T22:22:01.114 INFO:tasks.workunit.client.0.smithi114.stdout:6/471: rename d6/d2c/d38/f4f to d6/d2c/d38/f92 0 2024-02-16T22:22:01.123 INFO:tasks.workunit.client.0.smithi114.stdout:0/390: link d1/d16/d1f/d28/d51/d60/c39 d1/d1a/d3e/d41/c7a 0 2024-02-16T22:22:01.131 INFO:tasks.workunit.client.1.smithi123.stdout:8/645: sync 2024-02-16T22:22:01.135 INFO:tasks.workunit.client.1.smithi123.stdout:7/610: symlink d3/d5/lc8 0 2024-02-16T22:22:01.137 INFO:tasks.workunit.client.1.smithi123.stdout:7/611: symlink d3/d6e/d24/lc9 0 2024-02-16T22:22:01.137 INFO:tasks.workunit.client.1.smithi123.stdout:7/612: chown d3/c20 18998329 1 2024-02-16T22:22:01.138 INFO:tasks.workunit.client.1.smithi123.stdout:7/613: chown d3/d87/f5e 400630 1 2024-02-16T22:22:01.139 INFO:tasks.workunit.client.0.smithi114.stdout:0/391: dread d1/fe [4194304,4194304] 0 2024-02-16T22:22:01.140 INFO:tasks.workunit.client.0.smithi114.stdout:0/392: chown d1/d16/f5d 48937 1 2024-02-16T22:22:01.156 INFO:tasks.workunit.client.0.smithi114.stdout:0/393: dread d1/d1a/f4e [0,4194304] 0 2024-02-16T22:22:01.158 INFO:tasks.workunit.client.1.smithi123.stdout:4/635: dwrite d5/f1e [0,4194304] 0 2024-02-16T22:22:01.162 INFO:tasks.workunit.client.0.smithi114.stdout:3/436: sync 2024-02-16T22:22:01.162 INFO:tasks.workunit.client.0.smithi114.stdout:9/479: sync 2024-02-16T22:22:01.162 INFO:tasks.workunit.client.0.smithi114.stdout:7/505: sync 2024-02-16T22:22:01.162 INFO:tasks.workunit.client.1.smithi123.stdout:3/605: truncate d0/f12 3588677 0 2024-02-16T22:22:01.163 INFO:tasks.workunit.client.1.smithi123.stdout:4/636: dread d5/d1c/d23/f43 [0,4194304] 0 2024-02-16T22:22:01.163 INFO:tasks.workunit.client.0.smithi114.stdout:0/394: dwrite d1/d16/d1f/d28/f54 [0,4194304] 0 2024-02-16T22:22:01.166 INFO:tasks.workunit.client.1.smithi123.stdout:4/637: symlink d5/d1c/d23/d31/lc8 0 2024-02-16T22:22:01.166 INFO:tasks.workunit.client.1.smithi123.stdout:3/606: dread d0/d20/d29/f3d [0,4194304] 0 2024-02-16T22:22:01.169 INFO:tasks.workunit.client.0.smithi114.stdout:9/480: fdatasync d2/d1a/d23/d35/d47/d63/f6a 0 2024-02-16T22:22:01.170 INFO:tasks.workunit.client.0.smithi114.stdout:2/434: truncate dd/d3b/d8a/f7c 101426 0 2024-02-16T22:22:01.173 INFO:tasks.workunit.client.0.smithi114.stdout:0/395: rmdir d1/d16/d48 39 2024-02-16T22:22:01.175 INFO:tasks.workunit.client.1.smithi123.stdout:3/607: dwrite d0/d20/d76/d5d/dae/fb6 [0,4194304] 0 2024-02-16T22:22:01.175 INFO:tasks.workunit.client.0.smithi114.stdout:7/506: dread d3/d23/f5e [0,4194304] 0 2024-02-16T22:22:01.177 INFO:tasks.workunit.client.0.smithi114.stdout:3/437: dwrite d4/d24/f6d [0,4194304] 0 2024-02-16T22:22:01.177 INFO:tasks.workunit.client.0.smithi114.stdout:9/481: truncate d2/d1a/d65/f94 823128 0 2024-02-16T22:22:01.178 INFO:tasks.workunit.client.1.smithi123.stdout:4/638: symlink d5/dba/lc9 0 2024-02-16T22:22:01.178 INFO:tasks.workunit.client.0.smithi114.stdout:4/551: write dd/d59/f71 [998150,110894] 0 2024-02-16T22:22:01.179 INFO:tasks.workunit.client.0.smithi114.stdout:7/507: fsync d3/f6 0 2024-02-16T22:22:01.180 INFO:tasks.workunit.client.0.smithi114.stdout:8/448: dwrite d4/d24/f64 [0,4194304] 0 2024-02-16T22:22:01.183 INFO:tasks.workunit.client.0.smithi114.stdout:9/482: chown d2/d33/d3f/c60 2 1 2024-02-16T22:22:01.185 INFO:tasks.workunit.client.1.smithi123.stdout:3/608: dwrite d0/fc0 [0,4194304] 0 2024-02-16T22:22:01.186 INFO:tasks.workunit.client.0.smithi114.stdout:2/435: dwrite dd/f17 [0,4194304] 0 2024-02-16T22:22:01.187 INFO:tasks.workunit.client.1.smithi123.stdout:3/609: dread - d0/d20/d76/d37/fa1 zero size 2024-02-16T22:22:01.188 INFO:tasks.workunit.client.1.smithi123.stdout:3/610: chown d0/d50/l9e 0 1 2024-02-16T22:22:01.189 INFO:tasks.workunit.client.0.smithi114.stdout:2/436: stat dd/df/d1c/d21/f26 0 2024-02-16T22:22:01.190 INFO:tasks.workunit.client.1.smithi123.stdout:4/639: dwrite d5/f1d [0,4194304] 0 2024-02-16T22:22:01.191 INFO:tasks.workunit.client.0.smithi114.stdout:5/438: dread - d0/d28/f71 zero size 2024-02-16T22:22:01.196 INFO:tasks.workunit.client.1.smithi123.stdout:3/611: dread d0/d20/d76/d5d/dae/fb6 [0,4194304] 0 2024-02-16T22:22:01.198 INFO:tasks.workunit.client.0.smithi114.stdout:7/508: dwrite d3/d23/d6f/f90 [0,4194304] 0 2024-02-16T22:22:01.199 INFO:tasks.workunit.client.1.smithi123.stdout:4/640: dread d5/d1c/d23/d56/d6e/d9f/da1/fa4 [0,4194304] 0 2024-02-16T22:22:01.200 INFO:tasks.workunit.client.1.smithi123.stdout:9/710: write d4/d36/d57/f7d [3155095,112863] 0 2024-02-16T22:22:01.200 INFO:tasks.workunit.client.0.smithi114.stdout:9/483: symlink d2/d1a/d23/d32/d46/l9a 0 2024-02-16T22:22:01.207 INFO:tasks.workunit.client.0.smithi114.stdout:2/437: symlink dd/d3c/d4c/l95 0 2024-02-16T22:22:01.211 INFO:tasks.workunit.client.1.smithi123.stdout:5/569: symlink d6/lc5 0 2024-02-16T22:22:01.214 INFO:tasks.workunit.client.0.smithi114.stdout:3/438: symlink d4/dd/d33/d8f/l98 0 2024-02-16T22:22:01.216 INFO:tasks.workunit.client.0.smithi114.stdout:3/439: read d4/d46/d48/f5c [442772,51247] 0 2024-02-16T22:22:01.216 INFO:tasks.workunit.client.0.smithi114.stdout:1/425: symlink d4/l96 0 2024-02-16T22:22:01.218 INFO:tasks.workunit.client.0.smithi114.stdout:6/472: write d6/d83/f8f [4566888,58846] 0 2024-02-16T22:22:01.220 INFO:tasks.workunit.client.0.smithi114.stdout:3/440: write d4/d26/f63 [2758297,111305] 0 2024-02-16T22:22:01.220 INFO:tasks.workunit.client.0.smithi114.stdout:9/484: creat d2/d1a/d23/f9b x:0 0 0 2024-02-16T22:22:01.224 INFO:tasks.workunit.client.0.smithi114.stdout:7/509: rename d3/d23 to d3/d22/d73/da7 0 2024-02-16T22:22:01.225 INFO:tasks.workunit.client.0.smithi114.stdout:7/510: chown d3/d22/d73/da7/l3c 1 1 2024-02-16T22:22:01.226 INFO:tasks.workunit.client.0.smithi114.stdout:7/511: write d3/fc [4085973,115684] 0 2024-02-16T22:22:01.227 INFO:tasks.workunit.client.0.smithi114.stdout:7/512: chown d3/d22/c36 8838468 1 2024-02-16T22:22:01.228 INFO:tasks.workunit.client.0.smithi114.stdout:2/438: rmdir dd/d6b 39 2024-02-16T22:22:01.230 INFO:tasks.workunit.client.0.smithi114.stdout:5/439: creat d0/df/d16/d25/d3e/f82 x:0 0 0 2024-02-16T22:22:01.233 INFO:tasks.workunit.client.1.smithi123.stdout:2/573: rename d4/d1a/d1d/d13/d15/fac to d4/d1a/d1d/d13/d15/d6d/db1/fb2 0 2024-02-16T22:22:01.234 INFO:tasks.workunit.client.0.smithi114.stdout:1/426: chown d4/f16 1147 1 2024-02-16T22:22:01.235 INFO:tasks.workunit.client.0.smithi114.stdout:1/427: fsync d4/d9/f22 0 2024-02-16T22:22:01.235 INFO:tasks.workunit.client.1.smithi123.stdout:3/612: dread d0/f55 [0,4194304] 0 2024-02-16T22:22:01.237 INFO:tasks.workunit.client.1.smithi123.stdout:3/613: write d0/d20/d76/d5d/d60/f96 [819724,5453] 0 2024-02-16T22:22:01.238 INFO:tasks.workunit.client.1.smithi123.stdout:6/602: rename d0/d6/d1a/l35 to d0/d6/d91/d97/le0 0 2024-02-16T22:22:01.238 INFO:tasks.workunit.client.1.smithi123.stdout:6/603: fsync d0/d6/d2f/fad 0 2024-02-16T22:22:01.240 INFO:tasks.workunit.client.0.smithi114.stdout:8/449: link d4/d24/f7d d4/d11/d78/d7a/d31/fa5 0 2024-02-16T22:22:01.240 INFO:tasks.workunit.client.1.smithi123.stdout:6/604: read d0/d9/f6a [2167586,42943] 0 2024-02-16T22:22:01.242 INFO:tasks.workunit.client.0.smithi114.stdout:7/513: creat d3/d22/d73/da7/d6f/fa8 x:0 0 0 2024-02-16T22:22:01.243 INFO:tasks.workunit.client.1.smithi123.stdout:8/646: dwrite d0/d12/f58 [0,4194304] 0 2024-02-16T22:22:01.245 INFO:tasks.workunit.client.1.smithi123.stdout:8/647: chown d0/d29/f60 1 1 2024-02-16T22:22:01.246 INFO:tasks.workunit.client.0.smithi114.stdout:2/439: mknod dd/d3b/c96 0 2024-02-16T22:22:01.248 INFO:tasks.workunit.client.1.smithi123.stdout:0/527: rename d1/d8/l16 to d1/d8/lbb 0 2024-02-16T22:22:01.249 INFO:tasks.workunit.client.1.smithi123.stdout:0/528: chown d1/da0/fb9 83896 1 2024-02-16T22:22:01.249 INFO:tasks.workunit.client.0.smithi114.stdout:4/552: getdents dd/d1b/d37/d38/d87 0 2024-02-16T22:22:01.250 INFO:tasks.workunit.client.1.smithi123.stdout:7/614: write d3/d6e/d24/f6d [555797,75374] 0 2024-02-16T22:22:01.250 INFO:tasks.workunit.client.1.smithi123.stdout:9/711: dread d4/d33/f38 [0,4194304] 0 2024-02-16T22:22:01.254 INFO:tasks.workunit.client.0.smithi114.stdout:5/440: mknod d0/df/d16/d1e/c83 0 2024-02-16T22:22:01.255 INFO:tasks.workunit.client.0.smithi114.stdout:6/473: dread d6/d2c/d38/d54/f77 [0,4194304] 0 2024-02-16T22:22:01.255 INFO:tasks.workunit.client.0.smithi114.stdout:9/485: dread d2/d5c/d5f/d72/f79 [0,4194304] 0 2024-02-16T22:22:01.256 INFO:tasks.workunit.client.1.smithi123.stdout:6/605: read - d0/d9/dc/d3a/d70/f72 zero size 2024-02-16T22:22:01.256 INFO:tasks.workunit.client.1.smithi123.stdout:1/568: creat d3/d6/db/fbc x:0 0 0 2024-02-16T22:22:01.262 INFO:tasks.workunit.client.1.smithi123.stdout:1/569: dwrite d3/d6/d11/f62 [0,4194304] 0 2024-02-16T22:22:01.263 INFO:tasks.workunit.client.1.smithi123.stdout:1/570: chown d3/d6/d11/f62 150 1 2024-02-16T22:22:01.266 INFO:tasks.workunit.client.1.smithi123.stdout:2/574: creat d4/d1a/d1d/d59/fb3 x:0 0 0 2024-02-16T22:22:01.272 INFO:tasks.workunit.client.1.smithi123.stdout:0/529: rename d1/d8/d3d/d44 to d1/d8/d13/d1d/d64/dbc 0 2024-02-16T22:22:01.273 INFO:tasks.workunit.client.0.smithi114.stdout:7/514: symlink d3/d22/d73/da7/d6f/d89/la9 0 2024-02-16T22:22:01.273 INFO:tasks.workunit.client.1.smithi123.stdout:9/712: truncate d4/d36/d8e/d6d/d71/fa3 940886 0 2024-02-16T22:22:01.274 INFO:tasks.workunit.client.0.smithi114.stdout:2/440: rmdir dd/d3b 39 2024-02-16T22:22:01.276 INFO:tasks.workunit.client.1.smithi123.stdout:6/606: unlink d0/d6/d2f/fad 0 2024-02-16T22:22:01.277 INFO:tasks.workunit.client.1.smithi123.stdout:6/607: chown d0 5 1 2024-02-16T22:22:01.279 INFO:tasks.workunit.client.1.smithi123.stdout:8/648: symlink d0/d29/da7/ld2 0 2024-02-16T22:22:01.280 INFO:tasks.workunit.client.0.smithi114.stdout:5/441: symlink d0/df/d16/d25/d3e/l84 0 2024-02-16T22:22:01.284 INFO:tasks.workunit.client.0.smithi114.stdout:6/474: unlink d6/dd/d25/d34/d58/f81 0 2024-02-16T22:22:01.286 INFO:tasks.workunit.client.0.smithi114.stdout:0/396: dwrite d1/d16/f34 [0,4194304] 0 2024-02-16T22:22:01.287 INFO:tasks.workunit.client.0.smithi114.stdout:2/441: dread dd/df/d1c/d21/d61/d63/f73 [0,4194304] 0 2024-02-16T22:22:01.287 INFO:tasks.workunit.client.0.smithi114.stdout:2/442: stat dd/df/d1c/d21/d87/c66 0 2024-02-16T22:22:01.291 INFO:tasks.workunit.client.1.smithi123.stdout:0/530: rename d1/d4/d42/f65 to d1/d8/d13/d1d/d62/fbd 0 2024-02-16T22:22:01.298 INFO:tasks.workunit.client.1.smithi123.stdout:4/641: dwrite d5/d1c/d23/d31/f8f [0,4194304] 0 2024-02-16T22:22:01.299 INFO:tasks.workunit.client.1.smithi123.stdout:4/642: chown d5/l1a 199107511 1 2024-02-16T22:22:01.300 INFO:tasks.workunit.client.1.smithi123.stdout:1/571: creat d3/d6/d65/fbd x:0 0 0 2024-02-16T22:22:01.302 INFO:tasks.workunit.client.0.smithi114.stdout:5/442: rename d0/df/d53/f6f to d0/df/d16/d25/d2e/d3b/f85 0 2024-02-16T22:22:01.302 INFO:tasks.workunit.client.0.smithi114.stdout:5/443: chown d0/d28/f46 807264 1 2024-02-16T22:22:01.304 INFO:tasks.workunit.client.0.smithi114.stdout:5/444: truncate d0/df/d16/d25/f75 860874 0 2024-02-16T22:22:01.306 INFO:tasks.workunit.client.1.smithi123.stdout:0/531: readlink d1/d8/l77 0 2024-02-16T22:22:01.306 INFO:tasks.workunit.client.1.smithi123.stdout:0/532: chown d1/d8/d13/d1d/f8d 241195 1 2024-02-16T22:22:01.314 INFO:tasks.workunit.client.1.smithi123.stdout:8/649: unlink d0/df/c2b 0 2024-02-16T22:22:01.316 INFO:tasks.workunit.client.0.smithi114.stdout:8/450: link d4/d16/d25/d39/c49 d4/d11/d78/d7a/d7e/ca6 0 2024-02-16T22:22:01.316 INFO:tasks.workunit.client.0.smithi114.stdout:8/451: chown d4/d11/d78 4 1 2024-02-16T22:22:01.318 INFO:tasks.workunit.client.0.smithi114.stdout:7/515: fdatasync d3/d3d/f58 0 2024-02-16T22:22:01.319 INFO:tasks.workunit.client.0.smithi114.stdout:7/516: write d3/d14/f70 [3454980,45495] 0 2024-02-16T22:22:01.319 INFO:tasks.workunit.client.1.smithi123.stdout:3/614: truncate d0/fe 119331 0 2024-02-16T22:22:01.320 INFO:tasks.workunit.client.0.smithi114.stdout:7/517: write d3/d14/f82 [5272925,41226] 0 2024-02-16T22:22:01.320 INFO:tasks.workunit.client.1.smithi123.stdout:1/572: creat d3/d6/d49/d48/d84/d92/fbe x:0 0 0 2024-02-16T22:22:01.321 INFO:tasks.workunit.client.1.smithi123.stdout:1/573: chown d3/d6/db/d10/d4f/d9f 1 1 2024-02-16T22:22:01.322 INFO:tasks.workunit.client.0.smithi114.stdout:4/553: creat dd/d1b/d24/d57/fc9 x:0 0 0 2024-02-16T22:22:01.323 INFO:tasks.workunit.client.1.smithi123.stdout:1/574: write d3/d6/d49/d48/d84/d92/fbe [742140,45616] 0 2024-02-16T22:22:01.323 INFO:tasks.workunit.client.0.smithi114.stdout:3/441: truncate d4/d26/f58 4091639 0 2024-02-16T22:22:01.324 INFO:tasks.workunit.client.1.smithi123.stdout:1/575: write d3/d6/db/d10/d2d/f5a [784269,5690] 0 2024-02-16T22:22:01.326 INFO:tasks.workunit.client.1.smithi123.stdout:6/608: rmdir d0/d6/d2f/da5/dc6/ddd 0 2024-02-16T22:22:01.328 INFO:tasks.workunit.client.1.smithi123.stdout:6/609: readlink d0/d9/dc/lc9 0 2024-02-16T22:22:01.329 INFO:tasks.workunit.client.0.smithi114.stdout:6/475: rename d6/d2c/d38/l74 to d6/d2c/d5e/l93 0 2024-02-16T22:22:01.332 INFO:tasks.workunit.client.0.smithi114.stdout:9/486: link d2/d1a/d23/d32/c61 d2/d5c/d5f/d72/c9c 0 2024-02-16T22:22:01.332 INFO:tasks.workunit.client.1.smithi123.stdout:7/615: dwrite d3/d6e/d34/f9e [0,4194304] 0 2024-02-16T22:22:01.333 INFO:tasks.workunit.client.0.smithi114.stdout:6/476: dread d6/d2c/d75/f79 [0,4194304] 0 2024-02-16T22:22:01.334 INFO:tasks.workunit.client.1.smithi123.stdout:4/643: dread f4 [0,4194304] 0 2024-02-16T22:22:01.336 INFO:tasks.workunit.client.1.smithi123.stdout:4/644: write d5/d1c/d23/d31/fc1 [4121585,120423] 0 2024-02-16T22:22:01.339 INFO:tasks.workunit.client.1.smithi123.stdout:5/570: sync 2024-02-16T22:22:01.341 INFO:tasks.workunit.client.1.smithi123.stdout:1/576: dwrite d3/d6/d18/d9e/fae [4194304,4194304] 0 2024-02-16T22:22:01.342 INFO:tasks.workunit.client.1.smithi123.stdout:5/571: truncate d6/dd/df/d58/d3f/dbe/f1a 5114716 0 2024-02-16T22:22:01.343 INFO:tasks.workunit.client.0.smithi114.stdout:1/428: dwrite d4/d9/d3d/f41 [0,4194304] 0 2024-02-16T22:22:01.343 INFO:tasks.workunit.client.1.smithi123.stdout:8/650: symlink d0/ld3 0 2024-02-16T22:22:01.343 INFO:tasks.workunit.client.1.smithi123.stdout:5/572: chown d6/dd/df/d58/d3f/f9e 3 1 2024-02-16T22:22:01.343 INFO:tasks.workunit.client.1.smithi123.stdout:8/651: fdatasync d0/d12/f58 0 2024-02-16T22:22:01.344 INFO:tasks.workunit.client.0.smithi114.stdout:1/429: write d4/d9/f22 [2108269,16637] 0 2024-02-16T22:22:01.344 INFO:tasks.workunit.client.1.smithi123.stdout:5/573: fdatasync d6/dd/df/d1f/f6b 0 2024-02-16T22:22:01.344 INFO:tasks.workunit.client.1.smithi123.stdout:8/652: chown d0/d29/da7/d26/d32/d45/d67 2 1 2024-02-16T22:22:01.345 INFO:tasks.workunit.client.0.smithi114.stdout:1/430: stat d4/d9/d46/d67 0 2024-02-16T22:22:01.345 INFO:tasks.workunit.client.1.smithi123.stdout:8/653: readlink d0/d29/da7/d26/d2a/d3c/l3b 0 2024-02-16T22:22:01.349 INFO:tasks.workunit.client.1.smithi123.stdout:6/610: dread d0/d9/dc/d5f/f58 [0,4194304] 0 2024-02-16T22:22:01.353 INFO:tasks.workunit.client.0.smithi114.stdout:0/397: write d1/d1a/f38 [1005040,84991] 0 2024-02-16T22:22:01.355 INFO:tasks.workunit.client.1.smithi123.stdout:7/616: mkdir d3/d6e/d1d/d97/dca 0 2024-02-16T22:22:01.355 INFO:tasks.workunit.client.0.smithi114.stdout:2/443: write dd/f3a [1734336,45009] 0 2024-02-16T22:22:01.358 INFO:tasks.workunit.client.0.smithi114.stdout:5/445: rename d0/d28/d6e to d0/df/d16/d1e/d86 0 2024-02-16T22:22:01.364 INFO:tasks.workunit.client.1.smithi123.stdout:4/645: symlink d5/d1c/d23/d56/d6e/lca 0 2024-02-16T22:22:01.364 INFO:tasks.workunit.client.1.smithi123.stdout:4/646: chown d5/f1d 2 1 2024-02-16T22:22:01.365 INFO:tasks.workunit.client.0.smithi114.stdout:8/452: unlink d4/d11/c61 0 2024-02-16T22:22:01.365 INFO:tasks.workunit.client.1.smithi123.stdout:4/647: truncate d5/d1c/d64/f8b 5188026 0 2024-02-16T22:22:01.366 INFO:tasks.workunit.client.0.smithi114.stdout:8/453: dread - d4/d16/d25/d39/f82 zero size 2024-02-16T22:22:01.367 INFO:tasks.workunit.client.1.smithi123.stdout:1/577: rename d3/d47/f61 to d3/d6/d11/fbf 0 2024-02-16T22:22:01.369 INFO:tasks.workunit.client.1.smithi123.stdout:1/578: write d3/d6/db/d10/f4a [3957920,3378] 0 2024-02-16T22:22:01.370 INFO:tasks.workunit.client.1.smithi123.stdout:0/533: dwrite d1/d8/d13/f4d [0,4194304] 0 2024-02-16T22:22:01.370 INFO:tasks.workunit.client.1.smithi123.stdout:1/579: readlink d3/d6/db/l67 0 2024-02-16T22:22:01.371 INFO:tasks.workunit.client.1.smithi123.stdout:0/534: stat d1/d8/d13/d1d/d64/dbc 0 2024-02-16T22:22:01.371 INFO:tasks.workunit.client.0.smithi114.stdout:1/431: creat d4/d63/d69/d72/f97 x:0 0 0 2024-02-16T22:22:01.374 INFO:tasks.workunit.client.0.smithi114.stdout:3/442: fsync d4/d43/d53/f96 0 2024-02-16T22:22:01.375 INFO:tasks.workunit.client.0.smithi114.stdout:3/443: fsync d4/d46/d48/f57 0 2024-02-16T22:22:01.376 INFO:tasks.workunit.client.0.smithi114.stdout:0/398: chown d1/d16/d48/f6a 48397 1 2024-02-16T22:22:01.378 INFO:tasks.workunit.client.1.smithi123.stdout:6/611: mknod d0/d6/d1a/d2e/ce1 0 2024-02-16T22:22:01.378 INFO:tasks.workunit.client.1.smithi123.stdout:3/615: dwrite d0/d20/d76/d5d/d60/d93/fb9 [0,4194304] 0 2024-02-16T22:22:01.381 INFO:tasks.workunit.client.1.smithi123.stdout:2/575: sync 2024-02-16T22:22:01.382 INFO:tasks.workunit.client.0.smithi114.stdout:5/446: symlink d0/df/d16/d25/d2e/l87 0 2024-02-16T22:22:01.383 INFO:tasks.workunit.client.0.smithi114.stdout:5/447: chown d0/df/d16/d1e/f21 0 1 2024-02-16T22:22:01.387 INFO:tasks.workunit.client.0.smithi114.stdout:6/477: creat d6/dd/d25/d34/d61/d62/d8b/f94 x:0 0 0 2024-02-16T22:22:01.388 INFO:tasks.workunit.client.0.smithi114.stdout:6/478: read d6/d2c/f7c [4010355,52885] 0 2024-02-16T22:22:01.390 INFO:tasks.workunit.client.0.smithi114.stdout:6/479: write d6/d2c/d38/d41/f4a [6717934,73693] 0 2024-02-16T22:22:01.390 INFO:tasks.workunit.client.0.smithi114.stdout:8/454: truncate d4/d16/f4b 1547652 0 2024-02-16T22:22:01.391 INFO:tasks.workunit.client.1.smithi123.stdout:1/580: symlink d3/d6/d49/d48/d84/d92/lc0 0 2024-02-16T22:22:01.392 INFO:tasks.workunit.client.0.smithi114.stdout:1/432: mkdir d4/d9/d65/d98 0 2024-02-16T22:22:01.397 INFO:tasks.workunit.client.1.smithi123.stdout:8/654: mknod d0/cd4 0 2024-02-16T22:22:01.400 INFO:tasks.workunit.client.1.smithi123.stdout:5/574: write d6/dd/df/f62 [255443,130861] 0 2024-02-16T22:22:01.401 INFO:tasks.workunit.client.1.smithi123.stdout:6/612: creat d0/d6/d91/d97/fe2 x:0 0 0 2024-02-16T22:22:01.405 INFO:tasks.workunit.client.0.smithi114.stdout:2/444: unlink dd/d3b/d8a/f75 0 2024-02-16T22:22:01.406 INFO:tasks.workunit.client.0.smithi114.stdout:3/444: dwrite d4/d26/d4a/f65 [0,4194304] 0 2024-02-16T22:22:01.407 INFO:tasks.workunit.client.1.smithi123.stdout:3/616: creat d0/d20/d76/d37/d41/fd3 x:0 0 0 2024-02-16T22:22:01.410 INFO:tasks.workunit.client.1.smithi123.stdout:2/576: fsync d4/d1a/d1d/d13/d25/f3b 0 2024-02-16T22:22:01.413 INFO:tasks.workunit.client.1.smithi123.stdout:4/648: creat d5/d1c/d23/d74/d92/d9b/dbd/fcb x:0 0 0 2024-02-16T22:22:01.415 INFO:tasks.workunit.client.1.smithi123.stdout:4/649: fdatasync d5/d1c/d23/f3a 0 2024-02-16T22:22:01.415 INFO:tasks.workunit.client.1.smithi123.stdout:4/650: chown d5/d49/f89 112105 1 2024-02-16T22:22:01.419 INFO:tasks.workunit.client.1.smithi123.stdout:0/535: dwrite d1/d8/d13/f5a [0,4194304] 0 2024-02-16T22:22:01.426 INFO:tasks.workunit.client.1.smithi123.stdout:0/536: dwrite d1/d4/f11 [4194304,4194304] 0 2024-02-16T22:22:01.430 INFO:tasks.workunit.client.1.smithi123.stdout:9/713: sync 2024-02-16T22:22:01.455 INFO:tasks.workunit.client.0.smithi114.stdout:3/445: dread d4/d26/f34 [0,4194304] 0 2024-02-16T22:22:01.510 INFO:tasks.workunit.client.1.smithi123.stdout:8/655: symlink d0/d12/ld5 0 2024-02-16T22:22:01.525 INFO:tasks.workunit.client.1.smithi123.stdout:5/575: stat d6/dd/d44/dba 0 2024-02-16T22:22:01.526 INFO:tasks.workunit.client.1.smithi123.stdout:5/576: write d6/dd/d44/f47 [1389288,39525] 0 2024-02-16T22:22:01.527 INFO:tasks.workunit.client.0.smithi114.stdout:0/399: symlink d1/l7b 0 2024-02-16T22:22:01.528 INFO:tasks.workunit.client.0.smithi114.stdout:0/400: chown d1/d16/d1f/d28/d51 3 1 2024-02-16T22:22:01.528 INFO:tasks.workunit.client.0.smithi114.stdout:0/401: stat d1/d16/f19 0 2024-02-16T22:22:01.528 INFO:tasks.workunit.client.1.smithi123.stdout:6/613: fdatasync d0/f6d 0 2024-02-16T22:22:01.529 INFO:tasks.workunit.client.0.smithi114.stdout:0/402: chown d1/d1a/f76 27694 1 2024-02-16T22:22:01.529 INFO:tasks.workunit.client.1.smithi123.stdout:6/614: fsync d0/d6/d2f/d5d/fa3 0 2024-02-16T22:22:01.531 INFO:tasks.workunit.client.1.smithi123.stdout:7/617: creat d3/d6e/d1d/fcb x:0 0 0 2024-02-16T22:22:01.531 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:01 smithi114 ceph-mon[29235]: pgmap v28: 65 pgs: 65 active+clean; 1.8 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 53 MiB/s rd, 223 MiB/s wr, 382 op/s 2024-02-16T22:22:01.534 INFO:tasks.workunit.client.1.smithi123.stdout:3/617: creat d0/d20/d76/d5d/dae/dc7/fd4 x:0 0 0 2024-02-16T22:22:01.534 INFO:tasks.workunit.client.0.smithi114.stdout:6/480: dwrite d6/d2c/d38/d54/f2d [0,4194304] 0 2024-02-16T22:22:01.539 INFO:tasks.workunit.client.1.smithi123.stdout:2/577: truncate d4/d1a/d1d/f75 909124 0 2024-02-16T22:22:01.544 INFO:tasks.workunit.client.1.smithi123.stdout:1/581: link d3/d6/d11/f23 d3/d6/db/d10/d2a/fc1 0 2024-02-16T22:22:01.544 INFO:tasks.workunit.client.0.smithi114.stdout:4/554: link dd/d1b/d24/l7b dd/lca 0 2024-02-16T22:22:01.546 INFO:tasks.workunit.client.0.smithi114.stdout:2/445: dwrite dd/d4e/f90 [0,4194304] 0 2024-02-16T22:22:01.546 INFO:tasks.workunit.client.1.smithi123.stdout:9/714: creat d4/d36/daa/d7f/fe0 x:0 0 0 2024-02-16T22:22:01.547 INFO:tasks.workunit.client.1.smithi123.stdout:9/715: readlink d4/d36/d57/db2/l68 0 2024-02-16T22:22:01.548 INFO:tasks.workunit.client.1.smithi123.stdout:9/716: readlink d4/d36/d57/db2/l44 0 2024-02-16T22:22:01.548 INFO:tasks.workunit.client.0.smithi114.stdout:2/446: truncate dd/f3a 1886255 0 2024-02-16T22:22:01.553 INFO:tasks.workunit.client.1.smithi123.stdout:8/656: dwrite d0/d29/da7/d43/f27 [0,4194304] 0 2024-02-16T22:22:01.556 INFO:tasks.workunit.client.1.smithi123.stdout:7/618: fdatasync d3/d87/f4a 0 2024-02-16T22:22:01.559 INFO:tasks.workunit.client.1.smithi123.stdout:2/578: write d4/d1a/d1d/f2b [4390940,20012] 0 2024-02-16T22:22:01.560 INFO:tasks.workunit.client.1.smithi123.stdout:2/579: chown d4/d1a/d1d/d38/d7b/c83 9 1 2024-02-16T22:22:01.561 INFO:tasks.workunit.client.1.smithi123.stdout:4/651: rename d5/d1c/c38 to d5/d1c/d23/d56/ccc 0 2024-02-16T22:22:01.564 INFO:tasks.workunit.client.1.smithi123.stdout:1/582: creat d3/d6/db/d10/d4f/fc2 x:0 0 0 2024-02-16T22:22:01.569 INFO:tasks.workunit.client.1.smithi123.stdout:8/657: creat d0/d29/da7/d26/d2a/d77/fd6 x:0 0 0 2024-02-16T22:22:01.578 INFO:tasks.workunit.client.1.smithi123.stdout:0/537: rename d1/d4/d17/c61 to d1/d8/d13/d1d/d62/cbe 0 2024-02-16T22:22:01.578 INFO:tasks.workunit.client.1.smithi123.stdout:4/652: symlink d5/d1c/d23/d6d/lcd 0 2024-02-16T22:22:01.579 INFO:tasks.workunit.client.1.smithi123.stdout:4/653: chown d5/d1c/d9a/cbe 0 1 2024-02-16T22:22:01.579 INFO:tasks.workunit.client.1.smithi123.stdout:0/538: read d1/d4/f51 [2982528,28136] 0 2024-02-16T22:22:01.580 INFO:tasks.workunit.client.1.smithi123.stdout:1/583: creat d3/d6/db/d10/d4f/d9f/fc3 x:0 0 0 2024-02-16T22:22:01.581 INFO:tasks.workunit.client.1.smithi123.stdout:9/717: mkdir d4/d36/daa/d2b/de1 0 2024-02-16T22:22:01.585 INFO:tasks.workunit.client.0.smithi114.stdout:6/481: readlink d6/d2c/d38/d54/l2b 0 2024-02-16T22:22:01.585 INFO:tasks.workunit.client.1.smithi123.stdout:4/654: dwrite d5/d1c/d23/d74/d92/f86 [0,4194304] 0 2024-02-16T22:22:01.585 INFO:tasks.workunit.client.0.smithi114.stdout:6/482: chown d6/d83/f8f 559 1 2024-02-16T22:22:01.587 INFO:tasks.workunit.client.0.smithi114.stdout:6/483: write d6/dd/d25/f4c [1579047,98935] 0 2024-02-16T22:22:01.588 INFO:tasks.workunit.client.0.smithi114.stdout:6/484: chown d6/dd/d25/f53 5 1 2024-02-16T22:22:01.588 INFO:tasks.workunit.client.1.smithi123.stdout:4/655: read d5/d1c/d5e/dc3/f4a [834441,129272] 0 2024-02-16T22:22:01.590 INFO:tasks.workunit.client.1.smithi123.stdout:4/656: chown d5/d1c/d23/d74/d92/d9b 1130 1 2024-02-16T22:22:01.590 INFO:tasks.workunit.client.0.smithi114.stdout:8/455: rename d4/d24/d2c/l98 to d4/d16/d25/la7 0 2024-02-16T22:22:01.598 INFO:tasks.workunit.client.1.smithi123.stdout:8/658: dread d0/d29/da7/d26/d2a/f54 [0,4194304] 0 2024-02-16T22:22:01.599 INFO:tasks.workunit.client.1.smithi123.stdout:3/618: rename d0/d20/f42 to d0/d20/d94/fd5 0 2024-02-16T22:22:01.599 INFO:tasks.workunit.client.1.smithi123.stdout:0/539: dread d1/d4/d17/d74/d57/f75 [0,4194304] 0 2024-02-16T22:22:01.603 INFO:tasks.workunit.client.0.smithi114.stdout:0/403: unlink d1/d1a/f3b 0 2024-02-16T22:22:01.605 INFO:tasks.workunit.client.1.smithi123.stdout:1/584: symlink d3/d47/lc4 0 2024-02-16T22:22:01.607 INFO:tasks.workunit.client.1.smithi123.stdout:6/615: getdents d0/d9/dc 0 2024-02-16T22:22:01.607 INFO:tasks.workunit.client.1.smithi123.stdout:6/616: dread - d0/d74/fce zero size 2024-02-16T22:22:01.607 INFO:tasks.workunit.client.0.smithi114.stdout:6/485: fdatasync d6/dd/d25/f53 0 2024-02-16T22:22:01.613 INFO:tasks.workunit.client.0.smithi114.stdout:3/446: creat d4/d24/f99 x:0 0 0 2024-02-16T22:22:01.614 INFO:tasks.workunit.client.0.smithi114.stdout:2/447: dread dd/d3b/d8a/f93 [0,4194304] 0 2024-02-16T22:22:01.614 INFO:tasks.workunit.client.0.smithi114.stdout:3/447: readlink d4/d24/l4f 0 2024-02-16T22:22:01.615 INFO:tasks.workunit.client.0.smithi114.stdout:0/404: symlink d1/d16/d1f/d28/d51/d61/l7c 0 2024-02-16T22:22:01.616 INFO:tasks.workunit.client.0.smithi114.stdout:0/405: stat d1/d1a/d47/c68 0 2024-02-16T22:22:01.616 INFO:tasks.workunit.client.0.smithi114.stdout:8/456: dread d4/d11/f71 [0,4194304] 0 2024-02-16T22:22:01.619 INFO:tasks.workunit.client.0.smithi114.stdout:6/486: unlink d6/d2c/d38/d54/l2b 0 2024-02-16T22:22:01.619 INFO:tasks.workunit.client.0.smithi114.stdout:4/555: write dd/d59/f5c [201241,101512] 0 2024-02-16T22:22:01.621 INFO:tasks.workunit.client.0.smithi114.stdout:8/457: dread d4/d11/d78/d7a/d36/f5d [0,4194304] 0 2024-02-16T22:22:01.622 INFO:tasks.workunit.client.1.smithi123.stdout:7/619: dwrite d3/d6e/f14 [0,4194304] 0 2024-02-16T22:22:01.625 INFO:tasks.workunit.client.1.smithi123.stdout:3/619: rename d0/d85 to d0/d20/d76/d5d/d60/dd6 0 2024-02-16T22:22:01.627 INFO:tasks.workunit.client.0.smithi114.stdout:2/448: mkdir dd/d4e/d97 0 2024-02-16T22:22:01.631 INFO:tasks.workunit.client.0.smithi114.stdout:3/448: rename d4/f1c to d4/d46/d48/d75/f9a 0 2024-02-16T22:22:01.632 INFO:tasks.workunit.client.0.smithi114.stdout:3/449: read d4/d26/d4a/f65 [286364,22880] 0 2024-02-16T22:22:01.633 INFO:tasks.workunit.client.1.smithi123.stdout:1/585: unlink d3/d6/db/d10/d2a/d53/la7 0 2024-02-16T22:22:01.636 INFO:tasks.workunit.client.0.smithi114.stdout:6/487: creat d6/d2c/f95 x:0 0 0 2024-02-16T22:22:01.639 INFO:tasks.workunit.client.1.smithi123.stdout:0/540: write d1/d8/d13/f37 [633264,94549] 0 2024-02-16T22:22:01.639 INFO:tasks.workunit.client.1.smithi123.stdout:0/541: stat d1/d4/d17/f41 0 2024-02-16T22:22:01.640 INFO:tasks.workunit.client.1.smithi123.stdout:2/580: getdents d4/d1a/d1d/d13/d15/d6d/d4a 0 2024-02-16T22:22:01.641 INFO:tasks.workunit.client.0.smithi114.stdout:2/449: creat dd/df/d1c/d21/d61/d69/f98 x:0 0 0 2024-02-16T22:22:01.642 INFO:tasks.workunit.client.0.smithi114.stdout:2/450: chown dd/df/d1c/l4b 3 1 2024-02-16T22:22:01.646 INFO:tasks.workunit.client.0.smithi114.stdout:3/450: creat d4/dd/d33/d72/f9b x:0 0 0 2024-02-16T22:22:01.648 INFO:tasks.workunit.client.0.smithi114.stdout:0/406: creat d1/d78/f7d x:0 0 0 2024-02-16T22:22:01.649 INFO:tasks.workunit.client.1.smithi123.stdout:7/620: symlink d3/d59/lcc 0 2024-02-16T22:22:01.649 INFO:tasks.workunit.client.1.smithi123.stdout:9/718: write d4/d36/d57/db2/f69 [277054,18986] 0 2024-02-16T22:22:01.650 INFO:tasks.workunit.client.0.smithi114.stdout:6/488: symlink d6/d2c/l96 0 2024-02-16T22:22:01.652 INFO:tasks.workunit.client.1.smithi123.stdout:1/586: rename d3/d6/db/d10/d2a/f6f to d3/d6/d49/d48/d68/fc5 0 2024-02-16T22:22:01.654 INFO:tasks.workunit.client.1.smithi123.stdout:8/659: write d0/f75 [5767887,45141] 0 2024-02-16T22:22:01.654 INFO:tasks.workunit.client.0.smithi114.stdout:0/407: dwrite d1/d1a/f2e [0,4194304] 0 2024-02-16T22:22:01.657 INFO:tasks.workunit.client.0.smithi114.stdout:6/489: dread d6/dd/d25/d34/f8e [0,4194304] 0 2024-02-16T22:22:01.657 INFO:tasks.workunit.client.0.smithi114.stdout:8/458: write d4/d24/d2c/d5f/f93 [150420,57513] 0 2024-02-16T22:22:01.658 INFO:tasks.workunit.client.0.smithi114.stdout:6/490: truncate d6/dd/d25/f73 60629 0 2024-02-16T22:22:01.659 INFO:tasks.workunit.client.1.smithi123.stdout:6/617: truncate d0/d6/d91/d93/f95 913790 0 2024-02-16T22:22:01.659 INFO:tasks.workunit.client.0.smithi114.stdout:6/491: dread - d6/d2c/d66/f68 zero size 2024-02-16T22:22:01.659 INFO:tasks.workunit.client.1.smithi123.stdout:4/657: link d5/d1c/d23/d74/l37 d5/d1c/d23/d74/lce 0 2024-02-16T22:22:01.660 INFO:tasks.workunit.client.0.smithi114.stdout:6/492: write d6/d83/f87 [3016610,116431] 0 2024-02-16T22:22:01.660 INFO:tasks.workunit.client.0.smithi114.stdout:2/451: fsync dd/df/d1c/f40 0 2024-02-16T22:22:01.661 INFO:tasks.workunit.client.1.smithi123.stdout:4/658: dread - d5/d1c/d23/d76/fa6 zero size 2024-02-16T22:22:01.662 INFO:tasks.workunit.client.1.smithi123.stdout:0/542: symlink d1/da0/lbf 0 2024-02-16T22:22:01.663 INFO:tasks.workunit.client.1.smithi123.stdout:0/543: fdatasync d1/da0/faa 0 2024-02-16T22:22:01.664 INFO:tasks.workunit.client.0.smithi114.stdout:3/451: creat d4/d46/d48/d75/f9c x:0 0 0 2024-02-16T22:22:01.665 INFO:tasks.workunit.client.0.smithi114.stdout:3/452: fdatasync d4/d24/f6d 0 2024-02-16T22:22:01.666 INFO:tasks.workunit.client.0.smithi114.stdout:3/453: fdatasync d4/d26/d4a/f90 0 2024-02-16T22:22:01.666 INFO:tasks.workunit.client.0.smithi114.stdout:3/454: chown d4 5470539 1 2024-02-16T22:22:01.669 INFO:tasks.workunit.client.1.smithi123.stdout:9/719: creat d4/d33/fe2 x:0 0 0 2024-02-16T22:22:01.669 INFO:tasks.workunit.client.1.smithi123.stdout:9/720: chown d4/d36/d8e/d6f/d7e/c83 3701841 1 2024-02-16T22:22:01.670 INFO:tasks.workunit.client.0.smithi114.stdout:4/556: link dd/d1b/d37/d38/l4d dd/d2e/d3f/lcb 0 2024-02-16T22:22:01.672 INFO:tasks.workunit.client.0.smithi114.stdout:4/557: truncate dd/d1b/d41/d77/fb1 108099 0 2024-02-16T22:22:01.673 INFO:tasks.workunit.client.0.smithi114.stdout:3/455: dwrite d4/d24/f6d [0,4194304] 0 2024-02-16T22:22:01.674 INFO:tasks.workunit.client.1.smithi123.stdout:1/587: mknod d3/d52/cc6 0 2024-02-16T22:22:01.674 INFO:tasks.workunit.client.0.smithi114.stdout:3/456: dread - d4/dd/d33/d72/f9b zero size 2024-02-16T22:22:01.684 INFO:tasks.workunit.client.1.smithi123.stdout:7/621: write d3/d6e/d52/f88 [782179,17811] 0 2024-02-16T22:22:01.689 INFO:tasks.workunit.client.1.smithi123.stdout:5/577: sync 2024-02-16T22:22:01.690 INFO:tasks.workunit.client.0.smithi114.stdout:0/408: truncate d1/fa 303358 0 2024-02-16T22:22:01.692 INFO:tasks.workunit.client.0.smithi114.stdout:4/558: rename dd/d1b/d41/d77/l9a to dd/d1b/d41/dc3/lcc 0 2024-02-16T22:22:01.694 INFO:tasks.workunit.client.0.smithi114.stdout:3/457: symlink d4/dd/l9d 0 2024-02-16T22:22:01.695 INFO:tasks.workunit.client.0.smithi114.stdout:8/459: mkdir d4/d11/d9d/da8 0 2024-02-16T22:22:01.696 INFO:tasks.workunit.client.0.smithi114.stdout:2/452: dread dd/df/f52 [0,4194304] 0 2024-02-16T22:22:01.700 INFO:tasks.workunit.client.0.smithi114.stdout:0/409: mkdir d1/d16/d1f/d28/d51/d61/d7e 0 2024-02-16T22:22:01.701 INFO:tasks.workunit.client.0.smithi114.stdout:4/559: symlink dd/d2e/d3f/lcd 0 2024-02-16T22:22:01.703 INFO:tasks.workunit.client.0.smithi114.stdout:3/458: creat d4/dd/d33/d76/f9e x:0 0 0 2024-02-16T22:22:01.704 INFO:tasks.workunit.client.0.smithi114.stdout:2/453: fdatasync dd/d3b/d8a/f93 0 2024-02-16T22:22:01.706 INFO:tasks.workunit.client.0.smithi114.stdout:2/454: write dd/d3c/d4c/f86 [956637,127180] 0 2024-02-16T22:22:01.706 INFO:tasks.workunit.client.0.smithi114.stdout:6/493: readlink d6/d2c/d38/d41/l7e 0 2024-02-16T22:22:01.707 INFO:tasks.workunit.client.0.smithi114.stdout:2/455: write dd/df/d1c/d21/f26 [258941,59149] 0 2024-02-16T22:22:01.708 INFO:tasks.workunit.client.0.smithi114.stdout:0/410: mkdir d1/d1a/d47/d62/d7f 0 2024-02-16T22:22:01.710 INFO:tasks.workunit.client.0.smithi114.stdout:4/560: chown dd/d1b/d24/f6b 15350824 1 2024-02-16T22:22:01.717 INFO:tasks.workunit.client.0.smithi114.stdout:6/494: dread d6/d2c/d75/f79 [0,4194304] 0 2024-02-16T22:22:01.722 INFO:tasks.workunit.client.0.smithi114.stdout:4/561: mknod dd/d1b/d37/d38/d87/dad/cce 0 2024-02-16T22:22:01.728 INFO:tasks.workunit.client.0.smithi114.stdout:6/495: fdatasync d6/dd/d25/f27 0 2024-02-16T22:22:01.731 INFO:tasks.workunit.client.1.smithi123.stdout:3/620: sync 2024-02-16T22:22:01.731 INFO:tasks.workunit.client.0.smithi114.stdout:4/562: creat dd/d1b/d41/d77/fcf x:0 0 0 2024-02-16T22:22:01.732 INFO:tasks.workunit.client.1.smithi123.stdout:3/621: write d0/d20/d76/d5d/d60/d93/fb9 [5011095,57788] 0 2024-02-16T22:22:01.733 INFO:tasks.workunit.client.0.smithi114.stdout:3/459: link d4/c2a d4/dd/d33/d72/c9f 0 2024-02-16T22:22:01.733 INFO:tasks.workunit.client.1.smithi123.stdout:3/622: chown d0/d20/d2e 676997259 1 2024-02-16T22:22:01.733 INFO:tasks.workunit.client.0.smithi114.stdout:8/460: dwrite d4/d16/f91 [0,4194304] 0 2024-02-16T22:22:01.736 INFO:tasks.workunit.client.0.smithi114.stdout:2/456: write dd/d3c/d4c/f64 [24249,80395] 0 2024-02-16T22:22:01.737 INFO:tasks.workunit.client.1.smithi123.stdout:8/660: mkdir d0/d29/da7/d26/d68/d81/dd7 0 2024-02-16T22:22:01.739 INFO:tasks.workunit.client.0.smithi114.stdout:6/496: chown d6/d2c/d38/d54/f2d 11 1 2024-02-16T22:22:01.742 INFO:tasks.workunit.client.0.smithi114.stdout:3/460: dwrite d4/d24/d2d/d45/d68/f86 [0,4194304] 0 2024-02-16T22:22:01.747 INFO:tasks.workunit.client.1.smithi123.stdout:2/581: rmdir d4/d1a/d1d/d13/d25/d72 39 2024-02-16T22:22:01.751 INFO:tasks.workunit.client.1.smithi123.stdout:2/582: read d4/d1a/d1d/f9f [1933282,43081] 0 2024-02-16T22:22:01.753 INFO:tasks.workunit.client.1.smithi123.stdout:6/618: dread d0/d6/d1a/d2e/f7f [0,4194304] 0 2024-02-16T22:22:01.753 INFO:tasks.workunit.client.0.smithi114.stdout:6/497: dwrite d6/dd/d25/d34/f6f [0,4194304] 0 2024-02-16T22:22:01.755 INFO:tasks.workunit.client.1.smithi123.stdout:9/721: creat d4/d36/d8e/dbe/fe3 x:0 0 0 2024-02-16T22:22:01.756 INFO:tasks.workunit.client.0.smithi114.stdout:6/498: fdatasync d6/f18 0 2024-02-16T22:22:01.756 INFO:tasks.workunit.client.0.smithi114.stdout:3/461: dwrite d4/d24/f25 [0,4194304] 0 2024-02-16T22:22:01.758 INFO:tasks.workunit.client.1.smithi123.stdout:1/588: rename d3/d6/db/d10/d4f/d9f/fb9 to d3/d6/d11/fc7 0 2024-02-16T22:22:01.760 INFO:tasks.workunit.client.0.smithi114.stdout:4/563: dwrite dd/d1b/d24/d98/fc5 [0,4194304] 0 2024-02-16T22:22:01.766 INFO:tasks.workunit.client.0.smithi114.stdout:3/462: truncate d4/d24/f62 1529330 0 2024-02-16T22:22:01.767 INFO:tasks.workunit.client.1.smithi123.stdout:7/622: creat d3/d6e/d4d/d92/fcd x:0 0 0 2024-02-16T22:22:01.767 INFO:tasks.workunit.client.1.smithi123.stdout:5/578: dread - d6/f4c zero size 2024-02-16T22:22:01.770 INFO:tasks.workunit.client.1.smithi123.stdout:3/623: fsync d0/d20/d94/fd5 0 2024-02-16T22:22:01.771 INFO:tasks.workunit.client.0.smithi114.stdout:6/499: dwrite d6/f18 [4194304,4194304] 0 2024-02-16T22:22:01.773 INFO:tasks.workunit.client.1.smithi123.stdout:8/661: creat d0/d29/da7/d43/d34/fd8 x:0 0 0 2024-02-16T22:22:01.773 INFO:tasks.workunit.client.0.smithi114.stdout:2/457: write dd/df/d1c/d21/f30 [1039799,42437] 0 2024-02-16T22:22:01.775 INFO:tasks.workunit.client.0.smithi114.stdout:2/458: read - dd/d3c/f77 zero size 2024-02-16T22:22:01.775 INFO:tasks.workunit.client.1.smithi123.stdout:5/579: dwrite d6/f8 [0,4194304] 0 2024-02-16T22:22:01.776 INFO:tasks.workunit.client.0.smithi114.stdout:2/459: readlink l2 0 2024-02-16T22:22:01.776 INFO:tasks.workunit.client.0.smithi114.stdout:3/463: mknod d4/d24/ca0 0 2024-02-16T22:22:01.778 INFO:tasks.workunit.client.0.smithi114.stdout:6/500: dwrite d6/d2c/f46 [0,4194304] 0 2024-02-16T22:22:01.779 INFO:tasks.workunit.client.1.smithi123.stdout:2/583: creat d4/d1a/d1d/d13/d15/fb4 x:0 0 0 2024-02-16T22:22:01.780 INFO:tasks.workunit.client.1.smithi123.stdout:9/722: fdatasync d4/f6b 0 2024-02-16T22:22:01.780 INFO:tasks.workunit.client.1.smithi123.stdout:5/580: truncate d6/dd/df/d58/d89/f9c 374283 0 2024-02-16T22:22:01.780 INFO:tasks.workunit.client.0.smithi114.stdout:4/564: creat dd/d59/d8a/fd0 x:0 0 0 2024-02-16T22:22:01.785 INFO:tasks.workunit.client.0.smithi114.stdout:3/464: read - d4/d46/d48/f66 zero size 2024-02-16T22:22:01.786 INFO:tasks.workunit.client.1.smithi123.stdout:6/619: rename d0/d9/dc/d5f to d0/d96/de3 0 2024-02-16T22:22:01.786 INFO:tasks.workunit.client.1.smithi123.stdout:8/662: dwrite d0/d29/da7/d26/d68/d81/d47/fa6 [0,4194304] 0 2024-02-16T22:22:01.786 INFO:tasks.workunit.client.0.smithi114.stdout:3/465: dread - d4/dd/f35 zero size 2024-02-16T22:22:01.788 INFO:tasks.workunit.client.1.smithi123.stdout:7/623: mkdir d3/d6e/d1d/d97/dce 0 2024-02-16T22:22:01.790 INFO:tasks.workunit.client.1.smithi123.stdout:6/620: write d0/d6/f18 [1364110,44222] 0 2024-02-16T22:22:01.791 INFO:tasks.workunit.client.0.smithi114.stdout:8/461: write d4/d11/d78/d7a/d31/fa5 [266920,45463] 0 2024-02-16T22:22:01.793 INFO:tasks.workunit.client.1.smithi123.stdout:6/621: stat d0/d6/d2f/l1c 0 2024-02-16T22:22:01.793 INFO:tasks.workunit.client.0.smithi114.stdout:6/501: dwrite d6/d2c/f7c [0,4194304] 0 2024-02-16T22:22:01.795 INFO:tasks.workunit.client.0.smithi114.stdout:3/466: mknod d4/d87/ca1 0 2024-02-16T22:22:01.795 INFO:tasks.workunit.client.1.smithi123.stdout:5/581: dwrite d6/dd/df/d58/d3f/dbe/f1a [0,4194304] 0 2024-02-16T22:22:01.798 INFO:tasks.workunit.client.1.smithi123.stdout:8/663: dwrite d0/d29/da7/d43/f28 [0,4194304] 0 2024-02-16T22:22:01.800 INFO:tasks.workunit.client.1.smithi123.stdout:0/544: link d1/d8/d3d/f87 d1/d4/d42/fc0 0 2024-02-16T22:22:01.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:01 smithi123 ceph-mon[37161]: pgmap v28: 65 pgs: 65 active+clean; 1.8 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 53 MiB/s rd, 223 MiB/s wr, 382 op/s 2024-02-16T22:22:01.803 INFO:tasks.workunit.client.1.smithi123.stdout:0/545: write d1/d4/d17/d54/d8e/f78 [811549,90294] 0 2024-02-16T22:22:01.808 INFO:tasks.workunit.client.1.smithi123.stdout:2/584: creat d4/d1a/d1d/d13/d15/d6d/fb5 x:0 0 0 2024-02-16T22:22:01.811 INFO:tasks.workunit.client.1.smithi123.stdout:1/589: dread d3/d6/d11/d76/f59 [0,4194304] 0 2024-02-16T22:22:01.812 INFO:tasks.workunit.client.1.smithi123.stdout:1/590: readlink d3/d47/lc4 0 2024-02-16T22:22:01.814 INFO:tasks.workunit.client.0.smithi114.stdout:3/467: symlink d4/d26/d91/la2 0 2024-02-16T22:22:01.815 INFO:tasks.workunit.client.0.smithi114.stdout:3/468: write d4/d24/d2d/f4e [50933,113819] 0 2024-02-16T22:22:01.816 INFO:tasks.workunit.client.1.smithi123.stdout:3/624: getdents d0/d20/d2e/dc4 0 2024-02-16T22:22:01.817 INFO:tasks.workunit.client.1.smithi123.stdout:7/624: fsync d3/d6e/d1d/db2/f7e 0 2024-02-16T22:22:01.820 INFO:tasks.workunit.client.1.smithi123.stdout:8/664: symlink d0/d29/da7/d26/d2a/d3c/d62/ld9 0 2024-02-16T22:22:01.821 INFO:tasks.workunit.client.0.smithi114.stdout:2/460: dwrite dd/f92 [0,4194304] 0 2024-02-16T22:22:01.821 INFO:tasks.workunit.client.1.smithi123.stdout:8/665: chown d0/d29/da7/d43/d34 21 1 2024-02-16T22:22:01.821 INFO:tasks.workunit.client.1.smithi123.stdout:4/659: getdents d5/d1c/d23/d56/d6e 0 2024-02-16T22:22:01.824 INFO:tasks.workunit.client.1.smithi123.stdout:0/546: creat d1/d4/d17/d74/fc1 x:0 0 0 2024-02-16T22:22:01.827 INFO:tasks.workunit.client.1.smithi123.stdout:2/585: creat d4/d1a/d1d/d13/fb6 x:0 0 0 2024-02-16T22:22:01.827 INFO:tasks.workunit.client.0.smithi114.stdout:3/469: link d4/d24/d2d/d45/l4b d4/d46/d48/la3 0 2024-02-16T22:22:01.828 INFO:tasks.workunit.client.1.smithi123.stdout:2/586: chown d4/d1a/d1d/d13/d15/l5a 2626 1 2024-02-16T22:22:01.829 INFO:tasks.workunit.client.1.smithi123.stdout:9/723: mkdir d4/d36/d8e/d58/d59/d60/d9d/de4 0 2024-02-16T22:22:01.829 INFO:tasks.workunit.client.0.smithi114.stdout:2/461: dwrite dd/df/f1e [4194304,4194304] 0 2024-02-16T22:22:01.829 INFO:tasks.workunit.client.0.smithi114.stdout:3/470: fdatasync d4/dd/d33/d72/f9b 0 2024-02-16T22:22:01.830 INFO:tasks.workunit.client.0.smithi114.stdout:2/462: chown dd/df/d1c/d21/f26 6379 1 2024-02-16T22:22:01.833 INFO:tasks.workunit.client.0.smithi114.stdout:2/463: fsync dd/df/f41 0 2024-02-16T22:22:01.835 INFO:tasks.workunit.client.1.smithi123.stdout:0/547: symlink d1/da0/lc2 0 2024-02-16T22:22:01.839 INFO:tasks.workunit.client.1.smithi123.stdout:2/587: truncate d4/d1a/d1d/d13/d15/d6d/db1/fb2 452899 0 2024-02-16T22:22:01.840 INFO:tasks.workunit.client.1.smithi123.stdout:7/625: truncate d3/d6e/d34/f75 1940996 0 2024-02-16T22:22:01.841 INFO:tasks.workunit.client.0.smithi114.stdout:2/464: dwrite dd/d3c/d4c/f86 [0,4194304] 0 2024-02-16T22:22:01.841 INFO:tasks.workunit.client.1.smithi123.stdout:8/666: symlink d0/d61/d66/lda 0 2024-02-16T22:22:01.842 INFO:tasks.workunit.client.0.smithi114.stdout:4/565: write dd/f2b [1316031,87336] 0 2024-02-16T22:22:01.842 INFO:tasks.workunit.client.1.smithi123.stdout:8/667: stat d0/d29/da7/d26/d2a/f78 0 2024-02-16T22:22:01.843 INFO:tasks.workunit.client.1.smithi123.stdout:4/660: mkdir d5/d1c/d23/d56/d6e/d9f/da1/dc0/dc5/dcf 0 2024-02-16T22:22:01.846 INFO:tasks.workunit.client.1.smithi123.stdout:3/625: creat d0/d20/d76/d5d/fd7 x:0 0 0 2024-02-16T22:22:01.847 INFO:tasks.workunit.client.0.smithi114.stdout:2/465: symlink dd/df/d1c/d21/d61/d63/l99 0 2024-02-16T22:22:01.847 INFO:tasks.workunit.client.0.smithi114.stdout:8/462: write d4/d11/d78/d7a/d31/f8e [241845,51487] 0 2024-02-16T22:22:01.848 INFO:tasks.workunit.client.0.smithi114.stdout:6/502: dread d6/d83/f87 [0,4194304] 0 2024-02-16T22:22:01.849 INFO:tasks.workunit.client.1.smithi123.stdout:9/724: creat d4/d36/d8e/d6d/d71/da6/fe5 x:0 0 0 2024-02-16T22:22:01.851 INFO:tasks.workunit.client.1.smithi123.stdout:9/725: chown d4/d1e/fc8 4089 1 2024-02-16T22:22:01.851 INFO:tasks.workunit.client.1.smithi123.stdout:4/661: dwrite d5/d1c/d5e/dc3/f75 [0,4194304] 0 2024-02-16T22:22:01.851 INFO:tasks.workunit.client.1.smithi123.stdout:9/726: readlink d4/d36/l54 0 2024-02-16T22:22:01.852 INFO:tasks.workunit.client.0.smithi114.stdout:7/518: sync 2024-02-16T22:22:01.852 INFO:tasks.workunit.client.1.smithi123.stdout:2/588: creat d4/d1a/d1d/d13/d15/d6d/db1/fb7 x:0 0 0 2024-02-16T22:22:01.852 INFO:tasks.workunit.client.1.smithi123.stdout:4/662: stat d5/d49/f89 0 2024-02-16T22:22:01.853 INFO:tasks.workunit.client.1.smithi123.stdout:4/663: chown d5/d1c/d23/f9c 88331187 1 2024-02-16T22:22:01.856 INFO:tasks.workunit.client.1.smithi123.stdout:7/626: rmdir d3/d59 39 2024-02-16T22:22:01.856 INFO:tasks.workunit.client.1.smithi123.stdout:7/627: read - d3/d5/d6c/fac zero size 2024-02-16T22:22:01.857 INFO:tasks.workunit.client.0.smithi114.stdout:2/466: creat dd/d3b/d8a/f9a x:0 0 0 2024-02-16T22:22:01.864 INFO:tasks.workunit.client.0.smithi114.stdout:7/519: creat d3/d14/faa x:0 0 0 2024-02-16T22:22:01.865 INFO:tasks.workunit.client.1.smithi123.stdout:5/582: dwrite d6/dd/df/d58/d3f/dbe/f3a [4194304,4194304] 0 2024-02-16T22:22:01.866 INFO:tasks.workunit.client.0.smithi114.stdout:4/566: link dd/d1b/d24/l60 dd/d1b/d37/d38/d87/ld1 0 2024-02-16T22:22:01.869 INFO:tasks.workunit.client.0.smithi114.stdout:3/471: write d4/d46/d48/f66 [890566,113626] 0 2024-02-16T22:22:01.870 INFO:tasks.workunit.client.0.smithi114.stdout:2/467: write dd/d6b/f89 [979799,40628] 0 2024-02-16T22:22:01.870 INFO:tasks.workunit.client.1.smithi123.stdout:3/626: symlink d0/d20/d76/d5d/d60/d93/ld8 0 2024-02-16T22:22:01.874 INFO:tasks.workunit.client.1.smithi123.stdout:9/727: truncate d4/d36/daa/d20/f7b 4454358 0 2024-02-16T22:22:01.875 INFO:tasks.workunit.client.1.smithi123.stdout:1/591: sync 2024-02-16T22:22:01.875 INFO:tasks.workunit.client.1.smithi123.stdout:9/728: readlink d4/d33/l4b 0 2024-02-16T22:22:01.877 INFO:tasks.workunit.client.1.smithi123.stdout:2/589: write d4/d1a/d1d/f9f [937690,8257] 0 2024-02-16T22:22:01.877 INFO:tasks.workunit.client.1.smithi123.stdout:4/664: dread d5/d1c/d23/d47/f62 [0,4194304] 0 2024-02-16T22:22:01.878 INFO:tasks.workunit.client.0.smithi114.stdout:4/567: chown dd/d2e/l39 11 1 2024-02-16T22:22:01.878 INFO:tasks.workunit.client.0.smithi114.stdout:2/468: dwrite dd/f7a [0,4194304] 0 2024-02-16T22:22:01.879 INFO:tasks.workunit.client.0.smithi114.stdout:8/463: dread d4/d16/f1e [0,4194304] 0 2024-02-16T22:22:01.880 INFO:tasks.workunit.client.1.smithi123.stdout:7/628: rename d3/d6e/l6b to d3/d6e/d1d/db2/d35/lcf 0 2024-02-16T22:22:01.881 INFO:tasks.workunit.client.0.smithi114.stdout:8/464: dread - d4/d16/d25/d46/f73 zero size 2024-02-16T22:22:01.881 INFO:tasks.workunit.client.1.smithi123.stdout:6/622: dwrite d0/d9/dc/d3a/f40 [0,4194304] 0 2024-02-16T22:22:01.885 INFO:tasks.workunit.client.1.smithi123.stdout:4/665: dwrite d5/f1e [0,4194304] 0 2024-02-16T22:22:01.885 INFO:tasks.workunit.client.0.smithi114.stdout:3/472: creat d4/dd/d33/d8f/fa4 x:0 0 0 2024-02-16T22:22:01.886 INFO:tasks.workunit.client.0.smithi114.stdout:8/465: dread d4/d24/f3d [4194304,4194304] 0 2024-02-16T22:22:01.887 INFO:tasks.workunit.client.1.smithi123.stdout:0/548: link d1/da0/lc2 d1/d8/d13/lc3 0 2024-02-16T22:22:01.898 INFO:tasks.workunit.client.0.smithi114.stdout:4/568: unlink dd/d1b/d37/d99/l9c 0 2024-02-16T22:22:01.899 INFO:tasks.workunit.client.0.smithi114.stdout:9/487: sync 2024-02-16T22:22:01.899 INFO:tasks.workunit.client.0.smithi114.stdout:4/569: chown dd/d1b/c1f 764760600 1 2024-02-16T22:22:01.899 INFO:tasks.workunit.client.0.smithi114.stdout:1/433: sync 2024-02-16T22:22:01.899 INFO:tasks.workunit.client.0.smithi114.stdout:5/448: sync 2024-02-16T22:22:01.899 INFO:tasks.workunit.client.0.smithi114.stdout:0/411: sync 2024-02-16T22:22:01.900 INFO:tasks.workunit.client.1.smithi123.stdout:8/668: dwrite d0/d29/da7/d26/fa4 [0,4194304] 0 2024-02-16T22:22:01.905 INFO:tasks.workunit.client.1.smithi123.stdout:5/583: rename d6/dd/d44/f61 to d6/dd/df/d9f/fc6 0 2024-02-16T22:22:01.906 INFO:tasks.workunit.client.1.smithi123.stdout:5/584: readlink d6/l15 0 2024-02-16T22:22:01.908 INFO:tasks.workunit.client.0.smithi114.stdout:7/520: dwrite d3/d14/f29 [0,4194304] 0 2024-02-16T22:22:01.910 INFO:tasks.workunit.client.0.smithi114.stdout:6/503: getdents d6/d2c 0 2024-02-16T22:22:01.912 INFO:tasks.workunit.client.1.smithi123.stdout:6/623: mknod d0/d6/d2f/da5/dc6/ce4 0 2024-02-16T22:22:01.913 INFO:tasks.workunit.client.0.smithi114.stdout:6/504: write d6/dd/d25/d34/d61/d62/f8d [600030,125249] 0 2024-02-16T22:22:01.914 INFO:tasks.workunit.client.0.smithi114.stdout:8/466: creat d4/d11/d52/fa9 x:0 0 0 2024-02-16T22:22:01.914 INFO:tasks.workunit.client.0.smithi114.stdout:1/434: rmdir d4/d63/d69 39 2024-02-16T22:22:01.914 INFO:tasks.workunit.client.1.smithi123.stdout:4/666: mknod d5/dba/cd0 0 2024-02-16T22:22:01.914 INFO:tasks.workunit.client.1.smithi123.stdout:0/549: dread - d1/d8/d13/d1d/d62/d7a/f7f zero size 2024-02-16T22:22:01.915 INFO:tasks.workunit.client.0.smithi114.stdout:2/469: symlink dd/d3c/l9b 0 2024-02-16T22:22:01.915 INFO:tasks.workunit.client.0.smithi114.stdout:8/467: write d4/d11/d78/f86 [946981,129106] 0 2024-02-16T22:22:01.917 INFO:tasks.workunit.client.0.smithi114.stdout:3/473: creat d4/d46/d48/d80/fa5 x:0 0 0 2024-02-16T22:22:01.918 INFO:tasks.workunit.client.1.smithi123.stdout:0/550: write d1/f6a [931013,82748] 0 2024-02-16T22:22:01.919 INFO:tasks.workunit.client.0.smithi114.stdout:0/412: readlink d1/d16/d1f/d28/d51/d61/l75 0 2024-02-16T22:22:01.919 INFO:tasks.workunit.client.1.smithi123.stdout:1/592: dwrite d3/d6/db/d10/d2a/fc1 [0,4194304] 0 2024-02-16T22:22:01.922 INFO:tasks.workunit.client.0.smithi114.stdout:7/521: unlink d3/l1d 0 2024-02-16T22:22:01.927 INFO:tasks.workunit.client.0.smithi114.stdout:1/435: mknod d4/d9/d65/c99 0 2024-02-16T22:22:01.929 INFO:tasks.workunit.client.0.smithi114.stdout:2/470: creat dd/d4e/f9c x:0 0 0 2024-02-16T22:22:01.931 INFO:tasks.workunit.client.1.smithi123.stdout:4/667: read d5/d1c/d23/d47/f50 [1091604,120693] 0 2024-02-16T22:22:01.931 INFO:tasks.workunit.client.1.smithi123.stdout:4/668: chown d5/f1e 3 1 2024-02-16T22:22:01.932 INFO:tasks.workunit.client.1.smithi123.stdout:4/669: dread - d5/d1c/d23/d74/d92/d9b/fc2 zero size 2024-02-16T22:22:01.933 INFO:tasks.workunit.client.1.smithi123.stdout:9/729: creat d4/d36/d8e/d58/fe6 x:0 0 0 2024-02-16T22:22:01.935 INFO:tasks.workunit.client.0.smithi114.stdout:3/474: dread d4/dd/d33/f84 [0,4194304] 0 2024-02-16T22:22:01.937 INFO:tasks.workunit.client.1.smithi123.stdout:1/593: fsync d3/f7c 0 2024-02-16T22:22:01.940 INFO:tasks.workunit.client.1.smithi123.stdout:4/670: dwrite d5/d1c/d23/f3a [0,4194304] 0 2024-02-16T22:22:01.940 INFO:tasks.workunit.client.1.smithi123.stdout:3/627: write d0/fe [689708,119806] 0 2024-02-16T22:22:01.940 INFO:tasks.workunit.client.0.smithi114.stdout:4/570: dread dd/d1b/d37/d38/d7c/fc4 [0,4194304] 0 2024-02-16T22:22:01.942 INFO:tasks.workunit.client.1.smithi123.stdout:2/590: rename d4/d1a/d1d/d59/c7d to d4/d1a/d1d/d13/d15/d6d/d4a/cb8 0 2024-02-16T22:22:01.942 INFO:tasks.workunit.client.1.smithi123.stdout:7/629: dwrite d3/d6e/d24/f39 [0,4194304] 0 2024-02-16T22:22:01.943 INFO:tasks.workunit.client.1.smithi123.stdout:2/591: chown d4/d1a/l99 25191611 1 2024-02-16T22:22:01.950 INFO:tasks.workunit.client.0.smithi114.stdout:5/449: getdents d0 0 2024-02-16T22:22:01.952 INFO:tasks.workunit.client.0.smithi114.stdout:2/471: creat dd/df/d1c/d21/d87/f9d x:0 0 0 2024-02-16T22:22:01.952 INFO:tasks.workunit.client.0.smithi114.stdout:5/450: write d0/df/d16/d1e/d86/f7c [822315,62828] 0 2024-02-16T22:22:01.955 INFO:tasks.workunit.client.1.smithi123.stdout:5/585: truncate d6/dd/d5f/f60 1858367 0 2024-02-16T22:22:01.956 INFO:tasks.workunit.client.1.smithi123.stdout:5/586: stat d6/dd/df/d1f/l99 0 2024-02-16T22:22:01.956 INFO:tasks.workunit.client.1.smithi123.stdout:3/628: rmdir d0/d50/d7e 39 2024-02-16T22:22:01.957 INFO:tasks.workunit.client.0.smithi114.stdout:0/413: mkdir d1/d16/d5c/d80 0 2024-02-16T22:22:01.959 INFO:tasks.workunit.client.0.smithi114.stdout:3/475: unlink d4/d24/d2d/d45/d74/f8b 0 2024-02-16T22:22:01.960 INFO:tasks.workunit.client.1.smithi123.stdout:4/671: creat d5/d1c/d23/d56/d6e/d9f/da1/dc0/dc5/fd1 x:0 0 0 2024-02-16T22:22:01.962 INFO:tasks.workunit.client.0.smithi114.stdout:6/505: rmdir d6/d2c/d38/d54/d8a 0 2024-02-16T22:22:01.964 INFO:tasks.workunit.client.1.smithi123.stdout:7/630: unlink d3/d6e/d1d/db2/d3e/f63 0 2024-02-16T22:22:01.965 INFO:tasks.workunit.client.0.smithi114.stdout:0/414: dwrite d1/d16/d48/f6a [0,4194304] 0 2024-02-16T22:22:01.967 INFO:tasks.workunit.client.0.smithi114.stdout:4/571: mknod dd/d1b/d37/d38/d87/dad/cd2 0 2024-02-16T22:22:01.968 INFO:tasks.workunit.client.1.smithi123.stdout:2/592: truncate d4/d1a/d1d/d13/d15/f56 408867 0 2024-02-16T22:22:01.970 INFO:tasks.workunit.client.0.smithi114.stdout:9/488: getdents d2 0 2024-02-16T22:22:01.973 INFO:tasks.workunit.client.0.smithi114.stdout:1/436: symlink d4/d63/l9a 0 2024-02-16T22:22:01.973 INFO:tasks.workunit.client.0.smithi114.stdout:4/572: dread dd/d1b/d24/d98/fc5 [0,4194304] 0 2024-02-16T22:22:01.974 INFO:tasks.workunit.client.1.smithi123.stdout:8/669: getdents d0/d29/da7/d43/d34/dae 0 2024-02-16T22:22:01.975 INFO:tasks.workunit.client.1.smithi123.stdout:2/593: dwrite d4/d1a/d1d/d59/d53/fa8 [0,4194304] 0 2024-02-16T22:22:01.976 INFO:tasks.workunit.client.0.smithi114.stdout:2/472: unlink dd/df/f14 0 2024-02-16T22:22:01.977 INFO:tasks.workunit.client.1.smithi123.stdout:5/587: rmdir d6/dd/df/d76 39 2024-02-16T22:22:01.980 INFO:tasks.workunit.client.0.smithi114.stdout:8/468: rmdir d4/d16/d25/d39/d7b 0 2024-02-16T22:22:01.982 INFO:tasks.workunit.client.1.smithi123.stdout:2/594: dwrite d4/d1a/d1d/d13/d25/f77 [0,4194304] 0 2024-02-16T22:22:01.984 INFO:tasks.workunit.client.0.smithi114.stdout:3/476: mknod d4/d24/d2d/d45/d68/d77/ca6 0 2024-02-16T22:22:01.987 INFO:tasks.workunit.client.1.smithi123.stdout:9/730: creat d4/d1e/fe7 x:0 0 0 2024-02-16T22:22:01.988 INFO:tasks.workunit.client.0.smithi114.stdout:6/506: symlink d6/dd/d25/d34/d61/l97 0 2024-02-16T22:22:01.990 INFO:tasks.workunit.client.1.smithi123.stdout:8/670: creat d0/d29/da7/d26/d68/d81/d47/fdb x:0 0 0 2024-02-16T22:22:01.991 INFO:tasks.workunit.client.1.smithi123.stdout:8/671: stat d0/f6 0 2024-02-16T22:22:01.993 INFO:tasks.workunit.client.0.smithi114.stdout:9/489: creat d2/d1a/d23/d32/d46/f9d x:0 0 0 2024-02-16T22:22:01.994 INFO:tasks.workunit.client.0.smithi114.stdout:9/490: chown d2/d1a/d23/d35/d1d/f36 138856762 1 2024-02-16T22:22:02.000 INFO:tasks.workunit.client.0.smithi114.stdout:2/473: unlink dd/df/d1c/d21/d87/f9d 0 2024-02-16T22:22:02.001 INFO:tasks.workunit.client.1.smithi123.stdout:4/672: write d5/d1c/d23/d47/f50 [3811206,59554] 0 2024-02-16T22:22:02.003 INFO:tasks.workunit.client.1.smithi123.stdout:0/551: rename d1/d4/d17/d74/c76 to d1/d8/d13/d1d/cc4 0 2024-02-16T22:22:02.005 INFO:tasks.workunit.client.1.smithi123.stdout:2/595: creat d4/d1a/d1d/d13/d25/d98/fb9 x:0 0 0 2024-02-16T22:22:02.006 INFO:tasks.workunit.client.1.smithi123.stdout:5/588: dread d6/dd/df/d58/d3f/dbe/f5d [0,4194304] 0 2024-02-16T22:22:02.007 INFO:tasks.workunit.client.0.smithi114.stdout:3/477: unlink d4/d46/d48/c73 0 2024-02-16T22:22:02.007 INFO:tasks.workunit.client.0.smithi114.stdout:2/474: dwrite dd/df/f59 [0,4194304] 0 2024-02-16T22:22:02.008 INFO:tasks.workunit.client.0.smithi114.stdout:0/415: write d1/d1a/f45 [2924344,104132] 0 2024-02-16T22:22:02.009 INFO:tasks.workunit.client.0.smithi114.stdout:6/507: stat d6/d2c/d38/l89 0 2024-02-16T22:22:02.010 INFO:tasks.workunit.client.1.smithi123.stdout:9/731: creat d4/d36/d8e/d6d/d71/fe8 x:0 0 0 2024-02-16T22:22:02.011 INFO:tasks.workunit.client.0.smithi114.stdout:2/475: read dd/df/d1c/d21/d87/f5a [3923558,39072] 0 2024-02-16T22:22:02.011 INFO:tasks.workunit.client.1.smithi123.stdout:9/732: dread - d4/d1e/d3a/fcf zero size 2024-02-16T22:22:02.012 INFO:tasks.workunit.client.1.smithi123.stdout:9/733: read - d4/d36/d8e/d98/fbf zero size 2024-02-16T22:22:02.013 INFO:tasks.workunit.client.0.smithi114.stdout:0/416: dread d1/d16/f34 [0,4194304] 0 2024-02-16T22:22:02.014 INFO:tasks.workunit.client.0.smithi114.stdout:4/573: write dd/d2e/f6a [996571,15983] 0 2024-02-16T22:22:02.015 INFO:tasks.workunit.client.0.smithi114.stdout:0/417: chown d1/d1a/f38 18 1 2024-02-16T22:22:02.016 INFO:tasks.workunit.client.0.smithi114.stdout:1/437: mkdir d4/d63/d69/d7c/d9b 0 2024-02-16T22:22:02.017 INFO:tasks.workunit.client.1.smithi123.stdout:4/673: mknod d5/d1c/d23/d47/da2/daa/cd2 0 2024-02-16T22:22:02.019 INFO:tasks.workunit.client.1.smithi123.stdout:1/594: rename d3/d6/d49/d48/d84/d92/ca1 to d3/d6/d49/d48/d68/cc8 0 2024-02-16T22:22:02.020 INFO:tasks.workunit.client.1.smithi123.stdout:0/552: fsync d1/d4/d17/d74/d57/f73 0 2024-02-16T22:22:02.021 INFO:tasks.workunit.client.1.smithi123.stdout:2/596: symlink d4/d1a/d1d/d13/d15/d6d/d4a/lba 0 2024-02-16T22:22:02.021 INFO:tasks.workunit.client.1.smithi123.stdout:4/674: dread d5/f1e [0,4194304] 0 2024-02-16T22:22:02.025 INFO:tasks.workunit.client.1.smithi123.stdout:1/595: dread d3/d6/d49/d48/d84/d92/fbe [0,4194304] 0 2024-02-16T22:22:02.026 INFO:tasks.workunit.client.1.smithi123.stdout:9/734: creat d4/d36/d8e/d6f/d7e/fe9 x:0 0 0 2024-02-16T22:22:02.026 INFO:tasks.workunit.client.1.smithi123.stdout:1/596: fsync d3/d6/d49/d48/d84/f93 0 2024-02-16T22:22:02.028 INFO:tasks.workunit.client.0.smithi114.stdout:6/508: dwrite d6/d83/f87 [0,4194304] 0 2024-02-16T22:22:02.029 INFO:tasks.workunit.client.0.smithi114.stdout:4/574: mknod dd/d1b/d24/d57/cd3 0 2024-02-16T22:22:02.030 INFO:tasks.workunit.client.1.smithi123.stdout:9/735: dwrite d4/d33/f73 [0,4194304] 0 2024-02-16T22:22:02.030 INFO:tasks.workunit.client.1.smithi123.stdout:6/624: sync 2024-02-16T22:22:02.033 INFO:tasks.workunit.client.0.smithi114.stdout:0/418: stat d1/c12 0 2024-02-16T22:22:02.036 INFO:tasks.workunit.client.0.smithi114.stdout:1/438: truncate d4/d9/d46/d4d/f4c 5157769 0 2024-02-16T22:22:02.037 INFO:tasks.workunit.client.0.smithi114.stdout:4/575: dwrite dd/d1b/d37/d38/faf [0,4194304] 0 2024-02-16T22:22:02.040 INFO:tasks.workunit.client.0.smithi114.stdout:8/469: creat d4/faa x:0 0 0 2024-02-16T22:22:02.041 INFO:tasks.workunit.client.0.smithi114.stdout:1/439: write d4/d9/d46/d47/f94 [352583,30516] 0 2024-02-16T22:22:02.041 INFO:tasks.workunit.client.1.smithi123.stdout:4/675: rmdir d5/d1c/d23/d69 39 2024-02-16T22:22:02.046 INFO:tasks.workunit.client.1.smithi123.stdout:5/589: write d6/f8f [1366249,46065] 0 2024-02-16T22:22:02.049 INFO:tasks.workunit.client.0.smithi114.stdout:6/509: creat d6/d2c/d38/d41/d60/f98 x:0 0 0 2024-02-16T22:22:02.050 INFO:tasks.workunit.client.0.smithi114.stdout:6/510: read d6/d2c/d38/d54/f77 [2507990,16352] 0 2024-02-16T22:22:02.050 INFO:tasks.workunit.client.1.smithi123.stdout:3/629: getdents d0/d50/d7e 0 2024-02-16T22:22:02.055 INFO:tasks.workunit.client.1.smithi123.stdout:9/736: mkdir d4/d36/daa/d7f/dea 0 2024-02-16T22:22:02.056 INFO:tasks.workunit.client.0.smithi114.stdout:0/419: dread d1/d1a/f38 [0,4194304] 0 2024-02-16T22:22:02.057 INFO:tasks.workunit.client.0.smithi114.stdout:1/440: mknod d4/d33/d74/c9c 0 2024-02-16T22:22:02.057 INFO:tasks.workunit.client.1.smithi123.stdout:0/553: mkdir d1/d4/d17/d54/db4/dc5 0 2024-02-16T22:22:02.059 INFO:tasks.workunit.client.1.smithi123.stdout:0/554: write d1/d4/d17/d54/d8e/da5/fb0 [304945,82662] 0 2024-02-16T22:22:02.061 INFO:tasks.workunit.client.1.smithi123.stdout:5/590: fsync d6/dd/df/f1b 0 2024-02-16T22:22:02.062 INFO:tasks.workunit.client.0.smithi114.stdout:8/470: dwrite d4/d11/d78/d7a/f3a [0,4194304] 0 2024-02-16T22:22:02.064 INFO:tasks.workunit.client.0.smithi114.stdout:9/491: getdents d2/dd/d3c/d89 0 2024-02-16T22:22:02.064 INFO:tasks.workunit.client.1.smithi123.stdout:8/672: getdents d0/d29/da7/d26/d32/d45 0 2024-02-16T22:22:02.066 INFO:tasks.workunit.client.0.smithi114.stdout:6/511: dread - d6/d2c/d38/d54/f5f zero size 2024-02-16T22:22:02.070 INFO:tasks.workunit.client.0.smithi114.stdout:1/441: dwrite d4/d9/d46/d4d/f5c [0,4194304] 0 2024-02-16T22:22:02.070 INFO:tasks.workunit.client.1.smithi123.stdout:8/673: dwrite d0/d29/da7/d26/d2a/d4d/fc6 [0,4194304] 0 2024-02-16T22:22:02.071 INFO:tasks.workunit.client.1.smithi123.stdout:8/674: chown d0/d29/da7/d26/d2a/fb2 15 1 2024-02-16T22:22:02.076 INFO:tasks.workunit.client.1.smithi123.stdout:3/630: read d0/da0/faa [2453,3878] 0 2024-02-16T22:22:02.078 INFO:tasks.workunit.client.1.smithi123.stdout:9/737: rename d4/d36/d8e/d58/d6e to d4/d36/d8e/d58/d59/d60/deb 0 2024-02-16T22:22:02.079 INFO:tasks.workunit.client.0.smithi114.stdout:3/478: link d4/d24/d2d/d45/d68/d77/c85 d4/d6a/ca7 0 2024-02-16T22:22:02.080 INFO:tasks.workunit.client.1.smithi123.stdout:0/555: truncate d1/d8/d13/d1d/d64/dbc/f5c 1169874 0 2024-02-16T22:22:02.081 INFO:tasks.workunit.client.0.smithi114.stdout:3/479: truncate d4/d43/f8a 254480 0 2024-02-16T22:22:02.081 INFO:tasks.workunit.client.0.smithi114.stdout:4/576: dread dd/d1b/d24/f3d [0,4194304] 0 2024-02-16T22:22:02.082 INFO:tasks.workunit.client.0.smithi114.stdout:2/476: getdents dd/d3b 0 2024-02-16T22:22:02.082 INFO:tasks.workunit.client.1.smithi123.stdout:2/597: dwrite d4/d1a/d1d/d13/d25/d98/f76 [0,4194304] 0 2024-02-16T22:22:02.083 INFO:tasks.workunit.client.1.smithi123.stdout:9/738: write d4/d36/d8e/f61 [563755,52868] 0 2024-02-16T22:22:02.084 INFO:tasks.workunit.client.0.smithi114.stdout:9/492: mkdir d2/dd/d3c/d89/d9e 0 2024-02-16T22:22:02.085 INFO:tasks.workunit.client.1.smithi123.stdout:8/675: dwrite d0/d29/da7/d26/d2a/d4d/fc6 [0,4194304] 0 2024-02-16T22:22:02.086 INFO:tasks.workunit.client.0.smithi114.stdout:8/471: mknod d4/d16/d25/d39/cab 0 2024-02-16T22:22:02.087 INFO:tasks.workunit.client.0.smithi114.stdout:6/512: rmdir d6/d2c 39 2024-02-16T22:22:02.087 INFO:tasks.workunit.client.1.smithi123.stdout:9/739: stat d4/d1e/d3a 0 2024-02-16T22:22:02.088 INFO:tasks.workunit.client.1.smithi123.stdout:5/591: rmdir d6/dd/df/d58/d3f/dbe/d37 39 2024-02-16T22:22:02.090 INFO:tasks.workunit.client.1.smithi123.stdout:1/597: creat d3/d6/db/d10/fc9 x:0 0 0 2024-02-16T22:22:02.094 INFO:tasks.workunit.client.1.smithi123.stdout:5/592: truncate d6/fb8 511100 0 2024-02-16T22:22:02.094 INFO:tasks.workunit.client.1.smithi123.stdout:3/631: creat d0/d20/d76/d5d/d60/dd6/fd9 x:0 0 0 2024-02-16T22:22:02.095 INFO:tasks.workunit.client.1.smithi123.stdout:0/556: mknod d1/d4/cc6 0 2024-02-16T22:22:02.096 INFO:tasks.workunit.client.1.smithi123.stdout:7/631: sync 2024-02-16T22:22:02.097 INFO:tasks.workunit.client.0.smithi114.stdout:9/493: symlink d2/d1a/d23/d35/d47/d63/d98/l9f 0 2024-02-16T22:22:02.098 INFO:tasks.workunit.client.1.smithi123.stdout:2/598: mkdir d4/d1a/d1d/d13/d25/d98/dbb 0 2024-02-16T22:22:02.099 INFO:tasks.workunit.client.1.smithi123.stdout:8/676: dwrite d0/d29/f64 [0,4194304] 0 2024-02-16T22:22:02.100 INFO:tasks.workunit.client.0.smithi114.stdout:1/442: creat d4/d9/d65/d98/f9d x:0 0 0 2024-02-16T22:22:02.100 INFO:tasks.workunit.client.1.smithi123.stdout:4/676: dwrite d5/d1c/d23/d56/d6e/d9f/da1/fa4 [0,4194304] 0 2024-02-16T22:22:02.101 INFO:tasks.workunit.client.1.smithi123.stdout:2/599: readlink d4/d1a/d1d/d13/d15/l3f 0 2024-02-16T22:22:02.102 INFO:tasks.workunit.client.1.smithi123.stdout:9/740: creat d4/d36/d8e/d58/d59/d60/deb/fec x:0 0 0 2024-02-16T22:22:02.103 INFO:tasks.workunit.client.0.smithi114.stdout:0/420: creat d1/f81 x:0 0 0 2024-02-16T22:22:02.104 INFO:tasks.workunit.client.1.smithi123.stdout:1/598: mknod d3/d6/d65/cca 0 2024-02-16T22:22:02.104 INFO:tasks.workunit.client.0.smithi114.stdout:3/480: mknod d4/d24/d93/ca8 0 2024-02-16T22:22:02.107 INFO:tasks.workunit.client.1.smithi123.stdout:0/557: unlink d1/d4/l21 0 2024-02-16T22:22:02.110 INFO:tasks.workunit.client.0.smithi114.stdout:4/577: link dd/f2b dd/d2e/d56/fd4 0 2024-02-16T22:22:02.110 INFO:tasks.workunit.client.1.smithi123.stdout:3/632: dwrite d0/d20/d76/d5d/dae/dc7/fd4 [0,4194304] 0 2024-02-16T22:22:02.114 INFO:tasks.workunit.client.1.smithi123.stdout:5/593: rename d6/d3b/f7a to d6/dd/df/d58/d3f/dbe/da2/fc7 0 2024-02-16T22:22:02.114 INFO:tasks.workunit.client.1.smithi123.stdout:2/600: dwrite d4/fc [0,4194304] 0 2024-02-16T22:22:02.114 INFO:tasks.workunit.client.0.smithi114.stdout:6/513: stat d6/d2c/d38/l69 0 2024-02-16T22:22:02.115 INFO:tasks.workunit.client.1.smithi123.stdout:8/677: rmdir d0/d29/da7 39 2024-02-16T22:22:02.116 INFO:tasks.workunit.client.1.smithi123.stdout:5/594: chown d6/dd/d5f/fb2 119948 1 2024-02-16T22:22:02.116 INFO:tasks.workunit.client.1.smithi123.stdout:8/678: fsync d0/df/f96 0 2024-02-16T22:22:02.123 INFO:tasks.workunit.client.0.smithi114.stdout:3/481: creat d4/dd/d33/fa9 x:0 0 0 2024-02-16T22:22:02.124 INFO:tasks.workunit.client.0.smithi114.stdout:3/482: write d4/d24/d2d/d45/d68/f86 [4929696,34979] 0 2024-02-16T22:22:02.126 INFO:tasks.workunit.client.0.smithi114.stdout:3/483: write d4/d24/f62 [1628340,50620] 0 2024-02-16T22:22:02.127 INFO:tasks.workunit.client.0.smithi114.stdout:2/477: creat dd/df/f9e x:0 0 0 2024-02-16T22:22:02.127 INFO:tasks.workunit.client.0.smithi114.stdout:3/484: fdatasync d4/dd/d33/fa9 0 2024-02-16T22:22:02.130 INFO:tasks.workunit.client.1.smithi123.stdout:1/599: mkdir d3/d6/d18/dcb 0 2024-02-16T22:22:02.132 INFO:tasks.workunit.client.1.smithi123.stdout:6/625: sync 2024-02-16T22:22:02.133 INFO:tasks.workunit.client.1.smithi123.stdout:6/626: readlink d0/d6/l37 0 2024-02-16T22:22:02.136 INFO:tasks.workunit.client.1.smithi123.stdout:4/677: rename d5/d1c/d5e/dc3/f6a to d5/d1c/d23/d31/fd3 0 2024-02-16T22:22:02.138 INFO:tasks.workunit.client.1.smithi123.stdout:4/678: readlink d5/d1c/d23/d74/lb3 0 2024-02-16T22:22:02.140 INFO:tasks.workunit.client.1.smithi123.stdout:4/679: readlink d5/d1c/d23/d47/l85 0 2024-02-16T22:22:02.140 INFO:tasks.workunit.client.1.smithi123.stdout:9/741: write d4/d36/daa/d20/f6a [639096,7880] 0 2024-02-16T22:22:02.141 INFO:tasks.workunit.client.1.smithi123.stdout:8/679: write d0/d29/da7/d26/d2a/d4d/fc6 [3517694,112993] 0 2024-02-16T22:22:02.141 INFO:tasks.workunit.client.1.smithi123.stdout:6/627: dwrite d0/d6/d91/d97/fe2 [0,4194304] 0 2024-02-16T22:22:02.142 INFO:tasks.workunit.client.0.smithi114.stdout:3/485: mkdir d4/d46/d48/d75/daa 0 2024-02-16T22:22:02.146 INFO:tasks.workunit.client.1.smithi123.stdout:2/601: mknod d4/d1a/d1d/d13/d25/d72/cbc 0 2024-02-16T22:22:02.152 INFO:tasks.workunit.client.1.smithi123.stdout:7/632: rename d3/d5/l19 to d3/d59/ld0 0 2024-02-16T22:22:02.155 INFO:tasks.workunit.client.0.smithi114.stdout:3/486: creat d4/d24/d2d/d45/d68/d77/fab x:0 0 0 2024-02-16T22:22:02.156 INFO:tasks.workunit.client.0.smithi114.stdout:3/487: chown d4/d46 787 1 2024-02-16T22:22:02.156 INFO:tasks.workunit.client.1.smithi123.stdout:4/680: mknod d5/d1c/d64/cd4 0 2024-02-16T22:22:02.158 INFO:tasks.workunit.client.1.smithi123.stdout:4/681: write d5/d1c/d23/d74/d92/d9b/fc2 [915716,98380] 0 2024-02-16T22:22:02.158 INFO:tasks.workunit.client.0.smithi114.stdout:1/443: getdents d4/d9/d46/d67 0 2024-02-16T22:22:02.160 INFO:tasks.workunit.client.0.smithi114.stdout:0/421: dread d1/d1a/f26 [0,4194304] 0 2024-02-16T22:22:02.160 INFO:tasks.workunit.client.1.smithi123.stdout:1/600: dread d3/d6/d18/f7e [0,4194304] 0 2024-02-16T22:22:02.161 INFO:tasks.workunit.client.1.smithi123.stdout:0/558: rmdir d1/d4/d17/d74/d57/da3 0 2024-02-16T22:22:02.161 INFO:tasks.workunit.client.0.smithi114.stdout:3/488: fdatasync d4/dd/d33/f84 0 2024-02-16T22:22:02.161 INFO:tasks.workunit.client.1.smithi123.stdout:0/559: read d1/d8/d13/f5a [2622241,129222] 0 2024-02-16T22:22:02.162 INFO:tasks.workunit.client.1.smithi123.stdout:1/601: write d3/d6/db/d10/f85 [2146806,123707] 0 2024-02-16T22:22:02.163 INFO:tasks.workunit.client.1.smithi123.stdout:0/560: write d1/d4/d17/d74/fb8 [185988,19213] 0 2024-02-16T22:22:02.164 INFO:tasks.workunit.client.0.smithi114.stdout:1/444: read d4/f3c [115328,110396] 0 2024-02-16T22:22:02.168 INFO:tasks.workunit.client.0.smithi114.stdout:3/489: dwrite d4/d46/d48/f66 [0,4194304] 0 2024-02-16T22:22:02.170 INFO:tasks.workunit.client.1.smithi123.stdout:3/633: rename d0/d20/d76/d37/c8e to d0/d20/d76/d5d/d60/cda 0 2024-02-16T22:22:02.172 INFO:tasks.workunit.client.1.smithi123.stdout:7/633: creat d3/d6e/d4d/d92/fd1 x:0 0 0 2024-02-16T22:22:02.174 INFO:tasks.workunit.client.0.smithi114.stdout:1/445: rename d4/d33/f57 to d4/d9/d46/d47/f9e 0 2024-02-16T22:22:02.175 INFO:tasks.workunit.client.0.smithi114.stdout:3/490: dwrite d4/d26/f39 [0,4194304] 0 2024-02-16T22:22:02.177 INFO:tasks.workunit.client.0.smithi114.stdout:0/422: fdatasync d1/fa 0 2024-02-16T22:22:02.178 INFO:tasks.workunit.client.0.smithi114.stdout:9/494: truncate d2/d1a/f21 1694830 0 2024-02-16T22:22:02.180 INFO:tasks.workunit.client.0.smithi114.stdout:9/495: write d2/dd/d3c/f7f [3018190,129792] 0 2024-02-16T22:22:02.181 INFO:tasks.workunit.client.1.smithi123.stdout:4/682: creat d5/d1c/d5e/dc3/fd5 x:0 0 0 2024-02-16T22:22:02.182 INFO:tasks.workunit.client.1.smithi123.stdout:4/683: stat d5/d1c/d23/d31/f3e 0 2024-02-16T22:22:02.185 INFO:tasks.workunit.client.0.smithi114.stdout:1/446: rmdir d4/d33/d74 39 2024-02-16T22:22:02.188 INFO:tasks.workunit.client.0.smithi114.stdout:3/491: creat d4/d24/d2d/fac x:0 0 0 2024-02-16T22:22:02.190 INFO:tasks.workunit.client.0.smithi114.stdout:3/492: read - d4/dd/d33/d72/f94 zero size 2024-02-16T22:22:02.191 INFO:tasks.workunit.client.0.smithi114.stdout:4/578: write dd/d1b/d24/f76 [293845,3870] 0 2024-02-16T22:22:02.191 INFO:tasks.workunit.client.1.smithi123.stdout:1/602: write d3/f35 [874827,62528] 0 2024-02-16T22:22:02.192 INFO:tasks.workunit.client.0.smithi114.stdout:3/493: chown d4/d43/c78 35115 1 2024-02-16T22:22:02.192 INFO:tasks.workunit.client.0.smithi114.stdout:2/478: dwrite dd/df/f41 [0,4194304] 0 2024-02-16T22:22:02.195 INFO:tasks.workunit.client.0.smithi114.stdout:6/514: truncate d6/dd/f29 885522 0 2024-02-16T22:22:02.195 INFO:tasks.workunit.client.0.smithi114.stdout:2/479: write dd/f70 [3132528,44968] 0 2024-02-16T22:22:02.198 INFO:tasks.workunit.client.1.smithi123.stdout:0/561: unlink d1/d4/d17/d54/d8e/da5/fb0 0 2024-02-16T22:22:02.201 INFO:tasks.workunit.client.1.smithi123.stdout:2/602: unlink d4/d1a/d1d/d59/d45/c3e 0 2024-02-16T22:22:02.202 INFO:tasks.workunit.client.1.smithi123.stdout:2/603: write d4/d1a/d1d/d59/d53/fa8 [4587389,29187] 0 2024-02-16T22:22:02.203 INFO:tasks.workunit.client.0.smithi114.stdout:9/496: write d2/d5c/d5f/d72/f79 [1790274,113384] 0 2024-02-16T22:22:02.207 INFO:tasks.workunit.client.1.smithi123.stdout:6/628: sync 2024-02-16T22:22:02.215 INFO:tasks.workunit.client.1.smithi123.stdout:9/742: rmdir d4/d36/d8e/d84/d93/d96 0 2024-02-16T22:22:02.216 INFO:tasks.workunit.client.0.smithi114.stdout:3/494: dread d4/d43/d53/f96 [0,4194304] 0 2024-02-16T22:22:02.220 INFO:tasks.workunit.client.0.smithi114.stdout:6/515: chown d6/d2c/d38/l7a 32700725 1 2024-02-16T22:22:02.226 INFO:tasks.workunit.client.1.smithi123.stdout:8/680: truncate d0/d29/da7/d43/fc2 68486 0 2024-02-16T22:22:02.226 INFO:tasks.workunit.client.1.smithi123.stdout:2/604: dread d4/d1a/d1d/d13/d25/f26 [0,4194304] 0 2024-02-16T22:22:02.226 INFO:tasks.workunit.client.0.smithi114.stdout:4/579: dread dd/d1b/d24/f4e [0,4194304] 0 2024-02-16T22:22:02.227 INFO:tasks.workunit.client.1.smithi123.stdout:8/681: write d0/d29/da7/fb3 [515637,79824] 0 2024-02-16T22:22:02.227 INFO:tasks.workunit.client.0.smithi114.stdout:0/423: creat d1/d16/f82 x:0 0 0 2024-02-16T22:22:02.228 INFO:tasks.workunit.client.1.smithi123.stdout:8/682: write d0/d29/f64 [1683380,67988] 0 2024-02-16T22:22:02.228 INFO:tasks.workunit.client.1.smithi123.stdout:0/562: unlink d1/d8/f5d 0 2024-02-16T22:22:02.229 INFO:tasks.workunit.client.1.smithi123.stdout:0/563: chown d1/d4/cc6 7 1 2024-02-16T22:22:02.233 INFO:tasks.workunit.client.0.smithi114.stdout:4/580: dread dd/d1b/d37/fba [0,4194304] 0 2024-02-16T22:22:02.234 INFO:tasks.workunit.client.0.smithi114.stdout:4/581: fdatasync dd/d2e/d56/fb3 0 2024-02-16T22:22:02.234 INFO:tasks.workunit.client.0.smithi114.stdout:1/447: write d4/f37 [394617,54338] 0 2024-02-16T22:22:02.237 INFO:tasks.workunit.client.1.smithi123.stdout:5/595: rename d6/dd/df/d1f/f2e to d6/dd/df/d58/d89/d98/fc8 0 2024-02-16T22:22:02.237 INFO:tasks.workunit.client.1.smithi123.stdout:7/634: write d3/d6e/d24/f49 [41285,78438] 0 2024-02-16T22:22:02.238 INFO:tasks.workunit.client.1.smithi123.stdout:5/596: fsync d6/d3b/f65 0 2024-02-16T22:22:02.238 INFO:tasks.workunit.client.1.smithi123.stdout:7/635: dread - d3/d6e/d1d/d97/fc4 zero size 2024-02-16T22:22:02.241 INFO:tasks.workunit.client.1.smithi123.stdout:6/629: mkdir d0/d9/dc/d3a/d70/de5 0 2024-02-16T22:22:02.242 INFO:tasks.workunit.client.1.smithi123.stdout:6/630: chown d0/d9/dc/d3a/d70/dd5 3 1 2024-02-16T22:22:02.242 INFO:tasks.workunit.client.1.smithi123.stdout:8/683: dread d0/d29/da7/d26/d68/d81/d47/f6a [0,4194304] 0 2024-02-16T22:22:02.242 INFO:tasks.workunit.client.1.smithi123.stdout:1/603: write d3/d6/d11/f1f [3165588,110858] 0 2024-02-16T22:22:02.243 INFO:tasks.workunit.client.1.smithi123.stdout:8/684: chown d0/d29/da7/d26/d32/d45/d67/c71 706193955 1 2024-02-16T22:22:02.244 INFO:tasks.workunit.client.1.smithi123.stdout:9/743: creat d4/d36/d57/db2/fed x:0 0 0 2024-02-16T22:22:02.246 INFO:tasks.workunit.client.0.smithi114.stdout:9/497: dwrite d2/f15 [0,4194304] 0 2024-02-16T22:22:02.247 INFO:tasks.workunit.client.1.smithi123.stdout:9/744: chown d4/d36/d8e/d58/fa2 902 1 2024-02-16T22:22:02.248 INFO:tasks.workunit.client.0.smithi114.stdout:4/582: creat dd/d1b/d41/d74/fd5 x:0 0 0 2024-02-16T22:22:02.249 INFO:tasks.workunit.client.0.smithi114.stdout:4/583: readlink dd/d2e/d3f/l9f 0 2024-02-16T22:22:02.249 INFO:tasks.workunit.client.1.smithi123.stdout:6/631: dwrite d0/d74/f94 [0,4194304] 0 2024-02-16T22:22:02.250 INFO:tasks.workunit.client.0.smithi114.stdout:4/584: stat dd/d1b/d41 0 2024-02-16T22:22:02.250 INFO:tasks.workunit.client.0.smithi114.stdout:4/585: read - dd/d2e/d3f/fa6 zero size 2024-02-16T22:22:02.251 INFO:tasks.workunit.client.0.smithi114.stdout:1/448: symlink d4/d9/d46/d47/l9f 0 2024-02-16T22:22:02.254 INFO:tasks.workunit.client.1.smithi123.stdout:2/605: chown d4/d1a/d1d/d13/d15/d6d/c71 20 1 2024-02-16T22:22:02.254 INFO:tasks.workunit.client.0.smithi114.stdout:0/424: mknod d1/d16/d1f/d28/d51/d61/d6d/d79/c83 0 2024-02-16T22:22:02.254 INFO:tasks.workunit.client.0.smithi114.stdout:0/425: fsync d1/d1a/f4e 0 2024-02-16T22:22:02.260 INFO:tasks.workunit.client.0.smithi114.stdout:6/516: rename d6/d2c/d75/f85 to d6/dd/d25/f99 0 2024-02-16T22:22:02.261 INFO:tasks.workunit.client.0.smithi114.stdout:6/517: fdatasync d6/d2c/f7c 0 2024-02-16T22:22:02.262 INFO:tasks.workunit.client.1.smithi123.stdout:0/564: symlink d1/d4/d17/d54/db4/lc7 0 2024-02-16T22:22:02.265 INFO:tasks.workunit.client.1.smithi123.stdout:7/636: mknod d3/d87/cd2 0 2024-02-16T22:22:02.266 INFO:tasks.workunit.client.1.smithi123.stdout:3/634: creat d0/d20/d29/fdb x:0 0 0 2024-02-16T22:22:02.267 INFO:tasks.workunit.client.1.smithi123.stdout:5/597: creat d6/dd/df/d58/d3f/d6f/fc9 x:0 0 0 2024-02-16T22:22:02.268 INFO:tasks.workunit.client.0.smithi114.stdout:0/426: chown d1/f8 302035 1 2024-02-16T22:22:02.269 INFO:tasks.workunit.client.1.smithi123.stdout:0/565: dwrite d1/d8/d13/d1d/f8d [0,4194304] 0 2024-02-16T22:22:02.269 INFO:tasks.workunit.client.1.smithi123.stdout:5/598: write d6/dd/d44/f47 [1322151,56102] 0 2024-02-16T22:22:02.270 INFO:tasks.workunit.client.0.smithi114.stdout:0/427: stat d1/d16/d18/l27 0 2024-02-16T22:22:02.270 INFO:tasks.workunit.client.0.smithi114.stdout:3/495: getdents d4/d26 0 2024-02-16T22:22:02.270 INFO:tasks.workunit.client.1.smithi123.stdout:8/685: mknod d0/d29/da7/d26/d2a/d3c/d62/cdc 0 2024-02-16T22:22:02.271 INFO:tasks.workunit.client.0.smithi114.stdout:3/496: readlink d4/l1d 0 2024-02-16T22:22:02.272 INFO:tasks.workunit.client.0.smithi114.stdout:0/428: write d1/d16/d1f/d28/f54 [2250277,122467] 0 2024-02-16T22:22:02.277 INFO:tasks.workunit.client.0.smithi114.stdout:6/518: symlink d6/dd/d25/d34/d58/l9a 0 2024-02-16T22:22:02.277 INFO:tasks.workunit.client.0.smithi114.stdout:6/519: write d6/f18 [8562133,129411] 0 2024-02-16T22:22:02.278 INFO:tasks.workunit.client.1.smithi123.stdout:6/632: mkdir d0/d8a/de6 0 2024-02-16T22:22:02.279 INFO:tasks.workunit.client.0.smithi114.stdout:6/520: write d6/dd/d25/f73 [395710,71362] 0 2024-02-16T22:22:02.282 INFO:tasks.workunit.client.0.smithi114.stdout:1/449: symlink d4/d33/la0 0 2024-02-16T22:22:02.283 INFO:tasks.workunit.client.0.smithi114.stdout:6/521: read d6/dd/d25/d34/d58/f78 [1302430,784] 0 2024-02-16T22:22:02.283 INFO:tasks.workunit.client.0.smithi114.stdout:1/450: chown d4/d9/d65/c85 6 1 2024-02-16T22:22:02.284 INFO:tasks.workunit.client.0.smithi114.stdout:9/498: write d2/d1a/d23/f52 [614291,42669] 0 2024-02-16T22:22:02.286 INFO:tasks.workunit.client.0.smithi114.stdout:1/451: write d4/d9/d3d/d3f/f76 [1572416,36048] 0 2024-02-16T22:22:02.292 INFO:tasks.workunit.client.0.smithi114.stdout:1/452: dwrite d4/d9/d46/d4d/f7d [0,4194304] 0 2024-02-16T22:22:02.293 INFO:tasks.workunit.client.1.smithi123.stdout:9/745: write d4/d36/d57/db2/f49 [1953774,96154] 0 2024-02-16T22:22:02.293 INFO:tasks.workunit.client.1.smithi123.stdout:1/604: getdents d3/d6/db/d10/d4f/d9f/db4 0 2024-02-16T22:22:02.294 INFO:tasks.workunit.client.0.smithi114.stdout:1/453: fsync d4/d63/d69/d72/f97 0 2024-02-16T22:22:02.294 INFO:tasks.workunit.client.1.smithi123.stdout:9/746: write d4/d36/d8e/f61 [1726607,125380] 0 2024-02-16T22:22:02.295 INFO:tasks.workunit.client.0.smithi114.stdout:4/586: creat dd/d1b/d24/fd6 x:0 0 0 2024-02-16T22:22:02.295 INFO:tasks.workunit.client.1.smithi123.stdout:1/605: write d3/d6/db/d10/d2a/d80/f90 [2188743,19746] 0 2024-02-16T22:22:02.296 INFO:tasks.workunit.client.1.smithi123.stdout:9/747: write d4/d36/d57/db2/f69 [449197,59910] 0 2024-02-16T22:22:02.298 INFO:tasks.workunit.client.1.smithi123.stdout:9/748: chown d4/d36/d57/db2/d8c 1685660180 1 2024-02-16T22:22:02.298 INFO:tasks.workunit.client.1.smithi123.stdout:4/684: getdents d5/d1c/d23/d56/d6e/d9f/da1/dc0 0 2024-02-16T22:22:02.299 INFO:tasks.workunit.client.1.smithi123.stdout:4/685: readlink d5/l68 0 2024-02-16T22:22:02.300 INFO:tasks.workunit.client.1.smithi123.stdout:1/606: dread d3/d6/f78 [0,4194304] 0 2024-02-16T22:22:02.300 INFO:tasks.workunit.client.0.smithi114.stdout:6/522: rename d6/d2c/d66/f68 to d6/da/f9b 0 2024-02-16T22:22:02.301 INFO:tasks.workunit.client.1.smithi123.stdout:1/607: readlink d3/d6/d11/d76/l64 0 2024-02-16T22:22:02.302 INFO:tasks.workunit.client.1.smithi123.stdout:0/566: chown d1/d4/d42/fc0 1676759 1 2024-02-16T22:22:02.303 INFO:tasks.workunit.client.1.smithi123.stdout:1/608: write d3/d6/d65/fbd [360228,24562] 0 2024-02-16T22:22:02.305 INFO:tasks.workunit.client.1.smithi123.stdout:0/567: chown d1/d4/d17/d54/f80 20656 1 2024-02-16T22:22:02.305 INFO:tasks.workunit.client.1.smithi123.stdout:8/686: creat d0/d29/da7/d43/d34/fdd x:0 0 0 2024-02-16T22:22:02.306 INFO:tasks.workunit.client.0.smithi114.stdout:0/429: creat d1/d1a/d47/d62/d7f/f84 x:0 0 0 2024-02-16T22:22:02.307 INFO:tasks.workunit.client.1.smithi123.stdout:4/686: dwrite d5/d1c/d5e/dc3/f4a [0,4194304] 0 2024-02-16T22:22:02.307 INFO:tasks.workunit.client.0.smithi114.stdout:3/497: dwrite d4/d26/f42 [4194304,4194304] 0 2024-02-16T22:22:02.308 INFO:tasks.workunit.client.1.smithi123.stdout:8/687: chown d0/d29/da7/d43/d34/dae/fc1 42350 1 2024-02-16T22:22:02.308 INFO:tasks.workunit.client.1.smithi123.stdout:2/606: symlink d4/d1a/d1d/d59/lbd 0 2024-02-16T22:22:02.309 INFO:tasks.workunit.client.0.smithi114.stdout:3/498: stat d4/d26/f31 0 2024-02-16T22:22:02.309 INFO:tasks.workunit.client.0.smithi114.stdout:3/499: chown d4/dd/d33/d8f 4 1 2024-02-16T22:22:02.310 INFO:tasks.workunit.client.1.smithi123.stdout:2/607: read d4/f54 [3801524,22788] 0 2024-02-16T22:22:02.310 INFO:tasks.workunit.client.1.smithi123.stdout:7/637: symlink d3/d5/d6c/da6/ld3 0 2024-02-16T22:22:02.312 INFO:tasks.workunit.client.1.smithi123.stdout:2/608: read d4/fc [5742621,93806] 0 2024-02-16T22:22:02.313 INFO:tasks.workunit.client.1.smithi123.stdout:3/635: mknod d0/d50/cdc 0 2024-02-16T22:22:02.314 INFO:tasks.workunit.client.1.smithi123.stdout:2/609: chown d4/d1a/d1d/d13/d25/d98/fb9 424799 1 2024-02-16T22:22:02.318 INFO:tasks.workunit.client.1.smithi123.stdout:7/638: dwrite d3/d6e/d1d/db2/d3e/f5b [0,4194304] 0 2024-02-16T22:22:02.318 INFO:tasks.workunit.client.0.smithi114.stdout:9/499: write d2/d1a/f6b [430844,119719] 0 2024-02-16T22:22:02.320 INFO:tasks.workunit.client.0.smithi114.stdout:3/500: dwrite d4/d46/d48/f8e [0,4194304] 0 2024-02-16T22:22:02.320 INFO:tasks.workunit.client.0.smithi114.stdout:6/523: mknod d6/dd/c9c 0 2024-02-16T22:22:02.322 INFO:tasks.workunit.client.0.smithi114.stdout:9/500: fsync d2/d1a/d65/f8f 0 2024-02-16T22:22:02.322 INFO:tasks.workunit.client.0.smithi114.stdout:3/501: dread - d4/dd/d33/d76/f7f zero size 2024-02-16T22:22:02.323 INFO:tasks.workunit.client.0.smithi114.stdout:3/502: chown d4/d43/f6e 0 1 2024-02-16T22:22:02.325 INFO:tasks.workunit.client.1.smithi123.stdout:9/749: fdatasync d4/d36/d8e/d58/f91 0 2024-02-16T22:22:02.325 INFO:tasks.workunit.client.0.smithi114.stdout:1/454: symlink d4/d63/d6b/la1 0 2024-02-16T22:22:02.328 INFO:tasks.workunit.client.1.smithi123.stdout:3/636: dwrite d0/d20/d76/d5d/d60/f90 [0,4194304] 0 2024-02-16T22:22:02.330 INFO:tasks.workunit.client.0.smithi114.stdout:6/524: mknod d6/da/c9d 0 2024-02-16T22:22:02.331 INFO:tasks.workunit.client.0.smithi114.stdout:6/525: chown d6/f35 7697834 1 2024-02-16T22:22:02.333 INFO:tasks.workunit.client.0.smithi114.stdout:3/503: mknod d4/d24/d2d/d45/d68/d77/cad 0 2024-02-16T22:22:02.333 INFO:tasks.workunit.client.0.smithi114.stdout:8/472: sync 2024-02-16T22:22:02.334 INFO:tasks.workunit.client.0.smithi114.stdout:7/522: sync 2024-02-16T22:22:02.334 INFO:tasks.workunit.client.0.smithi114.stdout:5/451: sync 2024-02-16T22:22:02.335 INFO:tasks.workunit.client.1.smithi123.stdout:8/688: rmdir d0/d29/da7/d26/d2a/d3c/d62 39 2024-02-16T22:22:02.336 INFO:tasks.workunit.client.0.smithi114.stdout:3/504: write d4/d24/d2d/d45/d68/d77/fab [609005,77365] 0 2024-02-16T22:22:02.336 INFO:tasks.workunit.client.0.smithi114.stdout:1/455: creat d4/d9/d46/d47/fa2 x:0 0 0 2024-02-16T22:22:02.339 INFO:tasks.workunit.client.0.smithi114.stdout:5/452: mknod d0/c88 0 2024-02-16T22:22:02.340 INFO:tasks.workunit.client.0.smithi114.stdout:4/587: getdents dd/d2e 0 2024-02-16T22:22:02.341 INFO:tasks.workunit.client.1.smithi123.stdout:2/610: rename d4/d1a/d1d/d13/d15/d6d/l87 to d4/d1a/d1d/d13/d15/d6d/d4a/da2/lbe 0 2024-02-16T22:22:02.342 INFO:tasks.workunit.client.0.smithi114.stdout:8/473: symlink d4/d11/d78/d7a/d31/lac 0 2024-02-16T22:22:02.344 INFO:tasks.workunit.client.0.smithi114.stdout:3/505: mknod d4/d26/d91/cae 0 2024-02-16T22:22:02.346 INFO:tasks.workunit.client.0.smithi114.stdout:1/456: rename d4/d9/f8a to d4/d9/d3d/d3f/d79/fa3 0 2024-02-16T22:22:02.347 INFO:tasks.workunit.client.1.smithi123.stdout:5/599: creat d6/dd/df/d58/fca x:0 0 0 2024-02-16T22:22:02.347 INFO:tasks.workunit.client.1.smithi123.stdout:1/609: mknod d3/d6/d11/d8b/dbb/ccc 0 2024-02-16T22:22:02.349 INFO:tasks.workunit.client.1.smithi123.stdout:1/610: write d3/d6/db/d10/d4f/fc2 [827834,52215] 0 2024-02-16T22:22:02.349 INFO:tasks.workunit.client.0.smithi114.stdout:7/523: mknod d3/d22/d73/da7/d6f/d89/da6/cab 0 2024-02-16T22:22:02.350 INFO:tasks.workunit.client.0.smithi114.stdout:7/524: dread - d3/d22/d73/da7/d4e/f6e zero size 2024-02-16T22:22:02.352 INFO:tasks.workunit.client.0.smithi114.stdout:3/506: creat d4/d87/faf x:0 0 0 2024-02-16T22:22:02.353 INFO:tasks.workunit.client.0.smithi114.stdout:3/507: chown d4/d46/d48/f5c 399 1 2024-02-16T22:22:02.353 INFO:tasks.workunit.client.0.smithi114.stdout:3/508: stat d4/d46/d48 0 2024-02-16T22:22:02.353 INFO:tasks.workunit.client.1.smithi123.stdout:8/689: creat d0/d29/da7/d26/d2a/d77/fde x:0 0 0 2024-02-16T22:22:02.354 INFO:tasks.workunit.client.1.smithi123.stdout:8/690: fsync d0/d61/d66/d7e/fc7 0 2024-02-16T22:22:02.355 INFO:tasks.workunit.client.1.smithi123.stdout:8/691: chown d0/d29/da7/d26/f7a 13500522 1 2024-02-16T22:22:02.358 INFO:tasks.workunit.client.1.smithi123.stdout:2/611: rename d4/d1a/d1d/d59/fb3 to d4/d1a/d1d/d38/d5e/fbf 0 2024-02-16T22:22:02.358 INFO:tasks.workunit.client.0.smithi114.stdout:8/474: truncate d4/d11/d78/d7a/f10 245149 0 2024-02-16T22:22:02.359 INFO:tasks.workunit.client.1.smithi123.stdout:2/612: readlink d4/d1a/d1d/d59/lbd 0 2024-02-16T22:22:02.360 INFO:tasks.workunit.client.1.smithi123.stdout:7/639: link d3/d6e/d1d/db2/f7e d3/d5/d6c/da6/fd4 0 2024-02-16T22:22:02.364 INFO:tasks.workunit.client.1.smithi123.stdout:6/633: getdents d0/d6/d91/d97 0 2024-02-16T22:22:02.365 INFO:tasks.workunit.client.0.smithi114.stdout:0/430: write d1/d16/d1f/d28/d51/d61/d6d/d6e/f6f [831167,54419] 0 2024-02-16T22:22:02.366 INFO:tasks.workunit.client.1.smithi123.stdout:3/637: getdents d0/d20/d2e/dc4 0 2024-02-16T22:22:02.367 INFO:tasks.workunit.client.0.smithi114.stdout:0/431: stat d1/d16/d1f/d28/d51/d61/d7e 0 2024-02-16T22:22:02.367 INFO:tasks.workunit.client.0.smithi114.stdout:0/432: stat d1/f3d 0 2024-02-16T22:22:02.367 INFO:tasks.workunit.client.1.smithi123.stdout:2/613: dwrite d4/d1a/f9b [0,4194304] 0 2024-02-16T22:22:02.368 INFO:tasks.workunit.client.0.smithi114.stdout:0/433: fsync d1/d16/f73 0 2024-02-16T22:22:02.368 INFO:tasks.workunit.client.1.smithi123.stdout:3/638: stat d0/d20/d76/d5d/f82 0 2024-02-16T22:22:02.369 INFO:tasks.workunit.client.1.smithi123.stdout:0/568: write d1/d4/d17/d54/f99 [654191,64801] 0 2024-02-16T22:22:02.371 INFO:tasks.workunit.client.0.smithi114.stdout:8/475: creat d4/d11/d78/d7a/d7e/fad x:0 0 0 2024-02-16T22:22:02.373 INFO:tasks.workunit.client.0.smithi114.stdout:7/525: symlink d3/d46/da2/lac 0 2024-02-16T22:22:02.374 INFO:tasks.workunit.client.1.smithi123.stdout:0/569: dread d1/d4/d17/d74/fb8 [0,4194304] 0 2024-02-16T22:22:02.374 INFO:tasks.workunit.client.1.smithi123.stdout:7/640: stat d3/d87/da8/faf 0 2024-02-16T22:22:02.375 INFO:tasks.workunit.client.0.smithi114.stdout:9/501: dread d2/f13 [0,4194304] 0 2024-02-16T22:22:02.376 INFO:tasks.workunit.client.1.smithi123.stdout:4/687: dread d5/f29 [0,4194304] 0 2024-02-16T22:22:02.377 INFO:tasks.workunit.client.1.smithi123.stdout:5/600: dread d6/dd/df/d1f/f68 [0,4194304] 0 2024-02-16T22:22:02.380 INFO:tasks.workunit.client.0.smithi114.stdout:4/588: getdents dd/d1b/d41/d74/d9b 0 2024-02-16T22:22:02.380 INFO:tasks.workunit.client.1.smithi123.stdout:6/634: dread - d0/d6/d1a/f6f zero size 2024-02-16T22:22:02.381 INFO:tasks.workunit.client.1.smithi123.stdout:6/635: dread - d0/d9/dc/d3a/d70/dd5/fd8 zero size 2024-02-16T22:22:02.381 INFO:tasks.workunit.client.0.smithi114.stdout:0/434: rmdir d1/d16/d48 39 2024-02-16T22:22:02.382 INFO:tasks.workunit.client.0.smithi114.stdout:8/476: dwrite d4/d11/d78/d7a/d36/f6b [0,4194304] 0 2024-02-16T22:22:02.383 INFO:tasks.workunit.client.0.smithi114.stdout:6/526: dwrite d6/dd/d25/d34/f42 [0,4194304] 0 2024-02-16T22:22:02.385 INFO:tasks.workunit.client.0.smithi114.stdout:8/477: write d4/d11/d78/f9c [879854,118587] 0 2024-02-16T22:22:02.386 INFO:tasks.workunit.client.0.smithi114.stdout:9/502: dread d2/d1a/d23/d35/d47/d63/f6a [0,4194304] 0 2024-02-16T22:22:02.388 INFO:tasks.workunit.client.0.smithi114.stdout:7/526: mknod d3/d46/da2/cad 0 2024-02-16T22:22:02.390 INFO:tasks.workunit.client.1.smithi123.stdout:7/641: dread d3/d5/ff [0,4194304] 0 2024-02-16T22:22:02.391 INFO:tasks.workunit.client.1.smithi123.stdout:7/642: readlink d3/d6e/d24/laa 0 2024-02-16T22:22:02.391 INFO:tasks.workunit.client.0.smithi114.stdout:7/527: write d3/d22/d73/da7/f5e [738606,16699] 0 2024-02-16T22:22:02.392 INFO:tasks.workunit.client.0.smithi114.stdout:4/589: dwrite dd/d1b/d37/f5d [0,4194304] 0 2024-02-16T22:22:02.394 INFO:tasks.workunit.client.0.smithi114.stdout:7/528: write d3/d22/d73/da7/d4e/f6e [969425,24665] 0 2024-02-16T22:22:02.395 INFO:tasks.workunit.client.0.smithi114.stdout:2/480: sync 2024-02-16T22:22:02.395 INFO:tasks.workunit.client.0.smithi114.stdout:7/529: write d3/ff [6762691,64807] 0 2024-02-16T22:22:02.396 INFO:tasks.workunit.client.1.smithi123.stdout:2/614: rename d4/d1a/d1d/d38/d5e/c79 to d4/d1a/d1d/d13/d15/d6d/db1/cc0 0 2024-02-16T22:22:02.396 INFO:tasks.workunit.client.0.smithi114.stdout:4/590: chown dd/d2e 20 1 2024-02-16T22:22:02.399 INFO:tasks.workunit.client.0.smithi114.stdout:5/453: write d0/df/d16/d25/d2e/d3b/f59 [177277,35076] 0 2024-02-16T22:22:02.400 INFO:tasks.workunit.client.1.smithi123.stdout:4/688: mknod d5/d1c/d23/d74/d92/d9b/cd6 0 2024-02-16T22:22:02.403 INFO:tasks.workunit.client.0.smithi114.stdout:0/435: unlink d1/l43 0 2024-02-16T22:22:02.408 INFO:tasks.workunit.client.0.smithi114.stdout:3/509: write d4/f16 [1636884,99089] 0 2024-02-16T22:22:02.409 INFO:tasks.workunit.client.1.smithi123.stdout:8/692: creat d0/d29/da7/d43/fdf x:0 0 0 2024-02-16T22:22:02.410 INFO:tasks.workunit.client.0.smithi114.stdout:1/457: write d4/f30 [241046,99100] 0 2024-02-16T22:22:02.410 INFO:tasks.workunit.client.0.smithi114.stdout:3/510: chown d4/d24/l60 67456 1 2024-02-16T22:22:02.412 INFO:tasks.workunit.client.1.smithi123.stdout:9/750: dwrite d4/d36/d8e/d6d/d71/fa3 [0,4194304] 0 2024-02-16T22:22:02.412 INFO:tasks.workunit.client.0.smithi114.stdout:0/436: dwrite d1/d16/f19 [0,4194304] 0 2024-02-16T22:22:02.413 INFO:tasks.workunit.client.0.smithi114.stdout:8/478: creat d4/d11/d78/d7a/d7e/fae x:0 0 0 2024-02-16T22:22:02.413 INFO:tasks.workunit.client.0.smithi114.stdout:8/479: readlink d4/d24/l8a 0 2024-02-16T22:22:02.414 INFO:tasks.workunit.client.0.smithi114.stdout:8/480: truncate d4/f1d 4556938 0 2024-02-16T22:22:02.415 INFO:tasks.workunit.client.1.smithi123.stdout:9/751: truncate d4/d36/daa/f23 5197653 0 2024-02-16T22:22:02.416 INFO:tasks.workunit.client.1.smithi123.stdout:7/643: dread - d3/d6e/d52/f71 zero size 2024-02-16T22:22:02.417 INFO:tasks.workunit.client.0.smithi114.stdout:8/481: write d4/d11/d78/d7a/d7e/f80 [122162,72937] 0 2024-02-16T22:22:02.417 INFO:tasks.workunit.client.1.smithi123.stdout:2/615: rmdir d4/d1a/d1d 39 2024-02-16T22:22:02.418 INFO:tasks.workunit.client.0.smithi114.stdout:8/482: readlink d4/d24/d6d/l89 0 2024-02-16T22:22:02.418 INFO:tasks.workunit.client.1.smithi123.stdout:8/693: dwrite d0/d12/f58 [0,4194304] 0 2024-02-16T22:22:02.419 INFO:tasks.workunit.client.1.smithi123.stdout:9/752: truncate d4/d1e/d3a/fb7 1001952 0 2024-02-16T22:22:02.419 INFO:tasks.workunit.client.1.smithi123.stdout:7/644: chown d3/d87/l23 284523 1 2024-02-16T22:22:02.419 INFO:tasks.workunit.client.0.smithi114.stdout:7/530: rename d3/d3d/f41 to d3/d5a/fae 0 2024-02-16T22:22:02.424 INFO:tasks.workunit.client.0.smithi114.stdout:4/591: chown dd/d2e/d3f/lcb 503227 1 2024-02-16T22:22:02.426 INFO:tasks.workunit.client.1.smithi123.stdout:5/601: mknod d6/dd/df/d58/d89/dc3/ccb 0 2024-02-16T22:22:02.427 INFO:tasks.workunit.client.0.smithi114.stdout:6/527: write d6/da/f56 [890647,36314] 0 2024-02-16T22:22:02.427 INFO:tasks.workunit.client.1.smithi123.stdout:0/570: dwrite d1/d8/d13/d1d/d62/d7a/f7f [0,4194304] 0 2024-02-16T22:22:02.434 INFO:tasks.workunit.client.0.smithi114.stdout:1/458: chown d4/c14 34163 1 2024-02-16T22:22:02.435 INFO:tasks.workunit.client.0.smithi114.stdout:3/511: mknod d4/d24/d2d/cb0 0 2024-02-16T22:22:02.437 INFO:tasks.workunit.client.1.smithi123.stdout:8/694: mkdir d0/d29/da7/d26/d32/de0 0 2024-02-16T22:22:02.438 INFO:tasks.workunit.client.0.smithi114.stdout:9/503: symlink d2/dd/d3c/d89/d9e/la0 0 2024-02-16T22:22:02.439 INFO:tasks.workunit.client.1.smithi123.stdout:9/753: rename d4/d36/d8e/d6d/d71/d92/fcd to d4/d36/d8e/d6d/d71/da6/fee 0 2024-02-16T22:22:02.441 INFO:tasks.workunit.client.0.smithi114.stdout:0/437: creat d1/d1a/d3e/d41/f85 x:0 0 0 2024-02-16T22:22:02.442 INFO:tasks.workunit.client.1.smithi123.stdout:1/611: sync 2024-02-16T22:22:02.442 INFO:tasks.workunit.client.1.smithi123.stdout:4/689: fsync d5/d1c/d23/d31/fd3 0 2024-02-16T22:22:02.442 INFO:tasks.workunit.client.1.smithi123.stdout:1/612: chown d3/d6/d18/c83 30696340 1 2024-02-16T22:22:02.443 INFO:tasks.workunit.client.0.smithi114.stdout:2/481: mknod dd/df/d1c/d21/c9f 0 2024-02-16T22:22:02.443 INFO:tasks.workunit.client.1.smithi123.stdout:4/690: chown d5/d1c/d5e/dc3 0 1 2024-02-16T22:22:02.445 INFO:tasks.workunit.client.1.smithi123.stdout:0/571: rmdir d1 39 2024-02-16T22:22:02.447 INFO:tasks.workunit.client.0.smithi114.stdout:8/483: unlink d4/d11/d78/d7a/d31/lac 0 2024-02-16T22:22:02.449 INFO:tasks.workunit.client.0.smithi114.stdout:7/531: mknod d3/d46/da2/caf 0 2024-02-16T22:22:02.450 INFO:tasks.workunit.client.1.smithi123.stdout:3/639: getdents d0/d20/d76/d5d/d60/dd6 0 2024-02-16T22:22:02.451 INFO:tasks.workunit.client.1.smithi123.stdout:3/640: write d0/d20/f26 [406456,112101] 0 2024-02-16T22:22:02.451 INFO:tasks.workunit.client.1.smithi123.stdout:5/602: dread d6/dd/d44/f8b [0,4194304] 0 2024-02-16T22:22:02.453 INFO:tasks.workunit.client.1.smithi123.stdout:3/641: truncate d0/d40/fd0 127711 0 2024-02-16T22:22:02.454 INFO:tasks.workunit.client.1.smithi123.stdout:3/642: chown d0/d20/d76/d5d/d60/c77 0 1 2024-02-16T22:22:02.454 INFO:tasks.workunit.client.0.smithi114.stdout:5/454: mkdir d0/df/d16/d25/d89 0 2024-02-16T22:22:02.454 INFO:tasks.workunit.client.0.smithi114.stdout:8/484: dwrite d4/d11/d52/f83 [0,4194304] 0 2024-02-16T22:22:02.454 INFO:tasks.workunit.client.1.smithi123.stdout:3/643: write d0/d20/d29/fdb [592902,29730] 0 2024-02-16T22:22:02.455 INFO:tasks.workunit.client.1.smithi123.stdout:7/645: mknod d3/d87/da8/cd5 0 2024-02-16T22:22:02.455 INFO:tasks.workunit.client.1.smithi123.stdout:3/644: fdatasync d0/d20/d76/d5d/d60/dd6/fd9 0 2024-02-16T22:22:02.456 INFO:tasks.workunit.client.0.smithi114.stdout:6/528: symlink d6/dd/l9e 0 2024-02-16T22:22:02.459 INFO:tasks.workunit.client.0.smithi114.stdout:1/459: creat d4/d9/d46/d47/fa4 x:0 0 0 2024-02-16T22:22:02.459 INFO:tasks.workunit.client.1.smithi123.stdout:8/695: read d0/d29/da7/d26/d32/f56 [3088898,12779] 0 2024-02-16T22:22:02.462 INFO:tasks.workunit.client.0.smithi114.stdout:3/512: mknod d4/d24/d2d/d45/d74/cb1 0 2024-02-16T22:22:02.464 INFO:tasks.workunit.client.0.smithi114.stdout:9/504: mkdir d2/d1a/d23/d32/d46/da1 0 2024-02-16T22:22:02.465 INFO:tasks.workunit.client.0.smithi114.stdout:9/505: truncate d2/d1a/d23/d35/f75 1018705 0 2024-02-16T22:22:02.466 INFO:tasks.workunit.client.1.smithi123.stdout:4/691: rename d5/d1c/l5c to d5/d1c/d23/d74/ld7 0 2024-02-16T22:22:02.467 INFO:tasks.workunit.client.0.smithi114.stdout:0/438: mknod d1/d16/d1f/d28/d51/c86 0 2024-02-16T22:22:02.469 INFO:tasks.workunit.client.0.smithi114.stdout:2/482: mknod dd/d4e/ca0 0 2024-02-16T22:22:02.471 INFO:tasks.workunit.client.0.smithi114.stdout:7/532: creat d3/d22/d76/fb0 x:0 0 0 2024-02-16T22:22:02.472 INFO:tasks.workunit.client.1.smithi123.stdout:2/616: mknod d4/d1a/cc1 0 2024-02-16T22:22:02.476 INFO:tasks.workunit.client.0.smithi114.stdout:4/592: rename dd/d1b/d24/d57/fc7 to dd/d1b/d37/d99/fd7 0 2024-02-16T22:22:02.478 INFO:tasks.workunit.client.0.smithi114.stdout:2/483: dread dd/f3a [0,4194304] 0 2024-02-16T22:22:02.479 INFO:tasks.workunit.client.1.smithi123.stdout:7/646: mknod d3/d5/d6c/da6/cd6 0 2024-02-16T22:22:02.479 INFO:tasks.workunit.client.0.smithi114.stdout:8/485: symlink d4/d24/d6d/laf 0 2024-02-16T22:22:02.480 INFO:tasks.workunit.client.0.smithi114.stdout:8/486: write d4/d11/d78/d7a/d7e/fae [882905,83078] 0 2024-02-16T22:22:02.483 INFO:tasks.workunit.client.0.smithi114.stdout:3/513: symlink d4/d87/lb2 0 2024-02-16T22:22:02.484 INFO:tasks.workunit.client.1.smithi123.stdout:3/645: dread d0/d20/d2e/fd1 [0,4194304] 0 2024-02-16T22:22:02.484 INFO:tasks.workunit.client.1.smithi123.stdout:8/696: chown d0/d61/c94 90 1 2024-02-16T22:22:02.485 INFO:tasks.workunit.client.0.smithi114.stdout:9/506: fdatasync d2/d1a/d23/d35/d1d/f69 0 2024-02-16T22:22:02.512 INFO:tasks.workunit.client.0.smithi114.stdout:0/439: creat d1/d16/d1f/d28/d51/d61/f87 x:0 0 0 2024-02-16T22:22:02.512 INFO:tasks.workunit.client.0.smithi114.stdout:0/440: fdatasync d1/d16/d1f/f63 0 2024-02-16T22:22:02.514 INFO:tasks.workunit.client.1.smithi123.stdout:6/636: getdents d0/d9/dc/d39 0 2024-02-16T22:22:02.518 INFO:tasks.workunit.client.0.smithi114.stdout:0/441: dwrite d1/f74 [0,4194304] 0 2024-02-16T22:22:02.526 INFO:tasks.workunit.client.1.smithi123.stdout:5/603: creat d6/dd/d78/da5/db0/fcc x:0 0 0 2024-02-16T22:22:02.530 INFO:tasks.workunit.client.1.smithi123.stdout:5/604: dread d6/f8 [0,4194304] 0 2024-02-16T22:22:02.532 INFO:tasks.workunit.client.1.smithi123.stdout:3/646: unlink d0/d50/d7e/c88 0 2024-02-16T22:22:02.533 INFO:tasks.workunit.client.1.smithi123.stdout:1/613: sync 2024-02-16T22:22:02.534 INFO:tasks.workunit.client.1.smithi123.stdout:8/697: symlink d0/d29/da7/d43/d34/le1 0 2024-02-16T22:22:02.534 INFO:tasks.workunit.client.1.smithi123.stdout:7/647: dwrite d3/d6e/d52/f7d [0,4194304] 0 2024-02-16T22:22:02.534 INFO:tasks.workunit.client.1.smithi123.stdout:4/692: dwrite d5/f29 [0,4194304] 0 2024-02-16T22:22:02.535 INFO:tasks.workunit.client.1.smithi123.stdout:1/614: chown d3/d6/d49/fb1 137609 1 2024-02-16T22:22:02.536 INFO:tasks.workunit.client.1.smithi123.stdout:1/615: stat d3/d6/db/f3a 0 2024-02-16T22:22:02.538 INFO:tasks.workunit.client.1.smithi123.stdout:6/637: mknod d0/d9/dc/d3a/d70/dd5/ce7 0 2024-02-16T22:22:02.541 INFO:tasks.workunit.client.1.smithi123.stdout:9/754: getdents d4/d36/d57/db2/d8c 0 2024-02-16T22:22:02.543 INFO:tasks.workunit.client.1.smithi123.stdout:9/755: write d4/d36/d8e/d58/fdc [953500,78730] 0 2024-02-16T22:22:02.545 INFO:tasks.workunit.client.1.smithi123.stdout:9/756: fdatasync d4/d36/d8e/d58/d59/d60/deb/fc2 0 2024-02-16T22:22:02.550 INFO:tasks.workunit.client.1.smithi123.stdout:5/605: read d6/dd/df/d58/d3f/dbe/f66 [65775,86826] 0 2024-02-16T22:22:02.552 INFO:tasks.workunit.client.1.smithi123.stdout:7/648: rename d3/d6e/d1d/f2b to d3/d87/fd7 0 2024-02-16T22:22:02.560 INFO:tasks.workunit.client.1.smithi123.stdout:8/698: creat d0/df/dca/fe2 x:0 0 0 2024-02-16T22:22:02.560 INFO:tasks.workunit.client.1.smithi123.stdout:4/693: fdatasync d5/d1c/f35 0 2024-02-16T22:22:02.561 INFO:tasks.workunit.client.1.smithi123.stdout:8/699: chown d0/d29/da7/d43/d34/f38 1691829 1 2024-02-16T22:22:02.561 INFO:tasks.workunit.client.1.smithi123.stdout:9/757: rmdir d4/d36/d8e/d6f 39 2024-02-16T22:22:02.563 INFO:tasks.workunit.client.1.smithi123.stdout:0/572: link d1/l3 d1/d8/d13/d1d/lc8 0 2024-02-16T22:22:02.565 INFO:tasks.workunit.client.0.smithi114.stdout:6/529: rename d6/d2c/d38/d54/d8c to d6/dd/d25/d34/d58/d9f 0 2024-02-16T22:22:02.566 INFO:tasks.workunit.client.0.smithi114.stdout:6/530: write d6/f18 [7998645,1825] 0 2024-02-16T22:22:02.566 INFO:tasks.workunit.client.0.smithi114.stdout:7/533: chown d3/d22/d73/da7/d6f/d89/l80 0 1 2024-02-16T22:22:02.570 INFO:tasks.workunit.client.1.smithi123.stdout:1/616: rename d3/d6/db/d10/d2a/d80/f90 to d3/d6/db/d10/d4f/fcd 0 2024-02-16T22:22:02.572 INFO:tasks.workunit.client.1.smithi123.stdout:3/647: dwrite d0/d20/f68 [0,4194304] 0 2024-02-16T22:22:02.573 INFO:tasks.workunit.client.0.smithi114.stdout:7/534: dwrite d3/d14/f77 [0,4194304] 0 2024-02-16T22:22:02.575 INFO:tasks.workunit.client.0.smithi114.stdout:7/535: write d3/d22/d73/da7/f68 [1779238,14852] 0 2024-02-16T22:22:02.575 INFO:tasks.workunit.client.1.smithi123.stdout:3/648: read d0/d20/d76/f4e [2836396,80560] 0 2024-02-16T22:22:02.576 INFO:tasks.workunit.client.0.smithi114.stdout:7/536: write d3/d92/fa0 [812331,122344] 0 2024-02-16T22:22:02.577 INFO:tasks.workunit.client.1.smithi123.stdout:3/649: write d0/d20/f52 [2855983,119173] 0 2024-02-16T22:22:02.578 INFO:tasks.workunit.client.1.smithi123.stdout:4/694: truncate d5/d49/f4b 1885319 0 2024-02-16T22:22:02.582 INFO:tasks.workunit.client.0.smithi114.stdout:9/507: creat d2/dd/d3c/d89/d9e/fa2 x:0 0 0 2024-02-16T22:22:02.585 INFO:tasks.workunit.client.1.smithi123.stdout:9/758: creat d4/d36/d8e/dbe/fef x:0 0 0 2024-02-16T22:22:02.585 INFO:tasks.workunit.client.1.smithi123.stdout:0/573: rmdir d1/d8/d3d 39 2024-02-16T22:22:02.588 INFO:tasks.workunit.client.1.smithi123.stdout:0/574: readlink d1/d4/l18 0 2024-02-16T22:22:02.588 INFO:tasks.workunit.client.1.smithi123.stdout:6/638: dwrite d0/d74/f90 [0,4194304] 0 2024-02-16T22:22:02.590 INFO:tasks.workunit.client.0.smithi114.stdout:5/455: write d0/d2d/d30/f31 [2874902,66218] 0 2024-02-16T22:22:02.591 INFO:tasks.workunit.client.1.smithi123.stdout:5/606: truncate d6/dd/df/d58/d89/d98/fc8 188115 0 2024-02-16T22:22:02.592 INFO:tasks.workunit.client.1.smithi123.stdout:9/759: chown d4/d36/d8e/d6d/d71/d92/c89 38 1 2024-02-16T22:22:02.592 INFO:tasks.workunit.client.1.smithi123.stdout:2/617: dwrite d4/d1a/d1d/d13/f36 [4194304,4194304] 0 2024-02-16T22:22:02.592 INFO:tasks.workunit.client.1.smithi123.stdout:0/575: dread - d1/d4/d17/d54/fba zero size 2024-02-16T22:22:02.593 INFO:tasks.workunit.client.1.smithi123.stdout:1/617: dread d3/d6/d11/f1f [0,4194304] 0 2024-02-16T22:22:02.595 INFO:tasks.workunit.client.0.smithi114.stdout:6/531: fdatasync d6/d2c/d75/f79 0 2024-02-16T22:22:02.595 INFO:tasks.workunit.client.1.smithi123.stdout:1/618: fdatasync d3/d6/d49/d48/d68/fab 0 2024-02-16T22:22:02.599 INFO:tasks.workunit.client.1.smithi123.stdout:3/650: unlink d0/d20/d76/d5d/d60/d93/fb9 0 2024-02-16T22:22:02.601 INFO:tasks.workunit.client.0.smithi114.stdout:5/456: dwrite d0/df/d16/d25/d2e/f69 [0,4194304] 0 2024-02-16T22:22:02.604 INFO:tasks.workunit.client.0.smithi114.stdout:8/487: mknod d4/cb0 0 2024-02-16T22:22:02.605 INFO:tasks.workunit.client.0.smithi114.stdout:7/537: dread d3/d5a/fae [0,4194304] 0 2024-02-16T22:22:02.608 INFO:tasks.workunit.client.0.smithi114.stdout:0/442: rename d1/d1a/f36 to d1/d16/d5c/d80/f88 0 2024-02-16T22:22:02.609 INFO:tasks.workunit.client.1.smithi123.stdout:0/576: creat d1/da0/fc9 x:0 0 0 2024-02-16T22:22:02.610 INFO:tasks.workunit.client.0.smithi114.stdout:7/538: dread d3/f97 [0,4194304] 0 2024-02-16T22:22:02.613 INFO:tasks.workunit.client.1.smithi123.stdout:9/760: chown d4/d36/d57/db2/f28 4543 1 2024-02-16T22:22:02.614 INFO:tasks.workunit.client.1.smithi123.stdout:9/761: write d4/d36/d8e/dbe/fe3 [624875,82192] 0 2024-02-16T22:22:02.615 INFO:tasks.workunit.client.0.smithi114.stdout:0/443: dwrite d1/f74 [0,4194304] 0 2024-02-16T22:22:02.616 INFO:tasks.workunit.client.1.smithi123.stdout:7/649: link d3/d6e/d52/f67 d3/d6e/d24/fd8 0 2024-02-16T22:22:02.617 INFO:tasks.workunit.client.1.smithi123.stdout:7/650: truncate d3/d6e/d52/f88 1525090 0 2024-02-16T22:22:02.617 INFO:tasks.workunit.client.0.smithi114.stdout:0/444: write d1/d1a/d47/d62/f72 [298449,77210] 0 2024-02-16T22:22:02.618 INFO:tasks.workunit.client.1.smithi123.stdout:2/618: dread d4/d1a/f6a [0,4194304] 0 2024-02-16T22:22:02.624 INFO:tasks.workunit.client.0.smithi114.stdout:1/460: getdents d4/d9/d3d 0 2024-02-16T22:22:02.625 INFO:tasks.workunit.client.0.smithi114.stdout:1/461: dread - d4/d63/d69/d72/f97 zero size 2024-02-16T22:22:02.625 INFO:tasks.workunit.client.0.smithi114.stdout:6/532: dread d6/da/f56 [0,4194304] 0 2024-02-16T22:22:02.625 INFO:tasks.workunit.client.1.smithi123.stdout:5/607: rename d6/dd/df/cab to d6/dd/d44/dba/ccd 0 2024-02-16T22:22:02.628 INFO:tasks.workunit.client.1.smithi123.stdout:5/608: write d6/dd/df/f79 [1797559,56099] 0 2024-02-16T22:22:02.631 INFO:tasks.workunit.client.1.smithi123.stdout:1/619: dwrite d3/d6/d49/d48/d68/fc5 [0,4194304] 0 2024-02-16T22:22:02.632 INFO:tasks.workunit.client.1.smithi123.stdout:1/620: write d3/d6/d18/d9e/f86 [5083631,12710] 0 2024-02-16T22:22:02.635 INFO:tasks.workunit.client.0.smithi114.stdout:9/508: rename d2/d1a/d23/d32/c88 to d2/d1a/d65/ca3 0 2024-02-16T22:22:02.636 INFO:tasks.workunit.client.0.smithi114.stdout:7/539: mknod d3/d14/cb1 0 2024-02-16T22:22:02.638 INFO:tasks.workunit.client.1.smithi123.stdout:4/695: dread d5/d1c/d23/d31/f8f [0,4194304] 0 2024-02-16T22:22:02.638 INFO:tasks.workunit.client.1.smithi123.stdout:0/577: write d1/d8/d13/d1d/d62/fbd [1864920,82106] 0 2024-02-16T22:22:02.639 INFO:tasks.workunit.client.1.smithi123.stdout:4/696: stat d5/d1c/d23/d74/d92/da9 0 2024-02-16T22:22:02.639 INFO:tasks.workunit.client.1.smithi123.stdout:4/697: chown d5/d49/f89 395291 1 2024-02-16T22:22:02.640 INFO:tasks.workunit.client.0.smithi114.stdout:0/445: mkdir d1/d1a/d47/d62/d89 0 2024-02-16T22:22:02.642 INFO:tasks.workunit.client.0.smithi114.stdout:4/593: link dd/d1b/f55 dd/fd8 0 2024-02-16T22:22:02.644 INFO:tasks.workunit.client.0.smithi114.stdout:1/462: creat d4/d9/d65/fa5 x:0 0 0 2024-02-16T22:22:02.645 INFO:tasks.workunit.client.1.smithi123.stdout:6/639: dwrite d0/d6/d91/d93/fab [0,4194304] 0 2024-02-16T22:22:02.646 INFO:tasks.workunit.client.0.smithi114.stdout:6/533: stat d6/dd/d25/d34/f40 0 2024-02-16T22:22:02.648 INFO:tasks.workunit.client.0.smithi114.stdout:3/514: link d4/dd/d33/d72/l89 d4/dd/d33/d8f/lb3 0 2024-02-16T22:22:02.652 INFO:tasks.workunit.client.1.smithi123.stdout:6/640: dwrite d0/d6/d2f/d5d/fa3 [0,4194304] 0 2024-02-16T22:22:02.656 INFO:tasks.workunit.client.0.smithi114.stdout:5/457: rename d0/d10/f23 to d0/df/f8a 0 2024-02-16T22:22:02.658 INFO:tasks.workunit.client.0.smithi114.stdout:3/515: dwrite d4/d24/d2d/d45/d68/d77/fab [0,4194304] 0 2024-02-16T22:22:02.658 INFO:tasks.workunit.client.0.smithi114.stdout:5/458: truncate d0/df/f2b 3103850 0 2024-02-16T22:22:02.659 INFO:tasks.workunit.client.1.smithi123.stdout:1/621: dread d3/d6/d18/d9e/f86 [0,4194304] 0 2024-02-16T22:22:02.659 INFO:tasks.workunit.client.0.smithi114.stdout:9/509: symlink d2/d1a/d23/d35/d47/la4 0 2024-02-16T22:22:02.660 INFO:tasks.workunit.client.0.smithi114.stdout:3/516: dread - d4/d24/f99 zero size 2024-02-16T22:22:02.663 INFO:tasks.workunit.client.0.smithi114.stdout:9/510: dread d2/f15 [0,4194304] 0 2024-02-16T22:22:02.664 INFO:tasks.workunit.client.1.smithi123.stdout:8/700: link d0/d29/da7/d26/d2a/d3c/d62/lb9 d0/d29/da7/d26/d2a/d4d/le3 0 2024-02-16T22:22:02.664 INFO:tasks.workunit.client.1.smithi123.stdout:3/651: write d0/d20/d76/f35 [2102961,84222] 0 2024-02-16T22:22:02.665 INFO:tasks.workunit.client.0.smithi114.stdout:7/540: creat d3/d22/d73/da7/d6f/d89/fb2 x:0 0 0 2024-02-16T22:22:02.665 INFO:tasks.workunit.client.0.smithi114.stdout:7/541: stat d3/d3d/l3f 0 2024-02-16T22:22:02.666 INFO:tasks.workunit.client.0.smithi114.stdout:0/446: dread d1/d1a/d3e/d41/f42 [0,4194304] 0 2024-02-16T22:22:02.667 INFO:tasks.workunit.client.0.smithi114.stdout:0/447: dread - d1/d16/d1f/f63 zero size 2024-02-16T22:22:02.669 INFO:tasks.workunit.client.0.smithi114.stdout:0/448: dread d1/d1a/f38 [0,4194304] 0 2024-02-16T22:22:02.672 INFO:tasks.workunit.client.0.smithi114.stdout:4/594: creat dd/d1b/d41/d74/fd9 x:0 0 0 2024-02-16T22:22:02.675 INFO:tasks.workunit.client.1.smithi123.stdout:5/609: rename d6/dd/df/d58/d89/dc3/ccb to d6/dd/df/d58/d3f/dbe/da2/cce 0 2024-02-16T22:22:02.683 INFO:tasks.workunit.client.0.smithi114.stdout:8/488: dwrite d4/d16/d25/d7c/f8d [0,4194304] 0 2024-02-16T22:22:02.687 INFO:tasks.workunit.client.0.smithi114.stdout:5/459: creat d0/df/d53/f8b x:0 0 0 2024-02-16T22:22:02.688 INFO:tasks.workunit.client.0.smithi114.stdout:3/517: symlink d4/dd/d33/d8f/lb4 0 2024-02-16T22:22:02.691 INFO:tasks.workunit.client.0.smithi114.stdout:1/463: write d4/d9/d46/f5f [850412,106875] 0 2024-02-16T22:22:02.692 INFO:tasks.workunit.client.0.smithi114.stdout:1/464: chown d4/c14 27334 1 2024-02-16T22:22:02.693 INFO:tasks.workunit.client.1.smithi123.stdout:9/762: truncate d4/d36/daa/d20/f6a 3266276 0 2024-02-16T22:22:02.696 INFO:tasks.workunit.client.0.smithi114.stdout:7/542: creat d3/d22/d73/da7/d6f/d89/fb3 x:0 0 0 2024-02-16T22:22:02.697 INFO:tasks.workunit.client.0.smithi114.stdout:1/465: dread d4/f37 [0,4194304] 0 2024-02-16T22:22:02.698 INFO:tasks.workunit.client.1.smithi123.stdout:0/578: dread d1/d4/d17/d74/d57/f75 [0,4194304] 0 2024-02-16T22:22:02.701 INFO:tasks.workunit.client.0.smithi114.stdout:4/595: creat dd/d1b/d37/d38/d87/fda x:0 0 0 2024-02-16T22:22:02.701 INFO:tasks.workunit.client.0.smithi114.stdout:4/596: chown dd/d1b/d24 1 1 2024-02-16T22:22:02.702 INFO:tasks.workunit.client.1.smithi123.stdout:0/579: dread d1/d4/d17/d74/d57/f75 [0,4194304] 0 2024-02-16T22:22:02.702 INFO:tasks.workunit.client.1.smithi123.stdout:7/651: truncate d3/d6e/d1d/db2/f51 970883 0 2024-02-16T22:22:02.706 INFO:tasks.workunit.client.1.smithi123.stdout:7/652: dread d3/d6e/d1d/db2/d3e/f5b [0,4194304] 0 2024-02-16T22:22:02.707 INFO:tasks.workunit.client.0.smithi114.stdout:5/460: creat d0/df/d16/d1e/d86/f8c x:0 0 0 2024-02-16T22:22:02.708 INFO:tasks.workunit.client.0.smithi114.stdout:3/518: mkdir d4/d87/db5 0 2024-02-16T22:22:02.712 INFO:tasks.workunit.client.0.smithi114.stdout:9/511: write d2/dd/d3c/f59 [625713,78407] 0 2024-02-16T22:22:02.713 INFO:tasks.workunit.client.0.smithi114.stdout:7/543: mkdir d3/d22/d73/da7/db4 0 2024-02-16T22:22:02.715 INFO:tasks.workunit.client.0.smithi114.stdout:3/519: dwrite d4/d24/f67 [4194304,4194304] 0 2024-02-16T22:22:02.721 INFO:tasks.workunit.client.0.smithi114.stdout:4/597: mknod dd/d59/d8a/cdb 0 2024-02-16T22:22:02.721 INFO:tasks.workunit.client.0.smithi114.stdout:0/449: write d1/d16/d1f/d28/f35 [4634854,116142] 0 2024-02-16T22:22:02.722 INFO:tasks.workunit.client.0.smithi114.stdout:4/598: fsync dd/d1b/d41/d74/f79 0 2024-02-16T22:22:02.723 INFO:tasks.workunit.client.0.smithi114.stdout:8/489: mknod d4/cb1 0 2024-02-16T22:22:02.723 INFO:tasks.workunit.client.0.smithi114.stdout:4/599: chown dd/d1b/d37/fba 418258967 1 2024-02-16T22:22:02.726 INFO:tasks.workunit.client.0.smithi114.stdout:5/461: mkdir d0/df/d16/d25/d2e/d3b/d8d 0 2024-02-16T22:22:02.726 INFO:tasks.workunit.client.0.smithi114.stdout:5/462: readlink d0/df/d16/d25/d3e/l84 0 2024-02-16T22:22:02.727 INFO:tasks.workunit.client.0.smithi114.stdout:5/463: write d0/df/d16/f49 [3739665,97239] 0 2024-02-16T22:22:02.728 INFO:tasks.workunit.client.0.smithi114.stdout:5/464: readlink d0/df/d16/d25/d3e/l55 0 2024-02-16T22:22:02.734 INFO:tasks.workunit.client.0.smithi114.stdout:9/512: creat d2/d1a/d23/d35/d47/d63/fa5 x:0 0 0 2024-02-16T22:22:02.737 INFO:tasks.workunit.client.0.smithi114.stdout:7/544: symlink d3/d22/d73/da7/d6f/lb5 0 2024-02-16T22:22:02.739 INFO:tasks.workunit.client.0.smithi114.stdout:9/513: dwrite d2/d1a/d23/d35/d47/f4d [0,4194304] 0 2024-02-16T22:22:02.740 INFO:tasks.workunit.client.0.smithi114.stdout:3/520: mknod d4/d87/cb6 0 2024-02-16T22:22:02.740 INFO:tasks.workunit.client.0.smithi114.stdout:7/545: truncate d3/d22/d73/da7/d6f/d89/fb3 421101 0 2024-02-16T22:22:02.742 INFO:tasks.workunit.client.0.smithi114.stdout:0/450: unlink d1/f2a 0 2024-02-16T22:22:02.748 INFO:tasks.workunit.client.0.smithi114.stdout:8/490: fdatasync d4/d16/d25/f6e 0 2024-02-16T22:22:02.749 INFO:tasks.workunit.client.0.smithi114.stdout:0/451: dwrite d1/d1a/f45 [4194304,4194304] 0 2024-02-16T22:22:02.750 INFO:tasks.workunit.client.0.smithi114.stdout:0/452: chown d1/d16/c1e 30447 1 2024-02-16T22:22:02.752 INFO:tasks.workunit.client.0.smithi114.stdout:0/453: read d1/d16/d1f/d28/d51/d60/f5e [663112,72566] 0 2024-02-16T22:22:02.756 INFO:tasks.workunit.client.0.smithi114.stdout:1/466: rmdir d4/d9/d3d/d3f/d79/d7b 0 2024-02-16T22:22:02.758 INFO:tasks.workunit.client.0.smithi114.stdout:3/521: symlink d4/d26/lb7 0 2024-02-16T22:22:02.759 INFO:tasks.workunit.client.0.smithi114.stdout:3/522: write d4/d24/d2d/d45/d68/f86 [1943367,72476] 0 2024-02-16T22:22:02.765 INFO:tasks.workunit.client.0.smithi114.stdout:9/514: dread d2/d1a/d23/d35/d1d/f62 [0,4194304] 0 2024-02-16T22:22:02.766 INFO:tasks.workunit.client.0.smithi114.stdout:9/515: readlink d2/d33/d3f/l42 0 2024-02-16T22:22:02.766 INFO:tasks.workunit.client.0.smithi114.stdout:9/516: dread - d2/dd/d3c/d89/d9e/fa2 zero size 2024-02-16T22:22:02.766 INFO:tasks.workunit.client.1.smithi123.stdout:6/641: sync 2024-02-16T22:22:02.767 INFO:tasks.workunit.client.0.smithi114.stdout:9/517: write d2/d1a/d23/d35/d47/f92 [989617,26887] 0 2024-02-16T22:22:02.772 INFO:tasks.workunit.client.1.smithi123.stdout:1/622: symlink d3/d6/d49/d48/d68/lce 0 2024-02-16T22:22:02.773 INFO:tasks.workunit.client.1.smithi123.stdout:1/623: stat d3/d6/db/d10/c32 0 2024-02-16T22:22:02.773 INFO:tasks.workunit.client.1.smithi123.stdout:1/624: readlink d3/d6/db/d10/d4b/l73 0 2024-02-16T22:22:02.775 INFO:tasks.workunit.client.0.smithi114.stdout:9/518: dwrite d2/d1a/d65/f8f [0,4194304] 0 2024-02-16T22:22:02.776 INFO:tasks.workunit.client.0.smithi114.stdout:4/600: dwrite dd/f12 [0,4194304] 0 2024-02-16T22:22:02.776 INFO:tasks.workunit.client.1.smithi123.stdout:4/698: rename d5/d1c/d23/d31/d98 to d5/d1c/d9a/dd8 0 2024-02-16T22:22:02.779 INFO:tasks.workunit.client.0.smithi114.stdout:0/454: write d1/d16/d18/f77 [61269,123647] 0 2024-02-16T22:22:02.784 INFO:tasks.workunit.client.1.smithi123.stdout:9/763: creat d4/d36/d8e/d6d/d71/da6/ff0 x:0 0 0 2024-02-16T22:22:02.787 INFO:tasks.workunit.client.0.smithi114.stdout:3/523: creat d4/d24/d93/fb8 x:0 0 0 2024-02-16T22:22:02.787 INFO:tasks.workunit.client.0.smithi114.stdout:7/546: write d3/d22/d73/f7c [929466,115787] 0 2024-02-16T22:22:02.793 INFO:tasks.workunit.client.0.smithi114.stdout:5/465: truncate d0/df/d16/d1e/f29 8208636 0 2024-02-16T22:22:02.794 INFO:tasks.workunit.client.1.smithi123.stdout:1/625: symlink d3/d6/db/d10/d2a/d80/lcf 0 2024-02-16T22:22:02.794 INFO:tasks.workunit.client.1.smithi123.stdout:9/764: dread d4/d33/f38 [0,4194304] 0 2024-02-16T22:22:02.795 INFO:tasks.workunit.client.1.smithi123.stdout:7/653: dwrite d3/d87/fd7 [4194304,4194304] 0 2024-02-16T22:22:02.796 INFO:tasks.workunit.client.0.smithi114.stdout:4/601: mkdir dd/d2e/d56/ddc 0 2024-02-16T22:22:02.796 INFO:tasks.workunit.client.1.smithi123.stdout:9/765: readlink d4/d36/d8e/lae 0 2024-02-16T22:22:02.798 INFO:tasks.workunit.client.0.smithi114.stdout:0/455: rename d1/d1a/f76 to d1/d16/d1f/d28/d51/d61/d6d/d79/f8a 0 2024-02-16T22:22:02.799 INFO:tasks.workunit.client.0.smithi114.stdout:1/467: creat d4/d9/d46/d67/d92/fa6 x:0 0 0 2024-02-16T22:22:02.800 INFO:tasks.workunit.client.1.smithi123.stdout:2/619: link d4/f54 d4/d1a/d1d/d59/fc2 0 2024-02-16T22:22:02.801 INFO:tasks.workunit.client.0.smithi114.stdout:3/524: creat d4/d24/d2d/d45/d68/fb9 x:0 0 0 2024-02-16T22:22:02.802 INFO:tasks.workunit.client.1.smithi123.stdout:3/652: dwrite d0/d20/d29/f3c [0,4194304] 0 2024-02-16T22:22:02.805 INFO:tasks.workunit.client.1.smithi123.stdout:5/610: creat d6/dd/df/fcf x:0 0 0 2024-02-16T22:22:02.806 INFO:tasks.workunit.client.0.smithi114.stdout:5/466: rmdir d0/d2d 39 2024-02-16T22:22:02.810 INFO:tasks.workunit.client.0.smithi114.stdout:9/519: unlink d2/d33/d3f/c7a 0 2024-02-16T22:22:02.813 INFO:tasks.workunit.client.1.smithi123.stdout:0/580: rename d1/d8/d13/d1d/f25 to d1/d8/d13/fca 0 2024-02-16T22:22:02.815 INFO:tasks.workunit.client.0.smithi114.stdout:0/456: unlink d1/d1a/l29 0 2024-02-16T22:22:02.818 INFO:tasks.workunit.client.1.smithi123.stdout:1/626: dread d3/f8 [0,4194304] 0 2024-02-16T22:22:02.818 INFO:tasks.workunit.client.0.smithi114.stdout:9/520: dwrite d2/d5c/d5f/f8c [0,4194304] 0 2024-02-16T22:22:02.818 INFO:tasks.workunit.client.0.smithi114.stdout:3/525: unlink d4/d26/l3f 0 2024-02-16T22:22:02.819 INFO:tasks.workunit.client.1.smithi123.stdout:0/581: dwrite d1/d8/faf [0,4194304] 0 2024-02-16T22:22:02.821 INFO:tasks.workunit.client.0.smithi114.stdout:8/491: getdents d4/d24/d6d 0 2024-02-16T22:22:02.825 INFO:tasks.workunit.client.1.smithi123.stdout:8/701: getdents d0/d29 0 2024-02-16T22:22:02.826 INFO:tasks.workunit.client.0.smithi114.stdout:1/468: dwrite d4/d9/f11 [0,4194304] 0 2024-02-16T22:22:02.828 INFO:tasks.workunit.client.1.smithi123.stdout:5/611: mknod d6/dd/d78/cd0 0 2024-02-16T22:22:02.829 INFO:tasks.workunit.client.0.smithi114.stdout:3/526: dwrite d4/d26/d4a/f90 [0,4194304] 0 2024-02-16T22:22:02.832 INFO:tasks.workunit.client.1.smithi123.stdout:8/702: dwrite d0/d12/f74 [0,4194304] 0 2024-02-16T22:22:02.832 INFO:tasks.workunit.client.0.smithi114.stdout:3/527: truncate d4/dd/d33/fa9 666772 0 2024-02-16T22:22:02.833 INFO:tasks.workunit.client.0.smithi114.stdout:3/528: fsync d4/d24/d2d/d45/d68/d77/fab 0 2024-02-16T22:22:02.834 INFO:tasks.workunit.client.1.smithi123.stdout:7/654: mknod d3/d6e/d1d/db2/d3d/db1/cd9 0 2024-02-16T22:22:02.834 INFO:tasks.workunit.client.1.smithi123.stdout:8/703: fsync d0/d29/da7/fb3 0 2024-02-16T22:22:02.834 INFO:tasks.workunit.client.1.smithi123.stdout:7/655: stat l1 0 2024-02-16T22:22:02.839 INFO:tasks.workunit.client.1.smithi123.stdout:6/642: rename d0/d9/dc/d3a/fa1 to d0/d96/fe8 0 2024-02-16T22:22:02.840 INFO:tasks.workunit.client.0.smithi114.stdout:9/521: unlink d2/d1a/d23/d35/l25 0 2024-02-16T22:22:02.841 INFO:tasks.workunit.client.1.smithi123.stdout:9/766: mkdir d4/d36/daa/d2b/de1/df1 0 2024-02-16T22:22:02.842 INFO:tasks.workunit.client.1.smithi123.stdout:9/767: fsync d4/f85 0 2024-02-16T22:22:02.844 INFO:tasks.workunit.client.0.smithi114.stdout:7/547: link d3/f1a d3/d22/d76/fb6 0 2024-02-16T22:22:02.845 INFO:tasks.workunit.client.1.smithi123.stdout:4/699: dwrite d5/d1c/d23/f57 [0,4194304] 0 2024-02-16T22:22:02.847 INFO:tasks.workunit.client.1.smithi123.stdout:4/700: write d5/d1c/d23/f3a [1864171,59614] 0 2024-02-16T22:22:02.848 INFO:tasks.workunit.client.1.smithi123.stdout:4/701: fdatasync d5/d1c/d23/d31/f2d 0 2024-02-16T22:22:02.848 INFO:tasks.workunit.client.0.smithi114.stdout:9/522: dwrite d2/d1a/d23/f52 [0,4194304] 0 2024-02-16T22:22:02.849 INFO:tasks.workunit.client.1.smithi123.stdout:2/620: mknod d4/cc3 0 2024-02-16T22:22:02.851 INFO:tasks.workunit.client.0.smithi114.stdout:9/523: write d2/dd/d3c/f7f [3490128,44241] 0 2024-02-16T22:22:02.857 INFO:tasks.workunit.client.1.smithi123.stdout:3/653: symlink d0/d20/d76/d37/ldd 0 2024-02-16T22:22:02.860 INFO:tasks.workunit.client.0.smithi114.stdout:4/602: creat dd/d1b/d41/fdd x:0 0 0 2024-02-16T22:22:02.862 INFO:tasks.workunit.client.0.smithi114.stdout:1/469: rename d4/d9/d46/d67/l86 to d4/d63/d69/d7c/la7 0 2024-02-16T22:22:02.863 INFO:tasks.workunit.client.0.smithi114.stdout:1/470: write d4/d9/d3d/d3f/f61 [1590865,38758] 0 2024-02-16T22:22:02.864 INFO:tasks.workunit.client.0.smithi114.stdout:3/529: symlink d4/d46/d48/d80/lba 0 2024-02-16T22:22:02.867 INFO:tasks.workunit.client.1.smithi123.stdout:3/654: dread d0/d40/f56 [0,4194304] 0 2024-02-16T22:22:02.868 INFO:tasks.workunit.client.1.smithi123.stdout:3/655: readlink d0/d20/d2e/l47 0 2024-02-16T22:22:02.874 INFO:tasks.workunit.client.0.smithi114.stdout:5/467: write d0/df/d16/d25/d2e/d3b/f77 [2410,42057] 0 2024-02-16T22:22:02.903 INFO:tasks.workunit.client.0.smithi114.stdout:2/484: sync 2024-02-16T22:22:02.905 INFO:tasks.workunit.client.1.smithi123.stdout:1/627: rename d3/d6/d11/f23 to d3/d6/d11/d76/fd0 0 2024-02-16T22:22:02.910 INFO:tasks.workunit.client.0.smithi114.stdout:9/524: creat d2/d1a/d23/d32/d46/d78/fa6 x:0 0 0 2024-02-16T22:22:02.910 INFO:tasks.workunit.client.0.smithi114.stdout:9/525: readlink d2/d1a/d23/d35/l43 0 2024-02-16T22:22:02.911 INFO:tasks.workunit.client.1.smithi123.stdout:9/768: mkdir d4/d36/d8e/dbe/df2 0 2024-02-16T22:22:02.917 INFO:tasks.workunit.client.0.smithi114.stdout:4/603: rename dd/d1b/d37/d38/d7c to dd/d2e/d56/ddc/dde 0 2024-02-16T22:22:02.919 INFO:tasks.workunit.client.1.smithi123.stdout:4/702: unlink d5/d1c/d23/d74/c46 0 2024-02-16T22:22:02.919 INFO:tasks.workunit.client.1.smithi123.stdout:4/703: dread - d5/d1c/fac zero size 2024-02-16T22:22:02.921 INFO:tasks.workunit.client.1.smithi123.stdout:2/621: creat d4/d1a/d1d/d38/d7b/fc4 x:0 0 0 2024-02-16T22:22:02.923 INFO:tasks.workunit.client.0.smithi114.stdout:1/471: creat d4/d9/d46/d67/fa8 x:0 0 0 2024-02-16T22:22:02.923 INFO:tasks.workunit.client.0.smithi114.stdout:8/492: dwrite d4/d16/d25/d39/f4f [0,4194304] 0 2024-02-16T22:22:02.923 INFO:tasks.workunit.client.1.smithi123.stdout:0/582: symlink d1/d8/lcb 0 2024-02-16T22:22:02.924 INFO:tasks.workunit.client.1.smithi123.stdout:0/583: dread - d1/d8/da7/fae zero size 2024-02-16T22:22:02.925 INFO:tasks.workunit.client.1.smithi123.stdout:0/584: dread - d1/d8/d13/d1d/f97 zero size 2024-02-16T22:22:02.925 INFO:tasks.workunit.client.1.smithi123.stdout:5/612: fdatasync d6/dd/df/d58/f4f 0 2024-02-16T22:22:02.925 INFO:tasks.workunit.client.1.smithi123.stdout:0/585: fsync d1/d8/f2f 0 2024-02-16T22:22:02.930 INFO:tasks.workunit.client.0.smithi114.stdout:2/485: creat dd/d4e/fa1 x:0 0 0 2024-02-16T22:22:02.931 INFO:tasks.workunit.client.1.smithi123.stdout:4/704: dread d5/f1f [0,4194304] 0 2024-02-16T22:22:02.932 INFO:tasks.workunit.client.0.smithi114.stdout:0/457: truncate d1/d1a/f45 7138451 0 2024-02-16T22:22:02.933 INFO:tasks.workunit.client.0.smithi114.stdout:7/548: mkdir d3/d22/db7 0 2024-02-16T22:22:02.933 INFO:tasks.workunit.client.0.smithi114.stdout:7/549: chown d3/ff 0 1 2024-02-16T22:22:02.934 INFO:tasks.workunit.client.0.smithi114.stdout:7/550: dread - d3/d22/d61/f9d zero size 2024-02-16T22:22:02.935 INFO:tasks.workunit.client.0.smithi114.stdout:9/526: fsync d2/d5c/d5f/d7b/f22 0 2024-02-16T22:22:02.939 INFO:tasks.workunit.client.1.smithi123.stdout:1/628: mknod d3/d6/d11/d8b/cd1 0 2024-02-16T22:22:02.940 INFO:tasks.workunit.client.1.smithi123.stdout:6/643: dwrite d0/d9/dc/d3a/f80 [0,4194304] 0 2024-02-16T22:22:02.941 INFO:tasks.workunit.client.0.smithi114.stdout:4/604: creat dd/d2e/fdf x:0 0 0 2024-02-16T22:22:02.948 INFO:tasks.workunit.client.0.smithi114.stdout:1/472: rmdir d4/d9/d46/d4d 39 2024-02-16T22:22:02.952 INFO:tasks.workunit.client.0.smithi114.stdout:3/530: mkdir d4/d24/d2d/d45/dbb 0 2024-02-16T22:22:02.952 INFO:tasks.workunit.client.0.smithi114.stdout:3/531: fsync d4/f18 0 2024-02-16T22:22:02.954 INFO:tasks.workunit.client.0.smithi114.stdout:8/493: read d4/d11/d78/d7a/d36/f65 [120018,79711] 0 2024-02-16T22:22:02.956 INFO:tasks.workunit.client.0.smithi114.stdout:0/458: dread d1/d16/d18/f59 [0,4194304] 0 2024-02-16T22:22:02.956 INFO:tasks.workunit.client.0.smithi114.stdout:5/468: creat d0/df/d16/d25/d3e/d7f/f8e x:0 0 0 2024-02-16T22:22:02.958 INFO:tasks.workunit.client.0.smithi114.stdout:0/459: write d1/d16/d1f/d28/d51/d61/d6d/d6e/f6f [833333,74149] 0 2024-02-16T22:22:02.959 INFO:tasks.workunit.client.0.smithi114.stdout:0/460: write d1/d16/d1f/d28/d51/d61/f87 [615037,31522] 0 2024-02-16T22:22:02.960 INFO:tasks.workunit.client.0.smithi114.stdout:0/461: fdatasync d1/d16/f73 0 2024-02-16T22:22:02.962 INFO:tasks.workunit.client.1.smithi123.stdout:7/656: link d3/d5/c3b d3/d6e/d34/cda 0 2024-02-16T22:22:02.964 INFO:tasks.workunit.client.1.smithi123.stdout:8/704: link d0/d12/d3e/c53 d0/d29/da7/d26/d2a/d4d/ce4 0 2024-02-16T22:22:02.964 INFO:tasks.workunit.client.1.smithi123.stdout:8/705: stat d0/f6 0 2024-02-16T22:22:02.965 INFO:tasks.workunit.client.1.smithi123.stdout:8/706: write d0/d29/da7/d26/d68/d81/d47/fdb [714917,12963] 0 2024-02-16T22:22:02.966 INFO:tasks.workunit.client.0.smithi114.stdout:9/527: rename d2/f13 to d2/d1a/d23/d35/d47/d63/fa7 0 2024-02-16T22:22:02.966 INFO:tasks.workunit.client.1.smithi123.stdout:8/707: fsync d0/d29/da7/d26/d68/d81/d47/d9b/fb0 0 2024-02-16T22:22:02.967 INFO:tasks.workunit.client.1.smithi123.stdout:8/708: chown d0/d12/f3f 243786 1 2024-02-16T22:22:02.967 INFO:tasks.workunit.client.1.smithi123.stdout:4/705: rmdir d5/d1c/d23/d47 39 2024-02-16T22:22:02.969 INFO:tasks.workunit.client.0.smithi114.stdout:4/605: mknod dd/d1b/d41/d77/ce0 0 2024-02-16T22:22:02.970 INFO:tasks.workunit.client.1.smithi123.stdout:3/656: mknod d0/d20/d76/d5d/cde 0 2024-02-16T22:22:02.976 INFO:tasks.workunit.client.1.smithi123.stdout:6/644: mknod d0/d6/d91/ce9 0 2024-02-16T22:22:02.981 INFO:tasks.workunit.client.0.smithi114.stdout:7/551: dwrite d3/d22/d73/da7/d4e/f7f [0,4194304] 0 2024-02-16T22:22:02.982 INFO:tasks.workunit.client.1.smithi123.stdout:5/613: symlink d6/dd/ld1 0 2024-02-16T22:22:02.983 INFO:tasks.workunit.client.1.smithi123.stdout:9/769: truncate d4/d33/f73 2615264 0 2024-02-16T22:22:02.986 INFO:tasks.workunit.client.1.smithi123.stdout:7/657: creat d3/d6e/d1d/db2/d35/dba/fdb x:0 0 0 2024-02-16T22:22:02.987 INFO:tasks.workunit.client.0.smithi114.stdout:8/494: rename d4/d11/d78/d7a/d36/f65 to d4/d16/d25/d46/fb2 0 2024-02-16T22:22:02.990 INFO:tasks.workunit.client.1.smithi123.stdout:5/614: dwrite d6/dd/d5f/fb2 [0,4194304] 0 2024-02-16T22:22:02.991 INFO:tasks.workunit.client.1.smithi123.stdout:5/615: readlink d6/dd/df/d1f/la1 0 2024-02-16T22:22:02.992 INFO:tasks.workunit.client.1.smithi123.stdout:8/709: symlink d0/d29/da7/d26/d68/d81/d47/le5 0 2024-02-16T22:22:02.994 INFO:tasks.workunit.client.1.smithi123.stdout:1/629: getdents d3/d6/d49/d48/d84/db5 0 2024-02-16T22:22:02.994 INFO:tasks.workunit.client.0.smithi114.stdout:8/495: dwrite d4/d11/d78/d7a/d7e/fad [0,4194304] 0 2024-02-16T22:22:02.995 INFO:tasks.workunit.client.1.smithi123.stdout:6/645: mkdir d0/d9/dc/d3a/d70/dea 0 2024-02-16T22:22:02.997 INFO:tasks.workunit.client.1.smithi123.stdout:6/646: fsync d0/d96/de3/dcc/fde 0 2024-02-16T22:22:02.997 INFO:tasks.workunit.client.0.smithi114.stdout:5/469: creat d0/d10/f8f x:0 0 0 2024-02-16T22:22:02.998 INFO:tasks.workunit.client.1.smithi123.stdout:2/622: rename d4/d1a/d1d/d13/fb6 to d4/d1a/d1d/d59/d45/fc5 0 2024-02-16T22:22:02.998 INFO:tasks.workunit.client.1.smithi123.stdout:8/710: dread d0/d12/f3f [0,4194304] 0 2024-02-16T22:22:02.999 INFO:tasks.workunit.client.1.smithi123.stdout:0/586: link d1/d4/d42/f4a d1/d8/d13/d1d/d62/fcc 0 2024-02-16T22:22:03.000 INFO:tasks.workunit.client.0.smithi114.stdout:0/462: symlink d1/d16/d55/l8b 0 2024-02-16T22:22:03.001 INFO:tasks.workunit.client.0.smithi114.stdout:7/552: truncate d3/d22/d61/d6a/f94 1533917 0 2024-02-16T22:22:03.004 INFO:tasks.workunit.client.0.smithi114.stdout:8/496: dwrite d4/d11/d78/f9c [0,4194304] 0 2024-02-16T22:22:03.010 INFO:tasks.workunit.client.0.smithi114.stdout:4/606: write dd/f20 [2080078,123965] 0 2024-02-16T22:22:03.012 INFO:tasks.workunit.client.0.smithi114.stdout:4/607: dread - dd/d1b/d24/fc8 zero size 2024-02-16T22:22:03.012 INFO:tasks.workunit.client.0.smithi114.stdout:9/528: dwrite d2/d1a/d23/d35/d47/f92 [0,4194304] 0 2024-02-16T22:22:03.014 INFO:tasks.workunit.client.0.smithi114.stdout:9/529: readlink d2/d1a/d23/d32/d46/l6e 0 2024-02-16T22:22:03.016 INFO:tasks.workunit.client.1.smithi123.stdout:4/706: dwrite d5/d1c/d64/f8a [0,4194304] 0 2024-02-16T22:22:03.019 INFO:tasks.workunit.client.0.smithi114.stdout:0/463: readlink d1/d1a/d47/l5a 0 2024-02-16T22:22:03.021 INFO:tasks.workunit.client.0.smithi114.stdout:7/553: rename d3/d14/l16 to d3/d14/lb8 0 2024-02-16T22:22:03.022 INFO:tasks.workunit.client.0.smithi114.stdout:7/554: write d3/d22/f8f [283211,367] 0 2024-02-16T22:22:03.023 INFO:tasks.workunit.client.1.smithi123.stdout:3/657: dwrite d0/d20/d76/d5d/f61 [4194304,4194304] 0 2024-02-16T22:22:03.025 INFO:tasks.workunit.client.1.smithi123.stdout:0/587: creat d1/d4/d17/dad/fcd x:0 0 0 2024-02-16T22:22:03.026 INFO:tasks.workunit.client.1.smithi123.stdout:2/623: mknod d4/d1a/d1d/d13/d25/d72/cc6 0 2024-02-16T22:22:03.027 INFO:tasks.workunit.client.1.smithi123.stdout:0/588: chown d1/d8/d13/c33 517557829 1 2024-02-16T22:22:03.028 INFO:tasks.workunit.client.1.smithi123.stdout:2/624: fsync d4/d1a/d1d/d13/d15/fb4 0 2024-02-16T22:22:03.029 INFO:tasks.workunit.client.1.smithi123.stdout:8/711: dread d0/d12/f50 [0,4194304] 0 2024-02-16T22:22:03.031 INFO:tasks.workunit.client.0.smithi114.stdout:3/532: link d4/d24/d2d/d45/d68/d77/c85 d4/d24/d2d/d45/d68/cbc 0 2024-02-16T22:22:03.032 INFO:tasks.workunit.client.0.smithi114.stdout:3/533: write d4/d24/d2d/fac [992356,56825] 0 2024-02-16T22:22:03.035 INFO:tasks.workunit.client.0.smithi114.stdout:3/534: truncate d4/dd/d33/d76/f9e 30596 0 2024-02-16T22:22:03.035 INFO:tasks.workunit.client.0.smithi114.stdout:4/608: chown dd/d1b/d24/d98/c9e 4559 1 2024-02-16T22:22:03.036 INFO:tasks.workunit.client.0.smithi114.stdout:9/530: chown f1 18 1 2024-02-16T22:22:03.036 INFO:tasks.workunit.client.0.smithi114.stdout:3/535: write d4/dd/d33/d8f/fa4 [702200,42170] 0 2024-02-16T22:22:03.037 INFO:tasks.workunit.client.0.smithi114.stdout:9/531: write d2/d1a/d23/d35/f67 [43006,18229] 0 2024-02-16T22:22:03.038 INFO:tasks.workunit.client.0.smithi114.stdout:3/536: dread d4/dd/d33/fa9 [0,4194304] 0 2024-02-16T22:22:03.039 INFO:tasks.workunit.client.1.smithi123.stdout:7/658: rename d3/d6e/d1d/db2/d35/c4c to d3/d59/cdc 0 2024-02-16T22:22:03.039 INFO:tasks.workunit.client.1.smithi123.stdout:6/647: dwrite d0/d6/d1a/f36 [4194304,4194304] 0 2024-02-16T22:22:03.040 INFO:tasks.workunit.client.0.smithi114.stdout:0/464: write d1/d16/d18/f59 [8453976,60316] 0 2024-02-16T22:22:03.041 INFO:tasks.workunit.client.0.smithi114.stdout:7/555: symlink d3/d22/d73/lb9 0 2024-02-16T22:22:03.042 INFO:tasks.workunit.client.0.smithi114.stdout:0/465: readlink d1/d16/d1f/d28/d51/d61/l7c 0 2024-02-16T22:22:03.044 INFO:tasks.workunit.client.1.smithi123.stdout:3/658: unlink d0/d20/d76/d5d/d60/f96 0 2024-02-16T22:22:03.044 INFO:tasks.workunit.client.0.smithi114.stdout:7/556: write d3/d22/d73/da7/d6f/fa8 [506857,51961] 0 2024-02-16T22:22:03.045 INFO:tasks.workunit.client.0.smithi114.stdout:8/497: mknod d4/d16/d25/cb3 0 2024-02-16T22:22:03.045 INFO:tasks.workunit.client.1.smithi123.stdout:3/659: write d0/d20/d76/d5d/fd7 [12644,79999] 0 2024-02-16T22:22:03.047 INFO:tasks.workunit.client.1.smithi123.stdout:5/616: creat d6/fd2 x:0 0 0 2024-02-16T22:22:03.047 INFO:tasks.workunit.client.0.smithi114.stdout:1/473: link d4/d9/ce d4/d9/ca9 0 2024-02-16T22:22:03.048 INFO:tasks.workunit.client.1.smithi123.stdout:5/617: fsync d6/d3b/f65 0 2024-02-16T22:22:03.051 INFO:tasks.workunit.client.1.smithi123.stdout:5/618: read d6/d3b/fb5 [37209,95727] 0 2024-02-16T22:22:03.052 INFO:tasks.workunit.client.0.smithi114.stdout:5/470: link d0/df/d16/d1e/d86/f7e d0/d28/f90 0 2024-02-16T22:22:03.054 INFO:tasks.workunit.client.1.smithi123.stdout:8/712: unlink d0/d29/da7/d26/d68/d81/d47/f9d 0 2024-02-16T22:22:03.057 INFO:tasks.workunit.client.0.smithi114.stdout:3/537: mkdir d4/d46/d48/d80/dbd 0 2024-02-16T22:22:03.058 INFO:tasks.workunit.client.0.smithi114.stdout:5/471: dwrite d0/df/d16/d25/d2e/d3b/f59 [0,4194304] 0 2024-02-16T22:22:03.059 INFO:tasks.workunit.client.0.smithi114.stdout:4/609: dread dd/fd8 [0,4194304] 0 2024-02-16T22:22:03.060 INFO:tasks.workunit.client.1.smithi123.stdout:9/770: rename d4/d36/d8e/d58/d59/d60/deb/da7/dde/ldd to d4/d36/d8e/d58/d59/d60/lf3 0 2024-02-16T22:22:03.063 INFO:tasks.workunit.client.0.smithi114.stdout:0/466: mkdir d1/d1a/d47/d62/d7f/d8c 0 2024-02-16T22:22:03.065 INFO:tasks.workunit.client.1.smithi123.stdout:6/648: write d0/d6/f1e [5190154,59885] 0 2024-02-16T22:22:03.066 INFO:tasks.workunit.client.0.smithi114.stdout:3/538: dwrite d4/dd/d33/d76/f7f [0,4194304] 0 2024-02-16T22:22:03.067 INFO:tasks.workunit.client.0.smithi114.stdout:0/467: write d1/d16/d1f/d28/d51/d61/f87 [428705,1655] 0 2024-02-16T22:22:03.067 INFO:tasks.workunit.client.0.smithi114.stdout:8/498: rmdir d4/d11/d78/d7a/d36 39 2024-02-16T22:22:03.069 INFO:tasks.workunit.client.1.smithi123.stdout:3/660: truncate d0/d20/d76/d37/d49/f67 2901648 0 2024-02-16T22:22:03.070 INFO:tasks.workunit.client.0.smithi114.stdout:1/474: chown d4/l8 9076 1 2024-02-16T22:22:03.071 INFO:tasks.workunit.client.0.smithi114.stdout:1/475: chown d4/d9/d3d/d3f/f75 330 1 2024-02-16T22:22:03.071 INFO:tasks.workunit.client.0.smithi114.stdout:3/539: truncate d4/d24/d2d/d45/d68/fb9 161582 0 2024-02-16T22:22:03.072 INFO:tasks.workunit.client.1.smithi123.stdout:2/625: unlink d4/d1a/l89 0 2024-02-16T22:22:03.074 INFO:tasks.workunit.client.1.smithi123.stdout:5/619: creat d6/dd/d78/da5/fd3 x:0 0 0 2024-02-16T22:22:03.075 INFO:tasks.workunit.client.1.smithi123.stdout:5/620: readlink d6/dd/d5f/l94 0 2024-02-16T22:22:03.078 INFO:tasks.workunit.client.0.smithi114.stdout:5/472: creat d0/d10/f91 x:0 0 0 2024-02-16T22:22:03.080 INFO:tasks.workunit.client.0.smithi114.stdout:4/610: creat dd/d1b/d37/d38/d87/fe1 x:0 0 0 2024-02-16T22:22:03.083 INFO:tasks.workunit.client.0.smithi114.stdout:9/532: link d2/d1a/d23/d35/d47/f4d d2/d1a/d23/d35/d47/d63/fa8 0 2024-02-16T22:22:03.085 INFO:tasks.workunit.client.1.smithi123.stdout:9/771: mknod d4/d36/d8e/d58/d59/d60/deb/da7/dde/cf4 0 2024-02-16T22:22:03.090 INFO:tasks.workunit.client.1.smithi123.stdout:6/649: mknod d0/d6/d1a/d88/d8b/ceb 0 2024-02-16T22:22:03.091 INFO:tasks.workunit.client.1.smithi123.stdout:0/589: write d1/f22 [2970206,94513] 0 2024-02-16T22:22:03.092 INFO:tasks.workunit.client.0.smithi114.stdout:8/499: creat d4/d11/d52/fb4 x:0 0 0 2024-02-16T22:22:03.093 INFO:tasks.workunit.client.1.smithi123.stdout:3/661: rmdir d0/d20/d5a/da6 39 2024-02-16T22:22:03.094 INFO:tasks.workunit.client.1.smithi123.stdout:0/590: write d1/d8/d13/d1d/f97 [981097,55825] 0 2024-02-16T22:22:03.096 INFO:tasks.workunit.client.1.smithi123.stdout:8/713: dread d0/df/f16 [0,4194304] 0 2024-02-16T22:22:03.097 INFO:tasks.workunit.client.1.smithi123.stdout:0/591: chown d1/d4/d17/d54/f5b 2 1 2024-02-16T22:22:03.098 INFO:tasks.workunit.client.0.smithi114.stdout:1/476: dread d4/d9/d46/d67/f68 [0,4194304] 0 2024-02-16T22:22:03.098 INFO:tasks.workunit.client.1.smithi123.stdout:6/650: dwrite d0/d9/fb5 [0,4194304] 0 2024-02-16T22:22:03.099 INFO:tasks.workunit.client.1.smithi123.stdout:5/621: mkdir d6/dd/df/d90/dd4 0 2024-02-16T22:22:03.100 INFO:tasks.workunit.client.0.smithi114.stdout:8/500: dwrite d4/d11/d78/d7a/d7e/f9a [0,4194304] 0 2024-02-16T22:22:03.100 INFO:tasks.workunit.client.1.smithi123.stdout:1/630: sync 2024-02-16T22:22:03.101 INFO:tasks.workunit.client.0.smithi114.stdout:5/473: rename d0/d10/f8f to d0/df/d16/d1e/d86/f92 0 2024-02-16T22:22:03.103 INFO:tasks.workunit.client.1.smithi123.stdout:4/707: getdents d5/d1c 0 2024-02-16T22:22:03.103 INFO:tasks.workunit.client.0.smithi114.stdout:6/534: sync 2024-02-16T22:22:03.103 INFO:tasks.workunit.client.0.smithi114.stdout:2/486: sync 2024-02-16T22:22:03.106 INFO:tasks.workunit.client.0.smithi114.stdout:4/611: dread - dd/d2e/fdf zero size 2024-02-16T22:22:03.112 INFO:tasks.workunit.client.0.smithi114.stdout:3/540: getdents d4/d7e 0 2024-02-16T22:22:03.115 INFO:tasks.workunit.client.0.smithi114.stdout:3/541: write d4/d24/f62 [2547720,92794] 0 2024-02-16T22:22:03.115 INFO:tasks.workunit.client.0.smithi114.stdout:7/557: dwrite d3/d46/f47 [0,4194304] 0 2024-02-16T22:22:03.115 INFO:tasks.workunit.client.0.smithi114.stdout:1/477: mkdir d4/d9/d3d/d3f/d79/daa 0 2024-02-16T22:22:03.117 INFO:tasks.workunit.client.1.smithi123.stdout:5/622: mknod d6/dd/df/d9f/cd5 0 2024-02-16T22:22:03.118 INFO:tasks.workunit.client.0.smithi114.stdout:9/533: dwrite d2/d1a/d23/d35/d47/f6d [0,4194304] 0 2024-02-16T22:22:03.118 INFO:tasks.workunit.client.0.smithi114.stdout:7/558: read - d3/d14/fa4 zero size 2024-02-16T22:22:03.120 INFO:tasks.workunit.client.0.smithi114.stdout:1/478: read - d4/d9/d46/d67/d92/fa6 zero size 2024-02-16T22:22:03.122 INFO:tasks.workunit.client.0.smithi114.stdout:8/501: mkdir d4/d16/d25/d46/db5 0 2024-02-16T22:22:03.123 INFO:tasks.workunit.client.0.smithi114.stdout:1/479: chown d4/d9/d46 39154065 1 2024-02-16T22:22:03.124 INFO:tasks.workunit.client.0.smithi114.stdout:2/487: fsync dd/d3b/d8a/f93 0 2024-02-16T22:22:03.124 INFO:tasks.workunit.client.1.smithi123.stdout:7/659: getdents d3/d87 0 2024-02-16T22:22:03.125 INFO:tasks.workunit.client.0.smithi114.stdout:7/559: truncate d3/d22/d76/fb0 40140 0 2024-02-16T22:22:03.128 INFO:tasks.workunit.client.0.smithi114.stdout:4/612: write dd/d2e/d3f/fa6 [87963,115287] 0 2024-02-16T22:22:03.128 INFO:tasks.workunit.client.1.smithi123.stdout:2/626: creat d4/d1a/d1d/d59/fc7 x:0 0 0 2024-02-16T22:22:03.129 INFO:tasks.workunit.client.1.smithi123.stdout:8/714: creat d0/d61/d66/da5/fe6 x:0 0 0 2024-02-16T22:22:03.136 INFO:tasks.workunit.client.0.smithi114.stdout:3/542: dread d4/d43/f56 [0,4194304] 0 2024-02-16T22:22:03.136 INFO:tasks.workunit.client.0.smithi114.stdout:9/534: stat d2/d33/d3f 0 2024-02-16T22:22:03.136 INFO:tasks.workunit.client.0.smithi114.stdout:9/535: write d2/d1a/d23/d35/d1d/f3a [3051808,62586] 0 2024-02-16T22:22:03.137 INFO:tasks.workunit.client.1.smithi123.stdout:5/623: creat d6/dd/d5f/fd6 x:0 0 0 2024-02-16T22:22:03.137 INFO:tasks.workunit.client.0.smithi114.stdout:1/480: dread - d4/f91 zero size 2024-02-16T22:22:03.137 INFO:tasks.workunit.client.0.smithi114.stdout:0/468: truncate d1/d16/d1f/d28/f35 3448936 0 2024-02-16T22:22:03.137 INFO:tasks.workunit.client.0.smithi114.stdout:0/469: readlink d1/d1a/d47/l5b 0 2024-02-16T22:22:03.138 INFO:tasks.workunit.client.0.smithi114.stdout:0/470: write d1/d16/d18/f77 [4679912,100537] 0 2024-02-16T22:22:03.140 INFO:tasks.workunit.client.1.smithi123.stdout:9/772: link d4/d36/d8e/dbe/ccc d4/d33/cf5 0 2024-02-16T22:22:03.140 INFO:tasks.workunit.client.1.smithi123.stdout:6/651: write d0/f6d [14835,89920] 0 2024-02-16T22:22:03.141 INFO:tasks.workunit.client.1.smithi123.stdout:9/773: chown d4/d36/d8e/d58/d59/d60/d9d 576 1 2024-02-16T22:22:03.142 INFO:tasks.workunit.client.1.smithi123.stdout:9/774: stat d4/d36/d8e/dbe 0 2024-02-16T22:22:03.142 INFO:tasks.workunit.client.0.smithi114.stdout:8/502: mknod d4/d16/d25/d46/cb6 0 2024-02-16T22:22:03.143 INFO:tasks.workunit.client.0.smithi114.stdout:8/503: write d4/d16/d25/d7c/f87 [213233,17630] 0 2024-02-16T22:22:03.144 INFO:tasks.workunit.client.0.smithi114.stdout:6/535: chown d6/dd/d25/d34/d61/d62/d8b 470916 1 2024-02-16T22:22:03.145 INFO:tasks.workunit.client.1.smithi123.stdout:3/662: creat d0/d20/fdf x:0 0 0 2024-02-16T22:22:03.146 INFO:tasks.workunit.client.1.smithi123.stdout:6/652: dwrite d0/d6/d91/d97/fe2 [0,4194304] 0 2024-02-16T22:22:03.148 INFO:tasks.workunit.client.1.smithi123.stdout:8/715: fdatasync d0/df/f1c 0 2024-02-16T22:22:03.149 INFO:tasks.workunit.client.0.smithi114.stdout:7/560: mknod d3/d22/d73/da7/d6f/d89/cba 0 2024-02-16T22:22:03.150 INFO:tasks.workunit.client.0.smithi114.stdout:7/561: dread - d3/d3d/f86 zero size 2024-02-16T22:22:03.151 INFO:tasks.workunit.client.1.smithi123.stdout:2/627: mkdir d4/d1a/d1d/d13/d25/d72/dc8 0 2024-02-16T22:22:03.151 INFO:tasks.workunit.client.1.smithi123.stdout:2/628: write d4/d1a/d1d/d13/d15/d6d/fb5 [627633,7767] 0 2024-02-16T22:22:03.153 INFO:tasks.workunit.client.0.smithi114.stdout:4/613: chown dd/d1b/d24/c44 1840431 1 2024-02-16T22:22:03.157 INFO:tasks.workunit.client.0.smithi114.stdout:2/488: creat dd/d3c/d4c/fa2 x:0 0 0 2024-02-16T22:22:03.158 INFO:tasks.workunit.client.1.smithi123.stdout:5/624: creat d6/dd/df/d9f/fd7 x:0 0 0 2024-02-16T22:22:03.159 INFO:tasks.workunit.client.1.smithi123.stdout:1/631: link d3/d6/d11/d8b/cd1 d3/d6/db/d10/d4b/cd2 0 2024-02-16T22:22:03.165 INFO:tasks.workunit.client.0.smithi114.stdout:5/474: dwrite d0/f8 [0,4194304] 0 2024-02-16T22:22:03.166 INFO:tasks.workunit.client.1.smithi123.stdout:1/632: dwrite d3/d47/fa4 [0,4194304] 0 2024-02-16T22:22:03.168 INFO:tasks.workunit.client.1.smithi123.stdout:9/775: creat d4/d36/d8e/d6d/d71/d92/ff6 x:0 0 0 2024-02-16T22:22:03.170 INFO:tasks.workunit.client.1.smithi123.stdout:3/663: creat d0/d20/d94/fe0 x:0 0 0 2024-02-16T22:22:03.178 INFO:tasks.workunit.client.0.smithi114.stdout:9/536: write d2/d1a/d23/d35/d1d/f69 [1202912,27105] 0 2024-02-16T22:22:03.179 INFO:tasks.workunit.client.0.smithi114.stdout:0/471: dwrite d1/f6c [0,4194304] 0 2024-02-16T22:22:03.180 INFO:tasks.workunit.client.0.smithi114.stdout:4/614: dread dd/d1b/d24/d98/fc5 [0,4194304] 0 2024-02-16T22:22:03.181 INFO:tasks.workunit.client.0.smithi114.stdout:4/615: chown dd/d1b/d37 11467476 1 2024-02-16T22:22:03.183 INFO:tasks.workunit.client.0.smithi114.stdout:7/562: mknod d3/d46/cbb 0 2024-02-16T22:22:03.184 INFO:tasks.workunit.client.0.smithi114.stdout:3/543: mkdir d4/dbe 0 2024-02-16T22:22:03.185 INFO:tasks.workunit.client.0.smithi114.stdout:3/544: readlink d4/dd/l9d 0 2024-02-16T22:22:03.186 INFO:tasks.workunit.client.0.smithi114.stdout:5/475: creat d0/df/d53/f93 x:0 0 0 2024-02-16T22:22:03.186 INFO:tasks.workunit.client.0.smithi114.stdout:1/481: getdents d4/d9/d3d/d3f/d79/daa 0 2024-02-16T22:22:03.193 INFO:tasks.workunit.client.0.smithi114.stdout:8/504: getdents d4/d16/d25/d46/db5 0 2024-02-16T22:22:03.198 INFO:tasks.workunit.client.1.smithi123.stdout:6/653: creat d0/d6/d91/fec x:0 0 0 2024-02-16T22:22:03.198 INFO:tasks.workunit.client.1.smithi123.stdout:7/660: truncate d3/d6e/d24/f6d 1254330 0 2024-02-16T22:22:03.198 INFO:tasks.workunit.client.1.smithi123.stdout:8/716: rename d0/d29/da7/d26/d68/d81/d47/la3 to d0/df/dca/le7 0 2024-02-16T22:22:03.199 INFO:tasks.workunit.client.1.smithi123.stdout:0/592: getdents d1/d4/d17/d54 0 2024-02-16T22:22:03.200 INFO:tasks.workunit.client.0.smithi114.stdout:5/476: unlink d0/d28/f37 0 2024-02-16T22:22:03.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:02 smithi114 ceph-mon[29235]: pgmap v29: 65 pgs: 65 active+clean; 2.4 GiB data, 6.8 GiB used, 530 GiB / 536 GiB avail; 100 MiB/s rd, 312 MiB/s wr, 593 op/s 2024-02-16T22:22:03.201 INFO:tasks.workunit.client.0.smithi114.stdout:5/477: chown d0/df/d16/d25/d3e/l50 68 1 2024-02-16T22:22:03.204 INFO:tasks.workunit.client.1.smithi123.stdout:3/664: dwrite d0/d20/d76/d5d/f61 [0,4194304] 0 2024-02-16T22:22:03.204 INFO:tasks.workunit.client.0.smithi114.stdout:5/478: dread d0/df/d16/d1e/f27 [0,4194304] 0 2024-02-16T22:22:03.205 INFO:tasks.workunit.client.0.smithi114.stdout:1/482: creat d4/d63/d69/d7c/fab x:0 0 0 2024-02-16T22:22:03.206 INFO:tasks.workunit.client.0.smithi114.stdout:8/505: creat d4/d16/d25/d46/fb7 x:0 0 0 2024-02-16T22:22:03.207 INFO:tasks.workunit.client.0.smithi114.stdout:8/506: stat d4/d11/d78/d7a/d7e/fae 0 2024-02-16T22:22:03.208 INFO:tasks.workunit.client.0.smithi114.stdout:9/537: rmdir d2/dd/d3c/d89/d9e 39 2024-02-16T22:22:03.212 INFO:tasks.workunit.client.0.smithi114.stdout:4/616: symlink dd/d2e/d56/ddc/dde/le2 0 2024-02-16T22:22:03.214 INFO:tasks.workunit.client.0.smithi114.stdout:4/617: dread dd/f2b [0,4194304] 0 2024-02-16T22:22:03.214 INFO:tasks.workunit.client.1.smithi123.stdout:7/661: rename d3/d87/l81 to d3/d87/ldd 0 2024-02-16T22:22:03.217 INFO:tasks.workunit.client.0.smithi114.stdout:2/489: rename dd/d3c/f77 to dd/d3c/fa3 0 2024-02-16T22:22:03.219 INFO:tasks.workunit.client.1.smithi123.stdout:0/593: symlink d1/d4/d17/d54/d8e/da5/lce 0 2024-02-16T22:22:03.220 INFO:tasks.workunit.client.1.smithi123.stdout:6/654: write d0/d96/de3/f6c [3458165,66961] 0 2024-02-16T22:22:03.220 INFO:tasks.workunit.client.1.smithi123.stdout:3/665: creat d0/d20/d76/d5d/dae/dc7/fe1 x:0 0 0 2024-02-16T22:22:03.222 INFO:tasks.workunit.client.1.smithi123.stdout:2/629: rename d4/d1a/d1d/d13/d15/f56 to d4/d1a/d1d/d13/d15/d6d/d4a/da2/fc9 0 2024-02-16T22:22:03.223 INFO:tasks.workunit.client.0.smithi114.stdout:3/545: link d4/d46/d48/d75/f9c d4/dd/d33/d72/fbf 0 2024-02-16T22:22:03.225 INFO:tasks.workunit.client.0.smithi114.stdout:3/546: chown d4/d24/d2d/d45/d68/d77/cad 81 1 2024-02-16T22:22:03.226 INFO:tasks.workunit.client.0.smithi114.stdout:3/547: chown d4/dd/d33/f5f 166723403 1 2024-02-16T22:22:03.227 INFO:tasks.workunit.client.0.smithi114.stdout:1/483: unlink d4/d9/d46/l58 0 2024-02-16T22:22:03.228 INFO:tasks.workunit.client.0.smithi114.stdout:1/484: dread - d4/d33/f38 zero size 2024-02-16T22:22:03.229 INFO:tasks.workunit.client.1.smithi123.stdout:9/776: creat d4/d36/daa/d20/ff7 x:0 0 0 2024-02-16T22:22:03.230 INFO:tasks.workunit.client.0.smithi114.stdout:7/563: dwrite d3/d14/f5c [0,4194304] 0 2024-02-16T22:22:03.230 INFO:tasks.workunit.client.0.smithi114.stdout:5/479: stat d0/df/d16/d25/d3e/c6b 0 2024-02-16T22:22:03.230 INFO:tasks.workunit.client.1.smithi123.stdout:9/777: truncate d4/d36/d8e/d58/fdc 1765968 0 2024-02-16T22:22:03.231 INFO:tasks.workunit.client.0.smithi114.stdout:5/480: fsync d0/d10/f1a 0 2024-02-16T22:22:03.233 INFO:tasks.workunit.client.0.smithi114.stdout:5/481: chown d0/df/f1d 3 1 2024-02-16T22:22:03.235 INFO:tasks.workunit.client.1.smithi123.stdout:7/662: read d3/d5/f33 [30591,54703] 0 2024-02-16T22:22:03.237 INFO:tasks.workunit.client.0.smithi114.stdout:9/538: rmdir d2/dd 39 2024-02-16T22:22:03.237 INFO:tasks.workunit.client.1.smithi123.stdout:6/655: chown d0/d6/d1a/lac 7 1 2024-02-16T22:22:03.239 INFO:tasks.workunit.client.0.smithi114.stdout:7/564: dread d3/f27 [0,4194304] 0 2024-02-16T22:22:03.239 INFO:tasks.workunit.client.0.smithi114.stdout:1/485: dwrite d4/d9/d3d/d3f/f81 [0,4194304] 0 2024-02-16T22:22:03.239 INFO:tasks.workunit.client.0.smithi114.stdout:4/618: mkdir dd/d1b/db0/de3 0 2024-02-16T22:22:03.239 INFO:tasks.workunit.client.0.smithi114.stdout:6/536: getdents d6/d2c/d75 0 2024-02-16T22:22:03.240 INFO:tasks.workunit.client.1.smithi123.stdout:4/708: sync 2024-02-16T22:22:03.240 INFO:tasks.workunit.client.1.smithi123.stdout:4/709: readlink d5/l68 0 2024-02-16T22:22:03.241 INFO:tasks.workunit.client.1.smithi123.stdout:9/778: mknod d4/d1e/d3a/dac/cf8 0 2024-02-16T22:22:03.242 INFO:tasks.workunit.client.0.smithi114.stdout:4/619: chown dd/d1b/f3e 82451735 1 2024-02-16T22:22:03.242 INFO:tasks.workunit.client.0.smithi114.stdout:2/490: creat dd/df/d1c/fa4 x:0 0 0 2024-02-16T22:22:03.243 INFO:tasks.workunit.client.1.smithi123.stdout:1/633: link d3/d6/d49/d48/d84/d92/fbe d3/fd3 0 2024-02-16T22:22:03.244 INFO:tasks.workunit.client.0.smithi114.stdout:3/548: creat d4/d46/fc0 x:0 0 0 2024-02-16T22:22:03.245 INFO:tasks.workunit.client.0.smithi114.stdout:7/565: write d3/d22/d73/da7/f6b [2850575,125401] 0 2024-02-16T22:22:03.246 INFO:tasks.workunit.client.0.smithi114.stdout:2/491: fdatasync dd/d3c/d4c/f64 0 2024-02-16T22:22:03.246 INFO:tasks.workunit.client.1.smithi123.stdout:4/710: dread d5/d1c/d23/d74/d92/d9b/fc2 [0,4194304] 0 2024-02-16T22:22:03.247 INFO:tasks.workunit.client.0.smithi114.stdout:5/482: dwrite d0/d10/f74 [0,4194304] 0 2024-02-16T22:22:03.247 INFO:tasks.workunit.client.1.smithi123.stdout:7/663: symlink d3/d6e/d1d/d97/lde 0 2024-02-16T22:22:03.248 INFO:tasks.workunit.client.0.smithi114.stdout:2/492: write dd/f92 [3075636,81700] 0 2024-02-16T22:22:03.251 INFO:tasks.workunit.client.0.smithi114.stdout:8/507: mknod d4/d11/d9d/da8/cb8 0 2024-02-16T22:22:03.252 INFO:tasks.workunit.client.0.smithi114.stdout:9/539: dread - d2/d1a/d23/f56 zero size 2024-02-16T22:22:03.254 INFO:tasks.workunit.client.1.smithi123.stdout:7/664: dwrite d3/d6e/d24/fb9 [0,4194304] 0 2024-02-16T22:22:03.260 INFO:tasks.workunit.client.1.smithi123.stdout:9/779: rmdir d4/d36/daa/d20 39 2024-02-16T22:22:03.261 INFO:tasks.workunit.client.0.smithi114.stdout:1/486: mknod d4/d9/d46/d67/d92/cac 0 2024-02-16T22:22:03.263 INFO:tasks.workunit.client.1.smithi123.stdout:1/634: creat d3/d6/db/d10/d4f/fd4 x:0 0 0 2024-02-16T22:22:03.266 INFO:tasks.workunit.client.1.smithi123.stdout:8/717: getdents d0/d61/d66 0 2024-02-16T22:22:03.267 INFO:tasks.workunit.client.0.smithi114.stdout:4/620: symlink dd/d1b/db0/le4 0 2024-02-16T22:22:03.269 INFO:tasks.workunit.client.1.smithi123.stdout:1/635: dwrite d3/d6/db/d10/fc9 [0,4194304] 0 2024-02-16T22:22:03.271 INFO:tasks.workunit.client.1.smithi123.stdout:6/656: write d0/d96/de3/f3e [113994,120677] 0 2024-02-16T22:22:03.273 INFO:tasks.workunit.client.1.smithi123.stdout:2/630: write d4/d1a/d1d/f75 [806657,108296] 0 2024-02-16T22:22:03.274 INFO:tasks.workunit.client.0.smithi114.stdout:5/483: dread - d0/f65 zero size 2024-02-16T22:22:03.278 INFO:tasks.workunit.client.1.smithi123.stdout:1/636: dwrite d3/d6/db/d10/d2d/f5a [0,4194304] 0 2024-02-16T22:22:03.279 INFO:tasks.workunit.client.1.smithi123.stdout:9/780: fsync d4/d36/fad 0 2024-02-16T22:22:03.280 INFO:tasks.workunit.client.1.smithi123.stdout:1/637: chown d3/d6/db/d10/d2a/d80 3 1 2024-02-16T22:22:03.281 INFO:tasks.workunit.client.0.smithi114.stdout:6/537: symlink d6/la0 0 2024-02-16T22:22:03.283 INFO:tasks.workunit.client.1.smithi123.stdout:4/711: mknod d5/db8/cd9 0 2024-02-16T22:22:03.286 INFO:tasks.workunit.client.1.smithi123.stdout:6/657: dread - d0/d6/d91/d93/fae zero size 2024-02-16T22:22:03.287 INFO:tasks.workunit.client.0.smithi114.stdout:7/566: getdents d3/d22/d73/da7/d6f/d9c 0 2024-02-16T22:22:03.289 INFO:tasks.workunit.client.1.smithi123.stdout:3/666: getdents d0/d40 0 2024-02-16T22:22:03.290 INFO:tasks.workunit.client.0.smithi114.stdout:5/484: write d0/fa [4922280,83871] 0 2024-02-16T22:22:03.290 INFO:tasks.workunit.client.1.smithi123.stdout:2/631: creat d4/d1a/d1d/d13/d15/d6d/fca x:0 0 0 2024-02-16T22:22:03.291 INFO:tasks.workunit.client.1.smithi123.stdout:2/632: chown d4/d1a/d1d/d38/d5e/fbf 51 1 2024-02-16T22:22:03.292 INFO:tasks.workunit.client.0.smithi114.stdout:2/493: mkdir dd/d6b/da5 0 2024-02-16T22:22:03.292 INFO:tasks.workunit.client.1.smithi123.stdout:5/625: sync 2024-02-16T22:22:03.296 INFO:tasks.workunit.client.1.smithi123.stdout:3/667: dread d0/d20/d76/fa7 [0,4194304] 0 2024-02-16T22:22:03.297 INFO:tasks.workunit.client.1.smithi123.stdout:1/638: rmdir d3/d6/db/d55 39 2024-02-16T22:22:03.297 INFO:tasks.workunit.client.1.smithi123.stdout:8/718: write d0/df/f1c [2776183,24498] 0 2024-02-16T22:22:03.300 INFO:tasks.workunit.client.0.smithi114.stdout:4/621: fsync dd/d1b/d24/f4a 0 2024-02-16T22:22:03.300 INFO:tasks.workunit.client.1.smithi123.stdout:7/665: dwrite d3/d6e/d34/f57 [0,4194304] 0 2024-02-16T22:22:03.300 INFO:tasks.workunit.client.1.smithi123.stdout:3/668: truncate d0/d20/d29/d9b/fa8 768438 0 2024-02-16T22:22:03.301 INFO:tasks.workunit.client.0.smithi114.stdout:3/549: link d4/dd/f35 d4/d26/fc1 0 2024-02-16T22:22:03.302 INFO:tasks.workunit.client.1.smithi123.stdout:7/666: read - d3/d5/d6c/fac zero size 2024-02-16T22:22:03.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:02 smithi123 ceph-mon[37161]: pgmap v29: 65 pgs: 65 active+clean; 2.4 GiB data, 6.8 GiB used, 530 GiB / 536 GiB avail; 100 MiB/s rd, 312 MiB/s wr, 593 op/s 2024-02-16T22:22:03.302 INFO:tasks.workunit.client.1.smithi123.stdout:7/667: chown d3/d6e/d4d 5891 1 2024-02-16T22:22:03.303 INFO:tasks.workunit.client.1.smithi123.stdout:7/668: readlink d3/d59/la3 0 2024-02-16T22:22:03.303 INFO:tasks.workunit.client.1.smithi123.stdout:6/658: truncate d0/d6/f15 7068742 0 2024-02-16T22:22:03.304 INFO:tasks.workunit.client.1.smithi123.stdout:6/659: readlink d0/d6/d4d/l7c 0 2024-02-16T22:22:03.304 INFO:tasks.workunit.client.0.smithi114.stdout:7/567: creat d3/d22/d73/da7/fbc x:0 0 0 2024-02-16T22:22:03.308 INFO:tasks.workunit.client.0.smithi114.stdout:5/485: symlink d0/df/d16/d25/d2e/l94 0 2024-02-16T22:22:03.308 INFO:tasks.workunit.client.0.smithi114.stdout:5/486: fsync d0/d10/f1a 0 2024-02-16T22:22:03.310 INFO:tasks.workunit.client.1.smithi123.stdout:7/669: dwrite d3/d6e/d1d/db2/d35/db7/fbb [0,4194304] 0 2024-02-16T22:22:03.310 INFO:tasks.workunit.client.1.smithi123.stdout:2/633: unlink d4/d1a/d1d/d13/d15/d6d/d4a/f8b 0 2024-02-16T22:22:03.312 INFO:tasks.workunit.client.1.smithi123.stdout:2/634: write d4/d1a/d1d/f9f [4415837,17436] 0 2024-02-16T22:22:03.313 INFO:tasks.workunit.client.0.smithi114.stdout:1/487: write d4/f21 [5518627,15038] 0 2024-02-16T22:22:03.314 INFO:tasks.workunit.client.0.smithi114.stdout:9/540: dwrite d2/f4 [0,4194304] 0 2024-02-16T22:22:03.314 INFO:tasks.workunit.client.1.smithi123.stdout:2/635: fdatasync d4/d1a/d1d/d13/d15/d6d/d4a/faf 0 2024-02-16T22:22:03.315 INFO:tasks.workunit.client.1.smithi123.stdout:4/712: write d5/d1c/d23/d56/d6e/d73/f88 [1517088,40760] 0 2024-02-16T22:22:03.315 INFO:tasks.workunit.client.1.smithi123.stdout:9/781: dread d4/d36/f5f [0,4194304] 0 2024-02-16T22:22:03.316 INFO:tasks.workunit.client.0.smithi114.stdout:9/541: write d2/d1a/d23/d35/f31 [6815713,113371] 0 2024-02-16T22:22:03.316 INFO:tasks.workunit.client.1.smithi123.stdout:9/782: chown d4/d36/daa/f23 223843431 1 2024-02-16T22:22:03.317 INFO:tasks.workunit.client.1.smithi123.stdout:1/639: mknod d3/d6/db/d10/d4f/cd5 0 2024-02-16T22:22:03.322 INFO:tasks.workunit.client.1.smithi123.stdout:6/660: symlink d0/d6/d2f/da5/led 0 2024-02-16T22:22:03.322 INFO:tasks.workunit.client.0.smithi114.stdout:7/568: symlink d3/d5a/lbd 0 2024-02-16T22:22:03.323 INFO:tasks.workunit.client.1.smithi123.stdout:6/661: fdatasync d0/d9/dc/d3a/f7d 0 2024-02-16T22:22:03.323 INFO:tasks.workunit.client.0.smithi114.stdout:5/487: chown d0/c1 224542435 1 2024-02-16T22:22:03.324 INFO:tasks.workunit.client.1.smithi123.stdout:6/662: write d0/d6/d8e/fb6 [1142136,24846] 0 2024-02-16T22:22:03.326 INFO:tasks.workunit.client.0.smithi114.stdout:6/538: dread d6/d2c/d75/f82 [0,4194304] 0 2024-02-16T22:22:03.327 INFO:tasks.workunit.client.0.smithi114.stdout:2/494: dread - dd/df/d1c/d21/d61/d69/f98 zero size 2024-02-16T22:22:03.328 INFO:tasks.workunit.client.0.smithi114.stdout:1/488: unlink d4/fa 0 2024-02-16T22:22:03.330 INFO:tasks.workunit.client.0.smithi114.stdout:8/508: link d4/d11/d78/d7a/d36/d3b/f44 d4/d11/d78/d7a/d36/fb9 0 2024-02-16T22:22:03.330 INFO:tasks.workunit.client.0.smithi114.stdout:8/509: chown d4/f8 0 1 2024-02-16T22:22:03.331 INFO:tasks.workunit.client.1.smithi123.stdout:6/663: dread d0/d9/dc/d3a/f80 [0,4194304] 0 2024-02-16T22:22:03.332 INFO:tasks.workunit.client.1.smithi123.stdout:4/713: rename d5/d1c/d23/d74/c5a to d5/d1c/d9a/dd8/cda 0 2024-02-16T22:22:03.333 INFO:tasks.workunit.client.1.smithi123.stdout:3/669: symlink d0/d20/d2e/dc4/le2 0 2024-02-16T22:22:03.333 INFO:tasks.workunit.client.1.smithi123.stdout:2/636: dread d4/d1a/f86 [0,4194304] 0 2024-02-16T22:22:03.334 INFO:tasks.workunit.client.1.smithi123.stdout:2/637: chown d4/cc3 43278 1 2024-02-16T22:22:03.335 INFO:tasks.workunit.client.1.smithi123.stdout:2/638: write d4/d1a/d1d/d13/d25/f9a [583106,89615] 0 2024-02-16T22:22:03.336 INFO:tasks.workunit.client.1.smithi123.stdout:9/783: creat d4/d36/d8e/d58/d59/d60/deb/ff9 x:0 0 0 2024-02-16T22:22:03.337 INFO:tasks.workunit.client.0.smithi114.stdout:3/550: symlink d4/d7e/lc2 0 2024-02-16T22:22:03.339 INFO:tasks.workunit.client.0.smithi114.stdout:7/569: mkdir d3/d92/dbe 0 2024-02-16T22:22:03.339 INFO:tasks.workunit.client.0.smithi114.stdout:7/570: stat d3/f99 0 2024-02-16T22:22:03.342 INFO:tasks.workunit.client.1.smithi123.stdout:7/670: symlink d3/d6e/d1d/db2/ldf 0 2024-02-16T22:22:03.344 INFO:tasks.workunit.client.0.smithi114.stdout:7/571: read d3/d22/d73/da7/f5e [663699,110343] 0 2024-02-16T22:22:03.347 INFO:tasks.workunit.client.0.smithi114.stdout:6/539: rename d6/l16 to d6/dd/d25/d34/d61/d62/la1 0 2024-02-16T22:22:03.350 INFO:tasks.workunit.client.1.smithi123.stdout:6/664: mknod d0/d6/d1a/d88/d8b/cee 0 2024-02-16T22:22:03.351 INFO:tasks.workunit.client.0.smithi114.stdout:7/572: dwrite d3/d22/d73/da7/d6f/fa8 [0,4194304] 0 2024-02-16T22:22:03.354 INFO:tasks.workunit.client.0.smithi114.stdout:4/622: creat dd/d1b/d41/fe5 x:0 0 0 2024-02-16T22:22:03.355 INFO:tasks.workunit.client.1.smithi123.stdout:5/626: truncate d6/dd/df/d58/d3f/dbe/f5d 716335 0 2024-02-16T22:22:03.356 INFO:tasks.workunit.client.1.smithi123.stdout:1/640: fdatasync d3/d6/db/f28 0 2024-02-16T22:22:03.362 INFO:tasks.workunit.client.1.smithi123.stdout:2/639: creat d4/d1a/d1d/d13/d15/d6d/d4a/da2/fcb x:0 0 0 2024-02-16T22:22:03.363 INFO:tasks.workunit.client.1.smithi123.stdout:1/641: dwrite d3/d6/f26 [0,4194304] 0 2024-02-16T22:22:03.363 INFO:tasks.workunit.client.0.smithi114.stdout:5/488: symlink d0/d2d/d30/l95 0 2024-02-16T22:22:03.363 INFO:tasks.workunit.client.0.smithi114.stdout:3/551: dwrite d4/d43/f56 [0,4194304] 0 2024-02-16T22:22:03.368 INFO:tasks.workunit.client.0.smithi114.stdout:1/489: mknod d4/d63/d69/d80/cad 0 2024-02-16T22:22:03.368 INFO:tasks.workunit.client.0.smithi114.stdout:1/490: stat d4/d63/d6b 0 2024-02-16T22:22:03.368 INFO:tasks.workunit.client.0.smithi114.stdout:1/491: chown d4/d9 15079 1 2024-02-16T22:22:03.373 INFO:tasks.workunit.client.0.smithi114.stdout:6/540: rename d6/dd/c12 to d6/d2c/d38/d54/ca2 0 2024-02-16T22:22:03.374 INFO:tasks.workunit.client.0.smithi114.stdout:6/541: chown d6/dd/l59 139282 1 2024-02-16T22:22:03.374 INFO:tasks.workunit.client.1.smithi123.stdout:0/594: sync 2024-02-16T22:22:03.375 INFO:tasks.workunit.client.1.smithi123.stdout:4/714: fdatasync d5/d1c/d23/d69/f95 0 2024-02-16T22:22:03.377 INFO:tasks.workunit.client.0.smithi114.stdout:7/573: creat d3/d22/d73/da7/d4e/fbf x:0 0 0 2024-02-16T22:22:03.381 INFO:tasks.workunit.client.1.smithi123.stdout:5/627: unlink d6/dd/df/d9f/la9 0 2024-02-16T22:22:03.383 INFO:tasks.workunit.client.1.smithi123.stdout:3/670: dwrite d0/d20/f6f [0,4194304] 0 2024-02-16T22:22:03.384 INFO:tasks.workunit.client.0.smithi114.stdout:9/542: dwrite d2/f8 [0,4194304] 0 2024-02-16T22:22:03.386 INFO:tasks.workunit.client.0.smithi114.stdout:5/489: unlink d0/d10/f91 0 2024-02-16T22:22:03.388 INFO:tasks.workunit.client.1.smithi123.stdout:5/628: dwrite d6/f8f [0,4194304] 0 2024-02-16T22:22:03.394 INFO:tasks.workunit.client.0.smithi114.stdout:1/492: fsync d4/d33/d74/f87 0 2024-02-16T22:22:03.396 INFO:tasks.workunit.client.0.smithi114.stdout:1/493: chown d4/d9/d46/d47/l9f 1 1 2024-02-16T22:22:03.396 INFO:tasks.workunit.client.0.smithi114.stdout:6/542: mkdir d6/dd/d25/d34/d58/d9f/da3 0 2024-02-16T22:22:03.397 INFO:tasks.workunit.client.0.smithi114.stdout:3/552: dwrite d4/d46/d48/f5c [0,4194304] 0 2024-02-16T22:22:03.398 INFO:tasks.workunit.client.1.smithi123.stdout:9/784: mkdir d4/d36/d8e/d58/d59/d60/d9d/de4/dfa 0 2024-02-16T22:22:03.399 INFO:tasks.workunit.client.1.smithi123.stdout:9/785: chown d4/d36/d8e/d98 306185344 1 2024-02-16T22:22:03.399 INFO:tasks.workunit.client.0.smithi114.stdout:7/574: creat d3/d22/d73/da7/d4e/fc0 x:0 0 0 2024-02-16T22:22:03.402 INFO:tasks.workunit.client.1.smithi123.stdout:8/719: sync 2024-02-16T22:22:03.403 INFO:tasks.workunit.client.1.smithi123.stdout:7/671: mkdir d3/de0 0 2024-02-16T22:22:03.403 INFO:tasks.workunit.client.1.smithi123.stdout:7/672: chown d3/d5/c7 3 1 2024-02-16T22:22:03.408 INFO:tasks.workunit.client.1.smithi123.stdout:0/595: creat d1/d8/d13/d1d/d64/fcf x:0 0 0 2024-02-16T22:22:03.409 INFO:tasks.workunit.client.0.smithi114.stdout:9/543: dwrite d2/fc [0,4194304] 0 2024-02-16T22:22:03.411 INFO:tasks.workunit.client.1.smithi123.stdout:6/665: symlink d0/d6/d2f/dbd/lef 0 2024-02-16T22:22:03.413 INFO:tasks.workunit.client.0.smithi114.stdout:6/543: creat d6/dd/d25/d34/fa4 x:0 0 0 2024-02-16T22:22:03.414 INFO:tasks.workunit.client.1.smithi123.stdout:9/786: dread d4/d36/d8e/d58/d59/d60/deb/f9b [0,4194304] 0 2024-02-16T22:22:03.416 INFO:tasks.workunit.client.1.smithi123.stdout:3/671: creat d0/d20/d2e/fe3 x:0 0 0 2024-02-16T22:22:03.417 INFO:tasks.workunit.client.0.smithi114.stdout:7/575: rmdir d3/d46 39 2024-02-16T22:22:03.417 INFO:tasks.workunit.client.1.smithi123.stdout:3/672: write d0/d20/f6f [3069526,65570] 0 2024-02-16T22:22:03.418 INFO:tasks.workunit.client.1.smithi123.stdout:3/673: readlink d0/l38 0 2024-02-16T22:22:03.418 INFO:tasks.workunit.client.1.smithi123.stdout:5/629: mkdir d6/dd/df/d58/d89/d98/dd8 0 2024-02-16T22:22:03.420 INFO:tasks.workunit.client.0.smithi114.stdout:4/623: creat dd/d1b/d24/fe6 x:0 0 0 2024-02-16T22:22:03.421 INFO:tasks.workunit.client.1.smithi123.stdout:2/640: rename d4/d1a/d1d/c64 to d4/d1a/d1d/d38/d7b/ccc 0 2024-02-16T22:22:03.422 INFO:tasks.workunit.client.0.smithi114.stdout:6/544: dwrite d6/dd/d25/d34/f42 [0,4194304] 0 2024-02-16T22:22:03.423 INFO:tasks.workunit.client.0.smithi114.stdout:7/576: dread d3/d22/f8f [0,4194304] 0 2024-02-16T22:22:03.428 INFO:tasks.workunit.client.0.smithi114.stdout:9/544: mknod d2/d1a/d23/d35/d1d/ca9 0 2024-02-16T22:22:03.430 INFO:tasks.workunit.client.0.smithi114.stdout:1/494: mkdir d4/d63/d69/d7c/d9b/dae 0 2024-02-16T22:22:03.431 INFO:tasks.workunit.client.1.smithi123.stdout:0/596: symlink d1/d8/d13/ld0 0 2024-02-16T22:22:03.432 INFO:tasks.workunit.client.1.smithi123.stdout:1/642: write d3/d6/db/d10/d4b/f72 [181702,36993] 0 2024-02-16T22:22:03.433 INFO:tasks.workunit.client.1.smithi123.stdout:1/643: stat d3/d6/db/d10/d2a/d80/l97 0 2024-02-16T22:22:03.434 INFO:tasks.workunit.client.0.smithi114.stdout:4/624: dwrite dd/d1b/d24/fc8 [0,4194304] 0 2024-02-16T22:22:03.436 INFO:tasks.workunit.client.0.smithi114.stdout:7/577: dwrite d3/d22/f8f [0,4194304] 0 2024-02-16T22:22:03.436 INFO:tasks.workunit.client.0.smithi114.stdout:4/625: chown dd/d2e/d3f/l94 5 1 2024-02-16T22:22:03.437 INFO:tasks.workunit.client.1.smithi123.stdout:9/787: mknod d4/d36/d8e/d58/d59/cfb 0 2024-02-16T22:22:03.439 INFO:tasks.workunit.client.1.smithi123.stdout:9/788: fdatasync d4/d1e/d3a/fcf 0 2024-02-16T22:22:03.440 INFO:tasks.workunit.client.1.smithi123.stdout:4/715: dwrite d5/d1c/f30 [0,4194304] 0 2024-02-16T22:22:03.441 INFO:tasks.workunit.client.1.smithi123.stdout:5/630: readlink d6/dd/df/d1f/l4a 0 2024-02-16T22:22:03.442 INFO:tasks.workunit.client.0.smithi114.stdout:6/545: creat d6/d2c/d66/fa5 x:0 0 0 2024-02-16T22:22:03.444 INFO:tasks.workunit.client.1.smithi123.stdout:5/631: truncate d6/dd/d78/fa0 852070 0 2024-02-16T22:22:03.445 INFO:tasks.workunit.client.1.smithi123.stdout:8/720: rename d0/d29/da7/d26/d2a/f78 to d0/d29/da7/d26/d2a/d77/fe8 0 2024-02-16T22:22:03.449 INFO:tasks.workunit.client.0.smithi114.stdout:6/546: dwrite d6/f18 [4194304,4194304] 0 2024-02-16T22:22:03.451 INFO:tasks.workunit.client.0.smithi114.stdout:9/545: mknod d2/d1a/d23/d35/d47/d63/caa 0 2024-02-16T22:22:03.453 INFO:tasks.workunit.client.1.smithi123.stdout:0/597: creat d1/d4/d17/d74/d57/fd1 x:0 0 0 2024-02-16T22:22:03.453 INFO:tasks.workunit.client.1.smithi123.stdout:2/641: read d4/d1a/d1d/d59/f6f [1366806,122827] 0 2024-02-16T22:22:03.455 INFO:tasks.workunit.client.0.smithi114.stdout:1/495: mkdir d4/d9/d3d/d3f/daf 0 2024-02-16T22:22:03.455 INFO:tasks.workunit.client.0.smithi114.stdout:1/496: fdatasync d4/d33/f43 0 2024-02-16T22:22:03.456 INFO:tasks.workunit.client.1.smithi123.stdout:1/644: creat d3/d6/db/d10/d4b/d70/d87/fd6 x:0 0 0 2024-02-16T22:22:03.458 INFO:tasks.workunit.client.1.smithi123.stdout:4/716: dread d5/d1c/f97 [0,4194304] 0 2024-02-16T22:22:03.458 INFO:tasks.workunit.client.1.smithi123.stdout:3/674: dread d0/d20/d76/f36 [4194304,4194304] 0 2024-02-16T22:22:03.460 INFO:tasks.workunit.client.1.smithi123.stdout:3/675: write d0/d20/d2e/f95 [3300781,36607] 0 2024-02-16T22:22:03.461 INFO:tasks.workunit.client.1.smithi123.stdout:3/676: fdatasync d0/d20/d76/d5d/f86 0 2024-02-16T22:22:03.462 INFO:tasks.workunit.client.0.smithi114.stdout:3/553: truncate d4/dd/d33/d76/f7f 2551296 0 2024-02-16T22:22:03.465 INFO:tasks.workunit.client.0.smithi114.stdout:5/490: getdents d0/df/d16/d25/d3e 0 2024-02-16T22:22:03.466 INFO:tasks.workunit.client.1.smithi123.stdout:8/721: unlink d0/d12/f50 0 2024-02-16T22:22:03.467 INFO:tasks.workunit.client.0.smithi114.stdout:6/547: write d6/da/f9b [826459,36125] 0 2024-02-16T22:22:03.471 INFO:tasks.workunit.client.0.smithi114.stdout:4/626: mkdir dd/d2e/de7 0 2024-02-16T22:22:03.473 INFO:tasks.workunit.client.0.smithi114.stdout:1/497: rename d4/d33 to d4/d63/d69/d7c/d9b/db0 0 2024-02-16T22:22:03.474 INFO:tasks.workunit.client.0.smithi114.stdout:1/498: write d4/fb [155265,38393] 0 2024-02-16T22:22:03.475 INFO:tasks.workunit.client.0.smithi114.stdout:7/578: symlink d3/d22/d73/da7/db4/lc1 0 2024-02-16T22:22:03.480 INFO:tasks.workunit.client.0.smithi114.stdout:1/499: truncate d4/d9/d46/d47/fa2 114997 0 2024-02-16T22:22:03.481 INFO:tasks.workunit.client.0.smithi114.stdout:7/579: dread - d3/d22/d73/da7/fbc zero size 2024-02-16T22:22:03.484 INFO:tasks.workunit.client.0.smithi114.stdout:4/627: chown dd/d2e/l34 3 1 2024-02-16T22:22:03.495 INFO:tasks.workunit.client.1.smithi123.stdout:6/666: rename d0/d6/d2f/c82 to d0/d6/d2f/da5/dc6/cf0 0 2024-02-16T22:22:03.496 INFO:tasks.workunit.client.1.smithi123.stdout:2/642: creat d4/d1a/d1d/d13/d15/d6d/fcd x:0 0 0 2024-02-16T22:22:03.496 INFO:tasks.workunit.client.1.smithi123.stdout:1/645: unlink d3/d52/c58 0 2024-02-16T22:22:03.497 INFO:tasks.workunit.client.1.smithi123.stdout:4/717: fsync d5/f4c 0 2024-02-16T22:22:03.497 INFO:tasks.workunit.client.1.smithi123.stdout:9/789: truncate d4/d33/f73 2102261 0 2024-02-16T22:22:03.498 INFO:tasks.workunit.client.1.smithi123.stdout:6/667: write d0/d6/fd [5138778,93041] 0 2024-02-16T22:22:03.499 INFO:tasks.workunit.client.1.smithi123.stdout:5/632: mknod d6/dd/cd9 0 2024-02-16T22:22:03.499 INFO:tasks.workunit.client.1.smithi123.stdout:8/722: dread d0/d29/da7/d26/d2a/d4d/fb4 [0,4194304] 0 2024-02-16T22:22:03.500 INFO:tasks.workunit.client.1.smithi123.stdout:1/646: chown d3/d6/d49/d48/d84/d92 7523134 1 2024-02-16T22:22:03.502 INFO:tasks.workunit.client.0.smithi114.stdout:5/491: dwrite d0/df/d16/d25/d3e/f44 [0,4194304] 0 2024-02-16T22:22:03.508 INFO:tasks.workunit.client.1.smithi123.stdout:2/643: dwrite d4/d1a/d1d/d13/d15/d6d/db1/fb7 [0,4194304] 0 2024-02-16T22:22:03.511 INFO:tasks.workunit.client.1.smithi123.stdout:8/723: rmdir d0/d29/da7/d43/d34 39 2024-02-16T22:22:03.511 INFO:tasks.workunit.client.1.smithi123.stdout:5/633: creat d6/dd/df/d58/d56/fda x:0 0 0 2024-02-16T22:22:03.513 INFO:tasks.workunit.client.1.smithi123.stdout:3/677: dread d0/d20/d76/d5d/d60/f90 [0,4194304] 0 2024-02-16T22:22:03.513 INFO:tasks.workunit.client.1.smithi123.stdout:1/647: mkdir d3/d6/db/d10/d4b/d70/dd7 0 2024-02-16T22:22:03.515 INFO:tasks.workunit.client.1.smithi123.stdout:6/668: creat d0/d9/dc/d3a/d70/dd5/ff1 x:0 0 0 2024-02-16T22:22:03.517 INFO:tasks.workunit.client.1.smithi123.stdout:4/718: dwrite d5/d1c/d23/d47/da2/fb2 [0,4194304] 0 2024-02-16T22:22:03.519 INFO:tasks.workunit.client.1.smithi123.stdout:4/719: write d5/f90 [4700118,128693] 0 2024-02-16T22:22:03.519 INFO:tasks.workunit.client.0.smithi114.stdout:1/500: creat d4/d63/d69/d80/fb1 x:0 0 0 2024-02-16T22:22:03.520 INFO:tasks.workunit.client.0.smithi114.stdout:1/501: chown d4/d9 440 1 2024-02-16T22:22:03.521 INFO:tasks.workunit.client.0.smithi114.stdout:7/580: symlink d3/d22/d73/da7/d4e/lc2 0 2024-02-16T22:22:03.523 INFO:tasks.workunit.client.1.smithi123.stdout:2/644: mkdir d4/d1a/d1d/d13/d15/d6d/d4a/dce 0 2024-02-16T22:22:03.524 INFO:tasks.workunit.client.0.smithi114.stdout:9/546: creat d2/dd/d3c/d89/d9e/fab x:0 0 0 2024-02-16T22:22:03.525 INFO:tasks.workunit.client.1.smithi123.stdout:8/724: mknod d0/d61/ce9 0 2024-02-16T22:22:03.527 INFO:tasks.workunit.client.1.smithi123.stdout:5/634: creat d6/dd/df/d58/d3f/fdb x:0 0 0 2024-02-16T22:22:03.528 INFO:tasks.workunit.client.1.smithi123.stdout:7/673: sync 2024-02-16T22:22:03.528 INFO:tasks.workunit.client.1.smithi123.stdout:5/635: write d6/dd/df/d58/d3f/dbe/f3a [4362906,71281] 0 2024-02-16T22:22:03.529 INFO:tasks.workunit.client.0.smithi114.stdout:6/548: rename d6/dd/d25/d34/c7f to d6/d2c/d38/d41/d60/d84/ca6 0 2024-02-16T22:22:03.531 INFO:tasks.workunit.client.1.smithi123.stdout:5/636: write d6/dd/df/d58/d89/f9c [504201,33604] 0 2024-02-16T22:22:03.531 INFO:tasks.workunit.client.1.smithi123.stdout:1/648: mkdir d3/d6/d11/d8b/dbb/dd8 0 2024-02-16T22:22:03.532 INFO:tasks.workunit.client.1.smithi123.stdout:5/637: fsync d6/d3b/f65 0 2024-02-16T22:22:03.533 INFO:tasks.workunit.client.1.smithi123.stdout:1/649: readlink d3/d6/db/d10/d4b/l73 0 2024-02-16T22:22:03.534 INFO:tasks.workunit.client.1.smithi123.stdout:6/669: symlink d0/d9/dc/d3a/d70/dd5/lf2 0 2024-02-16T22:22:03.534 INFO:tasks.workunit.client.0.smithi114.stdout:6/549: dwrite d6/dd/d25/d34/f42 [0,4194304] 0 2024-02-16T22:22:03.542 INFO:tasks.workunit.client.1.smithi123.stdout:2/645: write d4/d1a/d1d/d38/d5e/fbf [84660,124093] 0 2024-02-16T22:22:03.545 INFO:tasks.workunit.client.1.smithi123.stdout:8/725: truncate d0/d12/f3f 2331242 0 2024-02-16T22:22:03.547 INFO:tasks.workunit.client.1.smithi123.stdout:8/726: read d0/d29/da7/d26/d68/d81/d47/d9b/fb0 [224912,78855] 0 2024-02-16T22:22:03.548 INFO:tasks.workunit.client.1.smithi123.stdout:3/678: mknod d0/d20/ce4 0 2024-02-16T22:22:03.552 INFO:tasks.workunit.client.1.smithi123.stdout:6/670: symlink d0/d6/d2f/d5d/lf3 0 2024-02-16T22:22:03.553 INFO:tasks.workunit.client.1.smithi123.stdout:9/790: creat d4/d36/daa/d20/dcb/ffc x:0 0 0 2024-02-16T22:22:03.554 INFO:tasks.workunit.client.1.smithi123.stdout:9/791: write d4/d36/d8e/d6d/d71/d92/ff6 [1018913,107388] 0 2024-02-16T22:22:03.558 INFO:tasks.workunit.client.1.smithi123.stdout:5/638: mkdir d6/dd/df/d1f/ddc 0 2024-02-16T22:22:03.561 INFO:tasks.workunit.client.1.smithi123.stdout:4/720: link d5/d1c/d23/d74/d92/d9b/fc2 d5/d1c/d64/d7b/fdb 0 2024-02-16T22:22:03.564 INFO:tasks.workunit.client.1.smithi123.stdout:1/650: link d3/d6/cad d3/d6/db/d10/d2a/cd9 0 2024-02-16T22:22:03.565 INFO:tasks.workunit.client.1.smithi123.stdout:6/671: fdatasync d0/d6/d1a/f64 0 2024-02-16T22:22:03.566 INFO:tasks.workunit.client.1.smithi123.stdout:4/721: symlink d5/d1c/d23/d56/d6e/d73/ldc 0 2024-02-16T22:22:03.586 INFO:tasks.workunit.client.1.smithi123.stdout:1/651: mknod d3/d6/db/d10/d2a/d80/cda 0 2024-02-16T22:22:03.587 INFO:tasks.workunit.client.1.smithi123.stdout:4/722: chown d5/d49/f89 237377 1 2024-02-16T22:22:03.588 INFO:tasks.workunit.client.1.smithi123.stdout:6/672: write d0/d6/d1a/d2e/f3c [13358,102886] 0 2024-02-16T22:22:03.589 INFO:tasks.workunit.client.1.smithi123.stdout:2/646: getdents d4/d1a 0 2024-02-16T22:22:03.590 INFO:tasks.workunit.client.1.smithi123.stdout:5/639: getdents d6/dd/df/d58/d89 0 2024-02-16T22:22:03.592 INFO:tasks.workunit.client.1.smithi123.stdout:1/652: write d3/d6/d49/d48/d84/f93 [252192,84991] 0 2024-02-16T22:22:03.594 INFO:tasks.workunit.client.1.smithi123.stdout:6/673: chown d0/d9/fc3 786 1 2024-02-16T22:22:03.595 INFO:tasks.workunit.client.1.smithi123.stdout:5/640: stat d6/dd/df/f1b 0 2024-02-16T22:22:03.595 INFO:tasks.workunit.client.1.smithi123.stdout:2/647: chown d4/d1a/d1d/d13/d25/c61 486962 1 2024-02-16T22:22:03.595 INFO:tasks.workunit.client.1.smithi123.stdout:8/727: dwrite d0/d61/d66/f98 [0,4194304] 0 2024-02-16T22:22:03.596 INFO:tasks.workunit.client.1.smithi123.stdout:6/674: stat d0/d6 0 2024-02-16T22:22:03.596 INFO:tasks.workunit.client.1.smithi123.stdout:3/679: dwrite d0/f17 [0,4194304] 0 2024-02-16T22:22:03.596 INFO:tasks.workunit.client.1.smithi123.stdout:1/653: symlink d3/d6/d11/ldb 0 2024-02-16T22:22:03.596 INFO:tasks.workunit.client.1.smithi123.stdout:2/648: dread - d4/fa1 zero size 2024-02-16T22:22:03.600 INFO:tasks.workunit.client.1.smithi123.stdout:3/680: dread d0/d40/fd0 [0,4194304] 0 2024-02-16T22:22:03.600 INFO:tasks.workunit.client.1.smithi123.stdout:6/675: mknod d0/d6/d4d/db0/cf4 0 2024-02-16T22:22:03.602 INFO:tasks.workunit.client.1.smithi123.stdout:5/641: fsync d6/dd/df/d58/d3f/d82/f83 0 2024-02-16T22:22:03.602 INFO:tasks.workunit.client.1.smithi123.stdout:6/676: stat d0/d6/d1a/d88/d8b/cee 0 2024-02-16T22:22:03.604 INFO:tasks.workunit.client.1.smithi123.stdout:5/642: write d6/fd2 [193412,5817] 0 2024-02-16T22:22:03.604 INFO:tasks.workunit.client.1.smithi123.stdout:1/654: rename d3/d6/db/d10/d4b/cd2 to d3/d6/d18/cdc 0 2024-02-16T22:22:03.608 INFO:tasks.workunit.client.1.smithi123.stdout:2/649: rename d4/d1a/d1d/d13/d15/d6d/d4a/f5c to d4/d1a/d1d/d38/d5e/fcf 0 2024-02-16T22:22:03.611 INFO:tasks.workunit.client.0.smithi114.stdout:1/502: creat d4/d63/d69/d72/d82/fb2 x:0 0 0 2024-02-16T22:22:03.611 INFO:tasks.workunit.client.0.smithi114.stdout:9/547: unlink d2/d33/d3f/c77 0 2024-02-16T22:22:03.613 INFO:tasks.workunit.client.0.smithi114.stdout:7/581: rename d3/d3d/c55 to d3/d22/d61/d6a/cc3 0 2024-02-16T22:22:03.613 INFO:tasks.workunit.client.0.smithi114.stdout:4/628: write dd/d1b/d24/f4e [2166522,57760] 0 2024-02-16T22:22:03.614 INFO:tasks.workunit.client.1.smithi123.stdout:1/655: unlink d3/d6/db/f3a 0 2024-02-16T22:22:03.616 INFO:tasks.workunit.client.1.smithi123.stdout:5/643: read d6/dd/df/d58/d89/d98/fc8 [101046,29087] 0 2024-02-16T22:22:03.620 INFO:tasks.workunit.client.1.smithi123.stdout:7/674: sync 2024-02-16T22:22:03.620 INFO:tasks.workunit.client.1.smithi123.stdout:0/598: sync 2024-02-16T22:22:03.620 INFO:tasks.workunit.client.1.smithi123.stdout:6/677: rename d0/d9/dc/d3a/d70/dd5/lf2 to d0/d96/de3/lf5 0 2024-02-16T22:22:03.622 INFO:tasks.workunit.client.1.smithi123.stdout:6/678: chown d0/d9/f62 599 1 2024-02-16T22:22:03.624 INFO:tasks.workunit.client.1.smithi123.stdout:6/679: chown d0/d6/l37 429276845 1 2024-02-16T22:22:03.626 INFO:tasks.workunit.client.1.smithi123.stdout:3/681: dwrite d0/d20/d76/d5d/f82 [0,4194304] 0 2024-02-16T22:22:03.639 INFO:tasks.workunit.client.1.smithi123.stdout:5/644: write d6/d3b/fb5 [1162836,90316] 0 2024-02-16T22:22:03.639 INFO:tasks.workunit.client.1.smithi123.stdout:2/650: link d4/d1a/d1d/d13/d25/f26 d4/d1a/d1d/d13/d15/d6d/d4a/da2/fd0 0 2024-02-16T22:22:03.641 INFO:tasks.workunit.client.1.smithi123.stdout:6/680: unlink d0/d6/d2f/d5d/cbc 0 2024-02-16T22:22:03.642 INFO:tasks.workunit.client.1.smithi123.stdout:6/681: chown d0/d9/dc/d3a/c48 4179209 1 2024-02-16T22:22:03.643 INFO:tasks.workunit.client.1.smithi123.stdout:1/656: creat d3/d6/db/fdd x:0 0 0 2024-02-16T22:22:03.644 INFO:tasks.workunit.client.1.smithi123.stdout:5/645: symlink d6/dd/d44/dba/ldd 0 2024-02-16T22:22:03.645 INFO:tasks.workunit.client.1.smithi123.stdout:7/675: dread d3/d6e/d1d/db2/d35/f9f [0,4194304] 0 2024-02-16T22:22:03.645 INFO:tasks.workunit.client.1.smithi123.stdout:3/682: rename d0/d20/f8b to d0/d20/d76/fe5 0 2024-02-16T22:22:03.647 INFO:tasks.workunit.client.1.smithi123.stdout:0/599: getdents d1/d8/d13/d1d 0 2024-02-16T22:22:03.649 INFO:tasks.workunit.client.1.smithi123.stdout:2/651: dwrite d4/d1a/d1d/d13/d25/d98/f96 [0,4194304] 0 2024-02-16T22:22:03.649 INFO:tasks.workunit.client.1.smithi123.stdout:7/676: dread - d3/d5/d6c/f8d zero size 2024-02-16T22:22:03.651 INFO:tasks.workunit.client.1.smithi123.stdout:7/677: write d3/f2a [6986183,116251] 0 2024-02-16T22:22:03.656 INFO:tasks.workunit.client.1.smithi123.stdout:3/683: mknod d0/d20/d29/d9b/dbd/ce6 0 2024-02-16T22:22:03.656 INFO:tasks.workunit.client.1.smithi123.stdout:6/682: write d0/d6/f63 [662629,46532] 0 2024-02-16T22:22:03.656 INFO:tasks.workunit.client.1.smithi123.stdout:0/600: read d1/d4/d17/d74/f58 [1153367,129165] 0 2024-02-16T22:22:03.656 INFO:tasks.workunit.client.1.smithi123.stdout:3/684: dread - d0/d20/d76/d37/fa1 zero size 2024-02-16T22:22:03.657 INFO:tasks.workunit.client.1.smithi123.stdout:6/683: chown d0/d6/d4d/db0/lc4 13 1 2024-02-16T22:22:03.657 INFO:tasks.workunit.client.1.smithi123.stdout:3/685: chown d0/d40/c4a 306438 1 2024-02-16T22:22:03.658 INFO:tasks.workunit.client.1.smithi123.stdout:7/678: link d3/d6e/d52/f6a d3/d59/fe1 0 2024-02-16T22:22:03.659 INFO:tasks.workunit.client.1.smithi123.stdout:2/652: rename d4/ce to d4/d1a/d1d/d13/d25/d98/cd1 0 2024-02-16T22:22:03.660 INFO:tasks.workunit.client.1.smithi123.stdout:0/601: stat d1/d8/d13/c3b 0 2024-02-16T22:22:03.661 INFO:tasks.workunit.client.1.smithi123.stdout:6/684: mknod d0/d6/d8e/cf6 0 2024-02-16T22:22:03.663 INFO:tasks.workunit.client.1.smithi123.stdout:3/686: creat d0/d20/d76/d5d/dae/db0/fe7 x:0 0 0 2024-02-16T22:22:03.663 INFO:tasks.workunit.client.1.smithi123.stdout:1/657: dwrite d3/f7 [0,4194304] 0 2024-02-16T22:22:03.668 INFO:tasks.workunit.client.1.smithi123.stdout:7/679: rename d3/d5/f61 to d3/d6e/d1d/db2/d35/db7/fe2 0 2024-02-16T22:22:03.671 INFO:tasks.workunit.client.1.smithi123.stdout:5/646: dwrite d6/dd/df/d58/d3f/dbe/f5d [0,4194304] 0 2024-02-16T22:22:03.672 INFO:tasks.workunit.client.1.smithi123.stdout:5/647: write d6/dd/d78/fa0 [192513,3696] 0 2024-02-16T22:22:03.675 INFO:tasks.workunit.client.1.smithi123.stdout:1/658: dwrite d3/d6/db/d10/d4b/d70/d87/fd6 [0,4194304] 0 2024-02-16T22:22:03.676 INFO:tasks.workunit.client.1.smithi123.stdout:1/659: readlink d3/d6/db/l67 0 2024-02-16T22:22:03.677 INFO:tasks.workunit.client.1.smithi123.stdout:2/653: creat d4/d1a/d1d/d38/d5e/fd2 x:0 0 0 2024-02-16T22:22:03.678 INFO:tasks.workunit.client.0.smithi114.stdout:1/503: mkdir d4/d9/d46/d47/db3 0 2024-02-16T22:22:03.679 INFO:tasks.workunit.client.0.smithi114.stdout:9/548: rmdir d2/d1a/d23/d32/d46 39 2024-02-16T22:22:03.680 INFO:tasks.workunit.client.1.smithi123.stdout:0/602: symlink d1/d4/d42/ld2 0 2024-02-16T22:22:03.680 INFO:tasks.workunit.client.1.smithi123.stdout:1/660: read d3/d6/db/d10/f29 [4109971,120598] 0 2024-02-16T22:22:03.681 INFO:tasks.workunit.client.1.smithi123.stdout:0/603: write d1/da0/fb9 [87152,103893] 0 2024-02-16T22:22:03.684 INFO:tasks.workunit.client.0.smithi114.stdout:5/492: dwrite d0/d28/f90 [0,4194304] 0 2024-02-16T22:22:03.685 INFO:tasks.workunit.client.0.smithi114.stdout:4/629: chown dd/d2e/d3f/f75 211 1 2024-02-16T22:22:03.689 INFO:tasks.workunit.client.0.smithi114.stdout:7/582: link d3/f6 d3/d14/fc4 0 2024-02-16T22:22:03.691 INFO:tasks.workunit.client.0.smithi114.stdout:6/550: dwrite d6/dd/d25/f99 [0,4194304] 0 2024-02-16T22:22:03.698 INFO:tasks.workunit.client.0.smithi114.stdout:4/630: readlink dd/d1b/d41/dc3/lcc 0 2024-02-16T22:22:03.700 INFO:tasks.workunit.client.0.smithi114.stdout:5/493: mkdir d0/df/d16/d1e/d63/d96 0 2024-02-16T22:22:03.701 INFO:tasks.workunit.client.0.smithi114.stdout:7/583: creat d3/d92/fc5 x:0 0 0 2024-02-16T22:22:03.702 INFO:tasks.workunit.client.0.smithi114.stdout:6/551: mknod d6/d2c/d38/d54/ca7 0 2024-02-16T22:22:03.708 INFO:tasks.workunit.client.0.smithi114.stdout:1/504: link d4/d63/d69/d7c/d9b/db0/f38 d4/d9/d46/d47/db3/fb4 0 2024-02-16T22:22:03.709 INFO:tasks.workunit.client.0.smithi114.stdout:1/505: write d4/d63/d69/d72/f97 [153515,65356] 0 2024-02-16T22:22:03.711 INFO:tasks.workunit.client.0.smithi114.stdout:7/584: creat d3/d22/d73/da7/d6f/d89/da6/fc6 x:0 0 0 2024-02-16T22:22:03.713 INFO:tasks.workunit.client.1.smithi123.stdout:9/792: sync 2024-02-16T22:22:03.715 INFO:tasks.workunit.client.1.smithi123.stdout:9/793: chown d4/d36/d8e/d6d/d71/fab 35 1 2024-02-16T22:22:03.720 INFO:tasks.workunit.client.0.smithi114.stdout:1/506: mknod d4/d9/d3d/d3f/d79/cb5 0 2024-02-16T22:22:03.721 INFO:tasks.workunit.client.0.smithi114.stdout:1/507: chown d4/d9/d46/d47/db3/fb4 4916 1 2024-02-16T22:22:03.722 INFO:tasks.workunit.client.0.smithi114.stdout:7/585: mknod d3/d22/d73/da7/d6f/cc7 0 2024-02-16T22:22:03.728 INFO:tasks.workunit.client.0.smithi114.stdout:7/586: creat d3/d22/d73/da7/d4e/fc8 x:0 0 0 2024-02-16T22:22:03.730 INFO:tasks.workunit.client.0.smithi114.stdout:5/494: getdents d0/df/d16 0 2024-02-16T22:22:03.731 INFO:tasks.workunit.client.0.smithi114.stdout:1/508: mknod d4/d63/d69/d7c/d9b/dae/cb6 0 2024-02-16T22:22:03.733 INFO:tasks.workunit.client.1.smithi123.stdout:6/685: rename d0/d6/d8e/fa0 to d0/d96/de3/ff7 0 2024-02-16T22:22:03.733 INFO:tasks.workunit.client.1.smithi123.stdout:6/686: chown d0/d9/dc/d39/fa9 0 1 2024-02-16T22:22:03.734 INFO:tasks.workunit.client.0.smithi114.stdout:7/587: creat d3/d22/d61/d6a/fc9 x:0 0 0 2024-02-16T22:22:03.738 INFO:tasks.workunit.client.0.smithi114.stdout:5/495: creat d0/df/d7a/f97 x:0 0 0 2024-02-16T22:22:03.738 INFO:tasks.workunit.client.0.smithi114.stdout:9/549: dwrite d2/d1a/d23/d35/d47/d63/f73 [0,4194304] 0 2024-02-16T22:22:03.746 INFO:tasks.workunit.client.1.smithi123.stdout:9/794: mkdir d4/d36/d8e/d58/d59/d60/deb/da7/dde/dfd 0 2024-02-16T22:22:03.746 INFO:tasks.workunit.client.0.smithi114.stdout:7/588: getdents d3/d92/dbe 0 2024-02-16T22:22:03.750 INFO:tasks.workunit.client.0.smithi114.stdout:5/496: rename d0/df/d16 to d0/d28/d98 0 2024-02-16T22:22:03.751 INFO:tasks.workunit.client.1.smithi123.stdout:8/728: sync 2024-02-16T22:22:03.752 INFO:tasks.workunit.client.1.smithi123.stdout:4/723: sync 2024-02-16T22:22:03.752 INFO:tasks.workunit.client.1.smithi123.stdout:3/687: rename d0/d20/c57 to d0/d20/d29/d9b/dbd/ce8 0 2024-02-16T22:22:03.754 INFO:tasks.workunit.client.1.smithi123.stdout:3/688: write d0/d20/d76/d5d/d60/d93/fbb [869176,75621] 0 2024-02-16T22:22:03.754 INFO:tasks.workunit.client.1.smithi123.stdout:6/687: read d0/f54 [1194025,67103] 0 2024-02-16T22:22:03.754 INFO:tasks.workunit.client.1.smithi123.stdout:3/689: fsync d0/d20/fdf 0 2024-02-16T22:22:03.756 INFO:tasks.workunit.client.0.smithi114.stdout:4/631: write dd/f19 [1103911,112254] 0 2024-02-16T22:22:03.757 INFO:tasks.workunit.client.1.smithi123.stdout:5/648: symlink d6/dd/df/lde 0 2024-02-16T22:22:03.759 INFO:tasks.workunit.client.0.smithi114.stdout:9/550: dread d2/d33/f7d [0,4194304] 0 2024-02-16T22:22:03.761 INFO:tasks.workunit.client.1.smithi123.stdout:2/654: unlink d4/d1a/d1d/c50 0 2024-02-16T22:22:03.762 INFO:tasks.workunit.client.0.smithi114.stdout:9/551: write d2/d1a/d23/d35/d47/f92 [1158154,22506] 0 2024-02-16T22:22:03.762 INFO:tasks.workunit.client.0.smithi114.stdout:4/632: fsync dd/d1b/d37/d99/fd7 0 2024-02-16T22:22:03.765 INFO:tasks.workunit.client.0.smithi114.stdout:6/552: write d6/dd/d25/f27 [2359264,55702] 0 2024-02-16T22:22:03.767 INFO:tasks.workunit.client.0.smithi114.stdout:7/589: dread d3/d14/f4d [0,4194304] 0 2024-02-16T22:22:03.769 INFO:tasks.workunit.client.1.smithi123.stdout:0/604: rename d1/d4/d17/d74/f58 to d1/d4/d17/d54/d8e/da5/fd3 0 2024-02-16T22:22:03.771 INFO:tasks.workunit.client.0.smithi114.stdout:6/553: dwrite d6/d2c/f7c [0,4194304] 0 2024-02-16T22:22:03.772 INFO:tasks.workunit.client.0.smithi114.stdout:9/552: mknod d2/d5c/d5f/d91/cac 0 2024-02-16T22:22:03.772 INFO:tasks.workunit.client.1.smithi123.stdout:8/729: dread d0/d12/f58 [0,4194304] 0 2024-02-16T22:22:03.774 INFO:tasks.workunit.client.0.smithi114.stdout:4/633: creat dd/d2e/d56/ddc/dde/fe8 x:0 0 0 2024-02-16T22:22:03.775 INFO:tasks.workunit.client.1.smithi123.stdout:3/690: mkdir d0/d20/d76/d5d/d60/de9 0 2024-02-16T22:22:03.777 INFO:tasks.workunit.client.0.smithi114.stdout:6/554: write d6/f35 [7186471,6819] 0 2024-02-16T22:22:03.777 INFO:tasks.workunit.client.1.smithi123.stdout:5/649: stat d6/dd/df/d58/d3f/dbe/c69 0 2024-02-16T22:22:03.778 INFO:tasks.workunit.client.0.smithi114.stdout:9/553: rmdir d2/d5c/d5f/d7b 39 2024-02-16T22:22:03.784 INFO:tasks.workunit.client.1.smithi123.stdout:4/724: rename d5/d1c/d5e/dc3/f75 to d5/d1c/d23/d56/d6e/d73/d87/fdd 0 2024-02-16T22:22:03.789 INFO:tasks.workunit.client.1.smithi123.stdout:7/680: getdents d3/d6e/d52 0 2024-02-16T22:22:03.789 INFO:tasks.workunit.client.1.smithi123.stdout:4/725: dwrite d5/d1c/f25 [0,4194304] 0 2024-02-16T22:22:03.791 INFO:tasks.workunit.client.0.smithi114.stdout:1/509: truncate d4/d9/d3d/f39 2404648 0 2024-02-16T22:22:03.791 INFO:tasks.workunit.client.1.smithi123.stdout:8/730: rmdir d0/d61 39 2024-02-16T22:22:03.791 INFO:tasks.workunit.client.1.smithi123.stdout:7/681: truncate d3/d6e/d4d/d92/fd1 577085 0 2024-02-16T22:22:03.792 INFO:tasks.workunit.client.1.smithi123.stdout:7/682: readlink d3/d59/lcc 0 2024-02-16T22:22:03.792 INFO:tasks.workunit.client.1.smithi123.stdout:7/683: stat d3/d6e/d1d/db2/d35/db7 0 2024-02-16T22:22:03.793 INFO:tasks.workunit.client.0.smithi114.stdout:9/554: rename d2/dd/f2a to d2/d1a/d23/d32/d46/fad 0 2024-02-16T22:22:03.793 INFO:tasks.workunit.client.1.smithi123.stdout:1/661: write d3/d6/d49/f50 [382010,95322] 0 2024-02-16T22:22:03.795 INFO:tasks.workunit.client.1.smithi123.stdout:3/691: creat d0/d20/d76/d5d/d60/dc6/fea x:0 0 0 2024-02-16T22:22:03.798 INFO:tasks.workunit.client.0.smithi114.stdout:6/555: dread d6/d2c/d38/d54/f2d [0,4194304] 0 2024-02-16T22:22:03.799 INFO:tasks.workunit.client.1.smithi123.stdout:2/655: symlink d4/d1a/d1d/d13/d25/d98/dbb/ld3 0 2024-02-16T22:22:03.800 INFO:tasks.workunit.client.0.smithi114.stdout:5/497: write d0/d28/f46 [466227,119663] 0 2024-02-16T22:22:03.801 INFO:tasks.workunit.client.0.smithi114.stdout:1/510: mknod d4/d63/d69/d72/d82/cb7 0 2024-02-16T22:22:03.804 INFO:tasks.workunit.client.0.smithi114.stdout:9/555: creat d2/d5c/d5f/d91/fae x:0 0 0 2024-02-16T22:22:03.808 INFO:tasks.workunit.client.1.smithi123.stdout:4/726: rename d5/f4c to d5/d1c/d23/d47/fde 0 2024-02-16T22:22:03.810 INFO:tasks.workunit.client.1.smithi123.stdout:4/727: stat d5/d1c/d23/d47/da2/daa/cd2 0 2024-02-16T22:22:03.810 INFO:tasks.workunit.client.1.smithi123.stdout:9/795: dwrite d4/d36/d57/db2/f28 [0,4194304] 0 2024-02-16T22:22:03.812 INFO:tasks.workunit.client.0.smithi114.stdout:9/556: unlink d2/d33/d3f/c60 0 2024-02-16T22:22:03.821 INFO:tasks.workunit.client.0.smithi114.stdout:5/498: mkdir d0/d28/d98/d25/d89/d99 0 2024-02-16T22:22:03.821 INFO:tasks.workunit.client.0.smithi114.stdout:4/634: dwrite dd/d1b/d24/f36 [4194304,4194304] 0 2024-02-16T22:22:03.822 INFO:tasks.workunit.client.1.smithi123.stdout:5/650: creat d6/dc1/fdf x:0 0 0 2024-02-16T22:22:03.824 INFO:tasks.workunit.client.0.smithi114.stdout:1/511: link d4/d63/d69/f7f d4/d63/d69/d72/d82/fb8 0 2024-02-16T22:22:03.829 INFO:tasks.workunit.client.0.smithi114.stdout:9/557: symlink d2/d5c/d5f/d72/laf 0 2024-02-16T22:22:03.832 INFO:tasks.workunit.client.1.smithi123.stdout:2/656: dread d4/d1a/d1d/d38/f58 [0,4194304] 0 2024-02-16T22:22:03.833 INFO:tasks.workunit.client.0.smithi114.stdout:4/635: creat dd/d1b/d41/dc3/fe9 x:0 0 0 2024-02-16T22:22:03.835 INFO:tasks.workunit.client.1.smithi123.stdout:8/731: mkdir d0/d29/da7/dea 0 2024-02-16T22:22:03.836 INFO:tasks.workunit.client.0.smithi114.stdout:6/556: truncate d6/d83/f87 2090505 0 2024-02-16T22:22:03.837 INFO:tasks.workunit.client.0.smithi114.stdout:1/512: mknod d4/d63/d69/d7c/cb9 0 2024-02-16T22:22:03.838 INFO:tasks.workunit.client.1.smithi123.stdout:4/728: symlink d5/d1c/d23/d74/d92/ldf 0 2024-02-16T22:22:03.841 INFO:tasks.workunit.client.0.smithi114.stdout:5/499: mkdir d0/d28/d98/d1e/d63/d96/d9a 0 2024-02-16T22:22:03.842 INFO:tasks.workunit.client.0.smithi114.stdout:6/557: dwrite d6/dd/d25/d34/f42 [0,4194304] 0 2024-02-16T22:22:03.844 INFO:tasks.workunit.client.0.smithi114.stdout:4/636: mkdir dd/d59/d8f/dea 0 2024-02-16T22:22:03.844 INFO:tasks.workunit.client.0.smithi114.stdout:4/637: stat dd/d1b 0 2024-02-16T22:22:03.850 INFO:tasks.workunit.client.1.smithi123.stdout:3/692: link d0/d20/d76/f35 d0/d20/d94/feb 0 2024-02-16T22:22:03.853 INFO:tasks.workunit.client.0.smithi114.stdout:1/513: creat d4/d9/d65/d98/fba x:0 0 0 2024-02-16T22:22:03.855 INFO:tasks.workunit.client.1.smithi123.stdout:0/605: getdents d1/d4 0 2024-02-16T22:22:03.855 INFO:tasks.workunit.client.0.smithi114.stdout:1/514: write d4/d9/d46/d47/fa4 [140166,107167] 0 2024-02-16T22:22:03.855 INFO:tasks.workunit.client.0.smithi114.stdout:5/500: mknod d0/d28/d98/d1e/d86/c9b 0 2024-02-16T22:22:03.857 INFO:tasks.workunit.client.1.smithi123.stdout:2/657: write d4/d1a/d1d/d59/d45/fc5 [169707,54593] 0 2024-02-16T22:22:03.857 INFO:tasks.workunit.client.0.smithi114.stdout:1/515: write d4/d63/d69/d80/fb1 [395823,25815] 0 2024-02-16T22:22:03.857 INFO:tasks.workunit.client.1.smithi123.stdout:3/693: dwrite d0/d40/f56 [0,4194304] 0 2024-02-16T22:22:03.860 INFO:tasks.workunit.client.1.smithi123.stdout:8/732: rename d0/cd4 to d0/d29/da7/d26/d2a/d4d/ceb 0 2024-02-16T22:22:03.861 INFO:tasks.workunit.client.0.smithi114.stdout:4/638: mknod dd/d1b/d37/d38/d87/ceb 0 2024-02-16T22:22:03.864 INFO:tasks.workunit.client.1.smithi123.stdout:4/729: creat d5/d1c/d64/fe0 x:0 0 0 2024-02-16T22:22:03.865 INFO:tasks.workunit.client.1.smithi123.stdout:4/730: write d5/f63 [1446535,9371] 0 2024-02-16T22:22:03.866 INFO:tasks.workunit.client.0.smithi114.stdout:5/501: dwrite d0/d28/d98/d25/d2e/d3b/f77 [0,4194304] 0 2024-02-16T22:22:03.866 INFO:tasks.workunit.client.1.smithi123.stdout:4/731: fdatasync d5/d1c/fac 0 2024-02-16T22:22:03.872 INFO:tasks.workunit.client.0.smithi114.stdout:9/558: getdents d2/d1a/d23 0 2024-02-16T22:22:03.874 INFO:tasks.workunit.client.0.smithi114.stdout:4/639: symlink dd/d1b/d24/lec 0 2024-02-16T22:22:03.875 INFO:tasks.workunit.client.1.smithi123.stdout:1/662: creat d3/d6/db/d10/fde x:0 0 0 2024-02-16T22:22:03.875 INFO:tasks.workunit.client.0.smithi114.stdout:5/502: creat d0/df/d5e/f9c x:0 0 0 2024-02-16T22:22:03.878 INFO:tasks.workunit.client.0.smithi114.stdout:9/559: unlink d2/dd/d3c/f59 0 2024-02-16T22:22:03.881 INFO:tasks.workunit.client.1.smithi123.stdout:0/606: symlink d1/d4/ld4 0 2024-02-16T22:22:03.881 INFO:tasks.workunit.client.1.smithi123.stdout:0/607: chown d1/f12 117517 1 2024-02-16T22:22:03.882 INFO:tasks.workunit.client.0.smithi114.stdout:4/640: mknod dd/d2e/d56/ddc/ced 0 2024-02-16T22:22:03.882 INFO:tasks.workunit.client.0.smithi114.stdout:5/503: dread d0/d28/d98/d25/d2e/f67 [0,4194304] 0 2024-02-16T22:22:03.884 INFO:tasks.workunit.client.0.smithi114.stdout:1/516: dread d4/d9/f22 [0,4194304] 0 2024-02-16T22:22:03.889 INFO:tasks.workunit.client.0.smithi114.stdout:1/517: dwrite d4/d9/d3d/f41 [0,4194304] 0 2024-02-16T22:22:03.893 INFO:tasks.workunit.client.1.smithi123.stdout:6/688: sync 2024-02-16T22:22:03.929 INFO:tasks.workunit.client.0.smithi114.stdout:4/641: chown dd/d1b/d24/l5a 233601140 1 2024-02-16T22:22:03.931 INFO:tasks.workunit.client.1.smithi123.stdout:4/732: creat d5/d1c/d23/d69/fe1 x:0 0 0 2024-02-16T22:22:03.932 INFO:tasks.workunit.client.0.smithi114.stdout:5/504: creat d0/d28/d98/d25/d89/f9d x:0 0 0 2024-02-16T22:22:03.934 INFO:tasks.workunit.client.1.smithi123.stdout:7/684: getdents d3/d6e/d34 0 2024-02-16T22:22:03.936 INFO:tasks.workunit.client.1.smithi123.stdout:9/796: truncate d4/d36/d8e/d6d/d71/da6/fee 153798 0 2024-02-16T22:22:03.936 INFO:tasks.workunit.client.0.smithi114.stdout:1/518: write d4/d9/f22 [1915539,101873] 0 2024-02-16T22:22:03.937 INFO:tasks.workunit.client.0.smithi114.stdout:6/558: truncate d6/d2c/d38/d54/f3d 832552 0 2024-02-16T22:22:03.940 INFO:tasks.workunit.client.1.smithi123.stdout:0/608: chown d1/d8/lbb 331 1 2024-02-16T22:22:03.943 INFO:tasks.workunit.client.0.smithi114.stdout:4/642: mknod dd/d1b/d37/d38/d87/dad/cee 0 2024-02-16T22:22:03.945 INFO:tasks.workunit.client.0.smithi114.stdout:5/505: mkdir d0/d28/d98/d1e/d63/d9e 0 2024-02-16T22:22:03.946 INFO:tasks.workunit.client.1.smithi123.stdout:2/658: mkdir d4/d1a/d1d/d13/d15/d6d/d4a/dce/dd4 0 2024-02-16T22:22:03.946 INFO:tasks.workunit.client.1.smithi123.stdout:2/659: dread - d4/fa1 zero size 2024-02-16T22:22:03.947 INFO:tasks.workunit.client.1.smithi123.stdout:8/733: creat d0/d29/da7/dea/fec x:0 0 0 2024-02-16T22:22:03.948 INFO:tasks.workunit.client.0.smithi114.stdout:9/560: fsync d2/d1a/d65/f71 0 2024-02-16T22:22:03.950 INFO:tasks.workunit.client.1.smithi123.stdout:8/734: write d0/d29/da7/d26/d68/d81/d47/d9b/fc5 [670195,92241] 0 2024-02-16T22:22:03.950 INFO:tasks.workunit.client.0.smithi114.stdout:4/643: chown dd/d2e/f6a 242530 1 2024-02-16T22:22:03.952 INFO:tasks.workunit.client.1.smithi123.stdout:3/694: dwrite d0/f58 [0,4194304] 0 2024-02-16T22:22:03.953 INFO:tasks.workunit.client.1.smithi123.stdout:3/695: readlink d0/d20/d76/d37/l66 0 2024-02-16T22:22:03.954 INFO:tasks.workunit.client.1.smithi123.stdout:5/651: creat d6/dd/df/d58/d3f/dbe/d37/fe0 x:0 0 0 2024-02-16T22:22:03.955 INFO:tasks.workunit.client.0.smithi114.stdout:1/519: link d4/d9/d46/d47/fa2 d4/d63/d69/d72/d82/fbb 0 2024-02-16T22:22:03.956 INFO:tasks.workunit.client.0.smithi114.stdout:4/644: dwrite dd/d2e/d3f/f75 [0,4194304] 0 2024-02-16T22:22:03.957 INFO:tasks.workunit.client.1.smithi123.stdout:8/735: dread d0/d29/da7/d26/d68/d81/d47/d9b/fb0 [4194304,4194304] 0 2024-02-16T22:22:03.958 INFO:tasks.workunit.client.1.smithi123.stdout:9/797: symlink d4/lfe 0 2024-02-16T22:22:03.963 INFO:tasks.workunit.client.1.smithi123.stdout:0/609: symlink d1/d4/d42/ld5 0 2024-02-16T22:22:03.966 INFO:tasks.workunit.client.0.smithi114.stdout:1/520: unlink d4/d9/c64 0 2024-02-16T22:22:03.968 INFO:tasks.workunit.client.0.smithi114.stdout:9/561: mknod d2/d1a/d23/d35/cb0 0 2024-02-16T22:22:03.969 INFO:tasks.workunit.client.1.smithi123.stdout:1/663: dwrite d3/d47/f7d [0,4194304] 0 2024-02-16T22:22:03.969 INFO:tasks.workunit.client.0.smithi114.stdout:9/562: read - d2/d1a/d23/d32/d46/f90 zero size 2024-02-16T22:22:03.970 INFO:tasks.workunit.client.0.smithi114.stdout:9/563: stat d2/d1a/d23/d35 0 2024-02-16T22:22:03.973 INFO:tasks.workunit.client.1.smithi123.stdout:5/652: mknod d6/d3b/ce1 0 2024-02-16T22:22:03.974 INFO:tasks.workunit.client.0.smithi114.stdout:4/645: fdatasync dd/d1b/d24/f3d 0 2024-02-16T22:22:03.976 INFO:tasks.workunit.client.1.smithi123.stdout:3/696: dwrite d0/d20/d76/d5d/d60/f90 [0,4194304] 0 2024-02-16T22:22:03.976 INFO:tasks.workunit.client.1.smithi123.stdout:7/685: dread d3/d6e/d34/f3c [0,4194304] 0 2024-02-16T22:22:03.977 INFO:tasks.workunit.client.0.smithi114.stdout:1/521: chown d4/d9/d46/d4d/f4c 74550 1 2024-02-16T22:22:03.978 INFO:tasks.workunit.client.1.smithi123.stdout:8/736: unlink d0/d29/da7/d26/d32/d45/d67/c71 0 2024-02-16T22:22:03.979 INFO:tasks.workunit.client.0.smithi114.stdout:9/564: dwrite d2/d1a/d23/d32/d46/f9d [0,4194304] 0 2024-02-16T22:22:03.980 INFO:tasks.workunit.client.0.smithi114.stdout:1/522: truncate d4/d63/d69/d7c/d9b/db0/f43 985882 0 2024-02-16T22:22:03.982 INFO:tasks.workunit.client.1.smithi123.stdout:9/798: dwrite d4/d36/daa/d20/f22 [0,4194304] 0 2024-02-16T22:22:03.991 INFO:tasks.workunit.client.1.smithi123.stdout:0/610: rename d1/d8/l77 to d1/d8/d13/d1d/ld6 0 2024-02-16T22:22:03.992 INFO:tasks.workunit.client.0.smithi114.stdout:1/523: creat d4/d9/d65/fbc x:0 0 0 2024-02-16T22:22:03.992 INFO:tasks.workunit.client.1.smithi123.stdout:6/689: write d0/d6/d91/d93/f95 [704834,115902] 0 2024-02-16T22:22:03.994 INFO:tasks.workunit.client.1.smithi123.stdout:6/690: write d0/d74/fce [112477,32747] 0 2024-02-16T22:22:03.999 INFO:tasks.workunit.client.0.smithi114.stdout:9/565: rename d2/d5c/d5f/d7b/f22 to d2/d1a/d23/d32/d46/d78/fb1 0 2024-02-16T22:22:03.999 INFO:tasks.workunit.client.1.smithi123.stdout:2/660: dwrite d4/d1a/d1d/d38/f92 [0,4194304] 0 2024-02-16T22:22:04.000 INFO:tasks.workunit.client.0.smithi114.stdout:1/524: creat d4/d63/d69/d7c/fbd x:0 0 0 2024-02-16T22:22:04.006 INFO:tasks.workunit.client.0.smithi114.stdout:9/566: mkdir d2/d1a/d23/d35/d47/db2 0 2024-02-16T22:22:04.007 INFO:tasks.workunit.client.0.smithi114.stdout:1/525: rename d4/d9/d46/d47/l9f to d4/d9/d46/d4d/lbe 0 2024-02-16T22:22:04.008 INFO:tasks.workunit.client.1.smithi123.stdout:4/733: getdents d5/d1c/d23/d74 0 2024-02-16T22:22:04.009 INFO:tasks.workunit.client.1.smithi123.stdout:0/611: mknod d1/d4/d17/d54/d8e/d96/cd7 0 2024-02-16T22:22:04.011 INFO:tasks.workunit.client.1.smithi123.stdout:0/612: fdatasync d1/d4/d17/d54/d8e/d96/f9e 0 2024-02-16T22:22:04.011 INFO:tasks.workunit.client.1.smithi123.stdout:1/664: unlink d3/d6/db/d10/c1b 0 2024-02-16T22:22:04.013 INFO:tasks.workunit.client.1.smithi123.stdout:5/653: mknod d6/dd/ce2 0 2024-02-16T22:22:04.016 INFO:tasks.workunit.client.1.smithi123.stdout:2/661: mkdir d4/d1a/dd5 0 2024-02-16T22:22:04.018 INFO:tasks.workunit.client.0.smithi114.stdout:4/646: write dd/d1b/d41/d74/d9b/fbc [1163636,56270] 0 2024-02-16T22:22:04.021 INFO:tasks.workunit.client.0.smithi114.stdout:1/526: rename d4/l32 to d4/lbf 0 2024-02-16T22:22:04.023 INFO:tasks.workunit.client.1.smithi123.stdout:8/737: unlink d0/d61/d66/f98 0 2024-02-16T22:22:04.026 INFO:tasks.workunit.client.0.smithi114.stdout:1/527: mkdir d4/d63/d69/d7c/dc0 0 2024-02-16T22:22:04.027 INFO:tasks.workunit.client.0.smithi114.stdout:1/528: fdatasync d4/d9/d46/d67/f6e 0 2024-02-16T22:22:04.027 INFO:tasks.workunit.client.1.smithi123.stdout:4/734: dread d5/d1c/d23/d47/f50 [0,4194304] 0 2024-02-16T22:22:04.032 INFO:tasks.workunit.client.0.smithi114.stdout:9/567: dwrite d2/d1a/f39 [0,4194304] 0 2024-02-16T22:22:04.033 INFO:tasks.workunit.client.0.smithi114.stdout:4/647: dread dd/d1b/d37/d38/f5e [0,4194304] 0 2024-02-16T22:22:04.033 INFO:tasks.workunit.client.0.smithi114.stdout:1/529: mknod d4/d63/d69/d72/cc1 0 2024-02-16T22:22:04.035 INFO:tasks.workunit.client.1.smithi123.stdout:6/691: creat d0/d96/dca/ff8 x:0 0 0 2024-02-16T22:22:04.040 INFO:tasks.workunit.client.1.smithi123.stdout:1/665: fdatasync d3/d6/db/d10/f29 0 2024-02-16T22:22:04.042 INFO:tasks.workunit.client.1.smithi123.stdout:7/686: truncate d3/d6e/d1d/fb6 333223 0 2024-02-16T22:22:04.043 INFO:tasks.workunit.client.1.smithi123.stdout:5/654: dread - d6/d3b/fa7 zero size 2024-02-16T22:22:04.044 INFO:tasks.workunit.client.0.smithi114.stdout:4/648: rename dd/d1b/d24/c44 to dd/d2e/d56/cef 0 2024-02-16T22:22:04.045 INFO:tasks.workunit.client.1.smithi123.stdout:9/799: write d4/f1b [852204,33187] 0 2024-02-16T22:22:04.045 INFO:tasks.workunit.client.0.smithi114.stdout:4/649: chown dd/d1b/d41/fe5 197042 1 2024-02-16T22:22:04.048 INFO:tasks.workunit.client.0.smithi114.stdout:9/568: mknod d2/d1a/d23/d32/cb3 0 2024-02-16T22:22:04.120 INFO:tasks.workunit.client.1.smithi123.stdout:5/655: sync 2024-02-16T22:22:04.165 INFO:tasks.workunit.client.1.smithi123.stdout:8/738: readlink d0/d29/da7/d26/d2a/d3c/d62/lb9 0 2024-02-16T22:22:04.169 INFO:tasks.workunit.client.1.smithi123.stdout:4/735: unlink d5/d1c/d64/f8b 0 2024-02-16T22:22:04.170 INFO:tasks.workunit.client.0.smithi114.stdout:9/569: symlink d2/d1a/d23/lb4 0 2024-02-16T22:22:04.173 INFO:tasks.workunit.client.1.smithi123.stdout:0/613: mknod d1/d8/d3d/cd8 0 2024-02-16T22:22:04.173 INFO:tasks.workunit.client.1.smithi123.stdout:0/614: fdatasync d1/d4/d17/d54/f99 0 2024-02-16T22:22:04.174 INFO:tasks.workunit.client.0.smithi114.stdout:1/530: dwrite d4/d9/d3d/d3f/f75 [0,4194304] 0 2024-02-16T22:22:04.175 INFO:tasks.workunit.client.1.smithi123.stdout:6/692: symlink d0/d9/dc/d3a/d52/lf9 0 2024-02-16T22:22:04.175 INFO:tasks.workunit.client.0.smithi114.stdout:4/650: rename dd/d59/l93 to dd/lf0 0 2024-02-16T22:22:04.175 INFO:tasks.workunit.client.1.smithi123.stdout:6/693: chown d0/d96/de3/f79 889017441 1 2024-02-16T22:22:04.176 INFO:tasks.workunit.client.0.smithi114.stdout:4/651: fdatasync dd/d2e/fdf 0 2024-02-16T22:22:04.179 INFO:tasks.workunit.client.0.smithi114.stdout:1/531: rename d4/l96 to d4/d9/d46/d47/db3/lc2 0 2024-02-16T22:22:04.181 INFO:tasks.workunit.client.0.smithi114.stdout:4/652: creat dd/d1b/d37/d38/d87/dad/ff1 x:0 0 0 2024-02-16T22:22:04.182 INFO:tasks.workunit.client.0.smithi114.stdout:4/653: chown dd/d1b/f55 464776 1 2024-02-16T22:22:04.182 INFO:tasks.workunit.client.1.smithi123.stdout:3/697: link d0/da0/cb7 d0/d40/cec 0 2024-02-16T22:22:04.187 INFO:tasks.workunit.client.1.smithi123.stdout:7/687: write d3/f4e [2654346,125293] 0 2024-02-16T22:22:04.188 INFO:tasks.workunit.client.1.smithi123.stdout:7/688: fdatasync d3/d6e/d34/f57 0 2024-02-16T22:22:04.188 INFO:tasks.workunit.client.1.smithi123.stdout:4/736: symlink d5/d1c/d5e/le2 0 2024-02-16T22:22:04.190 INFO:tasks.workunit.client.0.smithi114.stdout:4/654: link dd/d1b/d41/d77/fb1 dd/d2e/d56/ddc/ff2 0 2024-02-16T22:22:04.192 INFO:tasks.workunit.client.0.smithi114.stdout:1/532: getdents d4/d63/d69 0 2024-02-16T22:22:04.193 INFO:tasks.workunit.client.1.smithi123.stdout:0/615: dread d1/f1f [4194304,4194304] 0 2024-02-16T22:22:04.195 INFO:tasks.workunit.client.0.smithi114.stdout:4/655: creat dd/d1b/d41/dc3/ff3 x:0 0 0 2024-02-16T22:22:04.196 INFO:tasks.workunit.client.0.smithi114.stdout:1/533: creat d4/d9/d3d/d3f/d79/fc3 x:0 0 0 2024-02-16T22:22:04.197 INFO:tasks.workunit.client.1.smithi123.stdout:8/739: dwrite d0/d29/da7/d26/d68/d81/d47/f73 [0,4194304] 0 2024-02-16T22:22:04.198 INFO:tasks.workunit.client.1.smithi123.stdout:1/666: mkdir d3/d6/db/d10/d4f/d9f/db4/ddf 0 2024-02-16T22:22:04.198 INFO:tasks.workunit.client.0.smithi114.stdout:4/656: mkdir dd/d1b/d37/d61/df4 0 2024-02-16T22:22:04.199 INFO:tasks.workunit.client.1.smithi123.stdout:8/740: write d0/d29/da7/d26/d68/d81/d47/d9b/fa8 [857619,65080] 0 2024-02-16T22:22:04.200 INFO:tasks.workunit.client.1.smithi123.stdout:1/667: write d3/f7 [3390357,71133] 0 2024-02-16T22:22:04.200 INFO:tasks.workunit.client.1.smithi123.stdout:8/741: chown d0/d29/da7/d26/f7a 44158 1 2024-02-16T22:22:04.202 INFO:tasks.workunit.client.0.smithi114.stdout:1/534: mknod d4/d9/d46/d47/db3/cc4 0 2024-02-16T22:22:04.203 INFO:tasks.workunit.client.1.smithi123.stdout:6/694: truncate d0/d9/dc/d3a/f80 706649 0 2024-02-16T22:22:04.204 INFO:tasks.workunit.client.1.smithi123.stdout:9/800: mkdir d4/d36/d8e/dff 0 2024-02-16T22:22:04.204 INFO:tasks.workunit.client.0.smithi114.stdout:1/535: read d4/d9/d3d/d3f/f75 [2403007,43028] 0 2024-02-16T22:22:04.205 INFO:tasks.workunit.client.0.smithi114.stdout:4/657: dwrite dd/d1b/d24/fc8 [0,4194304] 0 2024-02-16T22:22:04.208 INFO:tasks.workunit.client.1.smithi123.stdout:3/698: mknod d0/d20/d76/d5d/dae/dc7/ced 0 2024-02-16T22:22:04.209 INFO:tasks.workunit.client.0.smithi114.stdout:1/536: symlink d4/d63/d69/d72/d82/lc5 0 2024-02-16T22:22:04.209 INFO:tasks.workunit.client.0.smithi114.stdout:4/658: dread dd/d2e/d3f/fa6 [0,4194304] 0 2024-02-16T22:22:04.210 INFO:tasks.workunit.client.1.smithi123.stdout:5/656: symlink d6/dd/df/d1f/le3 0 2024-02-16T22:22:04.216 INFO:tasks.workunit.client.0.smithi114.stdout:4/659: dwrite dd/d1b/d41/dc3/ff3 [0,4194304] 0 2024-02-16T22:22:04.216 INFO:tasks.workunit.client.1.smithi123.stdout:5/657: dwrite d6/dd/df/fcf [0,4194304] 0 2024-02-16T22:22:04.219 INFO:tasks.workunit.client.1.smithi123.stdout:4/737: creat d5/d1c/d23/d74/d92/d9b/fe3 x:0 0 0 2024-02-16T22:22:04.223 INFO:tasks.workunit.client.1.smithi123.stdout:7/689: dread d3/d6e/d1d/fb6 [0,4194304] 0 2024-02-16T22:22:04.226 INFO:tasks.workunit.client.0.smithi114.stdout:1/537: creat d4/d63/d69/d7c/d9b/db0/fc6 x:0 0 0 2024-02-16T22:22:04.228 INFO:tasks.workunit.client.1.smithi123.stdout:1/668: symlink d3/d6/d18/le0 0 2024-02-16T22:22:04.229 INFO:tasks.workunit.client.1.smithi123.stdout:8/742: write d0/d29/da7/d26/d68/d81/d47/f91 [563892,82726] 0 2024-02-16T22:22:04.230 INFO:tasks.workunit.client.0.smithi114.stdout:1/538: creat d4/d63/d69/d72/d82/fc7 x:0 0 0 2024-02-16T22:22:04.232 INFO:tasks.workunit.client.1.smithi123.stdout:6/695: unlink d0/d6/d1a/lcf 0 2024-02-16T22:22:04.233 INFO:tasks.workunit.client.1.smithi123.stdout:2/662: mknod d4/d1a/d1d/d13/d15/d6d/d4a/dce/dd4/cd6 0 2024-02-16T22:22:04.236 INFO:tasks.workunit.client.1.smithi123.stdout:3/699: write d0/d20/d76/fe5 [5345919,24014] 0 2024-02-16T22:22:04.239 INFO:tasks.workunit.client.1.smithi123.stdout:5/658: creat d6/dac/fe4 x:0 0 0 2024-02-16T22:22:04.241 INFO:tasks.workunit.client.1.smithi123.stdout:5/659: dread - d6/dd/df/d9f/fd7 zero size 2024-02-16T22:22:04.244 INFO:tasks.workunit.client.1.smithi123.stdout:0/616: dwrite d1/d4/d17/f53 [0,4194304] 0 2024-02-16T22:22:04.246 INFO:tasks.workunit.client.0.smithi114.stdout:1/539: dwrite d4/d9/d65/d98/fba [0,4194304] 0 2024-02-16T22:22:04.248 INFO:tasks.workunit.client.0.smithi114.stdout:4/660: dwrite dd/d2e/d56/fd4 [0,4194304] 0 2024-02-16T22:22:04.250 INFO:tasks.workunit.client.1.smithi123.stdout:6/696: truncate d0/d6/f5c 525704 0 2024-02-16T22:22:04.253 INFO:tasks.workunit.client.1.smithi123.stdout:4/738: dread d5/d1c/f25 [4194304,4194304] 0 2024-02-16T22:22:04.253 INFO:tasks.workunit.client.1.smithi123.stdout:5/660: dwrite d6/dd/df/d58/d3f/dbe/da2/faf [0,4194304] 0 2024-02-16T22:22:04.256 INFO:tasks.workunit.client.0.smithi114.stdout:4/661: dwrite dd/d1b/d24/f36 [4194304,4194304] 0 2024-02-16T22:22:04.257 INFO:tasks.workunit.client.1.smithi123.stdout:9/801: creat d4/d36/d8e/d58/d59/d60/deb/da7/dde/dfd/f100 x:0 0 0 2024-02-16T22:22:04.258 INFO:tasks.workunit.client.0.smithi114.stdout:4/662: dread - dd/d1b/d41/dc3/fe9 zero size 2024-02-16T22:22:04.262 INFO:tasks.workunit.client.1.smithi123.stdout:1/669: dwrite d3/d6/f5b [0,4194304] 0 2024-02-16T22:22:04.264 INFO:tasks.workunit.client.1.smithi123.stdout:3/700: dread d0/d20/f52 [0,4194304] 0 2024-02-16T22:22:04.267 INFO:tasks.workunit.client.0.smithi114.stdout:1/540: rmdir d4/d9/d3d/d3f/d79/daa 0 2024-02-16T22:22:04.268 INFO:tasks.workunit.client.1.smithi123.stdout:0/617: unlink d1/d8/d3d/la1 0 2024-02-16T22:22:04.269 INFO:tasks.workunit.client.0.smithi114.stdout:1/541: mknod d4/d9/d3d/d3f/cc8 0 2024-02-16T22:22:04.271 INFO:tasks.workunit.client.1.smithi123.stdout:0/618: fsync d1/d4/d17/d54/f99 0 2024-02-16T22:22:04.272 INFO:tasks.workunit.client.1.smithi123.stdout:0/619: stat d1/da0 0 2024-02-16T22:22:04.272 INFO:tasks.workunit.client.1.smithi123.stdout:2/663: mkdir d4/d1a/d1d/d59/dd7 0 2024-02-16T22:22:04.273 INFO:tasks.workunit.client.0.smithi114.stdout:1/542: readlink d4/d9/d46/d4d/lbe 0 2024-02-16T22:22:04.274 INFO:tasks.workunit.client.1.smithi123.stdout:5/661: fdatasync d6/f4c 0 2024-02-16T22:22:04.275 INFO:tasks.workunit.client.1.smithi123.stdout:3/701: dwrite d0/d20/d76/d5d/d60/dd6/fd9 [0,4194304] 0 2024-02-16T22:22:04.276 INFO:tasks.workunit.client.1.smithi123.stdout:6/697: symlink d0/d6/d8e/lfa 0 2024-02-16T22:22:04.277 INFO:tasks.workunit.client.1.smithi123.stdout:3/702: chown d0/d50/d75/f7d 10914 1 2024-02-16T22:22:04.277 INFO:tasks.workunit.client.1.smithi123.stdout:5/662: write d6/dac/fe4 [397801,19925] 0 2024-02-16T22:22:04.278 INFO:tasks.workunit.client.0.smithi114.stdout:1/543: mknod d4/d63/d69/d7c/dc0/cc9 0 2024-02-16T22:22:04.279 INFO:tasks.workunit.client.1.smithi123.stdout:3/703: fsync d0/d20/d76/d5d/f82 0 2024-02-16T22:22:04.282 INFO:tasks.workunit.client.1.smithi123.stdout:2/664: dwrite d4/d1a/d1d/d59/d45/fc5 [0,4194304] 0 2024-02-16T22:22:04.283 INFO:tasks.workunit.client.1.smithi123.stdout:8/743: rename d0/d61/d66/dc0 to d0/d12/ded 0 2024-02-16T22:22:04.286 INFO:tasks.workunit.client.1.smithi123.stdout:0/620: unlink d1/d4/l31 0 2024-02-16T22:22:04.287 INFO:tasks.workunit.client.0.smithi114.stdout:1/544: dwrite d4/d9/d46/d67/d92/fa6 [0,4194304] 0 2024-02-16T22:22:04.287 INFO:tasks.workunit.client.1.smithi123.stdout:2/665: dread d4/d1a/d1d/d13/d25/f9a [0,4194304] 0 2024-02-16T22:22:04.289 INFO:tasks.workunit.client.1.smithi123.stdout:2/666: write d4/d1a/d1d/d13/f36 [4293372,40497] 0 2024-02-16T22:22:04.291 INFO:tasks.workunit.client.1.smithi123.stdout:2/667: dread - d4/d1a/d1d/d13/d15/d6d/d4a/da2/fcb zero size 2024-02-16T22:22:04.292 INFO:tasks.workunit.client.0.smithi114.stdout:1/545: mkdir d4/d9/d46/d67/d92/dca 0 2024-02-16T22:22:04.294 INFO:tasks.workunit.client.1.smithi123.stdout:1/670: symlink d3/d6/le1 0 2024-02-16T22:22:04.294 INFO:tasks.workunit.client.1.smithi123.stdout:5/663: fdatasync d6/dd/df/d58/d3f/dbe/d37/f6e 0 2024-02-16T22:22:04.295 INFO:tasks.workunit.client.1.smithi123.stdout:3/704: symlink d0/d20/d76/d5d/dae/db0/lee 0 2024-02-16T22:22:04.296 INFO:tasks.workunit.client.1.smithi123.stdout:5/664: fdatasync d6/dd/df/d1f/f6b 0 2024-02-16T22:22:04.296 INFO:tasks.workunit.client.1.smithi123.stdout:1/671: write d3/d6/d18/d9e/fae [3457684,46655] 0 2024-02-16T22:22:04.298 INFO:tasks.workunit.client.1.smithi123.stdout:1/672: chown d3/d6/d18/d9e/fae 1633 1 2024-02-16T22:22:04.300 INFO:tasks.workunit.client.1.smithi123.stdout:6/698: rename d0/d9/f62 to d0/d6/d4d/db0/ffb 0 2024-02-16T22:22:04.300 INFO:tasks.workunit.client.0.smithi114.stdout:1/546: dwrite d4/d9/d46/f48 [0,4194304] 0 2024-02-16T22:22:04.302 INFO:tasks.workunit.client.1.smithi123.stdout:8/744: dread - d0/d29/da7/d43/d34/dae/fc1 zero size 2024-02-16T22:22:04.304 INFO:tasks.workunit.client.1.smithi123.stdout:4/739: dread d5/d1c/d5e/dc3/f4a [0,4194304] 0 2024-02-16T22:22:04.306 INFO:tasks.workunit.client.1.smithi123.stdout:3/705: creat d0/d20/d76/d37/d41/fef x:0 0 0 2024-02-16T22:22:04.307 INFO:tasks.workunit.client.0.smithi114.stdout:1/547: creat d4/d63/fcb x:0 0 0 2024-02-16T22:22:04.307 INFO:tasks.workunit.client.1.smithi123.stdout:5/665: dwrite d6/dd/d44/f47 [0,4194304] 0 2024-02-16T22:22:04.308 INFO:tasks.workunit.client.0.smithi114.stdout:1/548: write d4/d9/d46/d47/f94 [287910,24531] 0 2024-02-16T22:22:04.310 INFO:tasks.workunit.client.1.smithi123.stdout:8/745: symlink d0/d12/d3e/d7c/lee 0 2024-02-16T22:22:04.310 INFO:tasks.workunit.client.1.smithi123.stdout:6/699: dwrite d0/d96/dca/ff8 [0,4194304] 0 2024-02-16T22:22:04.311 INFO:tasks.workunit.client.0.smithi114.stdout:1/549: rename d4/d9/d65 to d4/d9/d46/d67/d92/dca/dcc 0 2024-02-16T22:22:04.312 INFO:tasks.workunit.client.0.smithi114.stdout:2/495: sync 2024-02-16T22:22:04.313 INFO:tasks.workunit.client.0.smithi114.stdout:0/472: sync 2024-02-16T22:22:04.332 INFO:tasks.workunit.client.1.smithi123.stdout:1/673: dread d3/d6/d49/f50 [0,4194304] 0 2024-02-16T22:22:04.336 INFO:tasks.workunit.client.1.smithi123.stdout:8/746: dwrite d0/d29/da7/d26/d2a/f92 [4194304,4194304] 0 2024-02-16T22:22:04.413 INFO:tasks.workunit.client.1.smithi123.stdout:3/706: mkdir d0/d20/d76/d5d/d60/dc6/df0 0 2024-02-16T22:22:04.414 INFO:tasks.workunit.client.1.smithi123.stdout:5/666: mknod d6/dd/df/d58/d89/ce5 0 2024-02-16T22:22:04.415 INFO:tasks.workunit.client.1.smithi123.stdout:5/667: write d6/dd/d5f/fd6 [571767,44468] 0 2024-02-16T22:22:04.416 INFO:tasks.workunit.client.1.smithi123.stdout:2/668: getdents d4/d1a/d1d/d38 0 2024-02-16T22:22:04.418 INFO:tasks.workunit.client.0.smithi114.stdout:0/473: symlink d1/d16/d18/l8d 0 2024-02-16T22:22:04.419 INFO:tasks.workunit.client.1.smithi123.stdout:2/669: chown d4/d1a/d1d/d13/d25/d72/dc8 60660 1 2024-02-16T22:22:04.421 INFO:tasks.workunit.client.1.smithi123.stdout:1/674: unlink d3/d52/l89 0 2024-02-16T22:22:04.421 INFO:tasks.workunit.client.1.smithi123.stdout:3/707: dwrite d0/d20/d29/fbf [0,4194304] 0 2024-02-16T22:22:04.422 INFO:tasks.workunit.client.1.smithi123.stdout:4/740: sync 2024-02-16T22:22:04.424 INFO:tasks.workunit.client.1.smithi123.stdout:7/690: dwrite d3/d6e/d1d/db2/f51 [0,4194304] 0 2024-02-16T22:22:04.426 INFO:tasks.workunit.client.0.smithi114.stdout:2/496: dread dd/d6b/f6c [0,4194304] 0 2024-02-16T22:22:04.426 INFO:tasks.workunit.client.0.smithi114.stdout:2/497: dread - dd/df/f9e zero size 2024-02-16T22:22:04.427 INFO:tasks.workunit.client.1.smithi123.stdout:5/668: rename d6/dd/d5f to d6/dc1/de6 0 2024-02-16T22:22:04.431 INFO:tasks.workunit.client.1.smithi123.stdout:6/700: getdents d0/d9/dc/d3a/d70/dea 0 2024-02-16T22:22:04.432 INFO:tasks.workunit.client.1.smithi123.stdout:3/708: dwrite d0/d20/d76/d5d/fd7 [0,4194304] 0 2024-02-16T22:22:04.434 INFO:tasks.workunit.client.1.smithi123.stdout:3/709: fdatasync d0/d20/d29/fdb 0 2024-02-16T22:22:04.436 INFO:tasks.workunit.client.0.smithi114.stdout:1/550: link d4/d9/d46/d47/f9e d4/d9/d46/d67/d92/dca/dcc/d98/fcd 0 2024-02-16T22:22:04.437 INFO:tasks.workunit.client.0.smithi114.stdout:1/551: chown d4/d9/d3d/d3f/d79 2303 1 2024-02-16T22:22:04.438 INFO:tasks.workunit.client.0.smithi114.stdout:1/552: fdatasync d4/d9/d46/d47/fa4 0 2024-02-16T22:22:04.439 INFO:tasks.workunit.client.1.smithi123.stdout:1/675: mknod d3/d6/db/d10/d2d/ce2 0 2024-02-16T22:22:04.439 INFO:tasks.workunit.client.1.smithi123.stdout:1/676: dread - d3/d6/db/d10/fba zero size 2024-02-16T22:22:04.440 INFO:tasks.workunit.client.1.smithi123.stdout:9/802: truncate d4/d36/d8e/d6d/d71/fa3 2802733 0 2024-02-16T22:22:04.442 INFO:tasks.workunit.client.1.smithi123.stdout:9/803: truncate d4/d36/d8e/d58/d59/d60/deb/fec 660902 0 2024-02-16T22:22:04.445 INFO:tasks.workunit.client.1.smithi123.stdout:8/747: sync 2024-02-16T22:22:04.446 INFO:tasks.workunit.client.1.smithi123.stdout:5/669: write d6/dd/df/d90/fae [859596,17846] 0 2024-02-16T22:22:04.451 INFO:tasks.workunit.client.1.smithi123.stdout:7/691: write d3/d6e/d1d/f22 [1545260,52158] 0 2024-02-16T22:22:04.451 INFO:tasks.workunit.client.1.smithi123.stdout:9/804: chown d4/d36/daa/d20/l2e 88665 1 2024-02-16T22:22:04.453 INFO:tasks.workunit.client.1.smithi123.stdout:2/670: rename d4/d1a/d1d/d13/d25/f26 to d4/d1a/dd5/fd8 0 2024-02-16T22:22:04.453 INFO:tasks.workunit.client.1.smithi123.stdout:0/621: dwrite d1/d8/d3d/f59 [0,4194304] 0 2024-02-16T22:22:04.456 INFO:tasks.workunit.client.1.smithi123.stdout:8/748: creat d0/d29/da7/d26/d68/dbe/fef x:0 0 0 2024-02-16T22:22:04.457 INFO:tasks.workunit.client.1.smithi123.stdout:9/805: dread - d4/d36/d8e/d58/d59/d60/deb/da7/dde/dfd/f100 zero size 2024-02-16T22:22:04.460 INFO:tasks.workunit.client.1.smithi123.stdout:4/741: rename d5/d49/f4e to d5/d1c/d23/d56/d6e/d73/d87/fe4 0 2024-02-16T22:22:04.463 INFO:tasks.workunit.client.1.smithi123.stdout:3/710: dread d0/d20/d94/feb [0,4194304] 0 2024-02-16T22:22:04.464 INFO:tasks.workunit.client.1.smithi123.stdout:5/670: symlink d6/dd/df/d1f/le7 0 2024-02-16T22:22:04.465 INFO:tasks.workunit.client.1.smithi123.stdout:8/749: dwrite d0/df/dca/fe2 [0,4194304] 0 2024-02-16T22:22:04.465 INFO:tasks.workunit.client.1.smithi123.stdout:2/671: mkdir d4/dd9 0 2024-02-16T22:22:04.465 INFO:tasks.workunit.client.1.smithi123.stdout:6/701: dread d0/d6/d8e/fc0 [0,4194304] 0 2024-02-16T22:22:04.468 INFO:tasks.workunit.client.1.smithi123.stdout:1/677: rename d3/d6/d11/c33 to d3/d6/d11/d8b/ce3 0 2024-02-16T22:22:04.468 INFO:tasks.workunit.client.1.smithi123.stdout:1/678: stat d3/d6/db/d10/d2d/d8c/cb7 0 2024-02-16T22:22:04.469 INFO:tasks.workunit.client.1.smithi123.stdout:1/679: read - d3/d6/d11/fbf zero size 2024-02-16T22:22:04.470 INFO:tasks.workunit.client.1.smithi123.stdout:4/742: mkdir d5/d1c/d23/d56/d6e/d9f/de5 0 2024-02-16T22:22:04.473 INFO:tasks.workunit.client.1.smithi123.stdout:3/711: truncate d0/fe 592689 0 2024-02-16T22:22:04.479 INFO:tasks.workunit.client.0.smithi114.stdout:0/474: dwrite d1/d16/d5c/d80/f88 [0,4194304] 0 2024-02-16T22:22:04.480 INFO:tasks.workunit.client.1.smithi123.stdout:9/806: rename d4/d36/d8e/dbe/df2 to d4/d36/d8e/d58/d59/d60/d101 0 2024-02-16T22:22:04.482 INFO:tasks.workunit.client.1.smithi123.stdout:1/680: creat d3/d6/d11/fe4 x:0 0 0 2024-02-16T22:22:04.484 INFO:tasks.workunit.client.0.smithi114.stdout:0/475: chown d1/d16/d1f/d28/d51/d61/d6d/d79/f8a 998 1 2024-02-16T22:22:04.485 INFO:tasks.workunit.client.1.smithi123.stdout:4/743: fsync d5/d1c/d23/d47/f7e 0 2024-02-16T22:22:04.488 INFO:tasks.workunit.client.0.smithi114.stdout:8/510: sync 2024-02-16T22:22:04.488 INFO:tasks.workunit.client.1.smithi123.stdout:0/622: getdents d1/d4/d17/d54/db4 0 2024-02-16T22:22:04.491 INFO:tasks.workunit.client.0.smithi114.stdout:1/553: dwrite d4/d63/d69/d72/d82/fbb [0,4194304] 0 2024-02-16T22:22:04.496 INFO:tasks.workunit.client.0.smithi114.stdout:8/511: dwrite d4/d24/d2c/d67/fa4 [0,4194304] 0 2024-02-16T22:22:04.499 INFO:tasks.workunit.client.1.smithi123.stdout:6/702: mkdir d0/d9/dc/d3a/d70/dea/dfc 0 2024-02-16T22:22:04.507 INFO:tasks.workunit.client.1.smithi123.stdout:9/807: rename d4/d36/d8e/d58/d59/d60/deb/f9b to d4/d1e/d3a/f102 0 2024-02-16T22:22:04.508 INFO:tasks.workunit.client.1.smithi123.stdout:1/681: mknod d3/d6/d11/d8b/dbb/ce5 0 2024-02-16T22:22:04.511 INFO:tasks.workunit.client.1.smithi123.stdout:4/744: creat d5/d1c/d23/d47/da2/daa/fe6 x:0 0 0 2024-02-16T22:22:04.512 INFO:tasks.workunit.client.1.smithi123.stdout:4/745: dread d5/d1c/f97 [0,4194304] 0 2024-02-16T22:22:04.515 INFO:tasks.workunit.client.1.smithi123.stdout:5/671: creat d6/dd/df/d58/d3f/dbe/d37/fe8 x:0 0 0 2024-02-16T22:22:04.516 INFO:tasks.workunit.client.1.smithi123.stdout:5/672: chown d6/dd/df/d90/fae 944 1 2024-02-16T22:22:04.523 INFO:tasks.workunit.client.1.smithi123.stdout:6/703: dwrite d0/d96/fbf [0,4194304] 0 2024-02-16T22:22:04.524 INFO:tasks.workunit.client.1.smithi123.stdout:0/623: rename d1/d4/d42/f95 to d1/d4/d17/d54/fd9 0 2024-02-16T22:22:04.525 INFO:tasks.workunit.client.1.smithi123.stdout:6/704: write d0/d96/dca/ff8 [4725613,53468] 0 2024-02-16T22:22:04.527 INFO:tasks.workunit.client.0.smithi114.stdout:0/476: rename d1/d16/d1f/l37 to d1/d16/d5c/l8e 0 2024-02-16T22:22:04.528 INFO:tasks.workunit.client.1.smithi123.stdout:7/692: sync 2024-02-16T22:22:04.529 INFO:tasks.workunit.client.0.smithi114.stdout:0/477: dread d1/d1a/f26 [4194304,4194304] 0 2024-02-16T22:22:04.532 INFO:tasks.workunit.client.1.smithi123.stdout:4/746: creat d5/d1c/d23/d56/d6e/fe7 x:0 0 0 2024-02-16T22:22:04.532 INFO:tasks.workunit.client.0.smithi114.stdout:1/554: rmdir d4/d63/d69 39 2024-02-16T22:22:04.533 INFO:tasks.workunit.client.0.smithi114.stdout:1/555: write d4/d9/f8f [442989,127272] 0 2024-02-16T22:22:04.533 INFO:tasks.workunit.client.1.smithi123.stdout:2/672: getdents d4/d1a/d1d/d13 0 2024-02-16T22:22:04.534 INFO:tasks.workunit.client.0.smithi114.stdout:8/512: creat d4/d11/d78/d7a/d7e/fba x:0 0 0 2024-02-16T22:22:04.535 INFO:tasks.workunit.client.1.smithi123.stdout:5/673: fdatasync d6/dd/df/d58/d3f/dbe/f17 0 2024-02-16T22:22:04.536 INFO:tasks.workunit.client.0.smithi114.stdout:0/478: creat d1/d16/d5c/d80/f8f x:0 0 0 2024-02-16T22:22:04.537 INFO:tasks.workunit.client.1.smithi123.stdout:8/750: getdents d0/d29/da7/d26/d2a 0 2024-02-16T22:22:04.538 INFO:tasks.workunit.client.0.smithi114.stdout:1/556: mkdir d4/d9/d46/d67/d92/dca/dce 0 2024-02-16T22:22:04.543 INFO:tasks.workunit.client.0.smithi114.stdout:0/479: dwrite d1/d16/d1f/f63 [0,4194304] 0 2024-02-16T22:22:04.550 INFO:tasks.workunit.client.0.smithi114.stdout:1/557: mknod d4/d63/d69/d7c/dc0/ccf 0 2024-02-16T22:22:04.550 INFO:tasks.workunit.client.0.smithi114.stdout:1/558: readlink l3 0 2024-02-16T22:22:04.551 INFO:tasks.workunit.client.0.smithi114.stdout:8/513: creat d4/fbb x:0 0 0 2024-02-16T22:22:04.553 INFO:tasks.workunit.client.1.smithi123.stdout:1/682: symlink d3/d6/d49/d48/d84/db5/le6 0 2024-02-16T22:22:04.554 INFO:tasks.workunit.client.1.smithi123.stdout:1/683: chown d3/d6/d11/ca8 7790 1 2024-02-16T22:22:04.554 INFO:tasks.workunit.client.1.smithi123.stdout:1/684: fsync d3/d47/fa4 0 2024-02-16T22:22:04.557 INFO:tasks.workunit.client.0.smithi114.stdout:0/480: rename d1/d1a/d3e/d41 to d1/d16/d1f/d28/d51/d61/d7e/d90 0 2024-02-16T22:22:04.558 INFO:tasks.workunit.client.1.smithi123.stdout:7/693: mknod d3/d5/d6c/da6/ce3 0 2024-02-16T22:22:04.558 INFO:tasks.workunit.client.0.smithi114.stdout:1/559: dread - d4/d9/d46/d4d/f78 zero size 2024-02-16T22:22:04.559 INFO:tasks.workunit.client.1.smithi123.stdout:7/694: chown d3/d6e/d34/f9e 30602499 1 2024-02-16T22:22:04.559 INFO:tasks.workunit.client.0.smithi114.stdout:1/560: fdatasync d4/d63/d69/d7c/d9b/db0/f43 0 2024-02-16T22:22:04.562 INFO:tasks.workunit.client.1.smithi123.stdout:4/747: fdatasync d5/d1c/d23/f43 0 2024-02-16T22:22:04.565 INFO:tasks.workunit.client.1.smithi123.stdout:2/673: truncate d4/d1a/d1d/d59/f6f 278412 0 2024-02-16T22:22:04.566 INFO:tasks.workunit.client.1.smithi123.stdout:3/712: truncate d0/d20/d76/fa7 558870 0 2024-02-16T22:22:04.568 INFO:tasks.workunit.client.0.smithi114.stdout:8/514: symlink d4/d16/d25/d46/db5/lbc 0 2024-02-16T22:22:04.568 INFO:tasks.workunit.client.1.smithi123.stdout:2/674: dread d4/d1a/d1d/d38/f92 [0,4194304] 0 2024-02-16T22:22:04.569 INFO:tasks.workunit.client.1.smithi123.stdout:1/685: dread d3/d6/d11/d76/fd0 [4194304,4194304] 0 2024-02-16T22:22:04.569 INFO:tasks.workunit.client.1.smithi123.stdout:5/674: symlink d6/dc1/le9 0 2024-02-16T22:22:04.570 INFO:tasks.workunit.client.0.smithi114.stdout:1/561: mkdir d4/d9/d3d/d3f/dd0 0 2024-02-16T22:22:04.570 INFO:tasks.workunit.client.1.smithi123.stdout:8/751: chown d0/d9c 33769878 1 2024-02-16T22:22:04.571 INFO:tasks.workunit.client.0.smithi114.stdout:1/562: chown d4/d63/d69/d72/cc1 144428 1 2024-02-16T22:22:04.574 INFO:tasks.workunit.client.1.smithi123.stdout:9/808: rename d4/d36/daa/d20/l31 to d4/d36/d8e/dff/l103 0 2024-02-16T22:22:04.576 INFO:tasks.workunit.client.0.smithi114.stdout:8/515: creat d4/d16/d25/d46/fbd x:0 0 0 2024-02-16T22:22:04.579 INFO:tasks.workunit.client.0.smithi114.stdout:1/563: unlink d4/l8 0 2024-02-16T22:22:04.581 INFO:tasks.workunit.client.0.smithi114.stdout:0/481: write d1/d1a/f45 [7061752,120565] 0 2024-02-16T22:22:04.583 INFO:tasks.workunit.client.0.smithi114.stdout:8/516: rename d4/d11/f75 to d4/d24/d2c/fbe 0 2024-02-16T22:22:04.584 INFO:tasks.workunit.client.0.smithi114.stdout:8/517: write d4/d16/d25/d46/fbd [538493,130317] 0 2024-02-16T22:22:04.585 INFO:tasks.workunit.client.1.smithi123.stdout:7/695: unlink d3/f8 0 2024-02-16T22:22:04.589 INFO:tasks.workunit.client.0.smithi114.stdout:0/482: creat d1/d16/d1f/d28/d51/f91 x:0 0 0 2024-02-16T22:22:04.591 INFO:tasks.workunit.client.0.smithi114.stdout:8/518: creat d4/d16/fbf x:0 0 0 2024-02-16T22:22:04.592 INFO:tasks.workunit.client.1.smithi123.stdout:3/713: symlink d0/d20/d76/d37/d49/lf1 0 2024-02-16T22:22:04.594 INFO:tasks.workunit.client.0.smithi114.stdout:0/483: dwrite d1/d1a/f45 [0,4194304] 0 2024-02-16T22:22:04.598 INFO:tasks.workunit.client.0.smithi114.stdout:1/564: write d4/d63/d69/d7c/d9b/db0/f45 [764600,30546] 0 2024-02-16T22:22:04.598 INFO:tasks.workunit.client.1.smithi123.stdout:3/714: dwrite d0/d20/d76/d5d/d60/d93/fce [0,4194304] 0 2024-02-16T22:22:04.600 INFO:tasks.workunit.client.0.smithi114.stdout:8/519: unlink d4/d11/d78/d7a/d31/f8e 0 2024-02-16T22:22:04.601 INFO:tasks.workunit.client.1.smithi123.stdout:1/686: symlink d3/d6/d18/d9e/le7 0 2024-02-16T22:22:04.601 INFO:tasks.workunit.client.0.smithi114.stdout:8/520: write d4/d11/d78/f86 [612598,124391] 0 2024-02-16T22:22:04.607 INFO:tasks.workunit.client.0.smithi114.stdout:0/484: mkdir d1/d16/d55/d92 0 2024-02-16T22:22:04.612 INFO:tasks.workunit.client.1.smithi123.stdout:7/696: mkdir d3/d6e/d1d/db2/d3e/de4 0 2024-02-16T22:22:04.613 INFO:tasks.workunit.client.1.smithi123.stdout:7/697: write d3/d6e/d34/f57 [2069319,25435] 0 2024-02-16T22:22:04.618 INFO:tasks.workunit.client.0.smithi114.stdout:0/485: creat d1/d16/d55/d92/f93 x:0 0 0 2024-02-16T22:22:04.619 INFO:tasks.workunit.client.1.smithi123.stdout:2/675: mkdir d4/dda 0 2024-02-16T22:22:04.622 INFO:tasks.workunit.client.1.smithi123.stdout:8/752: dwrite d0/d29/da7/d26/d2a/d4d/f57 [0,4194304] 0 2024-02-16T22:22:04.625 INFO:tasks.workunit.client.1.smithi123.stdout:5/675: mknod d6/dd/df/d76/cea 0 2024-02-16T22:22:04.626 INFO:tasks.workunit.client.0.smithi114.stdout:1/565: link d4/c2c d4/d63/d69/d7c/cd1 0 2024-02-16T22:22:04.626 INFO:tasks.workunit.client.1.smithi123.stdout:5/676: write d6/dc1/de6/fb2 [336474,82907] 0 2024-02-16T22:22:04.627 INFO:tasks.workunit.client.1.smithi123.stdout:3/715: creat d0/da0/ff2 x:0 0 0 2024-02-16T22:22:04.628 INFO:tasks.workunit.client.0.smithi114.stdout:1/566: dread - d4/d63/d69/d72/d82/fc7 zero size 2024-02-16T22:22:04.629 INFO:tasks.workunit.client.1.smithi123.stdout:3/716: write d0/d20/d76/d5d/f61 [7205703,30492] 0 2024-02-16T22:22:04.629 INFO:tasks.workunit.client.0.smithi114.stdout:0/486: dwrite d1/fe [4194304,4194304] 0 2024-02-16T22:22:04.629 INFO:tasks.workunit.client.1.smithi123.stdout:3/717: readlink d0/l38 0 2024-02-16T22:22:04.630 INFO:tasks.workunit.client.1.smithi123.stdout:3/718: write d0/f4c [7783131,73266] 0 2024-02-16T22:22:04.630 INFO:tasks.workunit.client.0.smithi114.stdout:8/521: dread d4/d24/f50 [0,4194304] 0 2024-02-16T22:22:04.631 INFO:tasks.workunit.client.1.smithi123.stdout:9/809: dread d4/d33/f73 [0,4194304] 0 2024-02-16T22:22:04.634 INFO:tasks.workunit.client.1.smithi123.stdout:1/687: rename d3/d6/d65 to d3/d6/db/d10/d4f/de8 0 2024-02-16T22:22:04.635 INFO:tasks.workunit.client.1.smithi123.stdout:1/688: chown d3/d6/d18/d9e/le7 0 1 2024-02-16T22:22:04.638 INFO:tasks.workunit.client.0.smithi114.stdout:0/487: dwrite d1/d1a/d47/d62/d7f/f84 [0,4194304] 0 2024-02-16T22:22:04.639 INFO:tasks.workunit.client.1.smithi123.stdout:0/624: creat d1/d8/d13/d1d/fda x:0 0 0 2024-02-16T22:22:04.641 INFO:tasks.workunit.client.0.smithi114.stdout:8/522: rmdir d4/d24 39 2024-02-16T22:22:04.642 INFO:tasks.workunit.client.1.smithi123.stdout:6/705: getdents d0/d9/dc/d3a/d70 0 2024-02-16T22:22:04.645 INFO:tasks.workunit.client.1.smithi123.stdout:4/748: sync 2024-02-16T22:22:04.646 INFO:tasks.workunit.client.0.smithi114.stdout:1/567: rename d4/d9/d46/d67 to d4/d9/d46/d67/d92/dca/dd2 22 2024-02-16T22:22:04.647 INFO:tasks.workunit.client.0.smithi114.stdout:0/488: creat d1/d16/d5c/d80/f94 x:0 0 0 2024-02-16T22:22:04.647 INFO:tasks.workunit.client.0.smithi114.stdout:1/568: write d4/d63/d69/d7c/f8b [406588,26618] 0 2024-02-16T22:22:04.649 INFO:tasks.workunit.client.1.smithi123.stdout:2/676: fsync d4/d1a/d1d/d13/d25/f3b 0 2024-02-16T22:22:04.650 INFO:tasks.workunit.client.0.smithi114.stdout:3/554: sync 2024-02-16T22:22:04.651 INFO:tasks.workunit.client.1.smithi123.stdout:8/753: unlink d0/d29/da7/d26/d68/d81/d47/d9b/fb0 0 2024-02-16T22:22:04.653 INFO:tasks.workunit.client.1.smithi123.stdout:5/677: creat d6/dd/df/d76/feb x:0 0 0 2024-02-16T22:22:04.655 INFO:tasks.workunit.client.1.smithi123.stdout:2/677: dwrite d4/d1a/d1d/d59/d45/fc5 [0,4194304] 0 2024-02-16T22:22:04.657 INFO:tasks.workunit.client.0.smithi114.stdout:3/555: rmdir d4/d24 39 2024-02-16T22:22:04.660 INFO:tasks.workunit.client.0.smithi114.stdout:1/569: mkdir d4/d9/d46/d67/d92/dca/dce/dd3 0 2024-02-16T22:22:04.661 INFO:tasks.workunit.client.1.smithi123.stdout:3/719: rmdir d0/d20/d5a 39 2024-02-16T22:22:04.663 INFO:tasks.workunit.client.0.smithi114.stdout:8/523: dread d4/d16/d25/f6e [0,4194304] 0 2024-02-16T22:22:04.664 INFO:tasks.workunit.client.0.smithi114.stdout:8/524: chown d4/d11/f17 273 1 2024-02-16T22:22:04.665 INFO:tasks.workunit.client.1.smithi123.stdout:7/698: dwrite d3/d6e/d1d/db2/f99 [0,4194304] 0 2024-02-16T22:22:04.669 INFO:tasks.workunit.client.0.smithi114.stdout:0/489: write d1/d16/d1f/d28/d51/d61/d7e/d90/f42 [1090852,96209] 0 2024-02-16T22:22:04.673 INFO:tasks.workunit.client.0.smithi114.stdout:3/556: symlink d4/d24/d2d/d45/d68/d77/lc3 0 2024-02-16T22:22:04.673 INFO:tasks.workunit.client.0.smithi114.stdout:3/557: fsync d4/d24/f6d 0 2024-02-16T22:22:04.674 INFO:tasks.workunit.client.1.smithi123.stdout:6/706: unlink d0/d6/d2f/da5/dc6/ld0 0 2024-02-16T22:22:04.675 INFO:tasks.workunit.client.1.smithi123.stdout:4/749: write d5/d1c/d23/d31/f8f [818812,93208] 0 2024-02-16T22:22:04.677 INFO:tasks.workunit.client.1.smithi123.stdout:8/754: creat d0/d29/da7/d26/d68/d81/d47/d9b/ff0 x:0 0 0 2024-02-16T22:22:04.678 INFO:tasks.workunit.client.1.smithi123.stdout:7/699: dwrite d3/d6e/d1d/db2/fc5 [0,4194304] 0 2024-02-16T22:22:04.678 INFO:tasks.workunit.client.0.smithi114.stdout:3/558: fsync d4/d26/f3c 0 2024-02-16T22:22:04.681 INFO:tasks.workunit.client.0.smithi114.stdout:1/570: link d4/d63/d69/c6a d4/d63/d69/d7c/d9b/db0/d74/cd4 0 2024-02-16T22:22:04.682 INFO:tasks.workunit.client.1.smithi123.stdout:2/678: creat d4/d1a/d1d/d38/d7b/fdb x:0 0 0 2024-02-16T22:22:04.683 INFO:tasks.workunit.client.1.smithi123.stdout:4/750: dread d5/d1c/d64/fb6 [0,4194304] 0 2024-02-16T22:22:04.683 INFO:tasks.workunit.client.1.smithi123.stdout:3/720: mknod d0/d50/d7e/cf3 0 2024-02-16T22:22:04.686 INFO:tasks.workunit.client.1.smithi123.stdout:5/678: dread d6/dd/df/d58/f63 [0,4194304] 0 2024-02-16T22:22:04.686 INFO:tasks.workunit.client.1.smithi123.stdout:2/679: readlink d4/d1a/d1d/d13/d25/l46 0 2024-02-16T22:22:04.690 INFO:tasks.workunit.client.0.smithi114.stdout:8/525: creat d4/d11/fc0 x:0 0 0 2024-02-16T22:22:04.690 INFO:tasks.workunit.client.0.smithi114.stdout:9/570: sync 2024-02-16T22:22:04.690 INFO:tasks.workunit.client.0.smithi114.stdout:7/590: sync 2024-02-16T22:22:04.690 INFO:tasks.workunit.client.0.smithi114.stdout:5/506: sync 2024-02-16T22:22:04.690 INFO:tasks.workunit.client.0.smithi114.stdout:4/663: sync 2024-02-16T22:22:04.690 INFO:tasks.workunit.client.0.smithi114.stdout:2/498: sync 2024-02-16T22:22:04.691 INFO:tasks.workunit.client.0.smithi114.stdout:6/559: sync 2024-02-16T22:22:04.691 INFO:tasks.workunit.client.1.smithi123.stdout:7/700: dwrite d3/d6e/d24/fb9 [0,4194304] 0 2024-02-16T22:22:04.692 INFO:tasks.workunit.client.1.smithi123.stdout:4/751: truncate d5/d1c/d64/fc6 5195366 0 2024-02-16T22:22:04.696 INFO:tasks.workunit.client.1.smithi123.stdout:3/721: dwrite d0/d20/d76/f36 [4194304,4194304] 0 2024-02-16T22:22:04.705 INFO:tasks.workunit.client.0.smithi114.stdout:9/571: dwrite d2/dd/d3c/d89/d9e/fa2 [0,4194304] 0 2024-02-16T22:22:04.707 INFO:tasks.workunit.client.1.smithi123.stdout:9/810: dread d4/d36/daa/d20/f6a [0,4194304] 0 2024-02-16T22:22:04.708 INFO:tasks.workunit.client.1.smithi123.stdout:9/811: chown d4/d36/daa/d20 1005 1 2024-02-16T22:22:04.710 INFO:tasks.workunit.client.1.smithi123.stdout:1/689: truncate d3/d47/fa4 1799783 0 2024-02-16T22:22:04.711 INFO:tasks.workunit.client.1.smithi123.stdout:1/690: read d3/d6/f5b [2219952,121089] 0 2024-02-16T22:22:04.713 INFO:tasks.workunit.client.1.smithi123.stdout:1/691: truncate d3/d6/db/d10/fba 105968 0 2024-02-16T22:22:04.713 INFO:tasks.workunit.client.0.smithi114.stdout:0/490: write d1/f7 [891309,27029] 0 2024-02-16T22:22:04.715 INFO:tasks.workunit.client.0.smithi114.stdout:0/491: chown d1/d16/d55/d92/f93 59 1 2024-02-16T22:22:04.716 INFO:tasks.workunit.client.0.smithi114.stdout:7/591: creat d3/d87/fca x:0 0 0 2024-02-16T22:22:04.718 INFO:tasks.workunit.client.0.smithi114.stdout:5/507: mkdir d0/d2d/d30/d9f 0 2024-02-16T22:22:04.720 INFO:tasks.workunit.client.0.smithi114.stdout:4/664: dread - dd/d2e/d56/f8c zero size 2024-02-16T22:22:04.722 INFO:tasks.workunit.client.1.smithi123.stdout:9/812: dread d4/f85 [0,4194304] 0 2024-02-16T22:22:04.730 INFO:tasks.workunit.client.0.smithi114.stdout:6/560: mkdir d6/d2c/d38/da8 0 2024-02-16T22:22:04.734 INFO:tasks.workunit.client.0.smithi114.stdout:8/526: dread d4/f8 [4194304,4194304] 0 2024-02-16T22:22:04.734 INFO:tasks.workunit.client.0.smithi114.stdout:8/527: write d4/d16/d25/d7c/f87 [521501,50295] 0 2024-02-16T22:22:04.744 INFO:tasks.workunit.client.0.smithi114.stdout:0/492: dread d1/d16/d1f/d28/d51/d61/d7e/d90/f42 [0,4194304] 0 2024-02-16T22:22:04.745 INFO:tasks.workunit.client.0.smithi114.stdout:2/499: dread dd/df/d1c/d21/d87/f34 [0,4194304] 0 2024-02-16T22:22:04.747 INFO:tasks.workunit.client.0.smithi114.stdout:3/559: truncate d4/d43/f56 3249962 0 2024-02-16T22:22:04.749 INFO:tasks.workunit.client.0.smithi114.stdout:3/560: fsync d4/d87/faf 0 2024-02-16T22:22:04.751 INFO:tasks.workunit.client.0.smithi114.stdout:6/561: mknod d6/d2c/d66/ca9 0 2024-02-16T22:22:04.752 INFO:tasks.workunit.client.0.smithi114.stdout:2/500: dwrite dd/df/f9e [0,4194304] 0 2024-02-16T22:22:04.755 INFO:tasks.workunit.client.0.smithi114.stdout:8/528: fsync d4/d11/f99 0 2024-02-16T22:22:04.756 INFO:tasks.workunit.client.0.smithi114.stdout:8/529: dread - d4/d11/d78/f90 zero size 2024-02-16T22:22:04.757 INFO:tasks.workunit.client.0.smithi114.stdout:5/508: creat d0/df/d6d/fa0 x:0 0 0 2024-02-16T22:22:04.761 INFO:tasks.workunit.client.0.smithi114.stdout:7/592: dwrite d3/d14/f77 [4194304,4194304] 0 2024-02-16T22:22:04.769 INFO:tasks.workunit.client.0.smithi114.stdout:4/665: dwrite dd/d1b/d37/d38/f5e [0,4194304] 0 2024-02-16T22:22:04.770 INFO:tasks.workunit.client.0.smithi114.stdout:1/571: rename d4/d9/d46/d47/db3/fb4 to d4/d9/d46/d67/d92/dca/dcc/fd5 0 2024-02-16T22:22:04.771 INFO:tasks.workunit.client.0.smithi114.stdout:8/530: dwrite d4/d16/fbf [0,4194304] 0 2024-02-16T22:22:04.772 INFO:tasks.workunit.client.0.smithi114.stdout:4/666: fdatasync dd/d2e/d56/fb3 0 2024-02-16T22:22:04.773 INFO:tasks.workunit.client.0.smithi114.stdout:4/667: fsync dd/d1b/d41/d74/d9b/fc6 0 2024-02-16T22:22:04.775 INFO:tasks.workunit.client.0.smithi114.stdout:3/561: write d4/dd/f64 [1843346,127748] 0 2024-02-16T22:22:04.780 INFO:tasks.workunit.client.0.smithi114.stdout:5/509: creat d0/df/d6d/fa1 x:0 0 0 2024-02-16T22:22:04.790 INFO:tasks.workunit.client.0.smithi114.stdout:9/572: link d2/dd/ff d2/d33/d3f/fb5 0 2024-02-16T22:22:04.790 INFO:tasks.workunit.client.0.smithi114.stdout:9/573: readlink d2/d1a/d23/d35/d47/la4 0 2024-02-16T22:22:04.793 INFO:tasks.workunit.client.0.smithi114.stdout:0/493: rename d1/d16/d5c to d1/d16/d1f/d28/d51/d61/d7e/d90/d95 0 2024-02-16T22:22:04.794 INFO:tasks.workunit.client.0.smithi114.stdout:1/572: readlink d4/d9/l1a 0 2024-02-16T22:22:04.800 INFO:tasks.workunit.client.0.smithi114.stdout:2/501: mkdir dd/da6 0 2024-02-16T22:22:04.800 INFO:tasks.workunit.client.0.smithi114.stdout:2/502: chown dd/d3c/d7f 377673640 1 2024-02-16T22:22:04.806 INFO:tasks.workunit.client.0.smithi114.stdout:5/510: unlink d0/df/d53/f62 0 2024-02-16T22:22:04.809 INFO:tasks.workunit.client.0.smithi114.stdout:7/593: symlink d3/d46/da2/lcb 0 2024-02-16T22:22:04.810 INFO:tasks.workunit.client.1.smithi123.stdout:6/707: dwrite d0/d96/de3/ff7 [4194304,4194304] 0 2024-02-16T22:22:04.810 INFO:tasks.workunit.client.0.smithi114.stdout:7/594: truncate d3/d92/fc5 169376 0 2024-02-16T22:22:04.811 INFO:tasks.workunit.client.1.smithi123.stdout:6/708: dread - d0/d6/d4d/f8f zero size 2024-02-16T22:22:04.811 INFO:tasks.workunit.client.1.smithi123.stdout:6/709: fsync d0/d74/f7b 0 2024-02-16T22:22:04.812 INFO:tasks.workunit.client.0.smithi114.stdout:3/562: dread d4/d26/f4d [0,4194304] 0 2024-02-16T22:22:04.813 INFO:tasks.workunit.client.0.smithi114.stdout:3/563: fdatasync d4/d24/f6d 0 2024-02-16T22:22:04.816 INFO:tasks.workunit.client.0.smithi114.stdout:0/494: mknod d1/d16/d1f/d28/d51/d61/d6d/d79/c96 0 2024-02-16T22:22:04.822 INFO:tasks.workunit.client.1.smithi123.stdout:5/679: truncate d6/dd/df/f57 131798 0 2024-02-16T22:22:04.825 INFO:tasks.workunit.client.1.smithi123.stdout:4/752: rename d5/d1c/d23/l8e to d5/d1c/d23/d74/d92/d9b/dbd/le8 0 2024-02-16T22:22:04.825 INFO:tasks.workunit.client.0.smithi114.stdout:2/503: rmdir dd/df/d1c 39 2024-02-16T22:22:04.826 INFO:tasks.workunit.client.1.smithi123.stdout:4/753: chown d5/d1c/d23/d56/d6e/d9f/da1/dc0 125 1 2024-02-16T22:22:04.827 INFO:tasks.workunit.client.0.smithi114.stdout:6/562: link d6/dd/d25/f53 d6/da/faa 0 2024-02-16T22:22:04.827 INFO:tasks.workunit.client.1.smithi123.stdout:6/710: dread d0/d9/fb5 [0,4194304] 0 2024-02-16T22:22:04.830 INFO:tasks.workunit.client.0.smithi114.stdout:5/511: creat d0/df/fa2 x:0 0 0 2024-02-16T22:22:04.832 INFO:tasks.workunit.client.1.smithi123.stdout:4/754: dwrite d5/d1c/fac [0,4194304] 0 2024-02-16T22:22:04.832 INFO:tasks.workunit.client.0.smithi114.stdout:0/495: dread d1/d1a/f45 [4194304,4194304] 0 2024-02-16T22:22:04.833 INFO:tasks.workunit.client.1.smithi123.stdout:4/755: write d5/d1c/d23/f2a [3556277,74204] 0 2024-02-16T22:22:04.834 INFO:tasks.workunit.client.0.smithi114.stdout:0/496: chown d1/d1a/d47/d62/d7f/d8c 89215612 1 2024-02-16T22:22:04.835 INFO:tasks.workunit.client.0.smithi114.stdout:7/595: creat d3/d22/d73/da7/db4/fcc x:0 0 0 2024-02-16T22:22:04.835 INFO:tasks.workunit.client.0.smithi114.stdout:4/668: dwrite dd/d1b/d41/d77/f90 [0,4194304] 0 2024-02-16T22:22:04.836 INFO:tasks.workunit.client.1.smithi123.stdout:4/756: read d5/d1c/d23/d47/f62 [1615736,93707] 0 2024-02-16T22:22:04.836 INFO:tasks.workunit.client.0.smithi114.stdout:0/497: chown d1/d1a/d47 1707 1 2024-02-16T22:22:04.838 INFO:tasks.workunit.client.0.smithi114.stdout:0/498: chown d1/f7 13609905 1 2024-02-16T22:22:04.838 INFO:tasks.workunit.client.1.smithi123.stdout:9/813: symlink d4/d36/d57/db2/l104 0 2024-02-16T22:22:04.844 INFO:tasks.workunit.client.1.smithi123.stdout:0/625: mkdir d1/d8/d13/d1d/d64/dbc/ddb 0 2024-02-16T22:22:04.851 INFO:tasks.workunit.client.0.smithi114.stdout:2/504: rename dd/d3c to dd/d3b/da7 0 2024-02-16T22:22:04.851 INFO:tasks.workunit.client.1.smithi123.stdout:2/680: dread d4/d1a/d1d/d59/fc2 [0,4194304] 0 2024-02-16T22:22:04.853 INFO:tasks.workunit.client.0.smithi114.stdout:5/512: symlink d0/df/la3 0 2024-02-16T22:22:04.854 INFO:tasks.workunit.client.1.smithi123.stdout:7/701: dread - d3/d5/fb8 zero size 2024-02-16T22:22:04.854 INFO:tasks.workunit.client.0.smithi114.stdout:5/513: chown d0/df/d6d 30810672 1 2024-02-16T22:22:04.856 INFO:tasks.workunit.client.1.smithi123.stdout:5/680: symlink d6/dd/d44/dba/lec 0 2024-02-16T22:22:04.857 INFO:tasks.workunit.client.0.smithi114.stdout:7/596: creat d3/d22/d61/d6a/fcd x:0 0 0 2024-02-16T22:22:04.859 INFO:tasks.workunit.client.0.smithi114.stdout:2/505: dwrite dd/df/f11 [0,4194304] 0 2024-02-16T22:22:04.859 INFO:tasks.workunit.client.0.smithi114.stdout:7/597: chown d3/d22/d61/d6a/f9e 69155749 1 2024-02-16T22:22:04.861 INFO:tasks.workunit.client.0.smithi114.stdout:2/506: write dd/df/f9e [4052213,18530] 0 2024-02-16T22:22:04.864 INFO:tasks.workunit.client.1.smithi123.stdout:3/722: rename d0/d20/d76 to d0/d20/d29/d9b/dbd/df4 0 2024-02-16T22:22:04.869 INFO:tasks.workunit.client.1.smithi123.stdout:2/681: dread d4/d1a/d1d/d13/d15/d6d/db1/fb7 [0,4194304] 0 2024-02-16T22:22:04.869 INFO:tasks.workunit.client.1.smithi123.stdout:6/711: truncate d0/d9/dc/d3a/f81 514999 0 2024-02-16T22:22:04.876 INFO:tasks.workunit.client.0.smithi114.stdout:3/564: dwrite d4/d26/f4d [0,4194304] 0 2024-02-16T22:22:04.876 INFO:tasks.workunit.client.0.smithi114.stdout:8/531: link d4/d11/d78/d7a/d36/c9e d4/d11/d9d/cc1 0 2024-02-16T22:22:04.879 INFO:tasks.workunit.client.0.smithi114.stdout:1/573: link d4/d9/d46/d47/c6c d4/d9/d46/d67/d92/dca/cd6 0 2024-02-16T22:22:04.883 INFO:tasks.workunit.client.1.smithi123.stdout:4/757: unlink d5/d1c/d23/f3c 0 2024-02-16T22:22:04.884 INFO:tasks.workunit.client.0.smithi114.stdout:6/563: symlink d6/d2c/d38/da8/lab 0 2024-02-16T22:22:04.886 INFO:tasks.workunit.client.0.smithi114.stdout:4/669: write dd/d2e/d3f/f50 [4064488,122001] 0 2024-02-16T22:22:04.887 INFO:tasks.workunit.client.0.smithi114.stdout:8/532: dwrite d4/d16/d25/f54 [0,4194304] 0 2024-02-16T22:22:04.892 INFO:tasks.workunit.client.1.smithi123.stdout:0/626: mkdir d1/d4/d42/ddc 0 2024-02-16T22:22:04.893 INFO:tasks.workunit.client.1.smithi123.stdout:8/755: creat d0/d61/d66/d7e/ff1 x:0 0 0 2024-02-16T22:22:04.894 INFO:tasks.workunit.client.1.smithi123.stdout:0/627: read - d1/d8/d13/d1d/fda zero size 2024-02-16T22:22:04.895 INFO:tasks.workunit.client.0.smithi114.stdout:7/598: symlink d3/d22/d73/da7/lce 0 2024-02-16T22:22:04.896 INFO:tasks.workunit.client.1.smithi123.stdout:0/628: write d1/d4/d17/d54/d8e/f78 [340700,7060] 0 2024-02-16T22:22:04.896 INFO:tasks.workunit.client.0.smithi114.stdout:7/599: stat d3/d22/d61/f84 0 2024-02-16T22:22:04.897 INFO:tasks.workunit.client.0.smithi114.stdout:6/564: dwrite d6/d2c/d66/fa5 [0,4194304] 0 2024-02-16T22:22:04.898 INFO:tasks.workunit.client.0.smithi114.stdout:7/600: readlink d3/d22/l4f 0 2024-02-16T22:22:04.904 INFO:tasks.workunit.client.0.smithi114.stdout:0/499: symlink d1/d1a/d47/d62/d89/l97 0 2024-02-16T22:22:04.905 INFO:tasks.workunit.client.0.smithi114.stdout:9/574: getdents d2/d5c 0 2024-02-16T22:22:04.906 INFO:tasks.workunit.client.0.smithi114.stdout:2/507: fdatasync dd/df/d1c/d21/d87/f4d 0 2024-02-16T22:22:04.908 INFO:tasks.workunit.client.1.smithi123.stdout:3/723: rename d0/d20/d29/d9b/dbd/df4/d37/d49 to d0/d20/d29/d9b/dbd/df5 0 2024-02-16T22:22:04.909 INFO:tasks.workunit.client.0.smithi114.stdout:6/565: dwrite d6/d2c/d38/d41/d60/f98 [0,4194304] 0 2024-02-16T22:22:04.909 INFO:tasks.workunit.client.0.smithi114.stdout:2/508: read - dd/d3b/d8a/f9a zero size 2024-02-16T22:22:04.912 INFO:tasks.workunit.client.1.smithi123.stdout:2/682: readlink d4/d1a/d1d/l6e 0 2024-02-16T22:22:04.912 INFO:tasks.workunit.client.1.smithi123.stdout:3/724: dread d0/d20/f52 [0,4194304] 0 2024-02-16T22:22:04.912 INFO:tasks.workunit.client.0.smithi114.stdout:0/500: write d1/d16/d18/f59 [3339019,43546] 0 2024-02-16T22:22:04.913 INFO:tasks.workunit.client.1.smithi123.stdout:2/683: fsync d4/d1a/d1d/f9f 0 2024-02-16T22:22:04.913 INFO:tasks.workunit.client.1.smithi123.stdout:3/725: readlink d0/d50/d75/l89 0 2024-02-16T22:22:04.914 INFO:tasks.workunit.client.0.smithi114.stdout:2/509: write dd/df/d1c/d21/d87/f94 [350689,117141] 0 2024-02-16T22:22:04.915 INFO:tasks.workunit.client.0.smithi114.stdout:1/574: creat d4/d9/d46/d67/d92/dca/dcc/d98/fd7 x:0 0 0 2024-02-16T22:22:04.915 INFO:tasks.workunit.client.0.smithi114.stdout:0/501: readlink d1/d1a/d47/d62/d89/l97 0 2024-02-16T22:22:04.917 INFO:tasks.workunit.client.0.smithi114.stdout:6/566: write d6/dd/d25/d34/d61/d62/f8d [332043,93276] 0 2024-02-16T22:22:04.920 INFO:tasks.workunit.client.1.smithi123.stdout:1/692: creat d3/d6/db/d10/fe9 x:0 0 0 2024-02-16T22:22:04.921 INFO:tasks.workunit.client.1.smithi123.stdout:2/684: dwrite d4/d1a/d1d/d13/d15/d6d/d4a/faf [0,4194304] 0 2024-02-16T22:22:04.927 INFO:tasks.workunit.client.0.smithi114.stdout:4/670: dread dd/d1b/d24/f4e [0,4194304] 0 2024-02-16T22:22:04.927 INFO:tasks.workunit.client.0.smithi114.stdout:7/601: dread d3/d22/f6c [0,4194304] 0 2024-02-16T22:22:04.932 INFO:tasks.workunit.client.0.smithi114.stdout:7/602: write d3/d14/fa4 [897055,4495] 0 2024-02-16T22:22:04.933 INFO:tasks.workunit.client.1.smithi123.stdout:0/629: mknod d1/d4/d17/cdd 0 2024-02-16T22:22:04.935 INFO:tasks.workunit.client.0.smithi114.stdout:9/575: chown d2/d1a/d65/ca3 59548781 1 2024-02-16T22:22:04.936 INFO:tasks.workunit.client.0.smithi114.stdout:3/565: mknod d4/d24/d2d/d45/dbb/cc4 0 2024-02-16T22:22:04.938 INFO:tasks.workunit.client.1.smithi123.stdout:7/702: creat d3/de0/fe5 x:0 0 0 2024-02-16T22:22:04.938 INFO:tasks.workunit.client.0.smithi114.stdout:5/514: write d0/df/f8a [3053372,83311] 0 2024-02-16T22:22:04.940 INFO:tasks.workunit.client.1.smithi123.stdout:5/681: link d6/dd/df/f62 d6/dd/df/d58/d89/fed 0 2024-02-16T22:22:04.945 INFO:tasks.workunit.client.1.smithi123.stdout:4/758: dwrite d5/d1c/d23/d56/d6e/fa5 [0,4194304] 0 2024-02-16T22:22:04.945 INFO:tasks.workunit.client.0.smithi114.stdout:2/510: symlink dd/d3b/d8a/la8 0 2024-02-16T22:22:04.945 INFO:tasks.workunit.client.1.smithi123.stdout:7/703: dwrite d3/d6e/d1d/db2/fc5 [4194304,4194304] 0 2024-02-16T22:22:04.946 INFO:tasks.workunit.client.1.smithi123.stdout:4/759: fsync d5/d1c/d23/d74/d92/f86 0 2024-02-16T22:22:04.947 INFO:tasks.workunit.client.1.smithi123.stdout:7/704: readlink d3/d6e/d1d/db2/d35/l85 0 2024-02-16T22:22:04.947 INFO:tasks.workunit.client.0.smithi114.stdout:1/575: write d4/d9/d46/f5f [307041,15413] 0 2024-02-16T22:22:04.949 INFO:tasks.workunit.client.1.smithi123.stdout:4/760: readlink d5/d1c/d23/d56/d6e/d73/d87/laf 0 2024-02-16T22:22:04.951 INFO:tasks.workunit.client.1.smithi123.stdout:9/814: creat d4/d1e/f105 x:0 0 0 2024-02-16T22:22:04.952 INFO:tasks.workunit.client.1.smithi123.stdout:4/761: fdatasync d5/d1c/d23/d56/d6e/d73/f88 0 2024-02-16T22:22:04.953 INFO:tasks.workunit.client.1.smithi123.stdout:8/756: link d0/d29/da7/d43/d34/dae/fc1 d0/d29/da7/d26/d68/d81/d8f/ff2 0 2024-02-16T22:22:04.962 INFO:tasks.workunit.client.0.smithi114.stdout:3/566: symlink d4/d87/lc5 0 2024-02-16T22:22:04.964 INFO:tasks.workunit.client.0.smithi114.stdout:9/576: rename d2/d1a/d23/d35/f1e to d2/d1a/d23/d32/d46/fb6 0 2024-02-16T22:22:04.965 INFO:tasks.workunit.client.1.smithi123.stdout:3/726: mkdir d0/d20/d29/df6 0 2024-02-16T22:22:04.965 INFO:tasks.workunit.client.1.smithi123.stdout:3/727: readlink d0/l16 0 2024-02-16T22:22:04.966 INFO:tasks.workunit.client.1.smithi123.stdout:3/728: stat d0/d20/d29/f84 0 2024-02-16T22:22:04.966 INFO:tasks.workunit.client.0.smithi114.stdout:5/515: unlink d0/df/d53/f8b 0 2024-02-16T22:22:04.967 INFO:tasks.workunit.client.1.smithi123.stdout:6/712: getdents d0/d9/dc/d39 0 2024-02-16T22:22:04.968 INFO:tasks.workunit.client.0.smithi114.stdout:8/533: truncate d4/d11/d78/d7a/d36/f6b 2600942 0 2024-02-16T22:22:04.969 INFO:tasks.workunit.client.0.smithi114.stdout:0/502: mkdir d1/d16/d48/d98 0 2024-02-16T22:22:04.971 INFO:tasks.workunit.client.1.smithi123.stdout:0/630: write d1/d8/d13/d1d/d64/dbc/f92 [387284,7660] 0 2024-02-16T22:22:04.972 INFO:tasks.workunit.client.1.smithi123.stdout:5/682: dread d6/dd/df/d58/f70 [0,4194304] 0 2024-02-16T22:22:04.973 INFO:tasks.workunit.client.1.smithi123.stdout:2/685: unlink d4/d1a/f6a 0 2024-02-16T22:22:04.975 INFO:tasks.workunit.client.1.smithi123.stdout:1/693: truncate d3/d6/d49/f66 86337 0 2024-02-16T22:22:04.978 INFO:tasks.workunit.client.1.smithi123.stdout:0/631: dwrite d1/d4/d17/f41 [0,4194304] 0 2024-02-16T22:22:04.978 INFO:tasks.workunit.client.0.smithi114.stdout:4/671: truncate dd/d1b/d24/f2f 1118124 0 2024-02-16T22:22:04.980 INFO:tasks.workunit.client.0.smithi114.stdout:7/603: creat d3/d22/d73/da7/d6f/d9c/fcf x:0 0 0 2024-02-16T22:22:04.982 INFO:tasks.workunit.client.1.smithi123.stdout:9/815: mkdir d4/d36/d8e/d58/d59/d60/d106 0 2024-02-16T22:22:04.989 INFO:tasks.workunit.client.0.smithi114.stdout:9/577: rename d2/d1a/d23/d35/d47/d63/f6a to d2/dd/d3c/fb7 0 2024-02-16T22:22:04.992 INFO:tasks.workunit.client.1.smithi123.stdout:7/705: write d3/d87/da8/faf [349082,48824] 0 2024-02-16T22:22:04.993 INFO:tasks.workunit.client.1.smithi123.stdout:7/706: read - d3/d6e/d52/f71 zero size 2024-02-16T22:22:04.995 INFO:tasks.workunit.client.0.smithi114.stdout:2/511: truncate dd/df/d1c/f67 597150 0 2024-02-16T22:22:04.997 INFO:tasks.workunit.client.1.smithi123.stdout:4/762: write d5/d1c/d23/d74/f5f [2340323,100406] 0 2024-02-16T22:22:04.998 INFO:tasks.workunit.client.0.smithi114.stdout:6/567: creat d6/d2c/d38/fac x:0 0 0 2024-02-16T22:22:05.000 INFO:tasks.workunit.client.0.smithi114.stdout:1/576: truncate d4/d9/d46/d67/f6e 7818898 0 2024-02-16T22:22:05.002 INFO:tasks.workunit.client.0.smithi114.stdout:1/577: write d4/d63/d69/d72/f97 [465963,125728] 0 2024-02-16T22:22:05.002 INFO:tasks.workunit.client.1.smithi123.stdout:8/757: dread d0/d12/fb5 [0,4194304] 0 2024-02-16T22:22:05.004 INFO:tasks.workunit.client.0.smithi114.stdout:4/672: fsync dd/d59/fa4 0 2024-02-16T22:22:05.006 INFO:tasks.workunit.client.0.smithi114.stdout:7/604: mknod d3/d22/d73/cd0 0 2024-02-16T22:22:05.007 INFO:tasks.workunit.client.1.smithi123.stdout:1/694: creat d3/d6/d18/d9e/fea x:0 0 0 2024-02-16T22:22:05.008 INFO:tasks.workunit.client.0.smithi114.stdout:3/567: creat d4/d26/d83/fc6 x:0 0 0 2024-02-16T22:22:05.012 INFO:tasks.workunit.client.1.smithi123.stdout:2/686: write d4/d1a/d1d/d38/d7b/f94 [642738,2688] 0 2024-02-16T22:22:05.013 INFO:tasks.workunit.client.1.smithi123.stdout:2/687: write d4/d1a/d1d/d13/d25/d98/f76 [4068929,8519] 0 2024-02-16T22:22:05.019 INFO:tasks.workunit.client.0.smithi114.stdout:8/534: write d4/d24/f38 [1000512,102643] 0 2024-02-16T22:22:05.021 INFO:tasks.workunit.client.1.smithi123.stdout:0/632: dwrite d1/d8/d13/d1d/d64/f84 [0,4194304] 0 2024-02-16T22:22:05.022 INFO:tasks.workunit.client.0.smithi114.stdout:2/512: rename dd/df/f41 to dd/df/d1c/d21/d61/fa9 0 2024-02-16T22:22:05.026 INFO:tasks.workunit.client.1.smithi123.stdout:5/683: mkdir d6/dd/df/d1f/ddc/dee 0 2024-02-16T22:22:05.027 INFO:tasks.workunit.client.1.smithi123.stdout:7/707: dwrite d3/d87/f4a [0,4194304] 0 2024-02-16T22:22:05.031 INFO:tasks.workunit.client.0.smithi114.stdout:1/578: write d4/d9/d46/d67/d92/dca/dcc/fa5 [805657,43458] 0 2024-02-16T22:22:05.034 INFO:tasks.workunit.client.1.smithi123.stdout:6/713: dwrite d0/d9/dc/d3a/f80 [0,4194304] 0 2024-02-16T22:22:05.035 INFO:tasks.workunit.client.0.smithi114.stdout:6/568: dwrite d6/dd/d25/d34/d61/f6a [0,4194304] 0 2024-02-16T22:22:05.035 INFO:tasks.workunit.client.0.smithi114.stdout:7/605: mkdir d3/d87/dd1 0 2024-02-16T22:22:05.036 INFO:tasks.workunit.client.1.smithi123.stdout:9/816: creat d4/d36/d8e/d6f/f107 x:0 0 0 2024-02-16T22:22:05.037 INFO:tasks.workunit.client.1.smithi123.stdout:2/688: write d4/d1a/d1d/d13/d15/d6d/db1/fb7 [3535423,21465] 0 2024-02-16T22:22:05.038 INFO:tasks.workunit.client.1.smithi123.stdout:2/689: stat d4/d1a/d1d/d13/d15/l5a 0 2024-02-16T22:22:05.040 INFO:tasks.workunit.client.1.smithi123.stdout:6/714: dread d0/d6/d91/d93/f95 [0,4194304] 0 2024-02-16T22:22:05.041 INFO:tasks.workunit.client.0.smithi114.stdout:3/568: dread - d4/d43/f6e zero size 2024-02-16T22:22:05.041 INFO:tasks.workunit.client.0.smithi114.stdout:6/569: chown d6/l9 86394294 1 2024-02-16T22:22:05.041 INFO:tasks.workunit.client.0.smithi114.stdout:7/606: write d3/d22/d73/da7/fbc [1032055,128463] 0 2024-02-16T22:22:05.043 INFO:tasks.workunit.client.0.smithi114.stdout:3/569: fsync d4/d24/d2d/d45/d68/d77/fab 0 2024-02-16T22:22:05.044 INFO:tasks.workunit.client.1.smithi123.stdout:0/633: fdatasync d1/d8/d13/d1d/d62/fbd 0 2024-02-16T22:22:05.046 INFO:tasks.workunit.client.0.smithi114.stdout:8/535: creat d4/d16/d25/d46/fc2 x:0 0 0 2024-02-16T22:22:05.048 INFO:tasks.workunit.client.1.smithi123.stdout:8/758: write d0/d29/da7/d43/d34/f38 [4609375,490] 0 2024-02-16T22:22:05.049 INFO:tasks.workunit.client.0.smithi114.stdout:2/513: chown dd/df/d1c/l35 660162981 1 2024-02-16T22:22:05.049 INFO:tasks.workunit.client.0.smithi114.stdout:2/514: stat dd/f5d 0 2024-02-16T22:22:05.051 INFO:tasks.workunit.client.1.smithi123.stdout:5/684: readlink d6/dd/df/d58/l2d 0 2024-02-16T22:22:05.053 INFO:tasks.workunit.client.1.smithi123.stdout:7/708: mknod d3/d6e/d1d/db2/d3d/db1/ce6 0 2024-02-16T22:22:05.056 INFO:tasks.workunit.client.1.smithi123.stdout:0/634: dread d1/da0/fb9 [0,4194304] 0 2024-02-16T22:22:05.059 INFO:tasks.workunit.client.0.smithi114.stdout:2/515: dwrite dd/f17 [4194304,4194304] 0 2024-02-16T22:22:05.060 INFO:tasks.workunit.client.1.smithi123.stdout:3/729: getdents d0/d20/d29/d9b/dbd/df4/d5d/d60/d93 0 2024-02-16T22:22:05.061 INFO:tasks.workunit.client.0.smithi114.stdout:5/516: getdents d0/d28/d98/d1e/d63 0 2024-02-16T22:22:05.061 INFO:tasks.workunit.client.0.smithi114.stdout:5/517: write d0/d28/d98/f5b [1718888,105985] 0 2024-02-16T22:22:05.061 INFO:tasks.workunit.client.0.smithi114.stdout:2/516: truncate dd/d3b/da7/d4c/f80 509728 0 2024-02-16T22:22:05.064 INFO:tasks.workunit.client.1.smithi123.stdout:2/690: rmdir d4/d1a/d1d/d59 39 2024-02-16T22:22:05.064 INFO:tasks.workunit.client.0.smithi114.stdout:0/503: getdents d1/d16/d1f/d28/d51 0 2024-02-16T22:22:05.065 INFO:tasks.workunit.client.0.smithi114.stdout:5/518: write d0/d28/f38 [748186,26880] 0 2024-02-16T22:22:05.066 INFO:tasks.workunit.client.1.smithi123.stdout:3/730: dwrite d0/d20/d94/fbc [4194304,4194304] 0 2024-02-16T22:22:05.066 INFO:tasks.workunit.client.0.smithi114.stdout:0/504: read d1/d1a/f38 [204022,111572] 0 2024-02-16T22:22:05.067 INFO:tasks.workunit.client.1.smithi123.stdout:3/731: readlink d0/d50/l9e 0 2024-02-16T22:22:05.070 INFO:tasks.workunit.client.1.smithi123.stdout:9/817: dread d4/d1e/d3a/fc3 [0,4194304] 0 2024-02-16T22:22:05.071 INFO:tasks.workunit.client.1.smithi123.stdout:6/715: creat d0/d6/d91/da2/ffd x:0 0 0 2024-02-16T22:22:05.072 INFO:tasks.workunit.client.0.smithi114.stdout:7/607: mknod d3/d22/d73/da7/cd2 0 2024-02-16T22:22:05.076 INFO:tasks.workunit.client.0.smithi114.stdout:3/570: mkdir d4/d24/d2d/d45/d74/dc7 0 2024-02-16T22:22:05.077 INFO:tasks.workunit.client.1.smithi123.stdout:4/763: creat d5/d1c/d23/fe9 x:0 0 0 2024-02-16T22:22:05.078 INFO:tasks.workunit.client.0.smithi114.stdout:8/536: symlink d4/d11/d78/d7a/d7e/lc3 0 2024-02-16T22:22:05.084 INFO:tasks.workunit.client.0.smithi114.stdout:4/673: creat dd/d1b/d37/ff5 x:0 0 0 2024-02-16T22:22:05.085 INFO:tasks.workunit.client.1.smithi123.stdout:7/709: creat d3/d6e/d34/fe7 x:0 0 0 2024-02-16T22:22:05.086 INFO:tasks.workunit.client.0.smithi114.stdout:4/674: write dd/d1b/d24/fd6 [216508,127541] 0 2024-02-16T22:22:05.086 INFO:tasks.workunit.client.1.smithi123.stdout:7/710: dread - d3/d6e/d52/f72 zero size 2024-02-16T22:22:05.086 INFO:tasks.workunit.client.0.smithi114.stdout:4/675: chown dd/d59/f71 1884 1 2024-02-16T22:22:05.087 INFO:tasks.workunit.client.1.smithi123.stdout:1/695: link d3/d6/db/l1a d3/d6/db/d10/d4b/d70/d87/leb 0 2024-02-16T22:22:05.089 INFO:tasks.workunit.client.0.smithi114.stdout:9/578: getdents d2/dd/d3c/d89 0 2024-02-16T22:22:05.094 INFO:tasks.workunit.client.1.smithi123.stdout:2/691: chown d4/d1a/d1d/d13/d15/d6d/c7f 15673 1 2024-02-16T22:22:05.094 INFO:tasks.workunit.client.0.smithi114.stdout:6/570: dread d6/d2c/d38/d41/d60/f65 [0,4194304] 0 2024-02-16T22:22:05.095 INFO:tasks.workunit.client.1.smithi123.stdout:2/692: write d4/d1a/d1d/d38/d7b/f94 [282572,2103] 0 2024-02-16T22:22:05.098 INFO:tasks.workunit.client.0.smithi114.stdout:6/571: dread d6/dd/d25/f73 [0,4194304] 0 2024-02-16T22:22:05.098 INFO:tasks.workunit.client.0.smithi114.stdout:5/519: rename d0/d28/f71 to d0/d28/d98/d1e/fa4 0 2024-02-16T22:22:05.100 INFO:tasks.workunit.client.1.smithi123.stdout:9/818: symlink d4/d36/d8e/d6d/l108 0 2024-02-16T22:22:05.101 INFO:tasks.workunit.client.1.smithi123.stdout:9/819: chown d4/d36/d8e/d6d/d71/fe8 103 1 2024-02-16T22:22:05.101 INFO:tasks.workunit.client.0.smithi114.stdout:0/505: fdatasync d1/d16/d1f/d28/d51/d61/d7e/d90/f42 0 2024-02-16T22:22:05.103 INFO:tasks.workunit.client.0.smithi114.stdout:7/608: symlink d3/d46/ld3 0 2024-02-16T22:22:05.104 INFO:tasks.workunit.client.0.smithi114.stdout:1/579: truncate d4/f30 96063 0 2024-02-16T22:22:05.105 INFO:tasks.workunit.client.0.smithi114.stdout:1/580: dread - d4/d63/d69/d7c/fbd zero size 2024-02-16T22:22:05.108 INFO:tasks.workunit.client.0.smithi114.stdout:0/506: dread d1/d16/d1f/d28/d51/d61/d6d/d6e/f6f [0,4194304] 0 2024-02-16T22:22:05.109 INFO:tasks.workunit.client.1.smithi123.stdout:0/635: dwrite d1/f12 [0,4194304] 0 2024-02-16T22:22:05.110 INFO:tasks.workunit.client.0.smithi114.stdout:0/507: write d1/d16/f73 [374840,16401] 0 2024-02-16T22:22:05.110 INFO:tasks.workunit.client.1.smithi123.stdout:0/636: chown d1/d8/d13/d1d/d64/f84 39548 1 2024-02-16T22:22:05.112 INFO:tasks.workunit.client.0.smithi114.stdout:4/676: symlink dd/d1b/d37/d38/d87/lf6 0 2024-02-16T22:22:05.113 INFO:tasks.workunit.client.1.smithi123.stdout:5/685: mknod d6/dd/df/d58/d3f/dbe/d37/cef 0 2024-02-16T22:22:05.114 INFO:tasks.workunit.client.0.smithi114.stdout:9/579: truncate d2/d1a/d23/f28 410143 0 2024-02-16T22:22:05.117 INFO:tasks.workunit.client.0.smithi114.stdout:2/517: mknod dd/df/d1c/caa 0 2024-02-16T22:22:05.120 INFO:tasks.workunit.client.1.smithi123.stdout:2/693: unlink d4/d1a/d1d/d13/f3c 0 2024-02-16T22:22:05.122 INFO:tasks.workunit.client.0.smithi114.stdout:5/520: symlink d0/d28/d98/d25/d2e/la5 0 2024-02-16T22:22:05.122 INFO:tasks.workunit.client.0.smithi114.stdout:0/508: dread d1/d1a/f26 [0,4194304] 0 2024-02-16T22:22:05.124 INFO:tasks.workunit.client.1.smithi123.stdout:9/820: symlink d4/d36/d8e/d58/d59/d60/d101/l109 0 2024-02-16T22:22:05.127 INFO:tasks.workunit.client.0.smithi114.stdout:7/609: rename d3/f6 to d3/d22/d73/da7/d6f/fd4 0 2024-02-16T22:22:05.127 INFO:tasks.workunit.client.1.smithi123.stdout:6/716: creat d0/d8a/de6/ffe x:0 0 0 2024-02-16T22:22:05.129 INFO:tasks.workunit.client.0.smithi114.stdout:3/571: creat d4/d46/d48/d80/dbd/fc8 x:0 0 0 2024-02-16T22:22:05.130 INFO:tasks.workunit.client.1.smithi123.stdout:4/764: mkdir d5/d1c/d23/dea 0 2024-02-16T22:22:05.131 INFO:tasks.workunit.client.0.smithi114.stdout:4/677: rmdir dd/d1b/d41/d74 39 2024-02-16T22:22:05.132 INFO:tasks.workunit.client.1.smithi123.stdout:8/759: link d0/d61/d66/d7e/f89 d0/d29/da7/d26/d68/ff3 0 2024-02-16T22:22:05.134 INFO:tasks.workunit.client.0.smithi114.stdout:4/678: truncate dd/d1b/d37/d38/d87/dad/fbf 1858843 0 2024-02-16T22:22:05.134 INFO:tasks.workunit.client.0.smithi114.stdout:9/580: creat d2/d5c/d5f/d72/fb8 x:0 0 0 2024-02-16T22:22:05.137 INFO:tasks.workunit.client.0.smithi114.stdout:8/537: dread d4/d11/f1f [0,4194304] 0 2024-02-16T22:22:05.137 INFO:tasks.workunit.client.1.smithi123.stdout:5/686: creat d6/dd/df/d58/d3f/d6f/ff0 x:0 0 0 2024-02-16T22:22:05.138 INFO:tasks.workunit.client.1.smithi123.stdout:5/687: readlink d6/l38 0 2024-02-16T22:22:05.138 INFO:tasks.workunit.client.0.smithi114.stdout:5/521: dread d0/d28/d98/d1e/f21 [0,4194304] 0 2024-02-16T22:22:05.139 INFO:tasks.workunit.client.0.smithi114.stdout:2/518: creat dd/d3b/fab x:0 0 0 2024-02-16T22:22:05.140 INFO:tasks.workunit.client.0.smithi114.stdout:5/522: chown d0/d28/d98/d25/d3e/d7f 64459 1 2024-02-16T22:22:05.141 INFO:tasks.workunit.client.0.smithi114.stdout:5/523: chown d0/d28/d98/d25/d89 6 1 2024-02-16T22:22:05.144 INFO:tasks.workunit.client.0.smithi114.stdout:2/519: read dd/df/d1c/d21/d61/d63/f73 [783878,31881] 0 2024-02-16T22:22:05.145 INFO:tasks.workunit.client.0.smithi114.stdout:2/520: chown dd/c6f 3868473 1 2024-02-16T22:22:05.148 INFO:tasks.workunit.client.0.smithi114.stdout:0/509: mknod d1/d16/d1f/d28/d51/d61/d7e/d90/d95/d80/c99 0 2024-02-16T22:22:05.152 INFO:tasks.workunit.client.1.smithi123.stdout:3/732: mknod d0/d20/d5a/da6/cf7 0 2024-02-16T22:22:05.152 INFO:tasks.workunit.client.1.smithi123.stdout:9/821: mkdir d4/d36/daa/d20/dcb/d10a 0 2024-02-16T22:22:05.153 INFO:tasks.workunit.client.0.smithi114.stdout:7/610: write d3/d22/f8f [1651898,106102] 0 2024-02-16T22:22:05.155 INFO:tasks.workunit.client.1.smithi123.stdout:6/717: creat d0/d6/d4d/fff x:0 0 0 2024-02-16T22:22:05.156 INFO:tasks.workunit.client.0.smithi114.stdout:3/572: rmdir d4/d43/d53 39 2024-02-16T22:22:05.157 INFO:tasks.workunit.client.1.smithi123.stdout:5/688: dread d6/dd/df/d58/d3f/dbe/d37/f6e [0,4194304] 0 2024-02-16T22:22:05.157 INFO:tasks.workunit.client.0.smithi114.stdout:3/573: chown d4/d46/d48/d80/dbd 161221540 1 2024-02-16T22:22:05.158 INFO:tasks.workunit.client.1.smithi123.stdout:4/765: creat d5/d1c/d23/d56/d6e/d9f/feb x:0 0 0 2024-02-16T22:22:05.159 INFO:tasks.workunit.client.0.smithi114.stdout:3/574: truncate d4/dd/d33/f5f 1387780 0 2024-02-16T22:22:05.160 INFO:tasks.workunit.client.0.smithi114.stdout:3/575: fsync d4/d46/d48/f57 0 2024-02-16T22:22:05.162 INFO:tasks.workunit.client.1.smithi123.stdout:8/760: mkdir d0/d61/dac/df4 0 2024-02-16T22:22:05.163 INFO:tasks.workunit.client.1.smithi123.stdout:4/766: dwrite d5/d1c/d64/d7b/fb1 [0,4194304] 0 2024-02-16T22:22:05.167 INFO:tasks.workunit.client.1.smithi123.stdout:8/761: dread d0/d29/da7/d26/d68/d81/d47/fdb [0,4194304] 0 2024-02-16T22:22:05.174 INFO:tasks.workunit.client.1.smithi123.stdout:2/694: rename d4/d1a/d1d/d59/d53 to d4/d1a/d1d/d59/dd7/ddc 0 2024-02-16T22:22:05.175 INFO:tasks.workunit.client.0.smithi114.stdout:2/521: symlink dd/d3b/d8a/lac 0 2024-02-16T22:22:05.175 INFO:tasks.workunit.client.1.smithi123.stdout:3/733: rmdir d0/d20/d5a 39 2024-02-16T22:22:05.177 INFO:tasks.workunit.client.1.smithi123.stdout:3/734: fdatasync d0/d20/fdf 0 2024-02-16T22:22:05.177 INFO:tasks.workunit.client.1.smithi123.stdout:8/762: read d0/d29/da7/d26/d2a/d77/f84 [1402562,30458] 0 2024-02-16T22:22:05.177 INFO:tasks.workunit.client.0.smithi114.stdout:6/572: write d6/dd/f29 [1706833,64284] 0 2024-02-16T22:22:05.178 INFO:tasks.workunit.client.1.smithi123.stdout:9/822: rmdir d4/d36/d8e/d6d/d71/da6 39 2024-02-16T22:22:05.179 INFO:tasks.workunit.client.1.smithi123.stdout:1/696: truncate d3/d6/d11/f17 7579797 0 2024-02-16T22:22:05.180 INFO:tasks.workunit.client.1.smithi123.stdout:9/823: readlink d4/d36/l54 0 2024-02-16T22:22:05.180 INFO:tasks.workunit.client.1.smithi123.stdout:7/711: dwrite d3/d6e/d24/f6d [0,4194304] 0 2024-02-16T22:22:05.182 INFO:tasks.workunit.client.1.smithi123.stdout:7/712: truncate d3/d6e/d4d/d92/fd1 1565525 0 2024-02-16T22:22:05.191 INFO:tasks.workunit.client.1.smithi123.stdout:5/689: dread d6/dc1/de6/f60 [0,4194304] 0 2024-02-16T22:22:05.194 INFO:tasks.workunit.client.1.smithi123.stdout:0/637: truncate d1/d8/d13/f4d 808742 0 2024-02-16T22:22:05.195 INFO:tasks.workunit.client.1.smithi123.stdout:4/767: symlink d5/d1c/d9a/dd8/lec 0 2024-02-16T22:22:05.199 INFO:tasks.workunit.client.1.smithi123.stdout:9/824: dread d4/d36/d8e/d6d/d71/d92/ff6 [0,4194304] 0 2024-02-16T22:22:05.201 INFO:tasks.workunit.client.0.smithi114.stdout:0/510: dread d1/d16/d1f/d28/f35 [0,4194304] 0 2024-02-16T22:22:05.202 INFO:tasks.workunit.client.0.smithi114.stdout:8/538: dwrite d4/d16/d25/f6e [0,4194304] 0 2024-02-16T22:22:05.205 INFO:tasks.workunit.client.1.smithi123.stdout:2/695: symlink d4/d1a/d1d/d13/d15/d6d/ldd 0 2024-02-16T22:22:05.206 INFO:tasks.workunit.client.1.smithi123.stdout:5/690: dread d6/dd/df/d58/d56/f71 [0,4194304] 0 2024-02-16T22:22:05.206 INFO:tasks.workunit.client.1.smithi123.stdout:3/735: read - d0/da0/fba zero size 2024-02-16T22:22:05.209 INFO:tasks.workunit.client.1.smithi123.stdout:8/763: symlink d0/d29/lf5 0 2024-02-16T22:22:05.211 INFO:tasks.workunit.client.0.smithi114.stdout:6/573: truncate d6/f21 849255 0 2024-02-16T22:22:05.212 INFO:tasks.workunit.client.0.smithi114.stdout:5/524: truncate d0/d28/d98/d25/d2e/d3b/f59 2409152 0 2024-02-16T22:22:05.214 INFO:tasks.workunit.client.1.smithi123.stdout:6/718: fsync d0/d6/d4d/db0/ffb 0 2024-02-16T22:22:05.216 INFO:tasks.workunit.client.1.smithi123.stdout:6/719: write d0/d96/de3/ff7 [6379788,40255] 0 2024-02-16T22:22:05.216 INFO:tasks.workunit.client.1.smithi123.stdout:1/697: mkdir d3/d6/d49/d48/d84/d92/dec 0 2024-02-16T22:22:05.219 INFO:tasks.workunit.client.1.smithi123.stdout:7/713: symlink d3/d87/le8 0 2024-02-16T22:22:05.219 INFO:tasks.workunit.client.1.smithi123.stdout:7/714: readlink d3/d6e/d1d/db2/d35/lc2 0 2024-02-16T22:22:05.220 INFO:tasks.workunit.client.1.smithi123.stdout:7/715: write d3/d87/fd7 [4741906,123513] 0 2024-02-16T22:22:05.222 INFO:tasks.workunit.client.1.smithi123.stdout:7/716: write d3/d6e/d34/fe7 [271565,626] 0 2024-02-16T22:22:05.223 INFO:tasks.workunit.client.0.smithi114.stdout:9/581: creat d2/d1a/d65/fb9 x:0 0 0 2024-02-16T22:22:05.225 INFO:tasks.workunit.client.1.smithi123.stdout:4/768: chown d5/d1c/d23/d56/d6e/d73/d87/fe4 0 1 2024-02-16T22:22:05.225 INFO:tasks.workunit.client.1.smithi123.stdout:4/769: read - d5/d1c/d23/d56/d6e/fe7 zero size 2024-02-16T22:22:05.226 INFO:tasks.workunit.client.1.smithi123.stdout:4/770: truncate d5/d1c/d23/d56/d6e/fe7 954087 0 2024-02-16T22:22:05.227 INFO:tasks.workunit.client.0.smithi114.stdout:8/539: rmdir d4/d16/d25/d46 39 2024-02-16T22:22:05.232 INFO:tasks.workunit.client.0.smithi114.stdout:2/522: mknod dd/cad 0 2024-02-16T22:22:05.236 INFO:tasks.workunit.client.1.smithi123.stdout:5/691: creat d6/dd/df/d1f/ddc/ff1 x:0 0 0 2024-02-16T22:22:05.238 INFO:tasks.workunit.client.0.smithi114.stdout:1/581: rename d4/l26 to d4/d9/d46/ld8 0 2024-02-16T22:22:05.240 INFO:tasks.workunit.client.1.smithi123.stdout:3/736: mknod d0/d20/d29/d9b/dbd/df4/d5d/d60/cf8 0 2024-02-16T22:22:05.243 INFO:tasks.workunit.client.0.smithi114.stdout:4/679: getdents dd/d1b/d41/dc3 0 2024-02-16T22:22:05.244 INFO:tasks.workunit.client.0.smithi114.stdout:3/576: truncate d4/dd/f1e 8076323 0 2024-02-16T22:22:05.245 INFO:tasks.workunit.client.0.smithi114.stdout:4/680: dread - dd/d1b/d37/d38/d87/dad/ff1 zero size 2024-02-16T22:22:05.248 INFO:tasks.workunit.client.0.smithi114.stdout:8/540: chown d4/d16/f4b 3 1 2024-02-16T22:22:05.249 INFO:tasks.workunit.client.1.smithi123.stdout:1/698: creat d3/d47/fed x:0 0 0 2024-02-16T22:22:05.251 INFO:tasks.workunit.client.0.smithi114.stdout:5/525: rename d0/d28/d98/d1e/f27 to d0/d28/d98/d1e/d63/d96/d9a/fa6 0 2024-02-16T22:22:05.252 INFO:tasks.workunit.client.1.smithi123.stdout:0/638: mknod d1/d8/cde 0 2024-02-16T22:22:05.253 INFO:tasks.workunit.client.0.smithi114.stdout:7/611: getdents d3/d22/d73 0 2024-02-16T22:22:05.253 INFO:tasks.workunit.client.1.smithi123.stdout:0/639: write d1/d8/d13/d1d/d62/d7a/f7f [1990058,2620] 0 2024-02-16T22:22:05.256 INFO:tasks.workunit.client.1.smithi123.stdout:4/771: fsync d5/d1c/d23/d76/f9e 0 2024-02-16T22:22:05.256 INFO:tasks.workunit.client.0.smithi114.stdout:4/681: mkdir dd/d59/d8f/df7 0 2024-02-16T22:22:05.256 INFO:tasks.workunit.client.0.smithi114.stdout:3/577: creat d4/d87/fc9 x:0 0 0 2024-02-16T22:22:05.258 INFO:tasks.workunit.client.1.smithi123.stdout:9/825: unlink d4/d36/d8e/d6d/d71/fa3 0 2024-02-16T22:22:05.262 INFO:tasks.workunit.client.1.smithi123.stdout:5/692: rename d6/dd/df/d1f/f5c to d6/dd/df/d90/ff2 0 2024-02-16T22:22:05.262 INFO:tasks.workunit.client.1.smithi123.stdout:5/693: stat d6/dac 0 2024-02-16T22:22:05.263 INFO:tasks.workunit.client.0.smithi114.stdout:8/541: write d4/d11/d78/d7a/d36/f5d [4635956,36345] 0 2024-02-16T22:22:05.263 INFO:tasks.workunit.client.1.smithi123.stdout:5/694: fdatasync d6/dc1/de6/fd6 0 2024-02-16T22:22:05.264 INFO:tasks.workunit.client.1.smithi123.stdout:8/764: dread - d0/d29/da7/d43/d34/fd8 zero size 2024-02-16T22:22:05.265 INFO:tasks.workunit.client.1.smithi123.stdout:1/699: chown d3/d6/db/d10/d2a/cd9 1397790 1 2024-02-16T22:22:05.266 INFO:tasks.workunit.client.0.smithi114.stdout:7/612: mkdir d3/d92/dd5 0 2024-02-16T22:22:05.266 INFO:tasks.workunit.client.1.smithi123.stdout:4/772: dread d5/d1c/d23/f2a [0,4194304] 0 2024-02-16T22:22:05.268 INFO:tasks.workunit.client.0.smithi114.stdout:4/682: rmdir dd/d2e/d56/ddc 39 2024-02-16T22:22:05.272 INFO:tasks.workunit.client.1.smithi123.stdout:1/700: dwrite d3/d6/d11/f62 [0,4194304] 0 2024-02-16T22:22:05.272 INFO:tasks.workunit.client.1.smithi123.stdout:2/696: link d4/d1a/d1d/d38/f90 d4/dda/fde 0 2024-02-16T22:22:05.274 INFO:tasks.workunit.client.0.smithi114.stdout:0/511: getdents d1/d16 0 2024-02-16T22:22:05.284 INFO:tasks.workunit.client.1.smithi123.stdout:8/765: dread d0/df/f1c [0,4194304] 0 2024-02-16T22:22:05.284 INFO:tasks.workunit.client.0.smithi114.stdout:5/526: mknod d0/d28/d98/d1e/d63/d9e/ca7 0 2024-02-16T22:22:05.286 INFO:tasks.workunit.client.1.smithi123.stdout:5/695: mknod d6/dd/df/d58/d3f/cf3 0 2024-02-16T22:22:05.287 INFO:tasks.workunit.client.1.smithi123.stdout:5/696: fdatasync d6/dd/df/d58/d3f/dbe/f5d 0 2024-02-16T22:22:05.289 INFO:tasks.workunit.client.1.smithi123.stdout:5/697: truncate d6/dd/df/d58/d3f/dbe/f5d 4546802 0 2024-02-16T22:22:05.290 INFO:tasks.workunit.client.1.smithi123.stdout:5/698: readlink d6/dd/df/d58/d3f/d6f/l87 0 2024-02-16T22:22:05.295 INFO:tasks.workunit.client.0.smithi114.stdout:3/578: rename d4/d24/d2d/d45/d68/d77/c85 to d4/d24/d2d/d45/d74/dc7/cca 0 2024-02-16T22:22:05.295 INFO:tasks.workunit.client.0.smithi114.stdout:6/574: dwrite d6/f21 [0,4194304] 0 2024-02-16T22:22:05.296 INFO:tasks.workunit.client.1.smithi123.stdout:4/773: symlink d5/d1c/d23/d74/d92/led 0 2024-02-16T22:22:05.300 INFO:tasks.workunit.client.1.smithi123.stdout:4/774: read d5/d1c/d5e/f72 [453174,48630] 0 2024-02-16T22:22:05.300 INFO:tasks.workunit.client.1.smithi123.stdout:4/775: readlink d5/d1c/d5e/le2 0 2024-02-16T22:22:05.306 INFO:tasks.workunit.client.1.smithi123.stdout:4/776: dwrite d5/d1c/d23/d74/d92/d9b/dbd/fcb [0,4194304] 0 2024-02-16T22:22:05.307 INFO:tasks.workunit.client.1.smithi123.stdout:4/777: chown d5/le 8 1 2024-02-16T22:22:05.308 INFO:tasks.workunit.client.0.smithi114.stdout:2/523: truncate dd/f92 2630619 0 2024-02-16T22:22:05.308 INFO:tasks.workunit.client.0.smithi114.stdout:9/582: dwrite d2/dd/f3b [0,4194304] 0 2024-02-16T22:22:05.310 INFO:tasks.workunit.client.0.smithi114.stdout:2/524: write dd/df/f33 [3441928,99867] 0 2024-02-16T22:22:05.311 INFO:tasks.workunit.client.0.smithi114.stdout:9/583: readlink d2/dd/d3c/d89/d9e/la0 0 2024-02-16T22:22:05.312 INFO:tasks.workunit.client.1.smithi123.stdout:3/737: creat d0/d20/d29/d9b/dbd/df4/ff9 x:0 0 0 2024-02-16T22:22:05.315 INFO:tasks.workunit.client.0.smithi114.stdout:5/527: dread d0/df/f1d [0,4194304] 0 2024-02-16T22:22:05.316 INFO:tasks.workunit.client.1.smithi123.stdout:8/766: creat d0/d29/da7/d26/d32/ff6 x:0 0 0 2024-02-16T22:22:05.318 INFO:tasks.workunit.client.0.smithi114.stdout:4/683: creat dd/d1b/d41/d74/d9b/ff8 x:0 0 0 2024-02-16T22:22:05.318 INFO:tasks.workunit.client.1.smithi123.stdout:8/767: write d0/d29/da7/d43/fdf [89917,90674] 0 2024-02-16T22:22:05.319 INFO:tasks.workunit.client.0.smithi114.stdout:4/684: write dd/d1b/d24/fe6 [997704,98503] 0 2024-02-16T22:22:05.320 INFO:tasks.workunit.client.0.smithi114.stdout:4/685: fdatasync dd/d2e/fdf 0 2024-02-16T22:22:05.324 INFO:tasks.workunit.client.1.smithi123.stdout:5/699: mknod d6/dd/d78/da5/cf4 0 2024-02-16T22:22:05.326 INFO:tasks.workunit.client.0.smithi114.stdout:6/575: creat d6/d2c/d38/d54/fad x:0 0 0 2024-02-16T22:22:05.327 INFO:tasks.workunit.client.0.smithi114.stdout:6/576: fsync d6/dd/d25/d34/d58/f90 0 2024-02-16T22:22:05.332 INFO:tasks.workunit.client.0.smithi114.stdout:7/613: dwrite d3/d22/d73/da7/d6f/fd4 [4194304,4194304] 0 2024-02-16T22:22:05.339 INFO:tasks.workunit.client.1.smithi123.stdout:4/778: symlink d5/d1c/d64/lee 0 2024-02-16T22:22:05.340 INFO:tasks.workunit.client.0.smithi114.stdout:7/614: dwrite d3/d22/d73/da7/d6f/fd4 [4194304,4194304] 0 2024-02-16T22:22:05.340 INFO:tasks.workunit.client.1.smithi123.stdout:4/779: write d5/d1c/d23/d56/d6e/d73/f88 [3192940,101760] 0 2024-02-16T22:22:05.342 INFO:tasks.workunit.client.1.smithi123.stdout:4/780: chown d5/d1c/d23/d56/d6e/d9f/da1 878113 1 2024-02-16T22:22:05.342 INFO:tasks.workunit.client.0.smithi114.stdout:7/615: read d3/fc [3080705,72095] 0 2024-02-16T22:22:05.343 INFO:tasks.workunit.client.1.smithi123.stdout:9/826: link d4/d36/daa/d2b/d6c/cc0 d4/d36/d8e/d58/d59/d60/d9d/de4/dfa/c10b 0 2024-02-16T22:22:05.346 INFO:tasks.workunit.client.0.smithi114.stdout:2/525: rmdir dd/df/d1c 39 2024-02-16T22:22:05.349 INFO:tasks.workunit.client.1.smithi123.stdout:3/738: read d0/d40/fd0 [116207,107581] 0 2024-02-16T22:22:05.349 INFO:tasks.workunit.client.0.smithi114.stdout:9/584: symlink d2/d5c/lba 0 2024-02-16T22:22:05.351 INFO:tasks.workunit.client.1.smithi123.stdout:9/827: dwrite d4/d36/d8e/d58/fe6 [0,4194304] 0 2024-02-16T22:22:05.351 INFO:tasks.workunit.client.1.smithi123.stdout:8/768: symlink d0/d29/da7/d26/d68/lf7 0 2024-02-16T22:22:05.352 INFO:tasks.workunit.client.1.smithi123.stdout:8/769: chown d0/d12/f3f 0 1 2024-02-16T22:22:05.354 INFO:tasks.workunit.client.0.smithi114.stdout:5/528: creat d0/d28/d98/d25/d3e/d7f/fa8 x:0 0 0 2024-02-16T22:22:05.355 INFO:tasks.workunit.client.1.smithi123.stdout:5/700: creat d6/dac/ff5 x:0 0 0 2024-02-16T22:22:05.358 INFO:tasks.workunit.client.1.smithi123.stdout:0/640: getdents d1/d4 0 2024-02-16T22:22:05.362 INFO:tasks.workunit.client.0.smithi114.stdout:0/512: truncate d1/d16/f5d 584626 0 2024-02-16T22:22:05.363 INFO:tasks.workunit.client.0.smithi114.stdout:5/529: dwrite d0/d28/d98/d25/d2e/f7d [0,4194304] 0 2024-02-16T22:22:05.364 INFO:tasks.workunit.client.0.smithi114.stdout:6/577: mknod d6/d2c/d75/cae 0 2024-02-16T22:22:05.364 INFO:tasks.workunit.client.0.smithi114.stdout:0/513: readlink d1/d1a/d47/d62/d89/l97 0 2024-02-16T22:22:05.370 INFO:tasks.workunit.client.1.smithi123.stdout:4/781: dread d5/d1c/d23/d47/da2/fb2 [0,4194304] 0 2024-02-16T22:22:05.373 INFO:tasks.workunit.client.1.smithi123.stdout:3/739: symlink d0/d20/d29/d9b/lfa 0 2024-02-16T22:22:05.373 INFO:tasks.workunit.client.0.smithi114.stdout:4/686: dread dd/f19 [0,4194304] 0 2024-02-16T22:22:05.375 INFO:tasks.workunit.client.0.smithi114.stdout:6/578: dwrite d6/d2c/d38/d41/d60/f98 [0,4194304] 0 2024-02-16T22:22:05.376 INFO:tasks.workunit.client.0.smithi114.stdout:7/616: creat d3/d22/d73/da7/d6f/fd6 x:0 0 0 2024-02-16T22:22:05.377 INFO:tasks.workunit.client.0.smithi114.stdout:7/617: chown d3/d22/d73/da7/l40 5474001 1 2024-02-16T22:22:05.379 INFO:tasks.workunit.client.0.smithi114.stdout:7/618: write d3/d22/d61/d6a/f9e [99442,4467] 0 2024-02-16T22:22:05.380 INFO:tasks.workunit.client.1.smithi123.stdout:8/770: rename d0/d29/da7/d26/d2a/d3c to d0/d29/da7/d26/d2a/d77/df8 0 2024-02-16T22:22:05.381 INFO:tasks.workunit.client.0.smithi114.stdout:9/585: mkdir d2/d1a/d23/d35/d47/d63/dbb 0 2024-02-16T22:22:05.390 INFO:tasks.workunit.client.1.smithi123.stdout:1/701: getdents d3/d6/d18 0 2024-02-16T22:22:05.391 INFO:tasks.workunit.client.1.smithi123.stdout:1/702: stat d3/d6/d18/d9e/fae 0 2024-02-16T22:22:05.393 INFO:tasks.workunit.client.1.smithi123.stdout:4/782: mknod d5/d49/cef 0 2024-02-16T22:22:05.396 INFO:tasks.workunit.client.0.smithi114.stdout:5/530: creat d0/d28/d98/d1e/d86/fa9 x:0 0 0 2024-02-16T22:22:05.398 INFO:tasks.workunit.client.1.smithi123.stdout:4/783: dwrite d5/f29 [4194304,4194304] 0 2024-02-16T22:22:05.398 INFO:tasks.workunit.client.1.smithi123.stdout:3/740: mkdir d0/d40/dfb 0 2024-02-16T22:22:05.399 INFO:tasks.workunit.client.1.smithi123.stdout:4/784: chown d5/d49/ca7 7127498 1 2024-02-16T22:22:05.399 INFO:tasks.workunit.client.1.smithi123.stdout:3/741: write d0/d20/d29/d9b/dbd/df4/d37/fa1 [578185,70025] 0 2024-02-16T22:22:05.400 INFO:tasks.workunit.client.0.smithi114.stdout:8/542: getdents d4/d24/d2c/d67 0 2024-02-16T22:22:05.402 INFO:tasks.workunit.client.1.smithi123.stdout:9/828: symlink d4/d36/d8e/d58/l10c 0 2024-02-16T22:22:05.404 INFO:tasks.workunit.client.1.smithi123.stdout:8/771: symlink d0/d12/ded/lf9 0 2024-02-16T22:22:05.404 INFO:tasks.workunit.client.1.smithi123.stdout:0/641: write d1/d8/d13/f5a [461157,107919] 0 2024-02-16T22:22:05.408 INFO:tasks.workunit.client.1.smithi123.stdout:5/701: dwrite d6/dd/df/d58/d3f/f9e [0,4194304] 0 2024-02-16T22:22:05.408 INFO:tasks.workunit.client.0.smithi114.stdout:4/687: write dd/d1b/d24/f76 [386845,90600] 0 2024-02-16T22:22:05.409 INFO:tasks.workunit.client.0.smithi114.stdout:6/579: creat d6/dd/d25/d34/d58/faf x:0 0 0 2024-02-16T22:22:05.411 INFO:tasks.workunit.client.0.smithi114.stdout:2/526: truncate dd/df/d1c/d21/d87/f5f 720995 0 2024-02-16T22:22:05.415 INFO:tasks.workunit.client.1.smithi123.stdout:1/703: dwrite d3/d6/d11/d76/f59 [4194304,4194304] 0 2024-02-16T22:22:05.416 INFO:tasks.workunit.client.1.smithi123.stdout:9/829: mkdir d4/d36/d8e/d58/d59/d60/deb/da7/dde/dfd/d10d 0 2024-02-16T22:22:05.419 INFO:tasks.workunit.client.1.smithi123.stdout:8/772: symlink d0/d29/da7/d26/d68/d81/lfa 0 2024-02-16T22:22:05.421 INFO:tasks.workunit.client.0.smithi114.stdout:5/531: rename d0/d28/d98/c1b to d0/d28/d98/d1e/caa 0 2024-02-16T22:22:05.422 INFO:tasks.workunit.client.0.smithi114.stdout:5/532: fdatasync d0/d2d/d30/f31 0 2024-02-16T22:22:05.423 INFO:tasks.workunit.client.0.smithi114.stdout:8/543: mknod d4/d11/d78/d7a/d36/cc4 0 2024-02-16T22:22:05.425 INFO:tasks.workunit.client.1.smithi123.stdout:5/702: creat d6/d3b/ff6 x:0 0 0 2024-02-16T22:22:05.427 INFO:tasks.workunit.client.1.smithi123.stdout:3/742: creat d0/d40/dfb/ffc x:0 0 0 2024-02-16T22:22:05.429 INFO:tasks.workunit.client.0.smithi114.stdout:6/580: rmdir d6/d2c 39 2024-02-16T22:22:05.432 INFO:tasks.workunit.client.0.smithi114.stdout:4/688: write dd/d2e/d56/ddc/dde/fe8 [491139,90086] 0 2024-02-16T22:22:05.433 INFO:tasks.workunit.client.0.smithi114.stdout:4/689: write dd/d1b/d41/fdd [357783,24960] 0 2024-02-16T22:22:05.436 INFO:tasks.workunit.client.1.smithi123.stdout:1/704: read d3/d6/d49/f66 [85407,41495] 0 2024-02-16T22:22:05.436 INFO:tasks.workunit.client.0.smithi114.stdout:7/619: link d3/d22/d61/f9d d3/d5a/fd7 0 2024-02-16T22:22:05.439 INFO:tasks.workunit.client.0.smithi114.stdout:0/514: truncate d1/f74 842887 0 2024-02-16T22:22:05.440 INFO:tasks.workunit.client.0.smithi114.stdout:7/620: write d3/d87/f8e [245362,71936] 0 2024-02-16T22:22:05.441 INFO:tasks.workunit.client.0.smithi114.stdout:9/586: write d2/d5c/d5f/d72/f79 [5783079,57889] 0 2024-02-16T22:22:05.444 INFO:tasks.workunit.client.1.smithi123.stdout:5/703: creat d6/dd/d78/da5/ff7 x:0 0 0 2024-02-16T22:22:05.445 INFO:tasks.workunit.client.0.smithi114.stdout:2/527: rename dd/d3b/f5c to dd/da6/fae 0 2024-02-16T22:22:05.446 INFO:tasks.workunit.client.0.smithi114.stdout:4/690: dwrite dd/d1b/d41/fe5 [0,4194304] 0 2024-02-16T22:22:05.446 INFO:tasks.workunit.client.0.smithi114.stdout:0/515: write d1/d1a/d3e/f4a [6783892,6716] 0 2024-02-16T22:22:05.446 INFO:tasks.workunit.client.0.smithi114.stdout:7/621: dread - d3/d22/d73/da7/d6f/fa3 zero size 2024-02-16T22:22:05.447 INFO:tasks.workunit.client.1.smithi123.stdout:3/743: dread - d0/d50/fa9 zero size 2024-02-16T22:22:05.448 INFO:tasks.workunit.client.1.smithi123.stdout:3/744: write d0/d20/d2e/f95 [4042008,38028] 0 2024-02-16T22:22:05.455 INFO:tasks.workunit.client.0.smithi114.stdout:9/587: dread d2/dd/d3c/d89/d9e/fa2 [0,4194304] 0 2024-02-16T22:22:05.459 INFO:tasks.workunit.client.1.smithi123.stdout:8/773: dread d0/d29/da7/d26/d32/d45/f5f [0,4194304] 0 2024-02-16T22:22:05.462 INFO:tasks.workunit.client.1.smithi123.stdout:0/642: getdents d1/d4/d17/d54/d8e/da5 0 2024-02-16T22:22:05.462 INFO:tasks.workunit.client.1.smithi123.stdout:9/830: dwrite d4/d36/d8e/d58/d59/fd5 [0,4194304] 0 2024-02-16T22:22:05.462 INFO:tasks.workunit.client.0.smithi114.stdout:6/581: dread d6/dd/d25/f99 [0,4194304] 0 2024-02-16T22:22:05.462 INFO:tasks.workunit.client.1.smithi123.stdout:3/745: dwrite d0/d20/d29/d9b/dbd/df4/d5d/d60/dd6/f9f [0,4194304] 0 2024-02-16T22:22:05.467 INFO:tasks.workunit.client.1.smithi123.stdout:1/705: dwrite d3/d6/d11/fbf [0,4194304] 0 2024-02-16T22:22:05.468 INFO:tasks.workunit.client.1.smithi123.stdout:6/720: sync 2024-02-16T22:22:05.469 INFO:tasks.workunit.client.1.smithi123.stdout:1/706: readlink d3/d6/d11/l56 0 2024-02-16T22:22:05.470 INFO:tasks.workunit.client.1.smithi123.stdout:6/721: write d0/d8a/de6/ffe [866987,122569] 0 2024-02-16T22:22:05.470 INFO:tasks.workunit.client.0.smithi114.stdout:2/528: creat dd/d3b/da7/d4c/faf x:0 0 0 2024-02-16T22:22:05.472 INFO:tasks.workunit.client.1.smithi123.stdout:6/722: chown d0/d96/de3/f2c 341751 1 2024-02-16T22:22:05.476 INFO:tasks.workunit.client.1.smithi123.stdout:5/704: getdents d6/dd/df/d58/d56 0 2024-02-16T22:22:05.476 INFO:tasks.workunit.client.1.smithi123.stdout:5/705: chown d6/dd/df/d58/d89/f9c 383 1 2024-02-16T22:22:05.479 INFO:tasks.workunit.client.1.smithi123.stdout:1/707: rmdir d3/d6/db/d10/d4f 39 2024-02-16T22:22:05.481 INFO:tasks.workunit.client.0.smithi114.stdout:7/622: mknod d3/d22/d73/da7/d6f/d89/da6/cd8 0 2024-02-16T22:22:05.486 INFO:tasks.workunit.client.0.smithi114.stdout:8/544: link d4/fbb d4/d11/d78/d7a/d7e/fc5 0 2024-02-16T22:22:05.487 INFO:tasks.workunit.client.1.smithi123.stdout:6/723: rename d0/d9/dc/d3a/d70/dea to d0/d8a/d100 0 2024-02-16T22:22:05.488 INFO:tasks.workunit.client.0.smithi114.stdout:8/545: write d4/d11/d78/d7a/d7e/fba [987907,122915] 0 2024-02-16T22:22:05.491 INFO:tasks.workunit.client.0.smithi114.stdout:6/582: mknod d6/da/cb0 0 2024-02-16T22:22:05.493 INFO:tasks.workunit.client.1.smithi123.stdout:5/706: mknod d6/dd/df/d58/d3f/d82/cf8 0 2024-02-16T22:22:05.496 INFO:tasks.workunit.client.1.smithi123.stdout:1/708: creat d3/d6/db/d10/d2d/fee x:0 0 0 2024-02-16T22:22:05.498 INFO:tasks.workunit.client.1.smithi123.stdout:9/831: dread d4/d36/fa8 [0,4194304] 0 2024-02-16T22:22:05.498 INFO:tasks.workunit.client.1.smithi123.stdout:6/724: symlink d0/d9/dc/d39/l101 0 2024-02-16T22:22:05.498 INFO:tasks.workunit.client.1.smithi123.stdout:9/832: dread - d4/d36/d57/db2/fed zero size 2024-02-16T22:22:05.499 INFO:tasks.workunit.client.0.smithi114.stdout:5/533: truncate d0/fa 329287 0 2024-02-16T22:22:05.499 INFO:tasks.workunit.client.1.smithi123.stdout:6/725: stat d0/d9/dc/d3a/f40 0 2024-02-16T22:22:05.501 INFO:tasks.workunit.client.1.smithi123.stdout:8/774: rmdir d0/d29/da7/d26/d68/d81/d47/d9b/da2 0 2024-02-16T22:22:05.503 INFO:tasks.workunit.client.0.smithi114.stdout:7/623: unlink d3/d22/d73/da7/c72 0 2024-02-16T22:22:05.503 INFO:tasks.workunit.client.0.smithi114.stdout:2/529: dwrite dd/df/f33 [0,4194304] 0 2024-02-16T22:22:05.504 INFO:tasks.workunit.client.1.smithi123.stdout:5/707: creat d6/dd/df/d58/d56/ff9 x:0 0 0 2024-02-16T22:22:05.505 INFO:tasks.workunit.client.1.smithi123.stdout:5/708: chown d6/dd/df/d1f/le7 1395145182 1 2024-02-16T22:22:05.511 INFO:tasks.workunit.client.0.smithi114.stdout:9/588: write d2/d33/f57 [2605663,39373] 0 2024-02-16T22:22:05.513 INFO:tasks.workunit.client.1.smithi123.stdout:9/833: creat d4/d1e/d3a/dac/f10e x:0 0 0 2024-02-16T22:22:05.513 INFO:tasks.workunit.client.1.smithi123.stdout:9/834: chown d4/d36/f5f 170474 1 2024-02-16T22:22:05.514 INFO:tasks.workunit.client.0.smithi114.stdout:7/624: dwrite d3/d22/d73/da7/d6f/d89/fb2 [0,4194304] 0 2024-02-16T22:22:05.515 INFO:tasks.workunit.client.1.smithi123.stdout:6/726: symlink d0/d6/d2f/l102 0 2024-02-16T22:22:05.516 INFO:tasks.workunit.client.0.smithi114.stdout:7/625: chown d3/d22/d73/da7/d4e/c9b 13107419 1 2024-02-16T22:22:05.521 INFO:tasks.workunit.client.1.smithi123.stdout:1/709: rename d3/d6/db/d10/d2a/d53/lb2 to d3/d6/db/d10/d4f/lef 0 2024-02-16T22:22:05.522 INFO:tasks.workunit.client.1.smithi123.stdout:1/710: chown d3/d6/d11/ca8 2140628 1 2024-02-16T22:22:05.525 INFO:tasks.workunit.client.1.smithi123.stdout:8/775: getdents d0/d29/da7/d26/d68/dbe/dbf 0 2024-02-16T22:22:05.526 INFO:tasks.workunit.client.1.smithi123.stdout:8/776: dread - d0/d29/da7/d26/d68/d81/d8f/ff2 zero size 2024-02-16T22:22:05.526 INFO:tasks.workunit.client.1.smithi123.stdout:1/711: dwrite d3/d6/f26 [0,4194304] 0 2024-02-16T22:22:05.527 INFO:tasks.workunit.client.1.smithi123.stdout:5/709: symlink d6/dd/df/d58/d3f/dbe/lfa 0 2024-02-16T22:22:05.529 INFO:tasks.workunit.client.1.smithi123.stdout:9/835: mknod d4/d1e/d3a/dac/dc5/c10f 0 2024-02-16T22:22:05.532 INFO:tasks.workunit.client.1.smithi123.stdout:6/727: mknod d0/d9/dc/d3a/d70/de5/c103 0 2024-02-16T22:22:05.534 INFO:tasks.workunit.client.1.smithi123.stdout:5/710: dwrite d6/dd/df/d58/d3f/d6f/fc9 [0,4194304] 0 2024-02-16T22:22:05.535 INFO:tasks.workunit.client.1.smithi123.stdout:5/711: readlink d6/dd/d44/dba/ldd 0 2024-02-16T22:22:05.537 INFO:tasks.workunit.client.1.smithi123.stdout:1/712: write d3/d6/db/d10/d2a/fc1 [591702,68234] 0 2024-02-16T22:22:05.538 INFO:tasks.workunit.client.1.smithi123.stdout:2/697: sync 2024-02-16T22:22:05.538 INFO:tasks.workunit.client.1.smithi123.stdout:4/785: sync 2024-02-16T22:22:05.538 INFO:tasks.workunit.client.1.smithi123.stdout:7/717: sync 2024-02-16T22:22:05.541 INFO:tasks.workunit.client.1.smithi123.stdout:7/718: chown d3/d6e/d1d/f2f 37506 1 2024-02-16T22:22:05.541 INFO:tasks.workunit.client.1.smithi123.stdout:9/836: creat d4/d1e/d3a/dac/dc5/f110 x:0 0 0 2024-02-16T22:22:05.543 INFO:tasks.workunit.client.1.smithi123.stdout:5/712: dwrite d6/dd/df/d58/d3f/dbe/d37/f6e [0,4194304] 0 2024-02-16T22:22:05.547 INFO:tasks.workunit.client.0.smithi114.stdout:8/546: dwrite d4/d16/d25/d46/fbd [0,4194304] 0 2024-02-16T22:22:05.549 INFO:tasks.workunit.client.1.smithi123.stdout:6/728: mkdir d0/d6/d4d/db0/d104 0 2024-02-16T22:22:05.551 INFO:tasks.workunit.client.0.smithi114.stdout:8/547: dread d4/d16/d25/f6e [0,4194304] 0 2024-02-16T22:22:05.554 INFO:tasks.workunit.client.1.smithi123.stdout:2/698: creat d4/d1a/dd5/fdf x:0 0 0 2024-02-16T22:22:05.557 INFO:tasks.workunit.client.0.smithi114.stdout:2/530: rename dd/df/d1c/d21/d61/c6d to dd/df/d1c/d21/d61/d63/d83/cb0 0 2024-02-16T22:22:05.562 INFO:tasks.workunit.client.1.smithi123.stdout:4/786: symlink d5/d1c/lf0 0 2024-02-16T22:22:05.570 INFO:tasks.workunit.client.1.smithi123.stdout:3/746: sync 2024-02-16T22:22:05.570 INFO:tasks.workunit.client.1.smithi123.stdout:0/643: sync 2024-02-16T22:22:05.572 INFO:tasks.workunit.client.1.smithi123.stdout:3/747: write d0/d20/d29/d9b/dbd/df4/d5d/d60/d93/fce [1590358,95173] 0 2024-02-16T22:22:05.573 INFO:tasks.workunit.client.1.smithi123.stdout:5/713: mknod d6/dd/df/d58/d56/cfb 0 2024-02-16T22:22:05.575 INFO:tasks.workunit.client.0.smithi114.stdout:0/516: getdents d1/d16/d1f 0 2024-02-16T22:22:05.577 INFO:tasks.workunit.client.0.smithi114.stdout:0/517: write d1/fe [5632322,12569] 0 2024-02-16T22:22:05.577 INFO:tasks.workunit.client.0.smithi114.stdout:0/518: stat d1/c9 0 2024-02-16T22:22:05.579 INFO:tasks.workunit.client.0.smithi114.stdout:0/519: write d1/d16/d1f/d28/d51/d61/d7e/d90/f85 [866920,89331] 0 2024-02-16T22:22:05.580 INFO:tasks.workunit.client.1.smithi123.stdout:2/699: mknod d4/d1a/d1d/d13/d15/d6d/d4a/dce/dd4/ce0 0 2024-02-16T22:22:05.580 INFO:tasks.workunit.client.0.smithi114.stdout:0/520: chown d1/f6c 51 1 2024-02-16T22:22:05.582 INFO:tasks.workunit.client.0.smithi114.stdout:8/548: chown d4/l5 127049749 1 2024-02-16T22:22:05.583 INFO:tasks.workunit.client.0.smithi114.stdout:8/549: dread - d4/faa zero size 2024-02-16T22:22:05.585 INFO:tasks.workunit.client.0.smithi114.stdout:8/550: chown d4/d11/d78/d7a/d7e/fae 24 1 2024-02-16T22:22:05.585 INFO:tasks.workunit.client.1.smithi123.stdout:1/713: mkdir d3/d6/db/d10/d4f/d9f/db4/ddf/df0 0 2024-02-16T22:22:05.586 INFO:tasks.workunit.client.0.smithi114.stdout:2/531: read - dd/d3b/d8a/f91 zero size 2024-02-16T22:22:05.587 INFO:tasks.workunit.client.0.smithi114.stdout:9/589: mknod d2/d1a/d23/d35/d47/d63/dbb/cbc 0 2024-02-16T22:22:05.590 INFO:tasks.workunit.client.0.smithi114.stdout:6/583: write d6/d2c/d38/f92 [794887,19401] 0 2024-02-16T22:22:05.592 INFO:tasks.workunit.client.1.smithi123.stdout:8/777: dwrite d0/d29/da7/d26/d68/d81/d47/fdb [0,4194304] 0 2024-02-16T22:22:05.595 INFO:tasks.workunit.client.1.smithi123.stdout:3/748: creat d0/d20/d2e/ffd x:0 0 0 2024-02-16T22:22:05.596 INFO:tasks.workunit.client.1.smithi123.stdout:3/749: write d0/d20/d2e/ffd [557629,101541] 0 2024-02-16T22:22:05.597 INFO:tasks.workunit.client.1.smithi123.stdout:3/750: write d0/d40/dfb/ffc [1045777,92388] 0 2024-02-16T22:22:05.600 INFO:tasks.workunit.client.0.smithi114.stdout:6/584: dwrite d6/d2c/d38/d41/f4a [0,4194304] 0 2024-02-16T22:22:05.602 INFO:tasks.workunit.client.0.smithi114.stdout:6/585: write d6/d2c/f7c [2473899,62256] 0 2024-02-16T22:22:05.604 INFO:tasks.workunit.client.0.smithi114.stdout:0/521: creat d1/d16/d1f/f9a x:0 0 0 2024-02-16T22:22:05.606 INFO:tasks.workunit.client.0.smithi114.stdout:5/534: write d0/d28/d98/d1e/fa4 [183926,4157] 0 2024-02-16T22:22:05.607 INFO:tasks.workunit.client.1.smithi123.stdout:1/714: truncate d3/d6/db/f28 764216 0 2024-02-16T22:22:05.607 INFO:tasks.workunit.client.0.smithi114.stdout:5/535: dread - d0/df/fa2 zero size 2024-02-16T22:22:05.611 INFO:tasks.workunit.client.0.smithi114.stdout:8/551: mkdir d4/d11/d78/d7a/d31/dc6 0 2024-02-16T22:22:05.614 INFO:tasks.workunit.client.1.smithi123.stdout:7/719: link d3/d87/c21 d3/d6e/d52/ce9 0 2024-02-16T22:22:05.615 INFO:tasks.workunit.client.0.smithi114.stdout:7/626: dwrite d3/fc [0,4194304] 0 2024-02-16T22:22:05.616 INFO:tasks.workunit.client.1.smithi123.stdout:9/837: dwrite d4/f6b [4194304,4194304] 0 2024-02-16T22:22:05.616 INFO:tasks.workunit.client.0.smithi114.stdout:9/590: symlink d2/dd/d3c/d89/d8b/lbd 0 2024-02-16T22:22:05.620 INFO:tasks.workunit.client.1.smithi123.stdout:9/838: read d4/d36/daa/f1a [4206707,20615] 0 2024-02-16T22:22:05.621 INFO:tasks.workunit.client.1.smithi123.stdout:6/729: dwrite d0/d6/d4d/db0/ffb [0,4194304] 0 2024-02-16T22:22:05.622 INFO:tasks.workunit.client.1.smithi123.stdout:8/778: unlink d0/d61/c94 0 2024-02-16T22:22:05.623 INFO:tasks.workunit.client.1.smithi123.stdout:6/730: chown d0/d9/dc/d3a/f80 15332878 1 2024-02-16T22:22:05.624 INFO:tasks.workunit.client.1.smithi123.stdout:5/714: creat d6/dd/df/d1f/ddc/dee/ffc x:0 0 0 2024-02-16T22:22:05.625 INFO:tasks.workunit.client.1.smithi123.stdout:3/751: chown d0/d20/c3b 0 1 2024-02-16T22:22:05.628 INFO:tasks.workunit.client.1.smithi123.stdout:1/715: creat d3/d6/db/d10/d4f/d9f/db4/ddf/ff1 x:0 0 0 2024-02-16T22:22:05.628 INFO:tasks.workunit.client.1.smithi123.stdout:4/787: dread d5/d1c/d23/d56/d6e/fe7 [0,4194304] 0 2024-02-16T22:22:05.628 INFO:tasks.workunit.client.1.smithi123.stdout:5/715: truncate d6/dd/df/d58/d3f/d6f/ff0 984454 0 2024-02-16T22:22:05.631 INFO:tasks.workunit.client.1.smithi123.stdout:9/839: dread d4/d36/d8e/dbe/fe3 [0,4194304] 0 2024-02-16T22:22:05.639 INFO:tasks.workunit.client.0.smithi114.stdout:7/627: symlink d3/d3d/ld9 0 2024-02-16T22:22:05.643 INFO:tasks.workunit.client.1.smithi123.stdout:0/644: write d1/d8/d13/f4d [674542,67761] 0 2024-02-16T22:22:05.644 INFO:tasks.workunit.client.0.smithi114.stdout:5/536: dread d0/d28/f46 [0,4194304] 0 2024-02-16T22:22:05.646 INFO:tasks.workunit.client.0.smithi114.stdout:0/522: symlink d1/d16/d1f/d28/d51/d61/d6d/d6e/l9b 0 2024-02-16T22:22:05.646 INFO:tasks.workunit.client.1.smithi123.stdout:6/731: symlink d0/d9/dc/d39/l105 0 2024-02-16T22:22:05.648 INFO:tasks.workunit.client.1.smithi123.stdout:8/779: rename d0/d29/da7/d43/d34/fdd to d0/d29/da7/d43/d34/ffb 0 2024-02-16T22:22:05.652 INFO:tasks.workunit.client.1.smithi123.stdout:3/752: mknod d0/d20/d29/d9b/dbd/df4/d5d/dae/db0/cfe 0 2024-02-16T22:22:05.653 INFO:tasks.workunit.client.0.smithi114.stdout:0/523: dwrite d1/d16/d48/f6a [0,4194304] 0 2024-02-16T22:22:05.655 INFO:tasks.workunit.client.0.smithi114.stdout:8/552: dread d4/f3c [0,4194304] 0 2024-02-16T22:22:05.656 INFO:tasks.workunit.client.0.smithi114.stdout:0/524: write d1/d16/d1f/d28/d51/d61/d7e/d90/d95/d80/f8f [130703,16498] 0 2024-02-16T22:22:05.657 INFO:tasks.workunit.client.0.smithi114.stdout:0/525: stat d1/d16/c1e 0 2024-02-16T22:22:05.660 INFO:tasks.workunit.client.0.smithi114.stdout:7/628: dread d3/d22/f8f [0,4194304] 0 2024-02-16T22:22:05.660 INFO:tasks.workunit.client.1.smithi123.stdout:1/716: mknod d3/d6/d49/d48/d84/db5/cf2 0 2024-02-16T22:22:05.662 INFO:tasks.workunit.client.0.smithi114.stdout:5/537: mknod d0/d28/d98/d1e/cab 0 2024-02-16T22:22:05.664 INFO:tasks.workunit.client.1.smithi123.stdout:9/840: mknod d4/d36/d8e/d6d/d71/d92/c111 0 2024-02-16T22:22:05.667 INFO:tasks.workunit.client.1.smithi123.stdout:7/720: chown d3/d5/d6c/c94 14 1 2024-02-16T22:22:05.673 INFO:tasks.workunit.client.0.smithi114.stdout:6/586: getdents d6 0 2024-02-16T22:22:05.681 INFO:tasks.workunit.client.1.smithi123.stdout:3/753: unlink d0/d50/d7e/cf3 0 2024-02-16T22:22:05.682 INFO:tasks.workunit.client.1.smithi123.stdout:3/754: stat d0/d20/d29/d9b/dbd/df4/d5d/d60/d93/cc9 0 2024-02-16T22:22:05.685 INFO:tasks.workunit.client.0.smithi114.stdout:2/532: link dd/df/d1c/f85 dd/df/d1c/d21/fb1 0 2024-02-16T22:22:05.689 INFO:tasks.workunit.client.0.smithi114.stdout:9/591: getdents d2 0 2024-02-16T22:22:05.690 INFO:tasks.workunit.client.1.smithi123.stdout:5/716: truncate d6/dd/df/d90/ff2 4686935 0 2024-02-16T22:22:05.692 INFO:tasks.workunit.client.1.smithi123.stdout:1/717: creat d3/d6/db/d10/d2d/d8c/ff3 x:0 0 0 2024-02-16T22:22:05.693 INFO:tasks.workunit.client.1.smithi123.stdout:4/788: dwrite d5/d1c/f30 [0,4194304] 0 2024-02-16T22:22:05.694 INFO:tasks.workunit.client.1.smithi123.stdout:8/780: dwrite d0/f6 [0,4194304] 0 2024-02-16T22:22:05.696 INFO:tasks.workunit.client.1.smithi123.stdout:9/841: rename d4/d33/l4b to d4/d1e/d3a/dac/dc5/l112 0 2024-02-16T22:22:05.696 INFO:tasks.workunit.client.0.smithi114.stdout:5/538: dread d0/d28/d98/d25/d2e/f69 [0,4194304] 0 2024-02-16T22:22:05.697 INFO:tasks.workunit.client.0.smithi114.stdout:5/539: dread - d0/d28/d98/d25/d3e/f82 zero size 2024-02-16T22:22:05.700 INFO:tasks.workunit.client.1.smithi123.stdout:9/842: truncate d4/d36/daa/d20/ff7 244568 0 2024-02-16T22:22:05.709 INFO:tasks.workunit.client.0.smithi114.stdout:6/587: mknod d6/dd/d25/cb1 0 2024-02-16T22:22:05.709 INFO:tasks.workunit.client.1.smithi123.stdout:2/700: sync 2024-02-16T22:22:05.710 INFO:tasks.workunit.client.0.smithi114.stdout:6/588: chown d6/d2c/d38/d54/f77 9643 1 2024-02-16T22:22:05.711 INFO:tasks.workunit.client.0.smithi114.stdout:9/592: symlink d2/d1a/d23/d35/d47/d63/dbb/lbe 0 2024-02-16T22:22:05.711 INFO:tasks.workunit.client.0.smithi114.stdout:0/526: truncate d1/f6c 325255 0 2024-02-16T22:22:05.712 INFO:tasks.workunit.client.0.smithi114.stdout:0/527: fsync d1/d78/f7d 0 2024-02-16T22:22:05.713 INFO:tasks.workunit.client.0.smithi114.stdout:2/533: read dd/df/d1c/f40 [2232856,35749] 0 2024-02-16T22:22:05.713 INFO:tasks.workunit.client.0.smithi114.stdout:0/528: read - d1/f81 zero size 2024-02-16T22:22:05.714 INFO:tasks.workunit.client.0.smithi114.stdout:7/629: rename d3/lb to d3/lda 0 2024-02-16T22:22:05.714 INFO:tasks.workunit.client.0.smithi114.stdout:2/534: chown dd/df/d1c/d21/c65 49284 1 2024-02-16T22:22:05.716 INFO:tasks.workunit.client.0.smithi114.stdout:5/540: rename d0/d2d/d30/l95 to d0/d28/d98/d1e/lac 0 2024-02-16T22:22:05.725 INFO:tasks.workunit.client.0.smithi114.stdout:7/630: mknod d3/d92/cdb 0 2024-02-16T22:22:05.732 INFO:tasks.workunit.client.0.smithi114.stdout:5/541: creat d0/d10/fad x:0 0 0 2024-02-16T22:22:05.735 INFO:tasks.workunit.client.0.smithi114.stdout:0/529: dread d1/d1a/f4e [0,4194304] 0 2024-02-16T22:22:05.737 INFO:tasks.workunit.client.0.smithi114.stdout:6/589: dread d6/d83/f87 [0,4194304] 0 2024-02-16T22:22:05.739 INFO:tasks.workunit.client.0.smithi114.stdout:6/590: rename d6 to d6/d2c/d38/d41/d60/d84/db2 22 2024-02-16T22:22:05.741 INFO:tasks.workunit.client.0.smithi114.stdout:0/530: dread d1/d16/d1f/d28/d51/d61/d7e/d90/f85 [0,4194304] 0 2024-02-16T22:22:05.741 INFO:tasks.workunit.client.0.smithi114.stdout:2/535: dread dd/df/f59 [0,4194304] 0 2024-02-16T22:22:05.752 INFO:tasks.workunit.client.0.smithi114.stdout:9/593: dwrite d2/f44 [4194304,4194304] 0 2024-02-16T22:22:05.754 INFO:tasks.workunit.client.1.smithi123.stdout:0/645: sync 2024-02-16T22:22:05.755 INFO:tasks.workunit.client.0.smithi114.stdout:0/531: creat d1/d16/d1f/d28/d51/d61/d6d/f9c x:0 0 0 2024-02-16T22:22:05.758 INFO:tasks.workunit.client.0.smithi114.stdout:5/542: creat d0/d2d/d30/d9f/fae x:0 0 0 2024-02-16T22:22:05.761 INFO:tasks.workunit.client.0.smithi114.stdout:2/536: creat dd/d3b/da7/d4c/fb2 x:0 0 0 2024-02-16T22:22:05.761 INFO:tasks.workunit.client.0.smithi114.stdout:2/537: readlink dd/l1a 0 2024-02-16T22:22:05.767 INFO:tasks.workunit.client.0.smithi114.stdout:7/631: getdents d3/d22/d73/da7 0 2024-02-16T22:22:05.771 INFO:tasks.workunit.client.0.smithi114.stdout:9/594: creat d2/d1a/d23/fbf x:0 0 0 2024-02-16T22:22:05.772 INFO:tasks.workunit.client.0.smithi114.stdout:9/595: dread - d2/dd/f2f zero size 2024-02-16T22:22:05.774 INFO:tasks.workunit.client.0.smithi114.stdout:6/591: dwrite d6/d83/f8f [4194304,4194304] 0 2024-02-16T22:22:05.775 INFO:tasks.workunit.client.0.smithi114.stdout:6/592: stat d6/d2c/d75/cae 0 2024-02-16T22:22:05.781 INFO:tasks.workunit.client.0.smithi114.stdout:0/532: truncate d1/f74 1327177 0 2024-02-16T22:22:05.789 INFO:tasks.workunit.client.0.smithi114.stdout:6/593: unlink d6/d2c/d66/ca9 0 2024-02-16T22:22:05.791 INFO:tasks.workunit.client.0.smithi114.stdout:7/632: mknod d3/d22/db7/cdc 0 2024-02-16T22:22:05.793 INFO:tasks.workunit.client.0.smithi114.stdout:7/633: write d3/d14/f18 [2473348,20559] 0 2024-02-16T22:22:05.795 INFO:tasks.workunit.client.0.smithi114.stdout:0/533: mkdir d1/d1a/d47/d62/d7f/d9d 0 2024-02-16T22:22:05.797 INFO:tasks.workunit.client.0.smithi114.stdout:5/543: getdents d0/d28/d98/d25/d3e 0 2024-02-16T22:22:05.799 INFO:tasks.workunit.client.0.smithi114.stdout:6/594: write d6/d2c/f6d [1670334,72254] 0 2024-02-16T22:22:05.800 INFO:tasks.workunit.client.0.smithi114.stdout:5/544: fsync d0/d28/d98/d25/d2e/d3b/f77 0 2024-02-16T22:22:05.801 INFO:tasks.workunit.client.0.smithi114.stdout:2/538: creat dd/d3b/da7/fb3 x:0 0 0 2024-02-16T22:22:05.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:05 smithi123 ceph-mon[37161]: pgmap v30: 65 pgs: 65 active+clean; 2.4 GiB data, 6.8 GiB used, 530 GiB / 536 GiB avail; 81 MiB/s rd, 193 MiB/s wr, 408 op/s 2024-02-16T22:22:05.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:05 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:22:05.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:05 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:22:05.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:05 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:22:05.803 INFO:tasks.workunit.client.0.smithi114.stdout:9/596: dwrite d2/d33/d3f/f7e [0,4194304] 0 2024-02-16T22:22:05.804 INFO:tasks.workunit.client.0.smithi114.stdout:2/539: write dd/d3b/da7/d4c/fb2 [350793,112228] 0 2024-02-16T22:22:05.804 INFO:tasks.workunit.client.1.smithi123.stdout:9/843: creat d4/d1e/d3a/dac/f113 x:0 0 0 2024-02-16T22:22:05.804 INFO:tasks.workunit.client.0.smithi114.stdout:9/597: fdatasync d2/f44 0 2024-02-16T22:22:05.804 INFO:tasks.workunit.client.1.smithi123.stdout:7/721: truncate d3/d6e/d1d/db2/f7e 1323863 0 2024-02-16T22:22:05.807 INFO:tasks.workunit.client.1.smithi123.stdout:2/701: truncate d4/d1a/d1d/d13/d25/d98/f67 3562582 0 2024-02-16T22:22:05.811 INFO:tasks.workunit.client.1.smithi123.stdout:0/646: write d1/d4/d17/d54/fd9 [695045,62780] 0 2024-02-16T22:22:05.811 INFO:tasks.workunit.client.1.smithi123.stdout:6/732: link d0/d6/d1a/lac d0/d6/d91/dda/l106 0 2024-02-16T22:22:05.813 INFO:tasks.workunit.client.0.smithi114.stdout:5/545: symlink d0/df/d6d/laf 0 2024-02-16T22:22:05.814 INFO:tasks.workunit.client.0.smithi114.stdout:5/546: chown d0/d28/d98/d25/d2e 914387 1 2024-02-16T22:22:05.815 INFO:tasks.workunit.client.1.smithi123.stdout:7/722: dwrite d3/d6e/d1d/db2/f99 [0,4194304] 0 2024-02-16T22:22:05.815 INFO:tasks.workunit.client.1.smithi123.stdout:4/789: dread d5/f40 [0,4194304] 0 2024-02-16T22:22:05.817 INFO:tasks.workunit.client.0.smithi114.stdout:7/634: link d3/d22/d73/f7c d3/d46/da2/fdd 0 2024-02-16T22:22:05.817 INFO:tasks.workunit.client.1.smithi123.stdout:4/790: write d5/d1c/d23/d74/d92/f86 [9095123,475] 0 2024-02-16T22:22:05.818 INFO:tasks.workunit.client.0.smithi114.stdout:7/635: dread - d3/d22/d61/f9d zero size 2024-02-16T22:22:05.818 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:05 smithi114 ceph-mon[29235]: pgmap v30: 65 pgs: 65 active+clean; 2.4 GiB data, 6.8 GiB used, 530 GiB / 536 GiB avail; 81 MiB/s rd, 193 MiB/s wr, 408 op/s 2024-02-16T22:22:05.818 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:05 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:22:05.819 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:05 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:22:05.819 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:05 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:22:05.819 INFO:tasks.workunit.client.1.smithi123.stdout:4/791: chown d5/d1c/d23/d56/d6e/d9f/de5 5196329 1 2024-02-16T22:22:05.820 INFO:tasks.workunit.client.1.smithi123.stdout:1/718: readlink d3/d6/db/d55/la9 0 2024-02-16T22:22:05.822 INFO:tasks.workunit.client.0.smithi114.stdout:2/540: creat dd/d3b/da7/d4c/fb4 x:0 0 0 2024-02-16T22:22:05.824 INFO:tasks.workunit.client.1.smithi123.stdout:9/844: chown d4/d1e/d3a/f102 0 1 2024-02-16T22:22:05.825 INFO:tasks.workunit.client.1.smithi123.stdout:9/845: chown d4/d1e/d3a/dac/f113 21 1 2024-02-16T22:22:05.825 INFO:tasks.workunit.client.1.smithi123.stdout:9/846: write d4/d1e/fe7 [293473,126895] 0 2024-02-16T22:22:05.829 INFO:tasks.workunit.client.0.smithi114.stdout:0/534: write d1/d16/d18/f22 [173389,32951] 0 2024-02-16T22:22:05.833 INFO:tasks.workunit.client.1.smithi123.stdout:5/717: chown d6/dd/df/d90/ff2 126043046 1 2024-02-16T22:22:05.834 INFO:tasks.workunit.client.1.smithi123.stdout:0/647: symlink d1/d8/da7/ldf 0 2024-02-16T22:22:05.835 INFO:tasks.workunit.client.1.smithi123.stdout:2/702: dread d4/d1a/d1d/d59/dd7/ddc/fa8 [0,4194304] 0 2024-02-16T22:22:05.836 INFO:tasks.workunit.client.1.smithi123.stdout:6/733: mknod d0/d9/dc/d39/d8d/c107 0 2024-02-16T22:22:05.838 INFO:tasks.workunit.client.1.smithi123.stdout:3/755: link d0/l16 d0/d20/d29/d9b/dbd/df4/d5d/d60/dd6/lff 0 2024-02-16T22:22:05.840 INFO:tasks.workunit.client.0.smithi114.stdout:7/636: mknod d3/d87/cde 0 2024-02-16T22:22:05.841 INFO:tasks.workunit.client.0.smithi114.stdout:6/595: dwrite d6/f1e [0,4194304] 0 2024-02-16T22:22:05.843 INFO:tasks.workunit.client.0.smithi114.stdout:2/541: mknod dd/d3b/da7/d7f/cb5 0 2024-02-16T22:22:05.846 INFO:tasks.workunit.client.1.smithi123.stdout:8/781: creat d0/ffc x:0 0 0 2024-02-16T22:22:05.846 INFO:tasks.workunit.client.0.smithi114.stdout:9/598: write d2/d5c/d5f/d91/f97 [3541482,129691] 0 2024-02-16T22:22:05.849 INFO:tasks.workunit.client.1.smithi123.stdout:9/847: creat d4/d36/d8e/d84/d93/f114 x:0 0 0 2024-02-16T22:22:05.850 INFO:tasks.workunit.client.0.smithi114.stdout:5/547: mkdir d0/d28/d98/d25/d89/d99/db0 0 2024-02-16T22:22:05.853 INFO:tasks.workunit.client.1.smithi123.stdout:5/718: creat d6/dc1/de6/ffd x:0 0 0 2024-02-16T22:22:05.854 INFO:tasks.workunit.client.1.smithi123.stdout:5/719: dread - d6/dd/df/d9f/fd7 zero size 2024-02-16T22:22:05.854 INFO:tasks.workunit.client.0.smithi114.stdout:7/637: mknod d3/d22/d73/da7/cdf 0 2024-02-16T22:22:05.855 INFO:tasks.workunit.client.0.smithi114.stdout:7/638: readlink d3/d22/d73/da7/l2f 0 2024-02-16T22:22:05.856 INFO:tasks.workunit.client.0.smithi114.stdout:2/542: creat dd/df/d1c/d21/d61/d63/d83/fb6 x:0 0 0 2024-02-16T22:22:05.856 INFO:tasks.workunit.client.1.smithi123.stdout:2/703: mknod d4/d1a/d1d/d13/d15/ce1 0 2024-02-16T22:22:05.858 INFO:tasks.workunit.client.0.smithi114.stdout:9/599: rmdir d2/d1a/d23/d35/d47/d63/d98 39 2024-02-16T22:22:05.860 INFO:tasks.workunit.client.0.smithi114.stdout:5/548: mkdir d0/df/d53/db1 0 2024-02-16T22:22:05.860 INFO:tasks.workunit.client.1.smithi123.stdout:4/792: symlink d5/d1c/lf1 0 2024-02-16T22:22:05.862 INFO:tasks.workunit.client.1.smithi123.stdout:1/719: mknod d3/d6/db/d10/d4f/d9f/db4/ddf/df0/cf4 0 2024-02-16T22:22:05.863 INFO:tasks.workunit.client.0.smithi114.stdout:7/639: creat d3/d22/d73/da7/d6f/d89/da6/fe0 x:0 0 0 2024-02-16T22:22:05.866 INFO:tasks.workunit.client.0.smithi114.stdout:0/535: link d1/d16/c50 d1/d16/d1f/d28/d51/d61/d6d/d79/c9e 0 2024-02-16T22:22:05.870 INFO:tasks.workunit.client.0.smithi114.stdout:9/600: creat d2/d1a/d23/d32/d46/fc0 x:0 0 0 2024-02-16T22:22:05.872 INFO:tasks.workunit.client.0.smithi114.stdout:7/640: dwrite d3/d22/d73/da7/d4e/fbf [0,4194304] 0 2024-02-16T22:22:05.873 INFO:tasks.workunit.client.0.smithi114.stdout:5/549: creat d0/d28/d98/d1e/d63/d9e/fb2 x:0 0 0 2024-02-16T22:22:05.874 INFO:tasks.workunit.client.1.smithi123.stdout:0/648: fdatasync d1/d8/d13/f14 0 2024-02-16T22:22:05.875 INFO:tasks.workunit.client.1.smithi123.stdout:8/782: dread d0/d29/f64 [0,4194304] 0 2024-02-16T22:22:05.877 INFO:tasks.workunit.client.1.smithi123.stdout:1/720: dread d3/d6/d18/f7e [0,4194304] 0 2024-02-16T22:22:05.877 INFO:tasks.workunit.client.1.smithi123.stdout:5/720: rename d6/dd/df/d1f/l2a to d6/dd/df/d58/d3f/d82/lfe 0 2024-02-16T22:22:05.899 INFO:tasks.workunit.client.1.smithi123.stdout:7/723: sync 2024-02-16T22:22:06.022 INFO:tasks.workunit.client.1.smithi123.stdout:7/724: sync 2024-02-16T22:22:06.081 INFO:tasks.workunit.client.0.smithi114.stdout:1/582: sync 2024-02-16T22:22:06.081 INFO:tasks.workunit.client.0.smithi114.stdout:3/579: sync 2024-02-16T22:22:06.081 INFO:tasks.workunit.client.0.smithi114.stdout:4/691: sync 2024-02-16T22:22:06.097 INFO:tasks.workunit.client.0.smithi114.stdout:4/692: dread dd/d1b/d24/d98/fc5 [0,4194304] 0 2024-02-16T22:22:06.098 INFO:tasks.workunit.client.0.smithi114.stdout:4/693: readlink dd/l28 0 2024-02-16T22:22:06.109 INFO:tasks.workunit.client.0.smithi114.stdout:0/536: symlink d1/d16/d1f/d28/d51/d61/d7e/d90/d95/l9f 0 2024-02-16T22:22:06.111 INFO:tasks.workunit.client.1.smithi123.stdout:3/756: symlink d0/d20/d29/d9b/dbd/df4/d5d/d60/dc6/df0/l100 0 2024-02-16T22:22:06.112 INFO:tasks.workunit.client.0.smithi114.stdout:7/641: mknod d3/d22/d73/da7/d6f/d89/da6/ce1 0 2024-02-16T22:22:06.113 INFO:tasks.workunit.client.0.smithi114.stdout:6/596: write d6/d2c/d38/d54/f3d [10991,75665] 0 2024-02-16T22:22:06.114 INFO:tasks.workunit.client.0.smithi114.stdout:7/642: chown d3/d22/d73/da7/l40 0 1 2024-02-16T22:22:06.115 INFO:tasks.workunit.client.0.smithi114.stdout:5/550: unlink d0/d28/f90 0 2024-02-16T22:22:06.122 INFO:tasks.workunit.client.1.smithi123.stdout:9/848: dwrite d4/d1e/d3a/fc3 [0,4194304] 0 2024-02-16T22:22:06.122 INFO:tasks.workunit.client.0.smithi114.stdout:6/597: dwrite d6/dd/d25/d34/fa4 [0,4194304] 0 2024-02-16T22:22:06.124 INFO:tasks.workunit.client.0.smithi114.stdout:3/580: read d4/d43/f56 [2844662,128518] 0 2024-02-16T22:22:06.131 INFO:tasks.workunit.client.0.smithi114.stdout:3/581: dread d4/dd/d33/d8f/fa4 [0,4194304] 0 2024-02-16T22:22:06.133 INFO:tasks.workunit.client.0.smithi114.stdout:4/694: symlink dd/d1b/d37/d38/lf9 0 2024-02-16T22:22:06.133 INFO:tasks.workunit.client.0.smithi114.stdout:3/582: write d4/d24/d2d/d45/d68/f86 [4694129,128243] 0 2024-02-16T22:22:06.134 INFO:tasks.workunit.client.0.smithi114.stdout:2/543: write dd/df/d1c/d21/d87/f5f [1311416,109489] 0 2024-02-16T22:22:06.138 INFO:tasks.workunit.client.0.smithi114.stdout:2/544: dread - dd/d3b/da7/fb3 zero size 2024-02-16T22:22:06.142 INFO:tasks.workunit.client.0.smithi114.stdout:5/551: dread d0/d10/f74 [0,4194304] 0 2024-02-16T22:22:06.142 INFO:tasks.workunit.client.1.smithi123.stdout:5/721: symlink d6/dd/d78/da5/lff 0 2024-02-16T22:22:06.144 INFO:tasks.workunit.client.0.smithi114.stdout:0/537: dwrite d1/d1a/f26 [0,4194304] 0 2024-02-16T22:22:06.147 INFO:tasks.workunit.client.0.smithi114.stdout:0/538: write d1/d16/f73 [736353,41100] 0 2024-02-16T22:22:06.148 INFO:tasks.workunit.client.0.smithi114.stdout:7/643: mknod d3/d87/ce2 0 2024-02-16T22:22:06.150 INFO:tasks.workunit.client.0.smithi114.stdout:0/539: truncate d1/d16/f82 388232 0 2024-02-16T22:22:06.156 INFO:tasks.workunit.client.1.smithi123.stdout:2/704: creat d4/dda/fe2 x:0 0 0 2024-02-16T22:22:06.157 INFO:tasks.workunit.client.1.smithi123.stdout:2/705: chown d4/d1a/d1d/d13/d15 1596805 1 2024-02-16T22:22:06.161 INFO:tasks.workunit.client.1.smithi123.stdout:9/849: dread d4/d36/d57/db2/f28 [0,4194304] 0 2024-02-16T22:22:06.163 INFO:tasks.workunit.client.0.smithi114.stdout:1/583: mknod d4/d9/d46/d67/d92/dca/dce/dd3/cd9 0 2024-02-16T22:22:06.164 INFO:tasks.workunit.client.0.smithi114.stdout:1/584: write d4/d9/d3d/f41 [4884242,101482] 0 2024-02-16T22:22:06.164 INFO:tasks.workunit.client.1.smithi123.stdout:3/757: symlink d0/d20/d29/d9b/dbd/df4/d5d/d60/dc6/df0/l101 0 2024-02-16T22:22:06.167 INFO:tasks.workunit.client.0.smithi114.stdout:0/540: dread d1/d1a/d3e/f4a [4194304,4194304] 0 2024-02-16T22:22:06.172 INFO:tasks.workunit.client.0.smithi114.stdout:6/598: dwrite d6/d2c/d75/f82 [0,4194304] 0 2024-02-16T22:22:06.174 INFO:tasks.workunit.client.1.smithi123.stdout:4/793: mknod d5/d1c/d23/d56/d6e/cf2 0 2024-02-16T22:22:06.181 INFO:tasks.workunit.client.0.smithi114.stdout:4/695: mkdir dd/d59/dfa 0 2024-02-16T22:22:06.183 INFO:tasks.workunit.client.0.smithi114.stdout:4/696: read dd/d1b/d37/d38/d87/dad/fbf [553623,67236] 0 2024-02-16T22:22:06.187 INFO:tasks.workunit.client.0.smithi114.stdout:2/545: creat dd/d4e/fb7 x:0 0 0 2024-02-16T22:22:06.190 INFO:tasks.workunit.client.1.smithi123.stdout:1/721: mkdir d3/d6/d49/df5 0 2024-02-16T22:22:06.192 INFO:tasks.workunit.client.0.smithi114.stdout:5/552: read d0/d28/d98/f49 [1622026,4513] 0 2024-02-16T22:22:06.195 INFO:tasks.workunit.client.1.smithi123.stdout:7/725: symlink d3/d6e/d1d/db2/d35/dbe/lea 0 2024-02-16T22:22:06.197 INFO:tasks.workunit.client.0.smithi114.stdout:7/644: mknod d3/d22/d73/da7/d6f/d9c/ce3 0 2024-02-16T22:22:06.202 INFO:tasks.workunit.client.1.smithi123.stdout:9/850: creat d4/d1e/d3a/dac/f115 x:0 0 0 2024-02-16T22:22:06.203 INFO:tasks.workunit.client.1.smithi123.stdout:3/758: truncate d0/da0/fba 739599 0 2024-02-16T22:22:06.206 INFO:tasks.workunit.client.1.smithi123.stdout:4/794: rename d5/d1c/d23/d74/d92/d9b/fe3 to d5/d1c/d23/d47/ff3 0 2024-02-16T22:22:06.206 INFO:tasks.workunit.client.0.smithi114.stdout:4/697: dread dd/d1b/d37/d38/faf [0,4194304] 0 2024-02-16T22:22:06.207 INFO:tasks.workunit.client.1.smithi123.stdout:7/726: dread d3/d6e/d1d/db2/d35/f9f [0,4194304] 0 2024-02-16T22:22:06.208 INFO:tasks.workunit.client.1.smithi123.stdout:2/706: dread d4/d1a/d1d/d13/d25/d98/f96 [0,4194304] 0 2024-02-16T22:22:06.209 INFO:tasks.workunit.client.0.smithi114.stdout:6/599: mknod d6/d83/cb3 0 2024-02-16T22:22:06.213 INFO:tasks.workunit.client.1.smithi123.stdout:8/783: creat d0/d29/da7/d26/d2a/d77/df8/d72/ffd x:0 0 0 2024-02-16T22:22:06.223 INFO:tasks.workunit.client.1.smithi123.stdout:1/722: rename d3/d6/db/d10/d4f/fb6 to d3/d6/d49/d48/d84/d92/dec/ff6 0 2024-02-16T22:22:06.223 INFO:tasks.workunit.client.0.smithi114.stdout:5/553: mkdir d0/d2d/d30/db3 0 2024-02-16T22:22:06.227 INFO:tasks.workunit.client.0.smithi114.stdout:7/645: unlink d3/d87/f8e 0 2024-02-16T22:22:06.232 INFO:tasks.workunit.client.1.smithi123.stdout:0/649: getdents d1/d8/d13/d1d/d64/dbc 0 2024-02-16T22:22:06.233 INFO:tasks.workunit.client.0.smithi114.stdout:0/541: mknod d1/d16/d1f/d28/d51/d60/d52/ca0 0 2024-02-16T22:22:06.237 INFO:tasks.workunit.client.0.smithi114.stdout:4/698: creat dd/d1b/d37/d38/d87/dad/ffb x:0 0 0 2024-02-16T22:22:06.247 INFO:tasks.workunit.client.1.smithi123.stdout:4/795: unlink d5/d1c/d64/d7b/fb1 0 2024-02-16T22:22:06.249 INFO:tasks.workunit.client.0.smithi114.stdout:2/546: mkdir dd/d6b/db8 0 2024-02-16T22:22:06.250 INFO:tasks.workunit.client.1.smithi123.stdout:7/727: truncate d3/d5/ff 1126648 0 2024-02-16T22:22:06.258 INFO:tasks.workunit.client.0.smithi114.stdout:5/554: dread d0/d28/d98/d25/d2e/f56 [0,4194304] 0 2024-02-16T22:22:06.259 INFO:tasks.workunit.client.0.smithi114.stdout:7/646: creat d3/d87/fe4 x:0 0 0 2024-02-16T22:22:06.265 INFO:tasks.workunit.client.1.smithi123.stdout:4/796: rename d5/d1c/d23/d74/d92/d9b/dbd to d5/df4 0 2024-02-16T22:22:06.267 INFO:tasks.workunit.client.0.smithi114.stdout:4/699: read dd/d1b/d37/d38/d87/dad/fbf [1317654,30367] 0 2024-02-16T22:22:06.275 INFO:tasks.workunit.client.1.smithi123.stdout:4/797: fsync d5/d1c/fac 0 2024-02-16T22:22:06.278 INFO:tasks.workunit.client.0.smithi114.stdout:5/555: read d0/d10/f13 [122732,61911] 0 2024-02-16T22:22:06.279 INFO:tasks.workunit.client.1.smithi123.stdout:4/798: dread d5/d1c/d23/d6d/f78 [0,4194304] 0 2024-02-16T22:22:06.279 INFO:tasks.workunit.client.1.smithi123.stdout:7/728: truncate d3/d5/f28 156043 0 2024-02-16T22:22:06.280 INFO:tasks.workunit.client.1.smithi123.stdout:7/729: write d3/f4e [876616,116366] 0 2024-02-16T22:22:06.280 INFO:tasks.workunit.client.1.smithi123.stdout:4/799: dread - d5/d1c/d23/d69/fe1 zero size 2024-02-16T22:22:06.280 INFO:tasks.workunit.client.0.smithi114.stdout:2/547: rmdir dd/d3b/da7/d4c 39 2024-02-16T22:22:06.281 INFO:tasks.workunit.client.1.smithi123.stdout:7/730: write d3/d6e/d1d/f22 [2814218,126977] 0 2024-02-16T22:22:06.287 INFO:tasks.workunit.client.0.smithi114.stdout:4/700: creat dd/d1b/d24/d98/ffc x:0 0 0 2024-02-16T22:22:06.294 INFO:tasks.workunit.client.0.smithi114.stdout:4/701: dwrite dd/d1b/d41/d74/d9b/ff8 [0,4194304] 0 2024-02-16T22:22:06.296 INFO:tasks.workunit.client.1.smithi123.stdout:4/800: unlink d5/d1c/d23/d74/d92/d9b/cd6 0 2024-02-16T22:22:06.297 INFO:tasks.workunit.client.0.smithi114.stdout:4/702: truncate dd/d1b/d41/dc3/ff3 4454116 0 2024-02-16T22:22:06.300 INFO:tasks.workunit.client.0.smithi114.stdout:4/703: dread dd/d1b/f55 [0,4194304] 0 2024-02-16T22:22:06.315 INFO:tasks.workunit.client.1.smithi123.stdout:9/851: sync 2024-02-16T22:22:06.319 INFO:tasks.workunit.client.1.smithi123.stdout:4/801: dread d5/d1c/d23/d31/f2d [0,4194304] 0 2024-02-16T22:22:06.319 INFO:tasks.workunit.client.1.smithi123.stdout:4/802: readlink d5/d1c/lf1 0 2024-02-16T22:22:06.320 INFO:tasks.workunit.client.1.smithi123.stdout:4/803: chown d5/d1c 21 1 2024-02-16T22:22:06.320 INFO:tasks.workunit.client.1.smithi123.stdout:4/804: readlink d5/l68 0 2024-02-16T22:22:06.324 INFO:tasks.workunit.client.0.smithi114.stdout:8/553: sync 2024-02-16T22:22:06.326 INFO:tasks.workunit.client.1.smithi123.stdout:7/731: mknod d3/d6e/d34/ceb 0 2024-02-16T22:22:06.326 INFO:tasks.workunit.client.1.smithi123.stdout:7/732: chown d3/d6e/d1d/db2/d35/db7/fbb 10157 1 2024-02-16T22:22:06.327 INFO:tasks.workunit.client.0.smithi114.stdout:5/556: mknod d0/d28/d98/d25/d2e/d3b/d8d/cb4 0 2024-02-16T22:22:06.328 INFO:tasks.workunit.client.1.smithi123.stdout:7/733: read d3/d6e/d34/f9e [3014687,10036] 0 2024-02-16T22:22:06.328 INFO:tasks.workunit.client.0.smithi114.stdout:5/557: write d0/d28/d98/d1e/f29 [7386512,29563] 0 2024-02-16T22:22:06.331 INFO:tasks.workunit.client.1.smithi123.stdout:4/805: mknod d5/d1c/d23/d6d/cf5 0 2024-02-16T22:22:06.335 INFO:tasks.workunit.client.0.smithi114.stdout:4/704: mkdir dd/d1b/d41/dc3/dfd 0 2024-02-16T22:22:06.337 INFO:tasks.workunit.client.0.smithi114.stdout:8/554: mkdir d4/d24/d6d/dc7 0 2024-02-16T22:22:06.340 INFO:tasks.workunit.client.1.smithi123.stdout:6/734: write d0/d6/f15 [6018674,117190] 0 2024-02-16T22:22:06.340 INFO:tasks.workunit.client.1.smithi123.stdout:4/806: chown d5/d1c/d9a/cbe 27798251 1 2024-02-16T22:22:06.340 INFO:tasks.workunit.client.1.smithi123.stdout:6/735: stat d0/d6/d4d/db0/d104 0 2024-02-16T22:22:06.342 INFO:tasks.workunit.client.0.smithi114.stdout:0/542: mknod d1/d16/d1f/d28/d51/d61/d6d/ca1 0 2024-02-16T22:22:06.345 INFO:tasks.workunit.client.0.smithi114.stdout:4/705: rename dd/d1b/d24/l5a to dd/d59/lfe 0 2024-02-16T22:22:06.346 INFO:tasks.workunit.client.0.smithi114.stdout:4/706: dread - dd/d1b/d37/d38/d87/dad/ffb zero size 2024-02-16T22:22:06.347 INFO:tasks.workunit.client.0.smithi114.stdout:5/558: dwrite d0/d28/d98/f5b [0,4194304] 0 2024-02-16T22:22:06.348 INFO:tasks.workunit.client.0.smithi114.stdout:5/559: chown d0/d28/d98/d1e/d86/f8c 1 1 2024-02-16T22:22:06.350 INFO:tasks.workunit.client.0.smithi114.stdout:5/560: fsync d0/d28/d98/d25/d2e/f7d 0 2024-02-16T22:22:06.350 INFO:tasks.workunit.client.0.smithi114.stdout:8/555: fdatasync d4/d11/d78/d7a/d31/f4d 0 2024-02-16T22:22:06.351 INFO:tasks.workunit.client.0.smithi114.stdout:8/556: chown d4/d24/l62 363 1 2024-02-16T22:22:06.352 INFO:tasks.workunit.client.1.smithi123.stdout:7/734: dread d3/f86 [0,4194304] 0 2024-02-16T22:22:06.353 INFO:tasks.workunit.client.1.smithi123.stdout:4/807: creat d5/d1c/d23/d47/da2/daa/ff6 x:0 0 0 2024-02-16T22:22:06.354 INFO:tasks.workunit.client.1.smithi123.stdout:7/735: write d3/d6e/d1d/db2/d3e/fbf [2453156,50123] 0 2024-02-16T22:22:06.358 INFO:tasks.workunit.client.0.smithi114.stdout:0/543: mknod d1/d16/d55/d92/ca2 0 2024-02-16T22:22:06.361 INFO:tasks.workunit.client.1.smithi123.stdout:9/852: getdents d4/d36/d8e/d58/d59/d60/deb 0 2024-02-16T22:22:06.361 INFO:tasks.workunit.client.1.smithi123.stdout:9/853: stat d4/d36/d8e/d58/f5a 0 2024-02-16T22:22:06.362 INFO:tasks.workunit.client.0.smithi114.stdout:4/707: readlink dd/lf0 0 2024-02-16T22:22:06.365 INFO:tasks.workunit.client.1.smithi123.stdout:0/650: truncate d1/d8/d13/f14 1260753 0 2024-02-16T22:22:06.370 INFO:tasks.workunit.client.1.smithi123.stdout:7/736: mknod d3/d6e/d1d/db2/d3d/cec 0 2024-02-16T22:22:06.374 INFO:tasks.workunit.client.1.smithi123.stdout:9/854: rename d4/c1c to d4/d36/d8e/d58/d59/d60/deb/da7/dde/dfd/d10d/c116 0 2024-02-16T22:22:06.378 INFO:tasks.workunit.client.0.smithi114.stdout:4/708: truncate dd/d1b/f55 529822 0 2024-02-16T22:22:06.379 INFO:tasks.workunit.client.1.smithi123.stdout:7/737: creat d3/d6e/d1d/db2/d3d/fed x:0 0 0 2024-02-16T22:22:06.383 INFO:tasks.workunit.client.0.smithi114.stdout:4/709: rmdir dd/d1b/d41/d74 39 2024-02-16T22:22:06.388 INFO:tasks.workunit.client.1.smithi123.stdout:2/707: rmdir d4/dda 39 2024-02-16T22:22:06.391 INFO:tasks.workunit.client.1.smithi123.stdout:7/738: getdents d3 0 2024-02-16T22:22:06.392 INFO:tasks.workunit.client.1.smithi123.stdout:6/736: creat d0/d9/dc/d3a/f108 x:0 0 0 2024-02-16T22:22:06.392 INFO:tasks.workunit.client.1.smithi123.stdout:6/737: readlink d0/d9/l87 0 2024-02-16T22:22:06.393 INFO:tasks.workunit.client.1.smithi123.stdout:2/708: mkdir d4/d1a/d1d/d13/d15/d6d/db1/de3 0 2024-02-16T22:22:06.394 INFO:tasks.workunit.client.1.smithi123.stdout:6/738: mknod d0/d9/dc/d3a/d70/dd5/c109 0 2024-02-16T22:22:06.395 INFO:tasks.workunit.client.1.smithi123.stdout:6/739: readlink d0/d9/dc/d3a/d52/lf9 0 2024-02-16T22:22:06.397 INFO:tasks.workunit.client.1.smithi123.stdout:7/739: truncate d3/d6e/d24/fd8 4556894 0 2024-02-16T22:22:06.397 INFO:tasks.workunit.client.1.smithi123.stdout:2/709: dread d4/d1a/d1d/d13/d15/d6d/d4a/faf [0,4194304] 0 2024-02-16T22:22:06.398 INFO:tasks.workunit.client.1.smithi123.stdout:6/740: truncate d0/d6/d8e/fc0 2446512 0 2024-02-16T22:22:06.399 INFO:tasks.workunit.client.1.smithi123.stdout:2/710: write d4/d1a/d1d/d38/d7b/fc4 [1010500,64838] 0 2024-02-16T22:22:06.404 INFO:tasks.workunit.client.1.smithi123.stdout:6/741: rename d0/d9/dc/d3a/c48 to d0/d9/dc/d3a/d70/de5/c10a 0 2024-02-16T22:22:06.407 INFO:tasks.workunit.client.1.smithi123.stdout:2/711: link d4/d1a/d1d/d13/d25/f43 d4/d1a/d1d/d59/dd7/ddc/fe4 0 2024-02-16T22:22:06.409 INFO:tasks.workunit.client.0.smithi114.stdout:3/583: write d4/dd/f19 [2744905,77930] 0 2024-02-16T22:22:06.410 INFO:tasks.workunit.client.1.smithi123.stdout:7/740: dread d3/d6e/f7c [0,4194304] 0 2024-02-16T22:22:06.411 INFO:tasks.workunit.client.1.smithi123.stdout:2/712: creat d4/d1a/fe5 x:0 0 0 2024-02-16T22:22:06.413 INFO:tasks.workunit.client.1.smithi123.stdout:7/741: rmdir d3/d87 39 2024-02-16T22:22:06.416 INFO:tasks.workunit.client.0.smithi114.stdout:3/584: dwrite d4/dd/f64 [0,4194304] 0 2024-02-16T22:22:06.418 INFO:tasks.workunit.client.1.smithi123.stdout:6/742: dread d0/d8a/fcd [0,4194304] 0 2024-02-16T22:22:06.425 INFO:tasks.workunit.client.1.smithi123.stdout:2/713: link d4/d1a/d1d/d13/d25/f66 d4/d1a/d1d/d38/fe6 0 2024-02-16T22:22:06.428 INFO:tasks.workunit.client.1.smithi123.stdout:6/743: rename d0/d9/dc/d39/c49 to d0/d96/de3/dcc/c10b 0 2024-02-16T22:22:06.429 INFO:tasks.workunit.client.1.smithi123.stdout:2/714: dread d4/d1a/d1d/d59/dd7/ddc/fa8 [0,4194304] 0 2024-02-16T22:22:06.432 INFO:tasks.workunit.client.1.smithi123.stdout:0/651: sync 2024-02-16T22:22:06.434 INFO:tasks.workunit.client.0.smithi114.stdout:3/585: getdents d4/d24 0 2024-02-16T22:22:06.435 INFO:tasks.workunit.client.0.smithi114.stdout:3/586: write d4/d46/d48/f66 [3022358,99935] 0 2024-02-16T22:22:06.436 INFO:tasks.workunit.client.1.smithi123.stdout:2/715: creat d4/d1a/d1d/d13/d15/d6d/d4a/da2/fe7 x:0 0 0 2024-02-16T22:22:06.441 INFO:tasks.workunit.client.1.smithi123.stdout:2/716: dwrite d4/d1a/d1d/d13/d25/d98/f76 [0,4194304] 0 2024-02-16T22:22:06.441 INFO:tasks.workunit.client.0.smithi114.stdout:3/587: creat d4/dd/fcb x:0 0 0 2024-02-16T22:22:06.442 INFO:tasks.workunit.client.0.smithi114.stdout:3/588: dread - d4/dd/d33/d72/f9b zero size 2024-02-16T22:22:06.443 INFO:tasks.workunit.client.0.smithi114.stdout:3/589: dread - d4/d43/f6e zero size 2024-02-16T22:22:06.445 INFO:tasks.workunit.client.1.smithi123.stdout:2/717: rmdir d4/d1a/d1d/d59/dd7/ddc 39 2024-02-16T22:22:06.447 INFO:tasks.workunit.client.0.smithi114.stdout:3/590: rename d4/d26/d83 to d4/d46/d48/dcc 0 2024-02-16T22:22:06.449 INFO:tasks.workunit.client.1.smithi123.stdout:2/718: mkdir d4/d1a/d1d/d38/d5e/de8 0 2024-02-16T22:22:06.454 INFO:tasks.workunit.client.1.smithi123.stdout:2/719: creat d4/d1a/d1d/fe9 x:0 0 0 2024-02-16T22:22:06.454 INFO:tasks.workunit.client.0.smithi114.stdout:3/591: dwrite d4/f18 [8388608,4194304] 0 2024-02-16T22:22:06.457 INFO:tasks.workunit.client.1.smithi123.stdout:2/720: truncate d4/d1a/d1d/d13/d25/f3b 584520 0 2024-02-16T22:22:06.458 INFO:tasks.workunit.client.0.smithi114.stdout:7/647: symlink d3/le5 0 2024-02-16T22:22:06.458 INFO:tasks.workunit.client.0.smithi114.stdout:7/648: readlink d3/d14/l6d 0 2024-02-16T22:22:06.459 INFO:tasks.workunit.client.0.smithi114.stdout:7/649: truncate d3/d14/fa4 970160 0 2024-02-16T22:22:06.461 INFO:tasks.workunit.client.0.smithi114.stdout:3/592: rename d4/f18 to d4/d24/d2d/d45/d74/dc7/fcd 0 2024-02-16T22:22:06.463 INFO:tasks.workunit.client.0.smithi114.stdout:3/593: chown d4/d24/d2d/d45/d68 14610300 1 2024-02-16T22:22:06.463 INFO:tasks.workunit.client.1.smithi123.stdout:2/721: unlink d4/d1a/d1d/f75 0 2024-02-16T22:22:06.467 INFO:tasks.workunit.client.0.smithi114.stdout:7/650: read d3/d3d/f4b [1579178,85242] 0 2024-02-16T22:22:06.468 INFO:tasks.workunit.client.1.smithi123.stdout:2/722: mknod d4/d1a/d1d/d13/d25/d72/cea 0 2024-02-16T22:22:06.468 INFO:tasks.workunit.client.0.smithi114.stdout:7/651: chown d3/d22/d61/f84 1391726 1 2024-02-16T22:22:06.470 INFO:tasks.workunit.client.0.smithi114.stdout:9/601: sync 2024-02-16T22:22:06.471 INFO:tasks.workunit.client.0.smithi114.stdout:9/602: chown d2/d1a/d23/d35/f86 418752 1 2024-02-16T22:22:06.472 INFO:tasks.workunit.client.0.smithi114.stdout:7/652: stat d3/d22/d61/d6a/f94 0 2024-02-16T22:22:06.477 INFO:tasks.workunit.client.1.smithi123.stdout:2/723: creat d4/d1a/feb x:0 0 0 2024-02-16T22:22:06.477 INFO:tasks.workunit.client.0.smithi114.stdout:3/594: creat d4/d43/fce x:0 0 0 2024-02-16T22:22:06.478 INFO:tasks.workunit.client.0.smithi114.stdout:9/603: readlink d2/d1a/d23/d35/d1d/l37 0 2024-02-16T22:22:06.479 INFO:tasks.workunit.client.1.smithi123.stdout:1/723: dwrite d3/d6/d49/d48/d68/fab [0,4194304] 0 2024-02-16T22:22:06.480 INFO:tasks.workunit.client.0.smithi114.stdout:7/653: mkdir d3/d92/dbe/de6 0 2024-02-16T22:22:06.483 INFO:tasks.workunit.client.1.smithi123.stdout:2/724: mkdir d4/d1a/d1d/d38/d5e/dec 0 2024-02-16T22:22:06.484 INFO:tasks.workunit.client.1.smithi123.stdout:2/725: chown d4/d1a/d1d/l6e 26098 1 2024-02-16T22:22:06.485 INFO:tasks.workunit.client.0.smithi114.stdout:3/595: rename d4/d24/d2d/d45/dbb to d4/d46/dcf 0 2024-02-16T22:22:06.491 INFO:tasks.workunit.client.1.smithi123.stdout:8/784: write d0/d29/da7/d26/d2a/d4d/fc8 [750997,66303] 0 2024-02-16T22:22:06.492 INFO:tasks.workunit.client.1.smithi123.stdout:8/785: chown d0/d29/da7/d43/d34/dae/lbd 22842887 1 2024-02-16T22:22:06.493 INFO:tasks.workunit.client.1.smithi123.stdout:2/726: stat d4/d1a/d1d/d38/fe6 0 2024-02-16T22:22:06.499 INFO:tasks.workunit.client.1.smithi123.stdout:3/759: unlink d0/d20/d94/fbc 0 2024-02-16T22:22:06.499 INFO:tasks.workunit.client.1.smithi123.stdout:3/760: chown d0/d20/d29/c30 908812199 1 2024-02-16T22:22:06.501 INFO:tasks.workunit.client.1.smithi123.stdout:8/786: mknod d0/df/cfe 0 2024-02-16T22:22:06.511 INFO:tasks.workunit.client.1.smithi123.stdout:2/727: creat d4/d1a/d1d/d59/d45/fed x:0 0 0 2024-02-16T22:22:06.513 INFO:tasks.workunit.client.1.smithi123.stdout:5/722: symlink d6/dd/d78/l100 0 2024-02-16T22:22:06.514 INFO:tasks.workunit.client.1.smithi123.stdout:5/723: readlink d6/dd/d44/dba/lec 0 2024-02-16T22:22:06.514 INFO:tasks.workunit.client.1.smithi123.stdout:3/761: dread d0/f19 [0,4194304] 0 2024-02-16T22:22:06.514 INFO:tasks.workunit.client.1.smithi123.stdout:2/728: creat d4/d1a/d1d/d13/d15/d6d/d4a/da2/fee x:0 0 0 2024-02-16T22:22:06.514 INFO:tasks.workunit.client.1.smithi123.stdout:5/724: read - d6/dd/df/d58/d56/fda zero size 2024-02-16T22:22:06.515 INFO:tasks.workunit.client.1.smithi123.stdout:5/725: fdatasync d6/dd/df/d58/d89/f9c 0 2024-02-16T22:22:06.516 INFO:tasks.workunit.client.1.smithi123.stdout:2/729: chown d4/d1a/d1d/d13/d15/d6d/d4a/da2/fcb 4195830 1 2024-02-16T22:22:06.516 INFO:tasks.workunit.client.1.smithi123.stdout:8/787: creat d0/fff x:0 0 0 2024-02-16T22:22:06.518 INFO:tasks.workunit.client.1.smithi123.stdout:2/730: truncate d4/d1a/d1d/d13/d15/d6d/d4a/da2/fee 631445 0 2024-02-16T22:22:06.518 INFO:tasks.workunit.client.1.smithi123.stdout:2/731: chown d4/d1a/d1d/l10 12525 1 2024-02-16T22:22:06.528 INFO:tasks.workunit.client.1.smithi123.stdout:3/762: mkdir d0/d102 0 2024-02-16T22:22:06.534 INFO:tasks.workunit.client.1.smithi123.stdout:7/742: rename d3/d6e/d1d/db2/ca2 to d3/d87/cee 0 2024-02-16T22:22:06.536 INFO:tasks.workunit.client.0.smithi114.stdout:6/600: sync 2024-02-16T22:22:06.536 INFO:tasks.workunit.client.0.smithi114.stdout:2/548: sync 2024-02-16T22:22:06.536 INFO:tasks.workunit.client.0.smithi114.stdout:1/585: sync 2024-02-16T22:22:06.538 INFO:tasks.workunit.client.0.smithi114.stdout:6/601: write d6/dd/d25/d34/f6f [2643440,115269] 0 2024-02-16T22:22:06.540 INFO:tasks.workunit.client.0.smithi114.stdout:6/602: write d6/d2c/d38/f92 [2954708,73020] 0 2024-02-16T22:22:06.543 INFO:tasks.workunit.client.1.smithi123.stdout:4/808: write d5/d1c/d23/d47/f7e [1031136,60635] 0 2024-02-16T22:22:06.544 INFO:tasks.workunit.client.1.smithi123.stdout:4/809: write d5/d1c/d23/d47/da2/daa/ff6 [794291,125656] 0 2024-02-16T22:22:06.545 INFO:tasks.workunit.client.1.smithi123.stdout:1/724: rename d3/d6/db/d55/la9 to d3/d6/db/d10/d4f/d9f/db4/ddf/lf7 0 2024-02-16T22:22:06.548 INFO:tasks.workunit.client.0.smithi114.stdout:8/557: write d4/d16/d25/f6e [2515755,76513] 0 2024-02-16T22:22:06.550 INFO:tasks.workunit.client.0.smithi114.stdout:9/604: creat d2/dd/fc1 x:0 0 0 2024-02-16T22:22:06.551 INFO:tasks.workunit.client.1.smithi123.stdout:1/725: dread d3/d6/db/d10/f85 [0,4194304] 0 2024-02-16T22:22:06.551 INFO:tasks.workunit.client.1.smithi123.stdout:1/726: dread - d3/d6/d11/fe4 zero size 2024-02-16T22:22:06.552 INFO:tasks.workunit.client.0.smithi114.stdout:0/544: write d1/d16/d1f/d28/d51/d61/d7e/d90/f85 [1235735,47459] 0 2024-02-16T22:22:06.556 INFO:tasks.workunit.client.1.smithi123.stdout:7/743: creat d3/d59/fef x:0 0 0 2024-02-16T22:22:06.557 INFO:tasks.workunit.client.0.smithi114.stdout:4/710: creat dd/fff x:0 0 0 2024-02-16T22:22:06.558 INFO:tasks.workunit.client.1.smithi123.stdout:9/855: write d4/d33/f73 [1656876,38526] 0 2024-02-16T22:22:06.559 INFO:tasks.workunit.client.1.smithi123.stdout:4/810: symlink d5/d1c/d23/d69/lf7 0 2024-02-16T22:22:06.560 INFO:tasks.workunit.client.1.smithi123.stdout:4/811: stat d5/d1c/d23/d74/lce 0 2024-02-16T22:22:06.564 INFO:tasks.workunit.client.1.smithi123.stdout:3/763: rmdir d0/d102 0 2024-02-16T22:22:06.568 INFO:tasks.workunit.client.0.smithi114.stdout:6/603: write d6/d2c/d38/d54/f2d [2689606,108223] 0 2024-02-16T22:22:06.569 INFO:tasks.workunit.client.0.smithi114.stdout:6/604: chown d6/da/l44 2220065 1 2024-02-16T22:22:06.571 INFO:tasks.workunit.client.1.smithi123.stdout:1/727: creat d3/d6/db/d10/d2a/ff8 x:0 0 0 2024-02-16T22:22:06.571 INFO:tasks.workunit.client.0.smithi114.stdout:4/711: dread dd/f12 [0,4194304] 0 2024-02-16T22:22:06.571 INFO:tasks.workunit.client.1.smithi123.stdout:1/728: dread - d3/d6/db/d10/d4f/d9f/fc3 zero size 2024-02-16T22:22:06.576 INFO:tasks.workunit.client.0.smithi114.stdout:5/561: sync 2024-02-16T22:22:06.577 INFO:tasks.workunit.client.1.smithi123.stdout:1/729: dwrite d3/d6/f26 [0,4194304] 0 2024-02-16T22:22:06.577 INFO:tasks.workunit.client.0.smithi114.stdout:9/605: mknod d2/dd/d3c/d89/d8b/cc2 0 2024-02-16T22:22:06.578 INFO:tasks.workunit.client.0.smithi114.stdout:5/562: dread - d0/d28/d98/d1e/d63/d9e/fb2 zero size 2024-02-16T22:22:06.578 INFO:tasks.workunit.client.1.smithi123.stdout:1/730: fdatasync d3/d6/d11/fe4 0 2024-02-16T22:22:06.579 INFO:tasks.workunit.client.1.smithi123.stdout:1/731: read d3/d6/d49/f66 [3257388,38069] 0 2024-02-16T22:22:06.581 INFO:tasks.workunit.client.0.smithi114.stdout:7/654: getdents d3/d22 0 2024-02-16T22:22:06.585 INFO:tasks.workunit.client.0.smithi114.stdout:0/545: read d1/d1a/f2e [7002891,105907] 0 2024-02-16T22:22:06.587 INFO:tasks.workunit.client.0.smithi114.stdout:0/546: fsync d1/d16/d48/f6a 0 2024-02-16T22:22:06.587 INFO:tasks.workunit.client.1.smithi123.stdout:2/732: getdents d4/dda 0 2024-02-16T22:22:06.590 INFO:tasks.workunit.client.0.smithi114.stdout:2/549: creat dd/d3b/da7/d88/fb9 x:0 0 0 2024-02-16T22:22:06.590 INFO:tasks.workunit.client.1.smithi123.stdout:0/652: write d1/d8/d13/d1d/d64/dbc/fac [928724,4052] 0 2024-02-16T22:22:06.593 INFO:tasks.workunit.client.0.smithi114.stdout:1/586: rename d4/d9/d46/d67/d92/dca/dcc/c6d to d4/d9/d46/d67/d92/dca/dce/cda 0 2024-02-16T22:22:06.595 INFO:tasks.workunit.client.1.smithi123.stdout:6/744: truncate d0/d96/de3/f58 318597 0 2024-02-16T22:22:06.600 INFO:tasks.workunit.client.0.smithi114.stdout:4/712: mknod dd/d1b/d24/d57/c100 0 2024-02-16T22:22:06.605 INFO:tasks.workunit.client.0.smithi114.stdout:5/563: mkdir d0/d28/d98/d25/d2e/d3b/db5 0 2024-02-16T22:22:06.607 INFO:tasks.workunit.client.1.smithi123.stdout:1/732: mkdir d3/d6/d49/d48/d84/d92/df9 0 2024-02-16T22:22:06.609 INFO:tasks.workunit.client.0.smithi114.stdout:7/655: rmdir d3 39 2024-02-16T22:22:06.612 INFO:tasks.workunit.client.1.smithi123.stdout:0/653: write d1/d8/d13/d1d/f8d [4265743,91944] 0 2024-02-16T22:22:06.613 INFO:tasks.workunit.client.1.smithi123.stdout:6/745: mkdir d0/d96/dca/d10c 0 2024-02-16T22:22:06.615 INFO:tasks.workunit.client.0.smithi114.stdout:0/547: unlink d1/d16/d1f/d28/d51/d61/d6d/d6e/l9b 0 2024-02-16T22:22:06.617 INFO:tasks.workunit.client.0.smithi114.stdout:2/550: fdatasync dd/f5d 0 2024-02-16T22:22:06.621 INFO:tasks.workunit.client.0.smithi114.stdout:1/587: symlink d4/d9/d46/d67/d92/dca/dce/ldb 0 2024-02-16T22:22:06.622 INFO:tasks.workunit.client.1.smithi123.stdout:1/733: mknod d3/d6/db/d10/d2d/cfa 0 2024-02-16T22:22:06.625 INFO:tasks.workunit.client.0.smithi114.stdout:3/596: dwrite d4/d26/d4a/f65 [4194304,4194304] 0 2024-02-16T22:22:06.625 INFO:tasks.workunit.client.1.smithi123.stdout:3/764: creat d0/d20/d29/f103 x:0 0 0 2024-02-16T22:22:06.628 INFO:tasks.workunit.client.1.smithi123.stdout:2/733: symlink d4/d1a/d1d/d13/d25/lef 0 2024-02-16T22:22:06.630 INFO:tasks.workunit.client.0.smithi114.stdout:5/564: symlink d0/df/d53/lb6 0 2024-02-16T22:22:06.631 INFO:tasks.workunit.client.0.smithi114.stdout:5/565: read d0/d28/d98/f5b [1374046,52215] 0 2024-02-16T22:22:06.632 INFO:tasks.workunit.client.1.smithi123.stdout:7/744: sync 2024-02-16T22:22:06.633 INFO:tasks.workunit.client.0.smithi114.stdout:9/606: creat d2/d1a/d23/d35/d47/db2/fc3 x:0 0 0 2024-02-16T22:22:06.634 INFO:tasks.workunit.client.1.smithi123.stdout:7/745: read d3/d6e/d34/f57 [1527521,22737] 0 2024-02-16T22:22:06.637 INFO:tasks.workunit.client.0.smithi114.stdout:0/548: creat d1/d1a/d3e/fa3 x:0 0 0 2024-02-16T22:22:06.638 INFO:tasks.workunit.client.1.smithi123.stdout:6/746: mknod d0/d6/d2f/c10d 0 2024-02-16T22:22:06.646 INFO:tasks.workunit.client.1.smithi123.stdout:3/765: rename d0/d20/d29/d9b/dbd/df4/d5d/dae/dc7/fe1 to d0/d20/d29/d9b/dbd/df4/d5d/dae/db0/f104 0 2024-02-16T22:22:06.646 INFO:tasks.workunit.client.1.smithi123.stdout:2/734: rename d4/d1a/d1d/d13/d15/d6d/d4a to d4/d1a/d1d/d13/d15/d6d/d4a/df0 22 2024-02-16T22:22:06.647 INFO:tasks.workunit.client.1.smithi123.stdout:3/766: write d0/d20/d29/f3c [391042,86334] 0 2024-02-16T22:22:06.647 INFO:tasks.workunit.client.1.smithi123.stdout:1/734: dread d3/d6/d49/d48/d84/d92/fbe [0,4194304] 0 2024-02-16T22:22:06.651 INFO:tasks.workunit.client.1.smithi123.stdout:0/654: creat d1/d4/d17/d54/db4/dc5/fe0 x:0 0 0 2024-02-16T22:22:06.651 INFO:tasks.workunit.client.1.smithi123.stdout:0/655: write d1/d8/da7/fae [291716,119971] 0 2024-02-16T22:22:06.654 INFO:tasks.workunit.client.1.smithi123.stdout:8/788: write d0/d29/da7/d43/d34/dae/fc1 [101128,111463] 0 2024-02-16T22:22:06.656 INFO:tasks.workunit.client.0.smithi114.stdout:5/566: mknod d0/d2d/d30/d9f/cb7 0 2024-02-16T22:22:06.656 INFO:tasks.workunit.client.1.smithi123.stdout:5/726: dwrite d6/dc1/de6/f60 [0,4194304] 0 2024-02-16T22:22:06.660 INFO:tasks.workunit.client.0.smithi114.stdout:7/656: mknod d3/d22/d73/da7/d6f/d89/ce7 0 2024-02-16T22:22:06.664 INFO:tasks.workunit.client.0.smithi114.stdout:0/549: creat d1/d16/d1f/d28/d51/d61/d6d/d79/fa4 x:0 0 0 2024-02-16T22:22:06.667 INFO:tasks.workunit.client.0.smithi114.stdout:0/550: write d1/d16/d1f/d28/d51/d61/d7e/d90/d95/d80/f8f [703820,22494] 0 2024-02-16T22:22:06.668 INFO:tasks.workunit.client.0.smithi114.stdout:2/551: rename dd/df/d1c/d21/d87/f34 to dd/d3b/d8a/fba 0 2024-02-16T22:22:06.669 INFO:tasks.workunit.client.0.smithi114.stdout:2/552: chown dd/f36 12392 1 2024-02-16T22:22:06.672 INFO:tasks.workunit.client.1.smithi123.stdout:3/767: rename d0/d40/l91 to d0/d20/d2e/l105 0 2024-02-16T22:22:06.675 INFO:tasks.workunit.client.1.smithi123.stdout:1/735: symlink d3/d6/db/d10/d2a/lfb 0 2024-02-16T22:22:06.678 INFO:tasks.workunit.client.0.smithi114.stdout:9/607: dread d2/d1a/d23/d32/d46/fb6 [0,4194304] 0 2024-02-16T22:22:06.681 INFO:tasks.workunit.client.1.smithi123.stdout:6/747: mknod d0/d6/d1a/daf/c10e 0 2024-02-16T22:22:06.681 INFO:tasks.workunit.client.1.smithi123.stdout:6/748: fdatasync d0/d6/d8e/fb6 0 2024-02-16T22:22:06.682 INFO:tasks.workunit.client.0.smithi114.stdout:7/657: dread d3/d14/fc4 [0,4194304] 0 2024-02-16T22:22:06.683 INFO:tasks.workunit.client.0.smithi114.stdout:5/567: write d0/df/f1d [3460322,88577] 0 2024-02-16T22:22:06.684 INFO:tasks.workunit.client.1.smithi123.stdout:8/789: symlink d0/d29/da7/dea/l100 0 2024-02-16T22:22:06.689 INFO:tasks.workunit.client.0.smithi114.stdout:8/558: write d4/d24/d2c/f56 [975763,33121] 0 2024-02-16T22:22:06.691 INFO:tasks.workunit.client.1.smithi123.stdout:2/735: fsync d4/f23 0 2024-02-16T22:22:06.694 INFO:tasks.workunit.client.0.smithi114.stdout:2/553: creat dd/d4e/fbb x:0 0 0 2024-02-16T22:22:06.695 INFO:tasks.workunit.client.0.smithi114.stdout:1/588: rmdir d4/d9/d3d/d3f/daf 0 2024-02-16T22:22:06.695 INFO:tasks.workunit.client.1.smithi123.stdout:9/856: dwrite d4/d1e/f42 [4194304,4194304] 0 2024-02-16T22:22:06.703 INFO:tasks.workunit.client.0.smithi114.stdout:4/713: getdents dd/d2e/d56/ddc 0 2024-02-16T22:22:06.707 INFO:tasks.workunit.client.0.smithi114.stdout:9/608: truncate d2/f15 4838770 0 2024-02-16T22:22:06.708 INFO:tasks.workunit.client.1.smithi123.stdout:4/812: write d5/d1c/d23/d47/fde [3706756,68719] 0 2024-02-16T22:22:06.710 INFO:tasks.workunit.client.0.smithi114.stdout:6/605: dwrite d6/f30 [0,4194304] 0 2024-02-16T22:22:06.718 INFO:tasks.workunit.client.0.smithi114.stdout:8/559: mkdir d4/d24/d2c/d67/dc8 0 2024-02-16T22:22:06.719 INFO:tasks.workunit.client.0.smithi114.stdout:8/560: chown d4/d11/d78/d7a/d36/d3b 139024562 1 2024-02-16T22:22:06.720 INFO:tasks.workunit.client.0.smithi114.stdout:8/561: write d4/d11/d78/d7a/d7e/f9a [2299668,122746] 0 2024-02-16T22:22:06.722 INFO:tasks.workunit.client.0.smithi114.stdout:0/551: mknod d1/d16/d1f/d28/d67/ca5 0 2024-02-16T22:22:06.726 INFO:tasks.workunit.client.0.smithi114.stdout:1/589: chown d4/d9/d46/d67/f6e 104 1 2024-02-16T22:22:06.728 INFO:tasks.workunit.client.0.smithi114.stdout:8/562: dread d4/d11/d78/d7a/d7e/fae [0,4194304] 0 2024-02-16T22:22:06.732 INFO:tasks.workunit.client.0.smithi114.stdout:7/658: rename d3/d22/l30 to d3/d3d/le8 0 2024-02-16T22:22:06.733 INFO:tasks.workunit.client.0.smithi114.stdout:6/606: creat d6/dd/d25/d34/fb4 x:0 0 0 2024-02-16T22:22:06.739 INFO:tasks.workunit.client.1.smithi123.stdout:7/746: sync 2024-02-16T22:22:06.740 INFO:tasks.workunit.client.0.smithi114.stdout:0/552: creat d1/d1a/d3e/fa6 x:0 0 0 2024-02-16T22:22:06.741 INFO:tasks.workunit.client.1.smithi123.stdout:7/747: dread d3/d6e/d34/f3c [4194304,4194304] 0 2024-02-16T22:22:06.742 INFO:tasks.workunit.client.0.smithi114.stdout:0/553: stat d1/d16/d1f/d28/d51/d61/d6d/f9c 0 2024-02-16T22:22:06.743 INFO:tasks.workunit.client.0.smithi114.stdout:0/554: chown d1/d16/d18/c49 759 1 2024-02-16T22:22:06.743 INFO:tasks.workunit.client.0.smithi114.stdout:3/597: dwrite d4/d43/f56 [0,4194304] 0 2024-02-16T22:22:06.744 INFO:tasks.workunit.client.0.smithi114.stdout:1/590: readlink d4/l3b 0 2024-02-16T22:22:06.746 INFO:tasks.workunit.client.0.smithi114.stdout:0/555: read d1/d1a/d47/d62/f72 [308277,98130] 0 2024-02-16T22:22:06.747 INFO:tasks.workunit.client.0.smithi114.stdout:4/714: mkdir dd/d1b/d37/d101 0 2024-02-16T22:22:06.750 INFO:tasks.workunit.client.0.smithi114.stdout:8/563: mknod d4/d11/d78/d7a/d36/cc9 0 2024-02-16T22:22:06.757 INFO:tasks.workunit.client.0.smithi114.stdout:6/607: mkdir d6/dd/d25/d34/d61/d62/db5 0 2024-02-16T22:22:06.758 INFO:tasks.workunit.client.0.smithi114.stdout:6/608: fsync d6/d2c/d38/d54/f3d 0 2024-02-16T22:22:06.763 INFO:tasks.workunit.client.1.smithi123.stdout:5/727: dwrite d6/dd/df/d58/d89/fed [0,4194304] 0 2024-02-16T22:22:06.767 INFO:tasks.workunit.client.1.smithi123.stdout:5/728: dread d6/dd/df/d58/d89/fed [0,4194304] 0 2024-02-16T22:22:06.767 INFO:tasks.workunit.client.1.smithi123.stdout:5/729: stat d6/dd/d78/da5/fd3 0 2024-02-16T22:22:06.767 INFO:tasks.workunit.client.0.smithi114.stdout:0/556: symlink d1/d16/d1f/d28/d51/d61/d6d/d79/la7 0 2024-02-16T22:22:06.770 INFO:tasks.workunit.client.0.smithi114.stdout:0/557: write d1/d16/d1f/d28/d51/d61/d7e/d90/f42 [1894498,112502] 0 2024-02-16T22:22:06.770 INFO:tasks.workunit.client.0.smithi114.stdout:4/715: mknod dd/d1b/d37/d38/d87/c102 0 2024-02-16T22:22:06.773 INFO:tasks.workunit.client.0.smithi114.stdout:8/564: mkdir d4/d16/d25/d46/db5/dca 0 2024-02-16T22:22:06.773 INFO:tasks.workunit.client.1.smithi123.stdout:3/768: sync 2024-02-16T22:22:06.774 INFO:tasks.workunit.client.1.smithi123.stdout:6/749: creat d0/d9/dc/d3a/d70/f10f x:0 0 0 2024-02-16T22:22:06.775 INFO:tasks.workunit.client.0.smithi114.stdout:7/659: mkdir d3/d92/dbe/de6/de9 0 2024-02-16T22:22:06.776 INFO:tasks.workunit.client.1.smithi123.stdout:8/790: dread - d0/d61/d66/da5/fe6 zero size 2024-02-16T22:22:06.780 INFO:tasks.workunit.client.0.smithi114.stdout:6/609: readlink d6/d2c/d38/l50 0 2024-02-16T22:22:06.781 INFO:tasks.workunit.client.0.smithi114.stdout:2/554: link dd/d3b/d8a/la8 dd/lbc 0 2024-02-16T22:22:06.783 INFO:tasks.workunit.client.0.smithi114.stdout:1/591: mknod d4/d63/cdc 0 2024-02-16T22:22:06.784 INFO:tasks.workunit.client.1.smithi123.stdout:1/736: symlink d3/d6/db/lfc 0 2024-02-16T22:22:06.785 INFO:tasks.workunit.client.0.smithi114.stdout:5/568: dwrite d0/d28/d98/d25/d2e/f56 [0,4194304] 0 2024-02-16T22:22:06.787 INFO:tasks.workunit.client.0.smithi114.stdout:1/592: write d4/d9/d46/d67/fa8 [175524,71633] 0 2024-02-16T22:22:06.788 INFO:tasks.workunit.client.1.smithi123.stdout:2/736: rename d4/d1a/d1d/d13/d15/d6d to d4/d1a/d1d/d38/d5e/dec/df1 0 2024-02-16T22:22:06.790 INFO:tasks.workunit.client.0.smithi114.stdout:2/555: dwrite dd/f36 [4194304,4194304] 0 2024-02-16T22:22:06.794 INFO:tasks.workunit.client.1.smithi123.stdout:4/813: write d5/d1c/d23/d47/f62 [1127744,111517] 0 2024-02-16T22:22:06.796 INFO:tasks.workunit.client.0.smithi114.stdout:4/716: mkdir dd/d1b/d37/d99/d103 0 2024-02-16T22:22:06.804 INFO:tasks.workunit.client.1.smithi123.stdout:5/730: unlink d6/dd/f50 0 2024-02-16T22:22:06.806 INFO:tasks.workunit.client.0.smithi114.stdout:6/610: creat d6/dd/d25/d34/d58/fb6 x:0 0 0 2024-02-16T22:22:06.806 INFO:tasks.workunit.client.0.smithi114.stdout:6/611: fdatasync d6/d2c/d66/fa5 0 2024-02-16T22:22:06.807 INFO:tasks.workunit.client.0.smithi114.stdout:6/612: fsync d6/d2c/d66/fa5 0 2024-02-16T22:22:06.811 INFO:tasks.workunit.client.1.smithi123.stdout:6/750: mknod d0/d6/d1a/d88/d8b/c110 0 2024-02-16T22:22:06.814 INFO:tasks.workunit.client.0.smithi114.stdout:5/569: rename d0/d10 to d0/d28/d98/d1e/d63/d96/d9a/db8 0 2024-02-16T22:22:06.816 INFO:tasks.workunit.client.1.smithi123.stdout:8/791: symlink d0/d29/da7/d43/d34/l101 0 2024-02-16T22:22:06.817 INFO:tasks.workunit.client.0.smithi114.stdout:2/556: write dd/df/f59 [1007367,38264] 0 2024-02-16T22:22:06.818 INFO:tasks.workunit.client.1.smithi123.stdout:8/792: write d0/d29/da7/d26/d2a/d4d/f57 [1675418,104315] 0 2024-02-16T22:22:06.819 INFO:tasks.workunit.client.1.smithi123.stdout:3/769: dread d0/fc [0,4194304] 0 2024-02-16T22:22:06.820 INFO:tasks.workunit.client.0.smithi114.stdout:1/593: mkdir d4/d9/d46/d67/d92/dca/dcc/d98/ddd 0 2024-02-16T22:22:06.822 INFO:tasks.workunit.client.1.smithi123.stdout:1/737: unlink d3/d6/db/lfc 0 2024-02-16T22:22:06.824 INFO:tasks.workunit.client.0.smithi114.stdout:0/558: unlink d1/d16/d1f/d28/d51/d61/d7e/d90/d95/l8e 0 2024-02-16T22:22:06.826 INFO:tasks.workunit.client.1.smithi123.stdout:7/748: rename d3/d6e/d1d/db2/c79 to d3/d59/cf0 0 2024-02-16T22:22:06.828 INFO:tasks.workunit.client.0.smithi114.stdout:6/613: dread d6/d2c/f6d [0,4194304] 0 2024-02-16T22:22:06.829 INFO:tasks.workunit.client.1.smithi123.stdout:2/737: creat d4/d1a/d1d/d13/d25/d98/dbb/ff2 x:0 0 0 2024-02-16T22:22:06.835 INFO:tasks.workunit.client.1.smithi123.stdout:2/738: dwrite d4/d1a/d1d/d59/d45/fc5 [0,4194304] 0 2024-02-16T22:22:06.835 INFO:tasks.workunit.client.0.smithi114.stdout:6/614: dread d6/d2c/d38/d54/f3d [0,4194304] 0 2024-02-16T22:22:06.835 INFO:tasks.workunit.client.0.smithi114.stdout:3/598: getdents d4/d46/dcf 0 2024-02-16T22:22:06.835 INFO:tasks.workunit.client.1.smithi123.stdout:2/739: read - d4/d1a/dd5/fdf zero size 2024-02-16T22:22:06.836 INFO:tasks.workunit.client.1.smithi123.stdout:2/740: stat d4/d1a/d1d/d38/d5e/de8 0 2024-02-16T22:22:06.837 INFO:tasks.workunit.client.0.smithi114.stdout:3/599: truncate d4/d46/d48/dcc/fc6 142214 0 2024-02-16T22:22:06.837 INFO:tasks.workunit.client.1.smithi123.stdout:2/741: write d4/d1a/d1d/d38/d5e/dec/df1/db1/fb7 [4443190,9048] 0 2024-02-16T22:22:06.844 INFO:tasks.workunit.client.0.smithi114.stdout:9/609: write d2/f15 [1537026,9679] 0 2024-02-16T22:22:06.845 INFO:tasks.workunit.client.1.smithi123.stdout:0/656: getdents d1/d4/d17/dad 0 2024-02-16T22:22:06.846 INFO:tasks.workunit.client.0.smithi114.stdout:1/594: creat d4/d63/d69/d72/fde x:0 0 0 2024-02-16T22:22:06.847 INFO:tasks.workunit.client.0.smithi114.stdout:1/595: chown d4/d9/d3d/d3f/f75 127078 1 2024-02-16T22:22:06.848 INFO:tasks.workunit.client.1.smithi123.stdout:6/751: symlink d0/d6/d2f/da5/l111 0 2024-02-16T22:22:06.848 INFO:tasks.workunit.client.1.smithi123.stdout:6/752: chown d0/d6/d2f/dbd/lef 16523407 1 2024-02-16T22:22:06.852 INFO:tasks.workunit.client.1.smithi123.stdout:3/770: fsync d0/d20/d29/d9b/dbd/df4/d5d/dae/db0/fb3 0 2024-02-16T22:22:06.852 INFO:tasks.workunit.client.1.smithi123.stdout:3/771: chown d0/d20/d29/d9b/dbd/df4/d5d/dae/db0 1229998 1 2024-02-16T22:22:06.853 INFO:tasks.workunit.client.0.smithi114.stdout:6/615: mkdir d6/dd/d25/d34/d61/d62/d8b/db7 0 2024-02-16T22:22:06.853 INFO:tasks.workunit.client.0.smithi114.stdout:8/565: write d4/d16/d25/d39/f82 [93442,53350] 0 2024-02-16T22:22:06.854 INFO:tasks.workunit.client.0.smithi114.stdout:8/566: chown d4/d24/d6d 906149 1 2024-02-16T22:22:06.855 INFO:tasks.workunit.client.0.smithi114.stdout:6/616: write d6/d2c/d38/d54/f3d [1282185,52440] 0 2024-02-16T22:22:06.856 INFO:tasks.workunit.client.0.smithi114.stdout:8/567: write d4/d11/d78/f86 [1856855,57984] 0 2024-02-16T22:22:06.860 INFO:tasks.workunit.client.0.smithi114.stdout:4/717: rename dd/d2e/c3b to dd/d1b/d37/d38/c104 0 2024-02-16T22:22:06.863 INFO:tasks.workunit.client.0.smithi114.stdout:2/557: mkdir dd/dbd 0 2024-02-16T22:22:06.863 INFO:tasks.workunit.client.0.smithi114.stdout:2/558: dread - dd/d3b/d8a/f9a zero size 2024-02-16T22:22:06.866 INFO:tasks.workunit.client.0.smithi114.stdout:1/596: mkdir d4/d9/d46/d47/ddf 0 2024-02-16T22:22:06.867 INFO:tasks.workunit.client.1.smithi123.stdout:2/742: truncate d4/d1a/d1d/d13/d25/f77 1865461 0 2024-02-16T22:22:06.867 INFO:tasks.workunit.client.0.smithi114.stdout:7/660: getdents d3/d22/d76 0 2024-02-16T22:22:06.868 INFO:tasks.workunit.client.1.smithi123.stdout:2/743: write d4/d1a/d1d/d38/d5e/dec/df1/d4a/da2/fcb [782990,44328] 0 2024-02-16T22:22:06.869 INFO:tasks.workunit.client.1.smithi123.stdout:2/744: chown d4/d1a/d1d/d38/d5e/c9c 16 1 2024-02-16T22:22:06.869 INFO:tasks.workunit.client.1.smithi123.stdout:5/731: link d6/dd/df/d58/d3f/dbe/d37/f52 d6/dd/df/d76/d7f/f101 0 2024-02-16T22:22:06.870 INFO:tasks.workunit.client.1.smithi123.stdout:5/732: dread - d6/dac/ff5 zero size 2024-02-16T22:22:06.870 INFO:tasks.workunit.client.0.smithi114.stdout:3/600: creat d4/d46/d48/d75/daa/fd0 x:0 0 0 2024-02-16T22:22:06.872 INFO:tasks.workunit.client.1.smithi123.stdout:0/657: readlink d1/d8/d3d/l49 0 2024-02-16T22:22:06.874 INFO:tasks.workunit.client.1.smithi123.stdout:6/753: chown d0/d96/de3/f58 832833 1 2024-02-16T22:22:06.875 INFO:tasks.workunit.client.1.smithi123.stdout:9/857: truncate d4/d36/daa/d20/ff7 22472 0 2024-02-16T22:22:06.876 INFO:tasks.workunit.client.1.smithi123.stdout:4/814: write d5/d1c/d23/d6d/f78 [2193093,55416] 0 2024-02-16T22:22:06.877 INFO:tasks.workunit.client.1.smithi123.stdout:3/772: creat d0/d20/d29/d9b/dbd/df4/d5d/d60/f106 x:0 0 0 2024-02-16T22:22:06.878 INFO:tasks.workunit.client.1.smithi123.stdout:4/815: readlink d5/d1c/d23/d74/d92/d9b/l9d 0 2024-02-16T22:22:06.879 INFO:tasks.workunit.client.0.smithi114.stdout:0/559: rename d1/d16/d1f/d28/d51/f91 to d1/d1a/d47/d62/d7f/fa8 0 2024-02-16T22:22:06.879 INFO:tasks.workunit.client.1.smithi123.stdout:4/816: chown d5/f29 2075869664 1 2024-02-16T22:22:06.882 INFO:tasks.workunit.client.1.smithi123.stdout:4/817: read d5/d1c/d23/d31/f66 [2842938,55756] 0 2024-02-16T22:22:06.882 INFO:tasks.workunit.client.1.smithi123.stdout:9/858: dwrite d4/d36/d8e/d6f/f107 [0,4194304] 0 2024-02-16T22:22:06.884 INFO:tasks.workunit.client.0.smithi114.stdout:0/560: chown d1/d1a/f26 12410868 1 2024-02-16T22:22:06.884 INFO:tasks.workunit.client.0.smithi114.stdout:5/570: creat d0/fb9 x:0 0 0 2024-02-16T22:22:06.885 INFO:tasks.workunit.client.0.smithi114.stdout:0/561: chown d1/d1a/f26 85300 1 2024-02-16T22:22:06.886 INFO:tasks.workunit.client.1.smithi123.stdout:7/749: write d3/d6e/d34/f75 [747742,2652] 0 2024-02-16T22:22:06.887 INFO:tasks.workunit.client.1.smithi123.stdout:3/773: dwrite d0/d20/d94/fe0 [0,4194304] 0 2024-02-16T22:22:06.888 INFO:tasks.workunit.client.0.smithi114.stdout:0/562: chown d1/d16/d1f/f63 1519640 1 2024-02-16T22:22:06.888 INFO:tasks.workunit.client.1.smithi123.stdout:7/750: write d3/d6e/d24/fb9 [4579393,76301] 0 2024-02-16T22:22:06.889 INFO:tasks.workunit.client.1.smithi123.stdout:6/754: creat d0/d6/d2f/da5/dc6/f112 x:0 0 0 2024-02-16T22:22:06.890 INFO:tasks.workunit.client.0.smithi114.stdout:7/661: mkdir d3/d22/db7/dea 0 2024-02-16T22:22:06.891 INFO:tasks.workunit.client.1.smithi123.stdout:7/751: write d3/d6e/d1d/d97/fc4 [746366,67905] 0 2024-02-16T22:22:06.894 INFO:tasks.workunit.client.0.smithi114.stdout:3/601: unlink d4/d46/d48/d80/fa5 0 2024-02-16T22:22:06.897 INFO:tasks.workunit.client.0.smithi114.stdout:8/568: symlink d4/d24/d6d/dc7/lcb 0 2024-02-16T22:22:06.903 INFO:tasks.workunit.client.0.smithi114.stdout:8/569: fsync d4/d11/d78/f86 0 2024-02-16T22:22:06.906 INFO:tasks.workunit.client.0.smithi114.stdout:3/602: dread d4/d24/d2d/d45/d74/dc7/fcd [8388608,4194304] 0 2024-02-16T22:22:06.906 INFO:tasks.workunit.client.0.smithi114.stdout:3/603: chown d4/dd/c59 1798 1 2024-02-16T22:22:06.906 INFO:tasks.workunit.client.0.smithi114.stdout:3/604: fdatasync d4/d24/d2d/f4e 0 2024-02-16T22:22:06.908 INFO:tasks.workunit.client.1.smithi123.stdout:2/745: write f2 [7463156,7395] 0 2024-02-16T22:22:06.909 INFO:tasks.workunit.client.1.smithi123.stdout:8/793: sync 2024-02-16T22:22:06.910 INFO:tasks.workunit.client.1.smithi123.stdout:4/818: mknod d5/d1c/d23/d74/d92/cf8 0 2024-02-16T22:22:06.911 INFO:tasks.workunit.client.1.smithi123.stdout:5/733: write d6/dd/df/d58/d3f/dbe/f66 [1448454,13354] 0 2024-02-16T22:22:06.915 INFO:tasks.workunit.client.0.smithi114.stdout:5/571: mknod d0/d28/d98/d25/d3e/cba 0 2024-02-16T22:22:06.920 INFO:tasks.workunit.client.0.smithi114.stdout:0/563: creat d1/d16/d1f/fa9 x:0 0 0 2024-02-16T22:22:06.920 INFO:tasks.workunit.client.0.smithi114.stdout:0/564: dread - d1/d16/d55/d92/f93 zero size 2024-02-16T22:22:06.921 INFO:tasks.workunit.client.1.smithi123.stdout:6/755: creat d0/d74/f113 x:0 0 0 2024-02-16T22:22:06.922 INFO:tasks.workunit.client.0.smithi114.stdout:7/662: creat d3/d22/feb x:0 0 0 2024-02-16T22:22:06.923 INFO:tasks.workunit.client.1.smithi123.stdout:3/774: truncate d0/d20/d2e/f2f 1817796 0 2024-02-16T22:22:06.924 INFO:tasks.workunit.client.0.smithi114.stdout:6/617: truncate d6/d2c/d38/d54/f3d 746403 0 2024-02-16T22:22:06.925 INFO:tasks.workunit.client.1.smithi123.stdout:3/775: read d0/f4c [4358624,101722] 0 2024-02-16T22:22:06.925 INFO:tasks.workunit.client.1.smithi123.stdout:1/738: getdents d3/d6/d11/d8b 0 2024-02-16T22:22:06.926 INFO:tasks.workunit.client.1.smithi123.stdout:1/739: dread - d3/d6/db/d10/d4f/fd4 zero size 2024-02-16T22:22:06.931 INFO:tasks.workunit.client.1.smithi123.stdout:4/819: chown d5/d49/f91 129557615 1 2024-02-16T22:22:06.934 INFO:tasks.workunit.client.1.smithi123.stdout:9/859: mknod d4/d36/daa/d20/dcb/d10a/c117 0 2024-02-16T22:22:06.934 INFO:tasks.workunit.client.1.smithi123.stdout:9/860: truncate d4/f85 780640 0 2024-02-16T22:22:06.937 INFO:tasks.workunit.client.0.smithi114.stdout:9/610: rename d2/dd/d3c/f4f to d2/d33/fc4 0 2024-02-16T22:22:06.940 INFO:tasks.workunit.client.1.smithi123.stdout:5/734: rename d6/dd/df/d76/d7f to d6/dd/df/d9f/d102 0 2024-02-16T22:22:06.940 INFO:tasks.workunit.client.1.smithi123.stdout:5/735: chown d6/dd/d44 2534603 1 2024-02-16T22:22:06.941 INFO:tasks.workunit.client.1.smithi123.stdout:5/736: fdatasync d6/dd/d78/da5/ff7 0 2024-02-16T22:22:06.943 INFO:tasks.workunit.client.1.smithi123.stdout:0/658: creat d1/d8/d13/d1d/fe1 x:0 0 0 2024-02-16T22:22:06.945 INFO:tasks.workunit.client.0.smithi114.stdout:5/572: symlink d0/d28/d98/d25/d3e/d7f/lbb 0 2024-02-16T22:22:06.951 INFO:tasks.workunit.client.0.smithi114.stdout:1/597: creat d4/d63/d69/d7c/d9b/db0/fe0 x:0 0 0 2024-02-16T22:22:06.953 INFO:tasks.workunit.client.1.smithi123.stdout:7/752: write d3/d6e/f2e [1003258,119712] 0 2024-02-16T22:22:06.957 INFO:tasks.workunit.client.0.smithi114.stdout:6/618: truncate d6/d2c/f6d 4853682 0 2024-02-16T22:22:06.958 INFO:tasks.workunit.client.0.smithi114.stdout:6/619: stat d6/d2c/l63 0 2024-02-16T22:22:06.958 INFO:tasks.workunit.client.0.smithi114.stdout:6/620: chown d6/d2c 66191 1 2024-02-16T22:22:06.959 INFO:tasks.workunit.client.0.smithi114.stdout:6/621: truncate d6/f18 9119454 0 2024-02-16T22:22:06.961 INFO:tasks.workunit.client.0.smithi114.stdout:8/570: symlink d4/d16/d25/d46/db5/dca/lcc 0 2024-02-16T22:22:06.962 INFO:tasks.workunit.client.1.smithi123.stdout:1/740: write d3/d6/d49/d48/d84/d92/dec/ff6 [768270,103575] 0 2024-02-16T22:22:06.965 INFO:tasks.workunit.client.0.smithi114.stdout:4/718: rename dd/d59/f71 to dd/d1b/d41/d74/f105 0 2024-02-16T22:22:06.966 INFO:tasks.workunit.client.1.smithi123.stdout:4/820: chown d5/d1c/d23/c41 31672319 1 2024-02-16T22:22:06.967 INFO:tasks.workunit.client.1.smithi123.stdout:4/821: chown d5/f63 0 1 2024-02-16T22:22:06.968 INFO:tasks.workunit.client.0.smithi114.stdout:2/559: getdents dd/df/d1c/d21/d87 0 2024-02-16T22:22:06.968 INFO:tasks.workunit.client.1.smithi123.stdout:5/737: symlink d6/dd/df/d1f/ddc/l103 0 2024-02-16T22:22:06.969 INFO:tasks.workunit.client.0.smithi114.stdout:5/573: mknod d0/d28/d98/d1e/d63/d96/d9a/cbc 0 2024-02-16T22:22:06.970 INFO:tasks.workunit.client.1.smithi123.stdout:6/756: dwrite d0/d6/d1a/f6f [0,4194304] 0 2024-02-16T22:22:06.970 INFO:tasks.workunit.client.1.smithi123.stdout:5/738: chown d6/dd/df/f79 112176 1 2024-02-16T22:22:06.970 INFO:tasks.workunit.client.1.smithi123.stdout:0/659: rmdir d1/d8/d13 39 2024-02-16T22:22:06.975 INFO:tasks.workunit.client.0.smithi114.stdout:7/663: creat d3/d87/dd1/fec x:0 0 0 2024-02-16T22:22:06.979 INFO:tasks.workunit.client.0.smithi114.stdout:6/622: mknod d6/d83/cb8 0 2024-02-16T22:22:06.980 INFO:tasks.workunit.client.0.smithi114.stdout:6/623: stat d6/d2c/d38/f92 0 2024-02-16T22:22:06.981 INFO:tasks.workunit.client.1.smithi123.stdout:3/776: truncate d0/fe 1593165 0 2024-02-16T22:22:06.985 INFO:tasks.workunit.client.0.smithi114.stdout:2/560: dread dd/d3b/d8a/f93 [0,4194304] 0 2024-02-16T22:22:06.985 INFO:tasks.workunit.client.0.smithi114.stdout:8/571: rename d4/d11/d78/d7a/d36/d3b/c5a to d4/d24/d6d/dc7/ccd 0 2024-02-16T22:22:06.988 INFO:tasks.workunit.client.1.smithi123.stdout:1/741: truncate d3/d6/d49/fb1 2103449 0 2024-02-16T22:22:06.990 INFO:tasks.workunit.client.0.smithi114.stdout:4/719: chown dd/d2e/d3f/caa 48 1 2024-02-16T22:22:06.992 INFO:tasks.workunit.client.1.smithi123.stdout:4/822: dread d5/d1c/fac [0,4194304] 0 2024-02-16T22:22:06.995 INFO:tasks.workunit.client.1.smithi123.stdout:6/757: mknod d0/d96/c114 0 2024-02-16T22:22:06.996 INFO:tasks.workunit.client.0.smithi114.stdout:1/598: dwrite d4/d9/d46/d67/d92/dca/dcc/fd5 [0,4194304] 0 2024-02-16T22:22:06.998 INFO:tasks.workunit.client.0.smithi114.stdout:5/574: truncate d0/d28/d98/d25/f75 1409562 0 2024-02-16T22:22:06.998 INFO:tasks.workunit.client.1.smithi123.stdout:6/758: dread d0/d6/f63 [0,4194304] 0 2024-02-16T22:22:07.003 INFO:tasks.workunit.client.1.smithi123.stdout:5/739: mknod d6/dd/df/d58/d89/d98/c104 0 2024-02-16T22:22:07.006 INFO:tasks.workunit.client.1.smithi123.stdout:7/753: truncate d3/fc7 789164 0 2024-02-16T22:22:07.007 INFO:tasks.workunit.client.0.smithi114.stdout:7/664: creat d3/d22/d73/da7/d6f/d9c/fed x:0 0 0 2024-02-16T22:22:07.008 INFO:tasks.workunit.client.1.smithi123.stdout:3/777: mkdir d0/d20/d29/d9b/dbd/df5/d107 0 2024-02-16T22:22:07.009 INFO:tasks.workunit.client.0.smithi114.stdout:6/624: dread d6/d2c/d38/d54/f2d [0,4194304] 0 2024-02-16T22:22:07.010 INFO:tasks.workunit.client.0.smithi114.stdout:3/605: getdents d4/d46/d48/d75/daa 0 2024-02-16T22:22:07.011 INFO:tasks.workunit.client.1.smithi123.stdout:4/823: dread - d5/d1c/d23/f9c zero size 2024-02-16T22:22:07.015 INFO:tasks.workunit.client.0.smithi114.stdout:8/572: unlink d4/d11/d78/d7a/d36/c9e 0 2024-02-16T22:22:07.019 INFO:tasks.workunit.client.1.smithi123.stdout:0/660: write d1/d8/d13/d1d/d64/dbc/fac [120477,129225] 0 2024-02-16T22:22:07.020 INFO:tasks.workunit.client.1.smithi123.stdout:0/661: write d1/d4/d17/f53 [114996,38108] 0 2024-02-16T22:22:07.020 INFO:tasks.workunit.client.0.smithi114.stdout:4/720: creat dd/d2e/d56/ddc/f106 x:0 0 0 2024-02-16T22:22:07.022 INFO:tasks.workunit.client.1.smithi123.stdout:6/759: creat d0/d6/d1a/daf/f115 x:0 0 0 2024-02-16T22:22:07.026 INFO:tasks.workunit.client.0.smithi114.stdout:4/721: dwrite dd/d1b/d24/fd6 [0,4194304] 0 2024-02-16T22:22:07.030 INFO:tasks.workunit.client.0.smithi114.stdout:1/599: dread - d4/d9/d46/d4d/f78 zero size 2024-02-16T22:22:07.034 INFO:tasks.workunit.client.0.smithi114.stdout:5/575: creat d0/d28/d98/d1e/d63/fbd x:0 0 0 2024-02-16T22:22:07.035 INFO:tasks.workunit.client.1.smithi123.stdout:9/861: getdents d4/d36/daa/d2b/d6c 0 2024-02-16T22:22:07.037 INFO:tasks.workunit.client.0.smithi114.stdout:0/565: getdents d1/d16/d1f/d28/d51/d61/d7e/d90/d95/d80 0 2024-02-16T22:22:07.037 INFO:tasks.workunit.client.1.smithi123.stdout:7/754: write d3/d6e/d52/f71 [1010803,92174] 0 2024-02-16T22:22:07.039 INFO:tasks.workunit.client.1.smithi123.stdout:4/824: mknod d5/d1c/d23/d56/d6e/d9f/da1/cf9 0 2024-02-16T22:22:07.040 INFO:tasks.workunit.client.0.smithi114.stdout:7/665: creat d3/d87/fee x:0 0 0 2024-02-16T22:22:07.042 INFO:tasks.workunit.client.0.smithi114.stdout:7/666: write d3/d22/d61/d6a/fcd [580646,50713] 0 2024-02-16T22:22:07.043 INFO:tasks.workunit.client.0.smithi114.stdout:6/625: fsync d6/da/f1b 0 2024-02-16T22:22:07.045 INFO:tasks.workunit.client.0.smithi114.stdout:3/606: creat d4/d26/d91/fd1 x:0 0 0 2024-02-16T22:22:07.045 INFO:tasks.workunit.client.1.smithi123.stdout:7/755: dread d3/d6e/d52/f88 [0,4194304] 0 2024-02-16T22:22:07.048 INFO:tasks.workunit.client.1.smithi123.stdout:2/746: sync 2024-02-16T22:22:07.048 INFO:tasks.workunit.client.1.smithi123.stdout:8/794: sync 2024-02-16T22:22:07.049 INFO:tasks.workunit.client.1.smithi123.stdout:0/662: symlink d1/da0/le2 0 2024-02-16T22:22:07.051 INFO:tasks.workunit.client.0.smithi114.stdout:2/561: rename dd/df/f11 to dd/d3b/da7/d4c/fbe 0 2024-02-16T22:22:07.057 INFO:tasks.workunit.client.0.smithi114.stdout:9/611: link f1 d2/d1a/d23/d35/d47/d63/fc5 0 2024-02-16T22:22:07.060 INFO:tasks.workunit.client.0.smithi114.stdout:5/576: stat d0/d28/d98/f1f 0 2024-02-16T22:22:07.061 INFO:tasks.workunit.client.0.smithi114.stdout:5/577: write d0/d28/d98/d25/d2e/d3b/f77 [2098223,60119] 0 2024-02-16T22:22:07.061 INFO:tasks.workunit.client.1.smithi123.stdout:6/760: dwrite d0/d9/dc/d3a/d70/f72 [0,4194304] 0 2024-02-16T22:22:07.064 INFO:tasks.workunit.client.1.smithi123.stdout:4/825: dread - d5/d1c/d23/d76/fa6 zero size 2024-02-16T22:22:07.069 INFO:tasks.workunit.client.0.smithi114.stdout:6/626: creat d6/da/fb9 x:0 0 0 2024-02-16T22:22:07.069 INFO:tasks.workunit.client.1.smithi123.stdout:2/747: fdatasync d4/d1a/d1d/f2c 0 2024-02-16T22:22:07.069 INFO:tasks.workunit.client.1.smithi123.stdout:6/761: dwrite d0/d6/d91/da2/ffd [0,4194304] 0 2024-02-16T22:22:07.069 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:06 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:22:07.069 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:06 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:22:07.069 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:06 smithi114 ceph-mon[29235]: pgmap v31: 65 pgs: 65 active+clean; 2.4 GiB data, 6.8 GiB used, 530 GiB / 536 GiB avail; 81 MiB/s rd, 193 MiB/s wr, 408 op/s 2024-02-16T22:22:07.072 INFO:tasks.workunit.client.0.smithi114.stdout:7/667: dwrite d3/fe [4194304,4194304] 0 2024-02-16T22:22:07.072 INFO:tasks.workunit.client.0.smithi114.stdout:8/573: mkdir d4/d11/d78/d7a/d31/dc6/dce 0 2024-02-16T22:22:07.075 INFO:tasks.workunit.client.1.smithi123.stdout:1/742: sync 2024-02-16T22:22:07.075 INFO:tasks.workunit.client.1.smithi123.stdout:0/663: mknod d1/d4/d17/d54/db4/dc5/ce3 0 2024-02-16T22:22:07.078 INFO:tasks.workunit.client.1.smithi123.stdout:5/740: creat d6/dd/df/d58/d3f/dbe/f105 x:0 0 0 2024-02-16T22:22:07.080 INFO:tasks.workunit.client.0.smithi114.stdout:0/566: rename d1/d16/d55/l8b to d1/d16/d48/laa 0 2024-02-16T22:22:07.082 INFO:tasks.workunit.client.1.smithi123.stdout:1/743: dwrite d3/d6/db/fdd [0,4194304] 0 2024-02-16T22:22:07.086 INFO:tasks.workunit.client.0.smithi114.stdout:6/627: creat d6/d2c/fba x:0 0 0 2024-02-16T22:22:07.088 INFO:tasks.workunit.client.1.smithi123.stdout:7/756: mkdir d3/d5/d6c/da6/df1 0 2024-02-16T22:22:07.092 INFO:tasks.workunit.client.0.smithi114.stdout:7/668: creat d3/d22/d73/da7/d6f/d9c/fef x:0 0 0 2024-02-16T22:22:07.092 INFO:tasks.workunit.client.0.smithi114.stdout:7/669: read - d3/d22/d73/da7/d6f/d9c/fef zero size 2024-02-16T22:22:07.092 INFO:tasks.workunit.client.1.smithi123.stdout:4/826: truncate d5/d1c/d23/d56/d6e/fe7 301155 0 2024-02-16T22:22:07.096 INFO:tasks.workunit.client.0.smithi114.stdout:3/607: rename d4/d26/f63 to d4/d46/d48/fd2 0 2024-02-16T22:22:07.097 INFO:tasks.workunit.client.0.smithi114.stdout:3/608: stat d4/d43/f8a 0 2024-02-16T22:22:07.097 INFO:tasks.workunit.client.1.smithi123.stdout:6/762: mkdir d0/d6/d91/d93/d116 0 2024-02-16T22:22:07.099 INFO:tasks.workunit.client.1.smithi123.stdout:2/748: mkdir d4/d1a/d1d/d13/d25/d72/df3 0 2024-02-16T22:22:07.099 INFO:tasks.workunit.client.0.smithi114.stdout:0/567: mknod d1/d1a/d47/d62/d7f/cab 0 2024-02-16T22:22:07.100 INFO:tasks.workunit.client.1.smithi123.stdout:0/664: mknod d1/d4/d17/d54/d8e/d96/ce4 0 2024-02-16T22:22:07.101 INFO:tasks.workunit.client.0.smithi114.stdout:4/722: link dd/d1b/d37/d38/d87/dad/cce dd/d1b/d41/dc3/c107 0 2024-02-16T22:22:07.101 INFO:tasks.workunit.client.1.smithi123.stdout:0/665: chown d1/d8/d13/l86 7 1 2024-02-16T22:22:07.101 INFO:tasks.workunit.client.0.smithi114.stdout:4/723: truncate dd/ff 1796434 0 2024-02-16T22:22:07.104 INFO:tasks.workunit.client.1.smithi123.stdout:3/778: creat d0/d20/d5a/f108 x:0 0 0 2024-02-16T22:22:07.104 INFO:tasks.workunit.client.0.smithi114.stdout:5/578: symlink d0/df/d53/db1/lbe 0 2024-02-16T22:22:07.106 INFO:tasks.workunit.client.1.smithi123.stdout:5/741: creat d6/dd/df/d58/d56/f106 x:0 0 0 2024-02-16T22:22:07.107 INFO:tasks.workunit.client.0.smithi114.stdout:6/628: creat d6/dd/d25/d34/d61/d62/d8b/fbb x:0 0 0 2024-02-16T22:22:07.107 INFO:tasks.workunit.client.1.smithi123.stdout:5/742: write d6/dd/d44/f47 [1386007,37069] 0 2024-02-16T22:22:07.107 INFO:tasks.workunit.client.0.smithi114.stdout:6/629: chown d6/d2c/f95 60187 1 2024-02-16T22:22:07.108 INFO:tasks.workunit.client.1.smithi123.stdout:9/862: creat d4/d36/daa/d20/f118 x:0 0 0 2024-02-16T22:22:07.109 INFO:tasks.workunit.client.1.smithi123.stdout:0/666: dread d1/d8/d13/f4d [0,4194304] 0 2024-02-16T22:22:07.109 INFO:tasks.workunit.client.1.smithi123.stdout:9/863: chown d4/d36/daa/d2b/de1/df1 31260131 1 2024-02-16T22:22:07.110 INFO:tasks.workunit.client.1.smithi123.stdout:9/864: read d4/d36/fa8 [1259344,124219] 0 2024-02-16T22:22:07.112 INFO:tasks.workunit.client.0.smithi114.stdout:2/562: creat dd/d6b/fbf x:0 0 0 2024-02-16T22:22:07.113 INFO:tasks.workunit.client.1.smithi123.stdout:8/795: creat d0/df/f102 x:0 0 0 2024-02-16T22:22:07.114 INFO:tasks.workunit.client.0.smithi114.stdout:2/563: truncate dd/df/d1c/fa4 456930 0 2024-02-16T22:22:07.114 INFO:tasks.workunit.client.1.smithi123.stdout:8/796: chown d0/d61/d66/d7e/fc7 5 1 2024-02-16T22:22:07.114 INFO:tasks.workunit.client.1.smithi123.stdout:8/797: write d0/d61/d66/d7e/ff1 [327034,116549] 0 2024-02-16T22:22:07.115 INFO:tasks.workunit.client.0.smithi114.stdout:7/670: rename d3/d22/d61/c71 to d3/d22/d73/da7/d6f/cf0 0 2024-02-16T22:22:07.117 INFO:tasks.workunit.client.1.smithi123.stdout:7/757: creat d3/d6e/d1d/db2/d3d/db1/ff2 x:0 0 0 2024-02-16T22:22:07.123 INFO:tasks.workunit.client.0.smithi114.stdout:0/568: creat d1/d16/d1f/d28/d51/d61/d7e/d90/fac x:0 0 0 2024-02-16T22:22:07.124 INFO:tasks.workunit.client.1.smithi123.stdout:6/763: truncate d0/d9/dc/d39/fa9 434015 0 2024-02-16T22:22:07.124 INFO:tasks.workunit.client.0.smithi114.stdout:1/600: link d4/d63/d69/d7c/d9b/db0/la0 d4/d9/d46/d67/d92/dca/dcc/d70/le1 0 2024-02-16T22:22:07.126 INFO:tasks.workunit.client.1.smithi123.stdout:2/749: dread - d4/d1a/d1d/d59/d45/fa9 zero size 2024-02-16T22:22:07.127 INFO:tasks.workunit.client.0.smithi114.stdout:5/579: creat d0/d28/d98/d25/d2e/d3b/fbf x:0 0 0 2024-02-16T22:22:07.129 INFO:tasks.workunit.client.0.smithi114.stdout:6/630: creat d6/d2c/d38/d41/d60/fbc x:0 0 0 2024-02-16T22:22:07.130 INFO:tasks.workunit.client.0.smithi114.stdout:5/580: truncate d0/d28/d98/d1e/d63/d9e/fb2 348772 0 2024-02-16T22:22:07.132 INFO:tasks.workunit.client.0.smithi114.stdout:5/581: read - d0/d28/d98/d1e/d86/f92 zero size 2024-02-16T22:22:07.133 INFO:tasks.workunit.client.0.smithi114.stdout:3/609: dread d4/d46/d48/d75/f9a [0,4194304] 0 2024-02-16T22:22:07.133 INFO:tasks.workunit.client.1.smithi123.stdout:5/743: creat d6/dd/df/d76/f107 x:0 0 0 2024-02-16T22:22:07.135 INFO:tasks.workunit.client.1.smithi123.stdout:8/798: dread d0/df/f16 [0,4194304] 0 2024-02-16T22:22:07.135 INFO:tasks.workunit.client.0.smithi114.stdout:4/724: rename dd/d1b/d41/d74/d9b/ff8 to dd/d1b/db0/f108 0 2024-02-16T22:22:07.136 INFO:tasks.workunit.client.1.smithi123.stdout:8/799: dread d0/d29/da7/d26/d2a/d4d/fb4 [0,4194304] 0 2024-02-16T22:22:07.137 INFO:tasks.workunit.client.0.smithi114.stdout:4/725: write dd/d1b/d41/d74/fd9 [803879,91737] 0 2024-02-16T22:22:07.139 INFO:tasks.workunit.client.0.smithi114.stdout:7/671: unlink d3/d22/d73/da7/db4/fcc 0 2024-02-16T22:22:07.140 INFO:tasks.workunit.client.1.smithi123.stdout:0/667: truncate d1/d4/d42/f7e 31486 0 2024-02-16T22:22:07.141 INFO:tasks.workunit.client.0.smithi114.stdout:9/612: getdents d2/d1a/d23/d35/d47/d63/dbb 0 2024-02-16T22:22:07.142 INFO:tasks.workunit.client.1.smithi123.stdout:1/744: truncate d3/f7c 3104160 0 2024-02-16T22:22:07.143 INFO:tasks.workunit.client.0.smithi114.stdout:9/613: fdatasync d2/d1a/d23/d35/d47/f92 0 2024-02-16T22:22:07.143 INFO:tasks.workunit.client.1.smithi123.stdout:1/745: write d3/d6/db/d10/d4f/d9f/db4/ddf/ff1 [899526,50037] 0 2024-02-16T22:22:07.144 INFO:tasks.workunit.client.0.smithi114.stdout:0/569: fdatasync d1/d16/d1f/d28/d51/d60/f5e 0 2024-02-16T22:22:07.146 INFO:tasks.workunit.client.0.smithi114.stdout:9/614: dread - d2/d1a/d23/d35/d47/db2/fc3 zero size 2024-02-16T22:22:07.148 INFO:tasks.workunit.client.0.smithi114.stdout:7/672: dwrite d3/d22/d61/d6a/fc9 [0,4194304] 0 2024-02-16T22:22:07.150 INFO:tasks.workunit.client.0.smithi114.stdout:8/574: getdents d4/d11/d78/d7a/d31 0 2024-02-16T22:22:07.153 INFO:tasks.workunit.client.1.smithi123.stdout:4/827: rename f4 to d5/d1c/d5e/dc3/ffa 0 2024-02-16T22:22:07.154 INFO:tasks.workunit.client.0.smithi114.stdout:2/564: getdents dd/d6b/da5 0 2024-02-16T22:22:07.155 INFO:tasks.workunit.client.0.smithi114.stdout:2/565: fdatasync dd/d3b/fab 0 2024-02-16T22:22:07.160 INFO:tasks.workunit.client.0.smithi114.stdout:3/610: mknod d4/dd/d33/d72/cd3 0 2024-02-16T22:22:07.160 INFO:tasks.workunit.client.1.smithi123.stdout:9/865: write d4/d36/daa/d2b/f53 [3609098,59747] 0 2024-02-16T22:22:07.161 INFO:tasks.workunit.client.0.smithi114.stdout:3/611: write d4/dd/f19 [2002370,21810] 0 2024-02-16T22:22:07.164 INFO:tasks.workunit.client.0.smithi114.stdout:4/726: unlink dd/d1b/d37/d38/fa3 0 2024-02-16T22:22:07.164 INFO:tasks.workunit.client.1.smithi123.stdout:6/764: dread d0/d9/dc/f23 [0,4194304] 0 2024-02-16T22:22:07.165 INFO:tasks.workunit.client.1.smithi123.stdout:5/744: mknod d6/dd/df/d58/d3f/d6f/c108 0 2024-02-16T22:22:07.171 INFO:tasks.workunit.client.0.smithi114.stdout:9/615: creat d2/d1a/d23/d35/d47/db2/fc6 x:0 0 0 2024-02-16T22:22:07.173 INFO:tasks.workunit.client.0.smithi114.stdout:7/673: mknod d3/d22/d76/cf1 0 2024-02-16T22:22:07.177 INFO:tasks.workunit.client.1.smithi123.stdout:1/746: rename d3/d6/db/d10/d4f to d3/d6/db/d10/d2d/d8c/dfd 0 2024-02-16T22:22:07.180 INFO:tasks.workunit.client.0.smithi114.stdout:7/674: dwrite d3/d22/d73/da7/d4e/fc8 [0,4194304] 0 2024-02-16T22:22:07.182 INFO:tasks.workunit.client.0.smithi114.stdout:3/612: readlink d4/d26/d4a/l7d 0 2024-02-16T22:22:07.183 INFO:tasks.workunit.client.1.smithi123.stdout:8/800: dwrite d0/d29/da7/d26/f5b [0,4194304] 0 2024-02-16T22:22:07.186 INFO:tasks.workunit.client.1.smithi123.stdout:9/866: creat d4/d36/d8e/dff/f119 x:0 0 0 2024-02-16T22:22:07.188 INFO:tasks.workunit.client.1.smithi123.stdout:7/758: dwrite d3/f86 [4194304,4194304] 0 2024-02-16T22:22:07.188 INFO:tasks.workunit.client.1.smithi123.stdout:5/745: truncate d6/f21 4017132 0 2024-02-16T22:22:07.189 INFO:tasks.workunit.client.1.smithi123.stdout:9/867: chown d4/d36/daa/d20/f6a 21860714 1 2024-02-16T22:22:07.192 INFO:tasks.workunit.client.0.smithi114.stdout:3/613: dwrite d4/dd/d33/d72/f9b [0,4194304] 0 2024-02-16T22:22:07.195 INFO:tasks.workunit.client.0.smithi114.stdout:3/614: write d4/d24/d2d/d45/d68/fb9 [625571,101597] 0 2024-02-16T22:22:07.196 INFO:tasks.workunit.client.1.smithi123.stdout:0/668: read d1/d8/d13/d1d/d62/fcc [249573,93232] 0 2024-02-16T22:22:07.199 INFO:tasks.workunit.client.1.smithi123.stdout:2/750: dwrite d4/d1a/d1d/d59/dd7/ddc/f6b [0,4194304] 0 2024-02-16T22:22:07.202 INFO:tasks.workunit.client.1.smithi123.stdout:6/765: rename d0/d9/l87 to d0/d8a/d100/dfc/l117 0 2024-02-16T22:22:07.203 INFO:tasks.workunit.client.0.smithi114.stdout:4/727: dwrite dd/d1b/d41/f62 [0,4194304] 0 2024-02-16T22:22:07.203 INFO:tasks.workunit.client.0.smithi114.stdout:5/582: creat d0/d28/d98/d25/fc0 x:0 0 0 2024-02-16T22:22:07.204 INFO:tasks.workunit.client.1.smithi123.stdout:4/828: truncate d5/d1c/f24 1867954 0 2024-02-16T22:22:07.205 INFO:tasks.workunit.client.1.smithi123.stdout:4/829: chown d5/db8 1145 1 2024-02-16T22:22:07.208 INFO:tasks.workunit.client.0.smithi114.stdout:7/675: rename d3/d22/d76/l7d to d3/d87/lf2 0 2024-02-16T22:22:07.209 INFO:tasks.workunit.client.0.smithi114.stdout:6/631: getdents d6/d2c 0 2024-02-16T22:22:07.212 INFO:tasks.workunit.client.1.smithi123.stdout:4/830: dwrite d5/d1c/d23/d74/f5f [4194304,4194304] 0 2024-02-16T22:22:07.214 INFO:tasks.workunit.client.1.smithi123.stdout:3/779: getdents d0/d20/d29/d9b/dbd/df4/d5d/d60/dc6/df0 0 2024-02-16T22:22:07.215 INFO:tasks.workunit.client.0.smithi114.stdout:3/615: truncate d4/d26/f31 214788 0 2024-02-16T22:22:07.219 INFO:tasks.workunit.client.1.smithi123.stdout:7/759: mknod d3/d5/d6c/cf3 0 2024-02-16T22:22:07.220 INFO:tasks.workunit.client.0.smithi114.stdout:8/575: mkdir d4/d24/d2c/d67/dcf 0 2024-02-16T22:22:07.221 INFO:tasks.workunit.client.1.smithi123.stdout:9/868: creat d4/d36/d8e/d58/d59/d60/d9d/de4/f11a x:0 0 0 2024-02-16T22:22:07.223 INFO:tasks.workunit.client.0.smithi114.stdout:5/583: fsync d0/d28/d98/d25/d2e/f7b 0 2024-02-16T22:22:07.224 INFO:tasks.workunit.client.0.smithi114.stdout:5/584: readlink d0/d28/d98/d25/d3e/l84 0 2024-02-16T22:22:07.225 INFO:tasks.workunit.client.0.smithi114.stdout:5/585: write d0/d28/d98/d1e/f29 [2316735,123438] 0 2024-02-16T22:22:07.228 INFO:tasks.workunit.client.0.smithi114.stdout:8/576: symlink d4/d11/d78/d7a/d31/ld0 0 2024-02-16T22:22:07.228 INFO:tasks.workunit.client.1.smithi123.stdout:0/669: dwrite d1/d8/d13/d1d/f97 [0,4194304] 0 2024-02-16T22:22:07.229 INFO:tasks.workunit.client.1.smithi123.stdout:0/670: fdatasync d1/d8/d3d/f59 0 2024-02-16T22:22:07.230 INFO:tasks.workunit.client.0.smithi114.stdout:0/570: truncate d1/d16/d18/f4b 858188 0 2024-02-16T22:22:07.231 INFO:tasks.workunit.client.0.smithi114.stdout:0/571: truncate d1/f7 3440317 0 2024-02-16T22:22:07.232 INFO:tasks.workunit.client.1.smithi123.stdout:0/671: fdatasync d1/d8/d13/d1d/d64/dbc/f92 0 2024-02-16T22:22:07.232 INFO:tasks.workunit.client.1.smithi123.stdout:6/766: truncate d0/d96/de3/f2d 1440556 0 2024-02-16T22:22:07.234 INFO:tasks.workunit.client.1.smithi123.stdout:6/767: write d0/d6/d1a/d2e/f3c [393472,8644] 0 2024-02-16T22:22:07.238 INFO:tasks.workunit.client.0.smithi114.stdout:9/616: getdents d2/dd 0 2024-02-16T22:22:07.241 INFO:tasks.workunit.client.0.smithi114.stdout:5/586: creat d0/d28/d98/d25/d2e/d3b/fc1 x:0 0 0 2024-02-16T22:22:07.242 INFO:tasks.workunit.client.0.smithi114.stdout:4/728: dread dd/d2e/d3f/f50 [0,4194304] 0 2024-02-16T22:22:07.245 INFO:tasks.workunit.client.1.smithi123.stdout:2/751: creat d4/d1a/d1d/d38/d5e/dec/df1/db1/de3/ff4 x:0 0 0 2024-02-16T22:22:07.247 INFO:tasks.workunit.client.0.smithi114.stdout:3/616: link d4/dd/d33/d72/fbf d4/d24/fd4 0 2024-02-16T22:22:07.247 INFO:tasks.workunit.client.0.smithi114.stdout:3/617: fsync d4/d26/f42 0 2024-02-16T22:22:07.248 INFO:tasks.workunit.client.0.smithi114.stdout:8/577: mkdir d4/d24/d2c/d5f/dd1 0 2024-02-16T22:22:07.248 INFO:tasks.workunit.client.1.smithi123.stdout:2/752: dread d4/d1a/d1d/d59/dd7/ddc/fa8 [0,4194304] 0 2024-02-16T22:22:07.249 INFO:tasks.workunit.client.0.smithi114.stdout:8/578: fsync d4/d24/f7d 0 2024-02-16T22:22:07.250 INFO:tasks.workunit.client.1.smithi123.stdout:2/753: chown d4/d1a/d1d/d13/d15/c7c 951 1 2024-02-16T22:22:07.250 INFO:tasks.workunit.client.0.smithi114.stdout:5/587: fsync d0/d28/d98/d1e/d63/f68 0 2024-02-16T22:22:07.250 INFO:tasks.workunit.client.1.smithi123.stdout:2/754: truncate d4/d1a/d1d/fe9 529820 0 2024-02-16T22:22:07.258 INFO:tasks.workunit.client.0.smithi114.stdout:5/588: dwrite d0/d28/d98/d25/d3e/d7f/fa8 [0,4194304] 0 2024-02-16T22:22:07.258 INFO:tasks.workunit.client.0.smithi114.stdout:4/729: creat dd/d1b/db0/f109 x:0 0 0 2024-02-16T22:22:07.260 INFO:tasks.workunit.client.0.smithi114.stdout:7/676: getdents d3/d87/dd1 0 2024-02-16T22:22:07.266 INFO:tasks.workunit.client.1.smithi123.stdout:5/746: creat d6/f109 x:0 0 0 2024-02-16T22:22:07.267 INFO:tasks.workunit.client.0.smithi114.stdout:3/618: creat d4/d46/d48/d75/fd5 x:0 0 0 2024-02-16T22:22:07.271 INFO:tasks.workunit.client.0.smithi114.stdout:8/579: symlink d4/d16/d25/d39/ld2 0 2024-02-16T22:22:07.271 INFO:tasks.workunit.client.0.smithi114.stdout:3/619: dread d4/dd/d33/d72/f9b [0,4194304] 0 2024-02-16T22:22:07.272 INFO:tasks.workunit.client.1.smithi123.stdout:2/755: symlink d4/d1a/d1d/d38/d5e/dec/lf5 0 2024-02-16T22:22:07.276 INFO:tasks.workunit.client.1.smithi123.stdout:0/672: rename d1/d8/d13/d1d/l5f to d1/d4/d17/d54/d8e/le5 0 2024-02-16T22:22:07.278 INFO:tasks.workunit.client.1.smithi123.stdout:1/747: dwrite d3/d6/db/f28 [0,4194304] 0 2024-02-16T22:22:07.280 INFO:tasks.workunit.client.0.smithi114.stdout:4/730: dread - dd/d1b/d37/d38/f96 zero size 2024-02-16T22:22:07.281 INFO:tasks.workunit.client.1.smithi123.stdout:4/831: write d5/d49/f91 [1605515,113427] 0 2024-02-16T22:22:07.282 INFO:tasks.workunit.client.1.smithi123.stdout:8/801: link d0/df/dca/le7 d0/d29/da7/d26/d68/dbe/dbf/l103 0 2024-02-16T22:22:07.282 INFO:tasks.workunit.client.1.smithi123.stdout:2/756: dwrite d4/d1a/d1d/d13/d25/d98/f76 [0,4194304] 0 2024-02-16T22:22:07.283 INFO:tasks.workunit.client.1.smithi123.stdout:4/832: stat d5/d1c/d9a/cbe 0 2024-02-16T22:22:07.283 INFO:tasks.workunit.client.1.smithi123.stdout:8/802: dread - d0/d29/da7/d26/d2a/d77/fde zero size 2024-02-16T22:22:07.284 INFO:tasks.workunit.client.1.smithi123.stdout:4/833: write d5/d1c/d23/d47/da2/daa/fc4 [667737,56925] 0 2024-02-16T22:22:07.285 INFO:tasks.workunit.client.1.smithi123.stdout:3/780: creat d0/d20/d29/d9b/dbd/df4/d5d/f109 x:0 0 0 2024-02-16T22:22:07.287 INFO:tasks.workunit.client.1.smithi123.stdout:9/869: write d4/d36/d8e/d58/f5a [2329306,103006] 0 2024-02-16T22:22:07.287 INFO:tasks.workunit.client.1.smithi123.stdout:5/747: mknod d6/dd/df/d58/d56/c10a 0 2024-02-16T22:22:07.289 INFO:tasks.workunit.client.1.smithi123.stdout:9/870: readlink d4/d36/d57/l5e 0 2024-02-16T22:22:07.292 INFO:tasks.workunit.client.0.smithi114.stdout:8/580: dwrite d4/f8 [0,4194304] 0 2024-02-16T22:22:07.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:06 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:22:07.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:06 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:22:07.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:06 smithi123 ceph-mon[37161]: pgmap v31: 65 pgs: 65 active+clean; 2.4 GiB data, 6.8 GiB used, 530 GiB / 536 GiB avail; 81 MiB/s rd, 193 MiB/s wr, 408 op/s 2024-02-16T22:22:07.469 INFO:tasks.workunit.client.0.smithi114.stdout:5/589: rename d0/c4 to d0/df/d5e/cc2 0 2024-02-16T22:22:07.470 INFO:tasks.workunit.client.0.smithi114.stdout:9/617: dwrite d2/d1a/d23/d32/d46/fb6 [0,4194304] 0 2024-02-16T22:22:07.472 INFO:tasks.workunit.client.0.smithi114.stdout:9/618: dread - d2/dd/d3c/d89/d9e/fab zero size 2024-02-16T22:22:07.476 INFO:tasks.workunit.client.0.smithi114.stdout:0/572: truncate d1/fe 571288 0 2024-02-16T22:22:07.478 INFO:tasks.workunit.client.1.smithi123.stdout:0/673: creat d1/d4/d17/d54/d8e/fe6 x:0 0 0 2024-02-16T22:22:07.482 INFO:tasks.workunit.client.1.smithi123.stdout:1/748: unlink d3/d6/db/d10/d2d/ce2 0 2024-02-16T22:22:07.487 INFO:tasks.workunit.client.0.smithi114.stdout:9/619: fsync d2/d1a/d23/d35/d1d/f3a 0 2024-02-16T22:22:07.494 INFO:tasks.workunit.client.1.smithi123.stdout:5/748: mkdir d6/dd/df/d58/d89/d98/d10b 0 2024-02-16T22:22:07.494 INFO:tasks.workunit.client.1.smithi123.stdout:5/749: stat d6/dd/d78/da5/lff 0 2024-02-16T22:22:07.497 INFO:tasks.workunit.client.1.smithi123.stdout:6/768: write d0/d9/dc/d39/fa9 [800752,8206] 0 2024-02-16T22:22:07.497 INFO:tasks.workunit.client.0.smithi114.stdout:5/590: symlink d0/d28/d98/d25/d2e/d3b/db5/lc3 0 2024-02-16T22:22:07.497 INFO:tasks.workunit.client.1.smithi123.stdout:6/769: stat d0/d6/d1a/d2e/c9c 0 2024-02-16T22:22:07.498 INFO:tasks.workunit.client.1.smithi123.stdout:9/871: mkdir d4/d36/d8e/d58/d59/d60/deb/da7/dde/dfd/d11b 0 2024-02-16T22:22:07.501 INFO:tasks.workunit.client.1.smithi123.stdout:9/872: dread d4/d1e/fe7 [0,4194304] 0 2024-02-16T22:22:07.501 INFO:tasks.workunit.client.1.smithi123.stdout:7/760: getdents d3 0 2024-02-16T22:22:07.502 INFO:tasks.workunit.client.0.smithi114.stdout:7/677: getdents d3/d22/d73/da7/d6f/d9c 0 2024-02-16T22:22:07.503 INFO:tasks.workunit.client.0.smithi114.stdout:3/620: dwrite d4/d24/d2d/d45/d74/dc7/fcd [4194304,4194304] 0 2024-02-16T22:22:07.505 INFO:tasks.workunit.client.0.smithi114.stdout:7/678: dread - d3/d22/d73/da7/d6f/fd6 zero size 2024-02-16T22:22:07.507 INFO:tasks.workunit.client.1.smithi123.stdout:1/749: creat d3/d6/d49/d48/d84/d92/ffe x:0 0 0 2024-02-16T22:22:07.508 INFO:tasks.workunit.client.1.smithi123.stdout:1/750: fsync d3/d6/db/d10/d2a/ff8 0 2024-02-16T22:22:07.510 INFO:tasks.workunit.client.0.smithi114.stdout:9/620: mknod d2/d5c/d5f/d91/cc7 0 2024-02-16T22:22:07.511 INFO:tasks.workunit.client.1.smithi123.stdout:2/757: mkdir d4/dae/df6 0 2024-02-16T22:22:07.514 INFO:tasks.workunit.client.0.smithi114.stdout:5/591: dread d0/d28/d98/d25/d2e/d3b/f85 [0,4194304] 0 2024-02-16T22:22:07.515 INFO:tasks.workunit.client.0.smithi114.stdout:0/573: creat d1/d1a/d47/d62/d7f/d9d/fad x:0 0 0 2024-02-16T22:22:07.518 INFO:tasks.workunit.client.1.smithi123.stdout:5/750: creat d6/dc1/f10c x:0 0 0 2024-02-16T22:22:07.520 INFO:tasks.workunit.client.1.smithi123.stdout:8/803: write d0/d12/fcc [1020069,51685] 0 2024-02-16T22:22:07.521 INFO:tasks.workunit.client.1.smithi123.stdout:3/781: write d0/d20/d29/d9b/dbd/df5/fb4 [668690,84799] 0 2024-02-16T22:22:07.523 INFO:tasks.workunit.client.1.smithi123.stdout:3/782: write d0/d20/d29/d9b/dbd/df4/d5d/d60/dd6/f9f [1074115,81684] 0 2024-02-16T22:22:07.527 INFO:tasks.workunit.client.1.smithi123.stdout:6/770: dread - d0/d96/de3/fc1 zero size 2024-02-16T22:22:07.540 INFO:tasks.workunit.client.1.smithi123.stdout:4/834: truncate d5/d1c/d23/d31/fd3 1436684 0 2024-02-16T22:22:07.541 INFO:tasks.workunit.client.1.smithi123.stdout:4/835: fsync d5/d1c/d23/d74/f5f 0 2024-02-16T22:22:07.541 INFO:tasks.workunit.client.1.smithi123.stdout:4/836: chown d5/d1c/d23/d56 23078 1 2024-02-16T22:22:07.545 INFO:tasks.workunit.client.1.smithi123.stdout:6/771: symlink d0/d6/d91/dda/l118 0 2024-02-16T22:22:07.549 INFO:tasks.workunit.client.1.smithi123.stdout:9/873: mkdir d4/d36/d8e/d6d/d71/da6/d11c 0 2024-02-16T22:22:07.552 INFO:tasks.workunit.client.1.smithi123.stdout:8/804: write d0/d12/fb5 [1212951,61054] 0 2024-02-16T22:22:07.553 INFO:tasks.workunit.client.1.smithi123.stdout:2/758: symlink d4/d1a/d1d/d59/lf7 0 2024-02-16T22:22:07.554 INFO:tasks.workunit.client.1.smithi123.stdout:2/759: chown d4/d1a/d1d/l10 71519 1 2024-02-16T22:22:07.559 INFO:tasks.workunit.client.1.smithi123.stdout:4/837: mkdir d5/d1c/d23/d56/d6e/d73/d87/dfb 0 2024-02-16T22:22:07.560 INFO:tasks.workunit.client.1.smithi123.stdout:3/783: truncate d0/d20/d29/d9b/dbd/df4/d37/f64 1074052 0 2024-02-16T22:22:07.562 INFO:tasks.workunit.client.1.smithi123.stdout:3/784: write d0/d20/d29/d9b/dbd/df4/d5d/d60/dc6/fea [917921,79939] 0 2024-02-16T22:22:07.563 INFO:tasks.workunit.client.1.smithi123.stdout:6/772: creat d0/d6/d2f/dbd/f119 x:0 0 0 2024-02-16T22:22:07.569 INFO:tasks.workunit.client.1.smithi123.stdout:8/805: truncate d0/d12/f1e 3363731 0 2024-02-16T22:22:07.571 INFO:tasks.workunit.client.1.smithi123.stdout:4/838: dread d5/d1c/d23/d56/d6e/fa5 [0,4194304] 0 2024-02-16T22:22:07.571 INFO:tasks.workunit.client.1.smithi123.stdout:2/760: creat d4/d1a/d1d/d38/d7b/ff8 x:0 0 0 2024-02-16T22:22:07.577 INFO:tasks.workunit.client.1.smithi123.stdout:3/785: stat d0/c11 0 2024-02-16T22:22:07.580 INFO:tasks.workunit.client.1.smithi123.stdout:7/761: getdents d3/d6e/d4d/d92 0 2024-02-16T22:22:07.583 INFO:tasks.workunit.client.1.smithi123.stdout:9/874: mkdir d4/d36/daa/d7f/dea/d11d 0 2024-02-16T22:22:07.583 INFO:tasks.workunit.client.1.smithi123.stdout:0/674: sync 2024-02-16T22:22:07.584 INFO:tasks.workunit.client.1.smithi123.stdout:0/675: chown d1/d8/d3d/f59 114 1 2024-02-16T22:22:07.584 INFO:tasks.workunit.client.1.smithi123.stdout:1/751: getdents d3/d6/db/d10/d2d/d8c/dfd/d9f/db4/ddf/df0 0 2024-02-16T22:22:07.591 INFO:tasks.workunit.client.1.smithi123.stdout:4/839: chown d5/d1c/d23/d74/d92/d9b/fc2 539198 1 2024-02-16T22:22:07.593 INFO:tasks.workunit.client.1.smithi123.stdout:5/751: link d6/dd/df/f6c d6/dd/df/d1f/f10d 0 2024-02-16T22:22:07.595 INFO:tasks.workunit.client.1.smithi123.stdout:3/786: dread d0/d20/d29/d9b/dbd/df4/d5d/fd7 [0,4194304] 0 2024-02-16T22:22:07.595 INFO:tasks.workunit.client.0.smithi114.stdout:8/581: rename d4/d11/d78/d7a/d36/cc9 to d4/d11/d78/d7a/cd3 0 2024-02-16T22:22:07.597 INFO:tasks.workunit.client.0.smithi114.stdout:3/621: mknod d4/d46/d48/d80/dbd/cd6 0 2024-02-16T22:22:07.599 INFO:tasks.workunit.client.0.smithi114.stdout:7/679: creat d3/d3d/ff3 x:0 0 0 2024-02-16T22:22:07.601 INFO:tasks.workunit.client.0.smithi114.stdout:9/621: stat d2/d1a/d23/d35/d47/d63/d98 0 2024-02-16T22:22:07.601 INFO:tasks.workunit.client.1.smithi123.stdout:6/773: truncate d0/d6/f5c 258331 0 2024-02-16T22:22:07.602 INFO:tasks.workunit.client.0.smithi114.stdout:5/592: mkdir d0/df/d7a/dc4 0 2024-02-16T22:22:07.603 INFO:tasks.workunit.client.1.smithi123.stdout:1/752: dread d3/d6/d18/d9e/fae [4194304,4194304] 0 2024-02-16T22:22:07.606 INFO:tasks.workunit.client.0.smithi114.stdout:8/582: dwrite d4/d11/d78/d7a/d31/fa5 [0,4194304] 0 2024-02-16T22:22:07.606 INFO:tasks.workunit.client.0.smithi114.stdout:0/574: chown d1/c10 14774 1 2024-02-16T22:22:07.607 INFO:tasks.workunit.client.1.smithi123.stdout:9/875: creat d4/d36/d8e/d58/d59/d60/deb/da7/f11e x:0 0 0 2024-02-16T22:22:07.608 INFO:tasks.workunit.client.1.smithi123.stdout:2/761: dread d4/f23 [0,4194304] 0 2024-02-16T22:22:07.609 INFO:tasks.workunit.client.0.smithi114.stdout:8/583: chown d4/d16/d25/d7c/l88 7 1 2024-02-16T22:22:07.609 INFO:tasks.workunit.client.1.smithi123.stdout:0/676: rmdir d1/d8/d3d 39 2024-02-16T22:22:07.611 INFO:tasks.workunit.client.0.smithi114.stdout:3/622: mkdir d4/d26/dd7 0 2024-02-16T22:22:07.611 INFO:tasks.workunit.client.0.smithi114.stdout:9/622: dwrite d2/d5c/d5f/d72/fb8 [0,4194304] 0 2024-02-16T22:22:07.615 INFO:tasks.workunit.client.1.smithi123.stdout:8/806: fdatasync d0/d29/da7/d26/d2a/d77/fe8 0 2024-02-16T22:22:07.616 INFO:tasks.workunit.client.1.smithi123.stdout:8/807: chown d0/d61/d66/d7e/c90 286277 1 2024-02-16T22:22:07.619 INFO:tasks.workunit.client.0.smithi114.stdout:3/623: dwrite d4/d46/d48/f66 [0,4194304] 0 2024-02-16T22:22:07.619 INFO:tasks.workunit.client.1.smithi123.stdout:4/840: mknod d5/d1c/d23/d56/d6e/d9f/da1/dc0/cfc 0 2024-02-16T22:22:07.622 INFO:tasks.workunit.client.0.smithi114.stdout:3/624: write d4/d24/d2d/f4e [164088,4414] 0 2024-02-16T22:22:07.626 INFO:tasks.workunit.client.1.smithi123.stdout:3/787: creat d0/d20/d29/d9b/dbd/df4/d5d/dae/dc7/f10a x:0 0 0 2024-02-16T22:22:07.627 INFO:tasks.workunit.client.1.smithi123.stdout:3/788: stat d0/d20/d29/d9b/dbd/df4/f35 0 2024-02-16T22:22:07.629 INFO:tasks.workunit.client.0.smithi114.stdout:0/575: creat d1/d16/d1f/d28/d51/d61/d6d/fae x:0 0 0 2024-02-16T22:22:07.630 INFO:tasks.workunit.client.1.smithi123.stdout:7/762: write d3/d6e/d24/f56 [778293,69611] 0 2024-02-16T22:22:07.631 INFO:tasks.workunit.client.1.smithi123.stdout:7/763: chown d3/d6e/d52/f88 24 1 2024-02-16T22:22:07.636 INFO:tasks.workunit.client.0.smithi114.stdout:7/680: creat d3/d22/db7/dea/ff4 x:0 0 0 2024-02-16T22:22:07.637 INFO:tasks.workunit.client.0.smithi114.stdout:5/593: write d0/fa [1165409,8625] 0 2024-02-16T22:22:07.639 INFO:tasks.workunit.client.1.smithi123.stdout:0/677: dread - d1/d8/d13/d1d/f6f zero size 2024-02-16T22:22:07.640 INFO:tasks.workunit.client.0.smithi114.stdout:3/625: write d4/d24/d93/fb8 [144718,129943] 0 2024-02-16T22:22:07.643 INFO:tasks.workunit.client.1.smithi123.stdout:4/841: mknod d5/d1c/d64/d7b/cfd 0 2024-02-16T22:22:07.646 INFO:tasks.workunit.client.1.smithi123.stdout:6/774: write d0/d96/de3/fc1 [595644,88920] 0 2024-02-16T22:22:07.646 INFO:tasks.workunit.client.1.smithi123.stdout:6/775: read - d0/d96/de3/dcc/fde zero size 2024-02-16T22:22:07.649 INFO:tasks.workunit.client.1.smithi123.stdout:7/764: chown d3/d87/ldd 1 1 2024-02-16T22:22:07.650 INFO:tasks.workunit.client.0.smithi114.stdout:8/584: dwrite d4/d11/f20 [0,4194304] 0 2024-02-16T22:22:07.652 INFO:tasks.workunit.client.0.smithi114.stdout:9/623: creat d2/d5c/fc8 x:0 0 0 2024-02-16T22:22:07.652 INFO:tasks.workunit.client.1.smithi123.stdout:9/876: symlink d4/d36/d8e/l11f 0 2024-02-16T22:22:07.652 INFO:tasks.workunit.client.1.smithi123.stdout:9/877: write d4/d1e/f42 [8925053,52519] 0 2024-02-16T22:22:07.654 INFO:tasks.workunit.client.0.smithi114.stdout:5/594: symlink d0/d28/d98/lc5 0 2024-02-16T22:22:07.655 INFO:tasks.workunit.client.1.smithi123.stdout:0/678: dread d1/d4/f43 [0,4194304] 0 2024-02-16T22:22:07.656 INFO:tasks.workunit.client.1.smithi123.stdout:0/679: write d1/d8/d13/d1d/f97 [852617,99937] 0 2024-02-16T22:22:07.659 INFO:tasks.workunit.client.0.smithi114.stdout:0/576: link d1/d1a/f2e d1/d1a/d3e/faf 0 2024-02-16T22:22:07.663 INFO:tasks.workunit.client.1.smithi123.stdout:6/776: fsync d0/d6/d91/d93/fae 0 2024-02-16T22:22:07.664 INFO:tasks.workunit.client.0.smithi114.stdout:9/624: truncate d2/f12 4034876 0 2024-02-16T22:22:07.664 INFO:tasks.workunit.client.1.smithi123.stdout:0/680: dwrite d1/f6a [0,4194304] 0 2024-02-16T22:22:07.664 INFO:tasks.workunit.client.1.smithi123.stdout:4/842: dread d5/d1c/d23/d74/f5f [0,4194304] 0 2024-02-16T22:22:07.665 INFO:tasks.workunit.client.1.smithi123.stdout:6/777: fdatasync d0/d9/dc/d3a/f108 0 2024-02-16T22:22:07.667 INFO:tasks.workunit.client.0.smithi114.stdout:5/595: creat d0/d28/d98/d1e/d63/fc6 x:0 0 0 2024-02-16T22:22:07.668 INFO:tasks.workunit.client.0.smithi114.stdout:5/596: read - d0/d28/d98/d25/d2e/f7b zero size 2024-02-16T22:22:07.669 INFO:tasks.workunit.client.0.smithi114.stdout:5/597: write d0/d28/d98/d25/d3e/f44 [1572647,60458] 0 2024-02-16T22:22:07.670 INFO:tasks.workunit.client.1.smithi123.stdout:0/681: dwrite d1/f12 [0,4194304] 0 2024-02-16T22:22:07.672 INFO:tasks.workunit.client.1.smithi123.stdout:2/762: write d4/d1a/d1d/d13/d25/f66 [1542834,38406] 0 2024-02-16T22:22:07.675 INFO:tasks.workunit.client.0.smithi114.stdout:3/626: rename d4/dd/d33/fa9 to d4/fd8 0 2024-02-16T22:22:07.675 INFO:tasks.workunit.client.0.smithi114.stdout:3/627: chown d4/dd/d33/d8f/fa4 515395 1 2024-02-16T22:22:07.677 INFO:tasks.workunit.client.1.smithi123.stdout:2/763: dread d4/f23 [0,4194304] 0 2024-02-16T22:22:07.686 INFO:tasks.workunit.client.0.smithi114.stdout:5/598: mkdir d0/df/d7a/dc7 0 2024-02-16T22:22:07.686 INFO:tasks.workunit.client.1.smithi123.stdout:5/752: getdents d6/dd/df/d1f/ddc/dee 0 2024-02-16T22:22:07.687 INFO:tasks.workunit.client.1.smithi123.stdout:3/789: creat d0/f10b x:0 0 0 2024-02-16T22:22:07.687 INFO:tasks.workunit.client.0.smithi114.stdout:0/577: dwrite d1/d16/f34 [0,4194304] 0 2024-02-16T22:22:07.693 INFO:tasks.workunit.client.0.smithi114.stdout:3/628: symlink d4/d46/d48/dcc/ld9 0 2024-02-16T22:22:07.698 INFO:tasks.workunit.client.0.smithi114.stdout:8/585: dwrite d4/d24/d2c/fbe [0,4194304] 0 2024-02-16T22:22:07.699 INFO:tasks.workunit.client.1.smithi123.stdout:7/765: dread d3/d6e/d1d/db2/d3e/f5b [0,4194304] 0 2024-02-16T22:22:07.703 INFO:tasks.workunit.client.0.smithi114.stdout:9/625: dwrite d2/dd/d3c/f3d [0,4194304] 0 2024-02-16T22:22:07.708 INFO:tasks.workunit.client.1.smithi123.stdout:0/682: dread d1/d4/d17/d74/d57/f75 [0,4194304] 0 2024-02-16T22:22:07.708 INFO:tasks.workunit.client.1.smithi123.stdout:2/764: creat d4/d1a/d1d/d59/dd7/ddc/ff9 x:0 0 0 2024-02-16T22:22:07.708 INFO:tasks.workunit.client.0.smithi114.stdout:5/599: rename d0/d28/d98/d1e/d63/d96/d9a/db8/f1a to d0/d28/d98/d1e/d63/fc8 0 2024-02-16T22:22:07.710 INFO:tasks.workunit.client.0.smithi114.stdout:0/578: creat d1/d16/d1f/d28/fb0 x:0 0 0 2024-02-16T22:22:07.718 INFO:tasks.workunit.client.0.smithi114.stdout:3/629: symlink d4/d7e/lda 0 2024-02-16T22:22:07.724 INFO:tasks.workunit.client.1.smithi123.stdout:9/878: dwrite d4/d36/daa/d20/ff7 [0,4194304] 0 2024-02-16T22:22:07.730 INFO:tasks.workunit.client.1.smithi123.stdout:1/753: sync 2024-02-16T22:22:07.735 INFO:tasks.workunit.client.0.smithi114.stdout:8/586: creat d4/d24/d2c/d67/dcf/fd4 x:0 0 0 2024-02-16T22:22:07.738 INFO:tasks.workunit.client.0.smithi114.stdout:9/626: rename d2/d1a/d23/d35 to d2/d5c/dc9 0 2024-02-16T22:22:07.740 INFO:tasks.workunit.client.0.smithi114.stdout:8/587: fdatasync d4/d24/f50 0 2024-02-16T22:22:07.744 INFO:tasks.workunit.client.0.smithi114.stdout:0/579: rename d1/d16/d55/d92 to d1/d1a/d47/d62/d7f/d8c/db1 0 2024-02-16T22:22:07.745 INFO:tasks.workunit.client.0.smithi114.stdout:8/588: dread d4/d11/d78/d7a/f1a [0,4194304] 0 2024-02-16T22:22:07.746 INFO:tasks.workunit.client.0.smithi114.stdout:8/589: stat d4/d11/d78/d7a/d7e/lc3 0 2024-02-16T22:22:07.746 INFO:tasks.workunit.client.0.smithi114.stdout:9/627: chown d2/d5c/f8d 588856 1 2024-02-16T22:22:07.752 INFO:tasks.workunit.client.0.smithi114.stdout:5/600: dwrite d0/d28/d98/d1e/d86/f7c [0,4194304] 0 2024-02-16T22:22:07.754 INFO:tasks.workunit.client.0.smithi114.stdout:0/580: truncate d1/fa 966218 0 2024-02-16T22:22:07.764 INFO:tasks.workunit.client.0.smithi114.stdout:3/630: rename d4/d24/d2d/d45/d74/c7a to d4/d24/d93/cdb 0 2024-02-16T22:22:07.765 INFO:tasks.workunit.client.0.smithi114.stdout:3/631: chown d4/d46/d48/f8e 27803 1 2024-02-16T22:22:07.767 INFO:tasks.workunit.client.0.smithi114.stdout:0/581: creat d1/d1a/d47/fb2 x:0 0 0 2024-02-16T22:22:07.769 INFO:tasks.workunit.client.0.smithi114.stdout:8/590: rename d4/d11/d78/d7a/f3a to d4/d24/d2c/d5f/dd1/fd5 0 2024-02-16T22:22:07.778 INFO:tasks.workunit.client.0.smithi114.stdout:0/582: dread d1/fe [0,4194304] 0 2024-02-16T22:22:07.778 INFO:tasks.workunit.client.0.smithi114.stdout:8/591: creat d4/d16/d25/d46/db5/fd6 x:0 0 0 2024-02-16T22:22:07.779 INFO:tasks.workunit.client.0.smithi114.stdout:9/628: dread d2/d5c/dc9/d47/d63/fa7 [0,4194304] 0 2024-02-16T22:22:07.783 INFO:tasks.workunit.client.1.smithi123.stdout:8/808: sync 2024-02-16T22:22:07.784 INFO:tasks.workunit.client.0.smithi114.stdout:0/583: unlink d1/f81 0 2024-02-16T22:22:07.785 INFO:tasks.workunit.client.0.smithi114.stdout:9/629: dwrite d2/d5c/dc9/d47/d63/f73 [4194304,4194304] 0 2024-02-16T22:22:07.790 INFO:tasks.workunit.client.0.smithi114.stdout:3/632: link d4/d43/c78 d4/d46/d48/d80/dbd/cdc 0 2024-02-16T22:22:07.792 INFO:tasks.workunit.client.0.smithi114.stdout:3/633: write d4/d46/d48/d75/fd5 [418005,54329] 0 2024-02-16T22:22:07.794 INFO:tasks.workunit.client.0.smithi114.stdout:0/584: unlink d1/d1a/c23 0 2024-02-16T22:22:07.797 INFO:tasks.workunit.client.0.smithi114.stdout:9/630: dread - d2/d5c/dc9/f86 zero size 2024-02-16T22:22:07.800 INFO:tasks.workunit.client.0.smithi114.stdout:3/634: dwrite d4/d24/d93/fb8 [0,4194304] 0 2024-02-16T22:22:07.802 INFO:tasks.workunit.client.0.smithi114.stdout:3/635: dread - d4/d46/d48/d75/daa/fd0 zero size 2024-02-16T22:22:07.804 INFO:tasks.workunit.client.0.smithi114.stdout:0/585: dwrite d1/d1a/d47/d62/d7f/f84 [4194304,4194304] 0 2024-02-16T22:22:07.804 INFO:tasks.workunit.client.0.smithi114.stdout:9/631: creat d2/d33/d3f/fca x:0 0 0 2024-02-16T22:22:07.808 INFO:tasks.workunit.client.0.smithi114.stdout:3/636: rmdir d4/d24/d2d/d45/d68/d77 39 2024-02-16T22:22:07.810 INFO:tasks.workunit.client.0.smithi114.stdout:0/586: creat d1/d16/d1f/d28/d51/fb3 x:0 0 0 2024-02-16T22:22:07.813 INFO:tasks.workunit.client.0.smithi114.stdout:9/632: creat d2/d5c/dc9/d47/d63/d98/fcb x:0 0 0 2024-02-16T22:22:07.818 INFO:tasks.workunit.client.1.smithi123.stdout:4/843: sync 2024-02-16T22:22:07.818 INFO:tasks.workunit.client.1.smithi123.stdout:7/766: sync 2024-02-16T22:22:07.819 INFO:tasks.workunit.client.1.smithi123.stdout:3/790: read d0/f55 [3981527,8410] 0 2024-02-16T22:22:07.821 INFO:tasks.workunit.client.0.smithi114.stdout:9/633: unlink d2/d5c/d5f/d72/f79 0 2024-02-16T22:22:07.823 INFO:tasks.workunit.client.0.smithi114.stdout:9/634: write d2/dd/d3c/f7f [562862,84773] 0 2024-02-16T22:22:07.832 INFO:tasks.workunit.client.1.smithi123.stdout:2/765: rename d4/d1a/d1d/d38/d5e/dec/df1/d4a/la3 to d4/dae/df6/lfa 0 2024-02-16T22:22:07.832 INFO:tasks.workunit.client.0.smithi114.stdout:8/592: truncate d4/d16/fbf 1959243 0 2024-02-16T22:22:07.835 INFO:tasks.workunit.client.0.smithi114.stdout:9/635: creat d2/dd/fcc x:0 0 0 2024-02-16T22:22:07.837 INFO:tasks.workunit.client.1.smithi123.stdout:1/754: mknod d3/d6/d49/d48/d84/d92/cff 0 2024-02-16T22:22:07.838 INFO:tasks.workunit.client.1.smithi123.stdout:8/809: chown d0/f4 11491629 1 2024-02-16T22:22:07.838 INFO:tasks.workunit.client.1.smithi123.stdout:8/810: fsync d0/f6 0 2024-02-16T22:22:07.840 INFO:tasks.workunit.client.0.smithi114.stdout:8/593: rename d4/d16/d25/d46/c94 to d4/d16/d25/d46/cd7 0 2024-02-16T22:22:07.840 INFO:tasks.workunit.client.0.smithi114.stdout:9/636: dwrite d2/d1a/d23/fbf [0,4194304] 0 2024-02-16T22:22:07.841 INFO:tasks.workunit.client.1.smithi123.stdout:8/811: dread d0/d29/da7/d43/fdf [0,4194304] 0 2024-02-16T22:22:07.845 INFO:tasks.workunit.client.1.smithi123.stdout:7/767: creat d3/d6e/d1d/db2/d35/dbe/ff4 x:0 0 0 2024-02-16T22:22:07.849 INFO:tasks.workunit.client.0.smithi114.stdout:9/637: creat d2/d1a/d23/d32/d46/d78/fcd x:0 0 0 2024-02-16T22:22:07.849 INFO:tasks.workunit.client.1.smithi123.stdout:6/778: link d0/d6/d1a/daf/c10e d0/d8a/c11a 0 2024-02-16T22:22:07.851 INFO:tasks.workunit.client.1.smithi123.stdout:6/779: truncate d0/d6/d1a/f36 8669830 0 2024-02-16T22:22:07.851 INFO:tasks.workunit.client.1.smithi123.stdout:6/780: stat d0/d6/f15 0 2024-02-16T22:22:07.852 INFO:tasks.workunit.client.0.smithi114.stdout:9/638: read d2/d1a/d65/f94 [104885,120007] 0 2024-02-16T22:22:07.854 INFO:tasks.workunit.client.1.smithi123.stdout:4/844: rename d5/d1c/d5e/le2 to d5/d1c/d23/d56/d6e/d9f/da1/dc0/dc5/lfe 0 2024-02-16T22:22:07.857 INFO:tasks.workunit.client.1.smithi123.stdout:2/766: unlink d4/d1a/d1d/d13/d25/d72/cea 0 2024-02-16T22:22:07.859 INFO:tasks.workunit.client.0.smithi114.stdout:3/637: dwrite d4/dd/d33/d72/fbf [0,4194304] 0 2024-02-16T22:22:07.860 INFO:tasks.workunit.client.1.smithi123.stdout:9/879: mknod d4/d36/daa/d7f/dea/d11d/c120 0 2024-02-16T22:22:07.861 INFO:tasks.workunit.client.1.smithi123.stdout:0/683: dwrite d1/d4/d17/d74/fc1 [0,4194304] 0 2024-02-16T22:22:07.862 INFO:tasks.workunit.client.0.smithi114.stdout:0/587: dwrite d1/d16/d18/f4b [0,4194304] 0 2024-02-16T22:22:07.865 INFO:tasks.workunit.client.1.smithi123.stdout:1/755: creat d3/d6/db/d10/d4b/d70/f100 x:0 0 0 2024-02-16T22:22:07.866 INFO:tasks.workunit.client.1.smithi123.stdout:1/756: chown d3/d6/d11/f62 232 1 2024-02-16T22:22:07.871 INFO:tasks.workunit.client.1.smithi123.stdout:8/812: mknod d0/d29/da7/d26/d68/d81/c104 0 2024-02-16T22:22:07.872 INFO:tasks.workunit.client.1.smithi123.stdout:8/813: chown d0/d29/da7/d26/d2a/d4d 62 1 2024-02-16T22:22:07.874 INFO:tasks.workunit.client.0.smithi114.stdout:0/588: creat d1/d16/d1f/fb4 x:0 0 0 2024-02-16T22:22:07.875 INFO:tasks.workunit.client.1.smithi123.stdout:5/753: dwrite d6/dd/df/d90/f9d [0,4194304] 0 2024-02-16T22:22:07.875 INFO:tasks.workunit.client.1.smithi123.stdout:7/768: mkdir d3/d87/df5 0 2024-02-16T22:22:07.877 INFO:tasks.workunit.client.0.smithi114.stdout:9/639: rmdir d2/d1a/d23/d32/d46/da1 0 2024-02-16T22:22:07.879 INFO:tasks.workunit.client.0.smithi114.stdout:9/640: write d2/d5c/dc9/d47/d63/f81 [2469486,62139] 0 2024-02-16T22:22:07.880 INFO:tasks.workunit.client.0.smithi114.stdout:0/589: stat d1/d16/d48/laa 0 2024-02-16T22:22:07.882 INFO:tasks.workunit.client.1.smithi123.stdout:4/845: symlink d5/d1c/d23/d56/d6e/d9f/da1/dc0/lff 0 2024-02-16T22:22:07.884 INFO:tasks.workunit.client.0.smithi114.stdout:8/594: dread d4/d16/f91 [0,4194304] 0 2024-02-16T22:22:07.887 INFO:tasks.workunit.client.1.smithi123.stdout:9/880: creat d4/d36/d57/f121 x:0 0 0 2024-02-16T22:22:07.889 INFO:tasks.workunit.client.1.smithi123.stdout:0/684: mkdir d1/d4/d17/d74/de7 0 2024-02-16T22:22:07.889 INFO:tasks.workunit.client.1.smithi123.stdout:9/881: chown d4/d36/daa/d2b/d6c/ca4 11 1 2024-02-16T22:22:07.890 INFO:tasks.workunit.client.0.smithi114.stdout:0/590: dwrite d1/d1a/d47/d62/d7f/d9d/fad [0,4194304] 0 2024-02-16T22:22:07.898 INFO:tasks.workunit.client.0.smithi114.stdout:0/591: fsync d1/d16/d1f/d28/d51/d60/f5e 0 2024-02-16T22:22:07.900 INFO:tasks.workunit.client.1.smithi123.stdout:7/769: creat d3/d5/d6c/ff6 x:0 0 0 2024-02-16T22:22:07.903 INFO:tasks.workunit.client.1.smithi123.stdout:5/754: creat d6/dd/df/d58/d89/dc3/f10e x:0 0 0 2024-02-16T22:22:07.905 INFO:tasks.workunit.client.1.smithi123.stdout:3/791: creat d0/d20/d29/d9b/dbd/df4/d37/f10c x:0 0 0 2024-02-16T22:22:07.906 INFO:tasks.workunit.client.1.smithi123.stdout:3/792: write d0/d20/d29/d9b/dbd/df4/d5d/dae/db0/fe7 [908295,51706] 0 2024-02-16T22:22:07.907 INFO:tasks.workunit.client.0.smithi114.stdout:0/592: dwrite d1/d16/d1f/d28/d51/d61/d6d/d79/fa4 [0,4194304] 0 2024-02-16T22:22:07.907 INFO:tasks.workunit.client.1.smithi123.stdout:6/781: fsync d0/d9/dc/f3f 0 2024-02-16T22:22:07.913 INFO:tasks.workunit.client.0.smithi114.stdout:0/593: creat d1/d1a/d47/d62/d89/fb5 x:0 0 0 2024-02-16T22:22:07.913 INFO:tasks.workunit.client.0.smithi114.stdout:3/638: write d4/d26/fc1 [76526,1690] 0 2024-02-16T22:22:07.913 INFO:tasks.workunit.client.1.smithi123.stdout:4/846: dread d5/d1c/d64/f8a [0,4194304] 0 2024-02-16T22:22:07.915 INFO:tasks.workunit.client.1.smithi123.stdout:4/847: fdatasync d5/d1c/d23/f3a 0 2024-02-16T22:22:07.916 INFO:tasks.workunit.client.1.smithi123.stdout:6/782: dwrite d0/d96/fbf [4194304,4194304] 0 2024-02-16T22:22:07.918 INFO:tasks.workunit.client.1.smithi123.stdout:6/783: chown d0/d6/d2f/l33 371 1 2024-02-16T22:22:07.920 INFO:tasks.workunit.client.1.smithi123.stdout:0/685: fsync d1/d8/d13/d1d/d64/dbc/f5c 0 2024-02-16T22:22:07.921 INFO:tasks.workunit.client.0.smithi114.stdout:3/639: dwrite d4/d46/d48/f66 [0,4194304] 0 2024-02-16T22:22:07.922 INFO:tasks.workunit.client.0.smithi114.stdout:3/640: fsync d4/d87/faf 0 2024-02-16T22:22:07.923 INFO:tasks.workunit.client.1.smithi123.stdout:0/686: dread d1/f12 [0,4194304] 0 2024-02-16T22:22:07.924 INFO:tasks.workunit.client.1.smithi123.stdout:0/687: chown d1/d4/d17/d54/d8e/da5 729673 1 2024-02-16T22:22:07.928 INFO:tasks.workunit.client.0.smithi114.stdout:8/595: write d4/d11/d52/f83 [755177,58980] 0 2024-02-16T22:22:07.930 INFO:tasks.workunit.client.0.smithi114.stdout:3/641: unlink d4/dd/d33/d8f/lb4 0 2024-02-16T22:22:07.933 INFO:tasks.workunit.client.1.smithi123.stdout:8/814: rename d0/d29/da7/d43/fc2 to d0/df/dca/f105 0 2024-02-16T22:22:07.939 INFO:tasks.workunit.client.0.smithi114.stdout:3/642: mknod d4/d46/dcf/cdd 0 2024-02-16T22:22:07.943 INFO:tasks.workunit.client.1.smithi123.stdout:2/767: truncate d4/d1a/d1d/d38/d5e/dec/df1/db1/fb7 1920089 0 2024-02-16T22:22:07.944 INFO:tasks.workunit.client.1.smithi123.stdout:2/768: dread d4/d1a/d1d/fe9 [0,4194304] 0 2024-02-16T22:22:07.945 INFO:tasks.workunit.client.1.smithi123.stdout:4/848: creat d5/d1c/d9a/dd8/f100 x:0 0 0 2024-02-16T22:22:07.946 INFO:tasks.workunit.client.1.smithi123.stdout:1/757: truncate d3/d6/d49/d48/d84/d92/dec/ff6 3805709 0 2024-02-16T22:22:07.947 INFO:tasks.workunit.client.0.smithi114.stdout:3/643: dread d4/d26/f3c [0,4194304] 0 2024-02-16T22:22:07.948 INFO:tasks.workunit.client.0.smithi114.stdout:0/594: dwrite d1/d16/d1f/d28/d51/d60/f5e [0,4194304] 0 2024-02-16T22:22:07.948 INFO:tasks.workunit.client.1.smithi123.stdout:6/784: creat d0/d9/dc/d3a/d70/de5/f11b x:0 0 0 2024-02-16T22:22:07.949 INFO:tasks.workunit.client.0.smithi114.stdout:3/644: write d4/dd/f64 [1804300,124076] 0 2024-02-16T22:22:07.949 INFO:tasks.workunit.client.1.smithi123.stdout:9/882: dwrite d4/d36/d57/db2/f28 [4194304,4194304] 0 2024-02-16T22:22:07.950 INFO:tasks.workunit.client.1.smithi123.stdout:1/758: write d3/d6/db/d10/d2a/ff8 [953212,111686] 0 2024-02-16T22:22:07.956 INFO:tasks.workunit.client.1.smithi123.stdout:2/769: dwrite d4/d1a/d1d/d38/f90 [4194304,4194304] 0 2024-02-16T22:22:07.957 INFO:tasks.workunit.client.1.smithi123.stdout:5/755: rename d6/dd/d78/da5/cf4 to d6/dd/df/d58/d56/c10f 0 2024-02-16T22:22:07.961 INFO:tasks.workunit.client.0.smithi114.stdout:0/595: creat d1/d16/d1f/d28/d51/d61/d6d/fb6 x:0 0 0 2024-02-16T22:22:07.966 INFO:tasks.workunit.client.0.smithi114.stdout:8/596: dwrite d4/d16/d25/d46/fb2 [0,4194304] 0 2024-02-16T22:22:07.971 INFO:tasks.workunit.client.1.smithi123.stdout:7/770: creat d3/d6e/d1d/db2/d3e/de4/ff7 x:0 0 0 2024-02-16T22:22:07.979 INFO:tasks.workunit.client.1.smithi123.stdout:0/688: write d1/d4/d42/f91 [771803,3036] 0 2024-02-16T22:22:07.983 INFO:tasks.workunit.client.0.smithi114.stdout:8/597: creat d4/d11/d78/fd8 x:0 0 0 2024-02-16T22:22:07.984 INFO:tasks.workunit.client.0.smithi114.stdout:8/598: write d4/d16/d25/d7c/f87 [181713,6080] 0 2024-02-16T22:22:07.985 INFO:tasks.workunit.client.0.smithi114.stdout:8/599: write d4/d24/f38 [4304517,17767] 0 2024-02-16T22:22:07.986 INFO:tasks.workunit.client.0.smithi114.stdout:3/645: creat d4/d43/fde x:0 0 0 2024-02-16T22:22:07.992 INFO:tasks.workunit.client.1.smithi123.stdout:1/759: truncate d3/d6/db/d10/f4a 8743 0 2024-02-16T22:22:07.992 INFO:tasks.workunit.client.1.smithi123.stdout:9/883: read d4/f16 [2283392,60448] 0 2024-02-16T22:22:07.993 INFO:tasks.workunit.client.0.smithi114.stdout:8/600: creat d4/d24/d2c/fd9 x:0 0 0 2024-02-16T22:22:07.998 INFO:tasks.workunit.client.1.smithi123.stdout:3/793: rename d0/d20/d29/d9b/dbd/df4/d5d/d60/c81 to d0/d40/dfb/c10d 0 2024-02-16T22:22:08.000 INFO:tasks.workunit.client.1.smithi123.stdout:8/815: creat d0/d29/da7/d26/d32/de0/f106 x:0 0 0 2024-02-16T22:22:08.000 INFO:tasks.workunit.client.1.smithi123.stdout:7/771: chown d3/d87/c21 507495 1 2024-02-16T22:22:08.006 INFO:tasks.workunit.client.1.smithi123.stdout:1/760: dread d3/f7 [0,4194304] 0 2024-02-16T22:22:08.007 INFO:tasks.workunit.client.1.smithi123.stdout:1/761: write d3/d6/db/d10/d2d/d8c/dfd/d9f/db4/ddf/ff1 [1465006,103591] 0 2024-02-16T22:22:08.009 INFO:tasks.workunit.client.1.smithi123.stdout:4/849: sync 2024-02-16T22:22:08.032 INFO:tasks.workunit.client.1.smithi123.stdout:5/756: write d6/f8 [1281682,104611] 0 2024-02-16T22:22:08.033 INFO:tasks.workunit.client.1.smithi123.stdout:5/757: dread - d6/dd/df/d76/f107 zero size 2024-02-16T22:22:08.049 INFO:tasks.workunit.client.0.smithi114.stdout:0/596: dwrite d1/fe [0,4194304] 0 2024-02-16T22:22:08.050 INFO:tasks.workunit.client.0.smithi114.stdout:3/646: read d4/dd/d33/d8f/fa4 [101772,128506] 0 2024-02-16T22:22:08.052 INFO:tasks.workunit.client.1.smithi123.stdout:2/770: fsync d4/d1a/d1d/d13/d25/f43 0 2024-02-16T22:22:08.055 INFO:tasks.workunit.client.0.smithi114.stdout:8/601: dread d4/d24/f7d [0,4194304] 0 2024-02-16T22:22:08.056 INFO:tasks.workunit.client.1.smithi123.stdout:3/794: mknod d0/d20/d2e/c10e 0 2024-02-16T22:22:08.058 INFO:tasks.workunit.client.0.smithi114.stdout:0/597: mknod d1/d16/d1f/d28/d51/d61/d7e/d90/d95/d80/cb7 0 2024-02-16T22:22:08.060 INFO:tasks.workunit.client.1.smithi123.stdout:8/816: creat d0/d29/da7/d26/d2a/d77/df8/d72/f107 x:0 0 0 2024-02-16T22:22:08.061 INFO:tasks.workunit.client.1.smithi123.stdout:2/771: dwrite d4/d1a/fe5 [0,4194304] 0 2024-02-16T22:22:08.061 INFO:tasks.workunit.client.1.smithi123.stdout:9/884: write d4/d36/d8e/d6d/d71/da6/fe5 [726719,4206] 0 2024-02-16T22:22:08.063 INFO:tasks.workunit.client.1.smithi123.stdout:7/772: rename d3/d6e/d1d/db2/d3d/l98 to d3/d87/lf8 0 2024-02-16T22:22:08.064 INFO:tasks.workunit.client.0.smithi114.stdout:0/598: dwrite d1/d1a/f4e [0,4194304] 0 2024-02-16T22:22:08.066 INFO:tasks.workunit.client.1.smithi123.stdout:0/689: dwrite d1/d4/d17/d74/d57/f72 [0,4194304] 0 2024-02-16T22:22:08.066 INFO:tasks.workunit.client.1.smithi123.stdout:9/885: truncate d4/d36/f5f 9344532 0 2024-02-16T22:22:08.066 INFO:tasks.workunit.client.1.smithi123.stdout:7/773: chown d3/d6e/d1d/cc0 2360634 1 2024-02-16T22:22:08.067 INFO:tasks.workunit.client.1.smithi123.stdout:1/762: symlink d3/d6/d49/d48/d84/d92/dec/l101 0 2024-02-16T22:22:08.068 INFO:tasks.workunit.client.1.smithi123.stdout:6/785: creat d0/d9/dc/f11c x:0 0 0 2024-02-16T22:22:08.070 INFO:tasks.workunit.client.0.smithi114.stdout:8/602: mknod d4/d11/d9d/da8/cda 0 2024-02-16T22:22:08.072 INFO:tasks.workunit.client.0.smithi114.stdout:0/599: rename d1/d16/l33 to d1/d16/d1f/d28/d51/d61/d6d/d6e/lb8 0 2024-02-16T22:22:08.072 INFO:tasks.workunit.client.1.smithi123.stdout:7/774: truncate d3/d6e/d34/fe7 1113727 0 2024-02-16T22:22:08.072 INFO:tasks.workunit.client.1.smithi123.stdout:3/795: dwrite d0/d20/d29/d9b/dbd/df4/ff9 [0,4194304] 0 2024-02-16T22:22:08.075 INFO:tasks.workunit.client.1.smithi123.stdout:3/796: chown d0/d20/d29/d9b/dbd/df4/d5d/d60/d93/ld8 1904198 1 2024-02-16T22:22:08.076 INFO:tasks.workunit.client.0.smithi114.stdout:0/600: creat d1/d16/d1f/d28/d51/d60/d52/fb9 x:0 0 0 2024-02-16T22:22:08.077 INFO:tasks.workunit.client.1.smithi123.stdout:5/758: dread d6/dd/df/f62 [0,4194304] 0 2024-02-16T22:22:08.077 INFO:tasks.workunit.client.0.smithi114.stdout:0/601: chown d1/d16/d1f/d28/c65 3847 1 2024-02-16T22:22:08.078 INFO:tasks.workunit.client.0.smithi114.stdout:8/603: creat d4/fdb x:0 0 0 2024-02-16T22:22:08.078 INFO:tasks.workunit.client.0.smithi114.stdout:0/602: fdatasync d1/d16/d18/f22 0 2024-02-16T22:22:08.079 INFO:tasks.workunit.client.1.smithi123.stdout:3/797: write d0/d20/d29/d9b/dbd/df4/d5d/dae/dc7/f10a [165595,101742] 0 2024-02-16T22:22:08.079 INFO:tasks.workunit.client.0.smithi114.stdout:8/604: readlink d4/d11/d78/la3 0 2024-02-16T22:22:08.080 INFO:tasks.workunit.client.0.smithi114.stdout:0/603: chown d1/d16/d1f/d28/d51/d61/d6d/d6e 36 1 2024-02-16T22:22:08.080 INFO:tasks.workunit.client.1.smithi123.stdout:8/817: creat d0/d29/da7/d26/d68/d81/d47/d9b/f108 x:0 0 0 2024-02-16T22:22:08.082 INFO:tasks.workunit.client.0.smithi114.stdout:8/605: dread - d4/d24/d2c/d5f/fa2 zero size 2024-02-16T22:22:08.082 INFO:tasks.workunit.client.0.smithi114.stdout:0/604: mkdir d1/d1a/d47/d62/d89/dba 0 2024-02-16T22:22:08.083 INFO:tasks.workunit.client.0.smithi114.stdout:0/605: readlink d1/d1a/d47/l5b 0 2024-02-16T22:22:08.084 INFO:tasks.workunit.client.0.smithi114.stdout:0/606: chown d1/d16/d1f/fa9 4088751 1 2024-02-16T22:22:08.086 INFO:tasks.workunit.client.0.smithi114.stdout:0/607: stat d1/d16/d1f/d28/d51/c86 0 2024-02-16T22:22:08.086 INFO:tasks.workunit.client.1.smithi123.stdout:9/886: truncate d4/d36/daa/d2b/f64 244982 0 2024-02-16T22:22:08.086 INFO:tasks.workunit.client.0.smithi114.stdout:0/608: stat d1/d16/d18/f77 0 2024-02-16T22:22:08.088 INFO:tasks.workunit.client.0.smithi114.stdout:8/606: dwrite d4/d16/d25/f54 [4194304,4194304] 0 2024-02-16T22:22:08.088 INFO:tasks.workunit.client.1.smithi123.stdout:3/798: creat d0/d20/d29/d9b/f10f x:0 0 0 2024-02-16T22:22:08.089 INFO:tasks.workunit.client.1.smithi123.stdout:3/799: chown d0/d20/d2e/dc4 1 1 2024-02-16T22:22:08.089 INFO:tasks.workunit.client.1.smithi123.stdout:5/759: dread - d6/dd/df/f5e zero size 2024-02-16T22:22:08.090 INFO:tasks.workunit.client.1.smithi123.stdout:5/760: readlink d6/l15 0 2024-02-16T22:22:08.091 INFO:tasks.workunit.client.1.smithi123.stdout:8/818: dread - d0/d29/da7/fc9 zero size 2024-02-16T22:22:08.092 INFO:tasks.workunit.client.1.smithi123.stdout:5/761: dread d6/dd/df/d58/d3f/d82/f83 [0,4194304] 0 2024-02-16T22:22:08.093 INFO:tasks.workunit.client.1.smithi123.stdout:0/690: mknod d1/d8/d13/d1d/ce8 0 2024-02-16T22:22:08.093 INFO:tasks.workunit.client.1.smithi123.stdout:0/691: chown d1/d4 257 1 2024-02-16T22:22:08.094 INFO:tasks.workunit.client.1.smithi123.stdout:6/786: mknod d0/d9/dc/c11d 0 2024-02-16T22:22:08.095 INFO:tasks.workunit.client.1.smithi123.stdout:0/692: chown d1/d4/d42/ld5 908087 1 2024-02-16T22:22:08.096 INFO:tasks.workunit.client.1.smithi123.stdout:9/887: write d4/f26 [308187,7994] 0 2024-02-16T22:22:08.097 INFO:tasks.workunit.client.1.smithi123.stdout:5/762: symlink d6/dd/df/d76/l110 0 2024-02-16T22:22:08.098 INFO:tasks.workunit.client.1.smithi123.stdout:8/819: mknod d0/d29/da7/d26/d2a/c109 0 2024-02-16T22:22:08.099 INFO:tasks.workunit.client.1.smithi123.stdout:5/763: stat d6/dd/df/d9f 0 2024-02-16T22:22:08.101 INFO:tasks.workunit.client.1.smithi123.stdout:8/820: rmdir d0/d61/d66/da5 39 2024-02-16T22:22:08.104 INFO:tasks.workunit.client.1.smithi123.stdout:5/764: write d6/dd/df/d58/d3f/dbe/da2/fc7 [694761,118873] 0 2024-02-16T22:22:08.104 INFO:tasks.workunit.client.1.smithi123.stdout:6/787: dwrite d0/d9/dc/f11c [0,4194304] 0 2024-02-16T22:22:08.106 INFO:tasks.workunit.client.1.smithi123.stdout:5/765: readlink d6/dd/df/d58/d89/lbf 0 2024-02-16T22:22:08.107 INFO:tasks.workunit.client.1.smithi123.stdout:0/693: dwrite d1/d8/d13/d1d/d62/d7a/f7f [0,4194304] 0 2024-02-16T22:22:08.111 INFO:tasks.workunit.client.1.smithi123.stdout:6/788: creat d0/d6/d91/d93/d116/f11e x:0 0 0 2024-02-16T22:22:08.113 INFO:tasks.workunit.client.1.smithi123.stdout:6/789: unlink d0/d6/d1a/la4 0 2024-02-16T22:22:08.114 INFO:tasks.workunit.client.1.smithi123.stdout:6/790: chown d0/d6/d2f 208389597 1 2024-02-16T22:22:08.115 INFO:tasks.workunit.client.1.smithi123.stdout:0/694: dwrite d1/d8/f2f [4194304,4194304] 0 2024-02-16T22:22:08.118 INFO:tasks.workunit.client.1.smithi123.stdout:0/695: chown d1/d8/d3d/c70 2300 1 2024-02-16T22:22:08.119 INFO:tasks.workunit.client.1.smithi123.stdout:0/696: write d1/d4/d17/d54/fd9 [1124716,111753] 0 2024-02-16T22:22:08.124 INFO:tasks.workunit.client.1.smithi123.stdout:0/697: rename d1/d8/d3d/c70 to d1/d8/d13/d1d/d64/ce9 0 2024-02-16T22:22:08.131 INFO:tasks.workunit.client.1.smithi123.stdout:6/791: dread d0/d96/de3/ff7 [4194304,4194304] 0 2024-02-16T22:22:08.134 INFO:tasks.workunit.client.1.smithi123.stdout:6/792: creat d0/d6/d91/d93/d116/f11f x:0 0 0 2024-02-16T22:22:08.139 INFO:tasks.workunit.client.1.smithi123.stdout:6/793: fdatasync d0/d9/dc/f11c 0 2024-02-16T22:22:08.143 INFO:tasks.workunit.client.1.smithi123.stdout:6/794: symlink d0/d8a/d100/l120 0 2024-02-16T22:22:08.143 INFO:tasks.workunit.client.1.smithi123.stdout:6/795: read - d0/d6/d2f/dbd/f119 zero size 2024-02-16T22:22:08.143 INFO:tasks.workunit.client.1.smithi123.stdout:8/821: dread d0/d29/da7/d43/d34/f38 [0,4194304] 0 2024-02-16T22:22:08.147 INFO:tasks.workunit.client.1.smithi123.stdout:6/796: rename d0/d6/d2f/l102 to d0/d9/dc/d3a/d70/l121 0 2024-02-16T22:22:08.149 INFO:tasks.workunit.client.1.smithi123.stdout:6/797: creat d0/d6/d2f/d5d/f122 x:0 0 0 2024-02-16T22:22:08.168 INFO:tasks.workunit.client.0.smithi114.stdout:0/609: read d1/f7 [2847155,104058] 0 2024-02-16T22:22:08.168 INFO:tasks.workunit.client.1.smithi123.stdout:3/800: read d0/d20/d29/d9b/dbd/df4/d5d/dae/db0/fc3 [3300877,73684] 0 2024-02-16T22:22:08.171 INFO:tasks.workunit.client.0.smithi114.stdout:0/610: mknod d1/d78/cbb 0 2024-02-16T22:22:08.173 INFO:tasks.workunit.client.0.smithi114.stdout:0/611: truncate d1/d16/d1f/d28/d51/fb3 246875 0 2024-02-16T22:22:08.174 INFO:tasks.workunit.client.1.smithi123.stdout:3/801: dwrite d0/d20/d29/f103 [0,4194304] 0 2024-02-16T22:22:08.176 INFO:tasks.workunit.client.0.smithi114.stdout:0/612: rename d1/d16/d1f/f63 to d1/d1a/d47/d62/d89/fbc 0 2024-02-16T22:22:08.179 INFO:tasks.workunit.client.1.smithi123.stdout:3/802: mknod d0/d20/d29/d9b/dbd/df5/c110 0 2024-02-16T22:22:08.180 INFO:tasks.workunit.client.0.smithi114.stdout:0/613: creat d1/d1a/d47/d62/d7f/d9d/fbd x:0 0 0 2024-02-16T22:22:08.182 INFO:tasks.workunit.client.0.smithi114.stdout:0/614: write d1/d16/d18/f22 [2103146,26091] 0 2024-02-16T22:22:08.185 INFO:tasks.workunit.client.1.smithi123.stdout:3/803: getdents d0/d20/d29/d9b/dbd/df4/d5d/d60/dc6/df0 0 2024-02-16T22:22:08.186 INFO:tasks.workunit.client.0.smithi114.stdout:0/615: symlink d1/d16/d1f/d28/d51/d61/d7e/d90/d95/d80/lbe 0 2024-02-16T22:22:08.187 INFO:tasks.workunit.client.1.smithi123.stdout:4/850: write d5/f1f [3767437,105084] 0 2024-02-16T22:22:08.193 INFO:tasks.workunit.client.1.smithi123.stdout:3/804: creat d0/d20/d29/d9b/dbd/df4/f111 x:0 0 0 2024-02-16T22:22:08.196 INFO:tasks.workunit.client.1.smithi123.stdout:3/805: creat d0/d20/d29/d9b/dbd/df4/d37/d41/f112 x:0 0 0 2024-02-16T22:22:08.198 INFO:tasks.workunit.client.1.smithi123.stdout:1/763: write d3/d6/f78 [1321411,25861] 0 2024-02-16T22:22:08.200 INFO:tasks.workunit.client.1.smithi123.stdout:7/775: write d3/fd [4033226,120666] 0 2024-02-16T22:22:08.202 INFO:tasks.workunit.client.1.smithi123.stdout:2/772: write d4/fc [2733824,125918] 0 2024-02-16T22:22:08.203 INFO:tasks.workunit.client.1.smithi123.stdout:1/764: write d3/d6/d18/d9e/fea [646994,92295] 0 2024-02-16T22:22:08.205 INFO:tasks.workunit.client.1.smithi123.stdout:3/806: dwrite d0/d20/d29/d9b/dbd/df4/ff9 [0,4194304] 0 2024-02-16T22:22:08.207 INFO:tasks.workunit.client.1.smithi123.stdout:9/888: write d4/f12 [4332721,112905] 0 2024-02-16T22:22:08.209 INFO:tasks.workunit.client.1.smithi123.stdout:3/807: fsync d0/d20/d29/d9b/dbd/df4/d37/f10c 0 2024-02-16T22:22:08.210 INFO:tasks.workunit.client.0.smithi114.stdout:8/607: dwrite d4/d16/fbf [0,4194304] 0 2024-02-16T22:22:08.210 INFO:tasks.workunit.client.1.smithi123.stdout:1/765: truncate d3/d6/d49/d48/d68/fc5 1246967 0 2024-02-16T22:22:08.210 INFO:tasks.workunit.client.1.smithi123.stdout:4/851: dread d5/d1c/d23/d74/f55 [0,4194304] 0 2024-02-16T22:22:08.212 INFO:tasks.workunit.client.1.smithi123.stdout:5/766: write d6/dd/df/f5e [499973,11728] 0 2024-02-16T22:22:08.213 INFO:tasks.workunit.client.1.smithi123.stdout:3/808: chown d0/d20/d29/f3d 43907 1 2024-02-16T22:22:08.214 INFO:tasks.workunit.client.1.smithi123.stdout:4/852: write d5/d1c/d9a/dd8/f100 [974871,37044] 0 2024-02-16T22:22:08.215 INFO:tasks.workunit.client.1.smithi123.stdout:4/853: chown d5/db8/cd9 482380 1 2024-02-16T22:22:08.216 INFO:tasks.workunit.client.1.smithi123.stdout:3/809: write d0/d20/d29/d9b/dbd/df4/d5d/f61 [7991601,59157] 0 2024-02-16T22:22:08.216 INFO:tasks.workunit.client.1.smithi123.stdout:0/698: write d1/d8/d13/d1d/d64/dbc/f5c [735059,21358] 0 2024-02-16T22:22:08.217 INFO:tasks.workunit.client.1.smithi123.stdout:2/773: getdents d4/d1a/d1d/d38/d5e/dec/df1/db1 0 2024-02-16T22:22:08.219 INFO:tasks.workunit.client.1.smithi123.stdout:0/699: chown d1/d4/d17/l98 0 1 2024-02-16T22:22:08.220 INFO:tasks.workunit.client.1.smithi123.stdout:5/767: fdatasync d6/dd/df/d58/d3f/d6f/f91 0 2024-02-16T22:22:08.222 INFO:tasks.workunit.client.1.smithi123.stdout:5/768: chown d6/dd/df/d1f 26 1 2024-02-16T22:22:08.222 INFO:tasks.workunit.client.1.smithi123.stdout:8/822: write d0/d29/da7/d26/d32/f6f [58750,118431] 0 2024-02-16T22:22:08.224 INFO:tasks.workunit.client.1.smithi123.stdout:4/854: mkdir d5/df4/d101 0 2024-02-16T22:22:08.225 INFO:tasks.workunit.client.1.smithi123.stdout:3/810: fsync d0/d20/d29/d9b/dbd/df4/d5d/fd7 0 2024-02-16T22:22:08.225 INFO:tasks.workunit.client.0.smithi114.stdout:8/608: dread d4/d16/d25/d7c/f87 [0,4194304] 0 2024-02-16T22:22:08.226 INFO:tasks.workunit.client.1.smithi123.stdout:4/855: write d5/d1c/d23/f57 [1244636,86611] 0 2024-02-16T22:22:08.227 INFO:tasks.workunit.client.1.smithi123.stdout:2/774: fsync d4/d1a/d1d/d59/d45/fa9 0 2024-02-16T22:22:08.228 INFO:tasks.workunit.client.1.smithi123.stdout:0/700: creat d1/d4/d17/fea x:0 0 0 2024-02-16T22:22:08.229 INFO:tasks.workunit.client.1.smithi123.stdout:3/811: dread - d0/d20/d5a/f108 zero size 2024-02-16T22:22:08.231 INFO:tasks.workunit.client.0.smithi114.stdout:8/609: creat d4/d11/d78/d7a/d36/d3b/fdc x:0 0 0 2024-02-16T22:22:08.232 INFO:tasks.workunit.client.1.smithi123.stdout:5/769: dwrite d6/dd/df/d1f/f68 [0,4194304] 0 2024-02-16T22:22:08.233 INFO:tasks.workunit.client.1.smithi123.stdout:8/823: creat d0/d61/d66/d7e/f10a x:0 0 0 2024-02-16T22:22:08.234 INFO:tasks.workunit.client.1.smithi123.stdout:6/798: dwrite d0/d9/dc/d3a/fc8 [0,4194304] 0 2024-02-16T22:22:08.235 INFO:tasks.workunit.client.0.smithi114.stdout:0/616: write d1/d1a/d3e/faf [5147166,41680] 0 2024-02-16T22:22:08.237 INFO:tasks.workunit.client.0.smithi114.stdout:0/617: truncate d1/d16/d1f/d28/d51/d61/d6d/fb6 472226 0 2024-02-16T22:22:08.238 INFO:tasks.workunit.client.1.smithi123.stdout:8/824: chown d0/d29/da7/d26/d32/ff6 1046137 1 2024-02-16T22:22:08.238 INFO:tasks.workunit.client.1.smithi123.stdout:5/770: write d6/dd/df/d76/feb [664972,122889] 0 2024-02-16T22:22:08.238 INFO:tasks.workunit.client.1.smithi123.stdout:6/799: fdatasync d0/d9/dc/d39/fa9 0 2024-02-16T22:22:08.238 INFO:tasks.workunit.client.1.smithi123.stdout:4/856: symlink d5/d1c/d23/d69/l102 0 2024-02-16T22:22:08.241 INFO:tasks.workunit.client.0.smithi114.stdout:8/610: creat d4/d11/d9d/fdd x:0 0 0 2024-02-16T22:22:08.241 INFO:tasks.workunit.client.1.smithi123.stdout:8/825: stat d0/d29/da7/d26/d2a/d77/df8/d62/cdc 0 2024-02-16T22:22:08.242 INFO:tasks.workunit.client.1.smithi123.stdout:7/776: write d3/d59/f7a [2637197,129406] 0 2024-02-16T22:22:08.244 INFO:tasks.workunit.client.1.smithi123.stdout:7/777: write d3/d6e/d4d/d92/fd1 [1190240,32509] 0 2024-02-16T22:22:08.245 INFO:tasks.workunit.client.1.smithi123.stdout:2/775: creat d4/dd9/ffb x:0 0 0 2024-02-16T22:22:08.246 INFO:tasks.workunit.client.1.smithi123.stdout:9/889: write d4/d36/d8e/d58/d59/d60/deb/fc2 [820525,25148] 0 2024-02-16T22:22:08.247 INFO:tasks.workunit.client.1.smithi123.stdout:9/890: chown d4/f6b 3674 1 2024-02-16T22:22:08.250 INFO:tasks.workunit.client.0.smithi114.stdout:8/611: truncate d4/d24/d2c/d67/fa4 4549966 0 2024-02-16T22:22:08.252 INFO:tasks.workunit.client.1.smithi123.stdout:4/857: mknod d5/d1c/d23/d6d/c103 0 2024-02-16T22:22:08.253 INFO:tasks.workunit.client.0.smithi114.stdout:0/618: getdents d1/d16/d1f/d28/d51/d60 0 2024-02-16T22:22:08.254 INFO:tasks.workunit.client.1.smithi123.stdout:8/826: mknod d0/d29/da7/d26/d68/c10b 0 2024-02-16T22:22:08.255 INFO:tasks.workunit.client.1.smithi123.stdout:3/812: dread d0/d20/d29/f3d [0,4194304] 0 2024-02-16T22:22:08.257 INFO:tasks.workunit.client.0.smithi114.stdout:8/612: creat d4/d16/d25/fde x:0 0 0 2024-02-16T22:22:08.260 INFO:tasks.workunit.client.1.smithi123.stdout:2/776: fsync d4/f23 0 2024-02-16T22:22:08.263 INFO:tasks.workunit.client.0.smithi114.stdout:8/613: mkdir d4/d11/d9d/ddf 0 2024-02-16T22:22:08.265 INFO:tasks.workunit.client.1.smithi123.stdout:4/858: rename d5/d1c/d23/d69/lf7 to d5/d1c/d9a/dd8/l104 0 2024-02-16T22:22:08.265 INFO:tasks.workunit.client.1.smithi123.stdout:9/891: dwrite d4/d36/f5f [0,4194304] 0 2024-02-16T22:22:08.269 INFO:tasks.workunit.client.0.smithi114.stdout:8/614: rename d4/d24/d6d/dc7/ccd to d4/d24/d2c/ce0 0 2024-02-16T22:22:08.275 INFO:tasks.workunit.client.1.smithi123.stdout:5/771: creat d6/dd/df/d58/d3f/dbe/f111 x:0 0 0 2024-02-16T22:22:08.276 INFO:tasks.workunit.client.1.smithi123.stdout:1/766: dwrite d3/d6/d49/d48/d68/fc5 [0,4194304] 0 2024-02-16T22:22:08.277 INFO:tasks.workunit.client.0.smithi114.stdout:8/615: dwrite d4/d11/d9d/fdd [0,4194304] 0 2024-02-16T22:22:08.277 INFO:tasks.workunit.client.0.smithi114.stdout:8/616: readlink d4/d11/l76 0 2024-02-16T22:22:08.278 INFO:tasks.workunit.client.1.smithi123.stdout:9/892: dwrite d4/d36/d57/db2/f28 [8388608,4194304] 0 2024-02-16T22:22:08.281 INFO:tasks.workunit.client.1.smithi123.stdout:4/859: write d5/d49/fb0 [2572419,119762] 0 2024-02-16T22:22:08.283 INFO:tasks.workunit.client.1.smithi123.stdout:4/860: stat d5/d1c/lf1 0 2024-02-16T22:22:08.284 INFO:tasks.workunit.client.1.smithi123.stdout:6/800: dwrite d0/d6/d1a/d2e/f77 [0,4194304] 0 2024-02-16T22:22:08.285 INFO:tasks.workunit.client.0.smithi114.stdout:0/619: dwrite d1/d1a/f45 [0,4194304] 0 2024-02-16T22:22:08.287 INFO:tasks.workunit.client.0.smithi114.stdout:8/617: unlink d4/d24/l62 0 2024-02-16T22:22:08.290 INFO:tasks.workunit.client.1.smithi123.stdout:2/777: mknod d4/d1a/d1d/d13/d25/d72/dc8/cfc 0 2024-02-16T22:22:08.290 INFO:tasks.workunit.client.0.smithi114.stdout:0/620: dread d1/d16/d1f/d28/d51/d61/d6d/d79/fa4 [0,4194304] 0 2024-02-16T22:22:08.292 INFO:tasks.workunit.client.1.smithi123.stdout:2/778: chown d4/d1a/d1d/d59/d45/l62 7 1 2024-02-16T22:22:08.294 INFO:tasks.workunit.client.1.smithi123.stdout:5/772: mknod d6/dd/df/d58/d89/d98/c112 0 2024-02-16T22:22:08.298 INFO:tasks.workunit.client.0.smithi114.stdout:0/621: mkdir d1/d16/dbf 0 2024-02-16T22:22:08.298 INFO:tasks.workunit.client.1.smithi123.stdout:8/827: dwrite d0/d29/da7/d26/d32/d45/f5f [4194304,4194304] 0 2024-02-16T22:22:08.301 INFO:tasks.workunit.client.1.smithi123.stdout:9/893: fsync d4/d1e/d3a/f9f 0 2024-02-16T22:22:08.301 INFO:tasks.workunit.client.1.smithi123.stdout:1/767: rmdir d3/d6/db/d10/d2d/d8c/dfd/d9f/db4/ddf 39 2024-02-16T22:22:08.302 INFO:tasks.workunit.client.1.smithi123.stdout:7/778: dwrite d3/d5/ff [0,4194304] 0 2024-02-16T22:22:08.303 INFO:tasks.workunit.client.1.smithi123.stdout:3/813: dwrite d0/d20/d29/d9b/dbd/df4/d37/d41/fd3 [0,4194304] 0 2024-02-16T22:22:08.303 INFO:tasks.workunit.client.1.smithi123.stdout:1/768: stat d3/d6/d49 0 2024-02-16T22:22:08.303 INFO:tasks.workunit.client.0.smithi114.stdout:8/618: dread d4/d11/d78/d7a/f10 [0,4194304] 0 2024-02-16T22:22:08.305 INFO:tasks.workunit.client.1.smithi123.stdout:9/894: stat d4/d36/daa/d2b/d6c/fdf 0 2024-02-16T22:22:08.307 INFO:tasks.workunit.client.0.smithi114.stdout:0/622: dwrite d1/d1a/f2e [4194304,4194304] 0 2024-02-16T22:22:08.308 INFO:tasks.workunit.client.1.smithi123.stdout:7/779: truncate d3/d5/d6c/ff6 884786 0 2024-02-16T22:22:08.308 INFO:tasks.workunit.client.1.smithi123.stdout:6/801: rmdir d0/d6/d2f/d5d 39 2024-02-16T22:22:08.308 INFO:tasks.workunit.client.1.smithi123.stdout:2/779: creat d4/d1a/d1d/d38/d5e/dec/df1/db1/de3/ffd x:0 0 0 2024-02-16T22:22:08.309 INFO:tasks.workunit.client.1.smithi123.stdout:5/773: fsync d6/dd/df/d58/fad 0 2024-02-16T22:22:08.313 INFO:tasks.workunit.client.1.smithi123.stdout:1/769: fsync d3/d6/d49/f50 0 2024-02-16T22:22:08.315 INFO:tasks.workunit.client.0.smithi114.stdout:0/623: unlink d1/d16/d1f/d28/l2f 0 2024-02-16T22:22:08.315 INFO:tasks.workunit.client.1.smithi123.stdout:3/814: write d0/d20/d29/d9b/dbd/df4/d5d/d60/dd6/f9f [948226,66700] 0 2024-02-16T22:22:08.316 INFO:tasks.workunit.client.1.smithi123.stdout:9/895: creat d4/d36/daa/d20/dcb/f122 x:0 0 0 2024-02-16T22:22:08.317 INFO:tasks.workunit.client.0.smithi114.stdout:0/624: truncate d1/d16/d1f/d28/d51/d60/d52/fb9 828443 0 2024-02-16T22:22:08.318 INFO:tasks.workunit.client.1.smithi123.stdout:4/861: creat d5/d1c/d23/f105 x:0 0 0 2024-02-16T22:22:08.318 INFO:tasks.workunit.client.1.smithi123.stdout:3/815: chown d0/d20/d94/fe0 0 1 2024-02-16T22:22:08.320 INFO:tasks.workunit.client.1.smithi123.stdout:3/816: stat d0/d20/d29/d9b/dbd/df4/d5d/d60/dc6 0 2024-02-16T22:22:08.320 INFO:tasks.workunit.client.1.smithi123.stdout:7/780: symlink d3/d6e/d1d/db2/lf9 0 2024-02-16T22:22:08.321 INFO:tasks.workunit.client.1.smithi123.stdout:9/896: truncate d4/d36/d8e/d58/d59/d60/deb/da7/f11e 209470 0 2024-02-16T22:22:08.321 INFO:tasks.workunit.client.1.smithi123.stdout:3/817: chown d0/d40/fd0 2 1 2024-02-16T22:22:08.322 INFO:tasks.workunit.client.0.smithi114.stdout:0/625: unlink d1/d1a/c5f 0 2024-02-16T22:22:08.322 INFO:tasks.workunit.client.1.smithi123.stdout:3/818: chown d0/d20/d29 7201 1 2024-02-16T22:22:08.323 INFO:tasks.workunit.client.1.smithi123.stdout:3/819: dread - d0/d20/d29/d9b/dbd/df4/f111 zero size 2024-02-16T22:22:08.323 INFO:tasks.workunit.client.1.smithi123.stdout:1/770: creat d3/d6/db/d10/d2d/d8c/dfd/de8/f102 x:0 0 0 2024-02-16T22:22:08.324 INFO:tasks.workunit.client.1.smithi123.stdout:7/781: read d3/d6e/d24/f6d [4152553,104595] 0 2024-02-16T22:22:08.326 INFO:tasks.workunit.client.0.smithi114.stdout:0/626: mknod d1/d16/d1f/d28/d51/d60/d52/cc0 0 2024-02-16T22:22:08.327 INFO:tasks.workunit.client.1.smithi123.stdout:2/780: dread - d4/d1a/d1d/d13/d25/d72/f73 zero size 2024-02-16T22:22:08.327 INFO:tasks.workunit.client.1.smithi123.stdout:0/701: sync 2024-02-16T22:22:08.329 INFO:tasks.workunit.client.1.smithi123.stdout:4/862: write d5/d1c/d23/d47/ff3 [975413,107922] 0 2024-02-16T22:22:08.330 INFO:tasks.workunit.client.1.smithi123.stdout:4/863: fsync d5/d1c/d23/d47/fde 0 2024-02-16T22:22:08.332 INFO:tasks.workunit.client.1.smithi123.stdout:9/897: symlink d4/d36/d57/l123 0 2024-02-16T22:22:08.334 INFO:tasks.workunit.client.0.smithi114.stdout:0/627: dwrite d1/d1a/d3e/fa6 [0,4194304] 0 2024-02-16T22:22:08.335 INFO:tasks.workunit.client.1.smithi123.stdout:7/782: symlink d3/d87/lfa 0 2024-02-16T22:22:08.336 INFO:tasks.workunit.client.1.smithi123.stdout:2/781: creat d4/dae/df6/ffe x:0 0 0 2024-02-16T22:22:08.337 INFO:tasks.workunit.client.1.smithi123.stdout:0/702: truncate d1/d4/d42/f4a 1774146 0 2024-02-16T22:22:08.339 INFO:tasks.workunit.client.0.smithi114.stdout:0/628: creat d1/d16/d1f/d28/d51/d61/d7e/d90/fc1 x:0 0 0 2024-02-16T22:22:08.341 INFO:tasks.workunit.client.1.smithi123.stdout:5/774: rmdir d6/dd/df/d90/dd4 0 2024-02-16T22:22:08.343 INFO:tasks.workunit.client.1.smithi123.stdout:1/771: read d3/f35 [3992398,90106] 0 2024-02-16T22:22:08.344 INFO:tasks.workunit.client.0.smithi114.stdout:0/629: creat d1/d16/d1f/d28/d51/d61/d7e/fc2 x:0 0 0 2024-02-16T22:22:08.348 INFO:tasks.workunit.client.1.smithi123.stdout:4/864: symlink d5/d1c/d23/d56/d6e/d9f/da1/dc0/dc5/l106 0 2024-02-16T22:22:08.348 INFO:tasks.workunit.client.1.smithi123.stdout:3/820: fdatasync d0/da0/fba 0 2024-02-16T22:22:08.348 INFO:tasks.workunit.client.1.smithi123.stdout:3/821: dread - d0/d20/d29/d9b/f10f zero size 2024-02-16T22:22:08.349 INFO:tasks.workunit.client.1.smithi123.stdout:3/822: chown d0/f58 0 1 2024-02-16T22:22:08.351 INFO:tasks.workunit.client.1.smithi123.stdout:7/783: dread - d3/d6e/d4d/d92/fcd zero size 2024-02-16T22:22:08.354 INFO:tasks.workunit.client.1.smithi123.stdout:0/703: creat d1/d4/feb x:0 0 0 2024-02-16T22:22:08.355 INFO:tasks.workunit.client.1.smithi123.stdout:0/704: read d1/f6a [970655,61976] 0 2024-02-16T22:22:08.355 INFO:tasks.workunit.client.1.smithi123.stdout:8/828: dread d0/fd [0,4194304] 0 2024-02-16T22:22:08.356 INFO:tasks.workunit.client.0.smithi114.stdout:8/619: dwrite d4/d16/d25/d46/f73 [0,4194304] 0 2024-02-16T22:22:08.357 INFO:tasks.workunit.client.1.smithi123.stdout:0/705: read d1/d4/d17/d74/d57/f75 [1742940,66332] 0 2024-02-16T22:22:08.361 INFO:tasks.workunit.client.1.smithi123.stdout:1/772: dread - d3/d6/db/d10/d4b/d70/d87/fb3 zero size 2024-02-16T22:22:08.361 INFO:tasks.workunit.client.0.smithi114.stdout:0/630: getdents d1/d1a/d47/d62/d7f/d9d 0 2024-02-16T22:22:08.362 INFO:tasks.workunit.client.1.smithi123.stdout:8/829: truncate d0/d29/da7/d43/d34/dae/fc1 417512 0 2024-02-16T22:22:08.362 INFO:tasks.workunit.client.1.smithi123.stdout:6/802: dwrite d0/d6/d1a/f64 [0,4194304] 0 2024-02-16T22:22:08.364 INFO:tasks.workunit.client.0.smithi114.stdout:8/620: rmdir d4/d16/d25/d46/db5 39 2024-02-16T22:22:08.366 INFO:tasks.workunit.client.1.smithi123.stdout:6/803: write d0/d6/d2f/da5/dc6/f112 [337499,7359] 0 2024-02-16T22:22:08.367 INFO:tasks.workunit.client.1.smithi123.stdout:6/804: readlink d0/d9/dc/d3a/d52/lb3 0 2024-02-16T22:22:08.368 INFO:tasks.workunit.client.1.smithi123.stdout:3/823: rmdir d0/d50 39 2024-02-16T22:22:08.371 INFO:tasks.workunit.client.1.smithi123.stdout:7/784: symlink d3/lfb 0 2024-02-16T22:22:08.374 INFO:tasks.workunit.client.1.smithi123.stdout:6/805: dwrite d0/d96/de3/dcc/fde [0,4194304] 0 2024-02-16T22:22:08.376 INFO:tasks.workunit.client.0.smithi114.stdout:8/621: dread d4/d11/d78/d7a/d7e/fba [0,4194304] 0 2024-02-16T22:22:08.380 INFO:tasks.workunit.client.0.smithi114.stdout:8/622: rmdir d4/d11/d9d 39 2024-02-16T22:22:08.385 INFO:tasks.workunit.client.1.smithi123.stdout:1/773: creat d3/d6/db/d55/f103 x:0 0 0 2024-02-16T22:22:08.385 INFO:tasks.workunit.client.1.smithi123.stdout:1/774: readlink d3/d6/l81 0 2024-02-16T22:22:08.387 INFO:tasks.workunit.client.1.smithi123.stdout:8/830: rename d0/d12/ded/fcd to d0/df/dca/f10c 0 2024-02-16T22:22:08.387 INFO:tasks.workunit.client.0.smithi114.stdout:0/631: dread d1/d16/d48/f6a [0,4194304] 0 2024-02-16T22:22:08.389 INFO:tasks.workunit.client.1.smithi123.stdout:3/824: mkdir d0/d20/d2e/dc4/d113 0 2024-02-16T22:22:08.391 INFO:tasks.workunit.client.1.smithi123.stdout:9/898: dwrite d4/d36/d8e/d58/f47 [0,4194304] 0 2024-02-16T22:22:08.392 INFO:tasks.workunit.client.0.smithi114.stdout:0/632: link d1/d1a/d47/d62/d7f/fa8 d1/d16/d1f/d28/d51/d61/d7e/fc3 0 2024-02-16T22:22:08.393 INFO:tasks.workunit.client.1.smithi123.stdout:4/865: dread d5/f1e [0,4194304] 0 2024-02-16T22:22:08.394 INFO:tasks.workunit.client.1.smithi123.stdout:0/706: symlink d1/d8/d13/d1d/lec 0 2024-02-16T22:22:08.396 INFO:tasks.workunit.client.1.smithi123.stdout:1/775: symlink d3/d6/db/d10/d2d/l104 0 2024-02-16T22:22:08.396 INFO:tasks.workunit.client.1.smithi123.stdout:5/775: dwrite d6/dd/df/d58/d3f/d82/f83 [0,4194304] 0 2024-02-16T22:22:08.397 INFO:tasks.workunit.client.1.smithi123.stdout:1/776: readlink d3/d6/le1 0 2024-02-16T22:22:08.399 INFO:tasks.workunit.client.1.smithi123.stdout:8/831: symlink d0/d61/d66/d7e/l10d 0 2024-02-16T22:22:08.402 INFO:tasks.workunit.client.1.smithi123.stdout:4/866: dread d5/d1c/d23/d56/d6e/fa5 [0,4194304] 0 2024-02-16T22:22:08.405 INFO:tasks.workunit.client.1.smithi123.stdout:2/782: getdents d4/d1a/d1d/d13/d25/d72/dc8 0 2024-02-16T22:22:08.408 INFO:tasks.workunit.client.1.smithi123.stdout:5/776: rename d6/dd/df/d58/f43 to d6/dd/df/d58/d56/f113 0 2024-02-16T22:22:08.409 INFO:tasks.workunit.client.1.smithi123.stdout:4/867: creat d5/d1c/d64/d7b/f107 x:0 0 0 2024-02-16T22:22:08.410 INFO:tasks.workunit.client.1.smithi123.stdout:9/899: unlink d4/d33/cf5 0 2024-02-16T22:22:08.410 INFO:tasks.workunit.client.1.smithi123.stdout:4/868: dread - d5/d1c/d23/d69/fe1 zero size 2024-02-16T22:22:08.410 INFO:tasks.workunit.client.1.smithi123.stdout:2/783: write d4/d1a/d1d/d38/f58 [5048681,68898] 0 2024-02-16T22:22:08.411 INFO:tasks.workunit.client.1.smithi123.stdout:8/832: symlink d0/d61/d66/l10e 0 2024-02-16T22:22:08.411 INFO:tasks.workunit.client.1.smithi123.stdout:9/900: write d4/d1e/d3a/fcf [432286,8213] 0 2024-02-16T22:22:08.412 INFO:tasks.workunit.client.1.smithi123.stdout:7/785: symlink d3/d6e/d1d/db2/d35/db7/lfc 0 2024-02-16T22:22:08.413 INFO:tasks.workunit.client.1.smithi123.stdout:1/777: rmdir d3/d6/d11/d8b/dbb/dd8 0 2024-02-16T22:22:08.414 INFO:tasks.workunit.client.1.smithi123.stdout:5/777: symlink d6/dd/df/d58/d89/d98/dd8/l114 0 2024-02-16T22:22:08.415 INFO:tasks.workunit.client.1.smithi123.stdout:9/901: readlink d4/d1e/l80 0 2024-02-16T22:22:08.416 INFO:tasks.workunit.client.1.smithi123.stdout:8/833: symlink d0/d61/dac/l10f 0 2024-02-16T22:22:08.417 INFO:tasks.workunit.client.1.smithi123.stdout:7/786: write d3/fd [96215,9690] 0 2024-02-16T22:22:08.417 INFO:tasks.workunit.client.1.smithi123.stdout:8/834: chown d0/d29/da7/d26/d2a/d4d/le3 736 1 2024-02-16T22:22:08.419 INFO:tasks.workunit.client.1.smithi123.stdout:4/869: rename d5/d1c/d23/d56/d6e/d9f/da1 to d5/df4/d108 0 2024-02-16T22:22:08.421 INFO:tasks.workunit.client.1.smithi123.stdout:2/784: mknod d4/cff 0 2024-02-16T22:22:08.422 INFO:tasks.workunit.client.1.smithi123.stdout:9/902: creat d4/f124 x:0 0 0 2024-02-16T22:22:08.423 INFO:tasks.workunit.client.0.smithi114.stdout:0/633: write d1/d16/d1f/d28/f35 [490716,16386] 0 2024-02-16T22:22:08.424 INFO:tasks.workunit.client.1.smithi123.stdout:7/787: mkdir d3/d6e/d1d/db2/d3e/de4/dfd 0 2024-02-16T22:22:08.426 INFO:tasks.workunit.client.1.smithi123.stdout:5/778: rename d6/f109 to d6/dd/df/d58/d89/dc3/f115 0 2024-02-16T22:22:08.427 INFO:tasks.workunit.client.1.smithi123.stdout:5/779: read - d6/dd/df/d9f/fd7 zero size 2024-02-16T22:22:08.427 INFO:tasks.workunit.client.1.smithi123.stdout:3/825: dread d0/d40/f56 [0,4194304] 0 2024-02-16T22:22:08.428 INFO:tasks.workunit.client.1.smithi123.stdout:4/870: creat d5/d1c/d5e/f109 x:0 0 0 2024-02-16T22:22:08.428 INFO:tasks.workunit.client.1.smithi123.stdout:3/826: readlink d0/l38 0 2024-02-16T22:22:08.428 INFO:tasks.workunit.client.1.smithi123.stdout:5/780: fdatasync d6/dd/df/f5e 0 2024-02-16T22:22:08.429 INFO:tasks.workunit.client.1.smithi123.stdout:6/806: truncate d0/d96/fe8 639465 0 2024-02-16T22:22:08.429 INFO:tasks.workunit.client.1.smithi123.stdout:5/781: chown d6/dd/cd9 4 1 2024-02-16T22:22:08.430 INFO:tasks.workunit.client.1.smithi123.stdout:5/782: dread - d6/d3b/fa7 zero size 2024-02-16T22:22:08.434 INFO:tasks.workunit.client.0.smithi114.stdout:0/634: link d1/d16/d1f/d28/d51/d61/d6d/d6e/f6f d1/d16/d48/d98/fc4 0 2024-02-16T22:22:08.434 INFO:tasks.workunit.client.1.smithi123.stdout:5/783: dread d6/dd/df/d58/d3f/dbe/f31 [0,4194304] 0 2024-02-16T22:22:08.435 INFO:tasks.workunit.client.0.smithi114.stdout:0/635: chown d1/d16/d18/f1c 11766370 1 2024-02-16T22:22:08.435 INFO:tasks.workunit.client.1.smithi123.stdout:2/785: truncate d4/d1a/d1d/d59/dd7/ddc/fe4 727221 0 2024-02-16T22:22:08.435 INFO:tasks.workunit.client.1.smithi123.stdout:5/784: fdatasync d6/dd/df/d58/d3f/dbe/f5d 0 2024-02-16T22:22:08.439 INFO:tasks.workunit.client.0.smithi114.stdout:0/636: dread d1/d16/f73 [0,4194304] 0 2024-02-16T22:22:08.440 INFO:tasks.workunit.client.0.smithi114.stdout:0/637: read d1/d1a/d47/d62/d7f/d9d/fad [2665732,4791] 0 2024-02-16T22:22:08.442 INFO:tasks.workunit.client.0.smithi114.stdout:0/638: chown d1/d16/d1f/d28/d51/d61/d6d/d6e 10083 1 2024-02-16T22:22:08.448 INFO:tasks.workunit.client.0.smithi114.stdout:0/639: rename d1/d1a/c1b to d1/d16/cc5 0 2024-02-16T22:22:08.449 INFO:tasks.workunit.client.0.smithi114.stdout:0/640: fsync d1/d16/d1f/fa9 0 2024-02-16T22:22:08.451 INFO:tasks.workunit.client.1.smithi123.stdout:6/807: dread d0/d9/dc/d3a/f80 [0,4194304] 0 2024-02-16T22:22:08.451 INFO:tasks.workunit.client.1.smithi123.stdout:9/903: dread d4/d36/d8e/f74 [0,4194304] 0 2024-02-16T22:22:08.452 INFO:tasks.workunit.client.1.smithi123.stdout:9/904: dread - d4/d36/d57/db2/fed zero size 2024-02-16T22:22:08.455 INFO:tasks.workunit.client.0.smithi114.stdout:0/641: rename d1/d1a/d3e/fa6 to d1/d16/dbf/fc6 0 2024-02-16T22:22:08.456 INFO:tasks.workunit.client.1.smithi123.stdout:1/778: write d3/f7c [76295,128365] 0 2024-02-16T22:22:08.457 INFO:tasks.workunit.client.1.smithi123.stdout:1/779: chown d3/d6/db/d10/d4b/d70/d87/fb3 50472 1 2024-02-16T22:22:08.459 INFO:tasks.workunit.client.1.smithi123.stdout:0/707: sync 2024-02-16T22:22:08.461 INFO:tasks.workunit.client.0.smithi114.stdout:0/642: creat d1/d16/d1f/d28/d51/d60/d52/fc7 x:0 0 0 2024-02-16T22:22:08.464 INFO:tasks.workunit.client.1.smithi123.stdout:8/835: rename d0/df/l10 to d0/d61/d66/d7e/l110 0 2024-02-16T22:22:08.464 INFO:tasks.workunit.client.1.smithi123.stdout:7/788: dwrite d3/d6e/d24/f90 [0,4194304] 0 2024-02-16T22:22:08.464 INFO:tasks.workunit.client.1.smithi123.stdout:3/827: symlink d0/d20/d29/d9b/dbd/df5/d107/l114 0 2024-02-16T22:22:08.465 INFO:tasks.workunit.client.1.smithi123.stdout:3/828: fsync d0/d20/d29/fbf 0 2024-02-16T22:22:08.467 INFO:tasks.workunit.client.0.smithi114.stdout:0/643: truncate d1/d16/d48/d98/fc4 1133607 0 2024-02-16T22:22:08.469 INFO:tasks.workunit.client.0.smithi114.stdout:0/644: chown d1/d16/d1f/d28/d51/d61/f87 7676 1 2024-02-16T22:22:08.469 INFO:tasks.workunit.client.1.smithi123.stdout:3/829: write d0/d20/d29/d9b/dbd/df4/d5d/f109 [797113,90135] 0 2024-02-16T22:22:08.469 INFO:tasks.workunit.client.1.smithi123.stdout:6/808: mkdir d0/d6/d8e/d123 0 2024-02-16T22:22:08.470 INFO:tasks.workunit.client.1.smithi123.stdout:6/809: chown d0/d9 29371794 1 2024-02-16T22:22:08.470 INFO:tasks.workunit.client.1.smithi123.stdout:0/708: dread d1/f22 [0,4194304] 0 2024-02-16T22:22:08.471 INFO:tasks.workunit.client.1.smithi123.stdout:3/830: write d0/d20/d29/d9b/dbd/df4/f111 [190368,3757] 0 2024-02-16T22:22:08.472 INFO:tasks.workunit.client.1.smithi123.stdout:9/905: fdatasync d4/d36/d8e/d6f/d7e/fe9 0 2024-02-16T22:22:08.475 INFO:tasks.workunit.client.1.smithi123.stdout:2/786: symlink d4/d1a/d1d/d59/dd7/l100 0 2024-02-16T22:22:08.475 INFO:tasks.workunit.client.1.smithi123.stdout:1/780: mkdir d3/d6/db/d10/d4b/d70/d87/d105 0 2024-02-16T22:22:08.477 INFO:tasks.workunit.client.0.smithi114.stdout:0/645: dwrite d1/d16/d1f/d28/d51/d61/d6d/fae [0,4194304] 0 2024-02-16T22:22:08.478 INFO:tasks.workunit.client.0.smithi114.stdout:0/646: dread - d1/d16/d1f/d28/fb0 zero size 2024-02-16T22:22:08.479 INFO:tasks.workunit.client.1.smithi123.stdout:7/789: readlink d3/d59/l95 0 2024-02-16T22:22:08.484 INFO:tasks.workunit.client.1.smithi123.stdout:8/836: rename d0/d29/da7/d26/d32/ff6 to d0/d12/ded/f111 0 2024-02-16T22:22:08.487 INFO:tasks.workunit.client.0.smithi114.stdout:0/647: rename d1/d1a/d47/c68 to d1/d16/d1f/d28/d51/d60/cc8 0 2024-02-16T22:22:08.491 INFO:tasks.workunit.client.1.smithi123.stdout:0/709: creat d1/d4/d17/dad/fed x:0 0 0 2024-02-16T22:22:08.492 INFO:tasks.workunit.client.1.smithi123.stdout:0/710: chown d1/d4/d17/d74/c4e 1290 1 2024-02-16T22:22:08.493 INFO:tasks.workunit.client.0.smithi114.stdout:0/648: creat d1/d16/d1f/d28/d51/d61/d7e/d90/d95/fc9 x:0 0 0 2024-02-16T22:22:08.494 INFO:tasks.workunit.client.0.smithi114.stdout:0/649: fdatasync d1/d16/d1f/d28/d51/d61/d6d/fae 0 2024-02-16T22:22:08.499 INFO:tasks.workunit.client.1.smithi123.stdout:2/787: symlink d4/d1a/d1d/d38/d5e/dec/df1/d4a/dce/dd4/l101 0 2024-02-16T22:22:08.504 INFO:tasks.workunit.client.1.smithi123.stdout:7/790: fsync d3/d6e/d4d/f50 0 2024-02-16T22:22:08.504 INFO:tasks.workunit.client.1.smithi123.stdout:2/788: dwrite d4/d1a/d1d/d38/d7b/fc4 [0,4194304] 0 2024-02-16T22:22:08.505 INFO:tasks.workunit.client.1.smithi123.stdout:4/871: link d5/d1c/d23/d76/f9e d5/d1c/d23/f10a 0 2024-02-16T22:22:08.511 INFO:tasks.workunit.client.1.smithi123.stdout:1/781: dread d3/d6/d11/f1f [0,4194304] 0 2024-02-16T22:22:08.513 INFO:tasks.workunit.client.0.smithi114.stdout:1/601: sync 2024-02-16T22:22:08.516 INFO:tasks.workunit.client.1.smithi123.stdout:0/711: mkdir d1/d4/d17/d54/d8e/d96/dee 0 2024-02-16T22:22:08.518 INFO:tasks.workunit.client.1.smithi123.stdout:9/906: mknod d4/d36/d8e/d58/c125 0 2024-02-16T22:22:08.522 INFO:tasks.workunit.client.1.smithi123.stdout:5/785: getdents d6/dd/df/d58/d89/d98 0 2024-02-16T22:22:08.524 INFO:tasks.workunit.client.1.smithi123.stdout:8/837: write d0/d29/da7/d26/d68/d81/d47/f6a [3714904,112609] 0 2024-02-16T22:22:08.525 INFO:tasks.workunit.client.1.smithi123.stdout:8/838: dread - d0/d29/da7/d26/d68/dbe/fef zero size 2024-02-16T22:22:08.526 INFO:tasks.workunit.client.1.smithi123.stdout:2/789: mknod d4/d1a/d1d/d38/c102 0 2024-02-16T22:22:08.526 INFO:tasks.workunit.client.1.smithi123.stdout:8/839: readlink d0/d29/da7/d43/d34/l101 0 2024-02-16T22:22:08.528 INFO:tasks.workunit.client.1.smithi123.stdout:4/872: mknod d5/d1c/d23/d69/c10b 0 2024-02-16T22:22:08.529 INFO:tasks.workunit.client.1.smithi123.stdout:6/810: truncate d0/d6/d4d/db0/ffb 2654980 0 2024-02-16T22:22:08.531 INFO:tasks.workunit.client.1.smithi123.stdout:3/831: rename d0/d20/d29/d9b/dbd/df4/d37/l9a to d0/d20/d29/d9b/dbd/df4/d5d/d60/d93/l115 0 2024-02-16T22:22:08.533 INFO:tasks.workunit.client.1.smithi123.stdout:9/907: mkdir d4/d36/d8e/dff/d126 0 2024-02-16T22:22:08.539 INFO:tasks.workunit.client.1.smithi123.stdout:7/791: dread d3/d5/f33 [0,4194304] 0 2024-02-16T22:22:08.540 INFO:tasks.workunit.client.1.smithi123.stdout:8/840: creat d0/d12/f112 x:0 0 0 2024-02-16T22:22:08.541 INFO:tasks.workunit.client.1.smithi123.stdout:4/873: creat d5/d1c/d23/d76/f10c x:0 0 0 2024-02-16T22:22:08.545 INFO:tasks.workunit.client.1.smithi123.stdout:1/782: rename d3/d6/d11/ca8 to d3/d6/d18/dcb/c106 0 2024-02-16T22:22:08.553 INFO:tasks.workunit.client.1.smithi123.stdout:2/790: fsync d4/d1a/d1d/d13/d25/f3b 0 2024-02-16T22:22:08.557 INFO:tasks.workunit.client.1.smithi123.stdout:9/908: dread d4/d36/daa/f1f [0,4194304] 0 2024-02-16T22:22:08.558 INFO:tasks.workunit.client.1.smithi123.stdout:5/786: dwrite d6/dd/df/d58/f4f [0,4194304] 0 2024-02-16T22:22:08.561 INFO:tasks.workunit.client.1.smithi123.stdout:3/832: write d0/f6 [1534851,89868] 0 2024-02-16T22:22:08.562 INFO:tasks.workunit.client.1.smithi123.stdout:5/787: read d6/dd/df/d58/d3f/dbe/f3a [2897377,92053] 0 2024-02-16T22:22:08.564 INFO:tasks.workunit.client.1.smithi123.stdout:6/811: rename d0/d6/d91/d93/d116/f11f to d0/d6/f124 0 2024-02-16T22:22:08.566 INFO:tasks.workunit.client.1.smithi123.stdout:9/909: dwrite d4/d36/d8e/d58/d59/d60/d9d/de4/f11a [0,4194304] 0 2024-02-16T22:22:08.571 INFO:tasks.workunit.client.0.smithi114.stdout:2/566: sync 2024-02-16T22:22:08.575 INFO:tasks.workunit.client.1.smithi123.stdout:9/910: dwrite d4/d1e/d3a/fcf [0,4194304] 0 2024-02-16T22:22:08.583 INFO:tasks.workunit.client.1.smithi123.stdout:4/874: write d5/d1c/d23/d56/d6e/d73/d87/fe4 [4817518,24030] 0 2024-02-16T22:22:08.592 INFO:tasks.workunit.client.1.smithi123.stdout:3/833: rename d0/d20/d29/d9b/dbd/df4/d37/d41/fef to d0/d20/d29/d9b/dbd/df4/d5d/d60/d93/f116 0 2024-02-16T22:22:08.593 INFO:tasks.workunit.client.1.smithi123.stdout:1/783: dwrite d3/d6/db/d10/d4b/d70/f94 [0,4194304] 0 2024-02-16T22:22:08.593 INFO:tasks.workunit.client.1.smithi123.stdout:1/784: chown d3/d6/d11 13 1 2024-02-16T22:22:08.596 INFO:tasks.workunit.client.1.smithi123.stdout:0/712: link d1/d4/c1a d1/d8/cef 0 2024-02-16T22:22:08.596 INFO:tasks.workunit.client.1.smithi123.stdout:0/713: chown d1/d8/d13/f15 3 1 2024-02-16T22:22:08.599 INFO:tasks.workunit.client.1.smithi123.stdout:2/791: symlink d4/d1a/d1d/d59/dd7/l103 0 2024-02-16T22:22:08.601 INFO:tasks.workunit.client.1.smithi123.stdout:2/792: write d4/d1a/d1d/d59/d45/fc5 [4377398,95155] 0 2024-02-16T22:22:08.602 INFO:tasks.workunit.client.0.smithi114.stdout:0/650: fsync d1/fa 0 2024-02-16T22:22:08.604 INFO:tasks.workunit.client.0.smithi114.stdout:0/651: write d1/d16/d1f/d28/d51/d61/d6d/fae [2526968,26918] 0 2024-02-16T22:22:08.604 INFO:tasks.workunit.client.1.smithi123.stdout:8/841: link d0/d29/da7/d26/d68/d81/d47/fa6 d0/d29/da7/d43/d34/f113 0 2024-02-16T22:22:08.607 INFO:tasks.workunit.client.1.smithi123.stdout:9/911: mkdir d4/d36/d8e/d98/d127 0 2024-02-16T22:22:08.607 INFO:tasks.workunit.client.0.smithi114.stdout:0/652: dread d1/d16/d48/f6a [0,4194304] 0 2024-02-16T22:22:08.611 INFO:tasks.workunit.client.1.smithi123.stdout:4/875: mkdir d5/d1c/d64/d10d 0 2024-02-16T22:22:08.613 INFO:tasks.workunit.client.1.smithi123.stdout:5/788: mkdir d6/dd/df/d116 0 2024-02-16T22:22:08.613 INFO:tasks.workunit.client.1.smithi123.stdout:5/789: fsync d6/dac/ff5 0 2024-02-16T22:22:08.613 INFO:tasks.workunit.client.0.smithi114.stdout:0/653: chown d1/d16/d1f/d28/d51/d61/d7e/d90/c7a 48032823 1 2024-02-16T22:22:08.615 INFO:tasks.workunit.client.1.smithi123.stdout:6/812: unlink d0/d6/d91/d93/c9f 0 2024-02-16T22:22:08.615 INFO:tasks.workunit.client.1.smithi123.stdout:6/813: stat d0/d6/d1a/f64 0 2024-02-16T22:22:08.618 INFO:tasks.workunit.client.0.smithi114.stdout:0/654: mkdir d1/d16/d1f/d28/d51/dca 0 2024-02-16T22:22:08.619 INFO:tasks.workunit.client.1.smithi123.stdout:8/842: dread d0/f4 [0,4194304] 0 2024-02-16T22:22:08.620 INFO:tasks.workunit.client.0.smithi114.stdout:1/602: getdents d4/d63/d69/d7c/dc0 0 2024-02-16T22:22:08.620 INFO:tasks.workunit.client.1.smithi123.stdout:8/843: fsync d0/d29/da7/d26/d68/d81/d47/f73 0 2024-02-16T22:22:08.622 INFO:tasks.workunit.client.1.smithi123.stdout:0/714: truncate d1/d8/d13/f4d 922407 0 2024-02-16T22:22:08.624 INFO:tasks.workunit.client.0.smithi114.stdout:0/655: rename d1/d16/d1f/d28/c3f to d1/d16/d1f/ccb 0 2024-02-16T22:22:08.625 INFO:tasks.workunit.client.0.smithi114.stdout:2/567: write dd/df/d1c/f40 [1528574,110776] 0 2024-02-16T22:22:08.627 INFO:tasks.workunit.client.0.smithi114.stdout:1/603: dwrite d4/d63/d69/d7c/d9b/db0/f43 [0,4194304] 0 2024-02-16T22:22:08.631 INFO:tasks.workunit.client.0.smithi114.stdout:0/656: creat d1/d16/d1f/d28/d51/d61/d6d/fcc x:0 0 0 2024-02-16T22:22:08.632 INFO:tasks.workunit.client.1.smithi123.stdout:4/876: creat d5/d1c/d23/d56/d6e/f10e x:0 0 0 2024-02-16T22:22:08.636 INFO:tasks.workunit.client.0.smithi114.stdout:1/604: symlink d4/d63/d69/d80/le2 0 2024-02-16T22:22:08.637 INFO:tasks.workunit.client.1.smithi123.stdout:6/814: creat d0/d6/d2f/da5/dc6/f125 x:0 0 0 2024-02-16T22:22:08.643 INFO:tasks.workunit.client.0.smithi114.stdout:0/657: creat d1/d16/d48/d98/fcd x:0 0 0 2024-02-16T22:22:08.643 INFO:tasks.workunit.client.1.smithi123.stdout:9/912: dwrite d4/fa [0,4194304] 0 2024-02-16T22:22:08.644 INFO:tasks.workunit.client.1.smithi123.stdout:9/913: fsync d4/d36/d8e/dff/f119 0 2024-02-16T22:22:08.644 INFO:tasks.workunit.client.1.smithi123.stdout:1/785: creat d3/d6/db/d10/d4b/d70/dd7/f107 x:0 0 0 2024-02-16T22:22:08.645 INFO:tasks.workunit.client.0.smithi114.stdout:0/658: truncate d1/d16/d1f/d28/d51/fb3 1152290 0 2024-02-16T22:22:08.646 INFO:tasks.workunit.client.1.smithi123.stdout:1/786: write d3/d6/db/d10/fc9 [1102072,96196] 0 2024-02-16T22:22:08.647 INFO:tasks.workunit.client.1.smithi123.stdout:0/715: rename d1/d8/d3d/l49 to d1/d8/da7/lf0 0 2024-02-16T22:22:08.651 INFO:tasks.workunit.client.0.smithi114.stdout:2/568: dread dd/df/f44 [0,4194304] 0 2024-02-16T22:22:08.651 INFO:tasks.workunit.client.1.smithi123.stdout:9/914: dwrite d4/d36/d8e/d58/d59/d60/deb/fc2 [0,4194304] 0 2024-02-16T22:22:08.652 INFO:tasks.workunit.client.1.smithi123.stdout:7/792: getdents d3/d87 0 2024-02-16T22:22:08.654 INFO:tasks.workunit.client.0.smithi114.stdout:1/605: write d4/d63/d69/f7f [1293847,90225] 0 2024-02-16T22:22:08.656 INFO:tasks.workunit.client.0.smithi114.stdout:0/659: rename d1/c14 to d1/d16/d1f/d28/d67/cce 0 2024-02-16T22:22:08.658 INFO:tasks.workunit.client.1.smithi123.stdout:4/877: dread d5/d1c/d5e/dc3/f4d [0,4194304] 0 2024-02-16T22:22:08.658 INFO:tasks.workunit.client.0.smithi114.stdout:0/660: fsync d1/d16/d1f/d28/d51/d61/d7e/d90/d95/fc9 0 2024-02-16T22:22:08.660 INFO:tasks.workunit.client.1.smithi123.stdout:5/790: symlink d6/dd/l117 0 2024-02-16T22:22:08.662 INFO:tasks.workunit.client.0.smithi114.stdout:1/606: truncate d4/d9/d3d/d3f/f5a 3087097 0 2024-02-16T22:22:08.663 INFO:tasks.workunit.client.1.smithi123.stdout:0/716: dread d1/f1f [0,4194304] 0 2024-02-16T22:22:08.663 INFO:tasks.workunit.client.1.smithi123.stdout:2/793: truncate f2 2979430 0 2024-02-16T22:22:08.664 INFO:tasks.workunit.client.0.smithi114.stdout:2/569: mkdir dd/d3b/da7/d8b/dc0 0 2024-02-16T22:22:08.666 INFO:tasks.workunit.client.0.smithi114.stdout:2/570: write dd/df/d1c/fa4 [480915,114335] 0 2024-02-16T22:22:08.666 INFO:tasks.workunit.client.0.smithi114.stdout:2/571: chown dd/f55 4992340 1 2024-02-16T22:22:08.667 INFO:tasks.workunit.client.0.smithi114.stdout:0/661: creat d1/d16/d1f/d28/d51/d61/d7e/d90/fcf x:0 0 0 2024-02-16T22:22:08.673 INFO:tasks.workunit.client.0.smithi114.stdout:2/572: unlink dd/d3b/da7/d4c/fa2 0 2024-02-16T22:22:08.675 INFO:tasks.workunit.client.1.smithi123.stdout:8/844: truncate d0/d61/d66/f86 905052 0 2024-02-16T22:22:08.684 INFO:tasks.workunit.client.1.smithi123.stdout:1/787: dwrite d3/d6/d49/f66 [0,4194304] 0 2024-02-16T22:22:08.684 INFO:tasks.workunit.client.1.smithi123.stdout:7/793: readlink d3/d6e/d1d/db2/d3e/l93 0 2024-02-16T22:22:08.686 INFO:tasks.workunit.client.0.smithi114.stdout:0/662: unlink d1/c32 0 2024-02-16T22:22:08.690 INFO:tasks.workunit.client.0.smithi114.stdout:2/573: creat dd/d6b/da5/fc1 x:0 0 0 2024-02-16T22:22:08.690 INFO:tasks.workunit.client.0.smithi114.stdout:1/607: dread d4/d9/d46/d67/d92/dca/dcc/fd5 [0,4194304] 0 2024-02-16T22:22:08.691 INFO:tasks.workunit.client.1.smithi123.stdout:9/915: unlink d4/d36/daa/d20/f7b 0 2024-02-16T22:22:08.692 INFO:tasks.workunit.client.1.smithi123.stdout:9/916: fdatasync d4/d36/d57/db2/fed 0 2024-02-16T22:22:08.693 INFO:tasks.workunit.client.1.smithi123.stdout:9/917: chown d4/d36/d8e/d58/d59/d60/deb/lb4 4185443 1 2024-02-16T22:22:08.696 INFO:tasks.workunit.client.1.smithi123.stdout:0/717: dread - d1/d8/f85 zero size 2024-02-16T22:22:08.697 INFO:tasks.workunit.client.1.smithi123.stdout:0/718: write d1/d8/d13/d1d/d64/f84 [5108251,85885] 0 2024-02-16T22:22:08.698 INFO:tasks.workunit.client.1.smithi123.stdout:0/719: stat d1/d4/d42/ld2 0 2024-02-16T22:22:08.699 INFO:tasks.workunit.client.1.smithi123.stdout:0/720: chown d1/d4/d42/ld2 135423612 1 2024-02-16T22:22:08.700 INFO:tasks.workunit.client.1.smithi123.stdout:3/834: getdents d0/d20/d29 0 2024-02-16T22:22:08.701 INFO:tasks.workunit.client.1.smithi123.stdout:8/845: write d0/d29/da7/d26/d2a/f92 [5397195,25979] 0 2024-02-16T22:22:08.706 INFO:tasks.workunit.client.1.smithi123.stdout:1/788: mkdir d3/d52/d108 0 2024-02-16T22:22:08.706 INFO:tasks.workunit.client.0.smithi114.stdout:2/574: write dd/d4e/f9c [800714,77133] 0 2024-02-16T22:22:08.707 INFO:tasks.workunit.client.1.smithi123.stdout:8/846: dwrite d0/d61/d66/d7e/ff1 [0,4194304] 0 2024-02-16T22:22:08.707 INFO:tasks.workunit.client.1.smithi123.stdout:1/789: fdatasync d3/d6/db/d55/f103 0 2024-02-16T22:22:08.718 INFO:tasks.workunit.client.0.smithi114.stdout:1/608: dread d4/d9/d46/d67/f6e [4194304,4194304] 0 2024-02-16T22:22:08.725 INFO:tasks.workunit.client.0.smithi114.stdout:1/609: creat d4/d63/d69/d7c/dc0/fe3 x:0 0 0 2024-02-16T22:22:08.728 INFO:tasks.workunit.client.0.smithi114.stdout:1/610: chown d4/d9/d46/l66 1028 1 2024-02-16T22:22:08.732 INFO:tasks.workunit.client.0.smithi114.stdout:1/611: mkdir d4/d9/d46/d67/d92/de4 0 2024-02-16T22:22:08.733 INFO:tasks.workunit.client.0.smithi114.stdout:1/612: readlink d4/d63/d6b/la1 0 2024-02-16T22:22:08.738 INFO:tasks.workunit.client.0.smithi114.stdout:2/575: write dd/df/d1c/d21/d61/d69/f72 [2524880,97983] 0 2024-02-16T22:22:08.744 INFO:tasks.workunit.client.0.smithi114.stdout:2/576: mkdir dd/df/d1c/d21/d61/d63/dc2 0 2024-02-16T22:22:08.748 INFO:tasks.workunit.client.0.smithi114.stdout:2/577: dread dd/df/d1c/d21/d87/f4d [0,4194304] 0 2024-02-16T22:22:08.756 INFO:tasks.workunit.client.0.smithi114.stdout:2/578: mkdir dd/df/d1c/d21/d61/d63/dc2/dc3 0 2024-02-16T22:22:08.756 INFO:tasks.workunit.client.0.smithi114.stdout:5/601: sync 2024-02-16T22:22:08.756 INFO:tasks.workunit.client.0.smithi114.stdout:4/731: sync 2024-02-16T22:22:08.756 INFO:tasks.workunit.client.0.smithi114.stdout:7/681: sync 2024-02-16T22:22:08.756 INFO:tasks.workunit.client.0.smithi114.stdout:6/632: sync 2024-02-16T22:22:08.762 INFO:tasks.workunit.client.0.smithi114.stdout:6/633: read - d6/da/faa zero size 2024-02-16T22:22:08.763 INFO:tasks.workunit.client.0.smithi114.stdout:1/613: getdents d4/d9/d46/d67/d92 0 2024-02-16T22:22:08.764 INFO:tasks.workunit.client.0.smithi114.stdout:1/614: readlink d4/d9/d46/l66 0 2024-02-16T22:22:08.771 INFO:tasks.workunit.client.0.smithi114.stdout:1/615: rmdir d4/d9/d46/d4d 39 2024-02-16T22:22:08.775 INFO:tasks.workunit.client.1.smithi123.stdout:7/794: rename d3/d5/d6c/c94 to d3/d5/d6c/da6/cfe 0 2024-02-16T22:22:08.775 INFO:tasks.workunit.client.0.smithi114.stdout:4/732: creat dd/d59/dfa/f10a x:0 0 0 2024-02-16T22:22:08.776 INFO:tasks.workunit.client.1.smithi123.stdout:7/795: fsync d3/d6e/d1d/db2/d3e/de4/ff7 0 2024-02-16T22:22:08.776 INFO:tasks.workunit.client.1.smithi123.stdout:4/878: mkdir d5/d1c/d23/d10f 0 2024-02-16T22:22:08.777 INFO:tasks.workunit.client.0.smithi114.stdout:4/733: read dd/d2e/d56/ddc/dde/f82 [1040033,82432] 0 2024-02-16T22:22:08.778 INFO:tasks.workunit.client.1.smithi123.stdout:4/879: read d5/d1c/d23/d31/f2d [2696465,29345] 0 2024-02-16T22:22:08.778 INFO:tasks.workunit.client.0.smithi114.stdout:6/634: mkdir d6/dd/d25/d34/d58/d9f/da3/dbd 0 2024-02-16T22:22:08.781 INFO:tasks.workunit.client.0.smithi114.stdout:1/616: mknod d4/d9/d46/d67/d92/dca/dcc/d98/ce5 0 2024-02-16T22:22:08.783 INFO:tasks.workunit.client.0.smithi114.stdout:2/579: link dd/df/c62 dd/df/d1c/d21/d61/cc4 0 2024-02-16T22:22:08.787 INFO:tasks.workunit.client.0.smithi114.stdout:6/635: creat d6/dd/fbe x:0 0 0 2024-02-16T22:22:08.789 INFO:tasks.workunit.client.0.smithi114.stdout:1/617: creat d4/d63/d69/d72/fe6 x:0 0 0 2024-02-16T22:22:08.791 INFO:tasks.workunit.client.0.smithi114.stdout:7/682: dwrite d3/d5a/fd7 [0,4194304] 0 2024-02-16T22:22:08.791 INFO:tasks.workunit.client.0.smithi114.stdout:2/580: symlink dd/d3b/da7/d7f/lc5 0 2024-02-16T22:22:08.793 INFO:tasks.workunit.client.0.smithi114.stdout:5/602: dwrite d0/df/f5d [0,4194304] 0 2024-02-16T22:22:08.794 INFO:tasks.workunit.client.0.smithi114.stdout:4/734: mknod dd/d1b/db0/de3/c10b 0 2024-02-16T22:22:08.799 INFO:tasks.workunit.client.1.smithi123.stdout:0/721: dread d1/d8/d13/f14 [0,4194304] 0 2024-02-16T22:22:08.801 INFO:tasks.workunit.client.0.smithi114.stdout:7/683: dwrite d3/d22/d61/d6a/fc9 [0,4194304] 0 2024-02-16T22:22:08.802 INFO:tasks.workunit.client.0.smithi114.stdout:7/684: dread - d3/d22/feb zero size 2024-02-16T22:22:08.808 INFO:tasks.workunit.client.1.smithi123.stdout:6/815: rmdir d0/d9/dc/d3a/d70/dd5 39 2024-02-16T22:22:08.808 INFO:tasks.workunit.client.1.smithi123.stdout:6/816: write d0/d74/f90 [3276385,57524] 0 2024-02-16T22:22:08.811 INFO:tasks.workunit.client.0.smithi114.stdout:2/581: creat dd/da6/fc6 x:0 0 0 2024-02-16T22:22:08.822 INFO:tasks.workunit.client.1.smithi123.stdout:9/918: mknod d4/d36/d8e/d58/d59/d60/dc1/c128 0 2024-02-16T22:22:08.824 INFO:tasks.workunit.client.1.smithi123.stdout:2/794: link d4/d1a/d1d/d38/d7b/c83 d4/d1a/d1d/d59/dd7/ddc/c104 0 2024-02-16T22:22:08.827 INFO:tasks.workunit.client.1.smithi123.stdout:3/835: unlink d0/d50/c51 0 2024-02-16T22:22:08.827 INFO:tasks.workunit.client.1.smithi123.stdout:7/796: write d3/d6e/d1d/fcb [987193,15673] 0 2024-02-16T22:22:08.828 INFO:tasks.workunit.client.1.smithi123.stdout:3/836: truncate d0/da0/ff2 115264 0 2024-02-16T22:22:08.828 INFO:tasks.workunit.client.0.smithi114.stdout:7/685: write d3/d14/f82 [3717348,69930] 0 2024-02-16T22:22:08.829 INFO:tasks.workunit.client.1.smithi123.stdout:0/722: fsync d1/d8/d13/d1d/d62/d7a/f90 0 2024-02-16T22:22:08.831 INFO:tasks.workunit.client.1.smithi123.stdout:6/817: symlink d0/d8a/d100/dfc/l126 0 2024-02-16T22:22:08.832 INFO:tasks.workunit.client.1.smithi123.stdout:6/818: dread - d0/d6/f124 zero size 2024-02-16T22:22:08.832 INFO:tasks.workunit.client.0.smithi114.stdout:6/636: truncate d6/dd/d25/d34/d58/f78 4743179 0 2024-02-16T22:22:08.836 INFO:tasks.workunit.client.1.smithi123.stdout:1/790: mkdir d3/d6/db/d109 0 2024-02-16T22:22:08.838 INFO:tasks.workunit.client.0.smithi114.stdout:2/582: dread dd/d3b/d8a/fba [0,4194304] 0 2024-02-16T22:22:08.839 INFO:tasks.workunit.client.0.smithi114.stdout:5/603: dread d0/d28/d98/d1e/d63/d96/d9a/db8/f74 [0,4194304] 0 2024-02-16T22:22:08.841 INFO:tasks.workunit.client.1.smithi123.stdout:8/847: link d0/d29/da7/d26/f5b d0/d61/f114 0 2024-02-16T22:22:08.841 INFO:tasks.workunit.client.0.smithi114.stdout:5/604: write d0/d28/d98/d25/d2e/f7d [4684789,17501] 0 2024-02-16T22:22:08.842 INFO:tasks.workunit.client.1.smithi123.stdout:2/795: dread d4/d1a/d1d/d38/d5e/dec/df1/d4a/faf [0,4194304] 0 2024-02-16T22:22:08.843 INFO:tasks.workunit.client.0.smithi114.stdout:6/637: creat d6/fbf x:0 0 0 2024-02-16T22:22:08.844 INFO:tasks.workunit.client.0.smithi114.stdout:4/735: getdents dd/d1b/d24/d57 0 2024-02-16T22:22:08.844 INFO:tasks.workunit.client.0.smithi114.stdout:6/638: write d6/d2c/fba [343536,21122] 0 2024-02-16T22:22:08.845 INFO:tasks.workunit.client.0.smithi114.stdout:6/639: stat d6/fbf 0 2024-02-16T22:22:08.847 INFO:tasks.workunit.client.0.smithi114.stdout:7/686: write d3/d22/d73/da7/f6b [3454617,29103] 0 2024-02-16T22:22:08.847 INFO:tasks.workunit.client.1.smithi123.stdout:5/791: getdents d6/dd/df/d58/d89/dc3 0 2024-02-16T22:22:08.849 INFO:tasks.workunit.client.0.smithi114.stdout:2/583: mkdir dd/d3b/da7/d4c/dc7 0 2024-02-16T22:22:08.852 INFO:tasks.workunit.client.0.smithi114.stdout:5/605: creat d0/d28/d98/d25/d2e/d3b/d8d/fc9 x:0 0 0 2024-02-16T22:22:08.853 INFO:tasks.workunit.client.1.smithi123.stdout:7/797: mknod d3/d5/d6c/cff 0 2024-02-16T22:22:08.855 INFO:tasks.workunit.client.1.smithi123.stdout:3/837: symlink d0/d40/dfb/l117 0 2024-02-16T22:22:08.856 INFO:tasks.workunit.client.1.smithi123.stdout:9/919: write d4/fa1 [3839575,8197] 0 2024-02-16T22:22:08.858 INFO:tasks.workunit.client.0.smithi114.stdout:7/687: creat d3/d46/da2/ff5 x:0 0 0 2024-02-16T22:22:08.859 INFO:tasks.workunit.client.1.smithi123.stdout:0/723: mkdir d1/d8/d13/df1 0 2024-02-16T22:22:08.859 INFO:tasks.workunit.client.0.smithi114.stdout:4/736: dwrite dd/f19 [0,4194304] 0 2024-02-16T22:22:08.860 INFO:tasks.workunit.client.0.smithi114.stdout:2/584: write dd/d3b/da7/d4c/f80 [1279672,110886] 0 2024-02-16T22:22:08.861 INFO:tasks.workunit.client.1.smithi123.stdout:6/819: creat d0/d6/d1a/d88/f127 x:0 0 0 2024-02-16T22:22:08.866 INFO:tasks.workunit.client.1.smithi123.stdout:2/796: creat d4/dae/f105 x:0 0 0 2024-02-16T22:22:08.866 INFO:tasks.workunit.client.1.smithi123.stdout:4/880: link d5/d1c/d5e/f72 d5/d1c/d5e/f110 0 2024-02-16T22:22:08.869 INFO:tasks.workunit.client.0.smithi114.stdout:4/737: truncate dd/d2e/d56/ddc/dde/fc4 666478 0 2024-02-16T22:22:08.870 INFO:tasks.workunit.client.1.smithi123.stdout:7/798: symlink d3/d6e/d34/l100 0 2024-02-16T22:22:08.871 INFO:tasks.workunit.client.0.smithi114.stdout:5/606: truncate d0/d28/d98/d25/d2e/d3b/f59 357321 0 2024-02-16T22:22:08.873 INFO:tasks.workunit.client.1.smithi123.stdout:3/838: write d0/d20/d29/d9b/dbd/df4/d5d/dae/db0/f104 [61936,15399] 0 2024-02-16T22:22:08.873 INFO:tasks.workunit.client.0.smithi114.stdout:5/607: fsync d0/d28/d98/d1e/d63/d96/d9a/fa6 0 2024-02-16T22:22:08.874 INFO:tasks.workunit.client.0.smithi114.stdout:5/608: chown d0/d28/d98/d25/d2e/d3b/d8d/cb4 70601 1 2024-02-16T22:22:08.875 INFO:tasks.workunit.client.0.smithi114.stdout:6/640: creat d6/dd/d25/fc0 x:0 0 0 2024-02-16T22:22:08.876 INFO:tasks.workunit.client.0.smithi114.stdout:5/609: truncate d0/d28/d98/d25/fc0 1038461 0 2024-02-16T22:22:08.877 INFO:tasks.workunit.client.0.smithi114.stdout:4/738: dread dd/d1b/d37/d61/f66 [0,4194304] 0 2024-02-16T22:22:08.883 INFO:tasks.workunit.client.0.smithi114.stdout:6/641: dwrite d6/dd/d25/d34/d61/d62/f8d [0,4194304] 0 2024-02-16T22:22:08.884 INFO:tasks.workunit.client.1.smithi123.stdout:9/920: dwrite d4/d36/fa8 [4194304,4194304] 0 2024-02-16T22:22:08.884 INFO:tasks.workunit.client.1.smithi123.stdout:1/791: dread d3/d6/d11/f17 [4194304,4194304] 0 2024-02-16T22:22:08.885 INFO:tasks.workunit.client.1.smithi123.stdout:5/792: mkdir d6/dd/df/d58/d118 0 2024-02-16T22:22:08.885 INFO:tasks.workunit.client.0.smithi114.stdout:7/688: dwrite d3/d14/f2c [0,4194304] 0 2024-02-16T22:22:08.886 INFO:tasks.workunit.client.0.smithi114.stdout:7/689: chown d3/d46/f7e 232 1 2024-02-16T22:22:08.887 INFO:tasks.workunit.client.0.smithi114.stdout:7/690: stat d3/d22/d73/da7/f5e 0 2024-02-16T22:22:08.890 INFO:tasks.workunit.client.1.smithi123.stdout:4/881: rename d5/d1c/d9a to d5/d1c/d23/d6d/d111 0 2024-02-16T22:22:08.895 INFO:tasks.workunit.client.0.smithi114.stdout:2/585: rename dd/df/d1c/d21/d61/d63/dc2/dc3 to dd/d3b/da7/dc8 0 2024-02-16T22:22:08.897 INFO:tasks.workunit.client.1.smithi123.stdout:4/882: dread d5/d1c/d23/d74/f55 [0,4194304] 0 2024-02-16T22:22:08.899 INFO:tasks.workunit.client.0.smithi114.stdout:5/610: symlink d0/df/d7a/dc7/lca 0 2024-02-16T22:22:08.899 INFO:tasks.workunit.client.1.smithi123.stdout:9/921: rmdir d4/d36/d8e/d58/d59/d60/dc1 39 2024-02-16T22:22:08.900 INFO:tasks.workunit.client.0.smithi114.stdout:7/691: mkdir d3/d22/df6 0 2024-02-16T22:22:08.903 INFO:tasks.workunit.client.1.smithi123.stdout:7/799: rename d3/d5/d6c/da6/ce3 to d3/d6e/d24/c101 0 2024-02-16T22:22:08.903 INFO:tasks.workunit.client.1.smithi123.stdout:2/797: dwrite d4/d1a/d1d/d13/d25/f9a [0,4194304] 0 2024-02-16T22:22:08.904 INFO:tasks.workunit.client.1.smithi123.stdout:7/800: write d3/d6e/d1d/db2/d3d/fed [278396,26704] 0 2024-02-16T22:22:08.906 INFO:tasks.workunit.client.0.smithi114.stdout:4/739: symlink dd/d2e/d56/l10c 0 2024-02-16T22:22:08.907 INFO:tasks.workunit.client.0.smithi114.stdout:5/611: dread - d0/f65 zero size 2024-02-16T22:22:08.909 INFO:tasks.workunit.client.0.smithi114.stdout:7/692: truncate d3/d22/f48 5163350 0 2024-02-16T22:22:08.913 INFO:tasks.workunit.client.0.smithi114.stdout:6/642: rename d6/dd/d25/d34/d61/d62/la1 to d6/dd/d25/d34/d58/d9f/da3/dbd/lc1 0 2024-02-16T22:22:08.914 INFO:tasks.workunit.client.1.smithi123.stdout:2/798: mknod d4/d1a/d1d/d13/d25/d72/dc8/c106 0 2024-02-16T22:22:08.917 INFO:tasks.workunit.client.1.smithi123.stdout:3/839: rename d0/l10 to d0/d40/l118 0 2024-02-16T22:22:08.917 INFO:tasks.workunit.client.1.smithi123.stdout:6/820: link d0/d9/dc/cc7 d0/d6/d91/dda/c128 0 2024-02-16T22:22:08.920 INFO:tasks.workunit.client.0.smithi114.stdout:4/740: rename dd/d1b/d37/ff5 to dd/d59/dfa/f10d 0 2024-02-16T22:22:08.922 INFO:tasks.workunit.client.1.smithi123.stdout:0/724: dread d1/d4/d17/d74/f52 [0,4194304] 0 2024-02-16T22:22:08.924 INFO:tasks.workunit.client.1.smithi123.stdout:0/725: write d1/d4/d17/d54/db4/dc5/fe0 [244253,14998] 0 2024-02-16T22:22:08.924 INFO:tasks.workunit.client.1.smithi123.stdout:5/793: rename d6/dd/df/d1f/f68 to d6/dd/df/d58/d118/f119 0 2024-02-16T22:22:08.925 INFO:tasks.workunit.client.1.smithi123.stdout:0/726: stat d1/d8/d13/d1d/lec 0 2024-02-16T22:22:08.926 INFO:tasks.workunit.client.1.smithi123.stdout:5/794: readlink d6/dd/df/d58/d3f/d6f/l87 0 2024-02-16T22:22:08.926 INFO:tasks.workunit.client.1.smithi123.stdout:3/840: creat d0/d40/dfb/f119 x:0 0 0 2024-02-16T22:22:08.929 INFO:tasks.workunit.client.1.smithi123.stdout:9/922: getdents d4/d36/d8e/d58/d59/d60/deb/da7 0 2024-02-16T22:22:08.930 INFO:tasks.workunit.client.1.smithi123.stdout:9/923: chown d4/d36/d8e/f95 118 1 2024-02-16T22:22:08.932 INFO:tasks.workunit.client.0.smithi114.stdout:5/612: dread d0/d28/d98/f79 [0,4194304] 0 2024-02-16T22:22:08.932 INFO:tasks.workunit.client.1.smithi123.stdout:6/821: mknod d0/d6/d1a/d88/d8b/c129 0 2024-02-16T22:22:08.932 INFO:tasks.workunit.client.1.smithi123.stdout:0/727: dwrite d1/d4/d17/d74/d57/f72 [0,4194304] 0 2024-02-16T22:22:08.933 INFO:tasks.workunit.client.0.smithi114.stdout:7/693: link d3/d22/d73/da7/d6f/d89/fb3 d3/d22/df6/ff7 0 2024-02-16T22:22:08.934 INFO:tasks.workunit.client.1.smithi123.stdout:2/799: mknod d4/d1a/d1d/d13/d25/c107 0 2024-02-16T22:22:08.935 INFO:tasks.workunit.client.0.smithi114.stdout:6/643: link d6/d2c/c31 d6/dd/d25/d34/d58/d9f/da3/cc2 0 2024-02-16T22:22:08.935 INFO:tasks.workunit.client.0.smithi114.stdout:6/644: chown d6/da/f56 5 1 2024-02-16T22:22:08.936 INFO:tasks.workunit.client.0.smithi114.stdout:6/645: readlink d6/d2c/l63 0 2024-02-16T22:22:08.937 INFO:tasks.workunit.client.1.smithi123.stdout:4/883: link d5/d1c/d23/c32 d5/df4/d108/dc0/dc5/c112 0 2024-02-16T22:22:08.940 INFO:tasks.workunit.client.1.smithi123.stdout:9/924: dwrite d4/fa1 [0,4194304] 0 2024-02-16T22:22:08.940 INFO:tasks.workunit.client.0.smithi114.stdout:4/741: link dd/d59/dfa/f10d dd/d1b/d41/d74/d9b/f10e 0 2024-02-16T22:22:08.942 INFO:tasks.workunit.client.0.smithi114.stdout:2/586: dwrite dd/d6b/f89 [0,4194304] 0 2024-02-16T22:22:08.943 INFO:tasks.workunit.client.1.smithi123.stdout:2/800: dwrite d4/fa1 [0,4194304] 0 2024-02-16T22:22:08.949 INFO:tasks.workunit.client.1.smithi123.stdout:1/792: dread d3/d6/db/d10/d4b/d70/d87/fd6 [0,4194304] 0 2024-02-16T22:22:08.951 INFO:tasks.workunit.client.0.smithi114.stdout:6/646: truncate d6/d2c/f95 415385 0 2024-02-16T22:22:08.952 INFO:tasks.workunit.client.0.smithi114.stdout:9/641: sync 2024-02-16T22:22:08.962 INFO:tasks.workunit.client.0.smithi114.stdout:2/587: creat dd/d3b/da7/d7f/fc9 x:0 0 0 2024-02-16T22:22:08.964 INFO:tasks.workunit.client.1.smithi123.stdout:0/728: symlink d1/d4/d17/d74/d57/lf2 0 2024-02-16T22:22:08.966 INFO:tasks.workunit.client.0.smithi114.stdout:6/647: mknod d6/cc3 0 2024-02-16T22:22:08.969 INFO:tasks.workunit.client.1.smithi123.stdout:3/841: write d0/fc8 [633870,98622] 0 2024-02-16T22:22:08.970 INFO:tasks.workunit.client.1.smithi123.stdout:9/925: creat d4/d36/d57/f129 x:0 0 0 2024-02-16T22:22:08.970 INFO:tasks.workunit.client.0.smithi114.stdout:4/742: truncate dd/d1b/d37/d38/d87/dad/fbf 12667 0 2024-02-16T22:22:08.971 INFO:tasks.workunit.client.1.smithi123.stdout:2/801: dread - d4/d1a/d1d/d13/d15/fb4 zero size 2024-02-16T22:22:08.972 INFO:tasks.workunit.client.0.smithi114.stdout:5/613: link d0/df/d53/db1/lbe d0/d28/d98/d1e/lcb 0 2024-02-16T22:22:08.973 INFO:tasks.workunit.client.0.smithi114.stdout:9/642: dread d2/f26 [0,4194304] 0 2024-02-16T22:22:08.975 INFO:tasks.workunit.client.1.smithi123.stdout:3/842: dwrite d0/f10b [0,4194304] 0 2024-02-16T22:22:08.977 INFO:tasks.workunit.client.1.smithi123.stdout:0/729: mknod d1/d4/d17/d74/cf3 0 2024-02-16T22:22:08.978 INFO:tasks.workunit.client.0.smithi114.stdout:5/614: rmdir d0/df/d7a/dc7 39 2024-02-16T22:22:08.983 INFO:tasks.workunit.client.0.smithi114.stdout:6/648: mknod d6/dd/d25/d34/d61/cc4 0 2024-02-16T22:22:08.984 INFO:tasks.workunit.client.1.smithi123.stdout:9/926: rmdir d4/d36/d8e 39 2024-02-16T22:22:08.986 INFO:tasks.workunit.client.1.smithi123.stdout:6/822: dwrite d0/d96/de3/f79 [0,4194304] 0 2024-02-16T22:22:08.986 INFO:tasks.workunit.client.0.smithi114.stdout:7/694: dread d3/d14/f43 [0,4194304] 0 2024-02-16T22:22:08.987 INFO:tasks.workunit.client.0.smithi114.stdout:5/615: dwrite d0/df/f8a [4194304,4194304] 0 2024-02-16T22:22:08.987 INFO:tasks.workunit.client.0.smithi114.stdout:7/695: chown d3/d22/d73/da7 78991023 1 2024-02-16T22:22:08.988 INFO:tasks.workunit.client.0.smithi114.stdout:7/696: chown d3/d22/l4f 3551 1 2024-02-16T22:22:08.989 INFO:tasks.workunit.client.0.smithi114.stdout:5/616: write d0/d28/d98/d1e/d86/f7c [3677060,46748] 0 2024-02-16T22:22:08.989 INFO:tasks.workunit.client.0.smithi114.stdout:2/588: creat dd/df/d1c/fca x:0 0 0 2024-02-16T22:22:08.993 INFO:tasks.workunit.client.0.smithi114.stdout:9/643: unlink d2/d1a/d23/d32/d46/f90 0 2024-02-16T22:22:08.994 INFO:tasks.workunit.client.0.smithi114.stdout:2/589: truncate dd/d3b/da7/d88/fb9 26693 0 2024-02-16T22:22:08.995 INFO:tasks.workunit.client.0.smithi114.stdout:2/590: chown dd/d4e/fb7 130355 1 2024-02-16T22:22:08.997 INFO:tasks.workunit.client.0.smithi114.stdout:2/591: write dd/df/f33 [140046,101751] 0 2024-02-16T22:22:08.997 INFO:tasks.workunit.client.1.smithi123.stdout:5/795: rename d6/f4c to d6/dd/df/d58/f11a 0 2024-02-16T22:22:08.997 INFO:tasks.workunit.client.1.smithi123.stdout:1/793: dread d3/d6/d18/d9e/fae [4194304,4194304] 0 2024-02-16T22:22:08.998 INFO:tasks.workunit.client.1.smithi123.stdout:8/848: sync 2024-02-16T22:22:09.000 INFO:tasks.workunit.client.1.smithi123.stdout:1/794: write d3/d6/db/d10/fc9 [1386521,67821] 0 2024-02-16T22:22:09.001 INFO:tasks.workunit.client.1.smithi123.stdout:0/730: fsync d1/d4/d17/d54/d8e/da5/fd3 0 2024-02-16T22:22:09.002 INFO:tasks.workunit.client.0.smithi114.stdout:3/647: sync 2024-02-16T22:22:09.002 INFO:tasks.workunit.client.0.smithi114.stdout:8/623: sync 2024-02-16T22:22:09.003 INFO:tasks.workunit.client.1.smithi123.stdout:0/731: chown d1/d8/d13/d1d/d64/dbc/fb3 963 1 2024-02-16T22:22:09.003 INFO:tasks.workunit.client.1.smithi123.stdout:4/884: link d5/d1c/d5e/dc3/f4d d5/d1c/d23/d47/f113 0 2024-02-16T22:22:09.003 INFO:tasks.workunit.client.1.smithi123.stdout:0/732: readlink d1/la2 0 2024-02-16T22:22:09.004 INFO:tasks.workunit.client.1.smithi123.stdout:0/733: write d1/d4/d17/d54/d8e/d96/f9e [8925,115763] 0 2024-02-16T22:22:09.004 INFO:tasks.workunit.client.0.smithi114.stdout:4/743: dwrite dd/d1b/d37/d38/d87/fe1 [0,4194304] 0 2024-02-16T22:22:09.005 INFO:tasks.workunit.client.1.smithi123.stdout:0/734: chown d1/la2 46522 1 2024-02-16T22:22:09.008 INFO:tasks.workunit.client.0.smithi114.stdout:6/649: dread d6/dd/d25/d34/f8e [0,4194304] 0 2024-02-16T22:22:09.009 INFO:tasks.workunit.client.0.smithi114.stdout:5/617: creat d0/d28/d98/d25/d89/d99/fcc x:0 0 0 2024-02-16T22:22:09.009 INFO:tasks.workunit.client.1.smithi123.stdout:4/885: dwrite d5/d1c/d23/d76/f10c [0,4194304] 0 2024-02-16T22:22:09.018 INFO:tasks.workunit.client.1.smithi123.stdout:1/795: mkdir d3/d6/db/d10/d2d/d8c/dfd/d9f/d10a 0 2024-02-16T22:22:09.021 INFO:tasks.workunit.client.0.smithi114.stdout:4/744: rename dd/d1b/d41/f83 to dd/d59/d8f/f10f 0 2024-02-16T22:22:09.035 INFO:tasks.workunit.client.1.smithi123.stdout:2/802: dread d4/d1a/f32 [0,4194304] 0 2024-02-16T22:22:09.037 INFO:tasks.workunit.client.1.smithi123.stdout:0/735: creat d1/d8/d3d/ff4 x:0 0 0 2024-02-16T22:22:09.040 INFO:tasks.workunit.client.1.smithi123.stdout:0/736: write d1/d4/d17/d54/d8e/fe6 [956582,96368] 0 2024-02-16T22:22:09.042 INFO:tasks.workunit.client.0.smithi114.stdout:6/650: rmdir d6/d2c/d66 39 2024-02-16T22:22:09.042 INFO:tasks.workunit.client.0.smithi114.stdout:9/644: mknod d2/d1a/d65/cce 0 2024-02-16T22:22:09.042 INFO:tasks.workunit.client.1.smithi123.stdout:2/803: dwrite d4/d1a/d1d/d59/dd7/ddc/ff9 [0,4194304] 0 2024-02-16T22:22:09.043 INFO:tasks.workunit.client.0.smithi114.stdout:4/745: chown dd/d1b/d24/d57/cd3 61408 1 2024-02-16T22:22:09.044 INFO:tasks.workunit.client.0.smithi114.stdout:9/645: dread - d2/d5c/dc9/d47/db2/fc3 zero size 2024-02-16T22:22:09.044 INFO:tasks.workunit.client.0.smithi114.stdout:9/646: chown d2/d1a/d23/f56 19 1 2024-02-16T22:22:09.057 INFO:tasks.workunit.client.0.smithi114.stdout:7/697: dwrite d3/d3d/f86 [0,4194304] 0 2024-02-16T22:22:09.058 INFO:tasks.workunit.client.1.smithi123.stdout:3/843: truncate d0/f10b 3822933 0 2024-02-16T22:22:09.058 INFO:tasks.workunit.client.0.smithi114.stdout:7/698: stat d3/d14/l53 0 2024-02-16T22:22:09.059 INFO:tasks.workunit.client.1.smithi123.stdout:2/804: dread d4/d1a/d1d/d13/d25/f77 [0,4194304] 0 2024-02-16T22:22:09.061 INFO:tasks.workunit.client.1.smithi123.stdout:9/927: dwrite d4/d33/f3b [0,4194304] 0 2024-02-16T22:22:09.061 INFO:tasks.workunit.client.1.smithi123.stdout:2/805: write d4/d1a/d1d/d13/d25/f77 [2235190,23092] 0 2024-02-16T22:22:09.062 INFO:tasks.workunit.client.1.smithi123.stdout:5/796: truncate d6/dd/df/d58/d3f/dbe/f5d 3532863 0 2024-02-16T22:22:09.063 INFO:tasks.workunit.client.0.smithi114.stdout:8/624: dwrite d4/d16/d25/d7c/f8d [0,4194304] 0 2024-02-16T22:22:09.065 INFO:tasks.workunit.client.0.smithi114.stdout:6/651: symlink d6/d2c/d5e/lc5 0 2024-02-16T22:22:09.065 INFO:tasks.workunit.client.1.smithi123.stdout:1/796: creat d3/d6/d49/d48/d84/d92/f10b x:0 0 0 2024-02-16T22:22:09.066 INFO:tasks.workunit.client.1.smithi123.stdout:6/823: creat d0/d6/d2f/f12a x:0 0 0 2024-02-16T22:22:09.072 INFO:tasks.workunit.client.1.smithi123.stdout:0/737: dread - d1/d4/d17/dad/fcd zero size 2024-02-16T22:22:09.073 INFO:tasks.workunit.client.1.smithi123.stdout:0/738: chown d1/d8/f85 925 1 2024-02-16T22:22:09.076 INFO:tasks.workunit.client.0.smithi114.stdout:5/618: link d0/d28/d98/d25/l6a d0/d28/d98/d25/d89/lcd 0 2024-02-16T22:22:09.078 INFO:tasks.workunit.client.0.smithi114.stdout:5/619: write d0/d28/d98/f79 [5809966,69318] 0 2024-02-16T22:22:09.078 INFO:tasks.workunit.client.0.smithi114.stdout:7/699: rmdir d3/d46/da2 39 2024-02-16T22:22:09.081 INFO:tasks.workunit.client.1.smithi123.stdout:7/801: sync 2024-02-16T22:22:09.081 INFO:tasks.workunit.client.0.smithi114.stdout:2/592: getdents dd/df 0 2024-02-16T22:22:09.083 INFO:tasks.workunit.client.0.smithi114.stdout:8/625: mknod d4/d11/d78/d7a/d36/d3b/ce1 0 2024-02-16T22:22:09.083 INFO:tasks.workunit.client.1.smithi123.stdout:5/797: dread - d6/dd/df/d58/f95 zero size 2024-02-16T22:22:09.083 INFO:tasks.workunit.client.0.smithi114.stdout:6/652: dread - d6/dd/d25/d34/d61/d62/d8b/f94 zero size 2024-02-16T22:22:09.086 INFO:tasks.workunit.client.1.smithi123.stdout:2/806: rename d4/d1a/d1d/d38/d7b/ff8 to d4/d1a/d1d/d13/d25/d72/dc8/f108 0 2024-02-16T22:22:09.086 INFO:tasks.workunit.client.0.smithi114.stdout:6/653: dread - d6/dd/fbe zero size 2024-02-16T22:22:09.087 INFO:tasks.workunit.client.1.smithi123.stdout:6/824: readlink d0/d9/dc/lc9 0 2024-02-16T22:22:09.088 INFO:tasks.workunit.client.0.smithi114.stdout:6/654: dread - d6/dd/d25/f53 zero size 2024-02-16T22:22:09.088 INFO:tasks.workunit.client.0.smithi114.stdout:7/700: dwrite d3/d87/fe4 [0,4194304] 0 2024-02-16T22:22:09.091 INFO:tasks.workunit.client.1.smithi123.stdout:1/797: mkdir d3/d6/db/d10/d2d/d8c/dfd/d9f/d10c 0 2024-02-16T22:22:09.091 INFO:tasks.workunit.client.0.smithi114.stdout:5/620: mknod d0/df/d53/db1/cce 0 2024-02-16T22:22:09.093 INFO:tasks.workunit.client.1.smithi123.stdout:8/849: getdents d0/d12/ded 0 2024-02-16T22:22:09.093 INFO:tasks.workunit.client.1.smithi123.stdout:4/886: sync 2024-02-16T22:22:09.096 INFO:tasks.workunit.client.0.smithi114.stdout:8/626: chown d4/d16/d25/d46/db5/dca 138 1 2024-02-16T22:22:09.096 INFO:tasks.workunit.client.0.smithi114.stdout:9/647: dwrite d2/dd/fc1 [0,4194304] 0 2024-02-16T22:22:09.101 INFO:tasks.workunit.client.0.smithi114.stdout:7/701: mkdir d3/d22/d73/da7/d6f/df8 0 2024-02-16T22:22:09.104 INFO:tasks.workunit.client.0.smithi114.stdout:2/593: fsync dd/df/d1c/f2e 0 2024-02-16T22:22:09.108 INFO:tasks.workunit.client.1.smithi123.stdout:3/844: dread d0/d20/d29/d9b/dbd/df4/d5d/dae/db0/fbe [0,4194304] 0 2024-02-16T22:22:09.109 INFO:tasks.workunit.client.1.smithi123.stdout:6/825: mknod d0/d96/de3/c12b 0 2024-02-16T22:22:09.111 INFO:tasks.workunit.client.0.smithi114.stdout:2/594: dwrite dd/d4e/f9c [0,4194304] 0 2024-02-16T22:22:09.113 INFO:tasks.workunit.client.1.smithi123.stdout:0/739: creat d1/d4/d17/d74/de7/ff5 x:0 0 0 2024-02-16T22:22:09.115 INFO:tasks.workunit.client.1.smithi123.stdout:0/740: rename d1/d4/d17/d54 to d1/d4/d17/d54/db4/dc5/df6 22 2024-02-16T22:22:09.115 INFO:tasks.workunit.client.1.smithi123.stdout:0/741: chown d1/d8/d13/d1d/fe1 79543091 1 2024-02-16T22:22:09.117 INFO:tasks.workunit.client.1.smithi123.stdout:1/798: creat d3/d6/db/d10/d4b/d70/d87/f10d x:0 0 0 2024-02-16T22:22:09.119 INFO:tasks.workunit.client.1.smithi123.stdout:1/799: write d3/d6/db/d55/f103 [360399,117777] 0 2024-02-16T22:22:09.120 INFO:tasks.workunit.client.0.smithi114.stdout:2/595: dwrite dd/df/d1c/d21/d61/d69/f72 [0,4194304] 0 2024-02-16T22:22:09.122 INFO:tasks.workunit.client.1.smithi123.stdout:8/850: creat d0/d12/f115 x:0 0 0 2024-02-16T22:22:09.124 INFO:tasks.workunit.client.0.smithi114.stdout:9/648: symlink d2/d1a/d23/d32/d46/lcf 0 2024-02-16T22:22:09.126 INFO:tasks.workunit.client.0.smithi114.stdout:2/596: dread dd/d6b/f6c [0,4194304] 0 2024-02-16T22:22:09.128 INFO:tasks.workunit.client.0.smithi114.stdout:2/597: fsync dd/d3b/da7/d4c/faf 0 2024-02-16T22:22:09.128 INFO:tasks.workunit.client.1.smithi123.stdout:4/887: unlink d5/d1c/fac 0 2024-02-16T22:22:09.131 INFO:tasks.workunit.client.1.smithi123.stdout:7/802: creat d3/d5/d6c/da6/df1/f102 x:0 0 0 2024-02-16T22:22:09.132 INFO:tasks.workunit.client.0.smithi114.stdout:4/746: write dd/d1b/d41/d74/f105 [914660,70379] 0 2024-02-16T22:22:09.133 INFO:tasks.workunit.client.1.smithi123.stdout:9/928: mknod d4/d36/d8e/d6d/d71/da6/d11c/c12a 0 2024-02-16T22:22:09.135 INFO:tasks.workunit.client.1.smithi123.stdout:3/845: dread d0/d20/d29/d9b/dbd/df4/d5d/d60/dd6/fd9 [0,4194304] 0 2024-02-16T22:22:09.136 INFO:tasks.workunit.client.0.smithi114.stdout:6/655: write d6/d2c/f6d [3952730,45806] 0 2024-02-16T22:22:09.139 INFO:tasks.workunit.client.0.smithi114.stdout:8/627: truncate d4/d11/d78/d7a/f6c 2840910 0 2024-02-16T22:22:09.139 INFO:tasks.workunit.client.1.smithi123.stdout:6/826: dread d0/d6/d8e/fb6 [0,4194304] 0 2024-02-16T22:22:09.140 INFO:tasks.workunit.client.0.smithi114.stdout:5/621: dread d0/d28/d98/d1e/d86/f7e [0,4194304] 0 2024-02-16T22:22:09.144 INFO:tasks.workunit.client.0.smithi114.stdout:1/618: sync 2024-02-16T22:22:09.144 INFO:tasks.workunit.client.0.smithi114.stdout:0/663: sync 2024-02-16T22:22:09.147 INFO:tasks.workunit.client.1.smithi123.stdout:8/851: creat d0/d29/da7/d26/d2a/d77/f116 x:0 0 0 2024-02-16T22:22:09.150 INFO:tasks.workunit.client.0.smithi114.stdout:7/702: dread d3/d22/d61/d6a/f7a [0,4194304] 0 2024-02-16T22:22:09.151 INFO:tasks.workunit.client.0.smithi114.stdout:4/747: truncate dd/d2e/d56/ddc/ff2 181520 0 2024-02-16T22:22:09.154 INFO:tasks.workunit.client.0.smithi114.stdout:6/656: dread d6/d2c/d38/d41/d60/f65 [0,4194304] 0 2024-02-16T22:22:09.154 INFO:tasks.workunit.client.1.smithi123.stdout:7/803: fsync d3/de0/fe5 0 2024-02-16T22:22:09.159 INFO:tasks.workunit.client.0.smithi114.stdout:8/628: dread d4/d11/d78/d7a/d7e/fae [0,4194304] 0 2024-02-16T22:22:09.160 INFO:tasks.workunit.client.0.smithi114.stdout:9/649: write d2/d5c/dc9/d47/f4d [3300076,114207] 0 2024-02-16T22:22:09.161 INFO:tasks.workunit.client.1.smithi123.stdout:3/846: mkdir d0/d20/d29/d11a 0 2024-02-16T22:22:09.161 INFO:tasks.workunit.client.0.smithi114.stdout:8/629: write d4/d16/d25/d39/f82 [831811,84861] 0 2024-02-16T22:22:09.163 INFO:tasks.workunit.client.1.smithi123.stdout:0/742: write d1/d8/d13/d1d/d62/d7a/f90 [446031,50106] 0 2024-02-16T22:22:09.163 INFO:tasks.workunit.client.0.smithi114.stdout:1/619: unlink d4/d63/d69/d7c/d9b/dae/cb6 0 2024-02-16T22:22:09.165 INFO:tasks.workunit.client.0.smithi114.stdout:1/620: truncate d4/d9/d3d/d3f/d79/fa3 148402 0 2024-02-16T22:22:09.166 INFO:tasks.workunit.client.0.smithi114.stdout:1/621: fdatasync d4/d63/d69/d72/d82/fb2 0 2024-02-16T22:22:09.167 INFO:tasks.workunit.client.0.smithi114.stdout:0/664: creat d1/d16/d1f/d28/d51/d60/d52/fd0 x:0 0 0 2024-02-16T22:22:09.171 INFO:tasks.workunit.client.0.smithi114.stdout:0/665: dread d1/fa [0,4194304] 0 2024-02-16T22:22:09.172 INFO:tasks.workunit.client.0.smithi114.stdout:0/666: chown d1/d16/d1f/d28/d51/d61/d6d/d79/c96 295 1 2024-02-16T22:22:09.173 INFO:tasks.workunit.client.0.smithi114.stdout:0/667: chown d1/d16/dbf 55 1 2024-02-16T22:22:09.174 INFO:tasks.workunit.client.0.smithi114.stdout:0/668: write d1/d16/f34 [1422873,65696] 0 2024-02-16T22:22:09.174 INFO:tasks.workunit.client.0.smithi114.stdout:0/669: stat d1/f7 0 2024-02-16T22:22:09.176 INFO:tasks.workunit.client.0.smithi114.stdout:0/670: read d1/d1a/d47/d62/d89/fbc [1857780,16043] 0 2024-02-16T22:22:09.176 INFO:tasks.workunit.client.0.smithi114.stdout:5/622: rename d0/d2d/d30/c45 to d0/d28/d98/d25/d89/d99/ccf 0 2024-02-16T22:22:09.177 INFO:tasks.workunit.client.1.smithi123.stdout:6/827: dwrite d0/d6/d8e/f9d [0,4194304] 0 2024-02-16T22:22:09.177 INFO:tasks.workunit.client.1.smithi123.stdout:1/800: mknod d3/d6/db/d10/d2d/d8c/dfd/d9f/d10c/c10e 0 2024-02-16T22:22:09.177 INFO:tasks.workunit.client.0.smithi114.stdout:5/623: stat d0/d28/d98/d25/d3e/d7f/f8e 0 2024-02-16T22:22:09.178 INFO:tasks.workunit.client.1.smithi123.stdout:1/801: chown d3/d6/db/d10/d4b/d70/dd7 71 1 2024-02-16T22:22:09.178 INFO:tasks.workunit.client.1.smithi123.stdout:4/888: mkdir d5/d1c/d23/d114 0 2024-02-16T22:22:09.180 INFO:tasks.workunit.client.1.smithi123.stdout:7/804: mkdir d3/d6e/d1d/db2/d3e/de4/d103 0 2024-02-16T22:22:09.181 INFO:tasks.workunit.client.0.smithi114.stdout:9/650: mkdir d2/dd0 0 2024-02-16T22:22:09.182 INFO:tasks.workunit.client.1.smithi123.stdout:5/798: link d6/dd/cd9 d6/dd/df/c11b 0 2024-02-16T22:22:09.185 INFO:tasks.workunit.client.0.smithi114.stdout:2/598: rmdir dd/d3b/da7/dc8 0 2024-02-16T22:22:09.188 INFO:tasks.workunit.client.1.smithi123.stdout:2/807: getdents d4/dd9 0 2024-02-16T22:22:09.191 INFO:tasks.workunit.client.1.smithi123.stdout:1/802: mkdir d3/d6/d49/d48/d10f 0 2024-02-16T22:22:09.193 INFO:tasks.workunit.client.0.smithi114.stdout:7/703: rename d3/d22/d61/f9d to d3/d22/d73/da7/d6f/df8/ff9 0 2024-02-16T22:22:09.194 INFO:tasks.workunit.client.1.smithi123.stdout:7/805: fdatasync d3/d5/fb8 0 2024-02-16T22:22:09.194 INFO:tasks.workunit.client.0.smithi114.stdout:7/704: read - d3/d22/d73/da7/d6f/fa3 zero size 2024-02-16T22:22:09.195 INFO:tasks.workunit.client.0.smithi114.stdout:5/624: rmdir d0/d28/d98/d1e 39 2024-02-16T22:22:09.196 INFO:tasks.workunit.client.0.smithi114.stdout:5/625: chown d0/df/d6d/laf 215995376 1 2024-02-16T22:22:09.198 INFO:tasks.workunit.client.0.smithi114.stdout:2/599: readlink dd/d3b/d8a/la8 0 2024-02-16T22:22:09.199 INFO:tasks.workunit.client.1.smithi123.stdout:9/929: link d4/d36/d8e/dbe/fef d4/d36/d8e/d6d/f12b 0 2024-02-16T22:22:09.199 INFO:tasks.workunit.client.0.smithi114.stdout:2/600: chown dd/df/d1c/d21/d61/d63/dc2 133 1 2024-02-16T22:22:09.200 INFO:tasks.workunit.client.0.smithi114.stdout:2/601: chown dd/d3b/da7/l49 0 1 2024-02-16T22:22:09.207 INFO:tasks.workunit.client.0.smithi114.stdout:0/671: truncate d1/f6c 1308639 0 2024-02-16T22:22:09.207 INFO:tasks.workunit.client.0.smithi114.stdout:6/657: truncate d6/dd/d25/d34/f42 2528930 0 2024-02-16T22:22:09.209 INFO:tasks.workunit.client.1.smithi123.stdout:2/808: mknod d4/d1a/d1d/d59/dd7/ddc/c109 0 2024-02-16T22:22:09.211 INFO:tasks.workunit.client.1.smithi123.stdout:1/803: symlink d3/d6/d49/d48/d68/l110 0 2024-02-16T22:22:09.217 INFO:tasks.workunit.client.1.smithi123.stdout:7/806: mkdir d3/d6e/d4d/d104 0 2024-02-16T22:22:09.219 INFO:tasks.workunit.client.1.smithi123.stdout:5/799: dwrite d6/dd/df/d58/fad [0,4194304] 0 2024-02-16T22:22:09.220 INFO:tasks.workunit.client.0.smithi114.stdout:1/622: dwrite d4/f30 [0,4194304] 0 2024-02-16T22:22:09.220 INFO:tasks.workunit.client.1.smithi123.stdout:5/800: chown d6/dd/df/d9f 24272 1 2024-02-16T22:22:09.221 INFO:tasks.workunit.client.1.smithi123.stdout:5/801: read d6/dc1/de6/f60 [1130433,1330] 0 2024-02-16T22:22:09.227 INFO:tasks.workunit.client.0.smithi114.stdout:6/658: mknod d6/d2c/d38/d41/d60/d84/cc6 0 2024-02-16T22:22:09.228 INFO:tasks.workunit.client.0.smithi114.stdout:7/705: write d3/d14/f5c [1619424,25125] 0 2024-02-16T22:22:09.230 INFO:tasks.workunit.client.0.smithi114.stdout:1/623: dwrite d4/d63/d69/d80/fb1 [0,4194304] 0 2024-02-16T22:22:09.231 INFO:tasks.workunit.client.0.smithi114.stdout:7/706: chown d3/d22/d73/da7/d4e/f6e 30641 1 2024-02-16T22:22:09.232 INFO:tasks.workunit.client.1.smithi123.stdout:0/743: link d1/d4/f11 d1/d8/d13/d1d/d64/ff7 0 2024-02-16T22:22:09.233 INFO:tasks.workunit.client.1.smithi123.stdout:8/852: sync 2024-02-16T22:22:09.237 INFO:tasks.workunit.client.1.smithi123.stdout:9/930: write d4/d36/d8e/d6d/d71/da6/ff0 [652032,13388] 0 2024-02-16T22:22:09.238 INFO:tasks.workunit.client.0.smithi114.stdout:5/626: mkdir d0/d28/d98/dd0 0 2024-02-16T22:22:09.238 INFO:tasks.workunit.client.0.smithi114.stdout:0/672: dwrite d1/d16/d18/f77 [0,4194304] 0 2024-02-16T22:22:09.242 INFO:tasks.workunit.client.1.smithi123.stdout:4/889: creat d5/d1c/f115 x:0 0 0 2024-02-16T22:22:09.244 INFO:tasks.workunit.client.1.smithi123.stdout:4/890: write d5/d1c/d23/d56/d6e/d73/d87/fe4 [5550338,43938] 0 2024-02-16T22:22:09.245 INFO:tasks.workunit.client.1.smithi123.stdout:7/807: creat d3/d6e/d1d/db2/d35/dba/f105 x:0 0 0 2024-02-16T22:22:09.248 INFO:tasks.workunit.client.0.smithi114.stdout:0/673: dwrite d1/d16/d1f/d28/d51/d61/d6d/f9c [0,4194304] 0 2024-02-16T22:22:09.251 INFO:tasks.workunit.client.1.smithi123.stdout:6/828: sync 2024-02-16T22:22:09.251 INFO:tasks.workunit.client.0.smithi114.stdout:9/651: rename d2/f8 to d2/d5c/d5f/fd1 0 2024-02-16T22:22:09.252 INFO:tasks.workunit.client.0.smithi114.stdout:9/652: chown d2/d5c/dc9/f86 350765 1 2024-02-16T22:22:09.253 INFO:tasks.workunit.client.1.smithi123.stdout:5/802: rename d6/dd/d44/dba/lec to d6/dd/df/d58/d3f/d82/l11c 0 2024-02-16T22:22:09.254 INFO:tasks.workunit.client.1.smithi123.stdout:5/803: chown d6/dd/df/d58/f9a 156577 1 2024-02-16T22:22:09.255 INFO:tasks.workunit.client.0.smithi114.stdout:1/624: rmdir d4/d9/d46/d67/d92/dca 39 2024-02-16T22:22:09.255 INFO:tasks.workunit.client.1.smithi123.stdout:2/809: write d4/d1a/f32 [4315312,128909] 0 2024-02-16T22:22:09.255 INFO:tasks.workunit.client.0.smithi114.stdout:7/707: fsync d3/d5a/fd7 0 2024-02-16T22:22:09.257 INFO:tasks.workunit.client.1.smithi123.stdout:1/804: write d3/d6/db/d10/d2d/f25 [5856834,4520] 0 2024-02-16T22:22:09.260 INFO:tasks.workunit.client.0.smithi114.stdout:5/627: read - d0/d28/d98/d25/d3e/d7f/f8e zero size 2024-02-16T22:22:09.266 INFO:tasks.workunit.client.1.smithi123.stdout:9/931: symlink d4/d1e/d3a/dac/dc5/l12c 0 2024-02-16T22:22:09.269 INFO:tasks.workunit.client.0.smithi114.stdout:5/628: mkdir d0/df/d5e/dd1 0 2024-02-16T22:22:09.270 INFO:tasks.workunit.client.0.smithi114.stdout:0/674: dwrite d1/d1a/d47/d62/d89/fbc [0,4194304] 0 2024-02-16T22:22:09.272 INFO:tasks.workunit.client.0.smithi114.stdout:6/659: dread d6/da/f56 [0,4194304] 0 2024-02-16T22:22:09.273 INFO:tasks.workunit.client.1.smithi123.stdout:7/808: read d3/d6e/d4d/f50 [1951989,107524] 0 2024-02-16T22:22:09.273 INFO:tasks.workunit.client.1.smithi123.stdout:4/891: dwrite d5/dba/fbf [0,4194304] 0 2024-02-16T22:22:09.273 INFO:tasks.workunit.client.1.smithi123.stdout:7/809: chown d3/d6e/d1d/db2/d35/f4b 384125043 1 2024-02-16T22:22:09.275 INFO:tasks.workunit.client.1.smithi123.stdout:3/847: truncate d0/d20/d29/d9b/dbd/df4/d5d/d60/dd6/fd9 1497912 0 2024-02-16T22:22:09.276 INFO:tasks.workunit.client.1.smithi123.stdout:6/829: rmdir d0/d8a/d100 39 2024-02-16T22:22:09.279 INFO:tasks.workunit.client.1.smithi123.stdout:2/810: chown d4/d1a/d1d/lb 435681 1 2024-02-16T22:22:09.282 INFO:tasks.workunit.client.0.smithi114.stdout:0/675: fdatasync d1/d16/d1f/d28/f54 0 2024-02-16T22:22:09.284 INFO:tasks.workunit.client.0.smithi114.stdout:0/676: chown d1/d1a/d47/c58 372624 1 2024-02-16T22:22:09.286 INFO:tasks.workunit.client.0.smithi114.stdout:8/630: rename d4/d11/c40 to d4/d11/d9d/ce2 0 2024-02-16T22:22:09.287 INFO:tasks.workunit.client.0.smithi114.stdout:0/677: write d1/d1a/f45 [7584473,20220] 0 2024-02-16T22:22:09.288 INFO:tasks.workunit.client.1.smithi123.stdout:8/853: creat d0/d29/da7/d26/d68/dbe/dbf/f117 x:0 0 0 2024-02-16T22:22:09.289 INFO:tasks.workunit.client.0.smithi114.stdout:9/653: creat d2/d1a/d23/d32/fd2 x:0 0 0 2024-02-16T22:22:09.290 INFO:tasks.workunit.client.0.smithi114.stdout:5/629: dwrite d0/d28/d98/d1e/d63/f68 [0,4194304] 0 2024-02-16T22:22:09.290 INFO:tasks.workunit.client.1.smithi123.stdout:9/932: mkdir d4/d36/d57/d12d 0 2024-02-16T22:22:09.290 INFO:tasks.workunit.client.1.smithi123.stdout:9/933: chown d4/d36/daa/d20/dcb/d10a 7864 1 2024-02-16T22:22:09.292 INFO:tasks.workunit.client.0.smithi114.stdout:0/678: write d1/d16/d1f/d28/d51/d61/d7e/d90/d95/d80/f8f [1503022,75906] 0 2024-02-16T22:22:09.298 INFO:tasks.workunit.client.0.smithi114.stdout:1/625: truncate d4/d63/d69/d7c/d9b/db0/f43 1686131 0 2024-02-16T22:22:09.300 INFO:tasks.workunit.client.1.smithi123.stdout:7/810: mkdir d3/d6e/d52/d106 0 2024-02-16T22:22:09.303 INFO:tasks.workunit.client.0.smithi114.stdout:2/602: rename dd/d3b/da7/d88 to dd/d3b/d8a/dcb 0 2024-02-16T22:22:09.304 INFO:tasks.workunit.client.1.smithi123.stdout:5/804: dwrite d6/dd/df/d9f/d102/fc0 [0,4194304] 0 2024-02-16T22:22:09.306 INFO:tasks.workunit.client.0.smithi114.stdout:8/631: fsync d4/d16/d25/d39/f4f 0 2024-02-16T22:22:09.306 INFO:tasks.workunit.client.1.smithi123.stdout:5/805: chown d6/dd/df/d58/d3f/d82/cf8 61819617 1 2024-02-16T22:22:09.308 INFO:tasks.workunit.client.0.smithi114.stdout:9/654: chown d2/d5c/d5f/fd1 343 1 2024-02-16T22:22:09.310 INFO:tasks.workunit.client.1.smithi123.stdout:2/811: chown d4/dae/df6/lfa 514609084 1 2024-02-16T22:22:09.313 INFO:tasks.workunit.client.1.smithi123.stdout:0/744: rmdir d1/d4/d42/ddc 0 2024-02-16T22:22:09.314 INFO:tasks.workunit.client.0.smithi114.stdout:6/660: write d6/dd/d25/d34/f42 [356465,42418] 0 2024-02-16T22:22:09.316 INFO:tasks.workunit.client.1.smithi123.stdout:1/805: creat d3/d6/db/d10/d2d/d8c/dfd/d9f/db4/ddf/f111 x:0 0 0 2024-02-16T22:22:09.317 INFO:tasks.workunit.client.1.smithi123.stdout:1/806: chown d3/d6/d11/fe4 10079801 1 2024-02-16T22:22:09.318 INFO:tasks.workunit.client.1.smithi123.stdout:1/807: stat d3/d6/db/d10/d2a/d80/cac 0 2024-02-16T22:22:09.322 INFO:tasks.workunit.client.1.smithi123.stdout:9/934: symlink d4/d36/daa/d7f/dea/l12e 0 2024-02-16T22:22:09.323 INFO:tasks.workunit.client.0.smithi114.stdout:1/626: chown d4/d9/d46/d4d/f4c 29446780 1 2024-02-16T22:22:09.325 INFO:tasks.workunit.client.1.smithi123.stdout:4/892: rename d5/d1c/d23/d56/d6e/d73/f88 to d5/d1c/d23/d74/f116 0 2024-02-16T22:22:09.327 INFO:tasks.workunit.client.0.smithi114.stdout:7/708: rename d3/d22/d73/da7/d6f/d89/c9a to d3/d22/db7/dea/cfa 0 2024-02-16T22:22:09.328 INFO:tasks.workunit.client.1.smithi123.stdout:7/811: fsync d3/d6e/d52/f8e 0 2024-02-16T22:22:09.328 INFO:tasks.workunit.client.0.smithi114.stdout:4/748: sync 2024-02-16T22:22:09.328 INFO:tasks.workunit.client.0.smithi114.stdout:3/648: sync 2024-02-16T22:22:09.329 INFO:tasks.workunit.client.0.smithi114.stdout:5/630: dwrite d0/d28/d98/d1e/f21 [0,4194304] 0 2024-02-16T22:22:09.331 INFO:tasks.workunit.client.0.smithi114.stdout:7/709: chown d3/d22/d73/da7/d6f/fd6 141 1 2024-02-16T22:22:09.334 INFO:tasks.workunit.client.0.smithi114.stdout:7/710: fsync d3/d22/db7/dea/ff4 0 2024-02-16T22:22:09.335 INFO:tasks.workunit.client.0.smithi114.stdout:7/711: stat d3/c19 0 2024-02-16T22:22:09.335 INFO:tasks.workunit.client.0.smithi114.stdout:7/712: stat d3/d22/l4f 0 2024-02-16T22:22:09.335 INFO:tasks.workunit.client.1.smithi123.stdout:6/830: truncate d0/d6/d8e/fc0 199912 0 2024-02-16T22:22:09.338 INFO:tasks.workunit.client.1.smithi123.stdout:5/806: dread d6/fb8 [0,4194304] 0 2024-02-16T22:22:09.338 INFO:tasks.workunit.client.0.smithi114.stdout:3/649: dwrite f1 [0,4194304] 0 2024-02-16T22:22:09.341 INFO:tasks.workunit.client.0.smithi114.stdout:0/679: dwrite d1/f7 [0,4194304] 0 2024-02-16T22:22:09.341 INFO:tasks.workunit.client.1.smithi123.stdout:3/848: sync 2024-02-16T22:22:09.344 INFO:tasks.workunit.client.1.smithi123.stdout:0/745: symlink d1/d4/d17/d74/d57/lf8 0 2024-02-16T22:22:09.346 INFO:tasks.workunit.client.0.smithi114.stdout:2/603: write dd/f3a [135186,117573] 0 2024-02-16T22:22:09.347 INFO:tasks.workunit.client.1.smithi123.stdout:8/854: mkdir d0/d61/dac/df4/d118 0 2024-02-16T22:22:09.349 INFO:tasks.workunit.client.1.smithi123.stdout:0/746: dread d1/d4/d17/d54/db4/dc5/fe0 [0,4194304] 0 2024-02-16T22:22:09.353 INFO:tasks.workunit.client.1.smithi123.stdout:1/808: rename d3/d6/d49/d48/d68/l110 to d3/d6/d49/d48/d84/db5/l112 0 2024-02-16T22:22:09.353 INFO:tasks.workunit.client.1.smithi123.stdout:1/809: chown l1 14559448 1 2024-02-16T22:22:09.354 INFO:tasks.workunit.client.1.smithi123.stdout:4/893: creat d5/d1c/d23/d47/da2/f117 x:0 0 0 2024-02-16T22:22:09.355 INFO:tasks.workunit.client.0.smithi114.stdout:6/661: dwrite d6/d83/f87 [0,4194304] 0 2024-02-16T22:22:09.355 INFO:tasks.workunit.client.0.smithi114.stdout:5/631: creat d0/df/d53/fd2 x:0 0 0 2024-02-16T22:22:09.356 INFO:tasks.workunit.client.0.smithi114.stdout:6/662: chown d6/fbf 265312 1 2024-02-16T22:22:09.356 INFO:tasks.workunit.client.1.smithi123.stdout:0/747: dwrite d1/d8/d13/d1d/d62/d7a/f90 [0,4194304] 0 2024-02-16T22:22:09.357 INFO:tasks.workunit.client.1.smithi123.stdout:7/812: creat d3/d6e/d1d/db2/d35/f107 x:0 0 0 2024-02-16T22:22:09.358 INFO:tasks.workunit.client.1.smithi123.stdout:6/831: creat d0/d74/f12c x:0 0 0 2024-02-16T22:22:09.359 INFO:tasks.workunit.client.1.smithi123.stdout:6/832: dread - d0/d6/d91/d93/d116/f11e zero size 2024-02-16T22:22:09.359 INFO:tasks.workunit.client.1.smithi123.stdout:2/812: mkdir d4/d1a/d1d/d59/dd7/d10a 0 2024-02-16T22:22:09.362 INFO:tasks.workunit.client.0.smithi114.stdout:7/713: chown d3/d3d/c42 64570 1 2024-02-16T22:22:09.367 INFO:tasks.workunit.client.0.smithi114.stdout:3/650: symlink d4/d46/dcf/ldf 0 2024-02-16T22:22:09.368 INFO:tasks.workunit.client.1.smithi123.stdout:9/935: mknod d4/d36/d8e/dff/d126/c12f 0 2024-02-16T22:22:09.368 INFO:tasks.workunit.client.0.smithi114.stdout:3/651: chown d4/d26/f58 0 1 2024-02-16T22:22:09.374 INFO:tasks.workunit.client.1.smithi123.stdout:1/810: creat d3/d6/db/d10/d4b/d70/d87/f113 x:0 0 0 2024-02-16T22:22:09.374 INFO:tasks.workunit.client.0.smithi114.stdout:1/627: creat d4/d9/d46/d67/d92/de4/fe7 x:0 0 0 2024-02-16T22:22:09.375 INFO:tasks.workunit.client.1.smithi123.stdout:1/811: chown d3/d6/d11/d76/fd0 11 1 2024-02-16T22:22:09.376 INFO:tasks.workunit.client.1.smithi123.stdout:7/813: creat d3/d6e/d1d/db2/d35/db7/f108 x:0 0 0 2024-02-16T22:22:09.376 INFO:tasks.workunit.client.1.smithi123.stdout:7/814: fdatasync d3/d6e/f2e 0 2024-02-16T22:22:09.378 INFO:tasks.workunit.client.1.smithi123.stdout:6/833: creat d0/d9/dc/d3a/d70/de5/f12d x:0 0 0 2024-02-16T22:22:09.382 INFO:tasks.workunit.client.1.smithi123.stdout:2/813: rename d4/d1a/d1d/d38/d5e/dec to d4/d1a/dd5/d10b 0 2024-02-16T22:22:09.382 INFO:tasks.workunit.client.0.smithi114.stdout:1/628: dread d4/d63/d69/d7c/f8b [0,4194304] 0 2024-02-16T22:22:09.382 INFO:tasks.workunit.client.1.smithi123.stdout:2/814: stat d4/dae/df6 0 2024-02-16T22:22:09.388 INFO:tasks.workunit.client.1.smithi123.stdout:4/894: rmdir d5/d1c/d23 39 2024-02-16T22:22:09.392 INFO:tasks.workunit.client.1.smithi123.stdout:8/855: dread d0/d12/f1e [0,4194304] 0 2024-02-16T22:22:09.393 INFO:tasks.workunit.client.1.smithi123.stdout:5/807: sync 2024-02-16T22:22:09.394 INFO:tasks.workunit.client.0.smithi114.stdout:8/632: rename d4/d16/l63 to d4/d16/d25/d46/le3 0 2024-02-16T22:22:09.396 INFO:tasks.workunit.client.1.smithi123.stdout:7/815: symlink d3/d6e/d1d/db2/d3e/de4/l109 0 2024-02-16T22:22:09.403 INFO:tasks.workunit.client.0.smithi114.stdout:9/655: dwrite d2/d5c/dc9/d1d/f62 [0,4194304] 0 2024-02-16T22:22:09.405 INFO:tasks.workunit.client.0.smithi114.stdout:1/629: creat d4/d63/d69/d7c/d9b/dae/fe8 x:0 0 0 2024-02-16T22:22:09.408 INFO:tasks.workunit.client.1.smithi123.stdout:0/748: dwrite d1/d8/d13/f37 [0,4194304] 0 2024-02-16T22:22:09.409 INFO:tasks.workunit.client.1.smithi123.stdout:6/834: dread d0/d9/f6a [0,4194304] 0 2024-02-16T22:22:09.413 INFO:tasks.workunit.client.1.smithi123.stdout:3/849: dwrite d0/f12 [0,4194304] 0 2024-02-16T22:22:09.415 INFO:tasks.workunit.client.1.smithi123.stdout:9/936: write d4/d1e/d3a/f40 [3133381,64581] 0 2024-02-16T22:22:09.415 INFO:tasks.workunit.client.0.smithi114.stdout:2/604: link dd/c6f dd/d6b/db8/ccc 0 2024-02-16T22:22:09.418 INFO:tasks.workunit.client.0.smithi114.stdout:6/663: dwrite d6/d2c/d5e/f67 [0,4194304] 0 2024-02-16T22:22:09.419 INFO:tasks.workunit.client.0.smithi114.stdout:6/664: chown d6/dd/d25/fc0 0 1 2024-02-16T22:22:09.420 INFO:tasks.workunit.client.0.smithi114.stdout:6/665: readlink d6/da/lb 0 2024-02-16T22:22:09.421 INFO:tasks.workunit.client.0.smithi114.stdout:7/714: rename d3/d22/d73/da7/l3e to d3/d22/d73/da7/d6f/d89/da6/lfb 0 2024-02-16T22:22:09.423 INFO:tasks.workunit.client.1.smithi123.stdout:9/937: dwrite d4/d36/daa/d20/ff7 [0,4194304] 0 2024-02-16T22:22:09.428 INFO:tasks.workunit.client.0.smithi114.stdout:8/633: mkdir d4/d11/d78/d7a/d36/d3b/de4 0 2024-02-16T22:22:09.429 INFO:tasks.workunit.client.1.smithi123.stdout:7/816: mkdir d3/d87/da8/d10a 0 2024-02-16T22:22:09.429 INFO:tasks.workunit.client.1.smithi123.stdout:2/815: write d4/d1a/d1d/f2c [1715130,89136] 0 2024-02-16T22:22:09.433 INFO:tasks.workunit.client.0.smithi114.stdout:7/715: dwrite d3/d22/d73/da7/d6f/d9c/fef [0,4194304] 0 2024-02-16T22:22:09.438 INFO:tasks.workunit.client.1.smithi123.stdout:1/812: truncate d3/d6/db/f28 769312 0 2024-02-16T22:22:09.439 INFO:tasks.workunit.client.0.smithi114.stdout:4/749: getdents dd 0 2024-02-16T22:22:09.439 INFO:tasks.workunit.client.0.smithi114.stdout:3/652: write d4/d24/f99 [837087,117362] 0 2024-02-16T22:22:09.441 INFO:tasks.workunit.client.0.smithi114.stdout:5/632: rmdir d0/df/d5e/dd1 0 2024-02-16T22:22:09.443 INFO:tasks.workunit.client.1.smithi123.stdout:6/835: creat d0/d6/d1a/d2e/f12e x:0 0 0 2024-02-16T22:22:09.443 INFO:tasks.workunit.client.1.smithi123.stdout:8/856: write d0/d29/da7/d26/fa4 [4539,67461] 0 2024-02-16T22:22:09.444 INFO:tasks.workunit.client.0.smithi114.stdout:9/656: fsync d2/dd/d3c/f3d 0 2024-02-16T22:22:09.447 INFO:tasks.workunit.client.0.smithi114.stdout:0/680: getdents d1/d16/d1f/d28/d51/d61/d6d 0 2024-02-16T22:22:09.448 INFO:tasks.workunit.client.0.smithi114.stdout:0/681: chown d1/d16/d1f/d28/d51/d60/d52 680 1 2024-02-16T22:22:09.450 INFO:tasks.workunit.client.0.smithi114.stdout:0/682: write d1/d16/d1f/d28/d51/d61/d7e/d90/d95/d80/f8f [1847834,15516] 0 2024-02-16T22:22:09.452 INFO:tasks.workunit.client.0.smithi114.stdout:0/683: dread - d1/d1a/d3e/fa3 zero size 2024-02-16T22:22:09.452 INFO:tasks.workunit.client.0.smithi114.stdout:6/666: mknod d6/d2c/d38/da8/cc7 0 2024-02-16T22:22:09.453 INFO:tasks.workunit.client.1.smithi123.stdout:2/816: mkdir d4/d1a/d1d/d13/d25/d98/d10c 0 2024-02-16T22:22:09.454 INFO:tasks.workunit.client.0.smithi114.stdout:8/634: creat d4/d16/fe5 x:0 0 0 2024-02-16T22:22:09.455 INFO:tasks.workunit.client.1.smithi123.stdout:2/817: read d4/d1a/d1d/d59/dd7/ddc/ff9 [3839421,69029] 0 2024-02-16T22:22:09.455 INFO:tasks.workunit.client.1.smithi123.stdout:0/749: dread d1/d8/d13/d1d/d62/d7a/f8f [0,4194304] 0 2024-02-16T22:22:09.455 INFO:tasks.workunit.client.1.smithi123.stdout:2/818: readlink d4/d1a/d1d/d13/d25/lef 0 2024-02-16T22:22:09.456 INFO:tasks.workunit.client.1.smithi123.stdout:4/895: truncate d5/d1c/d23/d47/f113 383380 0 2024-02-16T22:22:09.456 INFO:tasks.workunit.client.1.smithi123.stdout:4/896: chown d5/d1c 252228 1 2024-02-16T22:22:09.457 INFO:tasks.workunit.client.0.smithi114.stdout:8/635: chown d4/d16/d25/d7c/f8d 5116 1 2024-02-16T22:22:09.458 INFO:tasks.workunit.client.1.smithi123.stdout:3/850: creat d0/d20/d2e/dc4/d113/f11b x:0 0 0 2024-02-16T22:22:09.463 INFO:tasks.workunit.client.0.smithi114.stdout:5/633: symlink d0/d28/d98/d25/d3e/d7f/ld3 0 2024-02-16T22:22:09.465 INFO:tasks.workunit.client.0.smithi114.stdout:0/684: dwrite d1/d16/f19 [4194304,4194304] 0 2024-02-16T22:22:09.466 INFO:tasks.workunit.client.0.smithi114.stdout:5/634: dread - d0/d28/d98/d1e/d63/fbd zero size 2024-02-16T22:22:09.468 INFO:tasks.workunit.client.1.smithi123.stdout:5/808: dwrite d6/dd/df/d58/f63 [0,4194304] 0 2024-02-16T22:22:09.472 INFO:tasks.workunit.client.1.smithi123.stdout:1/813: mknod d3/d6/db/d10/d4b/d70/d87/d105/c114 0 2024-02-16T22:22:09.478 INFO:tasks.workunit.client.1.smithi123.stdout:9/938: write d4/d1e/fc8 [1974248,25056] 0 2024-02-16T22:22:09.479 INFO:tasks.workunit.client.1.smithi123.stdout:0/750: mknod d1/d4/d17/d74/cf9 0 2024-02-16T22:22:09.480 INFO:tasks.workunit.client.1.smithi123.stdout:0/751: dread - d1/d4/d17/d74/de7/ff5 zero size 2024-02-16T22:22:09.481 INFO:tasks.workunit.client.1.smithi123.stdout:2/819: chown d4/d1a/l2f 316991 1 2024-02-16T22:22:09.482 INFO:tasks.workunit.client.1.smithi123.stdout:2/820: truncate d4/dae/f105 295363 0 2024-02-16T22:22:09.483 INFO:tasks.workunit.client.0.smithi114.stdout:6/667: unlink d6/d2c/d75/cae 0 2024-02-16T22:22:09.485 INFO:tasks.workunit.client.1.smithi123.stdout:3/851: mkdir d0/d20/d29/d9b/dbd/df4/d5d/dae/db0/d11c 0 2024-02-16T22:22:09.486 INFO:tasks.workunit.client.0.smithi114.stdout:2/605: dwrite dd/f5d [0,4194304] 0 2024-02-16T22:22:09.491 INFO:tasks.workunit.client.0.smithi114.stdout:7/716: dread d3/d22/f6c [0,4194304] 0 2024-02-16T22:22:09.491 INFO:tasks.workunit.client.0.smithi114.stdout:7/717: fdatasync d3/d22/d73/da7/f6b 0 2024-02-16T22:22:09.499 INFO:tasks.workunit.client.0.smithi114.stdout:3/653: dwrite d4/dd/d33/f84 [0,4194304] 0 2024-02-16T22:22:09.508 INFO:tasks.workunit.client.1.smithi123.stdout:9/939: symlink d4/d36/d8e/d58/d59/d60/deb/da7/dde/dfd/l130 0 2024-02-16T22:22:09.512 INFO:tasks.workunit.client.0.smithi114.stdout:5/635: creat d0/d28/d98/d1e/d63/d96/fd4 x:0 0 0 2024-02-16T22:22:09.513 INFO:tasks.workunit.client.1.smithi123.stdout:0/752: creat d1/d8/d13/d1d/d64/dbc/ffa x:0 0 0 2024-02-16T22:22:09.515 INFO:tasks.workunit.client.0.smithi114.stdout:6/668: creat d6/d2c/d38/d41/d60/d84/fc8 x:0 0 0 2024-02-16T22:22:09.516 INFO:tasks.workunit.client.0.smithi114.stdout:4/750: write dd/fd8 [608803,54971] 0 2024-02-16T22:22:09.517 INFO:tasks.workunit.client.0.smithi114.stdout:6/669: chown d6/d2c/c43 1 1 2024-02-16T22:22:09.519 INFO:tasks.workunit.client.0.smithi114.stdout:0/685: write d1/d16/d18/f1c [7488252,121892] 0 2024-02-16T22:22:09.521 INFO:tasks.workunit.client.0.smithi114.stdout:0/686: chown d1/d16/d1f/d28/d51/dca 15140005 1 2024-02-16T22:22:09.522 INFO:tasks.workunit.client.0.smithi114.stdout:5/636: dread d0/d28/d98/f1f [0,4194304] 0 2024-02-16T22:22:09.522 INFO:tasks.workunit.client.0.smithi114.stdout:1/630: truncate d4/d9/d46/d67/d92/dca/dcc/d98/fba 2592009 0 2024-02-16T22:22:09.524 INFO:tasks.workunit.client.1.smithi123.stdout:8/857: rename d0/d12/f115 to d0/d29/da7/d26/d2a/f119 0 2024-02-16T22:22:09.524 INFO:tasks.workunit.client.0.smithi114.stdout:1/631: chown d4/d9/d46/d67/d92/dca/dce/dd3 140 1 2024-02-16T22:22:09.525 INFO:tasks.workunit.client.1.smithi123.stdout:5/809: dwrite d6/dd/df/d58/d89/d98/fc8 [0,4194304] 0 2024-02-16T22:22:09.526 INFO:tasks.workunit.client.1.smithi123.stdout:3/852: dwrite d0/d20/d29/f3c [0,4194304] 0 2024-02-16T22:22:09.526 INFO:tasks.workunit.client.0.smithi114.stdout:5/637: truncate d0/d28/d98/d25/d2e/f7d 5165365 0 2024-02-16T22:22:09.526 INFO:tasks.workunit.client.1.smithi123.stdout:3/853: fsync d0/d40/dfb/ffc 0 2024-02-16T22:22:09.528 INFO:tasks.workunit.client.0.smithi114.stdout:7/718: rmdir d3/d22/d73/da7/db4 39 2024-02-16T22:22:09.530 INFO:tasks.workunit.client.1.smithi123.stdout:7/817: getdents d3/d5 0 2024-02-16T22:22:09.530 INFO:tasks.workunit.client.1.smithi123.stdout:7/818: write d3/d6e/f2e [1297695,43973] 0 2024-02-16T22:22:09.534 INFO:tasks.workunit.client.0.smithi114.stdout:9/657: creat d2/d5c/dc9/d1d/fd3 x:0 0 0 2024-02-16T22:22:09.542 INFO:tasks.workunit.client.1.smithi123.stdout:6/836: getdents d0/d9/dc/d3a/d70/de5 0 2024-02-16T22:22:09.543 INFO:tasks.workunit.client.0.smithi114.stdout:5/638: dread d0/d28/d98/d1e/d63/d96/d9a/fa6 [0,4194304] 0 2024-02-16T22:22:09.547 INFO:tasks.workunit.client.1.smithi123.stdout:4/897: rename d5/d1c/d23/d56/d6e to d5/df4/d108/dc0/d118 0 2024-02-16T22:22:09.548 INFO:tasks.workunit.client.1.smithi123.stdout:4/898: stat d5/f40 0 2024-02-16T22:22:09.548 INFO:tasks.workunit.client.1.smithi123.stdout:1/814: sync 2024-02-16T22:22:09.548 INFO:tasks.workunit.client.1.smithi123.stdout:4/899: readlink d5/d1c/d23/d31/lc8 0 2024-02-16T22:22:09.549 INFO:tasks.workunit.client.1.smithi123.stdout:1/815: stat d3/d6/d11/c34 0 2024-02-16T22:22:09.557 INFO:tasks.workunit.client.1.smithi123.stdout:9/940: getdents d4/d36/daa/d2b/de1/df1 0 2024-02-16T22:22:09.558 INFO:tasks.workunit.client.1.smithi123.stdout:9/941: chown d4/d36/d57/f121 1181 1 2024-02-16T22:22:09.558 INFO:tasks.workunit.client.1.smithi123.stdout:9/942: readlink d4/d36/daa/d20/l2e 0 2024-02-16T22:22:09.559 INFO:tasks.workunit.client.1.smithi123.stdout:9/943: readlink d4/d36/d8e/d6f/d7e/lb9 0 2024-02-16T22:22:09.559 INFO:tasks.workunit.client.1.smithi123.stdout:2/821: getdents d4/d1a/dd5/d10b/df1/db1 0 2024-02-16T22:22:09.559 INFO:tasks.workunit.client.1.smithi123.stdout:9/944: stat d4/d36/daa/d20/l2e 0 2024-02-16T22:22:09.560 INFO:tasks.workunit.client.1.smithi123.stdout:2/822: stat d4/d1a/d1d/d13/d25/c61 0 2024-02-16T22:22:09.561 INFO:tasks.workunit.client.1.smithi123.stdout:6/837: mkdir d0/d9/dc/d3a/d70/de5/d12f 0 2024-02-16T22:22:09.564 INFO:tasks.workunit.client.1.smithi123.stdout:4/900: mknod d5/d1c/d23/d74/d92/d9b/c119 0 2024-02-16T22:22:09.566 INFO:tasks.workunit.client.1.smithi123.stdout:5/810: write d6/dd/d44/fbd [35690,117677] 0 2024-02-16T22:22:09.571 INFO:tasks.workunit.client.1.smithi123.stdout:3/854: unlink d0/d20/d29/d9b/dbd/df4/fa7 0 2024-02-16T22:22:09.571 INFO:tasks.workunit.client.1.smithi123.stdout:7/819: symlink d3/d6e/d1d/d97/dca/l10b 0 2024-02-16T22:22:09.573 INFO:tasks.workunit.client.1.smithi123.stdout:9/945: symlink d4/d36/d8e/d58/d59/d60/d101/l131 0 2024-02-16T22:22:09.575 INFO:tasks.workunit.client.1.smithi123.stdout:2/823: mkdir d4/d1a/d1d/d13/d25/d98/dbb/d10d 0 2024-02-16T22:22:09.579 INFO:tasks.workunit.client.1.smithi123.stdout:8/858: creat d0/d29/da7/d26/f11a x:0 0 0 2024-02-16T22:22:09.579 INFO:tasks.workunit.client.1.smithi123.stdout:2/824: dwrite d4/d1a/fe5 [0,4194304] 0 2024-02-16T22:22:09.581 INFO:tasks.workunit.client.1.smithi123.stdout:6/838: dread d0/d9/dc/f23 [0,4194304] 0 2024-02-16T22:22:09.584 INFO:tasks.workunit.client.1.smithi123.stdout:4/901: mknod d5/d1c/d64/c11a 0 2024-02-16T22:22:09.585 INFO:tasks.workunit.client.1.smithi123.stdout:1/816: write d3/d6/f5b [1991787,17956] 0 2024-02-16T22:22:09.586 INFO:tasks.workunit.client.0.smithi114.stdout:4/751: dread - dd/d59/d8f/fa0 zero size 2024-02-16T22:22:09.588 INFO:tasks.workunit.client.1.smithi123.stdout:5/811: rmdir d6/dac 39 2024-02-16T22:22:09.589 INFO:tasks.workunit.client.1.smithi123.stdout:5/812: chown d6/dd/df/c1e 605047 1 2024-02-16T22:22:09.590 INFO:tasks.workunit.client.0.smithi114.stdout:6/670: fsync d6/d2c/d38/d54/f5f 0 2024-02-16T22:22:09.592 INFO:tasks.workunit.client.0.smithi114.stdout:8/636: creat d4/d16/d25/fe6 x:0 0 0 2024-02-16T22:22:09.592 INFO:tasks.workunit.client.0.smithi114.stdout:6/671: dread - d6/dd/d25/fc0 zero size 2024-02-16T22:22:09.593 INFO:tasks.workunit.client.1.smithi123.stdout:0/753: getdents d1/d8/d13/d1d/d62 0 2024-02-16T22:22:09.594 INFO:tasks.workunit.client.0.smithi114.stdout:0/687: mkdir d1/d16/d1f/d28/d51/d60/dd1 0 2024-02-16T22:22:09.594 INFO:tasks.workunit.client.0.smithi114.stdout:4/752: dwrite dd/d1b/d41/dc3/ff3 [0,4194304] 0 2024-02-16T22:22:09.595 INFO:tasks.workunit.client.1.smithi123.stdout:9/946: unlink d4/d36/d8e/dbe/fe3 0 2024-02-16T22:22:09.597 INFO:tasks.workunit.client.1.smithi123.stdout:5/813: dwrite d6/f8 [0,4194304] 0 2024-02-16T22:22:09.597 INFO:tasks.workunit.client.0.smithi114.stdout:7/719: mkdir d3/d22/d73/da7/d6f/d89/da6/dfc 0 2024-02-16T22:22:09.598 INFO:tasks.workunit.client.1.smithi123.stdout:3/855: dwrite d0/d20/d29/d9b/dbd/df4/f36 [4194304,4194304] 0 2024-02-16T22:22:09.601 INFO:tasks.workunit.client.0.smithi114.stdout:0/688: dwrite d1/d1a/d47/d62/d7f/f84 [0,4194304] 0 2024-02-16T22:22:09.604 INFO:tasks.workunit.client.1.smithi123.stdout:8/859: mknod d0/d29/da7/d26/d2a/d77/df8/d62/c11b 0 2024-02-16T22:22:09.606 INFO:tasks.workunit.client.1.smithi123.stdout:6/839: chown d0/d9/dc/cd7 160011 1 2024-02-16T22:22:09.607 INFO:tasks.workunit.client.0.smithi114.stdout:0/689: dread d1/f7 [0,4194304] 0 2024-02-16T22:22:09.609 INFO:tasks.workunit.client.1.smithi123.stdout:4/902: dread - d5/d1c/d23/d31/f83 zero size 2024-02-16T22:22:09.614 INFO:tasks.workunit.client.1.smithi123.stdout:0/754: dread d1/d4/d17/d54/d8e/fe6 [0,4194304] 0 2024-02-16T22:22:09.615 INFO:tasks.workunit.client.1.smithi123.stdout:0/755: chown d1/d8/d13/l86 6099319 1 2024-02-16T22:22:09.616 INFO:tasks.workunit.client.0.smithi114.stdout:6/672: mkdir d6/d2c/d38/d41/d60/d84/dc9 0 2024-02-16T22:22:09.621 INFO:tasks.workunit.client.1.smithi123.stdout:2/825: mknod d4/d1a/d1d/d59/d45/c10e 0 2024-02-16T22:22:09.623 INFO:tasks.workunit.client.0.smithi114.stdout:2/606: rename dd/df/d1c/f85 to dd/df/d1c/d21/fcd 0 2024-02-16T22:22:09.625 INFO:tasks.workunit.client.0.smithi114.stdout:0/690: symlink d1/d1a/d47/d62/d89/ld2 0 2024-02-16T22:22:09.626 INFO:tasks.workunit.client.1.smithi123.stdout:8/860: creat d0/df/dca/f11c x:0 0 0 2024-02-16T22:22:09.627 INFO:tasks.workunit.client.0.smithi114.stdout:0/691: chown d1/d16/d18/f59 5189978 1 2024-02-16T22:22:09.628 INFO:tasks.workunit.client.0.smithi114.stdout:0/692: readlink d1/d1a/d47/l5a 0 2024-02-16T22:22:09.630 INFO:tasks.workunit.client.1.smithi123.stdout:6/840: unlink d0/d96/de3/f6c 0 2024-02-16T22:22:09.631 INFO:tasks.workunit.client.0.smithi114.stdout:2/607: dwrite dd/d3b/da7/d7f/fc9 [0,4194304] 0 2024-02-16T22:22:09.631 INFO:tasks.workunit.client.1.smithi123.stdout:6/841: chown d0/d9/dc/d3a/d70/de5/c103 2 1 2024-02-16T22:22:09.631 INFO:tasks.workunit.client.0.smithi114.stdout:2/608: stat dd/d3b/da7/fa3 0 2024-02-16T22:22:09.633 INFO:tasks.workunit.client.1.smithi123.stdout:4/903: creat d5/d1c/d64/f11b x:0 0 0 2024-02-16T22:22:09.633 INFO:tasks.workunit.client.0.smithi114.stdout:2/609: chown dd/d3b/da7/d7f/fc9 14866 1 2024-02-16T22:22:09.634 INFO:tasks.workunit.client.0.smithi114.stdout:2/610: fsync dd/f3a 0 2024-02-16T22:22:09.634 INFO:tasks.workunit.client.0.smithi114.stdout:7/720: mknod d3/d22/d73/da7/db4/cfd 0 2024-02-16T22:22:09.635 INFO:tasks.workunit.client.1.smithi123.stdout:1/817: mknod d3/d6/db/d109/c115 0 2024-02-16T22:22:09.635 INFO:tasks.workunit.client.1.smithi123.stdout:1/818: stat d3/d6/db/d10/fde 0 2024-02-16T22:22:09.636 INFO:tasks.workunit.client.0.smithi114.stdout:5/639: write d0/d28/d98/d25/d2e/f7b [195521,110051] 0 2024-02-16T22:22:09.637 INFO:tasks.workunit.client.1.smithi123.stdout:1/819: stat d3/d6/db/d10/c2e 0 2024-02-16T22:22:09.639 INFO:tasks.workunit.client.1.smithi123.stdout:7/820: creat d3/d6e/d1d/f10c x:0 0 0 2024-02-16T22:22:09.641 INFO:tasks.workunit.client.0.smithi114.stdout:0/693: creat d1/d78/fd3 x:0 0 0 2024-02-16T22:22:09.642 INFO:tasks.workunit.client.1.smithi123.stdout:1/820: dread d3/d6/d49/f66 [0,4194304] 0 2024-02-16T22:22:09.643 INFO:tasks.workunit.client.1.smithi123.stdout:5/814: mknod d6/dd/dbb/c11d 0 2024-02-16T22:22:09.647 INFO:tasks.workunit.client.1.smithi123.stdout:1/821: read d3/d6/d11/f17 [2078525,23850] 0 2024-02-16T22:22:09.648 INFO:tasks.workunit.client.1.smithi123.stdout:3/856: symlink d0/d20/d5a/l11d 0 2024-02-16T22:22:09.648 INFO:tasks.workunit.client.1.smithi123.stdout:9/947: write d4/d36/d57/db2/f69 [845300,42116] 0 2024-02-16T22:22:09.649 INFO:tasks.workunit.client.0.smithi114.stdout:8/637: dwrite d4/d24/d2c/d5f/f93 [0,4194304] 0 2024-02-16T22:22:09.649 INFO:tasks.workunit.client.0.smithi114.stdout:2/611: fdatasync dd/d3b/da7/d4c/fbe 0 2024-02-16T22:22:09.653 INFO:tasks.workunit.client.0.smithi114.stdout:3/654: rename d4/d24/d2d/d45/d68/cbc to d4/d24/d2d/d45/d74/ce0 0 2024-02-16T22:22:09.656 INFO:tasks.workunit.client.0.smithi114.stdout:3/655: write d4/d46/d48/dcc/fc6 [681588,51160] 0 2024-02-16T22:22:09.657 INFO:tasks.workunit.client.0.smithi114.stdout:3/656: chown d4/d24/d93/fb8 413 1 2024-02-16T22:22:09.661 INFO:tasks.workunit.client.1.smithi123.stdout:4/904: mknod d5/db8/c11c 0 2024-02-16T22:22:09.665 INFO:tasks.workunit.client.0.smithi114.stdout:6/673: dwrite d6/da/f9b [0,4194304] 0 2024-02-16T22:22:09.668 INFO:tasks.workunit.client.0.smithi114.stdout:5/640: dwrite d0/d28/d98/d1e/d86/f92 [0,4194304] 0 2024-02-16T22:22:09.668 INFO:tasks.workunit.client.0.smithi114.stdout:2/612: creat dd/df/d1c/d21/d61/d69/fce x:0 0 0 2024-02-16T22:22:09.670 INFO:tasks.workunit.client.1.smithi123.stdout:5/815: creat d6/dd/df/d58/d3f/dbe/da2/f11e x:0 0 0 2024-02-16T22:22:09.672 INFO:tasks.workunit.client.1.smithi123.stdout:2/826: unlink d4/d1a/f9b 0 2024-02-16T22:22:09.672 INFO:tasks.workunit.client.1.smithi123.stdout:2/827: readlink d4/d1a/d1d/l49 0 2024-02-16T22:22:09.673 INFO:tasks.workunit.client.1.smithi123.stdout:5/816: read d6/dd/df/d58/d56/f71 [4004822,42018] 0 2024-02-16T22:22:09.675 INFO:tasks.workunit.client.0.smithi114.stdout:1/632: rename d4/d9/d46/d47/f94 to d4/d63/d69/d7c/dc0/fe9 0 2024-02-16T22:22:09.676 INFO:tasks.workunit.client.1.smithi123.stdout:2/828: dread d4/d1a/fe5 [0,4194304] 0 2024-02-16T22:22:09.678 INFO:tasks.workunit.client.1.smithi123.stdout:3/857: dread - d0/d20/d29/f84 zero size 2024-02-16T22:22:09.679 INFO:tasks.workunit.client.0.smithi114.stdout:8/638: dwrite d4/d11/d78/d7a/d31/fa5 [0,4194304] 0 2024-02-16T22:22:09.680 INFO:tasks.workunit.client.0.smithi114.stdout:3/657: fdatasync d4/d46/d48/d75/f9a 0 2024-02-16T22:22:09.681 INFO:tasks.workunit.client.0.smithi114.stdout:0/694: dread d1/d16/f5d [0,4194304] 0 2024-02-16T22:22:09.683 INFO:tasks.workunit.client.1.smithi123.stdout:0/756: write d1/d4/d42/f7e [675534,30810] 0 2024-02-16T22:22:09.684 INFO:tasks.workunit.client.1.smithi123.stdout:2/829: dread d4/d1a/d1d/d38/d7b/f94 [0,4194304] 0 2024-02-16T22:22:09.685 INFO:tasks.workunit.client.1.smithi123.stdout:8/861: mknod d0/d61/dac/dd0/c11d 0 2024-02-16T22:22:09.687 INFO:tasks.workunit.client.1.smithi123.stdout:6/842: fsync d0/d6/d2f/d5d/f68 0 2024-02-16T22:22:09.689 INFO:tasks.workunit.client.0.smithi114.stdout:6/674: write d6/f1e [2757972,91668] 0 2024-02-16T22:22:09.689 INFO:tasks.workunit.client.1.smithi123.stdout:4/905: rmdir d5/df4/d108 39 2024-02-16T22:22:09.691 INFO:tasks.workunit.client.0.smithi114.stdout:7/721: creat d3/d46/ffe x:0 0 0 2024-02-16T22:22:09.692 INFO:tasks.workunit.client.0.smithi114.stdout:7/722: read d3/d22/f6c [1161290,38899] 0 2024-02-16T22:22:09.695 INFO:tasks.workunit.client.1.smithi123.stdout:4/906: read d5/f1f [2749130,34023] 0 2024-02-16T22:22:09.696 INFO:tasks.workunit.client.0.smithi114.stdout:7/723: truncate d3/d22/feb 588557 0 2024-02-16T22:22:09.700 INFO:tasks.workunit.client.0.smithi114.stdout:6/675: dwrite d6/dd/d25/d34/d61/d62/d8b/fbb [0,4194304] 0 2024-02-16T22:22:09.704 INFO:tasks.workunit.client.1.smithi123.stdout:6/843: dread d0/d74/f7b [0,4194304] 0 2024-02-16T22:22:09.704 INFO:tasks.workunit.client.0.smithi114.stdout:8/639: fsync d4/d11/d78/d7a/d31/fa5 0 2024-02-16T22:22:09.705 INFO:tasks.workunit.client.0.smithi114.stdout:1/633: unlink d4/d9/d46/d47/f8c 0 2024-02-16T22:22:09.712 INFO:tasks.workunit.client.0.smithi114.stdout:2/613: dwrite dd/df/d1c/f2e [0,4194304] 0 2024-02-16T22:22:09.712 INFO:tasks.workunit.client.1.smithi123.stdout:7/821: dwrite d3/d6e/d1d/db2/d35/db7/fe2 [0,4194304] 0 2024-02-16T22:22:09.715 INFO:tasks.workunit.client.0.smithi114.stdout:3/658: mknod d4/d26/ce1 0 2024-02-16T22:22:09.715 INFO:tasks.workunit.client.0.smithi114.stdout:3/659: chown d4/c69 323 1 2024-02-16T22:22:09.717 INFO:tasks.workunit.client.1.smithi123.stdout:5/817: dread d6/d3b/fb5 [0,4194304] 0 2024-02-16T22:22:09.722 INFO:tasks.workunit.client.0.smithi114.stdout:0/695: mkdir d1/d16/d1f/d28/d51/d61/d6d/d79/dd4 0 2024-02-16T22:22:09.723 INFO:tasks.workunit.client.0.smithi114.stdout:0/696: chown d1/d16/d1f/d28/d51/d61/d7e 63784420 1 2024-02-16T22:22:09.724 INFO:tasks.workunit.client.1.smithi123.stdout:0/757: creat d1/d4/d17/d54/db4/ffb x:0 0 0 2024-02-16T22:22:09.757 INFO:tasks.workunit.client.1.smithi123.stdout:3/858: dwrite d0/fc [0,4194304] 0 2024-02-16T22:22:09.759 INFO:tasks.workunit.client.0.smithi114.stdout:3/660: write d4/d24/d2d/d45/d68/d77/fab [845913,25610] 0 2024-02-16T22:22:09.759 INFO:tasks.workunit.client.0.smithi114.stdout:3/661: write d4/dd/d33/d72/fbf [1951839,66564] 0 2024-02-16T22:22:09.759 INFO:tasks.workunit.client.0.smithi114.stdout:0/697: read d1/f74 [456575,73947] 0 2024-02-16T22:22:09.759 INFO:tasks.workunit.client.0.smithi114.stdout:7/724: symlink d3/d22/d73/da7/d6f/d89/da6/dfc/lff 0 2024-02-16T22:22:09.759 INFO:tasks.workunit.client.0.smithi114.stdout:9/658: rename d2/d1a/f49 to d2/d5c/dc9/d1d/fd4 0 2024-02-16T22:22:09.759 INFO:tasks.workunit.client.0.smithi114.stdout:1/634: truncate d4/f25 2392015 0 2024-02-16T22:22:09.759 INFO:tasks.workunit.client.0.smithi114.stdout:7/725: dwrite d3/d22/d73/da7/d6f/df8/ff9 [0,4194304] 0 2024-02-16T22:22:09.759 INFO:tasks.workunit.client.0.smithi114.stdout:2/614: creat dd/d3b/da7/d8b/dc0/fcf x:0 0 0 2024-02-16T22:22:09.760 INFO:tasks.workunit.client.0.smithi114.stdout:2/615: dread - dd/df/d1c/d21/d61/d69/fce zero size 2024-02-16T22:22:09.760 INFO:tasks.workunit.client.0.smithi114.stdout:2/616: readlink dd/d3b/da7/l49 0 2024-02-16T22:22:09.760 INFO:tasks.workunit.client.0.smithi114.stdout:3/662: symlink d4/d46/dcf/le2 0 2024-02-16T22:22:09.760 INFO:tasks.workunit.client.0.smithi114.stdout:2/617: dread - dd/d3b/d8a/f9a zero size 2024-02-16T22:22:09.760 INFO:tasks.workunit.client.0.smithi114.stdout:0/698: mknod d1/d1a/d47/cd5 0 2024-02-16T22:22:09.760 INFO:tasks.workunit.client.0.smithi114.stdout:7/726: dwrite d3/d22/d73/da7/fbc [0,4194304] 0 2024-02-16T22:22:09.760 INFO:tasks.workunit.client.0.smithi114.stdout:7/727: stat d3/d87/c8b 0 2024-02-16T22:22:09.760 INFO:tasks.workunit.client.0.smithi114.stdout:5/641: truncate d0/d28/f46 908896 0 2024-02-16T22:22:09.760 INFO:tasks.workunit.client.0.smithi114.stdout:7/728: stat d3/d22/d73/da7/d6f/fa8 0 2024-02-16T22:22:09.764 INFO:tasks.workunit.client.0.smithi114.stdout:2/618: dwrite dd/d4e/f9c [0,4194304] 0 2024-02-16T22:22:09.766 INFO:tasks.workunit.client.0.smithi114.stdout:3/663: creat d4/d46/d48/d80/dbd/fe3 x:0 0 0 2024-02-16T22:22:09.768 INFO:tasks.workunit.client.0.smithi114.stdout:3/664: truncate d4/d46/d48/d75/fd5 820771 0 2024-02-16T22:22:09.769 INFO:tasks.workunit.client.0.smithi114.stdout:3/665: write d4/d87/faf [516360,98507] 0 2024-02-16T22:22:09.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:09 smithi123 ceph-mon[37161]: pgmap v32: 65 pgs: 65 active+clean; 2.9 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 122 MiB/s rd, 289 MiB/s wr, 611 op/s 2024-02-16T22:22:09.915 INFO:tasks.workunit.client.0.smithi114.stdout:0/699: mkdir d1/d16/d1f/d28/d51/d61/d7e/d90/dd6 0 2024-02-16T22:22:09.920 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:09 smithi114 ceph-mon[29235]: pgmap v32: 65 pgs: 65 active+clean; 2.9 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 122 MiB/s rd, 289 MiB/s wr, 611 op/s 2024-02-16T22:22:09.925 INFO:tasks.workunit.client.0.smithi114.stdout:9/659: mknod d2/dd0/cd5 0 2024-02-16T22:22:09.926 INFO:tasks.workunit.client.0.smithi114.stdout:9/660: chown d2/d5c/dc9/f75 58 1 2024-02-16T22:22:09.927 INFO:tasks.workunit.client.0.smithi114.stdout:9/661: dread - d2/d5c/dc9/d47/db2/fc6 zero size 2024-02-16T22:22:09.928 INFO:tasks.workunit.client.0.smithi114.stdout:2/619: creat dd/d6b/da5/fd0 x:0 0 0 2024-02-16T22:22:09.930 INFO:tasks.workunit.client.0.smithi114.stdout:0/700: mknod d1/d16/d1f/d28/d51/d61/d7e/d90/cd7 0 2024-02-16T22:22:09.931 INFO:tasks.workunit.client.0.smithi114.stdout:0/701: write d1/d16/f82 [372491,37677] 0 2024-02-16T22:22:09.934 INFO:tasks.workunit.client.0.smithi114.stdout:4/753: rename dd/d2e/f6a to dd/d1b/d24/f110 0 2024-02-16T22:22:09.937 INFO:tasks.workunit.client.0.smithi114.stdout:9/662: mkdir d2/d5c/d5f/d72/dd6 0 2024-02-16T22:22:09.938 INFO:tasks.workunit.client.0.smithi114.stdout:9/663: chown d2/d5c/d5f/d91/cc7 1854934870 1 2024-02-16T22:22:09.943 INFO:tasks.workunit.client.0.smithi114.stdout:0/702: creat d1/d1a/d47/d62/d7f/d8c/fd8 x:0 0 0 2024-02-16T22:22:09.945 INFO:tasks.workunit.client.0.smithi114.stdout:9/664: dread d2/d1a/f21 [0,4194304] 0 2024-02-16T22:22:09.947 INFO:tasks.workunit.client.0.smithi114.stdout:8/640: rename d4/d16/d25/cb3 to d4/d24/d2c/d5f/dd1/ce7 0 2024-02-16T22:22:09.950 INFO:tasks.workunit.client.0.smithi114.stdout:4/754: unlink dd/d1b/d41/d74/d9b/fc6 0 2024-02-16T22:22:09.956 INFO:tasks.workunit.client.0.smithi114.stdout:0/703: unlink d1/f15 0 2024-02-16T22:22:09.956 INFO:tasks.workunit.client.0.smithi114.stdout:7/729: dwrite d3/d22/d76/fb0 [0,4194304] 0 2024-02-16T22:22:09.959 INFO:tasks.workunit.client.0.smithi114.stdout:4/755: dwrite dd/d2e/fdf [0,4194304] 0 2024-02-16T22:22:09.962 INFO:tasks.workunit.client.0.smithi114.stdout:4/756: truncate dd/d1b/d41/d77/f90 5050397 0 2024-02-16T22:22:09.964 INFO:tasks.workunit.client.0.smithi114.stdout:1/635: rename d4/c90 to d4/d9/d46/d67/d92/dca/dcc/d98/cea 0 2024-02-16T22:22:09.977 INFO:tasks.workunit.client.0.smithi114.stdout:7/730: creat d3/d22/d73/f100 x:0 0 0 2024-02-16T22:22:09.980 INFO:tasks.workunit.client.0.smithi114.stdout:0/704: creat d1/d16/d1f/d28/d51/d60/d52/fd9 x:0 0 0 2024-02-16T22:22:09.981 INFO:tasks.workunit.client.0.smithi114.stdout:2/620: write dd/df/d1c/f67 [1283576,61654] 0 2024-02-16T22:22:09.985 INFO:tasks.workunit.client.1.smithi123.stdout:6/844: creat d0/d9/dc/d3a/d52/f130 x:0 0 0 2024-02-16T22:22:09.986 INFO:tasks.workunit.client.1.smithi123.stdout:6/845: stat d0/d9/dc 0 2024-02-16T22:22:09.986 INFO:tasks.workunit.client.0.smithi114.stdout:3/666: rename d4/d26/f3c to d4/d24/d2d/d45/d68/d77/fe4 0 2024-02-16T22:22:09.987 INFO:tasks.workunit.client.0.smithi114.stdout:9/665: dwrite d2/dd/d3c/d89/d8b/f95 [0,4194304] 0 2024-02-16T22:22:09.988 INFO:tasks.workunit.client.1.smithi123.stdout:1/822: rename d3/f7 to d3/d6/d49/f116 0 2024-02-16T22:22:09.990 INFO:tasks.workunit.client.1.smithi123.stdout:1/823: truncate d3/d6/d18/d9e/fea 1297802 0 2024-02-16T22:22:09.990 INFO:tasks.workunit.client.0.smithi114.stdout:1/636: readlink d4/d9/d46/l93 0 2024-02-16T22:22:09.991 INFO:tasks.workunit.client.1.smithi123.stdout:1/824: write d3/d6/db/d10/d2d/d8c/dfd/de8/f102 [22373,91972] 0 2024-02-16T22:22:09.995 INFO:tasks.workunit.client.0.smithi114.stdout:7/731: unlink d3/f1a 0 2024-02-16T22:22:09.996 INFO:tasks.workunit.client.0.smithi114.stdout:0/705: creat d1/d16/d1f/fda x:0 0 0 2024-02-16T22:22:09.998 INFO:tasks.workunit.client.0.smithi114.stdout:4/757: getdents dd/d1b/d37/d99/d103 0 2024-02-16T22:22:10.000 INFO:tasks.workunit.client.1.smithi123.stdout:7/822: dread d3/d6e/d4d/f5a [0,4194304] 0 2024-02-16T22:22:10.000 INFO:tasks.workunit.client.1.smithi123.stdout:9/948: creat d4/d36/daa/d20/f132 x:0 0 0 2024-02-16T22:22:10.001 INFO:tasks.workunit.client.1.smithi123.stdout:7/823: chown d3/d5/d6c/c9d 334756 1 2024-02-16T22:22:10.003 INFO:tasks.workunit.client.0.smithi114.stdout:9/666: creat d2/d5c/dc9/d47/d63/d98/fd7 x:0 0 0 2024-02-16T22:22:10.005 INFO:tasks.workunit.client.0.smithi114.stdout:9/667: write d2/d5c/dc9/d1d/f62 [1764630,15683] 0 2024-02-16T22:22:10.005 INFO:tasks.workunit.client.1.smithi123.stdout:0/758: rmdir d1/da0 39 2024-02-16T22:22:10.006 INFO:tasks.workunit.client.0.smithi114.stdout:9/668: chown d2/d1a/d65/f94 124 1 2024-02-16T22:22:10.006 INFO:tasks.workunit.client.1.smithi123.stdout:9/949: dwrite d4/d1e/d3a/dac/f115 [0,4194304] 0 2024-02-16T22:22:10.006 INFO:tasks.workunit.client.1.smithi123.stdout:2/830: getdents d4/d1a/d1d/d13/d25/d98/d10c 0 2024-02-16T22:22:10.017 INFO:tasks.workunit.client.1.smithi123.stdout:4/907: mkdir d5/df4/d108/dc0/d118/d73/d11d 0 2024-02-16T22:22:10.017 INFO:tasks.workunit.client.1.smithi123.stdout:5/818: write d6/dd/d44/f77 [353178,64122] 0 2024-02-16T22:22:10.020 INFO:tasks.workunit.client.0.smithi114.stdout:3/667: rename d4/d46/d48/d80/dbd/cdc to d4/d46/d48/d75/ce5 0 2024-02-16T22:22:10.024 INFO:tasks.workunit.client.0.smithi114.stdout:1/637: symlink d4/d63/leb 0 2024-02-16T22:22:10.026 INFO:tasks.workunit.client.0.smithi114.stdout:9/669: symlink d2/d5c/dc9/d47/d63/d98/ld8 0 2024-02-16T22:22:10.028 INFO:tasks.workunit.client.1.smithi123.stdout:8/862: creat d0/d29/dba/f11e x:0 0 0 2024-02-16T22:22:10.028 INFO:tasks.workunit.client.1.smithi123.stdout:9/950: dread d4/d36/f5f [0,4194304] 0 2024-02-16T22:22:10.029 INFO:tasks.workunit.client.0.smithi114.stdout:8/641: getdents d4/d11/d78/d7a/d36 0 2024-02-16T22:22:10.030 INFO:tasks.workunit.client.1.smithi123.stdout:7/824: creat d3/d6e/d4d/d92/f10d x:0 0 0 2024-02-16T22:22:10.031 INFO:tasks.workunit.client.1.smithi123.stdout:7/825: fdatasync d3/d6e/d24/f90 0 2024-02-16T22:22:10.031 INFO:tasks.workunit.client.0.smithi114.stdout:7/732: creat d3/d92/dd5/f101 x:0 0 0 2024-02-16T22:22:10.032 INFO:tasks.workunit.client.1.smithi123.stdout:7/826: read d3/d6e/d34/f75 [812996,46947] 0 2024-02-16T22:22:10.033 INFO:tasks.workunit.client.0.smithi114.stdout:2/621: creat dd/fd1 x:0 0 0 2024-02-16T22:22:10.034 INFO:tasks.workunit.client.1.smithi123.stdout:0/759: mknod d1/d4/d17/d54/d8e/d96/cfc 0 2024-02-16T22:22:10.037 INFO:tasks.workunit.client.1.smithi123.stdout:6/846: dread d0/d96/de3/f2c [0,4194304] 0 2024-02-16T22:22:10.040 INFO:tasks.workunit.client.0.smithi114.stdout:4/758: rename dd/d1b/d24/d57/fc9 to dd/d1b/d37/d99/d103/f111 0 2024-02-16T22:22:10.045 INFO:tasks.workunit.client.1.smithi123.stdout:4/908: unlink d5/d1c/d23/fe9 0 2024-02-16T22:22:10.045 INFO:tasks.workunit.client.1.smithi123.stdout:4/909: dread - d5/d1c/f115 zero size 2024-02-16T22:22:10.046 INFO:tasks.workunit.client.1.smithi123.stdout:4/910: write d5/f17 [6584637,92272] 0 2024-02-16T22:22:10.047 INFO:tasks.workunit.client.1.smithi123.stdout:5/819: creat d6/dd/df/d9f/d102/f11f x:0 0 0 2024-02-16T22:22:10.054 INFO:tasks.workunit.client.1.smithi123.stdout:3/859: write d0/d20/d29/d9b/dbd/df4/d37/f64 [1185317,30983] 0 2024-02-16T22:22:10.057 INFO:tasks.workunit.client.1.smithi123.stdout:9/951: mknod d4/d36/d8e/dff/c133 0 2024-02-16T22:22:10.059 INFO:tasks.workunit.client.0.smithi114.stdout:2/622: creat dd/df/d1c/d21/d61/d63/dc2/fd2 x:0 0 0 2024-02-16T22:22:10.063 INFO:tasks.workunit.client.0.smithi114.stdout:9/670: dwrite d2/d1a/f6b [0,4194304] 0 2024-02-16T22:22:10.063 INFO:tasks.workunit.client.1.smithi123.stdout:7/827: fdatasync d3/d6e/d24/f6d 0 2024-02-16T22:22:10.066 INFO:tasks.workunit.client.0.smithi114.stdout:4/759: mknod dd/d1b/d24/d57/c112 0 2024-02-16T22:22:10.066 INFO:tasks.workunit.client.0.smithi114.stdout:9/671: truncate d2/d5c/dc9/d47/db2/fc6 359165 0 2024-02-16T22:22:10.067 INFO:tasks.workunit.client.0.smithi114.stdout:9/672: chown d2/d33/f7d 2418 1 2024-02-16T22:22:10.067 INFO:tasks.workunit.client.1.smithi123.stdout:0/760: creat d1/d8/d13/ffd x:0 0 0 2024-02-16T22:22:10.068 INFO:tasks.workunit.client.1.smithi123.stdout:0/761: write d1/d8/d13/d1d/fda [384602,8080] 0 2024-02-16T22:22:10.069 INFO:tasks.workunit.client.0.smithi114.stdout:3/668: rename d4/d43/fde to d4/d43/d53/fe6 0 2024-02-16T22:22:10.072 INFO:tasks.workunit.client.1.smithi123.stdout:6/847: creat d0/d6/d2f/d5d/f131 x:0 0 0 2024-02-16T22:22:10.074 INFO:tasks.workunit.client.1.smithi123.stdout:2/831: rename d4/d1a/d1d/d38/d7b/ccc to d4/d1a/d1d/d38/d5e/de8/c10f 0 2024-02-16T22:22:10.075 INFO:tasks.workunit.client.0.smithi114.stdout:0/706: getdents d1/d1a/d47/d62/d7f/d8c 0 2024-02-16T22:22:10.076 INFO:tasks.workunit.client.0.smithi114.stdout:7/733: dread d3/d22/f48 [0,4194304] 0 2024-02-16T22:22:10.078 INFO:tasks.workunit.client.0.smithi114.stdout:2/623: creat dd/df/d1c/d21/d61/d63/d83/fd3 x:0 0 0 2024-02-16T22:22:10.078 INFO:tasks.workunit.client.1.smithi123.stdout:4/911: mkdir d5/db8/d11e 0 2024-02-16T22:22:10.079 INFO:tasks.workunit.client.1.smithi123.stdout:4/912: chown d5/db8 1 1 2024-02-16T22:22:10.079 INFO:tasks.workunit.client.0.smithi114.stdout:2/624: chown dd/d3b/d8a/f93 19 1 2024-02-16T22:22:10.080 INFO:tasks.workunit.client.0.smithi114.stdout:4/760: unlink dd/d2e/d56/ddc/ced 0 2024-02-16T22:22:10.080 INFO:tasks.workunit.client.0.smithi114.stdout:4/761: readlink dd/d1b/l64 0 2024-02-16T22:22:10.081 INFO:tasks.workunit.client.1.smithi123.stdout:5/820: mknod d6/dc1/de6/c120 0 2024-02-16T22:22:10.082 INFO:tasks.workunit.client.1.smithi123.stdout:1/825: link d3/d6/db/d10/d2a/lfb d3/d6/db/d10/d2d/d8c/dfd/d9f/l117 0 2024-02-16T22:22:10.083 INFO:tasks.workunit.client.1.smithi123.stdout:1/826: chown d3/d6/db/d10/d2a/d80/lcf 116158138 1 2024-02-16T22:22:10.084 INFO:tasks.workunit.client.1.smithi123.stdout:8/863: fsync d0/d29/f60 0 2024-02-16T22:22:10.086 INFO:tasks.workunit.client.0.smithi114.stdout:3/669: symlink d4/d26/d91/le7 0 2024-02-16T22:22:10.087 INFO:tasks.workunit.client.0.smithi114.stdout:3/670: fsync d4/d26/f4d 0 2024-02-16T22:22:10.088 INFO:tasks.workunit.client.1.smithi123.stdout:3/860: unlink d0/d40/dfb/f119 0 2024-02-16T22:22:10.090 INFO:tasks.workunit.client.1.smithi123.stdout:9/952: mknod d4/d36/d8e/d6d/d71/d92/c134 0 2024-02-16T22:22:10.091 INFO:tasks.workunit.client.0.smithi114.stdout:8/642: creat d4/d24/d2c/d5f/fe8 x:0 0 0 2024-02-16T22:22:10.093 INFO:tasks.workunit.client.0.smithi114.stdout:0/707: mknod d1/d1a/d47/d62/d89/cdb 0 2024-02-16T22:22:10.096 INFO:tasks.workunit.client.0.smithi114.stdout:7/734: rename d3/d22/d73/da7/d6f/d9c/fef to d3/d22/d73/da7/d6f/d89/da6/dfc/f102 0 2024-02-16T22:22:10.097 INFO:tasks.workunit.client.1.smithi123.stdout:7/828: chown d3/l47 11921415 1 2024-02-16T22:22:10.098 INFO:tasks.workunit.client.1.smithi123.stdout:7/829: write d3/d6e/d1d/db2/d3d/db1/ff2 [271346,119297] 0 2024-02-16T22:22:10.098 INFO:tasks.workunit.client.0.smithi114.stdout:2/625: unlink dd/l1a 0 2024-02-16T22:22:10.101 INFO:tasks.workunit.client.0.smithi114.stdout:0/708: dwrite d1/d78/fd3 [0,4194304] 0 2024-02-16T22:22:10.101 INFO:tasks.workunit.client.1.smithi123.stdout:9/953: dread d4/d36/d8e/d6f/f107 [0,4194304] 0 2024-02-16T22:22:10.102 INFO:tasks.workunit.client.1.smithi123.stdout:9/954: stat d4/d36/d8e/d6f/d7e 0 2024-02-16T22:22:10.105 INFO:tasks.workunit.client.1.smithi123.stdout:2/832: stat d4/d1a/f86 0 2024-02-16T22:22:10.106 INFO:tasks.workunit.client.1.smithi123.stdout:2/833: chown d4/d1a/d1d/d13/d25/f9a 3 1 2024-02-16T22:22:10.107 INFO:tasks.workunit.client.0.smithi114.stdout:3/671: dread d4/dd/d33/d76/f9e [0,4194304] 0 2024-02-16T22:22:10.107 INFO:tasks.workunit.client.1.smithi123.stdout:4/913: rmdir d5/d1c/d23/d47 39 2024-02-16T22:22:10.108 INFO:tasks.workunit.client.0.smithi114.stdout:1/638: getdents d4/d63/d69/d7c/d9b 0 2024-02-16T22:22:10.110 INFO:tasks.workunit.client.1.smithi123.stdout:6/848: dread d0/d96/fe8 [0,4194304] 0 2024-02-16T22:22:10.111 INFO:tasks.workunit.client.0.smithi114.stdout:2/626: dread dd/f92 [0,4194304] 0 2024-02-16T22:22:10.114 INFO:tasks.workunit.client.1.smithi123.stdout:1/827: mknod d3/d6/d49/d48/d84/c118 0 2024-02-16T22:22:10.114 INFO:tasks.workunit.client.0.smithi114.stdout:3/672: dread d4/d46/d48/d75/fd5 [0,4194304] 0 2024-02-16T22:22:10.116 INFO:tasks.workunit.client.0.smithi114.stdout:4/762: write dd/d1b/f97 [641675,115980] 0 2024-02-16T22:22:10.118 INFO:tasks.workunit.client.1.smithi123.stdout:0/762: dwrite d1/d8/d13/d1d/d62/d7a/f8f [0,4194304] 0 2024-02-16T22:22:10.121 INFO:tasks.workunit.client.0.smithi114.stdout:2/627: dwrite dd/f5d [0,4194304] 0 2024-02-16T22:22:10.121 INFO:tasks.workunit.client.0.smithi114.stdout:0/709: mkdir d1/d1a/d3e/ddc 0 2024-02-16T22:22:10.125 INFO:tasks.workunit.client.0.smithi114.stdout:7/735: dread d3/d22/f8f [0,4194304] 0 2024-02-16T22:22:10.128 INFO:tasks.workunit.client.1.smithi123.stdout:7/830: creat d3/d6e/d1d/db2/d3d/f10e x:0 0 0 2024-02-16T22:22:10.131 INFO:tasks.workunit.client.1.smithi123.stdout:5/821: dread d6/dd/df/f22 [0,4194304] 0 2024-02-16T22:22:10.133 INFO:tasks.workunit.client.1.smithi123.stdout:0/763: dread d1/d4/d17/d74/fc1 [0,4194304] 0 2024-02-16T22:22:10.133 INFO:tasks.workunit.client.1.smithi123.stdout:0/764: chown d1/d4 12236213 1 2024-02-16T22:22:10.134 INFO:tasks.workunit.client.1.smithi123.stdout:9/955: dread - d4/d33/fe2 zero size 2024-02-16T22:22:10.135 INFO:tasks.workunit.client.0.smithi114.stdout:9/673: link d2/d1a/d23/d32/d46/f9d d2/d5c/d5f/fd9 0 2024-02-16T22:22:10.140 INFO:tasks.workunit.client.0.smithi114.stdout:7/736: symlink d3/d22/d73/da7/d6f/l103 0 2024-02-16T22:22:10.141 INFO:tasks.workunit.client.0.smithi114.stdout:7/737: dread - d3/d22/d73/da7/d6f/fd6 zero size 2024-02-16T22:22:10.144 INFO:tasks.workunit.client.0.smithi114.stdout:0/710: dread d1/f74 [0,4194304] 0 2024-02-16T22:22:10.145 INFO:tasks.workunit.client.0.smithi114.stdout:1/639: dread d4/d9/d3d/d3f/f75 [0,4194304] 0 2024-02-16T22:22:10.147 INFO:tasks.workunit.client.1.smithi123.stdout:8/864: creat d0/d29/da7/d26/d68/d81/dd7/f11f x:0 0 0 2024-02-16T22:22:10.147 INFO:tasks.workunit.client.0.smithi114.stdout:8/643: link d4/d11/f1f d4/d11/d9d/fe9 0 2024-02-16T22:22:10.149 INFO:tasks.workunit.client.0.smithi114.stdout:1/640: dread d4/d9/d3d/d3f/f75 [0,4194304] 0 2024-02-16T22:22:10.149 INFO:tasks.workunit.client.1.smithi123.stdout:3/861: creat d0/d20/d29/d9b/dbd/df4/d5d/d60/de9/f11e x:0 0 0 2024-02-16T22:22:10.150 INFO:tasks.workunit.client.0.smithi114.stdout:7/738: symlink d3/d22/d73/da7/d6f/d89/da6/l104 0 2024-02-16T22:22:10.154 INFO:tasks.workunit.client.0.smithi114.stdout:0/711: unlink d1/d16/d18/f59 0 2024-02-16T22:22:10.156 INFO:tasks.workunit.client.0.smithi114.stdout:4/763: write dd/d2e/d3f/f6c [1660286,8182] 0 2024-02-16T22:22:10.157 INFO:tasks.workunit.client.0.smithi114.stdout:8/644: symlink d4/d16/d25/d46/db5/dca/lea 0 2024-02-16T22:22:10.159 INFO:tasks.workunit.client.1.smithi123.stdout:5/822: symlink d6/dd/df/d9f/l121 0 2024-02-16T22:22:10.161 INFO:tasks.workunit.client.0.smithi114.stdout:9/674: dread d2/d33/f57 [0,4194304] 0 2024-02-16T22:22:10.162 INFO:tasks.workunit.client.1.smithi123.stdout:0/765: creat d1/d4/d17/d54/d8e/d96/ffe x:0 0 0 2024-02-16T22:22:10.162 INFO:tasks.workunit.client.0.smithi114.stdout:2/628: dread dd/f28 [4194304,4194304] 0 2024-02-16T22:22:10.163 INFO:tasks.workunit.client.0.smithi114.stdout:7/739: symlink d3/d14/l105 0 2024-02-16T22:22:10.163 INFO:tasks.workunit.client.0.smithi114.stdout:7/740: chown d3/d87/cde 4384689 1 2024-02-16T22:22:10.165 INFO:tasks.workunit.client.1.smithi123.stdout:1/828: dwrite d3/f5 [4194304,4194304] 0 2024-02-16T22:22:10.167 INFO:tasks.workunit.client.1.smithi123.stdout:2/834: creat d4/d1a/d1d/d13/d25/d98/d10c/f110 x:0 0 0 2024-02-16T22:22:10.170 INFO:tasks.workunit.client.0.smithi114.stdout:7/741: dread d3/d22/d73/da7/f3b [0,4194304] 0 2024-02-16T22:22:10.171 INFO:tasks.workunit.client.1.smithi123.stdout:6/849: symlink d0/d6/d2f/l132 0 2024-02-16T22:22:10.175 INFO:tasks.workunit.client.1.smithi123.stdout:8/865: chown d0/d29/da7/d26/d2a/la9 0 1 2024-02-16T22:22:10.178 INFO:tasks.workunit.client.1.smithi123.stdout:3/862: symlink d0/d20/d29/d9b/dbd/df4/d37/d41/l11f 0 2024-02-16T22:22:10.179 INFO:tasks.workunit.client.1.smithi123.stdout:4/914: truncate d5/d1c/d23/f57 1417848 0 2024-02-16T22:22:10.181 INFO:tasks.workunit.client.1.smithi123.stdout:8/866: dwrite d0/d29/da7/d26/d68/d81/d47/f73 [0,4194304] 0 2024-02-16T22:22:10.182 INFO:tasks.workunit.client.0.smithi114.stdout:4/764: symlink dd/d2e/d56/l113 0 2024-02-16T22:22:10.183 INFO:tasks.workunit.client.0.smithi114.stdout:2/629: dread dd/df/d1c/d21/f26 [0,4194304] 0 2024-02-16T22:22:10.185 INFO:tasks.workunit.client.0.smithi114.stdout:4/765: read dd/d2e/d3f/f6c [2715781,112224] 0 2024-02-16T22:22:10.187 INFO:tasks.workunit.client.0.smithi114.stdout:4/766: chown dd/d1b/d24/d57/c112 15 1 2024-02-16T22:22:10.188 INFO:tasks.workunit.client.0.smithi114.stdout:0/712: truncate d1/d1a/f38 41701 0 2024-02-16T22:22:10.189 INFO:tasks.workunit.client.1.smithi123.stdout:5/823: dwrite d6/dd/df/d58/d118/f119 [0,4194304] 0 2024-02-16T22:22:10.189 INFO:tasks.workunit.client.0.smithi114.stdout:4/767: dread - dd/d59/d8f/fa0 zero size 2024-02-16T22:22:10.190 INFO:tasks.workunit.client.0.smithi114.stdout:9/675: dwrite d2/d5c/f8d [0,4194304] 0 2024-02-16T22:22:10.190 INFO:tasks.workunit.client.0.smithi114.stdout:0/713: stat d1/d16/f5d 0 2024-02-16T22:22:10.191 INFO:tasks.workunit.client.1.smithi123.stdout:1/829: rmdir d3/d6/d49 39 2024-02-16T22:22:10.193 INFO:tasks.workunit.client.0.smithi114.stdout:1/641: creat d4/fec x:0 0 0 2024-02-16T22:22:10.197 INFO:tasks.workunit.client.1.smithi123.stdout:5/824: dread d6/dd/df/d76/feb [0,4194304] 0 2024-02-16T22:22:10.202 INFO:tasks.workunit.client.1.smithi123.stdout:5/825: dwrite d6/f8 [0,4194304] 0 2024-02-16T22:22:10.206 INFO:tasks.workunit.client.1.smithi123.stdout:3/863: unlink d0/d20/d29/d9b/dbd/df4/f36 0 2024-02-16T22:22:10.207 INFO:tasks.workunit.client.1.smithi123.stdout:7/831: link d3/d6e/d1d/db2/d3d/cec d3/d59/c10f 0 2024-02-16T22:22:10.208 INFO:tasks.workunit.client.1.smithi123.stdout:0/766: dwrite d1/d4/d17/d54/d8e/da5/fd3 [4194304,4194304] 0 2024-02-16T22:22:10.209 INFO:tasks.workunit.client.1.smithi123.stdout:0/767: readlink d1/d8/d3d/l63 0 2024-02-16T22:22:10.210 INFO:tasks.workunit.client.1.smithi123.stdout:3/864: write d0/d20/d29/d9b/dbd/df4/d5d/dae/db0/f104 [144690,10138] 0 2024-02-16T22:22:10.211 INFO:tasks.workunit.client.0.smithi114.stdout:9/676: dread - d2/d1a/d23/f9b zero size 2024-02-16T22:22:10.213 INFO:tasks.workunit.client.1.smithi123.stdout:9/956: creat d4/d36/d8e/d6d/d71/f135 x:0 0 0 2024-02-16T22:22:10.216 INFO:tasks.workunit.client.0.smithi114.stdout:1/642: mkdir d4/d9/d46/d67/d92/dca/ded 0 2024-02-16T22:22:10.222 INFO:tasks.workunit.client.1.smithi123.stdout:4/915: write d5/f1d [5552117,52729] 0 2024-02-16T22:22:10.223 INFO:tasks.workunit.client.0.smithi114.stdout:8/645: link d4/d16/d25/d46/cd7 d4/d11/d78/d7a/d36/d3b/ceb 0 2024-02-16T22:22:10.225 INFO:tasks.workunit.client.0.smithi114.stdout:7/742: creat d3/f106 x:0 0 0 2024-02-16T22:22:10.228 INFO:tasks.workunit.client.0.smithi114.stdout:4/768: mkdir dd/d59/d8f/dea/d114 0 2024-02-16T22:22:10.228 INFO:tasks.workunit.client.0.smithi114.stdout:4/769: chown dd/l2d 1430808 1 2024-02-16T22:22:10.230 INFO:tasks.workunit.client.0.smithi114.stdout:9/677: mknod d2/d5c/dc9/d47/d63/dbb/cda 0 2024-02-16T22:22:10.234 INFO:tasks.workunit.client.1.smithi123.stdout:5/826: rename d6/dd/d78/da5/db0/fcc to d6/dd/df/d58/d89/d98/dd8/f122 0 2024-02-16T22:22:10.237 INFO:tasks.workunit.client.1.smithi123.stdout:6/850: truncate d0/d6/d2f/da5/dc6/f112 215734 0 2024-02-16T22:22:10.238 INFO:tasks.workunit.client.1.smithi123.stdout:6/851: write d0/d6/d1a/f64 [2901044,88227] 0 2024-02-16T22:22:10.238 INFO:tasks.workunit.client.0.smithi114.stdout:8/646: mknod d4/d16/cec 0 2024-02-16T22:22:10.239 INFO:tasks.workunit.client.0.smithi114.stdout:8/647: dread - d4/d16/d25/d46/db5/fd6 zero size 2024-02-16T22:22:10.240 INFO:tasks.workunit.client.1.smithi123.stdout:7/832: dread d3/d6e/d52/f67 [4194304,4194304] 0 2024-02-16T22:22:10.240 INFO:tasks.workunit.client.0.smithi114.stdout:7/743: mknod d3/d87/dd1/c107 0 2024-02-16T22:22:10.245 INFO:tasks.workunit.client.1.smithi123.stdout:8/867: truncate d0/d61/f114 224749 0 2024-02-16T22:22:10.248 INFO:tasks.workunit.client.0.smithi114.stdout:0/714: link d1/c12 d1/d16/d1f/d28/d51/d61/d7e/d90/dd6/cdd 0 2024-02-16T22:22:10.251 INFO:tasks.workunit.client.1.smithi123.stdout:3/865: creat d0/d20/d29/d9b/dbd/df4/d5d/d60/d93/f120 x:0 0 0 2024-02-16T22:22:10.252 INFO:tasks.workunit.client.1.smithi123.stdout:3/866: readlink d0/d20/d29/d9b/dbd/df4/d5d/d60/dc6/lcd 0 2024-02-16T22:22:10.252 INFO:tasks.workunit.client.0.smithi114.stdout:1/643: mknod d4/d9/d46/cee 0 2024-02-16T22:22:10.253 INFO:tasks.workunit.client.1.smithi123.stdout:3/867: fdatasync d0/d20/d29/d9b/dbd/df4/d5d/f109 0 2024-02-16T22:22:10.253 INFO:tasks.workunit.client.0.smithi114.stdout:0/715: dwrite d1/d16/d1f/fa9 [0,4194304] 0 2024-02-16T22:22:10.254 INFO:tasks.workunit.client.1.smithi123.stdout:3/868: read d0/da0/faa [240272,103978] 0 2024-02-16T22:22:10.254 INFO:tasks.workunit.client.1.smithi123.stdout:3/869: stat d0/f4c 0 2024-02-16T22:22:10.255 INFO:tasks.workunit.client.0.smithi114.stdout:1/644: write d4/d63/d69/d7c/dc0/fe3 [67671,113410] 0 2024-02-16T22:22:10.255 INFO:tasks.workunit.client.0.smithi114.stdout:0/716: fdatasync d1/d16/d18/f77 0 2024-02-16T22:22:10.257 INFO:tasks.workunit.client.0.smithi114.stdout:0/717: stat d1/d16/d1f/d28/d51/d61 0 2024-02-16T22:22:10.258 INFO:tasks.workunit.client.0.smithi114.stdout:1/645: dread - d4/d63/d69/d72/fde zero size 2024-02-16T22:22:10.258 INFO:tasks.workunit.client.0.smithi114.stdout:1/646: stat d4/d63/d6b 0 2024-02-16T22:22:10.258 INFO:tasks.workunit.client.1.smithi123.stdout:9/957: mknod d4/d36/daa/d7f/c136 0 2024-02-16T22:22:10.259 INFO:tasks.workunit.client.0.smithi114.stdout:1/647: fdatasync d4/d9/d46/d67/fa8 0 2024-02-16T22:22:10.260 INFO:tasks.workunit.client.0.smithi114.stdout:1/648: dread - d4/d63/d69/d72/fe6 zero size 2024-02-16T22:22:10.260 INFO:tasks.workunit.client.0.smithi114.stdout:8/648: creat d4/d11/d78/d7a/d36/fed x:0 0 0 2024-02-16T22:22:10.260 INFO:tasks.workunit.client.0.smithi114.stdout:4/770: write dd/d1b/d37/d61/f66 [701965,31248] 0 2024-02-16T22:22:10.264 INFO:tasks.workunit.client.0.smithi114.stdout:9/678: write d2/d5c/dc9/d47/d63/fa5 [782980,52723] 0 2024-02-16T22:22:10.266 INFO:tasks.workunit.client.1.smithi123.stdout:2/835: creat d4/d1a/f111 x:0 0 0 2024-02-16T22:22:10.267 INFO:tasks.workunit.client.0.smithi114.stdout:8/649: dread d4/d11/d78/d7a/d31/fa5 [0,4194304] 0 2024-02-16T22:22:10.268 INFO:tasks.workunit.client.1.smithi123.stdout:1/830: rename d3/d6/d49/d48/d84/d92/cff to d3/d6/db/d10/d4b/d70/c119 0 2024-02-16T22:22:10.270 INFO:tasks.workunit.client.1.smithi123.stdout:1/831: chown d3/d6/db/d10/d2d/d8c/dfd/d9f/db4/ddf 0 1 2024-02-16T22:22:10.272 INFO:tasks.workunit.client.0.smithi114.stdout:0/718: rename d1/fa to d1/d16/d48/d98/fde 0 2024-02-16T22:22:10.273 INFO:tasks.workunit.client.1.smithi123.stdout:7/833: creat d3/d6e/d52/f110 x:0 0 0 2024-02-16T22:22:10.275 INFO:tasks.workunit.client.1.smithi123.stdout:8/868: dread d0/d29/da7/d26/f6d [0,4194304] 0 2024-02-16T22:22:10.277 INFO:tasks.workunit.client.1.smithi123.stdout:3/870: dread d0/d20/d2e/f95 [0,4194304] 0 2024-02-16T22:22:10.278 INFO:tasks.workunit.client.1.smithi123.stdout:3/871: fsync d0/d20/d29/d9b/dbd/df4/d5d/dae/dc7/f10a 0 2024-02-16T22:22:10.279 INFO:tasks.workunit.client.0.smithi114.stdout:7/744: write d3/ff [9482275,9209] 0 2024-02-16T22:22:10.280 INFO:tasks.workunit.client.0.smithi114.stdout:7/745: chown d3/d22/d73/da7/d6f/lb5 16128 1 2024-02-16T22:22:10.321 INFO:tasks.workunit.client.1.smithi123.stdout:4/916: rename d5/df4/d108 to d5/db8/d11f 0 2024-02-16T22:22:10.322 INFO:tasks.workunit.client.0.smithi114.stdout:4/771: mknod dd/d1b/d41/c115 0 2024-02-16T22:22:10.325 INFO:tasks.workunit.client.1.smithi123.stdout:6/852: symlink d0/d6/d4d/db0/d104/l133 0 2024-02-16T22:22:10.328 INFO:tasks.workunit.client.0.smithi114.stdout:7/746: chown d3/d3d/l20 69888 1 2024-02-16T22:22:10.330 INFO:tasks.workunit.client.1.smithi123.stdout:7/834: unlink d3/d5/f28 0 2024-02-16T22:22:10.333 INFO:tasks.workunit.client.1.smithi123.stdout:2/836: dread d4/d1a/d1d/d13/f36 [0,4194304] 0 2024-02-16T22:22:10.333 INFO:tasks.workunit.client.0.smithi114.stdout:1/649: dwrite d4/d63/d69/d7c/d9b/db0/d74/f87 [0,4194304] 0 2024-02-16T22:22:10.340 INFO:tasks.workunit.client.0.smithi114.stdout:4/772: read dd/d1b/d24/f110 [481,110362] 0 2024-02-16T22:22:10.340 INFO:tasks.workunit.client.1.smithi123.stdout:0/768: link d1/d8/d13/d1d/d62/d7a/f9c d1/d4/d17/dad/fff 0 2024-02-16T22:22:10.341 INFO:tasks.workunit.client.1.smithi123.stdout:1/832: write d3/d6/d11/f17 [7533397,33442] 0 2024-02-16T22:22:10.345 INFO:tasks.workunit.client.1.smithi123.stdout:8/869: write d0/d29/da7/d43/d34/f113 [4257027,96415] 0 2024-02-16T22:22:10.347 INFO:tasks.workunit.client.0.smithi114.stdout:8/650: link d4/d24/d6d/dc7/lcb d4/d24/lee 0 2024-02-16T22:22:10.348 INFO:tasks.workunit.client.0.smithi114.stdout:8/651: truncate d4/d16/d25/fde 817774 0 2024-02-16T22:22:10.348 INFO:tasks.workunit.client.0.smithi114.stdout:0/719: mknod d1/d1a/d47/cdf 0 2024-02-16T22:22:10.349 INFO:tasks.workunit.client.1.smithi123.stdout:3/872: dread d0/d20/d29/f3d [0,4194304] 0 2024-02-16T22:22:10.350 INFO:tasks.workunit.client.1.smithi123.stdout:9/958: symlink d4/d36/d8e/d58/l137 0 2024-02-16T22:22:10.350 INFO:tasks.workunit.client.0.smithi114.stdout:8/652: write d4/d11/d78/fd8 [83342,81116] 0 2024-02-16T22:22:10.350 INFO:tasks.workunit.client.0.smithi114.stdout:0/720: write d1/fe [3951620,10900] 0 2024-02-16T22:22:10.351 INFO:tasks.workunit.client.0.smithi114.stdout:8/653: fdatasync d4/d16/d25/fde 0 2024-02-16T22:22:10.352 INFO:tasks.workunit.client.1.smithi123.stdout:4/917: unlink d5/dba/cd0 0 2024-02-16T22:22:10.353 INFO:tasks.workunit.client.1.smithi123.stdout:4/918: chown d5/d1c/d23/d74/l37 24 1 2024-02-16T22:22:10.353 INFO:tasks.workunit.client.0.smithi114.stdout:7/747: write d3/d22/d73/f7c [1708955,5341] 0 2024-02-16T22:22:10.354 INFO:tasks.workunit.client.0.smithi114.stdout:7/748: chown d3/d22/d73/da7/d6f 0 1 2024-02-16T22:22:10.354 INFO:tasks.workunit.client.1.smithi123.stdout:5/827: creat d6/dd/f123 x:0 0 0 2024-02-16T22:22:10.355 INFO:tasks.workunit.client.0.smithi114.stdout:4/773: symlink dd/d59/d8f/l116 0 2024-02-16T22:22:10.355 INFO:tasks.workunit.client.0.smithi114.stdout:7/749: write d3/d87/dd1/fec [758554,48032] 0 2024-02-16T22:22:10.356 INFO:tasks.workunit.client.1.smithi123.stdout:3/873: dwrite d0/d20/d29/d9b/dbd/df4/d5d/f61 [4194304,4194304] 0 2024-02-16T22:22:10.356 INFO:tasks.workunit.client.1.smithi123.stdout:5/828: dread - d6/dd/df/d76/f107 zero size 2024-02-16T22:22:10.357 INFO:tasks.workunit.client.0.smithi114.stdout:4/774: stat dd/d59/d8a/cdb 0 2024-02-16T22:22:10.357 INFO:tasks.workunit.client.0.smithi114.stdout:9/679: getdents d2/dd/d3c/d89 0 2024-02-16T22:22:10.357 INFO:tasks.workunit.client.0.smithi114.stdout:7/750: write d3/d14/f5c [3282188,3481] 0 2024-02-16T22:22:10.358 INFO:tasks.workunit.client.1.smithi123.stdout:7/835: chown d3/f86 1844788025 1 2024-02-16T22:22:10.365 INFO:tasks.workunit.client.1.smithi123.stdout:1/833: rename d3/d6/d18/d9e/le7 to d3/d6/d11/d8b/l11a 0 2024-02-16T22:22:10.369 INFO:tasks.workunit.client.0.smithi114.stdout:9/680: symlink d2/ldb 0 2024-02-16T22:22:10.371 INFO:tasks.workunit.client.1.smithi123.stdout:2/837: write d4/d1a/d1d/f2b [4161798,64659] 0 2024-02-16T22:22:10.373 INFO:tasks.workunit.client.1.smithi123.stdout:2/838: stat d4/d1a/d1d/d13/d15/c51 0 2024-02-16T22:22:10.375 INFO:tasks.workunit.client.0.smithi114.stdout:8/654: symlink d4/d11/d78/d7a/lef 0 2024-02-16T22:22:10.378 INFO:tasks.workunit.client.1.smithi123.stdout:4/919: mkdir d5/db8/d11f/dc0/d118/d9f/d120 0 2024-02-16T22:22:10.380 INFO:tasks.workunit.client.0.smithi114.stdout:4/775: write dd/d1b/d24/f110 [164534,99877] 0 2024-02-16T22:22:10.381 INFO:tasks.workunit.client.0.smithi114.stdout:9/681: mknod d2/d5c/d5f/d7b/cdc 0 2024-02-16T22:22:10.382 INFO:tasks.workunit.client.0.smithi114.stdout:4/776: write dd/d2e/d56/ddc/dde/fe8 [1217029,86650] 0 2024-02-16T22:22:10.384 INFO:tasks.workunit.client.1.smithi123.stdout:8/870: dwrite d0/d61/d66/d7e/fc7 [0,4194304] 0 2024-02-16T22:22:10.386 INFO:tasks.workunit.client.0.smithi114.stdout:0/721: truncate d1/d16/d1f/d28/d51/d60/d52/fb9 74394 0 2024-02-16T22:22:10.387 INFO:tasks.workunit.client.1.smithi123.stdout:8/871: read - d0/d29/da7/d26/f11a zero size 2024-02-16T22:22:10.388 INFO:tasks.workunit.client.1.smithi123.stdout:4/920: dwrite d5/db8/d11f/fa4 [0,4194304] 0 2024-02-16T22:22:10.389 INFO:tasks.workunit.client.1.smithi123.stdout:6/853: truncate d0/d96/de3/fc1 389179 0 2024-02-16T22:22:10.389 INFO:tasks.workunit.client.1.smithi123.stdout:9/959: dwrite d4/d36/daa/d2b/d6c/fdf [0,4194304] 0 2024-02-16T22:22:10.390 INFO:tasks.workunit.client.0.smithi114.stdout:8/655: write d4/d11/d78/d7a/f1a [4850699,49279] 0 2024-02-16T22:22:10.391 INFO:tasks.workunit.client.1.smithi123.stdout:7/836: mkdir d3/d6e/d1d/db2/d3d/db1/d111 0 2024-02-16T22:22:10.392 INFO:tasks.workunit.client.1.smithi123.stdout:5/829: write d6/dd/df/d58/d3f/dbe/d37/f52 [3201555,103575] 0 2024-02-16T22:22:10.394 INFO:tasks.workunit.client.1.smithi123.stdout:3/874: truncate d0/d20/d29/d9b/dbd/df4/d5d/d60/d93/fbb 24333 0 2024-02-16T22:22:10.396 INFO:tasks.workunit.client.0.smithi114.stdout:9/682: mkdir d2/d33/d3f/ddd 0 2024-02-16T22:22:10.398 INFO:tasks.workunit.client.1.smithi123.stdout:1/834: creat d3/d6/db/d10/d2a/f11b x:0 0 0 2024-02-16T22:22:10.400 INFO:tasks.workunit.client.1.smithi123.stdout:5/830: dread d6/dd/df/d58/f63 [0,4194304] 0 2024-02-16T22:22:10.400 INFO:tasks.workunit.client.1.smithi123.stdout:1/835: write d3/d6/d11/fe4 [525830,75356] 0 2024-02-16T22:22:10.401 INFO:tasks.workunit.client.1.smithi123.stdout:5/831: chown d6/fd2 26282 1 2024-02-16T22:22:10.402 INFO:tasks.workunit.client.0.smithi114.stdout:9/683: mknod d2/d5c/d5f/d72/cde 0 2024-02-16T22:22:10.405 INFO:tasks.workunit.client.0.smithi114.stdout:4/777: write dd/d1b/d24/f7a [1415934,4024] 0 2024-02-16T22:22:10.405 INFO:tasks.workunit.client.0.smithi114.stdout:9/684: truncate d2/d1a/d23/d32/d46/d78/fcd 427288 0 2024-02-16T22:22:10.405 INFO:tasks.workunit.client.1.smithi123.stdout:2/839: read f2 [2614878,115331] 0 2024-02-16T22:22:10.409 INFO:tasks.workunit.client.0.smithi114.stdout:7/751: rename d3/d87/lf2 to d3/d46/l108 0 2024-02-16T22:22:10.410 INFO:tasks.workunit.client.1.smithi123.stdout:4/921: mknod d5/d1c/d23/d56/c121 0 2024-02-16T22:22:10.411 INFO:tasks.workunit.client.0.smithi114.stdout:4/778: fdatasync dd/d1b/d24/f4a 0 2024-02-16T22:22:10.412 INFO:tasks.workunit.client.1.smithi123.stdout:9/960: fsync d4/d36/d8e/d58/d59/d60/deb/fec 0 2024-02-16T22:22:10.414 INFO:tasks.workunit.client.1.smithi123.stdout:7/837: fdatasync d3/d6e/d1d/db2/d35/f4b 0 2024-02-16T22:22:10.414 INFO:tasks.workunit.client.1.smithi123.stdout:7/838: chown d3/d87/da8 158516925 1 2024-02-16T22:22:10.416 INFO:tasks.workunit.client.0.smithi114.stdout:0/722: link d1/l7b d1/d16/d1f/d28/d51/d61/d7e/d90/le0 0 2024-02-16T22:22:10.419 INFO:tasks.workunit.client.0.smithi114.stdout:0/723: dread d1/d16/d1f/d28/d51/fb3 [0,4194304] 0 2024-02-16T22:22:10.420 INFO:tasks.workunit.client.1.smithi123.stdout:7/839: dread d3/d6e/d1d/fcb [0,4194304] 0 2024-02-16T22:22:10.421 INFO:tasks.workunit.client.1.smithi123.stdout:0/769: link d1/d4/f43 d1/d8/d13/f100 0 2024-02-16T22:22:10.422 INFO:tasks.workunit.client.1.smithi123.stdout:7/840: fdatasync d3/d6e/d1d/db2/f99 0 2024-02-16T22:22:10.425 INFO:tasks.workunit.client.1.smithi123.stdout:3/875: rename d0/d20/d29/d9b/dbd/df4/d5d/d60/dd6 to d0/d20/d29/d9b/dbd/df4/d5d/dae/dc7/d121 0 2024-02-16T22:22:10.429 INFO:tasks.workunit.client.0.smithi114.stdout:4/779: rename dd/d1b/d37/d38/d87/dad/cce to dd/d59/d8f/df7/c117 0 2024-02-16T22:22:10.430 INFO:tasks.workunit.client.1.smithi123.stdout:1/836: mkdir d3/d6/db/d10/d2a/d11c 0 2024-02-16T22:22:10.432 INFO:tasks.workunit.client.0.smithi114.stdout:0/724: dread d1/d16/d1f/f4d [0,4194304] 0 2024-02-16T22:22:10.433 INFO:tasks.workunit.client.0.smithi114.stdout:9/685: unlink d2/dd/d3c/fb7 0 2024-02-16T22:22:10.435 INFO:tasks.workunit.client.1.smithi123.stdout:5/832: mkdir d6/dc1/de6/d124 0 2024-02-16T22:22:10.436 INFO:tasks.workunit.client.0.smithi114.stdout:8/656: dwrite d4/d11/d52/fa9 [0,4194304] 0 2024-02-16T22:22:10.436 INFO:tasks.workunit.client.0.smithi114.stdout:8/657: stat d4/d16/c23 0 2024-02-16T22:22:10.437 INFO:tasks.workunit.client.1.smithi123.stdout:2/840: rmdir d4 39 2024-02-16T22:22:10.440 INFO:tasks.workunit.client.0.smithi114.stdout:7/752: link d3/d22/d61/d6a/fc9 d3/d22/d73/da7/d6f/d89/da6/dfc/f109 0 2024-02-16T22:22:10.442 INFO:tasks.workunit.client.1.smithi123.stdout:8/872: truncate d0/d61/f114 506349 0 2024-02-16T22:22:10.442 INFO:tasks.workunit.client.0.smithi114.stdout:7/753: chown d3/d22/d73/da7/d6f/f90 230 1 2024-02-16T22:22:10.451 INFO:tasks.workunit.client.0.smithi114.stdout:0/725: dread d1/d16/d48/d98/fde [0,4194304] 0 2024-02-16T22:22:10.451 INFO:tasks.workunit.client.0.smithi114.stdout:9/686: fsync d2/d5c/dc9/d47/d63/fa7 0 2024-02-16T22:22:10.453 INFO:tasks.workunit.client.0.smithi114.stdout:9/687: write d2/d5c/dc9/d1d/f62 [2193112,28336] 0 2024-02-16T22:22:10.456 INFO:tasks.workunit.client.0.smithi114.stdout:8/658: chown d4/d16/d25/la7 428860 1 2024-02-16T22:22:10.461 INFO:tasks.workunit.client.1.smithi123.stdout:0/770: creat d1/d4/d17/d54/d8e/d96/f101 x:0 0 0 2024-02-16T22:22:10.463 INFO:tasks.workunit.client.1.smithi123.stdout:7/841: dread - d3/d6e/d1d/db2/d35/dba/fc3 zero size 2024-02-16T22:22:10.466 INFO:tasks.workunit.client.0.smithi114.stdout:7/754: fsync d3/d22/d73/da7/d6f/d89/da6/dfc/f109 0 2024-02-16T22:22:10.467 INFO:tasks.workunit.client.0.smithi114.stdout:7/755: stat d3/d87/fe4 0 2024-02-16T22:22:10.469 INFO:tasks.workunit.client.1.smithi123.stdout:5/833: creat d6/dd/d44/dba/f125 x:0 0 0 2024-02-16T22:22:10.472 INFO:tasks.workunit.client.0.smithi114.stdout:0/726: rename d1/d16/d18/f77 to d1/d16/d1f/d28/d51/d61/d7e/fe1 0 2024-02-16T22:22:10.474 INFO:tasks.workunit.client.0.smithi114.stdout:8/659: mkdir d4/d24/d6d/dc7/df0 0 2024-02-16T22:22:10.475 INFO:tasks.workunit.client.0.smithi114.stdout:4/780: dwrite dd/d59/dfa/f10d [0,4194304] 0 2024-02-16T22:22:10.475 INFO:tasks.workunit.client.1.smithi123.stdout:8/873: creat d0/d29/da7/d26/d68/dbe/dbf/f120 x:0 0 0 2024-02-16T22:22:10.476 INFO:tasks.workunit.client.1.smithi123.stdout:8/874: write d0/d29/da7/d26/d2a/d77/fde [647483,99357] 0 2024-02-16T22:22:10.478 INFO:tasks.workunit.client.1.smithi123.stdout:4/922: fsync d5/d1c/f25 0 2024-02-16T22:22:10.481 INFO:tasks.workunit.client.0.smithi114.stdout:8/660: dwrite d4/d11/f20 [0,4194304] 0 2024-02-16T22:22:10.481 INFO:tasks.workunit.client.0.smithi114.stdout:6/676: sync 2024-02-16T22:22:10.484 INFO:tasks.workunit.client.1.smithi123.stdout:3/876: write d0/d20/d29/d9b/dbd/df4/d5d/dae/db0/fc3 [1548118,3116] 0 2024-02-16T22:22:10.488 INFO:tasks.workunit.client.0.smithi114.stdout:6/677: read d6/dd/d25/f99 [1260739,87218] 0 2024-02-16T22:22:10.489 INFO:tasks.workunit.client.0.smithi114.stdout:6/678: chown d6/dd/d25/d34/d61 3 1 2024-02-16T22:22:10.490 INFO:tasks.workunit.client.0.smithi114.stdout:0/727: mkdir d1/d16/d1f/d28/d51/d61/d6d/d6e/de2 0 2024-02-16T22:22:10.491 INFO:tasks.workunit.client.0.smithi114.stdout:0/728: write d1/d16/d1f/f9a [907785,109659] 0 2024-02-16T22:22:10.493 INFO:tasks.workunit.client.1.smithi123.stdout:0/771: dread - d1/d4/d17/d74/d57/fd1 zero size 2024-02-16T22:22:10.495 INFO:tasks.workunit.client.0.smithi114.stdout:4/781: creat dd/d59/d8f/dea/f118 x:0 0 0 2024-02-16T22:22:10.496 INFO:tasks.workunit.client.1.smithi123.stdout:7/842: fdatasync d3/d6e/d1d/db2/d35/f9f 0 2024-02-16T22:22:10.499 INFO:tasks.workunit.client.1.smithi123.stdout:1/837: link d3/d6/f5b d3/d6/d49/d48/d68/f11d 0 2024-02-16T22:22:10.501 INFO:tasks.workunit.client.0.smithi114.stdout:7/756: dwrite d3/d14/f4d [0,4194304] 0 2024-02-16T22:22:10.502 INFO:tasks.workunit.client.0.smithi114.stdout:0/729: creat d1/d16/d48/fe3 x:0 0 0 2024-02-16T22:22:10.503 INFO:tasks.workunit.client.1.smithi123.stdout:5/834: mkdir d6/dd/d44/dba/d126 0 2024-02-16T22:22:10.504 INFO:tasks.workunit.client.1.smithi123.stdout:5/835: chown d6/dd/df/d58/d3f/d82/fb1 392228833 1 2024-02-16T22:22:10.506 INFO:tasks.workunit.client.1.smithi123.stdout:5/836: read d6/dd/df/f5e [307094,41851] 0 2024-02-16T22:22:10.508 INFO:tasks.workunit.client.1.smithi123.stdout:5/837: fsync d6/dd/df/d58/d3f/dbe/da2/f11e 0 2024-02-16T22:22:10.509 INFO:tasks.workunit.client.1.smithi123.stdout:5/838: write d6/dd/df/d9f/fd7 [426501,50208] 0 2024-02-16T22:22:10.511 INFO:tasks.workunit.client.1.smithi123.stdout:4/923: creat d5/d1c/d64/d7b/f122 x:0 0 0 2024-02-16T22:22:10.512 INFO:tasks.workunit.client.1.smithi123.stdout:5/839: stat d6/dd/d78/da5/lff 0 2024-02-16T22:22:10.513 INFO:tasks.workunit.client.1.smithi123.stdout:8/875: rename d0/d12/f112 to d0/d29/da7/d26/d2a/d4d/f121 0 2024-02-16T22:22:10.513 INFO:tasks.workunit.client.1.smithi123.stdout:5/840: chown d6/dd/df/d76/f107 0 1 2024-02-16T22:22:10.515 INFO:tasks.workunit.client.1.smithi123.stdout:8/876: write d0/d29/da7/d26/d2a/d77/fe8 [2460413,85706] 0 2024-02-16T22:22:10.515 INFO:tasks.workunit.client.0.smithi114.stdout:6/679: rename d6/dd/l39 to d6/dd/d25/lca 0 2024-02-16T22:22:10.517 INFO:tasks.workunit.client.0.smithi114.stdout:8/661: dread d4/d16/d25/d39/f4f [0,4194304] 0 2024-02-16T22:22:10.518 INFO:tasks.workunit.client.1.smithi123.stdout:0/772: dread d1/d4/f11 [0,4194304] 0 2024-02-16T22:22:10.518 INFO:tasks.workunit.client.0.smithi114.stdout:8/662: chown d4/d11/d78/d7a/d31/ld0 2 1 2024-02-16T22:22:10.518 INFO:tasks.workunit.client.0.smithi114.stdout:8/663: stat d4/d24/d2c/c70 0 2024-02-16T22:22:10.520 INFO:tasks.workunit.client.1.smithi123.stdout:9/961: link d4/d36/d8e/cd3 d4/d36/d8e/d58/d59/d60/dc1/c138 0 2024-02-16T22:22:10.522 INFO:tasks.workunit.client.1.smithi123.stdout:4/924: fsync d5/d1c/d5e/f110 0 2024-02-16T22:22:10.525 INFO:tasks.workunit.client.1.smithi123.stdout:1/838: rename d3/d6/d49/d48/d84/db5 to d3/d6/db/d10/d4b/d70/d11e 0 2024-02-16T22:22:10.529 INFO:tasks.workunit.client.1.smithi123.stdout:8/877: dread d0/d29/da7/d26/d2a/d77/df8/fb1 [0,4194304] 0 2024-02-16T22:22:10.531 INFO:tasks.workunit.client.0.smithi114.stdout:8/664: dread d4/d24/d2c/fbe [0,4194304] 0 2024-02-16T22:22:10.535 INFO:tasks.workunit.client.1.smithi123.stdout:0/773: mkdir d1/d8/d13/d1d/d62/d102 0 2024-02-16T22:22:10.537 INFO:tasks.workunit.client.1.smithi123.stdout:9/962: unlink d4/d36/daa/d20/ff7 0 2024-02-16T22:22:10.540 INFO:tasks.workunit.client.1.smithi123.stdout:4/925: creat d5/db8/d11f/dc0/d118/d73/f123 x:0 0 0 2024-02-16T22:22:10.542 INFO:tasks.workunit.client.1.smithi123.stdout:3/877: rename d0/d20/d29/d9b/f10f to d0/d20/d29/df6/f122 0 2024-02-16T22:22:10.544 INFO:tasks.workunit.client.1.smithi123.stdout:5/841: link d6/dd/df/d76/f107 d6/dd/df/d58/d3f/d82/f127 0 2024-02-16T22:22:10.545 INFO:tasks.workunit.client.1.smithi123.stdout:1/839: mknod d3/d6/d18/d9e/c11f 0 2024-02-16T22:22:10.548 INFO:tasks.workunit.client.1.smithi123.stdout:7/843: creat d3/d5/d6c/da6/df1/f112 x:0 0 0 2024-02-16T22:22:10.549 INFO:tasks.workunit.client.1.smithi123.stdout:2/841: getdents d4/d1a/d1d/d13/d25/d98 0 2024-02-16T22:22:10.550 INFO:tasks.workunit.client.1.smithi123.stdout:0/774: rmdir d1/d4/d42 39 2024-02-16T22:22:10.552 INFO:tasks.workunit.client.1.smithi123.stdout:4/926: dread d5/d1c/d23/d74/f5f [4194304,4194304] 0 2024-02-16T22:22:10.553 INFO:tasks.workunit.client.1.smithi123.stdout:4/927: chown d5/db8/d11f/dc0/d118/d73/d11d 845683 1 2024-02-16T22:22:10.554 INFO:tasks.workunit.client.1.smithi123.stdout:4/928: write d5/f1d [4694705,45226] 0 2024-02-16T22:22:10.554 INFO:tasks.workunit.client.1.smithi123.stdout:4/929: chown d5/d1c/d64/fe0 3 1 2024-02-16T22:22:10.563 INFO:tasks.workunit.client.1.smithi123.stdout:0/775: write d1/d4/d17/d74/fc1 [128103,27805] 0 2024-02-16T22:22:10.565 INFO:tasks.workunit.client.1.smithi123.stdout:8/878: dwrite d0/d29/da7/d26/f7a [4194304,4194304] 0 2024-02-16T22:22:10.569 INFO:tasks.workunit.client.1.smithi123.stdout:5/842: write d6/dd/d44/fa3 [278016,84937] 0 2024-02-16T22:22:10.574 INFO:tasks.workunit.client.1.smithi123.stdout:1/840: dread d3/d6/d49/d48/d84/d92/dec/ff6 [0,4194304] 0 2024-02-16T22:22:10.575 INFO:tasks.workunit.client.1.smithi123.stdout:9/963: truncate d4/f85 500997 0 2024-02-16T22:22:10.588 INFO:tasks.workunit.client.1.smithi123.stdout:9/964: mknod d4/c139 0 2024-02-16T22:22:10.596 INFO:tasks.workunit.client.1.smithi123.stdout:3/878: getdents d0 0 2024-02-16T22:22:10.596 INFO:tasks.workunit.client.1.smithi123.stdout:4/930: dwrite d5/d1c/d5e/dc3/f4d [0,4194304] 0 2024-02-16T22:22:10.598 INFO:tasks.workunit.client.1.smithi123.stdout:5/843: creat d6/dd/d44/dba/d126/f128 x:0 0 0 2024-02-16T22:22:10.598 INFO:tasks.workunit.client.0.smithi114.stdout:7/757: creat d3/f10a x:0 0 0 2024-02-16T22:22:10.599 INFO:tasks.workunit.client.0.smithi114.stdout:7/758: read - d3/d22/db7/dea/ff4 zero size 2024-02-16T22:22:10.600 INFO:tasks.workunit.client.1.smithi123.stdout:9/965: symlink d4/d36/d8e/d58/d59/d60/deb/da7/dde/l13a 0 2024-02-16T22:22:10.601 INFO:tasks.workunit.client.0.smithi114.stdout:6/680: creat d6/d83/fcb x:0 0 0 2024-02-16T22:22:10.602 INFO:tasks.workunit.client.1.smithi123.stdout:8/879: write d0/d61/d66/d7e/f89 [3143902,101320] 0 2024-02-16T22:22:10.602 INFO:tasks.workunit.client.1.smithi123.stdout:6/854: sync 2024-02-16T22:22:10.603 INFO:tasks.workunit.client.0.smithi114.stdout:6/681: chown d6/d2c/d38/d41/d60/fbc 6 1 2024-02-16T22:22:10.606 INFO:tasks.workunit.client.1.smithi123.stdout:0/776: rename d1/d8/d13/lc3 to d1/da0/l103 0 2024-02-16T22:22:10.607 INFO:tasks.workunit.client.0.smithi114.stdout:0/730: fsync d1/d1a/d3e/faf 0 2024-02-16T22:22:10.608 INFO:tasks.workunit.client.0.smithi114.stdout:4/782: getdents dd/d2e/d56/ddc/dde 0 2024-02-16T22:22:10.609 INFO:tasks.workunit.client.0.smithi114.stdout:7/759: read - d3/d22/d73/da7/d6f/d9c/fcf zero size 2024-02-16T22:22:10.610 INFO:tasks.workunit.client.0.smithi114.stdout:7/760: chown d3/d22/d61 0 1 2024-02-16T22:22:10.617 INFO:tasks.workunit.client.1.smithi123.stdout:5/844: mknod d6/dd/df/d58/d89/d98/c129 0 2024-02-16T22:22:10.618 INFO:tasks.workunit.client.0.smithi114.stdout:0/731: creat d1/d16/d1f/d28/d51/d61/d7e/fe4 x:0 0 0 2024-02-16T22:22:10.619 INFO:tasks.workunit.client.1.smithi123.stdout:9/966: truncate d4/d36/d8e/f95 4025452 0 2024-02-16T22:22:10.620 INFO:tasks.workunit.client.0.smithi114.stdout:4/783: symlink dd/d2e/d56/ddc/dde/l119 0 2024-02-16T22:22:10.620 INFO:tasks.workunit.client.1.smithi123.stdout:9/967: fdatasync d4/d36/d8e/d84/d93/f114 0 2024-02-16T22:22:10.622 INFO:tasks.workunit.client.1.smithi123.stdout:8/880: symlink d0/d29/da7/d26/d32/d45/l122 0 2024-02-16T22:22:10.623 INFO:tasks.workunit.client.1.smithi123.stdout:5/845: dwrite d6/dd/d44/dba/d126/f128 [0,4194304] 0 2024-02-16T22:22:10.623 INFO:tasks.workunit.client.0.smithi114.stdout:8/665: dread d4/d11/d78/d7a/d36/f5d [0,4194304] 0 2024-02-16T22:22:10.623 INFO:tasks.workunit.client.0.smithi114.stdout:6/682: fdatasync d6/d2c/f46 0 2024-02-16T22:22:10.625 INFO:tasks.workunit.client.1.smithi123.stdout:1/841: rmdir d3/d6/db/d10/d2d/d8c/dfd/d9f/d10a 0 2024-02-16T22:22:10.626 INFO:tasks.workunit.client.0.smithi114.stdout:4/784: dread dd/d59/dfa/f10d [0,4194304] 0 2024-02-16T22:22:10.626 INFO:tasks.workunit.client.0.smithi114.stdout:7/761: symlink d3/d22/d73/da7/d6f/l10b 0 2024-02-16T22:22:10.629 INFO:tasks.workunit.client.1.smithi123.stdout:6/855: rename d0/d9/dc/d3a/d70/f10f to d0/d6/d1a/d88/f134 0 2024-02-16T22:22:10.631 INFO:tasks.workunit.client.1.smithi123.stdout:3/879: dwrite d0/da0/faa [0,4194304] 0 2024-02-16T22:22:10.633 INFO:tasks.workunit.client.1.smithi123.stdout:3/880: write d0/d20/d5a/f108 [830027,109221] 0 2024-02-16T22:22:10.634 INFO:tasks.workunit.client.1.smithi123.stdout:6/856: truncate d0/d6/d2f/dbd/f119 510724 0 2024-02-16T22:22:10.637 INFO:tasks.workunit.client.1.smithi123.stdout:3/881: write d0/d20/d29/d9b/dbd/df4/d5d/d60/dc6/fea [814068,56813] 0 2024-02-16T22:22:10.641 INFO:tasks.workunit.client.0.smithi114.stdout:4/785: creat dd/d2e/d56/ddc/dde/f11a x:0 0 0 2024-02-16T22:22:10.643 INFO:tasks.workunit.client.1.smithi123.stdout:5/846: mkdir d6/dd/df/d58/d3f/d6f/d12a 0 2024-02-16T22:22:10.647 INFO:tasks.workunit.client.0.smithi114.stdout:5/642: sync 2024-02-16T22:22:10.649 INFO:tasks.workunit.client.0.smithi114.stdout:5/643: write d0/d28/d98/d25/d3e/d7f/fa8 [3585862,2655] 0 2024-02-16T22:22:10.651 INFO:tasks.workunit.client.0.smithi114.stdout:5/644: truncate d0/d28/d98/d25/fc0 1298890 0 2024-02-16T22:22:10.651 INFO:tasks.workunit.client.0.smithi114.stdout:0/732: link d1/d1a/d47/cdf d1/d16/d1f/d28/d51/d61/d6d/d6e/ce5 0 2024-02-16T22:22:10.652 INFO:tasks.workunit.client.1.smithi123.stdout:1/842: dread d3/d6/d18/d9e/f86 [0,4194304] 0 2024-02-16T22:22:10.653 INFO:tasks.workunit.client.1.smithi123.stdout:7/844: sync 2024-02-16T22:22:10.654 INFO:tasks.workunit.client.0.smithi114.stdout:5/645: truncate d0/d28/d98/d25/d2e/d3b/fbf 491968 0 2024-02-16T22:22:10.656 INFO:tasks.workunit.client.0.smithi114.stdout:6/683: dread d6/dd/d25/f99 [0,4194304] 0 2024-02-16T22:22:10.657 INFO:tasks.workunit.client.0.smithi114.stdout:4/786: fdatasync dd/d1b/d41/fbb 0 2024-02-16T22:22:10.657 INFO:tasks.workunit.client.1.smithi123.stdout:0/777: dwrite d1/d4/d17/dad/fff [0,4194304] 0 2024-02-16T22:22:10.658 INFO:tasks.workunit.client.1.smithi123.stdout:4/931: truncate d5/db8/d11f/dc0/d118/d73/d87/fdd 1339975 0 2024-02-16T22:22:10.659 INFO:tasks.workunit.client.1.smithi123.stdout:9/968: mknod d4/d36/d8e/d58/d59/d60/d9d/de4/dfa/c13b 0 2024-02-16T22:22:10.663 INFO:tasks.workunit.client.1.smithi123.stdout:5/847: mkdir d6/dd/d44/d12b 0 2024-02-16T22:22:10.664 INFO:tasks.workunit.client.1.smithi123.stdout:5/848: write d6/dd/df/f5e [369627,71305] 0 2024-02-16T22:22:10.667 INFO:tasks.workunit.client.0.smithi114.stdout:8/666: getdents d4/d16/d25/d46 0 2024-02-16T22:22:10.668 INFO:tasks.workunit.client.0.smithi114.stdout:5/646: chown d0/d2d/d30/f40 15655 1 2024-02-16T22:22:10.670 INFO:tasks.workunit.client.1.smithi123.stdout:1/843: fsync d3/f35 0 2024-02-16T22:22:10.673 INFO:tasks.workunit.client.1.smithi123.stdout:6/857: mkdir d0/d8a/d100/d135 0 2024-02-16T22:22:10.676 INFO:tasks.workunit.client.0.smithi114.stdout:5/647: dwrite d0/d28/d98/d25/fc0 [0,4194304] 0 2024-02-16T22:22:10.678 INFO:tasks.workunit.client.0.smithi114.stdout:5/648: write d0/d28/d98/d25/d2e/d3b/fc1 [839129,108696] 0 2024-02-16T22:22:10.687 INFO:tasks.workunit.client.0.smithi114.stdout:7/762: rename d3/d22/d73/da7/d6f/f90 to d3/d46/f10c 0 2024-02-16T22:22:10.688 INFO:tasks.workunit.client.1.smithi123.stdout:7/845: dread d3/d5/f33 [0,4194304] 0 2024-02-16T22:22:10.690 INFO:tasks.workunit.client.0.smithi114.stdout:3/673: sync 2024-02-16T22:22:10.690 INFO:tasks.workunit.client.0.smithi114.stdout:1/650: sync 2024-02-16T22:22:10.690 INFO:tasks.workunit.client.0.smithi114.stdout:9/688: sync 2024-02-16T22:22:10.690 INFO:tasks.workunit.client.0.smithi114.stdout:2/630: sync 2024-02-16T22:22:10.691 INFO:tasks.workunit.client.0.smithi114.stdout:3/674: chown d4/d46/fc0 2 1 2024-02-16T22:22:10.693 INFO:tasks.workunit.client.0.smithi114.stdout:4/787: write dd/f20 [52577,54754] 0 2024-02-16T22:22:10.694 INFO:tasks.workunit.client.1.smithi123.stdout:2/842: sync 2024-02-16T22:22:10.700 INFO:tasks.workunit.client.1.smithi123.stdout:9/969: rmdir d4/d36/d8e/d58/d59/d60/deb/da7/dde 39 2024-02-16T22:22:10.700 INFO:tasks.workunit.client.1.smithi123.stdout:4/932: read d5/d49/fb7 [42045,91750] 0 2024-02-16T22:22:10.701 INFO:tasks.workunit.client.1.smithi123.stdout:8/881: rename d0/df to d0/d29/da7/d26/d2a/d123 0 2024-02-16T22:22:10.702 INFO:tasks.workunit.client.1.smithi123.stdout:8/882: chown d0/d61/d66/d7e/f10a 87593095 1 2024-02-16T22:22:10.703 INFO:tasks.workunit.client.0.smithi114.stdout:5/649: mknod d0/df/d5e/cd5 0 2024-02-16T22:22:10.704 INFO:tasks.workunit.client.0.smithi114.stdout:6/684: getdents d6/dd/d25/d34/d61/d62/d8b/db7 0 2024-02-16T22:22:10.707 INFO:tasks.workunit.client.0.smithi114.stdout:0/733: rename d1/d16/d48/d98/fcd to d1/d16/d1f/d28/fe6 0 2024-02-16T22:22:10.709 INFO:tasks.workunit.client.0.smithi114.stdout:7/763: truncate d3/d22/d73/da7/d4e/f6e 1305591 0 2024-02-16T22:22:10.714 INFO:tasks.workunit.client.1.smithi123.stdout:0/778: mknod d1/d4/d42/c104 0 2024-02-16T22:22:10.714 INFO:tasks.workunit.client.0.smithi114.stdout:1/651: chown d4/d9/d46/d67/d92/dca/dce/cda 1577119 1 2024-02-16T22:22:10.716 INFO:tasks.workunit.client.0.smithi114.stdout:8/667: dread d4/d24/d2c/f32 [0,4194304] 0 2024-02-16T22:22:10.717 INFO:tasks.workunit.client.0.smithi114.stdout:8/668: chown d4/cb1 963 1 2024-02-16T22:22:10.719 INFO:tasks.workunit.client.0.smithi114.stdout:7/764: dwrite d3/d5a/fd7 [0,4194304] 0 2024-02-16T22:22:10.719 INFO:tasks.workunit.client.1.smithi123.stdout:0/779: dwrite d1/d4/d17/d54/fd9 [0,4194304] 0 2024-02-16T22:22:10.724 INFO:tasks.workunit.client.1.smithi123.stdout:2/843: rename d4/dda/fe2 to d4/dda/f112 0 2024-02-16T22:22:10.726 INFO:tasks.workunit.client.0.smithi114.stdout:5/650: creat d0/d28/d98/d1e/d63/d96/d9a/fd6 x:0 0 0 2024-02-16T22:22:10.728 INFO:tasks.workunit.client.0.smithi114.stdout:5/651: write d0/d28/d98/d25/d2e/d3b/f77 [3846550,21007] 0 2024-02-16T22:22:10.731 INFO:tasks.workunit.client.1.smithi123.stdout:6/858: dread d0/f6d [0,4194304] 0 2024-02-16T22:22:10.732 INFO:tasks.workunit.client.1.smithi123.stdout:6/859: chown d0/d9/f6a 0 1 2024-02-16T22:22:10.734 INFO:tasks.workunit.client.0.smithi114.stdout:0/734: write d1/d16/d1f/d28/fe6 [289326,121130] 0 2024-02-16T22:22:10.740 INFO:tasks.workunit.client.0.smithi114.stdout:9/689: fsync d2/d1a/d65/f8f 0 2024-02-16T22:22:10.742 INFO:tasks.workunit.client.1.smithi123.stdout:7/846: dwrite d3/d5/f33 [0,4194304] 0 2024-02-16T22:22:10.743 INFO:tasks.workunit.client.0.smithi114.stdout:1/652: readlink d4/l60 0 2024-02-16T22:22:10.743 INFO:tasks.workunit.client.0.smithi114.stdout:1/653: chown d4/f30 0 1 2024-02-16T22:22:10.745 INFO:tasks.workunit.client.0.smithi114.stdout:7/765: fdatasync d3/d92/fa0 0 2024-02-16T22:22:10.748 INFO:tasks.workunit.client.0.smithi114.stdout:3/675: dwrite d4/d46/d48/f8e [0,4194304] 0 2024-02-16T22:22:10.752 INFO:tasks.workunit.client.0.smithi114.stdout:8/669: creat d4/d24/d2c/d67/dcf/ff1 x:0 0 0 2024-02-16T22:22:10.756 INFO:tasks.workunit.client.0.smithi114.stdout:4/788: symlink dd/d1b/d37/d61/df4/l11b 0 2024-02-16T22:22:10.756 INFO:tasks.workunit.client.0.smithi114.stdout:3/676: dread d4/d24/d2d/d45/d68/fb9 [0,4194304] 0 2024-02-16T22:22:10.757 INFO:tasks.workunit.client.0.smithi114.stdout:3/677: chown d4/dd/d33/d76 0 1 2024-02-16T22:22:10.758 INFO:tasks.workunit.client.0.smithi114.stdout:6/685: dwrite d6/dd/d25/f27 [0,4194304] 0 2024-02-16T22:22:10.759 INFO:tasks.workunit.client.0.smithi114.stdout:6/686: chown d6/le 7384 1 2024-02-16T22:22:10.761 INFO:tasks.workunit.client.0.smithi114.stdout:5/652: mkdir d0/d28/d98/d1e/dd7 0 2024-02-16T22:22:10.762 INFO:tasks.workunit.client.0.smithi114.stdout:5/653: dread - d0/d28/d98/d25/d2e/d3b/d8d/fc9 zero size 2024-02-16T22:22:10.765 INFO:tasks.workunit.client.0.smithi114.stdout:2/631: creat dd/fd4 x:0 0 0 2024-02-16T22:22:10.765 INFO:tasks.workunit.client.0.smithi114.stdout:2/632: readlink dd/lbc 0 2024-02-16T22:22:10.770 INFO:tasks.workunit.client.0.smithi114.stdout:1/654: mkdir d4/d9/d3d/def 0 2024-02-16T22:22:10.778 INFO:tasks.workunit.client.0.smithi114.stdout:4/789: mknod dd/d1b/db0/c11c 0 2024-02-16T22:22:10.781 INFO:tasks.workunit.client.0.smithi114.stdout:6/687: mkdir d6/d2c/d5e/dcc 0 2024-02-16T22:22:10.783 INFO:tasks.workunit.client.0.smithi114.stdout:3/678: mknod d4/d46/dcf/ce8 0 2024-02-16T22:22:10.786 INFO:tasks.workunit.client.0.smithi114.stdout:5/654: write d0/d28/d98/d25/f73 [955924,34274] 0 2024-02-16T22:22:10.790 INFO:tasks.workunit.client.0.smithi114.stdout:2/633: symlink dd/df/d1c/d21/d61/d69/ld5 0 2024-02-16T22:22:10.791 INFO:tasks.workunit.client.0.smithi114.stdout:3/679: dwrite d4/d87/faf [0,4194304] 0 2024-02-16T22:22:10.791 INFO:tasks.workunit.client.0.smithi114.stdout:3/680: chown d4/d43 157688 1 2024-02-16T22:22:10.793 INFO:tasks.workunit.client.0.smithi114.stdout:9/690: mkdir d2/d5c/ddf 0 2024-02-16T22:22:10.799 INFO:tasks.workunit.client.0.smithi114.stdout:7/766: write d3/d22/d73/da7/d4e/fc0 [917291,80811] 0 2024-02-16T22:22:10.801 INFO:tasks.workunit.client.0.smithi114.stdout:3/681: dwrite d4/d24/d2d/d45/d68/f86 [4194304,4194304] 0 2024-02-16T22:22:10.805 INFO:tasks.workunit.client.0.smithi114.stdout:1/655: fsync d4/d63/d69/d7c/fbd 0 2024-02-16T22:22:10.808 INFO:tasks.workunit.client.0.smithi114.stdout:4/790: mkdir dd/d1b/d37/d61/d11d 0 2024-02-16T22:22:10.809 INFO:tasks.workunit.client.0.smithi114.stdout:6/688: creat d6/fcd x:0 0 0 2024-02-16T22:22:10.815 INFO:tasks.workunit.client.0.smithi114.stdout:2/634: readlink dd/df/d1c/d21/d61/d63/d83/l8e 0 2024-02-16T22:22:10.817 INFO:tasks.workunit.client.0.smithi114.stdout:9/691: dread d2/f15 [0,4194304] 0 2024-02-16T22:22:10.822 INFO:tasks.workunit.client.0.smithi114.stdout:7/767: creat d3/d14/f10d x:0 0 0 2024-02-16T22:22:10.826 INFO:tasks.workunit.client.0.smithi114.stdout:1/656: mkdir d4/d63/d69/d7c/d9b/df0 0 2024-02-16T22:22:10.829 INFO:tasks.workunit.client.0.smithi114.stdout:3/682: rename d4/d26/f34 to d4/dd/fe9 0 2024-02-16T22:22:10.831 INFO:tasks.workunit.client.1.smithi123.stdout:4/933: symlink d5/d1c/d23/d6d/d111/dd8/l124 0 2024-02-16T22:22:10.833 INFO:tasks.workunit.client.0.smithi114.stdout:3/683: dread d4/d24/d2d/d45/d68/f86 [4194304,4194304] 0 2024-02-16T22:22:10.836 INFO:tasks.workunit.client.1.smithi123.stdout:3/882: rename d0/d20/d29/d9b/dbd/df4/d5d/d60/de9 to d0/d20/d29/d9b/dbd/df4/d5d/dae/db0/d123 0 2024-02-16T22:22:10.837 INFO:tasks.workunit.client.0.smithi114.stdout:4/791: creat dd/d1b/d24/d98/f11e x:0 0 0 2024-02-16T22:22:10.842 INFO:tasks.workunit.client.0.smithi114.stdout:5/655: creat d0/d2d/d30/db3/fd8 x:0 0 0 2024-02-16T22:22:10.846 INFO:tasks.workunit.client.0.smithi114.stdout:9/692: truncate d2/d5c/dc9/d47/d63/fa7 3038765 0 2024-02-16T22:22:10.846 INFO:tasks.workunit.client.0.smithi114.stdout:4/792: dwrite dd/d1b/d41/d74/fd9 [0,4194304] 0 2024-02-16T22:22:10.849 INFO:tasks.workunit.client.1.smithi123.stdout:0/780: rmdir d1/d4/d17 39 2024-02-16T22:22:10.856 INFO:tasks.workunit.client.0.smithi114.stdout:8/670: dwrite d4/d11/d78/d7a/d36/f6b [0,4194304] 0 2024-02-16T22:22:10.857 INFO:tasks.workunit.client.0.smithi114.stdout:6/689: dread d6/f35 [4194304,4194304] 0 2024-02-16T22:22:10.863 INFO:tasks.workunit.client.1.smithi123.stdout:9/970: truncate d4/f85 339052 0 2024-02-16T22:22:10.864 INFO:tasks.workunit.client.1.smithi123.stdout:9/971: chown d4/d1e/d3a/dac/dc5 16453 1 2024-02-16T22:22:10.865 INFO:tasks.workunit.client.1.smithi123.stdout:9/972: readlink d4/d1e/l80 0 2024-02-16T22:22:10.868 INFO:tasks.workunit.client.0.smithi114.stdout:5/656: unlink d0/d28/d98/d1e/d63/d9e/ca7 0 2024-02-16T22:22:10.874 INFO:tasks.workunit.client.0.smithi114.stdout:7/768: write d3/d46/da2/ff5 [810464,36506] 0 2024-02-16T22:22:10.875 INFO:tasks.workunit.client.1.smithi123.stdout:1/844: creat d3/d6/db/f120 x:0 0 0 2024-02-16T22:22:10.877 INFO:tasks.workunit.client.0.smithi114.stdout:9/693: rename d2/d33/d3f/f7e to d2/dd0/fe0 0 2024-02-16T22:22:10.881 INFO:tasks.workunit.client.1.smithi123.stdout:6/860: write d0/d9/f6a [1845268,80749] 0 2024-02-16T22:22:10.882 INFO:tasks.workunit.client.1.smithi123.stdout:2/844: symlink d4/d1a/l113 0 2024-02-16T22:22:10.885 INFO:tasks.workunit.client.0.smithi114.stdout:6/690: mkdir d6/dd/d25/d34/d58/d9f/da3/dce 0 2024-02-16T22:22:10.886 INFO:tasks.workunit.client.0.smithi114.stdout:5/657: dread d0/d2d/d30/f31 [0,4194304] 0 2024-02-16T22:22:10.887 INFO:tasks.workunit.client.1.smithi123.stdout:0/781: dwrite d1/d4/d17/d54/d8e/d96/ffe [0,4194304] 0 2024-02-16T22:22:10.891 INFO:tasks.workunit.client.1.smithi123.stdout:9/973: fdatasync d4/d36/d57/f7d 0 2024-02-16T22:22:10.894 INFO:tasks.workunit.client.1.smithi123.stdout:5/849: rename d6/dac to d6/dd/d44/dba/d126/d12c 0 2024-02-16T22:22:10.895 INFO:tasks.workunit.client.0.smithi114.stdout:1/657: dread d4/d9/d46/d67/d92/fa6 [0,4194304] 0 2024-02-16T22:22:10.896 INFO:tasks.workunit.client.1.smithi123.stdout:3/883: dwrite d0/d20/d2e/f48 [0,4194304] 0 2024-02-16T22:22:10.897 INFO:tasks.workunit.client.1.smithi123.stdout:4/934: dread d5/d1c/d64/fb6 [0,4194304] 0 2024-02-16T22:22:10.898 INFO:tasks.workunit.client.0.smithi114.stdout:4/793: mkdir dd/d2e/de7/d11f 0 2024-02-16T22:22:10.901 INFO:tasks.workunit.client.0.smithi114.stdout:7/769: mknod d3/d22/d61/c10e 0 2024-02-16T22:22:10.908 INFO:tasks.workunit.client.0.smithi114.stdout:6/691: dread d6/d2c/d38/f92 [0,4194304] 0 2024-02-16T22:22:10.913 INFO:tasks.workunit.client.0.smithi114.stdout:9/694: dread d2/d5c/d5f/d72/fb8 [0,4194304] 0 2024-02-16T22:22:10.913 INFO:tasks.workunit.client.0.smithi114.stdout:6/692: dread d6/da/f9b [0,4194304] 0 2024-02-16T22:22:10.916 INFO:tasks.workunit.client.1.smithi123.stdout:9/974: mknod d4/d36/d8e/d6f/c13c 0 2024-02-16T22:22:10.916 INFO:tasks.workunit.client.1.smithi123.stdout:9/975: fdatasync d4/d36/daa/d20/f132 0 2024-02-16T22:22:10.917 INFO:tasks.workunit.client.0.smithi114.stdout:3/684: creat d4/d24/d2d/d45/d68/fea x:0 0 0 2024-02-16T22:22:10.919 INFO:tasks.workunit.client.1.smithi123.stdout:8/883: rename d0/d29/da7/d26/d68/d81/d47/f91 to d0/d29/dba/f124 0 2024-02-16T22:22:10.921 INFO:tasks.workunit.client.1.smithi123.stdout:5/850: rmdir d6/dd/df/d90 39 2024-02-16T22:22:10.922 INFO:tasks.workunit.client.0.smithi114.stdout:2/635: getdents dd/d3b/da7/d8b 0 2024-02-16T22:22:10.923 INFO:tasks.workunit.client.1.smithi123.stdout:3/884: readlink d0/d20/d29/d9b/dbd/df4/d5d/d60/d93/l115 0 2024-02-16T22:22:10.923 INFO:tasks.workunit.client.1.smithi123.stdout:2/845: write d4/d1a/d1d/d13/d25/d72/f73 [97219,36972] 0 2024-02-16T22:22:10.924 INFO:tasks.workunit.client.1.smithi123.stdout:2/846: write d4/dae/f105 [28647,95484] 0 2024-02-16T22:22:10.925 INFO:tasks.workunit.client.0.smithi114.stdout:8/671: truncate d4/d24/d2c/f56 3918982 0 2024-02-16T22:22:10.926 INFO:tasks.workunit.client.1.smithi123.stdout:2/847: fsync d4/d1a/dd5/d10b/df1/d4a/da2/fee 0 2024-02-16T22:22:10.927 INFO:tasks.workunit.client.1.smithi123.stdout:0/782: write d1/da0/faa [817938,54944] 0 2024-02-16T22:22:10.928 INFO:tasks.workunit.client.1.smithi123.stdout:6/861: creat d0/d9/dc/d3a/d70/dd5/f136 x:0 0 0 2024-02-16T22:22:10.929 INFO:tasks.workunit.client.0.smithi114.stdout:1/658: creat d4/d9/d46/d4d/ff1 x:0 0 0 2024-02-16T22:22:10.931 INFO:tasks.workunit.client.0.smithi114.stdout:4/794: symlink dd/d1b/d24/d57/l120 0 2024-02-16T22:22:10.933 INFO:tasks.workunit.client.1.smithi123.stdout:7/847: getdents d3/d6e/d1d/db2/d35/db7 0 2024-02-16T22:22:10.936 INFO:tasks.workunit.client.0.smithi114.stdout:9/695: creat d2/d33/d3f/fe1 x:0 0 0 2024-02-16T22:22:10.936 INFO:tasks.workunit.client.0.smithi114.stdout:5/658: dwrite d0/d28/d98/d1e/d86/fa9 [0,4194304] 0 2024-02-16T22:22:10.936 INFO:tasks.workunit.client.1.smithi123.stdout:5/851: fdatasync d6/dd/df/d1f/f10d 0 2024-02-16T22:22:10.940 INFO:tasks.workunit.client.0.smithi114.stdout:3/685: mknod d4/d46/d48/ceb 0 2024-02-16T22:22:10.945 INFO:tasks.workunit.client.0.smithi114.stdout:1/659: fsync d4/d9/d46/d4d/f4c 0 2024-02-16T22:22:10.948 INFO:tasks.workunit.client.0.smithi114.stdout:8/672: read d4/d11/f60 [1041719,108641] 0 2024-02-16T22:22:10.948 INFO:tasks.workunit.client.0.smithi114.stdout:4/795: fsync dd/f78 0 2024-02-16T22:22:10.949 INFO:tasks.workunit.client.1.smithi123.stdout:3/885: dread d0/d20/d94/feb [0,4194304] 0 2024-02-16T22:22:10.949 INFO:tasks.workunit.client.1.smithi123.stdout:8/884: dread d0/d29/f64 [0,4194304] 0 2024-02-16T22:22:10.952 INFO:tasks.workunit.client.0.smithi114.stdout:7/770: link d3/ff d3/d22/d61/f10f 0 2024-02-16T22:22:10.954 INFO:tasks.workunit.client.1.smithi123.stdout:5/852: rename d6/dd/df/d58/d3f/cf3 to d6/dc1/de6/c12d 0 2024-02-16T22:22:10.955 INFO:tasks.workunit.client.1.smithi123.stdout:5/853: fsync d6/dd/df/d58/d89/d98/fc8 0 2024-02-16T22:22:10.959 INFO:tasks.workunit.client.1.smithi123.stdout:6/862: mknod d0/d9/dc/d3a/d70/c137 0 2024-02-16T22:22:10.964 INFO:tasks.workunit.client.1.smithi123.stdout:3/886: mknod d0/d20/d29/d9b/dbd/df4/d37/d41/c124 0 2024-02-16T22:22:10.965 INFO:tasks.workunit.client.0.smithi114.stdout:6/693: dwrite d6/dd/d25/d34/d61/d62/f8d [4194304,4194304] 0 2024-02-16T22:22:10.965 INFO:tasks.workunit.client.1.smithi123.stdout:3/887: chown d0/d20/d2e/c5b 5750902 1 2024-02-16T22:22:10.966 INFO:tasks.workunit.client.1.smithi123.stdout:7/848: write d3/d87/f4a [2671782,4629] 0 2024-02-16T22:22:10.966 INFO:tasks.workunit.client.1.smithi123.stdout:2/848: dwrite d4/d1a/d1d/d59/f3a [0,4194304] 0 2024-02-16T22:22:10.967 INFO:tasks.workunit.client.1.smithi123.stdout:7/849: readlink d3/d6e/d1d/db2/ldf 0 2024-02-16T22:22:10.972 INFO:tasks.workunit.client.0.smithi114.stdout:8/673: rename d4/d11/fc0 to d4/d24/d6d/ff2 0 2024-02-16T22:22:10.973 INFO:tasks.workunit.client.0.smithi114.stdout:8/674: chown d4/d11/d78/f9c 0 1 2024-02-16T22:22:10.978 INFO:tasks.workunit.client.0.smithi114.stdout:7/771: creat d3/d22/db7/dea/f110 x:0 0 0 2024-02-16T22:22:10.979 INFO:tasks.workunit.client.0.smithi114.stdout:9/696: write d2/d1a/f39 [2052543,81191] 0 2024-02-16T22:22:10.981 INFO:tasks.workunit.client.0.smithi114.stdout:3/686: symlink d4/dbe/lec 0 2024-02-16T22:22:10.981 INFO:tasks.workunit.client.0.smithi114.stdout:3/687: chown d4/dbe 131135 1 2024-02-16T22:22:10.982 INFO:tasks.workunit.client.1.smithi123.stdout:1/845: sync 2024-02-16T22:22:10.983 INFO:tasks.workunit.client.0.smithi114.stdout:3/688: write d4/dd/d33/d72/fbf [581574,96204] 0 2024-02-16T22:22:10.984 INFO:tasks.workunit.client.1.smithi123.stdout:4/935: getdents d5/db8/d11f/dc0/d118 0 2024-02-16T22:22:10.984 INFO:tasks.workunit.client.1.smithi123.stdout:2/849: chown d4/d1a/d1d/d38/d5e/c8c 20727562 1 2024-02-16T22:22:10.984 INFO:tasks.workunit.client.1.smithi123.stdout:5/854: creat d6/dd/df/d58/d3f/dbe/da2/f12e x:0 0 0 2024-02-16T22:22:10.984 INFO:tasks.workunit.client.1.smithi123.stdout:6/863: creat d0/d6/d4d/db0/f138 x:0 0 0 2024-02-16T22:22:10.984 INFO:tasks.workunit.client.0.smithi114.stdout:3/689: chown d4/d87/l97 473 1 2024-02-16T22:22:10.985 INFO:tasks.workunit.client.1.smithi123.stdout:8/885: mkdir d0/d29/da7/d125 0 2024-02-16T22:22:10.985 INFO:tasks.workunit.client.0.smithi114.stdout:1/660: mkdir d4/d9/d46/d67/d92/dca/dcc/d98/ddd/df2 0 2024-02-16T22:22:10.986 INFO:tasks.workunit.client.1.smithi123.stdout:8/886: dread - d0/d29/dba/f11e zero size 2024-02-16T22:22:10.986 INFO:tasks.workunit.client.1.smithi123.stdout:6/864: dread - d0/d74/f113 zero size 2024-02-16T22:22:10.987 INFO:tasks.workunit.client.1.smithi123.stdout:5/855: chown d6/dd/df/d58/d89/dc3/f115 15521 1 2024-02-16T22:22:10.990 INFO:tasks.workunit.client.1.smithi123.stdout:2/850: mknod d4/d1a/dd5/d10b/df1/db1/de3/c114 0 2024-02-16T22:22:10.991 INFO:tasks.workunit.client.1.smithi123.stdout:3/888: write d0/d20/d2e/f95 [4354868,77298] 0 2024-02-16T22:22:10.992 INFO:tasks.workunit.client.0.smithi114.stdout:8/675: rename d4/d11/d52/fb4 to d4/d16/d25/d46/db5/ff3 0 2024-02-16T22:22:10.992 INFO:tasks.workunit.client.0.smithi114.stdout:8/676: chown d4/d24/d2c/d67/dc8 106657660 1 2024-02-16T22:22:10.993 INFO:tasks.workunit.client.0.smithi114.stdout:4/796: symlink dd/d1b/d41/dc3/dfd/l121 0 2024-02-16T22:22:10.994 INFO:tasks.workunit.client.1.smithi123.stdout:4/936: dread d5/d1c/d23/d74/f116 [0,4194304] 0 2024-02-16T22:22:10.994 INFO:tasks.workunit.client.0.smithi114.stdout:8/677: write d4/d11/f71 [9404864,25614] 0 2024-02-16T22:22:10.995 INFO:tasks.workunit.client.0.smithi114.stdout:8/678: readlink d4/d11/d78/d7a/d36/d3b/l41 0 2024-02-16T22:22:10.996 INFO:tasks.workunit.client.1.smithi123.stdout:5/856: mkdir d6/dc1/d12f 0 2024-02-16T22:22:10.999 INFO:tasks.workunit.client.1.smithi123.stdout:6/865: rename d0/d9/dc/d3a/d52/l6b to d0/d96/dca/d10c/l139 0 2024-02-16T22:22:11.001 INFO:tasks.workunit.client.1.smithi123.stdout:6/866: dread d0/d6/d2f/dbd/f119 [0,4194304] 0 2024-02-16T22:22:11.044 INFO:tasks.workunit.client.1.smithi123.stdout:9/976: sync 2024-02-16T22:22:11.070 INFO:tasks.workunit.client.1.smithi123.stdout:0/783: sync 2024-02-16T22:22:11.077 INFO:tasks.workunit.client.0.smithi114.stdout:0/735: sync 2024-02-16T22:22:11.097 INFO:tasks.workunit.client.1.smithi123.stdout:1/846: truncate d3/d6/db/d10/d2d/d8c/dfd/de8/fbd 896217 0 2024-02-16T22:22:11.102 INFO:tasks.workunit.client.0.smithi114.stdout:3/690: readlink d4/d24/l60 0 2024-02-16T22:22:11.105 INFO:tasks.workunit.client.1.smithi123.stdout:8/887: dwrite d0/d29/da7/fc9 [0,4194304] 0 2024-02-16T22:22:11.112 INFO:tasks.workunit.client.1.smithi123.stdout:8/888: dread d0/d29/da7/d43/d34/f113 [4194304,4194304] 0 2024-02-16T22:22:11.120 INFO:tasks.workunit.client.0.smithi114.stdout:4/797: creat dd/d1b/d24/d57/f122 x:0 0 0 2024-02-16T22:22:11.120 INFO:tasks.workunit.client.1.smithi123.stdout:5/857: dread d6/fd2 [0,4194304] 0 2024-02-16T22:22:11.131 INFO:tasks.workunit.client.1.smithi123.stdout:6/867: mknod d0/d9/dc/d3a/c13a 0 2024-02-16T22:22:11.131 INFO:tasks.workunit.client.0.smithi114.stdout:5/659: getdents d0/d28/d98/d1e/d63/d9e 0 2024-02-16T22:22:11.132 INFO:tasks.workunit.client.0.smithi114.stdout:5/660: chown d0/df/d5e 0 1 2024-02-16T22:22:11.133 INFO:tasks.workunit.client.0.smithi114.stdout:5/661: write d0/d28/d98/d25/d2e/f56 [3781343,63901] 0 2024-02-16T22:22:11.141 INFO:tasks.workunit.client.0.smithi114.stdout:1/661: mkdir d4/d63/d69/d7c/d9b/db0/df3 0 2024-02-16T22:22:11.142 INFO:tasks.workunit.client.0.smithi114.stdout:1/662: stat d4/l3b 0 2024-02-16T22:22:11.154 INFO:tasks.workunit.client.0.smithi114.stdout:9/697: creat d2/d1a/d65/fe2 x:0 0 0 2024-02-16T22:22:11.155 INFO:tasks.workunit.client.0.smithi114.stdout:9/698: readlink d2/d1a/d23/lb4 0 2024-02-16T22:22:11.156 INFO:tasks.workunit.client.1.smithi123.stdout:5/858: fdatasync d6/d3b/fa7 0 2024-02-16T22:22:11.157 INFO:tasks.workunit.client.0.smithi114.stdout:5/662: creat d0/d28/d98/d1e/d63/d96/fd9 x:0 0 0 2024-02-16T22:22:11.159 INFO:tasks.workunit.client.0.smithi114.stdout:5/663: read d0/d28/d98/d25/d2e/d3b/f85 [831632,44303] 0 2024-02-16T22:22:11.161 INFO:tasks.workunit.client.1.smithi123.stdout:9/977: creat d4/d36/d57/d12d/f13d x:0 0 0 2024-02-16T22:22:11.167 INFO:tasks.workunit.client.0.smithi114.stdout:6/694: getdents d6/d2c 0 2024-02-16T22:22:11.169 INFO:tasks.workunit.client.1.smithi123.stdout:8/889: mkdir d0/d61/d66/da5/d126 0 2024-02-16T22:22:11.171 INFO:tasks.workunit.client.0.smithi114.stdout:1/663: dwrite d4/d9/d46/d67/f68 [4194304,4194304] 0 2024-02-16T22:22:11.173 INFO:tasks.workunit.client.0.smithi114.stdout:9/699: dread d2/d1a/d23/d32/d46/fb6 [0,4194304] 0 2024-02-16T22:22:11.174 INFO:tasks.workunit.client.0.smithi114.stdout:7/772: getdents d3/d46/da2 0 2024-02-16T22:22:11.178 INFO:tasks.workunit.client.1.smithi123.stdout:5/859: mkdir d6/dd/d78/da5/d130 0 2024-02-16T22:22:11.180 INFO:tasks.workunit.client.1.smithi123.stdout:9/978: symlink d4/d36/d8e/d6f/d7e/l13e 0 2024-02-16T22:22:11.183 INFO:tasks.workunit.client.0.smithi114.stdout:5/664: truncate d0/d28/d98/f49 408744 0 2024-02-16T22:22:11.191 INFO:tasks.workunit.client.1.smithi123.stdout:8/890: dread d0/d29/da7/fb3 [0,4194304] 0 2024-02-16T22:22:11.192 INFO:tasks.workunit.client.1.smithi123.stdout:5/860: link d6/dd/df/d58/d89/dc3/f115 d6/dc1/de6/d124/f131 0 2024-02-16T22:22:11.193 INFO:tasks.workunit.client.0.smithi114.stdout:9/700: mknod d2/d33/d3f/ce3 0 2024-02-16T22:22:11.196 INFO:tasks.workunit.client.0.smithi114.stdout:7/773: creat d3/d92/dbe/f111 x:0 0 0 2024-02-16T22:22:11.201 INFO:tasks.workunit.client.1.smithi123.stdout:5/861: creat d6/dd/df/d9f/f132 x:0 0 0 2024-02-16T22:22:11.206 INFO:tasks.workunit.client.0.smithi114.stdout:4/798: getdents dd/d1b/d37/d38/d87 0 2024-02-16T22:22:11.209 INFO:tasks.workunit.client.1.smithi123.stdout:9/979: getdents d4/d36/d8e/d6d 0 2024-02-16T22:22:11.209 INFO:tasks.workunit.client.1.smithi123.stdout:9/980: chown d4/d36/daa/d2b/d6c/ca4 395351 1 2024-02-16T22:22:11.211 INFO:tasks.workunit.client.0.smithi114.stdout:7/774: readlink d3/lda 0 2024-02-16T22:22:11.216 INFO:tasks.workunit.client.1.smithi123.stdout:9/981: creat d4/d36/d8e/d6d/f13f x:0 0 0 2024-02-16T22:22:11.217 INFO:tasks.workunit.client.1.smithi123.stdout:9/982: dread - d4/d36/daa/d20/f132 zero size 2024-02-16T22:22:11.217 INFO:tasks.workunit.client.0.smithi114.stdout:4/799: truncate dd/d2e/d56/f8c 318788 0 2024-02-16T22:22:11.221 INFO:tasks.workunit.client.0.smithi114.stdout:7/775: fsync d3/d14/f70 0 2024-02-16T22:22:11.233 INFO:tasks.workunit.client.0.smithi114.stdout:4/800: dread dd/f19 [0,4194304] 0 2024-02-16T22:22:11.242 INFO:tasks.workunit.client.0.smithi114.stdout:6/695: mknod d6/d2c/d38/ccf 0 2024-02-16T22:22:11.245 INFO:tasks.workunit.client.0.smithi114.stdout:6/696: symlink d6/d2c/d38/d41/d60/d84/ld0 0 2024-02-16T22:22:11.246 INFO:tasks.workunit.client.0.smithi114.stdout:6/697: write d6/dd/d25/d34/fb4 [1000773,42317] 0 2024-02-16T22:22:11.248 INFO:tasks.workunit.client.0.smithi114.stdout:6/698: write d6/dd/d25/f27 [2784212,9511] 0 2024-02-16T22:22:11.260 INFO:tasks.workunit.client.0.smithi114.stdout:0/736: rmdir d1/d1a/d3e/ddc 0 2024-02-16T22:22:11.265 INFO:tasks.workunit.client.0.smithi114.stdout:0/737: creat d1/d16/d1f/d28/d51/d60/dd1/fe7 x:0 0 0 2024-02-16T22:22:11.267 INFO:tasks.workunit.client.0.smithi114.stdout:0/738: creat d1/d16/d1f/d28/d51/d60/dd1/fe8 x:0 0 0 2024-02-16T22:22:11.270 INFO:tasks.workunit.client.0.smithi114.stdout:6/699: dread d6/dd/d25/d34/f6f [0,4194304] 0 2024-02-16T22:22:11.271 INFO:tasks.workunit.client.0.smithi114.stdout:0/739: mkdir d1/d16/d1f/d28/d51/d61/d6d/de9 0 2024-02-16T22:22:11.274 INFO:tasks.workunit.client.0.smithi114.stdout:6/700: getdents d6/d2c/d38/da8 0 2024-02-16T22:22:11.282 INFO:tasks.workunit.client.1.smithi123.stdout:3/889: dwrite d0/d50/fa9 [0,4194304] 0 2024-02-16T22:22:11.286 INFO:tasks.workunit.client.1.smithi123.stdout:3/890: mknod d0/c125 0 2024-02-16T22:22:11.287 INFO:tasks.workunit.client.1.smithi123.stdout:4/937: dwrite d5/f29 [4194304,4194304] 0 2024-02-16T22:22:11.290 INFO:tasks.workunit.client.1.smithi123.stdout:2/851: rename d4/dae to d4/d1a/dd5/d10b/df1/d4a/dce/d115 0 2024-02-16T22:22:11.292 INFO:tasks.workunit.client.1.smithi123.stdout:4/938: mkdir d5/db8/d11f/dc0/dc5/dcf/d125 0 2024-02-16T22:22:11.294 INFO:tasks.workunit.client.1.smithi123.stdout:1/847: rename d3/d6/db/d10/d4b/d70/f100 to d3/d6/d49/d48/f121 0 2024-02-16T22:22:11.297 INFO:tasks.workunit.client.0.smithi114.stdout:3/691: write d4/d46/fc0 [576003,88220] 0 2024-02-16T22:22:11.298 INFO:tasks.workunit.client.1.smithi123.stdout:1/848: creat d3/d6/db/d10/d2d/d8c/dfd/d9f/d10c/f122 x:0 0 0 2024-02-16T22:22:11.299 INFO:tasks.workunit.client.1.smithi123.stdout:2/852: creat d4/d1a/d1d/d59/d45/f116 x:0 0 0 2024-02-16T22:22:11.301 INFO:tasks.workunit.client.1.smithi123.stdout:4/939: dread d5/d1c/d23/d6d/d111/dd8/f100 [0,4194304] 0 2024-02-16T22:22:11.302 INFO:tasks.workunit.client.1.smithi123.stdout:6/868: rename d0/d74/f90 to d0/d9/dc/f13b 0 2024-02-16T22:22:11.304 INFO:tasks.workunit.client.1.smithi123.stdout:2/853: rmdir d4/d1a/dd5/d10b/df1/db1 39 2024-02-16T22:22:11.305 INFO:tasks.workunit.client.1.smithi123.stdout:4/940: write d5/d1c/d23/d6d/d111/dd8/f100 [736047,96573] 0 2024-02-16T22:22:11.305 INFO:tasks.workunit.client.1.smithi123.stdout:2/854: chown d4/d1a/d1d/d13/d25/d98/d10c 2512327 1 2024-02-16T22:22:11.306 INFO:tasks.workunit.client.1.smithi123.stdout:7/850: creat d3/d6e/f113 x:0 0 0 2024-02-16T22:22:11.306 INFO:tasks.workunit.client.1.smithi123.stdout:6/869: dread - d0/d6/d91/d93/fae zero size 2024-02-16T22:22:11.307 INFO:tasks.workunit.client.1.smithi123.stdout:6/870: stat d0/d6/d1a/d88/d8b/c110 0 2024-02-16T22:22:11.309 INFO:tasks.workunit.client.1.smithi123.stdout:7/851: chown d3/d5/d6c/da6/cfe 11 1 2024-02-16T22:22:11.310 INFO:tasks.workunit.client.1.smithi123.stdout:4/941: creat d5/db8/d11f/dc0/d118/d9f/d120/f126 x:0 0 0 2024-02-16T22:22:11.312 INFO:tasks.workunit.client.1.smithi123.stdout:6/871: mknod d0/d9/dc/d39/d8d/c13c 0 2024-02-16T22:22:11.312 INFO:tasks.workunit.client.1.smithi123.stdout:4/942: write d5/d1c/d23/d47/fde [2178332,39302] 0 2024-02-16T22:22:11.316 INFO:tasks.workunit.client.1.smithi123.stdout:7/852: dwrite d3/d5/d6c/da6/df1/f102 [0,4194304] 0 2024-02-16T22:22:11.317 INFO:tasks.workunit.client.0.smithi114.stdout:3/692: rename d4/d26/f42 to d4/d24/d2d/fed 0 2024-02-16T22:22:11.319 INFO:tasks.workunit.client.1.smithi123.stdout:0/784: dwrite d1/d8/d13/f100 [0,4194304] 0 2024-02-16T22:22:11.320 INFO:tasks.workunit.client.0.smithi114.stdout:3/693: rename d4/d24/d2d/d45/d74/cb1 to d4/d24/d2d/d45/d74/cee 0 2024-02-16T22:22:11.321 INFO:tasks.workunit.client.1.smithi123.stdout:7/853: truncate d3/f8c 1772978 0 2024-02-16T22:22:11.321 INFO:tasks.workunit.client.1.smithi123.stdout:7/854: readlink d3/d6e/d24/l74 0 2024-02-16T22:22:11.322 INFO:tasks.workunit.client.0.smithi114.stdout:3/694: dread d4/dd/f35 [0,4194304] 0 2024-02-16T22:22:11.322 INFO:tasks.workunit.client.1.smithi123.stdout:0/785: creat d1/d4/d42/f105 x:0 0 0 2024-02-16T22:22:11.323 INFO:tasks.workunit.client.0.smithi114.stdout:3/695: readlink d4/dd/d33/l82 0 2024-02-16T22:22:11.323 INFO:tasks.workunit.client.1.smithi123.stdout:7/855: dread d3/d6e/d4d/f5a [0,4194304] 0 2024-02-16T22:22:11.325 INFO:tasks.workunit.client.1.smithi123.stdout:0/786: truncate d1/da0/fc9 825219 0 2024-02-16T22:22:11.327 INFO:tasks.workunit.client.1.smithi123.stdout:2/855: mknod d4/d1a/dd5/d10b/df1/d4a/c117 0 2024-02-16T22:22:11.329 INFO:tasks.workunit.client.1.smithi123.stdout:0/787: chown d1/d8/ca6 4133 1 2024-02-16T22:22:11.334 INFO:tasks.workunit.client.1.smithi123.stdout:0/788: rename d1/d8/d13/c3b to d1/d4/d17/d54/d8e/c106 0 2024-02-16T22:22:11.334 INFO:tasks.workunit.client.1.smithi123.stdout:0/789: dread - d1/d8/d13/d1d/d64/dbc/fb3 zero size 2024-02-16T22:22:11.336 INFO:tasks.workunit.client.1.smithi123.stdout:7/856: getdents d3/d6e/d34 0 2024-02-16T22:22:11.338 INFO:tasks.workunit.client.1.smithi123.stdout:7/857: mknod d3/d6e/d1d/db2/d3e/c114 0 2024-02-16T22:22:11.338 INFO:tasks.workunit.client.1.smithi123.stdout:7/858: chown d3/d6e/fb5 0 1 2024-02-16T22:22:11.339 INFO:tasks.workunit.client.1.smithi123.stdout:0/790: dread d1/d4/d17/d74/fc1 [0,4194304] 0 2024-02-16T22:22:11.340 INFO:tasks.workunit.client.1.smithi123.stdout:0/791: write d1/d8/d13/d1d/d64/dbc/f92 [654195,72539] 0 2024-02-16T22:22:11.341 INFO:tasks.workunit.client.1.smithi123.stdout:5/862: write d6/dd/df/d58/d3f/dbe/f5d [2217346,104430] 0 2024-02-16T22:22:11.346 INFO:tasks.workunit.client.1.smithi123.stdout:0/792: rmdir d1/d8/d13/df1 0 2024-02-16T22:22:11.348 INFO:tasks.workunit.client.1.smithi123.stdout:0/793: creat d1/d4/d17/d74/f107 x:0 0 0 2024-02-16T22:22:11.349 INFO:tasks.workunit.client.1.smithi123.stdout:0/794: chown d1/d8/d13/d1d/d64/dbc 614520482 1 2024-02-16T22:22:11.349 INFO:tasks.workunit.client.1.smithi123.stdout:8/891: creat d0/f127 x:0 0 0 2024-02-16T22:22:11.350 INFO:tasks.workunit.client.1.smithi123.stdout:8/892: fdatasync d0/f127 0 2024-02-16T22:22:11.350 INFO:tasks.workunit.client.1.smithi123.stdout:9/983: dwrite d4/d36/daa/d20/f22 [0,4194304] 0 2024-02-16T22:22:11.355 INFO:tasks.workunit.client.1.smithi123.stdout:0/795: symlink d1/d4/d17/d54/d8e/d96/dee/l108 0 2024-02-16T22:22:11.355 INFO:tasks.workunit.client.1.smithi123.stdout:0/796: readlink d1/d8/lbb 0 2024-02-16T22:22:11.360 INFO:tasks.workunit.client.1.smithi123.stdout:0/797: dwrite d1/d8/d13/d1d/d62/fbd [0,4194304] 0 2024-02-16T22:22:11.361 INFO:tasks.workunit.client.0.smithi114.stdout:2/636: sync 2024-02-16T22:22:11.363 INFO:tasks.workunit.client.0.smithi114.stdout:5/665: write d0/d28/d98/d25/d2e/f67 [940766,56889] 0 2024-02-16T22:22:11.366 INFO:tasks.workunit.client.0.smithi114.stdout:9/701: write d2/f15 [218527,58349] 0 2024-02-16T22:22:11.368 INFO:tasks.workunit.client.0.smithi114.stdout:8/679: symlink d4/lf4 0 2024-02-16T22:22:11.370 INFO:tasks.workunit.client.0.smithi114.stdout:7/776: write d3/d22/d73/da7/d4e/f6e [2342311,62456] 0 2024-02-16T22:22:11.375 INFO:tasks.workunit.client.0.smithi114.stdout:3/696: dwrite d4/d26/fc1 [0,4194304] 0 2024-02-16T22:22:11.379 INFO:tasks.workunit.client.0.smithi114.stdout:7/777: write d3/d22/d73/da7/d4e/fc0 [1040721,80622] 0 2024-02-16T22:22:11.379 INFO:tasks.workunit.client.0.smithi114.stdout:1/664: dwrite d4/d9/d3d/d3f/f75 [0,4194304] 0 2024-02-16T22:22:11.384 INFO:tasks.workunit.client.0.smithi114.stdout:8/680: truncate d4/d24/d2c/d67/dcf/ff1 90288 0 2024-02-16T22:22:11.386 INFO:tasks.workunit.client.0.smithi114.stdout:4/801: dwrite dd/d59/d8f/f88 [0,4194304] 0 2024-02-16T22:22:11.390 INFO:tasks.workunit.client.1.smithi123.stdout:0/798: getdents d1/d4/d17/d54/d8e/da5 0 2024-02-16T22:22:11.393 INFO:tasks.workunit.client.0.smithi114.stdout:6/701: dread d6/f1e [0,4194304] 0 2024-02-16T22:22:11.396 INFO:tasks.workunit.client.0.smithi114.stdout:9/702: mkdir d2/d5c/d5f/d91/de4 0 2024-02-16T22:22:11.406 INFO:tasks.workunit.client.0.smithi114.stdout:9/703: dwrite d2/d5c/f8d [4194304,4194304] 0 2024-02-16T22:22:11.407 INFO:tasks.workunit.client.0.smithi114.stdout:2/637: mkdir dd/d4e/d97/dd6 0 2024-02-16T22:22:11.407 INFO:tasks.workunit.client.1.smithi123.stdout:3/891: read d0/f12 [3811351,29119] 0 2024-02-16T22:22:11.408 INFO:tasks.workunit.client.0.smithi114.stdout:8/681: dread d4/d11/d78/d7a/d7e/fae [0,4194304] 0 2024-02-16T22:22:11.414 INFO:tasks.workunit.client.1.smithi123.stdout:1/849: write d3/d6/db/d10/d2d/d8c/ff3 [938044,27392] 0 2024-02-16T22:22:11.415 INFO:tasks.workunit.client.1.smithi123.stdout:1/850: readlink d3/d6/d18/le0 0 2024-02-16T22:22:11.416 INFO:tasks.workunit.client.0.smithi114.stdout:4/802: fdatasync dd/d1b/d41/dc3/fe9 0 2024-02-16T22:22:11.416 INFO:tasks.workunit.client.1.smithi123.stdout:1/851: write d3/d6/db/d55/f103 [880510,71714] 0 2024-02-16T22:22:11.418 INFO:tasks.workunit.client.1.smithi123.stdout:0/799: getdents d1/d4/d17 0 2024-02-16T22:22:11.419 INFO:tasks.workunit.client.0.smithi114.stdout:6/702: symlink d6/ld1 0 2024-02-16T22:22:11.420 INFO:tasks.workunit.client.1.smithi123.stdout:4/943: dread d5/d1c/d5e/dc3/ffa [0,4194304] 0 2024-02-16T22:22:11.421 INFO:tasks.workunit.client.1.smithi123.stdout:1/852: dread d3/d6/db/d10/d2d/f25 [0,4194304] 0 2024-02-16T22:22:11.421 INFO:tasks.workunit.client.1.smithi123.stdout:6/872: write d0/d74/f7b [4247508,75022] 0 2024-02-16T22:22:11.427 INFO:tasks.workunit.client.0.smithi114.stdout:1/665: dread d4/d9/d46/d47/f9e [4194304,4194304] 0 2024-02-16T22:22:11.428 INFO:tasks.workunit.client.0.smithi114.stdout:1/666: readlink d4/d9/d46/l93 0 2024-02-16T22:22:11.429 INFO:tasks.workunit.client.1.smithi123.stdout:3/892: unlink d0/d20/d29/d9b/dbd/ce8 0 2024-02-16T22:22:11.430 INFO:tasks.workunit.client.1.smithi123.stdout:3/893: fsync d0/d20/d29/d9b/dbd/df4/d5d/d60/f106 0 2024-02-16T22:22:11.432 INFO:tasks.workunit.client.0.smithi114.stdout:2/638: creat dd/df/d1c/d21/d61/d69/fd7 x:0 0 0 2024-02-16T22:22:11.435 INFO:tasks.workunit.client.1.smithi123.stdout:6/873: dread - d0/d9/dc/d3a/d70/dd5/fd8 zero size 2024-02-16T22:22:11.437 INFO:tasks.workunit.client.1.smithi123.stdout:2/856: write d4/d1a/dd5/d10b/df1/db1/fb2 [1391036,60904] 0 2024-02-16T22:22:11.438 INFO:tasks.workunit.client.0.smithi114.stdout:4/803: mknod dd/d1b/d37/d99/c123 0 2024-02-16T22:22:11.440 INFO:tasks.workunit.client.1.smithi123.stdout:4/944: rename d5/d1c/d23/d31/cb5 to d5/d1c/d23/d74/d92/d9b/c127 0 2024-02-16T22:22:11.441 INFO:tasks.workunit.client.1.smithi123.stdout:4/945: readlink d5/d1c/d23/d47/l6b 0 2024-02-16T22:22:11.442 INFO:tasks.workunit.client.1.smithi123.stdout:7/859: write d3/d6e/d24/f49 [1449526,60448] 0 2024-02-16T22:22:11.442 INFO:tasks.workunit.client.0.smithi114.stdout:6/703: unlink d6/da/f1b 0 2024-02-16T22:22:11.443 INFO:tasks.workunit.client.1.smithi123.stdout:7/860: chown d3/d6e/d52/f71 4507 1 2024-02-16T22:22:11.444 INFO:tasks.workunit.client.0.smithi114.stdout:9/704: dread d2/d1a/d23/d32/d46/f9d [0,4194304] 0 2024-02-16T22:22:11.446 INFO:tasks.workunit.client.1.smithi123.stdout:5/863: dwrite d6/dd/df/f1b [0,4194304] 0 2024-02-16T22:22:11.447 INFO:tasks.workunit.client.1.smithi123.stdout:5/864: stat d6/dd/df/d1f/l80 0 2024-02-16T22:22:11.448 INFO:tasks.workunit.client.0.smithi114.stdout:1/667: write d4/d63/d69/d7c/d9b/db0/f38 [3725997,116999] 0 2024-02-16T22:22:11.452 INFO:tasks.workunit.client.1.smithi123.stdout:9/984: truncate d4/d1e/d3a/fcf 1087224 0 2024-02-16T22:22:11.453 INFO:tasks.workunit.client.1.smithi123.stdout:8/893: dwrite d0/d29/da7/d26/d68/d81/d47/d9b/fc5 [0,4194304] 0 2024-02-16T22:22:11.453 INFO:tasks.workunit.client.1.smithi123.stdout:2/857: dread d4/d1a/d1d/f9f [4194304,4194304] 0 2024-02-16T22:22:11.455 INFO:tasks.workunit.client.1.smithi123.stdout:8/894: write d0/d29/da7/fc9 [1830875,127318] 0 2024-02-16T22:22:11.461 INFO:tasks.workunit.client.0.smithi114.stdout:4/804: symlink dd/d1b/d41/dc3/dfd/l124 0 2024-02-16T22:22:11.461 INFO:tasks.workunit.client.0.smithi114.stdout:5/666: dwrite d0/f81 [0,4194304] 0 2024-02-16T22:22:11.464 INFO:tasks.workunit.client.0.smithi114.stdout:6/704: mknod d6/dd/d25/d34/d58/d9f/cd2 0 2024-02-16T22:22:11.465 INFO:tasks.workunit.client.0.smithi114.stdout:6/705: readlink d6/dd/d25/d34/d61/l97 0 2024-02-16T22:22:11.468 INFO:tasks.workunit.client.1.smithi123.stdout:6/874: mkdir d0/d9/dc/d3a/d70/d13d 0 2024-02-16T22:22:11.469 INFO:tasks.workunit.client.0.smithi114.stdout:9/705: write d2/d5c/d5f/d72/fb8 [1687293,6233] 0 2024-02-16T22:22:11.473 INFO:tasks.workunit.client.0.smithi114.stdout:7/778: getdents d3/d14 0 2024-02-16T22:22:11.475 INFO:tasks.workunit.client.0.smithi114.stdout:3/697: write d4/dd/d33/d76/f7f [777239,83980] 0 2024-02-16T22:22:11.477 INFO:tasks.workunit.client.0.smithi114.stdout:2/639: mknod dd/d3b/da7/cd8 0 2024-02-16T22:22:11.478 INFO:tasks.workunit.client.0.smithi114.stdout:2/640: write dd/d4e/f9c [5154071,32259] 0 2024-02-16T22:22:11.481 INFO:tasks.workunit.client.1.smithi123.stdout:0/800: dwrite d1/f1f [0,4194304] 0 2024-02-16T22:22:11.488 INFO:tasks.workunit.client.1.smithi123.stdout:8/895: creat d0/d29/da7/d26/d68/d81/d47/d9b/f128 x:0 0 0 2024-02-16T22:22:11.488 INFO:tasks.workunit.client.1.smithi123.stdout:8/896: chown d0/d29/da7/d26/d2a 80592920 1 2024-02-16T22:22:11.489 INFO:tasks.workunit.client.0.smithi114.stdout:6/706: dread - d6/d2c/d38/d54/fad zero size 2024-02-16T22:22:11.491 INFO:tasks.workunit.client.1.smithi123.stdout:1/853: getdents d3/d6/d49/d48 0 2024-02-16T22:22:11.491 INFO:tasks.workunit.client.1.smithi123.stdout:4/946: write d5/d1c/d23/d31/fc1 [3615487,113992] 0 2024-02-16T22:22:11.493 INFO:tasks.workunit.client.0.smithi114.stdout:8/682: write d4/d11/d78/d7a/d36/d3b/f44 [1217943,124104] 0 2024-02-16T22:22:11.496 INFO:tasks.workunit.client.1.smithi123.stdout:6/875: mknod d0/d6/d2f/dbd/c13e 0 2024-02-16T22:22:11.497 INFO:tasks.workunit.client.1.smithi123.stdout:5/865: write d6/dd/df/d58/d3f/dbe/da2/faf [4809159,99547] 0 2024-02-16T22:22:11.500 INFO:tasks.workunit.client.1.smithi123.stdout:3/894: rename d0/d20/d29/d9b/dbd/df4/d5d/dae/dc7/d121/f99 to d0/d20/d5a/f126 0 2024-02-16T22:22:11.501 INFO:tasks.workunit.client.0.smithi114.stdout:1/668: write d4/f91 [379448,60176] 0 2024-02-16T22:22:11.505 INFO:tasks.workunit.client.0.smithi114.stdout:4/805: write dd/d1b/d37/d38/d87/dad/ff1 [869657,58908] 0 2024-02-16T22:22:11.506 INFO:tasks.workunit.client.0.smithi114.stdout:5/667: mkdir d0/df/d7a/dc4/dda 0 2024-02-16T22:22:11.508 INFO:tasks.workunit.client.0.smithi114.stdout:9/706: write d2/f5 [3195263,17420] 0 2024-02-16T22:22:11.510 INFO:tasks.workunit.client.1.smithi123.stdout:0/801: creat d1/d8/d13/d1d/d62/f109 x:0 0 0 2024-02-16T22:22:11.514 INFO:tasks.workunit.client.0.smithi114.stdout:5/668: dwrite d0/d28/d98/d1e/d86/fa9 [4194304,4194304] 0 2024-02-16T22:22:11.516 INFO:tasks.workunit.client.1.smithi123.stdout:4/947: mkdir d5/d1c/d23/d6d/d128 0 2024-02-16T22:22:11.519 INFO:tasks.workunit.client.1.smithi123.stdout:7/861: creat d3/f115 x:0 0 0 2024-02-16T22:22:11.520 INFO:tasks.workunit.client.0.smithi114.stdout:8/683: mknod d4/d11/d78/cf5 0 2024-02-16T22:22:11.521 INFO:tasks.workunit.client.0.smithi114.stdout:8/684: chown d4/d16/d25/fe6 6696578 1 2024-02-16T22:22:11.524 INFO:tasks.workunit.client.0.smithi114.stdout:1/669: creat d4/d63/d69/d7c/dc0/ff4 x:0 0 0 2024-02-16T22:22:11.526 INFO:tasks.workunit.client.1.smithi123.stdout:5/866: creat d6/dd/df/d58/d3f/d6f/f133 x:0 0 0 2024-02-16T22:22:11.527 INFO:tasks.workunit.client.0.smithi114.stdout:2/641: creat dd/dbd/fd9 x:0 0 0 2024-02-16T22:22:11.529 INFO:tasks.workunit.client.1.smithi123.stdout:3/895: mkdir d0/d20/d29/d9b/dbd/df4/d5d/dae/db0/d123/d127 0 2024-02-16T22:22:11.531 INFO:tasks.workunit.client.0.smithi114.stdout:6/707: write d6/d2c/d38/d41/d60/f98 [5189109,16076] 0 2024-02-16T22:22:11.531 INFO:tasks.workunit.client.1.smithi123.stdout:9/985: link d4/d36/d8e/d58/l8f d4/d36/d8e/d58/d59/d60/deb/l140 0 2024-02-16T22:22:11.533 INFO:tasks.workunit.client.1.smithi123.stdout:1/854: dread d3/d6/d11/fbf [0,4194304] 0 2024-02-16T22:22:11.535 INFO:tasks.workunit.client.0.smithi114.stdout:9/707: dread - d2/dd/f2f zero size 2024-02-16T22:22:11.536 INFO:tasks.workunit.client.0.smithi114.stdout:6/708: read d6/dd/d25/d34/f8e [3593888,121382] 0 2024-02-16T22:22:11.537 INFO:tasks.workunit.client.1.smithi123.stdout:8/897: dwrite d0/d29/da7/d26/d32/f56 [0,4194304] 0 2024-02-16T22:22:11.537 INFO:tasks.workunit.client.0.smithi114.stdout:9/708: dread - d2/d5c/dc9/d47/db2/fc3 zero size 2024-02-16T22:22:11.539 INFO:tasks.workunit.client.1.smithi123.stdout:4/948: symlink d5/d1c/d23/d56/l129 0 2024-02-16T22:22:11.539 INFO:tasks.workunit.client.0.smithi114.stdout:7/779: dwrite d3/d46/f7e [4194304,4194304] 0 2024-02-16T22:22:11.540 INFO:tasks.workunit.client.1.smithi123.stdout:4/949: write d5/d1c/d64/d7b/f107 [680625,111835] 0 2024-02-16T22:22:11.543 INFO:tasks.workunit.client.0.smithi114.stdout:5/669: dread d0/d28/d98/d25/d3e/d7f/fa8 [0,4194304] 0 2024-02-16T22:22:11.546 INFO:tasks.workunit.client.0.smithi114.stdout:0/740: sync 2024-02-16T22:22:11.547 INFO:tasks.workunit.client.0.smithi114.stdout:0/741: write d1/d1a/d47/fb2 [817749,94960] 0 2024-02-16T22:22:11.548 INFO:tasks.workunit.client.1.smithi123.stdout:5/867: symlink d6/dd/d44/dba/d126/l134 0 2024-02-16T22:22:11.548 INFO:tasks.workunit.client.1.smithi123.stdout:6/876: dwrite d0/d96/dca/ff8 [4194304,4194304] 0 2024-02-16T22:22:11.549 INFO:tasks.workunit.client.0.smithi114.stdout:0/742: write d1/d16/d1f/d28/fe6 [82942,114221] 0 2024-02-16T22:22:11.552 INFO:tasks.workunit.client.0.smithi114.stdout:0/743: write d1/d16/d1f/d28/d51/d61/d7e/d90/fcf [370148,33135] 0 2024-02-16T22:22:11.553 INFO:tasks.workunit.client.0.smithi114.stdout:4/806: write dd/d1b/d37/d99/fd7 [885821,111979] 0 2024-02-16T22:22:11.556 INFO:tasks.workunit.client.1.smithi123.stdout:2/858: getdents d4/d1a/d1d/d13/d25 0 2024-02-16T22:22:11.565 INFO:tasks.workunit.client.1.smithi123.stdout:6/877: truncate d0/d6/d1a/d88/fcb 658774 0 2024-02-16T22:22:11.568 INFO:tasks.workunit.client.1.smithi123.stdout:2/859: rmdir d4/d1a/d1d/d59 39 2024-02-16T22:22:11.569 INFO:tasks.workunit.client.1.smithi123.stdout:0/802: dread d1/f6a [0,4194304] 0 2024-02-16T22:22:11.570 INFO:tasks.workunit.client.1.smithi123.stdout:0/803: dread - d1/d4/d42/fc0 zero size 2024-02-16T22:22:11.572 INFO:tasks.workunit.client.0.smithi114.stdout:0/744: dread d1/d16/f73 [0,4194304] 0 2024-02-16T22:22:11.573 INFO:tasks.workunit.client.0.smithi114.stdout:0/745: fsync d1/d16/d1f/f9a 0 2024-02-16T22:22:11.573 INFO:tasks.workunit.client.1.smithi123.stdout:3/896: write d0/d20/d29/f84 [501908,87813] 0 2024-02-16T22:22:11.574 INFO:tasks.workunit.client.1.smithi123.stdout:8/898: mknod d0/d29/da7/d43/c129 0 2024-02-16T22:22:11.574 INFO:tasks.workunit.client.0.smithi114.stdout:0/746: truncate d1/d16/d1f/d28/fb0 415724 0 2024-02-16T22:22:11.577 INFO:tasks.workunit.client.1.smithi123.stdout:7/862: dread d3/d5/ff [0,4194304] 0 2024-02-16T22:22:11.578 INFO:tasks.workunit.client.0.smithi114.stdout:5/670: dread d0/d28/d98/d1e/d63/fc8 [0,4194304] 0 2024-02-16T22:22:11.580 INFO:tasks.workunit.client.1.smithi123.stdout:5/868: unlink d6/dd/df/d90/f9d 0 2024-02-16T22:22:11.581 INFO:tasks.workunit.client.1.smithi123.stdout:6/878: rmdir d0/d6/d2f/da5 39 2024-02-16T22:22:11.582 INFO:tasks.workunit.client.1.smithi123.stdout:6/879: write d0/d6/d1a/d2e/f12e [163790,65222] 0 2024-02-16T22:22:11.585 INFO:tasks.workunit.client.1.smithi123.stdout:2/860: creat d4/d1a/dd5/f118 x:0 0 0 2024-02-16T22:22:11.586 INFO:tasks.workunit.client.1.smithi123.stdout:0/804: mknod d1/d4/d42/c10a 0 2024-02-16T22:22:11.587 INFO:tasks.workunit.client.1.smithi123.stdout:9/986: truncate d4/d36/d8e/d58/f5a 1006172 0 2024-02-16T22:22:11.588 INFO:tasks.workunit.client.1.smithi123.stdout:6/880: dwrite d0/d6/d1a/f6f [0,4194304] 0 2024-02-16T22:22:11.591 INFO:tasks.workunit.client.1.smithi123.stdout:6/881: write d0/d96/de3/f79 [2182504,3827] 0 2024-02-16T22:22:11.592 INFO:tasks.workunit.client.1.smithi123.stdout:4/950: truncate d5/d1c/d23/d47/fde 1858935 0 2024-02-16T22:22:11.598 INFO:tasks.workunit.client.1.smithi123.stdout:5/869: rename d6/dc1/de6/fd6 to d6/dd/df/d58/d89/d98/d10b/f135 0 2024-02-16T22:22:11.601 INFO:tasks.workunit.client.1.smithi123.stdout:0/805: creat d1/d8/da7/f10b x:0 0 0 2024-02-16T22:22:11.603 INFO:tasks.workunit.client.0.smithi114.stdout:3/698: rename d4/d6a to d4/d24/def 0 2024-02-16T22:22:11.606 INFO:tasks.workunit.client.1.smithi123.stdout:8/899: mknod d0/d29/c12a 0 2024-02-16T22:22:11.608 INFO:tasks.workunit.client.0.smithi114.stdout:1/670: mknod d4/d9/d46/d67/d92/dca/dcc/d98/cf5 0 2024-02-16T22:22:11.610 INFO:tasks.workunit.client.0.smithi114.stdout:2/642: creat dd/d3b/d8a/dcb/fda x:0 0 0 2024-02-16T22:22:11.611 INFO:tasks.workunit.client.0.smithi114.stdout:2/643: read - dd/d3b/da7/d4c/faf zero size 2024-02-16T22:22:11.613 INFO:tasks.workunit.client.1.smithi123.stdout:6/882: read d0/d8a/de6/ffe [308646,10065] 0 2024-02-16T22:22:11.615 INFO:tasks.workunit.client.1.smithi123.stdout:4/951: mkdir d5/d1c/d23/d47/da2/daa/d12a 0 2024-02-16T22:22:11.617 INFO:tasks.workunit.client.0.smithi114.stdout:6/709: truncate d6/dd/f29 1912124 0 2024-02-16T22:22:11.618 INFO:tasks.workunit.client.1.smithi123.stdout:5/870: truncate d6/dd/df/d58/f70 1877803 0 2024-02-16T22:22:11.621 INFO:tasks.workunit.client.0.smithi114.stdout:7/780: creat d3/d22/db7/f112 x:0 0 0 2024-02-16T22:22:11.625 INFO:tasks.workunit.client.0.smithi114.stdout:7/781: dread d3/d46/da2/ff5 [0,4194304] 0 2024-02-16T22:22:11.626 INFO:tasks.workunit.client.0.smithi114.stdout:7/782: stat d3/d22/d73/f7c 0 2024-02-16T22:22:11.629 INFO:tasks.workunit.client.0.smithi114.stdout:0/747: creat d1/d16/d48/fea x:0 0 0 2024-02-16T22:22:11.631 INFO:tasks.workunit.client.0.smithi114.stdout:5/671: creat d0/d2d/d30/d9f/fdb x:0 0 0 2024-02-16T22:22:11.632 INFO:tasks.workunit.client.0.smithi114.stdout:5/672: read - d0/d28/d98/d1e/d63/d96/fd9 zero size 2024-02-16T22:22:11.632 INFO:tasks.workunit.client.1.smithi123.stdout:9/987: link d4/d36/d8e/d6d/d71/f135 d4/d1e/d3a/dac/dc5/f141 0 2024-02-16T22:22:11.634 INFO:tasks.workunit.client.0.smithi114.stdout:8/685: rename d4/d11/d78/d7a/d31/ld0 to d4/d11/d78/d7a/d31/dc6/dce/lf6 0 2024-02-16T22:22:11.636 INFO:tasks.workunit.client.0.smithi114.stdout:6/710: dread d6/f30 [0,4194304] 0 2024-02-16T22:22:11.641 INFO:tasks.workunit.client.1.smithi123.stdout:6/883: creat d0/d74/f13f x:0 0 0 2024-02-16T22:22:11.642 INFO:tasks.workunit.client.1.smithi123.stdout:6/884: chown d0/d9/dc/d3a 235597 1 2024-02-16T22:22:11.643 INFO:tasks.workunit.client.1.smithi123.stdout:4/952: write d5/d1c/d23/d47/da2/fb2 [2805854,44107] 0 2024-02-16T22:22:11.645 INFO:tasks.workunit.client.0.smithi114.stdout:1/671: unlink d4/d9/d3d/l2b 0 2024-02-16T22:22:11.646 INFO:tasks.workunit.client.1.smithi123.stdout:7/863: write d3/d6e/d52/f7d [3471464,58996] 0 2024-02-16T22:22:11.648 INFO:tasks.workunit.client.0.smithi114.stdout:9/709: creat d2/d33/d3f/ddd/fe5 x:0 0 0 2024-02-16T22:22:11.655 INFO:tasks.workunit.client.0.smithi114.stdout:7/783: creat d3/d92/dd5/f113 x:0 0 0 2024-02-16T22:22:11.656 INFO:tasks.workunit.client.0.smithi114.stdout:7/784: write d3/d22/d73/da7/d6f/df8/ff9 [4380396,73459] 0 2024-02-16T22:22:11.657 INFO:tasks.workunit.client.1.smithi123.stdout:3/897: truncate d0/f6 793313 0 2024-02-16T22:22:11.659 INFO:tasks.workunit.client.1.smithi123.stdout:0/806: dwrite d1/d4/d17/d54/f5b [0,4194304] 0 2024-02-16T22:22:11.660 INFO:tasks.workunit.client.0.smithi114.stdout:5/673: symlink d0/d28/d98/d1e/d63/d96/d9a/db8/ldc 0 2024-02-16T22:22:11.660 INFO:tasks.workunit.client.1.smithi123.stdout:0/807: fsync d1/d4/d17/d74/de7/ff5 0 2024-02-16T22:22:11.661 INFO:tasks.workunit.client.0.smithi114.stdout:2/644: rename dd/d3b/da7/d7f to dd/df/ddb 0 2024-02-16T22:22:11.662 INFO:tasks.workunit.client.1.smithi123.stdout:0/808: write d1/d8/d13/d1d/d64/dbc/f5c [412949,49304] 0 2024-02-16T22:22:11.664 INFO:tasks.workunit.client.0.smithi114.stdout:8/686: fdatasync d4/d16/f4b 0 2024-02-16T22:22:11.668 INFO:tasks.workunit.client.1.smithi123.stdout:8/900: creat d0/d29/da7/d26/d2a/d77/df8/f12b x:0 0 0 2024-02-16T22:22:11.669 INFO:tasks.workunit.client.1.smithi123.stdout:0/809: dwrite d1/f1f [0,4194304] 0 2024-02-16T22:22:11.670 INFO:tasks.workunit.client.1.smithi123.stdout:4/953: dread d5/d1c/d23/f52 [0,4194304] 0 2024-02-16T22:22:11.676 INFO:tasks.workunit.client.0.smithi114.stdout:3/699: dwrite d4/dd/d33/d76/f9e [0,4194304] 0 2024-02-16T22:22:11.678 INFO:tasks.workunit.client.1.smithi123.stdout:6/885: creat d0/d8a/d100/d135/f140 x:0 0 0 2024-02-16T22:22:11.679 INFO:tasks.workunit.client.1.smithi123.stdout:5/871: dwrite d6/dd/df/d58/d3f/f9e [0,4194304] 0 2024-02-16T22:22:11.680 INFO:tasks.workunit.client.1.smithi123.stdout:5/872: dread - d6/dc1/de6/ffd zero size 2024-02-16T22:22:11.681 INFO:tasks.workunit.client.1.smithi123.stdout:5/873: stat d6/dd/df/d90/fae 0 2024-02-16T22:22:11.687 INFO:tasks.workunit.client.1.smithi123.stdout:7/864: mkdir d3/d87/d116 0 2024-02-16T22:22:11.689 INFO:tasks.workunit.client.0.smithi114.stdout:4/807: link dd/d2e/d56/ddc/dde/f82 dd/d1b/d37/d38/d87/f125 0 2024-02-16T22:22:11.694 INFO:tasks.workunit.client.0.smithi114.stdout:7/785: mkdir d3/d46/da2/d114 0 2024-02-16T22:22:11.694 INFO:tasks.workunit.client.0.smithi114.stdout:1/672: write d4/d9/d46/d67/f6e [6817426,43202] 0 2024-02-16T22:22:11.696 INFO:tasks.workunit.client.1.smithi123.stdout:8/901: rename d0/d29/da7/c21 to d0/d61/dac/dd0/c12c 0 2024-02-16T22:22:11.698 INFO:tasks.workunit.client.0.smithi114.stdout:9/710: write d2/d1a/d23/f9b [875901,97438] 0 2024-02-16T22:22:11.699 INFO:tasks.workunit.client.1.smithi123.stdout:0/810: symlink d1/d4/d17/d54/l10c 0 2024-02-16T22:22:11.700 INFO:tasks.workunit.client.0.smithi114.stdout:8/687: dread d4/d16/fbf [0,4194304] 0 2024-02-16T22:22:11.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:11 smithi114 ceph-mon[29235]: pgmap v33: 65 pgs: 65 active+clean; 2.9 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 88 MiB/s rd, 185 MiB/s wr, 413 op/s 2024-02-16T22:22:11.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:11 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:22:11.702 INFO:tasks.workunit.client.0.smithi114.stdout:8/688: write d4/d11/d78/fd8 [397319,13425] 0 2024-02-16T22:22:11.705 INFO:tasks.workunit.client.1.smithi123.stdout:4/954: truncate d5/d1c/d23/d74/d92/d9b/fc2 1616361 0 2024-02-16T22:22:11.708 INFO:tasks.workunit.client.1.smithi123.stdout:5/874: symlink d6/dd/d44/dba/d126/l136 0 2024-02-16T22:22:11.709 INFO:tasks.workunit.client.1.smithi123.stdout:5/875: stat d6/dd/d44/dba/d126/l136 0 2024-02-16T22:22:11.709 INFO:tasks.workunit.client.0.smithi114.stdout:6/711: mkdir d6/dd/d25/d34/d61/d62/d8b/db7/dd3 0 2024-02-16T22:22:11.710 INFO:tasks.workunit.client.1.smithi123.stdout:3/898: dwrite d0/d20/d5a/f9d [0,4194304] 0 2024-02-16T22:22:11.711 INFO:tasks.workunit.client.0.smithi114.stdout:6/712: truncate d6/dd/d25/fc0 926872 0 2024-02-16T22:22:11.711 INFO:tasks.workunit.client.0.smithi114.stdout:6/713: chown d6/d2c/d38/d54/fad 1853402226 1 2024-02-16T22:22:11.713 INFO:tasks.workunit.client.0.smithi114.stdout:6/714: write d6/dd/d25/d34/d61/d62/d8b/fbb [2782779,125213] 0 2024-02-16T22:22:11.714 INFO:tasks.workunit.client.1.smithi123.stdout:8/902: fsync d0/d29/da7/d43/fdf 0 2024-02-16T22:22:11.715 INFO:tasks.workunit.client.1.smithi123.stdout:8/903: chown d0/d29/da7 463 1 2024-02-16T22:22:11.716 INFO:tasks.workunit.client.1.smithi123.stdout:8/904: chown d0/d29/da7/d43/fdf 57043072 1 2024-02-16T22:22:11.722 INFO:tasks.workunit.client.0.smithi114.stdout:5/674: dwrite d0/d28/d98/d1e/d86/f8c [0,4194304] 0 2024-02-16T22:22:11.725 INFO:tasks.workunit.client.0.smithi114.stdout:5/675: chown d0/d28/d98/d25/d2e/d3b/f77 5454499 1 2024-02-16T22:22:11.726 INFO:tasks.workunit.client.0.smithi114.stdout:4/808: mknod dd/d1b/d37/d61/df4/c126 0 2024-02-16T22:22:11.741 INFO:tasks.workunit.client.0.smithi114.stdout:1/673: write d4/d9/d46/d67/d92/fa6 [2047654,74658] 0 2024-02-16T22:22:11.743 INFO:tasks.workunit.client.0.smithi114.stdout:4/809: chown dd/d1b/d24/c67 1 1 2024-02-16T22:22:11.744 INFO:tasks.workunit.client.0.smithi114.stdout:5/676: truncate d0/d28/d98/d1e/fa4 927754 0 2024-02-16T22:22:11.755 INFO:tasks.workunit.client.0.smithi114.stdout:5/677: unlink d0/df/f5d 0 2024-02-16T22:22:11.760 INFO:tasks.workunit.client.0.smithi114.stdout:1/674: dread d4/d9/d46/d67/d92/dca/dcc/d98/fcd [0,4194304] 0 2024-02-16T22:22:11.760 INFO:tasks.workunit.client.0.smithi114.stdout:4/810: dread dd/d2e/d3f/f50 [0,4194304] 0 2024-02-16T22:22:11.761 INFO:tasks.workunit.client.0.smithi114.stdout:3/700: getdents d4/dd/d33/d72 0 2024-02-16T22:22:11.761 INFO:tasks.workunit.client.0.smithi114.stdout:1/675: stat d4/d63/d69/d7c/d9b/db0/c40 0 2024-02-16T22:22:11.762 INFO:tasks.workunit.client.0.smithi114.stdout:2/645: write dd/df/d1c/d21/f30 [1410275,21474] 0 2024-02-16T22:22:11.762 INFO:tasks.workunit.client.0.smithi114.stdout:1/676: chown d4/d63/d69/l6f 9557 1 2024-02-16T22:22:11.764 INFO:tasks.workunit.client.1.smithi123.stdout:1/855: sync 2024-02-16T22:22:11.765 INFO:tasks.workunit.client.0.smithi114.stdout:5/678: mknod d0/d2d/cdd 0 2024-02-16T22:22:11.766 INFO:tasks.workunit.client.1.smithi123.stdout:0/811: chown d1/d8/da7/cb5 1575 1 2024-02-16T22:22:11.767 INFO:tasks.workunit.client.0.smithi114.stdout:9/711: getdents d2/d33 0 2024-02-16T22:22:11.767 INFO:tasks.workunit.client.1.smithi123.stdout:9/988: getdents d4/d1e 0 2024-02-16T22:22:11.770 INFO:tasks.workunit.client.0.smithi114.stdout:9/712: truncate d2/d5c/dc9/d47/db2/fc6 503656 0 2024-02-16T22:22:11.772 INFO:tasks.workunit.client.0.smithi114.stdout:5/679: dwrite d0/fb9 [0,4194304] 0 2024-02-16T22:22:11.774 INFO:tasks.workunit.client.1.smithi123.stdout:3/899: fdatasync d0/d20/fdf 0 2024-02-16T22:22:11.775 INFO:tasks.workunit.client.0.smithi114.stdout:3/701: dread d4/d26/f31 [0,4194304] 0 2024-02-16T22:22:11.775 INFO:tasks.workunit.client.0.smithi114.stdout:1/677: creat d4/d9/d46/d47/db3/ff6 x:0 0 0 2024-02-16T22:22:11.776 INFO:tasks.workunit.client.1.smithi123.stdout:4/955: dread d5/d1c/d23/d47/da2/daa/ff6 [0,4194304] 0 2024-02-16T22:22:11.777 INFO:tasks.workunit.client.1.smithi123.stdout:7/865: creat d3/d6e/d1d/d97/dce/f117 x:0 0 0 2024-02-16T22:22:11.778 INFO:tasks.workunit.client.0.smithi114.stdout:4/811: dwrite dd/d1b/d37/fab [0,4194304] 0 2024-02-16T22:22:11.778 INFO:tasks.workunit.client.0.smithi114.stdout:1/678: dread - d4/d63/d69/d72/fde zero size 2024-02-16T22:22:11.786 INFO:tasks.workunit.client.0.smithi114.stdout:3/702: rmdir d4/d26 39 2024-02-16T22:22:11.788 INFO:tasks.workunit.client.1.smithi123.stdout:1/856: mknod d3/d6/db/d10/d2d/d8c/dfd/d9f/db4/ddf/df0/c123 0 2024-02-16T22:22:11.788 INFO:tasks.workunit.client.1.smithi123.stdout:5/876: dwrite d6/f8f [0,4194304] 0 2024-02-16T22:22:11.789 INFO:tasks.workunit.client.1.smithi123.stdout:1/857: fdatasync d3/d6/db/fdd 0 2024-02-16T22:22:11.790 INFO:tasks.workunit.client.1.smithi123.stdout:0/812: mkdir d1/d4/d17/d54/d8e/da5/d10d 0 2024-02-16T22:22:11.792 INFO:tasks.workunit.client.0.smithi114.stdout:9/713: symlink d2/d5c/le6 0 2024-02-16T22:22:11.794 INFO:tasks.workunit.client.1.smithi123.stdout:9/989: dread - d4/d1e/d3a/dac/f113 zero size 2024-02-16T22:22:11.794 INFO:tasks.workunit.client.1.smithi123.stdout:8/905: write d0/d29/da7/d26/d68/d81/f87 [840749,122217] 0 2024-02-16T22:22:11.794 INFO:tasks.workunit.client.0.smithi114.stdout:9/714: stat d2/d33/d3f/fe1 0 2024-02-16T22:22:11.795 INFO:tasks.workunit.client.0.smithi114.stdout:2/646: dwrite dd/df/d1c/d21/d61/d63/d83/fb6 [0,4194304] 0 2024-02-16T22:22:11.795 INFO:tasks.workunit.client.1.smithi123.stdout:8/906: read - d0/d29/da7/d26/d2a/d123/dca/f11c zero size 2024-02-16T22:22:11.798 INFO:tasks.workunit.client.1.smithi123.stdout:3/900: mkdir d0/d20/d29/d9b/dbd/df4/d5d/d60/dc6/d128 0 2024-02-16T22:22:11.801 INFO:tasks.workunit.client.0.smithi114.stdout:3/703: rename d4/d24/d2d/d45/d68 to d4/d24/d2d/df0 0 2024-02-16T22:22:11.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:11 smithi123 ceph-mon[37161]: pgmap v33: 65 pgs: 65 active+clean; 2.9 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 88 MiB/s rd, 185 MiB/s wr, 413 op/s 2024-02-16T22:22:11.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:11 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:22:11.802 INFO:tasks.workunit.client.0.smithi114.stdout:3/704: chown d4/c69 1663 1 2024-02-16T22:22:11.803 INFO:tasks.workunit.client.0.smithi114.stdout:5/680: dwrite d0/d28/d98/f79 [0,4194304] 0 2024-02-16T22:22:11.804 INFO:tasks.workunit.client.0.smithi114.stdout:4/812: mkdir dd/d1b/d41/d127 0 2024-02-16T22:22:11.807 INFO:tasks.workunit.client.0.smithi114.stdout:9/715: read d2/d5c/dc9/f75 [949021,17191] 0 2024-02-16T22:22:11.808 INFO:tasks.workunit.client.0.smithi114.stdout:3/705: dread d4/d46/fc0 [0,4194304] 0 2024-02-16T22:22:11.809 INFO:tasks.workunit.client.0.smithi114.stdout:3/706: stat d4/d24/d2d/d45/c49 0 2024-02-16T22:22:11.809 INFO:tasks.workunit.client.0.smithi114.stdout:9/716: readlink d2/d33/d3f/l42 0 2024-02-16T22:22:11.809 INFO:tasks.workunit.client.1.smithi123.stdout:7/866: creat d3/d6e/d1d/db2/d35/dba/f118 x:0 0 0 2024-02-16T22:22:11.810 INFO:tasks.workunit.client.1.smithi123.stdout:7/867: fsync d3/d6e/d1d/db2/d35/f36 0 2024-02-16T22:22:11.811 INFO:tasks.workunit.client.0.smithi114.stdout:1/679: truncate d4/d9/d46/d67/d92/dca/dcc/d98/fba 2068072 0 2024-02-16T22:22:11.816 INFO:tasks.workunit.client.1.smithi123.stdout:5/877: rename d6/dd/df/d1f/ddc/dee to d6/dd/df/d90/d137 0 2024-02-16T22:22:11.819 INFO:tasks.workunit.client.0.smithi114.stdout:4/813: unlink dd/d2e/d3f/f50 0 2024-02-16T22:22:11.820 INFO:tasks.workunit.client.1.smithi123.stdout:1/858: mkdir d3/d6/d49/d48/d68/d124 0 2024-02-16T22:22:11.822 INFO:tasks.workunit.client.1.smithi123.stdout:0/813: creat d1/d4/d17/d54/db4/f10e x:0 0 0 2024-02-16T22:22:11.825 INFO:tasks.workunit.client.0.smithi114.stdout:2/647: link dd/df/d1c/d21/d61/fa9 dd/d3b/fdc 0 2024-02-16T22:22:11.827 INFO:tasks.workunit.client.0.smithi114.stdout:9/717: readlink d2/d5c/dc9/d1d/l87 0 2024-02-16T22:22:11.830 INFO:tasks.workunit.client.1.smithi123.stdout:4/956: dwrite d5/d1c/d5e/f110 [0,4194304] 0 2024-02-16T22:22:11.831 INFO:tasks.workunit.client.1.smithi123.stdout:8/907: rmdir d0/d29/da7/d26/d68/d81/d8f 39 2024-02-16T22:22:11.832 INFO:tasks.workunit.client.1.smithi123.stdout:8/908: chown d0/d29/da7/dea 207 1 2024-02-16T22:22:11.833 INFO:tasks.workunit.client.0.smithi114.stdout:9/718: dwrite d2/f5 [0,4194304] 0 2024-02-16T22:22:11.834 INFO:tasks.workunit.client.1.smithi123.stdout:3/901: creat d0/d20/d29/d9b/dbd/df4/d37/d41/f129 x:0 0 0 2024-02-16T22:22:11.835 INFO:tasks.workunit.client.1.smithi123.stdout:3/902: chown d0/d20/d29/d9b/dbd/df4/d5d 4074530 1 2024-02-16T22:22:11.836 INFO:tasks.workunit.client.1.smithi123.stdout:2/861: sync 2024-02-16T22:22:11.837 INFO:tasks.workunit.client.1.smithi123.stdout:6/886: sync 2024-02-16T22:22:11.837 INFO:tasks.workunit.client.0.smithi114.stdout:4/814: creat dd/d1b/d41/d77/f128 x:0 0 0 2024-02-16T22:22:11.837 INFO:tasks.workunit.client.1.smithi123.stdout:2/862: dread - d4/dda/f112 zero size 2024-02-16T22:22:11.837 INFO:tasks.workunit.client.0.smithi114.stdout:9/719: read d2/d5c/dc9/d47/d63/fa8 [1104,97617] 0 2024-02-16T22:22:11.839 INFO:tasks.workunit.client.0.smithi114.stdout:9/720: stat d2/d1a/d23/d32/d46/d78/fcd 0 2024-02-16T22:22:11.841 INFO:tasks.workunit.client.0.smithi114.stdout:5/681: unlink d0/df/d7a/dc7/lca 0 2024-02-16T22:22:11.847 INFO:tasks.workunit.client.0.smithi114.stdout:1/680: mknod d4/d9/d46/d67/d92/dca/dcc/d98/ddd/df2/cf7 0 2024-02-16T22:22:11.848 INFO:tasks.workunit.client.0.smithi114.stdout:1/681: chown d4/d9/d3d/d3f/d79 7733788 1 2024-02-16T22:22:11.849 INFO:tasks.workunit.client.0.smithi114.stdout:2/648: dread dd/d3b/da7/d4c/fb2 [0,4194304] 0 2024-02-16T22:22:11.849 INFO:tasks.workunit.client.1.smithi123.stdout:0/814: mknod d1/da0/c10f 0 2024-02-16T22:22:11.850 INFO:tasks.workunit.client.1.smithi123.stdout:9/990: mknod d4/d36/d8e/d58/d59/d60/deb/da7/dde/c142 0 2024-02-16T22:22:11.853 INFO:tasks.workunit.client.0.smithi114.stdout:5/682: symlink d0/d2d/d30/db3/lde 0 2024-02-16T22:22:11.855 INFO:tasks.workunit.client.0.smithi114.stdout:4/815: dwrite dd/f19 [0,4194304] 0 2024-02-16T22:22:11.858 INFO:tasks.workunit.client.1.smithi123.stdout:3/903: creat d0/d20/d29/d9b/dbd/df4/d5d/dae/db0/d123/f12a x:0 0 0 2024-02-16T22:22:11.864 INFO:tasks.workunit.client.1.smithi123.stdout:2/863: write d4/d1a/d1d/d59/f3a [1460784,102672] 0 2024-02-16T22:22:11.867 INFO:tasks.workunit.client.1.smithi123.stdout:5/878: mknod d6/dd/c138 0 2024-02-16T22:22:11.869 INFO:tasks.workunit.client.1.smithi123.stdout:6/887: dwrite d0/d6/d2f/da5/dc6/f125 [0,4194304] 0 2024-02-16T22:22:11.869 INFO:tasks.workunit.client.1.smithi123.stdout:0/815: truncate d1/d4/d42/fc0 398542 0 2024-02-16T22:22:11.870 INFO:tasks.workunit.client.0.smithi114.stdout:9/721: write d2/d1a/d23/f52 [4664761,97966] 0 2024-02-16T22:22:11.870 INFO:tasks.workunit.client.1.smithi123.stdout:5/879: truncate d6/dd/df/d58/d3f/dbe/f111 832875 0 2024-02-16T22:22:11.872 INFO:tasks.workunit.client.0.smithi114.stdout:4/816: mknod dd/d59/d8f/dea/c129 0 2024-02-16T22:22:11.873 INFO:tasks.workunit.client.1.smithi123.stdout:0/816: dread d1/d4/f43 [0,4194304] 0 2024-02-16T22:22:11.874 INFO:tasks.workunit.client.1.smithi123.stdout:9/991: rename d4/d36/d8e/d58/d59/d60/deb/da7/dde/l13a to d4/d36/d8e/d6d/l143 0 2024-02-16T22:22:11.874 INFO:tasks.workunit.client.1.smithi123.stdout:4/957: fsync d5/d49/f4b 0 2024-02-16T22:22:11.876 INFO:tasks.workunit.client.1.smithi123.stdout:1/859: dread d3/fe [0,4194304] 0 2024-02-16T22:22:11.876 INFO:tasks.workunit.client.0.smithi114.stdout:2/649: rename dd/df/d1c/d21/d61/c71 to dd/df/ddb/cdd 0 2024-02-16T22:22:11.878 INFO:tasks.workunit.client.1.smithi123.stdout:8/909: creat d0/d29/da7/d125/f12d x:0 0 0 2024-02-16T22:22:11.879 INFO:tasks.workunit.client.1.smithi123.stdout:8/910: chown d0/d29/da7/d26/d2a/d77/fd6 41540711 1 2024-02-16T22:22:11.880 INFO:tasks.workunit.client.0.smithi114.stdout:1/682: write d4/d9/d3d/d3f/d79/fc3 [781194,93119] 0 2024-02-16T22:22:11.880 INFO:tasks.workunit.client.1.smithi123.stdout:3/904: chown d0/d50/cdc 1 1 2024-02-16T22:22:11.881 INFO:tasks.workunit.client.1.smithi123.stdout:4/958: dwrite d5/d1c/d23/d76/f10c [0,4194304] 0 2024-02-16T22:22:11.882 INFO:tasks.workunit.client.0.smithi114.stdout:9/722: mknod d2/dd/d3c/ce7 0 2024-02-16T22:22:11.886 INFO:tasks.workunit.client.0.smithi114.stdout:5/683: write d0/d28/d98/d25/d2e/d3b/f85 [539488,110805] 0 2024-02-16T22:22:11.887 INFO:tasks.workunit.client.1.smithi123.stdout:7/868: dwrite d3/f4e [0,4194304] 0 2024-02-16T22:22:11.888 INFO:tasks.workunit.client.0.smithi114.stdout:5/684: chown d0/df/fa2 45549 1 2024-02-16T22:22:11.890 INFO:tasks.workunit.client.0.smithi114.stdout:1/683: dwrite d4/d9/d46/d67/fa8 [0,4194304] 0 2024-02-16T22:22:11.900 INFO:tasks.workunit.client.0.smithi114.stdout:2/650: rename dd/df/d1c/d21/d87/f4d to dd/df/d1c/d21/d61/d63/fde 0 2024-02-16T22:22:11.901 INFO:tasks.workunit.client.1.smithi123.stdout:2/864: creat d4/d1a/dd5/d10b/df1/db1/f119 x:0 0 0 2024-02-16T22:22:11.905 INFO:tasks.workunit.client.0.smithi114.stdout:5/685: chown d0/d28/c76 951 1 2024-02-16T22:22:11.908 INFO:tasks.workunit.client.0.smithi114.stdout:1/684: symlink d4/d9/d46/d67/d92/dca/lf8 0 2024-02-16T22:22:11.910 INFO:tasks.workunit.client.0.smithi114.stdout:2/651: symlink dd/df/ddb/ldf 0 2024-02-16T22:22:11.910 INFO:tasks.workunit.client.1.smithi123.stdout:0/817: fsync d1/d8/f85 0 2024-02-16T22:22:11.913 INFO:tasks.workunit.client.0.smithi114.stdout:4/817: dwrite dd/d1b/d37/f52 [0,4194304] 0 2024-02-16T22:22:11.913 INFO:tasks.workunit.client.0.smithi114.stdout:5/686: unlink d0/d28/d98/d1e/d63/d96/d9a/db8/f42 0 2024-02-16T22:22:11.913 INFO:tasks.workunit.client.0.smithi114.stdout:5/687: fdatasync d0/fa 0 2024-02-16T22:22:11.915 INFO:tasks.workunit.client.0.smithi114.stdout:4/818: write dd/d59/dfa/f10a [313468,128847] 0 2024-02-16T22:22:11.924 INFO:tasks.workunit.client.1.smithi123.stdout:4/959: mknod d5/db8/d11f/dc0/d118/d9f/c12b 0 2024-02-16T22:22:11.927 INFO:tasks.workunit.client.1.smithi123.stdout:7/869: creat d3/d6e/d1d/db2/d35/db7/f119 x:0 0 0 2024-02-16T22:22:11.927 INFO:tasks.workunit.client.1.smithi123.stdout:6/888: write d0/d6/d91/d97/fa6 [1612572,21350] 0 2024-02-16T22:22:11.929 INFO:tasks.workunit.client.1.smithi123.stdout:6/889: read d0/d96/fbf [6478295,93554] 0 2024-02-16T22:22:11.931 INFO:tasks.workunit.client.1.smithi123.stdout:2/865: rename d4/d1a/d1d/d13/l68 to d4/d1a/d1d/d13/d25/d98/dbb/d10d/l11a 0 2024-02-16T22:22:11.935 INFO:tasks.workunit.client.0.smithi114.stdout:5/688: write d0/f65 [144564,83382] 0 2024-02-16T22:22:11.937 INFO:tasks.workunit.client.1.smithi123.stdout:9/992: dwrite d4/d1e/fe7 [0,4194304] 0 2024-02-16T22:22:11.938 INFO:tasks.workunit.client.1.smithi123.stdout:9/993: write d4/d36/d57/db2/fed [850549,72471] 0 2024-02-16T22:22:11.938 INFO:tasks.workunit.client.1.smithi123.stdout:0/818: rmdir d1/d4/d17/d54/d8e/d96 39 2024-02-16T22:22:11.939 INFO:tasks.workunit.client.1.smithi123.stdout:0/819: chown d1/d4/d17/dad/fed 1 1 2024-02-16T22:22:11.940 INFO:tasks.workunit.client.0.smithi114.stdout:1/685: truncate d4/d63/d69/d7c/d9b/db0/f3e 3323454 0 2024-02-16T22:22:11.940 INFO:tasks.workunit.client.0.smithi114.stdout:2/652: dwrite dd/d3b/da7/d4c/f79 [0,4194304] 0 2024-02-16T22:22:11.944 INFO:tasks.workunit.client.0.smithi114.stdout:4/819: dwrite dd/d1b/d24/f4a [0,4194304] 0 2024-02-16T22:22:11.944 INFO:tasks.workunit.client.1.smithi123.stdout:1/860: creat d3/d6/db/d10/d2a/d11c/f125 x:0 0 0 2024-02-16T22:22:11.945 INFO:tasks.workunit.client.1.smithi123.stdout:1/861: write d3/d6/d49/d48/d84/d92/f10b [784669,19077] 0 2024-02-16T22:22:11.952 INFO:tasks.workunit.client.0.smithi114.stdout:4/820: fdatasync dd/d1b/d24/f76 0 2024-02-16T22:22:11.952 INFO:tasks.workunit.client.1.smithi123.stdout:1/862: dwrite d3/d6/d11/f17 [4194304,4194304] 0 2024-02-16T22:22:11.957 INFO:tasks.workunit.client.0.smithi114.stdout:4/821: mkdir dd/d1b/d41/d74/d12a 0 2024-02-16T22:22:11.960 INFO:tasks.workunit.client.1.smithi123.stdout:9/994: rename d4/d1e/fc8 to d4/d36/d8e/d84/d93/f144 0 2024-02-16T22:22:11.963 INFO:tasks.workunit.client.1.smithi123.stdout:1/863: dwrite d3/d6/d49/d48/d84/d92/ffe [0,4194304] 0 2024-02-16T22:22:11.967 INFO:tasks.workunit.client.1.smithi123.stdout:8/911: creat d0/d29/da7/d26/d2a/d123/f12e x:0 0 0 2024-02-16T22:22:11.968 INFO:tasks.workunit.client.1.smithi123.stdout:3/905: creat d0/d20/d29/d9b/dbd/df4/f12b x:0 0 0 2024-02-16T22:22:11.970 INFO:tasks.workunit.client.1.smithi123.stdout:4/960: mkdir d5/d1c/d23/d10f/d12c 0 2024-02-16T22:22:11.972 INFO:tasks.workunit.client.0.smithi114.stdout:2/653: write dd/d3b/da7/d4c/f64 [628978,77078] 0 2024-02-16T22:22:11.973 INFO:tasks.workunit.client.0.smithi114.stdout:1/686: write d4/d9/d3d/d3f/f81 [3627571,114347] 0 2024-02-16T22:22:11.976 INFO:tasks.workunit.client.1.smithi123.stdout:5/880: getdents d6/dc1/de6/d124 0 2024-02-16T22:22:11.978 INFO:tasks.workunit.client.0.smithi114.stdout:1/687: creat d4/d9/d46/d67/ff9 x:0 0 0 2024-02-16T22:22:11.980 INFO:tasks.workunit.client.0.smithi114.stdout:2/654: truncate dd/df/f52 992158 0 2024-02-16T22:22:11.983 INFO:tasks.workunit.client.1.smithi123.stdout:2/866: dread d4/d1a/d1d/d38/d7b/f94 [0,4194304] 0 2024-02-16T22:22:11.984 INFO:tasks.workunit.client.1.smithi123.stdout:9/995: read d4/d36/daa/f23 [3473402,30866] 0 2024-02-16T22:22:11.985 INFO:tasks.workunit.client.0.smithi114.stdout:2/655: creat dd/d4e/fe0 x:0 0 0 2024-02-16T22:22:11.985 INFO:tasks.workunit.client.0.smithi114.stdout:2/656: chown dd/fd4 1564908606 1 2024-02-16T22:22:11.986 INFO:tasks.workunit.client.1.smithi123.stdout:0/820: fsync d1/d8/f6b 0 2024-02-16T22:22:11.989 INFO:tasks.workunit.client.1.smithi123.stdout:1/864: unlink d3/d6/d11/c75 0 2024-02-16T22:22:11.991 INFO:tasks.workunit.client.1.smithi123.stdout:8/912: truncate d0/d29/da7/d26/d68/d81/d47/d9b/fa8 1743401 0 2024-02-16T22:22:11.992 INFO:tasks.workunit.client.0.smithi114.stdout:2/657: dread dd/f3a [0,4194304] 0 2024-02-16T22:22:11.994 INFO:tasks.workunit.client.0.smithi114.stdout:1/688: dread d4/d9/d46/d67/d92/dca/dcc/fd5 [0,4194304] 0 2024-02-16T22:22:11.994 INFO:tasks.workunit.client.0.smithi114.stdout:2/658: chown dd/df/d1c/d21/d61/d69/fce 3517 1 2024-02-16T22:22:11.994 INFO:tasks.workunit.client.1.smithi123.stdout:4/961: mkdir d5/d1c/d5e/dc3/d12d 0 2024-02-16T22:22:11.994 INFO:tasks.workunit.client.1.smithi123.stdout:1/865: dread d3/d6/d49/d48/d84/d92/f10b [0,4194304] 0 2024-02-16T22:22:11.996 INFO:tasks.workunit.client.1.smithi123.stdout:7/870: link d3/d6e/d1d/d97/dce/f117 d3/d6e/d1d/f11a 0 2024-02-16T22:22:12.000 INFO:tasks.workunit.client.1.smithi123.stdout:5/881: fdatasync d6/dd/d44/f7e 0 2024-02-16T22:22:12.002 INFO:tasks.workunit.client.0.smithi114.stdout:2/659: creat dd/d3b/d8a/dcb/fe1 x:0 0 0 2024-02-16T22:22:12.004 INFO:tasks.workunit.client.1.smithi123.stdout:2/867: rename d4/dda to d4/d1a/d1d/d13/d25/d98/d10c/d11b 0 2024-02-16T22:22:12.006 INFO:tasks.workunit.client.0.smithi114.stdout:4/822: dwrite dd/d1b/d24/d98/fc5 [0,4194304] 0 2024-02-16T22:22:12.006 INFO:tasks.workunit.client.1.smithi123.stdout:9/996: mkdir d4/d36/daa/d20/dcb/d145 0 2024-02-16T22:22:12.010 INFO:tasks.workunit.client.0.smithi114.stdout:2/660: mkdir dd/d3b/d8a/dcb/de2 0 2024-02-16T22:22:12.012 INFO:tasks.workunit.client.1.smithi123.stdout:0/821: symlink d1/d4/d17/d74/de7/l110 0 2024-02-16T22:22:12.012 INFO:tasks.workunit.client.1.smithi123.stdout:6/890: dwrite d0/d9/dc/d3a/f81 [0,4194304] 0 2024-02-16T22:22:12.014 INFO:tasks.workunit.client.0.smithi114.stdout:4/823: rename dd/d1b/d37 to dd/d1b/d24/d57/d12b 0 2024-02-16T22:22:12.017 INFO:tasks.workunit.client.1.smithi123.stdout:3/906: symlink d0/d20/d29/l12c 0 2024-02-16T22:22:12.018 INFO:tasks.workunit.client.1.smithi123.stdout:3/907: chown d0/d50/l9e 493704 1 2024-02-16T22:22:12.018 INFO:tasks.workunit.client.0.smithi114.stdout:2/661: creat dd/df/d1c/d21/d61/d69/fe3 x:0 0 0 2024-02-16T22:22:12.021 INFO:tasks.workunit.client.0.smithi114.stdout:4/824: readlink dd/d1b/d24/l8b 0 2024-02-16T22:22:12.022 INFO:tasks.workunit.client.0.smithi114.stdout:2/662: chown dd/d3b/c53 117701 1 2024-02-16T22:22:12.026 INFO:tasks.workunit.client.1.smithi123.stdout:1/866: unlink d3/d6/db/fdd 0 2024-02-16T22:22:12.028 INFO:tasks.workunit.client.1.smithi123.stdout:4/962: dwrite d5/d1c/f35 [0,4194304] 0 2024-02-16T22:22:12.032 INFO:tasks.workunit.client.1.smithi123.stdout:2/868: mkdir d4/d1a/dd5/d11c 0 2024-02-16T22:22:12.033 INFO:tasks.workunit.client.1.smithi123.stdout:9/997: read - d4/d36/d8e/d58/d59/d60/fd2 zero size 2024-02-16T22:22:12.033 INFO:tasks.workunit.client.0.smithi114.stdout:1/689: write d4/d9/d46/d4d/f5c [4152112,76377] 0 2024-02-16T22:22:12.034 INFO:tasks.workunit.client.1.smithi123.stdout:9/998: dread - d4/d36/d57/f129 zero size 2024-02-16T22:22:12.036 INFO:tasks.workunit.client.1.smithi123.stdout:0/822: read d1/d4/f43 [1636409,88358] 0 2024-02-16T22:22:12.037 INFO:tasks.workunit.client.1.smithi123.stdout:6/891: creat d0/d6/d91/d93/d116/f141 x:0 0 0 2024-02-16T22:22:12.038 INFO:tasks.workunit.client.1.smithi123.stdout:6/892: readlink d0/d6/d4d/db0/lc4 0 2024-02-16T22:22:12.039 INFO:tasks.workunit.client.1.smithi123.stdout:6/893: write d0/d6/d1a/f6f [3831998,60750] 0 2024-02-16T22:22:12.039 INFO:tasks.workunit.client.0.smithi114.stdout:1/690: symlink d4/d9/d46/d67/lfa 0 2024-02-16T22:22:12.043 INFO:tasks.workunit.client.1.smithi123.stdout:8/913: mkdir d0/d29/da7/d26/d2a/d12f 0 2024-02-16T22:22:12.043 INFO:tasks.workunit.client.1.smithi123.stdout:8/914: stat d0/d12/ded/f111 0 2024-02-16T22:22:12.046 INFO:tasks.workunit.client.0.smithi114.stdout:2/663: dread dd/f28 [0,4194304] 0 2024-02-16T22:22:12.046 INFO:tasks.workunit.client.0.smithi114.stdout:2/664: chown dd/df/d1c/fa4 74080811 1 2024-02-16T22:22:12.052 INFO:tasks.workunit.client.1.smithi123.stdout:1/867: mkdir d3/d6/d49/d48/d68/d126 0 2024-02-16T22:22:12.054 INFO:tasks.workunit.client.0.smithi114.stdout:2/665: creat dd/df/d1c/d21/d61/fe4 x:0 0 0 2024-02-16T22:22:12.059 INFO:tasks.workunit.client.0.smithi114.stdout:2/666: symlink dd/df/d1c/d21/d87/le5 0 2024-02-16T22:22:12.061 INFO:tasks.workunit.client.0.smithi114.stdout:2/667: truncate dd/df/d1c/d21/d61/d63/d83/fd3 371719 0 2024-02-16T22:22:12.061 INFO:tasks.workunit.client.1.smithi123.stdout:2/869: unlink d4/d1a/d1d/d13/d25/d98/dbb/ff2 0 2024-02-16T22:22:12.062 INFO:tasks.workunit.client.1.smithi123.stdout:5/882: dwrite d6/dd/df/d90/d137/ffc [0,4194304] 0 2024-02-16T22:22:12.063 INFO:tasks.workunit.client.1.smithi123.stdout:9/999: dread d4/fa [0,4194304] 0 2024-02-16T22:22:12.064 INFO:tasks.workunit.client.1.smithi123.stdout:5/883: chown d6/dd/df/d58/d3f/dbe/c23 33829952 1 2024-02-16T22:22:12.068 INFO:tasks.workunit.client.0.smithi114.stdout:1/691: truncate d4/d9/d46/d4d/f5c 330557 0 2024-02-16T22:22:12.070 INFO:tasks.workunit.client.1.smithi123.stdout:2/870: dwrite d4/d1a/dd5/d10b/df1/db1/de3/ff4 [0,4194304] 0 2024-02-16T22:22:12.073 INFO:tasks.workunit.client.0.smithi114.stdout:1/692: mknod d4/d9/d3d/d3f/cfb 0 2024-02-16T22:22:12.076 INFO:tasks.workunit.client.1.smithi123.stdout:5/884: dread d6/dd/df/f6c [0,4194304] 0 2024-02-16T22:22:12.076 INFO:tasks.workunit.client.1.smithi123.stdout:7/871: rename d3/d6e/d1d/db2/d3d/fed to d3/d5/f11b 0 2024-02-16T22:22:12.077 INFO:tasks.workunit.client.1.smithi123.stdout:4/963: dwrite d5/d1c/d23/d74/f116 [0,4194304] 0 2024-02-16T22:22:12.079 INFO:tasks.workunit.client.1.smithi123.stdout:4/964: chown d5/d1c/d64/fb6 104 1 2024-02-16T22:22:12.079 INFO:tasks.workunit.client.0.smithi114.stdout:1/693: creat d4/d63/d69/d7c/d9b/ffc x:0 0 0 2024-02-16T22:22:12.081 INFO:tasks.workunit.client.0.smithi114.stdout:2/668: truncate dd/d6b/f89 1473655 0 2024-02-16T22:22:12.087 INFO:tasks.workunit.client.1.smithi123.stdout:0/823: dwrite d1/f12 [0,4194304] 0 2024-02-16T22:22:12.089 INFO:tasks.workunit.client.1.smithi123.stdout:0/824: fsync d1/d8/d13/d1d/f97 0 2024-02-16T22:22:12.091 INFO:tasks.workunit.client.1.smithi123.stdout:5/885: dread - d6/d3b/ff6 zero size 2024-02-16T22:22:12.092 INFO:tasks.workunit.client.1.smithi123.stdout:7/872: truncate d3/d6e/d4d/d92/fb0 86398 0 2024-02-16T22:22:12.093 INFO:tasks.workunit.client.1.smithi123.stdout:5/886: dread - d6/dc1/de6/d124/f131 zero size 2024-02-16T22:22:12.095 INFO:tasks.workunit.client.1.smithi123.stdout:8/915: creat d0/d29/da7/d43/f130 x:0 0 0 2024-02-16T22:22:12.096 INFO:tasks.workunit.client.1.smithi123.stdout:5/887: truncate d6/dd/df/d58/d3f/d82/f127 729138 0 2024-02-16T22:22:12.096 INFO:tasks.workunit.client.1.smithi123.stdout:1/868: rmdir d3/d6/d49/d48/d68/d124 0 2024-02-16T22:22:12.098 INFO:tasks.workunit.client.1.smithi123.stdout:1/869: write d3/d6/db/d10/d4b/d70/d87/f10d [387064,99442] 0 2024-02-16T22:22:12.098 INFO:tasks.workunit.client.1.smithi123.stdout:7/873: mknod d3/d6e/d24/c11c 0 2024-02-16T22:22:12.101 INFO:tasks.workunit.client.1.smithi123.stdout:6/894: link d0/d9/dc/l24 d0/d6/d8e/d123/l142 0 2024-02-16T22:22:12.102 INFO:tasks.workunit.client.1.smithi123.stdout:8/916: stat d0/l8 0 2024-02-16T22:22:12.103 INFO:tasks.workunit.client.1.smithi123.stdout:5/888: mknod d6/dd/df/d58/d89/dc3/c139 0 2024-02-16T22:22:12.104 INFO:tasks.workunit.client.1.smithi123.stdout:0/825: rename d1/d4/d17/d54/d8e/d96/dee/l108 to d1/d4/d17/d54/db4/l111 0 2024-02-16T22:22:12.105 INFO:tasks.workunit.client.1.smithi123.stdout:1/870: symlink d3/d6/db/d10/d2a/l127 0 2024-02-16T22:22:12.105 INFO:tasks.workunit.client.1.smithi123.stdout:2/871: write d4/d1a/dd5/d10b/df1/fb5 [413241,100885] 0 2024-02-16T22:22:12.107 INFO:tasks.workunit.client.1.smithi123.stdout:7/874: dwrite d3/d6e/d1d/db2/d35/f107 [0,4194304] 0 2024-02-16T22:22:12.107 INFO:tasks.workunit.client.1.smithi123.stdout:0/826: chown d1/d4/d17/fea 246356971 1 2024-02-16T22:22:12.108 INFO:tasks.workunit.client.1.smithi123.stdout:0/827: chown d1/d4/d17/dad 41699 1 2024-02-16T22:22:12.109 INFO:tasks.workunit.client.1.smithi123.stdout:8/917: fsync d0/d12/f1e 0 2024-02-16T22:22:12.111 INFO:tasks.workunit.client.1.smithi123.stdout:5/889: creat d6/dd/df/d58/d89/d98/dd8/f13a x:0 0 0 2024-02-16T22:22:12.112 INFO:tasks.workunit.client.1.smithi123.stdout:4/965: dread d5/f63 [0,4194304] 0 2024-02-16T22:22:12.112 INFO:tasks.workunit.client.0.smithi114.stdout:2/669: write dd/df/d1c/d21/fb1 [931818,44834] 0 2024-02-16T22:22:12.114 INFO:tasks.workunit.client.1.smithi123.stdout:3/908: sync 2024-02-16T22:22:12.120 INFO:tasks.workunit.client.0.smithi114.stdout:2/670: dwrite dd/d6b/fbf [0,4194304] 0 2024-02-16T22:22:12.124 INFO:tasks.workunit.client.0.smithi114.stdout:2/671: rename dd/df/d1c/d21/l2a to dd/d6b/da5/le6 0 2024-02-16T22:22:12.125 INFO:tasks.workunit.client.0.smithi114.stdout:2/672: write dd/df/d1c/d21/d61/d69/f72 [2340674,38836] 0 2024-02-16T22:22:12.126 INFO:tasks.workunit.client.1.smithi123.stdout:2/872: rename d4/d1a/dd5/d10b/df1/d4a/dce/dd4/cd6 to d4/d1a/d1d/d38/d5e/de8/c11d 0 2024-02-16T22:22:12.129 INFO:tasks.workunit.client.1.smithi123.stdout:4/966: chown d5/d1c/d64/fb6 6 1 2024-02-16T22:22:12.129 INFO:tasks.workunit.client.1.smithi123.stdout:8/918: unlink d0/d29/da7/d43/d34/dae/fc1 0 2024-02-16T22:22:12.132 INFO:tasks.workunit.client.1.smithi123.stdout:6/895: write d0/f60 [39506,98756] 0 2024-02-16T22:22:12.135 INFO:tasks.workunit.client.1.smithi123.stdout:2/873: fsync d4/d1a/d1d/f9f 0 2024-02-16T22:22:12.136 INFO:tasks.workunit.client.1.smithi123.stdout:7/875: write d3/d5/d6c/fac [89939,118374] 0 2024-02-16T22:22:12.136 INFO:tasks.workunit.client.0.smithi114.stdout:2/673: truncate dd/d3b/da7/d4c/fbe 1851432 0 2024-02-16T22:22:12.141 INFO:tasks.workunit.client.1.smithi123.stdout:0/828: dwrite d1/d8/f34 [0,4194304] 0 2024-02-16T22:22:12.142 INFO:tasks.workunit.client.1.smithi123.stdout:0/829: chown d1/d4/l45 37688 1 2024-02-16T22:22:12.143 INFO:tasks.workunit.client.1.smithi123.stdout:0/830: fsync d1/d4/d17/d54/db4/f10e 0 2024-02-16T22:22:12.144 INFO:tasks.workunit.client.1.smithi123.stdout:7/876: dwrite d3/d6e/d24/f49 [0,4194304] 0 2024-02-16T22:22:12.145 INFO:tasks.workunit.client.1.smithi123.stdout:0/831: fsync d1/d4/d17/d54/d8e/d96/f9e 0 2024-02-16T22:22:12.155 INFO:tasks.workunit.client.0.smithi114.stdout:2/674: creat dd/d3b/d8a/fe7 x:0 0 0 2024-02-16T22:22:12.161 INFO:tasks.workunit.client.1.smithi123.stdout:8/919: dread d0/d61/d66/d7e/ff1 [0,4194304] 0 2024-02-16T22:22:12.162 INFO:tasks.workunit.client.1.smithi123.stdout:4/967: truncate d5/d1c/d23/d47/f50 1310508 0 2024-02-16T22:22:12.163 INFO:tasks.workunit.client.1.smithi123.stdout:8/920: readlink d0/d29/da7/d26/d68/d81/d47/le5 0 2024-02-16T22:22:12.167 INFO:tasks.workunit.client.1.smithi123.stdout:3/909: dread d0/fc0 [0,4194304] 0 2024-02-16T22:22:12.167 INFO:tasks.workunit.client.1.smithi123.stdout:2/874: symlink d4/d1a/dd5/d10b/df1/db1/l11e 0 2024-02-16T22:22:12.168 INFO:tasks.workunit.client.1.smithi123.stdout:7/877: truncate d3/d6e/d4d/f5a 1345356 0 2024-02-16T22:22:12.170 INFO:tasks.workunit.client.1.smithi123.stdout:0/832: creat d1/d4/d17/f112 x:0 0 0 2024-02-16T22:22:12.178 INFO:tasks.workunit.client.1.smithi123.stdout:5/890: link d6/dd/df/d58/d3f/d82/f127 d6/dd/d78/da5/db0/f13b 0 2024-02-16T22:22:12.178 INFO:tasks.workunit.client.1.smithi123.stdout:8/921: symlink d0/d29/da7/d26/d2a/d123/dca/l131 0 2024-02-16T22:22:12.178 INFO:tasks.workunit.client.1.smithi123.stdout:7/878: symlink d3/d6e/d34/l11d 0 2024-02-16T22:22:12.179 INFO:tasks.workunit.client.1.smithi123.stdout:7/879: fsync d3/d6e/f2e 0 2024-02-16T22:22:12.184 INFO:tasks.workunit.client.1.smithi123.stdout:0/833: mknod d1/d4/c113 0 2024-02-16T22:22:12.185 INFO:tasks.workunit.client.1.smithi123.stdout:7/880: dwrite d3/d6e/d1d/db2/d35/f36 [4194304,4194304] 0 2024-02-16T22:22:12.185 INFO:tasks.workunit.client.1.smithi123.stdout:0/834: fsync d1/d8/d13/d1d/d62/fbd 0 2024-02-16T22:22:12.187 INFO:tasks.workunit.client.1.smithi123.stdout:4/968: link d5/db8/c11c d5/d1c/d23/d74/d92/d9b/c12e 0 2024-02-16T22:22:12.188 INFO:tasks.workunit.client.0.smithi114.stdout:2/675: dwrite dd/f28 [0,4194304] 0 2024-02-16T22:22:12.190 INFO:tasks.workunit.client.0.smithi114.stdout:2/676: fdatasync dd/f5d 0 2024-02-16T22:22:12.190 INFO:tasks.workunit.client.1.smithi123.stdout:4/969: write d5/d1c/d23/d31/f2d [3832593,54512] 0 2024-02-16T22:22:12.191 INFO:tasks.workunit.client.0.smithi114.stdout:2/677: write dd/df/d1c/d21/f30 [1469590,56414] 0 2024-02-16T22:22:12.193 INFO:tasks.workunit.client.1.smithi123.stdout:6/896: dwrite d0/d9/dc/f23 [0,4194304] 0 2024-02-16T22:22:12.193 INFO:tasks.workunit.client.1.smithi123.stdout:5/891: unlink d6/dd/df/d58/d89/d98/dd8/f122 0 2024-02-16T22:22:12.208 INFO:tasks.workunit.client.1.smithi123.stdout:5/892: creat d6/dd/df/d58/d89/d98/dd8/f13c x:0 0 0 2024-02-16T22:22:12.213 INFO:tasks.workunit.client.1.smithi123.stdout:7/881: write d3/d5/ff [4977520,22140] 0 2024-02-16T22:22:12.214 INFO:tasks.workunit.client.1.smithi123.stdout:7/882: chown d3/d6e/d1d/db2/lf9 0 1 2024-02-16T22:22:12.216 INFO:tasks.workunit.client.1.smithi123.stdout:7/883: mknod d3/d6e/d1d/db2/d35/dba/c11e 0 2024-02-16T22:22:12.220 INFO:tasks.workunit.client.1.smithi123.stdout:7/884: link d3/f4e d3/d59/f11f 0 2024-02-16T22:22:12.221 INFO:tasks.workunit.client.1.smithi123.stdout:7/885: chown d3/l47 432 1 2024-02-16T22:22:12.223 INFO:tasks.workunit.client.1.smithi123.stdout:7/886: truncate d3/d6e/d24/f39 5041027 0 2024-02-16T22:22:12.224 INFO:tasks.workunit.client.1.smithi123.stdout:0/835: dread d1/d4/f51 [0,4194304] 0 2024-02-16T22:22:12.226 INFO:tasks.workunit.client.1.smithi123.stdout:0/836: fdatasync d1/d8/d13/d1d/d64/dbc/ffa 0 2024-02-16T22:22:12.226 INFO:tasks.workunit.client.1.smithi123.stdout:4/970: write d5/d1c/d23/f10a [435673,106039] 0 2024-02-16T22:22:12.228 INFO:tasks.workunit.client.1.smithi123.stdout:4/971: rmdir d5/db8/d11f 39 2024-02-16T22:22:12.230 INFO:tasks.workunit.client.0.smithi114.stdout:2/678: dread dd/df/f9e [0,4194304] 0 2024-02-16T22:22:12.230 INFO:tasks.workunit.client.1.smithi123.stdout:4/972: dread - d5/d1c/d23/d47/da2/daa/fe6 zero size 2024-02-16T22:22:12.231 INFO:tasks.workunit.client.1.smithi123.stdout:7/887: write d3/d6e/d24/fd8 [3602229,128122] 0 2024-02-16T22:22:12.231 INFO:tasks.workunit.client.0.smithi114.stdout:2/679: creat dd/d3b/d8a/fe8 x:0 0 0 2024-02-16T22:22:12.232 INFO:tasks.workunit.client.1.smithi123.stdout:5/893: dwrite d6/dd/df/d58/d3f/dbe/f97 [0,4194304] 0 2024-02-16T22:22:12.233 INFO:tasks.workunit.client.0.smithi114.stdout:2/680: mkdir dd/df/d1c/d21/d61/d69/de9 0 2024-02-16T22:22:12.234 INFO:tasks.workunit.client.1.smithi123.stdout:5/894: stat d6/dc1/de6/d124/f131 0 2024-02-16T22:22:12.235 INFO:tasks.workunit.client.1.smithi123.stdout:7/888: rmdir d3/d6e/d1d/db2/d3d 39 2024-02-16T22:22:12.236 INFO:tasks.workunit.client.0.smithi114.stdout:2/681: symlink dd/df/d1c/d21/d61/d63/d83/lea 0 2024-02-16T22:22:12.237 INFO:tasks.workunit.client.0.smithi114.stdout:2/682: dread - dd/df/d1c/d21/d61/d63/dc2/fd2 zero size 2024-02-16T22:22:12.238 INFO:tasks.workunit.client.1.smithi123.stdout:5/895: unlink d6/dd/df/d58/l2d 0 2024-02-16T22:22:12.239 INFO:tasks.workunit.client.0.smithi114.stdout:2/683: chown dd/df/d1c/d21/d61/d69/de9 195390569 1 2024-02-16T22:22:12.239 INFO:tasks.workunit.client.1.smithi123.stdout:4/973: mkdir d5/db8/d11f/dc0/d118/d73/d11d/d12f 0 2024-02-16T22:22:12.239 INFO:tasks.workunit.client.1.smithi123.stdout:5/896: write d6/dd/df/d9f/d102/f101 [427558,93526] 0 2024-02-16T22:22:12.240 INFO:tasks.workunit.client.0.smithi114.stdout:2/684: chown dd/d3b/c96 25743 1 2024-02-16T22:22:12.240 INFO:tasks.workunit.client.1.smithi123.stdout:7/889: stat d3/d5/c15 0 2024-02-16T22:22:12.241 INFO:tasks.workunit.client.1.smithi123.stdout:4/974: creat d5/d1c/d5e/dc3/f130 x:0 0 0 2024-02-16T22:22:12.242 INFO:tasks.workunit.client.1.smithi123.stdout:4/975: dread - d5/d1c/d23/d31/f83 zero size 2024-02-16T22:22:12.242 INFO:tasks.workunit.client.1.smithi123.stdout:4/976: fdatasync d5/d1c/d23/f10a 0 2024-02-16T22:22:12.242 INFO:tasks.workunit.client.1.smithi123.stdout:7/890: creat d3/d87/da8/f120 x:0 0 0 2024-02-16T22:22:12.243 INFO:tasks.workunit.client.1.smithi123.stdout:4/977: fdatasync d5/d1c/d5e/f109 0 2024-02-16T22:22:12.244 INFO:tasks.workunit.client.1.smithi123.stdout:1/871: sync 2024-02-16T22:22:12.246 INFO:tasks.workunit.client.1.smithi123.stdout:7/891: rename d3/d6e/d24/lc9 to d3/d87/l121 0 2024-02-16T22:22:12.249 INFO:tasks.workunit.client.0.smithi114.stdout:2/685: rename dd/df/d1c/d21/d61/d63/d83/cb0 to dd/d6b/ceb 0 2024-02-16T22:22:12.250 INFO:tasks.workunit.client.1.smithi123.stdout:1/872: dwrite d3/d6/db/d10/d2d/d8c/dfd/fcd [4194304,4194304] 0 2024-02-16T22:22:12.254 INFO:tasks.workunit.client.1.smithi123.stdout:7/892: rename d3/d5/d6c/fac to d3/d6e/d4d/d92/f122 0 2024-02-16T22:22:12.255 INFO:tasks.workunit.client.1.smithi123.stdout:5/897: write d6/dd/df/d58/d3f/f92 [4489630,48752] 0 2024-02-16T22:22:12.256 INFO:tasks.workunit.client.1.smithi123.stdout:5/898: dread - d6/dd/d78/da5/ff7 zero size 2024-02-16T22:22:12.262 INFO:tasks.workunit.client.1.smithi123.stdout:7/893: mknod d3/d5/d6c/da6/df1/c123 0 2024-02-16T22:22:12.263 INFO:tasks.workunit.client.1.smithi123.stdout:5/899: dwrite d6/dd/d44/dba/d126/d12c/ff5 [0,4194304] 0 2024-02-16T22:22:12.263 INFO:tasks.workunit.client.1.smithi123.stdout:4/978: truncate d5/d1c/d5e/f72 1400774 0 2024-02-16T22:22:12.270 INFO:tasks.workunit.client.1.smithi123.stdout:7/894: dread d3/d6e/d1d/fcb [0,4194304] 0 2024-02-16T22:22:12.274 INFO:tasks.workunit.client.0.smithi114.stdout:2/686: dread dd/df/f1e [4194304,4194304] 0 2024-02-16T22:22:12.281 INFO:tasks.workunit.client.1.smithi123.stdout:1/873: dread d3/d6/d49/d48/d68/fab [0,4194304] 0 2024-02-16T22:22:12.282 INFO:tasks.workunit.client.1.smithi123.stdout:1/874: chown d3/d6/d49/d48/d68/fc5 24165238 1 2024-02-16T22:22:12.314 INFO:tasks.workunit.client.0.smithi114.stdout:0/748: sync 2024-02-16T22:22:12.316 INFO:tasks.workunit.client.0.smithi114.stdout:0/749: truncate d1/d16/d1f/d28/d51/d60/d52/fd0 665496 0 2024-02-16T22:22:12.317 INFO:tasks.workunit.client.0.smithi114.stdout:0/750: chown d1/d16/d1f/d28/d51/d61 461 1 2024-02-16T22:22:12.326 INFO:tasks.workunit.client.1.smithi123.stdout:3/910: sync 2024-02-16T22:22:12.347 INFO:tasks.workunit.client.1.smithi123.stdout:7/895: symlink d3/d87/da8/l124 0 2024-02-16T22:22:12.349 INFO:tasks.workunit.client.0.smithi114.stdout:2/687: mkdir dd/df/d1c/d21/d61/dec 0 2024-02-16T22:22:12.352 INFO:tasks.workunit.client.0.smithi114.stdout:0/751: rmdir d1/d1a/d47/d62/d89 39 2024-02-16T22:22:12.354 INFO:tasks.workunit.client.1.smithi123.stdout:3/911: rmdir d0/d20/d29/d9b/dbd/df4/d5d/dae/db0/d123 39 2024-02-16T22:22:12.355 INFO:tasks.workunit.client.1.smithi123.stdout:3/912: read d0/d20/d29/d9b/dbd/df5/fb4 [589731,48064] 0 2024-02-16T22:22:12.359 INFO:tasks.workunit.client.0.smithi114.stdout:0/752: symlink d1/d16/d1f/d28/d51/d61/d7e/d90/d95/d80/leb 0 2024-02-16T22:22:12.361 INFO:tasks.workunit.client.1.smithi123.stdout:1/875: dwrite d3/d6/db/fbc [0,4194304] 0 2024-02-16T22:22:12.361 INFO:tasks.workunit.client.1.smithi123.stdout:2/875: sync 2024-02-16T22:22:12.363 INFO:tasks.workunit.client.0.smithi114.stdout:0/753: truncate d1/d16/d1f/d28/d51/fb3 1711685 0 2024-02-16T22:22:12.363 INFO:tasks.workunit.client.0.smithi114.stdout:2/688: getdents dd/d6b/db8 0 2024-02-16T22:22:12.365 INFO:tasks.workunit.client.1.smithi123.stdout:7/896: dwrite d3/d6e/d34/f75 [0,4194304] 0 2024-02-16T22:22:12.366 INFO:tasks.workunit.client.0.smithi114.stdout:0/754: write d1/d16/d1f/d28/d51/d61/d7e/fc2 [705974,81030] 0 2024-02-16T22:22:12.368 INFO:tasks.workunit.client.0.smithi114.stdout:2/689: mknod dd/d3b/da7/d8b/dc0/ced 0 2024-02-16T22:22:12.369 INFO:tasks.workunit.client.0.smithi114.stdout:2/690: stat dd/df/ddb/ldf 0 2024-02-16T22:22:12.370 INFO:tasks.workunit.client.1.smithi123.stdout:1/876: mkdir d3/d6/db/d10/d4b/d70/d87/d128 0 2024-02-16T22:22:12.370 INFO:tasks.workunit.client.1.smithi123.stdout:2/876: fdatasync d4/d1a/d1d/d59/dd7/ddc/ff9 0 2024-02-16T22:22:12.373 INFO:tasks.workunit.client.0.smithi114.stdout:0/755: mknod d1/d1a/d47/d62/d7f/d8c/db1/cec 0 2024-02-16T22:22:12.374 INFO:tasks.workunit.client.0.smithi114.stdout:0/756: stat d1/d16/d1f/d28/d51/d61/d7e/d90/f42 0 2024-02-16T22:22:12.376 INFO:tasks.workunit.client.0.smithi114.stdout:2/691: creat dd/d3b/d8a/dcb/fee x:0 0 0 2024-02-16T22:22:12.378 INFO:tasks.workunit.client.1.smithi123.stdout:1/877: mkdir d3/d6/db/d10/d2d/d8c/dfd/d9f/db4/ddf/df0/d129 0 2024-02-16T22:22:12.380 INFO:tasks.workunit.client.0.smithi114.stdout:0/757: rename d1/d16/d18/l8d to d1/d1a/d47/d62/d7f/led 0 2024-02-16T22:22:12.382 INFO:tasks.workunit.client.0.smithi114.stdout:2/692: truncate dd/d3b/da7/d4c/fb2 282516 0 2024-02-16T22:22:12.382 INFO:tasks.workunit.client.0.smithi114.stdout:2/693: readlink l2 0 2024-02-16T22:22:12.383 INFO:tasks.workunit.client.1.smithi123.stdout:2/877: dwrite d4/d1a/d1d/d38/d5e/fcf [0,4194304] 0 2024-02-16T22:22:12.384 INFO:tasks.workunit.client.1.smithi123.stdout:1/878: creat d3/d6/db/d10/d2d/d8c/dfd/de8/f12a x:0 0 0 2024-02-16T22:22:12.389 INFO:tasks.workunit.client.0.smithi114.stdout:2/694: write dd/d3b/da7/fa3 [728949,118190] 0 2024-02-16T22:22:12.389 INFO:tasks.workunit.client.1.smithi123.stdout:2/878: read d4/d1a/dd5/fd8 [2397280,80639] 0 2024-02-16T22:22:12.390 INFO:tasks.workunit.client.1.smithi123.stdout:2/879: chown d4/d1a/d1d/d13/d25/d98/d10c/d11b/fde 19900 1 2024-02-16T22:22:12.391 INFO:tasks.workunit.client.0.smithi114.stdout:0/758: creat d1/d1a/d47/d62/fee x:0 0 0 2024-02-16T22:22:12.397 INFO:tasks.workunit.client.0.smithi114.stdout:2/695: dwrite dd/d3b/d8a/fe8 [0,4194304] 0 2024-02-16T22:22:12.398 INFO:tasks.workunit.client.1.smithi123.stdout:7/897: dwrite d3/d5/d6c/da6/fd4 [0,4194304] 0 2024-02-16T22:22:12.398 INFO:tasks.workunit.client.0.smithi114.stdout:2/696: write dd/d6b/da5/fc1 [371206,58883] 0 2024-02-16T22:22:12.402 INFO:tasks.workunit.client.1.smithi123.stdout:2/880: creat d4/d1a/d1d/d59/f11f x:0 0 0 2024-02-16T22:22:12.403 INFO:tasks.workunit.client.1.smithi123.stdout:7/898: creat d3/d6e/d1d/d97/dca/f125 x:0 0 0 2024-02-16T22:22:12.405 INFO:tasks.workunit.client.1.smithi123.stdout:2/881: mkdir d4/d1a/dd5/d120 0 2024-02-16T22:22:12.406 INFO:tasks.workunit.client.1.smithi123.stdout:2/882: readlink d4/d1a/d1d/d59/d45/l9d 0 2024-02-16T22:22:12.407 INFO:tasks.workunit.client.1.smithi123.stdout:7/899: fdatasync d3/d5/d6c/f8d 0 2024-02-16T22:22:12.411 INFO:tasks.workunit.client.1.smithi123.stdout:7/900: unlink d3/d87/f5e 0 2024-02-16T22:22:12.412 INFO:tasks.workunit.client.1.smithi123.stdout:7/901: dread - d3/d6e/d1d/f10c zero size 2024-02-16T22:22:12.415 INFO:tasks.workunit.client.1.smithi123.stdout:7/902: rmdir d3/d87/df5 0 2024-02-16T22:22:12.417 INFO:tasks.workunit.client.1.smithi123.stdout:7/903: creat d3/d6e/d4d/d92/f126 x:0 0 0 2024-02-16T22:22:12.419 INFO:tasks.workunit.client.1.smithi123.stdout:7/904: mknod d3/d6e/d1d/c127 0 2024-02-16T22:22:12.419 INFO:tasks.workunit.client.1.smithi123.stdout:7/905: stat d3/d6e 0 2024-02-16T22:22:12.421 INFO:tasks.workunit.client.1.smithi123.stdout:7/906: read d3/d6e/f14 [252080,337] 0 2024-02-16T22:22:12.422 INFO:tasks.workunit.client.0.smithi114.stdout:0/759: dread d1/d16/f34 [0,4194304] 0 2024-02-16T22:22:12.423 INFO:tasks.workunit.client.0.smithi114.stdout:0/760: write d1/d1a/d3e/fa3 [604721,66477] 0 2024-02-16T22:22:12.423 INFO:tasks.workunit.client.1.smithi123.stdout:7/907: mknod d3/d87/da8/c128 0 2024-02-16T22:22:12.424 INFO:tasks.workunit.client.1.smithi123.stdout:2/883: write d4/d1a/d1d/d13/d25/f77 [1513025,44082] 0 2024-02-16T22:22:12.426 INFO:tasks.workunit.client.1.smithi123.stdout:7/908: creat d3/d6e/d1d/db2/d35/dba/f129 x:0 0 0 2024-02-16T22:22:12.426 INFO:tasks.workunit.client.0.smithi114.stdout:0/761: truncate d1/d1a/f38 183877 0 2024-02-16T22:22:12.427 INFO:tasks.workunit.client.1.smithi123.stdout:7/909: creat d3/d6e/d1d/db2/d35/f12a x:0 0 0 2024-02-16T22:22:12.428 INFO:tasks.workunit.client.0.smithi114.stdout:0/762: creat d1/d16/d48/d98/fef x:0 0 0 2024-02-16T22:22:12.429 INFO:tasks.workunit.client.0.smithi114.stdout:0/763: fdatasync d1/d16/d1f/d28/d51/d61/d6d/d79/f8a 0 2024-02-16T22:22:12.430 INFO:tasks.workunit.client.1.smithi123.stdout:7/910: dread d3/d6e/d1d/db2/d35/f36 [4194304,4194304] 0 2024-02-16T22:22:12.431 INFO:tasks.workunit.client.1.smithi123.stdout:7/911: chown d3/d6e/d1d/db2/d35/ca4 6 1 2024-02-16T22:22:12.431 INFO:tasks.workunit.client.1.smithi123.stdout:7/912: chown d3/d6e/f113 417144025 1 2024-02-16T22:22:12.432 INFO:tasks.workunit.client.0.smithi114.stdout:0/764: rename d1/d16/d55 to d1/d16/dbf/df0 0 2024-02-16T22:22:12.432 INFO:tasks.workunit.client.1.smithi123.stdout:8/922: sync 2024-02-16T22:22:12.432 INFO:tasks.workunit.client.1.smithi123.stdout:0/837: sync 2024-02-16T22:22:12.432 INFO:tasks.workunit.client.1.smithi123.stdout:4/979: sync 2024-02-16T22:22:12.432 INFO:tasks.workunit.client.1.smithi123.stdout:6/897: sync 2024-02-16T22:22:12.432 INFO:tasks.workunit.client.1.smithi123.stdout:5/900: sync 2024-02-16T22:22:12.433 INFO:tasks.workunit.client.0.smithi114.stdout:0/765: write d1/d16/d1f/d28/d51/d61/d7e/d90/f42 [257956,54743] 0 2024-02-16T22:22:12.435 INFO:tasks.workunit.client.1.smithi123.stdout:4/980: unlink d5/f1e 0 2024-02-16T22:22:12.436 INFO:tasks.workunit.client.1.smithi123.stdout:4/981: write d5/d1c/d64/fc6 [147203,112313] 0 2024-02-16T22:22:12.436 INFO:tasks.workunit.client.1.smithi123.stdout:0/838: mkdir d1/d8/d13/d1d/d64/dbc/d114 0 2024-02-16T22:22:12.438 INFO:tasks.workunit.client.0.smithi114.stdout:0/766: rename d1/d16/d1f/d28/d51/d61/d6d/d79 to d1/d16/d1f/d28/d51/d61/d6d/df1 0 2024-02-16T22:22:12.439 INFO:tasks.workunit.client.0.smithi114.stdout:0/767: truncate d1/d78/fd3 4573194 0 2024-02-16T22:22:12.440 INFO:tasks.workunit.client.1.smithi123.stdout:7/913: rename d3/d6e/d1d/d97/dce to d3/d6e/d4d/d92/d12b 0 2024-02-16T22:22:12.441 INFO:tasks.workunit.client.1.smithi123.stdout:6/898: mkdir d0/d6/d2f/da5/dc6/d143 0 2024-02-16T22:22:12.443 INFO:tasks.workunit.client.1.smithi123.stdout:8/923: symlink d0/d29/da7/d43/l132 0 2024-02-16T22:22:12.444 INFO:tasks.workunit.client.0.smithi114.stdout:0/768: dread d1/d16/d1f/d28/d51/fb3 [0,4194304] 0 2024-02-16T22:22:12.445 INFO:tasks.workunit.client.1.smithi123.stdout:0/839: creat d1/d4/d17/d54/db4/f115 x:0 0 0 2024-02-16T22:22:12.448 INFO:tasks.workunit.client.1.smithi123.stdout:8/924: dwrite d0/d29/da7/d26/d2a/d123/dca/f11c [0,4194304] 0 2024-02-16T22:22:12.450 INFO:tasks.workunit.client.1.smithi123.stdout:4/982: mknod d5/db8/d11f/dc0/d118/d9f/de5/c131 0 2024-02-16T22:22:12.451 INFO:tasks.workunit.client.1.smithi123.stdout:0/840: chown d1/d8/da7/lf0 22146 1 2024-02-16T22:22:12.476 INFO:tasks.workunit.client.0.smithi114.stdout:7/786: sync 2024-02-16T22:22:12.481 INFO:tasks.workunit.client.0.smithi114.stdout:7/787: symlink d3/d22/d73/da7/d6f/df8/l115 0 2024-02-16T22:22:12.481 INFO:tasks.workunit.client.1.smithi123.stdout:7/914: write d3/d6e/d1d/fb6 [1349779,73396] 0 2024-02-16T22:22:12.481 INFO:tasks.workunit.client.0.smithi114.stdout:7/788: readlink d3/d22/d73/da7/d6f/d89/da6/l104 0 2024-02-16T22:22:12.482 INFO:tasks.workunit.client.1.smithi123.stdout:0/841: getdents d1/d4/d42 0 2024-02-16T22:22:12.484 INFO:tasks.workunit.client.1.smithi123.stdout:7/915: creat d3/d6e/d4d/d92/d12b/f12c x:0 0 0 2024-02-16T22:22:12.486 INFO:tasks.workunit.client.1.smithi123.stdout:8/925: write d0/fff [980107,77452] 0 2024-02-16T22:22:12.487 INFO:tasks.workunit.client.1.smithi123.stdout:6/899: dwrite d0/d6/d91/d97/fe2 [0,4194304] 0 2024-02-16T22:22:12.487 INFO:tasks.workunit.client.1.smithi123.stdout:0/842: creat d1/d8/d13/d1d/d64/dbc/ddb/f116 x:0 0 0 2024-02-16T22:22:12.488 INFO:tasks.workunit.client.1.smithi123.stdout:7/916: rmdir d3/de0 39 2024-02-16T22:22:12.490 INFO:tasks.workunit.client.1.smithi123.stdout:6/900: creat d0/d8a/de6/f144 x:0 0 0 2024-02-16T22:22:12.493 INFO:tasks.workunit.client.1.smithi123.stdout:8/926: mknod d0/d29/da7/d26/d32/de0/c133 0 2024-02-16T22:22:12.494 INFO:tasks.workunit.client.1.smithi123.stdout:4/983: dwrite d5/db8/d11f/dc0/d118/fe7 [0,4194304] 0 2024-02-16T22:22:12.495 INFO:tasks.workunit.client.1.smithi123.stdout:8/927: write d0/fff [1220292,116115] 0 2024-02-16T22:22:12.499 INFO:tasks.workunit.client.1.smithi123.stdout:6/901: creat d0/d6/d1a/d2e/f145 x:0 0 0 2024-02-16T22:22:12.501 INFO:tasks.workunit.client.0.smithi114.stdout:7/789: dread d3/d46/f47 [0,4194304] 0 2024-02-16T22:22:12.503 INFO:tasks.workunit.client.1.smithi123.stdout:4/984: symlink d5/d1c/d23/l132 0 2024-02-16T22:22:12.503 INFO:tasks.workunit.client.1.smithi123.stdout:6/902: stat d0/d6/d8e/fc0 0 2024-02-16T22:22:12.505 INFO:tasks.workunit.client.0.smithi114.stdout:7/790: mknod d3/d22/d73/da7/d6f/d89/c116 0 2024-02-16T22:22:12.508 INFO:tasks.workunit.client.1.smithi123.stdout:4/985: truncate d5/d1c/d5e/dc3/f4a 1939948 0 2024-02-16T22:22:12.512 INFO:tasks.workunit.client.0.smithi114.stdout:7/791: dwrite d3/d22/d73/da7/d6f/d9c/fed [0,4194304] 0 2024-02-16T22:22:12.513 INFO:tasks.workunit.client.0.smithi114.stdout:7/792: write d3/d14/f5c [4026485,116905] 0 2024-02-16T22:22:12.515 INFO:tasks.workunit.client.1.smithi123.stdout:7/917: dwrite d3/d5/d6c/f8d [0,4194304] 0 2024-02-16T22:22:12.515 INFO:tasks.workunit.client.1.smithi123.stdout:0/843: dwrite d1/d8/d13/d1d/d62/fcc [0,4194304] 0 2024-02-16T22:22:12.516 INFO:tasks.workunit.client.1.smithi123.stdout:8/928: truncate d0/d29/da7/d26/d2a/d77/fde 633969 0 2024-02-16T22:22:12.517 INFO:tasks.workunit.client.1.smithi123.stdout:6/903: dwrite d0/d6/d1a/d2e/f7f [0,4194304] 0 2024-02-16T22:22:12.520 INFO:tasks.workunit.client.1.smithi123.stdout:4/986: rmdir d5/df4/d101 0 2024-02-16T22:22:12.522 INFO:tasks.workunit.client.1.smithi123.stdout:0/844: unlink d1/d4/d42/ld5 0 2024-02-16T22:22:12.525 INFO:tasks.workunit.client.1.smithi123.stdout:8/929: symlink d0/d12/d3e/d7c/l134 0 2024-02-16T22:22:12.526 INFO:tasks.workunit.client.1.smithi123.stdout:7/918: dwrite d3/d6e/f113 [0,4194304] 0 2024-02-16T22:22:12.528 INFO:tasks.workunit.client.1.smithi123.stdout:7/919: readlink d3/d6e/d1d/d97/lde 0 2024-02-16T22:22:12.530 INFO:tasks.workunit.client.1.smithi123.stdout:4/987: readlink d5/d1c/l53 0 2024-02-16T22:22:12.533 INFO:tasks.workunit.client.1.smithi123.stdout:3/913: sync 2024-02-16T22:22:12.536 INFO:tasks.workunit.client.1.smithi123.stdout:6/904: unlink d0/d6/d1a/daf/c10e 0 2024-02-16T22:22:12.537 INFO:tasks.workunit.client.1.smithi123.stdout:4/988: dread d5/d1c/d23/f52 [0,4194304] 0 2024-02-16T22:22:12.539 INFO:tasks.workunit.client.1.smithi123.stdout:4/989: write d5/db8/d11f/dc0/d118/d73/d87/fe4 [854659,66734] 0 2024-02-16T22:22:12.539 INFO:tasks.workunit.client.1.smithi123.stdout:8/930: mknod d0/d29/da7/d26/d32/de0/c135 0 2024-02-16T22:22:12.541 INFO:tasks.workunit.client.1.smithi123.stdout:8/931: chown d0/d29/da7/d26/f7a 16 1 2024-02-16T22:22:12.545 INFO:tasks.workunit.client.1.smithi123.stdout:4/990: creat d5/d1c/d23/d47/da2/f133 x:0 0 0 2024-02-16T22:22:12.545 INFO:tasks.workunit.client.0.smithi114.stdout:7/793: dread d3/d22/d61/d6a/f7a [0,4194304] 0 2024-02-16T22:22:12.547 INFO:tasks.workunit.client.1.smithi123.stdout:0/845: rename d1/d4/d17/d74/f107 to d1/d8/d13/d1d/f117 0 2024-02-16T22:22:12.549 INFO:tasks.workunit.client.1.smithi123.stdout:4/991: dread - d5/db8/d11f/dc0/d118/d9f/feb zero size 2024-02-16T22:22:12.551 INFO:tasks.workunit.client.1.smithi123.stdout:0/846: truncate d1/f6a 2590927 0 2024-02-16T22:22:12.553 INFO:tasks.workunit.client.1.smithi123.stdout:0/847: rmdir d1/d4/d17/d54/db4 39 2024-02-16T22:22:12.554 INFO:tasks.workunit.client.1.smithi123.stdout:0/848: stat d1/d4/d17/f41 0 2024-02-16T22:22:12.556 INFO:tasks.workunit.client.1.smithi123.stdout:0/849: creat d1/d4/d17/f118 x:0 0 0 2024-02-16T22:22:12.557 INFO:tasks.workunit.client.1.smithi123.stdout:0/850: truncate d1/d8/d13/ffd 87777 0 2024-02-16T22:22:12.562 INFO:tasks.workunit.client.1.smithi123.stdout:4/992: dread d5/d1c/d23/f43 [0,4194304] 0 2024-02-16T22:22:12.562 INFO:tasks.workunit.client.0.smithi114.stdout:8/689: sync 2024-02-16T22:22:12.568 INFO:tasks.workunit.client.1.smithi123.stdout:4/993: unlink d5/d1c/d5e/c65 0 2024-02-16T22:22:12.573 INFO:tasks.workunit.client.1.smithi123.stdout:4/994: rename d5/d1c/d64/d7b/c82 to d5/db8/d11f/c134 0 2024-02-16T22:22:12.573 INFO:tasks.workunit.client.1.smithi123.stdout:8/932: dread d0/d29/f60 [0,4194304] 0 2024-02-16T22:22:12.574 INFO:tasks.workunit.client.1.smithi123.stdout:4/995: write d5/d1c/d23/d31/f3e [1616768,111733] 0 2024-02-16T22:22:12.575 INFO:tasks.workunit.client.0.smithi114.stdout:7/794: mknod d3/d22/d73/da7/d6f/d89/da6/dfc/c117 0 2024-02-16T22:22:12.576 INFO:tasks.workunit.client.0.smithi114.stdout:8/690: fdatasync d4/d11/d78/d7a/d31/f4d 0 2024-02-16T22:22:12.580 INFO:tasks.workunit.client.1.smithi123.stdout:8/933: dwrite d0/d29/da7/d26/d32/de0/f106 [0,4194304] 0 2024-02-16T22:22:12.581 INFO:tasks.workunit.client.0.smithi114.stdout:7/795: read d3/d22/d73/da7/d4e/fc8 [347690,34696] 0 2024-02-16T22:22:12.582 INFO:tasks.workunit.client.1.smithi123.stdout:8/934: stat d0/d29/da7/d26/d2a/d123/f102 0 2024-02-16T22:22:12.583 INFO:tasks.workunit.client.1.smithi123.stdout:7/920: dwrite d3/d6e/d1d/f2f [0,4194304] 0 2024-02-16T22:22:12.588 INFO:tasks.workunit.client.0.smithi114.stdout:8/691: symlink d4/d11/lf7 0 2024-02-16T22:22:12.589 INFO:tasks.workunit.client.0.smithi114.stdout:7/796: mknod d3/d46/da2/d114/c118 0 2024-02-16T22:22:12.590 INFO:tasks.workunit.client.0.smithi114.stdout:7/797: chown d3/d22/d61/d6a/f94 996 1 2024-02-16T22:22:12.590 INFO:tasks.workunit.client.0.smithi114.stdout:7/798: chown d3/d92 580636090 1 2024-02-16T22:22:12.591 INFO:tasks.workunit.client.0.smithi114.stdout:7/799: dread - d3/d3d/ff3 zero size 2024-02-16T22:22:12.594 INFO:tasks.workunit.client.1.smithi123.stdout:7/921: rename d3/d5/d6c/c9d to d3/d6e/d52/d106/c12d 0 2024-02-16T22:22:12.596 INFO:tasks.workunit.client.0.smithi114.stdout:8/692: rename d4/d11/d78/d7a/l12 to d4/d16/d25/d46/lf8 0 2024-02-16T22:22:12.597 INFO:tasks.workunit.client.0.smithi114.stdout:8/693: write d4/d24/f38 [5554241,8898] 0 2024-02-16T22:22:12.603 INFO:tasks.workunit.client.1.smithi123.stdout:4/996: dread d5/d1c/d23/d31/f66 [0,4194304] 0 2024-02-16T22:22:12.603 INFO:tasks.workunit.client.1.smithi123.stdout:7/922: creat d3/d87/f12e x:0 0 0 2024-02-16T22:22:12.604 INFO:tasks.workunit.client.1.smithi123.stdout:7/923: chown d3/d59/fef 1801 1 2024-02-16T22:22:12.604 INFO:tasks.workunit.client.1.smithi123.stdout:2/884: sync 2024-02-16T22:22:12.604 INFO:tasks.workunit.client.1.smithi123.stdout:1/879: sync 2024-02-16T22:22:12.604 INFO:tasks.workunit.client.1.smithi123.stdout:5/901: sync 2024-02-16T22:22:12.605 INFO:tasks.workunit.client.1.smithi123.stdout:1/880: stat d3/d6/db/d10/d2a 0 2024-02-16T22:22:12.607 INFO:tasks.workunit.client.1.smithi123.stdout:5/902: chown d6/dd/d44/dba/ccd 18242989 1 2024-02-16T22:22:12.616 INFO:tasks.workunit.client.0.smithi114.stdout:8/694: dread d4/d16/d25/f54 [4194304,4194304] 0 2024-02-16T22:22:12.617 INFO:tasks.workunit.client.1.smithi123.stdout:5/903: chown d6/dd/d44/f8b 52324 1 2024-02-16T22:22:12.619 INFO:tasks.workunit.client.0.smithi114.stdout:8/695: unlink d4/d16/d25/l95 0 2024-02-16T22:22:12.619 INFO:tasks.workunit.client.1.smithi123.stdout:7/924: dwrite d3/d6e/d24/f90 [4194304,4194304] 0 2024-02-16T22:22:12.621 INFO:tasks.workunit.client.0.smithi114.stdout:8/696: creat d4/d24/ff9 x:0 0 0 2024-02-16T22:22:12.623 INFO:tasks.workunit.client.1.smithi123.stdout:0/851: dwrite d1/d4/d17/d54/f99 [0,4194304] 0 2024-02-16T22:22:12.630 INFO:tasks.workunit.client.0.smithi114.stdout:8/697: dwrite d4/d11/d9d/fdd [4194304,4194304] 0 2024-02-16T22:22:12.631 INFO:tasks.workunit.client.1.smithi123.stdout:7/925: dwrite d3/d6e/d1d/db2/d35/db7/f119 [0,4194304] 0 2024-02-16T22:22:12.633 INFO:tasks.workunit.client.1.smithi123.stdout:8/935: dwrite d0/d29/da7/d43/f28 [4194304,4194304] 0 2024-02-16T22:22:12.636 INFO:tasks.workunit.client.1.smithi123.stdout:0/852: dwrite d1/d4/d17/d54/f99 [0,4194304] 0 2024-02-16T22:22:12.636 INFO:tasks.workunit.client.1.smithi123.stdout:8/936: write d0/d29/da7/d26/d68/d81/d47/d9b/f128 [974908,6068] 0 2024-02-16T22:22:12.638 INFO:tasks.workunit.client.0.smithi114.stdout:7/800: dwrite d3/d22/f48 [4194304,4194304] 0 2024-02-16T22:22:12.640 INFO:tasks.workunit.client.0.smithi114.stdout:7/801: stat d3/d22/d73/da7/d6f/d89/da6/dfc/f109 0 2024-02-16T22:22:12.644 INFO:tasks.workunit.client.1.smithi123.stdout:2/885: write d4/d1a/d1d/f8f [721903,95024] 0 2024-02-16T22:22:12.647 INFO:tasks.workunit.client.1.smithi123.stdout:0/853: read d1/d4/d17/d54/d8e/d96/ffe [1402873,29966] 0 2024-02-16T22:22:12.647 INFO:tasks.workunit.client.1.smithi123.stdout:7/926: mkdir d3/d59/d12f 0 2024-02-16T22:22:12.647 INFO:tasks.workunit.client.0.smithi114.stdout:7/802: creat d3/d5a/f119 x:0 0 0 2024-02-16T22:22:12.650 INFO:tasks.workunit.client.1.smithi123.stdout:1/881: dwrite d3/d47/fed [0,4194304] 0 2024-02-16T22:22:12.650 INFO:tasks.workunit.client.0.smithi114.stdout:8/698: mknod d4/d16/d25/d46/db5/cfa 0 2024-02-16T22:22:12.653 INFO:tasks.workunit.client.1.smithi123.stdout:2/886: truncate d4/d1a/d1d/d13/d25/d72/dc8/f108 1036711 0 2024-02-16T22:22:12.654 INFO:tasks.workunit.client.0.smithi114.stdout:7/803: symlink d3/d46/da2/l11a 0 2024-02-16T22:22:12.657 INFO:tasks.workunit.client.1.smithi123.stdout:0/854: creat d1/da0/f119 x:0 0 0 2024-02-16T22:22:12.657 INFO:tasks.workunit.client.1.smithi123.stdout:2/887: truncate d4/d1a/d1d/d13/d25/d72/dc8/f108 1524994 0 2024-02-16T22:22:12.659 INFO:tasks.workunit.client.1.smithi123.stdout:7/927: truncate d3/d6e/d1d/db2/d3d/f10e 750668 0 2024-02-16T22:22:12.659 INFO:tasks.workunit.client.1.smithi123.stdout:0/855: write d1/d4/d17/d54/d8e/d96/f9e [174926,12101] 0 2024-02-16T22:22:12.661 INFO:tasks.workunit.client.1.smithi123.stdout:8/937: mkdir d0/d61/d66/d136 0 2024-02-16T22:22:12.662 INFO:tasks.workunit.client.0.smithi114.stdout:7/804: dwrite d3/d87/fe4 [0,4194304] 0 2024-02-16T22:22:12.668 INFO:tasks.workunit.client.0.smithi114.stdout:7/805: mknod d3/d92/dd5/c11b 0 2024-02-16T22:22:12.668 INFO:tasks.workunit.client.1.smithi123.stdout:2/888: dread d4/d1a/dd5/d10b/df1/d4a/da2/fc9 [0,4194304] 0 2024-02-16T22:22:12.670 INFO:tasks.workunit.client.1.smithi123.stdout:7/928: mknod d3/d6e/d1d/c130 0 2024-02-16T22:22:12.671 INFO:tasks.workunit.client.1.smithi123.stdout:8/938: symlink d0/d29/l137 0 2024-02-16T22:22:12.673 INFO:tasks.workunit.client.1.smithi123.stdout:2/889: creat d4/d1a/d1d/d38/d5e/de8/f121 x:0 0 0 2024-02-16T22:22:12.674 INFO:tasks.workunit.client.1.smithi123.stdout:1/882: getdents d3/d6/db/d10/d4b/d70/d11e 0 2024-02-16T22:22:12.675 INFO:tasks.workunit.client.1.smithi123.stdout:7/929: mknod d3/de0/c131 0 2024-02-16T22:22:12.675 INFO:tasks.workunit.client.1.smithi123.stdout:2/890: fdatasync d4/d1a/d1d/d59/dd7/ddc/ff9 0 2024-02-16T22:22:12.677 INFO:tasks.workunit.client.1.smithi123.stdout:8/939: link d0/d29/da7/d26/d2a/d4d/fb4 d0/d61/d66/d7e/f138 0 2024-02-16T22:22:12.678 INFO:tasks.workunit.client.1.smithi123.stdout:1/883: unlink d3/d6/db/d10/d2d/l104 0 2024-02-16T22:22:12.679 INFO:tasks.workunit.client.1.smithi123.stdout:7/930: mkdir d3/d6e/d1d/d132 0 2024-02-16T22:22:12.680 INFO:tasks.workunit.client.1.smithi123.stdout:2/891: creat d4/d1a/dd5/d120/f122 x:0 0 0 2024-02-16T22:22:12.680 INFO:tasks.workunit.client.1.smithi123.stdout:1/884: mknod d3/d6/db/d10/d2d/d8c/dfd/c12b 0 2024-02-16T22:22:12.680 INFO:tasks.workunit.client.1.smithi123.stdout:7/931: write d3/d6e/d1d/db2/d35/f107 [4690893,113292] 0 2024-02-16T22:22:12.683 INFO:tasks.workunit.client.1.smithi123.stdout:2/892: symlink d4/d1a/dd5/d10b/df1/db1/l123 0 2024-02-16T22:22:12.703 INFO:tasks.workunit.client.0.smithi114.stdout:6/715: sync 2024-02-16T22:22:12.703 INFO:tasks.workunit.client.0.smithi114.stdout:3/707: sync 2024-02-16T22:22:12.704 INFO:tasks.workunit.client.0.smithi114.stdout:3/708: chown d4/d87/lc5 9 1 2024-02-16T22:22:12.709 INFO:tasks.workunit.client.0.smithi114.stdout:6/716: read d6/d83/f87 [1918599,29911] 0 2024-02-16T22:22:12.713 INFO:tasks.workunit.client.0.smithi114.stdout:6/717: creat d6/d2c/d75/fd4 x:0 0 0 2024-02-16T22:22:12.716 INFO:tasks.workunit.client.1.smithi123.stdout:3/914: sync 2024-02-16T22:22:12.718 INFO:tasks.workunit.client.0.smithi114.stdout:6/718: rename d6/d2c/d38/d41/d60/f65 to d6/dd/d25/d34/d61/fd5 0 2024-02-16T22:22:12.719 INFO:tasks.workunit.client.0.smithi114.stdout:6/719: stat d6/d2c/d38/da8/lab 0 2024-02-16T22:22:12.720 INFO:tasks.workunit.client.0.smithi114.stdout:8/699: truncate d4/d11/f71 9305387 0 2024-02-16T22:22:12.721 INFO:tasks.workunit.client.0.smithi114.stdout:8/700: readlink d4/d11/d78/d7a/d36/d3b/l4e 0 2024-02-16T22:22:12.721 INFO:tasks.workunit.client.1.smithi123.stdout:7/932: rename d3/d6e/d52/fb4 to d3/d6e/d1d/db2/d3e/de4/d103/f133 0 2024-02-16T22:22:12.723 INFO:tasks.workunit.client.1.smithi123.stdout:3/915: readlink d0/d20/d2e/dc4/le2 0 2024-02-16T22:22:12.726 INFO:tasks.workunit.client.1.smithi123.stdout:5/904: dwrite d6/dd/df/d58/f95 [0,4194304] 0 2024-02-16T22:22:12.728 INFO:tasks.workunit.client.0.smithi114.stdout:6/720: mkdir d6/d2c/d38/dd6 0 2024-02-16T22:22:12.731 INFO:tasks.workunit.client.0.smithi114.stdout:9/723: sync 2024-02-16T22:22:12.733 INFO:tasks.workunit.client.0.smithi114.stdout:6/721: symlink d6/dd/d25/d34/d58/d9f/da3/ld7 0 2024-02-16T22:22:12.736 INFO:tasks.workunit.client.0.smithi114.stdout:7/806: dread d3/d22/d73/da7/d6f/df8/ff9 [0,4194304] 0 2024-02-16T22:22:12.736 INFO:tasks.workunit.client.0.smithi114.stdout:8/701: dread d4/d11/d52/f83 [0,4194304] 0 2024-02-16T22:22:12.737 INFO:tasks.workunit.client.0.smithi114.stdout:8/702: fsync d4/d24/ff9 0 2024-02-16T22:22:12.742 INFO:tasks.workunit.client.0.smithi114.stdout:7/807: creat d3/d92/dbe/de6/f11c x:0 0 0 2024-02-16T22:22:12.744 INFO:tasks.workunit.client.1.smithi123.stdout:1/885: dwrite d3/d6/d49/d48/d84/d92/f10b [0,4194304] 0 2024-02-16T22:22:12.745 INFO:tasks.workunit.client.0.smithi114.stdout:8/703: dwrite d4/d16/fe5 [0,4194304] 0 2024-02-16T22:22:12.753 INFO:tasks.workunit.client.0.smithi114.stdout:8/704: symlink d4/d11/d78/d7a/d7e/lfb 0 2024-02-16T22:22:12.753 INFO:tasks.workunit.client.0.smithi114.stdout:6/722: dread d6/dd/d25/d34/f42 [0,4194304] 0 2024-02-16T22:22:12.754 INFO:tasks.workunit.client.0.smithi114.stdout:3/709: write d4/d24/f67 [5374194,80805] 0 2024-02-16T22:22:12.765 INFO:tasks.workunit.client.0.smithi114.stdout:7/808: link d3/d3d/le8 d3/d46/da2/d114/l11d 0 2024-02-16T22:22:12.767 INFO:tasks.workunit.client.0.smithi114.stdout:6/723: dwrite d6/dd/d25/d34/f6f [0,4194304] 0 2024-02-16T22:22:12.771 INFO:tasks.workunit.client.0.smithi114.stdout:0/769: sync 2024-02-16T22:22:12.771 INFO:tasks.workunit.client.0.smithi114.stdout:1/694: sync 2024-02-16T22:22:12.771 INFO:tasks.workunit.client.0.smithi114.stdout:5/689: sync 2024-02-16T22:22:12.771 INFO:tasks.workunit.client.0.smithi114.stdout:2/697: sync 2024-02-16T22:22:12.771 INFO:tasks.workunit.client.0.smithi114.stdout:4/825: sync 2024-02-16T22:22:12.774 INFO:tasks.workunit.client.0.smithi114.stdout:1/695: readlink d4/d9/d46/l66 0 2024-02-16T22:22:12.774 INFO:tasks.workunit.client.0.smithi114.stdout:3/710: creat d4/d24/d2d/df0/ff1 x:0 0 0 2024-02-16T22:22:12.775 INFO:tasks.workunit.client.0.smithi114.stdout:9/724: write d2/d1a/d23/f56 [514583,49986] 0 2024-02-16T22:22:12.776 INFO:tasks.workunit.client.0.smithi114.stdout:7/809: rename d3/d92/dbe/de6/de9 to d3/d92/dbe/de6/d11e 0 2024-02-16T22:22:12.778 INFO:tasks.workunit.client.0.smithi114.stdout:9/725: write d2/d5c/dc9/d47/db2/fc6 [1214188,82951] 0 2024-02-16T22:22:12.778 INFO:tasks.workunit.client.0.smithi114.stdout:0/770: truncate d1/d16/d1f/d28/d51/d60/d52/fc7 308707 0 2024-02-16T22:22:12.781 INFO:tasks.workunit.client.0.smithi114.stdout:2/698: truncate dd/df/d1c/d21/d61/d69/fce 735790 0 2024-02-16T22:22:12.781 INFO:tasks.workunit.client.0.smithi114.stdout:6/724: unlink d6/d2c/d5e/l72 0 2024-02-16T22:22:12.782 INFO:tasks.workunit.client.0.smithi114.stdout:7/810: fdatasync d3/d22/d73/f100 0 2024-02-16T22:22:12.782 INFO:tasks.workunit.client.0.smithi114.stdout:8/705: link d4/d11/l2e d4/d11/d78/d7a/d31/dc6/dce/lfc 0 2024-02-16T22:22:12.783 INFO:tasks.workunit.client.0.smithi114.stdout:6/725: dread - d6/dd/fbe zero size 2024-02-16T22:22:12.785 INFO:tasks.workunit.client.0.smithi114.stdout:8/706: truncate d4/d24/ff9 116732 0 2024-02-16T22:22:12.786 INFO:tasks.workunit.client.0.smithi114.stdout:5/690: creat d0/d2d/fdf x:0 0 0 2024-02-16T22:22:12.786 INFO:tasks.workunit.client.1.smithi123.stdout:6/905: sync 2024-02-16T22:22:12.787 INFO:tasks.workunit.client.0.smithi114.stdout:3/711: dwrite d4/d24/d2d/df0/d77/fab [0,4194304] 0 2024-02-16T22:22:12.792 INFO:tasks.workunit.client.0.smithi114.stdout:1/696: creat d4/d9/d46/d67/ffd x:0 0 0 2024-02-16T22:22:12.793 INFO:tasks.workunit.client.0.smithi114.stdout:1/697: chown d4/d9/d3d/d3f/cfb 1 1 2024-02-16T22:22:12.794 INFO:tasks.workunit.client.0.smithi114.stdout:1/698: write d4/d63/d69/f7f [5390156,91236] 0 2024-02-16T22:22:12.801 INFO:tasks.workunit.client.0.smithi114.stdout:2/699: creat dd/df/d1c/d21/d87/fef x:0 0 0 2024-02-16T22:22:12.801 INFO:tasks.workunit.client.0.smithi114.stdout:2/700: readlink dd/d3b/l82 0 2024-02-16T22:22:12.803 INFO:tasks.workunit.client.0.smithi114.stdout:7/811: creat d3/d22/d73/da7/d4e/f11f x:0 0 0 2024-02-16T22:22:12.804 INFO:tasks.workunit.client.1.smithi123.stdout:2/893: mknod d4/d1a/d1d/d13/d15/c124 0 2024-02-16T22:22:12.808 INFO:tasks.workunit.client.1.smithi123.stdout:7/933: creat d3/d6e/d4d/d92/f134 x:0 0 0 2024-02-16T22:22:12.809 INFO:tasks.workunit.client.0.smithi114.stdout:3/712: symlink d4/dd/d33/lf2 0 2024-02-16T22:22:12.811 INFO:tasks.workunit.client.1.smithi123.stdout:7/934: dread d3/d6e/d1d/db2/d35/db7/f119 [0,4194304] 0 2024-02-16T22:22:12.812 INFO:tasks.workunit.client.0.smithi114.stdout:6/726: dread d6/dd/d25/f73 [0,4194304] 0 2024-02-16T22:22:12.813 INFO:tasks.workunit.client.0.smithi114.stdout:2/701: dread - dd/d3b/da7/fb3 zero size 2024-02-16T22:22:12.814 INFO:tasks.workunit.client.0.smithi114.stdout:2/702: read - dd/d4e/fe0 zero size 2024-02-16T22:22:12.814 INFO:tasks.workunit.client.1.smithi123.stdout:8/940: sync 2024-02-16T22:22:12.814 INFO:tasks.workunit.client.1.smithi123.stdout:4/997: sync 2024-02-16T22:22:12.814 INFO:tasks.workunit.client.1.smithi123.stdout:0/856: sync 2024-02-16T22:22:12.815 INFO:tasks.workunit.client.0.smithi114.stdout:2/703: chown dd/fd1 73893 1 2024-02-16T22:22:12.820 INFO:tasks.workunit.client.0.smithi114.stdout:3/713: fdatasync d4/d46/d48/d75/f9a 0 2024-02-16T22:22:12.823 INFO:tasks.workunit.client.0.smithi114.stdout:9/726: rename d2/d1a/d23/f56 to d2/d1a/d65/fe8 0 2024-02-16T22:22:12.824 INFO:tasks.workunit.client.1.smithi123.stdout:5/905: mkdir d6/dd/d13d 0 2024-02-16T22:22:12.826 INFO:tasks.workunit.client.0.smithi114.stdout:4/826: dread dd/d2e/f49 [0,4194304] 0 2024-02-16T22:22:12.831 INFO:tasks.workunit.client.1.smithi123.stdout:0/857: mkdir d1/d8/d13/d1d/d62/d7a/d11a 0 2024-02-16T22:22:12.833 INFO:tasks.workunit.client.0.smithi114.stdout:3/714: unlink d4/d46/d48/f5c 0 2024-02-16T22:22:12.838 INFO:tasks.workunit.client.0.smithi114.stdout:7/812: rename d3/d22/d73/da7/d4e/f6e to d3/d22/d73/da7/d6f/d89/da6/dfc/f120 0 2024-02-16T22:22:12.839 INFO:tasks.workunit.client.1.smithi123.stdout:2/894: mkdir d4/d1a/dd5/d10b/df1/d125 0 2024-02-16T22:22:12.840 INFO:tasks.workunit.client.0.smithi114.stdout:0/771: write d1/d16/d1f/d28/d51/d61/d7e/d90/fac [191941,89548] 0 2024-02-16T22:22:12.841 INFO:tasks.workunit.client.0.smithi114.stdout:9/727: symlink d2/d1a/d23/d32/d46/d78/le9 0 2024-02-16T22:22:12.841 INFO:tasks.workunit.client.1.smithi123.stdout:7/935: dread d3/d6e/d24/f6d [0,4194304] 0 2024-02-16T22:22:12.843 INFO:tasks.workunit.client.1.smithi123.stdout:3/916: link d0/d20/d2e/fe3 d0/d20/d29/df6/f12d 0 2024-02-16T22:22:12.847 INFO:tasks.workunit.client.0.smithi114.stdout:2/704: mkdir dd/df/d1c/df0 0 2024-02-16T22:22:12.849 INFO:tasks.workunit.client.1.smithi123.stdout:1/886: rmdir d3/d52/d108 0 2024-02-16T22:22:12.850 INFO:tasks.workunit.client.0.smithi114.stdout:2/705: fdatasync dd/df/d1c/fa4 0 2024-02-16T22:22:12.852 INFO:tasks.workunit.client.0.smithi114.stdout:5/691: dwrite d0/d28/d98/d25/d89/f9d [0,4194304] 0 2024-02-16T22:22:12.857 INFO:tasks.workunit.client.1.smithi123.stdout:4/998: creat d5/db8/d11f/dc0/d118/d73/d11d/d12f/f135 x:0 0 0 2024-02-16T22:22:12.858 INFO:tasks.workunit.client.0.smithi114.stdout:2/706: dread dd/f3a [0,4194304] 0 2024-02-16T22:22:12.859 INFO:tasks.workunit.client.0.smithi114.stdout:0/772: mkdir d1/d1a/d47/d62/d7f/d9d/df2 0 2024-02-16T22:22:12.859 INFO:tasks.workunit.client.1.smithi123.stdout:6/906: dwrite d0/d6/d2f/d5d/fa3 [0,4194304] 0 2024-02-16T22:22:12.860 INFO:tasks.workunit.client.1.smithi123.stdout:3/917: dread d0/d20/d29/d9b/dbd/df4/f35 [0,4194304] 0 2024-02-16T22:22:12.860 INFO:tasks.workunit.client.1.smithi123.stdout:2/895: rmdir d4/d1a/d1d/d13/d25/d98/dbb/d10d 39 2024-02-16T22:22:12.860 INFO:tasks.workunit.client.1.smithi123.stdout:6/907: stat d0/d9/dc/f23 0 2024-02-16T22:22:12.862 INFO:tasks.workunit.client.1.smithi123.stdout:3/918: write d0/d20/d94/dac/fb1 [4393701,32668] 0 2024-02-16T22:22:12.863 INFO:tasks.workunit.client.0.smithi114.stdout:8/707: dwrite d4/d24/d2c/d67/fa4 [0,4194304] 0 2024-02-16T22:22:12.864 INFO:tasks.workunit.client.1.smithi123.stdout:7/936: mkdir d3/de0/d135 0 2024-02-16T22:22:12.868 INFO:tasks.workunit.client.0.smithi114.stdout:4/827: creat dd/d2e/de7/d11f/f12c x:0 0 0 2024-02-16T22:22:12.869 INFO:tasks.workunit.client.1.smithi123.stdout:5/906: creat d6/dd/df/d90/d137/f13e x:0 0 0 2024-02-16T22:22:12.873 INFO:tasks.workunit.client.1.smithi123.stdout:8/941: creat d0/d61/d66/f139 x:0 0 0 2024-02-16T22:22:12.875 INFO:tasks.workunit.client.1.smithi123.stdout:0/858: truncate d1/d4/d17/d54/db4/dc5/fe0 517811 0 2024-02-16T22:22:12.879 INFO:tasks.workunit.client.0.smithi114.stdout:7/813: link d3/d92/dbe/de6/f11c d3/d22/d73/da7/d6f/d89/f121 0 2024-02-16T22:22:12.880 INFO:tasks.workunit.client.0.smithi114.stdout:6/727: write d6/dd/f29 [2432366,52377] 0 2024-02-16T22:22:12.880 INFO:tasks.workunit.client.1.smithi123.stdout:6/908: read - d0/d6/d4d/fff zero size 2024-02-16T22:22:12.881 INFO:tasks.workunit.client.0.smithi114.stdout:5/692: fdatasync d0/d28/d98/d25/d3e/d7f/f8e 0 2024-02-16T22:22:12.882 INFO:tasks.workunit.client.0.smithi114.stdout:5/693: fsync d0/d28/d98/d25/f73 0 2024-02-16T22:22:12.884 INFO:tasks.workunit.client.0.smithi114.stdout:5/694: dread - d0/d28/d98/d1e/d63/d96/d9a/fd6 zero size 2024-02-16T22:22:12.884 INFO:tasks.workunit.client.0.smithi114.stdout:2/707: mknod dd/df/d1c/d21/d61/d69/cf1 0 2024-02-16T22:22:12.885 INFO:tasks.workunit.client.0.smithi114.stdout:5/695: chown d0/d28/d98/d1e/d86/f92 14 1 2024-02-16T22:22:12.887 INFO:tasks.workunit.client.0.smithi114.stdout:0/773: mkdir d1/d16/d1f/d28/d51/d61/df3 0 2024-02-16T22:22:12.890 INFO:tasks.workunit.client.1.smithi123.stdout:7/937: mkdir d3/d6e/d34/d136 0 2024-02-16T22:22:12.891 INFO:tasks.workunit.client.1.smithi123.stdout:7/938: chown d3/d6e/d1d 1533836 1 2024-02-16T22:22:12.893 INFO:tasks.workunit.client.1.smithi123.stdout:5/907: stat d6/dd/df/d1f/ddc/ff1 0 2024-02-16T22:22:12.897 INFO:tasks.workunit.client.1.smithi123.stdout:1/887: dread d3/d6/db/d10/d2d/f5a [0,4194304] 0 2024-02-16T22:22:12.899 INFO:tasks.workunit.client.1.smithi123.stdout:4/999: truncate d5/d1c/d23/d47/f50 2284489 0 2024-02-16T22:22:12.899 INFO:tasks.workunit.client.1.smithi123.stdout:8/942: dread d0/d29/da7/d26/fa4 [0,4194304] 0 2024-02-16T22:22:12.901 INFO:tasks.workunit.client.1.smithi123.stdout:6/909: creat d0/d6/d8e/f146 x:0 0 0 2024-02-16T22:22:12.901 INFO:tasks.workunit.client.1.smithi123.stdout:6/910: chown d0/d6/d2f/d5d/fa3 200777579 1 2024-02-16T22:22:12.904 INFO:tasks.workunit.client.1.smithi123.stdout:2/896: write d4/d1a/d1d/d13/f36 [2417072,22807] 0 2024-02-16T22:22:12.905 INFO:tasks.workunit.client.1.smithi123.stdout:2/897: stat d4/d1a/d1d/d13/d25/d98/dbb 0 2024-02-16T22:22:12.909 INFO:tasks.workunit.client.1.smithi123.stdout:1/888: fsync d3/d6/db/d10/d4b/d70/d87/fb3 0 2024-02-16T22:22:12.910 INFO:tasks.workunit.client.1.smithi123.stdout:0/859: write d1/d8/f6b [3316907,10039] 0 2024-02-16T22:22:12.911 INFO:tasks.workunit.client.1.smithi123.stdout:8/943: unlink d0/d29/l137 0 2024-02-16T22:22:12.914 INFO:tasks.workunit.client.1.smithi123.stdout:3/919: creat d0/d20/d29/f12e x:0 0 0 2024-02-16T22:22:12.914 INFO:tasks.workunit.client.1.smithi123.stdout:3/920: chown d0/d20/d5a/da6/cf7 48454 1 2024-02-16T22:22:12.917 INFO:tasks.workunit.client.1.smithi123.stdout:5/908: rename d6/l14 to d6/dd/d13d/l13f 0 2024-02-16T22:22:12.917 INFO:tasks.workunit.client.1.smithi123.stdout:2/898: rename d4/d1a/d1d/d59 to d4/d1a/d1d/d59/dd7/d10a/d126 22 2024-02-16T22:22:12.921 INFO:tasks.workunit.client.1.smithi123.stdout:8/944: rmdir d0/d29/da7/d125 39 2024-02-16T22:22:12.924 INFO:tasks.workunit.client.1.smithi123.stdout:5/909: chown d6/dd/df/d58/f70 11100521 1 2024-02-16T22:22:12.925 INFO:tasks.workunit.client.1.smithi123.stdout:2/899: mkdir d4/d1a/d1d/d13/d25/d98/d127 0 2024-02-16T22:22:12.926 INFO:tasks.workunit.client.1.smithi123.stdout:1/889: write d3/d6/d49/d48/d84/f93 [982753,104310] 0 2024-02-16T22:22:12.927 INFO:tasks.workunit.client.1.smithi123.stdout:7/939: dwrite d3/d6e/d4d/d92/fb0 [0,4194304] 0 2024-02-16T22:22:12.929 INFO:tasks.workunit.client.1.smithi123.stdout:7/940: dread - d3/d6e/d1d/db2/d35/dba/f118 zero size 2024-02-16T22:22:12.929 INFO:tasks.workunit.client.1.smithi123.stdout:6/911: mkdir d0/d8a/d100/d147 0 2024-02-16T22:22:12.930 INFO:tasks.workunit.client.1.smithi123.stdout:3/921: mknod d0/d20/d29/c12f 0 2024-02-16T22:22:12.931 INFO:tasks.workunit.client.1.smithi123.stdout:0/860: truncate d1/d8/d13/d1d/d62/fcc 686436 0 2024-02-16T22:22:12.935 INFO:tasks.workunit.client.1.smithi123.stdout:6/912: write d0/d6/f124 [908384,51725] 0 2024-02-16T22:22:12.935 INFO:tasks.workunit.client.1.smithi123.stdout:0/861: read - d1/d4/d17/d74/d57/f73 zero size 2024-02-16T22:22:12.937 INFO:tasks.workunit.client.1.smithi123.stdout:3/922: unlink d0/d20/fdf 0 2024-02-16T22:22:12.940 INFO:tasks.workunit.client.1.smithi123.stdout:8/945: dwrite d0/d29/da7/d26/f6d [0,4194304] 0 2024-02-16T22:22:12.941 INFO:tasks.workunit.client.1.smithi123.stdout:8/946: write d0/fff [1451443,87435] 0 2024-02-16T22:22:12.942 INFO:tasks.workunit.client.0.smithi114.stdout:9/728: symlink d2/d1a/d65/lea 0 2024-02-16T22:22:12.942 INFO:tasks.workunit.client.0.smithi114.stdout:4/828: truncate dd/d1b/d24/fc8 2694842 0 2024-02-16T22:22:12.943 INFO:tasks.workunit.client.1.smithi123.stdout:6/913: mknod d0/d6/d91/d93/c148 0 2024-02-16T22:22:12.943 INFO:tasks.workunit.client.1.smithi123.stdout:5/910: getdents d6/d3b 0 2024-02-16T22:22:12.943 INFO:tasks.workunit.client.1.smithi123.stdout:7/941: dread d3/d6e/d4d/d92/fd1 [0,4194304] 0 2024-02-16T22:22:12.944 INFO:tasks.workunit.client.0.smithi114.stdout:6/728: fdatasync d6/dd/d25/d34/d61/d62/d8b/f94 0 2024-02-16T22:22:12.944 INFO:tasks.workunit.client.0.smithi114.stdout:2/708: rename dd/d6b/da5/fd0 to dd/d3b/da7/d4c/ff2 0 2024-02-16T22:22:12.944 INFO:tasks.workunit.client.1.smithi123.stdout:6/914: chown d0/d6/d4d 40 1 2024-02-16T22:22:12.945 INFO:tasks.workunit.client.0.smithi114.stdout:8/708: write d4/d11/f8f [854688,119923] 0 2024-02-16T22:22:12.948 INFO:tasks.workunit.client.1.smithi123.stdout:2/900: write d4/d1a/fe5 [4822527,128376] 0 2024-02-16T22:22:12.951 INFO:tasks.workunit.client.0.smithi114.stdout:8/709: truncate d4/d16/d25/d7c/f87 1369884 0 2024-02-16T22:22:12.952 INFO:tasks.workunit.client.0.smithi114.stdout:3/715: dwrite d4/dd/f1e [0,4194304] 0 2024-02-16T22:22:12.958 INFO:tasks.workunit.client.1.smithi123.stdout:1/890: dwrite d3/d6/db/d10/f4a [0,4194304] 0 2024-02-16T22:22:12.960 INFO:tasks.workunit.client.1.smithi123.stdout:0/862: dread d1/f6 [0,4194304] 0 2024-02-16T22:22:12.961 INFO:tasks.workunit.client.1.smithi123.stdout:6/915: creat d0/d6/d1a/d2e/f149 x:0 0 0 2024-02-16T22:22:12.963 INFO:tasks.workunit.client.0.smithi114.stdout:5/696: dwrite d0/d28/d98/d1e/d63/d96/d9a/db8/f74 [0,4194304] 0 2024-02-16T22:22:12.967 INFO:tasks.workunit.client.1.smithi123.stdout:2/901: rename d4/d1a/cc1 to d4/d1a/d1d/d38/d5e/c128 0 2024-02-16T22:22:12.968 INFO:tasks.workunit.client.0.smithi114.stdout:0/774: dwrite d1/d16/d1f/d28/d51/d61/d7e/fc3 [0,4194304] 0 2024-02-16T22:22:12.969 INFO:tasks.workunit.client.1.smithi123.stdout:6/916: dwrite d0/d9/dc/f23 [0,4194304] 0 2024-02-16T22:22:12.969 INFO:tasks.workunit.client.0.smithi114.stdout:4/829: creat dd/d59/d8f/dea/f12d x:0 0 0 2024-02-16T22:22:12.971 INFO:tasks.workunit.client.1.smithi123.stdout:8/947: mkdir d0/d29/da7/d26/d2a/d12f/d13a 0 2024-02-16T22:22:12.973 INFO:tasks.workunit.client.1.smithi123.stdout:6/917: truncate d0/d6/d91/d93/d116/f11e 369607 0 2024-02-16T22:22:12.974 INFO:tasks.workunit.client.0.smithi114.stdout:0/775: write d1/d16/d1f/d28/d51/d60/dd1/fe8 [583919,41033] 0 2024-02-16T22:22:12.974 INFO:tasks.workunit.client.0.smithi114.stdout:6/729: creat d6/dd/d25/d34/d58/d9f/da3/dbd/fd8 x:0 0 0 2024-02-16T22:22:12.977 INFO:tasks.workunit.client.0.smithi114.stdout:9/729: truncate d2/dd/f10 4828062 0 2024-02-16T22:22:12.978 INFO:tasks.workunit.client.0.smithi114.stdout:5/697: dwrite d0/d28/d98/d25/d2e/d3b/f85 [4194304,4194304] 0 2024-02-16T22:22:12.981 INFO:tasks.workunit.client.0.smithi114.stdout:9/730: write d2/d1a/d23/fbf [1276070,46994] 0 2024-02-16T22:22:12.981 INFO:tasks.workunit.client.1.smithi123.stdout:7/942: mkdir d3/d6e/d4d/d104/d137 0 2024-02-16T22:22:12.982 INFO:tasks.workunit.client.1.smithi123.stdout:1/891: dread - d3/d6/db/d10/fe9 zero size 2024-02-16T22:22:12.982 INFO:tasks.workunit.client.0.smithi114.stdout:3/716: symlink d4/d46/lf3 0 2024-02-16T22:22:12.983 INFO:tasks.workunit.client.1.smithi123.stdout:0/863: stat d1/d8/d13/d1d/l2b 0 2024-02-16T22:22:12.984 INFO:tasks.workunit.client.1.smithi123.stdout:0/864: stat d1/d4/d17/d54/d8e/d96/ce4 0 2024-02-16T22:22:12.985 INFO:tasks.workunit.client.1.smithi123.stdout:0/865: stat d1/d4/f11 0 2024-02-16T22:22:12.992 INFO:tasks.workunit.client.1.smithi123.stdout:6/918: fsync d0/d6/d91/d97/fb8 0 2024-02-16T22:22:12.993 INFO:tasks.workunit.client.0.smithi114.stdout:6/730: truncate d6/f21 189887 0 2024-02-16T22:22:12.996 INFO:tasks.workunit.client.1.smithi123.stdout:7/943: read d3/d6e/d1d/f22 [1880063,13135] 0 2024-02-16T22:22:12.998 INFO:tasks.workunit.client.0.smithi114.stdout:8/710: fsync d4/d11/d9d/fe9 0 2024-02-16T22:22:12.999 INFO:tasks.workunit.client.1.smithi123.stdout:0/866: rename d1/d4/d17/dad/fcd to d1/d8/d13/d1d/d64/dbc/ddb/f11b 0 2024-02-16T22:22:13.000 INFO:tasks.workunit.client.0.smithi114.stdout:9/731: fsync d2/d1a/f21 0 2024-02-16T22:22:13.000 INFO:tasks.workunit.client.0.smithi114.stdout:4/830: read dd/d2e/d3f/f6c [2186793,104097] 0 2024-02-16T22:22:13.001 INFO:tasks.workunit.client.1.smithi123.stdout:3/923: getdents d0/d20/d29/d9b/dbd/df4/d5d/d60/dc6 0 2024-02-16T22:22:13.002 INFO:tasks.workunit.client.1.smithi123.stdout:2/902: symlink d4/d1a/d1d/d13/d25/d98/d127/l129 0 2024-02-16T22:22:13.005 INFO:tasks.workunit.client.1.smithi123.stdout:6/919: fsync d0/d96/fe8 0 2024-02-16T22:22:13.007 INFO:tasks.workunit.client.1.smithi123.stdout:7/944: truncate d3/d6e/d52/f72 624151 0 2024-02-16T22:22:13.010 INFO:tasks.workunit.client.1.smithi123.stdout:3/924: stat d0/d20/d29/d9b/dbd/df4/d5d/dae/dc7/d121/fd9 0 2024-02-16T22:22:13.019 INFO:tasks.workunit.client.1.smithi123.stdout:3/925: dread d0/d20/d29/d9b/dbd/df4/d5d/dae/db0/fe7 [0,4194304] 0 2024-02-16T22:22:13.019 INFO:tasks.workunit.client.1.smithi123.stdout:6/920: symlink d0/d74/l14a 0 2024-02-16T22:22:13.019 INFO:tasks.workunit.client.0.smithi114.stdout:9/732: dwrite d2/d1a/d23/d32/d46/fad [0,4194304] 0 2024-02-16T22:22:13.022 INFO:tasks.workunit.client.0.smithi114.stdout:4/831: fdatasync dd/d1b/d24/d57/d12b/d38/d87/dad/ffb 0 2024-02-16T22:22:13.024 INFO:tasks.workunit.client.1.smithi123.stdout:5/911: write d6/dd/df/d76/feb [757025,94901] 0 2024-02-16T22:22:13.025 INFO:tasks.workunit.client.1.smithi123.stdout:5/912: fdatasync d6/f21 0 2024-02-16T22:22:13.026 INFO:tasks.workunit.client.0.smithi114.stdout:2/709: getdents dd 0 2024-02-16T22:22:13.028 INFO:tasks.workunit.client.1.smithi123.stdout:8/948: dread d0/d29/da7/d26/d68/d81/fa1 [0,4194304] 0 2024-02-16T22:22:13.028 INFO:tasks.workunit.client.1.smithi123.stdout:8/949: stat d0/f4 0 2024-02-16T22:22:13.033 INFO:tasks.workunit.client.0.smithi114.stdout:8/711: mknod d4/d11/d78/d7a/d36/d3b/de4/cfd 0 2024-02-16T22:22:13.035 INFO:tasks.workunit.client.1.smithi123.stdout:3/926: dread d0/da0/ff2 [0,4194304] 0 2024-02-16T22:22:13.036 INFO:tasks.workunit.client.1.smithi123.stdout:7/945: dread d3/d6e/d52/f6a [0,4194304] 0 2024-02-16T22:22:13.037 INFO:tasks.workunit.client.1.smithi123.stdout:6/921: dread d0/d74/f7b [0,4194304] 0 2024-02-16T22:22:13.037 INFO:tasks.workunit.client.1.smithi123.stdout:3/927: chown d0/d20/d29/df6/f122 63 1 2024-02-16T22:22:13.041 INFO:tasks.workunit.client.0.smithi114.stdout:4/832: rmdir dd/d1b/db0/de3 39 2024-02-16T22:22:13.042 INFO:tasks.workunit.client.1.smithi123.stdout:1/892: write d3/d6/db/d10/fde [60611,44758] 0 2024-02-16T22:22:13.042 INFO:tasks.workunit.client.0.smithi114.stdout:4/833: chown dd/d59/f80 108 1 2024-02-16T22:22:13.043 INFO:tasks.workunit.client.1.smithi123.stdout:6/922: dwrite d0/d9/dc/d3a/f80 [4194304,4194304] 0 2024-02-16T22:22:13.043 INFO:tasks.workunit.client.1.smithi123.stdout:5/913: creat d6/dd/df/d90/f140 x:0 0 0 2024-02-16T22:22:13.045 INFO:tasks.workunit.client.0.smithi114.stdout:2/710: read dd/d6b/f89 [705212,33334] 0 2024-02-16T22:22:13.046 INFO:tasks.workunit.client.0.smithi114.stdout:5/698: link d0/d28/d98/c4e d0/df/d53/db1/ce0 0 2024-02-16T22:22:13.050 INFO:tasks.workunit.client.1.smithi123.stdout:8/950: unlink d0/d29/da7/d26/d2a/d4d/fc8 0 2024-02-16T22:22:13.051 INFO:tasks.workunit.client.0.smithi114.stdout:9/733: dread d2/d1a/d23/fbf [0,4194304] 0 2024-02-16T22:22:13.051 INFO:tasks.workunit.client.0.smithi114.stdout:8/712: truncate d4/d11/d78/f90 42355 0 2024-02-16T22:22:13.056 INFO:tasks.workunit.client.0.smithi114.stdout:4/834: rename dd/d1b/d24/d57/d12b/d38/faf to dd/d1b/d24/d57/d12b/d61/df4/f12e 0 2024-02-16T22:22:13.056 INFO:tasks.workunit.client.0.smithi114.stdout:6/731: dwrite d6/d2c/d38/d41/f4a [4194304,4194304] 0 2024-02-16T22:22:13.056 INFO:tasks.workunit.client.1.smithi123.stdout:0/867: write d1/f6 [3432680,85792] 0 2024-02-16T22:22:13.058 INFO:tasks.workunit.client.0.smithi114.stdout:4/835: dread - dd/d2e/d56/ddc/dde/fb5 zero size 2024-02-16T22:22:13.059 INFO:tasks.workunit.client.0.smithi114.stdout:0/776: truncate d1/d16/d1f/d28/d51/d61/d7e/d90/f42 2976871 0 2024-02-16T22:22:13.059 INFO:tasks.workunit.client.0.smithi114.stdout:4/836: stat dd/d1b/d24/f3d 0 2024-02-16T22:22:13.063 INFO:tasks.workunit.client.0.smithi114.stdout:9/734: rmdir d2/d5c/d5f/d72 39 2024-02-16T22:22:13.064 INFO:tasks.workunit.client.1.smithi123.stdout:2/903: dwrite d4/d1a/d1d/d59/dd7/ddc/f6b [0,4194304] 0 2024-02-16T22:22:13.067 INFO:tasks.workunit.client.1.smithi123.stdout:3/928: stat d0/d20/d29/d9b/dbd/df4/d5d/d60/d93/fbb 0 2024-02-16T22:22:13.067 INFO:tasks.workunit.client.0.smithi114.stdout:0/777: dwrite d1/d16/d1f/d28/d51/d60/dd1/fe7 [0,4194304] 0 2024-02-16T22:22:13.069 INFO:tasks.workunit.client.0.smithi114.stdout:8/713: mkdir d4/d24/d6d/dc7/dfe 0 2024-02-16T22:22:13.071 INFO:tasks.workunit.client.1.smithi123.stdout:1/893: fsync d3/d6/db/d10/d4b/d70/d87/fb3 0 2024-02-16T22:22:13.074 INFO:tasks.workunit.client.1.smithi123.stdout:1/894: write d3/d6/db/d10/d2d/d8c/dfd/d9f/db4/ddf/f111 [28637,20602] 0 2024-02-16T22:22:13.076 INFO:tasks.workunit.client.1.smithi123.stdout:8/951: mknod d0/d29/da7/d26/d2a/d123/dca/c13b 0 2024-02-16T22:22:13.077 INFO:tasks.workunit.client.0.smithi114.stdout:4/837: rmdir dd/d1b/d24/d57/d12b/d38 39 2024-02-16T22:22:13.079 INFO:tasks.workunit.client.0.smithi114.stdout:5/699: truncate d0/d28/f38 696775 0 2024-02-16T22:22:13.081 INFO:tasks.workunit.client.0.smithi114.stdout:9/735: mkdir d2/dd/d3c/d89/deb 0 2024-02-16T22:22:13.086 INFO:tasks.workunit.client.0.smithi114.stdout:0/778: rmdir d1 39 2024-02-16T22:22:13.087 INFO:tasks.workunit.client.1.smithi123.stdout:2/904: symlink d4/d1a/dd5/d10b/df1/db1/l12a 0 2024-02-16T22:22:13.088 INFO:tasks.workunit.client.1.smithi123.stdout:2/905: stat d4/d1a/dd5/d10b/df1/fca 0 2024-02-16T22:22:13.088 INFO:tasks.workunit.client.0.smithi114.stdout:8/714: creat d4/d11/d78/d7a/d36/d3b/de4/fff x:0 0 0 2024-02-16T22:22:13.091 INFO:tasks.workunit.client.0.smithi114.stdout:2/711: write dd/f17 [8022860,91472] 0 2024-02-16T22:22:13.094 INFO:tasks.workunit.client.1.smithi123.stdout:1/895: truncate d3/d6/db/d10/d4b/d70/fb8 749192 0 2024-02-16T22:22:13.095 INFO:tasks.workunit.client.1.smithi123.stdout:1/896: chown d3/d6/d49/d48/d68 33522292 1 2024-02-16T22:22:13.095 INFO:tasks.workunit.client.1.smithi123.stdout:1/897: fsync d3/d6/db/d10/fc9 0 2024-02-16T22:22:13.097 INFO:tasks.workunit.client.1.smithi123.stdout:7/946: link d3/d87/lfa d3/de0/d135/l138 0 2024-02-16T22:22:13.098 INFO:tasks.workunit.client.0.smithi114.stdout:8/715: creat d4/d11/d78/d7a/d36/d3b/f100 x:0 0 0 2024-02-16T22:22:13.100 INFO:tasks.workunit.client.0.smithi114.stdout:2/712: unlink dd/df/d1c/d21/fb1 0 2024-02-16T22:22:13.100 INFO:tasks.workunit.client.0.smithi114.stdout:2/713: stat dd/f3a 0 2024-02-16T22:22:13.101 INFO:tasks.workunit.client.0.smithi114.stdout:4/838: unlink dd/d1b/d24/c33 0 2024-02-16T22:22:13.103 INFO:tasks.workunit.client.0.smithi114.stdout:5/700: symlink d0/d28/d98/d25/d89/d99/db0/le1 0 2024-02-16T22:22:13.103 INFO:tasks.workunit.client.1.smithi123.stdout:1/898: mkdir d3/d6/db/d55/d12c 0 2024-02-16T22:22:13.104 INFO:tasks.workunit.client.0.smithi114.stdout:4/839: chown dd/d2e/d56/ddc/dde/l119 72 1 2024-02-16T22:22:13.106 INFO:tasks.workunit.client.1.smithi123.stdout:2/906: mknod d4/d1a/d1d/d59/d45/c12b 0 2024-02-16T22:22:13.107 INFO:tasks.workunit.client.0.smithi114.stdout:2/714: dwrite dd/d6b/da5/fc1 [0,4194304] 0 2024-02-16T22:22:13.107 INFO:tasks.workunit.client.1.smithi123.stdout:3/929: link d0/d20/d2e/l3e d0/d20/d29/d9b/dbd/df4/d5d/d60/dc6/l130 0 2024-02-16T22:22:13.108 INFO:tasks.workunit.client.0.smithi114.stdout:2/715: chown dd/d4e/fe0 5 1 2024-02-16T22:22:13.110 INFO:tasks.workunit.client.0.smithi114.stdout:8/716: rename d4/d16/f4b to d4/d11/d9d/ddf/f101 0 2024-02-16T22:22:13.112 INFO:tasks.workunit.client.1.smithi123.stdout:7/947: mkdir d3/d139 0 2024-02-16T22:22:13.114 INFO:tasks.workunit.client.0.smithi114.stdout:0/779: dwrite d1/d16/f34 [0,4194304] 0 2024-02-16T22:22:13.114 INFO:tasks.workunit.client.1.smithi123.stdout:7/948: write d3/d5/d6c/ff6 [686720,105818] 0 2024-02-16T22:22:13.115 INFO:tasks.workunit.client.0.smithi114.stdout:9/736: dread d2/d5c/dc9/d47/d63/fa5 [0,4194304] 0 2024-02-16T22:22:13.115 INFO:tasks.workunit.client.1.smithi123.stdout:7/949: chown d3/d6e/d52/d106 747363 1 2024-02-16T22:22:13.116 INFO:tasks.workunit.client.0.smithi114.stdout:4/840: unlink dd/d1b/l26 0 2024-02-16T22:22:13.118 INFO:tasks.workunit.client.1.smithi123.stdout:3/930: dread - d0/d20/d29/d9b/dbd/df4/d37/d41/f69 zero size 2024-02-16T22:22:13.119 INFO:tasks.workunit.client.0.smithi114.stdout:6/732: write d6/d2c/d38/d54/f3d [1443452,58771] 0 2024-02-16T22:22:13.121 INFO:tasks.workunit.client.0.smithi114.stdout:0/780: truncate d1/d16/d1f/d28/d51/d61/d7e/d90/d95/fc9 254121 0 2024-02-16T22:22:13.123 INFO:tasks.workunit.client.1.smithi123.stdout:0/868: write d1/d4/d17/d54/fba [647776,9722] 0 2024-02-16T22:22:13.125 INFO:tasks.workunit.client.0.smithi114.stdout:5/701: rename d0/d2d/d30 to d0/d28/d98/d25/d89/de2 0 2024-02-16T22:22:13.125 INFO:tasks.workunit.client.0.smithi114.stdout:8/717: rename d4/d24/d2c/d5f to d4/d24/d2c/d5f/dd1/d102 22 2024-02-16T22:22:13.131 INFO:tasks.workunit.client.1.smithi123.stdout:3/931: creat d0/d20/d2e/dc4/f131 x:0 0 0 2024-02-16T22:22:13.132 INFO:tasks.workunit.client.1.smithi123.stdout:3/932: chown d0/d20/d29/d9b/dbd/df4/d5d/dae/dc7 3806089 1 2024-02-16T22:22:13.133 INFO:tasks.workunit.client.1.smithi123.stdout:5/914: truncate d6/dd/d44/dba/d126/d12c/ff5 1026618 0 2024-02-16T22:22:13.134 INFO:tasks.workunit.client.1.smithi123.stdout:5/915: dread - d6/dd/df/d58/d3f/d6f/f133 zero size 2024-02-16T22:22:13.134 INFO:tasks.workunit.client.1.smithi123.stdout:2/907: creat d4/d1a/d1d/d59/f12c x:0 0 0 2024-02-16T22:22:13.135 INFO:tasks.workunit.client.0.smithi114.stdout:6/733: dread - d6/d2c/d38/fac zero size 2024-02-16T22:22:13.135 INFO:tasks.workunit.client.0.smithi114.stdout:4/841: dread dd/d1b/d24/fe6 [0,4194304] 0 2024-02-16T22:22:13.138 INFO:tasks.workunit.client.0.smithi114.stdout:0/781: symlink d1/d16/d48/d98/lf4 0 2024-02-16T22:22:13.140 INFO:tasks.workunit.client.1.smithi123.stdout:3/933: dwrite d0/d20/d29/d9b/dbd/df4/d37/f10c [0,4194304] 0 2024-02-16T22:22:13.140 INFO:tasks.workunit.client.0.smithi114.stdout:2/716: creat dd/df/d1c/d21/d61/d69/de9/ff3 x:0 0 0 2024-02-16T22:22:13.141 INFO:tasks.workunit.client.0.smithi114.stdout:2/717: write dd/f5d [3106895,35446] 0 2024-02-16T22:22:13.144 INFO:tasks.workunit.client.1.smithi123.stdout:2/908: fdatasync d4/d1a/d1d/d13/d15/fb4 0 2024-02-16T22:22:13.145 INFO:tasks.workunit.client.1.smithi123.stdout:2/909: chown d4/dd9/ffb 2036605 1 2024-02-16T22:22:13.145 INFO:tasks.workunit.client.1.smithi123.stdout:8/952: dwrite d0/d29/da7/d26/d2a/d4d/f57 [0,4194304] 0 2024-02-16T22:22:13.153 INFO:tasks.workunit.client.1.smithi123.stdout:2/910: dwrite d4/d1a/dd5/d10b/df1/d4a/dce/d115/f105 [0,4194304] 0 2024-02-16T22:22:13.156 INFO:tasks.workunit.client.1.smithi123.stdout:7/950: creat d3/f13a x:0 0 0 2024-02-16T22:22:13.157 INFO:tasks.workunit.client.1.smithi123.stdout:6/923: sync 2024-02-16T22:22:13.158 INFO:tasks.workunit.client.0.smithi114.stdout:6/734: rename d6/d2c/d5e/f67 to d6/dd/d25/d34/d61/d62/d8b/db7/dd3/fd9 0 2024-02-16T22:22:13.160 INFO:tasks.workunit.client.1.smithi123.stdout:1/899: dread d3/d6/db/d10/fc9 [0,4194304] 0 2024-02-16T22:22:13.162 INFO:tasks.workunit.client.1.smithi123.stdout:1/900: truncate d3/d6/d11/fe4 1406538 0 2024-02-16T22:22:13.162 INFO:tasks.workunit.client.1.smithi123.stdout:5/916: rename d6/dd/df/d58/d89/d98/dd8 to d6/dd/df/d58/d56/d141 0 2024-02-16T22:22:13.163 INFO:tasks.workunit.client.1.smithi123.stdout:5/917: truncate d6/dd/d44/fbd 170339 0 2024-02-16T22:22:13.164 INFO:tasks.workunit.client.0.smithi114.stdout:9/737: link d2/d5c/dc9/d47/d63/fc5 d2/d1a/d23/fec 0 2024-02-16T22:22:13.168 INFO:tasks.workunit.client.1.smithi123.stdout:8/953: chown d0/d61/d66/lda 76768545 1 2024-02-16T22:22:13.174 INFO:tasks.workunit.client.0.smithi114.stdout:2/718: rename dd/d4e/ca0 to dd/df/d1c/d21/d87/cf4 0 2024-02-16T22:22:13.175 INFO:tasks.workunit.client.0.smithi114.stdout:2/719: chown dd/df/d1c/d21/d87/le5 10197 1 2024-02-16T22:22:13.176 INFO:tasks.workunit.client.0.smithi114.stdout:6/735: stat d6/dd/d25/d34/d58/d9f/da3/dbd/lc1 0 2024-02-16T22:22:13.176 INFO:tasks.workunit.client.0.smithi114.stdout:6/736: stat d6/d2c/l37 0 2024-02-16T22:22:13.177 INFO:tasks.workunit.client.1.smithi123.stdout:7/951: creat d3/d6e/d4d/d92/f13b x:0 0 0 2024-02-16T22:22:13.181 INFO:tasks.workunit.client.1.smithi123.stdout:2/911: dread d4/d1a/d1d/d38/d7b/f94 [0,4194304] 0 2024-02-16T22:22:13.181 INFO:tasks.workunit.client.1.smithi123.stdout:1/901: creat d3/d6/d18/dcb/f12d x:0 0 0 2024-02-16T22:22:13.183 INFO:tasks.workunit.client.1.smithi123.stdout:5/918: creat d6/dd/dbb/f142 x:0 0 0 2024-02-16T22:22:13.183 INFO:tasks.workunit.client.0.smithi114.stdout:6/737: dread d6/d2c/d38/d54/f3d [0,4194304] 0 2024-02-16T22:22:13.184 INFO:tasks.workunit.client.1.smithi123.stdout:3/934: truncate d0/fe 1548707 0 2024-02-16T22:22:13.184 INFO:tasks.workunit.client.0.smithi114.stdout:4/842: link dd/d1b/d41/c115 dd/d2e/de7/d11f/c12f 0 2024-02-16T22:22:13.186 INFO:tasks.workunit.client.0.smithi114.stdout:2/720: chown dd/df/d1c/c27 4669 1 2024-02-16T22:22:13.186 INFO:tasks.workunit.client.1.smithi123.stdout:0/869: link d1/d8/cb6 d1/d4/d17/d54/db4/dc5/c11c 0 2024-02-16T22:22:13.186 INFO:tasks.workunit.client.0.smithi114.stdout:2/721: read - dd/fd1 zero size 2024-02-16T22:22:13.189 INFO:tasks.workunit.client.0.smithi114.stdout:9/738: creat d2/d5c/d5f/d91/de4/fed x:0 0 0 2024-02-16T22:22:13.190 INFO:tasks.workunit.client.0.smithi114.stdout:2/722: dread dd/d3b/da7/fa3 [0,4194304] 0 2024-02-16T22:22:13.191 INFO:tasks.workunit.client.0.smithi114.stdout:2/723: read dd/f3a [1053576,68694] 0 2024-02-16T22:22:13.193 INFO:tasks.workunit.client.0.smithi114.stdout:2/724: chown dd/d3b/da7/d4c/fb4 249305 1 2024-02-16T22:22:13.194 INFO:tasks.workunit.client.0.smithi114.stdout:2/725: readlink dd/df/d1c/d21/d61/d63/l8f 0 2024-02-16T22:22:13.196 INFO:tasks.workunit.client.0.smithi114.stdout:6/738: mknod d6/dd/d25/d34/d61/d62/db5/cda 0 2024-02-16T22:22:13.198 INFO:tasks.workunit.client.1.smithi123.stdout:3/935: fsync d0/d20/d29/d9b/dbd/df4/d5d/dae/dc7/d121/fd9 0 2024-02-16T22:22:13.199 INFO:tasks.workunit.client.1.smithi123.stdout:2/912: dwrite d4/d1a/d1d/d13/d15/fb4 [0,4194304] 0 2024-02-16T22:22:13.201 INFO:tasks.workunit.client.0.smithi114.stdout:9/739: rename d2/d5c/d5f/d91/cac to d2/d5c/d5f/d91/cee 0 2024-02-16T22:22:13.201 INFO:tasks.workunit.client.1.smithi123.stdout:3/936: chown d0/d20/d29/d9b/dbd/df4/d5d/dae/db0/lee 1 1 2024-02-16T22:22:13.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:12 smithi114 ceph-mon[29235]: pgmap v34: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 128 MiB/s rd, 264 MiB/s wr, 614 op/s 2024-02-16T22:22:13.205 INFO:tasks.workunit.client.1.smithi123.stdout:0/870: rename d1/d4/l89 to d1/da0/l11d 0 2024-02-16T22:22:13.208 INFO:tasks.workunit.client.0.smithi114.stdout:0/782: dread d1/d1a/d47/d62/d7f/f84 [0,4194304] 0 2024-02-16T22:22:13.210 INFO:tasks.workunit.client.0.smithi114.stdout:0/783: fdatasync d1/d16/d1f/fa9 0 2024-02-16T22:22:13.213 INFO:tasks.workunit.client.1.smithi123.stdout:1/902: dread d3/d6/f26 [0,4194304] 0 2024-02-16T22:22:13.214 INFO:tasks.workunit.client.0.smithi114.stdout:6/739: rename d6/dd/d25/d34/d58/d9f/cd2 to d6/d2c/d38/dd6/cdb 0 2024-02-16T22:22:13.214 INFO:tasks.workunit.client.0.smithi114.stdout:8/718: dread d4/d16/d25/f6e [0,4194304] 0 2024-02-16T22:22:13.215 INFO:tasks.workunit.client.0.smithi114.stdout:6/740: chown d6/d2c/d38/dd6 0 1 2024-02-16T22:22:13.217 INFO:tasks.workunit.client.1.smithi123.stdout:5/919: dwrite d6/dd/df/d9f/fc6 [0,4194304] 0 2024-02-16T22:22:13.222 INFO:tasks.workunit.client.1.smithi123.stdout:2/913: mknod d4/d1a/d1d/d13/d25/d98/d10c/c12d 0 2024-02-16T22:22:13.224 INFO:tasks.workunit.client.1.smithi123.stdout:6/924: dwrite d0/d6/f8 [0,4194304] 0 2024-02-16T22:22:13.225 INFO:tasks.workunit.client.0.smithi114.stdout:0/784: mkdir d1/d16/d1f/d28/d51/d61/d7e/d90/df5 0 2024-02-16T22:22:13.226 INFO:tasks.workunit.client.0.smithi114.stdout:0/785: chown d1/d1a/d47/l5b 1264201 1 2024-02-16T22:22:13.227 INFO:tasks.workunit.client.1.smithi123.stdout:0/871: mkdir d1/d8/d13/d1d/d62/d7a/d11e 0 2024-02-16T22:22:13.227 INFO:tasks.workunit.client.1.smithi123.stdout:6/925: read d0/d6/f124 [481901,102530] 0 2024-02-16T22:22:13.229 INFO:tasks.workunit.client.1.smithi123.stdout:8/954: write d0/d12/f74 [3560942,99318] 0 2024-02-16T22:22:13.230 INFO:tasks.workunit.client.1.smithi123.stdout:7/952: link d3/d6e/d52/ce9 d3/d6e/d1d/db2/d3e/de4/dfd/c13c 0 2024-02-16T22:22:13.232 INFO:tasks.workunit.client.0.smithi114.stdout:9/740: rename d2/d33/d3f/fca to d2/d5c/dc9/d47/d63/dbb/fef 0 2024-02-16T22:22:13.234 INFO:tasks.workunit.client.0.smithi114.stdout:4/843: write dd/f2b [599530,63729] 0 2024-02-16T22:22:13.236 INFO:tasks.workunit.client.0.smithi114.stdout:8/719: fdatasync d4/d11/f17 0 2024-02-16T22:22:13.238 INFO:tasks.workunit.client.0.smithi114.stdout:6/741: dread d6/dd/d25/f27 [0,4194304] 0 2024-02-16T22:22:13.239 INFO:tasks.workunit.client.0.smithi114.stdout:6/742: read d6/d2c/d38/d41/f4a [4811338,22580] 0 2024-02-16T22:22:13.241 INFO:tasks.workunit.client.0.smithi114.stdout:0/786: truncate d1/d16/d1f/d28/f54 100138 0 2024-02-16T22:22:13.243 INFO:tasks.workunit.client.1.smithi123.stdout:2/914: chown d4/d1a/l48 67781 1 2024-02-16T22:22:13.250 INFO:tasks.workunit.client.1.smithi123.stdout:0/872: chown d1/da0/l103 73 1 2024-02-16T22:22:13.252 INFO:tasks.workunit.client.0.smithi114.stdout:2/726: truncate dd/f28 480593 0 2024-02-16T22:22:13.254 INFO:tasks.workunit.client.1.smithi123.stdout:8/955: unlink d0/d29/da7/d26/f7a 0 2024-02-16T22:22:13.257 INFO:tasks.workunit.client.0.smithi114.stdout:6/743: truncate d6/dd/f1d 1393648 0 2024-02-16T22:22:13.257 INFO:tasks.workunit.client.0.smithi114.stdout:6/744: readlink d6/l8 0 2024-02-16T22:22:13.258 INFO:tasks.workunit.client.0.smithi114.stdout:6/745: write d6/fbf [987052,73141] 0 2024-02-16T22:22:13.264 INFO:tasks.workunit.client.1.smithi123.stdout:3/937: rename d0/d20/d29/d9b/dbd/df4/d37/ldd to d0/d50/l132 0 2024-02-16T22:22:13.264 INFO:tasks.workunit.client.1.smithi123.stdout:5/920: write d6/dd/d44/dba/d126/d12c/fb4 [743279,127358] 0 2024-02-16T22:22:13.265 INFO:tasks.workunit.client.1.smithi123.stdout:1/903: dwrite d3/d6/db/d10/d2d/d8c/dfd/d9f/fc3 [0,4194304] 0 2024-02-16T22:22:13.265 INFO:tasks.workunit.client.0.smithi114.stdout:8/720: dread d4/d16/d25/d46/fb2 [0,4194304] 0 2024-02-16T22:22:13.267 INFO:tasks.workunit.client.1.smithi123.stdout:1/904: write d3/d6/db/d10/d2d/d8c/dfd/d9f/fc3 [1735205,4160] 0 2024-02-16T22:22:13.270 INFO:tasks.workunit.client.1.smithi123.stdout:0/873: creat d1/d8/d13/d1d/d62/d7a/f11f x:0 0 0 2024-02-16T22:22:13.270 INFO:tasks.workunit.client.1.smithi123.stdout:1/905: dread - d3/d6/db/d10/d2a/d11c/f125 zero size 2024-02-16T22:22:13.272 INFO:tasks.workunit.client.1.smithi123.stdout:1/906: write d3/d6/d18/dcb/f12d [880420,118224] 0 2024-02-16T22:22:13.273 INFO:tasks.workunit.client.1.smithi123.stdout:5/921: dread d6/dd/df/d1f/f10d [0,4194304] 0 2024-02-16T22:22:13.273 INFO:tasks.workunit.client.1.smithi123.stdout:1/907: fsync d3/d6/d49/d48/d84/f93 0 2024-02-16T22:22:13.275 INFO:tasks.workunit.client.0.smithi114.stdout:2/727: dread dd/df/d1c/d21/fcd [0,4194304] 0 2024-02-16T22:22:13.277 INFO:tasks.workunit.client.0.smithi114.stdout:2/728: write dd/d3b/d8a/fe7 [965671,108235] 0 2024-02-16T22:22:13.277 INFO:tasks.workunit.client.1.smithi123.stdout:8/956: fdatasync d0/d29/da7/dea/fec 0 2024-02-16T22:22:13.285 INFO:tasks.workunit.client.0.smithi114.stdout:0/787: dwrite d1/d78/f7d [0,4194304] 0 2024-02-16T22:22:13.287 INFO:tasks.workunit.client.1.smithi123.stdout:3/938: creat d0/da0/f133 x:0 0 0 2024-02-16T22:22:13.288 INFO:tasks.workunit.client.1.smithi123.stdout:3/939: fsync d0/d20/d29/d9b/dbd/df4/f12b 0 2024-02-16T22:22:13.295 INFO:tasks.workunit.client.0.smithi114.stdout:9/741: getdents d2/d33/d3f/ddd 0 2024-02-16T22:22:13.299 INFO:tasks.workunit.client.0.smithi114.stdout:4/844: getdents dd/d59/d8f/df7 0 2024-02-16T22:22:13.301 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:12 smithi123 ceph-mon[37161]: pgmap v34: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 128 MiB/s rd, 264 MiB/s wr, 614 op/s 2024-02-16T22:22:13.302 INFO:tasks.workunit.client.1.smithi123.stdout:8/957: creat d0/d29/da7/d26/d2a/d4d/f13c x:0 0 0 2024-02-16T22:22:13.303 INFO:tasks.workunit.client.1.smithi123.stdout:7/953: link d3/d59/la3 d3/d6e/d1d/db2/d3d/db1/d111/l13d 0 2024-02-16T22:22:13.303 INFO:tasks.workunit.client.1.smithi123.stdout:7/954: stat d3/d5/f33 0 2024-02-16T22:22:13.308 INFO:tasks.workunit.client.1.smithi123.stdout:0/874: mknod d1/d4/c120 0 2024-02-16T22:22:13.308 INFO:tasks.workunit.client.0.smithi114.stdout:2/729: dwrite dd/d4e/fbb [0,4194304] 0 2024-02-16T22:22:13.309 INFO:tasks.workunit.client.1.smithi123.stdout:8/958: dwrite d0/d29/da7/d26/d2a/d123/f12e [0,4194304] 0 2024-02-16T22:22:13.310 INFO:tasks.workunit.client.0.smithi114.stdout:6/746: rename d6/d2c/f7c to d6/d2c/d38/fdc 0 2024-02-16T22:22:13.312 INFO:tasks.workunit.client.1.smithi123.stdout:7/955: dread d3/d59/fe1 [0,4194304] 0 2024-02-16T22:22:13.313 INFO:tasks.workunit.client.1.smithi123.stdout:6/926: getdents d0/d9/dc 0 2024-02-16T22:22:13.313 INFO:tasks.workunit.client.1.smithi123.stdout:3/940: dread d0/d20/d2e/f48 [0,4194304] 0 2024-02-16T22:22:13.313 INFO:tasks.workunit.client.0.smithi114.stdout:9/742: symlink d2/d1a/d23/d32/d46/lf0 0 2024-02-16T22:22:13.315 INFO:tasks.workunit.client.1.smithi123.stdout:1/908: dwrite d3/fe [0,4194304] 0 2024-02-16T22:22:13.315 INFO:tasks.workunit.client.1.smithi123.stdout:2/915: sync 2024-02-16T22:22:13.315 INFO:tasks.workunit.client.0.smithi114.stdout:8/721: creat d4/d24/d2c/d5f/f103 x:0 0 0 2024-02-16T22:22:13.316 INFO:tasks.workunit.client.1.smithi123.stdout:8/959: write d0/d29/da7/d26/d68/d81/d47/d9b/fc5 [2289207,68203] 0 2024-02-16T22:22:13.317 INFO:tasks.workunit.client.0.smithi114.stdout:0/788: mknod d1/d1a/d47/d62/d89/cf6 0 2024-02-16T22:22:13.318 INFO:tasks.workunit.client.0.smithi114.stdout:0/789: write d1/d16/d48/f6a [4525491,125646] 0 2024-02-16T22:22:13.320 INFO:tasks.workunit.client.0.smithi114.stdout:0/790: chown d1/d1a/d47/d62/d89/l97 5 1 2024-02-16T22:22:13.321 INFO:tasks.workunit.client.1.smithi123.stdout:0/875: mkdir d1/d4/d17/dad/d121 0 2024-02-16T22:22:13.321 INFO:tasks.workunit.client.0.smithi114.stdout:1/699: sync 2024-02-16T22:22:13.324 INFO:tasks.workunit.client.0.smithi114.stdout:1/700: dread - d4/d63/d69/d72/d82/fb2 zero size 2024-02-16T22:22:13.326 INFO:tasks.workunit.client.0.smithi114.stdout:4/845: creat dd/d1b/d24/d57/d12b/d61/d11d/f130 x:0 0 0 2024-02-16T22:22:13.329 INFO:tasks.workunit.client.1.smithi123.stdout:2/916: write d4/d1a/d1d/d13/d25/d98/d10c/d11b/f112 [389925,68493] 0 2024-02-16T22:22:13.330 INFO:tasks.workunit.client.1.smithi123.stdout:3/941: symlink d0/d20/d29/d9b/dbd/df4/d5d/d60/dc6/l134 0 2024-02-16T22:22:13.330 INFO:tasks.workunit.client.1.smithi123.stdout:2/917: write d4/d1a/d1d/d59/dd7/ddc/f6b [3168079,51190] 0 2024-02-16T22:22:13.331 INFO:tasks.workunit.client.0.smithi114.stdout:0/791: truncate d1/d16/d1f/d28/d51/d61/d6d/df1/fa4 1559005 0 2024-02-16T22:22:13.332 INFO:tasks.workunit.client.1.smithi123.stdout:2/918: chown d4/d1a/d1d/d13/d25/d98/f76 6586047 1 2024-02-16T22:22:13.333 INFO:tasks.workunit.client.1.smithi123.stdout:8/960: mknod d0/d61/dac/df4/c13d 0 2024-02-16T22:22:13.334 INFO:tasks.workunit.client.1.smithi123.stdout:2/919: chown d4/d1a/d1d/d13/d25/d98/d10c/d11b/fde 3699 1 2024-02-16T22:22:13.336 INFO:tasks.workunit.client.1.smithi123.stdout:6/927: rename d0/d6/d1a/l4b to d0/d9/dc/d3a/d70/de5/d12f/l14b 0 2024-02-16T22:22:13.339 INFO:tasks.workunit.client.1.smithi123.stdout:5/922: getdents d6/dd/d44/dba/d126/d12c 0 2024-02-16T22:22:13.339 INFO:tasks.workunit.client.1.smithi123.stdout:5/923: chown d6/l15 0 1 2024-02-16T22:22:13.342 INFO:tasks.workunit.client.0.smithi114.stdout:4/846: rename dd/d1b/d41/d77/f90 to dd/d2e/d56/ddc/dde/f131 0 2024-02-16T22:22:13.344 INFO:tasks.workunit.client.1.smithi123.stdout:7/956: mkdir d3/d139/d13e 0 2024-02-16T22:22:13.345 INFO:tasks.workunit.client.0.smithi114.stdout:1/701: fsync d4/d63/d69/d7c/d9b/db0/f43 0 2024-02-16T22:22:13.349 INFO:tasks.workunit.client.1.smithi123.stdout:3/942: fsync d0/d20/d2e/fb8 0 2024-02-16T22:22:13.353 INFO:tasks.workunit.client.0.smithi114.stdout:4/847: creat dd/f132 x:0 0 0 2024-02-16T22:22:13.356 INFO:tasks.workunit.client.0.smithi114.stdout:2/730: dwrite dd/d6b/da5/fc1 [4194304,4194304] 0 2024-02-16T22:22:13.357 INFO:tasks.workunit.client.1.smithi123.stdout:2/920: symlink d4/d1a/d1d/d13/d25/d98/d10c/l12e 0 2024-02-16T22:22:13.357 INFO:tasks.workunit.client.0.smithi114.stdout:2/731: chown dd/d3b 209205380 1 2024-02-16T22:22:13.360 INFO:tasks.workunit.client.0.smithi114.stdout:2/732: read - dd/df/d1c/d21/d61/d69/fd7 zero size 2024-02-16T22:22:13.361 INFO:tasks.workunit.client.0.smithi114.stdout:8/722: dwrite d4/f3c [0,4194304] 0 2024-02-16T22:22:13.361 INFO:tasks.workunit.client.1.smithi123.stdout:5/924: fdatasync d6/dd/df/d58/d3f/fdb 0 2024-02-16T22:22:13.369 INFO:tasks.workunit.client.1.smithi123.stdout:0/876: creat d1/d4/d17/dad/db2/f122 x:0 0 0 2024-02-16T22:22:13.372 INFO:tasks.workunit.client.0.smithi114.stdout:9/743: write d2/d1a/f21 [747572,78862] 0 2024-02-16T22:22:13.374 INFO:tasks.workunit.client.0.smithi114.stdout:0/792: link d1/d16/d1f/d28/d51/d61/d7e/d90/fcf d1/d16/d1f/d28/d51/d60/dd1/ff7 0 2024-02-16T22:22:13.375 INFO:tasks.workunit.client.0.smithi114.stdout:8/723: dwrite d4/d24/d2c/d67/dcf/ff1 [0,4194304] 0 2024-02-16T22:22:13.376 INFO:tasks.workunit.client.0.smithi114.stdout:2/733: dwrite dd/d3b/d8a/dcb/fee [0,4194304] 0 2024-02-16T22:22:13.382 INFO:tasks.workunit.client.0.smithi114.stdout:2/734: fsync dd/df/d1c/d21/d61/d69/de9/ff3 0 2024-02-16T22:22:13.384 INFO:tasks.workunit.client.0.smithi114.stdout:7/814: sync 2024-02-16T22:22:13.384 INFO:tasks.workunit.client.0.smithi114.stdout:3/717: sync 2024-02-16T22:22:13.385 INFO:tasks.workunit.client.0.smithi114.stdout:5/702: sync 2024-02-16T22:22:13.386 INFO:tasks.workunit.client.1.smithi123.stdout:3/943: mknod d0/d20/d29/df6/c135 0 2024-02-16T22:22:13.387 INFO:tasks.workunit.client.1.smithi123.stdout:8/961: write d0/d29/da7/d26/d2a/d77/df8/f55 [2924542,83756] 0 2024-02-16T22:22:13.390 INFO:tasks.workunit.client.1.smithi123.stdout:2/921: truncate d4/d1a/d1d/d59/dd7/ddc/fa8 5480409 0 2024-02-16T22:22:13.393 INFO:tasks.workunit.client.1.smithi123.stdout:5/925: creat d6/dd/d78/da5/f143 x:0 0 0 2024-02-16T22:22:13.393 INFO:tasks.workunit.client.1.smithi123.stdout:5/926: dread - d6/dd/d44/dba/f125 zero size 2024-02-16T22:22:13.394 INFO:tasks.workunit.client.0.smithi114.stdout:7/815: dwrite d3/d14/f82 [4194304,4194304] 0 2024-02-16T22:22:13.395 INFO:tasks.workunit.client.1.smithi123.stdout:5/927: truncate d6/dd/d44/f77 514135 0 2024-02-16T22:22:13.396 INFO:tasks.workunit.client.1.smithi123.stdout:1/909: link d3/d6/d18/cdc d3/d47/c12e 0 2024-02-16T22:22:13.398 INFO:tasks.workunit.client.1.smithi123.stdout:3/944: truncate d0/d20/f26 3865230 0 2024-02-16T22:22:13.404 INFO:tasks.workunit.client.1.smithi123.stdout:7/957: write d3/d6e/d24/f31 [3577710,31717] 0 2024-02-16T22:22:13.405 INFO:tasks.workunit.client.1.smithi123.stdout:8/962: dwrite d0/d61/d66/d7e/ff1 [0,4194304] 0 2024-02-16T22:22:13.407 INFO:tasks.workunit.client.1.smithi123.stdout:8/963: fdatasync d0/d29/da7/d26/d2a/d4d/f13c 0 2024-02-16T22:22:13.407 INFO:tasks.workunit.client.1.smithi123.stdout:5/928: mknod d6/d3b/c144 0 2024-02-16T22:22:13.408 INFO:tasks.workunit.client.1.smithi123.stdout:8/964: write d0/d29/da7/d26/d68/d81/d47/f6a [2715962,46638] 0 2024-02-16T22:22:13.410 INFO:tasks.workunit.client.1.smithi123.stdout:8/965: truncate d0/d29/da7/d26/d68/d81/dd7/f11f 49426 0 2024-02-16T22:22:13.411 INFO:tasks.workunit.client.0.smithi114.stdout:6/747: dwrite d6/d2c/f95 [0,4194304] 0 2024-02-16T22:22:13.411 INFO:tasks.workunit.client.1.smithi123.stdout:7/958: dread d3/d6e/d24/f8b [0,4194304] 0 2024-02-16T22:22:13.412 INFO:tasks.workunit.client.1.smithi123.stdout:7/959: chown f2 5 1 2024-02-16T22:22:13.415 INFO:tasks.workunit.client.1.smithi123.stdout:1/910: rmdir d3/d6/db 39 2024-02-16T22:22:13.416 INFO:tasks.workunit.client.0.smithi114.stdout:5/703: dwrite d0/d28/d98/d1e/d63/fc8 [0,4194304] 0 2024-02-16T22:22:13.418 INFO:tasks.workunit.client.0.smithi114.stdout:2/735: dread dd/df/ddb/fc9 [0,4194304] 0 2024-02-16T22:22:13.419 INFO:tasks.workunit.client.0.smithi114.stdout:5/704: stat d0/d28/d98/d25/d2e/d3b/db5 0 2024-02-16T22:22:13.424 INFO:tasks.workunit.client.1.smithi123.stdout:6/928: sync 2024-02-16T22:22:13.427 INFO:tasks.workunit.client.0.smithi114.stdout:8/724: mkdir d4/d11/d9d/d104 0 2024-02-16T22:22:13.429 INFO:tasks.workunit.client.0.smithi114.stdout:7/816: symlink d3/d3d/l122 0 2024-02-16T22:22:13.430 INFO:tasks.workunit.client.0.smithi114.stdout:2/736: dwrite dd/df/d1c/d21/d87/fef [0,4194304] 0 2024-02-16T22:22:13.434 INFO:tasks.workunit.client.1.smithi123.stdout:5/929: mkdir d6/dd/df/d1f/ddc/d145 0 2024-02-16T22:22:13.439 INFO:tasks.workunit.client.1.smithi123.stdout:0/877: dwrite d1/d4/d17/d74/d57/f73 [0,4194304] 0 2024-02-16T22:22:13.439 INFO:tasks.workunit.client.0.smithi114.stdout:9/744: dwrite d2/d1a/d23/d32/d46/d78/fa6 [0,4194304] 0 2024-02-16T22:22:13.440 INFO:tasks.workunit.client.0.smithi114.stdout:0/793: link d1/d78/fd3 d1/d16/d1f/d28/d51/d61/df3/ff8 0 2024-02-16T22:22:13.441 INFO:tasks.workunit.client.1.smithi123.stdout:0/878: write d1/d4/d42/f91 [403910,20005] 0 2024-02-16T22:22:13.441 INFO:tasks.workunit.client.1.smithi123.stdout:8/966: chown d0/d29/dba/f124 2173033 1 2024-02-16T22:22:13.443 INFO:tasks.workunit.client.0.smithi114.stdout:9/745: fsync d2/d1a/f6b 0 2024-02-16T22:22:13.444 INFO:tasks.workunit.client.0.smithi114.stdout:9/746: truncate d2/d1a/d65/fe2 224277 0 2024-02-16T22:22:13.448 INFO:tasks.workunit.client.1.smithi123.stdout:1/911: truncate d3/d6/d49/d48/d68/fab 818728 0 2024-02-16T22:22:13.451 INFO:tasks.workunit.client.0.smithi114.stdout:6/748: rename d6/dd/d25/d34/d61/d62/d8b/db7/dd3/fd9 to d6/dd/d25/d34/d61/d62/d8b/fdd 0 2024-02-16T22:22:13.451 INFO:tasks.workunit.client.1.smithi123.stdout:6/929: rmdir d0/d8a/d100/dfc 39 2024-02-16T22:22:13.455 INFO:tasks.workunit.client.0.smithi114.stdout:8/725: chown d4/d24/d2c/l5c 254 1 2024-02-16T22:22:13.458 INFO:tasks.workunit.client.0.smithi114.stdout:7/817: creat d3/d87/dd1/f123 x:0 0 0 2024-02-16T22:22:13.458 INFO:tasks.workunit.client.0.smithi114.stdout:7/818: chown d3/d22/d73/da7/d6f/df8/ff9 4 1 2024-02-16T22:22:13.459 INFO:tasks.workunit.client.1.smithi123.stdout:5/930: creat d6/dd/d78/f146 x:0 0 0 2024-02-16T22:22:13.462 INFO:tasks.workunit.client.0.smithi114.stdout:1/702: link d4/d63/d69/d7c/d9b/db0/d74/cd4 d4/d9/d46/d67/d92/dca/dcc/d98/ddd/cfe 0 2024-02-16T22:22:13.466 INFO:tasks.workunit.client.1.smithi123.stdout:0/879: symlink d1/d8/d13/d1d/d64/dbc/ddb/l123 0 2024-02-16T22:22:13.470 INFO:tasks.workunit.client.1.smithi123.stdout:7/960: creat d3/d6e/d1d/d132/f13f x:0 0 0 2024-02-16T22:22:13.471 INFO:tasks.workunit.client.0.smithi114.stdout:9/747: fsync d2/d1a/d65/f94 0 2024-02-16T22:22:13.475 INFO:tasks.workunit.client.0.smithi114.stdout:4/848: link dd/d1b/db0/fb8 dd/d1b/d24/d57/d12b/d38/d87/dad/f133 0 2024-02-16T22:22:13.478 INFO:tasks.workunit.client.0.smithi114.stdout:5/705: creat d0/d28/d98/dd0/fe3 x:0 0 0 2024-02-16T22:22:13.480 INFO:tasks.workunit.client.1.smithi123.stdout:2/922: getdents d4/d1a/d1d/d38/d5e 0 2024-02-16T22:22:13.481 INFO:tasks.workunit.client.1.smithi123.stdout:3/945: dwrite d0/d20/d29/d9b/dbd/df4/f4e [0,4194304] 0 2024-02-16T22:22:13.481 INFO:tasks.workunit.client.1.smithi123.stdout:2/923: chown d4/d1a/dd5/d10b/df1/c2a 3 1 2024-02-16T22:22:13.483 INFO:tasks.workunit.client.0.smithi114.stdout:8/726: rmdir d4/d11/d78 39 2024-02-16T22:22:13.484 INFO:tasks.workunit.client.0.smithi114.stdout:5/706: dread d0/d28/d98/d25/d89/de2/f31 [4194304,4194304] 0 2024-02-16T22:22:13.485 INFO:tasks.workunit.client.0.smithi114.stdout:7/819: symlink d3/d22/d76/l124 0 2024-02-16T22:22:13.486 INFO:tasks.workunit.client.0.smithi114.stdout:7/820: chown d3/d14/c54 266218915 1 2024-02-16T22:22:13.487 INFO:tasks.workunit.client.0.smithi114.stdout:7/821: truncate d3/d22/db7/f112 733972 0 2024-02-16T22:22:13.490 INFO:tasks.workunit.client.0.smithi114.stdout:1/703: dread d4/d9/d46/d67/d92/dca/dcc/d70/f89 [0,4194304] 0 2024-02-16T22:22:13.491 INFO:tasks.workunit.client.0.smithi114.stdout:1/704: chown d4/d9/d46/d67/d92/dca/dcc/fa5 49 1 2024-02-16T22:22:13.492 INFO:tasks.workunit.client.1.smithi123.stdout:7/961: rmdir d3/d6e/d1d/d97/dca 39 2024-02-16T22:22:13.495 INFO:tasks.workunit.client.0.smithi114.stdout:0/794: link d1/d16/f34 d1/d1a/d47/d62/d89/ff9 0 2024-02-16T22:22:13.497 INFO:tasks.workunit.client.1.smithi123.stdout:1/912: creat d3/d6/db/d55/f12f x:0 0 0 2024-02-16T22:22:13.497 INFO:tasks.workunit.client.1.smithi123.stdout:6/930: mknod d0/d8a/d100/d147/c14c 0 2024-02-16T22:22:13.498 INFO:tasks.workunit.client.0.smithi114.stdout:9/748: truncate d2/d5c/d5f/fd9 2724451 0 2024-02-16T22:22:13.498 INFO:tasks.workunit.client.1.smithi123.stdout:1/913: dread - d3/d6/db/d10/d4b/d70/d87/fb3 zero size 2024-02-16T22:22:13.500 INFO:tasks.workunit.client.1.smithi123.stdout:3/946: readlink d0/l16 0 2024-02-16T22:22:13.505 INFO:tasks.workunit.client.1.smithi123.stdout:5/931: mknod d6/dd/c147 0 2024-02-16T22:22:13.509 INFO:tasks.workunit.client.1.smithi123.stdout:0/880: mkdir d1/d124 0 2024-02-16T22:22:13.511 INFO:tasks.workunit.client.0.smithi114.stdout:6/749: truncate d6/d2c/f95 515884 0 2024-02-16T22:22:13.511 INFO:tasks.workunit.client.0.smithi114.stdout:4/849: dread dd/d1b/d24/fc8 [0,4194304] 0 2024-02-16T22:22:13.513 INFO:tasks.workunit.client.0.smithi114.stdout:5/707: read d0/d28/d98/f49 [101573,31751] 0 2024-02-16T22:22:13.514 INFO:tasks.workunit.client.0.smithi114.stdout:5/708: write d0/d28/d98/d1e/d86/fa9 [6997470,103476] 0 2024-02-16T22:22:13.515 INFO:tasks.workunit.client.0.smithi114.stdout:2/737: link dd/d4e/fbb dd/d6b/ff5 0 2024-02-16T22:22:13.516 INFO:tasks.workunit.client.1.smithi123.stdout:7/962: symlink d3/d6e/d1d/db2/d3e/de4/d103/l140 0 2024-02-16T22:22:13.516 INFO:tasks.workunit.client.1.smithi123.stdout:7/963: stat d3/d139/d13e 0 2024-02-16T22:22:13.517 INFO:tasks.workunit.client.0.smithi114.stdout:5/709: read d0/d28/d98/d25/d2e/f7b [185530,51656] 0 2024-02-16T22:22:13.518 INFO:tasks.workunit.client.1.smithi123.stdout:1/914: creat d3/d6/db/d10/d2d/d8c/dfd/f130 x:0 0 0 2024-02-16T22:22:13.519 INFO:tasks.workunit.client.1.smithi123.stdout:1/915: fsync d3/d6/db/d10/d2d/d8c/ff3 0 2024-02-16T22:22:13.523 INFO:tasks.workunit.client.1.smithi123.stdout:5/932: fsync d6/dd/df/d58/d89/fed 0 2024-02-16T22:22:13.525 INFO:tasks.workunit.client.0.smithi114.stdout:4/850: mkdir dd/d59/d8f/dea/d134 0 2024-02-16T22:22:13.526 INFO:tasks.workunit.client.1.smithi123.stdout:8/967: getdents d0/d29/da7/d26/d32 0 2024-02-16T22:22:13.529 INFO:tasks.workunit.client.1.smithi123.stdout:6/931: symlink d0/l14d 0 2024-02-16T22:22:13.531 INFO:tasks.workunit.client.1.smithi123.stdout:7/964: mknod d3/d6e/d1d/db2/d3d/db1/c141 0 2024-02-16T22:22:13.535 INFO:tasks.workunit.client.0.smithi114.stdout:2/738: mkdir dd/d6b/da5/df6 0 2024-02-16T22:22:13.538 INFO:tasks.workunit.client.1.smithi123.stdout:2/924: creat d4/d1a/d1d/d59/d45/f12f x:0 0 0 2024-02-16T22:22:13.539 INFO:tasks.workunit.client.1.smithi123.stdout:2/925: write d4/d1a/d1d/d38/fe6 [8481448,85275] 0 2024-02-16T22:22:13.540 INFO:tasks.workunit.client.0.smithi114.stdout:2/739: dwrite dd/d4e/f9c [0,4194304] 0 2024-02-16T22:22:13.546 INFO:tasks.workunit.client.1.smithi123.stdout:5/933: symlink d6/dd/df/d1f/ddc/l148 0 2024-02-16T22:22:13.551 INFO:tasks.workunit.client.0.smithi114.stdout:8/727: rename d4/d11/d52/l58 to d4/d24/d2c/d67/l105 0 2024-02-16T22:22:13.551 INFO:tasks.workunit.client.0.smithi114.stdout:8/728: chown d4/d16/d25/l72 23365072 1 2024-02-16T22:22:13.553 INFO:tasks.workunit.client.1.smithi123.stdout:8/968: creat d0/d29/da7/d43/d34/f13e x:0 0 0 2024-02-16T22:22:13.554 INFO:tasks.workunit.client.1.smithi123.stdout:6/932: chown d0/d9/dc/cc7 21 1 2024-02-16T22:22:13.557 INFO:tasks.workunit.client.1.smithi123.stdout:3/947: rename d0/d40/dfb/c10d to d0/d50/d7e/c136 0 2024-02-16T22:22:13.559 INFO:tasks.workunit.client.0.smithi114.stdout:7/822: dwrite d3/d22/d73/da7/d6f/d89/fb3 [0,4194304] 0 2024-02-16T22:22:13.564 INFO:tasks.workunit.client.1.smithi123.stdout:2/926: dread d4/d1a/d1d/d38/f90 [4194304,4194304] 0 2024-02-16T22:22:13.566 INFO:tasks.workunit.client.1.smithi123.stdout:1/916: symlink d3/d6/db/d10/d2d/d8c/dfd/d9f/db4/ddf/df0/d129/l131 0 2024-02-16T22:22:13.570 INFO:tasks.workunit.client.1.smithi123.stdout:1/917: dread d3/d6/d11/fe4 [0,4194304] 0 2024-02-16T22:22:13.571 INFO:tasks.workunit.client.1.smithi123.stdout:5/934: unlink d6/dd/d44/f47 0 2024-02-16T22:22:13.572 INFO:tasks.workunit.client.1.smithi123.stdout:1/918: write d3/d6/d49/d48/d84/d92/ffe [4904543,17595] 0 2024-02-16T22:22:13.575 INFO:tasks.workunit.client.1.smithi123.stdout:8/969: creat d0/d29/da7/d26/d2a/d123/dca/f13f x:0 0 0 2024-02-16T22:22:13.579 INFO:tasks.workunit.client.1.smithi123.stdout:3/948: readlink d0/l18 0 2024-02-16T22:22:13.581 INFO:tasks.workunit.client.1.smithi123.stdout:2/927: mkdir d4/d1a/d1d/d13/d25/d130 0 2024-02-16T22:22:13.582 INFO:tasks.workunit.client.1.smithi123.stdout:2/928: write d4/d1a/dd5/d10b/df1/db1/f119 [827709,109558] 0 2024-02-16T22:22:13.583 INFO:tasks.workunit.client.1.smithi123.stdout:2/929: chown d4/d1a/dd5/d10b/df1/fcd 877399 1 2024-02-16T22:22:13.585 INFO:tasks.workunit.client.1.smithi123.stdout:2/930: write d4/d1a/dd5/d10b/df1/db1/f119 [899706,48336] 0 2024-02-16T22:22:13.586 INFO:tasks.workunit.client.1.smithi123.stdout:0/881: getdents d1/d4/d17/d74 0 2024-02-16T22:22:13.586 INFO:tasks.workunit.client.1.smithi123.stdout:6/933: dread d0/d6/d4d/db0/ffb [0,4194304] 0 2024-02-16T22:22:13.587 INFO:tasks.workunit.client.1.smithi123.stdout:0/882: read - d1/d8/d13/d1d/f6f zero size 2024-02-16T22:22:13.588 INFO:tasks.workunit.client.1.smithi123.stdout:0/883: chown d1/d8/d13/d1d/d62/cbe 162249 1 2024-02-16T22:22:13.590 INFO:tasks.workunit.client.1.smithi123.stdout:8/970: rmdir d0/d29/da7/d26/d68/d81/d47/d9b 39 2024-02-16T22:22:13.594 INFO:tasks.workunit.client.1.smithi123.stdout:3/949: unlink d0/d20/d29/d9b/dbd/df5/c87 0 2024-02-16T22:22:13.601 INFO:tasks.workunit.client.1.smithi123.stdout:2/931: creat d4/d1a/d1d/d38/d7b/f131 x:0 0 0 2024-02-16T22:22:13.602 INFO:tasks.workunit.client.1.smithi123.stdout:2/932: chown d4/d1a/d1d/d59/dd7/ddc/c81 446 1 2024-02-16T22:22:13.602 INFO:tasks.workunit.client.1.smithi123.stdout:1/919: mkdir d3/d6/d49/d48/d84/d92/df9/d132 0 2024-02-16T22:22:13.604 INFO:tasks.workunit.client.1.smithi123.stdout:6/934: creat d0/d9/dc/d39/d8d/f14e x:0 0 0 2024-02-16T22:22:13.606 INFO:tasks.workunit.client.1.smithi123.stdout:7/965: getdents d3/d6e 0 2024-02-16T22:22:13.607 INFO:tasks.workunit.client.0.smithi114.stdout:2/740: creat dd/df/ddb/ff7 x:0 0 0 2024-02-16T22:22:13.609 INFO:tasks.workunit.client.0.smithi114.stdout:0/795: creat d1/d1a/ffa x:0 0 0 2024-02-16T22:22:13.612 INFO:tasks.workunit.client.1.smithi123.stdout:5/935: link d6/dd/df/d1f/l4a d6/dd/df/d1f/ddc/d145/l149 0 2024-02-16T22:22:13.613 INFO:tasks.workunit.client.1.smithi123.stdout:5/936: truncate d6/dd/df/d58/d56/d141/f13c 973757 0 2024-02-16T22:22:13.614 INFO:tasks.workunit.client.1.smithi123.stdout:5/937: fdatasync d6/dd/df/d58/d3f/f9e 0 2024-02-16T22:22:13.616 INFO:tasks.workunit.client.1.smithi123.stdout:1/920: creat d3/d6/db/d10/d2d/d8c/dfd/d9f/f133 x:0 0 0 2024-02-16T22:22:13.619 INFO:tasks.workunit.client.0.smithi114.stdout:1/705: rename d4/d63/d69/d7c/dc0/fe3 to d4/d9/d46/d67/d92/dca/dcc/d70/fff 0 2024-02-16T22:22:13.623 INFO:tasks.workunit.client.1.smithi123.stdout:6/935: mkdir d0/d96/de3/dcc/d14f 0 2024-02-16T22:22:13.624 INFO:tasks.workunit.client.1.smithi123.stdout:8/971: write d0/d29/da7/d26/d2a/d123/dca/fe2 [3408580,63830] 0 2024-02-16T22:22:13.625 INFO:tasks.workunit.client.0.smithi114.stdout:7/823: truncate d3/d22/d73/da7/f52 1320701 0 2024-02-16T22:22:13.627 INFO:tasks.workunit.client.0.smithi114.stdout:6/750: link d6/dd/l55 d6/dd/d25/d34/d58/lde 0 2024-02-16T22:22:13.627 INFO:tasks.workunit.client.0.smithi114.stdout:7/824: read d3/d14/f4d [3775244,124792] 0 2024-02-16T22:22:13.628 INFO:tasks.workunit.client.0.smithi114.stdout:6/751: chown d6/dd/d25/d34/f6f 3984230 1 2024-02-16T22:22:13.628 INFO:tasks.workunit.client.0.smithi114.stdout:7/825: chown d3/d22/d61/c10e 1305 1 2024-02-16T22:22:13.628 INFO:tasks.workunit.client.1.smithi123.stdout:6/936: dwrite d0/d9/dc/d3a/f80 [4194304,4194304] 0 2024-02-16T22:22:13.629 INFO:tasks.workunit.client.1.smithi123.stdout:3/950: rename d0/d20/d29/d9b/dbd/df4/d5d/dae/db0/d123/d127 to d0/d20/d2e/d137 0 2024-02-16T22:22:13.632 INFO:tasks.workunit.client.1.smithi123.stdout:8/972: dwrite d0/d29/da7/d26/d2a/d4d/f57 [0,4194304] 0 2024-02-16T22:22:13.633 INFO:tasks.workunit.client.0.smithi114.stdout:7/826: dread d3/d22/f48 [4194304,4194304] 0 2024-02-16T22:22:13.635 INFO:tasks.workunit.client.1.smithi123.stdout:0/884: truncate d1/d4/d17/d74/fc1 3808576 0 2024-02-16T22:22:13.636 INFO:tasks.workunit.client.1.smithi123.stdout:0/885: write d1/da0/f119 [928668,73293] 0 2024-02-16T22:22:13.640 INFO:tasks.workunit.client.1.smithi123.stdout:6/937: dread d0/d9/dc/d39/f44 [0,4194304] 0 2024-02-16T22:22:13.641 INFO:tasks.workunit.client.0.smithi114.stdout:9/749: getdents d2/d33/d3f 0 2024-02-16T22:22:13.643 INFO:tasks.workunit.client.0.smithi114.stdout:9/750: write d2/d5c/d5f/d91/de4/fed [334052,78588] 0 2024-02-16T22:22:13.644 INFO:tasks.workunit.client.1.smithi123.stdout:1/921: symlink d3/d6/d49/d48/d68/l134 0 2024-02-16T22:22:13.645 INFO:tasks.workunit.client.0.smithi114.stdout:2/741: write dd/df/d1c/d21/d61/d69/f98 [464317,108563] 0 2024-02-16T22:22:13.648 INFO:tasks.workunit.client.1.smithi123.stdout:2/933: dread d4/d1a/d1d/d13/d25/d98/d10c/d11b/f112 [0,4194304] 0 2024-02-16T22:22:13.650 INFO:tasks.workunit.client.0.smithi114.stdout:1/706: rmdir d4/d9/d46/d47/db3 39 2024-02-16T22:22:13.652 INFO:tasks.workunit.client.0.smithi114.stdout:8/729: truncate d4/d11/d78/d7a/d36/f5d 1568631 0 2024-02-16T22:22:13.653 INFO:tasks.workunit.client.0.smithi114.stdout:8/730: dread - d4/d24/d2c/d5f/fa2 zero size 2024-02-16T22:22:13.654 INFO:tasks.workunit.client.0.smithi114.stdout:8/731: write d4/d11/d78/d7a/d36/d3b/f100 [583690,85375] 0 2024-02-16T22:22:13.655 INFO:tasks.workunit.client.1.smithi123.stdout:5/938: dwrite d6/dd/d78/da5/db0/f13b [0,4194304] 0 2024-02-16T22:22:13.657 INFO:tasks.workunit.client.0.smithi114.stdout:6/752: unlink d6/dd/d25/d34/d61/f6a 0 2024-02-16T22:22:13.659 INFO:tasks.workunit.client.1.smithi123.stdout:3/951: symlink d0/d20/d29/df6/l138 0 2024-02-16T22:22:13.662 INFO:tasks.workunit.client.1.smithi123.stdout:8/973: creat d0/d29/da7/d26/d2a/d123/dca/f140 x:0 0 0 2024-02-16T22:22:13.662 INFO:tasks.workunit.client.1.smithi123.stdout:5/939: dwrite d6/dd/df/d58/f4f [4194304,4194304] 0 2024-02-16T22:22:13.665 INFO:tasks.workunit.client.0.smithi114.stdout:7/827: mknod d3/d22/db7/dea/c125 0 2024-02-16T22:22:13.666 INFO:tasks.workunit.client.1.smithi123.stdout:1/922: dread d3/d6/db/d10/d4b/d70/d87/fd6 [0,4194304] 0 2024-02-16T22:22:13.668 INFO:tasks.workunit.client.0.smithi114.stdout:5/710: getdents d0/df/d7a/dc4 0 2024-02-16T22:22:13.672 INFO:tasks.workunit.client.0.smithi114.stdout:2/742: mkdir dd/df/d1c/d21/d61/d63/d83/df8 0 2024-02-16T22:22:13.673 INFO:tasks.workunit.client.0.smithi114.stdout:0/796: write d1/d1a/f38 [876912,13615] 0 2024-02-16T22:22:13.674 INFO:tasks.workunit.client.0.smithi114.stdout:1/707: dread - d4/d63/fcb zero size 2024-02-16T22:22:13.675 INFO:tasks.workunit.client.1.smithi123.stdout:2/934: mknod d4/d1a/d1d/d13/d25/d98/d10c/d11b/c132 0 2024-02-16T22:22:13.679 INFO:tasks.workunit.client.1.smithi123.stdout:0/886: write d1/f22 [809386,59028] 0 2024-02-16T22:22:13.680 INFO:tasks.workunit.client.1.smithi123.stdout:0/887: chown d1/d8/d13/d1d/d64 186 1 2024-02-16T22:22:13.684 INFO:tasks.workunit.client.1.smithi123.stdout:3/952: mkdir d0/d20/d29/d9b/d139 0 2024-02-16T22:22:13.687 INFO:tasks.workunit.client.0.smithi114.stdout:5/711: rename d0/d28/d98/d25/d3e/l50 to d0/df/d6d/le4 0 2024-02-16T22:22:13.688 INFO:tasks.workunit.client.1.smithi123.stdout:8/974: dread d0/d12/f74 [0,4194304] 0 2024-02-16T22:22:13.688 INFO:tasks.workunit.client.1.smithi123.stdout:5/940: symlink d6/dd/d78/l14a 0 2024-02-16T22:22:13.689 INFO:tasks.workunit.client.1.smithi123.stdout:8/975: chown d0/d61/d66/da5/fe6 1086345238 1 2024-02-16T22:22:13.690 INFO:tasks.workunit.client.0.smithi114.stdout:0/797: creat d1/d16/d18/ffb x:0 0 0 2024-02-16T22:22:13.690 INFO:tasks.workunit.client.1.smithi123.stdout:8/976: read - d0/d29/da7/d26/d2a/d77/f116 zero size 2024-02-16T22:22:13.694 INFO:tasks.workunit.client.0.smithi114.stdout:7/828: dread d3/d22/d73/da7/d6f/d89/da6/dfc/f102 [0,4194304] 0 2024-02-16T22:22:13.696 INFO:tasks.workunit.client.0.smithi114.stdout:8/732: symlink d4/d11/d9d/l106 0 2024-02-16T22:22:13.696 INFO:tasks.workunit.client.0.smithi114.stdout:7/829: write d3/d3d/f86 [4050450,109504] 0 2024-02-16T22:22:13.696 INFO:tasks.workunit.client.1.smithi123.stdout:7/966: getdents d3/d5/d6c/da6/df1 0 2024-02-16T22:22:13.697 INFO:tasks.workunit.client.0.smithi114.stdout:7/830: readlink d3/d22/d73/da7/d6f/la5 0 2024-02-16T22:22:13.701 INFO:tasks.workunit.client.0.smithi114.stdout:1/708: dread d4/d9/d3d/d3f/f75 [0,4194304] 0 2024-02-16T22:22:13.702 INFO:tasks.workunit.client.0.smithi114.stdout:5/712: creat d0/d28/d98/d25/d89/de2/fe5 x:0 0 0 2024-02-16T22:22:13.703 INFO:tasks.workunit.client.1.smithi123.stdout:0/888: dread d1/d8/d13/f4d [0,4194304] 0 2024-02-16T22:22:13.709 INFO:tasks.workunit.client.1.smithi123.stdout:8/977: mknod d0/d12/d3e/d7c/c141 0 2024-02-16T22:22:13.710 INFO:tasks.workunit.client.0.smithi114.stdout:0/798: mkdir d1/d16/d1f/d28/d51/d61/d6d/dfc 0 2024-02-16T22:22:13.711 INFO:tasks.workunit.client.1.smithi123.stdout:1/923: mknod d3/d6/db/d55/d12c/c135 0 2024-02-16T22:22:13.712 INFO:tasks.workunit.client.0.smithi114.stdout:2/743: truncate dd/d3b/da7/d4c/f64 387803 0 2024-02-16T22:22:13.713 INFO:tasks.workunit.client.1.smithi123.stdout:2/935: dwrite d4/d1a/d1d/d13/d25/d98/f67 [0,4194304] 0 2024-02-16T22:22:13.717 INFO:tasks.workunit.client.0.smithi114.stdout:1/709: dread d4/d63/d69/f7f [0,4194304] 0 2024-02-16T22:22:13.720 INFO:tasks.workunit.client.1.smithi123.stdout:3/953: link d0/d20/d5a/f126 d0/d20/d5a/da6/f13a 0 2024-02-16T22:22:13.720 INFO:tasks.workunit.client.0.smithi114.stdout:7/831: chown d3/d3d/le8 30375 1 2024-02-16T22:22:13.722 INFO:tasks.workunit.client.0.smithi114.stdout:9/751: getdents d2/d1a/d65 0 2024-02-16T22:22:13.725 INFO:tasks.workunit.client.0.smithi114.stdout:5/713: symlink d0/df/d53/db1/le6 0 2024-02-16T22:22:13.729 INFO:tasks.workunit.client.0.smithi114.stdout:0/799: symlink d1/d16/d1f/d28/lfd 0 2024-02-16T22:22:13.732 INFO:tasks.workunit.client.0.smithi114.stdout:3/718: sync 2024-02-16T22:22:13.732 INFO:tasks.workunit.client.0.smithi114.stdout:8/733: symlink d4/d11/d9d/d104/l107 0 2024-02-16T22:22:13.733 INFO:tasks.workunit.client.0.smithi114.stdout:8/734: write d4/d11/d78/d7a/d36/fed [887892,30575] 0 2024-02-16T22:22:13.734 INFO:tasks.workunit.client.0.smithi114.stdout:8/735: chown d4/d24/d2c/d67 86 1 2024-02-16T22:22:13.735 INFO:tasks.workunit.client.0.smithi114.stdout:6/753: getdents d6 0 2024-02-16T22:22:13.740 INFO:tasks.workunit.client.0.smithi114.stdout:7/832: creat d3/d92/dd5/f126 x:0 0 0 2024-02-16T22:22:13.743 INFO:tasks.workunit.client.1.smithi123.stdout:5/941: dwrite d6/dc1/fdf [0,4194304] 0 2024-02-16T22:22:13.745 INFO:tasks.workunit.client.1.smithi123.stdout:6/938: getdents d0 0 2024-02-16T22:22:13.745 INFO:tasks.workunit.client.1.smithi123.stdout:7/967: creat d3/d6e/d1d/d97/dca/f142 x:0 0 0 2024-02-16T22:22:13.746 INFO:tasks.workunit.client.0.smithi114.stdout:5/714: creat d0/d28/d98/d25/d89/d99/db0/fe7 x:0 0 0 2024-02-16T22:22:13.746 INFO:tasks.workunit.client.0.smithi114.stdout:1/710: write d4/d9/d46/d67/d92/dca/dcc/d98/fcd [1129550,129709] 0 2024-02-16T22:22:13.747 INFO:tasks.workunit.client.0.smithi114.stdout:0/800: symlink d1/d1a/d47/d62/d7f/lfe 0 2024-02-16T22:22:13.748 INFO:tasks.workunit.client.1.smithi123.stdout:7/968: chown d3/d6e/d4d/d92/d12b/f117 17190153 1 2024-02-16T22:22:13.748 INFO:tasks.workunit.client.0.smithi114.stdout:2/744: getdents dd/d6b/da5/df6 0 2024-02-16T22:22:13.751 INFO:tasks.workunit.client.0.smithi114.stdout:8/736: fdatasync d4/fdb 0 2024-02-16T22:22:13.753 INFO:tasks.workunit.client.0.smithi114.stdout:8/737: write d4/d24/d2c/d5f/f93 [3259817,27830] 0 2024-02-16T22:22:13.760 INFO:tasks.workunit.client.0.smithi114.stdout:9/752: mknod d2/d5c/cf1 0 2024-02-16T22:22:13.764 INFO:tasks.workunit.client.0.smithi114.stdout:5/715: read d0/d28/d98/d25/d3e/d7f/fa8 [4022939,94175] 0 2024-02-16T22:22:13.774 INFO:tasks.workunit.client.0.smithi114.stdout:8/738: read - d4/d24/d6d/ff2 zero size 2024-02-16T22:22:13.777 INFO:tasks.workunit.client.0.smithi114.stdout:6/754: dwrite d6/dd/d25/d34/d58/faf [0,4194304] 0 2024-02-16T22:22:13.777 INFO:tasks.workunit.client.0.smithi114.stdout:4/851: sync 2024-02-16T22:22:13.777 INFO:tasks.workunit.client.0.smithi114.stdout:8/739: truncate d4/d16/d25/d39/f82 1709377 0 2024-02-16T22:22:13.778 INFO:tasks.workunit.client.0.smithi114.stdout:7/833: write d3/d22/d73/da7/f5e [760464,41839] 0 2024-02-16T22:22:13.781 INFO:tasks.workunit.client.0.smithi114.stdout:9/753: dread d2/d1a/d23/f28 [0,4194304] 0 2024-02-16T22:22:13.782 INFO:tasks.workunit.client.0.smithi114.stdout:1/711: symlink d4/d9/l100 0 2024-02-16T22:22:13.790 INFO:tasks.workunit.client.0.smithi114.stdout:0/801: mkdir d1/d1a/d47/d62/dff 0 2024-02-16T22:22:13.794 INFO:tasks.workunit.client.0.smithi114.stdout:3/719: mkdir d4/d26/dd7/df4 0 2024-02-16T22:22:13.797 INFO:tasks.workunit.client.0.smithi114.stdout:6/755: fdatasync d6/dd/d25/d34/d61/fd5 0 2024-02-16T22:22:13.804 INFO:tasks.workunit.client.0.smithi114.stdout:8/740: dwrite d4/d16/fbf [0,4194304] 0 2024-02-16T22:22:13.807 INFO:tasks.workunit.client.1.smithi123.stdout:1/924: sync 2024-02-16T22:22:13.808 INFO:tasks.workunit.client.0.smithi114.stdout:7/834: creat d3/d46/da2/d114/f127 x:0 0 0 2024-02-16T22:22:13.812 INFO:tasks.workunit.client.0.smithi114.stdout:9/754: truncate d2/d5c/d5f/d91/fae 353495 0 2024-02-16T22:22:13.815 INFO:tasks.workunit.client.0.smithi114.stdout:2/745: truncate dd/d6b/fbf 2296428 0 2024-02-16T22:22:13.817 INFO:tasks.workunit.client.0.smithi114.stdout:5/716: creat d0/df/d7a/dc4/dda/fe8 x:0 0 0 2024-02-16T22:22:13.826 INFO:tasks.workunit.client.0.smithi114.stdout:4/852: getdents dd/d59/d8f/dea/d114 0 2024-02-16T22:22:13.826 INFO:tasks.workunit.client.1.smithi123.stdout:1/925: dread d3/d6/d11/f1f [0,4194304] 0 2024-02-16T22:22:13.827 INFO:tasks.workunit.client.0.smithi114.stdout:4/853: read - dd/d1b/d24/d57/d12b/d38/f96 zero size 2024-02-16T22:22:13.828 INFO:tasks.workunit.client.0.smithi114.stdout:4/854: readlink dd/d1b/d41/dc3/dfd/l124 0 2024-02-16T22:22:13.835 INFO:tasks.workunit.client.0.smithi114.stdout:1/712: dwrite d4/d9/d3d/f8e [0,4194304] 0 2024-02-16T22:22:13.841 INFO:tasks.workunit.client.0.smithi114.stdout:9/755: mkdir d2/dd/d3c/d89/d8b/df2 0 2024-02-16T22:22:13.841 INFO:tasks.workunit.client.0.smithi114.stdout:9/756: truncate d2/d5c/dc9/d47/db2/fc3 623816 0 2024-02-16T22:22:13.845 INFO:tasks.workunit.client.0.smithi114.stdout:2/746: unlink dd/df/d1c/d21/c38 0 2024-02-16T22:22:13.846 INFO:tasks.workunit.client.0.smithi114.stdout:7/835: write d3/d14/f43 [2119997,64045] 0 2024-02-16T22:22:13.846 INFO:tasks.workunit.client.0.smithi114.stdout:3/720: dwrite d4/dd/d33/d72/f9b [0,4194304] 0 2024-02-16T22:22:13.852 INFO:tasks.workunit.client.0.smithi114.stdout:2/747: write dd/d3b/da7/d8b/dc0/fcf [674575,53113] 0 2024-02-16T22:22:13.857 INFO:tasks.workunit.client.0.smithi114.stdout:6/756: rename d6/dd/d25/f73 to d6/d2c/d75/fdf 0 2024-02-16T22:22:13.861 INFO:tasks.workunit.client.0.smithi114.stdout:5/717: write d0/df/d5e/f9c [1008810,48259] 0 2024-02-16T22:22:13.866 INFO:tasks.workunit.client.0.smithi114.stdout:9/757: mknod d2/d1a/d23/d32/d46/d78/cf3 0 2024-02-16T22:22:13.867 INFO:tasks.workunit.client.0.smithi114.stdout:5/718: dwrite d0/d28/d98/d1e/f29 [4194304,4194304] 0 2024-02-16T22:22:13.872 INFO:tasks.workunit.client.0.smithi114.stdout:3/721: mknod d4/dd/d33/d76/cf5 0 2024-02-16T22:22:13.877 INFO:tasks.workunit.client.0.smithi114.stdout:2/748: mkdir dd/df/d1c/d21/d61/df9 0 2024-02-16T22:22:13.881 INFO:tasks.workunit.client.0.smithi114.stdout:1/713: rename d4/d63/d6b to d4/d9/d46/d67/d92/dca/ded/d101 0 2024-02-16T22:22:13.886 INFO:tasks.workunit.client.1.smithi123.stdout:0/889: symlink d1/l125 0 2024-02-16T22:22:13.888 INFO:tasks.workunit.client.1.smithi123.stdout:8/978: rename d0/c18 to d0/d61/dac/dd0/c142 0 2024-02-16T22:22:13.890 INFO:tasks.workunit.client.0.smithi114.stdout:9/758: creat d2/d33/ff4 x:0 0 0 2024-02-16T22:22:13.891 INFO:tasks.workunit.client.1.smithi123.stdout:1/926: sync 2024-02-16T22:22:13.896 INFO:tasks.workunit.client.1.smithi123.stdout:3/954: symlink d0/d20/d29/d9b/dbd/df4/d5d/d60/d93/l13b 0 2024-02-16T22:22:13.897 INFO:tasks.workunit.client.1.smithi123.stdout:3/955: read - d0/d20/d29/d9b/dbd/df4/d5d/d60/d93/f116 zero size 2024-02-16T22:22:13.898 INFO:tasks.workunit.client.0.smithi114.stdout:7/836: link d3/d22/d73/da7/d6f/d9c/fed d3/d22/d73/da7/d6f/df8/f128 0 2024-02-16T22:22:13.901 INFO:tasks.workunit.client.0.smithi114.stdout:0/802: getdents d1/d1a/d47/d62/d7f/d9d 0 2024-02-16T22:22:13.903 INFO:tasks.workunit.client.1.smithi123.stdout:7/969: creat d3/d5/d6c/da6/df1/f143 x:0 0 0 2024-02-16T22:22:13.904 INFO:tasks.workunit.client.0.smithi114.stdout:6/757: rmdir d6/d2c/d38/d41/d60/d84 39 2024-02-16T22:22:13.913 INFO:tasks.workunit.client.1.smithi123.stdout:1/927: mkdir d3/d6/d49/d48/d84/d136 0 2024-02-16T22:22:13.914 INFO:tasks.workunit.client.0.smithi114.stdout:8/741: link d4/d16/d25/d46/fc2 d4/d24/d6d/f108 0 2024-02-16T22:22:13.916 INFO:tasks.workunit.client.0.smithi114.stdout:4/855: link dd/d1b/f5b dd/d1b/d41/d74/f135 0 2024-02-16T22:22:13.917 INFO:tasks.workunit.client.1.smithi123.stdout:3/956: mkdir d0/d20/d29/d9b/dbd/df4/d5d/d60/d13c 0 2024-02-16T22:22:13.918 INFO:tasks.workunit.client.1.smithi123.stdout:3/957: stat d0/d20/d29/d9b/dbd 0 2024-02-16T22:22:13.918 INFO:tasks.workunit.client.1.smithi123.stdout:3/958: write d0/d20/d29/f84 [1339471,69888] 0 2024-02-16T22:22:13.919 INFO:tasks.workunit.client.0.smithi114.stdout:9/759: unlink d2/dd/d3c/f3d 0 2024-02-16T22:22:13.920 INFO:tasks.workunit.client.1.smithi123.stdout:2/936: write d4/d1a/d1d/d13/d25/f43 [1303595,104042] 0 2024-02-16T22:22:13.921 INFO:tasks.workunit.client.1.smithi123.stdout:2/937: write d4/d1a/dd5/d120/f122 [1019249,3151] 0 2024-02-16T22:22:13.927 INFO:tasks.workunit.client.0.smithi114.stdout:1/714: write d4/d9/d46/d4d/f5d [596035,80913] 0 2024-02-16T22:22:13.928 INFO:tasks.workunit.client.1.smithi123.stdout:8/979: chown d0/d29/da7/d26/d68/d81/d8f/ff2 251311 1 2024-02-16T22:22:13.928 INFO:tasks.workunit.client.0.smithi114.stdout:1/715: readlink d4/d63/d69/d80/le2 0 2024-02-16T22:22:13.929 INFO:tasks.workunit.client.1.smithi123.stdout:6/939: dwrite d0/d9/dc/f3f [0,4194304] 0 2024-02-16T22:22:13.930 INFO:tasks.workunit.client.1.smithi123.stdout:8/980: dread d0/d29/da7/d26/d68/d81/dd7/f11f [0,4194304] 0 2024-02-16T22:22:13.933 INFO:tasks.workunit.client.0.smithi114.stdout:0/803: creat d1/d16/d1f/d28/d51/d61/d6d/f100 x:0 0 0 2024-02-16T22:22:13.933 INFO:tasks.workunit.client.0.smithi114.stdout:3/722: truncate d4/d24/f67 1157096 0 2024-02-16T22:22:13.937 INFO:tasks.workunit.client.0.smithi114.stdout:7/837: write d3/d14/fc4 [9508616,103525] 0 2024-02-16T22:22:13.937 INFO:tasks.workunit.client.0.smithi114.stdout:2/749: symlink dd/df/d1c/d21/d61/dec/lfa 0 2024-02-16T22:22:13.939 INFO:tasks.workunit.client.0.smithi114.stdout:7/838: write d3/d3d/f86 [4053076,32630] 0 2024-02-16T22:22:13.939 INFO:tasks.workunit.client.0.smithi114.stdout:8/742: mkdir d4/d16/d25/d46/db5/d109 0 2024-02-16T22:22:13.941 INFO:tasks.workunit.client.0.smithi114.stdout:4/856: rmdir dd/d59 39 2024-02-16T22:22:13.943 INFO:tasks.workunit.client.0.smithi114.stdout:2/750: truncate dd/df/d1c/d21/d61/d63/d83/fb6 4205016 0 2024-02-16T22:22:13.943 INFO:tasks.workunit.client.1.smithi123.stdout:2/938: mknod d4/d1a/d1d/d13/d25/d98/c133 0 2024-02-16T22:22:13.944 INFO:tasks.workunit.client.0.smithi114.stdout:2/751: chown dd/df/d1c/fca 20971 1 2024-02-16T22:22:13.944 INFO:tasks.workunit.client.1.smithi123.stdout:0/890: link d1/d8/d13/d1d/f6f d1/d8/d13/f126 0 2024-02-16T22:22:13.944 INFO:tasks.workunit.client.0.smithi114.stdout:2/752: readlink dd/df/d1c/d21/l22 0 2024-02-16T22:22:13.945 INFO:tasks.workunit.client.1.smithi123.stdout:0/891: chown d1/d8/cde 2399390 1 2024-02-16T22:22:13.949 INFO:tasks.workunit.client.1.smithi123.stdout:6/940: rename d0/d9/dc/d3a/f108 to d0/d6/d1a/daf/f150 0 2024-02-16T22:22:13.950 INFO:tasks.workunit.client.0.smithi114.stdout:1/716: mknod d4/d9/d46/d67/d92/dca/dcc/d70/c102 0 2024-02-16T22:22:13.952 INFO:tasks.workunit.client.1.smithi123.stdout:8/981: creat d0/d29/da7/d26/d2a/d77/f143 x:0 0 0 2024-02-16T22:22:13.952 INFO:tasks.workunit.client.1.smithi123.stdout:8/982: stat d0/d12/f58 0 2024-02-16T22:22:13.956 INFO:tasks.workunit.client.1.smithi123.stdout:5/942: rmdir d6/dd/df/d116 0 2024-02-16T22:22:13.957 INFO:tasks.workunit.client.1.smithi123.stdout:5/943: write d6/dd/df/d58/d3f/dbe/d37/f52 [323145,54330] 0 2024-02-16T22:22:13.958 INFO:tasks.workunit.client.1.smithi123.stdout:1/928: creat d3/d6/d49/df5/f137 x:0 0 0 2024-02-16T22:22:13.961 INFO:tasks.workunit.client.0.smithi114.stdout:6/758: symlink d6/d2c/d38/d41/le0 0 2024-02-16T22:22:13.962 INFO:tasks.workunit.client.1.smithi123.stdout:3/959: symlink d0/l13d 0 2024-02-16T22:22:13.962 INFO:tasks.workunit.client.1.smithi123.stdout:7/970: dwrite d3/d6e/d1d/db2/d3e/de4/d103/f133 [0,4194304] 0 2024-02-16T22:22:13.967 INFO:tasks.workunit.client.0.smithi114.stdout:7/839: mknod d3/d22/d76/c129 0 2024-02-16T22:22:13.968 INFO:tasks.workunit.client.1.smithi123.stdout:7/971: dwrite d3/d6e/d1d/fb6 [0,4194304] 0 2024-02-16T22:22:13.970 INFO:tasks.workunit.client.0.smithi114.stdout:8/743: chown d4/d16/d25/d46/lf8 35 1 2024-02-16T22:22:13.974 INFO:tasks.workunit.client.0.smithi114.stdout:2/753: mknod dd/df/d1c/d21/d61/d69/cfb 0 2024-02-16T22:22:13.975 INFO:tasks.workunit.client.1.smithi123.stdout:6/941: chown d0/d9/dc/d39/c47 42 1 2024-02-16T22:22:13.975 INFO:tasks.workunit.client.0.smithi114.stdout:2/754: chown dd/fd1 7686 1 2024-02-16T22:22:13.977 INFO:tasks.workunit.client.0.smithi114.stdout:2/755: write dd/d3b/da7/d4c/faf [572686,14908] 0 2024-02-16T22:22:13.980 INFO:tasks.workunit.client.0.smithi114.stdout:5/719: getdents d0/d28/d98/dd0 0 2024-02-16T22:22:13.982 INFO:tasks.workunit.client.1.smithi123.stdout:1/929: fsync d3/d6/db/d10/fba 0 2024-02-16T22:22:13.983 INFO:tasks.workunit.client.0.smithi114.stdout:3/723: creat d4/d26/dd7/df4/ff6 x:0 0 0 2024-02-16T22:22:13.984 INFO:tasks.workunit.client.0.smithi114.stdout:3/724: stat d4/dd/l71 0 2024-02-16T22:22:13.992 INFO:tasks.workunit.client.0.smithi114.stdout:8/744: creat d4/d11/d78/d7a/d31/f10a x:0 0 0 2024-02-16T22:22:13.996 INFO:tasks.workunit.client.1.smithi123.stdout:0/892: truncate d1/d4/d17/d74/fc1 1401506 0 2024-02-16T22:22:13.997 INFO:tasks.workunit.client.1.smithi123.stdout:0/893: chown d1/d4/d17/d74/d57 2753966 1 2024-02-16T22:22:13.997 INFO:tasks.workunit.client.1.smithi123.stdout:0/894: chown d1/d8/d3d 1 1 2024-02-16T22:22:13.998 INFO:tasks.workunit.client.1.smithi123.stdout:0/895: write d1/d4/d17/d54/f99 [97012,21388] 0 2024-02-16T22:22:13.999 INFO:tasks.workunit.client.1.smithi123.stdout:8/983: dread - d0/d29/da7/d26/d68/d81/d47/d9b/ff0 zero size 2024-02-16T22:22:14.001 INFO:tasks.workunit.client.0.smithi114.stdout:4/857: dwrite dd/d1b/d24/f76 [0,4194304] 0 2024-02-16T22:22:14.001 INFO:tasks.workunit.client.1.smithi123.stdout:8/984: write d0/d61/d66/d7e/f89 [3844668,40711] 0 2024-02-16T22:22:14.004 INFO:tasks.workunit.client.1.smithi123.stdout:5/944: mkdir d6/d14b 0 2024-02-16T22:22:14.006 INFO:tasks.workunit.client.0.smithi114.stdout:1/717: truncate d4/d9/d3d/d3f/f5a 4134265 0 2024-02-16T22:22:14.006 INFO:tasks.workunit.client.0.smithi114.stdout:9/760: dwrite d2/d33/d3f/fb5 [0,4194304] 0 2024-02-16T22:22:14.007 INFO:tasks.workunit.client.1.smithi123.stdout:8/985: dwrite d0/d29/da7/d26/d2a/d77/f116 [0,4194304] 0 2024-02-16T22:22:14.009 INFO:tasks.workunit.client.0.smithi114.stdout:1/718: stat d4/d63/d69/d7c/d9b/db0/c40 0 2024-02-16T22:22:14.009 INFO:tasks.workunit.client.0.smithi114.stdout:3/725: mknod d4/d87/cf7 0 2024-02-16T22:22:14.010 INFO:tasks.workunit.client.0.smithi114.stdout:2/756: write dd/f92 [2550244,87190] 0 2024-02-16T22:22:14.010 INFO:tasks.workunit.client.1.smithi123.stdout:1/930: rmdir d3/d6/d49 39 2024-02-16T22:22:14.015 INFO:tasks.workunit.client.0.smithi114.stdout:6/759: fsync d6/d2c/f46 0 2024-02-16T22:22:14.019 INFO:tasks.workunit.client.1.smithi123.stdout:6/942: dwrite d0/d6/d1a/d2e/f77 [0,4194304] 0 2024-02-16T22:22:14.019 INFO:tasks.workunit.client.1.smithi123.stdout:3/960: mkdir d0/d20/d29/df6/d13e 0 2024-02-16T22:22:14.021 INFO:tasks.workunit.client.0.smithi114.stdout:7/840: dwrite d3/d14/faa [0,4194304] 0 2024-02-16T22:22:14.024 INFO:tasks.workunit.client.1.smithi123.stdout:7/972: truncate d3/d5/fb8 334623 0 2024-02-16T22:22:14.026 INFO:tasks.workunit.client.1.smithi123.stdout:7/973: write d3/d5/d6c/da6/df1/f143 [655179,98451] 0 2024-02-16T22:22:14.028 INFO:tasks.workunit.client.0.smithi114.stdout:8/745: chown d4/d16/d25/d39/c6a 14888 1 2024-02-16T22:22:14.028 INFO:tasks.workunit.client.0.smithi114.stdout:1/719: dwrite d4/d63/d69/d72/fde [0,4194304] 0 2024-02-16T22:22:14.032 INFO:tasks.workunit.client.0.smithi114.stdout:5/720: dwrite d0/f8 [0,4194304] 0 2024-02-16T22:22:14.036 INFO:tasks.workunit.client.0.smithi114.stdout:0/804: getdents d1/d1a/d3e 0 2024-02-16T22:22:14.037 INFO:tasks.workunit.client.0.smithi114.stdout:4/858: rmdir dd/d1b/d41/d74/d9b 39 2024-02-16T22:22:14.041 INFO:tasks.workunit.client.0.smithi114.stdout:5/721: dwrite d0/df/f8a [0,4194304] 0 2024-02-16T22:22:14.041 INFO:tasks.workunit.client.1.smithi123.stdout:5/945: rmdir d6/dd/df/d58/d3f/dbe/d37 39 2024-02-16T22:22:14.051 INFO:tasks.workunit.client.1.smithi123.stdout:3/961: fsync d0/fc 0 2024-02-16T22:22:14.052 INFO:tasks.workunit.client.1.smithi123.stdout:2/939: link d4/d1a/d1d/d13/d25/d98/d10c/d11b/c132 d4/d1a/d1d/d13/d25/d98/dbb/d10d/c134 0 2024-02-16T22:22:14.053 INFO:tasks.workunit.client.0.smithi114.stdout:9/761: creat d2/d5c/d5f/d7b/ff5 x:0 0 0 2024-02-16T22:22:14.054 INFO:tasks.workunit.client.0.smithi114.stdout:6/760: truncate d6/d2c/d38/d54/f5f 923378 0 2024-02-16T22:22:14.055 INFO:tasks.workunit.client.0.smithi114.stdout:7/841: creat d3/d22/f12a x:0 0 0 2024-02-16T22:22:14.057 INFO:tasks.workunit.client.0.smithi114.stdout:7/842: fdatasync d3/d14/f43 0 2024-02-16T22:22:14.060 INFO:tasks.workunit.client.0.smithi114.stdout:1/720: symlink d4/d63/d69/d7c/dc0/l103 0 2024-02-16T22:22:14.063 INFO:tasks.workunit.client.0.smithi114.stdout:9/762: dwrite d2/d5c/d5f/d91/de4/fed [0,4194304] 0 2024-02-16T22:22:14.063 INFO:tasks.workunit.client.1.smithi123.stdout:5/946: dwrite d6/dd/f73 [0,4194304] 0 2024-02-16T22:22:14.065 INFO:tasks.workunit.client.1.smithi123.stdout:8/986: creat d0/d29/da7/d125/f144 x:0 0 0 2024-02-16T22:22:14.066 INFO:tasks.workunit.client.1.smithi123.stdout:0/896: write d1/d8/d3d/f59 [2006284,52885] 0 2024-02-16T22:22:14.067 INFO:tasks.workunit.client.1.smithi123.stdout:8/987: chown d0/d29/da7/d26/d2a/d123/dca 95946465 1 2024-02-16T22:22:14.069 INFO:tasks.workunit.client.0.smithi114.stdout:0/805: creat d1/d16/d18/f101 x:0 0 0 2024-02-16T22:22:14.070 INFO:tasks.workunit.client.0.smithi114.stdout:0/806: dread - d1/d16/d48/d98/fef zero size 2024-02-16T22:22:14.072 INFO:tasks.workunit.client.0.smithi114.stdout:5/722: mkdir d0/d28/d98/d1e/d63/d96/d9a/de9 0 2024-02-16T22:22:14.073 INFO:tasks.workunit.client.0.smithi114.stdout:5/723: readlink d0/d28/l34 0 2024-02-16T22:22:14.075 INFO:tasks.workunit.client.0.smithi114.stdout:4/859: truncate dd/d1b/d24/d57/d12b/d38/f5e 2847514 0 2024-02-16T22:22:14.075 INFO:tasks.workunit.client.0.smithi114.stdout:4/860: fdatasync dd/f20 0 2024-02-16T22:22:14.075 INFO:tasks.workunit.client.1.smithi123.stdout:6/943: link d0/d6/d1a/f6f d0/d9/dc/d39/f151 0 2024-02-16T22:22:14.076 INFO:tasks.workunit.client.1.smithi123.stdout:6/944: chown d0/d96/fbf 2440 1 2024-02-16T22:22:14.076 INFO:tasks.workunit.client.0.smithi114.stdout:3/726: mknod d4/d24/d2d/d45/cf8 0 2024-02-16T22:22:14.081 INFO:tasks.workunit.client.0.smithi114.stdout:2/757: dwrite dd/df/f9e [0,4194304] 0 2024-02-16T22:22:14.084 INFO:tasks.workunit.client.1.smithi123.stdout:7/974: link d3/d6e/d4d/d92/d12b/f12c d3/d6e/d1d/db2/d35/db7/f144 0 2024-02-16T22:22:14.087 INFO:tasks.workunit.client.1.smithi123.stdout:3/962: dread d0/d40/dfb/ffc [0,4194304] 0 2024-02-16T22:22:14.087 INFO:tasks.workunit.client.1.smithi123.stdout:5/947: symlink d6/dd/df/d58/d56/l14c 0 2024-02-16T22:22:14.088 INFO:tasks.workunit.client.1.smithi123.stdout:5/948: fsync d6/dd/d78/da5/ff7 0 2024-02-16T22:22:14.090 INFO:tasks.workunit.client.0.smithi114.stdout:6/761: creat d6/dd/d25/d34/d61/d62/d8b/db7/dd3/fe1 x:0 0 0 2024-02-16T22:22:14.090 INFO:tasks.workunit.client.1.smithi123.stdout:8/988: mknod d0/d29/da7/d26/d68/dbe/dbf/c145 0 2024-02-16T22:22:14.093 INFO:tasks.workunit.client.0.smithi114.stdout:8/746: dwrite d4/d11/d78/d7a/f10 [0,4194304] 0 2024-02-16T22:22:14.093 INFO:tasks.workunit.client.1.smithi123.stdout:1/931: creat d3/d6/d49/d48/d84/d92/df9/d132/f138 x:0 0 0 2024-02-16T22:22:14.095 INFO:tasks.workunit.client.1.smithi123.stdout:7/975: mknod d3/d6e/d24/c145 0 2024-02-16T22:22:14.098 INFO:tasks.workunit.client.1.smithi123.stdout:3/963: write d0/da0/faa [84210,27140] 0 2024-02-16T22:22:14.098 INFO:tasks.workunit.client.0.smithi114.stdout:0/807: rmdir d1/d1a/d47/d62/d7f/d8c 39 2024-02-16T22:22:14.101 INFO:tasks.workunit.client.1.smithi123.stdout:5/949: mknod d6/dd/df/d58/d3f/c14d 0 2024-02-16T22:22:14.102 INFO:tasks.workunit.client.1.smithi123.stdout:5/950: dread - d6/dd/df/d9f/d102/f11f zero size 2024-02-16T22:22:14.103 INFO:tasks.workunit.client.1.smithi123.stdout:6/945: dread d0/d6/f124 [0,4194304] 0 2024-02-16T22:22:14.103 INFO:tasks.workunit.client.0.smithi114.stdout:4/861: dread - dd/d2e/d56/ddc/f106 zero size 2024-02-16T22:22:14.103 INFO:tasks.workunit.client.0.smithi114.stdout:9/763: dread d2/dd/ff [0,4194304] 0 2024-02-16T22:22:14.106 INFO:tasks.workunit.client.0.smithi114.stdout:9/764: read d2/d5c/f8d [1470153,43486] 0 2024-02-16T22:22:14.106 INFO:tasks.workunit.client.0.smithi114.stdout:2/758: mknod dd/d3b/da7/d4c/cfc 0 2024-02-16T22:22:14.108 INFO:tasks.workunit.client.0.smithi114.stdout:9/765: chown d2/d5c/dc9/d47/d63/dbb 37 1 2024-02-16T22:22:14.110 INFO:tasks.workunit.client.1.smithi123.stdout:2/940: dread d4/d1a/d1d/d59/d45/fc5 [0,4194304] 0 2024-02-16T22:22:14.110 INFO:tasks.workunit.client.0.smithi114.stdout:6/762: chown d6/d2c/c31 543886 1 2024-02-16T22:22:14.111 INFO:tasks.workunit.client.0.smithi114.stdout:1/721: symlink d4/d63/d69/d7c/d9b/df0/l104 0 2024-02-16T22:22:14.113 INFO:tasks.workunit.client.0.smithi114.stdout:8/747: fdatasync d4/d11/d78/d7a/d7e/fba 0 2024-02-16T22:22:14.114 INFO:tasks.workunit.client.1.smithi123.stdout:6/946: symlink d0/d9/dc/l152 0 2024-02-16T22:22:14.117 INFO:tasks.workunit.client.1.smithi123.stdout:3/964: dread d0/fe [0,4194304] 0 2024-02-16T22:22:14.118 INFO:tasks.workunit.client.1.smithi123.stdout:5/951: rename d6/dd/d78/da5/lff to d6/dc1/de6/d124/l14e 0 2024-02-16T22:22:14.122 INFO:tasks.workunit.client.0.smithi114.stdout:2/759: dread dd/f92 [0,4194304] 0 2024-02-16T22:22:14.122 INFO:tasks.workunit.client.0.smithi114.stdout:0/808: fdatasync d1/d16/d1f/d28/d51/d61/d7e/fe1 0 2024-02-16T22:22:14.123 INFO:tasks.workunit.client.0.smithi114.stdout:0/809: dread - d1/d16/d18/ffb zero size 2024-02-16T22:22:14.131 INFO:tasks.workunit.client.1.smithi123.stdout:2/941: write d4/d1a/dd5/d10b/df1/d4a/faf [886649,18586] 0 2024-02-16T22:22:14.134 INFO:tasks.workunit.client.1.smithi123.stdout:6/947: mkdir d0/d96/de3/dcc/d153 0 2024-02-16T22:22:14.135 INFO:tasks.workunit.client.0.smithi114.stdout:9/766: stat d2/d1a/d23/d32/d46/f9d 0 2024-02-16T22:22:14.139 INFO:tasks.workunit.client.1.smithi123.stdout:6/948: dwrite d0/d9/f6a [0,4194304] 0 2024-02-16T22:22:14.139 INFO:tasks.workunit.client.1.smithi123.stdout:3/965: creat d0/d20/d29/d9b/f13f x:0 0 0 2024-02-16T22:22:14.140 INFO:tasks.workunit.client.0.smithi114.stdout:2/760: dread dd/df/d1c/d21/d61/d69/f72 [0,4194304] 0 2024-02-16T22:22:14.140 INFO:tasks.workunit.client.1.smithi123.stdout:3/966: chown d0/d20/d2e/dc4/f131 102690 1 2024-02-16T22:22:14.140 INFO:tasks.workunit.client.1.smithi123.stdout:6/949: write d0/d6/d1a/d2e/f145 [154812,49510] 0 2024-02-16T22:22:14.141 INFO:tasks.workunit.client.0.smithi114.stdout:6/763: chown d6/d2c/d38/d41/d60/d84 15 1 2024-02-16T22:22:14.141 INFO:tasks.workunit.client.0.smithi114.stdout:2/761: write dd/df/d1c/d21/d61/fe4 [234254,28946] 0 2024-02-16T22:22:14.142 INFO:tasks.workunit.client.0.smithi114.stdout:6/764: write d6/dd/d25/fc0 [1061276,49401] 0 2024-02-16T22:22:14.142 INFO:tasks.workunit.client.1.smithi123.stdout:5/952: creat d6/dd/df/d9f/f14f x:0 0 0 2024-02-16T22:22:14.144 INFO:tasks.workunit.client.1.smithi123.stdout:0/897: write d1/d4/d42/f4a [349147,19850] 0 2024-02-16T22:22:14.146 INFO:tasks.workunit.client.0.smithi114.stdout:1/722: symlink d4/d63/d69/d7c/l105 0 2024-02-16T22:22:14.146 INFO:tasks.workunit.client.1.smithi123.stdout:0/898: write d1/d4/d17/d54/d8e/d96/f101 [44091,43739] 0 2024-02-16T22:22:14.146 INFO:tasks.workunit.client.0.smithi114.stdout:1/723: stat d4/f16 0 2024-02-16T22:22:14.147 INFO:tasks.workunit.client.1.smithi123.stdout:0/899: chown d1/d4/d17/d54/f88 102972 1 2024-02-16T22:22:14.149 INFO:tasks.workunit.client.0.smithi114.stdout:3/727: write d4/d46/d48/d80/dbd/fc8 [648483,60146] 0 2024-02-16T22:22:14.152 INFO:tasks.workunit.client.1.smithi123.stdout:8/989: getdents d0/d29/da7/d26/d68/dbe 0 2024-02-16T22:22:14.153 INFO:tasks.workunit.client.1.smithi123.stdout:8/990: chown d0/d29/da7/d26/d32/de0/c135 0 1 2024-02-16T22:22:14.155 INFO:tasks.workunit.client.0.smithi114.stdout:5/724: creat d0/df/d7a/fea x:0 0 0 2024-02-16T22:22:14.158 INFO:tasks.workunit.client.1.smithi123.stdout:3/967: mknod d0/d20/d29/d9b/dbd/df4/d5d/dae/dc7/c140 0 2024-02-16T22:22:14.159 INFO:tasks.workunit.client.0.smithi114.stdout:0/810: mkdir d1/d16/d1f/d28/d51/d61/d7e/d102 0 2024-02-16T22:22:14.160 INFO:tasks.workunit.client.1.smithi123.stdout:5/953: mknod d6/dd/d78/c150 0 2024-02-16T22:22:14.161 INFO:tasks.workunit.client.0.smithi114.stdout:4/862: write dd/d1b/f91 [1480078,111174] 0 2024-02-16T22:22:14.161 INFO:tasks.workunit.client.1.smithi123.stdout:7/976: getdents d3/d6e/d1d/d97/dca 0 2024-02-16T22:22:14.162 INFO:tasks.workunit.client.0.smithi114.stdout:4/863: chown dd/d1b/d24/d57/d12b/c51 7678106 1 2024-02-16T22:22:14.163 INFO:tasks.workunit.client.1.smithi123.stdout:7/977: chown d3/d6e/d4d/c66 6317 1 2024-02-16T22:22:14.163 INFO:tasks.workunit.client.1.smithi123.stdout:1/932: dwrite d3/d6/d49/fb1 [0,4194304] 0 2024-02-16T22:22:14.165 INFO:tasks.workunit.client.1.smithi123.stdout:0/900: rename d1/d8/d13/d1d/l2b to d1/d8/d13/d1d/d64/dbc/ddb/l127 0 2024-02-16T22:22:14.167 INFO:tasks.workunit.client.1.smithi123.stdout:8/991: unlink d0/d12/c2c 0 2024-02-16T22:22:14.171 INFO:tasks.workunit.client.0.smithi114.stdout:7/843: getdents d3/d22/d73/da7/d6f 0 2024-02-16T22:22:14.174 INFO:tasks.workunit.client.1.smithi123.stdout:3/968: rmdir d0/d20/d29/d9b/dbd/df4/d37 39 2024-02-16T22:22:14.175 INFO:tasks.workunit.client.1.smithi123.stdout:5/954: mkdir d6/dd/df/d90/d151 0 2024-02-16T22:22:14.177 INFO:tasks.workunit.client.0.smithi114.stdout:7/844: dread d3/d22/d73/da7/d6f/d89/da6/dfc/f102 [0,4194304] 0 2024-02-16T22:22:14.178 INFO:tasks.workunit.client.1.smithi123.stdout:1/933: creat d3/d6/d11/d8b/dbb/f139 x:0 0 0 2024-02-16T22:22:14.180 INFO:tasks.workunit.client.0.smithi114.stdout:3/728: mknod d4/d24/d2d/d45/d74/dc7/cf9 0 2024-02-16T22:22:14.180 INFO:tasks.workunit.client.1.smithi123.stdout:6/950: creat d0/d6/d1a/f154 x:0 0 0 2024-02-16T22:22:14.187 INFO:tasks.workunit.client.0.smithi114.stdout:5/725: unlink d0/df/f80 0 2024-02-16T22:22:14.191 INFO:tasks.workunit.client.0.smithi114.stdout:0/811: unlink d1/d16/d18/f101 0 2024-02-16T22:22:14.191 INFO:tasks.workunit.client.0.smithi114.stdout:0/812: stat d1/d1a/d47/d62/d89/fb5 0 2024-02-16T22:22:14.192 INFO:tasks.workunit.client.0.smithi114.stdout:0/813: dread - d1/d16/d1f/d28/d51/d61/d6d/f100 zero size 2024-02-16T22:22:14.193 INFO:tasks.workunit.client.1.smithi123.stdout:7/978: creat d3/d6e/d1d/db2/d3e/de4/dfd/f146 x:0 0 0 2024-02-16T22:22:14.197 INFO:tasks.workunit.client.1.smithi123.stdout:1/934: mknod d3/d6/d49/d48/d68/c13a 0 2024-02-16T22:22:14.199 INFO:tasks.workunit.client.0.smithi114.stdout:2/762: symlink dd/d3b/da7/d4c/dc7/lfd 0 2024-02-16T22:22:14.199 INFO:tasks.workunit.client.1.smithi123.stdout:6/951: creat d0/d96/f155 x:0 0 0 2024-02-16T22:22:14.200 INFO:tasks.workunit.client.1.smithi123.stdout:3/969: mknod d0/d50/d75/c141 0 2024-02-16T22:22:14.202 INFO:tasks.workunit.client.0.smithi114.stdout:8/748: rename d4/d24/d2c/d67 to d4/d16/d25/d46/d10b 0 2024-02-16T22:22:14.205 INFO:tasks.workunit.client.0.smithi114.stdout:3/729: chown d4/d24/l4c 128 1 2024-02-16T22:22:14.206 INFO:tasks.workunit.client.1.smithi123.stdout:0/901: rmdir d1/d8/d13/d1d/d62/d102 0 2024-02-16T22:22:14.207 INFO:tasks.workunit.client.0.smithi114.stdout:5/726: fdatasync d0/df/f57 0 2024-02-16T22:22:14.209 INFO:tasks.workunit.client.0.smithi114.stdout:5/727: stat d0/d28/d98/d25/d89/de2/l6c 0 2024-02-16T22:22:14.210 INFO:tasks.workunit.client.1.smithi123.stdout:8/992: rename d0/d29/dba/f124 to d0/d29/da7/d26/d32/d45/f146 0 2024-02-16T22:22:14.211 INFO:tasks.workunit.client.1.smithi123.stdout:0/902: dwrite d1/d4/d17/d54/fd9 [0,4194304] 0 2024-02-16T22:22:14.213 INFO:tasks.workunit.client.0.smithi114.stdout:4/864: symlink dd/d59/d8f/df7/l136 0 2024-02-16T22:22:14.213 INFO:tasks.workunit.client.1.smithi123.stdout:1/935: truncate d3/d6/d18/d9e/fae 1413795 0 2024-02-16T22:22:14.215 INFO:tasks.workunit.client.0.smithi114.stdout:9/767: symlink d2/d5c/d5f/d72/dd6/lf6 0 2024-02-16T22:22:14.216 INFO:tasks.workunit.client.0.smithi114.stdout:9/768: chown d2/d1a/d23/d32/d46/d78/fa6 813 1 2024-02-16T22:22:14.217 INFO:tasks.workunit.client.1.smithi123.stdout:3/970: mkdir d0/d20/d29/d9b/dbd/d142 0 2024-02-16T22:22:14.218 INFO:tasks.workunit.client.0.smithi114.stdout:6/765: symlink d6/d2c/d5e/dcc/le2 0 2024-02-16T22:22:14.221 INFO:tasks.workunit.client.0.smithi114.stdout:2/763: fdatasync dd/df/d1c/d21/fcd 0 2024-02-16T22:22:14.221 INFO:tasks.workunit.client.0.smithi114.stdout:2/764: chown dd/d3b 17612 1 2024-02-16T22:22:14.226 INFO:tasks.workunit.client.1.smithi123.stdout:5/955: write d6/dd/df/d58/f11a [334315,14057] 0 2024-02-16T22:22:14.226 INFO:tasks.workunit.client.0.smithi114.stdout:2/765: dread dd/df/ddb/fc9 [0,4194304] 0 2024-02-16T22:22:14.228 INFO:tasks.workunit.client.1.smithi123.stdout:8/993: read d0/d29/da7/d26/d2a/d123/dca/f105 [13566,33803] 0 2024-02-16T22:22:14.228 INFO:tasks.workunit.client.1.smithi123.stdout:2/942: sync 2024-02-16T22:22:14.228 INFO:tasks.workunit.client.1.smithi123.stdout:8/994: fdatasync d0/d61/d66/f139 0 2024-02-16T22:22:14.230 INFO:tasks.workunit.client.1.smithi123.stdout:0/903: mkdir d1/d4/d42/d128 0 2024-02-16T22:22:14.233 INFO:tasks.workunit.client.0.smithi114.stdout:8/749: fsync d4/fbb 0 2024-02-16T22:22:14.233 INFO:tasks.workunit.client.0.smithi114.stdout:2/766: dwrite dd/d6b/da5/fc1 [0,4194304] 0 2024-02-16T22:22:14.240 INFO:tasks.workunit.client.0.smithi114.stdout:3/730: symlink d4/dd/d33/d72/lfa 0 2024-02-16T22:22:14.240 INFO:tasks.workunit.client.0.smithi114.stdout:3/731: chown d4/dd/d33/lf2 391001 1 2024-02-16T22:22:14.242 INFO:tasks.workunit.client.0.smithi114.stdout:3/732: chown d4/d26/d4a/l81 13348 1 2024-02-16T22:22:14.242 INFO:tasks.workunit.client.1.smithi123.stdout:1/936: rename d3/d6/db/d10/d2d/d8c/dfd/d9f/db4/ddf/f111 to d3/d6/db/d10/d2d/d8c/dfd/de8/f13b 0 2024-02-16T22:22:14.243 INFO:tasks.workunit.client.1.smithi123.stdout:1/937: dread - d3/d6/db/d10/fe9 zero size 2024-02-16T22:22:14.244 INFO:tasks.workunit.client.0.smithi114.stdout:2/767: dwrite dd/df/d1c/d21/f30 [0,4194304] 0 2024-02-16T22:22:14.244 INFO:tasks.workunit.client.1.smithi123.stdout:3/971: symlink d0/d50/d7e/l143 0 2024-02-16T22:22:14.244 INFO:tasks.workunit.client.1.smithi123.stdout:1/938: chown d3/d6/d11/fe4 84091 1 2024-02-16T22:22:14.247 INFO:tasks.workunit.client.0.smithi114.stdout:5/728: unlink d0/f8 0 2024-02-16T22:22:14.247 INFO:tasks.workunit.client.0.smithi114.stdout:5/729: readlink d0/d28/l64 0 2024-02-16T22:22:14.250 INFO:tasks.workunit.client.1.smithi123.stdout:5/956: mkdir d6/dd/d13d/d152 0 2024-02-16T22:22:14.252 INFO:tasks.workunit.client.1.smithi123.stdout:2/943: mkdir d4/d1a/d1d/d13/d25/d98/d10c/d135 0 2024-02-16T22:22:14.254 INFO:tasks.workunit.client.0.smithi114.stdout:9/769: symlink d2/d1a/lf7 0 2024-02-16T22:22:14.255 INFO:tasks.workunit.client.0.smithi114.stdout:9/770: chown d2/d5c/dc9/d47/f4d 63347 1 2024-02-16T22:22:14.258 INFO:tasks.workunit.client.1.smithi123.stdout:6/952: creat d0/d6/d2f/f156 x:0 0 0 2024-02-16T22:22:14.259 INFO:tasks.workunit.client.0.smithi114.stdout:1/724: getdents d4/d9/d46/d67/d92/de4 0 2024-02-16T22:22:14.259 INFO:tasks.workunit.client.1.smithi123.stdout:6/953: chown d0/d9/dc/d39/c9e 186 1 2024-02-16T22:22:14.259 INFO:tasks.workunit.client.0.smithi114.stdout:6/766: dread d6/d2c/d38/d41/f4a [4194304,4194304] 0 2024-02-16T22:22:14.263 INFO:tasks.workunit.client.0.smithi114.stdout:0/814: rename d1/d16/d1f/d28/d51/d60/cc8 to d1/d16/d1f/d28/c103 0 2024-02-16T22:22:14.264 INFO:tasks.workunit.client.0.smithi114.stdout:0/815: chown d1/d16/d1f/d28/d51 7 1 2024-02-16T22:22:14.266 INFO:tasks.workunit.client.1.smithi123.stdout:2/944: truncate d4/d1a/dd5/d10b/df1/db1/fb7 2846690 0 2024-02-16T22:22:14.275 INFO:tasks.workunit.client.1.smithi123.stdout:3/972: dread d0/d50/fa9 [0,4194304] 0 2024-02-16T22:22:14.275 INFO:tasks.workunit.client.1.smithi123.stdout:7/979: sync 2024-02-16T22:22:14.279 INFO:tasks.workunit.client.0.smithi114.stdout:4/865: symlink dd/d2e/l137 0 2024-02-16T22:22:14.280 INFO:tasks.workunit.client.1.smithi123.stdout:5/957: mknod d6/dc1/d12f/c153 0 2024-02-16T22:22:14.284 INFO:tasks.workunit.client.0.smithi114.stdout:6/767: dread d6/d2c/f95 [0,4194304] 0 2024-02-16T22:22:14.285 INFO:tasks.workunit.client.1.smithi123.stdout:2/945: symlink d4/d1a/d1d/d13/d25/d98/d10c/l136 0 2024-02-16T22:22:14.285 INFO:tasks.workunit.client.1.smithi123.stdout:1/939: dwrite d3/d6/db/d10/d2d/f38 [0,4194304] 0 2024-02-16T22:22:14.286 INFO:tasks.workunit.client.1.smithi123.stdout:2/946: stat d4/d1a/d1d/d13/d25/d98/dbb/ld3 0 2024-02-16T22:22:14.287 INFO:tasks.workunit.client.1.smithi123.stdout:8/995: creat d0/d29/da7/d26/d68/d81/f147 x:0 0 0 2024-02-16T22:22:14.289 INFO:tasks.workunit.client.0.smithi114.stdout:3/733: rename d4/d46/d48/f66 to d4/d46/d48/d80/dbd/ffb 0 2024-02-16T22:22:14.291 INFO:tasks.workunit.client.1.smithi123.stdout:2/947: dwrite d4/d1a/dd5/d10b/df1/db1/fb2 [0,4194304] 0 2024-02-16T22:22:14.295 INFO:tasks.workunit.client.1.smithi123.stdout:6/954: unlink d0/d6/d2f/da5/dc6/cf0 0 2024-02-16T22:22:14.296 INFO:tasks.workunit.client.1.smithi123.stdout:6/955: stat d0/d6/d91/d97 0 2024-02-16T22:22:14.298 INFO:tasks.workunit.client.0.smithi114.stdout:2/768: dwrite dd/df/d1c/d21/d61/d63/f73 [0,4194304] 0 2024-02-16T22:22:14.303 INFO:tasks.workunit.client.0.smithi114.stdout:5/730: dwrite d0/df/d6d/fa1 [0,4194304] 0 2024-02-16T22:22:14.306 INFO:tasks.workunit.client.0.smithi114.stdout:2/769: dread - dd/df/d1c/d21/d61/d69/fd7 zero size 2024-02-16T22:22:14.307 INFO:tasks.workunit.client.0.smithi114.stdout:5/731: stat d0/d28/d98/d25/d89/de2/f31 0 2024-02-16T22:22:14.308 INFO:tasks.workunit.client.0.smithi114.stdout:3/734: creat d4/d46/d48/d75/daa/ffc x:0 0 0 2024-02-16T22:22:14.313 INFO:tasks.workunit.client.1.smithi123.stdout:0/904: getdents d1/d4/d17/dad/db2 0 2024-02-16T22:22:14.320 INFO:tasks.workunit.client.1.smithi123.stdout:3/973: write d0/d20/d29/d9b/dbd/df4/d5d/dae/dc7/d121/fd9 [748098,82440] 0 2024-02-16T22:22:14.321 INFO:tasks.workunit.client.0.smithi114.stdout:4/866: write dd/fff [409951,31490] 0 2024-02-16T22:22:14.322 INFO:tasks.workunit.client.0.smithi114.stdout:4/867: read - dd/d2e/de7/d11f/f12c zero size 2024-02-16T22:22:14.324 INFO:tasks.workunit.client.1.smithi123.stdout:5/958: read - d6/dd/df/d58/d3f/dbe/d37/fe0 zero size 2024-02-16T22:22:14.327 INFO:tasks.workunit.client.1.smithi123.stdout:0/905: mkdir d1/d8/d13/d1d/d64/d129 0 2024-02-16T22:22:14.329 INFO:tasks.workunit.client.1.smithi123.stdout:1/940: dwrite d3/d6/db/d10/d4b/d70/f94 [0,4194304] 0 2024-02-16T22:22:14.330 INFO:tasks.workunit.client.0.smithi114.stdout:5/732: unlink d0/d28/d98/d25/d2e/l87 0 2024-02-16T22:22:14.333 INFO:tasks.workunit.client.0.smithi114.stdout:8/750: getdents d4/d11/d78/d7a/d36/d3b/de4 0 2024-02-16T22:22:14.333 INFO:tasks.workunit.client.1.smithi123.stdout:0/906: dwrite d1/d8/d13/d1d/d62/f109 [0,4194304] 0 2024-02-16T22:22:14.336 INFO:tasks.workunit.client.0.smithi114.stdout:6/768: truncate d6/dd/d25/d34/fb4 922247 0 2024-02-16T22:22:14.338 INFO:tasks.workunit.client.0.smithi114.stdout:1/725: getdents d4/d63/d69/d72/d82 0 2024-02-16T22:22:14.338 INFO:tasks.workunit.client.1.smithi123.stdout:2/948: dwrite d4/d1a/d1d/d38/d7b/fc4 [0,4194304] 0 2024-02-16T22:22:14.343 INFO:tasks.workunit.client.1.smithi123.stdout:6/956: rename d0/d96/de3/fc1 to d0/d8a/d100/d135/f157 0 2024-02-16T22:22:14.349 INFO:tasks.workunit.client.1.smithi123.stdout:5/959: mkdir d6/dc1/de6/d154 0 2024-02-16T22:22:14.350 INFO:tasks.workunit.client.0.smithi114.stdout:3/735: symlink d4/d24/lfd 0 2024-02-16T22:22:14.352 INFO:tasks.workunit.client.1.smithi123.stdout:3/974: dwrite d0/d20/d29/d9b/dbd/df4/d37/f64 [0,4194304] 0 2024-02-16T22:22:14.352 INFO:tasks.workunit.client.0.smithi114.stdout:3/736: write d4/d46/d48/d80/dbd/fc8 [1455055,25266] 0 2024-02-16T22:22:14.353 INFO:tasks.workunit.client.1.smithi123.stdout:8/996: link d0/d29/da7/d26/d2a/d77/df8/f55 d0/d29/da7/d26/d2a/d77/df8/f148 0 2024-02-16T22:22:14.353 INFO:tasks.workunit.client.0.smithi114.stdout:3/737: readlink d4/d87/lc5 0 2024-02-16T22:22:14.357 INFO:tasks.workunit.client.1.smithi123.stdout:1/941: symlink d3/d6/d18/dcb/l13c 0 2024-02-16T22:22:14.361 INFO:tasks.workunit.client.0.smithi114.stdout:6/769: creat d6/dd/d25/d34/d61/d62/fe3 x:0 0 0 2024-02-16T22:22:14.362 INFO:tasks.workunit.client.1.smithi123.stdout:7/980: getdents d3/d6e/d1d/db2 0 2024-02-16T22:22:14.363 INFO:tasks.workunit.client.1.smithi123.stdout:7/981: write d3/d5/d6c/da6/df1/f102 [1037415,108769] 0 2024-02-16T22:22:14.364 INFO:tasks.workunit.client.0.smithi114.stdout:1/726: symlink d4/d9/d46/d67/d92/dca/l106 0 2024-02-16T22:22:14.365 INFO:tasks.workunit.client.1.smithi123.stdout:0/907: mknod d1/d8/d13/d1d/d64/c12a 0 2024-02-16T22:22:14.369 INFO:tasks.workunit.client.1.smithi123.stdout:2/949: chown d4/l65 8 1 2024-02-16T22:22:14.370 INFO:tasks.workunit.client.0.smithi114.stdout:4/868: write dd/f32 [168072,39852] 0 2024-02-16T22:22:14.371 INFO:tasks.workunit.client.0.smithi114.stdout:2/770: truncate dd/df/ddb/fc9 1730846 0 2024-02-16T22:22:14.374 INFO:tasks.workunit.client.0.smithi114.stdout:3/738: dread - d4/d46/d48/d75/daa/fd0 zero size 2024-02-16T22:22:14.378 INFO:tasks.workunit.client.1.smithi123.stdout:5/960: unlink d6/d3b/f65 0 2024-02-16T22:22:14.382 INFO:tasks.workunit.client.0.smithi114.stdout:6/770: truncate d6/d2c/d75/f79 1308309 0 2024-02-16T22:22:14.384 INFO:tasks.workunit.client.1.smithi123.stdout:8/997: mknod d0/d29/da7/d26/d2a/d123/dca/c149 0 2024-02-16T22:22:14.384 INFO:tasks.workunit.client.0.smithi114.stdout:5/733: dwrite d0/d28/d98/d1e/d86/f7e [0,4194304] 0 2024-02-16T22:22:14.390 INFO:tasks.workunit.client.0.smithi114.stdout:8/751: dwrite d4/d24/d2c/f32 [4194304,4194304] 0 2024-02-16T22:22:14.390 INFO:tasks.workunit.client.1.smithi123.stdout:1/942: rename d3/d6/d11/f1f to d3/d6/db/d55/d12c/f13d 0 2024-02-16T22:22:14.391 INFO:tasks.workunit.client.1.smithi123.stdout:1/943: dread - d3/d6/d49/d48/f121 zero size 2024-02-16T22:22:14.392 INFO:tasks.workunit.client.0.smithi114.stdout:4/869: rmdir dd/d59/d8f/df7 39 2024-02-16T22:22:14.396 INFO:tasks.workunit.client.0.smithi114.stdout:2/771: mknod dd/d3b/da7/d4c/cfe 0 2024-02-16T22:22:14.397 INFO:tasks.workunit.client.1.smithi123.stdout:7/982: creat d3/d6e/d4d/d92/d12b/f147 x:0 0 0 2024-02-16T22:22:14.400 INFO:tasks.workunit.client.0.smithi114.stdout:3/739: symlink d4/d87/lfe 0 2024-02-16T22:22:14.400 INFO:tasks.workunit.client.1.smithi123.stdout:1/944: dread d3/d6/d49/d48/d84/f93 [0,4194304] 0 2024-02-16T22:22:14.400 INFO:tasks.workunit.client.1.smithi123.stdout:7/983: truncate d3/d6e/d4d/d92/d12b/f117 558564 0 2024-02-16T22:22:14.401 INFO:tasks.workunit.client.1.smithi123.stdout:7/984: read - d3/d6e/d4d/d92/d12b/f12c zero size 2024-02-16T22:22:14.407 INFO:tasks.workunit.client.0.smithi114.stdout:5/734: dread d0/d28/d98/d1e/d86/fa9 [4194304,4194304] 0 2024-02-16T22:22:14.411 INFO:tasks.workunit.client.1.smithi123.stdout:5/961: rmdir d6/dd/d44/dba/d126 39 2024-02-16T22:22:14.414 INFO:tasks.workunit.client.0.smithi114.stdout:1/727: dwrite d4/d9/d3d/f88 [0,4194304] 0 2024-02-16T22:22:14.415 INFO:tasks.workunit.client.0.smithi114.stdout:1/728: chown d4/d9/d46/d4d/f4c 8280 1 2024-02-16T22:22:14.416 INFO:tasks.workunit.client.0.smithi114.stdout:8/752: mkdir d4/d11/d78/d10c 0 2024-02-16T22:22:14.418 INFO:tasks.workunit.client.0.smithi114.stdout:4/870: mkdir dd/d2e/d56/ddc/dde/d138 0 2024-02-16T22:22:14.436 INFO:tasks.workunit.client.1.smithi123.stdout:8/998: dread d0/d29/da7/d43/d34/f113 [0,4194304] 0 2024-02-16T22:22:14.543 INFO:tasks.workunit.client.0.smithi114.stdout:2/772: creat dd/df/d1c/d21/d61/d69/fff x:0 0 0 2024-02-16T22:22:14.544 INFO:tasks.workunit.client.1.smithi123.stdout:2/950: write d4/d1a/dd5/d10b/df1/d4a/da2/fe7 [609950,96598] 0 2024-02-16T22:22:14.545 INFO:tasks.workunit.client.1.smithi123.stdout:2/951: fsync d4/d1a/d1d/d59/f11f 0 2024-02-16T22:22:14.545 INFO:tasks.workunit.client.0.smithi114.stdout:3/740: mknod d4/d26/d4a/cff 0 2024-02-16T22:22:14.548 INFO:tasks.workunit.client.1.smithi123.stdout:6/957: write d0/d8a/d100/d135/f157 [139513,17839] 0 2024-02-16T22:22:14.549 INFO:tasks.workunit.client.0.smithi114.stdout:6/771: dwrite d6/dd/d25/d34/d61/d62/d8b/f94 [0,4194304] 0 2024-02-16T22:22:14.551 INFO:tasks.workunit.client.1.smithi123.stdout:7/985: creat d3/d87/da8/f148 x:0 0 0 2024-02-16T22:22:14.553 INFO:tasks.workunit.client.0.smithi114.stdout:5/735: mknod d0/d28/d98/d25/d2e/ceb 0 2024-02-16T22:22:14.554 INFO:tasks.workunit.client.0.smithi114.stdout:5/736: chown d0/d28/d98/d1e/d63/d96/d9a/db8/fad 125344680 1 2024-02-16T22:22:14.556 INFO:tasks.workunit.client.0.smithi114.stdout:1/729: fdatasync d4/d63/d69/d7c/fbd 0 2024-02-16T22:22:14.558 INFO:tasks.workunit.client.1.smithi123.stdout:3/975: rmdir d0/d20/d29/df6/d13e 0 2024-02-16T22:22:14.559 INFO:tasks.workunit.client.1.smithi123.stdout:0/908: link d1/d4/d17/l7d d1/d4/d17/dad/l12b 0 2024-02-16T22:22:14.559 INFO:tasks.workunit.client.1.smithi123.stdout:3/976: readlink d0/d20/d29/d9b/dbd/df4/d5d/d60/dc6/l134 0 2024-02-16T22:22:14.560 INFO:tasks.workunit.client.1.smithi123.stdout:0/909: write d1/d4/d42/f7e [293233,37640] 0 2024-02-16T22:22:14.561 INFO:tasks.workunit.client.0.smithi114.stdout:5/737: dread d0/d28/d98/d25/d2e/f67 [0,4194304] 0 2024-02-16T22:22:14.562 INFO:tasks.workunit.client.1.smithi123.stdout:8/999: dread d0/d61/d66/f86 [0,4194304] 0 2024-02-16T22:22:14.564 INFO:tasks.workunit.client.1.smithi123.stdout:2/952: rmdir d4/d1a/d1d/d13/d15 39 2024-02-16T22:22:14.566 INFO:tasks.workunit.client.1.smithi123.stdout:3/977: dwrite d0/d20/d29/d9b/dbd/df4/d5d/dae/db0/d123/f12a [0,4194304] 0 2024-02-16T22:22:14.569 INFO:tasks.workunit.client.1.smithi123.stdout:1/945: rename d3/d6/d49/d48/d68/fab to d3/d6/d49/d48/d84/d136/f13e 0 2024-02-16T22:22:14.576 INFO:tasks.workunit.client.1.smithi123.stdout:1/946: dwrite d3/d6/db/d10/d4b/d70/d87/f10d [0,4194304] 0 2024-02-16T22:22:14.578 INFO:tasks.workunit.client.1.smithi123.stdout:0/910: symlink d1/d4/d17/dad/db2/l12c 0 2024-02-16T22:22:14.578 INFO:tasks.workunit.client.1.smithi123.stdout:5/962: dwrite d6/dc1/f10c [0,4194304] 0 2024-02-16T22:22:14.581 INFO:tasks.workunit.client.0.smithi114.stdout:4/871: dwrite dd/d1b/d24/d57/d12b/f5d [0,4194304] 0 2024-02-16T22:22:14.583 INFO:tasks.workunit.client.0.smithi114.stdout:4/872: write dd/d1b/d41/d74/fd9 [463510,14021] 0 2024-02-16T22:22:14.585 INFO:tasks.workunit.client.1.smithi123.stdout:3/978: fsync d0/d20/d29/d9b/dbd/df4/d37/d41/f112 0 2024-02-16T22:22:14.594 INFO:tasks.workunit.client.1.smithi123.stdout:7/986: creat d3/de0/d135/f149 x:0 0 0 2024-02-16T22:22:14.595 INFO:tasks.workunit.client.1.smithi123.stdout:1/947: creat d3/d6/db/d10/d2d/d8c/dfd/d9f/db4/ddf/df0/d129/f13f x:0 0 0 2024-02-16T22:22:14.598 INFO:tasks.workunit.client.1.smithi123.stdout:0/911: dread d1/d8/d13/d1d/d64/dbc/f92 [0,4194304] 0 2024-02-16T22:22:14.604 INFO:tasks.workunit.client.1.smithi123.stdout:3/979: fdatasync d0/d20/d29/d9b/dbd/df5/fb4 0 2024-02-16T22:22:14.608 INFO:tasks.workunit.client.1.smithi123.stdout:6/958: rename d0/d6/d2f/da5/dc6/f112 to d0/d8a/d100/dfc/f158 0 2024-02-16T22:22:14.611 INFO:tasks.workunit.client.1.smithi123.stdout:0/912: dread d1/d4/f51 [0,4194304] 0 2024-02-16T22:22:14.627 INFO:tasks.workunit.client.1.smithi123.stdout:5/963: write d6/dd/df/d58/d3f/dbe/d37/fe0 [877018,12677] 0 2024-02-16T22:22:14.630 INFO:tasks.workunit.client.1.smithi123.stdout:0/913: mknod d1/d8/d13/d1d/d64/dbc/c12d 0 2024-02-16T22:22:14.634 INFO:tasks.workunit.client.1.smithi123.stdout:1/948: link d3/d6/d49/d48/d84/f93 d3/d6/db/d55/f140 0 2024-02-16T22:22:14.636 INFO:tasks.workunit.client.1.smithi123.stdout:2/953: link d4/d1a/f86 d4/d1a/d1d/d13/d25/f137 0 2024-02-16T22:22:14.643 INFO:tasks.workunit.client.1.smithi123.stdout:7/987: dwrite d3/f4e [0,4194304] 0 2024-02-16T22:22:14.644 INFO:tasks.workunit.client.1.smithi123.stdout:5/964: unlink d6/dd/df/d58/d3f/dbe/f105 0 2024-02-16T22:22:14.645 INFO:tasks.workunit.client.1.smithi123.stdout:0/914: dread d1/d8/f2f [4194304,4194304] 0 2024-02-16T22:22:14.647 INFO:tasks.workunit.client.1.smithi123.stdout:1/949: mkdir d3/d6/db/d10/d2d/d8c/dfd/d9f/d141 0 2024-02-16T22:22:14.652 INFO:tasks.workunit.client.1.smithi123.stdout:3/980: creat d0/d20/d29/d9b/dbd/df4/d5d/f144 x:0 0 0 2024-02-16T22:22:14.652 INFO:tasks.workunit.client.1.smithi123.stdout:6/959: write d0/d9/dc/d39/f44 [1479931,67285] 0 2024-02-16T22:22:14.657 INFO:tasks.workunit.client.1.smithi123.stdout:6/960: dwrite d0/d6/d91/d97/fa6 [0,4194304] 0 2024-02-16T22:22:14.658 INFO:tasks.workunit.client.1.smithi123.stdout:5/965: truncate d6/dd/df/d58/d89/dc3/f115 490846 0 2024-02-16T22:22:14.660 INFO:tasks.workunit.client.1.smithi123.stdout:6/961: write d0/d6/d91/d93/d116/f11e [784878,36083] 0 2024-02-16T22:22:14.660 INFO:tasks.workunit.client.1.smithi123.stdout:5/966: stat d6/dd/df/d58/d56/d141/f13c 0 2024-02-16T22:22:14.662 INFO:tasks.workunit.client.1.smithi123.stdout:1/950: symlink d3/d6/db/d55/l142 0 2024-02-16T22:22:14.664 INFO:tasks.workunit.client.1.smithi123.stdout:2/954: dread d4/d1a/dd5/d10b/df1/d4a/dce/d115/f105 [0,4194304] 0 2024-02-16T22:22:14.669 INFO:tasks.workunit.client.1.smithi123.stdout:3/981: fsync d0/d20/d5a/f126 0 2024-02-16T22:22:14.675 INFO:tasks.workunit.client.1.smithi123.stdout:5/967: symlink d6/dd/d13d/l155 0 2024-02-16T22:22:14.676 INFO:tasks.workunit.client.1.smithi123.stdout:0/915: dread d1/d8/d13/d1d/f8d [0,4194304] 0 2024-02-16T22:22:14.676 INFO:tasks.workunit.client.0.smithi114.stdout:8/753: symlink d4/d24/d2c/d5f/l10d 0 2024-02-16T22:22:14.677 INFO:tasks.workunit.client.1.smithi123.stdout:6/962: symlink d0/d6/d2f/da5/l159 0 2024-02-16T22:22:14.679 INFO:tasks.workunit.client.1.smithi123.stdout:2/955: rmdir d4/d1a/d1d/d13/d25/d98 39 2024-02-16T22:22:14.682 INFO:tasks.workunit.client.1.smithi123.stdout:5/968: dwrite d6/dd/df/d58/d3f/dbe/da2/f12e [0,4194304] 0 2024-02-16T22:22:14.683 INFO:tasks.workunit.client.0.smithi114.stdout:5/738: rename d0/d28/d98/d25/d89/de2/f40 to d0/d28/d98/d25/d89/d99/db0/fec 0 2024-02-16T22:22:14.686 INFO:tasks.workunit.client.1.smithi123.stdout:7/988: dread d3/d6e/d24/fb9 [0,4194304] 0 2024-02-16T22:22:14.687 INFO:tasks.workunit.client.0.smithi114.stdout:3/741: dread d4/dd/f19 [0,4194304] 0 2024-02-16T22:22:14.690 INFO:tasks.workunit.client.0.smithi114.stdout:6/772: write d6/dd/d25/f4c [1464499,38179] 0 2024-02-16T22:22:14.690 INFO:tasks.workunit.client.1.smithi123.stdout:6/963: dread - d0/d6/d91/fec zero size 2024-02-16T22:22:14.692 INFO:tasks.workunit.client.0.smithi114.stdout:2/773: getdents dd/d6b/db8 0 2024-02-16T22:22:14.696 INFO:tasks.workunit.client.0.smithi114.stdout:1/730: write d4/d63/d69/d7c/d9b/db0/fc6 [972909,112106] 0 2024-02-16T22:22:14.698 INFO:tasks.workunit.client.1.smithi123.stdout:1/951: dread d3/d6/db/d10/d2d/f25 [0,4194304] 0 2024-02-16T22:22:14.702 INFO:tasks.workunit.client.0.smithi114.stdout:5/739: creat d0/d28/d98/d25/d2e/d3b/d8d/fed x:0 0 0 2024-02-16T22:22:14.703 INFO:tasks.workunit.client.1.smithi123.stdout:3/982: write d0/d20/d29/d9b/dbd/df4/d37/fa1 [611715,3664] 0 2024-02-16T22:22:14.703 INFO:tasks.workunit.client.1.smithi123.stdout:7/989: creat d3/d6e/d1d/db2/d3e/de4/f14a x:0 0 0 2024-02-16T22:22:14.704 INFO:tasks.workunit.client.1.smithi123.stdout:7/990: chown d3/d87/da8/l124 411 1 2024-02-16T22:22:14.704 INFO:tasks.workunit.client.0.smithi114.stdout:3/742: symlink d4/dd/d33/d72/l100 0 2024-02-16T22:22:14.708 INFO:tasks.workunit.client.1.smithi123.stdout:6/964: creat d0/d6/d2f/da5/f15a x:0 0 0 2024-02-16T22:22:14.710 INFO:tasks.workunit.client.0.smithi114.stdout:6/773: fdatasync d6/dd/d25/f27 0 2024-02-16T22:22:14.714 INFO:tasks.workunit.client.0.smithi114.stdout:1/731: symlink d4/d9/d3d/d3f/d79/l107 0 2024-02-16T22:22:14.716 INFO:tasks.workunit.client.1.smithi123.stdout:1/952: creat d3/d6/db/d10/d2d/d8c/dfd/d9f/d10c/f143 x:0 0 0 2024-02-16T22:22:14.718 INFO:tasks.workunit.client.0.smithi114.stdout:5/740: mknod d0/d28/d98/d25/d2e/d3b/d8d/cee 0 2024-02-16T22:22:14.723 INFO:tasks.workunit.client.1.smithi123.stdout:3/983: unlink d0/d20/d2e/ffd 0 2024-02-16T22:22:14.725 INFO:tasks.workunit.client.0.smithi114.stdout:4/873: rmdir dd/d1b/d41/d127 0 2024-02-16T22:22:14.726 INFO:tasks.workunit.client.1.smithi123.stdout:7/991: creat d3/d6e/d1d/db2/d3e/de4/f14b x:0 0 0 2024-02-16T22:22:14.730 INFO:tasks.workunit.client.1.smithi123.stdout:0/916: truncate d1/d8/d13/d1d/d62/fcc 350398 0 2024-02-16T22:22:14.732 INFO:tasks.workunit.client.1.smithi123.stdout:0/917: dread - d1/d4/d17/d74/de7/ff5 zero size 2024-02-16T22:22:14.733 INFO:tasks.workunit.client.1.smithi123.stdout:7/992: read d3/d6e/d1d/f22 [1614522,106928] 0 2024-02-16T22:22:14.733 INFO:tasks.workunit.client.0.smithi114.stdout:1/732: mkdir d4/d63/d69/d7c/d9b/df0/d108 0 2024-02-16T22:22:14.734 INFO:tasks.workunit.client.1.smithi123.stdout:7/993: readlink d3/d6e/d1d/db2/d3e/de4/d103/l140 0 2024-02-16T22:22:14.736 INFO:tasks.workunit.client.0.smithi114.stdout:5/741: read - d0/d28/d98/d1e/d63/fbd zero size 2024-02-16T22:22:14.740 INFO:tasks.workunit.client.0.smithi114.stdout:2/774: dwrite dd/f7a [0,4194304] 0 2024-02-16T22:22:14.741 INFO:tasks.workunit.client.0.smithi114.stdout:2/775: chown dd/df/d1c/f2e 43144642 1 2024-02-16T22:22:14.768 INFO:tasks.workunit.client.0.smithi114.stdout:1/733: creat d4/d9/d46/d67/d92/dca/f109 x:0 0 0 2024-02-16T22:22:14.770 INFO:tasks.workunit.client.0.smithi114.stdout:5/742: fsync d0/d28/d98/d1e/d86/f7c 0 2024-02-16T22:22:14.770 INFO:tasks.workunit.client.1.smithi123.stdout:2/956: getdents d4/d1a/d1d/d59/d45 0 2024-02-16T22:22:14.771 INFO:tasks.workunit.client.0.smithi114.stdout:3/743: write d4/d26/d91/fd1 [742629,121527] 0 2024-02-16T22:22:14.772 INFO:tasks.workunit.client.0.smithi114.stdout:2/776: creat dd/df/ddb/f100 x:0 0 0 2024-02-16T22:22:14.774 INFO:tasks.workunit.client.0.smithi114.stdout:3/744: truncate d4/d46/d48/d80/dbd/fc8 2281455 0 2024-02-16T22:22:14.813 INFO:tasks.workunit.client.0.smithi114.stdout:1/734: mkdir d4/d63/d69/d7c/d9b/dae/d10a 0 2024-02-16T22:22:14.814 INFO:tasks.workunit.client.0.smithi114.stdout:1/735: chown d4/d63/d69/d72/cc1 3 1 2024-02-16T22:22:14.816 INFO:tasks.workunit.client.1.smithi123.stdout:5/969: link d6/dc1/de6/d124/l14e d6/dd/d78/da5/db0/l156 0 2024-02-16T22:22:14.818 INFO:tasks.workunit.client.0.smithi114.stdout:4/874: creat dd/d1b/f139 x:0 0 0 2024-02-16T22:22:14.823 INFO:tasks.workunit.client.1.smithi123.stdout:7/994: dread d3/d6e/d1d/db2/d35/db7/fe2 [0,4194304] 0 2024-02-16T22:22:14.824 INFO:tasks.workunit.client.0.smithi114.stdout:3/745: mknod d4/dd/c101 0 2024-02-16T22:22:14.828 INFO:tasks.workunit.client.0.smithi114.stdout:5/743: creat d0/d28/d98/d1e/d63/d96/d9a/de9/fef x:0 0 0 2024-02-16T22:22:14.830 INFO:tasks.workunit.client.0.smithi114.stdout:4/875: symlink dd/d1b/d24/d57/d12b/d61/l13a 0 2024-02-16T22:22:14.830 INFO:tasks.workunit.client.1.smithi123.stdout:6/965: dwrite d0/d9/dc/f13b [0,4194304] 0 2024-02-16T22:22:14.831 INFO:tasks.workunit.client.0.smithi114.stdout:4/876: dread - dd/d2e/d56/ddc/dde/f11a zero size 2024-02-16T22:22:14.832 INFO:tasks.workunit.client.0.smithi114.stdout:2/777: symlink dd/d3b/d8a/dcb/de2/l101 0 2024-02-16T22:22:14.833 INFO:tasks.workunit.client.0.smithi114.stdout:2/778: chown dd/cad 25092988 1 2024-02-16T22:22:14.833 INFO:tasks.workunit.client.0.smithi114.stdout:4/877: write dd/d1b/d24/d57/d12b/f52 [808355,8357] 0 2024-02-16T22:22:14.834 INFO:tasks.workunit.client.0.smithi114.stdout:2/779: chown dd/d4e/fb7 223161664 1 2024-02-16T22:22:14.834 INFO:tasks.workunit.client.0.smithi114.stdout:4/878: chown dd/d1b/d24/d98/fc5 107789391 1 2024-02-16T22:22:14.839 INFO:tasks.workunit.client.0.smithi114.stdout:5/744: creat d0/d28/d98/d1e/ff0 x:0 0 0 2024-02-16T22:22:14.843 INFO:tasks.workunit.client.0.smithi114.stdout:3/746: truncate d4/d24/f6d 4193908 0 2024-02-16T22:22:14.845 INFO:tasks.workunit.client.0.smithi114.stdout:4/879: rename dd/d59/d8f/dea/c129 to dd/d59/d8a/c13b 0 2024-02-16T22:22:14.855 INFO:tasks.workunit.client.0.smithi114.stdout:4/880: rename dd/d1b/d41/d77 to dd/d1b/d24/d57/d12b/d61/d13c 0 2024-02-16T22:22:14.856 INFO:tasks.workunit.client.0.smithi114.stdout:1/736: getdents d4/d63/d69/d7c/d9b/db0/d74 0 2024-02-16T22:22:14.865 INFO:tasks.workunit.client.0.smithi114.stdout:1/737: creat d4/d63/d69/d7c/d9b/db0/d74/f10b x:0 0 0 2024-02-16T22:22:14.870 INFO:tasks.workunit.client.0.smithi114.stdout:4/881: fsync dd/d1b/d41/d74/d9b/f10e 0 2024-02-16T22:22:14.876 INFO:tasks.workunit.client.0.smithi114.stdout:2/780: dwrite dd/d3b/da7/d4c/fb2 [0,4194304] 0 2024-02-16T22:22:14.881 INFO:tasks.workunit.client.0.smithi114.stdout:3/747: write d4/d24/f67 [275893,53361] 0 2024-02-16T22:22:14.885 INFO:tasks.workunit.client.0.smithi114.stdout:2/781: rmdir dd/df/d1c/d21/d61/dec 39 2024-02-16T22:22:14.891 INFO:tasks.workunit.client.0.smithi114.stdout:1/738: creat d4/d9/d46/f10c x:0 0 0 2024-02-16T22:22:14.892 INFO:tasks.workunit.client.0.smithi114.stdout:3/748: creat d4/d46/f102 x:0 0 0 2024-02-16T22:22:14.896 INFO:tasks.workunit.client.0.smithi114.stdout:2/782: truncate dd/df/d1c/d21/f26 432716 0 2024-02-16T22:22:14.897 INFO:tasks.workunit.client.0.smithi114.stdout:2/783: dread - dd/d3b/d8a/f9a zero size 2024-02-16T22:22:14.898 INFO:tasks.workunit.client.0.smithi114.stdout:4/882: creat dd/d2e/d56/f13d x:0 0 0 2024-02-16T22:22:14.903 INFO:tasks.workunit.client.0.smithi114.stdout:1/739: creat d4/d9/d46/d47/ddf/f10d x:0 0 0 2024-02-16T22:22:14.903 INFO:tasks.workunit.client.1.smithi123.stdout:0/918: getdents d1/d8/d13/d1d/d64/dbc/d114 0 2024-02-16T22:22:14.907 INFO:tasks.workunit.client.0.smithi114.stdout:4/883: getdents dd/d59/d8f/dea/d134 0 2024-02-16T22:22:14.908 INFO:tasks.workunit.client.0.smithi114.stdout:4/884: read - dd/d59/d8f/dea/f12d zero size 2024-02-16T22:22:14.908 INFO:tasks.workunit.client.1.smithi123.stdout:5/970: unlink d6/dd/l48 0 2024-02-16T22:22:14.910 INFO:tasks.workunit.client.0.smithi114.stdout:1/740: rename d4/fb to d4/d9/d46/d67/d92/dca/dcc/d70/f10e 0 2024-02-16T22:22:14.911 INFO:tasks.workunit.client.1.smithi123.stdout:3/984: creat d0/d20/d5a/f145 x:0 0 0 2024-02-16T22:22:14.917 INFO:tasks.workunit.client.0.smithi114.stdout:1/741: symlink d4/d9/d3d/d3f/d79/l10f 0 2024-02-16T22:22:14.919 INFO:tasks.workunit.client.0.smithi114.stdout:1/742: read d4/d9/d46/d67/f68 [7376615,71671] 0 2024-02-16T22:22:14.919 INFO:tasks.workunit.client.1.smithi123.stdout:6/966: rename d0/d6/d91/dda/c128 to d0/c15b 0 2024-02-16T22:22:14.924 INFO:tasks.workunit.client.1.smithi123.stdout:2/957: rmdir d4/d1a/d1d/d13/d25/d98/d10c/d11b 39 2024-02-16T22:22:14.924 INFO:tasks.workunit.client.0.smithi114.stdout:2/784: dwrite dd/f36 [4194304,4194304] 0 2024-02-16T22:22:14.926 INFO:tasks.workunit.client.0.smithi114.stdout:2/785: dread - dd/df/d1c/d21/d61/d69/de9/ff3 zero size 2024-02-16T22:22:14.928 INFO:tasks.workunit.client.1.smithi123.stdout:1/953: getdents d3/d6/db/d10/d2d/d8c/dfd/d9f/db4/ddf/df0/d129 0 2024-02-16T22:22:14.933 INFO:tasks.workunit.client.1.smithi123.stdout:5/971: mknod d6/dd/df/d90/c157 0 2024-02-16T22:22:14.933 INFO:tasks.workunit.client.1.smithi123.stdout:7/995: dwrite d3/d6e/d1d/db2/d3e/f5b [0,4194304] 0 2024-02-16T22:22:14.934 INFO:tasks.workunit.client.0.smithi114.stdout:4/885: dwrite dd/d1b/d24/d57/d12b/d38/d87/fc2 [0,4194304] 0 2024-02-16T22:22:14.934 INFO:tasks.workunit.client.1.smithi123.stdout:5/972: fdatasync d6/dd/d78/da5/db0/f13b 0 2024-02-16T22:22:14.935 INFO:tasks.workunit.client.0.smithi114.stdout:2/786: stat dd/d3b/da7/d4c/fbe 0 2024-02-16T22:22:14.940 INFO:tasks.workunit.client.1.smithi123.stdout:3/985: rename d0/d20/d29/d9b/dbd/df4/d5d/d60/d93/f120 to d0/d20/d29/d9b/dbd/df4/d5d/d60/dc6/df0/f146 0 2024-02-16T22:22:14.944 INFO:tasks.workunit.client.0.smithi114.stdout:2/787: dwrite dd/f5d [0,4194304] 0 2024-02-16T22:22:14.947 INFO:tasks.workunit.client.1.smithi123.stdout:2/958: unlink d4/d1a/d1d/fe9 0 2024-02-16T22:22:14.947 INFO:tasks.workunit.client.1.smithi123.stdout:0/919: symlink d1/d4/d17/d54/d8e/l12e 0 2024-02-16T22:22:14.949 INFO:tasks.workunit.client.1.smithi123.stdout:1/954: mknod d3/d6/d49/d48/d84/d136/c144 0 2024-02-16T22:22:14.951 INFO:tasks.workunit.client.0.smithi114.stdout:1/743: getdents d4/d9/d46/d47/ddf 0 2024-02-16T22:22:14.953 INFO:tasks.workunit.client.0.smithi114.stdout:2/788: chown dd/d6b/fbf 217 1 2024-02-16T22:22:14.954 INFO:tasks.workunit.client.1.smithi123.stdout:7/996: mknod d3/d6e/d34/c14c 0 2024-02-16T22:22:14.955 INFO:tasks.workunit.client.1.smithi123.stdout:7/997: write d3/d6e/d52/f110 [279271,54121] 0 2024-02-16T22:22:14.955 INFO:tasks.workunit.client.0.smithi114.stdout:4/886: mknod dd/d59/d8f/df7/c13e 0 2024-02-16T22:22:14.970 INFO:tasks.workunit.client.1.smithi123.stdout:0/920: mkdir d1/d4/d17/d54/d8e/d96/dee/d12f 0 2024-02-16T22:22:14.970 INFO:tasks.workunit.client.1.smithi123.stdout:5/973: rmdir d6/dd 39 2024-02-16T22:22:14.975 INFO:tasks.workunit.client.1.smithi123.stdout:1/955: unlink d3/d6/d49/d48/d68/l134 0 2024-02-16T22:22:14.977 INFO:tasks.workunit.client.0.smithi114.stdout:4/887: creat dd/d59/d8f/f13f x:0 0 0 2024-02-16T22:22:14.978 INFO:tasks.workunit.client.1.smithi123.stdout:7/998: write d3/d6e/d1d/db2/d35/f36 [8004785,42371] 0 2024-02-16T22:22:14.978 INFO:tasks.workunit.client.0.smithi114.stdout:2/789: getdents dd/d3b/d8a/dcb/de2 0 2024-02-16T22:22:14.980 INFO:tasks.workunit.client.0.smithi114.stdout:4/888: creat dd/d59/dfa/f140 x:0 0 0 2024-02-16T22:22:14.987 INFO:tasks.workunit.client.0.smithi114.stdout:4/889: truncate dd/d1b/d24/fe6 481981 0 2024-02-16T22:22:14.989 INFO:tasks.workunit.client.1.smithi123.stdout:0/921: rename d1/d8/d13/f20 to d1/d4/d17/d54/db4/dc5/f130 0 2024-02-16T22:22:14.992 INFO:tasks.workunit.client.0.smithi114.stdout:2/790: creat dd/df/d1c/d21/d61/f102 x:0 0 0 2024-02-16T22:22:14.993 INFO:tasks.workunit.client.0.smithi114.stdout:2/791: fsync dd/d3b/d8a/fe7 0 2024-02-16T22:22:14.994 INFO:tasks.workunit.client.1.smithi123.stdout:3/986: dwrite d0/d20/d29/d9b/dbd/df5/f67 [0,4194304] 0 2024-02-16T22:22:14.994 INFO:tasks.workunit.client.0.smithi114.stdout:4/890: mkdir dd/d1b/d41/d74/d141 0 2024-02-16T22:22:14.996 INFO:tasks.workunit.client.1.smithi123.stdout:6/967: mknod d0/d74/c15c 0 2024-02-16T22:22:14.999 INFO:tasks.workunit.client.1.smithi123.stdout:2/959: truncate d4/d1a/d1d/d38/d7b/fc4 1861321 0 2024-02-16T22:22:15.005 INFO:tasks.workunit.client.1.smithi123.stdout:1/956: rename d3/d6/d49/d48/d84/d92/lc0 to d3/d6/db/d55/l145 0 2024-02-16T22:22:15.008 INFO:tasks.workunit.client.1.smithi123.stdout:7/999: mkdir d3/d14d 0 2024-02-16T22:22:15.015 INFO:tasks.workunit.client.0.smithi114.stdout:4/891: getdents dd/d1b/db0 0 2024-02-16T22:22:15.016 INFO:tasks.workunit.client.1.smithi123.stdout:1/957: write d3/f8 [809769,20574] 0 2024-02-16T22:22:15.017 INFO:tasks.workunit.client.1.smithi123.stdout:1/958: chown d3/d6/db/d10/d2d/d8c/dfd/d9f/db4/ddf/df0/c123 1 1 2024-02-16T22:22:15.018 INFO:tasks.workunit.client.1.smithi123.stdout:6/968: creat d0/d8a/d100/d147/f15d x:0 0 0 2024-02-16T22:22:15.019 INFO:tasks.workunit.client.1.smithi123.stdout:2/960: mknod d4/d1a/d1d/d13/d25/d98/d10c/d135/c138 0 2024-02-16T22:22:15.020 INFO:tasks.workunit.client.0.smithi114.stdout:2/792: dwrite dd/df/f52 [0,4194304] 0 2024-02-16T22:22:15.022 INFO:tasks.workunit.client.1.smithi123.stdout:1/959: mknod d3/d6/db/d10/d2d/d8c/dfd/d9f/c146 0 2024-02-16T22:22:15.023 INFO:tasks.workunit.client.0.smithi114.stdout:4/892: creat dd/d59/d8f/dea/d134/f142 x:0 0 0 2024-02-16T22:22:15.024 INFO:tasks.workunit.client.1.smithi123.stdout:6/969: dread - d0/d6/d1a/d88/f134 zero size 2024-02-16T22:22:15.025 INFO:tasks.workunit.client.1.smithi123.stdout:5/974: getdents d6/dd/d78/da5 0 2024-02-16T22:22:15.025 INFO:tasks.workunit.client.0.smithi114.stdout:4/893: truncate dd/d1b/d24/d57/d12b/d61/f66 1704229 0 2024-02-16T22:22:15.026 INFO:tasks.workunit.client.0.smithi114.stdout:4/894: chown dd/c14 3234 1 2024-02-16T22:22:15.026 INFO:tasks.workunit.client.0.smithi114.stdout:2/793: rename dd/df/d1c/d21/d87/f5a to dd/df/d1c/d21/d61/d63/dc2/f103 0 2024-02-16T22:22:15.029 INFO:tasks.workunit.client.1.smithi123.stdout:6/970: creat d0/d6/d91/dda/f15e x:0 0 0 2024-02-16T22:22:15.030 INFO:tasks.workunit.client.1.smithi123.stdout:1/960: dwrite d3/d6/db/d10/d2d/d8c/dfd/f130 [0,4194304] 0 2024-02-16T22:22:15.030 INFO:tasks.workunit.client.0.smithi114.stdout:2/794: rmdir dd/d4e 39 2024-02-16T22:22:15.033 INFO:tasks.workunit.client.0.smithi114.stdout:4/895: mknod dd/d1b/d41/c143 0 2024-02-16T22:22:15.034 INFO:tasks.workunit.client.1.smithi123.stdout:5/975: dread d6/dd/df/f5e [0,4194304] 0 2024-02-16T22:22:15.035 INFO:tasks.workunit.client.1.smithi123.stdout:2/961: truncate d4/d1a/d1d/d59/f6f 715556 0 2024-02-16T22:22:15.035 INFO:tasks.workunit.client.1.smithi123.stdout:5/976: chown d6/dd/df/d58/d118/f119 182854 1 2024-02-16T22:22:15.037 INFO:tasks.workunit.client.0.smithi114.stdout:4/896: mkdir dd/d1b/d24/d57/d12b/d61/d13c/d144 0 2024-02-16T22:22:15.038 INFO:tasks.workunit.client.1.smithi123.stdout:1/961: mknod d3/d6/db/d109/c147 0 2024-02-16T22:22:15.039 INFO:tasks.workunit.client.0.smithi114.stdout:2/795: chown dd/df/ddb 644065 1 2024-02-16T22:22:15.041 INFO:tasks.workunit.client.0.smithi114.stdout:4/897: dread - dd/d1b/d24/d57/d12b/d99/d103/f111 zero size 2024-02-16T22:22:15.042 INFO:tasks.workunit.client.1.smithi123.stdout:5/977: truncate d6/dd/df/f62 1075383 0 2024-02-16T22:22:15.043 INFO:tasks.workunit.client.1.smithi123.stdout:1/962: creat d3/d6/db/d10/d2d/d8c/dfd/d9f/d141/f148 x:0 0 0 2024-02-16T22:22:15.044 INFO:tasks.workunit.client.0.smithi114.stdout:2/796: fdatasync dd/d3b/da7/d4c/fbe 0 2024-02-16T22:22:15.045 INFO:tasks.workunit.client.1.smithi123.stdout:5/978: fdatasync d6/dd/df/d58/d89/dc3/f10e 0 2024-02-16T22:22:15.048 INFO:tasks.workunit.client.0.smithi114.stdout:4/898: creat dd/d1b/d41/f145 x:0 0 0 2024-02-16T22:22:15.049 INFO:tasks.workunit.client.0.smithi114.stdout:4/899: write dd/d1b/d24/d98/fc5 [3974805,21045] 0 2024-02-16T22:22:15.050 INFO:tasks.workunit.client.1.smithi123.stdout:5/979: mkdir d6/dd/dbb/d158 0 2024-02-16T22:22:15.051 INFO:tasks.workunit.client.1.smithi123.stdout:5/980: chown d6/dd/df/d58/d3f/d6f/f8e 61 1 2024-02-16T22:22:15.052 INFO:tasks.workunit.client.1.smithi123.stdout:1/963: creat d3/d6/db/d10/d2d/d8c/dfd/d9f/db4/f149 x:0 0 0 2024-02-16T22:22:15.053 INFO:tasks.workunit.client.1.smithi123.stdout:5/981: unlink d6/f8 0 2024-02-16T22:22:15.054 INFO:tasks.workunit.client.0.smithi114.stdout:4/900: mknod dd/d59/d8f/c146 0 2024-02-16T22:22:15.057 INFO:tasks.workunit.client.1.smithi123.stdout:5/982: truncate d6/dd/df/f28 3620522 0 2024-02-16T22:22:15.059 INFO:tasks.workunit.client.1.smithi123.stdout:5/983: unlink d6/dd/df/d90/ff2 0 2024-02-16T22:22:15.062 INFO:tasks.workunit.client.1.smithi123.stdout:5/984: unlink d6/dd/df/d1f/ddc/l103 0 2024-02-16T22:22:15.068 INFO:tasks.workunit.client.0.smithi114.stdout:4/901: dread dd/d59/dfa/f10d [0,4194304] 0 2024-02-16T22:22:15.071 INFO:tasks.workunit.client.0.smithi114.stdout:4/902: unlink dd/d1b/d24/d98/cae 0 2024-02-16T22:22:15.076 INFO:tasks.workunit.client.0.smithi114.stdout:4/903: dwrite dd/d1b/d41/dc3/ff3 [0,4194304] 0 2024-02-16T22:22:15.079 INFO:tasks.workunit.client.0.smithi114.stdout:4/904: creat dd/d1b/d24/d57/d12b/d61/d13c/d144/f147 x:0 0 0 2024-02-16T22:22:15.082 INFO:tasks.workunit.client.0.smithi114.stdout:4/905: getdents dd/d59 0 2024-02-16T22:22:15.083 INFO:tasks.workunit.client.0.smithi114.stdout:4/906: read - dd/d2e/d56/ddc/dde/fb5 zero size 2024-02-16T22:22:15.087 INFO:tasks.workunit.client.0.smithi114.stdout:4/907: mknod dd/d1b/d24/d57/d12b/d38/d87/c148 0 2024-02-16T22:22:15.088 INFO:tasks.workunit.client.0.smithi114.stdout:4/908: write dd/d59/d8f/f88 [1395124,62026] 0 2024-02-16T22:22:15.088 INFO:tasks.workunit.client.1.smithi123.stdout:0/922: truncate d1/d8/f6b 2137902 0 2024-02-16T22:22:15.091 INFO:tasks.workunit.client.1.smithi123.stdout:3/987: dwrite d0/d20/d29/f3c [0,4194304] 0 2024-02-16T22:22:15.094 INFO:tasks.workunit.client.1.smithi123.stdout:0/923: creat d1/d4/d17/d74/f131 x:0 0 0 2024-02-16T22:22:15.096 INFO:tasks.workunit.client.1.smithi123.stdout:0/924: stat d1/d4/d17/d54/f88 0 2024-02-16T22:22:15.097 INFO:tasks.workunit.client.1.smithi123.stdout:0/925: chown d1/d8/d13/d1d/fe1 238007 1 2024-02-16T22:22:15.097 INFO:tasks.workunit.client.1.smithi123.stdout:3/988: mknod d0/d20/d29/d9b/dbd/df4/d5d/dae/c147 0 2024-02-16T22:22:15.098 INFO:tasks.workunit.client.1.smithi123.stdout:6/971: dread d0/d9/dc/d39/f5b [0,4194304] 0 2024-02-16T22:22:15.103 INFO:tasks.workunit.client.1.smithi123.stdout:6/972: creat d0/d9/dc/d39/f15f x:0 0 0 2024-02-16T22:22:15.104 INFO:tasks.workunit.client.1.smithi123.stdout:1/964: dread d3/d6/d11/d76/f59 [0,4194304] 0 2024-02-16T22:22:15.118 INFO:tasks.workunit.client.0.smithi114.stdout:2/797: dwrite dd/df/d1c/d21/d87/f94 [0,4194304] 0 2024-02-16T22:22:15.121 INFO:tasks.workunit.client.1.smithi123.stdout:0/926: dread d1/d8/d13/fca [0,4194304] 0 2024-02-16T22:22:15.127 INFO:tasks.workunit.client.0.smithi114.stdout:2/798: getdents dd/d6b/db8 0 2024-02-16T22:22:15.129 INFO:tasks.workunit.client.1.smithi123.stdout:0/927: truncate d1/d8/d13/d1d/d64/dbc/ddb/f11b 304984 0 2024-02-16T22:22:15.130 INFO:tasks.workunit.client.1.smithi123.stdout:5/985: dwrite d6/dd/df/d90/fae [0,4194304] 0 2024-02-16T22:22:15.130 INFO:tasks.workunit.client.0.smithi114.stdout:2/799: creat dd/d3b/f104 x:0 0 0 2024-02-16T22:22:15.131 INFO:tasks.workunit.client.0.smithi114.stdout:2/800: readlink dd/l20 0 2024-02-16T22:22:15.135 INFO:tasks.workunit.client.1.smithi123.stdout:0/928: unlink d1/d4/d17/d54/d8e/d96/cfc 0 2024-02-16T22:22:15.138 INFO:tasks.workunit.client.0.smithi114.stdout:4/909: write dd/f89 [416124,110922] 0 2024-02-16T22:22:15.138 INFO:tasks.workunit.client.1.smithi123.stdout:5/986: getdents d6/dc1/de6/d154 0 2024-02-16T22:22:15.139 INFO:tasks.workunit.client.0.smithi114.stdout:4/910: chown dd/d1b/d24/d57/d12b/fba 242513686 1 2024-02-16T22:22:15.141 INFO:tasks.workunit.client.0.smithi114.stdout:4/911: write dd/d1b/d24/f4a [2950691,113275] 0 2024-02-16T22:22:15.143 INFO:tasks.workunit.client.1.smithi123.stdout:6/973: write d0/d6/d2f/d5d/f7e [678967,86487] 0 2024-02-16T22:22:15.144 INFO:tasks.workunit.client.1.smithi123.stdout:5/987: dread d6/dd/df/d58/d89/dc3/f115 [0,4194304] 0 2024-02-16T22:22:15.147 INFO:tasks.workunit.client.1.smithi123.stdout:3/989: dwrite d0/d20/d2e/fe3 [0,4194304] 0 2024-02-16T22:22:15.147 INFO:tasks.workunit.client.1.smithi123.stdout:6/974: truncate d0/d6/d1a/d88/f134 749493 0 2024-02-16T22:22:15.149 INFO:tasks.workunit.client.0.smithi114.stdout:4/912: rmdir dd/d1b/d24/d98 39 2024-02-16T22:22:15.150 INFO:tasks.workunit.client.1.smithi123.stdout:3/990: chown d0/d20/d29/d9b/dbd/df4/d37/d41/fd3 6155 1 2024-02-16T22:22:15.151 INFO:tasks.workunit.client.1.smithi123.stdout:1/965: dwrite d3/d6/db/d10/d2d/d8c/dfd/fc2 [0,4194304] 0 2024-02-16T22:22:15.153 INFO:tasks.workunit.client.0.smithi114.stdout:2/801: rename dd/c15 to dd/df/c105 0 2024-02-16T22:22:15.154 INFO:tasks.workunit.client.1.smithi123.stdout:0/929: rmdir d1/d4/d42/d128 0 2024-02-16T22:22:15.156 INFO:tasks.workunit.client.0.smithi114.stdout:4/913: dwrite dd/d2e/de7/d11f/f12c [0,4194304] 0 2024-02-16T22:22:15.158 INFO:tasks.workunit.client.0.smithi114.stdout:4/914: write dd/d59/d8f/dea/d134/f142 [757452,109268] 0 2024-02-16T22:22:15.167 INFO:tasks.workunit.client.1.smithi123.stdout:2/962: sync 2024-02-16T22:22:15.182 INFO:tasks.workunit.client.0.smithi114.stdout:7/845: sync 2024-02-16T22:22:15.188 INFO:tasks.workunit.client.0.smithi114.stdout:2/802: mknod dd/d4e/c106 0 2024-02-16T22:22:15.194 INFO:tasks.workunit.client.1.smithi123.stdout:0/930: mknod d1/d4/d17/c132 0 2024-02-16T22:22:15.201 INFO:tasks.workunit.client.1.smithi123.stdout:0/931: unlink d1/d4/d17/d74/d57/f73 0 2024-02-16T22:22:15.202 INFO:tasks.workunit.client.1.smithi123.stdout:5/988: dread d6/dd/df/d58/d3f/dbe/d37/f6e [0,4194304] 0 2024-02-16T22:22:15.202 INFO:tasks.workunit.client.1.smithi123.stdout:0/932: readlink d1/d4/d17/d74/de7/l110 0 2024-02-16T22:22:15.203 INFO:tasks.workunit.client.1.smithi123.stdout:3/991: link d0/d20/d29/d9b/dbd/df4/d5d/d60/fad d0/d20/d29/d9b/dbd/df4/d5d/d60/dc6/f148 0 2024-02-16T22:22:15.207 INFO:tasks.workunit.client.1.smithi123.stdout:0/933: symlink d1/d8/d13/d1d/d64/dbc/ddb/l133 0 2024-02-16T22:22:15.208 INFO:tasks.workunit.client.1.smithi123.stdout:6/975: dread d0/d6/d1a/f36 [4194304,4194304] 0 2024-02-16T22:22:15.210 INFO:tasks.workunit.client.1.smithi123.stdout:5/989: mknod d6/dd/d44/d12b/c159 0 2024-02-16T22:22:15.212 INFO:tasks.workunit.client.1.smithi123.stdout:0/934: mknod d1/d8/d13/d1d/d64/dbc/c134 0 2024-02-16T22:22:15.213 INFO:tasks.workunit.client.0.smithi114.stdout:2/803: write dd/df/d1c/d21/d87/f5f [1893909,41463] 0 2024-02-16T22:22:15.216 INFO:tasks.workunit.client.1.smithi123.stdout:1/966: truncate d3/d6/db/d10/d2d/d8c/dfd/d9f/fc3 1082033 0 2024-02-16T22:22:15.218 INFO:tasks.workunit.client.0.smithi114.stdout:7/846: dwrite d3/d22/d73/da7/d4e/fbf [0,4194304] 0 2024-02-16T22:22:15.220 INFO:tasks.workunit.client.1.smithi123.stdout:1/967: creat d3/d6/db/d10/d2d/d8c/dfd/d9f/db4/ddf/df0/d129/f14a x:0 0 0 2024-02-16T22:22:15.220 INFO:tasks.workunit.client.0.smithi114.stdout:2/804: dwrite dd/d4e/f9c [0,4194304] 0 2024-02-16T22:22:15.223 INFO:tasks.workunit.client.1.smithi123.stdout:2/963: dwrite d4/d1a/d1d/d13/d25/d98/d10c/d11b/f112 [0,4194304] 0 2024-02-16T22:22:15.226 INFO:tasks.workunit.client.1.smithi123.stdout:1/968: creat d3/d6/d49/d48/d84/d92/f14b x:0 0 0 2024-02-16T22:22:15.227 INFO:tasks.workunit.client.1.smithi123.stdout:0/935: dread d1/d8/faf [0,4194304] 0 2024-02-16T22:22:15.227 INFO:tasks.workunit.client.1.smithi123.stdout:6/976: write d0/d9/fc3 [471753,130416] 0 2024-02-16T22:22:15.231 INFO:tasks.workunit.client.1.smithi123.stdout:3/992: dwrite d0/d50/f7c [0,4194304] 0 2024-02-16T22:22:15.232 INFO:tasks.workunit.client.1.smithi123.stdout:3/993: fsync d0/d20/d5a/f108 0 2024-02-16T22:22:15.233 INFO:tasks.workunit.client.0.smithi114.stdout:2/805: getdents dd/df/d1c 0 2024-02-16T22:22:15.235 INFO:tasks.workunit.client.1.smithi123.stdout:5/990: dread d6/dd/df/d58/d3f/f9e [0,4194304] 0 2024-02-16T22:22:15.238 INFO:tasks.workunit.client.1.smithi123.stdout:5/991: dread d6/dd/df/d58/d56/d141/f13c [0,4194304] 0 2024-02-16T22:22:15.239 INFO:tasks.workunit.client.1.smithi123.stdout:6/977: mknod d0/d8a/d100/dfc/c160 0 2024-02-16T22:22:15.240 INFO:tasks.workunit.client.0.smithi114.stdout:2/806: mknod dd/df/d1c/d21/d87/c107 0 2024-02-16T22:22:15.241 INFO:tasks.workunit.client.0.smithi114.stdout:7/847: write d3/d3d/f4b [2819975,80025] 0 2024-02-16T22:22:15.248 INFO:tasks.workunit.client.0.smithi114.stdout:2/807: rename dd/df/f59 to dd/d3b/d8a/f108 0 2024-02-16T22:22:15.251 INFO:tasks.workunit.client.1.smithi123.stdout:5/992: creat d6/dd/d13d/f15a x:0 0 0 2024-02-16T22:22:15.251 INFO:tasks.workunit.client.1.smithi123.stdout:5/993: fdatasync d6/dd/df/d58/f11a 0 2024-02-16T22:22:15.252 INFO:tasks.workunit.client.0.smithi114.stdout:2/808: chown dd/df/c105 469 1 2024-02-16T22:22:15.258 INFO:tasks.workunit.client.0.smithi114.stdout:2/809: unlink dd/df/f44 0 2024-02-16T22:22:15.258 INFO:tasks.workunit.client.0.smithi114.stdout:2/810: chown dd/d3b/c53 311 1 2024-02-16T22:22:15.261 INFO:tasks.workunit.client.1.smithi123.stdout:1/969: creat d3/d6/db/d10/d2d/d8c/dfd/d9f/db4/f14c x:0 0 0 2024-02-16T22:22:15.261 INFO:tasks.workunit.client.1.smithi123.stdout:0/936: link d1/d4/f51 d1/d4/d17/d54/d8e/d96/dee/f135 0 2024-02-16T22:22:15.264 INFO:tasks.workunit.client.0.smithi114.stdout:2/811: rename dd/d4e/fb7 to dd/df/d1c/d21/d61/d63/d83/df8/f109 0 2024-02-16T22:22:15.266 INFO:tasks.workunit.client.1.smithi123.stdout:0/937: mknod d1/d4/d17/dad/db2/c136 0 2024-02-16T22:22:15.267 INFO:tasks.workunit.client.0.smithi114.stdout:2/812: fdatasync dd/d3b/da7/d4c/fbe 0 2024-02-16T22:22:15.270 INFO:tasks.workunit.client.1.smithi123.stdout:1/970: symlink d3/d6/db/d10/d2d/d8c/dfd/d9f/d141/l14d 0 2024-02-16T22:22:15.271 INFO:tasks.workunit.client.1.smithi123.stdout:1/971: chown d3/d6/d49/d48/d84/d92/dec 116324 1 2024-02-16T22:22:15.271 INFO:tasks.workunit.client.1.smithi123.stdout:5/994: creat d6/dd/f15b x:0 0 0 2024-02-16T22:22:15.273 INFO:tasks.workunit.client.1.smithi123.stdout:6/978: getdents d0/d6/d2f/d5d 0 2024-02-16T22:22:15.273 INFO:tasks.workunit.client.1.smithi123.stdout:3/994: dread d0/d20/d29/d9b/dbd/df4/d5d/dae/db0/fc3 [0,4194304] 0 2024-02-16T22:22:15.274 INFO:tasks.workunit.client.1.smithi123.stdout:0/938: creat d1/d8/da7/f137 x:0 0 0 2024-02-16T22:22:15.276 INFO:tasks.workunit.client.1.smithi123.stdout:3/995: dread - d0/d20/d5a/f145 zero size 2024-02-16T22:22:15.278 INFO:tasks.workunit.client.1.smithi123.stdout:2/964: dwrite d4/d1a/d1d/d59/dd7/ddc/fa8 [4194304,4194304] 0 2024-02-16T22:22:15.278 INFO:tasks.workunit.client.1.smithi123.stdout:5/995: rmdir d6/dd/dbb 39 2024-02-16T22:22:15.279 INFO:tasks.workunit.client.0.smithi114.stdout:2/813: rename dd/d3b/fdc to dd/f10a 0 2024-02-16T22:22:15.280 INFO:tasks.workunit.client.1.smithi123.stdout:3/996: read - d0/d20/d29/d9b/dbd/df4/d37/d41/f112 zero size 2024-02-16T22:22:15.280 INFO:tasks.workunit.client.0.smithi114.stdout:2/814: chown dd/df/d1c/f67 12593686 1 2024-02-16T22:22:15.281 INFO:tasks.workunit.client.0.smithi114.stdout:2/815: chown dd/l20 20058920 1 2024-02-16T22:22:15.282 INFO:tasks.workunit.client.0.smithi114.stdout:2/816: write dd/df/d1c/d21/d61/d63/d83/fd3 [12745,32342] 0 2024-02-16T22:22:15.283 INFO:tasks.workunit.client.1.smithi123.stdout:0/939: mkdir d1/d4/d17/d54/db4/dc5/d138 0 2024-02-16T22:22:15.283 INFO:tasks.workunit.client.1.smithi123.stdout:5/996: write d6/dd/df/d58/d3f/dbe/d37/fe0 [766092,28436] 0 2024-02-16T22:22:15.284 INFO:tasks.workunit.client.1.smithi123.stdout:6/979: dwrite d0/d6/d2f/d5d/f7e [0,4194304] 0 2024-02-16T22:22:15.285 INFO:tasks.workunit.client.1.smithi123.stdout:2/965: mknod d4/d1a/d1d/d38/c139 0 2024-02-16T22:22:15.285 INFO:tasks.workunit.client.1.smithi123.stdout:1/972: dread d3/d6/d18/d9e/fae [0,4194304] 0 2024-02-16T22:22:15.289 INFO:tasks.workunit.client.0.smithi114.stdout:2/817: creat dd/d3b/d8a/f10b x:0 0 0 2024-02-16T22:22:15.291 INFO:tasks.workunit.client.1.smithi123.stdout:5/997: dread - d6/dd/df/f8c zero size 2024-02-16T22:22:15.292 INFO:tasks.workunit.client.1.smithi123.stdout:0/940: creat d1/d8/d13/d1d/d64/dbc/f139 x:0 0 0 2024-02-16T22:22:15.293 INFO:tasks.workunit.client.1.smithi123.stdout:0/941: fsync d1/d8/f34 0 2024-02-16T22:22:15.295 INFO:tasks.workunit.client.1.smithi123.stdout:6/980: dwrite d0/d96/f155 [0,4194304] 0 2024-02-16T22:22:15.296 INFO:tasks.workunit.client.1.smithi123.stdout:3/997: symlink d0/d20/d29/d9b/dbd/df4/d37/l149 0 2024-02-16T22:22:15.297 INFO:tasks.workunit.client.1.smithi123.stdout:6/981: chown d0/d6/d91 1039 1 2024-02-16T22:22:15.297 INFO:tasks.workunit.client.0.smithi114.stdout:2/818: link dd/d4e/c106 dd/d6b/db8/c10c 0 2024-02-16T22:22:15.297 INFO:tasks.workunit.client.1.smithi123.stdout:3/998: chown d0/da0/ff2 1368710 1 2024-02-16T22:22:15.300 INFO:tasks.workunit.client.1.smithi123.stdout:2/966: creat d4/d1a/dd5/d10b/df1/d125/f13a x:0 0 0 2024-02-16T22:22:15.302 INFO:tasks.workunit.client.1.smithi123.stdout:0/942: truncate d1/d8/d13/fca 4008436 0 2024-02-16T22:22:15.302 INFO:tasks.workunit.client.1.smithi123.stdout:3/999: fsync d0/d20/d29/d9b/dbd/df4/d5d/dae/db0/f104 0 2024-02-16T22:22:15.307 INFO:tasks.workunit.client.1.smithi123.stdout:2/967: creat d4/d1a/dd5/d10b/df1/db1/de3/f13b x:0 0 0 2024-02-16T22:22:15.308 INFO:tasks.workunit.client.1.smithi123.stdout:1/973: dread d3/d6/f78 [0,4194304] 0 2024-02-16T22:22:15.308 INFO:tasks.workunit.client.1.smithi123.stdout:5/998: creat d6/dd/df/f15c x:0 0 0 2024-02-16T22:22:15.309 INFO:tasks.workunit.client.1.smithi123.stdout:2/968: chown d4/d1a/dd5/d10b/df1/d125 17425298 1 2024-02-16T22:22:15.309 INFO:tasks.workunit.client.0.smithi114.stdout:2/819: getdents dd/d3b/da7/d4c/dc7 0 2024-02-16T22:22:15.311 INFO:tasks.workunit.client.0.smithi114.stdout:2/820: read dd/df/d1c/d21/d61/d63/f73 [1839203,119032] 0 2024-02-16T22:22:15.311 INFO:tasks.workunit.client.1.smithi123.stdout:1/974: truncate d3/d6/db/d10/d2d/d8c/dfd/d9f/d10c/f143 691470 0 2024-02-16T22:22:15.311 INFO:tasks.workunit.client.1.smithi123.stdout:6/982: mknod d0/d9/dc/d3a/d70/c161 0 2024-02-16T22:22:15.312 INFO:tasks.workunit.client.1.smithi123.stdout:5/999: creat d6/d3b/f15d x:0 0 0 2024-02-16T22:22:15.315 INFO:tasks.workunit.client.1.smithi123.stdout:2/969: rmdir d4/d1a/d1d/d38/d7b 39 2024-02-16T22:22:15.316 INFO:tasks.workunit.client.0.smithi114.stdout:2/821: fdatasync dd/d6b/fbf 0 2024-02-16T22:22:15.317 INFO:tasks.workunit.client.1.smithi123.stdout:0/943: dread d1/d8/f2f [0,4194304] 0 2024-02-16T22:22:15.321 INFO:tasks.workunit.client.1.smithi123.stdout:1/975: mknod d3/d6/d49/d48/d84/c14e 0 2024-02-16T22:22:15.322 INFO:tasks.workunit.client.1.smithi123.stdout:0/944: dwrite d1/da0/faa [0,4194304] 0 2024-02-16T22:22:15.323 INFO:tasks.workunit.client.1.smithi123.stdout:2/970: symlink d4/d1a/dd5/d10b/df1/d4a/dce/d115/l13c 0 2024-02-16T22:22:15.325 INFO:tasks.workunit.client.0.smithi114.stdout:2/822: symlink dd/d3b/d8a/dcb/de2/l10d 0 2024-02-16T22:22:15.327 INFO:tasks.workunit.client.1.smithi123.stdout:6/983: symlink d0/d8a/de6/l162 0 2024-02-16T22:22:15.331 INFO:tasks.workunit.client.0.smithi114.stdout:2/823: dread dd/d6b/da5/fc1 [0,4194304] 0 2024-02-16T22:22:15.332 INFO:tasks.workunit.client.1.smithi123.stdout:1/976: creat d3/d6/d11/f14f x:0 0 0 2024-02-16T22:22:15.334 INFO:tasks.workunit.client.1.smithi123.stdout:0/945: truncate d1/d8/d13/d1d/d64/f84 3571233 0 2024-02-16T22:22:15.336 INFO:tasks.workunit.client.1.smithi123.stdout:6/984: mkdir d0/d9/dc/d39/d163 0 2024-02-16T22:22:15.339 INFO:tasks.workunit.client.1.smithi123.stdout:1/977: rename d3/d6/db/d10/d2a/ca6 to d3/d6/d18/c150 0 2024-02-16T22:22:15.340 INFO:tasks.workunit.client.1.smithi123.stdout:0/946: symlink d1/d4/d17/d54/db4/l13a 0 2024-02-16T22:22:15.341 INFO:tasks.workunit.client.1.smithi123.stdout:0/947: chown d1/d4/c36 30343862 1 2024-02-16T22:22:15.341 INFO:tasks.workunit.client.0.smithi114.stdout:2/824: creat dd/df/d1c/d21/f10e x:0 0 0 2024-02-16T22:22:15.343 INFO:tasks.workunit.client.0.smithi114.stdout:2/825: write dd/d3b/d8a/dcb/fda [211725,67295] 0 2024-02-16T22:22:15.344 INFO:tasks.workunit.client.1.smithi123.stdout:0/948: creat d1/d8/da7/f13b x:0 0 0 2024-02-16T22:22:15.347 INFO:tasks.workunit.client.1.smithi123.stdout:1/978: creat d3/d6/d49/d48/d68/d126/f151 x:0 0 0 2024-02-16T22:22:15.348 INFO:tasks.workunit.client.1.smithi123.stdout:0/949: read d1/d8/d13/d1d/d64/dbc/ddb/f11b [152670,114230] 0 2024-02-16T22:22:15.348 INFO:tasks.workunit.client.1.smithi123.stdout:0/950: stat d1/d4/l18 0 2024-02-16T22:22:15.349 INFO:tasks.workunit.client.1.smithi123.stdout:1/979: symlink d3/d6/d49/d48/d84/d136/l152 0 2024-02-16T22:22:15.349 INFO:tasks.workunit.client.1.smithi123.stdout:1/980: stat d3/d6/db/d10/d2a/d80/cac 0 2024-02-16T22:22:15.350 INFO:tasks.workunit.client.0.smithi114.stdout:2/826: dwrite dd/d3b/d8a/fe8 [4194304,4194304] 0 2024-02-16T22:22:15.351 INFO:tasks.workunit.client.1.smithi123.stdout:1/981: read d3/d6/d49/d48/d84/d92/ffe [3857915,101736] 0 2024-02-16T22:22:15.380 INFO:tasks.workunit.client.0.smithi114.stdout:2/827: getdents dd/d6b/da5/df6 0 2024-02-16T22:22:15.386 INFO:tasks.workunit.client.1.smithi123.stdout:2/971: dwrite d4/d1a/d1d/d38/d7b/fc4 [0,4194304] 0 2024-02-16T22:22:15.386 INFO:tasks.workunit.client.0.smithi114.stdout:2/828: dwrite dd/d3b/da7/d4c/f79 [0,4194304] 0 2024-02-16T22:22:15.387 INFO:tasks.workunit.client.1.smithi123.stdout:1/982: rmdir d3/d6/db/d10/d4b/d70/d87/d128 0 2024-02-16T22:22:15.392 INFO:tasks.workunit.client.1.smithi123.stdout:0/951: write d1/d8/d13/f126 [1000363,55833] 0 2024-02-16T22:22:15.392 INFO:tasks.workunit.client.0.smithi114.stdout:2/829: creat dd/d6b/da5/f10f x:0 0 0 2024-02-16T22:22:15.393 INFO:tasks.workunit.client.1.smithi123.stdout:1/983: mknod d3/d6/db/d10/d4b/c153 0 2024-02-16T22:22:15.399 INFO:tasks.workunit.client.1.smithi123.stdout:1/984: stat d3/d47/c12e 0 2024-02-16T22:22:15.401 INFO:tasks.workunit.client.1.smithi123.stdout:0/952: symlink d1/d8/d13/d1d/d64/dbc/d114/l13c 0 2024-02-16T22:22:15.406 INFO:tasks.workunit.client.1.smithi123.stdout:0/953: dwrite d1/d8/d13/d1d/d64/dbc/fac [0,4194304] 0 2024-02-16T22:22:15.407 INFO:tasks.workunit.client.1.smithi123.stdout:0/954: chown d1/d4/d17/f112 2389 1 2024-02-16T22:22:15.411 INFO:tasks.workunit.client.1.smithi123.stdout:0/955: link d1/d8/f34 d1/d4/d17/d74/de7/f13d 0 2024-02-16T22:22:15.416 INFO:tasks.workunit.client.1.smithi123.stdout:0/956: link d1/d4/feb d1/d4/d17/d54/d8e/d96/dee/f13e 0 2024-02-16T22:22:15.416 INFO:tasks.workunit.client.1.smithi123.stdout:0/957: stat d1/d4/l45 0 2024-02-16T22:22:15.417 INFO:tasks.workunit.client.1.smithi123.stdout:2/972: truncate d4/d1a/d1d/f2b 3761072 0 2024-02-16T22:22:15.418 INFO:tasks.workunit.client.1.smithi123.stdout:6/985: sync 2024-02-16T22:22:15.418 INFO:tasks.workunit.client.1.smithi123.stdout:0/958: fdatasync d1/d4/d17/d54/f80 0 2024-02-16T22:22:15.420 INFO:tasks.workunit.client.1.smithi123.stdout:2/973: fdatasync d4/d1a/d1d/d13/d25/d72/f73 0 2024-02-16T22:22:15.422 INFO:tasks.workunit.client.0.smithi114.stdout:2/830: dwrite dd/d6b/f89 [0,4194304] 0 2024-02-16T22:22:15.423 INFO:tasks.workunit.client.1.smithi123.stdout:1/985: dwrite d3/d6/db/d10/d2a/fc1 [4194304,4194304] 0 2024-02-16T22:22:15.433 INFO:tasks.workunit.client.1.smithi123.stdout:1/986: rmdir d3/d6/d49/d48/d84/d92 39 2024-02-16T22:22:15.434 INFO:tasks.workunit.client.1.smithi123.stdout:6/986: write d0/d6/d4d/db0/ffb [1473645,54553] 0 2024-02-16T22:22:15.437 INFO:tasks.workunit.client.1.smithi123.stdout:1/987: read - d3/d6/db/d10/d4b/d70/d87/f113 zero size 2024-02-16T22:22:15.438 INFO:tasks.workunit.client.1.smithi123.stdout:6/987: symlink d0/d96/l164 0 2024-02-16T22:22:15.438 INFO:tasks.workunit.client.1.smithi123.stdout:0/959: dwrite d1/d8/d13/f37 [0,4194304] 0 2024-02-16T22:22:15.442 INFO:tasks.workunit.client.0.smithi114.stdout:2/831: getdents dd/df/d1c/d21/d61/dec 0 2024-02-16T22:22:15.442 INFO:tasks.workunit.client.1.smithi123.stdout:2/974: dread d4/d1a/d1d/d38/f92 [0,4194304] 0 2024-02-16T22:22:15.443 INFO:tasks.workunit.client.0.smithi114.stdout:2/832: stat dd/df/d1c/d21/d61/d63 0 2024-02-16T22:22:15.443 INFO:tasks.workunit.client.1.smithi123.stdout:6/988: creat d0/d6/d1a/d88/d8b/f165 x:0 0 0 2024-02-16T22:22:15.444 INFO:tasks.workunit.client.1.smithi123.stdout:0/960: mkdir d1/d4/d17/d54/db4/dc5/d13f 0 2024-02-16T22:22:15.444 INFO:tasks.workunit.client.1.smithi123.stdout:0/961: dread - d1/d4/d42/fb7 zero size 2024-02-16T22:22:15.445 INFO:tasks.workunit.client.0.smithi114.stdout:2/833: mknod dd/df/d1c/c110 0 2024-02-16T22:22:15.446 INFO:tasks.workunit.client.1.smithi123.stdout:2/975: fdatasync d4/d1a/d1d/d59/d45/fed 0 2024-02-16T22:22:15.446 INFO:tasks.workunit.client.0.smithi114.stdout:2/834: fsync dd/df/ddb/ff7 0 2024-02-16T22:22:15.448 INFO:tasks.workunit.client.1.smithi123.stdout:1/988: creat d3/d6/db/d10/f154 x:0 0 0 2024-02-16T22:22:15.450 INFO:tasks.workunit.client.0.smithi114.stdout:2/835: mkdir dd/d3b/da7/d8b/d111 0 2024-02-16T22:22:15.451 INFO:tasks.workunit.client.1.smithi123.stdout:2/976: mknod d4/d1a/d1d/d13/d25/d98/dbb/d10d/c13d 0 2024-02-16T22:22:15.453 INFO:tasks.workunit.client.1.smithi123.stdout:0/962: dread d1/d4/d17/d54/d8e/d96/dee/f135 [0,4194304] 0 2024-02-16T22:22:15.496 INFO:tasks.workunit.client.0.smithi114.stdout:9/771: sync 2024-02-16T22:22:15.497 INFO:tasks.workunit.client.0.smithi114.stdout:9/772: write d2/f15 [2858736,106498] 0 2024-02-16T22:22:15.508 INFO:tasks.workunit.client.1.smithi123.stdout:2/977: readlink d4/d1a/d1d/d13/d25/lef 0 2024-02-16T22:22:15.510 INFO:tasks.workunit.client.1.smithi123.stdout:0/963: mknod d1/d4/d42/c140 0 2024-02-16T22:22:15.514 INFO:tasks.workunit.client.1.smithi123.stdout:0/964: dwrite d1/d4/d17/d74/de7/ff5 [0,4194304] 0 2024-02-16T22:22:15.515 INFO:tasks.workunit.client.1.smithi123.stdout:0/965: dread - d1/d8/da7/f137 zero size 2024-02-16T22:22:15.587 INFO:tasks.workunit.client.1.smithi123.stdout:6/989: write d0/d6/d1a/d2e/f50 [1312938,40770] 0 2024-02-16T22:22:15.591 INFO:tasks.workunit.client.1.smithi123.stdout:2/978: truncate d4/d1a/d1d/d38/d5e/fd2 768134 0 2024-02-16T22:22:15.591 INFO:tasks.workunit.client.0.smithi114.stdout:2/836: dread dd/df/d1c/d21/d61/fe4 [0,4194304] 0 2024-02-16T22:22:15.592 INFO:tasks.workunit.client.0.smithi114.stdout:2/837: chown dd/df/d1c/d21/d61/d69 2 1 2024-02-16T22:22:15.593 INFO:tasks.workunit.client.1.smithi123.stdout:6/990: dwrite d0/d6/d2f/da5/f15a [0,4194304] 0 2024-02-16T22:22:15.593 INFO:tasks.workunit.client.0.smithi114.stdout:2/838: chown dd/d3b/d8a/f93 4 1 2024-02-16T22:22:15.593 INFO:tasks.workunit.client.1.smithi123.stdout:6/991: chown d0/d9/c25 251472100 1 2024-02-16T22:22:15.594 INFO:tasks.workunit.client.0.smithi114.stdout:2/839: dread - dd/d3b/da7/d4c/ff2 zero size 2024-02-16T22:22:15.596 INFO:tasks.workunit.client.0.smithi114.stdout:9/773: write d2/dd/d3c/d89/d9e/fa2 [2973896,97510] 0 2024-02-16T22:22:15.601 INFO:tasks.workunit.client.1.smithi123.stdout:1/989: getdents d3/d6/db/d10/d2a/d53 0 2024-02-16T22:22:15.601 INFO:tasks.workunit.client.1.smithi123.stdout:0/966: dread d1/d4/d17/d54/db4/dc5/fe0 [0,4194304] 0 2024-02-16T22:22:15.602 INFO:tasks.workunit.client.1.smithi123.stdout:1/990: chown d3/d6/db/d10/d2d/d8c/dfd/f51 1653 1 2024-02-16T22:22:15.603 INFO:tasks.workunit.client.1.smithi123.stdout:1/991: fdatasync d3/fe 0 2024-02-16T22:22:15.604 INFO:tasks.workunit.client.1.smithi123.stdout:2/979: stat d4/d1a/d1d/d13/d15 0 2024-02-16T22:22:15.612 INFO:tasks.workunit.client.1.smithi123.stdout:0/967: dread d1/d8/d13/fca [0,4194304] 0 2024-02-16T22:22:15.613 INFO:tasks.workunit.client.1.smithi123.stdout:0/968: stat d1/d4/d17/dad/db2/l12c 0 2024-02-16T22:22:15.615 INFO:tasks.workunit.client.1.smithi123.stdout:2/980: creat d4/d1a/d1d/d13/d25/d72/f13e x:0 0 0 2024-02-16T22:22:15.616 INFO:tasks.workunit.client.1.smithi123.stdout:6/992: mkdir d0/d96/de3/dcc/d14f/d166 0 2024-02-16T22:22:15.620 INFO:tasks.workunit.client.1.smithi123.stdout:0/969: creat d1/d4/d17/d54/d8e/d96/dee/f141 x:0 0 0 2024-02-16T22:22:15.622 INFO:tasks.workunit.client.1.smithi123.stdout:6/993: dwrite d0/d6/d1a/d88/d8b/f165 [0,4194304] 0 2024-02-16T22:22:15.623 INFO:tasks.workunit.client.1.smithi123.stdout:6/994: write d0/d6/d91/d93/d116/f141 [20321,75656] 0 2024-02-16T22:22:15.623 INFO:tasks.workunit.client.1.smithi123.stdout:1/992: write d3/d6/db/d10/f85 [2682713,130102] 0 2024-02-16T22:22:15.627 INFO:tasks.workunit.client.1.smithi123.stdout:6/995: mkdir d0/d74/d167 0 2024-02-16T22:22:15.629 INFO:tasks.workunit.client.1.smithi123.stdout:2/981: creat d4/d1a/d1d/d59/dd7/f13f x:0 0 0 2024-02-16T22:22:15.632 INFO:tasks.workunit.client.1.smithi123.stdout:1/993: unlink d3/d6/d11/d8b/dbb/f139 0 2024-02-16T22:22:15.634 INFO:tasks.workunit.client.1.smithi123.stdout:6/996: read - d0/d6/d91/da2/fbb zero size 2024-02-16T22:22:15.636 INFO:tasks.workunit.client.1.smithi123.stdout:2/982: mknod d4/d1a/d1d/d13/d25/d98/c140 0 2024-02-16T22:22:15.638 INFO:tasks.workunit.client.1.smithi123.stdout:0/970: write d1/d8/d13/d1d/d64/fcf [991684,52564] 0 2024-02-16T22:22:15.639 INFO:tasks.workunit.client.1.smithi123.stdout:0/971: dread - d1/d8/d13/d1d/d62/d7a/f11f zero size 2024-02-16T22:22:15.642 INFO:tasks.workunit.client.1.smithi123.stdout:6/997: mknod d0/d6/d2f/d5d/c168 0 2024-02-16T22:22:15.649 INFO:tasks.workunit.client.0.smithi114.stdout:0/816: sync 2024-02-16T22:22:15.655 INFO:tasks.workunit.client.1.smithi123.stdout:1/994: link d3/d6/d49/f50 d3/d6/db/d10/d4b/d70/d87/d105/f155 0 2024-02-16T22:22:15.658 INFO:tasks.workunit.client.1.smithi123.stdout:0/972: symlink d1/d8/l142 0 2024-02-16T22:22:15.658 INFO:tasks.workunit.client.1.smithi123.stdout:1/995: dread d3/d6/db/d55/f103 [0,4194304] 0 2024-02-16T22:22:15.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.658+0000 7f127cc83700 1 -- 172.21.15.114:0/2538757722 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1278100cb0 msgr2=0x7f1278101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:15.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.658+0000 7f127cc83700 1 --2- 172.21.15.114:0/2538757722 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1278100cb0 0x7f1278101110 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f126c009910 tx=0x7f126c009c20 comp rx=0 tx=0).stop 2024-02-16T22:22:15.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.659+0000 7f127cc83700 1 -- 172.21.15.114:0/2538757722 shutdown_connections 2024-02-16T22:22:15.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.659+0000 7f127cc83700 1 --2- 172.21.15.114:0/2538757722 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1278100cb0 0x7f1278101110 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:15.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.659+0000 7f127cc83700 1 --2- 172.21.15.114:0/2538757722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12780ffab0 0x7f12780ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:15.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.659+0000 7f127cc83700 1 -- 172.21.15.114:0/2538757722 >> 172.21.15.114:0/2538757722 conn(0x7f12780fb030 msgr2=0x7f12780fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:22:15.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.659+0000 7f127cc83700 1 -- 172.21.15.114:0/2538757722 shutdown_connections 2024-02-16T22:22:15.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.659+0000 7f127cc83700 1 -- 172.21.15.114:0/2538757722 wait complete. 2024-02-16T22:22:15.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.660+0000 7f127cc83700 1 Processor -- start 2024-02-16T22:22:15.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.660+0000 7f127cc83700 1 -- start start 2024-02-16T22:22:15.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.660+0000 7f127cc83700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12780ffab0 0x7f1278193f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:15.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.660+0000 7f127cc83700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1278100cb0 0x7f1278194480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:15.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.660+0000 7f127cc83700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1278194aa0 con 0x7f12780ffab0 2024-02-16T22:22:15.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.660+0000 7f127cc83700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1278194be0 con 0x7f1278100cb0 2024-02-16T22:22:15.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.660+0000 7f127659c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12780ffab0 0x7f1278193f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:15.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.660+0000 7f127659c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12780ffab0 0x7f1278193f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50070/0 (socket says 172.21.15.114:50070) 2024-02-16T22:22:15.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.660+0000 7f127659c700 1 -- 172.21.15.114:0/2042010792 learned_addr learned my addr 172.21.15.114:0/2042010792 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:22:15.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.660+0000 7f1275d9b700 1 --2- 172.21.15.114:0/2042010792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1278100cb0 0x7f1278194480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:15.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.661+0000 7f127659c700 1 -- 172.21.15.114:0/2042010792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1278100cb0 msgr2=0x7f1278194480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:15.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.661+0000 7f127659c700 1 --2- 172.21.15.114:0/2042010792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1278100cb0 0x7f1278194480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:15.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.661+0000 7f127659c700 1 -- 172.21.15.114:0/2042010792 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f126c0095f0 con 0x7f12780ffab0 2024-02-16T22:22:15.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.661+0000 7f127659c700 1 --2- 172.21.15.114:0/2042010792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12780ffab0 0x7f1278193f40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f126800e8f0 tx=0x7f126800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:22:15.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.661+0000 7f12677fe700 1 -- 172.21.15.114:0/2042010792 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1268009790 con 0x7f12780ffab0 2024-02-16T22:22:15.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.661+0000 7f12677fe700 1 -- 172.21.15.114:0/2042010792 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f126800c810 con 0x7f12780ffab0 2024-02-16T22:22:15.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.661+0000 7f12677fe700 1 -- 172.21.15.114:0/2042010792 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1268003d20 con 0x7f12780ffab0 2024-02-16T22:22:15.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.661+0000 7f127cc83700 1 -- 172.21.15.114:0/2042010792 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1278199690 con 0x7f12780ffab0 2024-02-16T22:22:15.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.662+0000 7f127cc83700 1 -- 172.21.15.114:0/2042010792 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1278199be0 con 0x7f12780ffab0 2024-02-16T22:22:15.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.663+0000 7f127cc83700 1 -- 172.21.15.114:0/2042010792 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1278106690 con 0x7f12780ffab0 2024-02-16T22:22:15.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.664+0000 7f12677fe700 1 -- 172.21.15.114:0/2042010792 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) v1 ==== 91450+0+0 (secure 0 0 0) 0x7f12680041f0 con 0x7f12780ffab0 2024-02-16T22:22:15.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.665+0000 7f12677fe700 1 --2- 172.21.15.114:0/2042010792 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f126006d6a0 0x7f126006fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:15.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.665+0000 7f12677fe700 1 -- 172.21.15.114:0/2042010792 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5726+0+0 (secure 0 0 0) 0x7f1268014070 con 0x7f12780ffab0 2024-02-16T22:22:15.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.665+0000 7f1275d9b700 1 --2- 172.21.15.114:0/2042010792 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f126006d6a0 0x7f126006fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:15.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.665+0000 7f1275d9b700 1 --2- 172.21.15.114:0/2042010792 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f126006d6a0 0x7f126006fb60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f126c006010 tx=0x7f126c009f70 comp rx=0 tx=0).ready entity=mgr.24441 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:22:15.670 INFO:tasks.workunit.client.1.smithi123.stdout:1/996: mknod d3/d6/db/d10/d2d/d8c/dfd/d9f/db4/ddf/df0/c156 0 2024-02-16T22:22:15.672 INFO:tasks.workunit.client.1.smithi123.stdout:2/983: dwrite d4/d1a/d1d/f9f [4194304,4194304] 0 2024-02-16T22:22:15.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:15.672+0000 7f12677fe700 1 -- 172.21.15.114:0/2042010792 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182869 (secure 0 0 0) 0x7f126805ac90 con 0x7f12780ffab0 2024-02-16T22:22:15.676 INFO:tasks.workunit.client.1.smithi123.stdout:0/973: truncate d1/f6a 3562568 0 2024-02-16T22:22:15.677 INFO:tasks.workunit.client.1.smithi123.stdout:6/998: write d0/d6/d2f/f2a [4997777,120350] 0 2024-02-16T22:22:15.677 INFO:tasks.workunit.client.1.smithi123.stdout:0/974: chown d1/da0/fb9 2 1 2024-02-16T22:22:15.677 INFO:tasks.workunit.client.1.smithi123.stdout:6/999: chown d0/d6/f8 863997988 1 2024-02-16T22:22:15.684 INFO:tasks.workunit.client.1.smithi123.stdout:2/984: rename d4/d1a/dd5/d10b/df1/d4a/c117 to d4/d1a/d1d/d59/dd7/ddc/c141 0 2024-02-16T22:22:15.684 INFO:tasks.workunit.client.1.smithi123.stdout:1/997: creat d3/d6/d49/d48/f157 x:0 0 0 2024-02-16T22:22:15.686 INFO:tasks.workunit.client.0.smithi114.stdout:0/817: chown d1/d16/d48/d98/fc4 928 1 2024-02-16T22:22:15.688 INFO:tasks.workunit.client.0.smithi114.stdout:9/774: mknod d2/d5c/cf8 0 2024-02-16T22:22:15.690 INFO:tasks.workunit.client.0.smithi114.stdout:0/818: mkdir d1/d1a/d47/d62/d89/d104 0 2024-02-16T22:22:15.691 INFO:tasks.workunit.client.1.smithi123.stdout:0/975: mknod d1/d4/d17/dad/d121/c143 0 2024-02-16T22:22:15.695 INFO:tasks.workunit.client.1.smithi123.stdout:2/985: mknod d4/d1a/d1d/d13/d25/d98/dbb/c142 0 2024-02-16T22:22:15.696 INFO:tasks.workunit.client.1.smithi123.stdout:0/976: mknod d1/d8/d13/d1d/d64/dbc/c144 0 2024-02-16T22:22:15.696 INFO:tasks.workunit.client.1.smithi123.stdout:0/977: chown d1/d8/d13/d1d/lec 1218976600 1 2024-02-16T22:22:15.697 INFO:tasks.workunit.client.0.smithi114.stdout:0/819: mknod d1/d16/d1f/d28/d51/d61/d6d/dfc/c105 0 2024-02-16T22:22:15.705 INFO:tasks.workunit.client.1.smithi123.stdout:1/998: link d3/d6/d49/d48/d84/d136/l152 d3/d6/d49/d48/d84/d92/df9/l158 0 2024-02-16T22:22:15.706 INFO:tasks.workunit.client.1.smithi123.stdout:1/999: fsync d3/d6/db/d10/d4b/d70/f94 0 2024-02-16T22:22:15.714 INFO:tasks.workunit.client.0.smithi114.stdout:9/775: truncate d2/d5c/dc9/d1d/f62 2229946 0 2024-02-16T22:22:15.714 INFO:tasks.workunit.client.0.smithi114.stdout:9/776: chown d2/fc 77260956 1 2024-02-16T22:22:15.716 INFO:tasks.workunit.client.0.smithi114.stdout:9/777: chown d2/d5c/dc9/d47/d63/dbb/fef 1239823 1 2024-02-16T22:22:15.717 INFO:tasks.workunit.client.0.smithi114.stdout:0/820: dread d1/d1a/f4e [0,4194304] 0 2024-02-16T22:22:15.727 INFO:tasks.workunit.client.0.smithi114.stdout:9/778: symlink d2/d5c/dc9/d47/d63/lf9 0 2024-02-16T22:22:15.728 INFO:tasks.workunit.client.0.smithi114.stdout:0/821: dread d1/d16/f73 [0,4194304] 0 2024-02-16T22:22:15.729 INFO:tasks.workunit.client.0.smithi114.stdout:0/822: chown d1/d16/d1f/d28/d51/d61/l75 139817 1 2024-02-16T22:22:15.729 INFO:tasks.workunit.client.1.smithi123.stdout:2/986: write d4/d1a/d1d/f2c [1568495,91667] 0 2024-02-16T22:22:15.731 INFO:tasks.workunit.client.0.smithi114.stdout:9/779: truncate d2/d1a/d65/f94 736331 0 2024-02-16T22:22:15.731 INFO:tasks.workunit.client.1.smithi123.stdout:0/978: write d1/d8/d13/d1d/f8d [317094,117993] 0 2024-02-16T22:22:15.735 INFO:tasks.workunit.client.1.smithi123.stdout:2/987: dwrite d4/d1a/d1d/d59/f3a [0,4194304] 0 2024-02-16T22:22:15.737 INFO:tasks.workunit.client.1.smithi123.stdout:0/979: mkdir d1/d4/d17/d74/de7/d145 0 2024-02-16T22:22:15.742 INFO:tasks.workunit.client.1.smithi123.stdout:2/988: dwrite d4/d1a/d1d/d59/dd7/f13f [0,4194304] 0 2024-02-16T22:22:15.746 INFO:tasks.workunit.client.1.smithi123.stdout:0/980: link d1/d8/da7/ldf d1/d4/d17/d54/db4/dc5/d138/l146 0 2024-02-16T22:22:15.751 INFO:tasks.workunit.client.0.smithi114.stdout:0/823: write d1/d16/d48/d98/fde [678866,115659] 0 2024-02-16T22:22:15.751 INFO:tasks.workunit.client.1.smithi123.stdout:2/989: rename d4/d1a/d1d/d13/d25/l4c to d4/d1a/d1d/d13/d25/d72/dc8/l143 0 2024-02-16T22:22:15.752 INFO:tasks.workunit.client.0.smithi114.stdout:9/780: write d2/fc [3441438,67878] 0 2024-02-16T22:22:15.757 INFO:tasks.workunit.client.0.smithi114.stdout:9/781: unlink d2/f5 0 2024-02-16T22:22:15.757 INFO:tasks.workunit.client.0.smithi114.stdout:9/782: write d2/f15 [5514581,70843] 0 2024-02-16T22:22:15.761 INFO:tasks.workunit.client.0.smithi114.stdout:0/824: creat d1/d16/d1f/d28/d51/d61/d7e/d90/d95/d80/f106 x:0 0 0 2024-02-16T22:22:15.772 INFO:tasks.workunit.client.0.smithi114.stdout:5/745: sync 2024-02-16T22:22:15.772 INFO:tasks.workunit.client.0.smithi114.stdout:3/749: sync 2024-02-16T22:22:15.772 INFO:tasks.workunit.client.0.smithi114.stdout:6/774: sync 2024-02-16T22:22:15.772 INFO:tasks.workunit.client.0.smithi114.stdout:8/754: sync 2024-02-16T22:22:15.772 INFO:tasks.workunit.client.0.smithi114.stdout:1/744: sync 2024-02-16T22:22:15.779 INFO:tasks.workunit.client.0.smithi114.stdout:1/745: readlink d4/l1f 0 2024-02-16T22:22:15.787 INFO:tasks.workunit.client.0.smithi114.stdout:9/783: write d2/dd/f2f [119499,41414] 0 2024-02-16T22:22:15.791 INFO:tasks.workunit.client.0.smithi114.stdout:0/825: write d1/d16/d1f/d28/d51/d60/dd1/ff7 [1192144,30373] 0 2024-02-16T22:22:15.793 INFO:tasks.workunit.client.0.smithi114.stdout:5/746: symlink d0/d28/d98/d25/d2e/d3b/db5/lf1 0 2024-02-16T22:22:15.796 INFO:tasks.workunit.client.0.smithi114.stdout:1/746: chown d4/f21 623 1 2024-02-16T22:22:15.798 INFO:tasks.workunit.client.0.smithi114.stdout:9/784: dwrite d2/d33/d3f/ddd/fe5 [0,4194304] 0 2024-02-16T22:22:15.800 INFO:tasks.workunit.client.0.smithi114.stdout:1/747: write d4/d9/d46/d67/f6e [5812526,8187] 0 2024-02-16T22:22:15.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:15 smithi123 ceph-mon[37161]: pgmap v35: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 81 MiB/s rd, 175 MiB/s wr, 403 op/s 2024-02-16T22:22:15.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:15 smithi123 ceph-mon[37161]: Standby manager daemon smithi114.vcullz restarted 2024-02-16T22:22:15.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:15 smithi123 ceph-mon[37161]: Standby manager daemon smithi114.vcullz started 2024-02-16T22:22:15.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:15 smithi123 ceph-mon[37161]: from='mgr.? 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi114.vcullz/crt"}]: dispatch 2024-02-16T22:22:15.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:15 smithi123 ceph-mon[37161]: from='mgr.? 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T22:22:15.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:15 smithi123 ceph-mon[37161]: from='mgr.? 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi114.vcullz/key"}]: dispatch 2024-02-16T22:22:15.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:15 smithi123 ceph-mon[37161]: from='mgr.? 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T22:22:15.805 INFO:tasks.workunit.client.0.smithi114.stdout:3/750: dread d4/d43/f56 [0,4194304] 0 2024-02-16T22:22:15.806 INFO:tasks.workunit.client.0.smithi114.stdout:0/826: unlink d1/d16/d1f/d28/d51/d61/d7e/d90/fc1 0 2024-02-16T22:22:15.807 INFO:tasks.workunit.client.0.smithi114.stdout:3/751: chown d4/d24/d2d/d45/d74/dc7/fcd 142918 1 2024-02-16T22:22:15.808 INFO:tasks.workunit.client.0.smithi114.stdout:0/827: fsync d1/d1a/d3e/fa3 0 2024-02-16T22:22:15.849 INFO:tasks.workunit.client.0.smithi114.stdout:4/915: sync 2024-02-16T22:22:15.849 INFO:tasks.workunit.client.0.smithi114.stdout:2/840: sync 2024-02-16T22:22:15.849 INFO:tasks.workunit.client.0.smithi114.stdout:7/848: sync 2024-02-16T22:22:15.850 INFO:tasks.workunit.client.0.smithi114.stdout:8/755: dwrite d4/d16/d25/d7c/f8d [4194304,4194304] 0 2024-02-16T22:22:15.850 INFO:tasks.workunit.client.0.smithi114.stdout:3/752: write d4/dd/f1e [7248099,79800] 0 2024-02-16T22:22:15.853 INFO:tasks.workunit.client.0.smithi114.stdout:5/747: chown d0/d28/d98/l72 168859 1 2024-02-16T22:22:15.859 INFO:tasks.workunit.client.0.smithi114.stdout:4/916: dread dd/d59/d8f/dea/d134/f142 [0,4194304] 0 2024-02-16T22:22:15.861 INFO:tasks.workunit.client.0.smithi114.stdout:4/917: dread - dd/d1b/d24/d57/d12b/d38/d87/dad/ffb zero size 2024-02-16T22:22:15.862 INFO:tasks.workunit.client.0.smithi114.stdout:8/756: dread d4/d11/d78/d7a/d7e/fba [0,4194304] 0 2024-02-16T22:22:15.871 INFO:tasks.workunit.client.0.smithi114.stdout:0/828: creat d1/d16/d48/d98/f107 x:0 0 0 2024-02-16T22:22:15.872 INFO:tasks.workunit.client.0.smithi114.stdout:0/829: chown d1/d16/c1e 6272 1 2024-02-16T22:22:15.873 INFO:tasks.workunit.client.0.smithi114.stdout:3/753: mknod d4/dbe/c103 0 2024-02-16T22:22:15.873 INFO:tasks.workunit.client.0.smithi114.stdout:0/830: chown d1/d16/d1f/d28/d51/d60/dd1/fe8 243998380 1 2024-02-16T22:22:15.876 INFO:tasks.workunit.client.0.smithi114.stdout:7/849: unlink d3/d22/d76/cf1 0 2024-02-16T22:22:15.884 INFO:tasks.workunit.client.0.smithi114.stdout:6/775: getdents d6/dd/d25/d34/d61/d62/d8b 0 2024-02-16T22:22:15.886 INFO:tasks.workunit.client.0.smithi114.stdout:1/748: mknod d4/d9/d3d/def/c110 0 2024-02-16T22:22:15.887 INFO:tasks.workunit.client.0.smithi114.stdout:1/749: chown d4/d9/d46/d4d/f5d 25161 1 2024-02-16T22:22:15.890 INFO:tasks.workunit.client.0.smithi114.stdout:2/841: dread dd/df/d1c/f40 [0,4194304] 0 2024-02-16T22:22:15.892 INFO:tasks.workunit.client.0.smithi114.stdout:4/918: creat dd/d2e/d3f/f149 x:0 0 0 2024-02-16T22:22:15.893 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:15 smithi114 ceph-mon[29235]: pgmap v35: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 81 MiB/s rd, 175 MiB/s wr, 403 op/s 2024-02-16T22:22:15.893 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:15 smithi114 ceph-mon[29235]: Standby manager daemon smithi114.vcullz restarted 2024-02-16T22:22:15.894 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:15 smithi114 ceph-mon[29235]: Standby manager daemon smithi114.vcullz started 2024-02-16T22:22:15.894 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:15 smithi114 ceph-mon[29235]: from='mgr.? 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi114.vcullz/crt"}]: dispatch 2024-02-16T22:22:15.894 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:15 smithi114 ceph-mon[29235]: from='mgr.? 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T22:22:15.894 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:15 smithi114 ceph-mon[29235]: from='mgr.? 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi114.vcullz/key"}]: dispatch 2024-02-16T22:22:15.894 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:15 smithi114 ceph-mon[29235]: from='mgr.? 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T22:22:15.895 INFO:tasks.workunit.client.0.smithi114.stdout:4/919: chown dd/d1b/d24/d57/d12b/d61/df4/c126 242380 1 2024-02-16T22:22:15.899 INFO:tasks.workunit.client.0.smithi114.stdout:2/842: dwrite dd/d3b/d8a/fe8 [4194304,4194304] 0 2024-02-16T22:22:15.899 INFO:tasks.workunit.client.0.smithi114.stdout:8/757: read d4/d16/d25/d46/fbd [2601503,58728] 0 2024-02-16T22:22:15.900 INFO:tasks.workunit.client.0.smithi114.stdout:3/754: rename d4/d46/d48 to d4/d26/dd7/d104 0 2024-02-16T22:22:15.907 INFO:tasks.workunit.client.0.smithi114.stdout:4/920: dwrite dd/d1b/d41/dc3/ff3 [0,4194304] 0 2024-02-16T22:22:15.913 INFO:tasks.workunit.client.0.smithi114.stdout:5/748: dwrite d0/d28/d98/d25/d89/de2/d9f/fae [0,4194304] 0 2024-02-16T22:22:15.916 INFO:tasks.workunit.client.1.smithi123.stdout:0/981: rename d1/d8/d13/d1d/d64/dbc/ddb/l127 to d1/d4/d17/d54/d8e/da5/d10d/l147 0 2024-02-16T22:22:15.920 INFO:tasks.workunit.client.1.smithi123.stdout:2/990: creat d4/f144 x:0 0 0 2024-02-16T22:22:15.922 INFO:tasks.workunit.client.1.smithi123.stdout:2/991: chown d4/d1a/dd5/d10b/df1/c2a 1766619 1 2024-02-16T22:22:15.922 INFO:tasks.workunit.client.0.smithi114.stdout:7/850: write d3/d22/d73/da7/d6f/d89/f96 [69380,113874] 0 2024-02-16T22:22:15.924 INFO:tasks.workunit.client.1.smithi123.stdout:0/982: creat d1/d8/d13/d1d/d64/dbc/d114/f148 x:0 0 0 2024-02-16T22:22:15.928 INFO:tasks.workunit.client.1.smithi123.stdout:2/992: creat d4/d1a/dd5/d11c/f145 x:0 0 0 2024-02-16T22:22:15.932 INFO:tasks.workunit.client.0.smithi114.stdout:8/758: rename d4/d16/d25/d46/d10b/dc8 to d4/d24/d6d/dc7/dfe/d10e 0 2024-02-16T22:22:15.935 INFO:tasks.workunit.client.1.smithi123.stdout:2/993: fsync d4/d1a/dd5/d10b/df1/f6c 0 2024-02-16T22:22:15.937 INFO:tasks.workunit.client.0.smithi114.stdout:4/921: mknod dd/d1b/d24/d57/d12b/d61/d13c/c14a 0 2024-02-16T22:22:15.941 INFO:tasks.workunit.client.0.smithi114.stdout:1/750: dwrite d4/f21 [0,4194304] 0 2024-02-16T22:22:15.941 INFO:tasks.workunit.client.1.smithi123.stdout:2/994: mkdir d4/d1a/dd5/d120/d146 0 2024-02-16T22:22:15.941 INFO:tasks.workunit.client.1.smithi123.stdout:2/995: read - d4/d1a/d1d/d59/f12c zero size 2024-02-16T22:22:15.944 INFO:tasks.workunit.client.0.smithi114.stdout:7/851: symlink d3/d92/l12b 0 2024-02-16T22:22:15.946 INFO:tasks.workunit.client.0.smithi114.stdout:4/922: dwrite dd/d59/dfa/f140 [0,4194304] 0 2024-02-16T22:22:15.946 INFO:tasks.workunit.client.1.smithi123.stdout:0/983: dwrite d1/d8/d13/f5a [0,4194304] 0 2024-02-16T22:22:15.951 INFO:tasks.workunit.client.0.smithi114.stdout:5/749: rename d0/d28/d98/d25/d3e/f82 to d0/df/d7a/dc4/ff2 0 2024-02-16T22:22:15.953 INFO:tasks.workunit.client.0.smithi114.stdout:0/831: link d1/d16/d1f/d28/d51/d60/dd1/ff7 d1/f108 0 2024-02-16T22:22:15.954 INFO:tasks.workunit.client.0.smithi114.stdout:5/750: write d0/d28/d98/d25/d2e/d3b/f85 [8475143,127690] 0 2024-02-16T22:22:15.954 INFO:tasks.workunit.client.1.smithi123.stdout:0/984: dwrite d1/d4/d17/f112 [0,4194304] 0 2024-02-16T22:22:15.956 INFO:tasks.workunit.client.0.smithi114.stdout:0/832: truncate d1/d1a/f38 1855292 0 2024-02-16T22:22:15.957 INFO:tasks.workunit.client.0.smithi114.stdout:0/833: chown d1/d16/d1f/d28/d51 324851208 1 2024-02-16T22:22:15.960 INFO:tasks.workunit.client.0.smithi114.stdout:4/923: fsync dd/d1b/d24/f36 0 2024-02-16T22:22:15.961 INFO:tasks.workunit.client.0.smithi114.stdout:3/755: rename d4/d43/d53 to d4/d43/d105 0 2024-02-16T22:22:15.964 INFO:tasks.workunit.client.0.smithi114.stdout:5/751: rmdir d0/d28/d98/d1e/d63/d96 39 2024-02-16T22:22:15.965 INFO:tasks.workunit.client.0.smithi114.stdout:6/776: link d6/d2c/d66/fa5 d6/dd/d25/d34/d58/d9f/fe4 0 2024-02-16T22:22:15.966 INFO:tasks.workunit.client.0.smithi114.stdout:1/751: symlink d4/d9/d46/d47/l111 0 2024-02-16T22:22:15.970 INFO:tasks.workunit.client.0.smithi114.stdout:4/924: rename dd/d59/f80 to dd/d59/d8a/f14b 0 2024-02-16T22:22:15.971 INFO:tasks.workunit.client.0.smithi114.stdout:4/925: chown dd/d1b/db0/f109 1 1 2024-02-16T22:22:15.972 INFO:tasks.workunit.client.0.smithi114.stdout:5/752: mkdir d0/d28/d98/d25/d89/de2/df3 0 2024-02-16T22:22:15.973 INFO:tasks.workunit.client.0.smithi114.stdout:4/926: stat dd/d1b/d24/d57/d12b/d61/d13c/c14a 0 2024-02-16T22:22:15.974 INFO:tasks.workunit.client.0.smithi114.stdout:6/777: read d6/d83/f8f [1614874,7978] 0 2024-02-16T22:22:15.978 INFO:tasks.workunit.client.0.smithi114.stdout:5/753: chown d0/d28/d98/d1e/d63/d96/d9a/db8 2480305 1 2024-02-16T22:22:15.981 INFO:tasks.workunit.client.0.smithi114.stdout:0/834: rmdir d1/d16/d1f/d28/d51/d61/d7e/d102 0 2024-02-16T22:22:15.984 INFO:tasks.workunit.client.0.smithi114.stdout:1/752: read d4/d9/d46/d67/f68 [4691370,94070] 0 2024-02-16T22:22:15.986 INFO:tasks.workunit.client.0.smithi114.stdout:5/754: rename d0/d28/d98/d25/d2e/l94 to d0/d2d/lf4 0 2024-02-16T22:22:15.988 INFO:tasks.workunit.client.0.smithi114.stdout:8/759: dread d4/d11/f20 [0,4194304] 0 2024-02-16T22:22:15.991 INFO:tasks.workunit.client.0.smithi114.stdout:2/843: dwrite dd/df/d1c/f67 [0,4194304] 0 2024-02-16T22:22:15.991 INFO:tasks.workunit.client.0.smithi114.stdout:1/753: write d4/d9/d46/d67/d92/dca/dcc/fd5 [1487715,125905] 0 2024-02-16T22:22:15.993 INFO:tasks.workunit.client.0.smithi114.stdout:5/755: dread - d0/df/d53/fd2 zero size 2024-02-16T22:22:15.995 INFO:tasks.workunit.client.0.smithi114.stdout:7/852: write d3/fc [2870247,108400] 0 2024-02-16T22:22:15.998 INFO:tasks.workunit.client.0.smithi114.stdout:8/760: read d4/d11/d52/fa9 [4190179,105373] 0 2024-02-16T22:22:16.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.012+0000 7f127cc83700 1 -- 172.21.15.114:0/2042010792 --> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f127819b1b0 con 0x7f126006d6a0 2024-02-16T22:22:16.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.014+0000 7f12677fe700 1 -- 172.21.15.114:0/2042010792 <== mgr.24441 v2:172.21.15.123:6828/1539029039 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7f127819b1b0 con 0x7f126006d6a0 2024-02-16T22:22:16.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.017+0000 7f127cc83700 1 -- 172.21.15.114:0/2042010792 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f126006d6a0 msgr2=0x7f126006fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:16.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.017+0000 7f127cc83700 1 --2- 172.21.15.114:0/2042010792 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f126006d6a0 0x7f126006fb60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f126c006010 tx=0x7f126c009f70 comp rx=0 tx=0).stop 2024-02-16T22:22:16.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.017+0000 7f127cc83700 1 -- 172.21.15.114:0/2042010792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12780ffab0 msgr2=0x7f1278193f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:16.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.017+0000 7f127cc83700 1 --2- 172.21.15.114:0/2042010792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12780ffab0 0x7f1278193f40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f126800e8f0 tx=0x7f126800ec00 comp rx=0 tx=0).stop 2024-02-16T22:22:16.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.017+0000 7f127cc83700 1 -- 172.21.15.114:0/2042010792 shutdown_connections 2024-02-16T22:22:16.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.017+0000 7f127cc83700 1 --2- 172.21.15.114:0/2042010792 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f126006d6a0 0x7f126006fb60 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:16.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.017+0000 7f127cc83700 1 --2- 172.21.15.114:0/2042010792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12780ffab0 0x7f1278193f40 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:16.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.017+0000 7f127cc83700 1 --2- 172.21.15.114:0/2042010792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1278100cb0 0x7f1278194480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:16.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.017+0000 7f127cc83700 1 -- 172.21.15.114:0/2042010792 >> 172.21.15.114:0/2042010792 conn(0x7f12780fb030 msgr2=0x7f12780fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:22:16.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.017+0000 7f127cc83700 1 -- 172.21.15.114:0/2042010792 shutdown_connections 2024-02-16T22:22:16.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.017+0000 7f127cc83700 1 -- 172.21.15.114:0/2042010792 wait complete. 2024-02-16T22:22:16.029 INFO:tasks.workunit.client.0.smithi114.stdout:1/754: dread d4/d63/d69/d80/fb1 [0,4194304] 0 2024-02-16T22:22:16.029 INFO:tasks.workunit.client.0.smithi114.stdout:1/755: chown d4/f16 93262418 1 2024-02-16T22:22:16.033 INFO:tasks.workunit.client.0.smithi114.stdout:9/785: sync 2024-02-16T22:22:16.035 INFO:teuthology.orchestra.run.smithi114.stdout:true 2024-02-16T22:22:16.099 INFO:tasks.workunit.client.0.smithi114.stdout:7/853: creat d3/d22/d73/da7/db4/f12c x:0 0 0 2024-02-16T22:22:16.100 INFO:tasks.workunit.client.0.smithi114.stdout:2/844: truncate dd/d6b/ff5 2565092 0 2024-02-16T22:22:16.103 INFO:tasks.workunit.client.0.smithi114.stdout:4/927: write dd/d2e/d3f/fa6 [464834,93649] 0 2024-02-16T22:22:16.108 INFO:tasks.workunit.client.0.smithi114.stdout:1/756: write d4/d9/d46/d47/db3/ff6 [150050,90970] 0 2024-02-16T22:22:16.109 INFO:tasks.workunit.client.0.smithi114.stdout:3/756: dwrite d4/d24/f62 [4194304,4194304] 0 2024-02-16T22:22:16.112 INFO:tasks.workunit.client.1.smithi123.stdout:2/996: sync 2024-02-16T22:22:16.113 INFO:tasks.workunit.client.1.smithi123.stdout:0/985: sync 2024-02-16T22:22:16.117 INFO:tasks.workunit.client.0.smithi114.stdout:6/778: dwrite d6/d2c/f95 [0,4194304] 0 2024-02-16T22:22:16.117 INFO:tasks.workunit.client.0.smithi114.stdout:2/845: creat dd/d3b/d8a/dcb/f112 x:0 0 0 2024-02-16T22:22:16.119 INFO:tasks.workunit.client.0.smithi114.stdout:5/756: write d0/d28/d98/d25/d89/de2/f31 [3991330,23661] 0 2024-02-16T22:22:16.121 INFO:tasks.workunit.client.0.smithi114.stdout:2/846: write dd/d3b/d8a/f10b [456814,59972] 0 2024-02-16T22:22:16.121 INFO:tasks.workunit.client.1.smithi123.stdout:0/986: dwrite d1/d4/d42/f105 [0,4194304] 0 2024-02-16T22:22:16.136 INFO:tasks.workunit.client.1.smithi123.stdout:0/987: rename d1/d4/d17/d74/de7/l110 to d1/d8/d3d/l149 0 2024-02-16T22:22:16.143 INFO:tasks.workunit.client.0.smithi114.stdout:3/757: dread d4/dd/f1e [0,4194304] 0 2024-02-16T22:22:16.143 INFO:tasks.workunit.client.0.smithi114.stdout:5/757: creat d0/df/d5e/ff5 x:0 0 0 2024-02-16T22:22:16.146 INFO:tasks.workunit.client.0.smithi114.stdout:4/928: mknod dd/d2e/d56/ddc/dde/d138/c14c 0 2024-02-16T22:22:16.151 INFO:tasks.workunit.client.0.smithi114.stdout:2/847: unlink dd/d3b/da7/d4c/ff2 0 2024-02-16T22:22:16.154 INFO:tasks.workunit.client.0.smithi114.stdout:4/929: dwrite dd/d1b/d24/d57/d12b/fba [4194304,4194304] 0 2024-02-16T22:22:16.160 INFO:tasks.workunit.client.0.smithi114.stdout:3/758: rename d4/d24/d2d/d45/d74/dc7/fcd to d4/d24/d2d/d45/d74/dc7/f106 0 2024-02-16T22:22:16.165 INFO:tasks.workunit.client.0.smithi114.stdout:7/854: getdents d3/d22 0 2024-02-16T22:22:16.167 INFO:tasks.workunit.client.0.smithi114.stdout:4/930: mknod dd/d1b/d24/d57/d12b/d61/d13c/d144/c14d 0 2024-02-16T22:22:16.169 INFO:tasks.workunit.client.0.smithi114.stdout:7/855: read d3/d22/d61/d6a/f7a [2036764,8211] 0 2024-02-16T22:22:16.172 INFO:tasks.workunit.client.0.smithi114.stdout:4/931: mkdir dd/d1b/d24/d57/d12b/d38/d14e 0 2024-02-16T22:22:16.174 INFO:tasks.workunit.client.0.smithi114.stdout:7/856: unlink d3/d22/d61/f10f 0 2024-02-16T22:22:16.177 INFO:tasks.workunit.client.0.smithi114.stdout:7/857: unlink d3/d22/c36 0 2024-02-16T22:22:16.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.188+0000 7f3d3a223700 1 -- 172.21.15.114:0/1938313672 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d340ffad0 msgr2=0x7f3d340ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:16.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.188+0000 7f3d3a223700 1 --2- 172.21.15.114:0/1938313672 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d340ffad0 0x7f3d340ffef0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009910 tx=0x7f3d24009c20 comp rx=0 tx=0).stop 2024-02-16T22:22:16.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.188+0000 7f3d3a223700 1 -- 172.21.15.114:0/1938313672 shutdown_connections 2024-02-16T22:22:16.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.188+0000 7f3d3a223700 1 --2- 172.21.15.114:0/1938313672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d34100cd0 0x7f3d34101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:16.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.188+0000 7f3d3a223700 1 --2- 172.21.15.114:0/1938313672 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d340ffad0 0x7f3d340ffef0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:16.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.188+0000 7f3d3a223700 1 -- 172.21.15.114:0/1938313672 >> 172.21.15.114:0/1938313672 conn(0x7f3d340fb050 msgr2=0x7f3d340fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:22:16.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.188+0000 7f3d3a223700 1 -- 172.21.15.114:0/1938313672 shutdown_connections 2024-02-16T22:22:16.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.188+0000 7f3d3a223700 1 -- 172.21.15.114:0/1938313672 wait complete. 2024-02-16T22:22:16.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.189+0000 7f3d3a223700 1 Processor -- start 2024-02-16T22:22:16.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.189+0000 7f3d3a223700 1 -- start start 2024-02-16T22:22:16.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.189+0000 7f3d3a223700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d340ffad0 0x7f3d34195280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:16.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.189+0000 7f3d3a223700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d34100cd0 0x7f3d341957c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:16.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.189+0000 7f3d3a223700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d34195de0 con 0x7f3d34100cd0 2024-02-16T22:22:16.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.189+0000 7f3d3a223700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d34195f20 con 0x7f3d340ffad0 2024-02-16T22:22:16.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.189+0000 7f3d32ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d34100cd0 0x7f3d341957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:16.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.189+0000 7f3d32ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d34100cd0 0x7f3d341957c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50090/0 (socket says 172.21.15.114:50090) 2024-02-16T22:22:16.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.189+0000 7f3d32ffd700 1 -- 172.21.15.114:0/2818611218 learned_addr learned my addr 172.21.15.114:0/2818611218 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:22:16.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.189+0000 7f3d337fe700 1 --2- 172.21.15.114:0/2818611218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d340ffad0 0x7f3d34195280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:16.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.190+0000 7f3d32ffd700 1 -- 172.21.15.114:0/2818611218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d340ffad0 msgr2=0x7f3d34195280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:16.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.190+0000 7f3d32ffd700 1 --2- 172.21.15.114:0/2818611218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d340ffad0 0x7f3d34195280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:16.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.190+0000 7f3d32ffd700 1 -- 172.21.15.114:0/2818611218 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d240095f0 con 0x7f3d34100cd0 2024-02-16T22:22:16.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.190+0000 7f3d337fe700 1 --2- 172.21.15.114:0/2818611218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d340ffad0 0x7f3d34195280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:22:16.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.190+0000 7f3d32ffd700 1 --2- 172.21.15.114:0/2818611218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d34100cd0 0x7f3d341957c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800e920 tx=0x7f3d2800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:22:16.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.190+0000 7f3d30ff9700 1 -- 172.21.15.114:0/2818611218 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d2800c9d0 con 0x7f3d34100cd0 2024-02-16T22:22:16.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.190+0000 7f3d30ff9700 1 -- 172.21.15.114:0/2818611218 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3d28010430 con 0x7f3d34100cd0 2024-02-16T22:22:16.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.190+0000 7f3d3a223700 1 -- 172.21.15.114:0/2818611218 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d3419a9d0 con 0x7f3d34100cd0 2024-02-16T22:22:16.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.190+0000 7f3d30ff9700 1 -- 172.21.15.114:0/2818611218 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d28018650 con 0x7f3d34100cd0 2024-02-16T22:22:16.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.190+0000 7f3d3a223700 1 -- 172.21.15.114:0/2818611218 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d3419ae60 con 0x7f3d34100cd0 2024-02-16T22:22:16.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.191+0000 7f3d3a223700 1 -- 172.21.15.114:0/2818611218 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d34108630 con 0x7f3d34100cd0 2024-02-16T22:22:16.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.192+0000 7f3d30ff9700 1 -- 172.21.15.114:0/2818611218 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) v1 ==== 91450+0+0 (secure 0 0 0) 0x7f3d280105a0 con 0x7f3d34100cd0 2024-02-16T22:22:16.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.193+0000 7f3d30ff9700 1 --2- 172.21.15.114:0/2818611218 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f3d1c06d6b0 0x7f3d1c06fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:16.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.193+0000 7f3d30ff9700 1 -- 172.21.15.114:0/2818611218 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5726+0+0 (secure 0 0 0) 0x7f3d28014070 con 0x7f3d34100cd0 2024-02-16T22:22:16.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.193+0000 7f3d337fe700 1 --2- 172.21.15.114:0/2818611218 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f3d1c06d6b0 0x7f3d1c06fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:16.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.194+0000 7f3d337fe700 1 --2- 172.21.15.114:0/2818611218 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f3d1c06d6b0 0x7f3d1c06fb70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3d2400b6f0 tx=0x7f3d24005fb0 comp rx=0 tx=0).ready entity=mgr.24441 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:22:16.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.198+0000 7f3d30ff9700 1 -- 172.21.15.114:0/2818611218 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182869 (secure 0 0 0) 0x7f3d2805afa0 con 0x7f3d34100cd0 2024-02-16T22:22:16.265 INFO:tasks.workunit.client.0.smithi114.stdout:0/835: rename d1/d16/d1f/d28/d51/d60/f5e to d1/d1a/f109 0 2024-02-16T22:22:16.269 INFO:tasks.workunit.client.0.smithi114.stdout:0/836: mknod d1/d1a/d47/d62/d89/d104/c10a 0 2024-02-16T22:22:16.272 INFO:tasks.workunit.client.0.smithi114.stdout:0/837: mkdir d1/d16/d1f/d28/d51/d61/d6d/d6e/d10b 0 2024-02-16T22:22:16.273 INFO:tasks.workunit.client.0.smithi114.stdout:0/838: chown d1/d16/d1f/d28/d67/cce 1400 1 2024-02-16T22:22:16.277 INFO:tasks.workunit.client.0.smithi114.stdout:0/839: mkdir d1/d1a/d47/d62/d7f/d8c/d10c 0 2024-02-16T22:22:16.279 INFO:tasks.workunit.client.0.smithi114.stdout:0/840: creat d1/d16/d1f/d28/d51/d60/dd1/f10d x:0 0 0 2024-02-16T22:22:16.280 INFO:tasks.workunit.client.0.smithi114.stdout:0/841: readlink d1/d1a/d47/d62/d89/ld2 0 2024-02-16T22:22:16.289 INFO:tasks.workunit.client.0.smithi114.stdout:0/842: dread d1/d16/d1f/d28/d51/d61/d7e/d90/d95/d80/f8f [0,4194304] 0 2024-02-16T22:22:16.291 INFO:tasks.workunit.client.0.smithi114.stdout:0/843: fdatasync d1/d16/d18/f22 0 2024-02-16T22:22:16.301 INFO:tasks.workunit.client.0.smithi114.stdout:8/761: dwrite d4/d24/d6d/f108 [0,4194304] 0 2024-02-16T22:22:16.302 INFO:tasks.workunit.client.0.smithi114.stdout:9/786: dwrite d2/f44 [0,4194304] 0 2024-02-16T22:22:16.308 INFO:tasks.workunit.client.0.smithi114.stdout:8/762: dread d4/d16/d25/d7c/f8d [4194304,4194304] 0 2024-02-16T22:22:16.310 INFO:tasks.workunit.client.1.smithi123.stdout:2/997: truncate d4/fc 58215 0 2024-02-16T22:22:16.313 INFO:tasks.workunit.client.1.smithi123.stdout:2/998: creat d4/d1a/d1d/d38/d5e/de8/f147 x:0 0 0 2024-02-16T22:22:16.314 INFO:tasks.workunit.client.0.smithi114.stdout:1/757: write d4/d9/d46/d67/f68 [4967303,85694] 0 2024-02-16T22:22:16.317 INFO:tasks.workunit.client.1.smithi123.stdout:2/999: mkdir d4/d1a/dd5/d10b/df1/d4a/dce/d115/df6/d148 0 2024-02-16T22:22:16.319 INFO:tasks.workunit.client.0.smithi114.stdout:1/758: mkdir d4/d9/d46/d67/d92/dca/dce/d112 0 2024-02-16T22:22:16.325 INFO:tasks.workunit.client.1.smithi123.stdout:0/988: dwrite d1/d4/d17/d74/f52 [0,4194304] 0 2024-02-16T22:22:16.327 INFO:tasks.workunit.client.0.smithi114.stdout:6/779: write d6/d2c/d38/d41/d60/fbc [536648,68498] 0 2024-02-16T22:22:16.327 INFO:tasks.workunit.client.1.smithi123.stdout:0/989: chown d1/d4/d42/f4a 186025072 1 2024-02-16T22:22:16.331 INFO:tasks.workunit.client.0.smithi114.stdout:9/787: dread d2/d5c/dc9/d47/f92 [0,4194304] 0 2024-02-16T22:22:16.332 INFO:tasks.workunit.client.0.smithi114.stdout:6/780: creat d6/dd/d25/d34/d58/d9f/da3/fe5 x:0 0 0 2024-02-16T22:22:16.334 INFO:tasks.workunit.client.1.smithi123.stdout:0/990: dwrite d1/d4/d42/f105 [0,4194304] 0 2024-02-16T22:22:16.337 INFO:tasks.workunit.client.0.smithi114.stdout:6/781: mkdir d6/d2c/de6 0 2024-02-16T22:22:16.338 INFO:tasks.workunit.client.1.smithi123.stdout:0/991: mknod d1/d4/d17/d74/d57/c14a 0 2024-02-16T22:22:16.339 INFO:tasks.workunit.client.1.smithi123.stdout:0/992: readlink d1/d4/l18 0 2024-02-16T22:22:16.340 INFO:tasks.workunit.client.1.smithi123.stdout:0/993: dread - d1/d8/da7/f10b zero size 2024-02-16T22:22:16.341 INFO:tasks.workunit.client.0.smithi114.stdout:2/848: symlink dd/d3b/da7/l113 0 2024-02-16T22:22:16.341 INFO:tasks.workunit.client.1.smithi123.stdout:0/994: chown d1/d4/d17/dad/db2/l12c 144446808 1 2024-02-16T22:22:16.342 INFO:tasks.workunit.client.0.smithi114.stdout:6/782: mkdir d6/d2c/d5e/dcc/de7 0 2024-02-16T22:22:16.343 INFO:tasks.workunit.client.0.smithi114.stdout:2/849: rmdir dd/d3b 39 2024-02-16T22:22:16.347 INFO:tasks.workunit.client.1.smithi123.stdout:0/995: creat d1/d8/d13/d1d/d64/d129/f14b x:0 0 0 2024-02-16T22:22:16.347 INFO:tasks.workunit.client.0.smithi114.stdout:3/759: write d4/d26/dd7/d104/dcc/fc6 [1766026,23346] 0 2024-02-16T22:22:16.348 INFO:tasks.workunit.client.0.smithi114.stdout:2/850: mkdir dd/df/d1c/d21/d61/d69/de9/d114 0 2024-02-16T22:22:16.350 INFO:tasks.workunit.client.0.smithi114.stdout:3/760: truncate d4/d26/dd7/d104/d75/daa/ffc 334944 0 2024-02-16T22:22:16.351 INFO:tasks.workunit.client.0.smithi114.stdout:2/851: chown dd/cad 237461 1 2024-02-16T22:22:16.354 INFO:tasks.workunit.client.1.smithi123.stdout:0/996: creat d1/d8/f14c x:0 0 0 2024-02-16T22:22:16.356 INFO:tasks.workunit.client.1.smithi123.stdout:0/997: mkdir d1/d8/d13/d1d/d64/dbc/ddb/d14d 0 2024-02-16T22:22:16.358 INFO:tasks.workunit.client.0.smithi114.stdout:4/932: truncate dd/d59/d8f/f88 1741592 0 2024-02-16T22:22:16.358 INFO:tasks.workunit.client.0.smithi114.stdout:2/852: getdents dd/df/d1c/d21/d61/d63/dc2 0 2024-02-16T22:22:16.359 INFO:tasks.workunit.client.0.smithi114.stdout:7/858: write d3/d92/dbe/de6/f11c [315464,56614] 0 2024-02-16T22:22:16.360 INFO:tasks.workunit.client.0.smithi114.stdout:4/933: chown dd/d1b/d24/d57/d12b/d61/d13c/ce0 0 1 2024-02-16T22:22:16.362 INFO:tasks.workunit.client.1.smithi123.stdout:0/998: fsync d1/d4/f11 0 2024-02-16T22:22:16.362 INFO:tasks.workunit.client.0.smithi114.stdout:4/934: chown dd/d1b/d24/d57/d12b/d38 39 1 2024-02-16T22:22:16.363 INFO:tasks.workunit.client.1.smithi123.stdout:0/999: stat d1/d8/d13/d1d/d64/c12a 0 2024-02-16T22:22:16.366 INFO:tasks.workunit.client.1.smithi123.stderr:+ rm -rf -- ./tmp.2GodLqhxmo 2024-02-16T22:22:16.368 INFO:tasks.workunit.client.0.smithi114.stdout:6/783: dread d6/dd/d25/d34/d58/d9f/fe4 [0,4194304] 0 2024-02-16T22:22:16.370 INFO:tasks.workunit.client.0.smithi114.stdout:7/859: dwrite d3/d22/d73/da7/db4/f12c [0,4194304] 0 2024-02-16T22:22:16.371 INFO:tasks.workunit.client.0.smithi114.stdout:6/784: dread - d6/dd/d25/d34/d61/d62/fe3 zero size 2024-02-16T22:22:16.374 INFO:tasks.workunit.client.0.smithi114.stdout:3/761: dread d4/d26/d4a/f90 [0,4194304] 0 2024-02-16T22:22:16.381 INFO:tasks.workunit.client.0.smithi114.stdout:3/762: mknod d4/d24/d93/c107 0 2024-02-16T22:22:16.381 INFO:tasks.workunit.client.0.smithi114.stdout:2/853: dread dd/da6/fae [0,4194304] 0 2024-02-16T22:22:16.382 INFO:tasks.workunit.client.0.smithi114.stdout:7/860: getdents d3/d92/dbe/de6/d11e 0 2024-02-16T22:22:16.384 INFO:tasks.workunit.client.0.smithi114.stdout:3/763: rmdir d4/d26/d4a 39 2024-02-16T22:22:16.384 INFO:tasks.workunit.client.0.smithi114.stdout:3/764: readlink d4/dd/d33/lf2 0 2024-02-16T22:22:16.385 INFO:tasks.workunit.client.0.smithi114.stdout:2/854: rmdir dd/d4e/d97 39 2024-02-16T22:22:16.386 INFO:tasks.workunit.client.0.smithi114.stdout:3/765: write d4/dd/d33/d72/fbf [1792268,30281] 0 2024-02-16T22:22:16.388 INFO:tasks.workunit.client.0.smithi114.stdout:7/861: creat d3/d22/d73/f12d x:0 0 0 2024-02-16T22:22:16.393 INFO:tasks.workunit.client.0.smithi114.stdout:2/855: mkdir dd/d115 0 2024-02-16T22:22:16.400 INFO:tasks.workunit.client.0.smithi114.stdout:2/856: dwrite dd/df/ddb/ff7 [0,4194304] 0 2024-02-16T22:22:16.407 INFO:tasks.workunit.client.0.smithi114.stdout:3/766: dread d4/dd/fe9 [0,4194304] 0 2024-02-16T22:22:16.408 INFO:tasks.workunit.client.0.smithi114.stdout:2/857: rename dd/d4e/fa1 to dd/d3b/da7/f116 0 2024-02-16T22:22:16.412 INFO:tasks.workunit.client.0.smithi114.stdout:3/767: mkdir d4/dd/d33/d72/d108 0 2024-02-16T22:22:16.413 INFO:tasks.workunit.client.0.smithi114.stdout:2/858: mkdir dd/d3b/da7/d8b/dc0/d117 0 2024-02-16T22:22:16.413 INFO:tasks.workunit.client.0.smithi114.stdout:3/768: chown d4/dd/d33/d72 33097376 1 2024-02-16T22:22:16.414 INFO:tasks.workunit.client.0.smithi114.stdout:3/769: chown d4/d26/lb7 447072919 1 2024-02-16T22:22:16.417 INFO:tasks.workunit.client.0.smithi114.stdout:0/844: dwrite d1/d16/d48/d98/fc4 [0,4194304] 0 2024-02-16T22:22:16.419 INFO:tasks.workunit.client.0.smithi114.stdout:3/770: unlink d4/d46/f102 0 2024-02-16T22:22:16.422 INFO:tasks.workunit.client.0.smithi114.stdout:3/771: write d4/d26/dd7/d104/dcc/fc6 [1149751,42053] 0 2024-02-16T22:22:16.423 INFO:tasks.workunit.client.0.smithi114.stdout:2/859: rename dd/df/ddb/f100 to dd/df/d1c/d21/d61/dec/f118 0 2024-02-16T22:22:16.427 INFO:tasks.workunit.client.0.smithi114.stdout:5/758: sync 2024-02-16T22:22:16.430 INFO:tasks.workunit.client.0.smithi114.stdout:3/772: mkdir d4/d26/dd7/d109 0 2024-02-16T22:22:16.430 INFO:tasks.workunit.client.0.smithi114.stdout:5/759: read d0/d28/d98/d1e/d63/fc8 [265378,122463] 0 2024-02-16T22:22:16.432 INFO:tasks.workunit.client.0.smithi114.stdout:8/763: dwrite d4/d11/d78/d7a/d31/fa5 [0,4194304] 0 2024-02-16T22:22:16.433 INFO:tasks.workunit.client.0.smithi114.stdout:0/845: creat d1/d1a/d47/d62/dff/f10e x:0 0 0 2024-02-16T22:22:16.437 INFO:tasks.workunit.client.0.smithi114.stdout:5/760: stat d0/d28/d98/d1e/lcb 0 2024-02-16T22:22:16.455 INFO:tasks.workunit.client.0.smithi114.stdout:5/761: write d0/d28/d98/d1e/d86/f7e [4682592,32537] 0 2024-02-16T22:22:16.457 INFO:tasks.workunit.client.0.smithi114.stdout:1/759: dwrite d4/d9/d46/d4d/f5c [0,4194304] 0 2024-02-16T22:22:16.459 INFO:tasks.workunit.client.0.smithi114.stdout:0/846: symlink d1/d16/d1f/d28/d51/d61/d6d/l10f 0 2024-02-16T22:22:16.460 INFO:tasks.workunit.client.0.smithi114.stdout:9/788: write d2/d1a/d23/d32/d46/fb6 [4605399,91188] 0 2024-02-16T22:22:16.467 INFO:tasks.workunit.client.0.smithi114.stdout:8/764: dread d4/f3c [0,4194304] 0 2024-02-16T22:22:16.469 INFO:tasks.workunit.client.0.smithi114.stdout:0/847: dwrite d1/d16/d1f/d28/d51/d61/d6d/f100 [0,4194304] 0 2024-02-16T22:22:16.470 INFO:tasks.workunit.client.0.smithi114.stdout:8/765: chown d4/d11/d78/d7a/d36/d3b/f44 3 1 2024-02-16T22:22:16.472 INFO:tasks.workunit.client.0.smithi114.stdout:5/762: fsync d0/d28/d98/d25/d2e/f7b 0 2024-02-16T22:22:16.472 INFO:tasks.workunit.client.0.smithi114.stdout:3/773: dread d4/d24/f25 [0,4194304] 0 2024-02-16T22:22:16.474 INFO:tasks.workunit.client.0.smithi114.stdout:1/760: mkdir d4/d9/d46/d67/d92/dca/dcc/d98/ddd/d113 0 2024-02-16T22:22:16.476 INFO:tasks.workunit.client.0.smithi114.stdout:1/761: write d4/d63/d69/d7c/dc0/ff4 [696349,115034] 0 2024-02-16T22:22:16.480 INFO:tasks.workunit.client.0.smithi114.stdout:0/848: symlink d1/d16/d1f/d28/d51/d61/d6d/df1/l110 0 2024-02-16T22:22:16.482 INFO:tasks.workunit.client.0.smithi114.stdout:8/766: mkdir d4/d16/d25/d7c/d10f 0 2024-02-16T22:22:16.483 INFO:tasks.workunit.client.0.smithi114.stdout:8/767: dread - d4/d24/d6d/ff2 zero size 2024-02-16T22:22:16.486 INFO:tasks.workunit.client.0.smithi114.stdout:3/774: mknod d4/dd/d33/d8f/c10a 0 2024-02-16T22:22:16.487 INFO:tasks.workunit.client.0.smithi114.stdout:4/935: truncate dd/d1b/f3e 3751012 0 2024-02-16T22:22:16.488 INFO:tasks.workunit.client.0.smithi114.stdout:5/763: rename d0/d28/d98/d25/d3e/l55 to d0/d28/d98/d25/d2e/d3b/lf6 0 2024-02-16T22:22:16.493 INFO:tasks.workunit.client.0.smithi114.stdout:6/785: dwrite d6/f35 [0,4194304] 0 2024-02-16T22:22:16.498 INFO:tasks.workunit.client.0.smithi114.stdout:3/775: rmdir d4/d26/dd7/d104/d75 39 2024-02-16T22:22:16.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.510+0000 7f3d3a223700 1 -- 172.21.15.114:0/2818611218 --> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d34105620 con 0x7f3d1c06d6b0 2024-02-16T22:22:16.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.511+0000 7f3d30ff9700 1 -- 172.21.15.114:0/2818611218 <== mgr.24441 v2:172.21.15.123:6828/1539029039 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7f3d34105620 con 0x7f3d1c06d6b0 2024-02-16T22:22:16.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.514+0000 7f3d3a223700 1 -- 172.21.15.114:0/2818611218 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f3d1c06d6b0 msgr2=0x7f3d1c06fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:16.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.514+0000 7f3d3a223700 1 --2- 172.21.15.114:0/2818611218 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f3d1c06d6b0 0x7f3d1c06fb70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3d2400b6f0 tx=0x7f3d24005fb0 comp rx=0 tx=0).stop 2024-02-16T22:22:16.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.514+0000 7f3d3a223700 1 -- 172.21.15.114:0/2818611218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d34100cd0 msgr2=0x7f3d341957c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:16.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.514+0000 7f3d3a223700 1 --2- 172.21.15.114:0/2818611218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d34100cd0 0x7f3d341957c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800e920 tx=0x7f3d2800ec30 comp rx=0 tx=0).stop 2024-02-16T22:22:16.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.515+0000 7f3d3a223700 1 -- 172.21.15.114:0/2818611218 shutdown_connections 2024-02-16T22:22:16.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.515+0000 7f3d3a223700 1 --2- 172.21.15.114:0/2818611218 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f3d1c06d6b0 0x7f3d1c06fb70 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:16.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.515+0000 7f3d3a223700 1 --2- 172.21.15.114:0/2818611218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d340ffad0 0x7f3d34195280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:16.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.515+0000 7f3d3a223700 1 --2- 172.21.15.114:0/2818611218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d34100cd0 0x7f3d341957c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:16.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.515+0000 7f3d3a223700 1 -- 172.21.15.114:0/2818611218 >> 172.21.15.114:0/2818611218 conn(0x7f3d340fb050 msgr2=0x7f3d34103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:22:16.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.515+0000 7f3d3a223700 1 -- 172.21.15.114:0/2818611218 shutdown_connections 2024-02-16T22:22:16.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.515+0000 7f3d3a223700 1 -- 172.21.15.114:0/2818611218 wait complete. 2024-02-16T22:22:16.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.630+0000 7f25d9830700 1 -- 172.21.15.114:0/3470840175 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25d40fee80 msgr2=0x7f25d40ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:16.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.630+0000 7f25d9830700 1 --2- 172.21.15.114:0/3470840175 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25d40fee80 0x7f25d40ff2a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f25c4009910 tx=0x7f25c4009c20 comp rx=0 tx=0).stop 2024-02-16T22:22:16.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.630+0000 7f25d9830700 1 -- 172.21.15.114:0/3470840175 shutdown_connections 2024-02-16T22:22:16.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.630+0000 7f25d9830700 1 --2- 172.21.15.114:0/3470840175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25d4101e60 0x7f25d41022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:16.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.630+0000 7f25d9830700 1 --2- 172.21.15.114:0/3470840175 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25d40fee80 0x7f25d40ff2a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:16.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.630+0000 7f25d9830700 1 -- 172.21.15.114:0/3470840175 >> 172.21.15.114:0/3470840175 conn(0x7f25d40fa7c0 msgr2=0x7f25d40fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:22:16.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.630+0000 7f25d9830700 1 -- 172.21.15.114:0/3470840175 shutdown_connections 2024-02-16T22:22:16.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.630+0000 7f25d9830700 1 -- 172.21.15.114:0/3470840175 wait complete. 2024-02-16T22:22:16.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.630+0000 7f25d9830700 1 Processor -- start 2024-02-16T22:22:16.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.631+0000 7f25d9830700 1 -- start start 2024-02-16T22:22:16.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.631+0000 7f25d9830700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25d40fee80 0x7f25d41963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:16.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.631+0000 7f25d9830700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25d4101e60 0x7f25d41968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:16.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.631+0000 7f25d9830700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d4196f00 con 0x7f25d4101e60 2024-02-16T22:22:16.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.631+0000 7f25d9830700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d4197040 con 0x7f25d40fee80 2024-02-16T22:22:16.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.631+0000 7f25d27fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25d4101e60 0x7f25d41968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:16.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.631+0000 7f25d2ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25d40fee80 0x7f25d41963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:16.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.631+0000 7f25d27fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25d4101e60 0x7f25d41968e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50118/0 (socket says 172.21.15.114:50118) 2024-02-16T22:22:16.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.631+0000 7f25d2ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25d40fee80 0x7f25d41963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.114:42266/0 (socket says 172.21.15.114:42266) 2024-02-16T22:22:16.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.631+0000 7f25d27fc700 1 -- 172.21.15.114:0/2495992948 learned_addr learned my addr 172.21.15.114:0/2495992948 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:22:16.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.631+0000 7f25d2ffd700 1 -- 172.21.15.114:0/2495992948 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25d4101e60 msgr2=0x7f25d41968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:16.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.631+0000 7f25d2ffd700 1 --2- 172.21.15.114:0/2495992948 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25d4101e60 0x7f25d41968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:16.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.631+0000 7f25d2ffd700 1 -- 172.21.15.114:0/2495992948 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c40095f0 con 0x7f25d40fee80 2024-02-16T22:22:16.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.632+0000 7f25d2ffd700 1 --2- 172.21.15.114:0/2495992948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25d40fee80 0x7f25d41963a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f25c4000c00 tx=0x7f25c400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:22:16.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.632+0000 7f25d882e700 1 -- 172.21.15.114:0/2495992948 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25c401d070 con 0x7f25d40fee80 2024-02-16T22:22:16.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.632+0000 7f25d9830700 1 -- 172.21.15.114:0/2495992948 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d419ba90 con 0x7f25d40fee80 2024-02-16T22:22:16.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.632+0000 7f25d882e700 1 -- 172.21.15.114:0/2495992948 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f25c4005dc0 con 0x7f25d40fee80 2024-02-16T22:22:16.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.632+0000 7f25d882e700 1 -- 172.21.15.114:0/2495992948 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25c4022470 con 0x7f25d40fee80 2024-02-16T22:22:16.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.632+0000 7f25d9830700 1 -- 172.21.15.114:0/2495992948 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d419bf80 con 0x7f25d40fee80 2024-02-16T22:22:16.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.633+0000 7f25d9830700 1 -- 172.21.15.114:0/2495992948 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25d4109870 con 0x7f25d40fee80 2024-02-16T22:22:16.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.633+0000 7f25d882e700 1 -- 172.21.15.114:0/2495992948 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) v1 ==== 91450+0+0 (secure 0 0 0) 0x7f25c4003810 con 0x7f25d40fee80 2024-02-16T22:22:16.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.634+0000 7f25d882e700 1 --2- 172.21.15.114:0/2495992948 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f25bc06d660 0x7f25bc06fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:16.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.634+0000 7f25d882e700 1 -- 172.21.15.114:0/2495992948 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5726+0+0 (secure 0 0 0) 0x7f25c408e490 con 0x7f25d40fee80 2024-02-16T22:22:16.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.634+0000 7f25d27fc700 1 --2- 172.21.15.114:0/2495992948 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f25bc06d660 0x7f25bc06fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:16.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.634+0000 7f25d27fc700 1 --2- 172.21.15.114:0/2495992948 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f25bc06d660 0x7f25bc06fb20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009d20 tx=0x7f25c8009260 comp rx=0 tx=0).ready entity=mgr.24441 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:22:16.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.639+0000 7f25d882e700 1 -- 172.21.15.114:0/2495992948 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182869 (secure 0 0 0) 0x7f25c405bc10 con 0x7f25d40fee80 2024-02-16T22:22:16.652 INFO:tasks.workunit.client.0.smithi114.stdout:7/862: truncate d3/d22/d73/da7/f5e 696672 0 2024-02-16T22:22:16.659 INFO:tasks.workunit.client.0.smithi114.stdout:5/764: creat d0/df/d7a/dc4/ff7 x:0 0 0 2024-02-16T22:22:16.660 INFO:tasks.workunit.client.0.smithi114.stdout:9/789: link d2/dd/ff d2/d1a/ffa 0 2024-02-16T22:22:16.661 INFO:tasks.workunit.client.0.smithi114.stdout:7/863: dread - d3/d3d/ff3 zero size 2024-02-16T22:22:16.662 INFO:tasks.workunit.client.0.smithi114.stdout:8/768: mknod d4/d24/d6d/dc7/df0/c110 0 2024-02-16T22:22:16.662 INFO:tasks.workunit.client.0.smithi114.stdout:1/762: dread d4/d9/d3d/f88 [0,4194304] 0 2024-02-16T22:22:16.667 INFO:tasks.workunit.client.0.smithi114.stdout:3/776: dwrite d4/d26/dd7/df4/ff6 [0,4194304] 0 2024-02-16T22:22:16.670 INFO:tasks.workunit.client.0.smithi114.stdout:0/849: write d1/fe [4242708,34240] 0 2024-02-16T22:22:16.671 INFO:tasks.workunit.client.0.smithi114.stdout:6/786: dwrite d6/dd/d25/d34/f42 [0,4194304] 0 2024-02-16T22:22:16.683 INFO:tasks.workunit.client.0.smithi114.stdout:8/769: dread d4/d11/d78/d7a/f10 [0,4194304] 0 2024-02-16T22:22:16.723 INFO:tasks.workunit.client.0.smithi114.stdout:5/765: symlink d0/d28/d98/d1e/d86/lf8 0 2024-02-16T22:22:16.725 INFO:tasks.workunit.client.0.smithi114.stdout:9/790: symlink d2/dd/d3c/lfb 0 2024-02-16T22:22:16.726 INFO:tasks.workunit.client.0.smithi114.stdout:1/763: fsync d4/d9/f12 0 2024-02-16T22:22:16.737 INFO:tasks.workunit.client.0.smithi114.stdout:0/850: readlink d1/d1a/d47/d62/d7f/led 0 2024-02-16T22:22:16.747 INFO:tasks.workunit.client.0.smithi114.stdout:8/770: mkdir d4/d11/d78/d7a/d111 0 2024-02-16T22:22:16.748 INFO:tasks.workunit.client.0.smithi114.stdout:0/851: creat d1/d16/d1f/d28/d51/d60/d52/f111 x:0 0 0 2024-02-16T22:22:16.749 INFO:tasks.workunit.client.0.smithi114.stdout:5/766: mkdir d0/d28/d98/d1e/d63/df9 0 2024-02-16T22:22:16.751 INFO:tasks.workunit.client.0.smithi114.stdout:3/777: link d4/d43/l8d d4/d7e/l10b 0 2024-02-16T22:22:16.752 INFO:tasks.workunit.client.0.smithi114.stdout:3/778: fsync d4/dd/d33/d72/f9b 0 2024-02-16T22:22:16.754 INFO:tasks.workunit.client.0.smithi114.stdout:0/852: dread - d1/d16/d48/fe3 zero size 2024-02-16T22:22:16.755 INFO:tasks.workunit.client.0.smithi114.stdout:0/853: chown d1/d16/f24 116547 1 2024-02-16T22:22:16.756 INFO:tasks.workunit.client.0.smithi114.stdout:9/791: creat d2/d5c/dc9/ffc x:0 0 0 2024-02-16T22:22:16.756 INFO:tasks.workunit.client.0.smithi114.stdout:0/854: write d1/d78/f7d [4979446,111049] 0 2024-02-16T22:22:16.759 INFO:tasks.workunit.client.0.smithi114.stdout:5/767: rename d0/d28/d98/d1e/d63/d96/d9a/de9 to d0/d28/d98/d1e/dfa 0 2024-02-16T22:22:16.759 INFO:tasks.workunit.client.0.smithi114.stdout:5/768: stat d0/d28/l34 0 2024-02-16T22:22:16.768 INFO:tasks.workunit.client.0.smithi114.stdout:9/792: fsync d2/d5c/dc9/d1d/f3a 0 2024-02-16T22:22:16.768 INFO:tasks.workunit.client.0.smithi114.stdout:9/793: readlink d2/d5c/d5f/d72/dd6/lf6 0 2024-02-16T22:22:16.771 INFO:tasks.workunit.client.0.smithi114.stdout:5/769: creat d0/d28/d98/d25/d2e/d3b/ffb x:0 0 0 2024-02-16T22:22:16.772 INFO:tasks.workunit.client.0.smithi114.stdout:3/779: symlink d4/dd/d33/d72/d108/l10c 0 2024-02-16T22:22:16.776 INFO:tasks.workunit.client.0.smithi114.stdout:0/855: creat d1/d1a/d47/d62/d89/dba/f112 x:0 0 0 2024-02-16T22:22:16.777 INFO:tasks.workunit.client.0.smithi114.stdout:9/794: truncate d2/fb 1541348 0 2024-02-16T22:22:16.779 INFO:tasks.workunit.client.0.smithi114.stdout:5/770: creat d0/d28/d98/d1e/d63/d9e/ffc x:0 0 0 2024-02-16T22:22:16.781 INFO:tasks.workunit.client.0.smithi114.stdout:3/780: rename d4/c69 to d4/d26/dd7/d109/c10d 0 2024-02-16T22:22:16.781 INFO:tasks.workunit.client.0.smithi114.stdout:7/864: write d3/d22/d61/d6a/fcd [1038455,53209] 0 2024-02-16T22:22:16.785 INFO:tasks.workunit.client.0.smithi114.stdout:9/795: mknod d2/d1a/d65/cfd 0 2024-02-16T22:22:16.793 INFO:tasks.workunit.client.0.smithi114.stdout:0/856: link d1/d16/d1f/fa9 d1/d16/d1f/d28/d51/d61/d6d/d6e/f113 0 2024-02-16T22:22:16.795 INFO:tasks.workunit.client.0.smithi114.stdout:6/787: dwrite d6/f21 [0,4194304] 0 2024-02-16T22:22:16.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:16 smithi123 ceph-mon[37161]: mgrmap e25: smithi123.inzgjd(active, since 65s), standbys: smithi114.vcullz 2024-02-16T22:22:16.808 INFO:tasks.workunit.client.0.smithi114.stdout:8/771: dwrite d4/d16/d25/d46/db5/ff3 [0,4194304] 0 2024-02-16T22:22:16.811 INFO:tasks.workunit.client.0.smithi114.stdout:0/857: mknod d1/d16/d1f/d28/d51/d60/d52/c114 0 2024-02-16T22:22:16.817 INFO:tasks.workunit.client.0.smithi114.stdout:9/796: dread d2/f12 [0,4194304] 0 2024-02-16T22:22:16.817 INFO:tasks.workunit.client.0.smithi114.stdout:9/797: readlink d2/d5c/dc9/l43 0 2024-02-16T22:22:16.819 INFO:tasks.workunit.client.0.smithi114.stdout:3/781: creat d4/d43/f10e x:0 0 0 2024-02-16T22:22:16.822 INFO:tasks.workunit.client.0.smithi114.stdout:7/865: link d3/d22/d73/da7/d4e/f7f d3/d22/d73/da7/d6f/df8/f12e 0 2024-02-16T22:22:16.829 INFO:tasks.workunit.client.0.smithi114.stdout:6/788: rmdir d6/d2c/d38/d41/d60 39 2024-02-16T22:22:16.834 INFO:tasks.workunit.client.0.smithi114.stdout:5/771: write d0/d28/d98/d25/d3e/d7f/fa8 [2525323,43004] 0 2024-02-16T22:22:16.838 INFO:tasks.workunit.client.0.smithi114.stdout:0/858: mknod d1/d16/d1f/d28/d51/d61/d6d/d6e/de2/c115 0 2024-02-16T22:22:16.843 INFO:tasks.workunit.client.0.smithi114.stdout:9/798: rename d2/d5c/dc9/d1d/l5b to d2/dd/d3c/d89/d9e/lfe 0 2024-02-16T22:22:16.848 INFO:tasks.workunit.client.0.smithi114.stdout:0/859: mknod d1/d16/d1f/d28/d51/c116 0 2024-02-16T22:22:16.850 INFO:tasks.workunit.client.0.smithi114.stdout:0/860: chown d1/d16/d1f/d28/d51/d61/d7e/d90/d95/l9f 519 1 2024-02-16T22:22:16.851 INFO:tasks.workunit.client.0.smithi114.stdout:0/861: read d1/d16/d1f/d28/d51/d61/d7e/d90/d95/fc9 [167826,71063] 0 2024-02-16T22:22:16.851 INFO:tasks.workunit.client.0.smithi114.stdout:0/862: chown d1/d16/dbf/df0 3698 1 2024-02-16T22:22:16.855 INFO:tasks.workunit.client.0.smithi114.stdout:9/799: rename d2/dd/fc1 to d2/d5c/dc9/d47/d63/dbb/fff 0 2024-02-16T22:22:16.861 INFO:tasks.workunit.client.0.smithi114.stdout:6/789: creat d6/d2c/d38/d41/d60/d84/dc9/fe8 x:0 0 0 2024-02-16T22:22:16.866 INFO:tasks.workunit.client.0.smithi114.stdout:8/772: dwrite d4/d16/d25/d39/f4f [0,4194304] 0 2024-02-16T22:22:16.866 INFO:tasks.workunit.client.0.smithi114.stdout:3/782: link d4/d26/d4a/cff d4/d26/dd7/d104/d80/dbd/c10f 0 2024-02-16T22:22:16.876 INFO:tasks.workunit.client.0.smithi114.stdout:7/866: dwrite d3/d14/f29 [0,4194304] 0 2024-02-16T22:22:16.877 INFO:tasks.workunit.client.0.smithi114.stdout:7/867: stat d3/d46/f7e 0 2024-02-16T22:22:16.877 INFO:tasks.workunit.client.0.smithi114.stdout:8/773: creat d4/d11/d9d/ddf/f112 x:0 0 0 2024-02-16T22:22:16.882 INFO:tasks.workunit.client.0.smithi114.stdout:6/790: creat d6/d2c/d75/fe9 x:0 0 0 2024-02-16T22:22:16.886 INFO:tasks.workunit.client.0.smithi114.stdout:7/868: mkdir d3/d22/d73/da7/db4/d12f 0 2024-02-16T22:22:16.887 INFO:tasks.workunit.client.0.smithi114.stdout:8/774: chown d4/d16/d25/d46/l9f 18351 1 2024-02-16T22:22:16.891 INFO:tasks.workunit.client.0.smithi114.stdout:6/791: mknod d6/dd/d25/d34/d58/d9f/da3/cea 0 2024-02-16T22:22:16.892 INFO:tasks.workunit.client.0.smithi114.stdout:5/772: truncate d0/d28/d98/d1e/d86/fa9 3389225 0 2024-02-16T22:22:16.895 INFO:tasks.workunit.client.0.smithi114.stdout:0/863: write d1/d16/d1f/d28/d51/d61/d6d/d6e/f113 [1458919,75977] 0 2024-02-16T22:22:16.897 INFO:tasks.workunit.client.0.smithi114.stdout:5/773: symlink d0/df/lfd 0 2024-02-16T22:22:16.902 INFO:tasks.workunit.client.0.smithi114.stdout:2/860: sync 2024-02-16T22:22:16.903 INFO:tasks.workunit.client.0.smithi114.stdout:0/864: dwrite d1/d1a/d47/d62/fee [0,4194304] 0 2024-02-16T22:22:16.907 INFO:tasks.workunit.client.0.smithi114.stdout:6/792: truncate d6/d2c/d38/fdc 1857743 0 2024-02-16T22:22:16.911 INFO:tasks.workunit.client.0.smithi114.stdout:3/783: dwrite d4/d26/f31 [0,4194304] 0 2024-02-16T22:22:16.911 INFO:tasks.workunit.client.0.smithi114.stdout:5/774: creat d0/d28/d98/d25/d89/d99/db0/ffe x:0 0 0 2024-02-16T22:22:16.912 INFO:tasks.workunit.client.0.smithi114.stdout:5/775: dread - d0/d2d/fdf zero size 2024-02-16T22:22:16.914 INFO:tasks.workunit.client.0.smithi114.stdout:9/800: creat d2/dd/f100 x:0 0 0 2024-02-16T22:22:16.927 INFO:tasks.workunit.client.0.smithi114.stdout:7/869: dwrite d3/d22/d73/da7/f68 [0,4194304] 0 2024-02-16T22:22:16.928 INFO:tasks.workunit.client.0.smithi114.stdout:8/775: rmdir d4/d11/d78/d7a/d111 0 2024-02-16T22:22:16.940 INFO:tasks.workunit.client.0.smithi114.stdout:6/793: mknod d6/dd/d25/d34/d58/ceb 0 2024-02-16T22:22:16.942 INFO:tasks.workunit.client.0.smithi114.stdout:5/776: symlink d0/d2d/lff 0 2024-02-16T22:22:16.943 INFO:tasks.workunit.client.0.smithi114.stdout:9/801: fsync d2/dd/fcc 0 2024-02-16T22:22:16.947 INFO:tasks.workunit.client.0.smithi114.stdout:7/870: rename d3/d3d/l20 to d3/d92/dbe/l130 0 2024-02-16T22:22:16.950 INFO:tasks.workunit.client.0.smithi114.stdout:2/861: unlink dd/df/l43 0 2024-02-16T22:22:16.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:16 smithi114 ceph-mon[29235]: mgrmap e25: smithi123.inzgjd(active, since 65s), standbys: smithi114.vcullz 2024-02-16T22:22:16.952 INFO:tasks.workunit.client.0.smithi114.stdout:8/776: truncate d4/d11/d78/d7a/d7e/fba 1808254 0 2024-02-16T22:22:16.953 INFO:tasks.workunit.client.0.smithi114.stdout:8/777: chown d4/d16/d25/d46/fbd 106278604 1 2024-02-16T22:22:16.953 INFO:tasks.workunit.client.0.smithi114.stdout:7/871: dread d3/d22/d61/d6a/f94 [0,4194304] 0 2024-02-16T22:22:16.956 INFO:tasks.workunit.client.0.smithi114.stdout:6/794: read - d6/fcd zero size 2024-02-16T22:22:16.956 INFO:tasks.workunit.client.0.smithi114.stdout:3/784: write d4/d24/d2d/df0/fea [655252,60348] 0 2024-02-16T22:22:16.960 INFO:tasks.workunit.client.0.smithi114.stdout:5/777: read d0/d28/d98/d25/d2e/d3b/f59 [94829,97794] 0 2024-02-16T22:22:16.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.958+0000 7f25d9830700 1 -- 172.21.15.114:0/2495992948 --> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f25d41067b0 con 0x7f25bc06d660 2024-02-16T22:22:16.963 INFO:tasks.workunit.client.0.smithi114.stdout:2/862: rename dd/d6b/f89 to dd/da6/f119 0 2024-02-16T22:22:16.965 INFO:tasks.workunit.client.0.smithi114.stdout:2/863: truncate dd/df/d1c/d21/d61/dec/f118 631122 0 2024-02-16T22:22:16.969 INFO:tasks.workunit.client.0.smithi114.stdout:9/802: write d2/d5c/dc9/f75 [2037550,128232] 0 2024-02-16T22:22:16.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.969+0000 7f25d882e700 1 -- 172.21.15.114:0/2495992948 <== mgr.24441 v2:172.21.15.123:6828/1539029039 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f25d41067b0 con 0x7f25bc06d660 2024-02-16T22:22:16.972 INFO:teuthology.orchestra.run.smithi114.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:22:16.972 INFO:teuthology.orchestra.run.smithi114.stdout:alertmanager.smithi114 smithi114 *:9093,9094 running (8m) 54s ago 9m 24.0M - 0.25.0 c8568f914cd2 e7404fea400e 2024-02-16T22:22:16.972 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi114 smithi114 running (9m) 54s ago 9m 7666k - 18.2.1 94eadd990e0a 7fd07162cd96 2024-02-16T22:22:16.972 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi123 smithi123 running (8m) 57s ago 8m 7402k - 18.2.1 94eadd990e0a 7687f5e99621 2024-02-16T22:22:16.972 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi114 smithi114 running (9m) 54s ago 9m 6660k - 18.2.1 94eadd990e0a fc8fbfe4fdd5 2024-02-16T22:22:16.972 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi123 smithi123 running (8m) 57s ago 8m 6665k - 18.2.1 94eadd990e0a a6249642b9bb 2024-02-16T22:22:16.972 INFO:teuthology.orchestra.run.smithi114.stdout:grafana.smithi114 smithi114 *:3000 running (8m) 54s ago 9m 89.8M - 9.4.7 954c08fa6188 db1c26c3bc20 2024-02-16T22:22:16.973 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.fdxpey smithi114 running (4m) 54s ago 4m 15.3M - 18.2.1 94eadd990e0a 704b40c144fb 2024-02-16T22:22:16.973 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.zzryke smithi114 running (4m) 54s ago 4m 150M - 18.2.1 94eadd990e0a 1f795d23c9a4 2024-02-16T22:22:16.973 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.gopcfa smithi123 running (4m) 57s ago 4m 15.2M - 18.2.1 94eadd990e0a 166134b7532a 2024-02-16T22:22:16.973 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.vhlqbe smithi123 running (4m) 57s ago 4m 16.7M - 18.2.1 94eadd990e0a 243c8bb96fa8 2024-02-16T22:22:16.973 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi114.vcullz smithi114 *:8443,9283,8765 starting - - - - 2024-02-16T22:22:16.973 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi123.inzgjd smithi123 *:8443,9283,8765 running (84s) 57s ago 8m 487M - 19.0.0-1450-gf78a58c0 f081f847070a c9d6ba733f8b 2024-02-16T22:22:16.973 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi114 smithi114 running (11m) 54s ago 11m 52.5M 2048M 18.2.1 94eadd990e0a b4c4433aa3a4 2024-02-16T22:22:16.973 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi123 smithi123 running (8m) 57s ago 8m 37.2M 2048M 18.2.1 94eadd990e0a c217fc30c152 2024-02-16T22:22:16.973 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi114 smithi114 *:9100 running (9m) 54s ago 9m 15.7M - 1.5.0 0da6a335fe13 744a090e06d0 2024-02-16T22:22:16.973 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi123 smithi123 *:9100 running (8m) 57s ago 8m 15.2M - 1.5.0 0da6a335fe13 3994b8c29b2e 2024-02-16T22:22:16.973 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 smithi114 running (7m) 54s ago 7m 74.7M 1216M 18.2.1 94eadd990e0a 24ca2e9714e0 2024-02-16T22:22:16.973 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 smithi114 running (7m) 54s ago 7m 77.5M 1216M 18.2.1 94eadd990e0a 32bb191b88e7 2024-02-16T22:22:16.973 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 smithi114 running (6m) 54s ago 6m 70.5M 1216M 18.2.1 94eadd990e0a 6608718d0818 2024-02-16T22:22:16.973 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 smithi123 running (6m) 57s ago 6m 86.3M 2240M 18.2.1 94eadd990e0a 575db379de3d 2024-02-16T22:22:16.973 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 smithi123 running (6m) 57s ago 6m 73.9M 2240M 18.2.1 94eadd990e0a 7e266866cd9c 2024-02-16T22:22:16.973 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 smithi123 running (5m) 57s ago 5m 69.8M 2240M 18.2.1 94eadd990e0a 24e5bef19fda 2024-02-16T22:22:16.974 INFO:teuthology.orchestra.run.smithi114.stdout:prometheus.smithi114 smithi114 *:9095 starting - - - - 2024-02-16T22:22:16.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.972+0000 7f25d9830700 1 -- 172.21.15.114:0/2495992948 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f25bc06d660 msgr2=0x7f25bc06fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:16.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.972+0000 7f25d9830700 1 --2- 172.21.15.114:0/2495992948 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f25bc06d660 0x7f25bc06fb20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009d20 tx=0x7f25c8009260 comp rx=0 tx=0).stop 2024-02-16T22:22:16.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.972+0000 7f25d9830700 1 -- 172.21.15.114:0/2495992948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25d40fee80 msgr2=0x7f25d41963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:16.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.972+0000 7f25d9830700 1 --2- 172.21.15.114:0/2495992948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25d40fee80 0x7f25d41963a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f25c4000c00 tx=0x7f25c400f690 comp rx=0 tx=0).stop 2024-02-16T22:22:16.974 INFO:tasks.workunit.client.0.smithi114.stdout:6/795: creat d6/fec x:0 0 0 2024-02-16T22:22:16.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.972+0000 7f25d9830700 1 -- 172.21.15.114:0/2495992948 shutdown_connections 2024-02-16T22:22:16.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.972+0000 7f25d9830700 1 --2- 172.21.15.114:0/2495992948 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f25bc06d660 0x7f25bc06fb20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:16.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.972+0000 7f25d9830700 1 --2- 172.21.15.114:0/2495992948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25d40fee80 0x7f25d41963a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:16.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.972+0000 7f25d9830700 1 --2- 172.21.15.114:0/2495992948 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25d4101e60 0x7f25d41968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:16.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.972+0000 7f25d9830700 1 -- 172.21.15.114:0/2495992948 >> 172.21.15.114:0/2495992948 conn(0x7f25d40fa7c0 msgr2=0x7f25d4105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:22:16.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.972+0000 7f25d9830700 1 -- 172.21.15.114:0/2495992948 shutdown_connections 2024-02-16T22:22:16.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:16.972+0000 7f25d9830700 1 -- 172.21.15.114:0/2495992948 wait complete. 2024-02-16T22:22:16.976 INFO:tasks.workunit.client.0.smithi114.stdout:3/785: rmdir d4/d24/d2d 39 2024-02-16T22:22:16.978 INFO:tasks.workunit.client.0.smithi114.stdout:7/872: write d3/d46/f7e [1824952,40445] 0 2024-02-16T22:22:16.980 INFO:tasks.workunit.client.0.smithi114.stdout:7/873: write d3/d92/dbe/de6/f11c [759525,129980] 0 2024-02-16T22:22:16.981 INFO:tasks.workunit.client.0.smithi114.stdout:3/786: dread d4/d24/f25 [0,4194304] 0 2024-02-16T22:22:16.985 INFO:tasks.workunit.client.0.smithi114.stdout:2/864: symlink dd/df/d1c/d21/d61/d63/d83/df8/l11a 0 2024-02-16T22:22:16.986 INFO:tasks.workunit.client.0.smithi114.stdout:0/865: getdents d1/d16/d1f/d28/d51/d60/d52 0 2024-02-16T22:22:16.989 INFO:tasks.workunit.client.0.smithi114.stdout:8/778: mknod d4/c113 0 2024-02-16T22:22:16.990 INFO:tasks.workunit.client.0.smithi114.stdout:9/803: creat d2/dd0/f101 x:0 0 0 2024-02-16T22:22:16.991 INFO:tasks.workunit.client.0.smithi114.stdout:9/804: readlink d2/d5c/lba 0 2024-02-16T22:22:16.993 INFO:tasks.workunit.client.0.smithi114.stdout:4/936: sync 2024-02-16T22:22:16.998 INFO:tasks.workunit.client.0.smithi114.stdout:9/805: dread d2/d5c/dc9/d47/d63/fc5 [0,4194304] 0 2024-02-16T22:22:17.002 INFO:tasks.workunit.client.0.smithi114.stdout:0/866: dread d1/d78/f7d [0,4194304] 0 2024-02-16T22:22:17.004 INFO:tasks.workunit.client.0.smithi114.stdout:6/796: write d6/f18 [1067434,103649] 0 2024-02-16T22:22:17.007 INFO:tasks.workunit.client.0.smithi114.stdout:2/865: creat dd/df/d1c/d21/d61/d63/d83/df8/f11b x:0 0 0 2024-02-16T22:22:17.007 INFO:tasks.workunit.client.0.smithi114.stdout:0/867: fsync d1/d16/d1f/d28/d51/d61/d6d/d6e/f6f 0 2024-02-16T22:22:17.007 INFO:tasks.workunit.client.0.smithi114.stdout:9/806: dwrite d2/d5c/dc9/ffc [0,4194304] 0 2024-02-16T22:22:17.010 INFO:tasks.workunit.client.0.smithi114.stdout:8/779: fdatasync d4/d11/d52/fa9 0 2024-02-16T22:22:17.013 INFO:tasks.workunit.client.0.smithi114.stdout:4/937: creat dd/d59/d8a/f14f x:0 0 0 2024-02-16T22:22:17.017 INFO:tasks.workunit.client.0.smithi114.stdout:5/778: link d0/df/d5e/cd5 d0/df/d7a/dc4/c100 0 2024-02-16T22:22:17.017 INFO:tasks.workunit.client.0.smithi114.stdout:2/866: dwrite dd/d3b/f104 [0,4194304] 0 2024-02-16T22:22:17.021 INFO:tasks.workunit.client.0.smithi114.stdout:3/787: creat d4/d24/d2d/df0/d77/f110 x:0 0 0 2024-02-16T22:22:17.026 INFO:tasks.workunit.client.0.smithi114.stdout:0/868: unlink d1/d16/d1f/d28/d51/d61/d7e/d90/c7a 0 2024-02-16T22:22:17.028 INFO:tasks.workunit.client.0.smithi114.stdout:0/869: chown d1/d16/d1f/d28/d51/d61/d7e/d90/d95/fc9 1203952 1 2024-02-16T22:22:17.031 INFO:tasks.workunit.client.0.smithi114.stdout:4/938: unlink dd/d1b/d41/fe5 0 2024-02-16T22:22:17.034 INFO:tasks.workunit.client.0.smithi114.stdout:2/867: chown dd/d3b/d8a/f7c 37 1 2024-02-16T22:22:17.039 INFO:tasks.workunit.client.0.smithi114.stdout:9/807: fsync d2/dd/d3c/f7f 0 2024-02-16T22:22:17.044 INFO:tasks.workunit.client.0.smithi114.stdout:1/764: sync 2024-02-16T22:22:17.046 INFO:tasks.workunit.client.0.smithi114.stdout:1/765: write d4/d9/d46/d47/db3/ff6 [914081,6094] 0 2024-02-16T22:22:17.048 INFO:tasks.workunit.client.0.smithi114.stdout:4/939: rename dd/d2e/d56/f13d to dd/d1b/d41/d74/d12a/f150 0 2024-02-16T22:22:17.050 INFO:tasks.workunit.client.0.smithi114.stdout:7/874: getdents d3/d87/dd1 0 2024-02-16T22:22:17.050 INFO:tasks.workunit.client.0.smithi114.stdout:8/780: dwrite d4/d11/d78/d7a/d31/f4d [0,4194304] 0 2024-02-16T22:22:17.057 INFO:tasks.workunit.client.0.smithi114.stdout:2/868: chown dd/f5d 156666 1 2024-02-16T22:22:17.060 INFO:tasks.workunit.client.0.smithi114.stdout:0/870: write d1/d16/d1f/d28/d51/d61/d7e/d90/fcf [658789,86778] 0 2024-02-16T22:22:17.062 INFO:tasks.workunit.client.0.smithi114.stdout:5/779: dwrite d0/d28/d98/d25/d2e/d3b/f59 [0,4194304] 0 2024-02-16T22:22:17.069 INFO:tasks.workunit.client.0.smithi114.stdout:1/766: readlink d4/d9/d46/l4f 0 2024-02-16T22:22:17.074 INFO:tasks.workunit.client.0.smithi114.stdout:1/767: read d4/d9/d3d/d3f/f81 [756494,84661] 0 2024-02-16T22:22:17.074 INFO:tasks.workunit.client.0.smithi114.stdout:9/808: rename d2/dd/d3c/ce7 to d2/d1a/d23/d32/d46/c102 0 2024-02-16T22:22:17.077 INFO:tasks.workunit.client.0.smithi114.stdout:7/875: mknod d3/d22/d73/da7/d6f/df8/c131 0 2024-02-16T22:22:17.078 INFO:tasks.workunit.client.0.smithi114.stdout:8/781: creat d4/d16/d25/d46/db5/dca/f114 x:0 0 0 2024-02-16T22:22:17.080 INFO:tasks.workunit.client.0.smithi114.stdout:3/788: creat d4/f111 x:0 0 0 2024-02-16T22:22:17.081 INFO:tasks.workunit.client.0.smithi114.stdout:3/789: chown d4/d87/lb2 36427 1 2024-02-16T22:22:17.082 INFO:tasks.workunit.client.0.smithi114.stdout:0/871: dread - d1/d1a/d47/d62/d7f/d8c/db1/f93 zero size 2024-02-16T22:22:17.083 INFO:tasks.workunit.client.0.smithi114.stdout:1/768: dwrite d4/d9/d46/d67/d92/dca/f109 [0,4194304] 0 2024-02-16T22:22:17.086 INFO:tasks.workunit.client.0.smithi114.stdout:0/872: read d1/d16/d1f/d28/d51/d61/d7e/fc2 [669272,4175] 0 2024-02-16T22:22:17.086 INFO:tasks.workunit.client.0.smithi114.stdout:6/797: getdents d6/d2c/d38/d41/d60/d84 0 2024-02-16T22:22:17.090 INFO:tasks.workunit.client.0.smithi114.stdout:4/940: getdents dd/d1b/d24/d57/d12b/d38/d14e 0 2024-02-16T22:22:17.091 INFO:tasks.workunit.client.0.smithi114.stdout:4/941: write dd/f19 [1924744,122107] 0 2024-02-16T22:22:17.095 INFO:tasks.workunit.client.0.smithi114.stdout:2/869: rename dd/d4e/c57 to dd/df/d1c/d21/d61/d63/dc2/c11c 0 2024-02-16T22:22:17.097 INFO:tasks.workunit.client.0.smithi114.stdout:7/876: fsync d3/d14/f2c 0 2024-02-16T22:22:17.098 INFO:tasks.workunit.client.0.smithi114.stdout:8/782: stat d4/d24/d2c/d5f/c9b 0 2024-02-16T22:22:17.102 INFO:tasks.workunit.client.0.smithi114.stdout:5/780: creat d0/d28/d98/d1e/dd7/f101 x:0 0 0 2024-02-16T22:22:17.105 INFO:tasks.workunit.client.0.smithi114.stdout:0/873: mkdir d1/d16/d1f/d28/d51/d61/d6d/d6e/d117 0 2024-02-16T22:22:17.106 INFO:tasks.workunit.client.0.smithi114.stdout:6/798: truncate d6/da/f9b 3002653 0 2024-02-16T22:22:17.107 INFO:tasks.workunit.client.0.smithi114.stdout:6/799: chown d6/dd/d25/d34/d61/d62/db5/cda 2996 1 2024-02-16T22:22:17.115 INFO:tasks.workunit.client.0.smithi114.stdout:1/769: rename d4/f37 to d4/d9/d46/d67/d92/dca/dcc/d98/ddd/d113/f114 0 2024-02-16T22:22:17.119 INFO:tasks.workunit.client.0.smithi114.stdout:7/877: write d3/d22/f48 [5785914,77779] 0 2024-02-16T22:22:17.123 INFO:tasks.workunit.client.0.smithi114.stdout:8/783: symlink d4/d11/d78/d7a/d36/d3b/de4/l115 0 2024-02-16T22:22:17.123 INFO:tasks.workunit.client.0.smithi114.stdout:8/784: chown d4/d16/d25/d46/db5/lbc 680898592 1 2024-02-16T22:22:17.125 INFO:tasks.workunit.client.0.smithi114.stdout:5/781: creat d0/df/d7a/dc7/f102 x:0 0 0 2024-02-16T22:22:17.127 INFO:tasks.workunit.client.0.smithi114.stdout:7/878: dwrite d3/d14/f70 [0,4194304] 0 2024-02-16T22:22:17.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.126+0000 7f588f532700 1 -- 172.21.15.114:0/671185050 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58880ffd50 msgr2=0x7f58881001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:17.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.126+0000 7f588f532700 1 --2- 172.21.15.114:0/671185050 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58880ffd50 0x7f58881001b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5878009910 tx=0x7f5878009c20 comp rx=0 tx=0).stop 2024-02-16T22:22:17.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.126+0000 7f588f532700 1 -- 172.21.15.114:0/671185050 shutdown_connections 2024-02-16T22:22:17.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.126+0000 7f588f532700 1 --2- 172.21.15.114:0/671185050 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58880ffd50 0x7f58881001b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:17.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.126+0000 7f588f532700 1 --2- 172.21.15.114:0/671185050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58881010a0 0x7f58881014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:17.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.126+0000 7f588f532700 1 -- 172.21.15.114:0/671185050 >> 172.21.15.114:0/671185050 conn(0x7f58880fa7c0 msgr2=0x7f58880fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:22:17.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.126+0000 7f588f532700 1 -- 172.21.15.114:0/671185050 shutdown_connections 2024-02-16T22:22:17.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.127+0000 7f588f532700 1 -- 172.21.15.114:0/671185050 wait complete. 2024-02-16T22:22:17.129 INFO:tasks.workunit.client.0.smithi114.stdout:5/782: fsync d0/d28/d98/d1e/d63/d96/d9a/fd6 0 2024-02-16T22:22:17.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.127+0000 7f588f532700 1 Processor -- start 2024-02-16T22:22:17.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.127+0000 7f588f532700 1 -- start start 2024-02-16T22:22:17.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.127+0000 7f588f532700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58880ffd50 0x7f58881985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:17.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.127+0000 7f588f532700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58881010a0 0x7f5888198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:17.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.128+0000 7f588f532700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5888199110 con 0x7f58881010a0 2024-02-16T22:22:17.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.128+0000 7f588f532700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5888199250 con 0x7f58880ffd50 2024-02-16T22:22:17.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.128+0000 7f588cacd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58881010a0 0x7f5888198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:17.130 INFO:tasks.workunit.client.0.smithi114.stdout:9/809: dwrite d2/f4 [0,4194304] 0 2024-02-16T22:22:17.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.128+0000 7f588d2ce700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58880ffd50 0x7f58881985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:17.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.128+0000 7f588cacd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58881010a0 0x7f5888198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50134/0 (socket says 172.21.15.114:50134) 2024-02-16T22:22:17.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.128+0000 7f588d2ce700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58880ffd50 0x7f58881985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.114:42282/0 (socket says 172.21.15.114:42282) 2024-02-16T22:22:17.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.128+0000 7f588cacd700 1 -- 172.21.15.114:0/1483746963 learned_addr learned my addr 172.21.15.114:0/1483746963 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:22:17.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.128+0000 7f588cacd700 1 -- 172.21.15.114:0/1483746963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58880ffd50 msgr2=0x7f58881985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:17.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.128+0000 7f588cacd700 1 --2- 172.21.15.114:0/1483746963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58880ffd50 0x7f58881985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:17.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.128+0000 7f588cacd700 1 -- 172.21.15.114:0/1483746963 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58780095f0 con 0x7f58881010a0 2024-02-16T22:22:17.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.129+0000 7f588cacd700 1 --2- 172.21.15.114:0/1483746963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58881010a0 0x7f5888198af0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5878000c00 tx=0x7f587800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:22:17.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.129+0000 7f587e7fc700 1 -- 172.21.15.114:0/1483746963 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f587801d070 con 0x7f58881010a0 2024-02-16T22:22:17.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.129+0000 7f588f532700 1 -- 172.21.15.114:0/1483746963 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f588819dca0 con 0x7f58881010a0 2024-02-16T22:22:17.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.129+0000 7f588f532700 1 -- 172.21.15.114:0/1483746963 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588819e190 con 0x7f58881010a0 2024-02-16T22:22:17.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.129+0000 7f587e7fc700 1 -- 172.21.15.114:0/1483746963 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5878005dc0 con 0x7f58881010a0 2024-02-16T22:22:17.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.129+0000 7f587e7fc700 1 -- 172.21.15.114:0/1483746963 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f587801d070 con 0x7f58881010a0 2024-02-16T22:22:17.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.131+0000 7f588f532700 1 -- 172.21.15.114:0/1483746963 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f586c005320 con 0x7f58881010a0 2024-02-16T22:22:17.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.131+0000 7f587e7fc700 1 -- 172.21.15.114:0/1483746963 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) v1 ==== 91450+0+0 (secure 0 0 0) 0x7f587800faf0 con 0x7f58881010a0 2024-02-16T22:22:17.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.132+0000 7f587e7fc700 1 --2- 172.21.15.114:0/1483746963 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f587406d6b0 0x7f587406fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:17.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.132+0000 7f587e7fc700 1 -- 172.21.15.114:0/1483746963 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5726+0+0 (secure 0 0 0) 0x7f587808e280 con 0x7f58881010a0 2024-02-16T22:22:17.134 INFO:tasks.workunit.client.0.smithi114.stdout:0/874: mkdir d1/d16/d48/d118 0 2024-02-16T22:22:17.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.132+0000 7f588d2ce700 1 --2- 172.21.15.114:0/1483746963 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f587406d6b0 0x7f587406fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:17.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.134+0000 7f588d2ce700 1 --2- 172.21.15.114:0/1483746963 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f587406d6b0 0x7f587406fb70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f58880681d0 tx=0x7f5884009260 comp rx=0 tx=0).ready entity=mgr.24441 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:22:17.137 INFO:tasks.workunit.client.0.smithi114.stdout:6/800: creat d6/dd/d25/d34/d61/d62/fed x:0 0 0 2024-02-16T22:22:17.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.140+0000 7f587e7fc700 1 -- 172.21.15.114:0/1483746963 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182869 (secure 0 0 0) 0x7f587805ba20 con 0x7f58881010a0 2024-02-16T22:22:17.142 INFO:tasks.workunit.client.0.smithi114.stdout:3/790: truncate d4/dd/d33/d76/f9e 1163137 0 2024-02-16T22:22:17.146 INFO:tasks.workunit.client.0.smithi114.stdout:1/770: symlink d4/d63/d69/d7c/d9b/dae/l115 0 2024-02-16T22:22:17.147 INFO:tasks.workunit.client.0.smithi114.stdout:2/870: symlink dd/d115/l11d 0 2024-02-16T22:22:17.155 INFO:tasks.workunit.client.0.smithi114.stdout:9/810: rename d2/d33/f7d to d2/d5c/dc9/d47/db2/f103 0 2024-02-16T22:22:17.157 INFO:tasks.workunit.client.0.smithi114.stdout:4/942: dwrite dd/d1b/f3e [0,4194304] 0 2024-02-16T22:22:17.163 INFO:tasks.workunit.client.0.smithi114.stdout:0/875: mknod d1/d16/d48/c119 0 2024-02-16T22:22:17.164 INFO:tasks.workunit.client.0.smithi114.stdout:8/785: dread d4/d11/d78/d7a/d36/f5d [0,4194304] 0 2024-02-16T22:22:17.165 INFO:tasks.workunit.client.0.smithi114.stdout:6/801: mkdir d6/dd/d25/d34/d58/d9f/da3/dbd/dee 0 2024-02-16T22:22:17.166 INFO:tasks.workunit.client.0.smithi114.stdout:1/771: fsync d4/d9/d46/f5f 0 2024-02-16T22:22:17.168 INFO:tasks.workunit.client.0.smithi114.stdout:0/876: stat d1/d1a/d47/d62/d89/fbc 0 2024-02-16T22:22:17.168 INFO:tasks.workunit.client.0.smithi114.stdout:7/879: truncate d3/d22/d73/da7/d4e/f7f 1405400 0 2024-02-16T22:22:17.171 INFO:tasks.workunit.client.0.smithi114.stdout:6/802: dread - d6/dd/d25/f53 zero size 2024-02-16T22:22:17.171 INFO:tasks.workunit.client.0.smithi114.stdout:1/772: chown d4/d9/ce 1663858 1 2024-02-16T22:22:17.176 INFO:tasks.workunit.client.0.smithi114.stdout:9/811: dwrite d2/d1a/f6b [0,4194304] 0 2024-02-16T22:22:17.178 INFO:tasks.workunit.client.0.smithi114.stdout:2/871: creat dd/d4e/f11e x:0 0 0 2024-02-16T22:22:17.181 INFO:tasks.workunit.client.0.smithi114.stdout:5/783: write d0/d28/d98/d25/d3e/d7f/f8e [635441,115740] 0 2024-02-16T22:22:17.182 INFO:tasks.workunit.client.0.smithi114.stdout:5/784: dread - d0/d28/d98/dd0/fe3 zero size 2024-02-16T22:22:17.187 INFO:tasks.workunit.client.0.smithi114.stdout:1/773: symlink d4/d9/d46/d67/d92/dca/dce/l116 0 2024-02-16T22:22:17.191 INFO:tasks.workunit.client.0.smithi114.stdout:0/877: truncate d1/d1a/d47/d62/d89/fb5 525766 0 2024-02-16T22:22:17.191 INFO:tasks.workunit.client.0.smithi114.stdout:0/878: stat d1/d78 0 2024-02-16T22:22:17.204 INFO:tasks.workunit.client.0.smithi114.stdout:8/786: dwrite d4/d11/d78/d7a/d7e/fba [0,4194304] 0 2024-02-16T22:22:17.207 INFO:tasks.workunit.client.0.smithi114.stdout:7/880: link d3/f99 d3/d14/f132 0 2024-02-16T22:22:17.208 INFO:tasks.workunit.client.0.smithi114.stdout:7/881: stat d3/fc 0 2024-02-16T22:22:17.209 INFO:tasks.workunit.client.0.smithi114.stdout:9/812: dread d2/d5c/dc9/d47/d63/dbb/fff [0,4194304] 0 2024-02-16T22:22:17.209 INFO:tasks.workunit.client.0.smithi114.stdout:7/882: read - d3/d5a/f119 zero size 2024-02-16T22:22:17.214 INFO:tasks.workunit.client.0.smithi114.stdout:0/879: fsync d1/d16/d1f/d28/fe6 0 2024-02-16T22:22:17.216 INFO:tasks.workunit.client.0.smithi114.stdout:6/803: dwrite d6/d2c/f6d [0,4194304] 0 2024-02-16T22:22:17.224 INFO:tasks.workunit.client.0.smithi114.stdout:1/774: dwrite d4/d63/d69/d7c/d9b/db0/d74/f87 [4194304,4194304] 0 2024-02-16T22:22:17.228 INFO:tasks.workunit.client.0.smithi114.stdout:4/943: link dd/d1b/d41/l8e dd/d1b/d24/d57/d12b/d61/d13c/l151 0 2024-02-16T22:22:17.229 INFO:tasks.workunit.client.0.smithi114.stdout:4/944: read - dd/d1b/db0/f109 zero size 2024-02-16T22:22:17.232 INFO:tasks.workunit.client.0.smithi114.stdout:1/775: dwrite d4/d63/d69/d7c/d9b/db0/f38 [0,4194304] 0 2024-02-16T22:22:17.232 INFO:tasks.workunit.client.0.smithi114.stdout:8/787: dread - d4/d11/d78/d7a/d7e/fc5 zero size 2024-02-16T22:22:17.235 INFO:tasks.workunit.client.0.smithi114.stdout:2/872: write dd/f92 [2230409,73816] 0 2024-02-16T22:22:17.251 INFO:tasks.workunit.client.0.smithi114.stdout:1/776: symlink d4/d9/d46/d67/d92/dca/dcc/d98/l117 0 2024-02-16T22:22:17.252 INFO:tasks.workunit.client.0.smithi114.stdout:9/813: write d2/d5c/dc9/d47/d63/f73 [4443142,42433] 0 2024-02-16T22:22:17.257 INFO:tasks.workunit.client.0.smithi114.stdout:7/883: dwrite d3/f27 [4194304,4194304] 0 2024-02-16T22:22:17.260 INFO:tasks.workunit.client.0.smithi114.stdout:0/880: mknod d1/d1a/d47/d62/d7f/d9d/df2/c11a 0 2024-02-16T22:22:17.263 INFO:tasks.workunit.client.0.smithi114.stdout:6/804: dwrite d6/dd/d25/d34/fa4 [0,4194304] 0 2024-02-16T22:22:17.263 INFO:tasks.workunit.client.0.smithi114.stdout:5/785: getdents d0/d2d 0 2024-02-16T22:22:17.266 INFO:tasks.workunit.client.0.smithi114.stdout:2/873: fsync dd/d3b/da7/d4c/f64 0 2024-02-16T22:22:17.268 INFO:tasks.workunit.client.0.smithi114.stdout:1/777: mkdir d4/d9/d46/d67/d92/dca/d118 0 2024-02-16T22:22:17.270 INFO:tasks.workunit.client.0.smithi114.stdout:8/788: dread d4/d16/d25/d46/fbd [0,4194304] 0 2024-02-16T22:22:17.271 INFO:tasks.workunit.client.0.smithi114.stdout:4/945: dwrite dd/d1b/d41/d74/f135 [0,4194304] 0 2024-02-16T22:22:17.272 INFO:tasks.workunit.client.0.smithi114.stdout:5/786: mkdir d0/d28/d103 0 2024-02-16T22:22:17.274 INFO:tasks.workunit.client.0.smithi114.stdout:9/814: dread d2/d1a/d65/fe2 [0,4194304] 0 2024-02-16T22:22:17.278 INFO:tasks.workunit.client.0.smithi114.stdout:0/881: dwrite d1/d16/d48/d98/fef [0,4194304] 0 2024-02-16T22:22:17.278 INFO:tasks.workunit.client.0.smithi114.stdout:6/805: creat d6/d2c/d5e/fef x:0 0 0 2024-02-16T22:22:17.286 INFO:tasks.workunit.client.0.smithi114.stdout:1/778: dwrite d4/d9/d46/d47/ddf/f10d [0,4194304] 0 2024-02-16T22:22:17.286 INFO:tasks.workunit.client.0.smithi114.stdout:9/815: rmdir d2/d1a/d65 39 2024-02-16T22:22:17.286 INFO:tasks.workunit.client.0.smithi114.stdout:7/884: rmdir d3/d92/dbe/de6/d11e 0 2024-02-16T22:22:17.288 INFO:tasks.workunit.client.0.smithi114.stdout:5/787: creat d0/d28/d98/d1e/d63/d96/f104 x:0 0 0 2024-02-16T22:22:17.289 INFO:tasks.workunit.client.0.smithi114.stdout:2/874: mknod dd/d6b/c11f 0 2024-02-16T22:22:17.294 INFO:tasks.workunit.client.0.smithi114.stdout:8/789: rename d4/d16/d25/d39/cab to d4/d16/d25/d7c/d10f/c116 0 2024-02-16T22:22:17.296 INFO:tasks.workunit.client.0.smithi114.stdout:5/788: dwrite d0/df/d6d/fa1 [0,4194304] 0 2024-02-16T22:22:17.305 INFO:tasks.workunit.client.0.smithi114.stdout:2/875: creat dd/df/d1c/d21/d61/d69/de9/f120 x:0 0 0 2024-02-16T22:22:17.310 INFO:tasks.workunit.client.0.smithi114.stdout:8/790: symlink d4/d11/d78/d7a/d36/d3b/l117 0 2024-02-16T22:22:17.311 INFO:tasks.workunit.client.0.smithi114.stdout:8/791: dread - d4/d11/d78/d7a/d7e/fc5 zero size 2024-02-16T22:22:17.312 INFO:tasks.workunit.client.0.smithi114.stdout:5/789: rmdir d0/d28/d98/d25/d89/de2/db3 39 2024-02-16T22:22:17.314 INFO:tasks.workunit.client.0.smithi114.stdout:9/816: fdatasync d2/dd/ff 0 2024-02-16T22:22:17.318 INFO:tasks.workunit.client.0.smithi114.stdout:9/817: dread d2/d5c/dc9/d47/db2/fc3 [0,4194304] 0 2024-02-16T22:22:17.319 INFO:tasks.workunit.client.0.smithi114.stdout:8/792: creat d4/d16/d25/d46/db5/dca/f118 x:0 0 0 2024-02-16T22:22:17.319 INFO:tasks.workunit.client.0.smithi114.stdout:2/876: read dd/df/d1c/d21/d61/d69/f72 [1212990,49097] 0 2024-02-16T22:22:17.320 INFO:tasks.workunit.client.0.smithi114.stdout:5/790: symlink d0/d28/d98/d1e/d63/d9e/l105 0 2024-02-16T22:22:17.324 INFO:tasks.workunit.client.0.smithi114.stdout:7/885: dwrite d3/d22/d73/da7/d6f/d89/da6/fe0 [0,4194304] 0 2024-02-16T22:22:17.326 INFO:tasks.workunit.client.0.smithi114.stdout:7/886: fdatasync d3/d92/dd5/f113 0 2024-02-16T22:22:17.328 INFO:tasks.workunit.client.0.smithi114.stdout:7/887: write d3/d3d/ff3 [600943,81469] 0 2024-02-16T22:22:17.331 INFO:tasks.workunit.client.0.smithi114.stdout:4/946: dread dd/f19 [0,4194304] 0 2024-02-16T22:22:17.331 INFO:tasks.workunit.client.0.smithi114.stdout:2/877: read dd/f36 [7330203,130476] 0 2024-02-16T22:22:17.331 INFO:tasks.workunit.client.0.smithi114.stdout:0/882: dread d1/d1a/d47/d62/d7f/fa8 [0,4194304] 0 2024-02-16T22:22:17.332 INFO:tasks.workunit.client.0.smithi114.stdout:9/818: symlink d2/d1a/l104 0 2024-02-16T22:22:17.333 INFO:tasks.workunit.client.0.smithi114.stdout:4/947: stat dd/d1b/d41/d74/fd9 0 2024-02-16T22:22:17.335 INFO:tasks.workunit.client.0.smithi114.stdout:1/779: dread d4/f30 [0,4194304] 0 2024-02-16T22:22:17.335 INFO:tasks.workunit.client.0.smithi114.stdout:2/878: unlink dd/d3b/d8a/la8 0 2024-02-16T22:22:17.337 INFO:tasks.workunit.client.0.smithi114.stdout:4/948: chown dd/f89 477155 1 2024-02-16T22:22:17.337 INFO:tasks.workunit.client.0.smithi114.stdout:9/819: chown d2/dd/d3c/d89/d8b/lbd 5302 1 2024-02-16T22:22:17.337 INFO:tasks.workunit.client.0.smithi114.stdout:2/879: fdatasync dd/d3b/d8a/fba 0 2024-02-16T22:22:17.339 INFO:tasks.workunit.client.0.smithi114.stdout:5/791: creat d0/d28/d98/d25/f106 x:0 0 0 2024-02-16T22:22:17.339 INFO:tasks.workunit.client.0.smithi114.stdout:1/780: chown d4/d63/d69/d7c/dc0/cc9 498143 1 2024-02-16T22:22:17.340 INFO:tasks.workunit.client.0.smithi114.stdout:7/888: dwrite d3/d14/f18 [0,4194304] 0 2024-02-16T22:22:17.346 INFO:tasks.workunit.client.0.smithi114.stdout:0/883: rename d1/d16/d1f/d28/d67 to d1/d16/d1f/d28/d51/d61/d7e/d11b 0 2024-02-16T22:22:17.349 INFO:tasks.workunit.client.0.smithi114.stdout:4/949: unlink dd/d1b/d41/c4f 0 2024-02-16T22:22:17.351 INFO:tasks.workunit.client.0.smithi114.stdout:2/880: creat dd/d6b/da5/f121 x:0 0 0 2024-02-16T22:22:17.352 INFO:tasks.workunit.client.0.smithi114.stdout:1/781: mkdir d4/d63/d69/d72/d119 0 2024-02-16T22:22:17.356 INFO:tasks.workunit.client.0.smithi114.stdout:0/884: unlink d1/d16/d1f/d28/d51/d61/d7e/d90/f85 0 2024-02-16T22:22:17.362 INFO:tasks.workunit.client.0.smithi114.stdout:7/889: link d3/f27 d3/d22/d76/f133 0 2024-02-16T22:22:17.364 INFO:tasks.workunit.client.0.smithi114.stdout:5/792: write d0/d28/d98/d25/d3e/f5c [3526550,100301] 0 2024-02-16T22:22:17.368 INFO:tasks.workunit.client.0.smithi114.stdout:2/881: mknod dd/c122 0 2024-02-16T22:22:17.374 INFO:tasks.workunit.client.0.smithi114.stdout:7/890: dread d3/d92/fc5 [0,4194304] 0 2024-02-16T22:22:17.377 INFO:tasks.workunit.client.0.smithi114.stdout:1/782: truncate d4/d9/d46/d67/fa8 3749344 0 2024-02-16T22:22:17.381 INFO:tasks.workunit.client.0.smithi114.stdout:4/950: rename dd/d1b/d24/fd6 to dd/d59/d8f/f152 0 2024-02-16T22:22:17.383 INFO:tasks.workunit.client.0.smithi114.stdout:5/793: unlink d0/d28/d98/d25/f106 0 2024-02-16T22:22:17.389 INFO:tasks.workunit.client.0.smithi114.stdout:9/820: link d2/d1a/d65/f94 d2/d5c/d5f/d7b/f105 0 2024-02-16T22:22:17.389 INFO:tasks.workunit.client.0.smithi114.stdout:9/821: readlink d2/l2b 0 2024-02-16T22:22:17.390 INFO:tasks.workunit.client.0.smithi114.stdout:0/885: truncate d1/d1a/f38 539392 0 2024-02-16T22:22:17.392 INFO:tasks.workunit.client.0.smithi114.stdout:1/783: symlink d4/d9/d3d/def/l11a 0 2024-02-16T22:22:17.393 INFO:tasks.workunit.client.0.smithi114.stdout:4/951: chown dd/d1b/d24/f70 50948717 1 2024-02-16T22:22:17.411 INFO:tasks.workunit.client.0.smithi114.stdout:2/882: chown dd/d3b/d8a/dcb/de2/l101 1 1 2024-02-16T22:22:17.411 INFO:tasks.workunit.client.0.smithi114.stdout:9/822: rmdir d2/d1a/d23/d32 39 2024-02-16T22:22:17.412 INFO:tasks.workunit.client.0.smithi114.stdout:0/886: creat d1/d16/d1f/d28/d51/d60/d52/f11c x:0 0 0 2024-02-16T22:22:17.412 INFO:tasks.workunit.client.0.smithi114.stdout:1/784: read - d4/d9/d46/d4d/ff1 zero size 2024-02-16T22:22:17.415 INFO:tasks.workunit.client.0.smithi114.stdout:4/952: write dd/d1b/d41/fbb [574899,4132] 0 2024-02-16T22:22:17.416 INFO:tasks.workunit.client.0.smithi114.stdout:7/891: rename d3/d22/d73/da7/d6f/d89/da6/ce1 to d3/d22/d73/da7/c134 0 2024-02-16T22:22:17.418 INFO:tasks.workunit.client.0.smithi114.stdout:4/953: readlink dd/d1b/d41/l8e 0 2024-02-16T22:22:17.418 INFO:tasks.workunit.client.0.smithi114.stdout:7/892: readlink d3/d22/d73/l95 0 2024-02-16T22:22:17.419 INFO:tasks.workunit.client.0.smithi114.stdout:5/794: dwrite d0/d28/d98/d1e/d86/f92 [0,4194304] 0 2024-02-16T22:22:17.425 INFO:tasks.workunit.client.0.smithi114.stdout:1/785: mkdir d4/d9/d46/d67/d92/dca/dce/d11b 0 2024-02-16T22:22:17.427 INFO:tasks.workunit.client.0.smithi114.stdout:0/887: rename d1/d1a/d47/d62/d89/fbc to d1/d16/d48/d98/f11d 0 2024-02-16T22:22:17.429 INFO:tasks.workunit.client.0.smithi114.stdout:7/893: mknod d3/d22/d61/d6a/c135 0 2024-02-16T22:22:17.437 INFO:tasks.workunit.client.0.smithi114.stdout:9/823: write d2/d5c/d5f/d91/de4/fed [4768462,35534] 0 2024-02-16T22:22:17.437 INFO:tasks.workunit.client.0.smithi114.stdout:4/954: dread dd/d1b/d24/d57/d12b/d38/d87/dad/fbf [0,4194304] 0 2024-02-16T22:22:17.439 INFO:tasks.workunit.client.0.smithi114.stdout:7/894: mknod d3/d92/dbe/c136 0 2024-02-16T22:22:17.440 INFO:tasks.workunit.client.0.smithi114.stdout:7/895: write d3/d5a/f119 [345186,78245] 0 2024-02-16T22:22:17.442 INFO:tasks.workunit.client.0.smithi114.stdout:5/795: write d0/df/f2b [3906072,118130] 0 2024-02-16T22:22:17.444 INFO:tasks.workunit.client.0.smithi114.stdout:5/796: write d0/df/d5e/ff5 [109301,83199] 0 2024-02-16T22:22:17.448 INFO:tasks.workunit.client.0.smithi114.stdout:2/883: dwrite dd/d6b/ff5 [0,4194304] 0 2024-02-16T22:22:17.456 INFO:tasks.workunit.client.0.smithi114.stdout:9/824: write d2/d5c/fc8 [1060,115517] 0 2024-02-16T22:22:17.457 INFO:tasks.workunit.client.0.smithi114.stdout:0/888: rename d1/d16/f24 to d1/d1a/d47/d62/f11e 0 2024-02-16T22:22:17.463 INFO:tasks.workunit.client.0.smithi114.stdout:4/955: dwrite dd/d1b/d24/d57/d12b/d38/d87/fda [0,4194304] 0 2024-02-16T22:22:17.463 INFO:tasks.workunit.client.0.smithi114.stdout:1/786: link d4/f2d d4/d9/d46/d47/f11c 0 2024-02-16T22:22:17.466 INFO:tasks.workunit.client.0.smithi114.stdout:0/889: creat d1/d16/d1f/d28/d51/f11f x:0 0 0 2024-02-16T22:22:17.467 INFO:tasks.workunit.client.0.smithi114.stdout:7/896: link d3/c66 d3/d22/d73/da7/d6f/c137 0 2024-02-16T22:22:17.468 INFO:tasks.workunit.client.0.smithi114.stdout:2/884: creat dd/df/d1c/d21/d61/d69/f123 x:0 0 0 2024-02-16T22:22:17.468 INFO:tasks.workunit.client.0.smithi114.stdout:0/890: stat d1/d16/d1f/d28/d51/d61/d7e/d90/d95/l9f 0 2024-02-16T22:22:17.469 INFO:tasks.workunit.client.0.smithi114.stdout:1/787: symlink d4/d63/d69/d7c/d9b/db0/d74/l11d 0 2024-02-16T22:22:17.470 INFO:tasks.workunit.client.0.smithi114.stdout:0/891: chown d1/fe 663505 1 2024-02-16T22:22:17.471 INFO:tasks.workunit.client.0.smithi114.stdout:4/956: symlink dd/d2e/d56/ddc/dde/l153 0 2024-02-16T22:22:17.473 INFO:tasks.workunit.client.0.smithi114.stdout:7/897: creat d3/d3d/f138 x:0 0 0 2024-02-16T22:22:17.474 INFO:tasks.workunit.client.0.smithi114.stdout:4/957: write dd/d59/dfa/f140 [1603490,120368] 0 2024-02-16T22:22:17.477 INFO:tasks.workunit.client.0.smithi114.stdout:9/825: dread d2/d5c/dc9/d47/d63/f73 [4194304,4194304] 0 2024-02-16T22:22:17.478 INFO:tasks.workunit.client.0.smithi114.stdout:1/788: symlink d4/d9/d46/d67/d92/dca/dcc/d98/l11e 0 2024-02-16T22:22:17.481 INFO:tasks.workunit.client.0.smithi114.stdout:9/826: dread d2/d5c/fc8 [0,4194304] 0 2024-02-16T22:22:17.482 INFO:tasks.workunit.client.0.smithi114.stdout:1/789: rename d4/d63 to d4/d63/d69/d7c/d9b/df0/d108/d11f 22 2024-02-16T22:22:17.483 INFO:tasks.workunit.client.0.smithi114.stdout:7/898: creat d3/d87/dd1/f139 x:0 0 0 2024-02-16T22:22:17.485 INFO:tasks.workunit.client.0.smithi114.stdout:5/797: dwrite d0/d28/d98/d25/d2e/d3b/f78 [0,4194304] 0 2024-02-16T22:22:17.486 INFO:tasks.workunit.client.0.smithi114.stdout:2/885: dread dd/d3b/da7/d4c/fbe [0,4194304] 0 2024-02-16T22:22:17.488 INFO:tasks.workunit.client.0.smithi114.stdout:4/958: mknod dd/d2e/d56/ddc/dde/c154 0 2024-02-16T22:22:17.489 INFO:tasks.workunit.client.0.smithi114.stdout:4/959: fdatasync dd/d2e/d3f/f149 0 2024-02-16T22:22:17.490 INFO:tasks.workunit.client.0.smithi114.stdout:4/960: fdatasync dd/d59/dfa/f10a 0 2024-02-16T22:22:17.493 INFO:tasks.workunit.client.0.smithi114.stdout:0/892: creat d1/d16/d1f/d28/d51/d61/d7e/d90/df5/f120 x:0 0 0 2024-02-16T22:22:17.493 INFO:tasks.workunit.client.0.smithi114.stdout:7/899: dwrite d3/d14/f18 [0,4194304] 0 2024-02-16T22:22:17.498 INFO:tasks.workunit.client.0.smithi114.stdout:1/790: mknod d4/d63/d69/d7c/d9b/db0/d74/c120 0 2024-02-16T22:22:17.499 INFO:tasks.workunit.client.0.smithi114.stdout:1/791: readlink d4/d9/d3d/d3f/d79/l10f 0 2024-02-16T22:22:17.504 INFO:tasks.workunit.client.0.smithi114.stdout:2/886: creat dd/df/ddb/f124 x:0 0 0 2024-02-16T22:22:17.507 INFO:tasks.workunit.client.0.smithi114.stdout:4/961: mkdir dd/d1b/db0/de3/d155 0 2024-02-16T22:22:17.509 INFO:tasks.workunit.client.0.smithi114.stdout:7/900: mknod d3/d22/db7/dea/c13a 0 2024-02-16T22:22:17.509 INFO:tasks.workunit.client.0.smithi114.stdout:7/901: chown d3/d46 414844 1 2024-02-16T22:22:17.510 INFO:tasks.workunit.client.0.smithi114.stdout:0/893: rename d1/f70 to d1/d16/d1f/d28/d51/d61/d7e/d90/f121 0 2024-02-16T22:22:17.512 INFO:tasks.workunit.client.0.smithi114.stdout:1/792: truncate d4/d63/d69/d72/f97 617600 0 2024-02-16T22:22:17.514 INFO:tasks.workunit.client.0.smithi114.stdout:7/902: dread - d3/d87/fee zero size 2024-02-16T22:22:17.514 INFO:tasks.workunit.client.0.smithi114.stdout:9/827: creat d2/d5c/dc9/d1d/f106 x:0 0 0 2024-02-16T22:22:17.516 INFO:tasks.workunit.client.0.smithi114.stdout:0/894: symlink d1/d1a/d47/d62/d89/dba/l122 0 2024-02-16T22:22:17.516 INFO:tasks.workunit.client.0.smithi114.stdout:0/895: chown d1/d16/l2b 0 1 2024-02-16T22:22:17.520 INFO:tasks.workunit.client.0.smithi114.stdout:5/798: write d0/df/f1d [1507201,55507] 0 2024-02-16T22:22:17.522 INFO:tasks.workunit.client.0.smithi114.stdout:1/793: rename l3 to d4/d9/d46/d67/l121 0 2024-02-16T22:22:17.524 INFO:tasks.workunit.client.0.smithi114.stdout:7/903: creat d3/d22/d73/da7/d4e/f13b x:0 0 0 2024-02-16T22:22:17.525 INFO:tasks.workunit.client.0.smithi114.stdout:9/828: fdatasync d2/d5c/d5f/fd9 0 2024-02-16T22:22:17.531 INFO:tasks.workunit.client.0.smithi114.stdout:0/896: unlink d1/d16/d1f/d28/d51/d60/d52/ca0 0 2024-02-16T22:22:17.533 INFO:tasks.workunit.client.0.smithi114.stdout:5/799: unlink d0/df/f1d 0 2024-02-16T22:22:17.533 INFO:tasks.workunit.client.0.smithi114.stdout:2/887: truncate dd/d3b/da7/d8b/dc0/fcf 691980 0 2024-02-16T22:22:17.534 INFO:tasks.workunit.client.0.smithi114.stdout:5/800: chown d0/d28/d98/d1e/d63/f68 1255 1 2024-02-16T22:22:17.536 INFO:tasks.workunit.client.0.smithi114.stdout:2/888: dread dd/df/d1c/d21/d61/dec/f118 [0,4194304] 0 2024-02-16T22:22:17.538 INFO:tasks.workunit.client.0.smithi114.stdout:7/904: creat d3/d92/dbe/f13c x:0 0 0 2024-02-16T22:22:17.544 INFO:tasks.workunit.client.0.smithi114.stdout:4/962: getdents dd/d59/dfa 0 2024-02-16T22:22:17.548 INFO:tasks.workunit.client.0.smithi114.stdout:5/801: rename d0/d28/d98/d1e/d63/d96/fd9 to d0/d28/d98/d1e/d63/d96/d9a/db8/f107 0 2024-02-16T22:22:17.551 INFO:tasks.workunit.client.0.smithi114.stdout:2/889: dread dd/df/d1c/f67 [0,4194304] 0 2024-02-16T22:22:17.552 INFO:tasks.workunit.client.0.smithi114.stdout:7/905: mkdir d3/d3d/d13d 0 2024-02-16T22:22:17.553 INFO:tasks.workunit.client.0.smithi114.stdout:2/890: write dd/df/d1c/d21/d61/d69/de9/f120 [104744,115230] 0 2024-02-16T22:22:17.560 INFO:tasks.workunit.client.0.smithi114.stdout:1/794: link d4/d63/d69/d7c/d9b/db0/c42 d4/d9/d3d/d3f/d79/c122 0 2024-02-16T22:22:17.562 INFO:tasks.workunit.client.0.smithi114.stdout:5/802: dread d0/d28/d98/d1e/f60 [0,4194304] 0 2024-02-16T22:22:17.563 INFO:tasks.workunit.client.0.smithi114.stdout:7/906: symlink d3/d14/l13e 0 2024-02-16T22:22:17.571 INFO:tasks.workunit.client.0.smithi114.stdout:0/897: write d1/d16/d1f/d28/f54 [778613,71112] 0 2024-02-16T22:22:17.572 INFO:tasks.workunit.client.0.smithi114.stdout:4/963: write dd/d1b/f47 [1821030,54737] 0 2024-02-16T22:22:17.575 INFO:tasks.workunit.client.0.smithi114.stdout:4/964: chown dd/d59/d8f/dea/d134/f142 456905732 1 2024-02-16T22:22:17.576 INFO:tasks.workunit.client.0.smithi114.stdout:2/891: write dd/d6b/fbf [885418,98977] 0 2024-02-16T22:22:17.579 INFO:tasks.workunit.client.0.smithi114.stdout:0/898: dread d1/d16/d48/d98/fef [0,4194304] 0 2024-02-16T22:22:17.581 INFO:tasks.workunit.client.0.smithi114.stdout:7/907: dread d3/d46/f47 [0,4194304] 0 2024-02-16T22:22:17.585 INFO:tasks.workunit.client.0.smithi114.stdout:7/908: fsync d3/d22/d73/da7/d4e/fbf 0 2024-02-16T22:22:17.585 INFO:tasks.workunit.client.0.smithi114.stdout:4/965: dwrite dd/d1b/f139 [0,4194304] 0 2024-02-16T22:22:17.586 INFO:tasks.workunit.client.0.smithi114.stdout:4/966: stat dd/d59/dfa 0 2024-02-16T22:22:17.587 INFO:tasks.workunit.client.0.smithi114.stdout:1/795: mkdir d4/d123 0 2024-02-16T22:22:17.593 INFO:tasks.workunit.client.0.smithi114.stdout:2/892: truncate dd/df/d1c/d21/d61/d69/f72 2591856 0 2024-02-16T22:22:17.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.612+0000 7f588f532700 1 -- 172.21.15.114:0/1483746963 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f586c006200 con 0x7f58881010a0 2024-02-16T22:22:17.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.612+0000 7f587e7fc700 1 -- 172.21.15.114:0/1483746963 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f587805ba20 con 0x7f58881010a0 2024-02-16T22:22:17.614 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:22:17.614 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": { 2024-02-16T22:22:17.614 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-02-16T22:22:17.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:22:17.614 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": { 2024-02-16T22:22:17.614 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:22:17.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:22:17.615 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": { 2024-02-16T22:22:17.615 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-02-16T22:22:17.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:22:17.615 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-02-16T22:22:17.615 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-02-16T22:22:17.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:22:17.615 INFO:teuthology.orchestra.run.smithi114.stdout: "overall": { 2024-02-16T22:22:17.615 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-02-16T22:22:17.615 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:22:17.615 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-02-16T22:22:17.615 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:22:17.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.614+0000 7f588f532700 1 -- 172.21.15.114:0/1483746963 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f587406d6b0 msgr2=0x7f587406fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:17.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.614+0000 7f588f532700 1 --2- 172.21.15.114:0/1483746963 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f587406d6b0 0x7f587406fb70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f58880681d0 tx=0x7f5884009260 comp rx=0 tx=0).stop 2024-02-16T22:22:17.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.614+0000 7f588f532700 1 -- 172.21.15.114:0/1483746963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58881010a0 msgr2=0x7f5888198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:17.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.614+0000 7f588f532700 1 --2- 172.21.15.114:0/1483746963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58881010a0 0x7f5888198af0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5878000c00 tx=0x7f587800f690 comp rx=0 tx=0).stop 2024-02-16T22:22:17.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.614+0000 7f588f532700 1 -- 172.21.15.114:0/1483746963 shutdown_connections 2024-02-16T22:22:17.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.614+0000 7f588f532700 1 --2- 172.21.15.114:0/1483746963 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f587406d6b0 0x7f587406fb70 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:17.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.614+0000 7f588f532700 1 --2- 172.21.15.114:0/1483746963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58880ffd50 0x7f58881985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:17.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.614+0000 7f588f532700 1 --2- 172.21.15.114:0/1483746963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58881010a0 0x7f5888198af0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:17.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.614+0000 7f588f532700 1 -- 172.21.15.114:0/1483746963 >> 172.21.15.114:0/1483746963 conn(0x7f58880fa7c0 msgr2=0x7f58880fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:22:17.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.614+0000 7f588f532700 1 -- 172.21.15.114:0/1483746963 shutdown_connections 2024-02-16T22:22:17.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.614+0000 7f588f532700 1 -- 172.21.15.114:0/1483746963 wait complete. 2024-02-16T22:22:17.629 INFO:tasks.workunit.client.0.smithi114.stdout:0/899: dwrite d1/d16/d1f/d28/d51/d61/d7e/d90/fcf [0,4194304] 0 2024-02-16T22:22:17.632 INFO:tasks.workunit.client.0.smithi114.stdout:4/967: dwrite dd/d59/dfa/f10a [0,4194304] 0 2024-02-16T22:22:17.633 INFO:tasks.workunit.client.0.smithi114.stdout:4/968: chown dd/fd8 27 1 2024-02-16T22:22:17.634 INFO:tasks.workunit.client.0.smithi114.stdout:5/803: chown d0/d28/d98/d25/d89/d99/db0/fec 4197869 1 2024-02-16T22:22:17.638 INFO:tasks.workunit.client.0.smithi114.stdout:7/909: mkdir d3/d22/d73/da7/db4/d12f/d13f 0 2024-02-16T22:22:17.641 INFO:tasks.workunit.client.0.smithi114.stdout:2/893: fsync dd/da6/f119 0 2024-02-16T22:22:17.642 INFO:tasks.workunit.client.0.smithi114.stdout:2/894: chown dd/d6b/da5/df6 26110101 1 2024-02-16T22:22:17.642 INFO:tasks.workunit.client.0.smithi114.stdout:4/969: mkdir dd/d2e/d156 0 2024-02-16T22:22:17.643 INFO:tasks.workunit.client.0.smithi114.stdout:7/910: truncate d3/d22/d73/f7c 2473289 0 2024-02-16T22:22:17.644 INFO:tasks.workunit.client.0.smithi114.stdout:1/796: dread d4/f91 [0,4194304] 0 2024-02-16T22:22:17.646 INFO:tasks.workunit.client.0.smithi114.stdout:1/797: stat d4/d9/d46/d67/d92/dca/d118 0 2024-02-16T22:22:17.646 INFO:tasks.workunit.client.0.smithi114.stdout:2/895: fdatasync dd/df/d1c/f2e 0 2024-02-16T22:22:17.647 INFO:tasks.workunit.client.0.smithi114.stdout:4/970: truncate dd/d1b/d24/f70 1018815 0 2024-02-16T22:22:17.647 INFO:tasks.workunit.client.0.smithi114.stdout:1/798: stat d4/d9 0 2024-02-16T22:22:17.648 INFO:tasks.workunit.client.0.smithi114.stdout:7/911: creat d3/d22/d73/da7/db4/d12f/f140 x:0 0 0 2024-02-16T22:22:17.652 INFO:tasks.workunit.client.0.smithi114.stdout:7/912: rmdir d3/d87 39 2024-02-16T22:22:17.654 INFO:tasks.workunit.client.0.smithi114.stdout:1/799: rename d4/d63/leb to d4/d9/d46/d67/d92/dca/dce/dd3/l124 0 2024-02-16T22:22:17.663 INFO:tasks.workunit.client.0.smithi114.stdout:2/896: creat dd/d3b/da7/f125 x:0 0 0 2024-02-16T22:22:17.665 INFO:tasks.workunit.client.0.smithi114.stdout:0/900: dread d1/d16/d1f/d28/d51/d61/d6d/f9c [0,4194304] 0 2024-02-16T22:22:17.666 INFO:tasks.workunit.client.0.smithi114.stdout:0/901: chown d1/d16/d1f/d28/lfd 1492989 1 2024-02-16T22:22:17.667 INFO:tasks.workunit.client.0.smithi114.stdout:0/902: write d1/d16/d1f/d28/d51/d60/dd1/f10d [122339,10567] 0 2024-02-16T22:22:17.676 INFO:tasks.workunit.client.0.smithi114.stdout:4/971: dwrite dd/d1b/d24/d57/d12b/d38/f5e [0,4194304] 0 2024-02-16T22:22:17.677 INFO:tasks.workunit.client.0.smithi114.stdout:1/800: dwrite d4/d63/d69/d7c/d9b/dae/fe8 [0,4194304] 0 2024-02-16T22:22:17.698 INFO:tasks.workunit.client.0.smithi114.stdout:0/903: rename d1/c9 to d1/d16/d1f/d28/d51/d60/d52/c123 0 2024-02-16T22:22:17.704 INFO:tasks.workunit.client.0.smithi114.stdout:0/904: fsync d1/d16/f73 0 2024-02-16T22:22:17.705 INFO:tasks.workunit.client.0.smithi114.stdout:0/905: write d1/fe [4040637,20076] 0 2024-02-16T22:22:17.714 INFO:tasks.workunit.client.0.smithi114.stdout:0/906: dwrite d1/d16/d1f/d28/d51/d61/d7e/d90/df5/f120 [0,4194304] 0 2024-02-16T22:22:17.715 INFO:tasks.workunit.client.0.smithi114.stdout:0/907: readlink d1/d1a/d47/l64 0 2024-02-16T22:22:17.716 INFO:tasks.workunit.client.0.smithi114.stdout:4/972: link dd/d1b/d24/d57/d12b/f73 dd/d1b/d24/d57/d12b/d38/d14e/f157 0 2024-02-16T22:22:17.723 INFO:tasks.workunit.client.0.smithi114.stdout:0/908: dread d1/d16/d1f/d28/d51/d61/d7e/fc2 [0,4194304] 0 2024-02-16T22:22:17.725 INFO:tasks.workunit.client.0.smithi114.stdout:4/973: mknod dd/d1b/d24/c158 0 2024-02-16T22:22:17.730 INFO:tasks.workunit.client.0.smithi114.stdout:4/974: creat dd/d2e/d56/ddc/dde/d138/f159 x:0 0 0 2024-02-16T22:22:17.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.756+0000 7fb666a7e700 1 -- 172.21.15.114:0/2238239949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb660103f70 msgr2=0x7fb660106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:17.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.756+0000 7fb666a7e700 1 --2- 172.21.15.114:0/2238239949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb660103f70 0x7fb660106360 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb64c009910 tx=0x7fb64c009c20 comp rx=0 tx=0).stop 2024-02-16T22:22:17.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.756+0000 7fb666a7e700 1 -- 172.21.15.114:0/2238239949 shutdown_connections 2024-02-16T22:22:17.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.756+0000 7fb666a7e700 1 --2- 172.21.15.114:0/2238239949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb660103f70 0x7fb660106360 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:17.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.756+0000 7fb666a7e700 1 --2- 172.21.15.114:0/2238239949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb660101640 0x7fb660103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:17.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.756+0000 7fb666a7e700 1 -- 172.21.15.114:0/2238239949 >> 172.21.15.114:0/2238239949 conn(0x7fb6600fb030 msgr2=0x7fb6600fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:22:17.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.756+0000 7fb666a7e700 1 -- 172.21.15.114:0/2238239949 shutdown_connections 2024-02-16T22:22:17.759 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:17 smithi114 ceph-mon[29235]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:22:17.759 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:17 smithi114 ceph-mon[29235]: pgmap v36: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 81 MiB/s rd, 175 MiB/s wr, 403 op/s 2024-02-16T22:22:17.759 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:17 smithi114 ceph-mon[29235]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:22:17.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.756+0000 7fb666a7e700 1 -- 172.21.15.114:0/2238239949 wait complete. 2024-02-16T22:22:17.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.757+0000 7fb666a7e700 1 Processor -- start 2024-02-16T22:22:17.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.757+0000 7fb666a7e700 1 -- start start 2024-02-16T22:22:17.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.757+0000 7fb666a7e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb660101640 0x7fb660198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:17.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.757+0000 7fb666a7e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb660103f70 0x7fb660198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:17.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.757+0000 7fb666a7e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6601990c0 con 0x7fb660101640 2024-02-16T22:22:17.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.757+0000 7fb666a7e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb660199200 con 0x7fb660103f70 2024-02-16T22:22:17.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.757+0000 7fb66481a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb660101640 0x7fb660198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:17.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.757+0000 7fb66481a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb660101640 0x7fb660198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50154/0 (socket says 172.21.15.114:50154) 2024-02-16T22:22:17.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.757+0000 7fb66481a700 1 -- 172.21.15.114:0/3226124302 learned_addr learned my addr 172.21.15.114:0/3226124302 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:22:17.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.757+0000 7fb65ffff700 1 --2- 172.21.15.114:0/3226124302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb660103f70 0x7fb660198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:17.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.757+0000 7fb66481a700 1 -- 172.21.15.114:0/3226124302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb660103f70 msgr2=0x7fb660198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:17.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.757+0000 7fb66481a700 1 --2- 172.21.15.114:0/3226124302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb660103f70 0x7fb660198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:17.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.757+0000 7fb66481a700 1 -- 172.21.15.114:0/3226124302 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64c0095f0 con 0x7fb660101640 2024-02-16T22:22:17.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.757+0000 7fb66481a700 1 --2- 172.21.15.114:0/3226124302 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb660101640 0x7fb660198560 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb65400d8d0 tx=0x7fb65400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:22:17.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.758+0000 7fb65dffb700 1 -- 172.21.15.114:0/3226124302 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6540098a0 con 0x7fb660101640 2024-02-16T22:22:17.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.758+0000 7fb65dffb700 1 -- 172.21.15.114:0/3226124302 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb654010460 con 0x7fb660101640 2024-02-16T22:22:17.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.758+0000 7fb666a7e700 1 -- 172.21.15.114:0/3226124302 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb66019dcb0 con 0x7fb660101640 2024-02-16T22:22:17.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.758+0000 7fb65dffb700 1 -- 172.21.15.114:0/3226124302 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb65400f5f0 con 0x7fb660101640 2024-02-16T22:22:17.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.758+0000 7fb666a7e700 1 -- 172.21.15.114:0/3226124302 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb66019e1b0 con 0x7fb660101640 2024-02-16T22:22:17.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.759+0000 7fb666a7e700 1 -- 172.21.15.114:0/3226124302 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6600fcc20 con 0x7fb660101640 2024-02-16T22:22:17.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.759+0000 7fb65dffb700 1 -- 172.21.15.114:0/3226124302 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) v1 ==== 91450+0+0 (secure 0 0 0) 0x7fb654009a00 con 0x7fb660101640 2024-02-16T22:22:17.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.760+0000 7fb65dffb700 1 --2- 172.21.15.114:0/3226124302 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fb65006d660 0x7fb65006fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:17.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.760+0000 7fb65dffb700 1 -- 172.21.15.114:0/3226124302 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5726+0+0 (secure 0 0 0) 0x7fb65408cf10 con 0x7fb660101640 2024-02-16T22:22:17.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.760+0000 7fb65ffff700 1 --2- 172.21.15.114:0/3226124302 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fb65006d660 0x7fb65006fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:17.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.760+0000 7fb65ffff700 1 --2- 172.21.15.114:0/3226124302 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fb65006d660 0x7fb65006fb20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00b5c0 tx=0x7fb64c005ee0 comp rx=0 tx=0).ready entity=mgr.24441 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:22:17.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:17.764+0000 7fb65dffb700 1 -- 172.21.15.114:0/3226124302 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182869 (secure 0 0 0) 0x7fb65405a6b0 con 0x7fb660101640 2024-02-16T22:22:17.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:17 smithi123 ceph-mon[37161]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:22:17.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:17 smithi123 ceph-mon[37161]: pgmap v36: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 81 MiB/s rd, 175 MiB/s wr, 403 op/s 2024-02-16T22:22:17.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:17 smithi123 ceph-mon[37161]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:22:17.836 INFO:tasks.workunit.client.0.smithi114.stdout:0/909: unlink d1/d16/cc5 0 2024-02-16T22:22:17.865 INFO:tasks.workunit.client.0.smithi114.stdout:0/910: creat d1/d1a/d47/d62/d89/d104/f124 x:0 0 0 2024-02-16T22:22:17.877 INFO:tasks.workunit.client.0.smithi114.stdout:0/911: rename d1/d1a/d47/l64 to d1/d16/d1f/d28/d51/d61/d6d/d6e/l125 0 2024-02-16T22:22:17.881 INFO:tasks.workunit.client.0.smithi114.stdout:0/912: fdatasync d1/d78/f7d 0 2024-02-16T22:22:17.887 INFO:tasks.workunit.client.0.smithi114.stdout:2/897: dwrite dd/df/d1c/fca [0,4194304] 0 2024-02-16T22:22:17.889 INFO:tasks.workunit.client.0.smithi114.stdout:1/801: dwrite d4/d9/f10 [0,4194304] 0 2024-02-16T22:22:17.899 INFO:tasks.workunit.client.0.smithi114.stdout:2/898: dwrite dd/d3b/d8a/dcb/f112 [0,4194304] 0 2024-02-16T22:22:17.900 INFO:tasks.workunit.client.0.smithi114.stdout:0/913: rename d1/d16/d48/d98/fc4 to d1/d16/d1f/d28/d51/d61/d7e/d11b/f126 0 2024-02-16T22:22:17.900 INFO:tasks.workunit.client.0.smithi114.stdout:1/802: dwrite d4/d63/d69/d7c/d9b/db0/fc6 [0,4194304] 0 2024-02-16T22:22:17.911 INFO:tasks.workunit.client.0.smithi114.stdout:2/899: readlink dd/d115/l11d 0 2024-02-16T22:22:17.913 INFO:tasks.workunit.client.0.smithi114.stdout:1/803: creat d4/d9/d46/d67/d92/dca/dcc/d98/ddd/df2/f125 x:0 0 0 2024-02-16T22:22:17.915 INFO:tasks.workunit.client.0.smithi114.stdout:2/900: creat dd/da6/f126 x:0 0 0 2024-02-16T22:22:17.917 INFO:tasks.workunit.client.0.smithi114.stdout:2/901: chown dd/df/d1c/d21/d61/d63/d83/l8d 49 1 2024-02-16T22:22:17.922 INFO:tasks.workunit.client.0.smithi114.stdout:1/804: mkdir d4/d9/d46/d67/d92/dca/d118/d126 0 2024-02-16T22:22:17.923 INFO:tasks.workunit.client.0.smithi114.stdout:4/975: dwrite dd/d59/dfa/f10d [0,4194304] 0 2024-02-16T22:22:17.927 INFO:tasks.workunit.client.0.smithi114.stdout:1/805: mkdir d4/d63/d69/d80/d127 0 2024-02-16T22:22:17.933 INFO:tasks.workunit.client.0.smithi114.stdout:4/976: creat dd/d1b/d24/d57/d12b/d38/f15a x:0 0 0 2024-02-16T22:22:17.940 INFO:tasks.workunit.client.0.smithi114.stdout:2/902: creat dd/d4e/d97/dd6/f127 x:0 0 0 2024-02-16T22:22:17.940 INFO:tasks.workunit.client.0.smithi114.stdout:2/903: write dd/da6/f126 [380584,69865] 0 2024-02-16T22:22:17.941 INFO:tasks.workunit.client.0.smithi114.stdout:4/977: unlink dd/d2e/d56/ddc/dde/f82 0 2024-02-16T22:22:17.947 INFO:tasks.workunit.client.0.smithi114.stdout:4/978: symlink dd/d1b/d41/d74/d12a/l15b 0 2024-02-16T22:22:17.948 INFO:tasks.workunit.client.0.smithi114.stdout:4/979: chown dd/d1b/d24/c67 1 1 2024-02-16T22:22:17.955 INFO:tasks.workunit.client.0.smithi114.stdout:4/980: dwrite dd/d1b/d24/d57/d12b/d38/d87/fda [0,4194304] 0 2024-02-16T22:22:17.967 INFO:tasks.workunit.client.0.smithi114.stdout:1/806: write d4/d63/d69/d7c/d9b/ffc [588147,102317] 0 2024-02-16T22:22:17.976 INFO:tasks.workunit.client.0.smithi114.stdout:1/807: creat d4/d63/d69/d7c/d9b/db0/f128 x:0 0 0 2024-02-16T22:22:17.980 INFO:tasks.workunit.client.0.smithi114.stdout:2/904: dwrite dd/df/d1c/d21/d61/d63/d83/df8/f109 [0,4194304] 0 2024-02-16T22:22:17.984 INFO:tasks.workunit.client.0.smithi114.stdout:4/981: write dd/d2e/d56/ddc/dde/fe8 [1109629,50595] 0 2024-02-16T22:22:17.996 INFO:tasks.workunit.client.0.smithi114.stdout:4/982: rmdir dd/d1b/d24/d57/d12b/d101 0 2024-02-16T22:22:18.010 INFO:tasks.workunit.client.0.smithi114.stdout:1/808: truncate d4/f21 2800165 0 2024-02-16T22:22:18.013 INFO:tasks.workunit.client.0.smithi114.stdout:1/809: getdents d4/d9/d3d/d3f/dd0 0 2024-02-16T22:22:18.015 INFO:tasks.workunit.client.0.smithi114.stdout:1/810: mkdir d4/d63/d69/d72/d82/d129 0 2024-02-16T22:22:18.017 INFO:tasks.workunit.client.0.smithi114.stdout:1/811: chown d4/d9/d46/d67/d92/dca/dcc/d98/cf5 112521 1 2024-02-16T22:22:18.030 INFO:tasks.workunit.client.0.smithi114.stdout:1/812: dwrite d4/d63/d69/d72/fe6 [0,4194304] 0 2024-02-16T22:22:18.031 INFO:tasks.workunit.client.0.smithi114.stdout:1/813: write d4/d9/f10 [2495067,71050] 0 2024-02-16T22:22:18.036 INFO:tasks.workunit.client.0.smithi114.stdout:1/814: read - d4/d9/d46/d67/d92/dca/dcc/fbc zero size 2024-02-16T22:22:18.055 INFO:tasks.workunit.client.0.smithi114.stdout:1/815: dread d4/d9/d3d/f39 [0,4194304] 0 2024-02-16T22:22:18.056 INFO:tasks.workunit.client.0.smithi114.stdout:1/816: dread - d4/d9/d46/d67/ff9 zero size 2024-02-16T22:22:18.062 INFO:tasks.workunit.client.0.smithi114.stdout:1/817: write d4/d9/d46/d47/fa2 [1613135,55157] 0 2024-02-16T22:22:18.069 INFO:tasks.workunit.client.0.smithi114.stdout:1/818: creat d4/d63/d69/d7c/d9b/db0/f12a x:0 0 0 2024-02-16T22:22:18.069 INFO:tasks.workunit.client.0.smithi114.stdout:1/819: chown d4/d63/d69/d7c/d9b/dae/d10a 140359 1 2024-02-16T22:22:18.070 INFO:tasks.workunit.client.0.smithi114.stdout:1/820: fdatasync d4/d9/d46/d47/fa2 0 2024-02-16T22:22:18.072 INFO:tasks.workunit.client.0.smithi114.stdout:1/821: symlink d4/d9/d3d/d3f/l12b 0 2024-02-16T22:22:18.074 INFO:tasks.workunit.client.0.smithi114.stdout:1/822: read d4/f2d [4109628,113927] 0 2024-02-16T22:22:18.075 INFO:tasks.workunit.client.0.smithi114.stdout:1/823: chown d4/d9/d46/d67/d92/dca/dcc/d98/fd7 26823 1 2024-02-16T22:22:18.079 INFO:tasks.workunit.client.0.smithi114.stdout:1/824: dread d4/d9/f10 [0,4194304] 0 2024-02-16T22:22:18.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.080+0000 7fb666a7e700 1 -- 172.21.15.114:0/3226124302 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb66002ced0 con 0x7fb660101640 2024-02-16T22:22:18.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.081+0000 7fb65dffb700 1 -- 172.21.15.114:0/3226124302 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1825 (secure 0 0 0) 0x7fb654016740 con 0x7fb660101640 2024-02-16T22:22:18.083 INFO:teuthology.orchestra.run.smithi114.stdout:e14 2024-02-16T22:22:18.083 INFO:teuthology.orchestra.run.smithi114.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:22:18.083 INFO:teuthology.orchestra.run.smithi114.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:22:18.083 INFO:teuthology.orchestra.run.smithi114.stdout:legacy client fscid: 1 2024-02-16T22:22:18.083 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:22:18.083 INFO:teuthology.orchestra.run.smithi114.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:22:18.084 INFO:teuthology.orchestra.run.smithi114.stdout:fs_name cephfs 2024-02-16T22:22:18.084 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 12 2024-02-16T22:22:18.084 INFO:teuthology.orchestra.run.smithi114.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:22:18.084 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-02-16T22:18:02.108825+0000 2024-02-16T22:22:18.084 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-02-16T22:18:16.413371+0000 2024-02-16T22:22:18.084 INFO:teuthology.orchestra.run.smithi114.stdout:tableserver 0 2024-02-16T22:22:18.084 INFO:teuthology.orchestra.run.smithi114.stdout:root 0 2024-02-16T22:22:18.084 INFO:teuthology.orchestra.run.smithi114.stdout:session_timeout 60 2024-02-16T22:22:18.084 INFO:teuthology.orchestra.run.smithi114.stdout:session_autoclose 300 2024-02-16T22:22:18.084 INFO:teuthology.orchestra.run.smithi114.stdout:max_file_size 1099511627776 2024-02-16T22:22:18.084 INFO:teuthology.orchestra.run.smithi114.stdout:required_client_features {} 2024-02-16T22:22:18.084 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure 0 2024-02-16T22:22:18.084 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure_osd_epoch 0 2024-02-16T22:22:18.085 INFO:teuthology.orchestra.run.smithi114.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:22:18.085 INFO:teuthology.orchestra.run.smithi114.stdout:max_mds 1 2024-02-16T22:22:18.085 INFO:teuthology.orchestra.run.smithi114.stdout:in 0 2024-02-16T22:22:18.085 INFO:teuthology.orchestra.run.smithi114.stdout:up {0=14468} 2024-02-16T22:22:18.085 INFO:teuthology.orchestra.run.smithi114.stdout:failed 2024-02-16T22:22:18.085 INFO:teuthology.orchestra.run.smithi114.stdout:damaged 2024-02-16T22:22:18.085 INFO:teuthology.orchestra.run.smithi114.stdout:stopped 2024-02-16T22:22:18.085 INFO:teuthology.orchestra.run.smithi114.stdout:data_pools [3] 2024-02-16T22:22:18.085 INFO:teuthology.orchestra.run.smithi114.stdout:metadata_pool 2 2024-02-16T22:22:18.085 INFO:teuthology.orchestra.run.smithi114.stdout:inline_data enabled 2024-02-16T22:22:18.085 INFO:teuthology.orchestra.run.smithi114.stdout:balancer 2024-02-16T22:22:18.085 INFO:teuthology.orchestra.run.smithi114.stdout:bal_rank_mask -1 2024-02-16T22:22:18.085 INFO:teuthology.orchestra.run.smithi114.stdout:standby_count_wanted 1 2024-02-16T22:22:18.086 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.zzryke{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.114:6826/277953535,v1:172.21.15.114:6827/277953535] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:22:18.086 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:22:18.086 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:22:18.086 INFO:teuthology.orchestra.run.smithi114.stdout:Standby daemons: 2024-02-16T22:22:18.086 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:22:18.086 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.fdxpey{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:22:18.086 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.vhlqbe{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:22:18.086 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.gopcfa{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:22:18.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.084+0000 7fb666a7e700 1 -- 172.21.15.114:0/3226124302 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fb65006d660 msgr2=0x7fb65006fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:18.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.084+0000 7fb666a7e700 1 --2- 172.21.15.114:0/3226124302 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fb65006d660 0x7fb65006fb20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00b5c0 tx=0x7fb64c005ee0 comp rx=0 tx=0).stop 2024-02-16T22:22:18.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.084+0000 7fb666a7e700 1 -- 172.21.15.114:0/3226124302 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb660101640 msgr2=0x7fb660198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:18.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.084+0000 7fb666a7e700 1 --2- 172.21.15.114:0/3226124302 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb660101640 0x7fb660198560 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb65400d8d0 tx=0x7fb65400dbe0 comp rx=0 tx=0).stop 2024-02-16T22:22:18.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.084+0000 7fb666a7e700 1 -- 172.21.15.114:0/3226124302 shutdown_connections 2024-02-16T22:22:18.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.084+0000 7fb666a7e700 1 --2- 172.21.15.114:0/3226124302 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fb65006d660 0x7fb65006fb20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:18.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.084+0000 7fb666a7e700 1 --2- 172.21.15.114:0/3226124302 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb660101640 0x7fb660198560 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:18.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.084+0000 7fb666a7e700 1 --2- 172.21.15.114:0/3226124302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb660103f70 0x7fb660198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:18.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.084+0000 7fb666a7e700 1 -- 172.21.15.114:0/3226124302 >> 172.21.15.114:0/3226124302 conn(0x7fb6600fb030 msgr2=0x7fb6600ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:22:18.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.084+0000 7fb666a7e700 1 -- 172.21.15.114:0/3226124302 shutdown_connections 2024-02-16T22:22:18.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.084+0000 7fb666a7e700 1 -- 172.21.15.114:0/3226124302 wait complete. 2024-02-16T22:22:18.088 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 14 2024-02-16T22:22:18.088 INFO:tasks.workunit.client.0.smithi114.stdout:1/825: write d4/fec [491623,95953] 0 2024-02-16T22:22:18.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.197+0000 7f1283815700 1 -- 172.21.15.114:0/2232133472 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f127c101640 msgr2=0x7f127c103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:18.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.197+0000 7f1283815700 1 --2- 172.21.15.114:0/2232133472 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f127c101640 0x7f127c103a30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1278009910 tx=0x7f1278009c20 comp rx=0 tx=0).stop 2024-02-16T22:22:18.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.197+0000 7f1283815700 1 -- 172.21.15.114:0/2232133472 shutdown_connections 2024-02-16T22:22:18.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.197+0000 7f1283815700 1 --2- 172.21.15.114:0/2232133472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f127c103f70 0x7f127c106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:18.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.197+0000 7f1283815700 1 --2- 172.21.15.114:0/2232133472 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f127c101640 0x7f127c103a30 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:18.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.197+0000 7f1283815700 1 -- 172.21.15.114:0/2232133472 >> 172.21.15.114:0/2232133472 conn(0x7f127c0fb030 msgr2=0x7f127c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:22:18.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.197+0000 7f1283815700 1 -- 172.21.15.114:0/2232133472 shutdown_connections 2024-02-16T22:22:18.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.197+0000 7f1283815700 1 -- 172.21.15.114:0/2232133472 wait complete. 2024-02-16T22:22:18.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.198+0000 7f1283815700 1 Processor -- start 2024-02-16T22:22:18.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.198+0000 7f1283815700 1 -- start start 2024-02-16T22:22:18.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.198+0000 7f1283815700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f127c101640 0x7f127c196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:18.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.198+0000 7f1283815700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f127c103f70 0x7f127c196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:18.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.198+0000 7f1283815700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127c196e60 con 0x7f127c103f70 2024-02-16T22:22:18.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.198+0000 7f1283815700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127c196fa0 con 0x7f127c101640 2024-02-16T22:22:18.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.198+0000 7f1280db0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f127c103f70 0x7f127c196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:18.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.198+0000 7f12815b1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f127c101640 0x7f127c196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:18.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.198+0000 7f12815b1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f127c101640 0x7f127c196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.114:42320/0 (socket says 172.21.15.114:42320) 2024-02-16T22:22:18.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.198+0000 7f1280db0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f127c103f70 0x7f127c196840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50174/0 (socket says 172.21.15.114:50174) 2024-02-16T22:22:18.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.198+0000 7f12815b1700 1 -- 172.21.15.114:0/4206487422 learned_addr learned my addr 172.21.15.114:0/4206487422 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:22:18.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.199+0000 7f1280db0700 1 -- 172.21.15.114:0/4206487422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f127c101640 msgr2=0x7f127c196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:18.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.199+0000 7f1280db0700 1 --2- 172.21.15.114:0/4206487422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f127c101640 0x7f127c196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:18.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.199+0000 7f1280db0700 1 -- 172.21.15.114:0/4206487422 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12780095f0 con 0x7f127c103f70 2024-02-16T22:22:18.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.199+0000 7f12815b1700 1 --2- 172.21.15.114:0/4206487422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f127c101640 0x7f127c196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:22:18.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.199+0000 7f1280db0700 1 --2- 172.21.15.114:0/4206487422 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f127c103f70 0x7f127c196840 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f126c00e970 tx=0x7f126c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:22:18.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.199+0000 7f12727fc700 1 -- 172.21.15.114:0/4206487422 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f126c00c9d0 con 0x7f127c103f70 2024-02-16T22:22:18.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.199+0000 7f1283815700 1 -- 172.21.15.114:0/4206487422 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f127c19ba50 con 0x7f127c103f70 2024-02-16T22:22:18.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.199+0000 7f12727fc700 1 -- 172.21.15.114:0/4206487422 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f126c010430 con 0x7f127c103f70 2024-02-16T22:22:18.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.199+0000 7f12727fc700 1 -- 172.21.15.114:0/4206487422 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f126c018650 con 0x7f127c103f70 2024-02-16T22:22:18.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.199+0000 7f1283815700 1 -- 172.21.15.114:0/4206487422 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f127c19bfa0 con 0x7f127c103f70 2024-02-16T22:22:18.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.200+0000 7f1283815700 1 -- 172.21.15.114:0/4206487422 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f127c190520 con 0x7f127c103f70 2024-02-16T22:22:18.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.200+0000 7f12727fc700 1 -- 172.21.15.114:0/4206487422 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) v1 ==== 91450+0+0 (secure 0 0 0) 0x7f126c0105a0 con 0x7f127c103f70 2024-02-16T22:22:18.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.201+0000 7f12727fc700 1 --2- 172.21.15.114:0/4206487422 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f126806d660 0x7f126806fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:18.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.201+0000 7f12727fc700 1 -- 172.21.15.114:0/4206487422 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5726+0+0 (secure 0 0 0) 0x7f126c014070 con 0x7f127c103f70 2024-02-16T22:22:18.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.201+0000 7f12815b1700 1 --2- 172.21.15.114:0/4206487422 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f126806d660 0x7f126806fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:18.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.201+0000 7f12815b1700 1 --2- 172.21.15.114:0/4206487422 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f126806d660 0x7f126806fb20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1278009fd0 tx=0x7f1278005940 comp rx=0 tx=0).ready entity=mgr.24441 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:22:18.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.205+0000 7f12727fc700 1 -- 172.21.15.114:0/4206487422 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182869 (secure 0 0 0) 0x7f126c052a70 con 0x7f127c103f70 2024-02-16T22:22:18.261 INFO:tasks.workunit.client.0.smithi114.stdout:6/806: sync 2024-02-16T22:22:18.261 INFO:tasks.workunit.client.0.smithi114.stdout:3/791: sync 2024-02-16T22:22:18.262 INFO:tasks.workunit.client.0.smithi114.stdout:6/807: chown d6/dd/d25/d34/d58/d9f/da3/dce 79644304 1 2024-02-16T22:22:18.263 INFO:tasks.workunit.client.0.smithi114.stdout:6/808: chown d6/d2c/d38/d54/c6c 121017 1 2024-02-16T22:22:18.274 INFO:tasks.workunit.client.0.smithi114.stdout:6/809: dread d6/d2c/d38/fdc [0,4194304] 0 2024-02-16T22:22:18.276 INFO:tasks.workunit.client.0.smithi114.stdout:3/792: dwrite d4/fd8 [0,4194304] 0 2024-02-16T22:22:18.279 INFO:tasks.workunit.client.0.smithi114.stdout:6/810: rename d6/da/fb9 to d6/d2c/d38/da8/ff0 0 2024-02-16T22:22:18.281 INFO:tasks.workunit.client.0.smithi114.stdout:3/793: mknod d4/d26/dd7/c112 0 2024-02-16T22:22:18.282 INFO:tasks.workunit.client.0.smithi114.stdout:6/811: dread d6/d2c/d38/fdc [0,4194304] 0 2024-02-16T22:22:18.283 INFO:tasks.workunit.client.0.smithi114.stdout:6/812: write d6/dd/d25/d34/fa4 [2207363,58277] 0 2024-02-16T22:22:18.296 INFO:tasks.workunit.client.0.smithi114.stdout:6/813: rename d6/d2c/l37 to d6/dd/d25/d34/d61/lf1 0 2024-02-16T22:22:18.300 INFO:tasks.workunit.client.0.smithi114.stdout:3/794: write d4/d24/d2d/df0/d77/fe4 [1196789,94871] 0 2024-02-16T22:22:18.302 INFO:tasks.workunit.client.0.smithi114.stdout:3/795: chown d4/c1b 56062217 1 2024-02-16T22:22:18.303 INFO:tasks.workunit.client.0.smithi114.stdout:6/814: getdents d6/dd/d25/d34/d61/d62/d8b/db7 0 2024-02-16T22:22:18.306 INFO:tasks.workunit.client.0.smithi114.stdout:6/815: fdatasync d6/f3e 0 2024-02-16T22:22:18.306 INFO:tasks.workunit.client.0.smithi114.stdout:6/816: stat d6/d2c/d38/l50 0 2024-02-16T22:22:18.311 INFO:tasks.workunit.client.0.smithi114.stdout:6/817: getdents d6/dd/d25/d34/d58/d9f/da3/dce 0 2024-02-16T22:22:18.314 INFO:tasks.workunit.client.0.smithi114.stdout:6/818: chown d6/d2c/c5a 39861 1 2024-02-16T22:22:18.318 INFO:tasks.workunit.client.0.smithi114.stdout:3/796: dwrite d4/d43/d105/f96 [0,4194304] 0 2024-02-16T22:22:18.323 INFO:tasks.workunit.client.0.smithi114.stdout:6/819: creat d6/d2c/de6/ff2 x:0 0 0 2024-02-16T22:22:18.325 INFO:tasks.workunit.client.0.smithi114.stdout:3/797: creat d4/d26/dd7/d104/dcc/f113 x:0 0 0 2024-02-16T22:22:18.325 INFO:tasks.workunit.client.0.smithi114.stdout:6/820: truncate d6/d2c/d5e/fef 210087 0 2024-02-16T22:22:18.328 INFO:tasks.workunit.client.0.smithi114.stdout:6/821: symlink d6/d2c/d75/lf3 0 2024-02-16T22:22:18.331 INFO:tasks.workunit.client.0.smithi114.stdout:3/798: dwrite d4/dd/f19 [0,4194304] 0 2024-02-16T22:22:18.332 INFO:tasks.workunit.client.0.smithi114.stdout:6/822: getdents d6/d83 0 2024-02-16T22:22:18.335 INFO:tasks.workunit.client.0.smithi114.stdout:3/799: creat d4/d26/dd7/d104/d80/f114 x:0 0 0 2024-02-16T22:22:18.336 INFO:tasks.workunit.client.0.smithi114.stdout:3/800: chown d4/d24/def 31472188 1 2024-02-16T22:22:18.337 INFO:tasks.workunit.client.0.smithi114.stdout:6/823: mkdir d6/dd/d25/d34/d58/d9f/df4 0 2024-02-16T22:22:18.343 INFO:tasks.workunit.client.0.smithi114.stdout:3/801: creat d4/d24/f115 x:0 0 0 2024-02-16T22:22:18.344 INFO:tasks.workunit.client.0.smithi114.stdout:3/802: chown d4/dd/fcb 88897322 1 2024-02-16T22:22:18.347 INFO:tasks.workunit.client.0.smithi114.stdout:6/824: getdents d6/d2c/d38/d41/d60/d84 0 2024-02-16T22:22:18.347 INFO:tasks.workunit.client.0.smithi114.stdout:3/803: creat d4/d24/d2d/d45/d74/f116 x:0 0 0 2024-02-16T22:22:18.348 INFO:tasks.workunit.client.0.smithi114.stdout:6/825: mknod d6/dd/cf5 0 2024-02-16T22:22:18.350 INFO:tasks.workunit.client.0.smithi114.stdout:3/804: unlink d4/d26/dd7/c112 0 2024-02-16T22:22:18.352 INFO:tasks.workunit.client.0.smithi114.stdout:3/805: chown d4/d26/dd7/d104/d75/f9a 25 1 2024-02-16T22:22:18.354 INFO:tasks.workunit.client.0.smithi114.stdout:3/806: write d4/d24/f115 [933410,86016] 0 2024-02-16T22:22:18.356 INFO:tasks.workunit.client.0.smithi114.stdout:3/807: creat d4/d26/dd7/d104/dcc/f117 x:0 0 0 2024-02-16T22:22:18.358 INFO:tasks.workunit.client.0.smithi114.stdout:3/808: creat d4/dd/d33/d72/f118 x:0 0 0 2024-02-16T22:22:18.362 INFO:tasks.workunit.client.0.smithi114.stdout:6/826: write d6/d2c/d38/da8/ff0 [196379,87002] 0 2024-02-16T22:22:18.362 INFO:tasks.workunit.client.0.smithi114.stdout:3/809: symlink d4/d24/d2d/df0/l119 0 2024-02-16T22:22:18.364 INFO:tasks.workunit.client.0.smithi114.stdout:6/827: fdatasync d6/dd/d25/f53 0 2024-02-16T22:22:18.366 INFO:tasks.workunit.client.0.smithi114.stdout:3/810: rmdir d4/d24/d2d/d45/d74 39 2024-02-16T22:22:18.370 INFO:tasks.workunit.client.0.smithi114.stdout:6/828: rmdir d6/d2c/d38/d41/d60/d84/dc9 39 2024-02-16T22:22:18.370 INFO:tasks.workunit.client.0.smithi114.stdout:6/829: stat d6/dd/f29 0 2024-02-16T22:22:18.375 INFO:tasks.workunit.client.0.smithi114.stdout:6/830: mkdir d6/dd/d25/d34/d61/d62/d8b/db7/dd3/df6 0 2024-02-16T22:22:18.379 INFO:tasks.workunit.client.0.smithi114.stdout:6/831: mkdir d6/d2c/d38/d41/d60/df7 0 2024-02-16T22:22:18.380 INFO:tasks.workunit.client.0.smithi114.stdout:3/811: dwrite d4/d26/fc1 [4194304,4194304] 0 2024-02-16T22:22:18.380 INFO:tasks.workunit.client.0.smithi114.stdout:8/793: sync 2024-02-16T22:22:18.382 INFO:tasks.workunit.client.0.smithi114.stdout:3/812: dread - d4/d43/d105/fe6 zero size 2024-02-16T22:22:18.384 INFO:tasks.workunit.client.0.smithi114.stdout:3/813: write d4/d26/dd7/d104/dcc/f113 [445964,2688] 0 2024-02-16T22:22:18.385 INFO:tasks.workunit.client.0.smithi114.stdout:6/832: creat d6/d2c/d66/ff8 x:0 0 0 2024-02-16T22:22:18.386 INFO:tasks.workunit.client.0.smithi114.stdout:3/814: chown d4/d26/dd7/d104/d80/dbd/cd6 24698 1 2024-02-16T22:22:18.387 INFO:tasks.workunit.client.0.smithi114.stdout:6/833: write d6/dd/d25/f4c [299945,57008] 0 2024-02-16T22:22:18.388 INFO:tasks.workunit.client.0.smithi114.stdout:3/815: fsync d4/d24/f67 0 2024-02-16T22:22:18.388 INFO:tasks.workunit.client.0.smithi114.stdout:8/794: mknod d4/d24/d2c/d5f/dd1/c119 0 2024-02-16T22:22:18.392 INFO:tasks.workunit.client.0.smithi114.stdout:6/834: fdatasync d6/dd/d25/f27 0 2024-02-16T22:22:18.397 INFO:tasks.workunit.client.0.smithi114.stdout:3/816: creat d4/dd/d33/d8f/f11a x:0 0 0 2024-02-16T22:22:18.399 INFO:tasks.workunit.client.0.smithi114.stdout:6/835: dwrite d6/fbf [0,4194304] 0 2024-02-16T22:22:18.406 INFO:tasks.workunit.client.0.smithi114.stdout:3/817: creat d4/d24/d2d/d45/d74/dc7/f11b x:0 0 0 2024-02-16T22:22:18.407 INFO:tasks.workunit.client.0.smithi114.stdout:6/836: mkdir d6/d2c/df9 0 2024-02-16T22:22:18.409 INFO:tasks.workunit.client.0.smithi114.stdout:6/837: read d6/f35 [6928466,113241] 0 2024-02-16T22:22:18.409 INFO:tasks.workunit.client.0.smithi114.stdout:8/795: write d4/d11/d9d/fdd [7583218,32955] 0 2024-02-16T22:22:18.410 INFO:tasks.workunit.client.0.smithi114.stdout:6/838: readlink d6/d2c/d38/d41/l7e 0 2024-02-16T22:22:18.417 INFO:tasks.workunit.client.0.smithi114.stdout:3/818: creat d4/d24/d2d/d45/f11c x:0 0 0 2024-02-16T22:22:18.417 INFO:tasks.workunit.client.0.smithi114.stdout:6/839: mkdir d6/d2c/d38/d41/d60/d84/dfa 0 2024-02-16T22:22:18.418 INFO:tasks.workunit.client.0.smithi114.stdout:3/819: chown d4/d26/dd7/d104/l5d 495484 1 2024-02-16T22:22:18.418 INFO:tasks.workunit.client.0.smithi114.stdout:6/840: dread - d6/fec zero size 2024-02-16T22:22:18.419 INFO:tasks.workunit.client.0.smithi114.stdout:8/796: truncate d4/d24/f3d 4582259 0 2024-02-16T22:22:18.422 INFO:tasks.workunit.client.0.smithi114.stdout:6/841: rmdir d6/d2c/d75 39 2024-02-16T22:22:18.423 INFO:tasks.workunit.client.0.smithi114.stdout:8/797: truncate d4/d24/d2c/d5f/fe8 82584 0 2024-02-16T22:22:18.424 INFO:tasks.workunit.client.0.smithi114.stdout:8/798: chown d4/d16/d25/d39/f4f 23 1 2024-02-16T22:22:18.430 INFO:tasks.workunit.client.0.smithi114.stdout:3/820: link d4/d24/def/ca7 d4/d26/dd7/d104/d75/daa/c11d 0 2024-02-16T22:22:18.431 INFO:tasks.workunit.client.0.smithi114.stdout:9/829: sync 2024-02-16T22:22:18.433 INFO:tasks.workunit.client.0.smithi114.stdout:9/830: chown d2/d5c/dc9/d47/la4 74491 1 2024-02-16T22:22:18.436 INFO:tasks.workunit.client.0.smithi114.stdout:6/842: dread d6/f1e [0,4194304] 0 2024-02-16T22:22:18.438 INFO:tasks.workunit.client.0.smithi114.stdout:3/821: symlink d4/dd/d33/d72/d108/l11e 0 2024-02-16T22:22:18.441 INFO:tasks.workunit.client.0.smithi114.stdout:3/822: write d4/d26/dd7/d104/d80/dbd/fc8 [1429958,124995] 0 2024-02-16T22:22:18.442 INFO:tasks.workunit.client.0.smithi114.stdout:6/843: truncate d6/dd/d25/d34/d58/f78 4588704 0 2024-02-16T22:22:18.444 INFO:tasks.workunit.client.0.smithi114.stdout:8/799: write d4/faa [207414,47406] 0 2024-02-16T22:22:18.446 INFO:tasks.workunit.client.0.smithi114.stdout:3/823: mkdir d4/dd/d11f 0 2024-02-16T22:22:18.449 INFO:tasks.workunit.client.0.smithi114.stdout:9/831: dread d2/fb [0,4194304] 0 2024-02-16T22:22:18.457 INFO:tasks.workunit.client.0.smithi114.stdout:6/844: write d6/dd/d25/f99 [2396945,18671] 0 2024-02-16T22:22:18.458 INFO:tasks.workunit.client.0.smithi114.stdout:8/800: symlink d4/d11/l11a 0 2024-02-16T22:22:18.458 INFO:tasks.workunit.client.0.smithi114.stdout:8/801: readlink d4/d11/d9d/l106 0 2024-02-16T22:22:18.459 INFO:tasks.workunit.client.0.smithi114.stdout:9/832: stat d2/d1a/d65/f71 0 2024-02-16T22:22:18.461 INFO:tasks.workunit.client.0.smithi114.stdout:9/833: readlink d2/dd/d3c/d89/d9e/la0 0 2024-02-16T22:22:18.465 INFO:tasks.workunit.client.0.smithi114.stdout:6/845: rename d6/d2c/d38/da8/lab to d6/d2c/d5e/dcc/lfb 0 2024-02-16T22:22:18.466 INFO:tasks.workunit.client.0.smithi114.stdout:6/846: write d6/fbf [2738364,66826] 0 2024-02-16T22:22:18.467 INFO:tasks.workunit.client.0.smithi114.stdout:3/824: creat d4/d26/dd7/d104/f120 x:0 0 0 2024-02-16T22:22:18.470 INFO:tasks.workunit.client.0.smithi114.stdout:0/914: sync 2024-02-16T22:22:18.470 INFO:tasks.workunit.client.0.smithi114.stdout:7/913: sync 2024-02-16T22:22:18.470 INFO:tasks.workunit.client.0.smithi114.stdout:2/905: sync 2024-02-16T22:22:18.470 INFO:tasks.workunit.client.0.smithi114.stdout:5/804: sync 2024-02-16T22:22:18.470 INFO:tasks.workunit.client.0.smithi114.stdout:4/983: sync 2024-02-16T22:22:18.470 INFO:tasks.workunit.client.0.smithi114.stdout:1/826: sync 2024-02-16T22:22:18.473 INFO:tasks.workunit.client.0.smithi114.stdout:0/915: dread - d1/d1a/d47/d62/d89/dba/f112 zero size 2024-02-16T22:22:18.473 INFO:tasks.workunit.client.0.smithi114.stdout:4/984: dread - dd/d1b/d24/d57/d12b/d61/d13c/d144/f147 zero size 2024-02-16T22:22:18.480 INFO:tasks.workunit.client.0.smithi114.stdout:0/916: chown d1/d16/d1f/d28/d51/d61/d6d/fb6 55882 1 2024-02-16T22:22:18.483 INFO:tasks.workunit.client.0.smithi114.stdout:8/802: dwrite d4/d24/d2c/d5f/fa2 [0,4194304] 0 2024-02-16T22:22:18.487 INFO:tasks.workunit.client.0.smithi114.stdout:8/803: write d4/d16/d25/d46/d10b/fa4 [1091665,122854] 0 2024-02-16T22:22:18.487 INFO:tasks.workunit.client.0.smithi114.stdout:4/985: dwrite dd/d1b/d24/d57/d12b/f5d [0,4194304] 0 2024-02-16T22:22:18.493 INFO:tasks.workunit.client.0.smithi114.stdout:3/825: write d4/d26/f58 [273796,18661] 0 2024-02-16T22:22:18.500 INFO:tasks.workunit.client.0.smithi114.stdout:1/827: symlink d4/d9/l12c 0 2024-02-16T22:22:18.502 INFO:tasks.workunit.client.0.smithi114.stdout:0/917: mkdir d1/d1a/d47/d62/d7f/d9d/df2/d127 0 2024-02-16T22:22:18.504 INFO:tasks.workunit.client.0.smithi114.stdout:6/847: dwrite d6/dd/d25/d34/d58/f90 [0,4194304] 0 2024-02-16T22:22:18.505 INFO:tasks.workunit.client.0.smithi114.stdout:6/848: chown d6/d2c/d38/d54/f2d 1719228787 1 2024-02-16T22:22:18.505 INFO:tasks.workunit.client.0.smithi114.stdout:9/834: mkdir d2/d1a/d23/d32/d107 0 2024-02-16T22:22:18.509 INFO:tasks.workunit.client.0.smithi114.stdout:8/804: creat d4/d24/d6d/dc7/dfe/d10e/f11b x:0 0 0 2024-02-16T22:22:18.510 INFO:tasks.workunit.client.0.smithi114.stdout:3/826: symlink d4/dd/d33/d8f/l121 0 2024-02-16T22:22:18.511 INFO:tasks.workunit.client.0.smithi114.stdout:8/805: chown d4/d11/f51 3261 1 2024-02-16T22:22:18.515 INFO:tasks.workunit.client.0.smithi114.stdout:1/828: creat d4/d9/d46/d47/db3/f12d x:0 0 0 2024-02-16T22:22:18.516 INFO:tasks.workunit.client.0.smithi114.stdout:4/986: mknod dd/d1b/d24/d57/d12b/d38/d14e/c15c 0 2024-02-16T22:22:18.520 INFO:tasks.workunit.client.0.smithi114.stdout:3/827: unlink d4/d26/d4a/l81 0 2024-02-16T22:22:18.525 INFO:tasks.workunit.client.0.smithi114.stdout:6/849: rename d6/d2c/d38/l7a to d6/dd/d25/d34/d61/d62/d8b/db7/dd3/df6/lfc 0 2024-02-16T22:22:18.525 INFO:tasks.workunit.client.0.smithi114.stdout:2/906: dwrite dd/d3b/da7/d4c/fbe [0,4194304] 0 2024-02-16T22:22:18.528 INFO:tasks.workunit.client.0.smithi114.stdout:9/835: mknod d2/dd/d3c/d89/deb/c108 0 2024-02-16T22:22:18.531 INFO:tasks.workunit.client.0.smithi114.stdout:1/829: creat d4/d9/d3d/def/f12e x:0 0 0 2024-02-16T22:22:18.532 INFO:tasks.workunit.client.0.smithi114.stdout:5/805: dread d0/d28/d98/d25/d2e/f69 [0,4194304] 0 2024-02-16T22:22:18.532 INFO:tasks.workunit.client.0.smithi114.stdout:1/830: fdatasync d4/d63/d69/d7c/d9b/db0/f12a 0 2024-02-16T22:22:18.534 INFO:tasks.workunit.client.0.smithi114.stdout:8/806: symlink d4/d11/d9d/l11c 0 2024-02-16T22:22:18.535 INFO:tasks.workunit.client.0.smithi114.stdout:7/914: dwrite d3/d22/d73/da7/f62 [4194304,4194304] 0 2024-02-16T22:22:18.537 INFO:tasks.workunit.client.0.smithi114.stdout:0/918: getdents d1/d1a/d47/d62/d7f/d8c/d10c 0 2024-02-16T22:22:18.539 INFO:tasks.workunit.client.0.smithi114.stdout:2/907: creat dd/d3b/da7/d8b/dc0/f128 x:0 0 0 2024-02-16T22:22:18.543 INFO:tasks.workunit.client.0.smithi114.stdout:5/806: dwrite d0/d28/d98/d25/d3e/d7f/fa8 [0,4194304] 0 2024-02-16T22:22:18.546 INFO:tasks.workunit.client.0.smithi114.stdout:9/836: dread - d2/d5c/dc9/d1d/fd3 zero size 2024-02-16T22:22:18.547 INFO:tasks.workunit.client.0.smithi114.stdout:5/807: readlink d0/d28/d98/d25/d2e/d3b/db5/lf1 0 2024-02-16T22:22:18.548 INFO:tasks.workunit.client.0.smithi114.stdout:2/908: truncate dd/df/d1c/d21/d61/d69/de9/ff3 3613 0 2024-02-16T22:22:18.548 INFO:tasks.workunit.client.0.smithi114.stdout:1/831: truncate d4/d9/d3d/f39 3329548 0 2024-02-16T22:22:18.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.550+0000 7f1283815700 1 -- 172.21.15.114:0/4206487422 --> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f127c0611d0 con 0x7f126806d660 2024-02-16T22:22:18.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.552+0000 7f12727fc700 1 -- 172.21.15.114:0/4206487422 <== mgr.24441 v2:172.21.15.123:6828/1539029039 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7f127c0611d0 con 0x7f126806d660 2024-02-16T22:22:18.554 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:22:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:22:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: "in_progress": true, 2024-02-16T22:22:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:22:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: "services_complete": [], 2024-02-16T22:22:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: "progress": "", 2024-02-16T22:22:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: "message": "Currently upgrading mgr daemons", 2024-02-16T22:22:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: "is_paused": false 2024-02-16T22:22:18.554 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:22:18.556 INFO:tasks.workunit.client.0.smithi114.stdout:8/807: dread d4/d11/d78/fd8 [0,4194304] 0 2024-02-16T22:22:18.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.555+0000 7f1283815700 1 -- 172.21.15.114:0/4206487422 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f126806d660 msgr2=0x7f126806fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:18.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.555+0000 7f1283815700 1 --2- 172.21.15.114:0/4206487422 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f126806d660 0x7f126806fb20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1278009fd0 tx=0x7f1278005940 comp rx=0 tx=0).stop 2024-02-16T22:22:18.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.555+0000 7f1283815700 1 -- 172.21.15.114:0/4206487422 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f127c103f70 msgr2=0x7f127c196840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:18.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.555+0000 7f1283815700 1 --2- 172.21.15.114:0/4206487422 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f127c103f70 0x7f127c196840 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f126c00e970 tx=0x7f126c00ec80 comp rx=0 tx=0).stop 2024-02-16T22:22:18.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.555+0000 7f1283815700 1 -- 172.21.15.114:0/4206487422 shutdown_connections 2024-02-16T22:22:18.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.555+0000 7f1283815700 1 --2- 172.21.15.114:0/4206487422 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7f126806d660 0x7f126806fb20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:18.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.555+0000 7f1283815700 1 --2- 172.21.15.114:0/4206487422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f127c101640 0x7f127c196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:18.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.555+0000 7f1283815700 1 --2- 172.21.15.114:0/4206487422 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f127c103f70 0x7f127c196840 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:18.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.555+0000 7f1283815700 1 -- 172.21.15.114:0/4206487422 >> 172.21.15.114:0/4206487422 conn(0x7f127c0fb030 msgr2=0x7f127c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:22:18.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.555+0000 7f1283815700 1 -- 172.21.15.114:0/4206487422 shutdown_connections 2024-02-16T22:22:18.558 INFO:tasks.workunit.client.0.smithi114.stdout:8/808: chown d4/d11/d78/d7a/d36/c57 66159 1 2024-02-16T22:22:18.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.556+0000 7f1283815700 1 -- 172.21.15.114:0/4206487422 wait complete. 2024-02-16T22:22:18.559 INFO:tasks.workunit.client.0.smithi114.stdout:3/828: creat d4/d24/f122 x:0 0 0 2024-02-16T22:22:18.559 INFO:tasks.workunit.client.0.smithi114.stdout:6/850: write d6/dd/d25/f53 [427385,24406] 0 2024-02-16T22:22:18.559 INFO:tasks.workunit.client.0.smithi114.stdout:3/829: fdatasync d4/d24/f115 0 2024-02-16T22:22:18.561 INFO:tasks.workunit.client.0.smithi114.stdout:3/830: stat d4/d26/dd7/d109 0 2024-02-16T22:22:18.561 INFO:tasks.workunit.client.0.smithi114.stdout:9/837: write d2/d1a/d23/fbf [1999257,55450] 0 2024-02-16T22:22:18.564 INFO:tasks.workunit.client.0.smithi114.stdout:1/832: creat d4/d63/d69/d7c/d9b/df0/f12f x:0 0 0 2024-02-16T22:22:18.568 INFO:tasks.workunit.client.0.smithi114.stdout:6/851: mkdir d6/dd/d25/d34/d58/dfd 0 2024-02-16T22:22:18.569 INFO:tasks.workunit.client.0.smithi114.stdout:3/831: mknod d4/d26/d91/c123 0 2024-02-16T22:22:18.571 INFO:tasks.workunit.client.0.smithi114.stdout:5/808: chown d0/df/d6d/le4 55445 1 2024-02-16T22:22:18.575 INFO:tasks.workunit.client.0.smithi114.stdout:4/987: link dd/d2e/d56/ddc/dde/l119 dd/d59/d8f/l15d 0 2024-02-16T22:22:18.575 INFO:tasks.workunit.client.0.smithi114.stdout:0/919: link d1/d16/d1f/f4d d1/d1a/d47/d62/d7f/d9d/f128 0 2024-02-16T22:22:18.577 INFO:tasks.workunit.client.0.smithi114.stdout:7/915: dwrite d3/d22/d73/da7/fbc [4194304,4194304] 0 2024-02-16T22:22:18.579 INFO:tasks.workunit.client.0.smithi114.stdout:0/920: chown d1/d16/d1f/d28/d51/d61/d7e/d90/d95/d80/f106 28762838 1 2024-02-16T22:22:18.582 INFO:tasks.workunit.client.0.smithi114.stdout:7/916: write d3/d22/d73/da7/f68 [1703574,14940] 0 2024-02-16T22:22:18.585 INFO:tasks.workunit.client.0.smithi114.stdout:8/809: dread d4/d11/d9d/fe9 [0,4194304] 0 2024-02-16T22:22:18.586 INFO:tasks.workunit.client.0.smithi114.stdout:7/917: truncate d3/d22/d73/da7/d6f/df8/f128 4815679 0 2024-02-16T22:22:18.586 INFO:tasks.workunit.client.0.smithi114.stdout:8/810: readlink d4/d11/d78/d7a/d7e/lfb 0 2024-02-16T22:22:18.587 INFO:tasks.workunit.client.0.smithi114.stdout:1/833: fdatasync d4/d9/d3d/d3f/f5a 0 2024-02-16T22:22:18.590 INFO:tasks.workunit.client.0.smithi114.stdout:4/988: fsync dd/d1b/d41/d74/fbe 0 2024-02-16T22:22:18.591 INFO:tasks.workunit.client.0.smithi114.stdout:4/989: write dd/d59/dfa/f10d [1437360,108588] 0 2024-02-16T22:22:18.593 INFO:tasks.workunit.client.0.smithi114.stdout:4/990: write dd/d59/d8f/dea/f12d [903077,94138] 0 2024-02-16T22:22:18.594 INFO:tasks.workunit.client.0.smithi114.stdout:0/921: symlink d1/d16/d48/l129 0 2024-02-16T22:22:18.594 INFO:tasks.workunit.client.0.smithi114.stdout:4/991: chown dd/d2e/d3f/f75 13659 1 2024-02-16T22:22:18.595 INFO:tasks.workunit.client.0.smithi114.stdout:2/909: getdents dd/df/d1c/d21/d61/d69 0 2024-02-16T22:22:18.596 INFO:tasks.workunit.client.0.smithi114.stdout:9/838: link d2/d5c/dc9/d47/f6d d2/dd/d3c/f109 0 2024-02-16T22:22:18.596 INFO:tasks.workunit.client.0.smithi114.stdout:0/922: write d1/d16/d48/d98/f107 [501745,3690] 0 2024-02-16T22:22:18.598 INFO:tasks.workunit.client.0.smithi114.stdout:1/834: creat d4/d9/d46/d67/d92/dca/f130 x:0 0 0 2024-02-16T22:22:18.602 INFO:tasks.workunit.client.0.smithi114.stdout:8/811: rename d4/d24/d2c/d5f/l10d to d4/d24/d6d/dc7/dfe/d10e/l11d 0 2024-02-16T22:22:18.644 INFO:tasks.workunit.client.0.smithi114.stdout:9/839: mknod d2/dd0/c10a 0 2024-02-16T22:22:18.647 INFO:tasks.workunit.client.0.smithi114.stdout:2/910: symlink dd/d3b/d8a/dcb/l129 0 2024-02-16T22:22:18.650 INFO:tasks.workunit.client.0.smithi114.stdout:5/809: write d0/d28/d98/d1e/f60 [3270473,11236] 0 2024-02-16T22:22:18.654 INFO:tasks.workunit.client.0.smithi114.stdout:2/911: dwrite dd/df/d1c/d21/d87/f94 [0,4194304] 0 2024-02-16T22:22:18.654 INFO:tasks.workunit.client.0.smithi114.stdout:0/923: dread d1/d16/d1f/d28/d51/d61/d6d/fb6 [0,4194304] 0 2024-02-16T22:22:18.658 INFO:tasks.workunit.client.0.smithi114.stdout:1/835: chown d4/f25 279971801 1 2024-02-16T22:22:18.659 INFO:tasks.workunit.client.0.smithi114.stdout:8/812: creat d4/d11/d78/d7a/d36/d3b/f11e x:0 0 0 2024-02-16T22:22:18.661 INFO:tasks.workunit.client.0.smithi114.stdout:1/836: truncate d4/d63/d69/d7c/d9b/db0/f12a 937471 0 2024-02-16T22:22:18.669 INFO:tasks.workunit.client.0.smithi114.stdout:6/852: getdents d6/d2c/d5e/dcc 0 2024-02-16T22:22:18.670 INFO:tasks.workunit.client.0.smithi114.stdout:8/813: read d4/d11/d78/d7a/d36/fed [437084,19969] 0 2024-02-16T22:22:18.670 INFO:tasks.workunit.client.0.smithi114.stdout:3/832: getdents d4/d26/dd7/d104/d75 0 2024-02-16T22:22:18.670 INFO:tasks.workunit.client.0.smithi114.stdout:1/837: write d4/d9/d46/d67/ffd [1002843,91086] 0 2024-02-16T22:22:18.672 INFO:tasks.workunit.client.0.smithi114.stdout:7/918: truncate d3/d92/dbe/de6/f11c 342480 0 2024-02-16T22:22:18.690 INFO:tasks.workunit.client.0.smithi114.stdout:3/833: dread d4/d24/d2d/fed [4194304,4194304] 0 2024-02-16T22:22:18.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.694+0000 7fc494a17700 1 -- 172.21.15.114:0/2464300274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4900ffae0 msgr2=0x7fc4900fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:18.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.694+0000 7fc494a17700 1 --2- 172.21.15.114:0/2464300274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4900ffae0 0x7fc4900fff00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc480009910 tx=0x7fc480009c20 comp rx=0 tx=0).stop 2024-02-16T22:22:18.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.694+0000 7fc494a17700 1 -- 172.21.15.114:0/2464300274 shutdown_connections 2024-02-16T22:22:18.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.694+0000 7fc494a17700 1 --2- 172.21.15.114:0/2464300274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc490100ce0 0x7fc490101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:18.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.694+0000 7fc494a17700 1 --2- 172.21.15.114:0/2464300274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4900ffae0 0x7fc4900fff00 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:18.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.694+0000 7fc494a17700 1 -- 172.21.15.114:0/2464300274 >> 172.21.15.114:0/2464300274 conn(0x7fc4900fb080 msgr2=0x7fc4900fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:22:18.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.694+0000 7fc494a17700 1 -- 172.21.15.114:0/2464300274 shutdown_connections 2024-02-16T22:22:18.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.694+0000 7fc494a17700 1 -- 172.21.15.114:0/2464300274 wait complete. 2024-02-16T22:22:18.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.695+0000 7fc494a17700 1 Processor -- start 2024-02-16T22:22:18.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.695+0000 7fc494a17700 1 -- start start 2024-02-16T22:22:18.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.695+0000 7fc494a17700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4900ffae0 0x7fc490196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:18.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.695+0000 7fc494a17700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc490100ce0 0x7fc4901968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:18.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.695+0000 7fc494a17700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc490196ef0 con 0x7fc490100ce0 2024-02-16T22:22:18.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.695+0000 7fc494a17700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc490197030 con 0x7fc4900ffae0 2024-02-16T22:22:18.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.695+0000 7fc48dd9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc490100ce0 0x7fc4901968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:18.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.695+0000 7fc48dd9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc490100ce0 0x7fc4901968d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50200/0 (socket says 172.21.15.114:50200) 2024-02-16T22:22:18.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.695+0000 7fc48dd9b700 1 -- 172.21.15.114:0/2024227009 learned_addr learned my addr 172.21.15.114:0/2024227009 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:22:18.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.695+0000 7fc48e59c700 1 --2- 172.21.15.114:0/2024227009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4900ffae0 0x7fc490196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:18.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.695+0000 7fc48dd9b700 1 -- 172.21.15.114:0/2024227009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4900ffae0 msgr2=0x7fc490196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:18.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.695+0000 7fc48dd9b700 1 --2- 172.21.15.114:0/2024227009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4900ffae0 0x7fc490196390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:18.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.695+0000 7fc48dd9b700 1 -- 172.21.15.114:0/2024227009 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4800095f0 con 0x7fc490100ce0 2024-02-16T22:22:18.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.696+0000 7fc48dd9b700 1 --2- 172.21.15.114:0/2024227009 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc490100ce0 0x7fc4901968d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc48400e920 tx=0x7fc48400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:22:18.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.696+0000 7fc47f7fe700 1 -- 172.21.15.114:0/2024227009 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc48400c9d0 con 0x7fc490100ce0 2024-02-16T22:22:18.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.696+0000 7fc47f7fe700 1 -- 172.21.15.114:0/2024227009 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc484010430 con 0x7fc490100ce0 2024-02-16T22:22:18.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.696+0000 7fc494a17700 1 -- 172.21.15.114:0/2024227009 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc49019bae0 con 0x7fc490100ce0 2024-02-16T22:22:18.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.696+0000 7fc47f7fe700 1 -- 172.21.15.114:0/2024227009 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc484018650 con 0x7fc490100ce0 2024-02-16T22:22:18.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.696+0000 7fc494a17700 1 -- 172.21.15.114:0/2024227009 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc49019bfe0 con 0x7fc490100ce0 2024-02-16T22:22:18.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.697+0000 7fc494a17700 1 -- 172.21.15.114:0/2024227009 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc490108790 con 0x7fc490100ce0 2024-02-16T22:22:18.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.698+0000 7fc47f7fe700 1 -- 172.21.15.114:0/2024227009 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) v1 ==== 91450+0+0 (secure 0 0 0) 0x7fc4840105a0 con 0x7fc490100ce0 2024-02-16T22:22:18.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.698+0000 7fc47f7fe700 1 --2- 172.21.15.114:0/2024227009 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fc47806d660 0x7fc47806fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:18.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.698+0000 7fc47f7fe700 1 -- 172.21.15.114:0/2024227009 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5726+0+0 (secure 0 0 0) 0x7fc484014070 con 0x7fc490100ce0 2024-02-16T22:22:18.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.698+0000 7fc48e59c700 1 --2- 172.21.15.114:0/2024227009 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fc47806d660 0x7fc47806fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:18.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.699+0000 7fc48e59c700 1 --2- 172.21.15.114:0/2024227009 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fc47806d660 0x7fc47806fb20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc48000b6f0 tx=0x7fc480005940 comp rx=0 tx=0).ready entity=mgr.24441 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:22:18.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:18.702+0000 7fc47f7fe700 1 -- 172.21.15.114:0/2024227009 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182869 (secure 0 0 0) 0x7fc484052a70 con 0x7fc490100ce0 2024-02-16T22:22:18.724 INFO:tasks.workunit.client.0.smithi114.stdout:8/814: symlink d4/d11/d9d/ddf/l11f 0 2024-02-16T22:22:18.726 INFO:tasks.workunit.client.0.smithi114.stdout:0/924: symlink d1/d1a/d47/d62/l12a 0 2024-02-16T22:22:18.731 INFO:tasks.workunit.client.0.smithi114.stdout:3/834: mkdir d4/dd/d33/d72/d124 0 2024-02-16T22:22:18.734 INFO:tasks.workunit.client.0.smithi114.stdout:6/853: dread d6/d83/f87 [0,4194304] 0 2024-02-16T22:22:18.735 INFO:tasks.workunit.client.0.smithi114.stdout:6/854: stat d6/d2c/d38/d54/fad 0 2024-02-16T22:22:18.737 INFO:tasks.workunit.client.0.smithi114.stdout:7/919: unlink d3/d22/d73/da7/d6f/c137 0 2024-02-16T22:22:18.738 INFO:tasks.workunit.client.0.smithi114.stdout:0/925: symlink d1/d16/d1f/d28/d51/d61/d7e/d90/l12b 0 2024-02-16T22:22:18.740 INFO:tasks.workunit.client.0.smithi114.stdout:3/835: creat d4/d24/d2d/d45/d74/f125 x:0 0 0 2024-02-16T22:22:18.741 INFO:tasks.workunit.client.0.smithi114.stdout:8/815: symlink d4/d24/d2c/d5f/l120 0 2024-02-16T22:22:18.747 INFO:tasks.workunit.client.0.smithi114.stdout:3/836: fdatasync d4/dd/d33/d76/f7f 0 2024-02-16T22:22:18.748 INFO:tasks.workunit.client.0.smithi114.stdout:8/816: creat d4/d11/d78/d7a/d36/d3b/f121 x:0 0 0 2024-02-16T22:22:18.752 INFO:tasks.workunit.client.0.smithi114.stdout:6/855: rename d6/d2c/df9 to d6/d2c/d75/dfe 0 2024-02-16T22:22:18.759 INFO:tasks.workunit.client.0.smithi114.stdout:7/920: creat d3/d46/f141 x:0 0 0 2024-02-16T22:22:18.760 INFO:tasks.workunit.client.0.smithi114.stdout:7/921: read - d3/d22/f12a zero size 2024-02-16T22:22:18.765 INFO:tasks.workunit.client.0.smithi114.stdout:8/817: dread d4/d11/f51 [0,4194304] 0 2024-02-16T22:22:18.767 INFO:tasks.workunit.client.0.smithi114.stdout:7/922: dwrite d3/d46/da2/d114/f127 [0,4194304] 0 2024-02-16T22:22:18.776 INFO:tasks.workunit.client.0.smithi114.stdout:6/856: rename d6/la0 to d6/d2c/d38/d41/lff 0 2024-02-16T22:22:18.793 INFO:tasks.workunit.client.0.smithi114.stdout:7/923: getdents d3/d22/d73 0 2024-02-16T22:22:18.799 INFO:tasks.workunit.client.0.smithi114.stdout:7/924: mkdir d3/d22/d73/da7/d4e/d142 0 2024-02-16T22:22:18.801 INFO:tasks.workunit.client.0.smithi114.stdout:6/857: dread d6/d2c/f46 [0,4194304] 0 2024-02-16T22:22:18.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:18 smithi123 ceph-mon[37161]: from='client.24531 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:22:18.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:18 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/1483746963' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:22:18.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:18 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/3226124302' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:22:18.805 INFO:tasks.workunit.client.0.smithi114.stdout:9/840: dwrite d2/d5c/dc9/d1d/f62 [0,4194304] 0 2024-02-16T22:22:18.808 INFO:tasks.workunit.client.0.smithi114.stdout:6/858: creat d6/da/f100 x:0 0 0 2024-02-16T22:22:18.812 INFO:tasks.workunit.client.0.smithi114.stdout:9/841: fdatasync d2/dd0/fe0 0 2024-02-16T22:22:18.817 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:18 smithi114 ceph-mon[29235]: from='client.24531 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:22:18.817 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:18 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/1483746963' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:22:18.817 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:18 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/3226124302' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:22:18.821 INFO:tasks.workunit.client.0.smithi114.stdout:9/842: link d2/d5c/dc9/cb0 d2/d5c/c10b 0 2024-02-16T22:22:18.838 INFO:tasks.workunit.client.0.smithi114.stdout:9/843: dread d2/d5c/dc9/d47/d63/f81 [0,4194304] 0 2024-02-16T22:22:18.841 INFO:tasks.workunit.client.0.smithi114.stdout:9/844: mknod d2/d33/c10c 0 2024-02-16T22:22:18.843 INFO:tasks.workunit.client.0.smithi114.stdout:3/837: dwrite d4/d24/d2d/fac [0,4194304] 0 2024-02-16T22:22:18.850 INFO:tasks.workunit.client.0.smithi114.stdout:9/845: dread d2/dd/f3b [0,4194304] 0 2024-02-16T22:22:18.854 INFO:tasks.workunit.client.0.smithi114.stdout:3/838: mknod d4/d24/d2d/d45/c126 0 2024-02-16T22:22:18.858 INFO:tasks.workunit.client.0.smithi114.stdout:9/846: mknod d2/d5c/d5f/d7b/c10d 0 2024-02-16T22:22:18.860 INFO:tasks.workunit.client.0.smithi114.stdout:3/839: symlink d4/d7e/l127 0 2024-02-16T22:22:18.867 INFO:tasks.workunit.client.0.smithi114.stdout:8/818: dwrite d4/d11/f20 [0,4194304] 0 2024-02-16T22:22:18.869 INFO:tasks.workunit.client.0.smithi114.stdout:3/840: unlink d4/d24/d2d/d45/c49 0 2024-02-16T22:22:18.873 INFO:tasks.workunit.client.0.smithi114.stdout:9/847: truncate d2/d5c/dc9/d1d/f36 174849 0 2024-02-16T22:22:18.876 INFO:tasks.workunit.client.0.smithi114.stdout:8/819: rename d4/d16/cec to d4/d11/d78/d7a/d31/dc6/c122 0 2024-02-16T22:22:18.877 INFO:tasks.workunit.client.0.smithi114.stdout:9/848: read d2/f4 [3165328,52974] 0 2024-02-16T22:22:18.878 INFO:tasks.workunit.client.0.smithi114.stdout:9/849: stat d2/d5c/dc9/f75 0 2024-02-16T22:22:18.884 INFO:tasks.workunit.client.0.smithi114.stdout:9/850: mknod d2/d33/c10e 0 2024-02-16T22:22:18.892 INFO:tasks.workunit.client.0.smithi114.stdout:7/925: truncate d3/d5a/f8a 2644987 0 2024-02-16T22:22:18.895 INFO:tasks.workunit.client.0.smithi114.stdout:7/926: unlink d3/d87/cde 0 2024-02-16T22:22:18.898 INFO:tasks.workunit.client.0.smithi114.stdout:7/927: creat d3/d22/d61/f143 x:0 0 0 2024-02-16T22:22:18.898 INFO:tasks.workunit.client.0.smithi114.stdout:6/859: dwrite d6/dd/d25/d34/d61/d62/d8b/fbb [4194304,4194304] 0 2024-02-16T22:22:18.903 INFO:tasks.workunit.client.0.smithi114.stdout:7/928: creat d3/d22/d73/da7/db4/d12f/d13f/f144 x:0 0 0 2024-02-16T22:22:18.919 INFO:tasks.workunit.client.0.smithi114.stdout:9/851: write d2/d5c/d5f/fd1 [6390659,16663] 0 2024-02-16T22:22:18.921 INFO:tasks.workunit.client.0.smithi114.stdout:3/841: dwrite d4/d26/dd7/d104/f8e [4194304,4194304] 0 2024-02-16T22:22:18.924 INFO:tasks.workunit.client.0.smithi114.stdout:8/820: dwrite d4/d11/d52/f83 [4194304,4194304] 0 2024-02-16T22:22:18.935 INFO:tasks.workunit.client.0.smithi114.stdout:9/852: mkdir d2/d5c/dc9/d10f 0 2024-02-16T22:22:18.935 INFO:tasks.workunit.client.0.smithi114.stdout:7/929: dread d3/d22/d73/da7/f6b [0,4194304] 0 2024-02-16T22:22:18.939 INFO:tasks.workunit.client.0.smithi114.stdout:9/853: dread d2/d5c/dc9/d47/db2/fc3 [0,4194304] 0 2024-02-16T22:22:18.944 INFO:tasks.workunit.client.0.smithi114.stdout:7/930: unlink d3/d46/da2/d114/l11d 0 2024-02-16T22:22:18.946 INFO:tasks.workunit.client.0.smithi114.stdout:8/821: getdents d4/d11/d78 0 2024-02-16T22:22:18.946 INFO:tasks.workunit.client.0.smithi114.stdout:3/842: link d4/dd/fe9 d4/d24/d2d/d45/f128 0 2024-02-16T22:22:18.947 INFO:tasks.workunit.client.0.smithi114.stdout:3/843: fdatasync d4/d43/d105/f96 0 2024-02-16T22:22:18.949 INFO:tasks.workunit.client.0.smithi114.stdout:7/931: rename d3/d14/f82 to d3/d22/f145 0 2024-02-16T22:22:18.950 INFO:tasks.workunit.client.0.smithi114.stdout:7/932: write d3/d22/d73/da7/f5e [868329,10619] 0 2024-02-16T22:22:18.977 INFO:tasks.workunit.client.0.smithi114.stdout:8/822: mknod d4/d16/d25/d46/db5/c123 0 2024-02-16T22:22:18.978 INFO:tasks.workunit.client.0.smithi114.stdout:3/844: fsync d4/d24/d2d/fed 0 2024-02-16T22:22:18.980 INFO:tasks.workunit.client.0.smithi114.stdout:3/845: read - d4/d24/d2d/d45/d74/f125 zero size 2024-02-16T22:22:18.980 INFO:tasks.workunit.client.0.smithi114.stdout:7/933: rename d3/d22/d73/da7/d6f/d89 to d3/d22/d73/da7/db4/d146 0 2024-02-16T22:22:18.982 INFO:tasks.workunit.client.0.smithi114.stdout:9/854: link d2/dd/d3c/c8e d2/d1a/c110 0 2024-02-16T22:22:18.985 INFO:tasks.workunit.client.0.smithi114.stdout:8/823: write d4/d11/f1f [5034188,85688] 0 2024-02-16T22:22:18.989 INFO:tasks.workunit.client.0.smithi114.stdout:3/846: write d4/dd/d33/f5f [1180421,29731] 0 2024-02-16T22:22:18.990 INFO:tasks.workunit.client.0.smithi114.stdout:9/855: dread d2/d5c/d5f/d91/fae [0,4194304] 0 2024-02-16T22:22:18.994 INFO:tasks.workunit.client.0.smithi114.stdout:7/934: dwrite d3/d14/f5c [0,4194304] 0 2024-02-16T22:22:18.996 INFO:tasks.workunit.client.0.smithi114.stdout:8/824: unlink d4/d16/d25/d46/d10b/l105 0 2024-02-16T22:22:18.996 INFO:tasks.workunit.client.0.smithi114.stdout:9/856: dread - d2/dd/d3c/d89/d9e/fab zero size 2024-02-16T22:22:18.998 INFO:tasks.workunit.client.0.smithi114.stdout:7/935: mknod d3/d22/d73/c147 0 2024-02-16T22:22:19.000 INFO:tasks.workunit.client.0.smithi114.stdout:3/847: creat d4/d26/f129 x:0 0 0 2024-02-16T22:22:19.001 INFO:tasks.workunit.client.0.smithi114.stdout:7/936: chown d3/d22/d73/da7/db4/d146/da6/dfc/f120 1 1 2024-02-16T22:22:19.002 INFO:tasks.workunit.client.0.smithi114.stdout:9/857: truncate d2/d1a/d65/fe8 1339304 0 2024-02-16T22:22:19.003 INFO:tasks.workunit.client.0.smithi114.stdout:8/825: rename d4/d11/d78/d7a/d36/d3b/c97 to d4/d11/d9d/da8/c124 0 2024-02-16T22:22:19.004 INFO:tasks.workunit.client.0.smithi114.stdout:8/826: chown d4/d16/d25/d46/d10b/dcf/fd4 9902436 1 2024-02-16T22:22:19.006 INFO:tasks.workunit.client.0.smithi114.stdout:8/827: write d4/d11/f20 [1558434,129528] 0 2024-02-16T22:22:19.007 INFO:tasks.workunit.client.0.smithi114.stdout:9/858: truncate d2/d5c/dc9/d1d/f3a 344988 0 2024-02-16T22:22:19.014 INFO:tasks.workunit.client.0.smithi114.stdout:3/848: write d4/d26/d4a/f65 [850663,69642] 0 2024-02-16T22:22:19.015 INFO:tasks.workunit.client.0.smithi114.stdout:7/937: write d3/d22/d73/da7/f52 [1710922,30134] 0 2024-02-16T22:22:19.017 INFO:tasks.workunit.client.0.smithi114.stdout:8/828: creat d4/d11/d9d/f125 x:0 0 0 2024-02-16T22:22:19.020 INFO:tasks.workunit.client.0.smithi114.stdout:9/859: write d2/dd/f10 [2438266,54384] 0 2024-02-16T22:22:19.024 INFO:tasks.workunit.client.0.smithi114.stdout:3/849: mknod d4/d43/d105/c12a 0 2024-02-16T22:22:19.029 INFO:tasks.workunit.client.0.smithi114.stdout:8/829: chown d4/d16/d25/d46/le3 52139687 1 2024-02-16T22:22:19.029 INFO:tasks.workunit.client.0.smithi114.stdout:8/830: dread - d4/d24/d6d/ff2 zero size 2024-02-16T22:22:19.032 INFO:tasks.workunit.client.0.smithi114.stdout:9/860: mknod d2/dd/c111 0 2024-02-16T22:22:19.033 INFO:tasks.workunit.client.0.smithi114.stdout:3/850: dread d4/d24/def/f7c [0,4194304] 0 2024-02-16T22:22:19.034 INFO:tasks.workunit.client.0.smithi114.stdout:8/831: symlink d4/d16/d25/d46/l126 0 2024-02-16T22:22:19.037 INFO:tasks.workunit.client.0.smithi114.stdout:8/832: unlink d4/faa 0 2024-02-16T22:22:19.040 INFO:tasks.workunit.client.0.smithi114.stdout:3/851: rename d4/d26/dd7/d104/d80/dbd/fe3 to d4/d87/f12b 0 2024-02-16T22:22:19.041 INFO:tasks.workunit.client.0.smithi114.stdout:7/938: dread d3/d22/d76/fb0 [0,4194304] 0 2024-02-16T22:22:19.042 INFO:tasks.workunit.client.0.smithi114.stdout:8/833: truncate d4/d24/d2c/fd9 761186 0 2024-02-16T22:22:19.043 INFO:tasks.workunit.client.0.smithi114.stdout:9/861: getdents d2/dd/d3c/d89/deb 0 2024-02-16T22:22:19.044 INFO:tasks.workunit.client.0.smithi114.stdout:3/852: creat d4/d26/dd7/d104/d75/daa/f12c x:0 0 0 2024-02-16T22:22:19.046 INFO:tasks.workunit.client.0.smithi114.stdout:8/834: dread - d4/d16/d25/d46/fb7 zero size 2024-02-16T22:22:19.047 INFO:tasks.workunit.client.0.smithi114.stdout:8/835: chown d4/d11/d78/d7a/d31/f10a 552791655 1 2024-02-16T22:22:19.061 INFO:tasks.workunit.client.0.smithi114.stdout:7/939: dread d3/d22/d73/da7/db4/d146/da6/dfc/f102 [0,4194304] 0 2024-02-16T22:22:19.062 INFO:tasks.workunit.client.0.smithi114.stdout:3/853: truncate d4/d26/d4a/f65 5722931 0 2024-02-16T22:22:19.063 INFO:tasks.workunit.client.0.smithi114.stdout:3/854: chown d4/d26/d91/c123 22114924 1 2024-02-16T22:22:19.064 INFO:tasks.workunit.client.0.smithi114.stdout:7/940: creat d3/d87/dd1/f148 x:0 0 0 2024-02-16T22:22:19.064 INFO:tasks.workunit.client.0.smithi114.stdout:9/862: write d2/d5c/dc9/d1d/f3a [443582,115778] 0 2024-02-16T22:22:19.066 INFO:tasks.workunit.client.0.smithi114.stdout:3/855: creat d4/dd/d33/d72/f12d x:0 0 0 2024-02-16T22:22:19.066 INFO:tasks.workunit.client.0.smithi114.stdout:7/941: dread - d3/d22/d73/da7/d6f/d9c/fcf zero size 2024-02-16T22:22:19.068 INFO:tasks.workunit.client.0.smithi114.stdout:3/856: truncate d4/dd/d33/d72/f12d 102230 0 2024-02-16T22:22:19.069 INFO:tasks.workunit.client.0.smithi114.stdout:9/863: creat d2/dd/f112 x:0 0 0 2024-02-16T22:22:19.069 INFO:tasks.workunit.client.0.smithi114.stdout:3/857: write d4/d26/f39 [467112,61998] 0 2024-02-16T22:22:19.072 INFO:tasks.workunit.client.0.smithi114.stdout:7/942: rename d3/d46/da2/cad to d3/d22/d73/da7/db4/d12f/c149 0 2024-02-16T22:22:19.073 INFO:tasks.workunit.client.0.smithi114.stdout:9/864: symlink d2/d33/d3f/l113 0 2024-02-16T22:22:19.078 INFO:tasks.workunit.client.0.smithi114.stdout:9/865: mknod d2/d5c/dc9/d1d/c114 0 2024-02-16T22:22:19.087 INFO:tasks.workunit.client.0.smithi114.stdout:7/943: dread d3/d22/d73/da7/d6f/fa8 [0,4194304] 0 2024-02-16T22:22:19.090 INFO:tasks.workunit.client.0.smithi114.stdout:9/866: getdents d2/d5c/d5f/d72/dd6 0 2024-02-16T22:22:19.099 INFO:tasks.workunit.client.0.smithi114.stdout:3/858: dwrite d4/d26/dd7/d104/fd2 [8388608,4194304] 0 2024-02-16T22:22:19.103 INFO:tasks.workunit.client.0.smithi114.stdout:3/859: unlink d4/d26/dd7/d104/d75/c95 0 2024-02-16T22:22:19.104 INFO:tasks.workunit.client.0.smithi114.stdout:3/860: dread - d4/dd/d33/d72/f118 zero size 2024-02-16T22:22:19.106 INFO:tasks.workunit.client.0.smithi114.stdout:9/867: getdents d2/dd/d3c/d89/d8b 0 2024-02-16T22:22:19.110 INFO:tasks.workunit.client.0.smithi114.stdout:3/861: creat d4/d46/dcf/f12e x:0 0 0 2024-02-16T22:22:19.113 INFO:tasks.workunit.client.0.smithi114.stdout:7/944: dwrite d3/d22/d73/da7/d6f/d9c/fed [0,4194304] 0 2024-02-16T22:22:19.117 INFO:tasks.workunit.client.0.smithi114.stdout:3/862: creat d4/dd/d33/d72/d108/f12f x:0 0 0 2024-02-16T22:22:19.118 INFO:tasks.workunit.client.0.smithi114.stdout:7/945: mkdir d3/d92/dbe/de6/d14a 0 2024-02-16T22:22:19.121 INFO:tasks.workunit.client.0.smithi114.stdout:3/863: fdatasync d4/d24/d2d/df0/ff1 0 2024-02-16T22:22:19.125 INFO:tasks.workunit.client.0.smithi114.stdout:7/946: mknod d3/d22/d73/da7/db4/d12f/c14b 0 2024-02-16T22:22:19.126 INFO:tasks.workunit.client.0.smithi114.stdout:9/868: dwrite d2/d5c/dc9/d1d/f69 [0,4194304] 0 2024-02-16T22:22:19.130 INFO:tasks.workunit.client.0.smithi114.stdout:3/864: creat d4/dd/d33/f130 x:0 0 0 2024-02-16T22:22:19.136 INFO:tasks.workunit.client.0.smithi114.stdout:9/869: creat d2/d1a/d23/d32/d46/f115 x:0 0 0 2024-02-16T22:22:19.138 INFO:tasks.workunit.client.0.smithi114.stdout:3/865: mkdir d4/d24/d2d/d45/d74/dc7/d131 0 2024-02-16T22:22:19.140 INFO:tasks.workunit.client.0.smithi114.stdout:7/947: write d3/d22/d73/da7/db4/d146/fb3 [2943635,80909] 0 2024-02-16T22:22:19.141 INFO:tasks.workunit.client.0.smithi114.stdout:9/870: dread - d2/d1a/d23/d32/d46/fc0 zero size 2024-02-16T22:22:19.142 INFO:tasks.workunit.client.0.smithi114.stdout:3/866: truncate d4/d87/faf 1374719 0 2024-02-16T22:22:19.142 INFO:tasks.workunit.client.0.smithi114.stdout:7/948: chown d3/d22/d73/c147 695 1 2024-02-16T22:22:19.145 INFO:tasks.workunit.client.0.smithi114.stdout:7/949: mknod d3/d22/d73/da7/c14c 0 2024-02-16T22:22:19.147 INFO:tasks.workunit.client.0.smithi114.stdout:3/867: mkdir d4/d24/d2d/d45/d132 0 2024-02-16T22:22:19.150 INFO:tasks.workunit.client.0.smithi114.stdout:7/950: creat d3/d92/dbe/f14d x:0 0 0 2024-02-16T22:22:19.151 INFO:tasks.workunit.client.0.smithi114.stdout:3/868: chown d4/d24/l3d 108005592 1 2024-02-16T22:22:19.153 INFO:tasks.workunit.client.0.smithi114.stdout:9/871: link d2/d5c/d5f/d7b/c3e d2/dd/d3c/d89/d8b/c116 0 2024-02-16T22:22:19.158 INFO:tasks.workunit.client.0.smithi114.stdout:3/869: dwrite d4/d26/f58 [4194304,4194304] 0 2024-02-16T22:22:19.159 INFO:tasks.workunit.client.0.smithi114.stdout:3/870: chown d4/d24/d2d/d45/d74 4730840 1 2024-02-16T22:22:19.164 INFO:tasks.workunit.client.0.smithi114.stdout:7/951: truncate d3/d14/fc4 131069 0 2024-02-16T22:22:19.169 INFO:tasks.workunit.client.0.smithi114.stdout:3/871: dread d4/dd/d33/f5f [0,4194304] 0 2024-02-16T22:22:19.169 INFO:tasks.workunit.client.0.smithi114.stdout:9/872: creat d2/d5c/dc9/d47/d63/f117 x:0 0 0 2024-02-16T22:22:19.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:19.168+0000 7fc494a17700 1 -- 172.21.15.114:0/2024227009 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc49019bc70 con 0x7fc490100ce0 2024-02-16T22:22:19.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:19.168+0000 7fc47f7fe700 1 -- 172.21.15.114:0/2024227009 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fc49019bc70 con 0x7fc490100ce0 2024-02-16T22:22:19.171 INFO:teuthology.orchestra.run.smithi114.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:22:19.171 INFO:teuthology.orchestra.run.smithi114.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:22:19.171 INFO:teuthology.orchestra.run.smithi114.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:22:19.172 INFO:tasks.workunit.client.0.smithi114.stdout:3/872: creat d4/dd/d33/f133 x:0 0 0 2024-02-16T22:22:19.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:19.171+0000 7fc494a17700 1 -- 172.21.15.114:0/2024227009 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fc47806d660 msgr2=0x7fc47806fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:19.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:19.171+0000 7fc494a17700 1 --2- 172.21.15.114:0/2024227009 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fc47806d660 0x7fc47806fb20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc48000b6f0 tx=0x7fc480005940 comp rx=0 tx=0).stop 2024-02-16T22:22:19.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:19.171+0000 7fc494a17700 1 -- 172.21.15.114:0/2024227009 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc490100ce0 msgr2=0x7fc4901968d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:19.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:19.171+0000 7fc494a17700 1 --2- 172.21.15.114:0/2024227009 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc490100ce0 0x7fc4901968d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc48400e920 tx=0x7fc48400ec30 comp rx=0 tx=0).stop 2024-02-16T22:22:19.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:19.171+0000 7fc494a17700 1 -- 172.21.15.114:0/2024227009 shutdown_connections 2024-02-16T22:22:19.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:19.171+0000 7fc494a17700 1 --2- 172.21.15.114:0/2024227009 >> [v2:172.21.15.123:6828/1539029039,v1:172.21.15.123:6829/1539029039] conn(0x7fc47806d660 0x7fc47806fb20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:19.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:19.171+0000 7fc494a17700 1 --2- 172.21.15.114:0/2024227009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4900ffae0 0x7fc490196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:19.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:19.171+0000 7fc494a17700 1 --2- 172.21.15.114:0/2024227009 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc490100ce0 0x7fc4901968d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:19.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:19.171+0000 7fc494a17700 1 -- 172.21.15.114:0/2024227009 >> 172.21.15.114:0/2024227009 conn(0x7fc4900fb080 msgr2=0x7fc490068dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:22:19.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:19.171+0000 7fc494a17700 1 -- 172.21.15.114:0/2024227009 shutdown_connections 2024-02-16T22:22:19.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:19.171+0000 7fc494a17700 1 -- 172.21.15.114:0/2024227009 wait complete. 2024-02-16T22:22:19.176 INFO:tasks.workunit.client.0.smithi114.stdout:9/873: symlink d2/l118 0 2024-02-16T22:22:19.176 INFO:tasks.workunit.client.0.smithi114.stdout:9/874: stat d2/dd0/cd5 0 2024-02-16T22:22:19.181 INFO:tasks.workunit.client.0.smithi114.stdout:9/875: dwrite d2/d5c/d5f/fd1 [8388608,4194304] 0 2024-02-16T22:22:19.188 INFO:tasks.workunit.client.0.smithi114.stdout:9/876: creat d2/dd/d3c/d89/d8b/df2/f119 x:0 0 0 2024-02-16T22:22:19.191 INFO:tasks.workunit.client.0.smithi114.stdout:3/873: dwrite d4/dd/d33/d8f/fa4 [0,4194304] 0 2024-02-16T22:22:19.193 INFO:tasks.workunit.client.0.smithi114.stdout:9/877: symlink d2/d5c/dc9/d47/d63/d98/l11a 0 2024-02-16T22:22:19.193 INFO:tasks.workunit.client.0.smithi114.stdout:9/878: dread - d2/dd/fcc zero size 2024-02-16T22:22:19.197 INFO:tasks.workunit.client.0.smithi114.stdout:3/874: unlink d4/d46/dcf/le2 0 2024-02-16T22:22:19.200 INFO:tasks.workunit.client.0.smithi114.stdout:3/875: unlink d4/d24/def/c6f 0 2024-02-16T22:22:19.200 INFO:tasks.workunit.client.0.smithi114.stdout:9/879: dwrite d2/d5c/d5f/d91/de4/fed [0,4194304] 0 2024-02-16T22:22:19.204 INFO:tasks.workunit.client.0.smithi114.stdout:9/880: mkdir d2/d5c/d5f/d91/de4/d11b 0 2024-02-16T22:22:19.205 INFO:tasks.workunit.client.0.smithi114.stdout:3/876: rename d4/d26/dd7/d104/la3 to d4/d26/d4a/l134 0 2024-02-16T22:22:19.206 INFO:tasks.workunit.client.0.smithi114.stdout:9/881: symlink d2/dd/d3c/d89/d8b/df2/l11c 0 2024-02-16T22:22:19.207 INFO:tasks.workunit.client.0.smithi114.stdout:3/877: truncate d4/dd/d33/d72/f9b 4595682 0 2024-02-16T22:22:19.207 INFO:tasks.workunit.client.0.smithi114.stdout:9/882: stat d2/dd/fcc 0 2024-02-16T22:22:19.209 INFO:tasks.workunit.client.0.smithi114.stdout:9/883: creat d2/d5c/dc9/d47/f11d x:0 0 0 2024-02-16T22:22:19.211 INFO:tasks.workunit.client.0.smithi114.stdout:3/878: truncate d4/dd/f19 3316175 0 2024-02-16T22:22:19.222 INFO:tasks.workunit.client.0.smithi114.stdout:9/884: rename d2/d5c/d5f/d72/c9c to d2/d5c/dc9/d1d/c11e 0 2024-02-16T22:22:19.223 INFO:tasks.workunit.client.0.smithi114.stdout:9/885: fdatasync d2/d1a/d23/fbf 0 2024-02-16T22:22:19.226 INFO:tasks.workunit.client.0.smithi114.stdout:3/879: write d4/d87/fc9 [60118,100845] 0 2024-02-16T22:22:19.227 INFO:tasks.workunit.client.0.smithi114.stdout:3/880: stat d4/d24/d2d/df0 0 2024-02-16T22:22:19.235 INFO:tasks.workunit.client.0.smithi114.stdout:9/886: write d2/d1a/d65/f8f [4532109,112040] 0 2024-02-16T22:22:19.237 INFO:tasks.workunit.client.0.smithi114.stdout:9/887: truncate d2/d5c/d5f/d72/fb8 2289398 0 2024-02-16T22:22:19.238 INFO:tasks.workunit.client.0.smithi114.stdout:9/888: write d2/d5c/dc9/d1d/f3a [1375226,33195] 0 2024-02-16T22:22:19.241 INFO:tasks.workunit.client.0.smithi114.stdout:9/889: rename d2/d1a/d23/d32/fd2 to d2/d5c/d5f/d7b/f11f 0 2024-02-16T22:22:19.243 INFO:tasks.workunit.client.0.smithi114.stdout:9/890: unlink d2/d5c/d5f/fd9 0 2024-02-16T22:22:19.245 INFO:tasks.workunit.client.0.smithi114.stdout:3/881: dread d4/d26/fc1 [0,4194304] 0 2024-02-16T22:22:19.252 INFO:tasks.workunit.client.0.smithi114.stdout:3/882: dwrite d4/d24/d2d/d45/f11c [0,4194304] 0 2024-02-16T22:22:19.253 INFO:tasks.workunit.client.0.smithi114.stdout:3/883: dread - d4/d26/dd7/d104/d75/daa/f12c zero size 2024-02-16T22:22:19.264 INFO:tasks.workunit.client.0.smithi114.stdout:9/891: dread d2/dd/f34 [0,4194304] 0 2024-02-16T22:22:19.267 INFO:tasks.workunit.client.0.smithi114.stdout:3/884: dread d4/d26/f39 [0,4194304] 0 2024-02-16T22:22:19.270 INFO:tasks.workunit.client.0.smithi114.stdout:9/892: dwrite d2/d5c/d5f/fd1 [4194304,4194304] 0 2024-02-16T22:22:19.272 INFO:tasks.workunit.client.0.smithi114.stdout:3/885: mkdir d4/d43/d105/d135 0 2024-02-16T22:22:19.275 INFO:tasks.workunit.client.0.smithi114.stdout:9/893: symlink d2/d5c/dc9/d47/db2/l120 0 2024-02-16T22:22:19.279 INFO:tasks.workunit.client.0.smithi114.stdout:3/886: read d4/d87/faf [235181,91380] 0 2024-02-16T22:22:19.289 INFO:tasks.workunit.client.0.smithi114.stdout:9/894: link d2/d5c/f8d d2/d5c/dc9/f121 0 2024-02-16T22:22:19.294 INFO:tasks.workunit.client.0.smithi114.stdout:3/887: dwrite d4/d24/d2d/df0/ff1 [0,4194304] 0 2024-02-16T22:22:19.297 INFO:tasks.workunit.client.0.smithi114.stdout:9/895: dwrite d2/d1a/d23/d32/d46/fb6 [0,4194304] 0 2024-02-16T22:22:19.299 INFO:tasks.workunit.client.0.smithi114.stdout:9/896: chown d2/d5c/dc9/d1d/f106 195 1 2024-02-16T22:22:19.302 INFO:tasks.workunit.client.0.smithi114.stdout:3/888: truncate d4/d43/fce 605739 0 2024-02-16T22:22:19.305 INFO:tasks.workunit.client.0.smithi114.stdout:3/889: mkdir d4/d24/def/d136 0 2024-02-16T22:22:19.309 INFO:tasks.workunit.client.0.smithi114.stdout:9/897: write d2/d5c/d5f/d7b/f64 [3774659,127046] 0 2024-02-16T22:22:19.310 INFO:tasks.workunit.client.0.smithi114.stdout:3/890: dwrite d4/fd8 [0,4194304] 0 2024-02-16T22:22:19.312 INFO:tasks.workunit.client.0.smithi114.stdout:3/891: truncate d4/dd/d33/d72/d108/f12f 540855 0 2024-02-16T22:22:19.313 INFO:tasks.workunit.client.0.smithi114.stdout:3/892: write d4/d24/fd4 [1011007,86568] 0 2024-02-16T22:22:19.314 INFO:tasks.workunit.client.0.smithi114.stdout:9/898: creat d2/d5c/d5f/d7b/f122 x:0 0 0 2024-02-16T22:22:19.317 INFO:tasks.workunit.client.0.smithi114.stdout:3/893: fsync d4/d26/fc1 0 2024-02-16T22:22:19.318 INFO:tasks.workunit.client.0.smithi114.stdout:9/899: mkdir d2/d5c/dc9/d47/d63/dbb/d123 0 2024-02-16T22:22:19.319 INFO:tasks.workunit.client.0.smithi114.stdout:3/894: write d4/d46/dcf/f12e [364406,116074] 0 2024-02-16T22:22:19.326 INFO:tasks.workunit.client.0.smithi114.stdout:9/900: rename d2/d5c/d5f/d7b/f11f to d2/d5c/d5f/d72/dd6/f124 0 2024-02-16T22:22:19.326 INFO:tasks.workunit.client.0.smithi114.stdout:3/895: read d4/d26/f31 [2183567,19557] 0 2024-02-16T22:22:19.330 INFO:tasks.workunit.client.0.smithi114.stdout:9/901: mkdir d2/d5c/d5f/d7b/d125 0 2024-02-16T22:22:19.335 INFO:tasks.workunit.client.0.smithi114.stdout:3/896: write d4/d87/faf [1763566,70442] 0 2024-02-16T22:22:19.418 INFO:tasks.workunit.client.0.smithi114.stdout:9/902: getdents d2/dd/d3c/d89/d8b 0 2024-02-16T22:22:19.419 INFO:tasks.workunit.client.0.smithi114.stdout:9/903: fdatasync d2/d1a/d23/d32/d46/fc0 0 2024-02-16T22:22:19.424 INFO:tasks.workunit.client.0.smithi114.stdout:9/904: getdents d2/d5c/dc9/d47/d63/dbb 0 2024-02-16T22:22:19.430 INFO:tasks.workunit.client.0.smithi114.stdout:9/905: write d2/d5c/dc9/d47/f6d [1266639,81036] 0 2024-02-16T22:22:19.431 INFO:tasks.workunit.client.0.smithi114.stdout:9/906: readlink d2/d5c/dc9/d47/d63/dbb/lbe 0 2024-02-16T22:22:19.437 INFO:tasks.workunit.client.0.smithi114.stdout:9/907: write d2/d1a/d65/f71 [171618,59736] 0 2024-02-16T22:22:19.443 INFO:tasks.workunit.client.0.smithi114.stdout:9/908: dwrite d2/dd0/f101 [0,4194304] 0 2024-02-16T22:22:19.446 INFO:tasks.workunit.client.0.smithi114.stdout:9/909: dread - d2/d5c/dc9/d47/d63/d98/fcb zero size 2024-02-16T22:22:19.447 INFO:tasks.workunit.client.0.smithi114.stdout:9/910: read d2/d1a/d65/f94 [508366,38326] 0 2024-02-16T22:22:19.448 INFO:tasks.workunit.client.0.smithi114.stdout:9/911: stat d2/dd/d3c/d89/d9e/la0 0 2024-02-16T22:22:19.451 INFO:tasks.workunit.client.0.smithi114.stdout:9/912: symlink d2/d5c/dc9/l126 0 2024-02-16T22:22:19.452 INFO:tasks.workunit.client.0.smithi114.stdout:9/913: write d2/d5c/d5f/d7b/ff5 [879048,2732] 0 2024-02-16T22:22:19.454 INFO:tasks.workunit.client.0.smithi114.stdout:9/914: creat d2/d5c/ddf/f127 x:0 0 0 2024-02-16T22:22:19.466 INFO:tasks.workunit.client.0.smithi114.stdout:9/915: write d2/d1a/d23/d32/d46/d78/fcd [1122668,89359] 0 2024-02-16T22:22:19.467 INFO:tasks.workunit.client.0.smithi114.stdout:9/916: readlink d2/d5c/dc9/d47/l96 0 2024-02-16T22:22:19.468 INFO:tasks.workunit.client.0.smithi114.stdout:9/917: symlink d2/dd/d3c/l128 0 2024-02-16T22:22:19.472 INFO:tasks.workunit.client.0.smithi114.stdout:9/918: getdents d2/dd0 0 2024-02-16T22:22:19.478 INFO:tasks.workunit.client.0.smithi114.stdout:9/919: creat d2/d1a/d65/f129 x:0 0 0 2024-02-16T22:22:19.479 INFO:tasks.workunit.client.0.smithi114.stdout:9/920: readlink d2/ldb 0 2024-02-16T22:22:19.490 INFO:tasks.workunit.client.0.smithi114.stdout:9/921: link d2/dd/d3c/lfb d2/dd/l12a 0 2024-02-16T22:22:19.494 INFO:tasks.workunit.client.0.smithi114.stdout:9/922: dread d2/d1a/d23/d32/d46/fb6 [0,4194304] 0 2024-02-16T22:22:19.496 INFO:tasks.workunit.client.0.smithi114.stdout:9/923: mknod d2/c12b 0 2024-02-16T22:22:19.500 INFO:tasks.workunit.client.0.smithi114.stdout:9/924: rename d2/dd/d3c/d89/deb to d2/d5c/d5f/d72/dd6/d12c 0 2024-02-16T22:22:19.505 INFO:tasks.workunit.client.0.smithi114.stdout:9/925: link d2/dd/d3c/d89/d8b/df2/l11c d2/d5c/ddf/l12d 0 2024-02-16T22:22:19.509 INFO:tasks.workunit.client.0.smithi114.stdout:9/926: rename d2/d5c/d5f/d7b/f64 to d2/d1a/d23/d32/d46/f12e 0 2024-02-16T22:22:19.511 INFO:tasks.workunit.client.0.smithi114.stdout:9/927: creat d2/d5c/dc9/d47/f12f x:0 0 0 2024-02-16T22:22:19.512 INFO:tasks.workunit.client.0.smithi114.stdout:9/928: write d2/d1a/f6b [2887733,19771] 0 2024-02-16T22:22:19.513 INFO:tasks.workunit.client.0.smithi114.stdout:9/929: creat d2/f130 x:0 0 0 2024-02-16T22:22:19.516 INFO:tasks.workunit.client.0.smithi114.stdout:9/930: creat d2/d5c/dc9/d47/d63/f131 x:0 0 0 2024-02-16T22:22:19.518 INFO:tasks.workunit.client.0.smithi114.stdout:4/992: sync 2024-02-16T22:22:19.525 INFO:tasks.workunit.client.0.smithi114.stdout:4/993: dwrite dd/d1b/d41/f145 [0,4194304] 0 2024-02-16T22:22:19.532 INFO:tasks.workunit.client.0.smithi114.stdout:9/931: dwrite d2/dd/d3c/d89/d8b/f95 [0,4194304] 0 2024-02-16T22:22:19.538 INFO:tasks.workunit.client.0.smithi114.stdout:9/932: mkdir d2/dd/d132 0 2024-02-16T22:22:19.558 INFO:tasks.workunit.client.0.smithi114.stdout:9/933: dread d2/d5c/dc9/ffc [0,4194304] 0 2024-02-16T22:22:19.561 INFO:tasks.workunit.client.0.smithi114.stdout:9/934: dread d2/d5c/dc9/d1d/f69 [0,4194304] 0 2024-02-16T22:22:19.565 INFO:tasks.workunit.client.0.smithi114.stdout:9/935: truncate d2/dd/d3c/d89/d9e/fab 745159 0 2024-02-16T22:22:19.568 INFO:tasks.workunit.client.0.smithi114.stdout:9/936: creat d2/dd/d132/f133 x:0 0 0 2024-02-16T22:22:19.584 INFO:tasks.workunit.client.0.smithi114.stdout:9/937: dread d2/d5c/dc9/d47/d63/fa8 [0,4194304] 0 2024-02-16T22:22:19.618 INFO:tasks.workunit.client.0.smithi114.stdout:5/810: sync 2024-02-16T22:22:19.620 INFO:tasks.workunit.client.0.smithi114.stdout:5/811: write d0/d28/d98/d1e/d86/f92 [4791439,94435] 0 2024-02-16T22:22:19.624 INFO:tasks.workunit.client.0.smithi114.stdout:5/812: creat d0/df/d6d/f108 x:0 0 0 2024-02-16T22:22:19.625 INFO:tasks.workunit.client.0.smithi114.stdout:5/813: creat d0/d28/d98/d25/d3e/f109 x:0 0 0 2024-02-16T22:22:19.626 INFO:tasks.workunit.client.0.smithi114.stdout:9/938: link d2/d5c/dc9/d47/f4d d2/d1a/d23/d32/d46/d78/f134 0 2024-02-16T22:22:19.627 INFO:tasks.workunit.client.0.smithi114.stdout:5/814: dread - d0/d28/d98/d25/d89/d99/db0/ffe zero size 2024-02-16T22:22:19.629 INFO:tasks.workunit.client.0.smithi114.stdout:5/815: truncate d0/d28/d98/d25/d2e/d3b/ffb 315900 0 2024-02-16T22:22:19.630 INFO:tasks.workunit.client.0.smithi114.stdout:9/939: symlink d2/d5c/dc9/d1d/l135 0 2024-02-16T22:22:19.631 INFO:tasks.workunit.client.0.smithi114.stdout:9/940: write d2/d5c/dc9/f75 [2460182,102547] 0 2024-02-16T22:22:19.632 INFO:tasks.workunit.client.0.smithi114.stdout:9/941: chown d2 258897 1 2024-02-16T22:22:19.635 INFO:tasks.workunit.client.0.smithi114.stdout:5/816: rename d0/df/d53/db1/ce0 to d0/d28/d98/d25/d3e/c10a 0 2024-02-16T22:22:19.636 INFO:tasks.workunit.client.0.smithi114.stdout:9/942: dread d2/d5c/d5f/d91/de4/fed [0,4194304] 0 2024-02-16T22:22:19.648 INFO:tasks.workunit.client.0.smithi114.stdout:9/943: rmdir d2/d5c/dc9/d47/d63/dbb/d123 0 2024-02-16T22:22:19.652 INFO:tasks.workunit.client.0.smithi114.stdout:5/817: dwrite d0/f81 [4194304,4194304] 0 2024-02-16T22:22:19.656 INFO:tasks.workunit.client.0.smithi114.stdout:9/944: creat d2/d1a/d23/d32/d46/f136 x:0 0 0 2024-02-16T22:22:19.659 INFO:tasks.workunit.client.0.smithi114.stdout:5/818: truncate d0/d28/d98/d25/d89/d99/db0/fec 3785517 0 2024-02-16T22:22:19.662 INFO:tasks.workunit.client.0.smithi114.stdout:9/945: symlink d2/d5c/dc9/d10f/l137 0 2024-02-16T22:22:19.662 INFO:tasks.workunit.client.0.smithi114.stdout:9/946: fdatasync d2/dd/f100 0 2024-02-16T22:22:19.663 INFO:tasks.workunit.client.0.smithi114.stdout:9/947: chown d2/dd/ff 0 1 2024-02-16T22:22:19.666 INFO:tasks.workunit.client.0.smithi114.stdout:5/819: rename d0/d28/d98/d25/d2e/d3b/d8d/cee to d0/d28/d98/d1e/dd7/c10b 0 2024-02-16T22:22:19.668 INFO:tasks.workunit.client.0.smithi114.stdout:9/948: rename d2/d33/d3f/fe1 to d2/d1a/d23/d32/d107/f138 0 2024-02-16T22:22:19.674 INFO:tasks.workunit.client.0.smithi114.stdout:5/820: symlink d0/df/d6d/l10c 0 2024-02-16T22:22:19.713 INFO:tasks.workunit.client.0.smithi114.stdout:9/949: creat d2/d33/d3f/ddd/f139 x:0 0 0 2024-02-16T22:22:19.713 INFO:tasks.workunit.client.0.smithi114.stdout:5/821: read d0/d28/d98/d25/d2e/d3b/fbf [310665,47761] 0 2024-02-16T22:22:19.716 INFO:tasks.workunit.client.0.smithi114.stdout:5/822: symlink d0/df/d53/db1/l10d 0 2024-02-16T22:22:19.723 INFO:tasks.workunit.client.0.smithi114.stdout:5/823: dwrite d0/d28/d98/d1e/d63/d96/d9a/fd6 [0,4194304] 0 2024-02-16T22:22:19.729 INFO:tasks.workunit.client.0.smithi114.stdout:5/824: read d0/d28/d98/d25/d2e/f67 [367464,89340] 0 2024-02-16T22:22:19.732 INFO:tasks.workunit.client.0.smithi114.stdout:5/825: chown d0/d28/d98/d25/d89/de2/db3/lde 172 1 2024-02-16T22:22:19.735 INFO:tasks.workunit.client.0.smithi114.stdout:5/826: rename d0/d28/d98/d25/d2e/la5 to d0/d28/d98/d1e/d63/d96/d9a/l10e 0 2024-02-16T22:22:19.737 INFO:tasks.workunit.client.0.smithi114.stdout:5/827: creat d0/df/d5e/f10f x:0 0 0 2024-02-16T22:22:19.740 INFO:tasks.workunit.client.0.smithi114.stdout:5/828: creat d0/d28/d98/f110 x:0 0 0 2024-02-16T22:22:19.744 INFO:tasks.workunit.client.0.smithi114.stdout:5/829: getdents d0/df/d7a 0 2024-02-16T22:22:19.747 INFO:tasks.workunit.client.0.smithi114.stdout:5/830: creat d0/d28/d98/d1e/f111 x:0 0 0 2024-02-16T22:22:19.747 INFO:tasks.workunit.client.0.smithi114.stdout:9/950: write d2/dd/ff [2024563,109888] 0 2024-02-16T22:22:19.761 INFO:tasks.workunit.client.0.smithi114.stdout:9/951: dread d2/d1a/f39 [0,4194304] 0 2024-02-16T22:22:19.763 INFO:tasks.workunit.client.0.smithi114.stdout:9/952: symlink d2/dd/l13a 0 2024-02-16T22:22:19.764 INFO:tasks.workunit.client.0.smithi114.stdout:1/838: sync 2024-02-16T22:22:19.764 INFO:tasks.workunit.client.0.smithi114.stdout:2/912: sync 2024-02-16T22:22:19.767 INFO:tasks.workunit.client.0.smithi114.stdout:9/953: getdents d2/d5c/d5f/d72/dd6/d12c 0 2024-02-16T22:22:19.775 INFO:tasks.workunit.client.0.smithi114.stdout:9/954: symlink d2/d5c/d5f/d72/dd6/l13b 0 2024-02-16T22:22:19.775 INFO:tasks.workunit.client.0.smithi114.stdout:5/831: dwrite d0/fa [0,4194304] 0 2024-02-16T22:22:19.784 INFO:tasks.workunit.client.0.smithi114.stdout:9/955: truncate d2/d5c/dc9/d47/d63/fa8 3213723 0 2024-02-16T22:22:19.789 INFO:tasks.workunit.client.0.smithi114.stdout:9/956: truncate d2/d5c/dc9/d47/d63/fc5 2428843 0 2024-02-16T22:22:19.792 INFO:tasks.workunit.client.0.smithi114.stdout:9/957: fsync d2/d5c/dc9/ffc 0 2024-02-16T22:22:19.793 INFO:tasks.workunit.client.0.smithi114.stdout:9/958: chown d2/d5c/dc9/d47/d63/d98/l9f 4234594 1 2024-02-16T22:22:19.795 INFO:tasks.workunit.client.0.smithi114.stdout:9/959: creat d2/dd0/f13c x:0 0 0 2024-02-16T22:22:19.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:19 smithi123 ceph-mon[37161]: pgmap v37: 65 pgs: 65 active+clean; 3.8 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 118 MiB/s rd, 253 MiB/s wr, 592 op/s 2024-02-16T22:22:19.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:19 smithi123 ceph-mon[37161]: from='client.14788 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:22:19.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:19 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/2024227009' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:22:19.802 INFO:tasks.workunit.client.0.smithi114.stdout:9/960: rename d2/dd/d3c/d89/d8b/cc2 to d2/d1a/d65/c13d 0 2024-02-16T22:22:19.808 INFO:tasks.workunit.client.0.smithi114.stdout:9/961: rename d2/f12 to d2/d1a/d23/d32/f13e 0 2024-02-16T22:22:19.817 INFO:tasks.workunit.client.0.smithi114.stdout:2/913: write dd/df/d1c/d21/d61/d69/fff [339500,54667] 0 2024-02-16T22:22:19.817 INFO:tasks.workunit.client.0.smithi114.stdout:1/839: dwrite d4/d63/d69/d72/d82/fc7 [0,4194304] 0 2024-02-16T22:22:19.819 INFO:tasks.workunit.client.0.smithi114.stdout:2/914: fdatasync dd/df/d1c/d21/d61/d63/d83/df8/f109 0 2024-02-16T22:22:19.820 INFO:tasks.workunit.client.0.smithi114.stdout:2/915: readlink dd/df/d1c/d21/d61/dec/lfa 0 2024-02-16T22:22:19.826 INFO:tasks.workunit.client.0.smithi114.stdout:1/840: dread d4/d9/f12 [0,4194304] 0 2024-02-16T22:22:19.827 INFO:tasks.workunit.client.0.smithi114.stdout:2/916: dwrite dd/d3b/d8a/fe7 [0,4194304] 0 2024-02-16T22:22:19.828 INFO:tasks.workunit.client.0.smithi114.stdout:2/917: chown dd/da6/fae 349 1 2024-02-16T22:22:19.834 INFO:tasks.workunit.client.0.smithi114.stdout:1/841: chown d4/d63/d69/d7c/cd1 395972536 1 2024-02-16T22:22:19.837 INFO:tasks.workunit.client.0.smithi114.stdout:2/918: dwrite dd/df/d1c/d21/d61/d69/fff [0,4194304] 0 2024-02-16T22:22:19.848 INFO:tasks.workunit.client.0.smithi114.stdout:9/962: truncate d2/d1a/d23/d32/d46/d78/f134 3243988 0 2024-02-16T22:22:19.849 INFO:tasks.workunit.client.0.smithi114.stdout:1/842: dwrite d4/d9/d46/d67/d92/dca/f130 [0,4194304] 0 2024-02-16T22:22:19.854 INFO:tasks.workunit.client.0.smithi114.stdout:5/832: dwrite d0/df/d53/fd2 [0,4194304] 0 2024-02-16T22:22:19.856 INFO:tasks.workunit.client.0.smithi114.stdout:2/919: dwrite dd/d6b/da5/fc1 [0,4194304] 0 2024-02-16T22:22:19.866 INFO:tasks.workunit.client.0.smithi114.stdout:9/963: mkdir d2/d5c/dc9/d13f 0 2024-02-16T22:22:19.872 INFO:tasks.workunit.client.0.smithi114.stdout:2/920: dwrite dd/d6b/da5/f121 [0,4194304] 0 2024-02-16T22:22:19.875 INFO:tasks.workunit.client.0.smithi114.stdout:2/921: write dd/d3b/da7/d8b/dc0/f128 [192699,118028] 0 2024-02-16T22:22:19.876 INFO:tasks.workunit.client.0.smithi114.stdout:6/860: sync 2024-02-16T22:22:19.876 INFO:tasks.workunit.client.0.smithi114.stdout:0/926: sync 2024-02-16T22:22:19.879 INFO:tasks.workunit.client.0.smithi114.stdout:0/927: readlink d1/d16/d1f/d28/d51/d61/l75 0 2024-02-16T22:22:19.883 INFO:tasks.workunit.client.0.smithi114.stdout:9/964: dwrite d2/d1a/d23/d32/d46/f12e [0,4194304] 0 2024-02-16T22:22:19.884 INFO:tasks.workunit.client.0.smithi114.stdout:9/965: write d2/d1a/f6b [4429116,47003] 0 2024-02-16T22:22:19.885 INFO:tasks.workunit.client.0.smithi114.stdout:6/861: mkdir d6/d2c/d38/da8/d101 0 2024-02-16T22:22:19.886 INFO:tasks.workunit.client.0.smithi114.stdout:5/833: getdents d0/d28/d98/d25/d3e 0 2024-02-16T22:22:19.887 INFO:tasks.workunit.client.0.smithi114.stdout:5/834: readlink d0/d28/d98/d25/d3e/d7f/ld3 0 2024-02-16T22:22:19.887 INFO:tasks.workunit.client.0.smithi114.stdout:5/835: readlink d0/df/d53/lb6 0 2024-02-16T22:22:19.891 INFO:tasks.workunit.client.0.smithi114.stdout:0/928: write d1/d16/dbf/fc6 [4666438,64967] 0 2024-02-16T22:22:19.900 INFO:tasks.workunit.client.0.smithi114.stdout:0/929: read d1/d16/f82 [236686,42976] 0 2024-02-16T22:22:19.904 INFO:tasks.workunit.client.0.smithi114.stdout:0/930: creat d1/d16/dbf/f12c x:0 0 0 2024-02-16T22:22:19.905 INFO:tasks.workunit.client.0.smithi114.stdout:2/922: link dd/df/d1c/fa4 dd/df/d1c/d21/d61/f12a 0 2024-02-16T22:22:19.910 INFO:tasks.workunit.client.0.smithi114.stdout:0/931: chown d1/d1a/f38 702 1 2024-02-16T22:22:19.923 INFO:tasks.workunit.client.0.smithi114.stdout:0/932: link d1/d16/d48/d98/lf4 d1/d16/d1f/d28/d51/d60/dd1/l12d 0 2024-02-16T22:22:19.926 INFO:tasks.workunit.client.0.smithi114.stdout:9/966: dread d2/d1a/d23/d32/d46/f9d [0,4194304] 0 2024-02-16T22:22:19.928 INFO:tasks.workunit.client.0.smithi114.stdout:9/967: write d2/dd/d3c/d89/d9e/fa2 [379393,120711] 0 2024-02-16T22:22:19.930 INFO:tasks.workunit.client.0.smithi114.stdout:0/933: rename d1/d16/d1f/d28/d51/d61/d6d/df1/c83 to d1/d16/d48/d118/c12e 0 2024-02-16T22:22:19.935 INFO:tasks.workunit.client.0.smithi114.stdout:9/968: chown d2/d5c/dc9/d47/d63/fa8 1 1 2024-02-16T22:22:19.936 INFO:tasks.workunit.client.0.smithi114.stdout:9/969: read - d2/dd/d3c/d89/d8b/df2/f119 zero size 2024-02-16T22:22:19.937 INFO:tasks.workunit.client.0.smithi114.stdout:2/923: dread dd/d4e/fbb [0,4194304] 0 2024-02-16T22:22:19.940 INFO:tasks.workunit.client.0.smithi114.stdout:2/924: creat dd/d3b/da7/d4c/dc7/f12b x:0 0 0 2024-02-16T22:22:19.941 INFO:tasks.workunit.client.0.smithi114.stdout:2/925: chown dd/df/d1c/d21/d87/le5 260148339 1 2024-02-16T22:22:19.941 INFO:tasks.workunit.client.0.smithi114.stdout:2/926: dread - dd/d4e/f11e zero size 2024-02-16T22:22:19.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:19 smithi114 ceph-mon[29235]: pgmap v37: 65 pgs: 65 active+clean; 3.8 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 118 MiB/s rd, 253 MiB/s wr, 592 op/s 2024-02-16T22:22:19.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:19 smithi114 ceph-mon[29235]: from='client.14788 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:22:19.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:19 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/2024227009' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:22:19.957 INFO:tasks.workunit.client.0.smithi114.stdout:1/843: dwrite d4/d9/d3d/f39 [0,4194304] 0 2024-02-16T22:22:19.960 INFO:tasks.workunit.client.0.smithi114.stdout:1/844: symlink d4/d9/d3d/l131 0 2024-02-16T22:22:19.960 INFO:tasks.workunit.client.0.smithi114.stdout:6/862: write d6/fcd [969008,25787] 0 2024-02-16T22:22:19.970 INFO:tasks.workunit.client.0.smithi114.stdout:6/863: rename d6/dd/d25/d34/d61/d62/d8b/db7/dd3/df6/lfc to d6/dd/d25/d34/d58/d9f/da3/l102 0 2024-02-16T22:22:19.970 INFO:tasks.workunit.client.0.smithi114.stdout:1/845: link d4/d9/d3d/f41 d4/d9/d3d/f132 0 2024-02-16T22:22:19.979 INFO:tasks.workunit.client.0.smithi114.stdout:9/970: dwrite d2/d1a/d65/f94 [0,4194304] 0 2024-02-16T22:22:19.979 INFO:tasks.workunit.client.0.smithi114.stdout:2/927: dwrite dd/df/d1c/d21/d61/fe4 [0,4194304] 0 2024-02-16T22:22:19.982 INFO:tasks.workunit.client.0.smithi114.stdout:9/971: creat d2/d33/d3f/f140 x:0 0 0 2024-02-16T22:22:19.985 INFO:tasks.workunit.client.0.smithi114.stdout:2/928: chown dd/df/c5b 16058175 1 2024-02-16T22:22:19.987 INFO:tasks.workunit.client.0.smithi114.stdout:6/864: rename d6/dd/d25/d34/d58/d9f/da3/l102 to d6/d2c/d5e/dcc/l103 0 2024-02-16T22:22:19.988 INFO:tasks.workunit.client.0.smithi114.stdout:6/865: chown d6/dd/l55 52657 1 2024-02-16T22:22:19.996 INFO:tasks.workunit.client.0.smithi114.stdout:1/846: rename d4/d9/d46/d67/d92/dca/dcc/d70/f10e to d4/d63/d69/d72/d82/f133 0 2024-02-16T22:22:19.997 INFO:tasks.workunit.client.0.smithi114.stdout:9/972: link d2/dd/d3c/d89/d9e/la0 d2/dd/d3c/d89/l141 0 2024-02-16T22:22:19.998 INFO:tasks.workunit.client.0.smithi114.stdout:1/847: write d4/d9/d46/d47/ddf/f10d [4995936,99573] 0 2024-02-16T22:22:20.000 INFO:tasks.workunit.client.0.smithi114.stdout:6/866: creat d6/dd/d25/d34/d58/d9f/df4/f104 x:0 0 0 2024-02-16T22:22:20.002 INFO:tasks.workunit.client.0.smithi114.stdout:6/867: readlink d6/dd/d25/d34/d61/l97 0 2024-02-16T22:22:20.002 INFO:tasks.workunit.client.0.smithi114.stdout:9/973: creat d2/dd/d132/f142 x:0 0 0 2024-02-16T22:22:20.004 INFO:tasks.workunit.client.0.smithi114.stdout:2/929: dwrite dd/df/d1c/d21/d61/d69/fce [0,4194304] 0 2024-02-16T22:22:20.007 INFO:tasks.workunit.client.0.smithi114.stdout:6/868: symlink d6/d83/l105 0 2024-02-16T22:22:20.011 INFO:tasks.workunit.client.0.smithi114.stdout:2/930: creat dd/d6b/db8/f12c x:0 0 0 2024-02-16T22:22:20.014 INFO:tasks.workunit.client.0.smithi114.stdout:6/869: symlink d6/dd/d25/d34/d61/d62/d8b/l106 0 2024-02-16T22:22:20.018 INFO:tasks.workunit.client.0.smithi114.stdout:2/931: dwrite dd/da6/f119 [0,4194304] 0 2024-02-16T22:22:20.019 INFO:tasks.workunit.client.0.smithi114.stdout:2/932: chown dd/d3b/da7/d8b/dc0/d117 33214363 1 2024-02-16T22:22:20.022 INFO:tasks.workunit.client.0.smithi114.stdout:9/974: dwrite d2/d33/fc4 [0,4194304] 0 2024-02-16T22:22:20.023 INFO:tasks.workunit.client.0.smithi114.stdout:2/933: creat dd/d6b/da5/df6/f12d x:0 0 0 2024-02-16T22:22:20.024 INFO:tasks.workunit.client.0.smithi114.stdout:6/870: dread d6/d2c/d38/d41/d60/fbc [0,4194304] 0 2024-02-16T22:22:20.025 INFO:tasks.workunit.client.0.smithi114.stdout:2/934: chown dd/d3b/da7/f125 1509 1 2024-02-16T22:22:20.029 INFO:tasks.workunit.client.0.smithi114.stdout:6/871: write d6/d2c/f95 [4919231,76874] 0 2024-02-16T22:22:20.031 INFO:tasks.workunit.client.0.smithi114.stdout:9/975: creat d2/d1a/d65/f143 x:0 0 0 2024-02-16T22:22:20.034 INFO:tasks.workunit.client.0.smithi114.stdout:9/976: creat d2/f144 x:0 0 0 2024-02-16T22:22:20.035 INFO:tasks.workunit.client.0.smithi114.stdout:2/935: link dd/df/d1c/d21/l7d dd/d6b/db8/l12e 0 2024-02-16T22:22:20.035 INFO:tasks.workunit.client.0.smithi114.stdout:6/872: dwrite d6/f21 [0,4194304] 0 2024-02-16T22:22:20.038 INFO:tasks.workunit.client.0.smithi114.stdout:2/936: symlink dd/da6/l12f 0 2024-02-16T22:22:20.039 INFO:tasks.workunit.client.0.smithi114.stdout:6/873: truncate d6/dd/d25/d34/d58/d9f/da3/fe5 593191 0 2024-02-16T22:22:20.043 INFO:tasks.workunit.client.0.smithi114.stdout:9/977: creat d2/d5c/d5f/d7b/d125/f145 x:0 0 0 2024-02-16T22:22:20.044 INFO:tasks.workunit.client.0.smithi114.stdout:9/978: stat d2/d5c/dc9/d47/d63/d98/ld8 0 2024-02-16T22:22:20.045 INFO:tasks.workunit.client.0.smithi114.stdout:2/937: creat dd/d3b/da7/d4c/dc7/f130 x:0 0 0 2024-02-16T22:22:20.053 INFO:tasks.workunit.client.0.smithi114.stdout:6/874: dwrite d6/da/f56 [0,4194304] 0 2024-02-16T22:22:20.066 INFO:tasks.workunit.client.0.smithi114.stdout:6/875: dwrite d6/dd/d25/d34/d61/d62/d8b/db7/dd3/fe1 [0,4194304] 0 2024-02-16T22:22:20.072 INFO:tasks.workunit.client.0.smithi114.stdout:2/938: dread dd/d3b/da7/d4c/f79 [0,4194304] 0 2024-02-16T22:22:20.074 INFO:tasks.workunit.client.0.smithi114.stdout:2/939: truncate dd/d3b/d8a/fe8 8777138 0 2024-02-16T22:22:20.075 INFO:tasks.workunit.client.0.smithi114.stdout:6/876: fsync d6/dd/d25/d34/d58/f78 0 2024-02-16T22:22:20.076 INFO:tasks.workunit.client.0.smithi114.stdout:2/940: fdatasync dd/df/f1e 0 2024-02-16T22:22:20.081 INFO:tasks.workunit.client.0.smithi114.stdout:6/877: write d6/d2c/d38/d41/d60/d84/fc8 [765789,4218] 0 2024-02-16T22:22:20.085 INFO:tasks.workunit.client.0.smithi114.stdout:6/878: unlink d6/d2c/d75/fdf 0 2024-02-16T22:22:20.091 INFO:tasks.workunit.client.0.smithi114.stdout:6/879: write d6/d2c/d66/fa5 [2067251,33117] 0 2024-02-16T22:22:20.094 INFO:tasks.workunit.client.0.smithi114.stdout:6/880: creat d6/d2c/d5e/f107 x:0 0 0 2024-02-16T22:22:20.103 INFO:tasks.workunit.client.0.smithi114.stdout:6/881: dwrite d6/dd/f17 [4194304,4194304] 0 2024-02-16T22:22:20.104 INFO:tasks.workunit.client.0.smithi114.stdout:6/882: readlink d6/dd/l55 0 2024-02-16T22:22:20.105 INFO:tasks.workunit.client.0.smithi114.stdout:6/883: write d6/d2c/de6/ff2 [982931,91518] 0 2024-02-16T22:22:20.109 INFO:tasks.workunit.client.0.smithi114.stdout:3/897: sync 2024-02-16T22:22:20.109 INFO:tasks.workunit.client.0.smithi114.stdout:4/994: sync 2024-02-16T22:22:20.110 INFO:tasks.workunit.client.0.smithi114.stdout:7/952: sync 2024-02-16T22:22:20.110 INFO:tasks.workunit.client.0.smithi114.stdout:8/836: sync 2024-02-16T22:22:20.119 INFO:tasks.workunit.client.0.smithi114.stdout:3/898: rmdir d4/d26/dd7/d104 39 2024-02-16T22:22:20.123 INFO:tasks.workunit.client.0.smithi114.stdout:4/995: unlink dd/d1b/d41/d74/f135 0 2024-02-16T22:22:20.126 INFO:tasks.workunit.client.0.smithi114.stdout:6/884: symlink d6/d2c/d38/da8/d101/l108 0 2024-02-16T22:22:20.127 INFO:tasks.workunit.client.0.smithi114.stdout:6/885: write d6/fec [312895,17883] 0 2024-02-16T22:22:20.130 INFO:tasks.workunit.client.0.smithi114.stdout:4/996: dwrite dd/d1b/d41/fbb [4194304,4194304] 0 2024-02-16T22:22:20.134 INFO:tasks.workunit.client.0.smithi114.stdout:7/953: link d3/d3d/l3f d3/d92/l14e 0 2024-02-16T22:22:20.138 INFO:tasks.workunit.client.0.smithi114.stdout:3/899: creat d4/d26/dd7/d104/dcc/f137 x:0 0 0 2024-02-16T22:22:20.140 INFO:tasks.workunit.client.0.smithi114.stdout:8/837: link d4/d16/d25/c48 d4/d24/c127 0 2024-02-16T22:22:20.143 INFO:tasks.workunit.client.0.smithi114.stdout:4/997: read dd/d1b/d24/d57/d12b/d61/df4/f12e [3106061,57443] 0 2024-02-16T22:22:20.146 INFO:tasks.workunit.client.0.smithi114.stdout:7/954: symlink d3/l14f 0 2024-02-16T22:22:20.154 INFO:tasks.workunit.client.0.smithi114.stdout:7/955: creat d3/d22/d61/f150 x:0 0 0 2024-02-16T22:22:20.155 INFO:tasks.workunit.client.0.smithi114.stdout:6/886: getdents d6/dd/d25/d34/d58/d9f/da3/dbd 0 2024-02-16T22:22:20.158 INFO:tasks.workunit.client.0.smithi114.stdout:7/956: chown d3/d3d/l3f 149048632 1 2024-02-16T22:22:20.159 INFO:tasks.workunit.client.0.smithi114.stdout:6/887: creat d6/d2c/d38/d54/f109 x:0 0 0 2024-02-16T22:22:20.163 INFO:tasks.workunit.client.0.smithi114.stdout:6/888: fsync d6/d2c/d38/d41/f4a 0 2024-02-16T22:22:20.164 INFO:tasks.workunit.client.0.smithi114.stdout:8/838: write d4/d11/d78/d7a/d36/fb9 [601506,98934] 0 2024-02-16T22:22:20.171 INFO:tasks.workunit.client.0.smithi114.stdout:3/900: dwrite d4/d26/dd7/d104/d80/dbd/ffb [4194304,4194304] 0 2024-02-16T22:22:20.173 INFO:tasks.workunit.client.0.smithi114.stdout:6/889: creat d6/dd/d25/d34/d61/f10a x:0 0 0 2024-02-16T22:22:20.174 INFO:tasks.workunit.client.0.smithi114.stdout:4/998: dwrite dd/d59/d8f/f152 [0,4194304] 0 2024-02-16T22:22:20.174 INFO:tasks.workunit.client.0.smithi114.stdout:3/901: readlink d4/dd/d33/d8f/l121 0 2024-02-16T22:22:20.176 INFO:tasks.workunit.client.0.smithi114.stdout:6/890: chown d6/dd/c9c 81224390 1 2024-02-16T22:22:20.177 INFO:tasks.workunit.client.0.smithi114.stdout:8/839: link d4/d11/d78/d7a/d7e/f80 d4/d11/d78/d7a/d36/d3b/f128 0 2024-02-16T22:22:20.177 INFO:tasks.workunit.client.0.smithi114.stdout:7/957: dwrite d3/d87/fe4 [0,4194304] 0 2024-02-16T22:22:20.178 INFO:tasks.workunit.client.0.smithi114.stdout:4/999: dread - dd/d1b/d24/d57/d12b/d99/d103/f111 zero size 2024-02-16T22:22:20.179 INFO:tasks.workunit.client.0.smithi114.stdout:7/958: stat d3/d46/da2 0 2024-02-16T22:22:20.179 INFO:tasks.workunit.client.0.smithi114.stdout:8/840: chown d4/d11/d78/d7a/d36/d3b/ce1 1 1 2024-02-16T22:22:20.180 INFO:tasks.workunit.client.0.smithi114.stdout:3/902: truncate d4/d26/dd7/d104/d75/daa/ffc 452854 0 2024-02-16T22:22:20.183 INFO:tasks.workunit.client.0.smithi114.stdout:6/891: read d6/dd/d25/d34/d61/d62/f8d [7957755,78982] 0 2024-02-16T22:22:20.184 INFO:tasks.workunit.client.0.smithi114.stdout:8/841: mkdir d4/d11/d78/d7a/d31/dc6/d129 0 2024-02-16T22:22:20.184 INFO:tasks.workunit.client.0.smithi114.stdout:7/959: getdents d3/d92/dbe/de6/d14a 0 2024-02-16T22:22:20.184 INFO:tasks.workunit.client.0.smithi114.stdout:8/842: fsync d4/d11/d9d/fdd 0 2024-02-16T22:22:20.188 INFO:tasks.workunit.client.0.smithi114.stdout:7/960: dread - d3/d22/d73/da7/db4/d146/da6/fc6 zero size 2024-02-16T22:22:20.194 INFO:tasks.workunit.client.0.smithi114.stdout:3/903: rename d4/d26/d4a/f65 to d4/d43/f138 0 2024-02-16T22:22:20.198 INFO:tasks.workunit.client.0.smithi114.stdout:3/904: rename d4/d43/d105 to d4/d46/dcf/d139 0 2024-02-16T22:22:20.201 INFO:tasks.workunit.client.0.smithi114.stdout:3/905: rename d4/d24/l60 to d4/d26/dd7/d104/d75/l13a 0 2024-02-16T22:22:20.203 INFO:tasks.workunit.client.0.smithi114.stdout:3/906: mknod d4/d43/c13b 0 2024-02-16T22:22:20.206 INFO:tasks.workunit.client.0.smithi114.stdout:3/907: creat d4/d24/def/d136/f13c x:0 0 0 2024-02-16T22:22:20.212 INFO:tasks.workunit.client.0.smithi114.stdout:3/908: dwrite d4/d26/dd7/d104/d80/dbd/ffb [4194304,4194304] 0 2024-02-16T22:22:20.217 INFO:tasks.workunit.client.0.smithi114.stdout:8/843: truncate d4/d24/f7d 3613104 0 2024-02-16T22:22:20.218 INFO:tasks.workunit.client.0.smithi114.stdout:8/844: chown d4/d11/l2e 1 1 2024-02-16T22:22:20.219 INFO:tasks.workunit.client.0.smithi114.stdout:6/892: dwrite d6/dd/d25/fc0 [0,4194304] 0 2024-02-16T22:22:20.225 INFO:tasks.workunit.client.0.smithi114.stdout:8/845: unlink d4/d11/d78/d7a/d36/d3b/l41 0 2024-02-16T22:22:20.228 INFO:tasks.workunit.client.0.smithi114.stdout:7/961: dwrite d3/ff [0,4194304] 0 2024-02-16T22:22:20.229 INFO:tasks.workunit.client.0.smithi114.stdout:7/962: write d3/d87/dd1/f139 [487265,89056] 0 2024-02-16T22:22:20.242 INFO:tasks.workunit.client.0.smithi114.stdout:7/963: rename d3/d22/f48 to d3/d22/d61/f151 0 2024-02-16T22:22:20.242 INFO:tasks.workunit.client.0.smithi114.stdout:6/893: link d6/da/f9b d6/dd/d25/d34/d58/d9f/da3/dbd/dee/f10b 0 2024-02-16T22:22:20.245 INFO:tasks.workunit.client.0.smithi114.stdout:8/846: getdents d4/d16/d25/d7c 0 2024-02-16T22:22:20.245 INFO:tasks.workunit.client.0.smithi114.stdout:3/909: dwrite d4/d43/f6e [0,4194304] 0 2024-02-16T22:22:20.246 INFO:tasks.workunit.client.0.smithi114.stdout:7/964: creat d3/d46/da2/f152 x:0 0 0 2024-02-16T22:22:20.251 INFO:tasks.workunit.client.0.smithi114.stdout:3/910: truncate d4/d24/def/d136/f13c 74342 0 2024-02-16T22:22:20.254 INFO:tasks.workunit.client.0.smithi114.stdout:6/894: dwrite d6/d2c/d38/d54/f77 [4194304,4194304] 0 2024-02-16T22:22:20.259 INFO:tasks.workunit.client.0.smithi114.stdout:3/911: dread d4/d26/dd7/d104/d75/fd5 [0,4194304] 0 2024-02-16T22:22:20.263 INFO:tasks.workunit.client.0.smithi114.stdout:7/965: getdents d3/d22/d61 0 2024-02-16T22:22:20.266 INFO:tasks.workunit.client.0.smithi114.stdout:6/895: getdents d6/d2c 0 2024-02-16T22:22:20.268 INFO:tasks.workunit.client.0.smithi114.stdout:7/966: rmdir d3/d22/d73/da7/d6f/df8 39 2024-02-16T22:22:20.269 INFO:tasks.workunit.client.0.smithi114.stdout:8/847: write d4/fbb [29527,39623] 0 2024-02-16T22:22:20.269 INFO:tasks.workunit.client.0.smithi114.stdout:3/912: symlink d4/l13d 0 2024-02-16T22:22:20.272 INFO:tasks.workunit.client.0.smithi114.stdout:6/896: chown d6/dd/d25/d34/d61/l64 413503366 1 2024-02-16T22:22:20.274 INFO:tasks.workunit.client.0.smithi114.stdout:3/913: creat d4/d46/dcf/f13e x:0 0 0 2024-02-16T22:22:20.276 INFO:tasks.workunit.client.0.smithi114.stdout:7/967: rename d3/d22/d73/da7/d4e/f7f to d3/d92/dbe/de6/f153 0 2024-02-16T22:22:20.277 INFO:tasks.workunit.client.0.smithi114.stdout:7/968: chown d3/le5 681247 1 2024-02-16T22:22:20.278 INFO:tasks.workunit.client.0.smithi114.stdout:7/969: fdatasync d3/d14/f29 0 2024-02-16T22:22:20.279 INFO:tasks.workunit.client.0.smithi114.stdout:3/914: mknod d4/d24/d2d/d45/c13f 0 2024-02-16T22:22:20.279 INFO:tasks.workunit.client.0.smithi114.stdout:7/970: chown d3/d22/d73/da7/f68 9 1 2024-02-16T22:22:20.281 INFO:tasks.workunit.client.0.smithi114.stdout:8/848: rename d4/d24/d2c/d5f to d4/d24/d6d/dc7/d12a 0 2024-02-16T22:22:20.281 INFO:tasks.workunit.client.0.smithi114.stdout:6/897: getdents d6/dd/d25/d34/d61/d62/d8b/db7 0 2024-02-16T22:22:20.283 INFO:tasks.workunit.client.0.smithi114.stdout:8/849: truncate d4/d11/d9d/ddf/f112 304907 0 2024-02-16T22:22:20.283 INFO:tasks.workunit.client.0.smithi114.stdout:3/915: truncate d4/dd/d33/d76/f7f 3275389 0 2024-02-16T22:22:20.286 INFO:tasks.workunit.client.0.smithi114.stdout:6/898: symlink d6/d2c/d66/l10c 0 2024-02-16T22:22:20.286 INFO:tasks.workunit.client.0.smithi114.stdout:8/850: rmdir d4/d16/d25/d46/d10b/dcf 39 2024-02-16T22:22:20.288 INFO:tasks.workunit.client.0.smithi114.stdout:3/916: dread d4/dd/f35 [0,4194304] 0 2024-02-16T22:22:20.288 INFO:tasks.workunit.client.0.smithi114.stdout:6/899: creat d6/d2c/d5e/f10d x:0 0 0 2024-02-16T22:22:20.290 INFO:tasks.workunit.client.0.smithi114.stdout:3/917: creat d4/d24/def/d136/f140 x:0 0 0 2024-02-16T22:22:20.292 INFO:tasks.workunit.client.0.smithi114.stdout:8/851: link d4/d24/d2c/fbe d4/d16/f12b 0 2024-02-16T22:22:20.293 INFO:tasks.workunit.client.0.smithi114.stdout:8/852: unlink d4/fdb 0 2024-02-16T22:22:20.295 INFO:tasks.workunit.client.0.smithi114.stdout:8/853: read d4/d11/d78/d7a/d31/f4d [3596833,130189] 0 2024-02-16T22:22:20.330 INFO:tasks.workunit.client.0.smithi114.stdout:8/854: getdents d4/d11/d78/d7a 0 2024-02-16T22:22:20.347 INFO:tasks.workunit.client.0.smithi114.stdout:3/918: write d4/d24/d2d/fed [47188,30212] 0 2024-02-16T22:22:20.354 INFO:tasks.workunit.client.0.smithi114.stdout:3/919: link d4/d24/d2d/df0/l6c d4/d46/dcf/d139/l141 0 2024-02-16T22:22:20.355 INFO:tasks.workunit.client.0.smithi114.stdout:8/855: dwrite d4/d11/d78/d7a/d7e/fad [0,4194304] 0 2024-02-16T22:22:20.359 INFO:tasks.workunit.client.0.smithi114.stdout:8/856: read d4/d11/d78/d7a/d7e/f9a [165808,127647] 0 2024-02-16T22:22:20.361 INFO:tasks.workunit.client.0.smithi114.stdout:3/920: rename d4/d87/l97 to d4/d26/dd7/d104/d75/daa/l142 0 2024-02-16T22:22:20.364 INFO:tasks.workunit.client.0.smithi114.stdout:3/921: fsync d4/d26/d4a/f90 0 2024-02-16T22:22:20.368 INFO:tasks.workunit.client.0.smithi114.stdout:3/922: mkdir d4/dd/d33/d8f/d143 0 2024-02-16T22:22:20.376 INFO:tasks.workunit.client.0.smithi114.stdout:8/857: write d4/d11/f60 [2131551,97581] 0 2024-02-16T22:22:20.378 INFO:tasks.workunit.client.0.smithi114.stdout:8/858: write d4/d11/d9d/f125 [150706,129078] 0 2024-02-16T22:22:20.380 INFO:tasks.workunit.client.0.smithi114.stdout:3/923: getdents d4/d46 0 2024-02-16T22:22:20.382 INFO:tasks.workunit.client.0.smithi114.stdout:8/859: creat d4/d16/d25/d7c/f12c x:0 0 0 2024-02-16T22:22:20.387 INFO:tasks.workunit.client.0.smithi114.stdout:3/924: mknod d4/d26/dd7/d104/d75/c144 0 2024-02-16T22:22:20.387 INFO:tasks.workunit.client.0.smithi114.stdout:8/860: mkdir d4/d11/d78/d7a/d36/d12d 0 2024-02-16T22:22:20.390 INFO:tasks.workunit.client.0.smithi114.stdout:3/925: dwrite d4/d43/f56 [0,4194304] 0 2024-02-16T22:22:20.393 INFO:tasks.workunit.client.0.smithi114.stdout:3/926: creat d4/d26/dd7/d104/d80/dbd/f145 x:0 0 0 2024-02-16T22:22:20.413 INFO:tasks.workunit.client.0.smithi114.stdout:3/927: write d4/d46/fc0 [520786,50794] 0 2024-02-16T22:22:20.415 INFO:tasks.workunit.client.0.smithi114.stdout:3/928: symlink d4/dbe/l146 0 2024-02-16T22:22:20.422 INFO:tasks.workunit.client.0.smithi114.stdout:3/929: link d4/dd/d33/d72/cd3 d4/d24/d2d/d45/d74/c147 0 2024-02-16T22:22:20.427 INFO:tasks.workunit.client.0.smithi114.stdout:3/930: dwrite d4/dd/d33/f133 [0,4194304] 0 2024-02-16T22:22:20.433 INFO:tasks.workunit.client.0.smithi114.stdout:3/931: creat d4/d26/dd7/d104/d75/f148 x:0 0 0 2024-02-16T22:22:20.439 INFO:tasks.workunit.client.0.smithi114.stdout:3/932: dwrite d4/d26/d91/fd1 [0,4194304] 0 2024-02-16T22:22:20.440 INFO:tasks.workunit.client.0.smithi114.stdout:3/933: fsync d4/d24/d2d/d45/d74/f116 0 2024-02-16T22:22:20.453 INFO:tasks.workunit.client.0.smithi114.stdout:3/934: dwrite d4/d24/d2d/f4e [4194304,4194304] 0 2024-02-16T22:22:20.456 INFO:tasks.workunit.client.0.smithi114.stdout:3/935: creat d4/d24/d2d/d45/d74/dc7/f149 x:0 0 0 2024-02-16T22:22:20.463 INFO:tasks.workunit.client.0.smithi114.stdout:3/936: dwrite d4/d24/d2d/df0/d77/f110 [0,4194304] 0 2024-02-16T22:22:20.464 INFO:tasks.workunit.client.0.smithi114.stdout:3/937: chown d4/l13d 78 1 2024-02-16T22:22:20.472 INFO:tasks.workunit.client.0.smithi114.stdout:3/938: dwrite d4/d26/f39 [0,4194304] 0 2024-02-16T22:22:20.477 INFO:tasks.workunit.client.0.smithi114.stdout:3/939: write d4/dd/d33/f5f [631696,47419] 0 2024-02-16T22:22:20.481 INFO:tasks.workunit.client.0.smithi114.stdout:3/940: rename d4/d26/dd7/d104/dcc/fc6 to d4/dd/d33/f14a 0 2024-02-16T22:22:20.486 INFO:tasks.workunit.client.0.smithi114.stdout:3/941: creat d4/d26/dd7/d104/d80/f14b x:0 0 0 2024-02-16T22:22:20.489 INFO:tasks.workunit.client.0.smithi114.stdout:3/942: creat d4/d7e/f14c x:0 0 0 2024-02-16T22:22:20.491 INFO:tasks.workunit.client.0.smithi114.stdout:3/943: rmdir d4/d46/dcf/d139 39 2024-02-16T22:22:20.492 INFO:tasks.workunit.client.0.smithi114.stdout:3/944: write d4/d26/dd7/d104/d75/f9c [1277348,129353] 0 2024-02-16T22:22:20.494 INFO:tasks.workunit.client.0.smithi114.stdout:3/945: symlink d4/dd/l14d 0 2024-02-16T22:22:20.496 INFO:tasks.workunit.client.0.smithi114.stdout:3/946: mkdir d4/d24/d2d/d45/d132/d14e 0 2024-02-16T22:22:20.515 INFO:tasks.workunit.client.0.smithi114.stdout:3/947: dwrite d4/d26/d91/fd1 [4194304,4194304] 0 2024-02-16T22:22:20.520 INFO:tasks.workunit.client.0.smithi114.stdout:3/948: creat d4/d26/dd7/d104/f14f x:0 0 0 2024-02-16T22:22:20.521 INFO:tasks.workunit.client.0.smithi114.stdout:3/949: write d4/fd8 [1328955,69864] 0 2024-02-16T22:22:20.521 INFO:tasks.workunit.client.0.smithi114.stdout:3/950: read - d4/d24/d2d/d45/d74/f116 zero size 2024-02-16T22:22:20.537 INFO:tasks.workunit.client.0.smithi114.stdout:5/836: sync 2024-02-16T22:22:20.540 INFO:tasks.workunit.client.0.smithi114.stdout:3/951: rename d4/d26/dd7/d104/d75/fd5 to d4/d24/d93/f150 0 2024-02-16T22:22:20.542 INFO:tasks.workunit.client.0.smithi114.stdout:3/952: mknod d4/dd/d33/c151 0 2024-02-16T22:22:20.544 INFO:tasks.workunit.client.0.smithi114.stdout:3/953: symlink d4/d46/dcf/d139/l152 0 2024-02-16T22:22:20.552 INFO:tasks.workunit.client.0.smithi114.stdout:5/837: dread d0/d28/d98/d1e/d63/fc8 [0,4194304] 0 2024-02-16T22:22:20.555 INFO:tasks.workunit.client.0.smithi114.stdout:5/838: mkdir d0/d28/d98/d25/d89/d112 0 2024-02-16T22:22:20.558 INFO:tasks.workunit.client.0.smithi114.stdout:5/839: truncate d0/d28/d98/d25/d2e/d3b/fbf 509115 0 2024-02-16T22:22:20.561 INFO:tasks.workunit.client.0.smithi114.stdout:3/954: dwrite d4/d43/f138 [0,4194304] 0 2024-02-16T22:22:20.561 INFO:tasks.workunit.client.0.smithi114.stdout:5/840: creat d0/d28/d98/d1e/d63/d96/f113 x:0 0 0 2024-02-16T22:22:20.569 INFO:tasks.workunit.client.0.smithi114.stdout:3/955: write d4/dd/f13 [177856,15907] 0 2024-02-16T22:22:20.573 INFO:tasks.workunit.client.0.smithi114.stdout:5/841: dwrite d0/df/f8a [4194304,4194304] 0 2024-02-16T22:22:20.576 INFO:tasks.workunit.client.0.smithi114.stdout:5/842: creat d0/df/d7a/dc4/f114 x:0 0 0 2024-02-16T22:22:20.579 INFO:tasks.workunit.client.0.smithi114.stdout:3/956: write d4/d26/d4a/f90 [4418125,116099] 0 2024-02-16T22:22:20.581 INFO:tasks.workunit.client.0.smithi114.stdout:5/843: creat d0/d28/d98/d25/f115 x:0 0 0 2024-02-16T22:22:20.583 INFO:tasks.workunit.client.0.smithi114.stdout:3/957: truncate d4/d24/f99 420299 0 2024-02-16T22:22:20.588 INFO:tasks.workunit.client.0.smithi114.stdout:5/844: dwrite d0/d28/d98/f110 [0,4194304] 0 2024-02-16T22:22:20.590 INFO:tasks.workunit.client.0.smithi114.stdout:5/845: symlink d0/d28/d98/d25/d3e/l116 0 2024-02-16T22:22:20.592 INFO:tasks.workunit.client.0.smithi114.stdout:3/958: link d4/d46/dcf/d139/l141 d4/d26/dd7/d109/l153 0 2024-02-16T22:22:20.595 INFO:tasks.workunit.client.0.smithi114.stdout:3/959: creat d4/d26/dd7/d104/d75/f154 x:0 0 0 2024-02-16T22:22:20.596 INFO:tasks.workunit.client.0.smithi114.stdout:3/960: chown d4/d24/def/d136 34447478 1 2024-02-16T22:22:20.597 INFO:tasks.workunit.client.0.smithi114.stdout:3/961: readlink d4/d46/dcf/ldf 0 2024-02-16T22:22:20.602 INFO:tasks.workunit.client.0.smithi114.stdout:3/962: dwrite d4/d43/f56 [0,4194304] 0 2024-02-16T22:22:20.604 INFO:tasks.workunit.client.0.smithi114.stdout:3/963: dread - d4/d26/dd7/d104/f120 zero size 2024-02-16T22:22:20.606 INFO:tasks.workunit.client.0.smithi114.stdout:3/964: unlink d4/dd/d33/d8f/l98 0 2024-02-16T22:22:20.607 INFO:tasks.workunit.client.0.smithi114.stdout:3/965: chown d4/dbe/lec 825 1 2024-02-16T22:22:20.608 INFO:tasks.workunit.client.0.smithi114.stdout:3/966: chown d4/d26/dd7/d104/d80/dbd/cd6 160590733 1 2024-02-16T22:22:20.609 INFO:tasks.workunit.client.0.smithi114.stdout:3/967: fdatasync d4/d26/fc1 0 2024-02-16T22:22:20.611 INFO:tasks.workunit.client.0.smithi114.stdout:3/968: read f1 [76247,90993] 0 2024-02-16T22:22:20.614 INFO:tasks.workunit.client.0.smithi114.stdout:3/969: truncate d4/d24/def/f7c 600273 0 2024-02-16T22:22:20.620 INFO:tasks.workunit.client.0.smithi114.stdout:3/970: unlink d4/d24/d93/f150 0 2024-02-16T22:22:20.621 INFO:tasks.workunit.client.0.smithi114.stdout:3/971: chown d4/d24/d93/c107 1 1 2024-02-16T22:22:20.622 INFO:tasks.workunit.client.0.smithi114.stdout:3/972: dread d4/dd/d33/d72/f12d [0,4194304] 0 2024-02-16T22:22:20.625 INFO:tasks.workunit.client.0.smithi114.stdout:3/973: dread d4/d43/fce [0,4194304] 0 2024-02-16T22:22:20.639 INFO:tasks.workunit.client.0.smithi114.stdout:3/974: write d4/d43/fce [551436,87592] 0 2024-02-16T22:22:20.646 INFO:tasks.workunit.client.0.smithi114.stdout:3/975: write d4/d24/d2d/df0/f86 [6057011,59471] 0 2024-02-16T22:22:20.647 INFO:tasks.workunit.client.0.smithi114.stdout:3/976: chown d4/d24/def/d136/f13c 0 1 2024-02-16T22:22:20.649 INFO:tasks.workunit.client.0.smithi114.stdout:3/977: dread d4/d43/f8a [0,4194304] 0 2024-02-16T22:22:20.652 INFO:tasks.workunit.client.0.smithi114.stdout:3/978: rename d4/dd/l3b to d4/d46/dcf/d139/l155 0 2024-02-16T22:22:20.656 INFO:tasks.workunit.client.0.smithi114.stdout:3/979: link d4/d7e/f14c d4/d24/d2d/df0/d77/f156 0 2024-02-16T22:22:20.657 INFO:tasks.workunit.client.0.smithi114.stdout:3/980: dread - d4/d87/f12b zero size 2024-02-16T22:22:20.658 INFO:tasks.workunit.client.0.smithi114.stdout:3/981: write d4/d26/dd7/d104/d80/f114 [776378,1191] 0 2024-02-16T22:22:20.659 INFO:tasks.workunit.client.0.smithi114.stdout:3/982: fdatasync d4/d43/f8a 0 2024-02-16T22:22:20.666 INFO:tasks.workunit.client.0.smithi114.stdout:3/983: dread d4/d26/dd7/d104/d80/dbd/fc8 [0,4194304] 0 2024-02-16T22:22:20.668 INFO:tasks.workunit.client.0.smithi114.stdout:3/984: chown d4/d46/dcf/d139/f96 102 1 2024-02-16T22:22:20.689 INFO:tasks.workunit.client.0.smithi114.stdout:3/985: dread d4/d26/dd7/d104/d80/dbd/ffb [0,4194304] 0 2024-02-16T22:22:20.715 INFO:tasks.workunit.client.0.smithi114.stdout:3/986: dwrite d4/d7e/f14c [0,4194304] 0 2024-02-16T22:22:20.727 INFO:tasks.workunit.client.0.smithi114.stdout:3/987: getdents d4/d26/dd7/d104 0 2024-02-16T22:22:21.010 INFO:tasks.workunit.client.0.smithi114.stdout:0/934: sync 2024-02-16T22:22:21.010 INFO:tasks.workunit.client.0.smithi114.stdout:0/935: fsync d1/d16/dbf/f12c 0 2024-02-16T22:22:21.047 INFO:tasks.workunit.client.0.smithi114.stdout:2/941: sync 2024-02-16T22:22:21.047 INFO:tasks.workunit.client.0.smithi114.stdout:9/979: sync 2024-02-16T22:22:21.047 INFO:tasks.workunit.client.0.smithi114.stdout:1/848: sync 2024-02-16T22:22:21.049 INFO:tasks.workunit.client.0.smithi114.stdout:9/980: write d2/dd/d132/f142 [273239,6428] 0 2024-02-16T22:22:21.097 INFO:tasks.workunit.client.0.smithi114.stdout:9/981: truncate d2/d5c/dc9/d47/d63/dbb/fff 2035076 0 2024-02-16T22:22:21.099 INFO:tasks.workunit.client.0.smithi114.stdout:2/942: mkdir dd/d3b/da7/d131 0 2024-02-16T22:22:21.103 INFO:tasks.workunit.client.0.smithi114.stdout:2/943: mknod dd/df/d1c/d21/d61/d69/de9/d114/c132 0 2024-02-16T22:22:21.104 INFO:tasks.workunit.client.0.smithi114.stdout:2/944: chown dd/df/d1c/d21/d61/d69/f72 1468 1 2024-02-16T22:22:21.107 INFO:tasks.workunit.client.0.smithi114.stdout:2/945: mkdir dd/d3b/da7/d8b/d133 0 2024-02-16T22:22:21.109 INFO:tasks.workunit.client.0.smithi114.stdout:9/982: dread d2/d5c/d5f/d91/de4/fed [0,4194304] 0 2024-02-16T22:22:21.114 INFO:tasks.workunit.client.0.smithi114.stdout:2/946: dwrite dd/da6/f126 [0,4194304] 0 2024-02-16T22:22:21.115 INFO:tasks.workunit.client.0.smithi114.stdout:2/947: fdatasync dd/d3b/da7/d4c/dc7/f130 0 2024-02-16T22:22:21.125 INFO:tasks.workunit.client.0.smithi114.stdout:2/948: dwrite dd/df/f33 [0,4194304] 0 2024-02-16T22:22:21.152 INFO:tasks.workunit.client.0.smithi114.stdout:7/971: sync 2024-02-16T22:22:21.152 INFO:tasks.workunit.client.0.smithi114.stdout:6/900: sync 2024-02-16T22:22:21.152 INFO:tasks.workunit.client.0.smithi114.stdout:8/861: sync 2024-02-16T22:22:21.153 INFO:tasks.workunit.client.0.smithi114.stdout:8/862: chown d4/d16/d25/d46/le3 77 1 2024-02-16T22:22:21.163 INFO:tasks.workunit.client.0.smithi114.stdout:0/936: rename d1/d16/d1f/d28/d51/d61/d7e/d90/d95/d80/lbe to d1/d1a/d47/d62/d7f/d8c/l12f 0 2024-02-16T22:22:21.165 INFO:tasks.workunit.client.0.smithi114.stdout:9/983: rename d2/d5c/dc9/d47/d63/d98/fd7 to d2/dd/d3c/d89/d8b/f146 0 2024-02-16T22:22:21.169 INFO:tasks.workunit.client.0.smithi114.stdout:9/984: dread d2/d1a/d23/d32/d46/f9d [0,4194304] 0 2024-02-16T22:22:21.170 INFO:tasks.workunit.client.0.smithi114.stdout:6/901: rename d6/dd/d25/cb1 to d6/d2c/d66/c10e 0 2024-02-16T22:22:21.171 INFO:tasks.workunit.client.0.smithi114.stdout:0/937: dread d1/d16/d1f/d28/d51/d61/f87 [0,4194304] 0 2024-02-16T22:22:21.174 INFO:tasks.workunit.client.0.smithi114.stdout:9/985: creat d2/d5c/dc9/d13f/f147 x:0 0 0 2024-02-16T22:22:21.175 INFO:tasks.workunit.client.0.smithi114.stdout:0/938: creat d1/d1a/d47/d62/d7f/d8c/f130 x:0 0 0 2024-02-16T22:22:21.178 INFO:tasks.workunit.client.0.smithi114.stdout:9/986: mknod d2/d5c/d5f/d72/dd6/d12c/c148 0 2024-02-16T22:22:21.179 INFO:tasks.workunit.client.0.smithi114.stdout:0/939: mkdir d1/d16/d48/d118/d131 0 2024-02-16T22:22:21.181 INFO:tasks.workunit.client.0.smithi114.stdout:6/902: rmdir d6/dd/d25/d34/d58/dfd 0 2024-02-16T22:22:21.183 INFO:tasks.workunit.client.0.smithi114.stdout:6/903: creat d6/d2c/d38/d54/f10f x:0 0 0 2024-02-16T22:22:21.195 INFO:tasks.workunit.client.0.smithi114.stdout:9/987: symlink d2/d33/d3f/l149 0 2024-02-16T22:22:21.197 INFO:tasks.workunit.client.0.smithi114.stdout:9/988: mknod d2/dd/d3c/d89/d8b/c14a 0 2024-02-16T22:22:21.200 INFO:tasks.workunit.client.0.smithi114.stdout:9/989: read d2/d5c/d5f/d91/f97 [3004035,127575] 0 2024-02-16T22:22:21.204 INFO:tasks.workunit.client.0.smithi114.stdout:8/863: mkdir d4/d11/d9d/d12e 0 2024-02-16T22:22:21.205 INFO:tasks.workunit.client.0.smithi114.stdout:8/864: dread - d4/d16/d25/fe6 zero size 2024-02-16T22:22:21.248 INFO:tasks.workunit.client.0.smithi114.stdout:7/972: write d3/f27 [7384325,128082] 0 2024-02-16T22:22:21.249 INFO:tasks.workunit.client.0.smithi114.stdout:7/973: readlink d3/d22/d73/da7/db4/d146/da6/l104 0 2024-02-16T22:22:21.250 INFO:tasks.workunit.client.0.smithi114.stdout:7/974: stat d3/d92 0 2024-02-16T22:22:21.253 INFO:tasks.workunit.client.0.smithi114.stdout:7/975: creat d3/d22/df6/f154 x:0 0 0 2024-02-16T22:22:21.265 INFO:tasks.workunit.client.0.smithi114.stdout:0/940: dwrite d1/f6c [0,4194304] 0 2024-02-16T22:22:21.265 INFO:tasks.workunit.client.0.smithi114.stdout:6/904: dwrite d6/dd/d25/d34/d61/d62/f8d [0,4194304] 0 2024-02-16T22:22:21.267 INFO:tasks.workunit.client.0.smithi114.stdout:6/905: write d6/d2c/d38/d54/f77 [5556274,20918] 0 2024-02-16T22:22:21.270 INFO:tasks.workunit.client.0.smithi114.stdout:8/865: write d4/d11/d78/d7a/d31/fa5 [1179808,111812] 0 2024-02-16T22:22:21.273 INFO:tasks.workunit.client.0.smithi114.stdout:0/941: mknod d1/d16/d1f/d28/d51/d61/df3/c132 0 2024-02-16T22:22:21.275 INFO:tasks.workunit.client.0.smithi114.stdout:9/990: link d2/d5c/dc9/d1d/l135 d2/d5c/dc9/d47/l14b 0 2024-02-16T22:22:21.284 INFO:tasks.workunit.client.0.smithi114.stdout:9/991: link d2/dd/d3c/d89/d9e/lfe d2/d5c/dc9/d47/l14c 0 2024-02-16T22:22:21.290 INFO:tasks.workunit.client.0.smithi114.stdout:6/906: mknod d6/d2c/d38/c110 0 2024-02-16T22:22:21.302 INFO:tasks.workunit.client.0.smithi114.stdout:6/907: dread d6/dd/d25/f4c [0,4194304] 0 2024-02-16T22:22:21.302 INFO:tasks.workunit.client.0.smithi114.stdout:7/976: dwrite d3/d22/db7/dea/ff4 [0,4194304] 0 2024-02-16T22:22:21.303 INFO:tasks.workunit.client.0.smithi114.stdout:0/942: write d1/d16/d1f/d28/d51/d61/d7e/d90/fac [865727,123629] 0 2024-02-16T22:22:21.308 INFO:tasks.workunit.client.0.smithi114.stdout:8/866: dwrite f0 [0,4194304] 0 2024-02-16T22:22:21.309 INFO:tasks.workunit.client.0.smithi114.stdout:9/992: write d2/d5c/dc9/d47/d63/fa8 [2175059,41770] 0 2024-02-16T22:22:21.314 INFO:tasks.workunit.client.0.smithi114.stdout:0/943: dread d1/d16/d1f/d28/d51/d60/d52/fc7 [0,4194304] 0 2024-02-16T22:22:21.317 INFO:tasks.workunit.client.0.smithi114.stdout:1/849: sync 2024-02-16T22:22:21.317 INFO:tasks.workunit.client.0.smithi114.stdout:5/846: sync 2024-02-16T22:22:21.317 INFO:tasks.workunit.client.0.smithi114.stdout:3/988: sync 2024-02-16T22:22:21.319 INFO:tasks.workunit.client.0.smithi114.stdout:7/977: dread d3/d22/d73/da7/fbc [0,4194304] 0 2024-02-16T22:22:21.320 INFO:tasks.workunit.client.0.smithi114.stdout:9/993: symlink d2/d1a/d23/d32/d107/l14d 0 2024-02-16T22:22:21.323 INFO:tasks.workunit.client.0.smithi114.stdout:6/908: creat d6/d2c/d38/d41/d60/f111 x:0 0 0 2024-02-16T22:22:21.324 INFO:tasks.workunit.client.0.smithi114.stdout:3/989: readlink d4/d7e/l10b 0 2024-02-16T22:22:21.328 INFO:tasks.workunit.client.0.smithi114.stdout:7/978: creat d3/d22/d73/da7/db4/d146/da6/dfc/f155 x:0 0 0 2024-02-16T22:22:21.329 INFO:tasks.workunit.client.0.smithi114.stdout:1/850: dread d4/d63/d69/d7c/dc0/ff4 [0,4194304] 0 2024-02-16T22:22:21.330 INFO:tasks.workunit.client.0.smithi114.stdout:7/979: chown d3/d22/db7/dea/ff4 14859 1 2024-02-16T22:22:21.331 INFO:tasks.workunit.client.0.smithi114.stdout:0/944: truncate d1/d1a/f2e 7410443 0 2024-02-16T22:22:21.332 INFO:tasks.workunit.client.0.smithi114.stdout:0/945: dread - d1/d16/dbf/f12c zero size 2024-02-16T22:22:21.335 INFO:tasks.workunit.client.0.smithi114.stdout:8/867: creat d4/d16/d25/f12f x:0 0 0 2024-02-16T22:22:21.336 INFO:tasks.workunit.client.0.smithi114.stdout:6/909: creat d6/dd/d25/d34/f112 x:0 0 0 2024-02-16T22:22:21.338 INFO:tasks.workunit.client.0.smithi114.stdout:5/847: dwrite d0/d28/d98/d1e/f21 [0,4194304] 0 2024-02-16T22:22:21.339 INFO:tasks.workunit.client.0.smithi114.stdout:5/848: write d0/d28/d98/d1e/d63/d96/f113 [723793,31220] 0 2024-02-16T22:22:21.341 INFO:tasks.workunit.client.0.smithi114.stdout:3/990: rename d4/d24/fd4 to d4/d46/dcf/d139/f157 0 2024-02-16T22:22:21.352 INFO:tasks.workunit.client.0.smithi114.stdout:6/910: mknod d6/d2c/d75/dfe/c113 0 2024-02-16T22:22:21.356 INFO:tasks.workunit.client.0.smithi114.stdout:2/949: sync 2024-02-16T22:22:21.356 INFO:tasks.workunit.client.0.smithi114.stdout:1/851: dwrite d4/d9/d46/d67/d92/dca/dcc/fbc [0,4194304] 0 2024-02-16T22:22:21.357 INFO:tasks.workunit.client.0.smithi114.stdout:0/946: write d1/d1a/ffa [516416,20879] 0 2024-02-16T22:22:21.357 INFO:tasks.workunit.client.0.smithi114.stdout:5/849: read d0/d28/d98/d25/d2e/f56 [3102363,2781] 0 2024-02-16T22:22:21.358 INFO:tasks.workunit.client.0.smithi114.stdout:0/947: chown d1/d16/d1f/d28/fe6 45 1 2024-02-16T22:22:21.360 INFO:tasks.workunit.client.0.smithi114.stdout:7/980: rename d3/d22/db7/dea/ff4 to d3/d22/d73/da7/db4/d146/f156 0 2024-02-16T22:22:21.361 INFO:tasks.workunit.client.0.smithi114.stdout:3/991: fdatasync d4/d26/dd7/d104/fd2 0 2024-02-16T22:22:21.363 INFO:tasks.workunit.client.0.smithi114.stdout:9/994: link d2/d1a/d23/lb4 d2/d5c/d5f/d91/de4/l14e 0 2024-02-16T22:22:21.364 INFO:tasks.workunit.client.0.smithi114.stdout:7/981: dread d3/d87/dd1/f139 [0,4194304] 0 2024-02-16T22:22:21.365 INFO:tasks.workunit.client.0.smithi114.stdout:8/868: fdatasync d4/d16/f12b 0 2024-02-16T22:22:21.367 INFO:tasks.workunit.client.0.smithi114.stdout:6/911: mkdir d6/d2c/d38/da8/d101/d114 0 2024-02-16T22:22:21.370 INFO:tasks.workunit.client.0.smithi114.stdout:1/852: readlink d4/d9/d46/ld8 0 2024-02-16T22:22:21.370 INFO:tasks.workunit.client.0.smithi114.stdout:2/950: dread - dd/df/d1c/d21/d61/f102 zero size 2024-02-16T22:22:21.370 INFO:tasks.workunit.client.0.smithi114.stdout:9/995: dwrite d2/dd/d132/f133 [0,4194304] 0 2024-02-16T22:22:21.372 INFO:tasks.workunit.client.0.smithi114.stdout:9/996: chown d2/dd/d3c/f7f 1420 1 2024-02-16T22:22:21.372 INFO:tasks.workunit.client.0.smithi114.stdout:2/951: dread - dd/d4e/f11e zero size 2024-02-16T22:22:21.375 INFO:tasks.workunit.client.0.smithi114.stdout:0/948: rmdir d1/d1a/d47/d62/d89 39 2024-02-16T22:22:21.377 INFO:tasks.workunit.client.0.smithi114.stdout:1/853: dread d4/d9/d46/d67/d92/dca/dcc/fbc [0,4194304] 0 2024-02-16T22:22:21.382 INFO:tasks.workunit.client.0.smithi114.stdout:3/992: unlink d4/d26/d91/le7 0 2024-02-16T22:22:21.382 INFO:tasks.workunit.client.0.smithi114.stdout:5/850: dread d0/d28/d98/d1e/d63/d96/d9a/fd6 [0,4194304] 0 2024-02-16T22:22:21.390 INFO:tasks.workunit.client.0.smithi114.stdout:2/952: read dd/df/d1c/fa4 [549924,91284] 0 2024-02-16T22:22:21.391 INFO:tasks.workunit.client.0.smithi114.stdout:9/997: dread d2/dd/f3b [0,4194304] 0 2024-02-16T22:22:21.392 INFO:tasks.workunit.client.0.smithi114.stdout:0/949: creat d1/d16/d1f/d28/d51/d61/d7e/d90/d95/f133 x:0 0 0 2024-02-16T22:22:21.393 INFO:tasks.workunit.client.0.smithi114.stdout:1/854: truncate d4/d9/d46/d67/d92/dca/dcc/d98/fcd 5206404 0 2024-02-16T22:22:21.399 INFO:tasks.workunit.client.0.smithi114.stdout:7/982: dwrite d3/d87/fca [0,4194304] 0 2024-02-16T22:22:21.402 INFO:tasks.workunit.client.0.smithi114.stdout:3/993: write d4/dd/f64 [734051,82348] 0 2024-02-16T22:22:21.403 INFO:tasks.workunit.client.0.smithi114.stdout:2/953: truncate dd/d3b/d8a/f9a 69331 0 2024-02-16T22:22:21.403 INFO:tasks.workunit.client.0.smithi114.stdout:5/851: write d0/d28/d98/d25/f73 [115981,49808] 0 2024-02-16T22:22:21.404 INFO:tasks.workunit.client.0.smithi114.stdout:1/855: dwrite d4/d63/d69/d7c/d9b/dae/fe8 [0,4194304] 0 2024-02-16T22:22:21.404 INFO:tasks.workunit.client.0.smithi114.stdout:8/869: dwrite d4/d11/f17 [0,4194304] 0 2024-02-16T22:22:21.410 INFO:tasks.workunit.client.0.smithi114.stdout:9/998: creat d2/d5c/dc9/d47/f14f x:0 0 0 2024-02-16T22:22:21.414 INFO:tasks.workunit.client.0.smithi114.stdout:7/983: truncate d3/d92/dbe/f111 174978 0 2024-02-16T22:22:21.418 INFO:tasks.workunit.client.0.smithi114.stdout:2/954: creat dd/df/d1c/d21/d61/d69/f134 x:0 0 0 2024-02-16T22:22:21.419 INFO:tasks.workunit.client.0.smithi114.stdout:5/852: mknod d0/d28/d98/d25/d3e/d7f/c117 0 2024-02-16T22:22:21.421 INFO:tasks.workunit.client.0.smithi114.stdout:9/999: symlink d2/d5c/dc9/d47/d63/d98/l150 0 2024-02-16T22:22:21.423 INFO:tasks.workunit.client.0.smithi114.stdout:6/912: link d6/c32 d6/d2c/d38/d41/d60/d84/c115 0 2024-02-16T22:22:21.423 INFO:tasks.workunit.client.0.smithi114.stdout:6/913: dread - d6/da/f100 zero size 2024-02-16T22:22:21.429 INFO:tasks.workunit.client.0.smithi114.stdout:6/914: unlink d6/dd/d25/l71 0 2024-02-16T22:22:21.432 INFO:tasks.workunit.client.0.smithi114.stdout:3/994: creat d4/d26/dd7/f158 x:0 0 0 2024-02-16T22:22:21.433 INFO:tasks.workunit.client.0.smithi114.stdout:3/995: rename d4/d26 to d4/d26/dd7/d104/d75/d159 22 2024-02-16T22:22:21.438 INFO:tasks.workunit.client.0.smithi114.stdout:6/915: creat d6/dd/d25/d34/f116 x:0 0 0 2024-02-16T22:22:21.441 INFO:tasks.workunit.client.0.smithi114.stdout:1/856: link d4/d9/l17 d4/d9/d46/d67/d92/dca/dce/d11b/l134 0 2024-02-16T22:22:21.442 INFO:tasks.workunit.client.0.smithi114.stdout:7/984: truncate d3/d87/fe4 1189920 0 2024-02-16T22:22:21.442 INFO:tasks.workunit.client.0.smithi114.stdout:0/950: dread d1/f7 [0,4194304] 0 2024-02-16T22:22:21.443 INFO:tasks.workunit.client.0.smithi114.stdout:1/857: chown d4/d9/d3d/d3f/d79/l10f 53189478 1 2024-02-16T22:22:21.443 INFO:tasks.workunit.client.0.smithi114.stdout:3/996: write d4/d26/dd7/d104/d80/dbd/fc8 [1575253,16881] 0 2024-02-16T22:22:21.445 INFO:tasks.workunit.client.0.smithi114.stdout:8/870: getdents d4/d24 0 2024-02-16T22:22:21.447 INFO:tasks.workunit.client.0.smithi114.stdout:6/916: unlink d6/d2c/f6d 0 2024-02-16T22:22:21.449 INFO:tasks.workunit.client.0.smithi114.stdout:7/985: mkdir d3/d22/d73/da7/d6f/d9c/d157 0 2024-02-16T22:22:21.451 INFO:tasks.workunit.client.0.smithi114.stdout:0/951: rename d1/d1a/d47/l5a to d1/d16/d1f/d28/d51/d61/d6d/df1/dd4/l134 0 2024-02-16T22:22:21.454 INFO:tasks.workunit.client.0.smithi114.stdout:7/986: rmdir d3/d22/d61 39 2024-02-16T22:22:21.463 INFO:tasks.workunit.client.0.smithi114.stdout:1/858: rename d4/d63/d69/d7c/l105 to d4/d63/d69/d7c/d9b/db0/df3/l135 0 2024-02-16T22:22:21.466 INFO:tasks.workunit.client.0.smithi114.stdout:0/952: mkdir d1/d16/d1f/d28/d51/d61/d6d/d6e/d135 0 2024-02-16T22:22:21.467 INFO:tasks.workunit.client.0.smithi114.stdout:0/953: dread - d1/d1a/d47/d62/d7f/d8c/f130 zero size 2024-02-16T22:22:21.468 INFO:tasks.workunit.client.0.smithi114.stdout:7/987: fdatasync d3/d22/d73/da7/db4/d146/da6/dfc/f109 0 2024-02-16T22:22:21.470 INFO:tasks.workunit.client.0.smithi114.stdout:8/871: truncate d4/d16/d25/d46/d10b/fa4 3806523 0 2024-02-16T22:22:21.472 INFO:tasks.workunit.client.0.smithi114.stdout:6/917: read d6/d2c/fba [147053,110458] 0 2024-02-16T22:22:21.473 INFO:tasks.workunit.client.0.smithi114.stdout:6/918: write d6/dd/d25/d34/d61/d62/f8d [5853340,37178] 0 2024-02-16T22:22:21.482 INFO:tasks.workunit.client.0.smithi114.stdout:1/859: rmdir d4/d9/d46/d47 39 2024-02-16T22:22:21.485 INFO:tasks.workunit.client.0.smithi114.stdout:0/954: mkdir d1/d1a/d47/d62/d7f/d9d/d136 0 2024-02-16T22:22:21.486 INFO:tasks.workunit.client.0.smithi114.stdout:3/997: getdents d4/d46/dcf 0 2024-02-16T22:22:21.489 INFO:tasks.workunit.client.0.smithi114.stdout:7/988: creat d3/d22/d73/da7/db4/f158 x:0 0 0 2024-02-16T22:22:21.490 INFO:tasks.workunit.client.0.smithi114.stdout:1/860: dwrite d4/d63/d69/d7c/d9b/db0/f38 [4194304,4194304] 0 2024-02-16T22:22:21.491 INFO:tasks.workunit.client.0.smithi114.stdout:6/919: dread d6/f30 [0,4194304] 0 2024-02-16T22:22:21.494 INFO:tasks.workunit.client.0.smithi114.stdout:0/955: dread - d1/d16/d1f/fb4 zero size 2024-02-16T22:22:21.495 INFO:tasks.workunit.client.0.smithi114.stdout:3/998: mkdir d4/d26/dd7/d104/d75/daa/d15a 0 2024-02-16T22:22:21.496 INFO:tasks.workunit.client.0.smithi114.stdout:8/872: mknod d4/d11/d78/d10c/c130 0 2024-02-16T22:22:21.499 INFO:tasks.workunit.client.0.smithi114.stdout:6/920: rename d6/le to d6/dd/d25/d34/d58/l117 0 2024-02-16T22:22:21.499 INFO:tasks.workunit.client.0.smithi114.stdout:0/956: rename d1/d16/d1f/d28/d51/d61 to d1/d16/d1f/d28/d51/d61/d7e/d11b/d137 22 2024-02-16T22:22:21.500 INFO:tasks.workunit.client.0.smithi114.stdout:1/861: truncate d4/d9/d46/d67/d92/de4/fe7 770949 0 2024-02-16T22:22:21.501 INFO:tasks.workunit.client.0.smithi114.stdout:0/957: write d1/d16/dbf/fc6 [3599769,115285] 0 2024-02-16T22:22:21.505 INFO:tasks.workunit.client.0.smithi114.stdout:8/873: rename d4/d11/d9d to d4/d24/d6d/dc7/d12a/dd1/d131 0 2024-02-16T22:22:21.511 INFO:tasks.workunit.client.0.smithi114.stdout:0/958: creat d1/d1a/d3e/f138 x:0 0 0 2024-02-16T22:22:21.512 INFO:tasks.workunit.client.0.smithi114.stdout:0/959: dread - d1/d16/d1f/d28/d51/d61/d7e/d90/d95/f133 zero size 2024-02-16T22:22:21.513 INFO:tasks.workunit.client.0.smithi114.stdout:8/874: mknod d4/d11/d78/d7a/d36/c132 0 2024-02-16T22:22:21.517 INFO:tasks.workunit.client.0.smithi114.stdout:3/999: creat d4/d24/d2d/df0/f15b x:0 0 0 2024-02-16T22:22:21.517 INFO:tasks.workunit.client.0.smithi114.stdout:7/989: write d3/d14/fa4 [1347753,29139] 0 2024-02-16T22:22:21.518 INFO:tasks.workunit.client.0.smithi114.stdout:6/921: dread d6/dd/d25/d34/d58/d9f/fe4 [0,4194304] 0 2024-02-16T22:22:21.520 INFO:tasks.workunit.client.0.smithi114.stdout:0/960: dread - d1/d16/d1f/d28/d51/d61/d7e/d90/f121 zero size 2024-02-16T22:22:21.521 INFO:tasks.workunit.client.0.smithi114.stdout:8/875: chown d4/d24/f3d 8 1 2024-02-16T22:22:21.521 INFO:tasks.workunit.client.0.smithi114.stdout:0/961: write d1/d1a/d3e/fa3 [218118,57387] 0 2024-02-16T22:22:21.524 INFO:tasks.workunit.client.0.smithi114.stdout:1/862: rename d4/d9/d3d/f8e to d4/d9/d46/d67/d92/f136 0 2024-02-16T22:22:21.525 INFO:tasks.workunit.client.0.smithi114.stdout:1/863: write d4/d63/d69/d7c/d9b/db0/f38 [2134553,85931] 0 2024-02-16T22:22:21.525 INFO:tasks.workunit.client.0.smithi114.stdout:7/990: unlink d3/d46/da2/lcb 0 2024-02-16T22:22:21.527 INFO:tasks.workunit.client.0.smithi114.stdout:7/991: chown d3/d22/d73/f12d 1 1 2024-02-16T22:22:21.530 INFO:tasks.workunit.client.0.smithi114.stdout:8/876: creat d4/d24/d6d/f133 x:0 0 0 2024-02-16T22:22:21.531 INFO:tasks.workunit.client.0.smithi114.stdout:0/962: write d1/d1a/d47/d62/d89/dba/f112 [29737,664] 0 2024-02-16T22:22:21.533 INFO:tasks.workunit.client.0.smithi114.stdout:1/864: rename d4/d63/d69/d72/fde to d4/d9/d46/d67/d92/dca/dce/dd3/f137 0 2024-02-16T22:22:21.533 INFO:tasks.workunit.client.0.smithi114.stdout:1/865: chown d4/l1f 135 1 2024-02-16T22:22:21.536 INFO:tasks.workunit.client.0.smithi114.stdout:7/992: creat d3/d92/f159 x:0 0 0 2024-02-16T22:22:21.536 INFO:tasks.workunit.client.0.smithi114.stdout:6/922: truncate d6/d2c/d75/fd4 367961 0 2024-02-16T22:22:21.536 INFO:tasks.workunit.client.0.smithi114.stdout:7/993: stat d3/d46/f47 0 2024-02-16T22:22:21.537 INFO:tasks.workunit.client.0.smithi114.stdout:7/994: chown d3/d22/d73/da7/d4e/f13b 319 1 2024-02-16T22:22:21.538 INFO:tasks.workunit.client.0.smithi114.stdout:7/995: readlink d3/d14/l6d 0 2024-02-16T22:22:21.539 INFO:tasks.workunit.client.0.smithi114.stdout:8/877: unlink d4/d16/d25/d46/db5/ff3 0 2024-02-16T22:22:21.546 INFO:tasks.workunit.client.0.smithi114.stdout:0/963: symlink d1/d16/d1f/d28/d51/dca/l139 0 2024-02-16T22:22:21.549 INFO:tasks.workunit.client.0.smithi114.stdout:6/923: mkdir d6/d2c/d5e/d118 0 2024-02-16T22:22:21.550 INFO:tasks.workunit.client.0.smithi114.stdout:7/996: dread d3/d22/d73/da7/d4e/fc0 [0,4194304] 0 2024-02-16T22:22:21.554 INFO:tasks.workunit.client.0.smithi114.stdout:6/924: symlink d6/dd/d25/d34/d58/l119 0 2024-02-16T22:22:21.555 INFO:tasks.workunit.client.0.smithi114.stdout:1/866: dread d4/d9/d46/d47/f11c [4194304,4194304] 0 2024-02-16T22:22:21.563 INFO:tasks.workunit.client.0.smithi114.stdout:8/878: dwrite d4/d24/f64 [0,4194304] 0 2024-02-16T22:22:21.564 INFO:tasks.workunit.client.0.smithi114.stdout:1/867: creat d4/d9/d3d/def/f138 x:0 0 0 2024-02-16T22:22:21.565 INFO:tasks.workunit.client.0.smithi114.stdout:8/879: dread - d4/d16/d25/fe6 zero size 2024-02-16T22:22:21.569 INFO:tasks.workunit.client.0.smithi114.stdout:6/925: truncate d6/d2c/d38/fdc 845414 0 2024-02-16T22:22:21.572 INFO:tasks.workunit.client.0.smithi114.stdout:0/964: dwrite d1/d16/d1f/d28/d51/d61/d7e/d90/f42 [0,4194304] 0 2024-02-16T22:22:21.574 INFO:tasks.workunit.client.0.smithi114.stdout:1/868: truncate d4/f16 4333007 0 2024-02-16T22:22:21.577 INFO:tasks.workunit.client.0.smithi114.stdout:7/997: dwrite d3/d3d/f58 [0,4194304] 0 2024-02-16T22:22:21.589 INFO:tasks.workunit.client.0.smithi114.stdout:8/880: write d4/f8 [3298223,47413] 0 2024-02-16T22:22:21.589 INFO:tasks.workunit.client.0.smithi114.stdout:1/869: getdents d4/d9/d46/d67/d92/dca/d118/d126 0 2024-02-16T22:22:21.593 INFO:tasks.workunit.client.0.smithi114.stdout:7/998: mknod d3/c15a 0 2024-02-16T22:22:21.597 INFO:tasks.workunit.client.0.smithi114.stdout:6/926: dread d6/f3e [0,4194304] 0 2024-02-16T22:22:21.599 INFO:tasks.workunit.client.0.smithi114.stdout:0/965: dwrite d1/d16/d18/f4b [4194304,4194304] 0 2024-02-16T22:22:21.600 INFO:tasks.workunit.client.0.smithi114.stdout:7/999: truncate d3/d22/db7/dea/f110 650833 0 2024-02-16T22:22:21.600 INFO:tasks.workunit.client.0.smithi114.stdout:0/966: chown d1/d1a/d47/d62/d89/dba 12 1 2024-02-16T22:22:21.605 INFO:tasks.workunit.client.0.smithi114.stdout:6/927: rename d6/dd/d25/d34/d61/l97 to d6/dd/d25/d34/l11a 0 2024-02-16T22:22:21.606 INFO:tasks.workunit.client.0.smithi114.stdout:6/928: write d6/da/f56 [5086605,125031] 0 2024-02-16T22:22:21.608 INFO:tasks.workunit.client.0.smithi114.stdout:1/870: write d4/d9/d46/d67/d92/f136 [4792418,42183] 0 2024-02-16T22:22:21.612 INFO:tasks.workunit.client.0.smithi114.stdout:1/871: chown d4/d9/d3d/f88 3 1 2024-02-16T22:22:21.612 INFO:tasks.workunit.client.0.smithi114.stdout:1/872: stat d4/d63/d69/d7c/d9b/df0/f12f 0 2024-02-16T22:22:21.614 INFO:tasks.workunit.client.0.smithi114.stdout:0/967: symlink d1/l13a 0 2024-02-16T22:22:21.615 INFO:tasks.workunit.client.0.smithi114.stdout:0/968: write d1/d1a/d47/d62/fee [4356221,1090] 0 2024-02-16T22:22:21.616 INFO:tasks.workunit.client.0.smithi114.stdout:8/881: link d4/c113 d4/d11/d52/c134 0 2024-02-16T22:22:21.619 INFO:tasks.workunit.client.0.smithi114.stdout:1/873: creat d4/d63/d69/d7c/d9b/df0/d108/f139 x:0 0 0 2024-02-16T22:22:21.622 INFO:tasks.workunit.client.0.smithi114.stdout:0/969: read d1/d1a/f26 [2999530,125117] 0 2024-02-16T22:22:21.623 INFO:tasks.workunit.client.0.smithi114.stdout:8/882: dwrite d4/d24/f3d [0,4194304] 0 2024-02-16T22:22:21.623 INFO:tasks.workunit.client.0.smithi114.stdout:1/874: stat d4/d9/d3d/d3f/d79/c122 0 2024-02-16T22:22:21.625 INFO:tasks.workunit.client.0.smithi114.stdout:8/883: write d4/d16/d25/f12f [614009,11604] 0 2024-02-16T22:22:21.629 INFO:tasks.workunit.client.0.smithi114.stdout:1/875: rename d4/c2c to d4/d9/d3d/d3f/c13a 0 2024-02-16T22:22:21.633 INFO:tasks.workunit.client.0.smithi114.stdout:1/876: unlink d4/d9/d46/d47/fa4 0 2024-02-16T22:22:21.634 INFO:tasks.workunit.client.0.smithi114.stdout:6/929: write d6/d2c/d38/d41/d60/f98 [5114558,70419] 0 2024-02-16T22:22:21.637 INFO:tasks.workunit.client.0.smithi114.stdout:6/930: rmdir d6/dd/d25/d34/d58/d9f/da3 39 2024-02-16T22:22:21.638 INFO:tasks.workunit.client.0.smithi114.stdout:6/931: stat d6/d2c/d38/da8/cc7 0 2024-02-16T22:22:21.641 INFO:tasks.workunit.client.0.smithi114.stdout:8/884: getdents d4/d11/d78/d7a/d31/dc6 0 2024-02-16T22:22:21.644 INFO:tasks.workunit.client.0.smithi114.stdout:6/932: creat d6/d2c/d38/da8/f11b x:0 0 0 2024-02-16T22:22:21.645 INFO:tasks.workunit.client.0.smithi114.stdout:0/970: dwrite d1/d16/d18/f1c [0,4194304] 0 2024-02-16T22:22:21.646 INFO:tasks.workunit.client.0.smithi114.stdout:0/971: chown d1/d1a/d47/d62/d7f/lfe 96 1 2024-02-16T22:22:21.647 INFO:tasks.workunit.client.0.smithi114.stdout:1/877: rename d4/d9/d46/c52 to d4/d9/d46/d67/d92/dca/dcc/d98/ddd/d113/c13b 0 2024-02-16T22:22:21.651 INFO:tasks.workunit.client.0.smithi114.stdout:8/885: mkdir d4/d11/d78/d7a/d31/dc6/d135 0 2024-02-16T22:22:21.653 INFO:tasks.workunit.client.0.smithi114.stdout:6/933: dwrite d6/da/faa [0,4194304] 0 2024-02-16T22:22:21.653 INFO:tasks.workunit.client.0.smithi114.stdout:1/878: symlink d4/d9/d46/d67/d92/dca/dce/l13c 0 2024-02-16T22:22:21.656 INFO:tasks.workunit.client.0.smithi114.stdout:0/972: mkdir d1/d16/d1f/d28/d51/d61/d6d/d6e/d117/d13b 0 2024-02-16T22:22:21.657 INFO:tasks.workunit.client.0.smithi114.stdout:8/886: creat d4/d11/d78/d7a/d36/d3b/f136 x:0 0 0 2024-02-16T22:22:21.661 INFO:tasks.workunit.client.0.smithi114.stdout:0/973: symlink d1/d16/d1f/d28/d51/d61/d6d/d6e/l13c 0 2024-02-16T22:22:21.662 INFO:tasks.workunit.client.0.smithi114.stdout:8/887: fdatasync d4/d11/d78/f90 0 2024-02-16T22:22:21.664 INFO:tasks.workunit.client.0.smithi114.stdout:8/888: write d4/d11/d78/d7a/d36/fb9 [2029341,90787] 0 2024-02-16T22:22:21.670 INFO:tasks.workunit.client.0.smithi114.stdout:0/974: symlink d1/d16/d48/d118/d131/l13d 0 2024-02-16T22:22:21.671 INFO:tasks.workunit.client.0.smithi114.stdout:6/934: dwrite d6/d2c/d38/d54/f10f [0,4194304] 0 2024-02-16T22:22:21.672 INFO:tasks.workunit.client.0.smithi114.stdout:6/935: dread - d6/dd/d25/d34/f112 zero size 2024-02-16T22:22:21.684 INFO:tasks.workunit.client.0.smithi114.stdout:1/879: unlink d4/d9/d46/d67/d92/dca/dcc/d98/ddd/d113/c13b 0 2024-02-16T22:22:21.685 INFO:tasks.workunit.client.0.smithi114.stdout:6/936: fsync d6/d2c/d38/d54/f2d 0 2024-02-16T22:22:21.685 INFO:tasks.workunit.client.0.smithi114.stdout:8/889: link d4/d16/d25/d46/db5/dca/f114 d4/d24/d6d/dc7/d12a/dd1/d131/d12e/f137 0 2024-02-16T22:22:21.686 INFO:tasks.workunit.client.0.smithi114.stdout:6/937: write d6/dd/f17 [7997040,63349] 0 2024-02-16T22:22:21.687 INFO:tasks.workunit.client.0.smithi114.stdout:6/938: chown d6/dd/d25/d34/d58/faf 118873527 1 2024-02-16T22:22:21.699 INFO:tasks.workunit.client.0.smithi114.stdout:0/975: dwrite d1/d78/f7d [0,4194304] 0 2024-02-16T22:22:21.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:21 smithi114 ceph-mon[29235]: pgmap v38: 65 pgs: 65 active+clean; 3.8 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 77 MiB/s rd, 157 MiB/s wr, 389 op/s 2024-02-16T22:22:21.717 INFO:tasks.workunit.client.0.smithi114.stdout:8/890: mkdir d4/d11/d78/d10c/d138 0 2024-02-16T22:22:21.718 INFO:tasks.workunit.client.0.smithi114.stdout:0/976: mknod d1/d1a/d47/d62/d7f/d9d/df2/c13e 0 2024-02-16T22:22:21.719 INFO:tasks.workunit.client.0.smithi114.stdout:1/880: mkdir d4/d13d 0 2024-02-16T22:22:21.721 INFO:tasks.workunit.client.0.smithi114.stdout:8/891: dread d4/d11/f71 [0,4194304] 0 2024-02-16T22:22:21.721 INFO:tasks.workunit.client.0.smithi114.stdout:6/939: symlink d6/dd/d25/d34/d58/d9f/da3/dbd/l11c 0 2024-02-16T22:22:21.722 INFO:tasks.workunit.client.0.smithi114.stdout:0/977: creat d1/d1a/d47/d62/d7f/d8c/f13f x:0 0 0 2024-02-16T22:22:21.724 INFO:tasks.workunit.client.0.smithi114.stdout:1/881: mkdir d4/d9/d46/d4d/d13e 0 2024-02-16T22:22:21.727 INFO:tasks.workunit.client.0.smithi114.stdout:0/978: mkdir d1/d1a/d47/d62/d89/dba/d140 0 2024-02-16T22:22:21.728 INFO:tasks.workunit.client.0.smithi114.stdout:0/979: readlink d1/d16/d1f/d28/d51/d61/d6d/df1/l110 0 2024-02-16T22:22:21.739 INFO:tasks.workunit.client.0.smithi114.stdout:0/980: mknod d1/d16/d1f/d28/d51/d60/c141 0 2024-02-16T22:22:21.743 INFO:tasks.workunit.client.0.smithi114.stdout:0/981: creat d1/d16/d1f/d28/d51/d61/d6d/d6e/d135/f142 x:0 0 0 2024-02-16T22:22:21.746 INFO:tasks.workunit.client.0.smithi114.stdout:0/982: creat d1/d1a/d47/d62/d89/dba/d140/f143 x:0 0 0 2024-02-16T22:22:21.749 INFO:tasks.workunit.client.0.smithi114.stdout:0/983: rename d1/d16/d1f/d28/c4f to d1/d16/d18/c144 0 2024-02-16T22:22:21.752 INFO:tasks.workunit.client.0.smithi114.stdout:0/984: creat d1/d16/d1f/d28/d51/d61/d6d/d6e/de2/f145 x:0 0 0 2024-02-16T22:22:21.755 INFO:tasks.workunit.client.0.smithi114.stdout:0/985: dread d1/d16/d1f/d28/d51/d60/d52/fd0 [0,4194304] 0 2024-02-16T22:22:21.756 INFO:tasks.workunit.client.0.smithi114.stdout:0/986: write d1/d1a/d47/d62/d89/dba/d140/f143 [955936,106391] 0 2024-02-16T22:22:21.761 INFO:tasks.workunit.client.0.smithi114.stdout:8/892: write d4/d24/d2c/fd9 [1389478,10843] 0 2024-02-16T22:22:21.766 INFO:tasks.workunit.client.0.smithi114.stdout:6/940: dwrite d6/d83/fcb [0,4194304] 0 2024-02-16T22:22:21.768 INFO:tasks.workunit.client.0.smithi114.stdout:1/882: dwrite d4/d9/f8f [0,4194304] 0 2024-02-16T22:22:21.772 INFO:tasks.workunit.client.0.smithi114.stdout:6/941: mknod d6/dd/d25/d34/d61/d62/d8b/db7/dd3/c11d 0 2024-02-16T22:22:21.776 INFO:tasks.workunit.client.0.smithi114.stdout:1/883: mkdir d4/d9/d46/d67/d92/dca/dcc/d98/ddd/df2/d13f 0 2024-02-16T22:22:21.777 INFO:tasks.workunit.client.0.smithi114.stdout:6/942: mkdir d6/d11e 0 2024-02-16T22:22:21.781 INFO:tasks.workunit.client.0.smithi114.stdout:6/943: symlink d6/dd/d25/d34/l11f 0 2024-02-16T22:22:21.792 INFO:tasks.workunit.client.0.smithi114.stdout:0/987: dwrite d1/d1a/f26 [0,4194304] 0 2024-02-16T22:22:21.792 INFO:tasks.workunit.client.0.smithi114.stdout:2/955: sync 2024-02-16T22:22:21.798 INFO:tasks.workunit.client.0.smithi114.stdout:0/988: mknod d1/d16/d1f/d28/d51/d61/d6d/de9/c146 0 2024-02-16T22:22:21.799 INFO:tasks.workunit.client.0.smithi114.stdout:0/989: chown d1/d1a/d47/d62/d89/cf6 170748 1 2024-02-16T22:22:21.800 INFO:tasks.workunit.client.0.smithi114.stdout:0/990: readlink d1/d16/d18/l20 0 2024-02-16T22:22:21.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:21 smithi123 ceph-mon[37161]: pgmap v38: 65 pgs: 65 active+clean; 3.8 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 77 MiB/s rd, 157 MiB/s wr, 389 op/s 2024-02-16T22:22:21.803 INFO:tasks.workunit.client.0.smithi114.stdout:0/991: creat d1/d16/d1f/d28/d51/d61/d6d/f147 x:0 0 0 2024-02-16T22:22:21.805 INFO:tasks.workunit.client.0.smithi114.stdout:8/893: write d4/d11/d78/f86 [50988,125928] 0 2024-02-16T22:22:21.811 INFO:tasks.workunit.client.0.smithi114.stdout:1/884: write d4/f25 [2683160,56953] 0 2024-02-16T22:22:21.813 INFO:tasks.workunit.client.0.smithi114.stdout:1/885: fsync d4/d9/d46/d67/d92/f136 0 2024-02-16T22:22:21.813 INFO:tasks.workunit.client.0.smithi114.stdout:6/944: getdents d6/dd/d25/d34 0 2024-02-16T22:22:21.814 INFO:tasks.workunit.client.0.smithi114.stdout:6/945: chown d6/d2c/d38/l50 5 1 2024-02-16T22:22:21.817 INFO:tasks.workunit.client.0.smithi114.stdout:1/886: mkdir d4/d9/d46/d67/d92/dca/d118/d126/d140 0 2024-02-16T22:22:21.820 INFO:tasks.workunit.client.0.smithi114.stdout:1/887: chown d4/d63/d69/d7c/dc0/fe9 33207 1 2024-02-16T22:22:21.821 INFO:tasks.workunit.client.0.smithi114.stdout:0/992: dread d1/d1a/f45 [4194304,4194304] 0 2024-02-16T22:22:21.821 INFO:tasks.workunit.client.0.smithi114.stdout:6/946: rename d6/dd/d25/d34/d61/d62/fe3 to d6/dd/d25/d34/d58/d9f/f120 0 2024-02-16T22:22:21.824 INFO:tasks.workunit.client.0.smithi114.stdout:6/947: creat d6/d2c/d38/da8/d101/f121 x:0 0 0 2024-02-16T22:22:21.826 INFO:tasks.workunit.client.0.smithi114.stdout:1/888: getdents d4/d63/d69/d72/d82 0 2024-02-16T22:22:21.828 INFO:tasks.workunit.client.0.smithi114.stdout:6/948: fdatasync d6/dd/d25/d34/f6f 0 2024-02-16T22:22:21.829 INFO:tasks.workunit.client.0.smithi114.stdout:0/993: dwrite d1/d1a/d47/d62/d7f/f84 [0,4194304] 0 2024-02-16T22:22:21.834 INFO:tasks.workunit.client.0.smithi114.stdout:0/994: rmdir d1/d16/d1f/d28/d51/dca 39 2024-02-16T22:22:21.837 INFO:tasks.workunit.client.0.smithi114.stdout:2/956: dwrite dd/df/f9e [0,4194304] 0 2024-02-16T22:22:21.838 INFO:tasks.workunit.client.0.smithi114.stdout:0/995: dread d1/d1a/d47/d62/d89/dba/f112 [0,4194304] 0 2024-02-16T22:22:21.840 INFO:tasks.workunit.client.0.smithi114.stdout:1/889: link d4/d9/d3d/d3f/f75 d4/d9/d46/d67/d92/dca/dcc/d70/f141 0 2024-02-16T22:22:21.841 INFO:tasks.workunit.client.0.smithi114.stdout:1/890: chown d4/d63/d69/d7c/d9b/db0/d74/cd4 19638781 1 2024-02-16T22:22:21.844 INFO:tasks.workunit.client.0.smithi114.stdout:2/957: truncate dd/d4e/f9c 6136299 0 2024-02-16T22:22:21.846 INFO:tasks.workunit.client.0.smithi114.stdout:2/958: chown dd/df/d1c/d21/d61/d63/dc2/f103 36 1 2024-02-16T22:22:21.847 INFO:tasks.workunit.client.0.smithi114.stdout:1/891: rename d4/d63/d69/d7c/d9b/db0/c35 to d4/d9/d3d/d3f/dd0/c142 0 2024-02-16T22:22:21.849 INFO:tasks.workunit.client.0.smithi114.stdout:1/892: symlink d4/d63/d69/d7c/d9b/l143 0 2024-02-16T22:22:21.851 INFO:tasks.workunit.client.0.smithi114.stdout:1/893: write d4/d63/d69/d7c/d9b/dae/fe8 [1896978,40859] 0 2024-02-16T22:22:21.856 INFO:tasks.workunit.client.0.smithi114.stdout:1/894: dread d4/f91 [0,4194304] 0 2024-02-16T22:22:21.857 INFO:tasks.workunit.client.0.smithi114.stdout:8/894: write d4/d16/d25/fde [837931,28705] 0 2024-02-16T22:22:21.861 INFO:tasks.workunit.client.0.smithi114.stdout:5/853: sync 2024-02-16T22:22:21.862 INFO:tasks.workunit.client.0.smithi114.stdout:1/895: creat d4/d9/d46/d67/d92/f144 x:0 0 0 2024-02-16T22:22:21.866 INFO:tasks.workunit.client.0.smithi114.stdout:5/854: creat d0/d28/d98/d25/d89/f118 x:0 0 0 2024-02-16T22:22:21.870 INFO:tasks.workunit.client.0.smithi114.stdout:2/959: dread dd/f5d [0,4194304] 0 2024-02-16T22:22:21.880 INFO:tasks.workunit.client.0.smithi114.stdout:1/896: dread d4/d9/f22 [0,4194304] 0 2024-02-16T22:22:21.883 INFO:tasks.workunit.client.0.smithi114.stdout:1/897: creat d4/d63/d69/d72/d82/f145 x:0 0 0 2024-02-16T22:22:21.887 INFO:tasks.workunit.client.0.smithi114.stdout:1/898: mkdir d4/d63/d69/d7c/d9b/db0/d74/d146 0 2024-02-16T22:22:21.888 INFO:tasks.workunit.client.0.smithi114.stdout:6/949: dwrite d6/dd/d25/d34/fb4 [0,4194304] 0 2024-02-16T22:22:21.893 INFO:tasks.workunit.client.0.smithi114.stdout:1/899: creat d4/d9/d46/d67/d92/dca/dcc/d70/f147 x:0 0 0 2024-02-16T22:22:21.894 INFO:tasks.workunit.client.0.smithi114.stdout:6/950: symlink d6/dd/d25/d34/d58/d9f/da3/dbd/l122 0 2024-02-16T22:22:21.897 INFO:tasks.workunit.client.0.smithi114.stdout:1/900: unlink d4/d9/d46/d67/d92/dca/dcc/fa5 0 2024-02-16T22:22:21.898 INFO:tasks.workunit.client.0.smithi114.stdout:1/901: chown d4/fec 1401 1 2024-02-16T22:22:21.898 INFO:tasks.workunit.client.0.smithi114.stdout:6/951: dread d6/d2c/d38/d41/d60/d84/fc8 [0,4194304] 0 2024-02-16T22:22:21.901 INFO:tasks.workunit.client.0.smithi114.stdout:0/996: write d1/d16/d1f/fb4 [794993,64652] 0 2024-02-16T22:22:21.906 INFO:tasks.workunit.client.0.smithi114.stdout:1/902: creat d4/d63/d69/d7c/d9b/dae/f148 x:0 0 0 2024-02-16T22:22:21.921 INFO:tasks.workunit.client.0.smithi114.stdout:1/903: link d4/c24 d4/d9/d46/d67/d92/dca/dcc/d98/ddd/d113/c149 0 2024-02-16T22:22:21.924 INFO:tasks.workunit.client.0.smithi114.stdout:1/904: symlink d4/d63/d69/d80/l14a 0 2024-02-16T22:22:21.926 INFO:tasks.workunit.client.0.smithi114.stdout:1/905: dread - d4/d63/d69/d72/d82/f145 zero size 2024-02-16T22:22:21.928 INFO:tasks.workunit.client.0.smithi114.stdout:8/895: write d4/d16/d25/d46/d10b/fa4 [2831334,85880] 0 2024-02-16T22:22:21.931 INFO:tasks.workunit.client.0.smithi114.stdout:8/896: read d4/d16/d25/d46/fbd [1508809,56907] 0 2024-02-16T22:22:21.932 INFO:tasks.workunit.client.0.smithi114.stdout:1/906: creat d4/d9/d46/d67/d92/dca/ded/d101/f14b x:0 0 0 2024-02-16T22:22:21.941 INFO:tasks.workunit.client.0.smithi114.stdout:8/897: link d4/d11/d78/d7a/d36/d3b/f121 d4/d24/d6d/dc7/df0/f139 0 2024-02-16T22:22:21.943 INFO:tasks.workunit.client.0.smithi114.stdout:2/960: write dd/df/d1c/d21/d61/d63/d83/fd3 [975846,82094] 0 2024-02-16T22:22:21.947 INFO:tasks.workunit.client.0.smithi114.stdout:8/898: unlink d4/d16/d25/f12f 0 2024-02-16T22:22:21.952 INFO:tasks.workunit.client.0.smithi114.stdout:2/961: symlink dd/df/d1c/d21/d61/d63/dc2/l135 0 2024-02-16T22:22:21.954 INFO:tasks.workunit.client.0.smithi114.stdout:2/962: fdatasync dd/df/d1c/d21/d61/d63/d83/df8/f11b 0 2024-02-16T22:22:21.955 INFO:tasks.workunit.client.0.smithi114.stdout:2/963: chown dd/da6/fc6 1647 1 2024-02-16T22:22:21.957 INFO:tasks.workunit.client.0.smithi114.stdout:2/964: symlink dd/da6/l136 0 2024-02-16T22:22:21.958 INFO:tasks.workunit.client.0.smithi114.stdout:8/899: chown d4/d11/d52/fa9 1852101 1 2024-02-16T22:22:21.964 INFO:tasks.workunit.client.0.smithi114.stdout:2/965: read - dd/dbd/fd9 zero size 2024-02-16T22:22:21.966 INFO:tasks.workunit.client.0.smithi114.stdout:8/900: fsync d4/d11/d78/d7a/d36/d3b/f128 0 2024-02-16T22:22:21.972 INFO:tasks.workunit.client.0.smithi114.stdout:8/901: fdatasync d4/d11/d78/d7a/f1a 0 2024-02-16T22:22:21.974 INFO:tasks.workunit.client.0.smithi114.stdout:2/966: mkdir dd/d6b/d137 0 2024-02-16T22:22:22.161 INFO:tasks.workunit.client.0.smithi114.stdout:0/997: write d1/d1a/f109 [2839573,76134] 0 2024-02-16T22:22:22.162 INFO:tasks.workunit.client.0.smithi114.stdout:8/902: rmdir d4/d24/d6d 39 2024-02-16T22:22:22.166 INFO:tasks.workunit.client.0.smithi114.stdout:2/967: symlink dd/df/d1c/d21/l138 0 2024-02-16T22:22:22.169 INFO:tasks.workunit.client.0.smithi114.stdout:0/998: unlink d1/d1a/d47/d62/d7f/f84 0 2024-02-16T22:22:22.170 INFO:tasks.workunit.client.0.smithi114.stdout:8/903: read d4/d11/d78/d7a/d36/d3b/f44 [1691038,125429] 0 2024-02-16T22:22:22.170 INFO:tasks.workunit.client.0.smithi114.stdout:6/952: dwrite d6/d2c/d75/f82 [0,4194304] 0 2024-02-16T22:22:22.173 INFO:tasks.workunit.client.0.smithi114.stdout:1/907: dwrite d4/d9/d3d/d3f/d79/fc3 [0,4194304] 0 2024-02-16T22:22:22.175 INFO:tasks.workunit.client.0.smithi114.stdout:2/968: symlink dd/df/ddb/l139 0 2024-02-16T22:22:22.176 INFO:tasks.workunit.client.0.smithi114.stdout:6/953: read d6/da/f56 [1224725,99964] 0 2024-02-16T22:22:22.177 INFO:tasks.workunit.client.0.smithi114.stdout:2/969: chown dd/d3b/fab 1 1 2024-02-16T22:22:22.186 INFO:tasks.workunit.client.0.smithi114.stdout:8/904: truncate d4/d11/d78/d7a/d7e/f80 704831 0 2024-02-16T22:22:22.188 INFO:tasks.workunit.client.0.smithi114.stdout:2/970: fsync dd/d3b/d8a/dcb/fb9 0 2024-02-16T22:22:22.192 INFO:tasks.workunit.client.0.smithi114.stdout:8/905: mkdir d4/d16/d25/d46/d10b/d13a 0 2024-02-16T22:22:22.193 INFO:tasks.workunit.client.0.smithi114.stdout:2/971: mknod dd/df/d1c/c13a 0 2024-02-16T22:22:22.197 INFO:tasks.workunit.client.0.smithi114.stdout:2/972: stat dd/df/d1c/d21/d61/d69/c6e 0 2024-02-16T22:22:22.202 INFO:tasks.workunit.client.0.smithi114.stdout:2/973: mkdir dd/d3b/da7/d8b/d111/d13b 0 2024-02-16T22:22:22.202 INFO:tasks.workunit.client.0.smithi114.stdout:1/908: write d4/d9/d46/f10c [747139,28777] 0 2024-02-16T22:22:22.206 INFO:tasks.workunit.client.0.smithi114.stdout:1/909: creat d4/d9/d3d/def/f14c x:0 0 0 2024-02-16T22:22:22.208 INFO:tasks.workunit.client.0.smithi114.stdout:2/974: getdents dd/d6b/d137 0 2024-02-16T22:22:22.210 INFO:tasks.workunit.client.0.smithi114.stdout:0/999: dread d1/d1a/d47/d62/d7f/d9d/f128 [0,4194304] 0 2024-02-16T22:22:22.211 INFO:tasks.workunit.client.0.smithi114.stdout:1/910: mkdir d4/d9/d3d/d3f/d14d 0 2024-02-16T22:22:22.211 INFO:tasks.workunit.client.0.smithi114.stdout:6/954: dread d6/dd/d25/d34/d61/d62/d8b/f94 [0,4194304] 0 2024-02-16T22:22:22.213 INFO:tasks.workunit.client.0.smithi114.stdout:2/975: read dd/f28 [360823,15284] 0 2024-02-16T22:22:22.215 INFO:tasks.workunit.client.0.smithi114.stdout:6/955: mkdir d6/dd/d25/d34/d58/d123 0 2024-02-16T22:22:22.216 INFO:tasks.workunit.client.0.smithi114.stdout:1/911: creat d4/d63/d69/d7c/d9b/db0/d74/f14e x:0 0 0 2024-02-16T22:22:22.216 INFO:tasks.workunit.client.0.smithi114.stdout:6/956: chown d6/d2c/d75/dfe 4046 1 2024-02-16T22:22:22.217 INFO:tasks.workunit.client.0.smithi114.stdout:8/906: dwrite d4/d11/d78/d7a/d36/d3b/f100 [0,4194304] 0 2024-02-16T22:22:22.218 INFO:tasks.workunit.client.0.smithi114.stdout:2/976: truncate dd/d3b/da7/d4c/fb2 2429657 0 2024-02-16T22:22:22.220 INFO:tasks.workunit.client.0.smithi114.stdout:8/907: dread d4/d11/d78/d7a/d7e/fc5 [0,4194304] 0 2024-02-16T22:22:22.225 INFO:tasks.workunit.client.0.smithi114.stdout:2/977: creat dd/d3b/da7/d8b/dc0/f13c x:0 0 0 2024-02-16T22:22:22.228 INFO:tasks.workunit.client.0.smithi114.stdout:5/855: sync 2024-02-16T22:22:22.231 INFO:tasks.workunit.client.0.smithi114.stdout:1/912: mknod d4/d9/d46/d67/d92/dca/dce/d112/c14f 0 2024-02-16T22:22:22.233 INFO:tasks.workunit.client.0.smithi114.stdout:2/978: mkdir dd/d3b/d8a/d13d 0 2024-02-16T22:22:22.234 INFO:tasks.workunit.client.0.smithi114.stdout:2/979: chown dd/df/d1c/d21/c65 56101732 1 2024-02-16T22:22:22.234 INFO:tasks.workunit.client.0.smithi114.stdout:2/980: chown dd/d3b/f104 1579480429 1 2024-02-16T22:22:22.237 INFO:tasks.workunit.client.0.smithi114.stdout:6/957: creat d6/dd/d25/d34/d58/d9f/f124 x:0 0 0 2024-02-16T22:22:22.242 INFO:tasks.workunit.client.0.smithi114.stdout:8/908: creat d4/d24/d6d/dc7/dfe/f13b x:0 0 0 2024-02-16T22:22:22.243 INFO:tasks.workunit.client.0.smithi114.stdout:8/909: chown d4/d16/d25/d39/c6a 32084 1 2024-02-16T22:22:22.246 INFO:tasks.workunit.client.0.smithi114.stdout:1/913: symlink d4/d9/d46/d67/d92/dca/dcc/d98/ddd/l150 0 2024-02-16T22:22:22.247 INFO:tasks.workunit.client.0.smithi114.stdout:2/981: mknod dd/df/d1c/d21/d61/d63/d83/c13e 0 2024-02-16T22:22:22.250 INFO:tasks.workunit.client.0.smithi114.stdout:5/856: write d0/df/d7a/dc4/ff7 [564482,22801] 0 2024-02-16T22:22:22.254 INFO:tasks.workunit.client.0.smithi114.stdout:6/958: write d6/d2c/d38/d41/f4a [759286,110005] 0 2024-02-16T22:22:22.256 INFO:tasks.workunit.client.0.smithi114.stdout:2/982: write dd/df/d1c/d21/d61/d63/fde [1285387,54922] 0 2024-02-16T22:22:22.259 INFO:tasks.workunit.client.0.smithi114.stdout:1/914: write d4/d9/f12 [755994,6579] 0 2024-02-16T22:22:22.261 INFO:tasks.workunit.client.0.smithi114.stdout:2/983: creat dd/d6b/db8/f13f x:0 0 0 2024-02-16T22:22:22.261 INFO:tasks.workunit.client.0.smithi114.stdout:2/984: read - dd/d3b/da7/f125 zero size 2024-02-16T22:22:22.263 INFO:tasks.workunit.client.0.smithi114.stdout:5/857: unlink d0/d28/d98/l48 0 2024-02-16T22:22:22.264 INFO:tasks.workunit.client.0.smithi114.stdout:8/910: rmdir d4/d16/d25/d46/db5/d109 0 2024-02-16T22:22:22.266 INFO:tasks.workunit.client.0.smithi114.stdout:6/959: symlink d6/d2c/d5e/dcc/de7/l125 0 2024-02-16T22:22:22.267 INFO:tasks.workunit.client.0.smithi114.stdout:2/985: mkdir dd/d115/d140 0 2024-02-16T22:22:22.269 INFO:tasks.workunit.client.0.smithi114.stdout:5/858: mkdir d0/df/d53/db1/d119 0 2024-02-16T22:22:22.270 INFO:tasks.workunit.client.0.smithi114.stdout:8/911: symlink d4/d11/d78/d7a/d31/dc6/l13c 0 2024-02-16T22:22:22.270 INFO:tasks.workunit.client.0.smithi114.stdout:5/859: chown d0/d28/d98/d25/d89/de2/db3 14 1 2024-02-16T22:22:22.271 INFO:tasks.workunit.client.0.smithi114.stdout:8/912: fsync d4/d11/f20 0 2024-02-16T22:22:22.271 INFO:tasks.workunit.client.0.smithi114.stdout:1/915: symlink d4/d63/d69/d7c/d9b/db0/d74/d146/l151 0 2024-02-16T22:22:22.273 INFO:tasks.workunit.client.0.smithi114.stdout:1/916: readlink d4/d9/d3d/d3f/l51 0 2024-02-16T22:22:22.273 INFO:tasks.workunit.client.0.smithi114.stdout:6/960: creat d6/dd/d25/d34/d61/d62/f126 x:0 0 0 2024-02-16T22:22:22.274 INFO:tasks.workunit.client.0.smithi114.stdout:6/961: chown d6/d2c/d38/fac 56850365 1 2024-02-16T22:22:22.275 INFO:tasks.workunit.client.0.smithi114.stdout:2/986: creat dd/d6b/da5/df6/f141 x:0 0 0 2024-02-16T22:22:22.278 INFO:tasks.workunit.client.0.smithi114.stdout:5/860: rename d0/df/f57 to d0/d28/d98/d25/d89/d99/f11a 0 2024-02-16T22:22:22.279 INFO:tasks.workunit.client.0.smithi114.stdout:8/913: truncate d4/d16/d25/d46/db5/fd6 535870 0 2024-02-16T22:22:22.284 INFO:tasks.workunit.client.0.smithi114.stdout:1/917: rename d4/d9/d46/d67/d92 to d4/d9/d3d/d3f/dd0/d152 0 2024-02-16T22:22:22.285 INFO:tasks.workunit.client.0.smithi114.stdout:1/918: write d4/d9/d3d/f39 [2416017,105466] 0 2024-02-16T22:22:22.285 INFO:tasks.workunit.client.0.smithi114.stdout:5/861: mkdir d0/d28/d98/dd0/d11b 0 2024-02-16T22:22:22.287 INFO:tasks.workunit.client.0.smithi114.stdout:1/919: write d4/d63/d69/d7c/d9b/ffc [1523448,89420] 0 2024-02-16T22:22:22.292 INFO:tasks.workunit.client.0.smithi114.stdout:2/987: rename dd/c122 to dd/df/ddb/c142 0 2024-02-16T22:22:22.300 INFO:tasks.workunit.client.0.smithi114.stdout:1/920: truncate d4/f2d 3793841 0 2024-02-16T22:22:22.301 INFO:tasks.workunit.client.0.smithi114.stdout:1/921: stat d4/d9/d3d/d3f/dd0/d152/f136 0 2024-02-16T22:22:22.305 INFO:tasks.workunit.client.0.smithi114.stdout:8/914: creat d4/d16/d25/d46/d10b/d13a/f13d x:0 0 0 2024-02-16T22:22:22.309 INFO:tasks.workunit.client.0.smithi114.stdout:5/862: dwrite d0/df/d7a/f97 [0,4194304] 0 2024-02-16T22:22:22.311 INFO:tasks.workunit.client.0.smithi114.stdout:6/962: getdents d6/d2c/d38/d41/d60/d84 0 2024-02-16T22:22:22.312 INFO:tasks.workunit.client.0.smithi114.stdout:6/963: readlink d6/d2c/d38/l69 0 2024-02-16T22:22:22.315 INFO:tasks.workunit.client.0.smithi114.stdout:2/988: fdatasync dd/f70 0 2024-02-16T22:22:22.317 INFO:tasks.workunit.client.0.smithi114.stdout:2/989: readlink dd/df/d1c/d21/d61/d63/d83/l8e 0 2024-02-16T22:22:22.328 INFO:tasks.workunit.client.0.smithi114.stdout:5/863: mkdir d0/d28/d98/d1e/d63/d96/d9a/d11c 0 2024-02-16T22:22:22.330 INFO:tasks.workunit.client.0.smithi114.stdout:6/964: creat d6/dd/d25/d34/d58/f127 x:0 0 0 2024-02-16T22:22:22.330 INFO:tasks.workunit.client.0.smithi114.stdout:5/864: readlink d0/d28/d98/d1e/d63/d96/d9a/db8/l3c 0 2024-02-16T22:22:22.331 INFO:tasks.workunit.client.0.smithi114.stdout:6/965: chown d6/d2c/d38/d41/l7e 8059 1 2024-02-16T22:22:22.332 INFO:tasks.workunit.client.0.smithi114.stdout:2/990: symlink dd/d3b/da7/d8b/d111/l143 0 2024-02-16T22:22:22.333 INFO:tasks.workunit.client.0.smithi114.stdout:8/915: dwrite d4/f3c [0,4194304] 0 2024-02-16T22:22:22.333 INFO:tasks.workunit.client.0.smithi114.stdout:5/865: write d0/d28/d98/d25/d3e/f5c [1800178,126472] 0 2024-02-16T22:22:22.336 INFO:tasks.workunit.client.0.smithi114.stdout:1/922: truncate d4/d9/d3d/d3f/dd0/d152/dca/dcc/fd5 6611879 0 2024-02-16T22:22:22.338 INFO:tasks.workunit.client.0.smithi114.stdout:6/966: symlink d6/dd/l128 0 2024-02-16T22:22:22.340 INFO:tasks.workunit.client.0.smithi114.stdout:2/991: rmdir dd/df/d1c/d21/d61/d69/de9 39 2024-02-16T22:22:22.342 INFO:tasks.workunit.client.0.smithi114.stdout:5/866: chown d0/d28/d98/d25/d89/lcd 6720266 1 2024-02-16T22:22:22.343 INFO:tasks.workunit.client.0.smithi114.stdout:5/867: dread - d0/df/d7a/fea zero size 2024-02-16T22:22:22.344 INFO:tasks.workunit.client.0.smithi114.stdout:1/923: creat d4/d63/d69/d72/d82/f153 x:0 0 0 2024-02-16T22:22:22.348 INFO:tasks.workunit.client.0.smithi114.stdout:6/967: symlink d6/d2c/l129 0 2024-02-16T22:22:22.350 INFO:tasks.workunit.client.0.smithi114.stdout:5/868: rename d0/d28/d98/f49 to d0/d28/d98/dd0/d11b/f11d 0 2024-02-16T22:22:22.360 INFO:tasks.workunit.client.0.smithi114.stdout:8/916: getdents d4/d11/d78/d7a/d36/d3b 0 2024-02-16T22:22:22.363 INFO:tasks.workunit.client.0.smithi114.stdout:6/968: creat d6/d2c/d38/da8/d101/d114/f12a x:0 0 0 2024-02-16T22:22:22.364 INFO:tasks.workunit.client.0.smithi114.stdout:2/992: write dd/d3b/d8a/f10b [502937,75959] 0 2024-02-16T22:22:22.365 INFO:tasks.workunit.client.0.smithi114.stdout:8/917: symlink d4/d16/d25/d46/db5/dca/l13e 0 2024-02-16T22:22:22.365 INFO:tasks.workunit.client.0.smithi114.stdout:8/918: readlink d4/d16/d25/d46/l126 0 2024-02-16T22:22:22.368 INFO:tasks.workunit.client.0.smithi114.stdout:1/924: rename d4/d9/d3d/d3f/d79 to d4/d9/d46/d47/d154 0 2024-02-16T22:22:22.368 INFO:tasks.workunit.client.0.smithi114.stdout:1/925: chown d4/d9/d3d/f39 0 1 2024-02-16T22:22:22.369 INFO:tasks.workunit.client.0.smithi114.stdout:6/969: unlink d6/l9 0 2024-02-16T22:22:22.371 INFO:tasks.workunit.client.0.smithi114.stdout:2/993: creat dd/d3b/da7/d8b/d111/f144 x:0 0 0 2024-02-16T22:22:22.371 INFO:tasks.workunit.client.0.smithi114.stdout:2/994: read - dd/dbd/fd9 zero size 2024-02-16T22:22:22.373 INFO:tasks.workunit.client.0.smithi114.stdout:8/919: read d4/d16/fe5 [1885863,30749] 0 2024-02-16T22:22:22.379 INFO:tasks.workunit.client.0.smithi114.stdout:5/869: dwrite d0/d28/d98/d25/d2e/f7d [0,4194304] 0 2024-02-16T22:22:22.382 INFO:tasks.workunit.client.0.smithi114.stdout:8/920: rmdir d4/d11/d78/d7a/d31/dc6/dce 39 2024-02-16T22:22:22.386 INFO:tasks.workunit.client.0.smithi114.stdout:6/970: write d6/dd/d25/d34/d58/fb6 [1020217,75642] 0 2024-02-16T22:22:22.386 INFO:tasks.workunit.client.0.smithi114.stdout:1/926: write d4/d9/d3d/d3f/f81 [2617546,72560] 0 2024-02-16T22:22:22.388 INFO:tasks.workunit.client.0.smithi114.stdout:5/870: symlink d0/d28/d98/d25/d2e/l11e 0 2024-02-16T22:22:22.389 INFO:tasks.workunit.client.0.smithi114.stdout:8/921: rename d4/d11/d78/d7a/d31/l79 to d4/d11/d78/d7a/d31/l13f 0 2024-02-16T22:22:22.390 INFO:tasks.workunit.client.0.smithi114.stdout:6/971: rmdir d6/dd/d25/d34 39 2024-02-16T22:22:22.391 INFO:tasks.workunit.client.0.smithi114.stdout:6/972: chown d6/dd/f1d 556 1 2024-02-16T22:22:22.393 INFO:tasks.workunit.client.0.smithi114.stdout:5/871: creat d0/df/d53/db1/d119/f11f x:0 0 0 2024-02-16T22:22:22.394 INFO:tasks.workunit.client.0.smithi114.stdout:5/872: chown d0/d28/d98/dd0 395 1 2024-02-16T22:22:22.394 INFO:tasks.workunit.client.0.smithi114.stdout:8/922: truncate d4/d24/d6d/dc7/d12a/f93 3925589 0 2024-02-16T22:22:22.394 INFO:tasks.workunit.client.0.smithi114.stdout:6/973: truncate d6/dd/f29 2274581 0 2024-02-16T22:22:22.396 INFO:tasks.workunit.client.0.smithi114.stdout:8/923: chown d4/d16/d25/d46/db5/dca 1965298579 1 2024-02-16T22:22:22.396 INFO:tasks.workunit.client.0.smithi114.stdout:2/995: dread dd/d3b/d8a/dcb/fee [0,4194304] 0 2024-02-16T22:22:22.397 INFO:tasks.workunit.client.0.smithi114.stdout:2/996: chown dd/da6 6 1 2024-02-16T22:22:22.401 INFO:tasks.workunit.client.0.smithi114.stdout:2/997: dread dd/f5d [0,4194304] 0 2024-02-16T22:22:22.402 INFO:tasks.workunit.client.0.smithi114.stdout:8/924: creat d4/d24/d6d/dc7/df0/f140 x:0 0 0 2024-02-16T22:22:22.402 INFO:tasks.workunit.client.0.smithi114.stdout:2/998: chown dd/d3b/da7/l49 496 1 2024-02-16T22:22:22.403 INFO:tasks.workunit.client.0.smithi114.stdout:6/974: mkdir d6/dd/d25/d34/d58/d9f/da3/dbd/d12b 0 2024-02-16T22:22:22.404 INFO:tasks.workunit.client.0.smithi114.stdout:6/975: write d6/fcd [935591,129531] 0 2024-02-16T22:22:22.407 INFO:tasks.workunit.client.0.smithi114.stdout:8/925: chown d4/d24/d6d/dc7/d12a/fe8 0 1 2024-02-16T22:22:22.408 INFO:tasks.workunit.client.0.smithi114.stdout:1/927: dread d4/d63/d69/d7c/d9b/db0/d74/f77 [0,4194304] 0 2024-02-16T22:22:22.411 INFO:tasks.workunit.client.0.smithi114.stdout:2/999: mknod dd/d6b/da5/c145 0 2024-02-16T22:22:22.415 INFO:tasks.workunit.client.0.smithi114.stdout:6/976: symlink d6/dd/d25/d34/d61/d62/d8b/l12c 0 2024-02-16T22:22:22.418 INFO:tasks.workunit.client.0.smithi114.stdout:1/928: creat d4/d63/d69/d7c/d9b/db0/df3/f155 x:0 0 0 2024-02-16T22:22:22.419 INFO:tasks.workunit.client.0.smithi114.stdout:5/873: dwrite d0/d28/d98/d1e/ff0 [0,4194304] 0 2024-02-16T22:22:22.419 INFO:tasks.workunit.client.0.smithi114.stdout:8/926: mknod d4/c141 0 2024-02-16T22:22:22.422 INFO:tasks.workunit.client.0.smithi114.stdout:5/874: write d0/d28/d98/f110 [1876606,1739] 0 2024-02-16T22:22:22.424 INFO:tasks.workunit.client.0.smithi114.stdout:6/977: creat d6/d2c/d38/d41/d60/d84/f12d x:0 0 0 2024-02-16T22:22:22.431 INFO:tasks.workunit.client.0.smithi114.stdout:8/927: chown d4/d24/d6d/dc7/d12a/dd1/ce7 432196 1 2024-02-16T22:22:22.435 INFO:tasks.workunit.client.0.smithi114.stdout:1/929: truncate d4/d9/d3d/d3f/dd0/d152/de4/fe7 1767129 0 2024-02-16T22:22:22.437 INFO:tasks.workunit.client.0.smithi114.stdout:8/928: chown d4/d11/c1b 2 1 2024-02-16T22:22:22.439 INFO:tasks.workunit.client.0.smithi114.stdout:1/930: mknod d4/d9/d46/d47/d154/c156 0 2024-02-16T22:22:22.441 INFO:tasks.workunit.client.0.smithi114.stdout:5/875: dwrite d0/d28/d98/d25/fc0 [0,4194304] 0 2024-02-16T22:22:22.443 INFO:tasks.workunit.client.0.smithi114.stdout:5/876: chown d0/d28/d98/d1e/f60 4 1 2024-02-16T22:22:22.444 INFO:tasks.workunit.client.0.smithi114.stdout:1/931: creat d4/d9/d46/d47/db3/f157 x:0 0 0 2024-02-16T22:22:22.445 INFO:tasks.workunit.client.0.smithi114.stdout:6/978: getdents d6/dd/d25/d34/d58 0 2024-02-16T22:22:22.447 INFO:tasks.workunit.client.0.smithi114.stdout:8/929: symlink d4/d11/d78/d7a/d31/dc6/l142 0 2024-02-16T22:22:22.449 INFO:tasks.workunit.client.0.smithi114.stdout:5/877: chown d0/d2d/f51 5 1 2024-02-16T22:22:22.451 INFO:tasks.workunit.client.0.smithi114.stdout:1/932: creat d4/d9/d46/d47/ddf/f158 x:0 0 0 2024-02-16T22:22:22.453 INFO:tasks.workunit.client.0.smithi114.stdout:6/979: dread - d6/d2c/d5e/f107 zero size 2024-02-16T22:22:22.454 INFO:tasks.workunit.client.0.smithi114.stdout:8/930: creat d4/d24/d6d/f143 x:0 0 0 2024-02-16T22:22:22.456 INFO:tasks.workunit.client.0.smithi114.stdout:5/878: creat d0/d28/d98/d1e/d86/f120 x:0 0 0 2024-02-16T22:22:22.458 INFO:tasks.workunit.client.0.smithi114.stdout:6/980: unlink d6/dd/d25/d34/d61/d62/d8b/f94 0 2024-02-16T22:22:22.461 INFO:tasks.workunit.client.0.smithi114.stdout:1/933: symlink d4/d9/d46/d47/l159 0 2024-02-16T22:22:22.462 INFO:tasks.workunit.client.0.smithi114.stdout:1/934: chown d4/d9/d46/d4d/f5c 82936043 1 2024-02-16T22:22:22.462 INFO:tasks.workunit.client.0.smithi114.stdout:6/981: symlink d6/d2c/d38/da8/d101/l12e 0 2024-02-16T22:22:22.465 INFO:tasks.workunit.client.0.smithi114.stdout:5/879: read d0/d28/d98/d1e/d86/f7c [2757012,128797] 0 2024-02-16T22:22:22.474 INFO:tasks.workunit.client.0.smithi114.stdout:8/931: dwrite d4/d24/ff9 [0,4194304] 0 2024-02-16T22:22:22.476 INFO:tasks.workunit.client.0.smithi114.stdout:6/982: dread d6/d2c/d38/d41/d60/d84/fc8 [0,4194304] 0 2024-02-16T22:22:22.479 INFO:tasks.workunit.client.0.smithi114.stdout:1/935: dread d4/d63/d69/d80/f95 [0,4194304] 0 2024-02-16T22:22:22.481 INFO:tasks.workunit.client.0.smithi114.stdout:8/932: truncate d4/d11/d52/fa9 4447341 0 2024-02-16T22:22:22.484 INFO:tasks.workunit.client.0.smithi114.stdout:1/936: unlink d4/d9/f8f 0 2024-02-16T22:22:22.484 INFO:tasks.workunit.client.0.smithi114.stdout:8/933: read d4/d11/d78/d7a/f10 [1511448,105313] 0 2024-02-16T22:22:22.485 INFO:tasks.workunit.client.0.smithi114.stdout:1/937: fsync d4/d9/d3d/f132 0 2024-02-16T22:22:22.489 INFO:tasks.workunit.client.0.smithi114.stdout:5/880: link d0/d28/d98/d1e/lac d0/d28/d98/d1e/dfa/l121 0 2024-02-16T22:22:22.494 INFO:tasks.workunit.client.0.smithi114.stdout:8/934: mkdir d4/d11/d78/d7a/d7e/d144 0 2024-02-16T22:22:22.502 INFO:tasks.workunit.client.0.smithi114.stdout:6/983: getdents d6/d2c/d38/d41/d60 0 2024-02-16T22:22:22.505 INFO:tasks.workunit.client.0.smithi114.stdout:5/881: creat d0/d28/d103/f122 x:0 0 0 2024-02-16T22:22:22.506 INFO:tasks.workunit.client.0.smithi114.stdout:5/882: write d0/d28/d98/d1e/f111 [793433,77574] 0 2024-02-16T22:22:22.509 INFO:tasks.workunit.client.0.smithi114.stdout:8/935: creat d4/d11/d78/d7a/d31/dc6/d135/f145 x:0 0 0 2024-02-16T22:22:22.511 INFO:tasks.workunit.client.0.smithi114.stdout:1/938: write d4/d63/d69/d72/f97 [331386,109437] 0 2024-02-16T22:22:22.528 INFO:tasks.workunit.client.0.smithi114.stdout:8/936: dwrite d4/f1d [0,4194304] 0 2024-02-16T22:22:22.529 INFO:tasks.workunit.client.0.smithi114.stdout:5/883: creat d0/d28/d98/d25/f123 x:0 0 0 2024-02-16T22:22:22.532 INFO:tasks.workunit.client.0.smithi114.stdout:1/939: mkdir d4/d9/d3d/d3f/dd0/d152/dca/dcc/d98/ddd/df2/d13f/d15a 0 2024-02-16T22:22:22.536 INFO:tasks.workunit.client.0.smithi114.stdout:5/884: rmdir d0/d28/d98/d25/d2e/d3b 39 2024-02-16T22:22:22.537 INFO:tasks.workunit.client.0.smithi114.stdout:8/937: truncate d4/d24/d6d/f108 2452167 0 2024-02-16T22:22:22.538 INFO:tasks.workunit.client.0.smithi114.stdout:6/984: dwrite d6/d2c/d75/f79 [0,4194304] 0 2024-02-16T22:22:22.549 INFO:tasks.workunit.client.0.smithi114.stdout:8/938: truncate d4/d11/d78/d7a/f1a 2395799 0 2024-02-16T22:22:22.550 INFO:tasks.workunit.client.0.smithi114.stdout:1/940: creat d4/d9/d3d/d3f/dd0/d152/f15b x:0 0 0 2024-02-16T22:22:22.551 INFO:tasks.workunit.client.0.smithi114.stdout:6/985: creat d6/d2c/d38/d41/d60/d84/dfa/f12f x:0 0 0 2024-02-16T22:22:22.553 INFO:tasks.workunit.client.0.smithi114.stdout:8/939: unlink d4/d11/d78/f90 0 2024-02-16T22:22:22.554 INFO:tasks.workunit.client.0.smithi114.stdout:5/885: getdents d0/df 0 2024-02-16T22:22:22.555 INFO:tasks.workunit.client.0.smithi114.stdout:1/941: creat d4/d9/d3d/d3f/dd0/d152/dca/ded/d101/f15c x:0 0 0 2024-02-16T22:22:22.564 INFO:tasks.workunit.client.0.smithi114.stdout:1/942: mknod d4/d9/d3d/d3f/dd0/d152/dca/dce/d11b/c15d 0 2024-02-16T22:22:22.567 INFO:tasks.workunit.client.0.smithi114.stdout:5/886: dwrite d0/d28/d98/d25/d89/d99/db0/ffe [0,4194304] 0 2024-02-16T22:22:22.567 INFO:tasks.workunit.client.0.smithi114.stdout:6/986: dwrite d6/d2c/de6/ff2 [0,4194304] 0 2024-02-16T22:22:22.568 INFO:tasks.workunit.client.0.smithi114.stdout:1/943: write d4/d63/d69/d72/fe6 [5170414,57178] 0 2024-02-16T22:22:22.577 INFO:tasks.workunit.client.0.smithi114.stdout:6/987: creat d6/d2c/d5e/f130 x:0 0 0 2024-02-16T22:22:22.579 INFO:tasks.workunit.client.0.smithi114.stdout:6/988: chown d6/d83/fcb 27187686 1 2024-02-16T22:22:22.583 INFO:tasks.workunit.client.0.smithi114.stdout:8/940: write d4/d16/d25/d7c/f8d [2095922,48455] 0 2024-02-16T22:22:22.597 INFO:tasks.workunit.client.0.smithi114.stdout:8/941: truncate d4/d24/d6d/dc7/d12a/dd1/d131/fdd 5796032 0 2024-02-16T22:22:22.598 INFO:tasks.workunit.client.0.smithi114.stdout:6/989: dread d6/d2c/d38/d41/f4a [4194304,4194304] 0 2024-02-16T22:22:22.604 INFO:tasks.workunit.client.0.smithi114.stdout:6/990: rename d6/dd/d25/d34/d58/d9f/f124 to d6/dd/d25/d34/d61/f131 0 2024-02-16T22:22:22.604 INFO:tasks.workunit.client.0.smithi114.stdout:1/944: dwrite d4/d63/d69/d72/d82/fb2 [0,4194304] 0 2024-02-16T22:22:22.607 INFO:tasks.workunit.client.0.smithi114.stdout:6/991: stat d6/d2c/d75/dfe/c113 0 2024-02-16T22:22:22.607 INFO:tasks.workunit.client.0.smithi114.stdout:1/945: fsync d4/d63/d69/d72/fe6 0 2024-02-16T22:22:22.611 INFO:tasks.workunit.client.0.smithi114.stdout:5/887: dwrite d0/d28/d98/d1e/d63/d96/d9a/db8/f107 [0,4194304] 0 2024-02-16T22:22:22.614 INFO:tasks.workunit.client.0.smithi114.stdout:8/942: dread d4/d11/d78/f9c [0,4194304] 0 2024-02-16T22:22:22.615 INFO:tasks.workunit.client.0.smithi114.stdout:6/992: symlink d6/dd/d25/d34/d61/d62/d8b/db7/dd3/l132 0 2024-02-16T22:22:22.618 INFO:tasks.workunit.client.0.smithi114.stdout:1/946: creat d4/d9/d3d/d3f/dd0/d152/dca/dcc/d98/f15e x:0 0 0 2024-02-16T22:22:22.624 INFO:tasks.workunit.client.0.smithi114.stdout:5/888: rename d0/d28/d98/d1e/d63/d96/fd4 to d0/d28/d98/d25/d3e/d7f/f124 0 2024-02-16T22:22:22.625 INFO:tasks.workunit.client.0.smithi114.stdout:8/943: dwrite d4/d16/d25/d46/db5/dca/f118 [0,4194304] 0 2024-02-16T22:22:22.634 INFO:tasks.workunit.client.0.smithi114.stdout:6/993: chown d6/d2c/d38/fdc 439 1 2024-02-16T22:22:22.635 INFO:tasks.workunit.client.0.smithi114.stdout:6/994: chown d6/d2c/d38/da8/d101/l108 634 1 2024-02-16T22:22:22.637 INFO:tasks.workunit.client.0.smithi114.stdout:5/889: mknod d0/d28/d98/d25/c125 0 2024-02-16T22:22:22.638 INFO:tasks.workunit.client.0.smithi114.stdout:8/944: read d4/d16/f91 [2292003,14957] 0 2024-02-16T22:22:22.640 INFO:tasks.workunit.client.0.smithi114.stdout:6/995: symlink d6/d2c/d38/d54/l133 0 2024-02-16T22:22:22.643 INFO:tasks.workunit.client.0.smithi114.stdout:8/945: dread d4/d16/d25/d7c/f87 [0,4194304] 0 2024-02-16T22:22:22.643 INFO:tasks.workunit.client.0.smithi114.stdout:5/890: rmdir d0/d28/d98/d1e/d63/d9e 39 2024-02-16T22:22:22.644 INFO:tasks.workunit.client.0.smithi114.stdout:8/946: write d4/f1d [4740633,13967] 0 2024-02-16T22:22:22.653 INFO:tasks.workunit.client.0.smithi114.stdout:6/996: creat d6/d2c/d66/f134 x:0 0 0 2024-02-16T22:22:22.654 INFO:tasks.workunit.client.0.smithi114.stdout:6/997: readlink d6/da/l13 0 2024-02-16T22:22:22.656 INFO:tasks.workunit.client.0.smithi114.stdout:5/891: creat d0/d28/d98/d25/d89/de2/df3/f126 x:0 0 0 2024-02-16T22:22:22.658 INFO:tasks.workunit.client.0.smithi114.stdout:5/892: truncate d0/df/d7a/dc4/ff7 1063994 0 2024-02-16T22:22:22.660 INFO:tasks.workunit.client.0.smithi114.stdout:6/998: creat d6/d2c/d5e/d118/f135 x:0 0 0 2024-02-16T22:22:22.660 INFO:tasks.workunit.client.0.smithi114.stdout:8/947: write d4/d11/f71 [369262,105250] 0 2024-02-16T22:22:22.662 INFO:tasks.workunit.client.0.smithi114.stdout:5/893: creat d0/d28/d98/d1e/dfa/f127 x:0 0 0 2024-02-16T22:22:22.663 INFO:tasks.workunit.client.0.smithi114.stdout:6/999: write d6/d2c/f95 [2889950,27691] 0 2024-02-16T22:22:22.664 INFO:tasks.workunit.client.0.smithi114.stdout:5/894: fsync d0/d28/d98/d25/d3e/d7f/fa8 0 2024-02-16T22:22:22.668 INFO:tasks.workunit.client.0.smithi114.stdout:8/948: link d4/d11/d78/d7a/d36/d3b/ceb d4/d24/d6d/dc7/d12a/dd1/c146 0 2024-02-16T22:22:22.669 INFO:tasks.workunit.client.0.smithi114.stdout:8/949: readlink d4/d16/d25/d46/db5/lbc 0 2024-02-16T22:22:22.670 INFO:tasks.workunit.client.0.smithi114.stdout:8/950: write d4/d11/f60 [3228575,120760] 0 2024-02-16T22:22:22.672 INFO:tasks.workunit.client.0.smithi114.stdout:8/951: chown d4/d16/d25/d39/ld2 4244361 1 2024-02-16T22:22:22.674 INFO:tasks.workunit.client.0.smithi114.stdout:8/952: mknod d4/d16/d25/d46/db5/c147 0 2024-02-16T22:22:22.676 INFO:tasks.workunit.client.0.smithi114.stdout:5/895: write d0/df/d7a/dc4/ff2 [588407,67059] 0 2024-02-16T22:22:22.677 INFO:tasks.workunit.client.0.smithi114.stdout:8/953: symlink d4/d24/d6d/dc7/d12a/dd1/d131/ddf/l148 0 2024-02-16T22:22:22.680 INFO:tasks.workunit.client.0.smithi114.stdout:5/896: creat d0/d28/d98/d25/d89/de2/db3/f128 x:0 0 0 2024-02-16T22:22:22.681 INFO:tasks.workunit.client.0.smithi114.stdout:8/954: creat d4/d16/d25/d46/db5/f149 x:0 0 0 2024-02-16T22:22:22.690 INFO:tasks.workunit.client.0.smithi114.stdout:8/955: mknod d4/c14a 0 2024-02-16T22:22:22.694 INFO:tasks.workunit.client.0.smithi114.stdout:5/897: dwrite d0/d28/d98/d25/d2e/d3b/f59 [0,4194304] 0 2024-02-16T22:22:22.740 INFO:tasks.workunit.client.0.smithi114.stdout:8/956: truncate d4/d24/d6d/dc7/d12a/dd1/fd5 4437986 0 2024-02-16T22:22:22.760 INFO:tasks.workunit.client.0.smithi114.stdout:5/898: write d0/d28/d98/f5b [2761949,85243] 0 2024-02-16T22:22:22.761 INFO:tasks.workunit.client.0.smithi114.stdout:8/957: write d4/d24/d6d/dc7/d12a/f93 [4530485,423] 0 2024-02-16T22:22:22.761 INFO:tasks.workunit.client.0.smithi114.stdout:5/899: chown d0/d28/d98/d25/d2e/d3b/l47 3 1 2024-02-16T22:22:22.768 INFO:tasks.workunit.client.0.smithi114.stdout:8/958: creat d4/d16/d25/f14b x:0 0 0 2024-02-16T22:22:22.770 INFO:tasks.workunit.client.0.smithi114.stdout:8/959: dread - d4/d11/d78/d7a/d36/d3b/de4/fff zero size 2024-02-16T22:22:22.773 INFO:tasks.workunit.client.0.smithi114.stdout:5/900: dwrite d0/d28/d98/d1e/d86/f7e [4194304,4194304] 0 2024-02-16T22:22:22.776 INFO:tasks.workunit.client.0.smithi114.stdout:8/960: truncate d4/d11/d78/d7a/d36/f6b 2022847 0 2024-02-16T22:22:22.777 INFO:tasks.workunit.client.0.smithi114.stdout:5/901: fdatasync d0/d28/d98/d1e/d86/f8c 0 2024-02-16T22:22:22.780 INFO:tasks.workunit.client.0.smithi114.stdout:8/961: creat d4/d24/d6d/dc7/dfe/f14c x:0 0 0 2024-02-16T22:22:22.781 INFO:tasks.workunit.client.0.smithi114.stdout:8/962: fsync d4/d11/d78/d7a/d31/dc6/d135/f145 0 2024-02-16T22:22:22.787 INFO:tasks.workunit.client.0.smithi114.stdout:5/902: mknod d0/d28/d98/d1e/d63/d96/d9a/d11c/c129 0 2024-02-16T22:22:22.790 INFO:tasks.workunit.client.0.smithi114.stdout:5/903: creat d0/d28/d98/d1e/d86/f12a x:0 0 0 2024-02-16T22:22:22.791 INFO:tasks.workunit.client.0.smithi114.stdout:5/904: dread - d0/df/d7a/dc4/f114 zero size 2024-02-16T22:22:22.795 INFO:tasks.workunit.client.0.smithi114.stdout:5/905: mknod d0/d28/d98/d1e/d63/d96/c12b 0 2024-02-16T22:22:22.798 INFO:tasks.workunit.client.0.smithi114.stdout:5/906: truncate d0/d28/d98/f79 5986430 0 2024-02-16T22:22:22.800 INFO:tasks.workunit.client.0.smithi114.stdout:8/963: dread d4/d24/d6d/dc7/d12a/f93 [0,4194304] 0 2024-02-16T22:22:22.807 INFO:tasks.workunit.client.0.smithi114.stdout:8/964: dwrite d4/d16/d25/f14b [0,4194304] 0 2024-02-16T22:22:22.808 INFO:tasks.workunit.client.0.smithi114.stdout:5/907: write d0/d28/d98/d25/f66 [914809,54106] 0 2024-02-16T22:22:22.811 INFO:tasks.workunit.client.0.smithi114.stdout:5/908: creat d0/df/d53/db1/d119/f12c x:0 0 0 2024-02-16T22:22:22.814 INFO:tasks.workunit.client.0.smithi114.stdout:8/965: getdents d4/d16/d25/d46 0 2024-02-16T22:22:22.815 INFO:tasks.workunit.client.0.smithi114.stdout:8/966: write d4/f1d [901575,23837] 0 2024-02-16T22:22:22.817 INFO:tasks.workunit.client.0.smithi114.stdout:8/967: fsync d4/d16/f12b 0 2024-02-16T22:22:22.820 INFO:tasks.workunit.client.0.smithi114.stdout:8/968: truncate d4/d16/d25/d46/f73 3234541 0 2024-02-16T22:22:22.831 INFO:tasks.workunit.client.0.smithi114.stdout:8/969: truncate d4/d11/f17 1966243 0 2024-02-16T22:22:22.832 INFO:tasks.workunit.client.0.smithi114.stdout:8/970: rmdir d4 39 2024-02-16T22:22:22.834 INFO:tasks.workunit.client.0.smithi114.stdout:8/971: write d4/d16/d25/fde [1650298,78590] 0 2024-02-16T22:22:22.837 INFO:tasks.workunit.client.0.smithi114.stdout:8/972: chown d4/d24/d6d/dc7/d12a/dd1/d131/ce2 121710577 1 2024-02-16T22:22:22.839 INFO:tasks.workunit.client.0.smithi114.stdout:8/973: mkdir d4/d11/d78/d7a/d14d 0 2024-02-16T22:22:22.841 INFO:tasks.workunit.client.0.smithi114.stdout:8/974: mkdir d4/d11/d52/d14e 0 2024-02-16T22:22:22.846 INFO:tasks.workunit.client.0.smithi114.stdout:8/975: dwrite d4/f8 [0,4194304] 0 2024-02-16T22:22:22.847 INFO:tasks.workunit.client.0.smithi114.stdout:8/976: readlink d4/d24/lee 0 2024-02-16T22:22:22.856 INFO:tasks.workunit.client.0.smithi114.stdout:8/977: symlink d4/d11/d78/d7a/d14d/l14f 0 2024-02-16T22:22:22.857 INFO:tasks.workunit.client.0.smithi114.stdout:8/978: stat d4/d16/d25/d39/f82 0 2024-02-16T22:22:22.860 INFO:tasks.workunit.client.0.smithi114.stdout:8/979: creat d4/d24/d6d/dc7/d12a/dd1/d131/ddf/f150 x:0 0 0 2024-02-16T22:22:22.861 INFO:tasks.workunit.client.0.smithi114.stdout:8/980: dread - d4/d16/d25/d46/fb7 zero size 2024-02-16T22:22:22.863 INFO:tasks.workunit.client.0.smithi114.stdout:8/981: rmdir d4/d24/d2c 39 2024-02-16T22:22:22.877 INFO:tasks.workunit.client.0.smithi114.stdout:8/982: dwrite d4/d11/d78/d7a/f10 [0,4194304] 0 2024-02-16T22:22:22.883 INFO:tasks.workunit.client.0.smithi114.stdout:8/983: truncate d4/d24/d6d/dc7/d12a/dd1/d131/ddf/f101 1656447 0 2024-02-16T22:22:22.883 INFO:tasks.workunit.client.0.smithi114.stdout:8/984: chown d4/d16/d25/d46/fb2 473 1 2024-02-16T22:22:22.890 INFO:tasks.workunit.client.0.smithi114.stdout:8/985: link d4/d11/d78/d7a/d31/dc6/d135/f145 d4/d24/d2c/f151 0 2024-02-16T22:22:22.908 INFO:tasks.workunit.client.0.smithi114.stdout:8/986: dwrite d4/d24/d6d/dc7/d12a/fe8 [0,4194304] 0 2024-02-16T22:22:22.909 INFO:tasks.workunit.client.0.smithi114.stdout:8/987: chown d4/d16/l43 513 1 2024-02-16T22:22:22.910 INFO:tasks.workunit.client.0.smithi114.stdout:8/988: readlink d4/d24/d6d/dc7/d12a/dd1/d131/ddf/l148 0 2024-02-16T22:22:22.915 INFO:tasks.workunit.client.0.smithi114.stdout:8/989: rmdir d4/d11/d78/d7a/d31/dc6/d135 39 2024-02-16T22:22:22.916 INFO:tasks.workunit.client.0.smithi114.stdout:8/990: stat d4/d16/d25/d46/l92 0 2024-02-16T22:22:22.919 INFO:tasks.workunit.client.0.smithi114.stdout:8/991: mknod d4/d24/d6d/dc7/d12a/dd1/d131/d104/c152 0 2024-02-16T22:22:23.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:22 smithi114 ceph-mon[29235]: pgmap v39: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 93 MiB/s rd, 206 MiB/s wr, 521 op/s 2024-02-16T22:22:23.203 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:22 smithi123 ceph-mon[37161]: pgmap v39: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 93 MiB/s rd, 206 MiB/s wr, 521 op/s 2024-02-16T22:22:23.390 INFO:tasks.workunit.client.0.smithi114.stdout:1/947: sync 2024-02-16T22:22:23.394 INFO:tasks.workunit.client.0.smithi114.stdout:1/948: rename d4/d63/d69/d7c/d9b/ffc to d4/d9/d3d/d3f/dd0/d152/dca/dcc/d98/f15f 0 2024-02-16T22:22:23.397 INFO:tasks.workunit.client.0.smithi114.stdout:1/949: unlink d4/d9/d3d/d3f/dd0/d152/f136 0 2024-02-16T22:22:23.402 INFO:tasks.workunit.client.0.smithi114.stdout:1/950: link d4/d63/d69/d7c/d9b/l143 d4/l160 0 2024-02-16T22:22:23.404 INFO:tasks.workunit.client.0.smithi114.stdout:1/951: creat d4/d9/d46/d47/db3/f161 x:0 0 0 2024-02-16T22:22:23.530 INFO:tasks.workunit.client.0.smithi114.stdout:5/909: sync 2024-02-16T22:22:23.530 INFO:tasks.workunit.client.0.smithi114.stdout:8/992: sync 2024-02-16T22:22:23.536 INFO:tasks.workunit.client.0.smithi114.stdout:5/910: dread d0/d28/d98/d25/d2e/d3b/fc1 [0,4194304] 0 2024-02-16T22:22:23.538 INFO:tasks.workunit.client.0.smithi114.stdout:5/911: creat d0/d28/d98/dd0/f12d x:0 0 0 2024-02-16T22:22:23.544 INFO:tasks.workunit.client.0.smithi114.stdout:8/993: dread d4/d16/fbf [0,4194304] 0 2024-02-16T22:22:23.545 INFO:tasks.workunit.client.0.smithi114.stdout:8/994: chown d4/d24/l8a 0 1 2024-02-16T22:22:23.546 INFO:tasks.workunit.client.0.smithi114.stdout:1/952: sync 2024-02-16T22:22:23.547 INFO:tasks.workunit.client.0.smithi114.stdout:8/995: chown d4/d16/d25/d46/cd7 2 1 2024-02-16T22:22:23.550 INFO:tasks.workunit.client.0.smithi114.stdout:1/953: write d4/d9/d3d/d3f/dd0/d152/dca/dcc/d70/fff [471357,52129] 0 2024-02-16T22:22:23.551 INFO:tasks.workunit.client.0.smithi114.stdout:8/996: symlink d4/d24/d6d/dc7/df0/l153 0 2024-02-16T22:22:23.552 INFO:tasks.workunit.client.0.smithi114.stdout:5/912: dwrite d0/d28/d98/d1e/d86/fa9 [0,4194304] 0 2024-02-16T22:22:23.553 INFO:tasks.workunit.client.0.smithi114.stdout:1/954: creat d4/d63/d69/d7c/d9b/f162 x:0 0 0 2024-02-16T22:22:23.558 INFO:tasks.workunit.client.0.smithi114.stdout:1/955: mknod d4/d9/d3d/d3f/dd0/d152/dca/dce/c163 0 2024-02-16T22:22:23.563 INFO:tasks.workunit.client.0.smithi114.stdout:5/913: creat d0/d28/d98/f12e x:0 0 0 2024-02-16T22:22:23.564 INFO:tasks.workunit.client.0.smithi114.stdout:1/956: dwrite d4/d63/d69/d72/fe6 [0,4194304] 0 2024-02-16T22:22:23.565 INFO:tasks.workunit.client.0.smithi114.stdout:5/914: symlink d0/df/l12f 0 2024-02-16T22:22:23.568 INFO:tasks.workunit.client.0.smithi114.stdout:5/915: mknod d0/d28/d98/d25/d89/de2/df3/c130 0 2024-02-16T22:22:23.569 INFO:tasks.workunit.client.0.smithi114.stdout:1/957: creat d4/d9/d3d/d3f/dd0/d152/dca/dcc/d70/f164 x:0 0 0 2024-02-16T22:22:23.572 INFO:tasks.workunit.client.0.smithi114.stdout:5/916: unlink d0/d28/d98/d25/d2e/d3b/db5/lf1 0 2024-02-16T22:22:23.572 INFO:tasks.workunit.client.0.smithi114.stdout:5/917: chown d0/d28/d98/d25/d3e 70872359 1 2024-02-16T22:22:23.574 INFO:tasks.workunit.client.0.smithi114.stdout:1/958: creat d4/d63/d69/d7c/d9b/df0/f165 x:0 0 0 2024-02-16T22:22:23.575 INFO:tasks.workunit.client.0.smithi114.stdout:5/918: mkdir d0/d28/d98/d25/d89/de2/d9f/d131 0 2024-02-16T22:22:23.576 INFO:tasks.workunit.client.0.smithi114.stdout:1/959: write d4/d63/d69/d7c/d9b/f162 [87344,88974] 0 2024-02-16T22:22:23.579 INFO:tasks.workunit.client.0.smithi114.stdout:8/997: dwrite d4/d16/d25/d46/fc2 [0,4194304] 0 2024-02-16T22:22:23.580 INFO:tasks.workunit.client.0.smithi114.stdout:1/960: write d4/d9/d3d/d3f/dd0/d152/f15b [213550,44059] 0 2024-02-16T22:22:23.585 INFO:tasks.workunit.client.0.smithi114.stdout:1/961: creat d4/d63/d69/d7c/d9b/df0/d108/f166 x:0 0 0 2024-02-16T22:22:23.585 INFO:tasks.workunit.client.0.smithi114.stdout:5/919: link d0/d28/d98/d1e/d86/f7c d0/d2d/f132 0 2024-02-16T22:22:23.587 INFO:tasks.workunit.client.0.smithi114.stdout:5/920: readlink d0/d2d/lf4 0 2024-02-16T22:22:23.587 INFO:tasks.workunit.client.0.smithi114.stdout:8/998: symlink d4/d11/d78/d7a/d7e/d144/l154 0 2024-02-16T22:22:23.588 INFO:tasks.workunit.client.0.smithi114.stdout:5/921: write d0/d28/d98/d25/d3e/d7f/fa8 [3762605,43734] 0 2024-02-16T22:22:23.589 INFO:tasks.workunit.client.0.smithi114.stdout:1/962: mkdir d4/d9/d46/d4d/d167 0 2024-02-16T22:22:23.591 INFO:tasks.workunit.client.0.smithi114.stdout:8/999: symlink d4/d16/d25/d46/d10b/l155 0 2024-02-16T22:22:23.592 INFO:tasks.workunit.client.0.smithi114.stdout:1/963: creat d4/d63/d69/d7c/d9b/dae/f168 x:0 0 0 2024-02-16T22:22:23.594 INFO:tasks.workunit.client.0.smithi114.stdout:1/964: mknod d4/d9/d3d/d3f/dd0/d152/dca/ded/c169 0 2024-02-16T22:22:23.612 INFO:tasks.workunit.client.0.smithi114.stdout:5/922: dread d0/f81 [4194304,4194304] 0 2024-02-16T22:22:23.620 INFO:tasks.workunit.client.0.smithi114.stdout:5/923: dwrite d0/d28/d98/d25/d89/d99/fcc [0,4194304] 0 2024-02-16T22:22:23.673 INFO:tasks.workunit.client.0.smithi114.stdout:5/924: rename d0/df/c4b to d0/d28/d98/d1e/d63/c133 0 2024-02-16T22:22:23.677 INFO:tasks.workunit.client.0.smithi114.stdout:5/925: dread d0/d28/d98/d1e/fa4 [0,4194304] 0 2024-02-16T22:22:23.679 INFO:tasks.workunit.client.0.smithi114.stdout:5/926: symlink d0/df/d5e/l134 0 2024-02-16T22:22:23.683 INFO:tasks.workunit.client.0.smithi114.stdout:5/927: write d0/df/d5e/ff5 [926192,72406] 0 2024-02-16T22:22:23.686 INFO:tasks.workunit.client.0.smithi114.stdout:5/928: creat d0/d28/d98/d25/d89/de2/d9f/d131/f135 x:0 0 0 2024-02-16T22:22:23.687 INFO:tasks.workunit.client.0.smithi114.stdout:5/929: read d0/d28/d98/d25/d2e/f67 [371684,61411] 0 2024-02-16T22:22:23.934 INFO:tasks.workunit.client.0.smithi114.stdout:1/965: sync 2024-02-16T22:22:23.934 INFO:tasks.workunit.client.0.smithi114.stdout:1/966: chown d4/d63/l9a 223895 1 2024-02-16T22:22:23.937 INFO:tasks.workunit.client.0.smithi114.stdout:1/967: unlink d4/d63/cdc 0 2024-02-16T22:22:23.941 INFO:tasks.workunit.client.0.smithi114.stdout:1/968: creat d4/d9/d3d/d3f/dd0/d152/dca/dce/f16a x:0 0 0 2024-02-16T22:22:23.942 INFO:tasks.workunit.client.0.smithi114.stdout:1/969: read d4/d63/d69/d7c/d9b/f162 [158570,17919] 0 2024-02-16T22:22:23.949 INFO:tasks.workunit.client.0.smithi114.stdout:1/970: rename d4/d9/d3d/d3f/f75 to d4/d123/f16b 0 2024-02-16T22:22:23.951 INFO:tasks.workunit.client.0.smithi114.stdout:5/930: sync 2024-02-16T22:22:23.954 INFO:tasks.workunit.client.0.smithi114.stdout:5/931: fdatasync d0/fb9 0 2024-02-16T22:22:23.958 INFO:tasks.workunit.client.0.smithi114.stdout:1/971: dwrite d4/d9/d3d/d3f/dd0/d152/dca/dcc/d98/f15f [0,4194304] 0 2024-02-16T22:22:23.977 INFO:tasks.workunit.client.0.smithi114.stdout:5/932: write d0/d28/d98/f79 [6386597,14960] 0 2024-02-16T22:22:23.977 INFO:tasks.workunit.client.0.smithi114.stdout:1/972: write d4/d9/d3d/d3f/dd0/d152/dca/dcc/d70/f141 [4673318,128630] 0 2024-02-16T22:22:23.983 INFO:tasks.workunit.client.0.smithi114.stdout:5/933: fsync d0/d2d/f132 0 2024-02-16T22:22:23.985 INFO:tasks.workunit.client.0.smithi114.stdout:5/934: write d0/d28/d98/d25/d89/de2/d9f/d131/f135 [152653,11917] 0 2024-02-16T22:22:23.985 INFO:tasks.workunit.client.0.smithi114.stdout:5/935: readlink d0/df/d6d/l10c 0 2024-02-16T22:22:23.987 INFO:tasks.workunit.client.0.smithi114.stdout:1/973: write d4/d9/d46/d47/d154/fa3 [1158690,2068] 0 2024-02-16T22:22:23.987 INFO:tasks.workunit.client.0.smithi114.stdout:5/936: dread - d0/df/d7a/dc4/dda/fe8 zero size 2024-02-16T22:22:23.989 INFO:tasks.workunit.client.0.smithi114.stdout:5/937: write d0/df/d5e/ff5 [1522554,104860] 0 2024-02-16T22:22:23.990 INFO:tasks.workunit.client.0.smithi114.stdout:1/974: creat d4/d9/d3d/d3f/d14d/f16c x:0 0 0 2024-02-16T22:22:23.997 INFO:tasks.workunit.client.0.smithi114.stdout:5/938: mknod d0/d28/d98/d25/c136 0 2024-02-16T22:22:24.001 INFO:tasks.workunit.client.0.smithi114.stdout:1/975: getdents d4/d9/d3d/d3f/dd0/d152/dca/dcc 0 2024-02-16T22:22:24.015 INFO:tasks.workunit.client.0.smithi114.stdout:5/939: dwrite d0/d28/d98/d1e/d86/f8c [0,4194304] 0 2024-02-16T22:22:24.016 INFO:tasks.workunit.client.0.smithi114.stdout:1/976: dwrite d4/d9/d3d/d3f/dd0/d152/dca/dcc/d98/ddd/df2/f125 [0,4194304] 0 2024-02-16T22:22:24.018 INFO:tasks.workunit.client.0.smithi114.stdout:1/977: write d4/f25 [2011427,51958] 0 2024-02-16T22:22:24.021 INFO:tasks.workunit.client.0.smithi114.stdout:1/978: readlink d4/d63/d69/d7c/d9b/df0/l104 0 2024-02-16T22:22:24.024 INFO:tasks.workunit.client.0.smithi114.stdout:5/940: symlink d0/d28/d98/d25/d89/de2/d9f/l137 0 2024-02-16T22:22:24.027 INFO:tasks.workunit.client.0.smithi114.stdout:1/979: mkdir d4/d9/d3d/d3f/dd0/d152/de4/d16d 0 2024-02-16T22:22:24.035 INFO:tasks.workunit.client.0.smithi114.stdout:1/980: rename d4/d9/d3d/d3f/dd0/d152/de4/fe7 to d4/d9/d3d/d3f/dd0/d152/dca/dcc/d98/f16e 0 2024-02-16T22:22:24.037 INFO:tasks.workunit.client.0.smithi114.stdout:5/941: getdents d0/d28/d98/d25/d2e/d3b 0 2024-02-16T22:22:24.038 INFO:tasks.workunit.client.0.smithi114.stdout:1/981: mknod d4/d9/d3d/d3f/dd0/d152/dca/dcc/d70/c16f 0 2024-02-16T22:22:24.039 INFO:tasks.workunit.client.0.smithi114.stdout:5/942: dread - d0/df/d7a/dc7/f102 zero size 2024-02-16T22:22:24.041 INFO:tasks.workunit.client.0.smithi114.stdout:1/982: symlink d4/d9/d3d/d3f/dd0/d152/dca/l170 0 2024-02-16T22:22:24.042 INFO:tasks.workunit.client.0.smithi114.stdout:5/943: mkdir d0/d138 0 2024-02-16T22:22:24.044 INFO:tasks.workunit.client.0.smithi114.stdout:1/983: symlink d4/d63/d69/d7c/d9b/df0/l171 0 2024-02-16T22:22:24.045 INFO:tasks.workunit.client.0.smithi114.stdout:5/944: truncate d0/d28/d98/d1e/d63/fbd 127191 0 2024-02-16T22:22:24.047 INFO:tasks.workunit.client.0.smithi114.stdout:1/984: mknod d4/d9/d46/d47/db3/c172 0 2024-02-16T22:22:24.048 INFO:tasks.workunit.client.0.smithi114.stdout:5/945: creat d0/df/d7a/dc4/f139 x:0 0 0 2024-02-16T22:22:24.049 INFO:tasks.workunit.client.0.smithi114.stdout:5/946: readlink d0/d28/d98/d1e/d86/lf8 0 2024-02-16T22:22:24.057 INFO:tasks.workunit.client.0.smithi114.stdout:5/947: dwrite d0/d28/d98/d1e/d86/f12a [0,4194304] 0 2024-02-16T22:22:24.071 INFO:tasks.workunit.client.0.smithi114.stdout:5/948: truncate d0/df/d7a/f97 3376156 0 2024-02-16T22:22:24.078 INFO:tasks.workunit.client.0.smithi114.stdout:5/949: dwrite d0/d28/d98/d25/d89/de2/d9f/d131/f135 [0,4194304] 0 2024-02-16T22:22:24.081 INFO:tasks.workunit.client.0.smithi114.stdout:5/950: mknod d0/d28/d98/d1e/d63/d96/d9a/c13a 0 2024-02-16T22:22:24.345 INFO:tasks.workunit.client.0.smithi114.stdout:1/985: sync 2024-02-16T22:22:24.350 INFO:tasks.workunit.client.0.smithi114.stdout:1/986: dwrite d4/d9/d3d/def/f138 [0,4194304] 0 2024-02-16T22:22:24.352 INFO:tasks.workunit.client.0.smithi114.stdout:1/987: fdatasync d4/d9/d3d/d3f/dd0/d152/dca/dce/f16a 0 2024-02-16T22:22:24.358 INFO:tasks.workunit.client.0.smithi114.stdout:1/988: dread d4/d9/d46/d47/d154/fa3 [0,4194304] 0 2024-02-16T22:22:24.358 INFO:tasks.workunit.client.0.smithi114.stdout:1/989: chown d4 0 1 2024-02-16T22:22:24.364 INFO:tasks.workunit.client.0.smithi114.stdout:1/990: creat d4/d63/f173 x:0 0 0 2024-02-16T22:22:24.368 INFO:tasks.workunit.client.0.smithi114.stdout:1/991: rename d4/d9/d3d/d3f/dd0/d152/dca/dce/dd3/cd9 to d4/d9/d3d/d3f/d14d/c174 0 2024-02-16T22:22:24.370 INFO:tasks.workunit.client.0.smithi114.stdout:1/992: getdents d4/d63/d69/d72/d119 0 2024-02-16T22:22:24.372 INFO:tasks.workunit.client.0.smithi114.stdout:1/993: creat d4/d9/d46/d4d/f175 x:0 0 0 2024-02-16T22:22:24.375 INFO:tasks.workunit.client.0.smithi114.stdout:1/994: mknod d4/d9/d3d/d3f/dd0/d152/dca/dcc/d98/ddd/df2/d13f/c176 0 2024-02-16T22:22:24.378 INFO:tasks.workunit.client.0.smithi114.stdout:1/995: getdents d4/d63/d69/d80/d127 0 2024-02-16T22:22:24.380 INFO:tasks.workunit.client.0.smithi114.stdout:1/996: creat d4/d63/d69/d80/d127/f177 x:0 0 0 2024-02-16T22:22:24.381 INFO:tasks.workunit.client.0.smithi114.stdout:1/997: stat d4/d9/d3d/d3f/dd0/d152/dca/dcc/d98/ddd 0 2024-02-16T22:22:24.384 INFO:tasks.workunit.client.0.smithi114.stdout:1/998: mknod d4/d63/d69/d72/d119/c178 0 2024-02-16T22:22:24.387 INFO:tasks.workunit.client.0.smithi114.stdout:1/999: symlink d4/d63/d69/d7c/d9b/dae/l179 0 2024-02-16T22:22:24.439 INFO:tasks.workunit.client.0.smithi114.stdout:5/951: sync 2024-02-16T22:22:24.439 INFO:tasks.workunit.client.0.smithi114.stdout:5/952: readlink d0/df/l12f 0 2024-02-16T22:22:24.442 INFO:tasks.workunit.client.0.smithi114.stdout:5/953: symlink d0/df/d7a/dc7/l13b 0 2024-02-16T22:22:24.442 INFO:tasks.workunit.client.0.smithi114.stdout:5/954: readlink d0/d28/d98/d25/d89/lcd 0 2024-02-16T22:22:24.445 INFO:tasks.workunit.client.0.smithi114.stdout:5/955: symlink d0/d28/d98/d1e/d63/d9e/l13c 0 2024-02-16T22:22:24.447 INFO:tasks.workunit.client.0.smithi114.stdout:5/956: mknod d0/d28/d98/d25/d89/de2/db3/c13d 0 2024-02-16T22:22:24.449 INFO:tasks.workunit.client.0.smithi114.stdout:5/957: unlink d0/d28/d98/d1e/dd7/f101 0 2024-02-16T22:22:24.451 INFO:tasks.workunit.client.0.smithi114.stdout:5/958: mknod d0/d28/d98/dd0/d11b/c13e 0 2024-02-16T22:22:24.455 INFO:tasks.workunit.client.0.smithi114.stdout:5/959: rmdir d0/d138 0 2024-02-16T22:22:24.456 INFO:tasks.workunit.client.0.smithi114.stdout:5/960: read d0/d28/d98/d25/d2e/f7b [252233,25458] 0 2024-02-16T22:22:24.457 INFO:tasks.workunit.client.0.smithi114.stdout:5/961: fsync d0/df/d5e/f10f 0 2024-02-16T22:22:24.463 INFO:tasks.workunit.client.0.smithi114.stdout:5/962: dwrite d0/d28/d98/d1e/d86/f12a [0,4194304] 0 2024-02-16T22:22:24.464 INFO:tasks.workunit.client.0.smithi114.stdout:5/963: truncate d0/d28/d98/d25/f123 806570 0 2024-02-16T22:22:24.466 INFO:tasks.workunit.client.0.smithi114.stdout:5/964: read d0/d28/d98/d25/d2e/f69 [2345782,34478] 0 2024-02-16T22:22:24.485 INFO:tasks.workunit.client.0.smithi114.stdout:5/965: dread d0/d28/d98/d1e/d86/f7c [0,4194304] 0 2024-02-16T22:22:24.492 INFO:tasks.workunit.client.0.smithi114.stdout:5/966: dwrite d0/df/d6d/f108 [0,4194304] 0 2024-02-16T22:22:24.496 INFO:tasks.workunit.client.0.smithi114.stdout:5/967: creat d0/d28/d98/d25/d89/de2/d9f/f13f x:0 0 0 2024-02-16T22:22:24.746 INFO:tasks.workunit.client.0.smithi114.stdout:5/968: sync 2024-02-16T22:22:24.752 INFO:tasks.workunit.client.0.smithi114.stdout:5/969: dwrite d0/d28/d98/f5b [4194304,4194304] 0 2024-02-16T22:22:24.755 INFO:tasks.workunit.client.0.smithi114.stdout:5/970: creat d0/d28/d98/d1e/dfa/f140 x:0 0 0 2024-02-16T22:22:24.763 INFO:tasks.workunit.client.0.smithi114.stdout:5/971: dwrite d0/df/d53/db1/d119/f12c [0,4194304] 0 2024-02-16T22:22:24.765 INFO:tasks.workunit.client.0.smithi114.stdout:5/972: readlink d0/d28/d98/d25/d89/de2/l6c 0 2024-02-16T22:22:24.795 INFO:tasks.workunit.client.0.smithi114.stdout:5/973: dread d0/d28/d98/d25/d2e/d3b/f85 [0,4194304] 0 2024-02-16T22:22:24.814 INFO:tasks.workunit.client.0.smithi114.stdout:5/974: dwrite d0/d28/d98/d25/d2e/f69 [0,4194304] 0 2024-02-16T22:22:24.817 INFO:tasks.workunit.client.0.smithi114.stdout:5/975: symlink d0/d28/d98/d25/d89/l141 0 2024-02-16T22:22:24.819 INFO:tasks.workunit.client.0.smithi114.stdout:5/976: readlink d0/l6 0 2024-02-16T22:22:24.824 INFO:tasks.workunit.client.0.smithi114.stdout:5/977: mkdir d0/d28/d98/d25/d89/de2/d9f/d131/d142 0 2024-02-16T22:22:24.825 INFO:tasks.workunit.client.0.smithi114.stdout:5/978: chown d0/df/d53/db1/d119/f11f 1 1 2024-02-16T22:22:24.829 INFO:tasks.workunit.client.0.smithi114.stdout:5/979: link d0/d28/d98/d25/c125 d0/d28/d98/d25/d89/d99/db0/c143 0 2024-02-16T22:22:24.830 INFO:tasks.workunit.client.0.smithi114.stdout:5/980: mknod d0/df/d7a/dc7/c144 0 2024-02-16T22:22:24.831 INFO:tasks.workunit.client.0.smithi114.stdout:5/981: stat d0/d28/d98/f12e 0 2024-02-16T22:22:24.831 INFO:tasks.workunit.client.0.smithi114.stdout:5/982: readlink d0/df/d53/db1/lbe 0 2024-02-16T22:22:24.833 INFO:tasks.workunit.client.0.smithi114.stdout:5/983: creat d0/d28/d98/d25/d3e/d7f/f145 x:0 0 0 2024-02-16T22:22:24.835 INFO:tasks.workunit.client.0.smithi114.stdout:5/984: mknod d0/d28/d98/d1e/d63/df9/c146 0 2024-02-16T22:22:24.837 INFO:tasks.workunit.client.0.smithi114.stdout:5/985: rmdir d0/d28/d98/d1e/dfa 39 2024-02-16T22:22:24.843 INFO:tasks.workunit.client.0.smithi114.stdout:5/986: creat d0/d28/d98/d25/d89/de2/d9f/d131/d142/f147 x:0 0 0 2024-02-16T22:22:24.845 INFO:tasks.workunit.client.0.smithi114.stdout:5/987: creat d0/d28/d98/d25/d89/f148 x:0 0 0 2024-02-16T22:22:24.853 INFO:tasks.workunit.client.0.smithi114.stdout:5/988: write d0/df/d7a/fea [509392,50458] 0 2024-02-16T22:22:24.855 INFO:tasks.workunit.client.0.smithi114.stdout:5/989: symlink d0/d28/l149 0 2024-02-16T22:22:24.859 INFO:tasks.workunit.client.0.smithi114.stdout:5/990: mknod d0/d28/d98/d1e/d63/d9e/c14a 0 2024-02-16T22:22:24.862 INFO:tasks.workunit.client.0.smithi114.stdout:5/991: rename d0/d28/d98/d1e/d63/d96/d9a/fa6 to d0/d28/d98/d25/d2e/d3b/f14b 0 2024-02-16T22:22:24.867 INFO:tasks.workunit.client.0.smithi114.stdout:5/992: rename d0/df/d53/db1 to d0/d28/d98/d1e/d63/df9/d14c 0 2024-02-16T22:22:24.868 INFO:tasks.workunit.client.0.smithi114.stdout:5/993: chown d0/d28/d98/d25/d2e/d3b/ffb 18300324 1 2024-02-16T22:22:24.870 INFO:tasks.workunit.client.0.smithi114.stdout:5/994: symlink d0/d28/d98/d25/d3e/l14d 0 2024-02-16T22:22:24.871 INFO:tasks.workunit.client.0.smithi114.stdout:5/995: chown d0/d28/d98/d1e/d63/d9e/fb2 1 1 2024-02-16T22:22:24.873 INFO:tasks.workunit.client.0.smithi114.stdout:5/996: read - d0/d28/d98/d1e/d63/d96/f104 zero size 2024-02-16T22:22:24.875 INFO:tasks.workunit.client.0.smithi114.stdout:5/997: creat d0/d28/d98/d25/d89/f14e x:0 0 0 2024-02-16T22:22:24.879 INFO:tasks.workunit.client.0.smithi114.stdout:5/998: truncate d0/d28/d98/dd0/d11b/f11d 1113968 0 2024-02-16T22:22:24.883 INFO:tasks.workunit.client.0.smithi114.stdout:5/999: link d0/df/l12f d0/d28/d98/d1e/d86/l14f 0 2024-02-16T22:22:24.887 INFO:tasks.workunit.client.0.smithi114.stderr:+ rm -rf -- ./tmp.WttGWFrPDP 2024-02-16T22:22:25.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:25 smithi114 ceph-mon[29235]: pgmap v40: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 53 MiB/s rd, 127 MiB/s wr, 320 op/s 2024-02-16T22:22:25.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:25 smithi123 ceph-mon[37161]: pgmap v40: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 53 MiB/s rd, 127 MiB/s wr, 320 op/s 2024-02-16T22:22:26.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:26 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:22:26.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:26 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:22:26.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:26 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:22:26.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:26 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:22:26.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:26 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:22:26.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:26 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:22:27.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:27 smithi123 ceph-mon[37161]: pgmap v41: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 53 MiB/s rd, 127 MiB/s wr, 320 op/s 2024-02-16T22:22:27.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:27 smithi114 ceph-mon[29235]: pgmap v41: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 53 MiB/s rd, 127 MiB/s wr, 320 op/s 2024-02-16T22:22:28.568 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:28 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:22:28.568 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:28 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:22:28.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:28 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:22:28.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:28 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:22:29.651 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:29 smithi114 ceph-mon[29235]: pgmap v42: 65 pgs: 65 active+clean; 3.6 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 60 MiB/s rd, 147 MiB/s wr, 412 op/s 2024-02-16T22:22:29.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:29 smithi123 ceph-mon[37161]: pgmap v42: 65 pgs: 65 active+clean; 3.6 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 60 MiB/s rd, 147 MiB/s wr, 412 op/s 2024-02-16T22:22:31.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:31 smithi114 ceph-mon[29235]: pgmap v43: 65 pgs: 65 active+clean; 3.6 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 23 MiB/s rd, 68 MiB/s wr, 223 op/s 2024-02-16T22:22:31.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:31 smithi123 ceph-mon[37161]: pgmap v43: 65 pgs: 65 active+clean; 3.6 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 23 MiB/s rd, 68 MiB/s wr, 223 op/s 2024-02-16T22:22:32.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:22:32.953 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:22:32.953 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:22:32.953 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:22:32.953 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:22:32.953 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:22:32.953 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "mgr fail", "who": "smithi123.inzgjd"}]: dispatch 2024-02-16T22:22:32.953 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "mgr fail", "who": "smithi123.inzgjd"}]: dispatch 2024-02-16T22:22:32.953 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: osdmap e47: 6 total, 6 up, 6 in 2024-02-16T22:22:32.953 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' cmd='[{"prefix": "mgr fail", "who": "smithi123.inzgjd"}]': finished 2024-02-16T22:22:32.953 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: mgrmap e26: smithi114.vcullz(active, starting, since 0.00639034s) 2024-02-16T22:22:32.953 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi114"}]: dispatch 2024-02-16T22:22:32.954 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi123"}]: dispatch 2024-02-16T22:22:32.954 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi114.zzryke"}]: dispatch 2024-02-16T22:22:32.954 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi114.fdxpey"}]: dispatch 2024-02-16T22:22:32.954 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi123.vhlqbe"}]: dispatch 2024-02-16T22:22:32.954 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi123.gopcfa"}]: dispatch 2024-02-16T22:22:32.954 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mgr metadata", "who": "smithi114.vcullz", "id": "smithi114.vcullz"}]: dispatch 2024-02-16T22:22:32.954 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:22:32.954 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:22:32.954 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:22:32.954 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:22:32.954 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:22:32.954 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:22:32.954 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T22:22:32.954 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T22:22:32.954 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T22:22:32.955 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: Manager daemon smithi114.vcullz is now available 2024-02-16T22:22:32.955 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:22:32.955 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:22:32.955 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:22:32.955 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi114.vcullz/mirror_snapshot_schedule"}]: dispatch 2024-02-16T22:22:32.955 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi114.vcullz/trash_purge_schedule"}]: dispatch 2024-02-16T22:22:32.955 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: [16/Feb/2024:22:22:32] ENGINE Bus STARTING 2024-02-16T22:22:32.955 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:32 smithi114 ceph-mon[29235]: [16/Feb/2024:22:22:32] ENGINE Serving on https://172.21.15.114:7150 2024-02-16T22:22:33.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:22:33.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:22:33.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:22:33.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:22:33.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' 2024-02-16T22:22:33.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:22:33.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: from='mgr.24441 172.21.15.123:0/2756904252' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "mgr fail", "who": "smithi123.inzgjd"}]: dispatch 2024-02-16T22:22:33.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "mgr fail", "who": "smithi123.inzgjd"}]: dispatch 2024-02-16T22:22:33.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: osdmap e47: 6 total, 6 up, 6 in 2024-02-16T22:22:33.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: from='mgr.24441 ' entity='mgr.smithi123.inzgjd' cmd='[{"prefix": "mgr fail", "who": "smithi123.inzgjd"}]': finished 2024-02-16T22:22:33.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: mgrmap e26: smithi114.vcullz(active, starting, since 0.00639034s) 2024-02-16T22:22:33.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi114"}]: dispatch 2024-02-16T22:22:33.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi123"}]: dispatch 2024-02-16T22:22:33.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi114.zzryke"}]: dispatch 2024-02-16T22:22:33.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi114.fdxpey"}]: dispatch 2024-02-16T22:22:33.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi123.vhlqbe"}]: dispatch 2024-02-16T22:22:33.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi123.gopcfa"}]: dispatch 2024-02-16T22:22:33.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mgr metadata", "who": "smithi114.vcullz", "id": "smithi114.vcullz"}]: dispatch 2024-02-16T22:22:33.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:22:33.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:22:33.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:22:33.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:22:33.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:22:33.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:22:33.055 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T22:22:33.055 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T22:22:33.055 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T22:22:33.055 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: Manager daemon smithi114.vcullz is now available 2024-02-16T22:22:33.055 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:22:33.055 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:22:33.055 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:22:33.055 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi114.vcullz/mirror_snapshot_schedule"}]: dispatch 2024-02-16T22:22:33.055 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi114.vcullz/trash_purge_schedule"}]: dispatch 2024-02-16T22:22:33.055 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: [16/Feb/2024:22:22:32] ENGINE Bus STARTING 2024-02-16T22:22:33.055 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:32 smithi123 ceph-mon[37161]: [16/Feb/2024:22:22:32] ENGINE Serving on https://172.21.15.114:7150 2024-02-16T22:22:34.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:33 smithi114 ceph-mon[29235]: [16/Feb/2024:22:22:32] ENGINE Serving on http://172.21.15.114:8765 2024-02-16T22:22:34.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:33 smithi114 ceph-mon[29235]: [16/Feb/2024:22:22:32] ENGINE Bus STARTED 2024-02-16T22:22:34.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:33 smithi114 ceph-mon[29235]: mgrmap e27: smithi114.vcullz(active, since 1.01767s) 2024-02-16T22:22:34.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:33 smithi114 ceph-mon[29235]: pgmap v3: 65 pgs: 65 active+clean; 2.0 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail 2024-02-16T22:22:34.238 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:33 smithi123 ceph-mon[37161]: [16/Feb/2024:22:22:32] ENGINE Serving on http://172.21.15.114:8765 2024-02-16T22:22:34.238 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:33 smithi123 ceph-mon[37161]: [16/Feb/2024:22:22:32] ENGINE Bus STARTED 2024-02-16T22:22:34.238 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:33 smithi123 ceph-mon[37161]: mgrmap e27: smithi114.vcullz(active, since 1.01767s) 2024-02-16T22:22:34.239 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:33 smithi123 ceph-mon[37161]: pgmap v3: 65 pgs: 65 active+clean; 2.0 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail 2024-02-16T22:22:35.132 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:34 smithi123 ceph-mon[37161]: pgmap v4: 65 pgs: 65 active+clean; 2.0 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail 2024-02-16T22:22:35.132 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:34 smithi123 ceph-mon[37161]: mgrmap e28: smithi114.vcullz(active, since 2s) 2024-02-16T22:22:35.223 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-16T22:22:35.223 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-02-16T22:22:35.324 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:34 smithi114 ceph-mon[29235]: pgmap v4: 65 pgs: 65 active+clean; 2.0 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail 2024-02-16T22:22:35.324 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:34 smithi114 ceph-mon[29235]: mgrmap e28: smithi114.vcullz(active, since 2s) 2024-02-16T22:22:36.892 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:36 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:22:36.892 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:36 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:22:36.892 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:36 smithi123 ceph-mon[37161]: pgmap v5: 65 pgs: 65 active+clean; 2.0 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail 2024-02-16T22:22:36.892 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:36 smithi123 ceph-mon[37161]: mgrmap e29: smithi114.vcullz(active, since 4s) 2024-02-16T22:22:37.114 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:36 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:22:37.114 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:36 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:22:37.114 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:36 smithi114 ceph-mon[29235]: pgmap v5: 65 pgs: 65 active+clean; 2.0 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail 2024-02-16T22:22:37.114 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:36 smithi114 ceph-mon[29235]: mgrmap e29: smithi114.vcullz(active, since 4s) 2024-02-16T22:22:38.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:38 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:22:38.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:38 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:22:38.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:38 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:22:38.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:38 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:22:39.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:39 smithi123 ceph-mon[37161]: pgmap v6: 65 pgs: 65 active+clean; 2.0 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail 2024-02-16T22:22:39.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:39 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:22:39.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:39 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:22:39.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:39 smithi114 ceph-mon[29235]: pgmap v6: 65 pgs: 65 active+clean; 2.0 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail 2024-02-16T22:22:39.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:39 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:22:39.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:39 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:22:41.000 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:40 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:22:41.000 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:40 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:22:41.000 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:40 smithi114 ceph-mon[29235]: Standby manager daemon smithi123.inzgjd started 2024-02-16T22:22:41.000 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:40 smithi114 ceph-mon[29235]: from='mgr.? 172.21.15.123:0/1515253631' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi123.inzgjd/crt"}]: dispatch 2024-02-16T22:22:41.000 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:40 smithi114 ceph-mon[29235]: from='mgr.? 172.21.15.123:0/1515253631' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T22:22:41.000 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:40 smithi114 ceph-mon[29235]: from='mgr.? 172.21.15.123:0/1515253631' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi123.inzgjd/key"}]: dispatch 2024-02-16T22:22:41.000 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:40 smithi114 ceph-mon[29235]: from='mgr.? 172.21.15.123:0/1515253631' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T22:22:41.001 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:40 smithi114 ceph-mon[29235]: pgmap v7: 65 pgs: 65 active+clean; 984 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 51 KiB/s rd, 1.1 MiB/s wr, 144 op/s 2024-02-16T22:22:41.203 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:40 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:22:41.204 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:40 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:22:41.204 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:40 smithi123 ceph-mon[37161]: Standby manager daemon smithi123.inzgjd started 2024-02-16T22:22:41.204 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:40 smithi123 ceph-mon[37161]: from='mgr.? 172.21.15.123:0/1515253631' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi123.inzgjd/crt"}]: dispatch 2024-02-16T22:22:41.204 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:40 smithi123 ceph-mon[37161]: from='mgr.? 172.21.15.123:0/1515253631' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T22:22:41.204 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:40 smithi123 ceph-mon[37161]: from='mgr.? 172.21.15.123:0/1515253631' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi123.inzgjd/key"}]: dispatch 2024-02-16T22:22:41.204 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:40 smithi123 ceph-mon[37161]: from='mgr.? 172.21.15.123:0/1515253631' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T22:22:41.204 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:40 smithi123 ceph-mon[37161]: pgmap v7: 65 pgs: 65 active+clean; 984 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 51 KiB/s rd, 1.1 MiB/s wr, 144 op/s 2024-02-16T22:22:41.680 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-16T22:22:41.681 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-16T22:22:42.020 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:41 smithi114 ceph-mon[29235]: mgrmap e30: smithi114.vcullz(active, since 8s), standbys: smithi123.inzgjd 2024-02-16T22:22:42.021 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:41 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mgr metadata", "who": "smithi123.inzgjd", "id": "smithi123.inzgjd"}]: dispatch 2024-02-16T22:22:42.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:41 smithi123 ceph-mon[37161]: mgrmap e30: smithi114.vcullz(active, since 8s), standbys: smithi123.inzgjd 2024-02-16T22:22:42.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:41 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mgr metadata", "who": "smithi123.inzgjd", "id": "smithi123.inzgjd"}]: dispatch 2024-02-16T22:22:43.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:42 smithi114 ceph-mon[29235]: pgmap v8: 65 pgs: 65 active+clean; 984 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 40 KiB/s rd, 840 KiB/s wr, 112 op/s 2024-02-16T22:22:43.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:42 smithi123 ceph-mon[37161]: pgmap v8: 65 pgs: 65 active+clean; 984 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 40 KiB/s rd, 840 KiB/s wr, 112 op/s 2024-02-16T22:22:45.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:45 smithi114 ceph-mon[29235]: pgmap v9: 65 pgs: 65 active+clean; 430 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 45 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-02-16T22:22:45.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:45 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:22:45.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:45 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:22:45.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:45 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:22:45.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:45 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:22:45.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:45 smithi114 ceph-mon[29235]: Updating smithi114:/etc/ceph/ceph.conf 2024-02-16T22:22:45.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:45 smithi114 ceph-mon[29235]: Updating smithi123:/etc/ceph/ceph.conf 2024-02-16T22:22:45.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:45 smithi114 ceph-mon[29235]: Updating smithi114:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:22:45.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:45 smithi114 ceph-mon[29235]: Updating smithi123:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:22:45.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:45 smithi123 ceph-mon[37161]: pgmap v9: 65 pgs: 65 active+clean; 430 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 45 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-02-16T22:22:45.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:45 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:22:45.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:45 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:22:45.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:45 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:22:45.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:45 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:22:45.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:45 smithi123 ceph-mon[37161]: Updating smithi114:/etc/ceph/ceph.conf 2024-02-16T22:22:45.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:45 smithi123 ceph-mon[37161]: Updating smithi123:/etc/ceph/ceph.conf 2024-02-16T22:22:45.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:45 smithi123 ceph-mon[37161]: Updating smithi114:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:22:45.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:45 smithi123 ceph-mon[37161]: Updating smithi123:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:22:46.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:46 smithi114 ceph-mon[29235]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:22:46.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:46 smithi114 ceph-mon[29235]: Updating smithi114:/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:22:46.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:46 smithi114 ceph-mon[29235]: Updating smithi123:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T22:22:46.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:46 smithi114 ceph-mon[29235]: Updating smithi114:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T22:22:46.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:46 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:22:46.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:46 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:22:46.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:46 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:22:46.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:46 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:22:46.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:46 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:22:46.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:46 smithi114 ceph-mon[29235]: Reconfiguring prometheus.smithi114 (dependencies changed)... 2024-02-16T22:22:46.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:46 smithi123 ceph-mon[37161]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:22:46.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:46 smithi123 ceph-mon[37161]: Updating smithi114:/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:22:46.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:46 smithi123 ceph-mon[37161]: Updating smithi123:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T22:22:46.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:46 smithi123 ceph-mon[37161]: Updating smithi114:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T22:22:46.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:46 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:22:46.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:46 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:22:46.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:46 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:22:46.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:46 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:22:46.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:46 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:22:46.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:46 smithi123 ceph-mon[37161]: Reconfiguring prometheus.smithi114 (dependencies changed)... 2024-02-16T22:22:47.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:47 smithi114 ceph-mon[29235]: Reconfiguring daemon prometheus.smithi114 on smithi114 2024-02-16T22:22:47.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:47 smithi114 ceph-mon[29235]: pgmap v10: 65 pgs: 65 active+clean; 430 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 41 KiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-02-16T22:22:47.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:47 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:22:48.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:47 smithi123 ceph-mon[37161]: Reconfiguring daemon prometheus.smithi114 on smithi114 2024-02-16T22:22:48.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:47 smithi123 ceph-mon[37161]: pgmap v10: 65 pgs: 65 active+clean; 430 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 41 KiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-02-16T22:22:48.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:47 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:22:49.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.320+0000 7f7cdacb5700 1 -- 172.21.15.114:0/1307824127 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cd4103fe0 msgr2=0x7f7cd41063d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:49.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.320+0000 7f7cdacb5700 1 --2- 172.21.15.114:0/1307824127 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cd4103fe0 0x7f7cd41063d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0009910 tx=0x7f7cc0009c20 comp rx=0 tx=0).stop 2024-02-16T22:22:49.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.320+0000 7f7cdacb5700 1 -- 172.21.15.114:0/1307824127 shutdown_connections 2024-02-16T22:22:49.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.320+0000 7f7cdacb5700 1 --2- 172.21.15.114:0/1307824127 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cd4103fe0 0x7f7cd41063d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:49.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.320+0000 7f7cdacb5700 1 --2- 172.21.15.114:0/1307824127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cd41016b0 0x7f7cd4103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:49.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.320+0000 7f7cdacb5700 1 -- 172.21.15.114:0/1307824127 >> 172.21.15.114:0/1307824127 conn(0x7f7cd40fb060 msgr2=0x7f7cd40fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:22:49.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.320+0000 7f7cdacb5700 1 -- 172.21.15.114:0/1307824127 shutdown_connections 2024-02-16T22:22:49.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.320+0000 7f7cdacb5700 1 -- 172.21.15.114:0/1307824127 wait complete. 2024-02-16T22:22:49.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.321+0000 7f7cdacb5700 1 Processor -- start 2024-02-16T22:22:49.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.321+0000 7f7cdacb5700 1 -- start start 2024-02-16T22:22:49.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.321+0000 7f7cdacb5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cd41016b0 0x7f7cd4101010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:49.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.321+0000 7f7cdacb5700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cd4103fe0 0x7f7cd40ff660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:49.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.321+0000 7f7cdacb5700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd40ffba0 con 0x7f7cd4103fe0 2024-02-16T22:22:49.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.321+0000 7f7cdacb5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd40ffce0 con 0x7f7cd41016b0 2024-02-16T22:22:49.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.321+0000 7f7cd3fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cd4103fe0 0x7f7cd40ff660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:49.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.321+0000 7f7cd3fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cd4103fe0 0x7f7cd40ff660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57722/0 (socket says 172.21.15.114:57722) 2024-02-16T22:22:49.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.321+0000 7f7cd3fff700 1 -- 172.21.15.114:0/692866538 learned_addr learned my addr 172.21.15.114:0/692866538 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:22:49.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.321+0000 7f7cd8a51700 1 --2- 172.21.15.114:0/692866538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cd41016b0 0x7f7cd4101010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:49.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.321+0000 7f7cd3fff700 1 -- 172.21.15.114:0/692866538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cd41016b0 msgr2=0x7f7cd4101010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:49.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.321+0000 7f7cd3fff700 1 --2- 172.21.15.114:0/692866538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cd41016b0 0x7f7cd4101010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:49.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.321+0000 7f7cd3fff700 1 -- 172.21.15.114:0/692866538 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc00095f0 con 0x7f7cd4103fe0 2024-02-16T22:22:49.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.322+0000 7f7cd3fff700 1 --2- 172.21.15.114:0/692866538 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cd4103fe0 0x7f7cd40ff660 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7cc000bd70 tx=0x7f7cc000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:22:49.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.322+0000 7f7cd1ffb700 1 -- 172.21.15.114:0/692866538 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cc001d070 con 0x7f7cd4103fe0 2024-02-16T22:22:49.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.322+0000 7f7cd1ffb700 1 -- 172.21.15.114:0/692866538 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7cc0005dc0 con 0x7f7cd4103fe0 2024-02-16T22:22:49.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.322+0000 7f7cdacb5700 1 -- 172.21.15.114:0/692866538 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cd40fff60 con 0x7f7cd4103fe0 2024-02-16T22:22:49.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.322+0000 7f7cd1ffb700 1 -- 172.21.15.114:0/692866538 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cc0021410 con 0x7f7cd4103fe0 2024-02-16T22:22:49.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.322+0000 7f7cdacb5700 1 -- 172.21.15.114:0/692866538 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cd4100450 con 0x7f7cd4103fe0 2024-02-16T22:22:49.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.323+0000 7f7cdacb5700 1 -- 172.21.15.114:0/692866538 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cd4190530 con 0x7f7cd4103fe0 2024-02-16T22:22:49.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.323+0000 7f7cd1ffb700 1 -- 172.21.15.114:0/692866538 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 30) v1 ==== 91450+0+0 (secure 0 0 0) 0x7f7cc000f7d0 con 0x7f7cd4103fe0 2024-02-16T22:22:49.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.324+0000 7f7cd1ffb700 1 --2- 172.21.15.114:0/692866538 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f7cc406d660 0x7f7cc406fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:49.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.324+0000 7f7cd1ffb700 1 -- 172.21.15.114:0/692866538 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f7cc008e480 con 0x7f7cd4103fe0 2024-02-16T22:22:49.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.324+0000 7f7cd8a51700 1 --2- 172.21.15.114:0/692866538 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f7cc406d660 0x7f7cc406fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:49.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.324+0000 7f7cd8a51700 1 --2- 172.21.15.114:0/692866538 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f7cc406d660 0x7f7cc406fb20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8005950 tx=0x7f7cc800b410 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:22:49.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.329+0000 7f7cd1ffb700 1 -- 172.21.15.114:0/692866538 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182869 (secure 0 0 0) 0x7f7cc005bac0 con 0x7f7cd4103fe0 2024-02-16T22:22:49.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.626+0000 7f7cdacb5700 1 -- 172.21.15.114:0/692866538 --> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7cd40611d0 con 0x7f7cc406d660 2024-02-16T22:22:49.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.627+0000 7f7cd1ffb700 1 -- 172.21.15.114:0/692866538 <== mgr.14764 v2:172.21.15.114:6800/3051780151 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f7cd40611d0 con 0x7f7cc406d660 2024-02-16T22:22:49.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.630+0000 7f7cdacb5700 1 -- 172.21.15.114:0/692866538 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f7cc406d660 msgr2=0x7f7cc406fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:49.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.630+0000 7f7cdacb5700 1 --2- 172.21.15.114:0/692866538 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f7cc406d660 0x7f7cc406fb20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8005950 tx=0x7f7cc800b410 comp rx=0 tx=0).stop 2024-02-16T22:22:49.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.630+0000 7f7cdacb5700 1 -- 172.21.15.114:0/692866538 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cd4103fe0 msgr2=0x7f7cd40ff660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:49.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.630+0000 7f7cdacb5700 1 --2- 172.21.15.114:0/692866538 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cd4103fe0 0x7f7cd40ff660 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7cc000bd70 tx=0x7f7cc000bda0 comp rx=0 tx=0).stop 2024-02-16T22:22:49.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.630+0000 7f7cdacb5700 1 -- 172.21.15.114:0/692866538 shutdown_connections 2024-02-16T22:22:49.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.630+0000 7f7cdacb5700 1 --2- 172.21.15.114:0/692866538 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f7cc406d660 0x7f7cc406fb20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:49.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.630+0000 7f7cdacb5700 1 --2- 172.21.15.114:0/692866538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cd41016b0 0x7f7cd4101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:49.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.630+0000 7f7cdacb5700 1 --2- 172.21.15.114:0/692866538 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cd4103fe0 0x7f7cd40ff660 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:49.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.630+0000 7f7cdacb5700 1 -- 172.21.15.114:0/692866538 >> 172.21.15.114:0/692866538 conn(0x7f7cd40fb060 msgr2=0x7f7cd40fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:22:49.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.630+0000 7f7cdacb5700 1 -- 172.21.15.114:0/692866538 shutdown_connections 2024-02-16T22:22:49.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.630+0000 7f7cdacb5700 1 -- 172.21.15.114:0/692866538 wait complete. 2024-02-16T22:22:49.643 INFO:teuthology.orchestra.run.smithi114.stdout:true 2024-02-16T22:22:49.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.764+0000 7fc84e7c7700 1 -- 172.21.15.114:0/2072096827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc848101070 msgr2=0x7fc848101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:49.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.764+0000 7fc84e7c7700 1 --2- 172.21.15.114:0/2072096827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc848101070 0x7fc848101490 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc838009910 tx=0x7fc838009c20 comp rx=0 tx=0).stop 2024-02-16T22:22:49.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.764+0000 7fc84e7c7700 1 -- 172.21.15.114:0/2072096827 shutdown_connections 2024-02-16T22:22:49.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.764+0000 7fc84e7c7700 1 --2- 172.21.15.114:0/2072096827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8480ffcd0 0x7fc848100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:49.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.764+0000 7fc84e7c7700 1 --2- 172.21.15.114:0/2072096827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc848101070 0x7fc848101490 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:49.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.764+0000 7fc84e7c7700 1 -- 172.21.15.114:0/2072096827 >> 172.21.15.114:0/2072096827 conn(0x7fc8480fa7c0 msgr2=0x7fc8480fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:22:49.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.764+0000 7fc84e7c7700 1 -- 172.21.15.114:0/2072096827 shutdown_connections 2024-02-16T22:22:49.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.764+0000 7fc84e7c7700 1 -- 172.21.15.114:0/2072096827 wait complete. 2024-02-16T22:22:49.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.765+0000 7fc84e7c7700 1 Processor -- start 2024-02-16T22:22:49.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.765+0000 7fc84e7c7700 1 -- start start 2024-02-16T22:22:49.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.765+0000 7fc84e7c7700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8480ffcd0 0x7fc848196350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:49.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.765+0000 7fc84e7c7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc848101070 0x7fc848196890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:49.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.765+0000 7fc84e7c7700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc848196eb0 con 0x7fc8480ffcd0 2024-02-16T22:22:49.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.765+0000 7fc84e7c7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc848196ff0 con 0x7fc848101070 2024-02-16T22:22:49.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.765+0000 7fc847fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8480ffcd0 0x7fc848196350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:49.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.765+0000 7fc847fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8480ffcd0 0x7fc848196350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57732/0 (socket says 172.21.15.114:57732) 2024-02-16T22:22:49.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.765+0000 7fc847fff700 1 -- 172.21.15.114:0/2160726147 learned_addr learned my addr 172.21.15.114:0/2160726147 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:22:49.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.765+0000 7fc8477fe700 1 --2- 172.21.15.114:0/2160726147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc848101070 0x7fc848196890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:49.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.766+0000 7fc847fff700 1 -- 172.21.15.114:0/2160726147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc848101070 msgr2=0x7fc848196890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:49.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.766+0000 7fc847fff700 1 --2- 172.21.15.114:0/2160726147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc848101070 0x7fc848196890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:49.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.766+0000 7fc847fff700 1 -- 172.21.15.114:0/2160726147 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8380095f0 con 0x7fc8480ffcd0 2024-02-16T22:22:49.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.766+0000 7fc847fff700 1 --2- 172.21.15.114:0/2160726147 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8480ffcd0 0x7fc848196350 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc83800bd70 tx=0x7fc83800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:22:49.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.766+0000 7fc8457fa700 1 -- 172.21.15.114:0/2160726147 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc83801d070 con 0x7fc8480ffcd0 2024-02-16T22:22:49.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.766+0000 7fc8457fa700 1 -- 172.21.15.114:0/2160726147 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc838005dc0 con 0x7fc8480ffcd0 2024-02-16T22:22:49.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.766+0000 7fc8457fa700 1 -- 172.21.15.114:0/2160726147 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc838021410 con 0x7fc8480ffcd0 2024-02-16T22:22:49.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.766+0000 7fc84e7c7700 1 -- 172.21.15.114:0/2160726147 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc84819ba40 con 0x7fc8480ffcd0 2024-02-16T22:22:49.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.766+0000 7fc84e7c7700 1 -- 172.21.15.114:0/2160726147 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc84819bf30 con 0x7fc8480ffcd0 2024-02-16T22:22:49.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.767+0000 7fc84e7c7700 1 -- 172.21.15.114:0/2160726147 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc848108610 con 0x7fc8480ffcd0 2024-02-16T22:22:49.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.768+0000 7fc8457fa700 1 -- 172.21.15.114:0/2160726147 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 30) v1 ==== 91450+0+0 (secure 0 0 0) 0x7fc83800f7d0 con 0x7fc8480ffcd0 2024-02-16T22:22:49.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.768+0000 7fc8457fa700 1 --2- 172.21.15.114:0/2160726147 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7fc83006d660 0x7fc83006fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:49.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.768+0000 7fc8457fa700 1 -- 172.21.15.114:0/2160726147 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7fc83808e390 con 0x7fc8480ffcd0 2024-02-16T22:22:49.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.768+0000 7fc8477fe700 1 --2- 172.21.15.114:0/2160726147 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7fc83006d660 0x7fc83006fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:49.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.768+0000 7fc8477fe700 1 --2- 172.21.15.114:0/2160726147 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7fc83006d660 0x7fc83006fb20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc83c007c60 tx=0x7fc83c0073d0 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:22:49.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:49.773+0000 7fc8457fa700 1 -- 172.21.15.114:0/2160726147 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182869 (secure 0 0 0) 0x7fc83805b9d0 con 0x7fc8480ffcd0 2024-02-16T22:22:49.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:49 smithi114 ceph-mon[29235]: pgmap v11: 65 pgs: 65 active+clean; 430 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 41 KiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-02-16T22:22:50.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:49 smithi123 ceph-mon[37161]: pgmap v11: 65 pgs: 65 active+clean; 430 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 41 KiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-02-16T22:22:50.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.079+0000 7fc84e7c7700 1 -- 172.21.15.114:0/2160726147 --> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8480ff2b0 con 0x7fc83006d660 2024-02-16T22:22:50.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.080+0000 7fc8457fa700 1 -- 172.21.15.114:0/2160726147 <== mgr.14764 v2:172.21.15.114:6800/3051780151 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fc8480ff2b0 con 0x7fc83006d660 2024-02-16T22:22:50.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.082+0000 7fc84e7c7700 1 -- 172.21.15.114:0/2160726147 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7fc83006d660 msgr2=0x7fc83006fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:50.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.082+0000 7fc84e7c7700 1 --2- 172.21.15.114:0/2160726147 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7fc83006d660 0x7fc83006fb20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc83c007c60 tx=0x7fc83c0073d0 comp rx=0 tx=0).stop 2024-02-16T22:22:50.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.082+0000 7fc84e7c7700 1 -- 172.21.15.114:0/2160726147 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8480ffcd0 msgr2=0x7fc848196350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:50.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.082+0000 7fc84e7c7700 1 --2- 172.21.15.114:0/2160726147 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8480ffcd0 0x7fc848196350 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc83800bd70 tx=0x7fc83800bda0 comp rx=0 tx=0).stop 2024-02-16T22:22:50.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.083+0000 7fc84e7c7700 1 -- 172.21.15.114:0/2160726147 shutdown_connections 2024-02-16T22:22:50.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.083+0000 7fc84e7c7700 1 --2- 172.21.15.114:0/2160726147 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7fc83006d660 0x7fc83006fb20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:50.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.083+0000 7fc84e7c7700 1 --2- 172.21.15.114:0/2160726147 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8480ffcd0 0x7fc848196350 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:50.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.083+0000 7fc84e7c7700 1 --2- 172.21.15.114:0/2160726147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc848101070 0x7fc848196890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:50.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.083+0000 7fc84e7c7700 1 -- 172.21.15.114:0/2160726147 >> 172.21.15.114:0/2160726147 conn(0x7fc8480fa7c0 msgr2=0x7fc8480feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:22:50.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.083+0000 7fc84e7c7700 1 -- 172.21.15.114:0/2160726147 shutdown_connections 2024-02-16T22:22:50.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.083+0000 7fc84e7c7700 1 -- 172.21.15.114:0/2160726147 wait complete. 2024-02-16T22:22:50.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.221+0000 7fa66affd700 1 -- 172.21.15.114:0/2350379377 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6600054b0 con 0x7fa66c101db0 2024-02-16T22:22:50.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.222+0000 7fa6724cb700 1 -- 172.21.15.114:0/2350379377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa66c101db0 msgr2=0x7fa66c102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:50.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.222+0000 7fa6724cb700 1 --2- 172.21.15.114:0/2350379377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa66c101db0 0x7fa66c102210 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa660009910 tx=0x7fa660009c20 comp rx=0 tx=0).stop 2024-02-16T22:22:50.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.222+0000 7fa6724cb700 1 -- 172.21.15.114:0/2350379377 shutdown_connections 2024-02-16T22:22:50.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.222+0000 7fa6724cb700 1 --2- 172.21.15.114:0/2350379377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa66c101db0 0x7fa66c102210 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:50.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.222+0000 7fa6724cb700 1 --2- 172.21.15.114:0/2350379377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa66c100bb0 0x7fa66c100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:50.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.222+0000 7fa6724cb700 1 -- 172.21.15.114:0/2350379377 >> 172.21.15.114:0/2350379377 conn(0x7fa66c0fc110 msgr2=0x7fa66c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:22:50.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.222+0000 7fa6724cb700 1 -- 172.21.15.114:0/2350379377 shutdown_connections 2024-02-16T22:22:50.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.222+0000 7fa6724cb700 1 -- 172.21.15.114:0/2350379377 wait complete. 2024-02-16T22:22:50.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.223+0000 7fa6724cb700 1 Processor -- start 2024-02-16T22:22:50.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.223+0000 7fa6724cb700 1 -- start start 2024-02-16T22:22:50.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.223+0000 7fa6724cb700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa66c100bb0 0x7fa66c196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:50.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.223+0000 7fa6724cb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa66c101db0 0x7fa66c1968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:50.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.223+0000 7fa6724cb700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c196ec0 con 0x7fa66c100bb0 2024-02-16T22:22:50.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.223+0000 7fa6724cb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c197000 con 0x7fa66c101db0 2024-02-16T22:22:50.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.223+0000 7fa66bfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa66c100bb0 0x7fa66c196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:50.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.223+0000 7fa66b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa66c101db0 0x7fa66c1968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:50.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.223+0000 7fa66b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa66c101db0 0x7fa66c1968a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.114:50044/0 (socket says 172.21.15.114:50044) 2024-02-16T22:22:50.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.223+0000 7fa66b7fe700 1 -- 172.21.15.114:0/2635291037 learned_addr learned my addr 172.21.15.114:0/2635291037 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:22:50.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.224+0000 7fa66b7fe700 1 -- 172.21.15.114:0/2635291037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa66c100bb0 msgr2=0x7fa66c196360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:50.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.224+0000 7fa66b7fe700 1 --2- 172.21.15.114:0/2635291037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa66c100bb0 0x7fa66c196360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:50.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.224+0000 7fa66b7fe700 1 -- 172.21.15.114:0/2635291037 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6600095f0 con 0x7fa66c101db0 2024-02-16T22:22:50.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.224+0000 7fa66b7fe700 1 --2- 172.21.15.114:0/2635291037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa66c101db0 0x7fa66c1968a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa660006010 tx=0x7fa66000bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:22:50.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.224+0000 7fa6697fa700 1 -- 172.21.15.114:0/2635291037 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa66001d070 con 0x7fa66c101db0 2024-02-16T22:22:50.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.224+0000 7fa6697fa700 1 -- 172.21.15.114:0/2635291037 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa66000f460 con 0x7fa66c101db0 2024-02-16T22:22:50.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.224+0000 7fa6724cb700 1 -- 172.21.15.114:0/2635291037 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa66c19ba50 con 0x7fa66c101db0 2024-02-16T22:22:50.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.224+0000 7fa6697fa700 1 -- 172.21.15.114:0/2635291037 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa660021640 con 0x7fa66c101db0 2024-02-16T22:22:50.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.224+0000 7fa6724cb700 1 -- 172.21.15.114:0/2635291037 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa66c19bf40 con 0x7fa66c101db0 2024-02-16T22:22:50.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.225+0000 7fa6724cb700 1 -- 172.21.15.114:0/2635291037 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa66c109710 con 0x7fa66c101db0 2024-02-16T22:22:50.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.226+0000 7fa6697fa700 1 -- 172.21.15.114:0/2635291037 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 30) v1 ==== 91450+0+0 (secure 0 0 0) 0x7fa66000fa70 con 0x7fa66c101db0 2024-02-16T22:22:50.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.226+0000 7fa6697fa700 1 --2- 172.21.15.114:0/2635291037 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7fa65406d660 0x7fa65406fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:50.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.226+0000 7fa6697fa700 1 -- 172.21.15.114:0/2635291037 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7fa66008e3e0 con 0x7fa66c101db0 2024-02-16T22:22:50.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.226+0000 7fa66bfff700 1 --2- 172.21.15.114:0/2635291037 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7fa65406d660 0x7fa65406fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:50.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.227+0000 7fa66bfff700 1 --2- 172.21.15.114:0/2635291037 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7fa65406d660 0x7fa65406fb20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa66c101c10 tx=0x7fa65c00b410 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:22:50.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.231+0000 7fa6697fa700 1 -- 172.21.15.114:0/2635291037 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182869 (secure 0 0 0) 0x7fa66005baa0 con 0x7fa66c101db0 2024-02-16T22:22:50.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.473+0000 7fa6724cb700 1 -- 172.21.15.114:0/2635291037 --> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa66c106700 con 0x7fa65406d660 2024-02-16T22:22:50.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.488+0000 7fa6697fa700 1 -- 172.21.15.114:0/2635291037 <== mgr.14764 v2:172.21.15.114:6800/3051780151 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fa66c106700 con 0x7fa65406d660 2024-02-16T22:22:50.491 INFO:teuthology.orchestra.run.smithi114.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:22:50.491 INFO:teuthology.orchestra.run.smithi114.stdout:alertmanager.smithi114 smithi114 *:9093,9094 running (8m) 11s ago 10m 24.3M - 0.25.0 c8568f914cd2 e7404fea400e 2024-02-16T22:22:50.491 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi114 smithi114 running (10m) 11s ago 10m 7772k - 18.2.1 94eadd990e0a 7fd07162cd96 2024-02-16T22:22:50.491 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi123 smithi123 running (9m) 14s ago 9m 7503k - 18.2.1 94eadd990e0a 7687f5e99621 2024-02-16T22:22:50.491 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi114 smithi114 running (10m) 11s ago 10m 6660k - 18.2.1 94eadd990e0a fc8fbfe4fdd5 2024-02-16T22:22:50.491 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi123 smithi123 running (9m) 14s ago 9m 6665k - 18.2.1 94eadd990e0a a6249642b9bb 2024-02-16T22:22:50.491 INFO:teuthology.orchestra.run.smithi114.stdout:grafana.smithi114 smithi114 *:3000 running (8m) 11s ago 9m 90.8M - 9.4.7 954c08fa6188 db1c26c3bc20 2024-02-16T22:22:50.491 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.fdxpey smithi114 running (4m) 11s ago 4m 15.9M - 18.2.1 94eadd990e0a 704b40c144fb 2024-02-16T22:22:50.491 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.zzryke smithi114 running (4m) 11s ago 4m 258M - 18.2.1 94eadd990e0a 1f795d23c9a4 2024-02-16T22:22:50.491 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.gopcfa smithi123 running (4m) 14s ago 4m 15.7M - 18.2.1 94eadd990e0a 166134b7532a 2024-02-16T22:22:50.491 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.vhlqbe smithi123 running (4m) 14s ago 4m 17.3M - 18.2.1 94eadd990e0a 243c8bb96fa8 2024-02-16T22:22:50.491 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi114.vcullz smithi114 *:8443,9283,8765 running (48s) 11s ago 11m 478M - 19.0.0-1450-gf78a58c0 f081f847070a 14ebeb7888c0 2024-02-16T22:22:50.491 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi123.inzgjd smithi123 *:8443,9283,8765 running (118s) 14s ago 9m 82.3M - 19.0.0-1450-gf78a58c0 f081f847070a c9d6ba733f8b 2024-02-16T22:22:50.492 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi114 smithi114 running (11m) 11s ago 11m 56.1M 2048M 18.2.1 94eadd990e0a b4c4433aa3a4 2024-02-16T22:22:50.492 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi123 smithi123 running (9m) 14s ago 9m 42.4M 2048M 18.2.1 94eadd990e0a c217fc30c152 2024-02-16T22:22:50.492 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi114 smithi114 *:9100 running (10m) 11s ago 10m 15.7M - 1.5.0 0da6a335fe13 744a090e06d0 2024-02-16T22:22:50.492 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi123 smithi123 *:9100 running (9m) 14s ago 9m 15.0M - 1.5.0 0da6a335fe13 3994b8c29b2e 2024-02-16T22:22:50.492 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 smithi114 running (8m) 11s ago 8m 181M 1216M 18.2.1 94eadd990e0a 24ca2e9714e0 2024-02-16T22:22:50.492 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 smithi114 running (7m) 11s ago 7m 184M 1216M 18.2.1 94eadd990e0a 32bb191b88e7 2024-02-16T22:22:50.492 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 smithi114 running (7m) 11s ago 7m 156M 1216M 18.2.1 94eadd990e0a 6608718d0818 2024-02-16T22:22:50.492 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 smithi123 running (7m) 14s ago 7m 209M 2240M 18.2.1 94eadd990e0a 575db379de3d 2024-02-16T22:22:50.492 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 smithi123 running (6m) 14s ago 6m 175M 2240M 18.2.1 94eadd990e0a 7e266866cd9c 2024-02-16T22:22:50.492 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 smithi123 running (6m) 14s ago 6m 173M 2240M 18.2.1 94eadd990e0a 24e5bef19fda 2024-02-16T22:22:50.492 INFO:teuthology.orchestra.run.smithi114.stdout:prometheus.smithi114 smithi114 *:9095 running (74s) 11s ago 9m 54.3M - 2.43.0 a07b618ecd1d 318298946ec0 2024-02-16T22:22:50.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.492+0000 7fa6724cb700 1 -- 172.21.15.114:0/2635291037 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7fa65406d660 msgr2=0x7fa65406fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:50.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.492+0000 7fa6724cb700 1 --2- 172.21.15.114:0/2635291037 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7fa65406d660 0x7fa65406fb20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa66c101c10 tx=0x7fa65c00b410 comp rx=0 tx=0).stop 2024-02-16T22:22:50.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.492+0000 7fa6724cb700 1 -- 172.21.15.114:0/2635291037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa66c101db0 msgr2=0x7fa66c1968a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:50.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.492+0000 7fa6724cb700 1 --2- 172.21.15.114:0/2635291037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa66c101db0 0x7fa66c1968a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa660006010 tx=0x7fa66000bd70 comp rx=0 tx=0).stop 2024-02-16T22:22:50.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.492+0000 7fa6724cb700 1 -- 172.21.15.114:0/2635291037 shutdown_connections 2024-02-16T22:22:50.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.492+0000 7fa6724cb700 1 --2- 172.21.15.114:0/2635291037 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7fa65406d660 0x7fa65406fb20 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:50.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.492+0000 7fa6724cb700 1 --2- 172.21.15.114:0/2635291037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa66c100bb0 0x7fa66c196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:50.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.492+0000 7fa6724cb700 1 --2- 172.21.15.114:0/2635291037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa66c101db0 0x7fa66c1968a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:50.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.492+0000 7fa6724cb700 1 -- 172.21.15.114:0/2635291037 >> 172.21.15.114:0/2635291037 conn(0x7fa66c0fc110 msgr2=0x7fa66c104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:22:50.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.492+0000 7fa6724cb700 1 -- 172.21.15.114:0/2635291037 shutdown_connections 2024-02-16T22:22:50.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.492+0000 7fa6724cb700 1 -- 172.21.15.114:0/2635291037 wait complete. 2024-02-16T22:22:50.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.648+0000 7f709b59e700 1 -- 172.21.15.114:0/2153013035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f709c103e20 msgr2=0x7f709c106210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:50.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.648+0000 7f709b59e700 1 --2- 172.21.15.114:0/2153013035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f709c103e20 0x7f709c106210 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f708c009870 tx=0x7f708c009b80 comp rx=0 tx=0).stop 2024-02-16T22:22:50.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.648+0000 7f709b59e700 1 -- 172.21.15.114:0/2153013035 shutdown_connections 2024-02-16T22:22:50.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.648+0000 7f709b59e700 1 --2- 172.21.15.114:0/2153013035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f709c103e20 0x7f709c106210 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:50.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.648+0000 7f709b59e700 1 --2- 172.21.15.114:0/2153013035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f709c1014f0 0x7f709c1038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:50.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.648+0000 7f709b59e700 1 -- 172.21.15.114:0/2153013035 >> 172.21.15.114:0/2153013035 conn(0x7f709c0fae40 msgr2=0x7f709c0fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:22:50.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.648+0000 7f709b59e700 1 -- 172.21.15.114:0/2153013035 shutdown_connections 2024-02-16T22:22:50.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.648+0000 7f709b59e700 1 -- 172.21.15.114:0/2153013035 wait complete. 2024-02-16T22:22:50.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.649+0000 7f709b59e700 1 Processor -- start 2024-02-16T22:22:50.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.649+0000 7f709b59e700 1 -- start start 2024-02-16T22:22:50.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.650+0000 7f709b59e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f709c1014f0 0x7f709c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:50.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.650+0000 7f709b59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f709c103e20 0x7f709c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:50.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.650+0000 7f709b59e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c198ed0 con 0x7f709c1014f0 2024-02-16T22:22:50.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.650+0000 7f709b59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c199010 con 0x7f709c103e20 2024-02-16T22:22:50.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.650+0000 7f709a59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f709c1014f0 0x7f709c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:50.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.650+0000 7f709a59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f709c1014f0 0x7f709c198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57778/0 (socket says 172.21.15.114:57778) 2024-02-16T22:22:50.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.650+0000 7f709a59c700 1 -- 172.21.15.114:0/85833513 learned_addr learned my addr 172.21.15.114:0/85833513 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:22:50.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.650+0000 7f7099d9b700 1 --2- 172.21.15.114:0/85833513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f709c103e20 0x7f709c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:50.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.650+0000 7f709a59c700 1 -- 172.21.15.114:0/85833513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f709c103e20 msgr2=0x7f709c1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:50.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.650+0000 7f709a59c700 1 --2- 172.21.15.114:0/85833513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f709c103e20 0x7f709c1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:50.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.650+0000 7f709a59c700 1 -- 172.21.15.114:0/85833513 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70900095f0 con 0x7f709c1014f0 2024-02-16T22:22:50.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.650+0000 7f7099d9b700 1 --2- 172.21.15.114:0/85833513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f709c103e20 0x7f709c1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:22:50.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.650+0000 7f709a59c700 1 --2- 172.21.15.114:0/85833513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f709c1014f0 0x7f709c198370 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f709000eb90 tx=0x7f709000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:22:50.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.651+0000 7f708b7fe700 1 -- 172.21.15.114:0/85833513 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f709000ccc0 con 0x7f709c1014f0 2024-02-16T22:22:50.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.651+0000 7f709b59e700 1 -- 172.21.15.114:0/85833513 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f708c009520 con 0x7f709c1014f0 2024-02-16T22:22:50.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.651+0000 7f708b7fe700 1 -- 172.21.15.114:0/85833513 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f709000ce20 con 0x7f709c1014f0 2024-02-16T22:22:50.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.651+0000 7f709b59e700 1 -- 172.21.15.114:0/85833513 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f709c19df00 con 0x7f709c1014f0 2024-02-16T22:22:50.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.651+0000 7f708b7fe700 1 -- 172.21.15.114:0/85833513 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7090010430 con 0x7f709c1014f0 2024-02-16T22:22:50.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.652+0000 7f709b59e700 1 -- 172.21.15.114:0/85833513 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f707c005320 con 0x7f709c1014f0 2024-02-16T22:22:50.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.652+0000 7f708b7fe700 1 -- 172.21.15.114:0/85833513 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 30) v1 ==== 91450+0+0 (secure 0 0 0) 0x7f70900106a0 con 0x7f709c1014f0 2024-02-16T22:22:50.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.653+0000 7f708b7fe700 1 --2- 172.21.15.114:0/85833513 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f708406d650 0x7f708406fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:50.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.653+0000 7f708b7fe700 1 -- 172.21.15.114:0/85833513 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f7090014070 con 0x7f709c1014f0 2024-02-16T22:22:50.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.653+0000 7f7099d9b700 1 --2- 172.21.15.114:0/85833513 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f708406d650 0x7f708406fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:50.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.654+0000 7f7099d9b700 1 --2- 172.21.15.114:0/85833513 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f708406d650 0x7f708406fb10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f708c00b6f0 tx=0x7f708c019040 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:22:50.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:50.658+0000 7f708b7fe700 1 -- 172.21.15.114:0/85833513 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182869 (secure 0 0 0) 0x7f709005ae30 con 0x7f709c1014f0 2024-02-16T22:22:50.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:50 smithi114 ceph-mon[29235]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:22:51.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.031+0000 7f709b59e700 1 -- 172.21.15.114:0/85833513 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f707c005cc0 con 0x7f709c1014f0 2024-02-16T22:22:51.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.031+0000 7f708b7fe700 1 -- 172.21.15.114:0/85833513 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f709005a9c0 con 0x7f709c1014f0 2024-02-16T22:22:51.033 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:22:51.033 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": { 2024-02-16T22:22:51.034 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-02-16T22:22:51.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:22:51.034 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": { 2024-02-16T22:22:51.034 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:22:51.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:22:51.034 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": { 2024-02-16T22:22:51.034 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-02-16T22:22:51.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:22:51.034 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-02-16T22:22:51.034 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-02-16T22:22:51.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:22:51.034 INFO:teuthology.orchestra.run.smithi114.stdout: "overall": { 2024-02-16T22:22:51.034 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-02-16T22:22:51.034 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:22:51.034 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-02-16T22:22:51.034 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:22:51.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.035+0000 7f709b59e700 1 -- 172.21.15.114:0/85833513 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f708406d650 msgr2=0x7f708406fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:51.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.035+0000 7f709b59e700 1 --2- 172.21.15.114:0/85833513 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f708406d650 0x7f708406fb10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f708c00b6f0 tx=0x7f708c019040 comp rx=0 tx=0).stop 2024-02-16T22:22:51.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.035+0000 7f709b59e700 1 -- 172.21.15.114:0/85833513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f709c1014f0 msgr2=0x7f709c198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:51.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.035+0000 7f709b59e700 1 --2- 172.21.15.114:0/85833513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f709c1014f0 0x7f709c198370 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f709000eb90 tx=0x7f709000eea0 comp rx=0 tx=0).stop 2024-02-16T22:22:51.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.035+0000 7f709b59e700 1 -- 172.21.15.114:0/85833513 shutdown_connections 2024-02-16T22:22:51.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.035+0000 7f709b59e700 1 --2- 172.21.15.114:0/85833513 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f708406d650 0x7f708406fb10 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:51.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.035+0000 7f709b59e700 1 --2- 172.21.15.114:0/85833513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f709c1014f0 0x7f709c198370 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:51.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.035+0000 7f709b59e700 1 --2- 172.21.15.114:0/85833513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f709c103e20 0x7f709c1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:51.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.035+0000 7f709b59e700 1 -- 172.21.15.114:0/85833513 >> 172.21.15.114:0/85833513 conn(0x7f709c0fae40 msgr2=0x7f709c0fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:22:51.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.035+0000 7f709b59e700 1 -- 172.21.15.114:0/85833513 shutdown_connections 2024-02-16T22:22:51.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.035+0000 7f709b59e700 1 -- 172.21.15.114:0/85833513 wait complete. 2024-02-16T22:22:51.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:50 smithi123 ceph-mon[37161]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:22:51.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.174+0000 7efc19979700 1 -- 172.21.15.114:0/2894830013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc14101e00 msgr2=0x7efc14102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:51.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.174+0000 7efc19979700 1 --2- 172.21.15.114:0/2894830013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc14101e00 0x7efc14102260 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7efc08009910 tx=0x7efc08009c20 comp rx=0 tx=0).stop 2024-02-16T22:22:51.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.174+0000 7efc19979700 1 -- 172.21.15.114:0/2894830013 shutdown_connections 2024-02-16T22:22:51.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.174+0000 7efc19979700 1 --2- 172.21.15.114:0/2894830013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc14101e00 0x7efc14102260 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:51.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.174+0000 7efc19979700 1 --2- 172.21.15.114:0/2894830013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc14100c00 0x7efc14101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:51.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.174+0000 7efc19979700 1 -- 172.21.15.114:0/2894830013 >> 172.21.15.114:0/2894830013 conn(0x7efc140fc180 msgr2=0x7efc140fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:22:51.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.174+0000 7efc19979700 1 -- 172.21.15.114:0/2894830013 shutdown_connections 2024-02-16T22:22:51.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.174+0000 7efc19979700 1 -- 172.21.15.114:0/2894830013 wait complete. 2024-02-16T22:22:51.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.175+0000 7efc19979700 1 Processor -- start 2024-02-16T22:22:51.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.175+0000 7efc19979700 1 -- start start 2024-02-16T22:22:51.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.176+0000 7efc19979700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc14100c00 0x7efc14198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:51.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.176+0000 7efc19979700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc14101e00 0x7efc14198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:51.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.176+0000 7efc19979700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc141990a0 con 0x7efc14101e00 2024-02-16T22:22:51.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.176+0000 7efc127fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc14101e00 0x7efc14198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:51.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.176+0000 7efc127fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc14101e00 0x7efc14198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57794/0 (socket says 172.21.15.114:57794) 2024-02-16T22:22:51.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.176+0000 7efc127fc700 1 -- 172.21.15.114:0/1252051850 learned_addr learned my addr 172.21.15.114:0/1252051850 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:22:51.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.176+0000 7efc19979700 1 -- 172.21.15.114:0/1252051850 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc141991e0 con 0x7efc14100c00 2024-02-16T22:22:51.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.176+0000 7efc12ffd700 1 --2- 172.21.15.114:0/1252051850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc14100c00 0x7efc14198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:51.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.176+0000 7efc12ffd700 1 -- 172.21.15.114:0/1252051850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc14101e00 msgr2=0x7efc14198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:51.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.176+0000 7efc12ffd700 1 --2- 172.21.15.114:0/1252051850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc14101e00 0x7efc14198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:51.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.176+0000 7efc12ffd700 1 -- 172.21.15.114:0/1252051850 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc080095f0 con 0x7efc14100c00 2024-02-16T22:22:51.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.176+0000 7efc12ffd700 1 --2- 172.21.15.114:0/1252051850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc14100c00 0x7efc14198540 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efc0400d6c0 tx=0x7efc0400d9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:22:51.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.177+0000 7efc18977700 1 -- 172.21.15.114:0/1252051850 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc040041f0 con 0x7efc14100c00 2024-02-16T22:22:51.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.177+0000 7efc18977700 1 -- 172.21.15.114:0/1252051850 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efc04004350 con 0x7efc14100c00 2024-02-16T22:22:51.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.177+0000 7efc19979700 1 -- 172.21.15.114:0/1252051850 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc1419dc90 con 0x7efc14100c00 2024-02-16T22:22:51.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.177+0000 7efc18977700 1 -- 172.21.15.114:0/1252051850 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc04003d20 con 0x7efc14100c00 2024-02-16T22:22:51.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.177+0000 7efc19979700 1 -- 172.21.15.114:0/1252051850 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc1419e190 con 0x7efc14100c00 2024-02-16T22:22:51.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.178+0000 7efc19979700 1 -- 172.21.15.114:0/1252051850 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc14073930 con 0x7efc14100c00 2024-02-16T22:22:51.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.179+0000 7efc18977700 1 -- 172.21.15.114:0/1252051850 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 30) v1 ==== 91450+0+0 (secure 0 0 0) 0x7efc04010460 con 0x7efc14100c00 2024-02-16T22:22:51.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.179+0000 7efc18977700 1 --2- 172.21.15.114:0/1252051850 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7efbfc06d650 0x7efbfc06fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:51.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.179+0000 7efc18977700 1 -- 172.21.15.114:0/1252051850 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7efc0408cb70 con 0x7efc14100c00 2024-02-16T22:22:51.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.179+0000 7efc127fc700 1 --2- 172.21.15.114:0/1252051850 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7efbfc06d650 0x7efbfc06fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:51.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.179+0000 7efc127fc700 1 --2- 172.21.15.114:0/1252051850 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7efbfc06d650 0x7efbfc06fb10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efc08006010 tx=0x7efc08005ab0 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:22:51.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.184+0000 7efc18977700 1 -- 172.21.15.114:0/1252051850 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182869 (secure 0 0 0) 0x7efc0405a230 con 0x7efc14100c00 2024-02-16T22:22:51.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.517+0000 7efc19979700 1 -- 172.21.15.114:0/1252051850 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7efc1419e7d0 con 0x7efc14100c00 2024-02-16T22:22:51.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.518+0000 7efc18977700 1 -- 172.21.15.114:0/1252051850 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1825 (secure 0 0 0) 0x7efc0401f740 con 0x7efc14100c00 2024-02-16T22:22:51.520 INFO:teuthology.orchestra.run.smithi114.stdout:e14 2024-02-16T22:22:51.520 INFO:teuthology.orchestra.run.smithi114.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:22:51.521 INFO:teuthology.orchestra.run.smithi114.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:22:51.521 INFO:teuthology.orchestra.run.smithi114.stdout:legacy client fscid: 1 2024-02-16T22:22:51.521 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:22:51.521 INFO:teuthology.orchestra.run.smithi114.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:22:51.521 INFO:teuthology.orchestra.run.smithi114.stdout:fs_name cephfs 2024-02-16T22:22:51.521 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 12 2024-02-16T22:22:51.521 INFO:teuthology.orchestra.run.smithi114.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:22:51.521 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-02-16T22:18:02.108825+0000 2024-02-16T22:22:51.521 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-02-16T22:18:16.413371+0000 2024-02-16T22:22:51.521 INFO:teuthology.orchestra.run.smithi114.stdout:tableserver 0 2024-02-16T22:22:51.521 INFO:teuthology.orchestra.run.smithi114.stdout:root 0 2024-02-16T22:22:51.521 INFO:teuthology.orchestra.run.smithi114.stdout:session_timeout 60 2024-02-16T22:22:51.522 INFO:teuthology.orchestra.run.smithi114.stdout:session_autoclose 300 2024-02-16T22:22:51.522 INFO:teuthology.orchestra.run.smithi114.stdout:max_file_size 1099511627776 2024-02-16T22:22:51.522 INFO:teuthology.orchestra.run.smithi114.stdout:required_client_features {} 2024-02-16T22:22:51.522 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure 0 2024-02-16T22:22:51.522 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure_osd_epoch 0 2024-02-16T22:22:51.522 INFO:teuthology.orchestra.run.smithi114.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:22:51.522 INFO:teuthology.orchestra.run.smithi114.stdout:max_mds 1 2024-02-16T22:22:51.522 INFO:teuthology.orchestra.run.smithi114.stdout:in 0 2024-02-16T22:22:51.522 INFO:teuthology.orchestra.run.smithi114.stdout:up {0=14468} 2024-02-16T22:22:51.522 INFO:teuthology.orchestra.run.smithi114.stdout:failed 2024-02-16T22:22:51.522 INFO:teuthology.orchestra.run.smithi114.stdout:damaged 2024-02-16T22:22:51.522 INFO:teuthology.orchestra.run.smithi114.stdout:stopped 2024-02-16T22:22:51.522 INFO:teuthology.orchestra.run.smithi114.stdout:data_pools [3] 2024-02-16T22:22:51.522 INFO:teuthology.orchestra.run.smithi114.stdout:metadata_pool 2 2024-02-16T22:22:51.523 INFO:teuthology.orchestra.run.smithi114.stdout:inline_data enabled 2024-02-16T22:22:51.523 INFO:teuthology.orchestra.run.smithi114.stdout:balancer 2024-02-16T22:22:51.523 INFO:teuthology.orchestra.run.smithi114.stdout:bal_rank_mask -1 2024-02-16T22:22:51.523 INFO:teuthology.orchestra.run.smithi114.stdout:standby_count_wanted 1 2024-02-16T22:22:51.523 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.zzryke{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.114:6826/277953535,v1:172.21.15.114:6827/277953535] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:22:51.523 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:22:51.523 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:22:51.523 INFO:teuthology.orchestra.run.smithi114.stdout:Standby daemons: 2024-02-16T22:22:51.523 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:22:51.523 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.fdxpey{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:22:51.523 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.vhlqbe{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:22:51.523 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.gopcfa{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:22:51.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.520+0000 7efc19979700 1 -- 172.21.15.114:0/1252051850 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7efbfc06d650 msgr2=0x7efbfc06fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:51.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.520+0000 7efc19979700 1 --2- 172.21.15.114:0/1252051850 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7efbfc06d650 0x7efbfc06fb10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efc08006010 tx=0x7efc08005ab0 comp rx=0 tx=0).stop 2024-02-16T22:22:51.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.520+0000 7efc19979700 1 -- 172.21.15.114:0/1252051850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc14100c00 msgr2=0x7efc14198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:51.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.520+0000 7efc19979700 1 --2- 172.21.15.114:0/1252051850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc14100c00 0x7efc14198540 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efc0400d6c0 tx=0x7efc0400d9d0 comp rx=0 tx=0).stop 2024-02-16T22:22:51.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.520+0000 7efc19979700 1 -- 172.21.15.114:0/1252051850 shutdown_connections 2024-02-16T22:22:51.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.520+0000 7efc19979700 1 --2- 172.21.15.114:0/1252051850 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7efbfc06d650 0x7efbfc06fb10 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:51.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.520+0000 7efc19979700 1 --2- 172.21.15.114:0/1252051850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc14100c00 0x7efc14198540 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:51.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.520+0000 7efc19979700 1 --2- 172.21.15.114:0/1252051850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc14101e00 0x7efc14198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:51.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.520+0000 7efc19979700 1 -- 172.21.15.114:0/1252051850 >> 172.21.15.114:0/1252051850 conn(0x7efc140fc180 msgr2=0x7efc140fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:22:51.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.520+0000 7efc19979700 1 -- 172.21.15.114:0/1252051850 shutdown_connections 2024-02-16T22:22:51.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.520+0000 7efc19979700 1 -- 172.21.15.114:0/1252051850 wait complete. 2024-02-16T22:22:51.525 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 14 2024-02-16T22:22:51.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.660+0000 7f3c75b64700 1 -- 172.21.15.114:0/3595041846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3c70101640 msgr2=0x7f3c70103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:51.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.660+0000 7f3c75b64700 1 --2- 172.21.15.114:0/3595041846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3c70101640 0x7f3c70103a30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3c60009910 tx=0x7f3c60009c20 comp rx=0 tx=0).stop 2024-02-16T22:22:51.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.660+0000 7f3c75b64700 1 -- 172.21.15.114:0/3595041846 shutdown_connections 2024-02-16T22:22:51.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.660+0000 7f3c75b64700 1 --2- 172.21.15.114:0/3595041846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c70103f70 0x7f3c70106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:51.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.660+0000 7f3c75b64700 1 --2- 172.21.15.114:0/3595041846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3c70101640 0x7f3c70103a30 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:51.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.660+0000 7f3c75b64700 1 -- 172.21.15.114:0/3595041846 >> 172.21.15.114:0/3595041846 conn(0x7f3c700fb030 msgr2=0x7f3c700fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:22:51.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.660+0000 7f3c75b64700 1 -- 172.21.15.114:0/3595041846 shutdown_connections 2024-02-16T22:22:51.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.660+0000 7f3c75b64700 1 -- 172.21.15.114:0/3595041846 wait complete. 2024-02-16T22:22:51.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.661+0000 7f3c75b64700 1 Processor -- start 2024-02-16T22:22:51.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.661+0000 7f3c75b64700 1 -- start start 2024-02-16T22:22:51.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.661+0000 7f3c75b64700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c70101640 0x7f3c70196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:51.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.661+0000 7f3c75b64700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3c70103f70 0x7f3c701968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:51.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.661+0000 7f3c75b64700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c70196ee0 con 0x7f3c70103f70 2024-02-16T22:22:51.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.661+0000 7f3c75b64700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c70197020 con 0x7f3c70101640 2024-02-16T22:22:51.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.661+0000 7f3c6effd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3c70103f70 0x7f3c701968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:51.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.661+0000 7f3c6effd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3c70103f70 0x7f3c701968c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40252/0 (socket says 172.21.15.114:40252) 2024-02-16T22:22:51.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.661+0000 7f3c6effd700 1 -- 172.21.15.114:0/2232950236 learned_addr learned my addr 172.21.15.114:0/2232950236 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:22:51.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.661+0000 7f3c6f7fe700 1 --2- 172.21.15.114:0/2232950236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c70101640 0x7f3c70196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:51.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.662+0000 7f3c6effd700 1 -- 172.21.15.114:0/2232950236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c70101640 msgr2=0x7f3c70196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:51.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.662+0000 7f3c6effd700 1 --2- 172.21.15.114:0/2232950236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c70101640 0x7f3c70196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:51.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.662+0000 7f3c6effd700 1 -- 172.21.15.114:0/2232950236 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c600095f0 con 0x7f3c70103f70 2024-02-16T22:22:51.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.662+0000 7f3c6effd700 1 --2- 172.21.15.114:0/2232950236 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3c70103f70 0x7f3c701968c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3c6400b4b0 tx=0x7f3c6400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:22:51.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.662+0000 7f3c6cff9700 1 -- 172.21.15.114:0/2232950236 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c64011800 con 0x7f3c70103f70 2024-02-16T22:22:51.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.662+0000 7f3c6cff9700 1 -- 172.21.15.114:0/2232950236 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3c64011e10 con 0x7f3c70103f70 2024-02-16T22:22:51.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.662+0000 7f3c75b64700 1 -- 172.21.15.114:0/2232950236 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c700ff550 con 0x7f3c70103f70 2024-02-16T22:22:51.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.662+0000 7f3c6cff9700 1 -- 172.21.15.114:0/2232950236 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c6400f2e0 con 0x7f3c70103f70 2024-02-16T22:22:51.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.662+0000 7f3c75b64700 1 -- 172.21.15.114:0/2232950236 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c700ffb20 con 0x7f3c70103f70 2024-02-16T22:22:51.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.663+0000 7f3c75b64700 1 -- 172.21.15.114:0/2232950236 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c70109790 con 0x7f3c70103f70 2024-02-16T22:22:51.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.664+0000 7f3c6cff9700 1 -- 172.21.15.114:0/2232950236 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 30) v1 ==== 91450+0+0 (secure 0 0 0) 0x7f3c64010380 con 0x7f3c70103f70 2024-02-16T22:22:51.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.665+0000 7f3c6cff9700 1 --2- 172.21.15.114:0/2232950236 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f3c5806d6a0 0x7f3c5806fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:51.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.665+0000 7f3c6cff9700 1 -- 172.21.15.114:0/2232950236 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f3c6408d380 con 0x7f3c70103f70 2024-02-16T22:22:51.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.665+0000 7f3c6f7fe700 1 --2- 172.21.15.114:0/2232950236 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f3c5806d6a0 0x7f3c5806fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:51.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.666+0000 7f3c6f7fe700 1 --2- 172.21.15.114:0/2232950236 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f3c5806d6a0 0x7f3c5806fb60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3c60006010 tx=0x7f3c6000b540 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:22:51.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:51.669+0000 7f3c6cff9700 1 -- 172.21.15.114:0/2232950236 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182869 (secure 0 0 0) 0x7f3c6405aab0 con 0x7f3c70103f70 2024-02-16T22:22:51.928 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:51 smithi123 ceph-mon[37161]: pgmap v12: 65 pgs: 65 active+clean; 291 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 56 KiB/s rd, 2.0 MiB/s wr, 196 op/s 2024-02-16T22:22:51.928 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:51 smithi123 ceph-mon[37161]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:22:51.928 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:51 smithi123 ceph-mon[37161]: from='client.24583 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:22:51.929 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:51 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/85833513' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:22:51.929 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:51 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/1252051850' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:22:51.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:51 smithi114 ceph-mon[29235]: pgmap v12: 65 pgs: 65 active+clean; 291 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 56 KiB/s rd, 2.0 MiB/s wr, 196 op/s 2024-02-16T22:22:51.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:51 smithi114 ceph-mon[29235]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:22:51.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:51 smithi114 ceph-mon[29235]: from='client.24583 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:22:51.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:51 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/85833513' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:22:51.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:51 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/1252051850' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:22:52.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.008+0000 7f3c75b64700 1 -- 172.21.15.114:0/2232950236 --> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c70190140 con 0x7f3c5806d6a0 2024-02-16T22:22:52.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.009+0000 7f3c6cff9700 1 -- 172.21.15.114:0/2232950236 <== mgr.14764 v2:172.21.15.114:6800/3051780151 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f3c70190140 con 0x7f3c5806d6a0 2024-02-16T22:22:52.012 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:22:52.012 INFO:teuthology.orchestra.run.smithi114.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:22:52.012 INFO:teuthology.orchestra.run.smithi114.stdout: "in_progress": true, 2024-02-16T22:22:52.012 INFO:teuthology.orchestra.run.smithi114.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:22:52.012 INFO:teuthology.orchestra.run.smithi114.stdout: "services_complete": [ 2024-02-16T22:22:52.012 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr" 2024-02-16T22:22:52.012 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-02-16T22:22:52.012 INFO:teuthology.orchestra.run.smithi114.stdout: "progress": "2/23 daemons upgraded", 2024-02-16T22:22:52.012 INFO:teuthology.orchestra.run.smithi114.stdout: "message": "", 2024-02-16T22:22:52.012 INFO:teuthology.orchestra.run.smithi114.stdout: "is_paused": false 2024-02-16T22:22:52.012 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:22:52.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.012+0000 7f3c75b64700 1 -- 172.21.15.114:0/2232950236 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f3c5806d6a0 msgr2=0x7f3c5806fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:52.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.012+0000 7f3c75b64700 1 --2- 172.21.15.114:0/2232950236 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f3c5806d6a0 0x7f3c5806fb60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3c60006010 tx=0x7f3c6000b540 comp rx=0 tx=0).stop 2024-02-16T22:22:52.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.012+0000 7f3c75b64700 1 -- 172.21.15.114:0/2232950236 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3c70103f70 msgr2=0x7f3c701968c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:52.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.012+0000 7f3c75b64700 1 --2- 172.21.15.114:0/2232950236 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3c70103f70 0x7f3c701968c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3c6400b4b0 tx=0x7f3c6400b870 comp rx=0 tx=0).stop 2024-02-16T22:22:52.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.012+0000 7f3c75b64700 1 -- 172.21.15.114:0/2232950236 shutdown_connections 2024-02-16T22:22:52.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.012+0000 7f3c75b64700 1 --2- 172.21.15.114:0/2232950236 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f3c5806d6a0 0x7f3c5806fb60 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:52.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.012+0000 7f3c75b64700 1 --2- 172.21.15.114:0/2232950236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c70101640 0x7f3c70196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:52.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.012+0000 7f3c75b64700 1 --2- 172.21.15.114:0/2232950236 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3c70103f70 0x7f3c701968c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:52.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.012+0000 7f3c75b64700 1 -- 172.21.15.114:0/2232950236 >> 172.21.15.114:0/2232950236 conn(0x7f3c700fb030 msgr2=0x7f3c700fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:22:52.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.012+0000 7f3c75b64700 1 -- 172.21.15.114:0/2232950236 shutdown_connections 2024-02-16T22:22:52.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.012+0000 7f3c75b64700 1 -- 172.21.15.114:0/2232950236 wait complete. 2024-02-16T22:22:52.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.141+0000 7f45b37cc700 1 -- 172.21.15.114:0/1005904252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45ac103f70 msgr2=0x7f45ac106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:52.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.141+0000 7f45b37cc700 1 --2- 172.21.15.114:0/1005904252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45ac103f70 0x7f45ac106360 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f459c009910 tx=0x7f459c009c20 comp rx=0 tx=0).stop 2024-02-16T22:22:52.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.141+0000 7f45b37cc700 1 -- 172.21.15.114:0/1005904252 shutdown_connections 2024-02-16T22:22:52.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.141+0000 7f45b37cc700 1 --2- 172.21.15.114:0/1005904252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45ac103f70 0x7f45ac106360 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:52.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.141+0000 7f45b37cc700 1 --2- 172.21.15.114:0/1005904252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45ac101640 0x7f45ac103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:52.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.141+0000 7f45b37cc700 1 -- 172.21.15.114:0/1005904252 >> 172.21.15.114:0/1005904252 conn(0x7f45ac0fb030 msgr2=0x7f45ac0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:22:52.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.142+0000 7f45b37cc700 1 -- 172.21.15.114:0/1005904252 shutdown_connections 2024-02-16T22:22:52.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.142+0000 7f45b37cc700 1 -- 172.21.15.114:0/1005904252 wait complete. 2024-02-16T22:22:52.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.142+0000 7f45b37cc700 1 Processor -- start 2024-02-16T22:22:52.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.142+0000 7f45b37cc700 1 -- start start 2024-02-16T22:22:52.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.143+0000 7f45b37cc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45ac101640 0x7f45ac196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:52.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.143+0000 7f45b37cc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45ac103f70 0x7f45ac196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:52.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.143+0000 7f45b37cc700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac196e90 con 0x7f45ac103f70 2024-02-16T22:22:52.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.143+0000 7f45b37cc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac196fd0 con 0x7f45ac101640 2024-02-16T22:22:52.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.143+0000 7f45b0d67700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45ac103f70 0x7f45ac196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:52.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.143+0000 7f45b0d67700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45ac103f70 0x7f45ac196870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40284/0 (socket says 172.21.15.114:40284) 2024-02-16T22:22:52.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.143+0000 7f45b0d67700 1 -- 172.21.15.114:0/4120169120 learned_addr learned my addr 172.21.15.114:0/4120169120 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:22:52.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.143+0000 7f45b1568700 1 --2- 172.21.15.114:0/4120169120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45ac101640 0x7f45ac196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:52.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.143+0000 7f45b0d67700 1 -- 172.21.15.114:0/4120169120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45ac101640 msgr2=0x7f45ac196330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:52.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.143+0000 7f45b0d67700 1 --2- 172.21.15.114:0/4120169120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45ac101640 0x7f45ac196330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:52.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.143+0000 7f45b0d67700 1 -- 172.21.15.114:0/4120169120 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f459c0095f0 con 0x7f45ac103f70 2024-02-16T22:22:52.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.143+0000 7f45b0d67700 1 --2- 172.21.15.114:0/4120169120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45ac103f70 0x7f45ac196870 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f459c000c00 tx=0x7f459c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:22:52.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.143+0000 7f45a27fc700 1 -- 172.21.15.114:0/4120169120 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f459c01d070 con 0x7f45ac103f70 2024-02-16T22:22:52.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.144+0000 7f45a27fc700 1 -- 172.21.15.114:0/4120169120 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f459c005dc0 con 0x7f45ac103f70 2024-02-16T22:22:52.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.144+0000 7f45a27fc700 1 -- 172.21.15.114:0/4120169120 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f459c0226a0 con 0x7f45ac103f70 2024-02-16T22:22:52.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.144+0000 7f45b37cc700 1 -- 172.21.15.114:0/4120169120 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45ac19ba20 con 0x7f45ac103f70 2024-02-16T22:22:52.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.144+0000 7f45b37cc700 1 -- 172.21.15.114:0/4120169120 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45ac19be50 con 0x7f45ac103f70 2024-02-16T22:22:52.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.145+0000 7f45b37cc700 1 -- 172.21.15.114:0/4120169120 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45ac190520 con 0x7f45ac103f70 2024-02-16T22:22:52.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.146+0000 7f45a27fc700 1 -- 172.21.15.114:0/4120169120 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 30) v1 ==== 91450+0+0 (secure 0 0 0) 0x7f459c0038f0 con 0x7f45ac103f70 2024-02-16T22:22:52.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.146+0000 7f45a27fc700 1 --2- 172.21.15.114:0/4120169120 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f459806d650 0x7f459806fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:22:52.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.146+0000 7f45a27fc700 1 -- 172.21.15.114:0/4120169120 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f459c08e7e0 con 0x7f45ac103f70 2024-02-16T22:22:52.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.147+0000 7f45b1568700 1 --2- 172.21.15.114:0/4120169120 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f459806d650 0x7f459806fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:22:52.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.147+0000 7f45b1568700 1 --2- 172.21.15.114:0/4120169120 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f459806d650 0x7f459806fb10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f45a8009c60 tx=0x7f45a8009260 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:22:52.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.151+0000 7f45a27fc700 1 -- 172.21.15.114:0/4120169120 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182869 (secure 0 0 0) 0x7f459c05bf40 con 0x7f45ac103f70 2024-02-16T22:22:52.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.522+0000 7f45b37cc700 1 -- 172.21.15.114:0/4120169120 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f45ac02ced0 con 0x7f45ac103f70 2024-02-16T22:22:52.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.523+0000 7f45a27fc700 1 -- 172.21.15.114:0/4120169120 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f459c05bad0 con 0x7f45ac103f70 2024-02-16T22:22:52.525 INFO:teuthology.orchestra.run.smithi114.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:22:52.525 INFO:teuthology.orchestra.run.smithi114.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:22:52.525 INFO:teuthology.orchestra.run.smithi114.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:22:52.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.527+0000 7f45b37cc700 1 -- 172.21.15.114:0/4120169120 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f459806d650 msgr2=0x7f459806fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:52.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.527+0000 7f45b37cc700 1 --2- 172.21.15.114:0/4120169120 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f459806d650 0x7f459806fb10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f45a8009c60 tx=0x7f45a8009260 comp rx=0 tx=0).stop 2024-02-16T22:22:52.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.527+0000 7f45b37cc700 1 -- 172.21.15.114:0/4120169120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45ac103f70 msgr2=0x7f45ac196870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:22:52.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.527+0000 7f45b37cc700 1 --2- 172.21.15.114:0/4120169120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45ac103f70 0x7f45ac196870 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f459c000c00 tx=0x7f459c00f690 comp rx=0 tx=0).stop 2024-02-16T22:22:52.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.527+0000 7f45b37cc700 1 -- 172.21.15.114:0/4120169120 shutdown_connections 2024-02-16T22:22:52.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.527+0000 7f45b37cc700 1 --2- 172.21.15.114:0/4120169120 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f459806d650 0x7f459806fb10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:52.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.527+0000 7f45b37cc700 1 --2- 172.21.15.114:0/4120169120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45ac101640 0x7f45ac196330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:52.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.527+0000 7f45b37cc700 1 --2- 172.21.15.114:0/4120169120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45ac103f70 0x7f45ac196870 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:22:52.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.527+0000 7f45b37cc700 1 -- 172.21.15.114:0/4120169120 >> 172.21.15.114:0/4120169120 conn(0x7f45ac0fb030 msgr2=0x7f45ac0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:22:52.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.528+0000 7f45b37cc700 1 -- 172.21.15.114:0/4120169120 shutdown_connections 2024-02-16T22:22:52.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:22:52.529+0000 7f45b37cc700 1 -- 172.21.15.114:0/4120169120 wait complete. 2024-02-16T22:22:53.199 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:52 smithi114 ceph-mon[29235]: pgmap v13: 65 pgs: 65 active+clean; 291 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-02-16T22:22:53.199 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:52 smithi114 ceph-mon[29235]: from='client.14842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:22:53.199 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:52 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/4120169120' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:22:53.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:52 smithi123 ceph-mon[37161]: pgmap v13: 65 pgs: 65 active+clean; 291 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-02-16T22:22:53.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:52 smithi123 ceph-mon[37161]: from='client.14842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:22:53.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:52 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/4120169120' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:22:55.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:54 smithi114 ceph-mon[29235]: pgmap v14: 65 pgs: 65 active+clean; 298 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 41 KiB/s rd, 2.1 MiB/s wr, 152 op/s 2024-02-16T22:22:55.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:54 smithi123 ceph-mon[37161]: pgmap v14: 65 pgs: 65 active+clean; 298 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 41 KiB/s rd, 2.1 MiB/s wr, 152 op/s 2024-02-16T22:22:57.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:56 smithi114 ceph-mon[29235]: pgmap v15: 65 pgs: 65 active+clean; 298 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.4 MiB/s wr, 88 op/s 2024-02-16T22:22:57.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:56 smithi123 ceph-mon[37161]: pgmap v15: 65 pgs: 65 active+clean; 298 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.4 MiB/s wr, 88 op/s 2024-02-16T22:22:59.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:22:58 smithi123 ceph-mon[37161]: pgmap v16: 65 pgs: 65 active+clean; 298 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.4 MiB/s wr, 88 op/s 2024-02-16T22:22:59.381 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:22:58 smithi114 ceph-mon[29235]: pgmap v16: 65 pgs: 65 active+clean; 298 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.4 MiB/s wr, 88 op/s 2024-02-16T22:23:01.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:00 smithi123 ceph-mon[37161]: pgmap v17: 65 pgs: 65 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 43 KiB/s rd, 2.1 MiB/s wr, 127 op/s 2024-02-16T22:23:01.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:00 smithi114 ceph-mon[29235]: pgmap v17: 65 pgs: 65 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 43 KiB/s rd, 2.1 MiB/s wr, 127 op/s 2024-02-16T22:23:03.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:02 smithi114 ceph-mon[29235]: pgmap v18: 65 pgs: 65 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 1.4 MiB/s wr, 78 op/s 2024-02-16T22:23:03.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:02 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:23:03.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:02 smithi123 ceph-mon[37161]: pgmap v18: 65 pgs: 65 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 1.4 MiB/s wr, 78 op/s 2024-02-16T22:23:03.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:02 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:23:05.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:04 smithi123 ceph-mon[37161]: pgmap v19: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 45 KiB/s rd, 2.2 MiB/s wr, 124 op/s 2024-02-16T22:23:05.378 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:04 smithi114 ceph-mon[29235]: pgmap v19: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 45 KiB/s rd, 2.2 MiB/s wr, 124 op/s 2024-02-16T22:23:07.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:06 smithi114 ceph-mon[29235]: pgmap v20: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.5 MiB/s wr, 84 op/s 2024-02-16T22:23:07.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:06 smithi123 ceph-mon[37161]: pgmap v20: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.5 MiB/s wr, 84 op/s 2024-02-16T22:23:09.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:09 smithi123 ceph-mon[37161]: pgmap v21: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.5 MiB/s wr, 84 op/s 2024-02-16T22:23:09.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:09 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:09.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:09 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:09.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:09 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T22:23:09.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:09 smithi123 ceph-mon[37161]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T22:23:09.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:09 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:23:09.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:09 smithi114 ceph-mon[29235]: pgmap v21: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.5 MiB/s wr, 84 op/s 2024-02-16T22:23:09.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:09 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:09.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:09 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:09.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:09 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T22:23:09.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:09 smithi114 ceph-mon[29235]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T22:23:09.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:09 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:23:11.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:11 smithi123 ceph-mon[37161]: pgmap v22: 65 pgs: 65 active+clean; 307 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 48 KiB/s rd, 2.4 MiB/s wr, 131 op/s 2024-02-16T22:23:11.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:11 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi123.inzgjd", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:23:11.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:11 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T22:23:11.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:11 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:23:11.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:11 smithi114 ceph-mon[29235]: pgmap v22: 65 pgs: 65 active+clean; 307 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 48 KiB/s rd, 2.4 MiB/s wr, 131 op/s 2024-02-16T22:23:11.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:11 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi123.inzgjd", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:23:11.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:11 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T22:23:11.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:11 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:23:12.692 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:12 smithi123 ceph-mon[37161]: Upgrade: Updating mgr.smithi123.inzgjd 2024-02-16T22:23:12.692 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:12 smithi123 ceph-mon[37161]: Deploying daemon mgr.smithi123.inzgjd on smithi123 2024-02-16T22:23:12.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:12 smithi114 ceph-mon[29235]: Upgrade: Updating mgr.smithi123.inzgjd 2024-02-16T22:23:12.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:12 smithi114 ceph-mon[29235]: Deploying daemon mgr.smithi123.inzgjd on smithi123 2024-02-16T22:23:13.625 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:13 smithi123 ceph-mon[37161]: pgmap v23: 65 pgs: 65 active+clean; 307 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.8 MiB/s wr, 93 op/s 2024-02-16T22:23:13.852 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:13 smithi114 ceph-mon[29235]: pgmap v23: 65 pgs: 65 active+clean; 307 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.8 MiB/s wr, 93 op/s 2024-02-16T22:23:15.671 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:15 smithi123 ceph-mon[37161]: pgmap v24: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 52 KiB/s rd, 2.7 MiB/s wr, 150 op/s 2024-02-16T22:23:15.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:15 smithi114 ceph-mon[29235]: pgmap v24: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 52 KiB/s rd, 2.7 MiB/s wr, 150 op/s 2024-02-16T22:23:17.919 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:17 smithi114 ceph-mon[29235]: pgmap v25: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 1.8 MiB/s wr, 104 op/s 2024-02-16T22:23:17.920 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:17 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:17.920 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:17 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:17.920 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:17 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:23:17.920 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:17 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:23:17.920 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:17 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:17.920 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:17 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:18.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:17 smithi123 ceph-mon[37161]: pgmap v25: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 1.8 MiB/s wr, 104 op/s 2024-02-16T22:23:18.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:17 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:18.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:17 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:18.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:17 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:23:18.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:17 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:23:18.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:17 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:18.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:17 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:19.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:19 smithi114 ceph-mon[29235]: pgmap v26: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 1.8 MiB/s wr, 104 op/s 2024-02-16T22:23:20.022 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:19 smithi123 ceph-mon[37161]: pgmap v26: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 1.8 MiB/s wr, 104 op/s 2024-02-16T22:23:21.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:21 smithi114 ceph-mon[29235]: pgmap v27: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53 KiB/s rd, 2.9 MiB/s wr, 445 op/s 2024-02-16T22:23:21.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:21 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:21.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:21 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:22.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:21 smithi123 ceph-mon[37161]: pgmap v27: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53 KiB/s rd, 2.9 MiB/s wr, 445 op/s 2024-02-16T22:23:22.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:21 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:22.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:21 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:22.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.653+0000 7f684072a700 1 -- 172.21.15.114:0/3145618834 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6838101640 msgr2=0x7f6838103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:22.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.653+0000 7f684072a700 1 --2- 172.21.15.114:0/3145618834 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6838101640 0x7f6838103a30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6834009910 tx=0x7f6834009c20 comp rx=0 tx=0).stop 2024-02-16T22:23:22.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.653+0000 7f684072a700 1 -- 172.21.15.114:0/3145618834 shutdown_connections 2024-02-16T22:23:22.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.653+0000 7f684072a700 1 --2- 172.21.15.114:0/3145618834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6838103f70 0x7f6838106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:22.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.653+0000 7f684072a700 1 --2- 172.21.15.114:0/3145618834 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6838101640 0x7f6838103a30 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:22.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.653+0000 7f684072a700 1 -- 172.21.15.114:0/3145618834 >> 172.21.15.114:0/3145618834 conn(0x7f68380fb030 msgr2=0x7f68380fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:23:22.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.653+0000 7f684072a700 1 -- 172.21.15.114:0/3145618834 shutdown_connections 2024-02-16T22:23:22.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.653+0000 7f684072a700 1 -- 172.21.15.114:0/3145618834 wait complete. 2024-02-16T22:23:22.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.654+0000 7f684072a700 1 Processor -- start 2024-02-16T22:23:22.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.654+0000 7f684072a700 1 -- start start 2024-02-16T22:23:22.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.654+0000 7f684072a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6838101640 0x7f6838196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:22.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.654+0000 7f684072a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6838103f70 0x7f6838196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:22.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.654+0000 7f684072a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6838196e60 con 0x7f6838101640 2024-02-16T22:23:22.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.654+0000 7f684072a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6838196fa0 con 0x7f6838103f70 2024-02-16T22:23:22.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.654+0000 7f683e4c6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6838101640 0x7f6838196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:22.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.654+0000 7f683e4c6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6838101640 0x7f6838196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:45052/0 (socket says 172.21.15.114:45052) 2024-02-16T22:23:22.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.654+0000 7f683e4c6700 1 -- 172.21.15.114:0/3173228327 learned_addr learned my addr 172.21.15.114:0/3173228327 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:23:22.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.654+0000 7f683dcc5700 1 --2- 172.21.15.114:0/3173228327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6838103f70 0x7f6838196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:22.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.654+0000 7f683e4c6700 1 -- 172.21.15.114:0/3173228327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6838103f70 msgr2=0x7f6838196840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:22.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.654+0000 7f683e4c6700 1 --2- 172.21.15.114:0/3173228327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6838103f70 0x7f6838196840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:22.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.654+0000 7f683e4c6700 1 -- 172.21.15.114:0/3173228327 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68340095f0 con 0x7f6838101640 2024-02-16T22:23:22.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.654+0000 7f683e4c6700 1 --2- 172.21.15.114:0/3173228327 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6838101640 0x7f6838196300 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6834006010 tx=0x7f683400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:23:22.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.655+0000 7f682f7fe700 1 -- 172.21.15.114:0/3173228327 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f683401d070 con 0x7f6838101640 2024-02-16T22:23:22.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.655+0000 7f682f7fe700 1 -- 172.21.15.114:0/3173228327 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f683400f460 con 0x7f6838101640 2024-02-16T22:23:22.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.655+0000 7f682f7fe700 1 -- 172.21.15.114:0/3173228327 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6834021640 con 0x7f6838101640 2024-02-16T22:23:22.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.655+0000 7f684072a700 1 -- 172.21.15.114:0/3173228327 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f683819b9f0 con 0x7f6838101640 2024-02-16T22:23:22.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.655+0000 7f684072a700 1 -- 172.21.15.114:0/3173228327 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f683819bee0 con 0x7f6838101640 2024-02-16T22:23:22.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.656+0000 7f684072a700 1 -- 172.21.15.114:0/3173228327 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6838190520 con 0x7f6838101640 2024-02-16T22:23:22.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.656+0000 7f682f7fe700 1 -- 172.21.15.114:0/3173228327 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 30) v1 ==== 91450+0+0 (secure 0 0 0) 0x7f683400fab0 con 0x7f6838101640 2024-02-16T22:23:22.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.657+0000 7f682f7fe700 1 --2- 172.21.15.114:0/3173228327 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f682406d660 0x7f682406fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:22.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.657+0000 7f682f7fe700 1 -- 172.21.15.114:0/3173228327 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f683408e3b0 con 0x7f6838101640 2024-02-16T22:23:22.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.657+0000 7f683dcc5700 1 --2- 172.21.15.114:0/3173228327 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f682406d660 0x7f682406fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:22.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.657+0000 7f683dcc5700 1 --2- 172.21.15.114:0/3173228327 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f682406d660 0x7f682406fb20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6828005950 tx=0x7f682800b410 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:23:22.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.661+0000 7f682f7fe700 1 -- 172.21.15.114:0/3173228327 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182869 (secure 0 0 0) 0x7f683405ba70 con 0x7f6838101640 2024-02-16T22:23:22.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.985+0000 7f684072a700 1 -- 172.21.15.114:0/3173228327 --> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f68380611d0 con 0x7f682406d660 2024-02-16T22:23:22.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.986+0000 7f682f7fe700 1 -- 172.21.15.114:0/3173228327 <== mgr.14764 v2:172.21.15.114:6800/3051780151 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f68380611d0 con 0x7f682406d660 2024-02-16T22:23:22.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.989+0000 7f684072a700 1 -- 172.21.15.114:0/3173228327 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f682406d660 msgr2=0x7f682406fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:22.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.989+0000 7f684072a700 1 --2- 172.21.15.114:0/3173228327 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f682406d660 0x7f682406fb20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6828005950 tx=0x7f682800b410 comp rx=0 tx=0).stop 2024-02-16T22:23:22.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.990+0000 7f684072a700 1 -- 172.21.15.114:0/3173228327 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6838101640 msgr2=0x7f6838196300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:22.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.990+0000 7f684072a700 1 --2- 172.21.15.114:0/3173228327 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6838101640 0x7f6838196300 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6834006010 tx=0x7f683400bd70 comp rx=0 tx=0).stop 2024-02-16T22:23:22.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.990+0000 7f684072a700 1 -- 172.21.15.114:0/3173228327 shutdown_connections 2024-02-16T22:23:22.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.990+0000 7f684072a700 1 --2- 172.21.15.114:0/3173228327 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f682406d660 0x7f682406fb20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:22.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.990+0000 7f684072a700 1 --2- 172.21.15.114:0/3173228327 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6838101640 0x7f6838196300 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:22.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.990+0000 7f684072a700 1 --2- 172.21.15.114:0/3173228327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6838103f70 0x7f6838196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:22.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.990+0000 7f684072a700 1 -- 172.21.15.114:0/3173228327 >> 172.21.15.114:0/3173228327 conn(0x7f68380fb030 msgr2=0x7f68380fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:23:22.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.990+0000 7f684072a700 1 -- 172.21.15.114:0/3173228327 shutdown_connections 2024-02-16T22:23:22.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:22.990+0000 7f684072a700 1 -- 172.21.15.114:0/3173228327 wait complete. 2024-02-16T22:23:23.003 INFO:teuthology.orchestra.run.smithi114.stdout:true 2024-02-16T22:23:23.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.109+0000 7f383740b700 1 -- 172.21.15.114:0/2533348180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3830100bd0 msgr2=0x7f3830100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:23.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.109+0000 7f383740b700 1 --2- 172.21.15.114:0/2533348180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3830100bd0 0x7f3830100ff0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f382c009910 tx=0x7f382c009c20 comp rx=0 tx=0).stop 2024-02-16T22:23:23.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.109+0000 7f383740b700 1 -- 172.21.15.114:0/2533348180 shutdown_connections 2024-02-16T22:23:23.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.109+0000 7f383740b700 1 --2- 172.21.15.114:0/2533348180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3830101dd0 0x7f3830102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:23.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.109+0000 7f383740b700 1 --2- 172.21.15.114:0/2533348180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3830100bd0 0x7f3830100ff0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:23.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.109+0000 7f383740b700 1 -- 172.21.15.114:0/2533348180 >> 172.21.15.114:0/2533348180 conn(0x7f38300fc190 msgr2=0x7f38300fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:23:23.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.109+0000 7f383740b700 1 -- 172.21.15.114:0/2533348180 shutdown_connections 2024-02-16T22:23:23.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.109+0000 7f383740b700 1 -- 172.21.15.114:0/2533348180 wait complete. 2024-02-16T22:23:23.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.109+0000 7f383740b700 1 Processor -- start 2024-02-16T22:23:23.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.110+0000 7f383740b700 1 -- start start 2024-02-16T22:23:23.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.110+0000 7f383740b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3830100bd0 0x7f3830074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:23.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.110+0000 7f383740b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3830101dd0 0x7f3830073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:23.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.110+0000 7f383740b700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38300736c0 con 0x7f3830101dd0 2024-02-16T22:23:23.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.110+0000 7f383740b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3830073800 con 0x7f3830100bd0 2024-02-16T22:23:23.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.110+0000 7f38351a7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3830100bd0 0x7f3830074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:23.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.110+0000 7f38349a6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3830101dd0 0x7f3830073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:23.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.110+0000 7f38351a7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3830100bd0 0x7f3830074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.114:38644/0 (socket says 172.21.15.114:38644) 2024-02-16T22:23:23.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.110+0000 7f38351a7700 1 -- 172.21.15.114:0/3728586639 learned_addr learned my addr 172.21.15.114:0/3728586639 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:23:23.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.110+0000 7f38349a6700 1 -- 172.21.15.114:0/3728586639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3830100bd0 msgr2=0x7f3830074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:23.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.110+0000 7f38349a6700 1 --2- 172.21.15.114:0/3728586639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3830100bd0 0x7f3830074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:23.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.110+0000 7f38349a6700 1 -- 172.21.15.114:0/3728586639 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f382c0095f0 con 0x7f3830101dd0 2024-02-16T22:23:23.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.110+0000 7f38351a7700 1 --2- 172.21.15.114:0/3728586639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3830100bd0 0x7f3830074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:23:23.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.111+0000 7f38349a6700 1 --2- 172.21.15.114:0/3728586639 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3830101dd0 0x7f3830073180 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f382000e970 tx=0x7f382000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:23:23.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.111+0000 7f38267fc700 1 -- 172.21.15.114:0/3728586639 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f382000c9d0 con 0x7f3830101dd0 2024-02-16T22:23:23.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.111+0000 7f38267fc700 1 -- 172.21.15.114:0/3728586639 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3820010430 con 0x7f3830101dd0 2024-02-16T22:23:23.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.111+0000 7f383740b700 1 -- 172.21.15.114:0/3728586639 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3830073ae0 con 0x7f3830101dd0 2024-02-16T22:23:23.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.111+0000 7f38267fc700 1 -- 172.21.15.114:0/3728586639 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3820018650 con 0x7f3830101dd0 2024-02-16T22:23:23.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.111+0000 7f383740b700 1 -- 172.21.15.114:0/3728586639 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f383019e3c0 con 0x7f3830101dd0 2024-02-16T22:23:23.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.112+0000 7f383740b700 1 -- 172.21.15.114:0/3728586639 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3830109730 con 0x7f3830101dd0 2024-02-16T22:23:23.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.112+0000 7f38267fc700 1 -- 172.21.15.114:0/3728586639 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 30) v1 ==== 91450+0+0 (secure 0 0 0) 0x7f38200105a0 con 0x7f3830101dd0 2024-02-16T22:23:23.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.113+0000 7f38267fc700 1 --2- 172.21.15.114:0/3728586639 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f381c06d660 0x7f381c06fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:23.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.113+0000 7f38267fc700 1 -- 172.21.15.114:0/3728586639 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f3820014070 con 0x7f3830101dd0 2024-02-16T22:23:23.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.113+0000 7f38351a7700 1 --2- 172.21.15.114:0/3728586639 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f381c06d660 0x7f381c06fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:23.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.114+0000 7f38351a7700 1 --2- 172.21.15.114:0/3728586639 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f381c06d660 0x7f381c06fb20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f382c009fd0 tx=0x7f382c005940 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:23:23.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.118+0000 7f38267fc700 1 -- 172.21.15.114:0/3728586639 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182869 (secure 0 0 0) 0x7f3820052a70 con 0x7f3830101dd0 2024-02-16T22:23:23.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:23 smithi123 ceph-mon[37161]: pgmap v28: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 36 KiB/s rd, 2.0 MiB/s wr, 398 op/s 2024-02-16T22:23:23.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:23 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:23.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:23 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:23.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.396+0000 7f383740b700 1 -- 172.21.15.114:0/3728586639 --> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3830106720 con 0x7f381c06d660 2024-02-16T22:23:23.399 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:23 smithi114 ceph-mon[29235]: pgmap v28: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 36 KiB/s rd, 2.0 MiB/s wr, 398 op/s 2024-02-16T22:23:23.399 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:23 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:23.399 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:23 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:23.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.398+0000 7f38267fc700 1 -- 172.21.15.114:0/3728586639 <== mgr.14764 v2:172.21.15.114:6800/3051780151 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f3830106720 con 0x7f381c06d660 2024-02-16T22:23:23.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.401+0000 7f383740b700 1 -- 172.21.15.114:0/3728586639 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f381c06d660 msgr2=0x7f381c06fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:23.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.401+0000 7f383740b700 1 --2- 172.21.15.114:0/3728586639 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f381c06d660 0x7f381c06fb20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f382c009fd0 tx=0x7f382c005940 comp rx=0 tx=0).stop 2024-02-16T22:23:23.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.401+0000 7f383740b700 1 -- 172.21.15.114:0/3728586639 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3830101dd0 msgr2=0x7f3830073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:23.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.401+0000 7f383740b700 1 --2- 172.21.15.114:0/3728586639 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3830101dd0 0x7f3830073180 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f382000e970 tx=0x7f382000ec80 comp rx=0 tx=0).stop 2024-02-16T22:23:23.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.401+0000 7f383740b700 1 -- 172.21.15.114:0/3728586639 shutdown_connections 2024-02-16T22:23:23.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.401+0000 7f383740b700 1 --2- 172.21.15.114:0/3728586639 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f381c06d660 0x7f381c06fb20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:23.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.401+0000 7f383740b700 1 --2- 172.21.15.114:0/3728586639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3830100bd0 0x7f3830074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:23.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.401+0000 7f383740b700 1 --2- 172.21.15.114:0/3728586639 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3830101dd0 0x7f3830073180 secure :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f382000e970 tx=0x7f382000ec80 comp rx=0 tx=0).stop 2024-02-16T22:23:23.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.401+0000 7f383740b700 1 -- 172.21.15.114:0/3728586639 >> 172.21.15.114:0/3728586639 conn(0x7f38300fc190 msgr2=0x7f3830105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:23:23.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.401+0000 7f383740b700 1 -- 172.21.15.114:0/3728586639 shutdown_connections 2024-02-16T22:23:23.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.401+0000 7f383740b700 1 -- 172.21.15.114:0/3728586639 wait complete. 2024-02-16T22:23:23.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.542+0000 7fe00593a700 1 -- 172.21.15.114:0/755061970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe000073790 msgr2=0x7fe000073c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:23.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.542+0000 7fe00593a700 1 --2- 172.21.15.114:0/755061970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe000073790 0x7fe000073c10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdff4009910 tx=0x7fdff4009c20 comp rx=0 tx=0).stop 2024-02-16T22:23:23.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.543+0000 7fe00593a700 1 -- 172.21.15.114:0/755061970 shutdown_connections 2024-02-16T22:23:23.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.543+0000 7fe00593a700 1 --2- 172.21.15.114:0/755061970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe000073790 0x7fe000073c10 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:23.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.543+0000 7fe00593a700 1 --2- 172.21.15.114:0/755061970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe000074d60 0x7fe0000731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:23.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.543+0000 7fe00593a700 1 -- 172.21.15.114:0/755061970 >> 172.21.15.114:0/755061970 conn(0x7fe0000fc170 msgr2=0x7fe0000fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:23:23.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.543+0000 7fe00593a700 1 -- 172.21.15.114:0/755061970 shutdown_connections 2024-02-16T22:23:23.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.543+0000 7fe00593a700 1 -- 172.21.15.114:0/755061970 wait complete. 2024-02-16T22:23:23.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.543+0000 7fe00593a700 1 Processor -- start 2024-02-16T22:23:23.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.544+0000 7fe00593a700 1 -- start start 2024-02-16T22:23:23.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.544+0000 7fe00593a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe000073790 0x7fe000198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:23.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.544+0000 7fe00593a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe000074d60 0x7fe000198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:23.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.544+0000 7fe00593a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe000198e80 con 0x7fe000073790 2024-02-16T22:23:23.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.544+0000 7fe00593a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe000198fc0 con 0x7fe000074d60 2024-02-16T22:23:23.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.544+0000 7fdffeffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe000073790 0x7fe000198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:23.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.544+0000 7fdffe7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe000074d60 0x7fe000198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:23.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.544+0000 7fdffe7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe000074d60 0x7fe000198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.114:38664/0 (socket says 172.21.15.114:38664) 2024-02-16T22:23:23.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.544+0000 7fdffeffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe000073790 0x7fe000198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:45080/0 (socket says 172.21.15.114:45080) 2024-02-16T22:23:23.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.544+0000 7fdffe7fc700 1 -- 172.21.15.114:0/1082799525 learned_addr learned my addr 172.21.15.114:0/1082799525 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:23:23.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.545+0000 7fdffeffd700 1 -- 172.21.15.114:0/1082799525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe000074d60 msgr2=0x7fe000198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:23.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.545+0000 7fdffeffd700 1 --2- 172.21.15.114:0/1082799525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe000074d60 0x7fe000198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:23.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.545+0000 7fdffeffd700 1 -- 172.21.15.114:0/1082799525 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff40095f0 con 0x7fe000073790 2024-02-16T22:23:23.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.545+0000 7fdffeffd700 1 --2- 172.21.15.114:0/1082799525 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe000073790 0x7fe000198320 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdff000e970 tx=0x7fdff000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:23:23.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.545+0000 7fe004938700 1 -- 172.21.15.114:0/1082799525 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdff000c9d0 con 0x7fe000073790 2024-02-16T22:23:23.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.545+0000 7fe004938700 1 -- 172.21.15.114:0/1082799525 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdff0010430 con 0x7fe000073790 2024-02-16T22:23:23.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.545+0000 7fe004938700 1 -- 172.21.15.114:0/1082799525 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdff0018650 con 0x7fe000073790 2024-02-16T22:23:23.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.545+0000 7fe00593a700 1 -- 172.21.15.114:0/1082799525 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe00019da70 con 0x7fe000073790 2024-02-16T22:23:23.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.545+0000 7fe00593a700 1 -- 172.21.15.114:0/1082799525 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe00019dfc0 con 0x7fe000073790 2024-02-16T22:23:23.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.547+0000 7fe00593a700 1 -- 172.21.15.114:0/1082799525 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe00010b920 con 0x7fe000073790 2024-02-16T22:23:23.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.547+0000 7fe004938700 1 -- 172.21.15.114:0/1082799525 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 30) v1 ==== 91450+0+0 (secure 0 0 0) 0x7fdff00105a0 con 0x7fe000073790 2024-02-16T22:23:23.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.548+0000 7fe004938700 1 --2- 172.21.15.114:0/1082799525 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7fdfe806d6a0 0x7fdfe806fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:23.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.548+0000 7fe004938700 1 -- 172.21.15.114:0/1082799525 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7fdff0014070 con 0x7fe000073790 2024-02-16T22:23:23.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.548+0000 7fdffe7fc700 1 --2- 172.21.15.114:0/1082799525 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7fdfe806d6a0 0x7fdfe806fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:23.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.549+0000 7fdffe7fc700 1 --2- 172.21.15.114:0/1082799525 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7fdfe806d6a0 0x7fdfe806fb60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdff4006010 tx=0x7fdff4009f90 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:23:23.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.553+0000 7fe004938700 1 -- 172.21.15.114:0/1082799525 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182869 (secure 0 0 0) 0x7fdff0052a70 con 0x7fe000073790 2024-02-16T22:23:23.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.832+0000 7fe00593a700 1 -- 172.21.15.114:0/1082799525 --> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe000107dc0 con 0x7fdfe806d6a0 2024-02-16T22:23:23.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.843+0000 7fe004938700 1 -- 172.21.15.114:0/1082799525 <== mgr.14764 v2:172.21.15.114:6800/3051780151 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe000107dc0 con 0x7fdfe806d6a0 2024-02-16T22:23:23.845 INFO:teuthology.orchestra.run.smithi114.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:23:23.845 INFO:teuthology.orchestra.run.smithi114.stdout:alertmanager.smithi114 smithi114 *:9093,9094 running (9m) 45s ago 10m 24.3M - 0.25.0 c8568f914cd2 e7404fea400e 2024-02-16T22:23:23.845 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi114 smithi114 running (11m) 45s ago 11m 7772k - 18.2.1 94eadd990e0a 7fd07162cd96 2024-02-16T22:23:23.845 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi123 smithi123 running (9m) 3s ago 9m 7562k - 18.2.1 94eadd990e0a 7687f5e99621 2024-02-16T22:23:23.846 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi114 smithi114 running (11m) 45s ago 11m 6660k - 18.2.1 94eadd990e0a fc8fbfe4fdd5 2024-02-16T22:23:23.846 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi123 smithi123 running (9m) 3s ago 9m 6665k - 18.2.1 94eadd990e0a a6249642b9bb 2024-02-16T22:23:23.846 INFO:teuthology.orchestra.run.smithi114.stdout:grafana.smithi114 smithi114 *:3000 running (9m) 45s ago 10m 90.8M - 9.4.7 954c08fa6188 db1c26c3bc20 2024-02-16T22:23:23.846 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.fdxpey smithi114 running (5m) 45s ago 5m 15.9M - 18.2.1 94eadd990e0a 704b40c144fb 2024-02-16T22:23:23.846 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.zzryke smithi114 running (5m) 45s ago 5m 258M - 18.2.1 94eadd990e0a 1f795d23c9a4 2024-02-16T22:23:23.846 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.gopcfa smithi123 running (5m) 3s ago 5m 16.0M - 18.2.1 94eadd990e0a 166134b7532a 2024-02-16T22:23:23.846 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.vhlqbe smithi123 running (5m) 3s ago 5m 17.7M - 18.2.1 94eadd990e0a 243c8bb96fa8 2024-02-16T22:23:23.846 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi114.vcullz smithi114 *:8443,9283,8765 running (81s) 45s ago 12m 478M - 19.0.0-1450-gf78a58c0 f081f847070a 14ebeb7888c0 2024-02-16T22:23:23.846 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi123.inzgjd smithi123 *:8443,9283,8765 running (7s) 3s ago 9m 91.9M - 19.0.0-1450-gf78a58c0 f081f847070a f10aab9faf9e 2024-02-16T22:23:23.846 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi114 smithi114 running (12m) 45s ago 12m 56.1M 2048M 18.2.1 94eadd990e0a b4c4433aa3a4 2024-02-16T22:23:23.846 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi123 smithi123 running (9m) 3s ago 9m 44.0M 2048M 18.2.1 94eadd990e0a c217fc30c152 2024-02-16T22:23:23.846 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi114 smithi114 *:9100 running (10m) 45s ago 10m 15.7M - 1.5.0 0da6a335fe13 744a090e06d0 2024-02-16T22:23:23.846 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi123 smithi123 *:9100 running (9m) 3s ago 9m 15.6M - 1.5.0 0da6a335fe13 3994b8c29b2e 2024-02-16T22:23:23.846 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 smithi114 running (8m) 45s ago 8m 181M 1216M 18.2.1 94eadd990e0a 24ca2e9714e0 2024-02-16T22:23:23.846 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 smithi114 running (8m) 45s ago 8m 184M 1216M 18.2.1 94eadd990e0a 32bb191b88e7 2024-02-16T22:23:23.847 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 smithi114 running (7m) 45s ago 7m 156M 1216M 18.2.1 94eadd990e0a 6608718d0818 2024-02-16T22:23:23.847 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 smithi123 running (7m) 3s ago 7m 243M 2240M 18.2.1 94eadd990e0a 575db379de3d 2024-02-16T22:23:23.847 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 smithi123 running (7m) 3s ago 7m 204M 2240M 18.2.1 94eadd990e0a 7e266866cd9c 2024-02-16T22:23:23.847 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 smithi123 running (6m) 3s ago 6m 188M 2240M 18.2.1 94eadd990e0a 24e5bef19fda 2024-02-16T22:23:23.847 INFO:teuthology.orchestra.run.smithi114.stdout:prometheus.smithi114 smithi114 *:9095 starting - - - - 2024-02-16T22:23:23.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.846+0000 7fe00593a700 1 -- 172.21.15.114:0/1082799525 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7fdfe806d6a0 msgr2=0x7fdfe806fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:23.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.846+0000 7fe00593a700 1 --2- 172.21.15.114:0/1082799525 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7fdfe806d6a0 0x7fdfe806fb60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdff4006010 tx=0x7fdff4009f90 comp rx=0 tx=0).stop 2024-02-16T22:23:23.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.846+0000 7fe00593a700 1 -- 172.21.15.114:0/1082799525 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe000073790 msgr2=0x7fe000198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:23.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.846+0000 7fe00593a700 1 --2- 172.21.15.114:0/1082799525 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe000073790 0x7fe000198320 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdff000e970 tx=0x7fdff000ec80 comp rx=0 tx=0).stop 2024-02-16T22:23:23.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.846+0000 7fe00593a700 1 -- 172.21.15.114:0/1082799525 shutdown_connections 2024-02-16T22:23:23.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.846+0000 7fe00593a700 1 --2- 172.21.15.114:0/1082799525 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7fdfe806d6a0 0x7fdfe806fb60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:23.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.846+0000 7fe00593a700 1 --2- 172.21.15.114:0/1082799525 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe000073790 0x7fe000198320 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:23.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.846+0000 7fe00593a700 1 --2- 172.21.15.114:0/1082799525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe000074d60 0x7fe000198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:23.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.846+0000 7fe00593a700 1 -- 172.21.15.114:0/1082799525 >> 172.21.15.114:0/1082799525 conn(0x7fe0000fc170 msgr2=0x7fe0000fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:23:23.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.847+0000 7fe00593a700 1 -- 172.21.15.114:0/1082799525 shutdown_connections 2024-02-16T22:23:23.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.847+0000 7fe00593a700 1 -- 172.21.15.114:0/1082799525 wait complete. 2024-02-16T22:23:23.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.994+0000 7f3281b58700 1 -- 172.21.15.114:0/657778701 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f327c101640 msgr2=0x7f327c103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:23.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.994+0000 7f3281b58700 1 --2- 172.21.15.114:0/657778701 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f327c101640 0x7f327c103a30 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f326c009910 tx=0x7f326c009c20 comp rx=0 tx=0).stop 2024-02-16T22:23:23.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.995+0000 7f3281b58700 1 -- 172.21.15.114:0/657778701 shutdown_connections 2024-02-16T22:23:23.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.995+0000 7f3281b58700 1 --2- 172.21.15.114:0/657778701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f327c103f70 0x7f327c106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:23.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.995+0000 7f3281b58700 1 --2- 172.21.15.114:0/657778701 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f327c101640 0x7f327c103a30 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:23.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.995+0000 7f3281b58700 1 -- 172.21.15.114:0/657778701 >> 172.21.15.114:0/657778701 conn(0x7f327c0fb030 msgr2=0x7f327c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:23:23.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.995+0000 7f3281b58700 1 -- 172.21.15.114:0/657778701 shutdown_connections 2024-02-16T22:23:23.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.995+0000 7f3281b58700 1 -- 172.21.15.114:0/657778701 wait complete. 2024-02-16T22:23:23.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.995+0000 7f3281b58700 1 Processor -- start 2024-02-16T22:23:23.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.995+0000 7f3281b58700 1 -- start start 2024-02-16T22:23:24.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.996+0000 7f3281b58700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f327c101640 0x7f327c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:24.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.996+0000 7f3281b58700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f327c103f70 0x7f327c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:24.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.996+0000 7f3281b58700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c199070 con 0x7f327c101640 2024-02-16T22:23:24.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.996+0000 7f3281b58700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c1991b0 con 0x7f327c103f70 2024-02-16T22:23:24.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.996+0000 7f327b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f327c101640 0x7f327c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:24.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.996+0000 7f327b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f327c101640 0x7f327c198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:45086/0 (socket says 172.21.15.114:45086) 2024-02-16T22:23:24.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.996+0000 7f327b7fe700 1 -- 172.21.15.114:0/154966847 learned_addr learned my addr 172.21.15.114:0/154966847 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:23:24.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.996+0000 7f327affd700 1 --2- 172.21.15.114:0/154966847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f327c103f70 0x7f327c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:24.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.996+0000 7f327b7fe700 1 -- 172.21.15.114:0/154966847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f327c103f70 msgr2=0x7f327c198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:24.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.996+0000 7f327b7fe700 1 --2- 172.21.15.114:0/154966847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f327c103f70 0x7f327c198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:24.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.996+0000 7f327b7fe700 1 -- 172.21.15.114:0/154966847 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326c0095f0 con 0x7f327c101640 2024-02-16T22:23:24.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.996+0000 7f327b7fe700 1 --2- 172.21.15.114:0/154966847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f327c101640 0x7f327c198510 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f326c005b40 tx=0x7f326c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:23:24.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.997+0000 7f3278ff9700 1 -- 172.21.15.114:0/154966847 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f326c01d070 con 0x7f327c101640 2024-02-16T22:23:24.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.997+0000 7f3278ff9700 1 -- 172.21.15.114:0/154966847 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f326c00f460 con 0x7f327c101640 2024-02-16T22:23:24.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.997+0000 7f3281b58700 1 -- 172.21.15.114:0/154966847 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f327c19dc00 con 0x7f327c101640 2024-02-16T22:23:24.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.997+0000 7f3278ff9700 1 -- 172.21.15.114:0/154966847 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f326c005720 con 0x7f327c101640 2024-02-16T22:23:24.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.997+0000 7f3281b58700 1 -- 172.21.15.114:0/154966847 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f327c19e0f0 con 0x7f327c101640 2024-02-16T22:23:24.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.998+0000 7f3281b58700 1 -- 172.21.15.114:0/154966847 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f327c0fcc20 con 0x7f327c101640 2024-02-16T22:23:24.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.999+0000 7f3278ff9700 1 -- 172.21.15.114:0/154966847 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 30) v1 ==== 91450+0+0 (secure 0 0 0) 0x7f326c00f5d0 con 0x7f327c101640 2024-02-16T22:23:24.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.999+0000 7f3278ff9700 1 --2- 172.21.15.114:0/154966847 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f326406d660 0x7f326406fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:24.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.999+0000 7f3278ff9700 1 -- 172.21.15.114:0/154966847 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f326c08e590 con 0x7f327c101640 2024-02-16T22:23:24.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:23.999+0000 7f327affd700 1 --2- 172.21.15.114:0/154966847 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f326406d660 0x7f326406fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:24.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.000+0000 7f327affd700 1 --2- 172.21.15.114:0/154966847 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f326406d660 0x7f326406fb20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3270005950 tx=0x7f327000a300 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:23:24.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.004+0000 7f3278ff9700 1 -- 172.21.15.114:0/154966847 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182869 (secure 0 0 0) 0x7f326c05bbd0 con 0x7f327c101640 2024-02-16T22:23:24.325 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:24 smithi123 ceph-mon[37161]: from='client.14852 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:23:24.325 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:24 smithi123 ceph-mon[37161]: from='client.14856 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:23:24.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.409+0000 7f3281b58700 1 -- 172.21.15.114:0/154966847 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f327c02ce70 con 0x7f327c101640 2024-02-16T22:23:24.411 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:24 smithi114 ceph-mon[29235]: from='client.14852 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:23:24.411 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:24 smithi114 ceph-mon[29235]: from='client.14856 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:23:24.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.409+0000 7f3278ff9700 1 -- 172.21.15.114:0/154966847 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f326c05b760 con 0x7f327c101640 2024-02-16T22:23:24.412 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:23:24.412 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": { 2024-02-16T22:23:24.412 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-02-16T22:23:24.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:23:24.412 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": { 2024-02-16T22:23:24.412 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:23:24.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:23:24.412 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": { 2024-02-16T22:23:24.412 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-02-16T22:23:24.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:23:24.413 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-02-16T22:23:24.413 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-02-16T22:23:24.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:23:24.413 INFO:teuthology.orchestra.run.smithi114.stdout: "overall": { 2024-02-16T22:23:24.413 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-02-16T22:23:24.413 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:23:24.413 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-02-16T22:23:24.413 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:23:24.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.411+0000 7f3281b58700 1 -- 172.21.15.114:0/154966847 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f326406d660 msgr2=0x7f326406fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:24.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.411+0000 7f3281b58700 1 --2- 172.21.15.114:0/154966847 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f326406d660 0x7f326406fb20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3270005950 tx=0x7f327000a300 comp rx=0 tx=0).stop 2024-02-16T22:23:24.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.412+0000 7f3281b58700 1 -- 172.21.15.114:0/154966847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f327c101640 msgr2=0x7f327c198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:24.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.412+0000 7f3281b58700 1 --2- 172.21.15.114:0/154966847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f327c101640 0x7f327c198510 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f326c005b40 tx=0x7f326c00bd70 comp rx=0 tx=0).stop 2024-02-16T22:23:24.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.412+0000 7f3281b58700 1 -- 172.21.15.114:0/154966847 shutdown_connections 2024-02-16T22:23:24.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.412+0000 7f3281b58700 1 --2- 172.21.15.114:0/154966847 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f326406d660 0x7f326406fb20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:24.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.412+0000 7f3281b58700 1 --2- 172.21.15.114:0/154966847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f327c101640 0x7f327c198510 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:24.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.412+0000 7f3281b58700 1 --2- 172.21.15.114:0/154966847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f327c103f70 0x7f327c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:24.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.412+0000 7f3281b58700 1 -- 172.21.15.114:0/154966847 >> 172.21.15.114:0/154966847 conn(0x7f327c0fb030 msgr2=0x7f327c104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:23:24.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.412+0000 7f3281b58700 1 -- 172.21.15.114:0/154966847 shutdown_connections 2024-02-16T22:23:24.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.412+0000 7f3281b58700 1 -- 172.21.15.114:0/154966847 wait complete. 2024-02-16T22:23:24.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.539+0000 7fb66211d700 1 -- 172.21.15.114:0/3328454798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb65c100cb0 msgr2=0x7fb65c101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:24.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.539+0000 7fb66211d700 1 --2- 172.21.15.114:0/3328454798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb65c100cb0 0x7fb65c101110 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb650009910 tx=0x7fb650009c20 comp rx=0 tx=0).stop 2024-02-16T22:23:24.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.539+0000 7fb66211d700 1 -- 172.21.15.114:0/3328454798 shutdown_connections 2024-02-16T22:23:24.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.539+0000 7fb66211d700 1 --2- 172.21.15.114:0/3328454798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb65c100cb0 0x7fb65c101110 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:24.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.539+0000 7fb66211d700 1 --2- 172.21.15.114:0/3328454798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb65c0ffab0 0x7fb65c0ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:24.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.539+0000 7fb66211d700 1 -- 172.21.15.114:0/3328454798 >> 172.21.15.114:0/3328454798 conn(0x7fb65c0fb050 msgr2=0x7fb65c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:23:24.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.539+0000 7fb66211d700 1 -- 172.21.15.114:0/3328454798 shutdown_connections 2024-02-16T22:23:24.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.539+0000 7fb66211d700 1 -- 172.21.15.114:0/3328454798 wait complete. 2024-02-16T22:23:24.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.540+0000 7fb66211d700 1 Processor -- start 2024-02-16T22:23:24.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.540+0000 7fb66211d700 1 -- start start 2024-02-16T22:23:24.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.540+0000 7fb66211d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb65c0ffab0 0x7fb65c069860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:24.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.540+0000 7fb66211d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb65c100cb0 0x7fb65c19b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:24.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.540+0000 7fb66211d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c19b680 con 0x7fb65c0ffab0 2024-02-16T22:23:24.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.540+0000 7fb66211d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c19b7c0 con 0x7fb65c100cb0 2024-02-16T22:23:24.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.540+0000 7fb65b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb65c0ffab0 0x7fb65c069860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:24.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.540+0000 7fb65b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb65c0ffab0 0x7fb65c069860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:45108/0 (socket says 172.21.15.114:45108) 2024-02-16T22:23:24.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.540+0000 7fb65b7fe700 1 -- 172.21.15.114:0/1643162114 learned_addr learned my addr 172.21.15.114:0/1643162114 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:23:24.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.541+0000 7fb65affd700 1 --2- 172.21.15.114:0/1643162114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb65c100cb0 0x7fb65c19b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:24.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.541+0000 7fb65b7fe700 1 -- 172.21.15.114:0/1643162114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb65c100cb0 msgr2=0x7fb65c19b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:24.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.541+0000 7fb65b7fe700 1 --2- 172.21.15.114:0/1643162114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb65c100cb0 0x7fb65c19b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:24.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.541+0000 7fb65b7fe700 1 -- 172.21.15.114:0/1643162114 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6500095f0 con 0x7fb65c0ffab0 2024-02-16T22:23:24.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.541+0000 7fb65affd700 1 --2- 172.21.15.114:0/1643162114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb65c100cb0 0x7fb65c19b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:23:24.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.541+0000 7fb65b7fe700 1 --2- 172.21.15.114:0/1643162114 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb65c0ffab0 0x7fb65c069860 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00e8f0 tx=0x7fb64c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:23:24.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.541+0000 7fb658ff9700 1 -- 172.21.15.114:0/1643162114 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb64c009790 con 0x7fb65c0ffab0 2024-02-16T22:23:24.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.541+0000 7fb658ff9700 1 -- 172.21.15.114:0/1643162114 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb64c00c810 con 0x7fb65c0ffab0 2024-02-16T22:23:24.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.541+0000 7fb66211d700 1 -- 172.21.15.114:0/1643162114 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb65c19baa0 con 0x7fb65c0ffab0 2024-02-16T22:23:24.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.541+0000 7fb658ff9700 1 -- 172.21.15.114:0/1643162114 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb64c003d20 con 0x7fb65c0ffab0 2024-02-16T22:23:24.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.541+0000 7fb66211d700 1 -- 172.21.15.114:0/1643162114 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb65c19bff0 con 0x7fb65c0ffab0 2024-02-16T22:23:24.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.543+0000 7fb66211d700 1 -- 172.21.15.114:0/1643162114 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb65c108610 con 0x7fb65c0ffab0 2024-02-16T22:23:24.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.543+0000 7fb658ff9700 1 -- 172.21.15.114:0/1643162114 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 30) v1 ==== 91450+0+0 (secure 0 0 0) 0x7fb64c0041f0 con 0x7fb65c0ffab0 2024-02-16T22:23:24.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.544+0000 7fb658ff9700 1 --2- 172.21.15.114:0/1643162114 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7fb64406d650 0x7fb64406fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:24.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.544+0000 7fb658ff9700 1 -- 172.21.15.114:0/1643162114 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7fb64c014070 con 0x7fb65c0ffab0 2024-02-16T22:23:24.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.544+0000 7fb65affd700 1 --2- 172.21.15.114:0/1643162114 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7fb64406d650 0x7fb64406fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:24.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.544+0000 7fb65affd700 1 --2- 172.21.15.114:0/1643162114 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7fb64406d650 0x7fb64406fb10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb65000b6f0 tx=0x7fb650005a90 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:23:24.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.549+0000 7fb658ff9700 1 -- 172.21.15.114:0/1643162114 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182869 (secure 0 0 0) 0x7fb64c05aa90 con 0x7fb65c0ffab0 2024-02-16T22:23:24.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.881+0000 7fb66211d700 1 -- 172.21.15.114:0/1643162114 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb65c108800 con 0x7fb65c0ffab0 2024-02-16T22:23:24.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.881+0000 7fb658ff9700 1 -- 172.21.15.114:0/1643162114 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1825 (secure 0 0 0) 0x7fb64c05a620 con 0x7fb65c0ffab0 2024-02-16T22:23:24.883 INFO:teuthology.orchestra.run.smithi114.stdout:e14 2024-02-16T22:23:24.884 INFO:teuthology.orchestra.run.smithi114.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:23:24.884 INFO:teuthology.orchestra.run.smithi114.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:23:24.884 INFO:teuthology.orchestra.run.smithi114.stdout:legacy client fscid: 1 2024-02-16T22:23:24.884 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:23:24.884 INFO:teuthology.orchestra.run.smithi114.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:23:24.884 INFO:teuthology.orchestra.run.smithi114.stdout:fs_name cephfs 2024-02-16T22:23:24.884 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 12 2024-02-16T22:23:24.884 INFO:teuthology.orchestra.run.smithi114.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:23:24.884 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-02-16T22:18:02.108825+0000 2024-02-16T22:23:24.884 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-02-16T22:18:16.413371+0000 2024-02-16T22:23:24.884 INFO:teuthology.orchestra.run.smithi114.stdout:tableserver 0 2024-02-16T22:23:24.884 INFO:teuthology.orchestra.run.smithi114.stdout:root 0 2024-02-16T22:23:24.884 INFO:teuthology.orchestra.run.smithi114.stdout:session_timeout 60 2024-02-16T22:23:24.885 INFO:teuthology.orchestra.run.smithi114.stdout:session_autoclose 300 2024-02-16T22:23:24.885 INFO:teuthology.orchestra.run.smithi114.stdout:max_file_size 1099511627776 2024-02-16T22:23:24.885 INFO:teuthology.orchestra.run.smithi114.stdout:required_client_features {} 2024-02-16T22:23:24.885 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure 0 2024-02-16T22:23:24.885 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure_osd_epoch 0 2024-02-16T22:23:24.885 INFO:teuthology.orchestra.run.smithi114.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:23:24.885 INFO:teuthology.orchestra.run.smithi114.stdout:max_mds 1 2024-02-16T22:23:24.885 INFO:teuthology.orchestra.run.smithi114.stdout:in 0 2024-02-16T22:23:24.885 INFO:teuthology.orchestra.run.smithi114.stdout:up {0=14468} 2024-02-16T22:23:24.885 INFO:teuthology.orchestra.run.smithi114.stdout:failed 2024-02-16T22:23:24.885 INFO:teuthology.orchestra.run.smithi114.stdout:damaged 2024-02-16T22:23:24.885 INFO:teuthology.orchestra.run.smithi114.stdout:stopped 2024-02-16T22:23:24.885 INFO:teuthology.orchestra.run.smithi114.stdout:data_pools [3] 2024-02-16T22:23:24.885 INFO:teuthology.orchestra.run.smithi114.stdout:metadata_pool 2 2024-02-16T22:23:24.885 INFO:teuthology.orchestra.run.smithi114.stdout:inline_data enabled 2024-02-16T22:23:24.886 INFO:teuthology.orchestra.run.smithi114.stdout:balancer 2024-02-16T22:23:24.886 INFO:teuthology.orchestra.run.smithi114.stdout:bal_rank_mask -1 2024-02-16T22:23:24.886 INFO:teuthology.orchestra.run.smithi114.stdout:standby_count_wanted 1 2024-02-16T22:23:24.886 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.zzryke{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.114:6826/277953535,v1:172.21.15.114:6827/277953535] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:23:24.886 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:23:24.886 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:23:24.886 INFO:teuthology.orchestra.run.smithi114.stdout:Standby daemons: 2024-02-16T22:23:24.886 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:23:24.886 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.fdxpey{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:23:24.886 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.vhlqbe{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:23:24.886 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.gopcfa{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:23:24.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.883+0000 7fb66211d700 1 -- 172.21.15.114:0/1643162114 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7fb64406d650 msgr2=0x7fb64406fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:24.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.883+0000 7fb66211d700 1 --2- 172.21.15.114:0/1643162114 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7fb64406d650 0x7fb64406fb10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb65000b6f0 tx=0x7fb650005a90 comp rx=0 tx=0).stop 2024-02-16T22:23:24.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.883+0000 7fb66211d700 1 -- 172.21.15.114:0/1643162114 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb65c0ffab0 msgr2=0x7fb65c069860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:24.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.883+0000 7fb66211d700 1 --2- 172.21.15.114:0/1643162114 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb65c0ffab0 0x7fb65c069860 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00e8f0 tx=0x7fb64c00ec00 comp rx=0 tx=0).stop 2024-02-16T22:23:24.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.883+0000 7fb66211d700 1 -- 172.21.15.114:0/1643162114 shutdown_connections 2024-02-16T22:23:24.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.883+0000 7fb66211d700 1 --2- 172.21.15.114:0/1643162114 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7fb64406d650 0x7fb64406fb10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:24.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.883+0000 7fb66211d700 1 --2- 172.21.15.114:0/1643162114 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb65c0ffab0 0x7fb65c069860 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:24.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.883+0000 7fb66211d700 1 --2- 172.21.15.114:0/1643162114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb65c100cb0 0x7fb65c19b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:24.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.883+0000 7fb66211d700 1 -- 172.21.15.114:0/1643162114 >> 172.21.15.114:0/1643162114 conn(0x7fb65c0fb050 msgr2=0x7fb65c103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:23:24.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.884+0000 7fb66211d700 1 -- 172.21.15.114:0/1643162114 shutdown_connections 2024-02-16T22:23:24.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:24.884+0000 7fb66211d700 1 -- 172.21.15.114:0/1643162114 wait complete. 2024-02-16T22:23:24.888 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 14 2024-02-16T22:23:25.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.001+0000 7f1f754dc700 1 -- 172.21.15.114:0/925562237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f700ffab0 msgr2=0x7f1f700ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:25.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.001+0000 7f1f754dc700 1 --2- 172.21.15.114:0/925562237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f700ffab0 0x7f1f700ffed0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1f60009910 tx=0x7f1f60009c20 comp rx=0 tx=0).stop 2024-02-16T22:23:25.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.001+0000 7f1f754dc700 1 -- 172.21.15.114:0/925562237 shutdown_connections 2024-02-16T22:23:25.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.001+0000 7f1f754dc700 1 --2- 172.21.15.114:0/925562237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f70100cb0 0x7f1f70101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:25.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.001+0000 7f1f754dc700 1 --2- 172.21.15.114:0/925562237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f700ffab0 0x7f1f700ffed0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:25.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.001+0000 7f1f754dc700 1 -- 172.21.15.114:0/925562237 >> 172.21.15.114:0/925562237 conn(0x7f1f700fb050 msgr2=0x7f1f700fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:23:25.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.001+0000 7f1f754dc700 1 -- 172.21.15.114:0/925562237 shutdown_connections 2024-02-16T22:23:25.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.002+0000 7f1f754dc700 1 -- 172.21.15.114:0/925562237 wait complete. 2024-02-16T22:23:25.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.002+0000 7f1f754dc700 1 Processor -- start 2024-02-16T22:23:25.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.002+0000 7f1f754dc700 1 -- start start 2024-02-16T22:23:25.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.002+0000 7f1f754dc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f700ffab0 0x7f1f70069860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:25.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.002+0000 7f1f754dc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f70100cb0 0x7f1f7019b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:25.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.002+0000 7f1f754dc700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7019b6a0 con 0x7f1f70100cb0 2024-02-16T22:23:25.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.002+0000 7f1f754dc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7019b7e0 con 0x7f1f700ffab0 2024-02-16T22:23:25.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.003+0000 7f1f6e7fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f70100cb0 0x7f1f7019b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:25.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.003+0000 7f1f6e7fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f70100cb0 0x7f1f7019b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:45120/0 (socket says 172.21.15.114:45120) 2024-02-16T22:23:25.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.003+0000 7f1f6e7fc700 1 -- 172.21.15.114:0/3841268567 learned_addr learned my addr 172.21.15.114:0/3841268567 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:23:25.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.003+0000 7f1f6effd700 1 --2- 172.21.15.114:0/3841268567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f700ffab0 0x7f1f70069860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:25.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.003+0000 7f1f6e7fc700 1 -- 172.21.15.114:0/3841268567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f700ffab0 msgr2=0x7f1f70069860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:25.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.003+0000 7f1f6e7fc700 1 --2- 172.21.15.114:0/3841268567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f700ffab0 0x7f1f70069860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:25.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.003+0000 7f1f6e7fc700 1 -- 172.21.15.114:0/3841268567 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f600095f0 con 0x7f1f70100cb0 2024-02-16T22:23:25.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.003+0000 7f1f6e7fc700 1 --2- 172.21.15.114:0/3841268567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f70100cb0 0x7f1f7019b080 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1f6400d900 tx=0x7f1f6400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:23:25.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.003+0000 7f1f57fff700 1 -- 172.21.15.114:0/3841268567 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f640041f0 con 0x7f1f70100cb0 2024-02-16T22:23:25.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.003+0000 7f1f754dc700 1 -- 172.21.15.114:0/3841268567 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f7019bac0 con 0x7f1f70100cb0 2024-02-16T22:23:25.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.003+0000 7f1f57fff700 1 -- 172.21.15.114:0/3841268567 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1f64004ce0 con 0x7f1f70100cb0 2024-02-16T22:23:25.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.003+0000 7f1f57fff700 1 -- 172.21.15.114:0/3841268567 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f6400b750 con 0x7f1f70100cb0 2024-02-16T22:23:25.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.003+0000 7f1f754dc700 1 -- 172.21.15.114:0/3841268567 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f7019c010 con 0x7f1f70100cb0 2024-02-16T22:23:25.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.005+0000 7f1f754dc700 1 -- 172.21.15.114:0/3841268567 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f70108610 con 0x7f1f70100cb0 2024-02-16T22:23:25.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.006+0000 7f1f57fff700 1 -- 172.21.15.114:0/3841268567 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 30) v1 ==== 91450+0+0 (secure 0 0 0) 0x7f1f64004e50 con 0x7f1f70100cb0 2024-02-16T22:23:25.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.006+0000 7f1f57fff700 1 --2- 172.21.15.114:0/3841268567 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f1f5806d650 0x7f1f5806fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:25.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.006+0000 7f1f57fff700 1 -- 172.21.15.114:0/3841268567 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7f1f6408d090 con 0x7f1f70100cb0 2024-02-16T22:23:25.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.006+0000 7f1f6effd700 1 --2- 172.21.15.114:0/3841268567 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f1f5806d650 0x7f1f5806fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:25.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.007+0000 7f1f6effd700 1 --2- 172.21.15.114:0/3841268567 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f1f5806d650 0x7f1f5806fb10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1f60006010 tx=0x7f1f6000b560 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:23:25.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.012+0000 7f1f57fff700 1 -- 172.21.15.114:0/3841268567 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182869 (secure 0 0 0) 0x7f1f6405a6d0 con 0x7f1f70100cb0 2024-02-16T22:23:25.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.118+0000 7f1f57fff700 1 -- 172.21.15.114:0/3841268567 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mgrmap(e 31) v1 ==== 91450+0+0 (secure 0 0 0) 0x7f1f6408b440 con 0x7f1f70100cb0 2024-02-16T22:23:25.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:25 smithi123 ceph-mon[37161]: from='client.14860 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:23:25.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:25 smithi123 ceph-mon[37161]: pgmap v29: 65 pgs: 65 active+clean; 310 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 47 KiB/s rd, 2.6 MiB/s wr, 523 op/s 2024-02-16T22:23:25.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:25 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/154966847' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:23:25.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:25 smithi123 ceph-mon[37161]: Standby manager daemon smithi123.inzgjd restarted 2024-02-16T22:23:25.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:25 smithi123 ceph-mon[37161]: Standby manager daemon smithi123.inzgjd started 2024-02-16T22:23:25.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:25 smithi123 ceph-mon[37161]: from='mgr.? 172.21.15.123:0/2853168152' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi123.inzgjd/crt"}]: dispatch 2024-02-16T22:23:25.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:25 smithi123 ceph-mon[37161]: from='mgr.? 172.21.15.123:0/2853168152' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T22:23:25.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:25 smithi123 ceph-mon[37161]: from='mgr.? 172.21.15.123:0/2853168152' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi123.inzgjd/key"}]: dispatch 2024-02-16T22:23:25.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:25 smithi123 ceph-mon[37161]: from='mgr.? 172.21.15.123:0/2853168152' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T22:23:25.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:25 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/1643162114' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:23:25.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.368+0000 7f1f754dc700 1 -- 172.21.15.114:0/3841268567 --> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f70105600 con 0x7f1f5806d650 2024-02-16T22:23:25.370 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:25 smithi114 ceph-mon[29235]: from='client.14860 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:23:25.370 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:25 smithi114 ceph-mon[29235]: pgmap v29: 65 pgs: 65 active+clean; 310 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 47 KiB/s rd, 2.6 MiB/s wr, 523 op/s 2024-02-16T22:23:25.370 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:25 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/154966847' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:23:25.370 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:25 smithi114 ceph-mon[29235]: Standby manager daemon smithi123.inzgjd restarted 2024-02-16T22:23:25.371 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:25 smithi114 ceph-mon[29235]: Standby manager daemon smithi123.inzgjd started 2024-02-16T22:23:25.371 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:25 smithi114 ceph-mon[29235]: from='mgr.? 172.21.15.123:0/2853168152' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi123.inzgjd/crt"}]: dispatch 2024-02-16T22:23:25.371 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:25 smithi114 ceph-mon[29235]: from='mgr.? 172.21.15.123:0/2853168152' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T22:23:25.371 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:25 smithi114 ceph-mon[29235]: from='mgr.? 172.21.15.123:0/2853168152' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi123.inzgjd/key"}]: dispatch 2024-02-16T22:23:25.371 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:25 smithi114 ceph-mon[29235]: from='mgr.? 172.21.15.123:0/2853168152' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T22:23:25.371 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:25 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/1643162114' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:23:25.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.369+0000 7f1f57fff700 1 -- 172.21.15.114:0/3841268567 <== mgr.14764 v2:172.21.15.114:6800/3051780151 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f1f70105600 con 0x7f1f5806d650 2024-02-16T22:23:25.371 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:23:25.372 INFO:teuthology.orchestra.run.smithi114.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:23:25.372 INFO:teuthology.orchestra.run.smithi114.stdout: "in_progress": true, 2024-02-16T22:23:25.372 INFO:teuthology.orchestra.run.smithi114.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:23:25.372 INFO:teuthology.orchestra.run.smithi114.stdout: "services_complete": [ 2024-02-16T22:23:25.372 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr" 2024-02-16T22:23:25.372 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-02-16T22:23:25.372 INFO:teuthology.orchestra.run.smithi114.stdout: "progress": "2/23 daemons upgraded", 2024-02-16T22:23:25.372 INFO:teuthology.orchestra.run.smithi114.stdout: "message": "Currently upgrading mgr daemons", 2024-02-16T22:23:25.372 INFO:teuthology.orchestra.run.smithi114.stdout: "is_paused": false 2024-02-16T22:23:25.372 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:23:25.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.373+0000 7f1f754dc700 1 -- 172.21.15.114:0/3841268567 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f1f5806d650 msgr2=0x7f1f5806fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:25.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.373+0000 7f1f754dc700 1 --2- 172.21.15.114:0/3841268567 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f1f5806d650 0x7f1f5806fb10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1f60006010 tx=0x7f1f6000b560 comp rx=0 tx=0).stop 2024-02-16T22:23:25.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.373+0000 7f1f754dc700 1 -- 172.21.15.114:0/3841268567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f70100cb0 msgr2=0x7f1f7019b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:25.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.373+0000 7f1f754dc700 1 --2- 172.21.15.114:0/3841268567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f70100cb0 0x7f1f7019b080 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1f6400d900 tx=0x7f1f6400dc10 comp rx=0 tx=0).stop 2024-02-16T22:23:25.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.373+0000 7f1f754dc700 1 -- 172.21.15.114:0/3841268567 shutdown_connections 2024-02-16T22:23:25.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.373+0000 7f1f754dc700 1 --2- 172.21.15.114:0/3841268567 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7f1f5806d650 0x7f1f5806fb10 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:25.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.373+0000 7f1f754dc700 1 --2- 172.21.15.114:0/3841268567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f700ffab0 0x7f1f70069860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:25.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.373+0000 7f1f754dc700 1 --2- 172.21.15.114:0/3841268567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f70100cb0 0x7f1f7019b080 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:25.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.373+0000 7f1f754dc700 1 -- 172.21.15.114:0/3841268567 >> 172.21.15.114:0/3841268567 conn(0x7f1f700fb050 msgr2=0x7f1f70103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:23:25.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.373+0000 7f1f754dc700 1 -- 172.21.15.114:0/3841268567 shutdown_connections 2024-02-16T22:23:25.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.373+0000 7f1f754dc700 1 -- 172.21.15.114:0/3841268567 wait complete. 2024-02-16T22:23:25.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.506+0000 7fe890078700 1 -- 172.21.15.114:0/2797109110 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe888100ce0 msgr2=0x7fe888101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:25.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.506+0000 7fe890078700 1 --2- 172.21.15.114:0/2797109110 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe888100ce0 0x7fe888101140 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe878009910 tx=0x7fe878009c20 comp rx=0 tx=0).stop 2024-02-16T22:23:25.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.506+0000 7fe890078700 1 -- 172.21.15.114:0/2797109110 shutdown_connections 2024-02-16T22:23:25.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.506+0000 7fe890078700 1 --2- 172.21.15.114:0/2797109110 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe888100ce0 0x7fe888101140 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:25.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.506+0000 7fe890078700 1 --2- 172.21.15.114:0/2797109110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8880ffae0 0x7fe8880fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:25.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.507+0000 7fe890078700 1 -- 172.21.15.114:0/2797109110 >> 172.21.15.114:0/2797109110 conn(0x7fe8880fb080 msgr2=0x7fe8880fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:23:25.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.507+0000 7fe890078700 1 -- 172.21.15.114:0/2797109110 shutdown_connections 2024-02-16T22:23:25.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.507+0000 7fe890078700 1 -- 172.21.15.114:0/2797109110 wait complete. 2024-02-16T22:23:25.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.507+0000 7fe890078700 1 Processor -- start 2024-02-16T22:23:25.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.507+0000 7fe890078700 1 -- start start 2024-02-16T22:23:25.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.508+0000 7fe890078700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8880ffae0 0x7fe888103a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:25.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.508+0000 7fe890078700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe888100ce0 0x7fe8881020c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:25.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.508+0000 7fe890078700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe888102600 con 0x7fe8880ffae0 2024-02-16T22:23:25.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.508+0000 7fe890078700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe888102740 con 0x7fe888100ce0 2024-02-16T22:23:25.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.508+0000 7fe88d613700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe888100ce0 0x7fe8881020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:25.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.508+0000 7fe88de14700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8880ffae0 0x7fe888103a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:25.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.508+0000 7fe88d613700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe888100ce0 0x7fe8881020c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.114:38732/0 (socket says 172.21.15.114:38732) 2024-02-16T22:23:25.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.508+0000 7fe88d613700 1 -- 172.21.15.114:0/2707290858 learned_addr learned my addr 172.21.15.114:0/2707290858 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:23:25.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.508+0000 7fe88de14700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8880ffae0 0x7fe888103a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:45146/0 (socket says 172.21.15.114:45146) 2024-02-16T22:23:25.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.508+0000 7fe88d613700 1 -- 172.21.15.114:0/2707290858 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8880ffae0 msgr2=0x7fe888103a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:25.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.508+0000 7fe88d613700 1 --2- 172.21.15.114:0/2707290858 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8880ffae0 0x7fe888103a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:25.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.508+0000 7fe88d613700 1 -- 172.21.15.114:0/2707290858 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8780095f0 con 0x7fe888100ce0 2024-02-16T22:23:25.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.508+0000 7fe88d613700 1 --2- 172.21.15.114:0/2707290858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe888100ce0 0x7fe8881020c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe878000c00 tx=0x7fe87800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:23:25.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.508+0000 7fe87effd700 1 -- 172.21.15.114:0/2707290858 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe87801d070 con 0x7fe888100ce0 2024-02-16T22:23:25.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.508+0000 7fe890078700 1 -- 172.21.15.114:0/2707290858 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8881029c0 con 0x7fe888100ce0 2024-02-16T22:23:25.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.509+0000 7fe890078700 1 -- 172.21.15.114:0/2707290858 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe888102eb0 con 0x7fe888100ce0 2024-02-16T22:23:25.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.509+0000 7fe87effd700 1 -- 172.21.15.114:0/2707290858 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe878005dc0 con 0x7fe888100ce0 2024-02-16T22:23:25.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.509+0000 7fe87effd700 1 -- 172.21.15.114:0/2707290858 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8780226a0 con 0x7fe888100ce0 2024-02-16T22:23:25.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.510+0000 7fe890078700 1 -- 172.21.15.114:0/2707290858 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe86c005320 con 0x7fe888100ce0 2024-02-16T22:23:25.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.510+0000 7fe87effd700 1 -- 172.21.15.114:0/2707290858 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 31) v1 ==== 91450+0+0 (secure 0 0 0) 0x7fe87800faf0 con 0x7fe888100ce0 2024-02-16T22:23:25.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.511+0000 7fe87effd700 1 --2- 172.21.15.114:0/2707290858 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7fe87406d650 0x7fe87406fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:25.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.511+0000 7fe87effd700 1 -- 172.21.15.114:0/2707290858 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6070+0+0 (secure 0 0 0) 0x7fe87808e570 con 0x7fe888100ce0 2024-02-16T22:23:25.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.511+0000 7fe88de14700 1 --2- 172.21.15.114:0/2707290858 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7fe87406d650 0x7fe87406fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:25.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.511+0000 7fe88de14700 1 --2- 172.21.15.114:0/2707290858 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7fe87406d650 0x7fe87406fb10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe888100b40 tx=0x7fe884009260 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:23:25.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.516+0000 7fe87effd700 1 -- 172.21.15.114:0/2707290858 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182869 (secure 0 0 0) 0x7fe87805bc30 con 0x7fe888100ce0 2024-02-16T22:23:25.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.932+0000 7fe890078700 1 -- 172.21.15.114:0/2707290858 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe86c005190 con 0x7fe888100ce0 2024-02-16T22:23:25.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.933+0000 7fe87effd700 1 -- 172.21.15.114:0/2707290858 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fe87805b7c0 con 0x7fe888100ce0 2024-02-16T22:23:25.935 INFO:teuthology.orchestra.run.smithi114.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:23:25.935 INFO:teuthology.orchestra.run.smithi114.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:23:25.935 INFO:teuthology.orchestra.run.smithi114.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:23:25.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.935+0000 7fe890078700 1 -- 172.21.15.114:0/2707290858 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7fe87406d650 msgr2=0x7fe87406fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:25.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.935+0000 7fe890078700 1 --2- 172.21.15.114:0/2707290858 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7fe87406d650 0x7fe87406fb10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe888100b40 tx=0x7fe884009260 comp rx=0 tx=0).stop 2024-02-16T22:23:25.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.935+0000 7fe890078700 1 -- 172.21.15.114:0/2707290858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe888100ce0 msgr2=0x7fe8881020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:25.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.935+0000 7fe890078700 1 --2- 172.21.15.114:0/2707290858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe888100ce0 0x7fe8881020c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe878000c00 tx=0x7fe87800f690 comp rx=0 tx=0).stop 2024-02-16T22:23:25.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.935+0000 7fe890078700 1 -- 172.21.15.114:0/2707290858 shutdown_connections 2024-02-16T22:23:25.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.935+0000 7fe890078700 1 --2- 172.21.15.114:0/2707290858 >> [v2:172.21.15.114:6800/3051780151,v1:172.21.15.114:6801/3051780151] conn(0x7fe87406d650 0x7fe87406fb10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:25.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.935+0000 7fe890078700 1 --2- 172.21.15.114:0/2707290858 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8880ffae0 0x7fe888103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:25.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.935+0000 7fe890078700 1 --2- 172.21.15.114:0/2707290858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe888100ce0 0x7fe8881020c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:25.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.935+0000 7fe890078700 1 -- 172.21.15.114:0/2707290858 >> 172.21.15.114:0/2707290858 conn(0x7fe8880fb080 msgr2=0x7fe888068d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:23:25.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.935+0000 7fe890078700 1 -- 172.21.15.114:0/2707290858 shutdown_connections 2024-02-16T22:23:25.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:25.935+0000 7fe890078700 1 -- 172.21.15.114:0/2707290858 wait complete. 2024-02-16T22:23:26.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:26 smithi114 ceph-mon[29235]: mgrmap e31: smithi114.vcullz(active, since 53s), standbys: smithi123.inzgjd 2024-02-16T22:23:26.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:26 smithi114 ceph-mon[29235]: from='client.14872 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:23:26.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:26 smithi114 ceph-mon[29235]: from='client.? 172.21.15.114:0/2707290858' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:23:26.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:26 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:26.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:26 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:26.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:26 smithi123 ceph-mon[37161]: mgrmap e31: smithi114.vcullz(active, since 53s), standbys: smithi123.inzgjd 2024-02-16T22:23:26.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:26 smithi123 ceph-mon[37161]: from='client.14872 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:23:26.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:26 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/2707290858' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:23:26.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:26 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:26.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:26 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:27.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:27 smithi114 ceph-mon[29235]: pgmap v30: 65 pgs: 65 active+clean; 310 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.7 MiB/s wr, 466 op/s 2024-02-16T22:23:27.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:27 smithi123 ceph-mon[37161]: pgmap v30: 65 pgs: 65 active+clean; 310 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.7 MiB/s wr, 466 op/s 2024-02-16T22:23:29.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:29 smithi114 ceph-mon[29235]: pgmap v31: 65 pgs: 65 active+clean; 310 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.7 MiB/s wr, 466 op/s 2024-02-16T22:23:29.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:29 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:29.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:29 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:29.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:29 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:23:29.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:29 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:23:29.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:29 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:29.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:29 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:23:29.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:29 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:23:29.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:29 smithi114 ceph-mon[29235]: Upgrade: Setting container_image for all mgr 2024-02-16T22:23:29.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:29 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:29.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:29 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi114.vcullz"}]: dispatch 2024-02-16T22:23:29.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:29 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi114.vcullz"}]': finished 2024-02-16T22:23:29.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:29 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi123.inzgjd"}]: dispatch 2024-02-16T22:23:29.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:29 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi123.inzgjd"}]': finished 2024-02-16T22:23:29.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:29 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-16T22:23:29.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:29 smithi123 ceph-mon[37161]: pgmap v31: 65 pgs: 65 active+clean; 310 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.7 MiB/s wr, 466 op/s 2024-02-16T22:23:29.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:29 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:29.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:29 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:29.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:29 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:23:29.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:29 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:23:29.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:29 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:29.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:29 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:23:29.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:29 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:23:29.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:29 smithi123 ceph-mon[37161]: Upgrade: Setting container_image for all mgr 2024-02-16T22:23:29.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:29 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:29.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:29 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi114.vcullz"}]: dispatch 2024-02-16T22:23:29.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:29 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi114.vcullz"}]': finished 2024-02-16T22:23:29.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:29 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi123.inzgjd"}]: dispatch 2024-02-16T22:23:29.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:29 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi123.inzgjd"}]': finished 2024-02-16T22:23:29.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:29 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-16T22:23:31.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:31 smithi114 ceph-mon[29235]: pgmap v32: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 43 KiB/s rd, 2.3 MiB/s wr, 567 op/s 2024-02-16T22:23:31.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:31 smithi114 ceph-mon[29235]: Upgrade: Updating mon.smithi114 2024-02-16T22:23:31.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:31 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:31.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:31 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T22:23:31.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:31 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T22:23:31.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:31 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:23:31.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:31 smithi114 ceph-mon[29235]: Deploying daemon mon.smithi114 on smithi114 2024-02-16T22:23:32.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:31 smithi123 ceph-mon[37161]: pgmap v32: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 43 KiB/s rd, 2.3 MiB/s wr, 567 op/s 2024-02-16T22:23:32.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:31 smithi123 ceph-mon[37161]: Upgrade: Updating mon.smithi114 2024-02-16T22:23:32.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:31 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:32.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:31 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T22:23:32.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:31 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T22:23:32.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:31 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:23:32.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:31 smithi123 ceph-mon[37161]: Deploying daemon mon.smithi114 on smithi114 2024-02-16T22:23:33.136 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:32 smithi114 systemd[1]: Stopping Ceph mon.smithi114 for 1dcbaabe-cd18-11ee-95ba-87774f69a715... 2024-02-16T22:23:33.136 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:32 smithi114 ceph-mon[29235]: pgmap v33: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 1.3 MiB/s wr, 225 op/s 2024-02-16T22:23:33.136 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:32 smithi114 ceph-mon[29235]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:23:33.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:32 smithi123 ceph-mon[37161]: pgmap v33: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 1.3 MiB/s wr, 225 op/s 2024-02-16T22:23:33.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:32 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:23:33.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:33 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi114[29231]: 2024-02-16T22:23:33.133+0000 7fcda4506700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi114 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T22:23:33.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:33 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi114[29231]: 2024-02-16T22:23:33.133+0000 7fcda4506700 -1 mon.smithi114@0(leader) e2 *** Got Signal Terminated *** 2024-02-16T22:23:33.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:33 smithi114 podman[94285]: 2024-02-16 22:23:33.195129787 +0000 UTC m=+0.213112559 container died b4c4433aa3a498943b424ae37b066e6165519cec63521ab0fb8159651676d519 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi114, CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, org.label-schema.build-date=20240214) 2024-02-16T22:23:33.939 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:33 smithi114 podman[94285]: 2024-02-16 22:23:33.693820574 +0000 UTC m=+0.711803350 container cleanup b4c4433aa3a498943b424ae37b066e6165519cec63521ab0fb8159651676d519 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi114, CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, ceph=True, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-02-16T22:23:33.939 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:33 smithi114 bash[94285]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi114 2024-02-16T22:23:34.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:33 smithi114 podman[94307]: 2024-02-16 22:23:33.937677622 +0000 UTC m=+0.739715001 container remove b4c4433aa3a498943b424ae37b066e6165519cec63521ab0fb8159651676d519 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi114, RELEASE=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, ceph=True) 2024-02-16T22:23:34.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:34 smithi114 systemd[1]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@mon.smithi114.service: Deactivated successfully. 2024-02-16T22:23:34.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:34 smithi114 systemd[1]: Stopped Ceph mon.smithi114 for 1dcbaabe-cd18-11ee-95ba-87774f69a715. 2024-02-16T22:23:34.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:34 smithi114 systemd[1]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@mon.smithi114.service: Consumed 12.430s CPU time. 2024-02-16T22:23:34.640 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:34 smithi114 systemd[1]: Starting Ceph mon.smithi114 for 1dcbaabe-cd18-11ee-95ba-87774f69a715... 2024-02-16T22:23:34.640 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:34 smithi114 podman[94434]: 2024-02-16 22:23:34.506152469 +0000 UTC m=+0.022908931 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T22:23:34.640 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:34 smithi114 podman[94434]: 2024-02-16 22:23:34.638954093 +0000 UTC m=+0.155710549 container create 58ac063725235a5943ffb3c6a4284229d105f7cbf6736c59d56b3decc54325ea (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi114, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True) 2024-02-16T22:23:35.372 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:34 smithi114 podman[94434]: 2024-02-16 22:23:34.977120068 +0000 UTC m=+0.493876523 container init 58ac063725235a5943ffb3c6a4284229d105f7cbf6736c59d56b3decc54325ea (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi114, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, org.label-schema.build-date=20240214) 2024-02-16T22:23:35.372 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:34 smithi114 podman[94434]: 2024-02-16 22:23:34.981793342 +0000 UTC m=+0.498549805 container start 58ac063725235a5943ffb3c6a4284229d105f7cbf6736c59d56b3decc54325ea (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi114, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-02-16T22:23:35.372 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: set uid:gid to 167:167 (ceph:ceph) 2024-02-16T22:23:35.372 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev), process ceph-mon, pid 2 2024-02-16T22:23:35.372 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: pidfile_write: ignore empty --pid-file 2024-02-16T22:23:35.372 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: load: jerasure load: lrc 2024-02-16T22:23:35.372 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: RocksDB version: 7.9.2 2024-02-16T22:23:35.372 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Git sha 0 2024-02-16T22:23:35.372 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Compile date 2024-02-15 18:05:23 2024-02-16T22:23:35.372 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: DB SUMMARY 2024-02-16T22:23:35.373 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: DB Session ID: JLZVS1BLW423XL8B0VKS 2024-02-16T22:23:35.373 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: CURRENT file: CURRENT 2024-02-16T22:23:35.373 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: IDENTITY file: IDENTITY 2024-02-16T22:23:35.373 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1510 Bytes 2024-02-16T22:23:35.373 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi114/store.db dir, Total Num: 1, files: 000032.sst 2024-02-16T22:23:35.373 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi114/store.db: 000030.log size: 2752211 ; 2024-02-16T22:23:35.373 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.error_if_exists: 0 2024-02-16T22:23:35.373 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.create_if_missing: 0 2024-02-16T22:23:35.373 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.paranoid_checks: 1 2024-02-16T22:23:35.373 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-16T22:23:35.373 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-16T22:23:35.373 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-16T22:23:35.373 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.env: 0x55eb1a70cae0 2024-02-16T22:23:35.373 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.fs: PosixFileSystem 2024-02-16T22:23:35.373 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.info_log: 0x55eb1bb77760 2024-02-16T22:23:35.374 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.max_file_opening_threads: 16 2024-02-16T22:23:35.374 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.statistics: (nil) 2024-02-16T22:23:35.375 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.use_fsync: 0 2024-02-16T22:23:35.375 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.max_log_file_size: 0 2024-02-16T22:23:35.375 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-16T22:23:35.375 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-16T22:23:35.375 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.keep_log_file_num: 1000 2024-02-16T22:23:35.375 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.recycle_log_file_num: 0 2024-02-16T22:23:35.375 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.allow_fallocate: 1 2024-02-16T22:23:35.375 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.allow_mmap_reads: 0 2024-02-16T22:23:35.375 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.allow_mmap_writes: 0 2024-02-16T22:23:35.375 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.use_direct_reads: 0 2024-02-16T22:23:35.375 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-16T22:23:35.375 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.create_missing_column_families: 0 2024-02-16T22:23:35.375 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.db_log_dir: 2024-02-16T22:23:35.375 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.wal_dir: 2024-02-16T22:23:35.375 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-16T22:23:35.376 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-16T22:23:35.376 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-16T22:23:35.376 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-16T22:23:35.376 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-16T22:23:35.376 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-16T22:23:35.376 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.advise_random_on_open: 1 2024-02-16T22:23:35.376 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.db_write_buffer_size: 0 2024-02-16T22:23:35.376 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.write_buffer_manager: 0x55eb1bb4b400 2024-02-16T22:23:35.376 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-16T22:23:35.376 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-16T22:23:35.376 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-16T22:23:35.376 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.rate_limiter: (nil) 2024-02-16T22:23:35.376 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-16T22:23:35.377 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.wal_recovery_mode: 2 2024-02-16T22:23:35.377 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.enable_thread_tracking: 0 2024-02-16T22:23:35.377 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.enable_pipelined_write: 0 2024-02-16T22:23:35.377 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.unordered_write: 0 2024-02-16T22:23:35.377 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-16T22:23:35.377 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-16T22:23:35.377 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-16T22:23:35.377 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-16T22:23:35.377 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.row_cache: None 2024-02-16T22:23:35.377 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.wal_filter: None 2024-02-16T22:23:35.377 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-16T22:23:35.377 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.allow_ingest_behind: 0 2024-02-16T22:23:35.377 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.two_write_queues: 0 2024-02-16T22:23:35.377 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.manual_wal_flush: 0 2024-02-16T22:23:35.377 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.wal_compression: 0 2024-02-16T22:23:35.377 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.atomic_flush: 0 2024-02-16T22:23:35.378 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-16T22:23:35.378 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-16T22:23:35.378 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-16T22:23:35.378 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.log_readahead_size: 0 2024-02-16T22:23:35.378 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-16T22:23:35.378 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.best_efforts_recovery: 0 2024-02-16T22:23:35.378 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-16T22:23:35.378 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-16T22:23:35.378 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.allow_data_in_errors: 0 2024-02-16T22:23:35.378 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.db_host_id: __hostname__ 2024-02-16T22:23:35.378 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-16T22:23:35.378 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.max_background_jobs: 2 2024-02-16T22:23:35.378 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.max_background_compactions: -1 2024-02-16T22:23:35.378 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.max_subcompactions: 1 2024-02-16T22:23:35.378 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-16T22:23:35.378 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-16T22:23:35.379 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-16T22:23:35.379 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.max_total_wal_size: 0 2024-02-16T22:23:35.379 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-16T22:23:35.379 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-16T22:23:35.379 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-16T22:23:35.379 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-16T22:23:35.379 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.max_open_files: -1 2024-02-16T22:23:35.379 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.bytes_per_sync: 0 2024-02-16T22:23:35.379 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-16T22:23:35.379 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-16T22:23:35.379 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.compaction_readahead_size: 0 2024-02-16T22:23:35.379 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.max_background_flushes: -1 2024-02-16T22:23:35.379 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Compression algorithms supported: 2024-02-16T22:23:35.379 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-16T22:23:35.379 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: kZSTD supported: 0 2024-02-16T22:23:35.379 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: kXpressCompression supported: 0 2024-02-16T22:23:35.380 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: kLZ4HCCompression supported: 1 2024-02-16T22:23:35.380 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: kZlibCompression supported: 1 2024-02-16T22:23:35.380 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: kSnappyCompression supported: 1 2024-02-16T22:23:35.380 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: kLZ4Compression supported: 1 2024-02-16T22:23:35.380 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: kBZip2Compression supported: 0 2024-02-16T22:23:35.380 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-16T22:23:35.380 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-16T22:23:35.380 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi114/store.db/MANIFEST-000015 2024-02-16T22:23:35.380 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-16T22:23:35.380 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-16T22:23:35.380 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.merge_operator: 2024-02-16T22:23:35.380 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.compaction_filter: None 2024-02-16T22:23:35.380 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.compaction_filter_factory: None 2024-02-16T22:23:35.381 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.sst_partitioner_factory: None 2024-02-16T22:23:35.381 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-16T22:23:35.381 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-16T22:23:35.381 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55eb1bb77880) 2024-02-16T22:23:35.381 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: cache_index_and_filter_blocks: 1 2024-02-16T22:23:35.381 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-16T22:23:35.381 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-16T22:23:35.381 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: pin_top_level_index_and_filter: 1 2024-02-16T22:23:35.381 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: index_type: 0 2024-02-16T22:23:35.381 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: data_block_index_type: 0 2024-02-16T22:23:35.381 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: index_shortening: 1 2024-02-16T22:23:35.381 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-16T22:23:35.381 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: checksum: 4 2024-02-16T22:23:35.381 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: no_block_cache: 0 2024-02-16T22:23:35.381 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: block_cache: 0x55eb1bb7b090 2024-02-16T22:23:35.381 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: block_cache_name: BinnedLRUCache 2024-02-16T22:23:35.382 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: block_cache_options: 2024-02-16T22:23:35.382 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: capacity : 536870912 2024-02-16T22:23:35.382 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: num_shard_bits : 4 2024-02-16T22:23:35.382 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: strict_capacity_limit : 0 2024-02-16T22:23:35.382 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: high_pri_pool_ratio: 0.000 2024-02-16T22:23:35.382 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: block_cache_compressed: (nil) 2024-02-16T22:23:35.382 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: persistent_cache: (nil) 2024-02-16T22:23:35.382 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: block_size: 4096 2024-02-16T22:23:35.382 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: block_size_deviation: 10 2024-02-16T22:23:35.382 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: block_restart_interval: 16 2024-02-16T22:23:35.382 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: index_block_restart_interval: 1 2024-02-16T22:23:35.382 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: metadata_block_size: 4096 2024-02-16T22:23:35.382 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: partition_filters: 0 2024-02-16T22:23:35.382 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: use_delta_encoding: 1 2024-02-16T22:23:35.383 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: filter_policy: bloomfilter 2024-02-16T22:23:35.383 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: whole_key_filtering: 1 2024-02-16T22:23:35.383 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: verify_compression: 0 2024-02-16T22:23:35.383 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: read_amp_bytes_per_bit: 0 2024-02-16T22:23:35.383 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: format_version: 5 2024-02-16T22:23:35.383 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: enable_index_compression: 1 2024-02-16T22:23:35.383 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: block_align: 0 2024-02-16T22:23:35.383 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: max_auto_readahead_size: 262144 2024-02-16T22:23:35.383 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: prepopulate_block_cache: 0 2024-02-16T22:23:35.383 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: initial_auto_readahead_size: 8192 2024-02-16T22:23:35.383 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: num_file_reads_for_auto_readahead: 2 2024-02-16T22:23:35.383 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.write_buffer_size: 33554432 2024-02-16T22:23:35.383 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.max_write_buffer_number: 2 2024-02-16T22:23:35.383 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.compression: NoCompression 2024-02-16T22:23:35.383 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.bottommost_compression: Disabled 2024-02-16T22:23:35.384 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.prefix_extractor: nullptr 2024-02-16T22:23:35.384 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-16T22:23:35.384 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.num_levels: 7 2024-02-16T22:23:35.384 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-16T22:23:35.384 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-16T22:23:35.384 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-16T22:23:35.384 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-16T22:23:35.384 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-16T22:23:35.384 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-16T22:23:35.384 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-16T22:23:35.384 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-16T22:23:35.384 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-16T22:23:35.384 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-16T22:23:35.384 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-16T22:23:35.385 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-16T22:23:35.385 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-16T22:23:35.385 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.compression_opts.level: 32767 2024-02-16T22:23:35.385 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.compression_opts.strategy: 0 2024-02-16T22:23:35.385 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-16T22:23:35.385 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-16T22:23:35.385 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-16T22:23:35.385 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-16T22:23:35.385 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.compression_opts.enabled: false 2024-02-16T22:23:35.385 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-16T22:23:35.385 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-16T22:23:35.385 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-16T22:23:35.385 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-16T22:23:35.385 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.target_file_size_base: 67108864 2024-02-16T22:23:35.385 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-16T22:23:35.386 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-16T22:23:35.386 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-16T22:23:35.386 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-16T22:23:35.386 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-16T22:23:35.386 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-16T22:23:35.386 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-16T22:23:35.386 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-16T22:23:35.386 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-16T22:23:35.386 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-16T22:23:35.386 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-16T22:23:35.386 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-16T22:23:35.386 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-16T22:23:35.386 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-16T22:23:35.386 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.arena_block_size: 1048576 2024-02-16T22:23:35.386 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-16T22:23:35.386 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-16T22:23:35.387 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.disable_auto_compactions: 0 2024-02-16T22:23:35.387 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-16T22:23:35.387 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-16T22:23:35.387 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-16T22:23:35.387 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-16T22:23:35.387 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-16T22:23:35.387 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-16T22:23:35.387 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-16T22:23:35.387 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-16T22:23:35.387 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-16T22:23:35.387 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-16T22:23:35.387 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.table_properties_collectors: 2024-02-16T22:23:35.387 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.inplace_update_support: 0 2024-02-16T22:23:35.387 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-16T22:23:35.387 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-16T22:23:35.387 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-16T22:23:35.388 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-16T22:23:35.388 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.bloom_locality: 0 2024-02-16T22:23:35.388 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.max_successive_merges: 0 2024-02-16T22:23:35.388 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-16T22:23:35.388 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.paranoid_file_checks: 0 2024-02-16T22:23:35.388 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.force_consistency_checks: 1 2024-02-16T22:23:35.388 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.report_bg_io_stats: 0 2024-02-16T22:23:35.388 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.ttl: 2592000 2024-02-16T22:23:35.388 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-16T22:23:35.388 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-16T22:23:35.388 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-16T22:23:35.389 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.enable_blob_files: false 2024-02-16T22:23:35.389 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.min_blob_size: 0 2024-02-16T22:23:35.389 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.blob_file_size: 268435456 2024-02-16T22:23:35.389 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-16T22:23:35.389 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-16T22:23:35.389 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-16T22:23:35.389 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-16T22:23:35.389 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-16T22:23:35.390 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.blob_file_starting_level: 0 2024-02-16T22:23:35.390 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-16T22:23:35.390 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-16T22:23:35.390 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi114/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 34, last_sequence is 12018, log_number is 30,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 30 2024-02-16T22:23:35.390 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 30 2024-02-16T22:23:35.390 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 09eac42e-19a1-425c-924d-a6b3942204ce 2024-02-16T22:23:35.390 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708122215014274, "job": 1, "event": "recovery_started", "wal_files": [30]} 2024-02-16T22:23:35.390 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #30 mode 2 2024-02-16T22:23:35.390 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708122215024107, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 35, "file_size": 2515190, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12019, "largest_seqno": 13349, "table_properties": {"data_size": 2508963, "index_size": 3800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1413, "raw_key_size": 12441, "raw_average_key_size": 22, "raw_value_size": 2497119, "raw_average_value_size": 4556, "num_data_blocks": 173, "num_entries": 548, "num_filter_entries": 548, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708122215, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "09eac42e-19a1-425c-924d-a6b3942204ce", "db_session_id": "JLZVS1BLW423XL8B0VKS", "orig_file_number": 35, "seqno_to_time_mapping": "N/A"}} 2024-02-16T22:23:35.390 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708122215024249, "job": 1, "event": "recovery_finished"} 2024-02-16T22:23:35.390 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: [db/version_set.cc:5047] Creating manifest 37 2024-02-16T22:23:35.390 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-16T22:23:35.390 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi114/store.db/000030.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-16T22:23:35.390 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55eb1bc6a000 2024-02-16T22:23:35.391 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: DB pointer 0x55eb1bc56000 2024-02-16T22:23:35.391 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-16T22:23:35.391 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-16T22:23:35.391 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: ** DB Stats ** 2024-02-16T22:23:35.391 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-16T22:23:35.391 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-16T22:23:35.391 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-16T22:23:35.391 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-16T22:23:35.391 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-16T22:23:35.391 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-16T22:23:35.391 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-16T22:23:35.391 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: 2024-02-16T22:23:35.391 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: ** Compaction Stats [default] ** 2024-02-16T22:23:35.391 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-16T22:23:35.391 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-16T22:23:35.392 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: L0 1/0 2.40 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 593.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-02-16T22:23:35.392 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: L6 1/0 4.74 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-02-16T22:23:35.392 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Sum 2/0 7.13 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 593.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-02-16T22:23:35.392 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 593.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-02-16T22:23:35.392 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: 2024-02-16T22:23:35.392 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: ** Compaction Stats [default] ** 2024-02-16T22:23:35.392 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-16T22:23:35.392 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-16T22:23:35.392 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 593.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-02-16T22:23:35.392 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: 2024-02-16T22:23:35.392 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-16T22:23:35.392 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: 2024-02-16T22:23:35.392 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-16T22:23:35.392 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-02-16T22:23:35.393 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-16T22:23:35.393 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-16T22:23:35.393 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-16T22:23:35.393 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-16T22:23:35.393 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Cumulative compaction: 0.00 GB write, 162.09 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-16T22:23:35.393 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Interval compaction: 0.00 GB write, 162.09 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-16T22:23:35.393 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-16T22:23:35.393 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Block cache BinnedLRUCache@0x55eb1bb7b090#2 capacity: 512.00 MB usage: 5.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-02-16T22:23:35.393 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.53 KB,0.000292063%) IndexBlock(1,4.08 KB,0.000777841%) Misc(1,0.00 KB,0%) 2024-02-16T22:23:35.393 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: 2024-02-16T22:23:35.393 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-16T22:23:35.393 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: starting mon.smithi114 rank 0 at public addrs [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] at bind addrs [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi114 fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 2024-02-16T22:23:35.393 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: mon.smithi114@-1(???) e2 preinit fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 2024-02-16T22:23:35.393 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: mon.smithi114@-1(???).mds e14 new map 2024-02-16T22:23:35.394 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: mon.smithi114@-1(???).mds e14 print_map 2024-02-16T22:23:35.394 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: e14 2024-02-16T22:23:35.394 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:23:35.394 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:23:35.394 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: legacy client fscid: 1 2024-02-16T22:23:35.394 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: 2024-02-16T22:23:35.394 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Filesystem 'cephfs' (1) 2024-02-16T22:23:35.394 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: fs_name cephfs 2024-02-16T22:23:35.394 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: epoch 12 2024-02-16T22:23:35.394 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:23:35.394 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: created 2024-02-16T22:18:02.108825+0000 2024-02-16T22:23:35.394 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: modified 2024-02-16T22:18:16.413371+0000 2024-02-16T22:23:35.394 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: tableserver 0 2024-02-16T22:23:35.394 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: root 0 2024-02-16T22:23:35.394 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: session_timeout 60 2024-02-16T22:23:35.394 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: session_autoclose 300 2024-02-16T22:23:35.394 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: max_file_size 1099511627776 2024-02-16T22:23:35.395 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: max_xattr_size 65536 2024-02-16T22:23:35.395 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: required_client_features {} 2024-02-16T22:23:35.395 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: last_failure 0 2024-02-16T22:23:35.395 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: last_failure_osd_epoch 0 2024-02-16T22:23:35.395 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:23:35.395 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: max_mds 1 2024-02-16T22:23:35.395 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: in 0 2024-02-16T22:23:35.395 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: up {0=14468} 2024-02-16T22:23:35.395 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: failed 2024-02-16T22:23:35.395 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: damaged 2024-02-16T22:23:35.395 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: stopped 2024-02-16T22:23:35.395 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: data_pools [3] 2024-02-16T22:23:35.395 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: metadata_pool 2 2024-02-16T22:23:35.395 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: inline_data enabled 2024-02-16T22:23:35.395 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: balancer 2024-02-16T22:23:35.395 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: bal_rank_mask -1 2024-02-16T22:23:35.395 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: standby_count_wanted 1 2024-02-16T22:23:35.396 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: [mds.cephfs.smithi114.zzryke{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.114:6826/277953535,v1:172.21.15.114:6827/277953535] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:23:35.396 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: 2024-02-16T22:23:35.396 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: 2024-02-16T22:23:35.396 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: Standby daemons: 2024-02-16T22:23:35.396 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: 2024-02-16T22:23:35.396 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: [mds.cephfs.smithi114.fdxpey{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:23:35.396 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: [mds.cephfs.smithi123.vhlqbe{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:23:35.396 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout: [mds.cephfs.smithi123.gopcfa{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:23:35.396 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: mon.smithi114@-1(???).osd e47 crush map has features 3314933000852226048, adjusting msgr requires 2024-02-16T22:23:35.396 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: mon.smithi114@-1(???).osd e47 crush map has features 288514051259236352, adjusting msgr requires 2024-02-16T22:23:35.396 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: mon.smithi114@-1(???).osd e47 crush map has features 288514051259236352, adjusting msgr requires 2024-02-16T22:23:35.396 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: mon.smithi114@-1(???).osd e47 crush map has features 288514051259236352, adjusting msgr requires 2024-02-16T22:23:35.396 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 ceph-mon[94452]: mon.smithi114@-1(???).paxosservice(auth 1..23) refresh upgraded, format 0 -> 3 2024-02-16T22:23:35.396 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 bash[94434]: 58ac063725235a5943ffb3c6a4284229d105f7cbf6736c59d56b3decc54325ea 2024-02-16T22:23:35.396 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:35 smithi114 systemd[1]: Started Ceph mon.smithi114 for 1dcbaabe-cd18-11ee-95ba-87774f69a715. 2024-02-16T22:23:36.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:36 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi114"}]: dispatch 2024-02-16T22:23:36.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:36 smithi123 ceph-mon[37161]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi123"}]: dispatch 2024-02-16T22:23:36.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:36 smithi123 ceph-mon[37161]: pgmap v34: 65 pgs: 65 active+clean; 301 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 2.0 MiB/s wr, 308 op/s 2024-02-16T22:23:36.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:36 smithi123 ceph-mon[37161]: mon.smithi114 calling monitor election 2024-02-16T22:23:36.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:36 smithi123 ceph-mon[37161]: mon.smithi114 is new leader, mons smithi114,smithi123 in quorum (ranks 0,1) 2024-02-16T22:23:36.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:36 smithi123 ceph-mon[37161]: monmap e2: 2 mons at {smithi114=[v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0],smithi123=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-16T22:23:36.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:36 smithi123 ceph-mon[37161]: fsmap cephfs:1 {0=cephfs.smithi114.zzryke=up:active} 3 up:standby 2024-02-16T22:23:36.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:36 smithi123 ceph-mon[37161]: osdmap e47: 6 total, 6 up, 6 in 2024-02-16T22:23:36.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:36 smithi123 ceph-mon[37161]: mgrmap e31: smithi114.vcullz(active, since 64s), standbys: smithi123.inzgjd 2024-02-16T22:23:36.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:36 smithi123 ceph-mon[37161]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:23:36.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:36 smithi123 ceph-mon[37161]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:23:36.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:36 smithi123 ceph-mon[37161]: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:23:36.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:36 smithi123 ceph-mon[37161]: from='mgr.14764 ' entity='' 2024-02-16T22:23:36.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:36 smithi123 ceph-mon[37161]: mgrmap e32: smithi114.vcullz(active, since 64s), standbys: smithi123.inzgjd 2024-02-16T22:23:36.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:36 smithi114 ceph-mon[94452]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi114"}]: dispatch 2024-02-16T22:23:36.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:36 smithi114 ceph-mon[94452]: from='mgr.14764 172.21.15.114:0/2955245034' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi123"}]: dispatch 2024-02-16T22:23:36.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:36 smithi114 ceph-mon[94452]: pgmap v34: 65 pgs: 65 active+clean; 301 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 2.0 MiB/s wr, 308 op/s 2024-02-16T22:23:36.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:36 smithi114 ceph-mon[94452]: mon.smithi114 calling monitor election 2024-02-16T22:23:36.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:36 smithi114 ceph-mon[94452]: mon.smithi114 is new leader, mons smithi114,smithi123 in quorum (ranks 0,1) 2024-02-16T22:23:36.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:36 smithi114 ceph-mon[94452]: monmap e2: 2 mons at {smithi114=[v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0],smithi123=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-16T22:23:36.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:36 smithi114 ceph-mon[94452]: fsmap cephfs:1 {0=cephfs.smithi114.zzryke=up:active} 3 up:standby 2024-02-16T22:23:36.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:36 smithi114 ceph-mon[94452]: osdmap e47: 6 total, 6 up, 6 in 2024-02-16T22:23:36.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:36 smithi114 ceph-mon[94452]: mgrmap e31: smithi114.vcullz(active, since 64s), standbys: smithi123.inzgjd 2024-02-16T22:23:36.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:36 smithi114 ceph-mon[94452]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:23:36.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:36 smithi114 ceph-mon[94452]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:23:36.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:36 smithi114 ceph-mon[94452]: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:23:36.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:36 smithi114 ceph-mon[94452]: from='mgr.14764 ' entity='' 2024-02-16T22:23:36.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:36 smithi114 ceph-mon[94452]: mgrmap e32: smithi114.vcullz(active, since 64s), standbys: smithi123.inzgjd 2024-02-16T22:23:44.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:44 smithi123 ceph-mon[37161]: Standby manager daemon smithi123.inzgjd restarted 2024-02-16T22:23:44.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:44 smithi123 ceph-mon[37161]: Standby manager daemon smithi123.inzgjd started 2024-02-16T22:23:44.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:44 smithi123 ceph-mon[37161]: from='mgr.? 172.21.15.123:0/1898327462' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi123.inzgjd/crt"}]: dispatch 2024-02-16T22:23:44.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:44 smithi123 ceph-mon[37161]: from='mgr.? 172.21.15.123:0/1898327462' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T22:23:44.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:44 smithi123 ceph-mon[37161]: from='mgr.? 172.21.15.123:0/1898327462' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi123.inzgjd/key"}]: dispatch 2024-02-16T22:23:44.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:44 smithi123 ceph-mon[37161]: from='mgr.? 172.21.15.123:0/1898327462' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T22:23:44.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:44 smithi114 ceph-mon[94452]: Standby manager daemon smithi123.inzgjd restarted 2024-02-16T22:23:44.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:44 smithi114 ceph-mon[94452]: Standby manager daemon smithi123.inzgjd started 2024-02-16T22:23:44.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:44 smithi114 ceph-mon[94452]: from='mgr.? 172.21.15.123:0/1898327462' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi123.inzgjd/crt"}]: dispatch 2024-02-16T22:23:44.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:44 smithi114 ceph-mon[94452]: from='mgr.? 172.21.15.123:0/1898327462' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T22:23:44.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:44 smithi114 ceph-mon[94452]: from='mgr.? 172.21.15.123:0/1898327462' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi123.inzgjd/key"}]: dispatch 2024-02-16T22:23:44.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:44 smithi114 ceph-mon[94452]: from='mgr.? 172.21.15.123:0/1898327462' entity='mgr.smithi123.inzgjd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T22:23:45.119 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-02-16T22:23:45.119 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-02-16T22:23:45.326 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:45 smithi114 ceph-mon[94452]: mgrmap e33: smithi114.vcullz(active, since 72s), standbys: smithi123.inzgjd 2024-02-16T22:23:45.460 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:45 smithi123 ceph-mon[37161]: mgrmap e33: smithi114.vcullz(active, since 72s), standbys: smithi123.inzgjd 2024-02-16T22:23:45.461 DEBUG:teuthology.parallel:result is None 2024-02-16T22:23:46.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:46 smithi114 ceph-mon[94452]: Active manager daemon smithi114.vcullz restarted 2024-02-16T22:23:46.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:46 smithi114 ceph-mon[94452]: Activating manager daemon smithi114.vcullz 2024-02-16T22:23:46.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:46 smithi114 ceph-mon[94452]: osdmap e48: 6 total, 6 up, 6 in 2024-02-16T22:23:46.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:46 smithi114 ceph-mon[94452]: mgrmap e34: smithi114.vcullz(active, starting, since 0.00696737s), standbys: smithi123.inzgjd 2024-02-16T22:23:46.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi114"}]: dispatch 2024-02-16T22:23:46.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi123"}]: dispatch 2024-02-16T22:23:46.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi114.zzryke"}]: dispatch 2024-02-16T22:23:46.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi114.fdxpey"}]: dispatch 2024-02-16T22:23:46.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi123.vhlqbe"}]: dispatch 2024-02-16T22:23:46.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi123.gopcfa"}]: dispatch 2024-02-16T22:23:46.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mgr metadata", "who": "smithi114.vcullz", "id": "smithi114.vcullz"}]: dispatch 2024-02-16T22:23:46.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mgr metadata", "who": "smithi123.inzgjd", "id": "smithi123.inzgjd"}]: dispatch 2024-02-16T22:23:46.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:23:46.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:23:46.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:23:46.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:23:46.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:23:46.204 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:23:46.204 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T22:23:46.204 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T22:23:46.204 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T22:23:46.204 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:46 smithi114 ceph-mon[94452]: Manager daemon smithi114.vcullz is now available 2024-02-16T22:23:46.204 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:23:46.204 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:23:46.204 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:23:46.204 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi114.vcullz/mirror_snapshot_schedule"}]: dispatch 2024-02-16T22:23:46.204 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi114.vcullz/trash_purge_schedule"}]: dispatch 2024-02-16T22:23:46.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:46 smithi123 ceph-mon[37161]: Active manager daemon smithi114.vcullz restarted 2024-02-16T22:23:46.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:46 smithi123 ceph-mon[37161]: Activating manager daemon smithi114.vcullz 2024-02-16T22:23:46.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:46 smithi123 ceph-mon[37161]: osdmap e48: 6 total, 6 up, 6 in 2024-02-16T22:23:46.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:46 smithi123 ceph-mon[37161]: mgrmap e34: smithi114.vcullz(active, starting, since 0.00696737s), standbys: smithi123.inzgjd 2024-02-16T22:23:46.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:46 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi114"}]: dispatch 2024-02-16T22:23:46.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:46 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi123"}]: dispatch 2024-02-16T22:23:46.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:46 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi114.zzryke"}]: dispatch 2024-02-16T22:23:46.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:46 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi114.fdxpey"}]: dispatch 2024-02-16T22:23:46.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:46 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi123.vhlqbe"}]: dispatch 2024-02-16T22:23:46.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:46 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi123.gopcfa"}]: dispatch 2024-02-16T22:23:46.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:46 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mgr metadata", "who": "smithi114.vcullz", "id": "smithi114.vcullz"}]: dispatch 2024-02-16T22:23:46.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:46 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mgr metadata", "who": "smithi123.inzgjd", "id": "smithi123.inzgjd"}]: dispatch 2024-02-16T22:23:46.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:46 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:23:46.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:46 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:23:46.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:46 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:23:46.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:46 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:23:46.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:46 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:23:46.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:46 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:23:46.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:46 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T22:23:46.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:46 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T22:23:46.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:46 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T22:23:46.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:46 smithi123 ceph-mon[37161]: Manager daemon smithi114.vcullz is now available 2024-02-16T22:23:46.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:46 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:23:46.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:46 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:23:46.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:46 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:23:46.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:46 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi114.vcullz/mirror_snapshot_schedule"}]: dispatch 2024-02-16T22:23:46.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:46 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi114.vcullz/trash_purge_schedule"}]: dispatch 2024-02-16T22:23:48.021 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:47 smithi123 ceph-mon[37161]: [16/Feb/2024:22:23:46] ENGINE Bus STARTING 2024-02-16T22:23:48.021 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:47 smithi123 ceph-mon[37161]: mgrmap e35: smithi114.vcullz(active, since 1.01069s), standbys: smithi123.inzgjd 2024-02-16T22:23:48.021 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:47 smithi123 ceph-mon[37161]: pgmap v3: 65 pgs: 65 active+clean; 267 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:23:48.021 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:47 smithi123 ceph-mon[37161]: [16/Feb/2024:22:23:46] ENGINE Serving on http://172.21.15.114:8765 2024-02-16T22:23:48.021 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:47 smithi123 ceph-mon[37161]: [16/Feb/2024:22:23:46] ENGINE Serving on https://172.21.15.114:7150 2024-02-16T22:23:48.021 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:47 smithi123 ceph-mon[37161]: [16/Feb/2024:22:23:46] ENGINE Bus STARTED 2024-02-16T22:23:48.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:47 smithi114 ceph-mon[94452]: [16/Feb/2024:22:23:46] ENGINE Bus STARTING 2024-02-16T22:23:48.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:47 smithi114 ceph-mon[94452]: mgrmap e35: smithi114.vcullz(active, since 1.01069s), standbys: smithi123.inzgjd 2024-02-16T22:23:48.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:47 smithi114 ceph-mon[94452]: pgmap v3: 65 pgs: 65 active+clean; 267 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:23:48.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:47 smithi114 ceph-mon[94452]: [16/Feb/2024:22:23:46] ENGINE Serving on http://172.21.15.114:8765 2024-02-16T22:23:48.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:47 smithi114 ceph-mon[94452]: [16/Feb/2024:22:23:46] ENGINE Serving on https://172.21.15.114:7150 2024-02-16T22:23:48.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:47 smithi114 ceph-mon[94452]: [16/Feb/2024:22:23:46] ENGINE Bus STARTED 2024-02-16T22:23:48.312 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-02-16T22:23:48.312 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-16T22:23:48.637 DEBUG:teuthology.parallel:result is None 2024-02-16T22:23:48.638 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-02-16T22:23:48.673 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-02-16T22:23:48.673 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-02-16T22:23:48.710 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-02-16T22:23:48.710 DEBUG:teuthology.parallel:result is None 2024-02-16T22:23:48.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:48 smithi114 ceph-mon[94452]: pgmap v4: 65 pgs: 65 active+clean; 267 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:23:49.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:48 smithi123 ceph-mon[37161]: pgmap v4: 65 pgs: 65 active+clean; 267 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:23:50.031 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:49 smithi114 ceph-mon[94452]: mgrmap e36: smithi114.vcullz(active, since 3s), standbys: smithi123.inzgjd 2024-02-16T22:23:50.031 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:49 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:50.031 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:49 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:50.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:49 smithi123 ceph-mon[37161]: mgrmap e36: smithi114.vcullz(active, since 3s), standbys: smithi123.inzgjd 2024-02-16T22:23:50.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:49 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:50.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:49 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:51.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:51 smithi123 ceph-mon[37161]: pgmap v5: 65 pgs: 65 active+clean; 267 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:23:51.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:51 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:51.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:51 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:51.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:51 smithi123 ceph-mon[37161]: mgrmap e37: smithi114.vcullz(active, since 5s), standbys: smithi123.inzgjd 2024-02-16T22:23:51.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:51 smithi114 ceph-mon[94452]: pgmap v5: 65 pgs: 65 active+clean; 267 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:23:51.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:51 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:51.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:51 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:51.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:51 smithi114 ceph-mon[94452]: mgrmap e37: smithi114.vcullz(active, since 5s), standbys: smithi123.inzgjd 2024-02-16T22:23:53.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:53 smithi123 ceph-mon[37161]: pgmap v6: 65 pgs: 65 active+clean; 267 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:23:53.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:53 smithi114 ceph-mon[94452]: pgmap v6: 65 pgs: 65 active+clean; 267 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:23:55.277 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:55 smithi114 ceph-mon[94452]: pgmap v7: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 47 KiB/s rd, 373 KiB/s wr, 100 op/s 2024-02-16T22:23:55.277 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:55 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:55.277 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:55 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:55.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:55 smithi123 ceph-mon[37161]: pgmap v7: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 47 KiB/s rd, 373 KiB/s wr, 100 op/s 2024-02-16T22:23:55.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:55 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:55.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:55 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:23:56.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.045+0000 7fdabc558700 1 -- 172.21.15.114:0/822639861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdab4103fb0 msgr2=0x7fdab41063a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:56.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.045+0000 7fdabc558700 1 --2- 172.21.15.114:0/822639861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdab4103fb0 0x7fdab41063a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4009910 tx=0x7fdaa4009c20 comp rx=0 tx=0).stop 2024-02-16T22:23:56.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.046+0000 7fdabc558700 1 -- 172.21.15.114:0/822639861 shutdown_connections 2024-02-16T22:23:56.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.046+0000 7fdabc558700 1 --2- 172.21.15.114:0/822639861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdab4103fb0 0x7fdab41063a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:56.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.046+0000 7fdabc558700 1 --2- 172.21.15.114:0/822639861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdab4101680 0x7fdab4103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:56.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.046+0000 7fdabc558700 1 -- 172.21.15.114:0/822639861 >> 172.21.15.114:0/822639861 conn(0x7fdab40fb070 msgr2=0x7fdab40fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:23:56.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.046+0000 7fdabc558700 1 -- 172.21.15.114:0/822639861 shutdown_connections 2024-02-16T22:23:56.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.046+0000 7fdabc558700 1 -- 172.21.15.114:0/822639861 wait complete. 2024-02-16T22:23:56.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.046+0000 7fdabc558700 1 Processor -- start 2024-02-16T22:23:56.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.046+0000 7fdabc558700 1 -- start start 2024-02-16T22:23:56.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.046+0000 7fdabc558700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdab4101680 0x7fdab41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:56.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.046+0000 7fdabc558700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdab4103fb0 0x7fdab4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:56.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.046+0000 7fdabc558700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab4199110 con 0x7fdab4101680 2024-02-16T22:23:56.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.046+0000 7fdabc558700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab4199250 con 0x7fdab4103fb0 2024-02-16T22:23:56.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.047+0000 7fdaba2f4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdab4101680 0x7fdab41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:56.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.047+0000 7fdaba2f4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdab4101680 0x7fdab41985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33838/0 (socket says 172.21.15.114:33838) 2024-02-16T22:23:56.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.047+0000 7fdaba2f4700 1 -- 172.21.15.114:0/1001295821 learned_addr learned my addr 172.21.15.114:0/1001295821 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:23:56.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.047+0000 7fdab9af3700 1 --2- 172.21.15.114:0/1001295821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdab4103fb0 0x7fdab4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:56.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.047+0000 7fdaba2f4700 1 -- 172.21.15.114:0/1001295821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdab4103fb0 msgr2=0x7fdab4198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:56.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.047+0000 7fdaba2f4700 1 --2- 172.21.15.114:0/1001295821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdab4103fb0 0x7fdab4198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:56.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.047+0000 7fdaba2f4700 1 -- 172.21.15.114:0/1001295821 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa40095f0 con 0x7fdab4101680 2024-02-16T22:23:56.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.047+0000 7fdaba2f4700 1 --2- 172.21.15.114:0/1001295821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdab4101680 0x7fdab41985b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdab000b4b0 tx=0x7fdab000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:23:56.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.047+0000 7fdaab7fe700 1 -- 172.21.15.114:0/1001295821 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdab0011800 con 0x7fdab4101680 2024-02-16T22:23:56.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.047+0000 7fdaab7fe700 1 -- 172.21.15.114:0/1001295821 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdab0011e10 con 0x7fdab4101680 2024-02-16T22:23:56.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.047+0000 7fdabc558700 1 -- 172.21.15.114:0/1001295821 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdab419dd00 con 0x7fdab4101680 2024-02-16T22:23:56.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.047+0000 7fdaab7fe700 1 -- 172.21.15.114:0/1001295821 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdab000f2e0 con 0x7fdab4101680 2024-02-16T22:23:56.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.047+0000 7fdabc558700 1 -- 172.21.15.114:0/1001295821 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab419e270 con 0x7fdab4101680 2024-02-16T22:23:56.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.048+0000 7fdabc558700 1 -- 172.21.15.114:0/1001295821 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdab40fcc60 con 0x7fdab4101680 2024-02-16T22:23:56.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.049+0000 7fdaab7fe700 1 -- 172.21.15.114:0/1001295821 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 37) v1 ==== 91597+0+0 (secure 0 0 0) 0x7fdab0010380 con 0x7fdab4101680 2024-02-16T22:23:56.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.049+0000 7fdaab7fe700 1 --2- 172.21.15.114:0/1001295821 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fdaa006d9e0 0x7fdaa006fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:56.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.049+0000 7fdab9af3700 1 --2- 172.21.15.114:0/1001295821 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fdaa006d9e0 0x7fdaa006fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:56.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.049+0000 7fdaab7fe700 1 -- 172.21.15.114:0/1001295821 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fdab008d250 con 0x7fdab4101680 2024-02-16T22:23:56.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.050+0000 7fdab9af3700 1 --2- 172.21.15.114:0/1001295821 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fdaa006d9e0 0x7fdaa006fea0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdaa40059d0 tx=0x7fdaa4005920 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:23:56.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.054+0000 7fdaab7fe700 1 -- 172.21.15.114:0/1001295821 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdab005a4d0 con 0x7fdab4101680 2024-02-16T22:23:56.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.277+0000 7fdabc558700 1 -- 172.21.15.114:0/1001295821 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdab40611d0 con 0x7fdaa006d9e0 2024-02-16T22:23:56.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.279+0000 7fdaab7fe700 1 -- 172.21.15.114:0/1001295821 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fdab40611d0 con 0x7fdaa006d9e0 2024-02-16T22:23:56.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.281+0000 7fdabc558700 1 -- 172.21.15.114:0/1001295821 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fdaa006d9e0 msgr2=0x7fdaa006fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:56.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.281+0000 7fdabc558700 1 --2- 172.21.15.114:0/1001295821 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fdaa006d9e0 0x7fdaa006fea0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdaa40059d0 tx=0x7fdaa4005920 comp rx=0 tx=0).stop 2024-02-16T22:23:56.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.281+0000 7fdabc558700 1 -- 172.21.15.114:0/1001295821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdab4101680 msgr2=0x7fdab41985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:56.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.281+0000 7fdabc558700 1 --2- 172.21.15.114:0/1001295821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdab4101680 0x7fdab41985b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdab000b4b0 tx=0x7fdab000b870 comp rx=0 tx=0).stop 2024-02-16T22:23:56.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.281+0000 7fdabc558700 1 -- 172.21.15.114:0/1001295821 shutdown_connections 2024-02-16T22:23:56.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.281+0000 7fdabc558700 1 --2- 172.21.15.114:0/1001295821 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fdaa006d9e0 0x7fdaa006fea0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:56.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.281+0000 7fdabc558700 1 --2- 172.21.15.114:0/1001295821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdab4101680 0x7fdab41985b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:56.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.281+0000 7fdabc558700 1 --2- 172.21.15.114:0/1001295821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdab4103fb0 0x7fdab4198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:56.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.281+0000 7fdabc558700 1 -- 172.21.15.114:0/1001295821 >> 172.21.15.114:0/1001295821 conn(0x7fdab40fb070 msgr2=0x7fdab4104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:23:56.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.281+0000 7fdabc558700 1 -- 172.21.15.114:0/1001295821 shutdown_connections 2024-02-16T22:23:56.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.281+0000 7fdabc558700 1 -- 172.21.15.114:0/1001295821 wait complete. 2024-02-16T22:23:56.292 INFO:teuthology.orchestra.run.smithi114.stdout:true 2024-02-16T22:23:56.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.387+0000 7f6bb1a6b700 1 -- 172.21.15.114:0/3850566945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bac104010 msgr2=0x7f6bac106400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:56.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.387+0000 7f6bb1a6b700 1 --2- 172.21.15.114:0/3850566945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bac104010 0x7f6bac106400 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009910 tx=0x7f6ba0009c20 comp rx=0 tx=0).stop 2024-02-16T22:23:56.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.387+0000 7f6bb1a6b700 1 -- 172.21.15.114:0/3850566945 shutdown_connections 2024-02-16T22:23:56.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.387+0000 7f6bb1a6b700 1 --2- 172.21.15.114:0/3850566945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bac104010 0x7f6bac106400 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:56.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.387+0000 7f6bb1a6b700 1 --2- 172.21.15.114:0/3850566945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bac1016e0 0x7f6bac103ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:56.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.387+0000 7f6bb1a6b700 1 -- 172.21.15.114:0/3850566945 >> 172.21.15.114:0/3850566945 conn(0x7f6bac0fb030 msgr2=0x7f6bac0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:23:56.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.388+0000 7f6bb1a6b700 1 -- 172.21.15.114:0/3850566945 shutdown_connections 2024-02-16T22:23:56.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.388+0000 7f6bb1a6b700 1 -- 172.21.15.114:0/3850566945 wait complete. 2024-02-16T22:23:56.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.388+0000 7f6bb1a6b700 1 Processor -- start 2024-02-16T22:23:56.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.388+0000 7f6bb1a6b700 1 -- start start 2024-02-16T22:23:56.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.388+0000 7f6bb1a6b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bac1016e0 0x7f6bac1963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:56.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.388+0000 7f6bb1a6b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bac104010 0x7f6bac1968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:56.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.388+0000 7f6bb1a6b700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac196f10 con 0x7f6bac1016e0 2024-02-16T22:23:56.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.388+0000 7f6bb1a6b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac197050 con 0x7f6bac104010 2024-02-16T22:23:56.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.389+0000 7f6bab7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bac1016e0 0x7f6bac1963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:56.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.389+0000 7f6bab7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bac1016e0 0x7f6bac1963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33850/0 (socket says 172.21.15.114:33850) 2024-02-16T22:23:56.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.389+0000 7f6bab7fe700 1 -- 172.21.15.114:0/1286469727 learned_addr learned my addr 172.21.15.114:0/1286469727 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:23:56.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.389+0000 7f6baaffd700 1 --2- 172.21.15.114:0/1286469727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bac104010 0x7f6bac1968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:56.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.389+0000 7f6bab7fe700 1 -- 172.21.15.114:0/1286469727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bac104010 msgr2=0x7f6bac1968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:56.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.389+0000 7f6bab7fe700 1 --2- 172.21.15.114:0/1286469727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bac104010 0x7f6bac1968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:56.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.389+0000 7f6bab7fe700 1 -- 172.21.15.114:0/1286469727 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba00095f0 con 0x7f6bac1016e0 2024-02-16T22:23:56.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.389+0000 7f6bab7fe700 1 --2- 172.21.15.114:0/1286469727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bac1016e0 0x7f6bac1963b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00b4b0 tx=0x7f6b9c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:23:56.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.389+0000 7f6ba8ff9700 1 -- 172.21.15.114:0/1286469727 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b9c011800 con 0x7f6bac1016e0 2024-02-16T22:23:56.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.389+0000 7f6ba8ff9700 1 -- 172.21.15.114:0/1286469727 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6b9c011e10 con 0x7f6bac1016e0 2024-02-16T22:23:56.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.389+0000 7f6bb1a6b700 1 -- 172.21.15.114:0/1286469727 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bac19bb00 con 0x7f6bac1016e0 2024-02-16T22:23:56.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.389+0000 7f6ba8ff9700 1 -- 172.21.15.114:0/1286469727 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b9c00f2e0 con 0x7f6bac1016e0 2024-02-16T22:23:56.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.389+0000 7f6bb1a6b700 1 -- 172.21.15.114:0/1286469727 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bac19c050 con 0x7f6bac1016e0 2024-02-16T22:23:56.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.390+0000 7f6bb1a6b700 1 -- 172.21.15.114:0/1286469727 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bac1905b0 con 0x7f6bac1016e0 2024-02-16T22:23:56.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.391+0000 7f6ba8ff9700 1 -- 172.21.15.114:0/1286469727 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 37) v1 ==== 91597+0+0 (secure 0 0 0) 0x7f6b9c011960 con 0x7f6bac1016e0 2024-02-16T22:23:56.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.391+0000 7f6ba8ff9700 1 --2- 172.21.15.114:0/1286469727 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6b9406d9e0 0x7f6b9406fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:56.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.391+0000 7f6ba8ff9700 1 -- 172.21.15.114:0/1286469727 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f6b9c08d230 con 0x7f6bac1016e0 2024-02-16T22:23:56.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.391+0000 7f6baaffd700 1 --2- 172.21.15.114:0/1286469727 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6b9406d9e0 0x7f6b9406fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:56.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.391+0000 7f6baaffd700 1 --2- 172.21.15.114:0/1286469727 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6b9406d9e0 0x7f6b9406fea0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0006010 tx=0x7f6ba000b580 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:23:56.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.396+0000 7f6ba8ff9700 1 -- 172.21.15.114:0/1286469727 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6b9c05a4b0 con 0x7f6bac1016e0 2024-02-16T22:23:56.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.608+0000 7f6bb1a6b700 1 -- 172.21.15.114:0/1286469727 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6bac0611d0 con 0x7f6b9406d9e0 2024-02-16T22:23:56.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.609+0000 7f6ba8ff9700 1 -- 172.21.15.114:0/1286469727 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f6bac0611d0 con 0x7f6b9406d9e0 2024-02-16T22:23:56.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.611+0000 7f6bb1a6b700 1 -- 172.21.15.114:0/1286469727 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6b9406d9e0 msgr2=0x7f6b9406fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:56.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.611+0000 7f6bb1a6b700 1 --2- 172.21.15.114:0/1286469727 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6b9406d9e0 0x7f6b9406fea0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0006010 tx=0x7f6ba000b580 comp rx=0 tx=0).stop 2024-02-16T22:23:56.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.611+0000 7f6bb1a6b700 1 -- 172.21.15.114:0/1286469727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bac1016e0 msgr2=0x7f6bac1963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:56.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.611+0000 7f6bb1a6b700 1 --2- 172.21.15.114:0/1286469727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bac1016e0 0x7f6bac1963b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00b4b0 tx=0x7f6b9c00b870 comp rx=0 tx=0).stop 2024-02-16T22:23:56.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.611+0000 7f6bb1a6b700 1 -- 172.21.15.114:0/1286469727 shutdown_connections 2024-02-16T22:23:56.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.611+0000 7f6bb1a6b700 1 --2- 172.21.15.114:0/1286469727 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6b9406d9e0 0x7f6b9406fea0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:56.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.611+0000 7f6bb1a6b700 1 --2- 172.21.15.114:0/1286469727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bac1016e0 0x7f6bac1963b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:56.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.611+0000 7f6bb1a6b700 1 --2- 172.21.15.114:0/1286469727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bac104010 0x7f6bac1968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:56.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.611+0000 7f6bb1a6b700 1 -- 172.21.15.114:0/1286469727 >> 172.21.15.114:0/1286469727 conn(0x7f6bac0fb030 msgr2=0x7f6bac0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:23:56.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.612+0000 7f6bb1a6b700 1 -- 172.21.15.114:0/1286469727 shutdown_connections 2024-02-16T22:23:56.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.612+0000 7f6bb1a6b700 1 -- 172.21.15.114:0/1286469727 wait complete. 2024-02-16T22:23:56.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.725+0000 7f64ebc99700 1 -- 172.21.15.114:0/3226564784 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64e4100c10 msgr2=0x7f64e4101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:56.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.725+0000 7f64ebc99700 1 --2- 172.21.15.114:0/3226564784 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64e4100c10 0x7f64e4101030 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009910 tx=0x7f64e0009c20 comp rx=0 tx=0).stop 2024-02-16T22:23:56.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.725+0000 7f64ebc99700 1 -- 172.21.15.114:0/3226564784 shutdown_connections 2024-02-16T22:23:56.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.725+0000 7f64ebc99700 1 --2- 172.21.15.114:0/3226564784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64e4101e10 0x7f64e4102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:56.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.725+0000 7f64ebc99700 1 --2- 172.21.15.114:0/3226564784 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64e4100c10 0x7f64e4101030 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:56.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.725+0000 7f64ebc99700 1 -- 172.21.15.114:0/3226564784 >> 172.21.15.114:0/3226564784 conn(0x7f64e40fc1b0 msgr2=0x7f64e40fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:23:56.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.725+0000 7f64ebc99700 1 -- 172.21.15.114:0/3226564784 shutdown_connections 2024-02-16T22:23:56.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.725+0000 7f64ebc99700 1 -- 172.21.15.114:0/3226564784 wait complete. 2024-02-16T22:23:56.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.726+0000 7f64ebc99700 1 Processor -- start 2024-02-16T22:23:56.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.726+0000 7f64ebc99700 1 -- start start 2024-02-16T22:23:56.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.726+0000 7f64ebc99700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64e4100c10 0x7f64e4196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:56.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.726+0000 7f64ebc99700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64e4101e10 0x7f64e41968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:56.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.726+0000 7f64ebc99700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e4196ef0 con 0x7f64e4100c10 2024-02-16T22:23:56.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.726+0000 7f64ebc99700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e4197030 con 0x7f64e4101e10 2024-02-16T22:23:56.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.726+0000 7f64e9a35700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64e4100c10 0x7f64e4196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:56.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.726+0000 7f64e9a35700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64e4100c10 0x7f64e4196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33862/0 (socket says 172.21.15.114:33862) 2024-02-16T22:23:56.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.726+0000 7f64e9a35700 1 -- 172.21.15.114:0/1377427307 learned_addr learned my addr 172.21.15.114:0/1377427307 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:23:56.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.726+0000 7f64e9234700 1 --2- 172.21.15.114:0/1377427307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64e4101e10 0x7f64e41968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:56.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.727+0000 7f64e9a35700 1 -- 172.21.15.114:0/1377427307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64e4101e10 msgr2=0x7f64e41968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:56.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.727+0000 7f64e9a35700 1 --2- 172.21.15.114:0/1377427307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64e4101e10 0x7f64e41968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:56.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.727+0000 7f64e9a35700 1 -- 172.21.15.114:0/1377427307 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e00095f0 con 0x7f64e4100c10 2024-02-16T22:23:56.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.727+0000 7f64e9a35700 1 --2- 172.21.15.114:0/1377427307 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64e4100c10 0x7f64e4196390 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f64e000bda0 tx=0x7f64e000bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:23:56.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.727+0000 7f64daffd700 1 -- 172.21.15.114:0/1377427307 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64e001d070 con 0x7f64e4100c10 2024-02-16T22:23:56.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.727+0000 7f64daffd700 1 -- 172.21.15.114:0/1377427307 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f64e000f460 con 0x7f64e4100c10 2024-02-16T22:23:56.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.727+0000 7f64daffd700 1 -- 172.21.15.114:0/1377427307 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64e0021620 con 0x7f64e4100c10 2024-02-16T22:23:56.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.727+0000 7f64ebc99700 1 -- 172.21.15.114:0/1377427307 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e419ba80 con 0x7f64e4100c10 2024-02-16T22:23:56.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.727+0000 7f64ebc99700 1 -- 172.21.15.114:0/1377427307 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e419bf70 con 0x7f64e4100c10 2024-02-16T22:23:56.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.728+0000 7f64ebc99700 1 -- 172.21.15.114:0/1377427307 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e4109770 con 0x7f64e4100c10 2024-02-16T22:23:56.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.729+0000 7f64daffd700 1 -- 172.21.15.114:0/1377427307 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 37) v1 ==== 91597+0+0 (secure 0 0 0) 0x7f64e000f5d0 con 0x7f64e4100c10 2024-02-16T22:23:56.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.729+0000 7f64daffd700 1 --2- 172.21.15.114:0/1377427307 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f64d006d9e0 0x7f64d006fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:56.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.729+0000 7f64daffd700 1 -- 172.21.15.114:0/1377427307 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f64e008e920 con 0x7f64e4100c10 2024-02-16T22:23:56.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.730+0000 7f64e9234700 1 --2- 172.21.15.114:0/1377427307 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f64d006d9e0 0x7f64d006fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:56.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.730+0000 7f64e9234700 1 --2- 172.21.15.114:0/1377427307 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f64d006d9e0 0x7f64d006fea0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f64d4005950 tx=0x7f64d400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:23:56.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.734+0000 7f64daffd700 1 -- 172.21.15.114:0/1377427307 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f64e005bb10 con 0x7f64e4100c10 2024-02-16T22:23:56.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.949+0000 7f64ebc99700 1 -- 172.21.15.114:0/1377427307 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f64e4106760 con 0x7f64d006d9e0 2024-02-16T22:23:56.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.960+0000 7f64daffd700 1 -- 172.21.15.114:0/1377427307 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f64e4106760 con 0x7f64d006d9e0 2024-02-16T22:23:56.962 INFO:teuthology.orchestra.run.smithi114.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:23:56.962 INFO:teuthology.orchestra.run.smithi114.stdout:alertmanager.smithi114 smithi114 *:9093,9094 running (9m) 28s ago 11m 24.5M - 0.25.0 c8568f914cd2 e7404fea400e 2024-02-16T22:23:56.962 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi114 smithi114 running (11m) 28s ago 11m 7894k - 18.2.1 94eadd990e0a 7fd07162cd96 2024-02-16T22:23:56.962 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi123 smithi123 running (10m) 7s ago 10m 7604k - 18.2.1 94eadd990e0a 7687f5e99621 2024-02-16T22:23:56.963 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi114 smithi114 running (11m) 28s ago 11m 6660k - 18.2.1 94eadd990e0a fc8fbfe4fdd5 2024-02-16T22:23:56.963 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi123 smithi123 running (10m) 7s ago 10m 6665k - 18.2.1 94eadd990e0a a6249642b9bb 2024-02-16T22:23:56.963 INFO:teuthology.orchestra.run.smithi114.stdout:grafana.smithi114 smithi114 *:3000 running (9m) 28s ago 11m 89.9M - 9.4.7 954c08fa6188 db1c26c3bc20 2024-02-16T22:23:56.963 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.fdxpey smithi114 running (5m) 28s ago 5m 16.3M - 18.2.1 94eadd990e0a 704b40c144fb 2024-02-16T22:23:56.963 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.zzryke smithi114 running (5m) 28s ago 5m 250M - 18.2.1 94eadd990e0a 1f795d23c9a4 2024-02-16T22:23:56.963 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.gopcfa smithi123 running (5m) 7s ago 5m 16.4M - 18.2.1 94eadd990e0a 166134b7532a 2024-02-16T22:23:56.963 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.vhlqbe smithi123 running (5m) 7s ago 5m 17.9M - 18.2.1 94eadd990e0a 243c8bb96fa8 2024-02-16T22:23:56.963 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi114.vcullz smithi114 *:8443,9283,8765 running (114s) 28s ago 12m 497M - 19.0.0-1450-gf78a58c0 f081f847070a 14ebeb7888c0 2024-02-16T22:23:56.963 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi123.inzgjd smithi123 *:8443,9283,8765 running (40s) 7s ago 10m 446M - 19.0.0-1450-gf78a58c0 f081f847070a f10aab9faf9e 2024-02-16T22:23:56.963 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi114 smithi114 running (12m) 28s ago 12m 58.4M 2048M 18.2.1 94eadd990e0a b4c4433aa3a4 2024-02-16T22:23:56.963 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi123 smithi123 running (10m) 7s ago 10m 49.4M 2048M 18.2.1 94eadd990e0a c217fc30c152 2024-02-16T22:23:56.963 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi114 smithi114 *:9100 running (11m) 28s ago 11m 15.4M - 1.5.0 0da6a335fe13 744a090e06d0 2024-02-16T22:23:56.963 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi123 smithi123 *:9100 running (10m) 7s ago 10m 15.6M - 1.5.0 0da6a335fe13 3994b8c29b2e 2024-02-16T22:23:56.963 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 smithi114 running (9m) 28s ago 9m 194M 1216M 18.2.1 94eadd990e0a 24ca2e9714e0 2024-02-16T22:23:56.963 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 smithi114 running (9m) 28s ago 9m 205M 1216M 18.2.1 94eadd990e0a 32bb191b88e7 2024-02-16T22:23:56.964 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 smithi114 running (8m) 28s ago 8m 174M 1216M 18.2.1 94eadd990e0a 6608718d0818 2024-02-16T22:23:56.964 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 smithi123 running (8m) 7s ago 8m 258M 2240M 18.2.1 94eadd990e0a 575db379de3d 2024-02-16T22:23:56.964 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 smithi123 running (7m) 7s ago 7m 218M 2240M 18.2.1 94eadd990e0a 7e266866cd9c 2024-02-16T22:23:56.964 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 smithi123 running (7m) 7s ago 7m 200M 2240M 18.2.1 94eadd990e0a 24e5bef19fda 2024-02-16T22:23:56.964 INFO:teuthology.orchestra.run.smithi114.stdout:prometheus.smithi114 smithi114 *:9095 running (49s) 28s ago 10m 39.6M - 2.43.0 a07b618ecd1d 37aac985cdd7 2024-02-16T22:23:56.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.962+0000 7f64ebc99700 1 -- 172.21.15.114:0/1377427307 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f64d006d9e0 msgr2=0x7f64d006fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:56.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.962+0000 7f64ebc99700 1 --2- 172.21.15.114:0/1377427307 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f64d006d9e0 0x7f64d006fea0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f64d4005950 tx=0x7f64d400b410 comp rx=0 tx=0).stop 2024-02-16T22:23:56.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.962+0000 7f64ebc99700 1 -- 172.21.15.114:0/1377427307 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64e4100c10 msgr2=0x7f64e4196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:56.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.962+0000 7f64ebc99700 1 --2- 172.21.15.114:0/1377427307 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64e4100c10 0x7f64e4196390 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f64e000bda0 tx=0x7f64e000bdd0 comp rx=0 tx=0).stop 2024-02-16T22:23:56.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.962+0000 7f64ebc99700 1 -- 172.21.15.114:0/1377427307 shutdown_connections 2024-02-16T22:23:56.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.962+0000 7f64ebc99700 1 --2- 172.21.15.114:0/1377427307 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f64d006d9e0 0x7f64d006fea0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:56.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.962+0000 7f64ebc99700 1 --2- 172.21.15.114:0/1377427307 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64e4100c10 0x7f64e4196390 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:56.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.962+0000 7f64ebc99700 1 --2- 172.21.15.114:0/1377427307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64e4101e10 0x7f64e41968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:56.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.962+0000 7f64ebc99700 1 -- 172.21.15.114:0/1377427307 >> 172.21.15.114:0/1377427307 conn(0x7f64e40fc1b0 msgr2=0x7f64e4105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:23:56.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.963+0000 7f64ebc99700 1 -- 172.21.15.114:0/1377427307 shutdown_connections 2024-02-16T22:23:56.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:56.963+0000 7f64ebc99700 1 -- 172.21.15.114:0/1377427307 wait complete. 2024-02-16T22:23:57.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.073+0000 7fb6be3a7700 1 -- 172.21.15.114:0/1949383343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6b8100bc0 msgr2=0x7fb6b8100fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:57.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.073+0000 7fb6be3a7700 1 --2- 172.21.15.114:0/1949383343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6b8100bc0 0x7fb6b8100fe0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009910 tx=0x7fb6a8009c20 comp rx=0 tx=0).stop 2024-02-16T22:23:57.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.073+0000 7fb6be3a7700 1 -- 172.21.15.114:0/1949383343 shutdown_connections 2024-02-16T22:23:57.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.073+0000 7fb6be3a7700 1 --2- 172.21.15.114:0/1949383343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6b8101dc0 0x7fb6b8102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:57.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.073+0000 7fb6be3a7700 1 --2- 172.21.15.114:0/1949383343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6b8100bc0 0x7fb6b8100fe0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:57.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.073+0000 7fb6be3a7700 1 -- 172.21.15.114:0/1949383343 >> 172.21.15.114:0/1949383343 conn(0x7fb6b80fc140 msgr2=0x7fb6b80fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:23:57.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.073+0000 7fb6be3a7700 1 -- 172.21.15.114:0/1949383343 shutdown_connections 2024-02-16T22:23:57.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.073+0000 7fb6be3a7700 1 -- 172.21.15.114:0/1949383343 wait complete. 2024-02-16T22:23:57.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.074+0000 7fb6be3a7700 1 Processor -- start 2024-02-16T22:23:57.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.074+0000 7fb6be3a7700 1 -- start start 2024-02-16T22:23:57.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.074+0000 7fb6be3a7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6b8100bc0 0x7fb6b8196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:57.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.074+0000 7fb6be3a7700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6b8101dc0 0x7fb6b81968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:57.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.074+0000 7fb6be3a7700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b8196ef0 con 0x7fb6b8101dc0 2024-02-16T22:23:57.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.074+0000 7fb6be3a7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b8197030 con 0x7fb6b8100bc0 2024-02-16T22:23:57.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.074+0000 7fb6affff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6b8101dc0 0x7fb6b81968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:57.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.074+0000 7fb6affff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6b8101dc0 0x7fb6b81968d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33882/0 (socket says 172.21.15.114:33882) 2024-02-16T22:23:57.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.074+0000 7fb6affff700 1 -- 172.21.15.114:0/3130892273 learned_addr learned my addr 172.21.15.114:0/3130892273 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:23:57.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.074+0000 7fb6affff700 1 -- 172.21.15.114:0/3130892273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6b8100bc0 msgr2=0x7fb6b8196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:57.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.074+0000 7fb6affff700 1 --2- 172.21.15.114:0/3130892273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6b8100bc0 0x7fb6b8196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:57.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.074+0000 7fb6affff700 1 -- 172.21.15.114:0/3130892273 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a80095f0 con 0x7fb6b8101dc0 2024-02-16T22:23:57.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.074+0000 7fb6affff700 1 --2- 172.21.15.114:0/3130892273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6b8101dc0 0x7fb6b81968d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000d8d0 tx=0x7fb6a000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:23:57.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.075+0000 7fb6b5ffb700 1 -- 172.21.15.114:0/3130892273 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6a0009940 con 0x7fb6b8101dc0 2024-02-16T22:23:57.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.075+0000 7fb6b5ffb700 1 -- 172.21.15.114:0/3130892273 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb6a0010460 con 0x7fb6b8101dc0 2024-02-16T22:23:57.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.075+0000 7fb6be3a7700 1 -- 172.21.15.114:0/3130892273 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b819bae0 con 0x7fb6b8101dc0 2024-02-16T22:23:57.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.075+0000 7fb6b5ffb700 1 -- 172.21.15.114:0/3130892273 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6a000f5f0 con 0x7fb6b8101dc0 2024-02-16T22:23:57.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.075+0000 7fb6be3a7700 1 -- 172.21.15.114:0/3130892273 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b819c030 con 0x7fb6b8101dc0 2024-02-16T22:23:57.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.076+0000 7fb6be3a7700 1 -- 172.21.15.114:0/3130892273 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6b8109720 con 0x7fb6b8101dc0 2024-02-16T22:23:57.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.076+0000 7fb6b5ffb700 1 -- 172.21.15.114:0/3130892273 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 37) v1 ==== 91597+0+0 (secure 0 0 0) 0x7fb6a00105d0 con 0x7fb6b8101dc0 2024-02-16T22:23:57.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.076+0000 7fb6b5ffb700 1 --2- 172.21.15.114:0/3130892273 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb6a406d9e0 0x7fb6a406fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:57.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.076+0000 7fb6b5ffb700 1 -- 172.21.15.114:0/3130892273 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fb6a008d600 con 0x7fb6b8101dc0 2024-02-16T22:23:57.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.077+0000 7fb6b7fff700 1 --2- 172.21.15.114:0/3130892273 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb6a406d9e0 0x7fb6a406fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:57.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.077+0000 7fb6b7fff700 1 --2- 172.21.15.114:0/3130892273 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb6a406d9e0 0x7fb6a406fea0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb6a800b5c0 tx=0x7fb6a8005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:23:57.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.081+0000 7fb6b5ffb700 1 -- 172.21.15.114:0/3130892273 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb6a005a7f0 con 0x7fb6b8101dc0 2024-02-16T22:23:57.221 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:57 smithi114 ceph-mon[94452]: pgmap v8: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 36 KiB/s rd, 290 KiB/s wr, 77 op/s 2024-02-16T22:23:57.221 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:57 smithi114 ceph-mon[94452]: from='client.34130 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:23:57.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.368+0000 7fb6be3a7700 1 -- 172.21.15.114:0/3130892273 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb6b819c690 con 0x7fb6b8101dc0 2024-02-16T22:23:57.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.368+0000 7fb6b5ffb700 1 -- 172.21.15.114:0/3130892273 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+779 (secure 0 0 0) 0x7fb6a005a380 con 0x7fb6b8101dc0 2024-02-16T22:23:57.371 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:23:57.371 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": { 2024-02-16T22:23:57.371 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-02-16T22:23:57.371 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T22:23:57.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:23:57.371 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": { 2024-02-16T22:23:57.371 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:23:57.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:23:57.371 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": { 2024-02-16T22:23:57.371 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-02-16T22:23:57.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:23:57.371 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-02-16T22:23:57.371 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-02-16T22:23:57.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:23:57.372 INFO:teuthology.orchestra.run.smithi114.stdout: "overall": { 2024-02-16T22:23:57.372 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 11, 2024-02-16T22:23:57.372 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-16T22:23:57.372 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-02-16T22:23:57.372 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:23:57.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.370+0000 7fb6be3a7700 1 -- 172.21.15.114:0/3130892273 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb6a406d9e0 msgr2=0x7fb6a406fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:57.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.370+0000 7fb6be3a7700 1 --2- 172.21.15.114:0/3130892273 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb6a406d9e0 0x7fb6a406fea0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb6a800b5c0 tx=0x7fb6a8005960 comp rx=0 tx=0).stop 2024-02-16T22:23:57.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.370+0000 7fb6be3a7700 1 -- 172.21.15.114:0/3130892273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6b8101dc0 msgr2=0x7fb6b81968d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:57.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.370+0000 7fb6be3a7700 1 --2- 172.21.15.114:0/3130892273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6b8101dc0 0x7fb6b81968d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000d8d0 tx=0x7fb6a000dc90 comp rx=0 tx=0).stop 2024-02-16T22:23:57.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.371+0000 7fb6be3a7700 1 -- 172.21.15.114:0/3130892273 shutdown_connections 2024-02-16T22:23:57.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.371+0000 7fb6be3a7700 1 --2- 172.21.15.114:0/3130892273 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb6a406d9e0 0x7fb6a406fea0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:57.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.371+0000 7fb6be3a7700 1 --2- 172.21.15.114:0/3130892273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6b8100bc0 0x7fb6b8196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:57.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.371+0000 7fb6be3a7700 1 --2- 172.21.15.114:0/3130892273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6b8101dc0 0x7fb6b81968d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:57.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.371+0000 7fb6be3a7700 1 -- 172.21.15.114:0/3130892273 >> 172.21.15.114:0/3130892273 conn(0x7fb6b80fc140 msgr2=0x7fb6b8104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:23:57.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.371+0000 7fb6be3a7700 1 -- 172.21.15.114:0/3130892273 shutdown_connections 2024-02-16T22:23:57.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.371+0000 7fb6be3a7700 1 -- 172.21.15.114:0/3130892273 wait complete. 2024-02-16T22:23:57.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.498+0000 7fd909c1d700 1 -- 172.21.15.114:0/3912516251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd904103f70 msgr2=0x7fd904106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:57.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.498+0000 7fd909c1d700 1 --2- 172.21.15.114:0/3912516251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd904103f70 0x7fd904106360 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8009910 tx=0x7fd8f8009c20 comp rx=0 tx=0).stop 2024-02-16T22:23:57.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.498+0000 7fd909c1d700 1 -- 172.21.15.114:0/3912516251 shutdown_connections 2024-02-16T22:23:57.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.498+0000 7fd909c1d700 1 --2- 172.21.15.114:0/3912516251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd904103f70 0x7fd904106360 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:57.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.498+0000 7fd909c1d700 1 --2- 172.21.15.114:0/3912516251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd904101640 0x7fd904103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:57.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.498+0000 7fd909c1d700 1 -- 172.21.15.114:0/3912516251 >> 172.21.15.114:0/3912516251 conn(0x7fd9040fb030 msgr2=0x7fd9040fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:23:57.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.498+0000 7fd909c1d700 1 -- 172.21.15.114:0/3912516251 shutdown_connections 2024-02-16T22:23:57.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.499+0000 7fd909c1d700 1 -- 172.21.15.114:0/3912516251 wait complete. 2024-02-16T22:23:57.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.499+0000 7fd909c1d700 1 Processor -- start 2024-02-16T22:23:57.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.499+0000 7fd909c1d700 1 -- start start 2024-02-16T22:23:57.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.500+0000 7fd909c1d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd904101640 0x7fd904196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:57.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.500+0000 7fd909c1d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd904103f70 0x7fd904196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:57.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.500+0000 7fd909c1d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd904196e90 con 0x7fd904103f70 2024-02-16T22:23:57.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.500+0000 7fd909c1d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd904196fd0 con 0x7fd904101640 2024-02-16T22:23:57.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.500+0000 7fd902ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd904103f70 0x7fd904196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:57.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.500+0000 7fd902ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd904103f70 0x7fd904196870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33902/0 (socket says 172.21.15.114:33902) 2024-02-16T22:23:57.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.500+0000 7fd902ffd700 1 -- 172.21.15.114:0/2401960351 learned_addr learned my addr 172.21.15.114:0/2401960351 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:23:57.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.500+0000 7fd9037fe700 1 --2- 172.21.15.114:0/2401960351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd904101640 0x7fd904196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:57.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.500+0000 7fd902ffd700 1 -- 172.21.15.114:0/2401960351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd904101640 msgr2=0x7fd904196330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:57.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.500+0000 7fd902ffd700 1 --2- 172.21.15.114:0/2401960351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd904101640 0x7fd904196330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:57.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.500+0000 7fd902ffd700 1 -- 172.21.15.114:0/2401960351 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f80095f0 con 0x7fd904103f70 2024-02-16T22:23:57.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.500+0000 7fd902ffd700 1 --2- 172.21.15.114:0/2401960351 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd904103f70 0x7fd904196870 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8006010 tx=0x7fd8f800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:23:57.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.500+0000 7fd900ff9700 1 -- 172.21.15.114:0/2401960351 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8f801d070 con 0x7fd904103f70 2024-02-16T22:23:57.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.500+0000 7fd900ff9700 1 -- 172.21.15.114:0/2401960351 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd8f800f460 con 0x7fd904103f70 2024-02-16T22:23:57.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.501+0000 7fd900ff9700 1 -- 172.21.15.114:0/2401960351 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8f8021640 con 0x7fd904103f70 2024-02-16T22:23:57.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.501+0000 7fd909c1d700 1 -- 172.21.15.114:0/2401960351 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd90419ba20 con 0x7fd904103f70 2024-02-16T22:23:57.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.501+0000 7fd909c1d700 1 -- 172.21.15.114:0/2401960351 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd90419bf10 con 0x7fd904103f70 2024-02-16T22:23:57.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.502+0000 7fd909c1d700 1 -- 172.21.15.114:0/2401960351 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd904190520 con 0x7fd904103f70 2024-02-16T22:23:57.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.502+0000 7fd900ff9700 1 -- 172.21.15.114:0/2401960351 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 37) v1 ==== 91597+0+0 (secure 0 0 0) 0x7fd8f800fab0 con 0x7fd904103f70 2024-02-16T22:23:57.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.503+0000 7fd900ff9700 1 --2- 172.21.15.114:0/2401960351 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd8ec06d9e0 0x7fd8ec06fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:57.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.503+0000 7fd900ff9700 1 -- 172.21.15.114:0/2401960351 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd8f808ea30 con 0x7fd904103f70 2024-02-16T22:23:57.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.503+0000 7fd9037fe700 1 --2- 172.21.15.114:0/2401960351 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd8ec06d9e0 0x7fd8ec06fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:57.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.503+0000 7fd9037fe700 1 --2- 172.21.15.114:0/2401960351 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd8ec06d9e0 0x7fd8ec06fea0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4005950 tx=0x7fd8f400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:23:57.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.508+0000 7fd900ff9700 1 -- 172.21.15.114:0/2401960351 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd8f805bba0 con 0x7fd904103f70 2024-02-16T22:23:57.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:57 smithi123 ceph-mon[37161]: pgmap v8: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 36 KiB/s rd, 290 KiB/s wr, 77 op/s 2024-02-16T22:23:57.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:57 smithi123 ceph-mon[37161]: from='client.34130 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:23:57.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.749+0000 7fd909c1d700 1 -- 172.21.15.114:0/2401960351 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd90402ce50 con 0x7fd904103f70 2024-02-16T22:23:57.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.750+0000 7fd900ff9700 1 -- 172.21.15.114:0/2401960351 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1846 (secure 0 0 0) 0x7fd8f805b730 con 0x7fd904103f70 2024-02-16T22:23:57.752 INFO:teuthology.orchestra.run.smithi114.stdout:e14 2024-02-16T22:23:57.752 INFO:teuthology.orchestra.run.smithi114.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:23:57.752 INFO:teuthology.orchestra.run.smithi114.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:23:57.752 INFO:teuthology.orchestra.run.smithi114.stdout:legacy client fscid: 1 2024-02-16T22:23:57.753 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:23:57.753 INFO:teuthology.orchestra.run.smithi114.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:23:57.753 INFO:teuthology.orchestra.run.smithi114.stdout:fs_name cephfs 2024-02-16T22:23:57.753 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 12 2024-02-16T22:23:57.753 INFO:teuthology.orchestra.run.smithi114.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:23:57.753 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-02-16T22:18:02.108825+0000 2024-02-16T22:23:57.753 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-02-16T22:18:16.413371+0000 2024-02-16T22:23:57.753 INFO:teuthology.orchestra.run.smithi114.stdout:tableserver 0 2024-02-16T22:23:57.753 INFO:teuthology.orchestra.run.smithi114.stdout:root 0 2024-02-16T22:23:57.753 INFO:teuthology.orchestra.run.smithi114.stdout:session_timeout 60 2024-02-16T22:23:57.753 INFO:teuthology.orchestra.run.smithi114.stdout:session_autoclose 300 2024-02-16T22:23:57.753 INFO:teuthology.orchestra.run.smithi114.stdout:max_file_size 1099511627776 2024-02-16T22:23:57.753 INFO:teuthology.orchestra.run.smithi114.stdout:max_xattr_size 65536 2024-02-16T22:23:57.753 INFO:teuthology.orchestra.run.smithi114.stdout:required_client_features {} 2024-02-16T22:23:57.753 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure 0 2024-02-16T22:23:57.754 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure_osd_epoch 0 2024-02-16T22:23:57.754 INFO:teuthology.orchestra.run.smithi114.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:23:57.754 INFO:teuthology.orchestra.run.smithi114.stdout:max_mds 1 2024-02-16T22:23:57.754 INFO:teuthology.orchestra.run.smithi114.stdout:in 0 2024-02-16T22:23:57.754 INFO:teuthology.orchestra.run.smithi114.stdout:up {0=14468} 2024-02-16T22:23:57.754 INFO:teuthology.orchestra.run.smithi114.stdout:failed 2024-02-16T22:23:57.754 INFO:teuthology.orchestra.run.smithi114.stdout:damaged 2024-02-16T22:23:57.754 INFO:teuthology.orchestra.run.smithi114.stdout:stopped 2024-02-16T22:23:57.754 INFO:teuthology.orchestra.run.smithi114.stdout:data_pools [3] 2024-02-16T22:23:57.754 INFO:teuthology.orchestra.run.smithi114.stdout:metadata_pool 2 2024-02-16T22:23:57.754 INFO:teuthology.orchestra.run.smithi114.stdout:inline_data enabled 2024-02-16T22:23:57.754 INFO:teuthology.orchestra.run.smithi114.stdout:balancer 2024-02-16T22:23:57.754 INFO:teuthology.orchestra.run.smithi114.stdout:bal_rank_mask -1 2024-02-16T22:23:57.754 INFO:teuthology.orchestra.run.smithi114.stdout:standby_count_wanted 1 2024-02-16T22:23:57.755 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.zzryke{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.114:6826/277953535,v1:172.21.15.114:6827/277953535] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:23:57.755 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:23:57.755 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:23:57.755 INFO:teuthology.orchestra.run.smithi114.stdout:Standby daemons: 2024-02-16T22:23:57.755 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:23:57.755 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.fdxpey{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:23:57.755 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.vhlqbe{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:23:57.755 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.gopcfa{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:23:57.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.752+0000 7fd909c1d700 1 -- 172.21.15.114:0/2401960351 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd8ec06d9e0 msgr2=0x7fd8ec06fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:57.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.752+0000 7fd909c1d700 1 --2- 172.21.15.114:0/2401960351 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd8ec06d9e0 0x7fd8ec06fea0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4005950 tx=0x7fd8f400b410 comp rx=0 tx=0).stop 2024-02-16T22:23:57.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.752+0000 7fd909c1d700 1 -- 172.21.15.114:0/2401960351 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd904103f70 msgr2=0x7fd904196870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:57.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.752+0000 7fd909c1d700 1 --2- 172.21.15.114:0/2401960351 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd904103f70 0x7fd904196870 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8006010 tx=0x7fd8f800bd70 comp rx=0 tx=0).stop 2024-02-16T22:23:57.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.752+0000 7fd909c1d700 1 -- 172.21.15.114:0/2401960351 shutdown_connections 2024-02-16T22:23:57.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.752+0000 7fd909c1d700 1 --2- 172.21.15.114:0/2401960351 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd8ec06d9e0 0x7fd8ec06fea0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:57.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.752+0000 7fd909c1d700 1 --2- 172.21.15.114:0/2401960351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd904101640 0x7fd904196330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:57.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.752+0000 7fd909c1d700 1 --2- 172.21.15.114:0/2401960351 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd904103f70 0x7fd904196870 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:57.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.752+0000 7fd909c1d700 1 -- 172.21.15.114:0/2401960351 >> 172.21.15.114:0/2401960351 conn(0x7fd9040fb030 msgr2=0x7fd9040fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:23:57.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.752+0000 7fd909c1d700 1 -- 172.21.15.114:0/2401960351 shutdown_connections 2024-02-16T22:23:57.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.752+0000 7fd909c1d700 1 -- 172.21.15.114:0/2401960351 wait complete. 2024-02-16T22:23:57.757 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 14 2024-02-16T22:23:57.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.863+0000 7f00f4dd9700 1 -- 172.21.15.114:0/3888422155 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00f0103f70 msgr2=0x7f00f0106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:57.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.863+0000 7f00f4dd9700 1 --2- 172.21.15.114:0/3888422155 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00f0103f70 0x7f00f0106360 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f00e4009910 tx=0x7f00e4009c20 comp rx=0 tx=0).stop 2024-02-16T22:23:57.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.864+0000 7f00f4dd9700 1 -- 172.21.15.114:0/3888422155 shutdown_connections 2024-02-16T22:23:57.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.864+0000 7f00f4dd9700 1 --2- 172.21.15.114:0/3888422155 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00f0103f70 0x7f00f0106360 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:57.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.864+0000 7f00f4dd9700 1 --2- 172.21.15.114:0/3888422155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00f0101640 0x7f00f0103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:57.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.864+0000 7f00f4dd9700 1 -- 172.21.15.114:0/3888422155 >> 172.21.15.114:0/3888422155 conn(0x7f00f00fb050 msgr2=0x7f00f00fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:23:57.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.864+0000 7f00f4dd9700 1 -- 172.21.15.114:0/3888422155 shutdown_connections 2024-02-16T22:23:57.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.864+0000 7f00f4dd9700 1 -- 172.21.15.114:0/3888422155 wait complete. 2024-02-16T22:23:57.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.864+0000 7f00f4dd9700 1 Processor -- start 2024-02-16T22:23:57.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.864+0000 7f00f4dd9700 1 -- start start 2024-02-16T22:23:57.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.865+0000 7f00f4dd9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00f0101640 0x7f00f0196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:57.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.865+0000 7f00f4dd9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00f0103f70 0x7f00f0196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:57.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.865+0000 7f00f4dd9700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f0196e80 con 0x7f00f0103f70 2024-02-16T22:23:57.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.865+0000 7f00f4dd9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f0196fc0 con 0x7f00f0101640 2024-02-16T22:23:57.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.865+0000 7f00edd9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00f0103f70 0x7f00f0196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:57.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.865+0000 7f00edd9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00f0103f70 0x7f00f0196860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33910/0 (socket says 172.21.15.114:33910) 2024-02-16T22:23:57.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.865+0000 7f00edd9b700 1 -- 172.21.15.114:0/2655705294 learned_addr learned my addr 172.21.15.114:0/2655705294 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:23:57.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.865+0000 7f00edd9b700 1 -- 172.21.15.114:0/2655705294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00f0101640 msgr2=0x7f00f0196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:57.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.865+0000 7f00edd9b700 1 --2- 172.21.15.114:0/2655705294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00f0101640 0x7f00f0196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:57.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.865+0000 7f00edd9b700 1 -- 172.21.15.114:0/2655705294 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e40095f0 con 0x7f00f0103f70 2024-02-16T22:23:57.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.865+0000 7f00edd9b700 1 --2- 172.21.15.114:0/2655705294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00f0103f70 0x7f00f0196860 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f00e4006010 tx=0x7f00e400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:23:57.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.865+0000 7f00df7fe700 1 -- 172.21.15.114:0/2655705294 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00e401d070 con 0x7f00f0103f70 2024-02-16T22:23:57.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.865+0000 7f00df7fe700 1 -- 172.21.15.114:0/2655705294 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f00e400f460 con 0x7f00f0103f70 2024-02-16T22:23:57.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.865+0000 7f00df7fe700 1 -- 172.21.15.114:0/2655705294 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00e4017630 con 0x7f00f0103f70 2024-02-16T22:23:57.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.865+0000 7f00f4dd9700 1 -- 172.21.15.114:0/2655705294 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00f019ba10 con 0x7f00f0103f70 2024-02-16T22:23:57.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.866+0000 7f00f4dd9700 1 -- 172.21.15.114:0/2655705294 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f019bf00 con 0x7f00f0103f70 2024-02-16T22:23:57.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.866+0000 7f00f4dd9700 1 -- 172.21.15.114:0/2655705294 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00f0190520 con 0x7f00f0103f70 2024-02-16T22:23:57.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.867+0000 7f00df7fe700 1 -- 172.21.15.114:0/2655705294 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 37) v1 ==== 91597+0+0 (secure 0 0 0) 0x7f00e4003a60 con 0x7f00f0103f70 2024-02-16T22:23:57.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.868+0000 7f00df7fe700 1 --2- 172.21.15.114:0/2655705294 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f00d806d9e0 0x7f00d806fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:57.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.868+0000 7f00ee59c700 1 --2- 172.21.15.114:0/2655705294 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f00d806d9e0 0x7f00d806fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:57.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.868+0000 7f00df7fe700 1 -- 172.21.15.114:0/2655705294 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f00e408eba0 con 0x7f00f0103f70 2024-02-16T22:23:57.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.869+0000 7f00ee59c700 1 --2- 172.21.15.114:0/2655705294 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f00d806d9e0 0x7f00d806fea0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f00e0005950 tx=0x7f00e00058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:23:57.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:57.874+0000 7f00df7fe700 1 -- 172.21.15.114:0/2655705294 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f00e405bd90 con 0x7f00f0103f70 2024-02-16T22:23:58.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.092+0000 7f00f4dd9700 1 -- 172.21.15.114:0/2655705294 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f00f00611d0 con 0x7f00d806d9e0 2024-02-16T22:23:58.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.094+0000 7f00df7fe700 1 -- 172.21.15.114:0/2655705294 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f00f00611d0 con 0x7f00d806d9e0 2024-02-16T22:23:58.096 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:23:58.096 INFO:teuthology.orchestra.run.smithi114.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:23:58.096 INFO:teuthology.orchestra.run.smithi114.stdout: "in_progress": true, 2024-02-16T22:23:58.096 INFO:teuthology.orchestra.run.smithi114.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:23:58.096 INFO:teuthology.orchestra.run.smithi114.stdout: "services_complete": [ 2024-02-16T22:23:58.096 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr" 2024-02-16T22:23:58.096 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-02-16T22:23:58.096 INFO:teuthology.orchestra.run.smithi114.stdout: "progress": "2/23 daemons upgraded", 2024-02-16T22:23:58.096 INFO:teuthology.orchestra.run.smithi114.stdout: "message": "", 2024-02-16T22:23:58.096 INFO:teuthology.orchestra.run.smithi114.stdout: "is_paused": false 2024-02-16T22:23:58.096 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:23:58.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.095+0000 7f00f4dd9700 1 -- 172.21.15.114:0/2655705294 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f00d806d9e0 msgr2=0x7f00d806fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:58.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.096+0000 7f00f4dd9700 1 --2- 172.21.15.114:0/2655705294 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f00d806d9e0 0x7f00d806fea0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f00e0005950 tx=0x7f00e00058e0 comp rx=0 tx=0).stop 2024-02-16T22:23:58.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.096+0000 7f00f4dd9700 1 -- 172.21.15.114:0/2655705294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00f0103f70 msgr2=0x7f00f0196860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:58.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.096+0000 7f00f4dd9700 1 --2- 172.21.15.114:0/2655705294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00f0103f70 0x7f00f0196860 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f00e4006010 tx=0x7f00e400be20 comp rx=0 tx=0).stop 2024-02-16T22:23:58.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.096+0000 7f00f4dd9700 1 -- 172.21.15.114:0/2655705294 shutdown_connections 2024-02-16T22:23:58.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.096+0000 7f00f4dd9700 1 --2- 172.21.15.114:0/2655705294 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f00d806d9e0 0x7f00d806fea0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:58.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.096+0000 7f00f4dd9700 1 --2- 172.21.15.114:0/2655705294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00f0101640 0x7f00f0196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:58.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.096+0000 7f00f4dd9700 1 --2- 172.21.15.114:0/2655705294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00f0103f70 0x7f00f0196860 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:58.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.096+0000 7f00f4dd9700 1 -- 172.21.15.114:0/2655705294 >> 172.21.15.114:0/2655705294 conn(0x7f00f00fb050 msgr2=0x7f00f00fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:23:58.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.096+0000 7f00f4dd9700 1 -- 172.21.15.114:0/2655705294 shutdown_connections 2024-02-16T22:23:58.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.096+0000 7f00f4dd9700 1 -- 172.21.15.114:0/2655705294 wait complete. 2024-02-16T22:23:58.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.204+0000 7f5c8e3cd700 1 -- 172.21.15.114:0/4216511488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c88103f70 msgr2=0x7f5c88106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:58.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.204+0000 7f5c8e3cd700 1 --2- 172.21.15.114:0/4216511488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c88103f70 0x7f5c88106360 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c009910 tx=0x7f5c7c009c20 comp rx=0 tx=0).stop 2024-02-16T22:23:58.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.204+0000 7f5c8e3cd700 1 -- 172.21.15.114:0/4216511488 shutdown_connections 2024-02-16T22:23:58.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.204+0000 7f5c8e3cd700 1 --2- 172.21.15.114:0/4216511488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c88103f70 0x7f5c88106360 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:58.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.204+0000 7f5c8e3cd700 1 --2- 172.21.15.114:0/4216511488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c88101640 0x7f5c88103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:58.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.204+0000 7f5c8e3cd700 1 -- 172.21.15.114:0/4216511488 >> 172.21.15.114:0/4216511488 conn(0x7f5c880fb030 msgr2=0x7f5c880fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:23:58.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.204+0000 7f5c8e3cd700 1 -- 172.21.15.114:0/4216511488 shutdown_connections 2024-02-16T22:23:58.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.204+0000 7f5c8e3cd700 1 -- 172.21.15.114:0/4216511488 wait complete. 2024-02-16T22:23:58.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.205+0000 7f5c8e3cd700 1 Processor -- start 2024-02-16T22:23:58.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.205+0000 7f5c8e3cd700 1 -- start start 2024-02-16T22:23:58.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.205+0000 7f5c8e3cd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c88101640 0x7f5c88196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:58.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.205+0000 7f5c8e3cd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c88103f70 0x7f5c88196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:58.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.205+0000 7f5c8e3cd700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c88196e90 con 0x7f5c88101640 2024-02-16T22:23:58.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.205+0000 7f5c8e3cd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c88196fd0 con 0x7f5c88103f70 2024-02-16T22:23:58.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.206+0000 7f5c87fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c88101640 0x7f5c88196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:58.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.206+0000 7f5c87fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c88101640 0x7f5c88196330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33924/0 (socket says 172.21.15.114:33924) 2024-02-16T22:23:58.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.206+0000 7f5c87fff700 1 -- 172.21.15.114:0/825613841 learned_addr learned my addr 172.21.15.114:0/825613841 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:23:58.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.206+0000 7f5c877fe700 1 --2- 172.21.15.114:0/825613841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c88103f70 0x7f5c88196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:58.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.206+0000 7f5c87fff700 1 -- 172.21.15.114:0/825613841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c88103f70 msgr2=0x7f5c88196870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:58.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.206+0000 7f5c87fff700 1 --2- 172.21.15.114:0/825613841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c88103f70 0x7f5c88196870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:58.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.206+0000 7f5c87fff700 1 -- 172.21.15.114:0/825613841 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7c0095f0 con 0x7f5c88101640 2024-02-16T22:23:58.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.206+0000 7f5c87fff700 1 --2- 172.21.15.114:0/825613841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c88101640 0x7f5c88196330 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800d8d0 tx=0x7f5c7800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:23:58.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.206+0000 7f5c857fa700 1 -- 172.21.15.114:0/825613841 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c780098a0 con 0x7f5c88101640 2024-02-16T22:23:58.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.206+0000 7f5c857fa700 1 -- 172.21.15.114:0/825613841 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5c78010460 con 0x7f5c88101640 2024-02-16T22:23:58.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.206+0000 7f5c8e3cd700 1 -- 172.21.15.114:0/825613841 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c8819ba80 con 0x7f5c88101640 2024-02-16T22:23:58.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.206+0000 7f5c857fa700 1 -- 172.21.15.114:0/825613841 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c7800f5f0 con 0x7f5c88101640 2024-02-16T22:23:58.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.207+0000 7f5c8e3cd700 1 -- 172.21.15.114:0/825613841 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c8819bfd0 con 0x7f5c88101640 2024-02-16T22:23:58.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.208+0000 7f5c8e3cd700 1 -- 172.21.15.114:0/825613841 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c88190520 con 0x7f5c88101640 2024-02-16T22:23:58.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.208+0000 7f5c857fa700 1 -- 172.21.15.114:0/825613841 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 37) v1 ==== 91597+0+0 (secure 0 0 0) 0x7f5c78009a00 con 0x7f5c88101640 2024-02-16T22:23:58.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.209+0000 7f5c857fa700 1 --2- 172.21.15.114:0/825613841 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f5c7006d9e0 0x7f5c7006fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:23:58.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.209+0000 7f5c857fa700 1 -- 172.21.15.114:0/825613841 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f5c7808d530 con 0x7f5c88101640 2024-02-16T22:23:58.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.209+0000 7f5c877fe700 1 --2- 172.21.15.114:0/825613841 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f5c7006d9e0 0x7f5c7006fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:23:58.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.209+0000 7f5c877fe700 1 --2- 172.21.15.114:0/825613841 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f5c7006d9e0 0x7f5c7006fea0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c006010 tx=0x7f5c7c00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:23:58.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.214+0000 7f5c857fa700 1 -- 172.21.15.114:0/825613841 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5c7805a720 con 0x7f5c88101640 2024-02-16T22:23:58.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:58 smithi114 ceph-mon[94452]: from='client.34134 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:23:58.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:58 smithi114 ceph-mon[94452]: from='client.34138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:23:58.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:58 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/3130892273' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:23:58.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:58 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/2401960351' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:23:58.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.514+0000 7f5c8e3cd700 1 -- 172.21.15.114:0/825613841 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5c88109860 con 0x7f5c88101640 2024-02-16T22:23:58.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.514+0000 7f5c857fa700 1 -- 172.21.15.114:0/825613841 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f5c7805a2b0 con 0x7f5c88101640 2024-02-16T22:23:58.516 INFO:teuthology.orchestra.run.smithi114.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:23:58.516 INFO:teuthology.orchestra.run.smithi114.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:23:58.516 INFO:teuthology.orchestra.run.smithi114.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:23:58.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.516+0000 7f5c8e3cd700 1 -- 172.21.15.114:0/825613841 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f5c7006d9e0 msgr2=0x7f5c7006fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:58.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.516+0000 7f5c8e3cd700 1 --2- 172.21.15.114:0/825613841 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f5c7006d9e0 0x7f5c7006fea0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c006010 tx=0x7f5c7c00b540 comp rx=0 tx=0).stop 2024-02-16T22:23:58.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.516+0000 7f5c8e3cd700 1 -- 172.21.15.114:0/825613841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c88101640 msgr2=0x7f5c88196330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:23:58.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.516+0000 7f5c8e3cd700 1 --2- 172.21.15.114:0/825613841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c88101640 0x7f5c88196330 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800d8d0 tx=0x7f5c7800dbe0 comp rx=0 tx=0).stop 2024-02-16T22:23:58.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.516+0000 7f5c8e3cd700 1 -- 172.21.15.114:0/825613841 shutdown_connections 2024-02-16T22:23:58.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.516+0000 7f5c8e3cd700 1 --2- 172.21.15.114:0/825613841 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f5c7006d9e0 0x7f5c7006fea0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:58.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.516+0000 7f5c8e3cd700 1 --2- 172.21.15.114:0/825613841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c88101640 0x7f5c88196330 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:58.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.516+0000 7f5c8e3cd700 1 --2- 172.21.15.114:0/825613841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c88103f70 0x7f5c88196870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:23:58.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.516+0000 7f5c8e3cd700 1 -- 172.21.15.114:0/825613841 >> 172.21.15.114:0/825613841 conn(0x7f5c880fb030 msgr2=0x7f5c880fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:23:58.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.516+0000 7f5c8e3cd700 1 -- 172.21.15.114:0/825613841 shutdown_connections 2024-02-16T22:23:58.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:23:58.516+0000 7f5c8e3cd700 1 -- 172.21.15.114:0/825613841 wait complete. 2024-02-16T22:23:58.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:58 smithi123 ceph-mon[37161]: from='client.34134 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:23:58.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:58 smithi123 ceph-mon[37161]: from='client.34138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:23:58.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:58 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/3130892273' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:23:58.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:58 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/2401960351' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:23:59.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:59 smithi114 ceph-mon[94452]: pgmap v9: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 238 KiB/s wr, 64 op/s 2024-02-16T22:23:59.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:59 smithi114 ceph-mon[94452]: from='client.34150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:23:59.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:23:59 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/825613841' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:23:59.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:59 smithi123 ceph-mon[37161]: pgmap v9: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 238 KiB/s wr, 64 op/s 2024-02-16T22:23:59.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:59 smithi123 ceph-mon[37161]: from='client.34150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:23:59.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:23:59 smithi123 ceph-mon[37161]: from='client.? 172.21.15.114:0/825613841' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:24:00.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:00 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:00.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:00 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:00.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:00 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:00.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:00 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:01.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:01 smithi114 ceph-mon[94452]: pgmap v10: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 218 KiB/s wr, 58 op/s 2024-02-16T22:24:01.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:01.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:01.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:01.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:24:01.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:01 smithi114 ceph-mon[94452]: Updating smithi114:/etc/ceph/ceph.conf 2024-02-16T22:24:01.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:01 smithi114 ceph-mon[94452]: Updating smithi123:/etc/ceph/ceph.conf 2024-02-16T22:24:01.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:24:01.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:01 smithi123 ceph-mon[37161]: pgmap v10: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 218 KiB/s wr, 58 op/s 2024-02-16T22:24:01.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:01 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:01.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:01 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:01.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:01 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:01.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:01 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:24:01.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:01 smithi123 ceph-mon[37161]: Updating smithi114:/etc/ceph/ceph.conf 2024-02-16T22:24:01.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:01 smithi123 ceph-mon[37161]: Updating smithi123:/etc/ceph/ceph.conf 2024-02-16T22:24:01.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:01 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:24:02.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:02 smithi114 ceph-mon[94452]: Updating smithi114:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:24:02.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:02 smithi114 ceph-mon[94452]: Updating smithi123:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:24:02.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:02 smithi114 ceph-mon[94452]: Updating smithi114:/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:24:02.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:02 smithi114 ceph-mon[94452]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:24:02.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:02 smithi114 ceph-mon[94452]: Updating smithi114:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T22:24:02.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:02 smithi114 ceph-mon[94452]: Updating smithi123:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T22:24:02.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:02 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:02.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:02 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:02.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:02 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:02.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:02 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:02.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:02 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:02.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:02 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:24:02.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:02 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:24:02.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:02 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-16T22:24:02.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:02 smithi123 ceph-mon[37161]: Updating smithi114:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:24:02.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:02 smithi123 ceph-mon[37161]: Updating smithi123:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T22:24:02.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:02 smithi123 ceph-mon[37161]: Updating smithi114:/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:24:02.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:02 smithi123 ceph-mon[37161]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-02-16T22:24:02.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:02 smithi123 ceph-mon[37161]: Updating smithi114:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T22:24:02.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:02 smithi123 ceph-mon[37161]: Updating smithi123:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T22:24:02.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:02 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:02.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:02 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:02.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:02 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:02.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:02 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:02.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:02 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:02.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:02 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:24:02.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:02 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:24:02.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:02 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-16T22:24:03.642 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:03 smithi123 ceph-mon[37161]: pgmap v11: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 218 KiB/s wr, 58 op/s 2024-02-16T22:24:03.642 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:03 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:03.642 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:03 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T22:24:03.642 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:03 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T22:24:03.642 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:03 smithi123 ceph-mon[37161]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:03.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:03 smithi114 ceph-mon[94452]: pgmap v11: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 218 KiB/s wr, 58 op/s 2024-02-16T22:24:03.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:03 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:03.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:03 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T22:24:03.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:03 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T22:24:03.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:03 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:04.528 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:04 smithi123 ceph-mon[37161]: Upgrade: Updating mon.smithi123 2024-02-16T22:24:04.528 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:04 smithi123 ceph-mon[37161]: Deploying daemon mon.smithi123 on smithi123 2024-02-16T22:24:04.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:04 smithi114 ceph-mon[94452]: Upgrade: Updating mon.smithi123 2024-02-16T22:24:04.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:04 smithi114 ceph-mon[94452]: Deploying daemon mon.smithi123 on smithi123 2024-02-16T22:24:05.066 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:04 smithi123 systemd[1]: Stopping Ceph mon.smithi123 for 1dcbaabe-cd18-11ee-95ba-87774f69a715... 2024-02-16T22:24:05.408 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:05 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi123[37157]: 2024-02-16T22:24:05.064+0000 7fe588198700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi123 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T22:24:05.408 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:05 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi123[37157]: 2024-02-16T22:24:05.064+0000 7fe588198700 -1 mon.smithi123@1(peon) e2 *** Got Signal Terminated *** 2024-02-16T22:24:05.408 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:05 smithi123 podman[82164]: 2024-02-16 22:24:05.146342973 +0000 UTC m=+0.197621029 container died c217fc30c152c3428292c035b804fdf2838198fc562d347a62d5f1df62fa2e37 (image=quay.io/ceph/ceph@sha256:6176526d5ef4047f148d47c951653dcc6b0de49091fda006d3a088e7a1f404d0, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi123, GIT_CLEAN=True, RELEASE=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1, ceph=True) 2024-02-16T22:24:05.800 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:05 smithi123 podman[82164]: 2024-02-16 22:24:05.497564457 +0000 UTC m=+0.548842512 container cleanup c217fc30c152c3428292c035b804fdf2838198fc562d347a62d5f1df62fa2e37 (image=quay.io/ceph/ceph@sha256:6176526d5ef4047f148d47c951653dcc6b0de49091fda006d3a088e7a1f404d0, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi123, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T22:24:05.800 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:05 smithi123 bash[82164]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi123 2024-02-16T22:24:06.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:05 smithi123 podman[82178]: 2024-02-16 22:24:05.799017059 +0000 UTC m=+0.651058135 container remove c217fc30c152c3428292c035b804fdf2838198fc562d347a62d5f1df62fa2e37 (image=quay.io/ceph/ceph@sha256:6176526d5ef4047f148d47c951653dcc6b0de49091fda006d3a088e7a1f404d0, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi123, org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=HEAD, maintainer=Guillaume Abrioux ) 2024-02-16T22:24:06.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:05 smithi123 systemd[1]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@mon.smithi123.service: Deactivated successfully. 2024-02-16T22:24:06.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:05 smithi123 systemd[1]: Stopped Ceph mon.smithi123 for 1dcbaabe-cd18-11ee-95ba-87774f69a715. 2024-02-16T22:24:06.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:05 smithi123 systemd[1]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@mon.smithi123.service: Consumed 6.575s CPU time. 2024-02-16T22:24:06.490 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 systemd[1]: Starting Ceph mon.smithi123 for 1dcbaabe-cd18-11ee-95ba-87774f69a715... 2024-02-16T22:24:06.491 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 podman[82312]: 2024-02-16 22:24:06.271205784 +0000 UTC m=+0.017470561 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T22:24:06.491 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 podman[82312]: 2024-02-16 22:24:06.391893541 +0000 UTC m=+0.138158361 container create f76485e0ea24d21414f3dc5d1ff1b3351f9dd1c9422ff2307d0d27a43419a078 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi123, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-02-16T22:24:07.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 podman[82312]: 2024-02-16 22:24:06.594113121 +0000 UTC m=+0.340377898 container init f76485e0ea24d21414f3dc5d1ff1b3351f9dd1c9422ff2307d0d27a43419a078 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi123, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-02-16T22:24:07.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 podman[82312]: 2024-02-16 22:24:06.59712425 +0000 UTC m=+0.343389036 container start f76485e0ea24d21414f3dc5d1ff1b3351f9dd1c9422ff2307d0d27a43419a078 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi123, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2) 2024-02-16T22:24:07.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: set uid:gid to 167:167 (ceph:ceph) 2024-02-16T22:24:07.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev), process ceph-mon, pid 2 2024-02-16T22:24:07.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: pidfile_write: ignore empty --pid-file 2024-02-16T22:24:07.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: load: jerasure load: lrc 2024-02-16T22:24:07.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: RocksDB version: 7.9.2 2024-02-16T22:24:07.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Git sha 0 2024-02-16T22:24:07.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Compile date 2024-02-15 18:05:23 2024-02-16T22:24:07.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: DB SUMMARY 2024-02-16T22:24:07.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: DB Session ID: IDZ75YPS6JDDW5OJIXJB 2024-02-16T22:24:07.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: CURRENT file: CURRENT 2024-02-16T22:24:07.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: IDENTITY file: IDENTITY 2024-02-16T22:24:07.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1404 Bytes 2024-02-16T22:24:07.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi123/store.db dir, Total Num: 1, files: 000027.sst 2024-02-16T22:24:07.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi123/store.db: 000025.log size: 7048654 ; 2024-02-16T22:24:07.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.error_if_exists: 0 2024-02-16T22:24:07.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.create_if_missing: 0 2024-02-16T22:24:07.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.paranoid_checks: 1 2024-02-16T22:24:07.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-16T22:24:07.055 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-16T22:24:07.055 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-16T22:24:07.055 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.env: 0x561288e49ae0 2024-02-16T22:24:07.055 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.fs: PosixFileSystem 2024-02-16T22:24:07.055 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.info_log: 0x5612897ad760 2024-02-16T22:24:07.055 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.max_file_opening_threads: 16 2024-02-16T22:24:07.055 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.statistics: (nil) 2024-02-16T22:24:07.055 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.use_fsync: 0 2024-02-16T22:24:07.055 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.max_log_file_size: 0 2024-02-16T22:24:07.055 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-16T22:24:07.055 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-16T22:24:07.055 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.keep_log_file_num: 1000 2024-02-16T22:24:07.055 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.recycle_log_file_num: 0 2024-02-16T22:24:07.056 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.allow_fallocate: 1 2024-02-16T22:24:07.056 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.allow_mmap_reads: 0 2024-02-16T22:24:07.056 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.allow_mmap_writes: 0 2024-02-16T22:24:07.056 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.use_direct_reads: 0 2024-02-16T22:24:07.056 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-16T22:24:07.056 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.create_missing_column_families: 0 2024-02-16T22:24:07.056 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.db_log_dir: 2024-02-16T22:24:07.056 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.wal_dir: 2024-02-16T22:24:07.056 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-16T22:24:07.056 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-16T22:24:07.056 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-16T22:24:07.056 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-16T22:24:07.057 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-16T22:24:07.057 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-16T22:24:07.057 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.advise_random_on_open: 1 2024-02-16T22:24:07.057 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.db_write_buffer_size: 0 2024-02-16T22:24:07.057 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.write_buffer_manager: 0x561289781400 2024-02-16T22:24:07.057 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-16T22:24:07.057 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-16T22:24:07.057 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-16T22:24:07.057 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.rate_limiter: (nil) 2024-02-16T22:24:07.057 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-16T22:24:07.057 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.wal_recovery_mode: 2 2024-02-16T22:24:07.057 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.enable_thread_tracking: 0 2024-02-16T22:24:07.058 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.enable_pipelined_write: 0 2024-02-16T22:24:07.058 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.unordered_write: 0 2024-02-16T22:24:07.058 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-16T22:24:07.058 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-16T22:24:07.058 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-16T22:24:07.058 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-16T22:24:07.058 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.row_cache: None 2024-02-16T22:24:07.058 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.wal_filter: None 2024-02-16T22:24:07.059 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-16T22:24:07.060 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.allow_ingest_behind: 0 2024-02-16T22:24:07.060 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.two_write_queues: 0 2024-02-16T22:24:07.060 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.manual_wal_flush: 0 2024-02-16T22:24:07.060 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.wal_compression: 0 2024-02-16T22:24:07.060 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.atomic_flush: 0 2024-02-16T22:24:07.060 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-16T22:24:07.060 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-16T22:24:07.060 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-16T22:24:07.060 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.log_readahead_size: 0 2024-02-16T22:24:07.060 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-16T22:24:07.060 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.best_efforts_recovery: 0 2024-02-16T22:24:07.061 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-16T22:24:07.061 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-16T22:24:07.061 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.allow_data_in_errors: 0 2024-02-16T22:24:07.061 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.db_host_id: __hostname__ 2024-02-16T22:24:07.061 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-16T22:24:07.061 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.max_background_jobs: 2 2024-02-16T22:24:07.061 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.max_background_compactions: -1 2024-02-16T22:24:07.061 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.max_subcompactions: 1 2024-02-16T22:24:07.061 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-16T22:24:07.061 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-16T22:24:07.061 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-16T22:24:07.061 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.max_total_wal_size: 0 2024-02-16T22:24:07.062 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-16T22:24:07.062 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-16T22:24:07.062 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-16T22:24:07.062 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-16T22:24:07.062 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.max_open_files: -1 2024-02-16T22:24:07.062 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.bytes_per_sync: 0 2024-02-16T22:24:07.062 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-16T22:24:07.062 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-16T22:24:07.062 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.compaction_readahead_size: 0 2024-02-16T22:24:07.062 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.max_background_flushes: -1 2024-02-16T22:24:07.062 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Compression algorithms supported: 2024-02-16T22:24:07.062 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-16T22:24:07.063 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: kZSTD supported: 0 2024-02-16T22:24:07.063 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: kXpressCompression supported: 0 2024-02-16T22:24:07.063 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: kLZ4HCCompression supported: 1 2024-02-16T22:24:07.063 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: kZlibCompression supported: 1 2024-02-16T22:24:07.063 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: kSnappyCompression supported: 1 2024-02-16T22:24:07.063 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: kLZ4Compression supported: 1 2024-02-16T22:24:07.063 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: kBZip2Compression supported: 0 2024-02-16T22:24:07.063 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-16T22:24:07.063 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-16T22:24:07.063 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi123/store.db/MANIFEST-000010 2024-02-16T22:24:07.063 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-16T22:24:07.063 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-16T22:24:07.064 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.merge_operator: 2024-02-16T22:24:07.064 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.compaction_filter: None 2024-02-16T22:24:07.064 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.compaction_filter_factory: None 2024-02-16T22:24:07.064 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.sst_partitioner_factory: None 2024-02-16T22:24:07.064 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-16T22:24:07.064 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-16T22:24:07.064 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5612897ad880) 2024-02-16T22:24:07.064 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: cache_index_and_filter_blocks: 1 2024-02-16T22:24:07.064 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-16T22:24:07.064 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-16T22:24:07.064 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: pin_top_level_index_and_filter: 1 2024-02-16T22:24:07.064 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: index_type: 0 2024-02-16T22:24:07.065 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: data_block_index_type: 0 2024-02-16T22:24:07.065 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: index_shortening: 1 2024-02-16T22:24:07.065 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-16T22:24:07.065 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: checksum: 4 2024-02-16T22:24:07.065 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: no_block_cache: 0 2024-02-16T22:24:07.065 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: block_cache: 0x5612897b1090 2024-02-16T22:24:07.065 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: block_cache_name: BinnedLRUCache 2024-02-16T22:24:07.065 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: block_cache_options: 2024-02-16T22:24:07.065 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: capacity : 536870912 2024-02-16T22:24:07.065 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: num_shard_bits : 4 2024-02-16T22:24:07.065 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: strict_capacity_limit : 0 2024-02-16T22:24:07.065 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: high_pri_pool_ratio: 0.000 2024-02-16T22:24:07.066 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: block_cache_compressed: (nil) 2024-02-16T22:24:07.066 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: persistent_cache: (nil) 2024-02-16T22:24:07.066 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: block_size: 4096 2024-02-16T22:24:07.066 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: block_size_deviation: 10 2024-02-16T22:24:07.066 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: block_restart_interval: 16 2024-02-16T22:24:07.066 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: index_block_restart_interval: 1 2024-02-16T22:24:07.066 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: metadata_block_size: 4096 2024-02-16T22:24:07.066 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: partition_filters: 0 2024-02-16T22:24:07.066 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: use_delta_encoding: 1 2024-02-16T22:24:07.066 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: filter_policy: bloomfilter 2024-02-16T22:24:07.066 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: whole_key_filtering: 1 2024-02-16T22:24:07.066 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: verify_compression: 0 2024-02-16T22:24:07.066 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: read_amp_bytes_per_bit: 0 2024-02-16T22:24:07.067 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: format_version: 5 2024-02-16T22:24:07.067 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: enable_index_compression: 1 2024-02-16T22:24:07.067 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: block_align: 0 2024-02-16T22:24:07.067 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: max_auto_readahead_size: 262144 2024-02-16T22:24:07.067 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: prepopulate_block_cache: 0 2024-02-16T22:24:07.067 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: initial_auto_readahead_size: 8192 2024-02-16T22:24:07.067 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: num_file_reads_for_auto_readahead: 2 2024-02-16T22:24:07.067 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.write_buffer_size: 33554432 2024-02-16T22:24:07.067 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.max_write_buffer_number: 2 2024-02-16T22:24:07.067 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.compression: NoCompression 2024-02-16T22:24:07.067 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.bottommost_compression: Disabled 2024-02-16T22:24:07.067 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.prefix_extractor: nullptr 2024-02-16T22:24:07.068 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-16T22:24:07.068 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.num_levels: 7 2024-02-16T22:24:07.068 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-16T22:24:07.068 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-16T22:24:07.068 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-16T22:24:07.068 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-16T22:24:07.068 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-16T22:24:07.068 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-16T22:24:07.068 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-16T22:24:07.068 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-16T22:24:07.068 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-16T22:24:07.068 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-16T22:24:07.069 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-16T22:24:07.069 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-16T22:24:07.069 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-16T22:24:07.069 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.compression_opts.level: 32767 2024-02-16T22:24:07.069 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.compression_opts.strategy: 0 2024-02-16T22:24:07.069 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-16T22:24:07.069 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-16T22:24:07.069 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-16T22:24:07.069 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-16T22:24:07.069 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.compression_opts.enabled: false 2024-02-16T22:24:07.069 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-16T22:24:07.069 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-16T22:24:07.069 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-16T22:24:07.070 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-16T22:24:07.070 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.target_file_size_base: 67108864 2024-02-16T22:24:07.070 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-16T22:24:07.070 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-16T22:24:07.070 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-16T22:24:07.070 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-16T22:24:07.070 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-16T22:24:07.070 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-16T22:24:07.070 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-16T22:24:07.070 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-16T22:24:07.070 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-16T22:24:07.070 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-16T22:24:07.070 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-16T22:24:07.071 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-16T22:24:07.071 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-16T22:24:07.071 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-16T22:24:07.071 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.arena_block_size: 1048576 2024-02-16T22:24:07.071 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-16T22:24:07.071 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-16T22:24:07.071 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.disable_auto_compactions: 0 2024-02-16T22:24:07.071 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-16T22:24:07.071 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-16T22:24:07.071 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-16T22:24:07.071 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-16T22:24:07.071 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-16T22:24:07.072 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-16T22:24:07.072 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-16T22:24:07.072 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-16T22:24:07.072 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-16T22:24:07.072 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-16T22:24:07.072 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.table_properties_collectors: 2024-02-16T22:24:07.072 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.inplace_update_support: 0 2024-02-16T22:24:07.072 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-16T22:24:07.072 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-16T22:24:07.072 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-16T22:24:07.072 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-16T22:24:07.072 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.bloom_locality: 0 2024-02-16T22:24:07.072 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.max_successive_merges: 0 2024-02-16T22:24:07.073 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-16T22:24:07.073 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.paranoid_file_checks: 0 2024-02-16T22:24:07.073 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.force_consistency_checks: 1 2024-02-16T22:24:07.073 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.report_bg_io_stats: 0 2024-02-16T22:24:07.073 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.ttl: 2592000 2024-02-16T22:24:07.073 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-16T22:24:07.073 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-16T22:24:07.073 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-16T22:24:07.073 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.enable_blob_files: false 2024-02-16T22:24:07.073 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.min_blob_size: 0 2024-02-16T22:24:07.073 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.blob_file_size: 268435456 2024-02-16T22:24:07.073 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-16T22:24:07.074 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-16T22:24:07.074 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-16T22:24:07.074 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-16T22:24:07.074 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-16T22:24:07.074 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.blob_file_starting_level: 0 2024-02-16T22:24:07.074 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-16T22:24:07.074 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-16T22:24:07.074 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi123/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 29, last_sequence is 12036, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 25 2024-02-16T22:24:07.074 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-02-16T22:24:07.074 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c5131855-ce63-4261-b504-a2929d749d93 2024-02-16T22:24:07.074 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708122246626710, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-02-16T22:24:07.074 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-02-16T22:24:07.075 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708122246645170, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 4255779, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12041, "largest_seqno": 14056, "table_properties": {"data_size": 4247459, "index_size": 5380, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1925, "raw_key_size": 17466, "raw_average_key_size": 22, "raw_value_size": 4230996, "raw_average_value_size": 5516, "num_data_blocks": 244, "num_entries": 767, "num_filter_entries": 767, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708122246, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c5131855-ce63-4261-b504-a2929d749d93", "db_session_id": "IDZ75YPS6JDDW5OJIXJB", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-02-16T22:24:07.075 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708122246645278, "job": 1, "event": "recovery_finished"} 2024-02-16T22:24:07.075 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-02-16T22:24:07.075 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-16T22:24:07.075 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi123/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-16T22:24:07.075 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5612898a0000 2024-02-16T22:24:07.075 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: DB pointer 0x56128988c000 2024-02-16T22:24:07.075 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-16T22:24:07.075 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-16T22:24:07.075 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: ** DB Stats ** 2024-02-16T22:24:07.075 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-16T22:24:07.075 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-16T22:24:07.075 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-16T22:24:07.076 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-16T22:24:07.076 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-16T22:24:07.076 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-16T22:24:07.076 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-16T22:24:07.076 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: 2024-02-16T22:24:07.076 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: ** Compaction Stats [default] ** 2024-02-16T22:24:07.076 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-16T22:24:07.076 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-16T22:24:07.076 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: L0 1/0 4.06 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 546.5 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-02-16T22:24:07.076 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: L6 1/0 4.74 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-02-16T22:24:07.076 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: Sum 2/0 8.80 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 546.5 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-02-16T22:24:07.076 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 546.5 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-02-16T22:24:07.077 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: 2024-02-16T22:24:07.077 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: ** Compaction Stats [default] ** 2024-02-16T22:24:07.077 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-16T22:24:07.077 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-16T22:24:07.077 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 546.5 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-02-16T22:24:07.077 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: 2024-02-16T22:24:07.077 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-16T22:24:07.077 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: 2024-02-16T22:24:07.077 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-16T22:24:07.077 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: Flush(GB): cumulative 0.004, interval 0.004 2024-02-16T22:24:07.077 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-16T22:24:07.077 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-16T22:24:07.078 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-16T22:24:07.078 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-16T22:24:07.078 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: Cumulative compaction: 0.00 GB write, 189.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-16T22:24:07.078 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: Interval compaction: 0.00 GB write, 189.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-16T22:24:07.078 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-16T22:24:07.078 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: Block cache BinnedLRUCache@0x5612897b1090#2 capacity: 512.00 MB usage: 8.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-02-16T22:24:07.078 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,2.03 KB,0.00038743%) IndexBlock(1,6.08 KB,0.00115931%) Misc(1,0.00 KB,0%) 2024-02-16T22:24:07.078 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: 2024-02-16T22:24:07.078 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-16T22:24:07.078 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: starting mon.smithi123 rank 1 at public addrs [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] at bind addrs [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi123 fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 2024-02-16T22:24:07.078 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: mon.smithi123@-1(???) e2 preinit fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 2024-02-16T22:24:07.078 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: mon.smithi123@-1(???).mds e14 new map 2024-02-16T22:24:07.079 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: mon.smithi123@-1(???).mds e14 print_map 2024-02-16T22:24:07.079 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: e14 2024-02-16T22:24:07.079 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:24:07.079 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:24:07.079 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: legacy client fscid: 1 2024-02-16T22:24:07.079 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: 2024-02-16T22:24:07.079 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: Filesystem 'cephfs' (1) 2024-02-16T22:24:07.079 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: fs_name cephfs 2024-02-16T22:24:07.079 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: epoch 12 2024-02-16T22:24:07.079 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:24:07.079 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: created 2024-02-16T22:18:02.108825+0000 2024-02-16T22:24:07.079 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: modified 2024-02-16T22:18:16.413371+0000 2024-02-16T22:24:07.079 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: tableserver 0 2024-02-16T22:24:07.080 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: root 0 2024-02-16T22:24:07.080 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: session_timeout 60 2024-02-16T22:24:07.080 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: session_autoclose 300 2024-02-16T22:24:07.080 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: max_file_size 1099511627776 2024-02-16T22:24:07.080 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: max_xattr_size 65536 2024-02-16T22:24:07.080 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: required_client_features {} 2024-02-16T22:24:07.080 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: last_failure 0 2024-02-16T22:24:07.080 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: last_failure_osd_epoch 0 2024-02-16T22:24:07.080 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:24:07.080 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: max_mds 1 2024-02-16T22:24:07.080 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: in 0 2024-02-16T22:24:07.080 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: up {0=14468} 2024-02-16T22:24:07.081 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: failed 2024-02-16T22:24:07.081 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: damaged 2024-02-16T22:24:07.081 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: stopped 2024-02-16T22:24:07.081 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: data_pools [3] 2024-02-16T22:24:07.081 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: metadata_pool 2 2024-02-16T22:24:07.081 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: inline_data enabled 2024-02-16T22:24:07.081 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: balancer 2024-02-16T22:24:07.081 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: bal_rank_mask -1 2024-02-16T22:24:07.081 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: standby_count_wanted 1 2024-02-16T22:24:07.081 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: [mds.cephfs.smithi114.zzryke{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.114:6826/277953535,v1:172.21.15.114:6827/277953535] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:24:07.081 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: 2024-02-16T22:24:07.081 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: 2024-02-16T22:24:07.081 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: Standby daemons: 2024-02-16T22:24:07.082 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: 2024-02-16T22:24:07.082 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: [mds.cephfs.smithi114.fdxpey{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:24:07.082 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: [mds.cephfs.smithi123.vhlqbe{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:24:07.082 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout: [mds.cephfs.smithi123.gopcfa{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:24:07.082 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: mon.smithi123@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-02-16T22:24:07.082 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: mon.smithi123@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-02-16T22:24:07.082 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: mon.smithi123@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-02-16T22:24:07.082 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: mon.smithi123@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-02-16T22:24:07.082 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: mon.smithi123@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-02-16T22:24:07.082 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 bash[82312]: f76485e0ea24d21414f3dc5d1ff1b3351f9dd1c9422ff2307d0d27a43419a078 2024-02-16T22:24:07.082 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 systemd[1]: Started Ceph mon.smithi123 for 1dcbaabe-cd18-11ee-95ba-87774f69a715. 2024-02-16T22:24:07.082 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:06 smithi123 ceph-mon[82331]: pgmap v12: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 218 KiB/s wr, 58 op/s 2024-02-16T22:24:07.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:06 smithi114 ceph-mon[94452]: pgmap v12: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 218 KiB/s wr, 58 op/s 2024-02-16T22:24:08.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:07 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi114"}]: dispatch 2024-02-16T22:24:08.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:07 smithi123 ceph-mon[82331]: mon.smithi114 calling monitor election 2024-02-16T22:24:08.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:07 smithi123 ceph-mon[82331]: mon.smithi123 calling monitor election 2024-02-16T22:24:08.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:07 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi123"}]: dispatch 2024-02-16T22:24:08.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:07 smithi123 ceph-mon[82331]: mon.smithi114 is new leader, mons smithi114,smithi123 in quorum (ranks 0,1) 2024-02-16T22:24:08.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:07 smithi123 ceph-mon[82331]: monmap e3: 2 mons at {smithi114=[v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0],smithi123=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-16T22:24:08.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:07 smithi123 ceph-mon[82331]: fsmap cephfs:1 {0=cephfs.smithi114.zzryke=up:active} 3 up:standby 2024-02-16T22:24:08.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:07 smithi123 ceph-mon[82331]: osdmap e48: 6 total, 6 up, 6 in 2024-02-16T22:24:08.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:07 smithi123 ceph-mon[82331]: mgrmap e37: smithi114.vcullz(active, since 21s), standbys: smithi123.inzgjd 2024-02-16T22:24:08.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:07 smithi123 ceph-mon[82331]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:24:08.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:07 smithi123 ceph-mon[82331]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:24:08.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:07 smithi123 ceph-mon[82331]: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:24:08.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:07 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:08.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:07 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:08.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:07 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:24:08.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:07 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi114"}]: dispatch 2024-02-16T22:24:08.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:07 smithi114 ceph-mon[94452]: mon.smithi114 calling monitor election 2024-02-16T22:24:08.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:07 smithi114 ceph-mon[94452]: mon.smithi123 calling monitor election 2024-02-16T22:24:08.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:07 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mon metadata", "id": "smithi123"}]: dispatch 2024-02-16T22:24:08.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:07 smithi114 ceph-mon[94452]: mon.smithi114 is new leader, mons smithi114,smithi123 in quorum (ranks 0,1) 2024-02-16T22:24:08.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:07 smithi114 ceph-mon[94452]: monmap e3: 2 mons at {smithi114=[v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0],smithi123=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-16T22:24:08.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:07 smithi114 ceph-mon[94452]: fsmap cephfs:1 {0=cephfs.smithi114.zzryke=up:active} 3 up:standby 2024-02-16T22:24:08.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:07 smithi114 ceph-mon[94452]: osdmap e48: 6 total, 6 up, 6 in 2024-02-16T22:24:08.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:07 smithi114 ceph-mon[94452]: mgrmap e37: smithi114.vcullz(active, since 21s), standbys: smithi123.inzgjd 2024-02-16T22:24:08.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:07 smithi114 ceph-mon[94452]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:24:08.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:07 smithi114 ceph-mon[94452]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:24:08.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:07 smithi114 ceph-mon[94452]: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:24:08.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:07 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:08.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:07 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:08.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:07 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:24:09.119 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:08 smithi123 ceph-mon[82331]: pgmap v14: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 255 B/s wr, 0 op/s 2024-02-16T22:24:09.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:08 smithi114 ceph-mon[94452]: pgmap v14: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 255 B/s wr, 0 op/s 2024-02-16T22:24:11.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:10 smithi123 ceph-mon[82331]: pgmap v15: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:11.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:10 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:11.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:10 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:11.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:10 smithi114 ceph-mon[94452]: pgmap v15: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:11.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:10 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:11.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:10 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:11.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:11 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:11.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:11 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:12.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:11 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:12.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:11 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:13.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:12 smithi114 ceph-mon[94452]: pgmap v16: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:13.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:12 smithi123 ceph-mon[82331]: pgmap v16: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:15.607 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:15 smithi114 ceph-mon[94452]: pgmap v17: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:15.607 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:15 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:15.608 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:15 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:15.608 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:15 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:15.608 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:15 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:24:15.608 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:15 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:15.608 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:15 smithi114 ceph-mon[94452]: Reconfiguring mon.smithi114 (monmap changed)... 2024-02-16T22:24:15.608 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:15 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T22:24:15.608 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:15 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T22:24:15.608 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:15 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:15.608 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:15 smithi114 ceph-mon[94452]: Reconfiguring daemon mon.smithi114 on smithi114 2024-02-16T22:24:15.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:15 smithi123 ceph-mon[82331]: pgmap v17: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:15.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:15 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:15.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:15 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:15.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:15 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:15.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:15 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:24:15.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:15 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:15.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:15 smithi123 ceph-mon[82331]: Reconfiguring mon.smithi114 (monmap changed)... 2024-02-16T22:24:15.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:15 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T22:24:15.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:15 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T22:24:15.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:15 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:15.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:15 smithi123 ceph-mon[82331]: Reconfiguring daemon mon.smithi114 on smithi114 2024-02-16T22:24:16.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:16 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:24:16.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:16 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:16.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:16 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:16.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:16 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi114.vcullz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:24:16.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:16 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T22:24:16.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:16 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:16.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:16 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:24:16.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:16 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:16.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:16 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:16.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:16 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi114.vcullz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:24:16.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:16 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T22:24:16.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:16 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:17.691 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:17 smithi114 ceph-mon[94452]: pgmap v18: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:17.691 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:17 smithi114 ceph-mon[94452]: Reconfiguring mgr.smithi114.vcullz (monmap changed)... 2024-02-16T22:24:17.692 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:17 smithi114 ceph-mon[94452]: Reconfiguring daemon mgr.smithi114.vcullz on smithi114 2024-02-16T22:24:17.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:17 smithi123 ceph-mon[82331]: pgmap v18: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:17.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:17 smithi123 ceph-mon[82331]: Reconfiguring mgr.smithi114.vcullz (monmap changed)... 2024-02-16T22:24:17.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:17 smithi123 ceph-mon[82331]: Reconfiguring daemon mgr.smithi114.vcullz on smithi114 2024-02-16T22:24:19.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:18 smithi123 ceph-mon[82331]: pgmap v19: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:19.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:19.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:19.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:18 smithi123 ceph-mon[82331]: Reconfiguring ceph-exporter.smithi114 (monmap changed)... 2024-02-16T22:24:19.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi114", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T22:24:19.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi114", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T22:24:19.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi114", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-02-16T22:24:19.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:18 smithi123 ceph-mon[82331]: Unable to update caps for client.ceph-exporter.smithi114 2024-02-16T22:24:19.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi114"}]: dispatch 2024-02-16T22:24:19.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:19.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:18 smithi123 ceph-mon[82331]: Reconfiguring daemon ceph-exporter.smithi114 on smithi114 2024-02-16T22:24:19.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:18 smithi114 ceph-mon[94452]: pgmap v19: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:19.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:19.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:19.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:18 smithi114 ceph-mon[94452]: Reconfiguring ceph-exporter.smithi114 (monmap changed)... 2024-02-16T22:24:19.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi114", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T22:24:19.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi114", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T22:24:19.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi114", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-02-16T22:24:19.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:18 smithi114 ceph-mon[94452]: Unable to update caps for client.ceph-exporter.smithi114 2024-02-16T22:24:19.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi114"}]: dispatch 2024-02-16T22:24:19.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:19.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:18 smithi114 ceph-mon[94452]: Reconfiguring daemon ceph-exporter.smithi114 on smithi114 2024-02-16T22:24:20.363 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:20 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:20.364 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:20 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:20.364 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:20 smithi114 ceph-mon[94452]: Reconfiguring crash.smithi114 (monmap changed)... 2024-02-16T22:24:20.364 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:20 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi114", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T22:24:20.364 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:20 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:20.364 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:20 smithi114 ceph-mon[94452]: Reconfiguring daemon crash.smithi114 on smithi114 2024-02-16T22:24:20.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:20 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:20.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:20 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:20.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:20 smithi123 ceph-mon[82331]: Reconfiguring crash.smithi114 (monmap changed)... 2024-02-16T22:24:20.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:20 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi114", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T22:24:20.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:20 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:20.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:20 smithi123 ceph-mon[82331]: Reconfiguring daemon crash.smithi114 on smithi114 2024-02-16T22:24:21.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:21 smithi123 ceph-mon[82331]: pgmap v20: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:21.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:21 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:21.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:21 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:21.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:21 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-16T22:24:21.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:21 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:21.599 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:21 smithi114 ceph-mon[94452]: pgmap v20: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:21.599 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:21 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:21.599 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:21 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:21.599 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:21 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-16T22:24:21.599 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:21 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:22.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:22 smithi123 ceph-mon[82331]: Reconfiguring osd.0 (monmap changed)... 2024-02-16T22:24:22.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:22 smithi123 ceph-mon[82331]: Reconfiguring daemon osd.0 on smithi114 2024-02-16T22:24:22.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:22 smithi114 ceph-mon[94452]: Reconfiguring osd.0 (monmap changed)... 2024-02-16T22:24:22.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:22 smithi114 ceph-mon[94452]: Reconfiguring daemon osd.0 on smithi114 2024-02-16T22:24:23.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:23 smithi123 ceph-mon[82331]: pgmap v21: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:23.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:23 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:23.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:23 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:23.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:23 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-16T22:24:23.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:23 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:23.570 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:23 smithi114 ceph-mon[94452]: pgmap v21: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:23.570 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:23 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:23.570 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:23 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:23.570 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:23 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-16T22:24:23.570 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:23 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:24.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:24 smithi114 ceph-mon[94452]: Reconfiguring osd.1 (monmap changed)... 2024-02-16T22:24:24.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:24 smithi114 ceph-mon[94452]: Reconfiguring daemon osd.1 on smithi114 2024-02-16T22:24:24.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:24 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:24.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:24 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:24.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:24 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-16T22:24:24.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:24 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:24.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:24 smithi123 ceph-mon[82331]: Reconfiguring osd.1 (monmap changed)... 2024-02-16T22:24:24.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:24 smithi123 ceph-mon[82331]: Reconfiguring daemon osd.1 on smithi114 2024-02-16T22:24:24.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:24 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:24.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:24 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:24.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:24 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-16T22:24:24.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:24 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:25.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:25 smithi114 ceph-mon[94452]: pgmap v22: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:25.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:25 smithi114 ceph-mon[94452]: Reconfiguring osd.2 (monmap changed)... 2024-02-16T22:24:25.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:25 smithi114 ceph-mon[94452]: Reconfiguring daemon osd.2 on smithi114 2024-02-16T22:24:25.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:25 smithi123 ceph-mon[82331]: pgmap v22: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:25.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:25 smithi123 ceph-mon[82331]: Reconfiguring osd.2 (monmap changed)... 2024-02-16T22:24:25.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:25 smithi123 ceph-mon[82331]: Reconfiguring daemon osd.2 on smithi114 2024-02-16T22:24:27.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:26 smithi114 ceph-mon[94452]: pgmap v23: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:27.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:26 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:27.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:26 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:27.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:26 smithi114 ceph-mon[94452]: Reconfiguring mds.cephfs.smithi114.zzryke (monmap changed)... 2024-02-16T22:24:27.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:26 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi114.zzryke", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:24:27.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:26 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:27.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:26 smithi114 ceph-mon[94452]: Reconfiguring daemon mds.cephfs.smithi114.zzryke on smithi114 2024-02-16T22:24:27.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:26 smithi123 ceph-mon[82331]: pgmap v23: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:27.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:26 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:27.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:26 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:27.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:26 smithi123 ceph-mon[82331]: Reconfiguring mds.cephfs.smithi114.zzryke (monmap changed)... 2024-02-16T22:24:27.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:26 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi114.zzryke", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:24:27.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:26 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:27.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:26 smithi123 ceph-mon[82331]: Reconfiguring daemon mds.cephfs.smithi114.zzryke on smithi114 2024-02-16T22:24:28.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.629+0000 7fe61ee1c700 1 -- 172.21.15.114:0/3723597720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6180ffad0 msgr2=0x7fe6180ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:24:28.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.629+0000 7fe61ee1c700 1 --2- 172.21.15.114:0/3723597720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6180ffad0 0x7fe6180ffef0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe60c009910 tx=0x7fe60c009c20 comp rx=0 tx=0).stop 2024-02-16T22:24:28.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.629+0000 7fe61ee1c700 1 -- 172.21.15.114:0/3723597720 shutdown_connections 2024-02-16T22:24:28.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.629+0000 7fe61ee1c700 1 --2- 172.21.15.114:0/3723597720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe618100cd0 0x7fe618101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:28.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.629+0000 7fe61ee1c700 1 --2- 172.21.15.114:0/3723597720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6180ffad0 0x7fe6180ffef0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:28.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.629+0000 7fe61ee1c700 1 -- 172.21.15.114:0/3723597720 >> 172.21.15.114:0/3723597720 conn(0x7fe6180fb050 msgr2=0x7fe6180fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:24:28.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.629+0000 7fe61ee1c700 1 -- 172.21.15.114:0/3723597720 shutdown_connections 2024-02-16T22:24:28.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.629+0000 7fe61ee1c700 1 -- 172.21.15.114:0/3723597720 wait complete. 2024-02-16T22:24:28.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.630+0000 7fe61ee1c700 1 Processor -- start 2024-02-16T22:24:28.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.630+0000 7fe61ee1c700 1 -- start start 2024-02-16T22:24:28.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.630+0000 7fe61ee1c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6180ffad0 0x7fe618198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:24:28.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.630+0000 7fe61ee1c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe618100cd0 0x7fe618198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:24:28.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.630+0000 7fe61ee1c700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe618199180 con 0x7fe618100cd0 2024-02-16T22:24:28.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.630+0000 7fe61ee1c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6181992c0 con 0x7fe6180ffad0 2024-02-16T22:24:28.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.630+0000 7fe617fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe618100cd0 0x7fe618198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:24:28.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.630+0000 7fe617fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe618100cd0 0x7fe618198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41986/0 (socket says 172.21.15.114:41986) 2024-02-16T22:24:28.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.630+0000 7fe617fff700 1 -- 172.21.15.114:0/4129331534 learned_addr learned my addr 172.21.15.114:0/4129331534 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:24:28.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.630+0000 7fe61cbb8700 1 --2- 172.21.15.114:0/4129331534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6180ffad0 0x7fe618198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:24:28.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.630+0000 7fe617fff700 1 -- 172.21.15.114:0/4129331534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6180ffad0 msgr2=0x7fe618198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:24:28.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.630+0000 7fe617fff700 1 --2- 172.21.15.114:0/4129331534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6180ffad0 0x7fe618198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:28.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.630+0000 7fe617fff700 1 -- 172.21.15.114:0/4129331534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe60c0095f0 con 0x7fe618100cd0 2024-02-16T22:24:28.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.630+0000 7fe61cbb8700 1 --2- 172.21.15.114:0/4129331534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6180ffad0 0x7fe618198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T22:24:28.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.630+0000 7fe617fff700 1 --2- 172.21.15.114:0/4129331534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe618100cd0 0x7fe618198b60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe60400b4b0 tx=0x7fe60400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:24:28.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.631+0000 7fe615ffb700 1 -- 172.21.15.114:0/4129331534 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe604011800 con 0x7fe618100cd0 2024-02-16T22:24:28.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.631+0000 7fe615ffb700 1 -- 172.21.15.114:0/4129331534 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe604011e10 con 0x7fe618100cd0 2024-02-16T22:24:28.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.631+0000 7fe61ee1c700 1 -- 172.21.15.114:0/4129331534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe61819dd70 con 0x7fe618100cd0 2024-02-16T22:24:28.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.631+0000 7fe615ffb700 1 -- 172.21.15.114:0/4129331534 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe60400f2e0 con 0x7fe618100cd0 2024-02-16T22:24:28.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.631+0000 7fe61ee1c700 1 -- 172.21.15.114:0/4129331534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe61819e340 con 0x7fe618100cd0 2024-02-16T22:24:28.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.632+0000 7fe61ee1c700 1 -- 172.21.15.114:0/4129331534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6181086b0 con 0x7fe618100cd0 2024-02-16T22:24:28.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.632+0000 7fe615ffb700 1 -- 172.21.15.114:0/4129331534 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 37) v1 ==== 91597+0+0 (secure 0 0 0) 0x7fe604010380 con 0x7fe618100cd0 2024-02-16T22:24:28.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.632+0000 7fe615ffb700 1 --2- 172.21.15.114:0/4129331534 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe60806d9e0 0x7fe60806fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:24:28.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.633+0000 7fe615ffb700 1 -- 172.21.15.114:0/4129331534 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe60408d150 con 0x7fe618100cd0 2024-02-16T22:24:28.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.633+0000 7fe61cbb8700 1 --2- 172.21.15.114:0/4129331534 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe60806d9e0 0x7fe60806fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:24:28.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.633+0000 7fe61cbb8700 1 --2- 172.21.15.114:0/4129331534 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe60806d9e0 0x7fe60806fea0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00b5c0 tx=0x7fe60c005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:24:28.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.638+0000 7fe615ffb700 1 -- 172.21.15.114:0/4129331534 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe60405a450 con 0x7fe618100cd0 2024-02-16T22:24:28.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:28 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:28.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:28 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:28.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:28 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi114.fdxpey", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:24:28.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:28 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:28.839 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:28 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:28.839 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:28 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:28.840 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:28 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi114.fdxpey", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:24:28.840 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:28 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:28.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.858+0000 7fe61ee1c700 1 -- 172.21.15.114:0/4129331534 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6180694a0 con 0x7fe60806d9e0 2024-02-16T22:24:28.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.859+0000 7fe615ffb700 1 -- 172.21.15.114:0/4129331534 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fe6180694a0 con 0x7fe60806d9e0 2024-02-16T22:24:28.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.861+0000 7fe61ee1c700 1 -- 172.21.15.114:0/4129331534 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe60806d9e0 msgr2=0x7fe60806fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:24:28.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.861+0000 7fe61ee1c700 1 --2- 172.21.15.114:0/4129331534 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe60806d9e0 0x7fe60806fea0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00b5c0 tx=0x7fe60c005e20 comp rx=0 tx=0).stop 2024-02-16T22:24:28.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.861+0000 7fe61ee1c700 1 -- 172.21.15.114:0/4129331534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe618100cd0 msgr2=0x7fe618198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:24:28.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.861+0000 7fe61ee1c700 1 --2- 172.21.15.114:0/4129331534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe618100cd0 0x7fe618198b60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe60400b4b0 tx=0x7fe60400b870 comp rx=0 tx=0).stop 2024-02-16T22:24:28.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.861+0000 7fe61ee1c700 1 -- 172.21.15.114:0/4129331534 shutdown_connections 2024-02-16T22:24:28.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.861+0000 7fe61ee1c700 1 --2- 172.21.15.114:0/4129331534 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe60806d9e0 0x7fe60806fea0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:28.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.861+0000 7fe61ee1c700 1 --2- 172.21.15.114:0/4129331534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6180ffad0 0x7fe618198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:28.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.861+0000 7fe61ee1c700 1 --2- 172.21.15.114:0/4129331534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe618100cd0 0x7fe618198b60 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:28.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.861+0000 7fe61ee1c700 1 -- 172.21.15.114:0/4129331534 >> 172.21.15.114:0/4129331534 conn(0x7fe6180fb050 msgr2=0x7fe618068d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:24:28.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.862+0000 7fe61ee1c700 1 -- 172.21.15.114:0/4129331534 shutdown_connections 2024-02-16T22:24:28.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.862+0000 7fe61ee1c700 1 -- 172.21.15.114:0/4129331534 wait complete. 2024-02-16T22:24:28.874 INFO:teuthology.orchestra.run.smithi114.stdout:true 2024-02-16T22:24:28.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.967+0000 7f6bb0bde700 1 -- 172.21.15.114:0/2949359933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bac100bb0 msgr2=0x7f6bac100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:24:28.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.967+0000 7f6bb0bde700 1 --2- 172.21.15.114:0/2949359933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bac100bb0 0x7f6bac100fd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009910 tx=0x7f6b9c009c20 comp rx=0 tx=0).stop 2024-02-16T22:24:28.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.967+0000 7f6bb0bde700 1 -- 172.21.15.114:0/2949359933 shutdown_connections 2024-02-16T22:24:28.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.967+0000 7f6bb0bde700 1 --2- 172.21.15.114:0/2949359933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bac101db0 0x7f6bac102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:28.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.967+0000 7f6bb0bde700 1 --2- 172.21.15.114:0/2949359933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bac100bb0 0x7f6bac100fd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:28.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.967+0000 7f6bb0bde700 1 -- 172.21.15.114:0/2949359933 >> 172.21.15.114:0/2949359933 conn(0x7f6bac0fc110 msgr2=0x7f6bac0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:24:28.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.967+0000 7f6bb0bde700 1 -- 172.21.15.114:0/2949359933 shutdown_connections 2024-02-16T22:24:28.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.967+0000 7f6bb0bde700 1 -- 172.21.15.114:0/2949359933 wait complete. 2024-02-16T22:24:28.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.968+0000 7f6bb0bde700 1 Processor -- start 2024-02-16T22:24:28.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.968+0000 7f6bb0bde700 1 -- start start 2024-02-16T22:24:28.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.968+0000 7f6bb0bde700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bac100bb0 0x7f6bac196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:24:28.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.968+0000 7f6bb0bde700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bac101db0 0x7f6bac1968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:24:28.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.968+0000 7f6bb0bde700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac196ee0 con 0x7f6bac100bb0 2024-02-16T22:24:28.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.968+0000 7f6bb0bde700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac197020 con 0x7f6bac101db0 2024-02-16T22:24:28.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.968+0000 7f6baa59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bac100bb0 0x7f6bac196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:24:28.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.968+0000 7f6baa59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bac100bb0 0x7f6bac196380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41998/0 (socket says 172.21.15.114:41998) 2024-02-16T22:24:28.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.968+0000 7f6baa59c700 1 -- 172.21.15.114:0/235164030 learned_addr learned my addr 172.21.15.114:0/235164030 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:24:28.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.968+0000 7f6ba9d9b700 1 --2- 172.21.15.114:0/235164030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bac101db0 0x7f6bac1968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:24:28.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.968+0000 7f6baa59c700 1 -- 172.21.15.114:0/235164030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bac101db0 msgr2=0x7f6bac1968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:24:28.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.969+0000 7f6baa59c700 1 --2- 172.21.15.114:0/235164030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bac101db0 0x7f6bac1968c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:28.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.969+0000 7f6baa59c700 1 -- 172.21.15.114:0/235164030 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b9c0095f0 con 0x7f6bac100bb0 2024-02-16T22:24:28.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.969+0000 7f6ba9d9b700 1 --2- 172.21.15.114:0/235164030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bac101db0 0x7f6bac1968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:24:28.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.969+0000 7f6baa59c700 1 --2- 172.21.15.114:0/235164030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bac100bb0 0x7f6bac196380 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00bda0 tx=0x7f6b9c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:24:28.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.969+0000 7f6b9b7fe700 1 -- 172.21.15.114:0/235164030 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b9c01d070 con 0x7f6bac100bb0 2024-02-16T22:24:28.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.969+0000 7f6b9b7fe700 1 -- 172.21.15.114:0/235164030 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6b9c00f460 con 0x7f6bac100bb0 2024-02-16T22:24:28.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.969+0000 7f6bb0bde700 1 -- 172.21.15.114:0/235164030 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bac19ba70 con 0x7f6bac100bb0 2024-02-16T22:24:28.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.969+0000 7f6b9b7fe700 1 -- 172.21.15.114:0/235164030 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b9c021640 con 0x7f6bac100bb0 2024-02-16T22:24:28.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.969+0000 7f6bb0bde700 1 -- 172.21.15.114:0/235164030 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bac19bf60 con 0x7f6bac100bb0 2024-02-16T22:24:28.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.970+0000 7f6bb0bde700 1 -- 172.21.15.114:0/235164030 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bac109710 con 0x7f6bac100bb0 2024-02-16T22:24:28.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.970+0000 7f6b9b7fe700 1 -- 172.21.15.114:0/235164030 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 37) v1 ==== 91597+0+0 (secure 0 0 0) 0x7f6b9c00fa00 con 0x7f6bac100bb0 2024-02-16T22:24:28.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.971+0000 7f6b9b7fe700 1 --2- 172.21.15.114:0/235164030 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6b9406d9e0 0x7f6b9406fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:24:28.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.971+0000 7f6b9b7fe700 1 -- 172.21.15.114:0/235164030 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f6b9c08ead0 con 0x7f6bac100bb0 2024-02-16T22:24:28.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.971+0000 7f6ba9d9b700 1 --2- 172.21.15.114:0/235164030 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6b9406d9e0 0x7f6b9406fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:24:28.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.971+0000 7f6ba9d9b700 1 --2- 172.21.15.114:0/235164030 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6b9406d9e0 0x7f6b9406fea0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0005950 tx=0x7f6ba00058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:24:28.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:28.975+0000 7f6b9b7fe700 1 -- 172.21.15.114:0/235164030 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6b9c05bc40 con 0x7f6bac100bb0 2024-02-16T22:24:29.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.193+0000 7f6bb0bde700 1 -- 172.21.15.114:0/235164030 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6bac19c660 con 0x7f6b9406d9e0 2024-02-16T22:24:29.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.195+0000 7f6b9b7fe700 1 -- 172.21.15.114:0/235164030 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f6bac19c660 con 0x7f6b9406d9e0 2024-02-16T22:24:29.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.197+0000 7f6bb0bde700 1 -- 172.21.15.114:0/235164030 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6b9406d9e0 msgr2=0x7f6b9406fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:24:29.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.197+0000 7f6bb0bde700 1 --2- 172.21.15.114:0/235164030 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6b9406d9e0 0x7f6b9406fea0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0005950 tx=0x7f6ba00058e0 comp rx=0 tx=0).stop 2024-02-16T22:24:29.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.197+0000 7f6bb0bde700 1 -- 172.21.15.114:0/235164030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bac100bb0 msgr2=0x7f6bac196380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:24:29.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.197+0000 7f6bb0bde700 1 --2- 172.21.15.114:0/235164030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bac100bb0 0x7f6bac196380 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00bda0 tx=0x7f6b9c00be80 comp rx=0 tx=0).stop 2024-02-16T22:24:29.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.197+0000 7f6bb0bde700 1 -- 172.21.15.114:0/235164030 shutdown_connections 2024-02-16T22:24:29.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.197+0000 7f6bb0bde700 1 --2- 172.21.15.114:0/235164030 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6b9406d9e0 0x7f6b9406fea0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:29.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.197+0000 7f6bb0bde700 1 --2- 172.21.15.114:0/235164030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bac100bb0 0x7f6bac196380 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:29.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.197+0000 7f6bb0bde700 1 --2- 172.21.15.114:0/235164030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bac101db0 0x7f6bac1968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:29.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.197+0000 7f6bb0bde700 1 -- 172.21.15.114:0/235164030 >> 172.21.15.114:0/235164030 conn(0x7f6bac0fc110 msgr2=0x7f6bac104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:24:29.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.197+0000 7f6bb0bde700 1 -- 172.21.15.114:0/235164030 shutdown_connections 2024-02-16T22:24:29.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.197+0000 7f6bb0bde700 1 -- 172.21.15.114:0/235164030 wait complete. 2024-02-16T22:24:29.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.304+0000 7fb336d51700 1 -- 172.21.15.114:0/2729527369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3300ffb00 msgr2=0x7fb3300fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:24:29.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.304+0000 7fb336d51700 1 --2- 172.21.15.114:0/2729527369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3300ffb00 0x7fb3300fff20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb324009910 tx=0x7fb324009c20 comp rx=0 tx=0).stop 2024-02-16T22:24:29.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.304+0000 7fb336d51700 1 -- 172.21.15.114:0/2729527369 shutdown_connections 2024-02-16T22:24:29.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.304+0000 7fb336d51700 1 --2- 172.21.15.114:0/2729527369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb330100d00 0x7fb330101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:29.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.304+0000 7fb336d51700 1 --2- 172.21.15.114:0/2729527369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3300ffb00 0x7fb3300fff20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:29.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.304+0000 7fb336d51700 1 -- 172.21.15.114:0/2729527369 >> 172.21.15.114:0/2729527369 conn(0x7fb3300fb080 msgr2=0x7fb3300fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:24:29.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.304+0000 7fb336d51700 1 -- 172.21.15.114:0/2729527369 shutdown_connections 2024-02-16T22:24:29.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.304+0000 7fb336d51700 1 -- 172.21.15.114:0/2729527369 wait complete. 2024-02-16T22:24:29.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.304+0000 7fb336d51700 1 Processor -- start 2024-02-16T22:24:29.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.304+0000 7fb336d51700 1 -- start start 2024-02-16T22:24:29.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.304+0000 7fb336d51700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3300ffb00 0x7fb330196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:24:29.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.304+0000 7fb336d51700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb330100d00 0x7fb3301968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:24:29.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.304+0000 7fb336d51700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb330196ee0 con 0x7fb330100d00 2024-02-16T22:24:29.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.304+0000 7fb336d51700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb330197020 con 0x7fb3300ffb00 2024-02-16T22:24:29.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.304+0000 7fb32ffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb330100d00 0x7fb3301968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:24:29.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.304+0000 7fb32ffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb330100d00 0x7fb3301968c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42014/0 (socket says 172.21.15.114:42014) 2024-02-16T22:24:29.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.304+0000 7fb32ffff700 1 -- 172.21.15.114:0/1726939985 learned_addr learned my addr 172.21.15.114:0/1726939985 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:24:29.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.305+0000 7fb334aed700 1 --2- 172.21.15.114:0/1726939985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3300ffb00 0x7fb330196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:24:29.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.305+0000 7fb32ffff700 1 -- 172.21.15.114:0/1726939985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3300ffb00 msgr2=0x7fb330196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:24:29.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.305+0000 7fb32ffff700 1 --2- 172.21.15.114:0/1726939985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3300ffb00 0x7fb330196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:29.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.305+0000 7fb32ffff700 1 -- 172.21.15.114:0/1726939985 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3240095f0 con 0x7fb330100d00 2024-02-16T22:24:29.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.305+0000 7fb32ffff700 1 --2- 172.21.15.114:0/1726939985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb330100d00 0x7fb3301968c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb31c00e920 tx=0x7fb31c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:24:29.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.305+0000 7fb32dffb700 1 -- 172.21.15.114:0/1726939985 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb31c00c9d0 con 0x7fb330100d00 2024-02-16T22:24:29.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.305+0000 7fb32dffb700 1 -- 172.21.15.114:0/1726939985 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb31c010430 con 0x7fb330100d00 2024-02-16T22:24:29.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.305+0000 7fb336d51700 1 -- 172.21.15.114:0/1726939985 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb33019bad0 con 0x7fb330100d00 2024-02-16T22:24:29.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.305+0000 7fb32dffb700 1 -- 172.21.15.114:0/1726939985 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb31c018650 con 0x7fb330100d00 2024-02-16T22:24:29.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.305+0000 7fb336d51700 1 -- 172.21.15.114:0/1726939985 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33019c020 con 0x7fb330100d00 2024-02-16T22:24:29.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.306+0000 7fb336d51700 1 -- 172.21.15.114:0/1726939985 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb330108660 con 0x7fb330100d00 2024-02-16T22:24:29.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.307+0000 7fb32dffb700 1 -- 172.21.15.114:0/1726939985 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 37) v1 ==== 91597+0+0 (secure 0 0 0) 0x7fb31c0105a0 con 0x7fb330100d00 2024-02-16T22:24:29.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.307+0000 7fb32dffb700 1 --2- 172.21.15.114:0/1726939985 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb32006d9e0 0x7fb32006fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:24:29.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.307+0000 7fb334aed700 1 --2- 172.21.15.114:0/1726939985 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb32006d9e0 0x7fb32006fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:24:29.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.307+0000 7fb32dffb700 1 -- 172.21.15.114:0/1726939985 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fb31c014070 con 0x7fb330100d00 2024-02-16T22:24:29.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.308+0000 7fb334aed700 1 --2- 172.21.15.114:0/1726939985 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb32006d9e0 0x7fb32006fea0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb32400b6f0 tx=0x7fb324005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:24:29.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.312+0000 7fb32dffb700 1 -- 172.21.15.114:0/1726939985 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb31c052d20 con 0x7fb330100d00 2024-02-16T22:24:29.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.513+0000 7fb336d51700 1 -- 172.21.15.114:0/1726939985 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb330069480 con 0x7fb32006d9e0 2024-02-16T22:24:29.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.524+0000 7fb32dffb700 1 -- 172.21.15.114:0/1726939985 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fb330069480 con 0x7fb32006d9e0 2024-02-16T22:24:29.526 INFO:teuthology.orchestra.run.smithi114.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:24:29.526 INFO:teuthology.orchestra.run.smithi114.stdout:alertmanager.smithi114 smithi114 *:9093,9094 running (10m) 30s ago 11m 24.6M - 0.25.0 c8568f914cd2 e7404fea400e 2024-02-16T22:24:29.526 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi114 smithi114 running (12m) 30s ago 12m 7931k - 18.2.1 94eadd990e0a 7fd07162cd96 2024-02-16T22:24:29.526 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi123 smithi123 running (11m) 19s ago 11m 7629k - 18.2.1 94eadd990e0a 7687f5e99621 2024-02-16T22:24:29.526 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi114 smithi114 running (12m) 30s ago 12m 6660k - 18.2.1 94eadd990e0a fc8fbfe4fdd5 2024-02-16T22:24:29.526 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi123 smithi123 running (11m) 19s ago 11m 6665k - 18.2.1 94eadd990e0a a6249642b9bb 2024-02-16T22:24:29.526 INFO:teuthology.orchestra.run.smithi114.stdout:grafana.smithi114 smithi114 *:3000 running (10m) 30s ago 11m 89.9M - 9.4.7 954c08fa6188 db1c26c3bc20 2024-02-16T22:24:29.526 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.fdxpey smithi114 running (6m) 30s ago 6m 17.1M - 18.2.1 94eadd990e0a 704b40c144fb 2024-02-16T22:24:29.527 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.zzryke smithi114 running (6m) 30s ago 6m 183M - 18.2.1 94eadd990e0a 1f795d23c9a4 2024-02-16T22:24:29.527 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.gopcfa smithi123 running (6m) 19s ago 6m 17.0M - 18.2.1 94eadd990e0a 166134b7532a 2024-02-16T22:24:29.527 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.vhlqbe smithi123 running (6m) 19s ago 6m 18.5M - 18.2.1 94eadd990e0a 243c8bb96fa8 2024-02-16T22:24:29.527 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi114.vcullz smithi114 *:8443,9283,8765 running (2m) 30s ago 13m 475M - 19.0.0-1450-gf78a58c0 f081f847070a 14ebeb7888c0 2024-02-16T22:24:29.527 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi123.inzgjd smithi123 *:8443,9283,8765 running (73s) 19s ago 10m 448M - 19.0.0-1450-gf78a58c0 f081f847070a f10aab9faf9e 2024-02-16T22:24:29.527 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi114 smithi114 running (55s) 30s ago 13m 30.6M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 58ac06372523 2024-02-16T22:24:29.527 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi123 smithi123 running (23s) 19s ago 10m 24.5M 2048M 19.0.0-1450-gf78a58c0 f081f847070a f76485e0ea24 2024-02-16T22:24:29.527 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi114 smithi114 *:9100 running (12m) 30s ago 12m 15.5M - 1.5.0 0da6a335fe13 744a090e06d0 2024-02-16T22:24:29.527 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi123 smithi123 *:9100 running (10m) 19s ago 10m 15.5M - 1.5.0 0da6a335fe13 3994b8c29b2e 2024-02-16T22:24:29.527 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 smithi114 running (10m) 30s ago 10m 211M 1216M 18.2.1 94eadd990e0a 24ca2e9714e0 2024-02-16T22:24:29.527 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 smithi114 running (9m) 30s ago 9m 222M 1216M 18.2.1 94eadd990e0a 32bb191b88e7 2024-02-16T22:24:29.527 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 smithi114 running (9m) 30s ago 9m 189M 1216M 18.2.1 94eadd990e0a 6608718d0818 2024-02-16T22:24:29.527 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 smithi123 running (8m) 19s ago 8m 258M 2240M 18.2.1 94eadd990e0a 575db379de3d 2024-02-16T22:24:29.527 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 smithi123 running (8m) 19s ago 8m 219M 2240M 18.2.1 94eadd990e0a 7e266866cd9c 2024-02-16T22:24:29.527 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 smithi123 running (7m) 19s ago 7m 198M 2240M 18.2.1 94eadd990e0a 24e5bef19fda 2024-02-16T22:24:29.527 INFO:teuthology.orchestra.run.smithi114.stdout:prometheus.smithi114 smithi114 *:9095 running (81s) 30s ago 11m 50.7M - 2.43.0 a07b618ecd1d 37aac985cdd7 2024-02-16T22:24:29.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.526+0000 7fb336d51700 1 -- 172.21.15.114:0/1726939985 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb32006d9e0 msgr2=0x7fb32006fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:24:29.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.526+0000 7fb336d51700 1 --2- 172.21.15.114:0/1726939985 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb32006d9e0 0x7fb32006fea0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb32400b6f0 tx=0x7fb324005940 comp rx=0 tx=0).stop 2024-02-16T22:24:29.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.526+0000 7fb336d51700 1 -- 172.21.15.114:0/1726939985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb330100d00 msgr2=0x7fb3301968c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:24:29.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.526+0000 7fb336d51700 1 --2- 172.21.15.114:0/1726939985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb330100d00 0x7fb3301968c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb31c00e920 tx=0x7fb31c00ec30 comp rx=0 tx=0).stop 2024-02-16T22:24:29.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.526+0000 7fb336d51700 1 -- 172.21.15.114:0/1726939985 shutdown_connections 2024-02-16T22:24:29.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.526+0000 7fb336d51700 1 --2- 172.21.15.114:0/1726939985 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb32006d9e0 0x7fb32006fea0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:29.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.526+0000 7fb336d51700 1 --2- 172.21.15.114:0/1726939985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3300ffb00 0x7fb330196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:29.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.526+0000 7fb336d51700 1 --2- 172.21.15.114:0/1726939985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb330100d00 0x7fb3301968c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:29.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.526+0000 7fb336d51700 1 -- 172.21.15.114:0/1726939985 >> 172.21.15.114:0/1726939985 conn(0x7fb3300fb080 msgr2=0x7fb330068d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:24:29.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.526+0000 7fb336d51700 1 -- 172.21.15.114:0/1726939985 shutdown_connections 2024-02-16T22:24:29.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.526+0000 7fb336d51700 1 -- 172.21.15.114:0/1726939985 wait complete. 2024-02-16T22:24:29.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.629+0000 7f96f72a1700 1 -- 172.21.15.114:0/998730362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96f0100bc0 msgr2=0x7f96f0100fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:24:29.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.629+0000 7f96f72a1700 1 --2- 172.21.15.114:0/998730362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96f0100bc0 0x7f96f0100fe0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f96ec009910 tx=0x7f96ec009c20 comp rx=0 tx=0).stop 2024-02-16T22:24:29.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.629+0000 7f96f72a1700 1 -- 172.21.15.114:0/998730362 shutdown_connections 2024-02-16T22:24:29.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.629+0000 7f96f72a1700 1 --2- 172.21.15.114:0/998730362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96f0101dc0 0x7f96f0102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:29.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.629+0000 7f96f72a1700 1 --2- 172.21.15.114:0/998730362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96f0100bc0 0x7f96f0100fe0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:29.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.629+0000 7f96f72a1700 1 -- 172.21.15.114:0/998730362 >> 172.21.15.114:0/998730362 conn(0x7f96f00fc140 msgr2=0x7f96f00fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:24:29.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.630+0000 7f96f72a1700 1 -- 172.21.15.114:0/998730362 shutdown_connections 2024-02-16T22:24:29.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.630+0000 7f96f72a1700 1 -- 172.21.15.114:0/998730362 wait complete. 2024-02-16T22:24:29.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.630+0000 7f96f72a1700 1 Processor -- start 2024-02-16T22:24:29.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.630+0000 7f96f72a1700 1 -- start start 2024-02-16T22:24:29.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.630+0000 7f96f72a1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96f0100bc0 0x7f96f0196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:24:29.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.630+0000 7f96f72a1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96f0101dc0 0x7f96f01968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:24:29.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.630+0000 7f96f72a1700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f0196ef0 con 0x7f96f0101dc0 2024-02-16T22:24:29.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.630+0000 7f96f72a1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f0197030 con 0x7f96f0100bc0 2024-02-16T22:24:29.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.630+0000 7f96f483c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96f0101dc0 0x7f96f01968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:24:29.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.630+0000 7f96f503d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96f0100bc0 0x7f96f0196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:24:29.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.630+0000 7f96f483c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96f0101dc0 0x7f96f01968d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42040/0 (socket says 172.21.15.114:42040) 2024-02-16T22:24:29.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.630+0000 7f96f483c700 1 -- 172.21.15.114:0/449611891 learned_addr learned my addr 172.21.15.114:0/449611891 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:24:29.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.631+0000 7f96f483c700 1 -- 172.21.15.114:0/449611891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96f0100bc0 msgr2=0x7f96f0196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:24:29.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.631+0000 7f96f483c700 1 --2- 172.21.15.114:0/449611891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96f0100bc0 0x7f96f0196390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:29.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.631+0000 7f96f483c700 1 -- 172.21.15.114:0/449611891 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ec0095f0 con 0x7f96f0101dc0 2024-02-16T22:24:29.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.631+0000 7f96f503d700 1 --2- 172.21.15.114:0/449611891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96f0100bc0 0x7f96f0196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:24:29.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.631+0000 7f96f483c700 1 --2- 172.21.15.114:0/449611891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96f0101dc0 0x7f96f01968d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f96e000e920 tx=0x7f96e000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:24:29.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.631+0000 7f96e67fc700 1 -- 172.21.15.114:0/449611891 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96e000c9d0 con 0x7f96f0101dc0 2024-02-16T22:24:29.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.631+0000 7f96e67fc700 1 -- 172.21.15.114:0/449611891 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f96e0010430 con 0x7f96f0101dc0 2024-02-16T22:24:29.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.631+0000 7f96e67fc700 1 -- 172.21.15.114:0/449611891 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96e0018650 con 0x7f96f0101dc0 2024-02-16T22:24:29.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.631+0000 7f96f72a1700 1 -- 172.21.15.114:0/449611891 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f019bae0 con 0x7f96f0101dc0 2024-02-16T22:24:29.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.631+0000 7f96f72a1700 1 -- 172.21.15.114:0/449611891 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f019c030 con 0x7f96f0101dc0 2024-02-16T22:24:29.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.632+0000 7f96f72a1700 1 -- 172.21.15.114:0/449611891 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f0109720 con 0x7f96f0101dc0 2024-02-16T22:24:29.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.633+0000 7f96e67fc700 1 -- 172.21.15.114:0/449611891 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 37) v1 ==== 91597+0+0 (secure 0 0 0) 0x7f96e00105a0 con 0x7f96f0101dc0 2024-02-16T22:24:29.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.633+0000 7f96e67fc700 1 --2- 172.21.15.114:0/449611891 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f96dc06d9e0 0x7f96dc06fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:24:29.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.633+0000 7f96f503d700 1 --2- 172.21.15.114:0/449611891 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f96dc06d9e0 0x7f96dc06fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:24:29.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.633+0000 7f96e67fc700 1 -- 172.21.15.114:0/449611891 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f96e0014070 con 0x7f96f0101dc0 2024-02-16T22:24:29.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.633+0000 7f96f503d700 1 --2- 172.21.15.114:0/449611891 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f96dc06d9e0 0x7f96dc06fea0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f96ec00b6f0 tx=0x7f96ec005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:24:29.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.638+0000 7f96e67fc700 1 -- 172.21.15.114:0/449611891 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f96e0052d20 con 0x7f96f0101dc0 2024-02-16T22:24:29.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:29 smithi123 ceph-mon[82331]: Reconfiguring mds.cephfs.smithi114.fdxpey (monmap changed)... 2024-02-16T22:24:29.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:29 smithi123 ceph-mon[82331]: Reconfiguring daemon mds.cephfs.smithi114.fdxpey on smithi114 2024-02-16T22:24:29.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:29 smithi123 ceph-mon[82331]: pgmap v24: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:29.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:29 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:29.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:29 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:29.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:29 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi123", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T22:24:29.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:29 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi123", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T22:24:29.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:29 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi123", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-02-16T22:24:29.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:29 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi123"}]: dispatch 2024-02-16T22:24:29.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:29 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:29.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.911+0000 7f96f72a1700 1 -- 172.21.15.114:0/449611891 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f96f019c640 con 0x7f96f0101dc0 2024-02-16T22:24:29.914 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:29 smithi114 ceph-mon[94452]: Reconfiguring mds.cephfs.smithi114.fdxpey (monmap changed)... 2024-02-16T22:24:29.914 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:29 smithi114 ceph-mon[94452]: Reconfiguring daemon mds.cephfs.smithi114.fdxpey on smithi114 2024-02-16T22:24:29.914 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:29 smithi114 ceph-mon[94452]: pgmap v24: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:29.914 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:29 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:29.914 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:29 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:29.914 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:29 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi123", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T22:24:29.914 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:29 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi123", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T22:24:29.914 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:29 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi123", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-02-16T22:24:29.914 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:29 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi123"}]: dispatch 2024-02-16T22:24:29.914 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:29 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:29.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.912+0000 7f96e67fc700 1 -- 172.21.15.114:0/449611891 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7f96e005aba0 con 0x7f96f0101dc0 2024-02-16T22:24:29.915 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:24:29.915 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": { 2024-02-16T22:24:29.915 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:24:29.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:24:29.915 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": { 2024-02-16T22:24:29.915 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:24:29.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:24:29.915 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": { 2024-02-16T22:24:29.915 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-02-16T22:24:29.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:24:29.915 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-02-16T22:24:29.916 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-02-16T22:24:29.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:24:29.916 INFO:teuthology.orchestra.run.smithi114.stdout: "overall": { 2024-02-16T22:24:29.916 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 10, 2024-02-16T22:24:29.916 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T22:24:29.916 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-02-16T22:24:29.916 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:24:29.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.915+0000 7f96f72a1700 1 -- 172.21.15.114:0/449611891 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f96dc06d9e0 msgr2=0x7f96dc06fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:24:29.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.915+0000 7f96f72a1700 1 --2- 172.21.15.114:0/449611891 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f96dc06d9e0 0x7f96dc06fea0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f96ec00b6f0 tx=0x7f96ec005940 comp rx=0 tx=0).stop 2024-02-16T22:24:29.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.915+0000 7f96f72a1700 1 -- 172.21.15.114:0/449611891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96f0101dc0 msgr2=0x7f96f01968d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:24:29.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.915+0000 7f96f72a1700 1 --2- 172.21.15.114:0/449611891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96f0101dc0 0x7f96f01968d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f96e000e920 tx=0x7f96e000ec30 comp rx=0 tx=0).stop 2024-02-16T22:24:29.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.915+0000 7f96f72a1700 1 -- 172.21.15.114:0/449611891 shutdown_connections 2024-02-16T22:24:29.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.915+0000 7f96f72a1700 1 --2- 172.21.15.114:0/449611891 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f96dc06d9e0 0x7f96dc06fea0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:29.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.915+0000 7f96f72a1700 1 --2- 172.21.15.114:0/449611891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96f0100bc0 0x7f96f0196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:29.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.915+0000 7f96f72a1700 1 --2- 172.21.15.114:0/449611891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96f0101dc0 0x7f96f01968d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:29.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.915+0000 7f96f72a1700 1 -- 172.21.15.114:0/449611891 >> 172.21.15.114:0/449611891 conn(0x7f96f00fc140 msgr2=0x7f96f0104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:24:29.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.915+0000 7f96f72a1700 1 -- 172.21.15.114:0/449611891 shutdown_connections 2024-02-16T22:24:29.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:29.915+0000 7f96f72a1700 1 -- 172.21.15.114:0/449611891 wait complete. 2024-02-16T22:24:30.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.020+0000 7f0b52239700 1 -- 172.21.15.114:0/1599264259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b4c101f30 msgr2=0x7f0b4c102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:24:30.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.020+0000 7f0b52239700 1 --2- 172.21.15.114:0/1599264259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b4c101f30 0x7f0b4c102390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0b40009910 tx=0x7f0b40009c20 comp rx=0 tx=0).stop 2024-02-16T22:24:30.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.020+0000 7f0b52239700 1 -- 172.21.15.114:0/1599264259 shutdown_connections 2024-02-16T22:24:30.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.020+0000 7f0b52239700 1 --2- 172.21.15.114:0/1599264259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b4c101f30 0x7f0b4c102390 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:30.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.020+0000 7f0b52239700 1 --2- 172.21.15.114:0/1599264259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b4c0feeb0 0x7f0b4c0ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:30.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.020+0000 7f0b52239700 1 -- 172.21.15.114:0/1599264259 >> 172.21.15.114:0/1599264259 conn(0x7f0b4c0fa7f0 msgr2=0x7f0b4c0fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:24:30.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.020+0000 7f0b52239700 1 -- 172.21.15.114:0/1599264259 shutdown_connections 2024-02-16T22:24:30.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.020+0000 7f0b52239700 1 -- 172.21.15.114:0/1599264259 wait complete. 2024-02-16T22:24:30.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.021+0000 7f0b52239700 1 Processor -- start 2024-02-16T22:24:30.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.021+0000 7f0b52239700 1 -- start start 2024-02-16T22:24:30.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.021+0000 7f0b52239700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b4c0feeb0 0x7f0b4c196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:24:30.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.021+0000 7f0b52239700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b4c101f30 0x7f0b4c1968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:24:30.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.021+0000 7f0b52239700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c196ec0 con 0x7f0b4c0feeb0 2024-02-16T22:24:30.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.021+0000 7f0b52239700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c197000 con 0x7f0b4c101f30 2024-02-16T22:24:30.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.021+0000 7f0b4b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b4c0feeb0 0x7f0b4c196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:24:30.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.021+0000 7f0b4b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b4c0feeb0 0x7f0b4c196360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42058/0 (socket says 172.21.15.114:42058) 2024-02-16T22:24:30.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.021+0000 7f0b4b7fe700 1 -- 172.21.15.114:0/875011151 learned_addr learned my addr 172.21.15.114:0/875011151 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:24:30.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.021+0000 7f0b4affd700 1 --2- 172.21.15.114:0/875011151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b4c101f30 0x7f0b4c1968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:24:30.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.022+0000 7f0b4b7fe700 1 -- 172.21.15.114:0/875011151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b4c101f30 msgr2=0x7f0b4c1968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:24:30.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.022+0000 7f0b4b7fe700 1 --2- 172.21.15.114:0/875011151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b4c101f30 0x7f0b4c1968a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:30.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.022+0000 7f0b4b7fe700 1 -- 172.21.15.114:0/875011151 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b400095f0 con 0x7f0b4c0feeb0 2024-02-16T22:24:30.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.022+0000 7f0b4b7fe700 1 --2- 172.21.15.114:0/875011151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b4c0feeb0 0x7f0b4c196360 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00d900 tx=0x7f0b3c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:24:30.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.022+0000 7f0b48ff9700 1 -- 172.21.15.114:0/875011151 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b3c0041f0 con 0x7f0b4c0feeb0 2024-02-16T22:24:30.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.022+0000 7f0b48ff9700 1 -- 172.21.15.114:0/875011151 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0b3c004350 con 0x7f0b4c0feeb0 2024-02-16T22:24:30.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.022+0000 7f0b48ff9700 1 -- 172.21.15.114:0/875011151 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b3c003d20 con 0x7f0b4c0feeb0 2024-02-16T22:24:30.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.022+0000 7f0b52239700 1 -- 172.21.15.114:0/875011151 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b4c19bab0 con 0x7f0b4c0feeb0 2024-02-16T22:24:30.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.022+0000 7f0b52239700 1 -- 172.21.15.114:0/875011151 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b4c19c000 con 0x7f0b4c0feeb0 2024-02-16T22:24:30.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.023+0000 7f0b52239700 1 -- 172.21.15.114:0/875011151 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b4c109730 con 0x7f0b4c0feeb0 2024-02-16T22:24:30.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.023+0000 7f0b48ff9700 1 -- 172.21.15.114:0/875011151 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 37) v1 ==== 91597+0+0 (secure 0 0 0) 0x7f0b3c009730 con 0x7f0b4c0feeb0 2024-02-16T22:24:30.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.024+0000 7f0b48ff9700 1 --2- 172.21.15.114:0/875011151 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f0b3406d9e0 0x7f0b3406fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:24:30.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.024+0000 7f0b48ff9700 1 -- 172.21.15.114:0/875011151 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f0b3c08d5d0 con 0x7f0b4c0feeb0 2024-02-16T22:24:30.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.024+0000 7f0b4affd700 1 --2- 172.21.15.114:0/875011151 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f0b3406d9e0 0x7f0b3406fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:24:30.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.024+0000 7f0b4affd700 1 --2- 172.21.15.114:0/875011151 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f0b3406d9e0 0x7f0b3406fea0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0b40006010 tx=0x7f0b4000b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:24:30.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.029+0000 7f0b48ff9700 1 -- 172.21.15.114:0/875011151 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0b3c05a740 con 0x7f0b4c0feeb0 2024-02-16T22:24:30.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.268+0000 7f0b52239700 1 -- 172.21.15.114:0/875011151 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0b4c19c610 con 0x7f0b4c0feeb0 2024-02-16T22:24:30.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.268+0000 7f0b48ff9700 1 -- 172.21.15.114:0/875011151 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1846 (secure 0 0 0) 0x7f0b3c003e80 con 0x7f0b4c0feeb0 2024-02-16T22:24:30.270 INFO:teuthology.orchestra.run.smithi114.stdout:e14 2024-02-16T22:24:30.270 INFO:teuthology.orchestra.run.smithi114.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:24:30.270 INFO:teuthology.orchestra.run.smithi114.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:24:30.270 INFO:teuthology.orchestra.run.smithi114.stdout:legacy client fscid: 1 2024-02-16T22:24:30.270 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:24:30.270 INFO:teuthology.orchestra.run.smithi114.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:24:30.271 INFO:teuthology.orchestra.run.smithi114.stdout:fs_name cephfs 2024-02-16T22:24:30.271 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 12 2024-02-16T22:24:30.271 INFO:teuthology.orchestra.run.smithi114.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:24:30.271 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-02-16T22:18:02.108825+0000 2024-02-16T22:24:30.271 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-02-16T22:18:16.413371+0000 2024-02-16T22:24:30.271 INFO:teuthology.orchestra.run.smithi114.stdout:tableserver 0 2024-02-16T22:24:30.271 INFO:teuthology.orchestra.run.smithi114.stdout:root 0 2024-02-16T22:24:30.271 INFO:teuthology.orchestra.run.smithi114.stdout:session_timeout 60 2024-02-16T22:24:30.271 INFO:teuthology.orchestra.run.smithi114.stdout:session_autoclose 300 2024-02-16T22:24:30.271 INFO:teuthology.orchestra.run.smithi114.stdout:max_file_size 1099511627776 2024-02-16T22:24:30.271 INFO:teuthology.orchestra.run.smithi114.stdout:max_xattr_size 65536 2024-02-16T22:24:30.271 INFO:teuthology.orchestra.run.smithi114.stdout:required_client_features {} 2024-02-16T22:24:30.271 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure 0 2024-02-16T22:24:30.271 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure_osd_epoch 0 2024-02-16T22:24:30.271 INFO:teuthology.orchestra.run.smithi114.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:24:30.271 INFO:teuthology.orchestra.run.smithi114.stdout:max_mds 1 2024-02-16T22:24:30.272 INFO:teuthology.orchestra.run.smithi114.stdout:in 0 2024-02-16T22:24:30.272 INFO:teuthology.orchestra.run.smithi114.stdout:up {0=14468} 2024-02-16T22:24:30.272 INFO:teuthology.orchestra.run.smithi114.stdout:failed 2024-02-16T22:24:30.272 INFO:teuthology.orchestra.run.smithi114.stdout:damaged 2024-02-16T22:24:30.272 INFO:teuthology.orchestra.run.smithi114.stdout:stopped 2024-02-16T22:24:30.272 INFO:teuthology.orchestra.run.smithi114.stdout:data_pools [3] 2024-02-16T22:24:30.272 INFO:teuthology.orchestra.run.smithi114.stdout:metadata_pool 2 2024-02-16T22:24:30.272 INFO:teuthology.orchestra.run.smithi114.stdout:inline_data enabled 2024-02-16T22:24:30.272 INFO:teuthology.orchestra.run.smithi114.stdout:balancer 2024-02-16T22:24:30.272 INFO:teuthology.orchestra.run.smithi114.stdout:bal_rank_mask -1 2024-02-16T22:24:30.272 INFO:teuthology.orchestra.run.smithi114.stdout:standby_count_wanted 1 2024-02-16T22:24:30.272 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.zzryke{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.114:6826/277953535,v1:172.21.15.114:6827/277953535] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:24:30.272 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:24:30.272 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:24:30.272 INFO:teuthology.orchestra.run.smithi114.stdout:Standby daemons: 2024-02-16T22:24:30.273 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:24:30.273 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.fdxpey{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:24:30.273 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.vhlqbe{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:24:30.273 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.gopcfa{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:24:30.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.270+0000 7f0b52239700 1 -- 172.21.15.114:0/875011151 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f0b3406d9e0 msgr2=0x7f0b3406fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:24:30.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.270+0000 7f0b52239700 1 --2- 172.21.15.114:0/875011151 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f0b3406d9e0 0x7f0b3406fea0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0b40006010 tx=0x7f0b4000b540 comp rx=0 tx=0).stop 2024-02-16T22:24:30.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.270+0000 7f0b52239700 1 -- 172.21.15.114:0/875011151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b4c0feeb0 msgr2=0x7f0b4c196360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:24:30.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.270+0000 7f0b52239700 1 --2- 172.21.15.114:0/875011151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b4c0feeb0 0x7f0b4c196360 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00d900 tx=0x7f0b3c00dc10 comp rx=0 tx=0).stop 2024-02-16T22:24:30.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.270+0000 7f0b52239700 1 -- 172.21.15.114:0/875011151 shutdown_connections 2024-02-16T22:24:30.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.270+0000 7f0b52239700 1 --2- 172.21.15.114:0/875011151 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f0b3406d9e0 0x7f0b3406fea0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:30.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.270+0000 7f0b52239700 1 --2- 172.21.15.114:0/875011151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b4c0feeb0 0x7f0b4c196360 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:30.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.270+0000 7f0b52239700 1 --2- 172.21.15.114:0/875011151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b4c101f30 0x7f0b4c1968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:30.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.270+0000 7f0b52239700 1 -- 172.21.15.114:0/875011151 >> 172.21.15.114:0/875011151 conn(0x7f0b4c0fa7f0 msgr2=0x7f0b4c105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:24:30.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.270+0000 7f0b52239700 1 -- 172.21.15.114:0/875011151 shutdown_connections 2024-02-16T22:24:30.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.270+0000 7f0b52239700 1 -- 172.21.15.114:0/875011151 wait complete. 2024-02-16T22:24:30.274 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 14 2024-02-16T22:24:30.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.374+0000 7eff01e45700 1 -- 172.21.15.114:0/1191337515 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efefc103f90 msgr2=0x7efefc106380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:24:30.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.374+0000 7eff01e45700 1 --2- 172.21.15.114:0/1191337515 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efefc103f90 0x7efefc106380 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efeec009910 tx=0x7efeec009c20 comp rx=0 tx=0).stop 2024-02-16T22:24:30.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.375+0000 7eff01e45700 1 -- 172.21.15.114:0/1191337515 shutdown_connections 2024-02-16T22:24:30.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.375+0000 7eff01e45700 1 --2- 172.21.15.114:0/1191337515 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efefc103f90 0x7efefc106380 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:30.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.375+0000 7eff01e45700 1 --2- 172.21.15.114:0/1191337515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efefc101660 0x7efefc103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:30.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.375+0000 7eff01e45700 1 -- 172.21.15.114:0/1191337515 >> 172.21.15.114:0/1191337515 conn(0x7efefc0fb050 msgr2=0x7efefc0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:24:30.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.375+0000 7eff01e45700 1 -- 172.21.15.114:0/1191337515 shutdown_connections 2024-02-16T22:24:30.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.375+0000 7eff01e45700 1 -- 172.21.15.114:0/1191337515 wait complete. 2024-02-16T22:24:30.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.375+0000 7eff01e45700 1 Processor -- start 2024-02-16T22:24:30.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.375+0000 7eff01e45700 1 -- start start 2024-02-16T22:24:30.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.375+0000 7eff01e45700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efefc101660 0x7efefc196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:24:30.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.375+0000 7eff01e45700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efefc103f90 0x7efefc1968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:24:30.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.375+0000 7eff01e45700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efefc196ec0 con 0x7efefc103f90 2024-02-16T22:24:30.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.375+0000 7eff01e45700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efefc197000 con 0x7efefc101660 2024-02-16T22:24:30.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.376+0000 7efefaffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efefc103f90 0x7efefc1968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:24:30.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.376+0000 7efefaffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efefc103f90 0x7efefc1968a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42072/0 (socket says 172.21.15.114:42072) 2024-02-16T22:24:30.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.376+0000 7efefaffd700 1 -- 172.21.15.114:0/2237111116 learned_addr learned my addr 172.21.15.114:0/2237111116 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:24:30.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.376+0000 7efefb7fe700 1 --2- 172.21.15.114:0/2237111116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efefc101660 0x7efefc196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:24:30.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.376+0000 7efefaffd700 1 -- 172.21.15.114:0/2237111116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efefc101660 msgr2=0x7efefc196360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:24:30.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.376+0000 7efefaffd700 1 --2- 172.21.15.114:0/2237111116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efefc101660 0x7efefc196360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:30.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.376+0000 7efefaffd700 1 -- 172.21.15.114:0/2237111116 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeec0095f0 con 0x7efefc103f90 2024-02-16T22:24:30.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.376+0000 7efefaffd700 1 --2- 172.21.15.114:0/2237111116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efefc103f90 0x7efefc1968a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efeec000c00 tx=0x7efeec00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:24:30.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.376+0000 7efef8ff9700 1 -- 172.21.15.114:0/2237111116 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efeec01d070 con 0x7efefc103f90 2024-02-16T22:24:30.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.376+0000 7efef8ff9700 1 -- 172.21.15.114:0/2237111116 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efeec005dc0 con 0x7efefc103f90 2024-02-16T22:24:30.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.376+0000 7eff01e45700 1 -- 172.21.15.114:0/2237111116 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efefc19ba50 con 0x7efefc103f90 2024-02-16T22:24:30.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.376+0000 7efef8ff9700 1 -- 172.21.15.114:0/2237111116 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efeec0226a0 con 0x7efefc103f90 2024-02-16T22:24:30.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.376+0000 7eff01e45700 1 -- 172.21.15.114:0/2237111116 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efefc19bf40 con 0x7efefc103f90 2024-02-16T22:24:30.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.377+0000 7eff01e45700 1 -- 172.21.15.114:0/2237111116 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efefc190540 con 0x7efefc103f90 2024-02-16T22:24:30.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.378+0000 7efef8ff9700 1 -- 172.21.15.114:0/2237111116 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 37) v1 ==== 91597+0+0 (secure 0 0 0) 0x7efeec0038f0 con 0x7efefc103f90 2024-02-16T22:24:30.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.378+0000 7efef8ff9700 1 --2- 172.21.15.114:0/2237111116 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7efee806d9e0 0x7efee806fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:24:30.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.378+0000 7efef8ff9700 1 -- 172.21.15.114:0/2237111116 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7efeec08eb90 con 0x7efefc103f90 2024-02-16T22:24:30.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.378+0000 7efefb7fe700 1 --2- 172.21.15.114:0/2237111116 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7efee806d9e0 0x7efee806fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:24:30.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.379+0000 7efefb7fe700 1 --2- 172.21.15.114:0/2237111116 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7efee806d9e0 0x7efee806fea0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efee4009c60 tx=0x7efee4009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:24:30.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.383+0000 7efef8ff9700 1 -- 172.21.15.114:0/2237111116 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efeec05bd80 con 0x7efefc103f90 2024-02-16T22:24:30.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.602+0000 7eff01e45700 1 -- 172.21.15.114:0/2237111116 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efefc0611d0 con 0x7efee806d9e0 2024-02-16T22:24:30.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.603+0000 7efef8ff9700 1 -- 172.21.15.114:0/2237111116 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7efefc0611d0 con 0x7efee806d9e0 2024-02-16T22:24:30.605 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:24:30.605 INFO:teuthology.orchestra.run.smithi114.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:24:30.605 INFO:teuthology.orchestra.run.smithi114.stdout: "in_progress": true, 2024-02-16T22:24:30.605 INFO:teuthology.orchestra.run.smithi114.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:24:30.605 INFO:teuthology.orchestra.run.smithi114.stdout: "services_complete": [ 2024-02-16T22:24:30.605 INFO:teuthology.orchestra.run.smithi114.stdout: "mon", 2024-02-16T22:24:30.605 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr" 2024-02-16T22:24:30.605 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-02-16T22:24:30.605 INFO:teuthology.orchestra.run.smithi114.stdout: "progress": "4/23 daemons upgraded", 2024-02-16T22:24:30.605 INFO:teuthology.orchestra.run.smithi114.stdout: "message": "Currently upgrading mon daemons", 2024-02-16T22:24:30.605 INFO:teuthology.orchestra.run.smithi114.stdout: "is_paused": false 2024-02-16T22:24:30.605 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:24:30.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.605+0000 7eff01e45700 1 -- 172.21.15.114:0/2237111116 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7efee806d9e0 msgr2=0x7efee806fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:24:30.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.605+0000 7eff01e45700 1 --2- 172.21.15.114:0/2237111116 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7efee806d9e0 0x7efee806fea0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efee4009c60 tx=0x7efee4009260 comp rx=0 tx=0).stop 2024-02-16T22:24:30.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.605+0000 7eff01e45700 1 -- 172.21.15.114:0/2237111116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efefc103f90 msgr2=0x7efefc1968a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:24:30.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.605+0000 7eff01e45700 1 --2- 172.21.15.114:0/2237111116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efefc103f90 0x7efefc1968a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efeec000c00 tx=0x7efeec00f690 comp rx=0 tx=0).stop 2024-02-16T22:24:30.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.605+0000 7eff01e45700 1 -- 172.21.15.114:0/2237111116 shutdown_connections 2024-02-16T22:24:30.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.605+0000 7eff01e45700 1 --2- 172.21.15.114:0/2237111116 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7efee806d9e0 0x7efee806fea0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:30.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.605+0000 7eff01e45700 1 --2- 172.21.15.114:0/2237111116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efefc101660 0x7efefc196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:30.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.605+0000 7eff01e45700 1 --2- 172.21.15.114:0/2237111116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efefc103f90 0x7efefc1968a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:30.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.605+0000 7eff01e45700 1 -- 172.21.15.114:0/2237111116 >> 172.21.15.114:0/2237111116 conn(0x7efefc0fb050 msgr2=0x7efefc0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:24:30.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.605+0000 7eff01e45700 1 -- 172.21.15.114:0/2237111116 shutdown_connections 2024-02-16T22:24:30.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.605+0000 7eff01e45700 1 -- 172.21.15.114:0/2237111116 wait complete. 2024-02-16T22:24:30.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.708+0000 7f7102c6b700 1 -- 172.21.15.114:0/12349916 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70fc100be0 msgr2=0x7f70fc101000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:24:30.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.708+0000 7f7102c6b700 1 --2- 172.21.15.114:0/12349916 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70fc100be0 0x7f70fc101000 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009910 tx=0x7f70f0009c20 comp rx=0 tx=0).stop 2024-02-16T22:24:30.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.708+0000 7f7102c6b700 1 -- 172.21.15.114:0/12349916 shutdown_connections 2024-02-16T22:24:30.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.708+0000 7f7102c6b700 1 --2- 172.21.15.114:0/12349916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70fc101de0 0x7f70fc102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:30.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.708+0000 7f7102c6b700 1 --2- 172.21.15.114:0/12349916 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70fc100be0 0x7f70fc101000 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:30.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.708+0000 7f7102c6b700 1 -- 172.21.15.114:0/12349916 >> 172.21.15.114:0/12349916 conn(0x7f70fc0fc140 msgr2=0x7f70fc0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:24:30.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.708+0000 7f7102c6b700 1 -- 172.21.15.114:0/12349916 shutdown_connections 2024-02-16T22:24:30.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.708+0000 7f7102c6b700 1 -- 172.21.15.114:0/12349916 wait complete. 2024-02-16T22:24:30.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.709+0000 7f7102c6b700 1 Processor -- start 2024-02-16T22:24:30.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.709+0000 7f7102c6b700 1 -- start start 2024-02-16T22:24:30.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.709+0000 7f7102c6b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70fc100be0 0x7f70fc196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:24:30.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.709+0000 7f7102c6b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70fc101de0 0x7f70fc1968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:24:30.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.709+0000 7f7102c6b700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc196ef0 con 0x7f70fc100be0 2024-02-16T22:24:30.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.709+0000 7f7102c6b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc197030 con 0x7f70fc101de0 2024-02-16T22:24:30.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.709+0000 7f7100a07700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70fc100be0 0x7f70fc196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:24:30.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.709+0000 7f7100a07700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70fc100be0 0x7f70fc196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42094/0 (socket says 172.21.15.114:42094) 2024-02-16T22:24:30.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.709+0000 7f7100a07700 1 -- 172.21.15.114:0/1556130357 learned_addr learned my addr 172.21.15.114:0/1556130357 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:24:30.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.709+0000 7f7100a07700 1 -- 172.21.15.114:0/1556130357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70fc101de0 msgr2=0x7f70fc1968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:24:30.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.709+0000 7f7100a07700 1 --2- 172.21.15.114:0/1556130357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70fc101de0 0x7f70fc1968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:30.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.709+0000 7f7100a07700 1 -- 172.21.15.114:0/1556130357 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f00095f0 con 0x7f70fc100be0 2024-02-16T22:24:30.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.710+0000 7f7100a07700 1 --2- 172.21.15.114:0/1556130357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70fc100be0 0x7f70fc196390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f70f000bda0 tx=0x7f70f000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:24:30.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.710+0000 7f70f9ffb700 1 -- 172.21.15.114:0/1556130357 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70f001d070 con 0x7f70fc100be0 2024-02-16T22:24:30.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.710+0000 7f70f9ffb700 1 -- 172.21.15.114:0/1556130357 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f70f000f460 con 0x7f70fc100be0 2024-02-16T22:24:30.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.710+0000 7f7102c6b700 1 -- 172.21.15.114:0/1556130357 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70fc19ba80 con 0x7f70fc100be0 2024-02-16T22:24:30.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.710+0000 7f70f9ffb700 1 -- 172.21.15.114:0/1556130357 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70f0021640 con 0x7f70fc100be0 2024-02-16T22:24:30.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.710+0000 7f7102c6b700 1 -- 172.21.15.114:0/1556130357 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70fc19bf70 con 0x7f70fc100be0 2024-02-16T22:24:30.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.711+0000 7f7102c6b700 1 -- 172.21.15.114:0/1556130357 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70fc109740 con 0x7f70fc100be0 2024-02-16T22:24:30.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.711+0000 7f70f9ffb700 1 -- 172.21.15.114:0/1556130357 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 37) v1 ==== 91597+0+0 (secure 0 0 0) 0x7f70f000fa00 con 0x7f70fc100be0 2024-02-16T22:24:30.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.712+0000 7f70f9ffb700 1 --2- 172.21.15.114:0/1556130357 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f70ec06d9e0 0x7f70ec06fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:24:30.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.712+0000 7f70f9ffb700 1 -- 172.21.15.114:0/1556130357 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f70f008ea20 con 0x7f70fc100be0 2024-02-16T22:24:30.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.712+0000 7f70fbfff700 1 --2- 172.21.15.114:0/1556130357 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f70ec06d9e0 0x7f70ec06fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:24:30.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.712+0000 7f70fbfff700 1 --2- 172.21.15.114:0/1556130357 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f70ec06d9e0 0x7f70ec06fea0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f70e8005950 tx=0x7f70e80058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:24:30.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:30.716+0000 7f70f9ffb700 1 -- 172.21.15.114:0/1556130357 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f70f005bb90 con 0x7f70fc100be0 2024-02-16T22:24:30.764 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:30 smithi123 ceph-mon[82331]: from='client.34158 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:24:30.764 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:30 smithi123 ceph-mon[82331]: from='client.34162 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:24:30.764 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:30 smithi123 ceph-mon[82331]: Reconfiguring ceph-exporter.smithi123 (monmap changed)... 2024-02-16T22:24:30.764 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:30 smithi123 ceph-mon[82331]: Unable to update caps for client.ceph-exporter.smithi123 2024-02-16T22:24:30.764 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:30 smithi123 ceph-mon[82331]: Reconfiguring daemon ceph-exporter.smithi123 on smithi123 2024-02-16T22:24:30.764 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:30 smithi123 ceph-mon[82331]: from='client.34166 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:24:30.764 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:30 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/449611891' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:24:30.764 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:30 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/875011151' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:24:30.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:30 smithi114 ceph-mon[94452]: from='client.34158 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:24:30.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:30 smithi114 ceph-mon[94452]: from='client.34162 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:24:30.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:30 smithi114 ceph-mon[94452]: Reconfiguring ceph-exporter.smithi123 (monmap changed)... 2024-02-16T22:24:30.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:30 smithi114 ceph-mon[94452]: Unable to update caps for client.ceph-exporter.smithi123 2024-02-16T22:24:30.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:30 smithi114 ceph-mon[94452]: Reconfiguring daemon ceph-exporter.smithi123 on smithi123 2024-02-16T22:24:30.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:30 smithi114 ceph-mon[94452]: from='client.34166 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:24:30.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:30 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/449611891' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:24:30.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:30 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/875011151' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:24:31.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:31.009+0000 7f7102c6b700 1 -- 172.21.15.114:0/1556130357 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f70fc19c5b0 con 0x7f70fc100be0 2024-02-16T22:24:31.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:31.009+0000 7f70f9ffb700 1 -- 172.21.15.114:0/1556130357 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f70f005b720 con 0x7f70fc100be0 2024-02-16T22:24:31.011 INFO:teuthology.orchestra.run.smithi114.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:24:31.011 INFO:teuthology.orchestra.run.smithi114.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:24:31.011 INFO:teuthology.orchestra.run.smithi114.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:24:31.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:31.011+0000 7f7102c6b700 1 -- 172.21.15.114:0/1556130357 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f70ec06d9e0 msgr2=0x7f70ec06fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:24:31.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:31.011+0000 7f7102c6b700 1 --2- 172.21.15.114:0/1556130357 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f70ec06d9e0 0x7f70ec06fea0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f70e8005950 tx=0x7f70e80058e0 comp rx=0 tx=0).stop 2024-02-16T22:24:31.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:31.011+0000 7f7102c6b700 1 -- 172.21.15.114:0/1556130357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70fc100be0 msgr2=0x7f70fc196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:24:31.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:31.011+0000 7f7102c6b700 1 --2- 172.21.15.114:0/1556130357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70fc100be0 0x7f70fc196390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f70f000bda0 tx=0x7f70f000be80 comp rx=0 tx=0).stop 2024-02-16T22:24:31.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:31.011+0000 7f7102c6b700 1 -- 172.21.15.114:0/1556130357 shutdown_connections 2024-02-16T22:24:31.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:31.011+0000 7f7102c6b700 1 --2- 172.21.15.114:0/1556130357 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f70ec06d9e0 0x7f70ec06fea0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:31.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:31.011+0000 7f7102c6b700 1 --2- 172.21.15.114:0/1556130357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70fc100be0 0x7f70fc196390 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:31.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:31.011+0000 7f7102c6b700 1 --2- 172.21.15.114:0/1556130357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70fc101de0 0x7f70fc1968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:24:31.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:31.011+0000 7f7102c6b700 1 -- 172.21.15.114:0/1556130357 >> 172.21.15.114:0/1556130357 conn(0x7f70fc0fc140 msgr2=0x7f70fc105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:24:31.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:31.011+0000 7f7102c6b700 1 -- 172.21.15.114:0/1556130357 shutdown_connections 2024-02-16T22:24:31.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:24:31.011+0000 7f7102c6b700 1 -- 172.21.15.114:0/1556130357 wait complete. 2024-02-16T22:24:31.708 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:31 smithi123 ceph-mon[82331]: pgmap v25: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:31.708 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:31 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:31.708 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:31 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:31.709 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:31 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:24:31.709 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:31 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi123", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T22:24:31.709 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:31 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:31.709 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:31 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/1556130357' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:24:31.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:31 smithi114 ceph-mon[94452]: pgmap v25: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:31.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:31 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:31.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:31 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:31.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:31 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:24:31.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:31 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi123", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T22:24:31.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:31 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:31.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:31 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/1556130357' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:24:32.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:32 smithi123 ceph-mon[82331]: from='client.34178 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:24:32.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:32 smithi123 ceph-mon[82331]: Reconfiguring crash.smithi123 (monmap changed)... 2024-02-16T22:24:32.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:32 smithi123 ceph-mon[82331]: Reconfiguring daemon crash.smithi123 on smithi123 2024-02-16T22:24:32.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:32 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:32.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:32 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:32.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:32 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi123.inzgjd", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:24:32.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:32 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T22:24:32.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:32 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:32.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:32 smithi114 ceph-mon[94452]: from='client.34178 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:24:32.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:32 smithi114 ceph-mon[94452]: Reconfiguring crash.smithi123 (monmap changed)... 2024-02-16T22:24:32.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:32 smithi114 ceph-mon[94452]: Reconfiguring daemon crash.smithi123 on smithi123 2024-02-16T22:24:32.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:32 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:32.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:32 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:32.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:32 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi123.inzgjd", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T22:24:32.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:32 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T22:24:32.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:32 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:33.676 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:33 smithi123 ceph-mon[82331]: pgmap v26: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:33.676 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:33 smithi123 ceph-mon[82331]: Reconfiguring mgr.smithi123.inzgjd (monmap changed)... 2024-02-16T22:24:33.676 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:33 smithi123 ceph-mon[82331]: Reconfiguring daemon mgr.smithi123.inzgjd on smithi123 2024-02-16T22:24:33.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:33 smithi114 ceph-mon[94452]: pgmap v26: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:33.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:33 smithi114 ceph-mon[94452]: Reconfiguring mgr.smithi123.inzgjd (monmap changed)... 2024-02-16T22:24:33.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:33 smithi114 ceph-mon[94452]: Reconfiguring daemon mgr.smithi123.inzgjd on smithi123 2024-02-16T22:24:35.112 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:35 smithi123 ceph-mon[82331]: pgmap v27: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:35.113 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:35 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:35.113 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:35 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:35.113 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:35 smithi123 ceph-mon[82331]: Reconfiguring mon.smithi123 (monmap changed)... 2024-02-16T22:24:35.113 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:35 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T22:24:35.113 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:35 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T22:24:35.113 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:35 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:35.113 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:35 smithi123 ceph-mon[82331]: Reconfiguring daemon mon.smithi123 on smithi123 2024-02-16T22:24:35.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:35 smithi114 ceph-mon[94452]: pgmap v27: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:35.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:35 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:35.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:35 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:35.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:35 smithi114 ceph-mon[94452]: Reconfiguring mon.smithi123 (monmap changed)... 2024-02-16T22:24:35.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:35 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T22:24:35.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:35 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T22:24:35.453 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:35 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:35.453 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:35 smithi114 ceph-mon[94452]: Reconfiguring daemon mon.smithi123 on smithi123 2024-02-16T22:24:36.996 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:36 smithi123 ceph-mon[82331]: pgmap v28: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:36.997 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:36.997 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:36.997 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:36 smithi123 ceph-mon[82331]: Reconfiguring osd.3 (monmap changed)... 2024-02-16T22:24:36.997 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-16T22:24:36.997 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:36.997 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:36 smithi123 ceph-mon[82331]: Reconfiguring daemon osd.3 on smithi123 2024-02-16T22:24:37.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:36 smithi114 ceph-mon[94452]: pgmap v28: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:37.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:37.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:37.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:36 smithi114 ceph-mon[94452]: Reconfiguring osd.3 (monmap changed)... 2024-02-16T22:24:37.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-16T22:24:37.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:37.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:36 smithi114 ceph-mon[94452]: Reconfiguring daemon osd.3 on smithi123 2024-02-16T22:24:38.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:38 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:38.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:38 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:38.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:38 smithi123 ceph-mon[82331]: Reconfiguring osd.4 (monmap changed)... 2024-02-16T22:24:38.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:38 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-16T22:24:38.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:38 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:38.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:38 smithi123 ceph-mon[82331]: Reconfiguring daemon osd.4 on smithi123 2024-02-16T22:24:38.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:38 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:38.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:38 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:38.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:38 smithi114 ceph-mon[94452]: Reconfiguring osd.4 (monmap changed)... 2024-02-16T22:24:38.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:38 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-16T22:24:38.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:38 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:38.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:38 smithi114 ceph-mon[94452]: Reconfiguring daemon osd.4 on smithi123 2024-02-16T22:24:39.689 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:39 smithi123 ceph-mon[82331]: pgmap v29: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:39.689 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:39 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:39.689 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:39 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:39.689 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:39 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-16T22:24:39.689 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:39 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:39.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:39 smithi114 ceph-mon[94452]: pgmap v29: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:39.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:39 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:39.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:39 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:39.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:39 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-16T22:24:39.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:39 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:40.684 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:40 smithi123 ceph-mon[82331]: Reconfiguring osd.5 (monmap changed)... 2024-02-16T22:24:40.684 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:40 smithi123 ceph-mon[82331]: Reconfiguring daemon osd.5 on smithi123 2024-02-16T22:24:40.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:40 smithi114 ceph-mon[94452]: Reconfiguring osd.5 (monmap changed)... 2024-02-16T22:24:40.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:40 smithi114 ceph-mon[94452]: Reconfiguring daemon osd.5 on smithi123 2024-02-16T22:24:41.662 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:41 smithi123 ceph-mon[82331]: pgmap v30: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:41.663 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:41 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:41.663 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:41 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:41.663 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:41 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi123.vhlqbe", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:24:41.663 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:41 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:41.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:41 smithi114 ceph-mon[94452]: pgmap v30: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:41.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:41 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:41.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:41 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:41.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:41 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi123.vhlqbe", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:24:41.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:41 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:42.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:42 smithi114 ceph-mon[94452]: Reconfiguring mds.cephfs.smithi123.vhlqbe (monmap changed)... 2024-02-16T22:24:42.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:42 smithi114 ceph-mon[94452]: Reconfiguring daemon mds.cephfs.smithi123.vhlqbe on smithi123 2024-02-16T22:24:42.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:42 smithi123 ceph-mon[82331]: Reconfiguring mds.cephfs.smithi123.vhlqbe (monmap changed)... 2024-02-16T22:24:42.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:42 smithi123 ceph-mon[82331]: Reconfiguring daemon mds.cephfs.smithi123.vhlqbe on smithi123 2024-02-16T22:24:43.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:43 smithi123 ceph-mon[82331]: pgmap v31: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:43.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:43 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:43.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:43 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:43.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:43 smithi123 ceph-mon[82331]: Reconfiguring mds.cephfs.smithi123.gopcfa (monmap changed)... 2024-02-16T22:24:43.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:43 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi123.gopcfa", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:24:43.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:43 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:43.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:43 smithi123 ceph-mon[82331]: Reconfiguring daemon mds.cephfs.smithi123.gopcfa on smithi123 2024-02-16T22:24:43.870 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:43 smithi114 ceph-mon[94452]: pgmap v31: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:43.870 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:43 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:43.870 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:43 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:43.870 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:43 smithi114 ceph-mon[94452]: Reconfiguring mds.cephfs.smithi123.gopcfa (monmap changed)... 2024-02-16T22:24:43.871 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:43 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi123.gopcfa", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:24:43.871 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:43 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:43.871 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:43 smithi114 ceph-mon[94452]: Reconfiguring daemon mds.cephfs.smithi123.gopcfa on smithi123 2024-02-16T22:24:45.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:45 smithi114 ceph-mon[94452]: pgmap v32: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:45.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:45 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:45.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:45 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:45.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:45 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:24:45.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:45 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:24:45.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:45 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:24:45.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:45 smithi114 ceph-mon[94452]: Upgrade: Setting container_image for all mon 2024-02-16T22:24:45.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:45 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:45.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:45 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi114"}]: dispatch 2024-02-16T22:24:45.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:45 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi114"}]': finished 2024-02-16T22:24:45.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:45 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi123"}]: dispatch 2024-02-16T22:24:45.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:45 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi123"}]': finished 2024-02-16T22:24:45.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:45 smithi123 ceph-mon[82331]: pgmap v32: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:45.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:45 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:45.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:45 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:45.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:45 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:24:45.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:45 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:24:45.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:45 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:24:45.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:45 smithi123 ceph-mon[82331]: Upgrade: Setting container_image for all mon 2024-02-16T22:24:45.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:45 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:45.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:45 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi114"}]: dispatch 2024-02-16T22:24:45.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:45 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi114"}]': finished 2024-02-16T22:24:45.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:45 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi123"}]: dispatch 2024-02-16T22:24:45.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:45 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi123"}]': finished 2024-02-16T22:24:46.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:24:46.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:46.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi114", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T22:24:46.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:46.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:46 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:24:46.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:46 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:46.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:46 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi114", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T22:24:46.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:46 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:47.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:47 smithi114 ceph-mon[94452]: pgmap v33: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:47.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:47 smithi114 ceph-mon[94452]: Upgrade: Updating crash.smithi114 (1/2) 2024-02-16T22:24:47.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:47 smithi114 ceph-mon[94452]: Deploying daemon crash.smithi114 on smithi114 2024-02-16T22:24:47.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:47 smithi123 ceph-mon[82331]: pgmap v33: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:47.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:47 smithi123 ceph-mon[82331]: Upgrade: Updating crash.smithi114 (1/2) 2024-02-16T22:24:47.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:47 smithi123 ceph-mon[82331]: Deploying daemon crash.smithi114 on smithi114 2024-02-16T22:24:49.385 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:49 smithi114 ceph-mon[94452]: pgmap v34: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:49.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:49 smithi123 ceph-mon[82331]: pgmap v34: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:50.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:50 smithi114 ceph-mon[94452]: pgmap v35: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:50.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:50 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:50.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:50 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:51.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:50 smithi123 ceph-mon[82331]: pgmap v35: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:51.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:50 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:51.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:50 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:52.583 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:52 smithi123 ceph-mon[82331]: Upgrade: Updating crash.smithi123 (2/2) 2024-02-16T22:24:52.584 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:52 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:52.584 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:52 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi123", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T22:24:52.584 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:52 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:52.584 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:52 smithi123 ceph-mon[82331]: Deploying daemon crash.smithi123 on smithi123 2024-02-16T22:24:52.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:52 smithi114 ceph-mon[94452]: Upgrade: Updating crash.smithi123 (2/2) 2024-02-16T22:24:52.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:52 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:52.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:52 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi123", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T22:24:52.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:52 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:24:52.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:52 smithi114 ceph-mon[94452]: Deploying daemon crash.smithi123 on smithi123 2024-02-16T22:24:53.404 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:53 smithi123 ceph-mon[82331]: pgmap v36: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:53.581 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:53 smithi114 ceph-mon[94452]: pgmap v36: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:55.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:55 smithi114 ceph-mon[94452]: pgmap v37: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:55.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:55 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:55.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:55 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:55.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:55 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:24:55.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:55 smithi123 ceph-mon[82331]: pgmap v37: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:55.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:55 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:55.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:55 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:55.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:55 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:24:57.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:57 smithi123 ceph-mon[82331]: pgmap v38: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:57.674 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:57 smithi114 ceph-mon[94452]: pgmap v38: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:59.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:58 smithi123 ceph-mon[82331]: pgmap v39: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:59.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:58 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:59.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:24:58 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:59.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:58 smithi114 ceph-mon[94452]: pgmap v39: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:24:59.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:58 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:24:59.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:24:58 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:00.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:00 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:00.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:00 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:00.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:00 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:00.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:00 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:01.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.120+0000 7fe136115700 1 -- 172.21.15.114:0/2963387924 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe130100c00 msgr2=0x7fe130101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:01.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.120+0000 7fe136115700 1 --2- 172.21.15.114:0/2963387924 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe130100c00 0x7fe130101020 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe120009910 tx=0x7fe120009c20 comp rx=0 tx=0).stop 2024-02-16T22:25:01.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.120+0000 7fe136115700 1 -- 172.21.15.114:0/2963387924 shutdown_connections 2024-02-16T22:25:01.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.120+0000 7fe136115700 1 --2- 172.21.15.114:0/2963387924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe130101e00 0x7fe130102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:01.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.120+0000 7fe136115700 1 --2- 172.21.15.114:0/2963387924 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe130100c00 0x7fe130101020 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:01.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.120+0000 7fe136115700 1 -- 172.21.15.114:0/2963387924 >> 172.21.15.114:0/2963387924 conn(0x7fe1300fc180 msgr2=0x7fe1300fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:25:01.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.120+0000 7fe136115700 1 -- 172.21.15.114:0/2963387924 shutdown_connections 2024-02-16T22:25:01.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.120+0000 7fe136115700 1 -- 172.21.15.114:0/2963387924 wait complete. 2024-02-16T22:25:01.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.121+0000 7fe136115700 1 Processor -- start 2024-02-16T22:25:01.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.121+0000 7fe136115700 1 -- start start 2024-02-16T22:25:01.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.121+0000 7fe136115700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe130100c00 0x7fe130074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:01.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.121+0000 7fe136115700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe130101e00 0x7fe130073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:01.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.121+0000 7fe136115700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1300736c0 con 0x7fe130100c00 2024-02-16T22:25:01.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.121+0000 7fe136115700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe130073800 con 0x7fe130101e00 2024-02-16T22:25:01.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.121+0000 7fe12f7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe130100c00 0x7fe130074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:01.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.121+0000 7fe12f7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe130100c00 0x7fe130074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33920/0 (socket says 172.21.15.114:33920) 2024-02-16T22:25:01.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.121+0000 7fe12f7fe700 1 -- 172.21.15.114:0/1126175069 learned_addr learned my addr 172.21.15.114:0/1126175069 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:25:01.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.121+0000 7fe12effd700 1 --2- 172.21.15.114:0/1126175069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe130101e00 0x7fe130073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:01.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.122+0000 7fe12f7fe700 1 -- 172.21.15.114:0/1126175069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe130101e00 msgr2=0x7fe130073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:01.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.122+0000 7fe12f7fe700 1 --2- 172.21.15.114:0/1126175069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe130101e00 0x7fe130073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:01.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.122+0000 7fe12f7fe700 1 -- 172.21.15.114:0/1126175069 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1200095f0 con 0x7fe130100c00 2024-02-16T22:25:01.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.122+0000 7fe12f7fe700 1 --2- 172.21.15.114:0/1126175069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe130100c00 0x7fe130074b30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe12000bda0 tx=0x7fe12000bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:25:01.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.122+0000 7fe12cff9700 1 -- 172.21.15.114:0/1126175069 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe12001d070 con 0x7fe130100c00 2024-02-16T22:25:01.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.122+0000 7fe136115700 1 -- 172.21.15.114:0/1126175069 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe130073a80 con 0x7fe130100c00 2024-02-16T22:25:01.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.122+0000 7fe12cff9700 1 -- 172.21.15.114:0/1126175069 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe120022470 con 0x7fe130100c00 2024-02-16T22:25:01.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.122+0000 7fe12cff9700 1 -- 172.21.15.114:0/1126175069 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe12000f670 con 0x7fe130100c00 2024-02-16T22:25:01.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.122+0000 7fe136115700 1 -- 172.21.15.114:0/1126175069 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe130073f70 con 0x7fe130100c00 2024-02-16T22:25:01.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.123+0000 7fe12cff9700 1 -- 172.21.15.114:0/1126175069 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 37) v1 ==== 91597+0+0 (secure 0 0 0) 0x7fe120004af0 con 0x7fe130100c00 2024-02-16T22:25:01.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.124+0000 7fe136115700 1 -- 172.21.15.114:0/1126175069 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1301097e0 con 0x7fe130100c00 2024-02-16T22:25:01.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.124+0000 7fe12cff9700 1 --2- 172.21.15.114:0/1126175069 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe11806d9e0 0x7fe11806fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:01.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.124+0000 7fe12cff9700 1 -- 172.21.15.114:0/1126175069 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe12008eed0 con 0x7fe130100c00 2024-02-16T22:25:01.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.124+0000 7fe12effd700 1 --2- 172.21.15.114:0/1126175069 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe11806d9e0 0x7fe11806fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:01.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.124+0000 7fe12effd700 1 --2- 172.21.15.114:0/1126175069 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe11806d9e0 0x7fe11806fea0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe124009a10 tx=0x7fe124009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:25:01.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.129+0000 7fe12cff9700 1 -- 172.21.15.114:0/1126175069 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe12005c210 con 0x7fe130100c00 2024-02-16T22:25:01.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.368+0000 7fe136115700 1 -- 172.21.15.114:0/1126175069 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe130106750 con 0x7fe11806d9e0 2024-02-16T22:25:01.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.370+0000 7fe12cff9700 1 -- 172.21.15.114:0/1126175069 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+402 (secure 0 0 0) 0x7fe130106750 con 0x7fe11806d9e0 2024-02-16T22:25:01.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.372+0000 7fe136115700 1 -- 172.21.15.114:0/1126175069 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe11806d9e0 msgr2=0x7fe11806fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:01.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.372+0000 7fe136115700 1 --2- 172.21.15.114:0/1126175069 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe11806d9e0 0x7fe11806fea0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe124009a10 tx=0x7fe124009190 comp rx=0 tx=0).stop 2024-02-16T22:25:01.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.372+0000 7fe136115700 1 -- 172.21.15.114:0/1126175069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe130100c00 msgr2=0x7fe130074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:01.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.372+0000 7fe136115700 1 --2- 172.21.15.114:0/1126175069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe130100c00 0x7fe130074b30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe12000bda0 tx=0x7fe12000bdd0 comp rx=0 tx=0).stop 2024-02-16T22:25:01.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.372+0000 7fe136115700 1 -- 172.21.15.114:0/1126175069 shutdown_connections 2024-02-16T22:25:01.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.372+0000 7fe136115700 1 --2- 172.21.15.114:0/1126175069 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe11806d9e0 0x7fe11806fea0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:01.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.372+0000 7fe136115700 1 --2- 172.21.15.114:0/1126175069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe130100c00 0x7fe130074b30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:01.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.372+0000 7fe136115700 1 --2- 172.21.15.114:0/1126175069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe130101e00 0x7fe130073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:01.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.372+0000 7fe136115700 1 -- 172.21.15.114:0/1126175069 >> 172.21.15.114:0/1126175069 conn(0x7fe1300fc180 msgr2=0x7fe130105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:25:01.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.372+0000 7fe136115700 1 -- 172.21.15.114:0/1126175069 shutdown_connections 2024-02-16T22:25:01.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.372+0000 7fe136115700 1 -- 172.21.15.114:0/1126175069 wait complete. 2024-02-16T22:25:01.385 INFO:teuthology.orchestra.run.smithi114.stdout:true 2024-02-16T22:25:01.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.480+0000 7faea6588700 1 -- 172.21.15.114:0/2940888066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faea00ffab0 msgr2=0x7faea00ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:01.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.480+0000 7faea6588700 1 --2- 172.21.15.114:0/2940888066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faea00ffab0 0x7faea00ffed0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fae94009910 tx=0x7fae94009c20 comp rx=0 tx=0).stop 2024-02-16T22:25:01.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.481+0000 7faea6588700 1 -- 172.21.15.114:0/2940888066 shutdown_connections 2024-02-16T22:25:01.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.481+0000 7faea6588700 1 --2- 172.21.15.114:0/2940888066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faea0100cb0 0x7faea0101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:01.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.481+0000 7faea6588700 1 --2- 172.21.15.114:0/2940888066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faea00ffab0 0x7faea00ffed0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:01.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.481+0000 7faea6588700 1 -- 172.21.15.114:0/2940888066 >> 172.21.15.114:0/2940888066 conn(0x7faea00fb050 msgr2=0x7faea00fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:25:01.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.481+0000 7faea6588700 1 -- 172.21.15.114:0/2940888066 shutdown_connections 2024-02-16T22:25:01.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.481+0000 7faea6588700 1 -- 172.21.15.114:0/2940888066 wait complete. 2024-02-16T22:25:01.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.481+0000 7faea6588700 1 Processor -- start 2024-02-16T22:25:01.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.481+0000 7faea6588700 1 -- start start 2024-02-16T22:25:01.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.481+0000 7faea6588700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faea00ffab0 0x7faea0103a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:01.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.482+0000 7faea6588700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faea0100cb0 0x7faea0102090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:01.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.482+0000 7faea6588700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea01025d0 con 0x7faea00ffab0 2024-02-16T22:25:01.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.482+0000 7faea6588700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea0102710 con 0x7faea0100cb0 2024-02-16T22:25:01.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.482+0000 7fae9ffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faea00ffab0 0x7faea0103a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:01.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.482+0000 7fae9ffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faea00ffab0 0x7faea0103a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47416/0 (socket says 172.21.15.114:47416) 2024-02-16T22:25:01.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.482+0000 7fae9ffff700 1 -- 172.21.15.114:0/568296672 learned_addr learned my addr 172.21.15.114:0/568296672 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:25:01.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.482+0000 7fae9f7fe700 1 --2- 172.21.15.114:0/568296672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faea0100cb0 0x7faea0102090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:01.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.482+0000 7fae9ffff700 1 -- 172.21.15.114:0/568296672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faea0100cb0 msgr2=0x7faea0102090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:01.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.482+0000 7fae9ffff700 1 --2- 172.21.15.114:0/568296672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faea0100cb0 0x7faea0102090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:01.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.482+0000 7fae9ffff700 1 -- 172.21.15.114:0/568296672 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae940095f0 con 0x7faea00ffab0 2024-02-16T22:25:01.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.482+0000 7fae9ffff700 1 --2- 172.21.15.114:0/568296672 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faea00ffab0 0x7faea0103a40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fae94000c00 tx=0x7fae9400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:25:01.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.482+0000 7fae9d7fa700 1 -- 172.21.15.114:0/568296672 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae9401d070 con 0x7faea00ffab0 2024-02-16T22:25:01.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.482+0000 7fae9d7fa700 1 -- 172.21.15.114:0/568296672 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fae94005dc0 con 0x7faea00ffab0 2024-02-16T22:25:01.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.482+0000 7faea6588700 1 -- 172.21.15.114:0/568296672 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea0102990 con 0x7faea00ffab0 2024-02-16T22:25:01.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.482+0000 7faea6588700 1 -- 172.21.15.114:0/568296672 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea0102e80 con 0x7faea00ffab0 2024-02-16T22:25:01.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.482+0000 7fae9d7fa700 1 -- 172.21.15.114:0/568296672 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae94022470 con 0x7faea00ffab0 2024-02-16T22:25:01.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.484+0000 7faea6588700 1 -- 172.21.15.114:0/568296672 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faea0108630 con 0x7faea00ffab0 2024-02-16T22:25:01.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.484+0000 7fae9d7fa700 1 -- 172.21.15.114:0/568296672 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 37) v1 ==== 91597+0+0 (secure 0 0 0) 0x7fae94003810 con 0x7faea00ffab0 2024-02-16T22:25:01.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.484+0000 7fae9d7fa700 1 --2- 172.21.15.114:0/568296672 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fae8806d9e0 0x7fae8806fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:01.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.484+0000 7fae9d7fa700 1 -- 172.21.15.114:0/568296672 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fae9408eca0 con 0x7faea00ffab0 2024-02-16T22:25:01.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.484+0000 7fae9f7fe700 1 --2- 172.21.15.114:0/568296672 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fae8806d9e0 0x7fae8806fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:01.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.485+0000 7fae9f7fe700 1 --2- 172.21.15.114:0/568296672 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fae8806d9e0 0x7fae8806fea0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fae90009c60 tx=0x7fae90009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:25:01.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.490+0000 7fae9d7fa700 1 -- 172.21.15.114:0/568296672 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fae94093070 con 0x7faea00ffab0 2024-02-16T22:25:01.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:01 smithi123 ceph-mon[82331]: pgmap v40: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:25:01.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:01 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:01.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:01 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:01.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:01 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:25:01.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:01 smithi114 ceph-mon[94452]: pgmap v40: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:25:01.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:01.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:01.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:25:01.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.719+0000 7faea6588700 1 -- 172.21.15.114:0/568296672 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faea00693b0 con 0x7fae8806d9e0 2024-02-16T22:25:01.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.720+0000 7fae9d7fa700 1 -- 172.21.15.114:0/568296672 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+402 (secure 0 0 0) 0x7faea00693b0 con 0x7fae8806d9e0 2024-02-16T22:25:01.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.722+0000 7faea6588700 1 -- 172.21.15.114:0/568296672 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fae8806d9e0 msgr2=0x7fae8806fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:01.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.722+0000 7faea6588700 1 --2- 172.21.15.114:0/568296672 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fae8806d9e0 0x7fae8806fea0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fae90009c60 tx=0x7fae90009260 comp rx=0 tx=0).stop 2024-02-16T22:25:01.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.722+0000 7faea6588700 1 -- 172.21.15.114:0/568296672 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faea00ffab0 msgr2=0x7faea0103a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:01.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.722+0000 7faea6588700 1 --2- 172.21.15.114:0/568296672 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faea00ffab0 0x7faea0103a40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fae94000c00 tx=0x7fae9400f690 comp rx=0 tx=0).stop 2024-02-16T22:25:01.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.722+0000 7faea6588700 1 -- 172.21.15.114:0/568296672 shutdown_connections 2024-02-16T22:25:01.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.722+0000 7faea6588700 1 --2- 172.21.15.114:0/568296672 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fae8806d9e0 0x7fae8806fea0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:01.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.722+0000 7faea6588700 1 --2- 172.21.15.114:0/568296672 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faea00ffab0 0x7faea0103a40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:01.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.722+0000 7faea6588700 1 --2- 172.21.15.114:0/568296672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faea0100cb0 0x7faea0102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:01.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.722+0000 7faea6588700 1 -- 172.21.15.114:0/568296672 >> 172.21.15.114:0/568296672 conn(0x7faea00fb050 msgr2=0x7faea0068ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:25:01.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.722+0000 7faea6588700 1 -- 172.21.15.114:0/568296672 shutdown_connections 2024-02-16T22:25:01.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.722+0000 7faea6588700 1 -- 172.21.15.114:0/568296672 wait complete. 2024-02-16T22:25:01.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.847+0000 7f95e610e700 1 -- 172.21.15.114:0/3925460939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95e00feee0 msgr2=0x7f95e00ff300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:01.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.847+0000 7f95e610e700 1 --2- 172.21.15.114:0/3925460939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95e00feee0 0x7f95e00ff300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f95d0009910 tx=0x7f95d0009c20 comp rx=0 tx=0).stop 2024-02-16T22:25:01.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.847+0000 7f95e610e700 1 -- 172.21.15.114:0/3925460939 shutdown_connections 2024-02-16T22:25:01.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.847+0000 7f95e610e700 1 --2- 172.21.15.114:0/3925460939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e0101f60 0x7f95e01023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:01.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.847+0000 7f95e610e700 1 --2- 172.21.15.114:0/3925460939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95e00feee0 0x7f95e00ff300 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:01.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.847+0000 7f95e610e700 1 -- 172.21.15.114:0/3925460939 >> 172.21.15.114:0/3925460939 conn(0x7f95e00fa800 msgr2=0x7f95e00fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:25:01.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.847+0000 7f95e610e700 1 -- 172.21.15.114:0/3925460939 shutdown_connections 2024-02-16T22:25:01.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.847+0000 7f95e610e700 1 -- 172.21.15.114:0/3925460939 wait complete. 2024-02-16T22:25:01.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.848+0000 7f95e610e700 1 Processor -- start 2024-02-16T22:25:01.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.848+0000 7f95e610e700 1 -- start start 2024-02-16T22:25:01.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.848+0000 7f95e610e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95e00feee0 0x7f95e01963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:01.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.848+0000 7f95e610e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e0101f60 0x7f95e0196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:01.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.848+0000 7f95e610e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e0196f30 con 0x7f95e00feee0 2024-02-16T22:25:01.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.848+0000 7f95e610e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e0197070 con 0x7f95e0101f60 2024-02-16T22:25:01.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.848+0000 7f95df7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95e00feee0 0x7f95e01963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:01.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.848+0000 7f95df7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95e00feee0 0x7f95e01963d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47432/0 (socket says 172.21.15.114:47432) 2024-02-16T22:25:01.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.848+0000 7f95deffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e0101f60 0x7f95e0196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:01.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.848+0000 7f95df7fe700 1 -- 172.21.15.114:0/2717542050 learned_addr learned my addr 172.21.15.114:0/2717542050 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:25:01.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.849+0000 7f95deffd700 1 -- 172.21.15.114:0/2717542050 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95e00feee0 msgr2=0x7f95e01963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:01.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.849+0000 7f95deffd700 1 --2- 172.21.15.114:0/2717542050 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95e00feee0 0x7f95e01963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:01.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.849+0000 7f95deffd700 1 -- 172.21.15.114:0/2717542050 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d00095f0 con 0x7f95e0101f60 2024-02-16T22:25:01.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.849+0000 7f95df7fe700 1 --2- 172.21.15.114:0/2717542050 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95e00feee0 0x7f95e01963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:25:01.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.849+0000 7f95deffd700 1 --2- 172.21.15.114:0/2717542050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e0101f60 0x7f95e0196910 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f95d400e970 tx=0x7f95d400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:25:01.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.849+0000 7f95dcff9700 1 -- 172.21.15.114:0/2717542050 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95d400cba0 con 0x7f95e0101f60 2024-02-16T22:25:01.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.849+0000 7f95dcff9700 1 -- 172.21.15.114:0/2717542050 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f95d400cd00 con 0x7f95e0101f60 2024-02-16T22:25:01.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.849+0000 7f95e610e700 1 -- 172.21.15.114:0/2717542050 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e019bb20 con 0x7f95e0101f60 2024-02-16T22:25:01.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.849+0000 7f95dcff9700 1 -- 172.21.15.114:0/2717542050 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95d4018770 con 0x7f95e0101f60 2024-02-16T22:25:01.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.849+0000 7f95e610e700 1 -- 172.21.15.114:0/2717542050 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e019bfb0 con 0x7f95e0101f60 2024-02-16T22:25:01.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.851+0000 7f95e610e700 1 -- 172.21.15.114:0/2717542050 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e0109760 con 0x7f95e0101f60 2024-02-16T22:25:01.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.851+0000 7f95dcff9700 1 -- 172.21.15.114:0/2717542050 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 37) v1 ==== 91597+0+0 (secure 0 0 0) 0x7f95d4018960 con 0x7f95e0101f60 2024-02-16T22:25:01.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.851+0000 7f95dcff9700 1 --2- 172.21.15.114:0/2717542050 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f95c806da30 0x7f95c806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:01.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.852+0000 7f95dcff9700 1 -- 172.21.15.114:0/2717542050 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f95d4014070 con 0x7f95e0101f60 2024-02-16T22:25:01.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.852+0000 7f95df7fe700 1 --2- 172.21.15.114:0/2717542050 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f95c806da30 0x7f95c806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:01.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.852+0000 7f95df7fe700 1 --2- 172.21.15.114:0/2717542050 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f95c806da30 0x7f95c806fef0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f95d0009fd0 tx=0x7f95d0005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:25:01.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:01.856+0000 7f95dcff9700 1 -- 172.21.15.114:0/2717542050 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f95d4052d20 con 0x7f95e0101f60 2024-02-16T22:25:02.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.072+0000 7f95e610e700 1 -- 172.21.15.114:0/2717542050 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f95e01068b0 con 0x7f95c806da30 2024-02-16T22:25:02.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.084+0000 7f95dcff9700 1 -- 172.21.15.114:0/2717542050 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f95e01068b0 con 0x7f95c806da30 2024-02-16T22:25:02.086 INFO:teuthology.orchestra.run.smithi114.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:25:02.086 INFO:teuthology.orchestra.run.smithi114.stdout:alertmanager.smithi114 smithi114 *:9093,9094 running (10m) 1s ago 12m 24.7M - 0.25.0 c8568f914cd2 e7404fea400e 2024-02-16T22:25:02.086 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi114 smithi114 running (12m) 1s ago 12m 8019k - 18.2.1 94eadd990e0a 7fd07162cd96 2024-02-16T22:25:02.087 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi123 smithi123 running (11m) 4s ago 11m 7696k - 18.2.1 94eadd990e0a 7687f5e99621 2024-02-16T22:25:02.087 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi114 smithi114 running (12s) 1s ago 12m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 9b8ae9efe83b 2024-02-16T22:25:02.087 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi123 smithi123 running (7s) 4s ago 11m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 6d97263a0ffa 2024-02-16T22:25:02.087 INFO:teuthology.orchestra.run.smithi114.stdout:grafana.smithi114 smithi114 *:3000 running (10m) 1s ago 12m 90.2M - 9.4.7 954c08fa6188 db1c26c3bc20 2024-02-16T22:25:02.087 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.fdxpey smithi114 running (6m) 1s ago 6m 17.9M - 18.2.1 94eadd990e0a 704b40c144fb 2024-02-16T22:25:02.087 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.zzryke smithi114 running (6m) 1s ago 6m 158M - 18.2.1 94eadd990e0a 1f795d23c9a4 2024-02-16T22:25:02.087 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.gopcfa smithi123 running (6m) 4s ago 6m 17.3M - 18.2.1 94eadd990e0a 166134b7532a 2024-02-16T22:25:02.087 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.vhlqbe smithi123 running (6m) 4s ago 6m 18.8M - 18.2.1 94eadd990e0a 243c8bb96fa8 2024-02-16T22:25:02.087 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi114.vcullz smithi114 *:8443,9283,8765 running (2m) 1s ago 13m 495M - 19.0.0-1450-gf78a58c0 f081f847070a 14ebeb7888c0 2024-02-16T22:25:02.087 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi123.inzgjd smithi123 *:8443,9283,8765 running (105s) 4s ago 11m 449M - 19.0.0-1450-gf78a58c0 f081f847070a f10aab9faf9e 2024-02-16T22:25:02.087 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi114 smithi114 running (87s) 1s ago 13m 42.7M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 58ac06372523 2024-02-16T22:25:02.087 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi123 smithi123 running (55s) 4s ago 11m 34.0M 2048M 19.0.0-1450-gf78a58c0 f081f847070a f76485e0ea24 2024-02-16T22:25:02.087 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi114 smithi114 *:9100 running (12m) 1s ago 12m 15.4M - 1.5.0 0da6a335fe13 744a090e06d0 2024-02-16T22:25:02.087 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi123 smithi123 *:9100 running (11m) 4s ago 11m 16.0M - 1.5.0 0da6a335fe13 3994b8c29b2e 2024-02-16T22:25:02.087 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 smithi114 running (10m) 1s ago 10m 214M 1216M 18.2.1 94eadd990e0a 24ca2e9714e0 2024-02-16T22:25:02.088 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 smithi114 running (10m) 1s ago 10m 223M 1216M 18.2.1 94eadd990e0a 32bb191b88e7 2024-02-16T22:25:02.088 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 smithi114 running (9m) 1s ago 9m 195M 1216M 18.2.1 94eadd990e0a 6608718d0818 2024-02-16T22:25:02.088 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 smithi123 running (9m) 4s ago 9m 259M 2240M 18.2.1 94eadd990e0a 575db379de3d 2024-02-16T22:25:02.088 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 smithi123 running (8m) 4s ago 8m 220M 2240M 18.2.1 94eadd990e0a 7e266866cd9c 2024-02-16T22:25:02.088 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 smithi123 running (8m) 4s ago 8m 199M 2240M 18.2.1 94eadd990e0a 24e5bef19fda 2024-02-16T22:25:02.088 INFO:teuthology.orchestra.run.smithi114.stdout:prometheus.smithi114 smithi114 *:9095 running (114s) 1s ago 11m 57.2M - 2.43.0 a07b618ecd1d 37aac985cdd7 2024-02-16T22:25:02.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.087+0000 7f95e610e700 1 -- 172.21.15.114:0/2717542050 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f95c806da30 msgr2=0x7f95c806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:02.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.087+0000 7f95e610e700 1 --2- 172.21.15.114:0/2717542050 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f95c806da30 0x7f95c806fef0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f95d0009fd0 tx=0x7f95d0005940 comp rx=0 tx=0).stop 2024-02-16T22:25:02.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.087+0000 7f95e610e700 1 -- 172.21.15.114:0/2717542050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e0101f60 msgr2=0x7f95e0196910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:02.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.087+0000 7f95e610e700 1 --2- 172.21.15.114:0/2717542050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e0101f60 0x7f95e0196910 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f95d400e970 tx=0x7f95d400ed30 comp rx=0 tx=0).stop 2024-02-16T22:25:02.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.087+0000 7f95e610e700 1 -- 172.21.15.114:0/2717542050 shutdown_connections 2024-02-16T22:25:02.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.087+0000 7f95e610e700 1 --2- 172.21.15.114:0/2717542050 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f95c806da30 0x7f95c806fef0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:02.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.087+0000 7f95e610e700 1 --2- 172.21.15.114:0/2717542050 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95e00feee0 0x7f95e01963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:02.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.087+0000 7f95e610e700 1 --2- 172.21.15.114:0/2717542050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e0101f60 0x7f95e0196910 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:02.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.087+0000 7f95e610e700 1 -- 172.21.15.114:0/2717542050 >> 172.21.15.114:0/2717542050 conn(0x7f95e00fa800 msgr2=0x7f95e0105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:25:02.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.087+0000 7f95e610e700 1 -- 172.21.15.114:0/2717542050 shutdown_connections 2024-02-16T22:25:02.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.087+0000 7f95e610e700 1 -- 172.21.15.114:0/2717542050 wait complete. 2024-02-16T22:25:02.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.197+0000 7f42d3b83700 1 -- 172.21.15.114:0/1950025696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42cc101640 msgr2=0x7f42cc103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:02.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.197+0000 7f42d3b83700 1 --2- 172.21.15.114:0/1950025696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42cc101640 0x7f42cc103a30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009910 tx=0x7f42c4009c20 comp rx=0 tx=0).stop 2024-02-16T22:25:02.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.197+0000 7f42d3b83700 1 -- 172.21.15.114:0/1950025696 shutdown_connections 2024-02-16T22:25:02.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.197+0000 7f42d3b83700 1 --2- 172.21.15.114:0/1950025696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42cc103f70 0x7f42cc106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:02.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.197+0000 7f42d3b83700 1 --2- 172.21.15.114:0/1950025696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42cc101640 0x7f42cc103a30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:02.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.197+0000 7f42d3b83700 1 -- 172.21.15.114:0/1950025696 >> 172.21.15.114:0/1950025696 conn(0x7f42cc0fb030 msgr2=0x7f42cc0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:25:02.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.197+0000 7f42d3b83700 1 -- 172.21.15.114:0/1950025696 shutdown_connections 2024-02-16T22:25:02.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.197+0000 7f42d3b83700 1 -- 172.21.15.114:0/1950025696 wait complete. 2024-02-16T22:25:02.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.197+0000 7f42d3b83700 1 Processor -- start 2024-02-16T22:25:02.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.197+0000 7f42d3b83700 1 -- start start 2024-02-16T22:25:02.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.198+0000 7f42d3b83700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42cc101640 0x7f42cc196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:02.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.198+0000 7f42d3b83700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42cc103f70 0x7f42cc196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:02.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.198+0000 7f42d3b83700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc196e60 con 0x7f42cc101640 2024-02-16T22:25:02.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.198+0000 7f42d3b83700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc196fa0 con 0x7f42cc103f70 2024-02-16T22:25:02.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.198+0000 7f42d191f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42cc101640 0x7f42cc196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:02.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.198+0000 7f42d191f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42cc101640 0x7f42cc196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47436/0 (socket says 172.21.15.114:47436) 2024-02-16T22:25:02.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.198+0000 7f42d191f700 1 -- 172.21.15.114:0/3182401550 learned_addr learned my addr 172.21.15.114:0/3182401550 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:25:02.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.198+0000 7f42d111e700 1 --2- 172.21.15.114:0/3182401550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42cc103f70 0x7f42cc196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:02.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.198+0000 7f42d191f700 1 -- 172.21.15.114:0/3182401550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42cc103f70 msgr2=0x7f42cc196840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:02.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.198+0000 7f42d191f700 1 --2- 172.21.15.114:0/3182401550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42cc103f70 0x7f42cc196840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:02.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.198+0000 7f42d191f700 1 -- 172.21.15.114:0/3182401550 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c40095f0 con 0x7f42cc101640 2024-02-16T22:25:02.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.198+0000 7f42d191f700 1 --2- 172.21.15.114:0/3182401550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42cc101640 0x7f42cc196300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f42c4006010 tx=0x7f42c400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:25:02.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.198+0000 7f42c2ffd700 1 -- 172.21.15.114:0/3182401550 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42c401d070 con 0x7f42cc101640 2024-02-16T22:25:02.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.198+0000 7f42d3b83700 1 -- 172.21.15.114:0/3182401550 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42cc19b9f0 con 0x7f42cc101640 2024-02-16T22:25:02.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.198+0000 7f42c2ffd700 1 -- 172.21.15.114:0/3182401550 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f42c400f460 con 0x7f42cc101640 2024-02-16T22:25:02.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.198+0000 7f42c2ffd700 1 -- 172.21.15.114:0/3182401550 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42c4021640 con 0x7f42cc101640 2024-02-16T22:25:02.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.198+0000 7f42d3b83700 1 -- 172.21.15.114:0/3182401550 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42cc19bee0 con 0x7f42cc101640 2024-02-16T22:25:02.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.199+0000 7f42d3b83700 1 -- 172.21.15.114:0/3182401550 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42cc190520 con 0x7f42cc101640 2024-02-16T22:25:02.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.200+0000 7f42c2ffd700 1 -- 172.21.15.114:0/3182401550 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 37) v1 ==== 91597+0+0 (secure 0 0 0) 0x7f42c400fab0 con 0x7f42cc101640 2024-02-16T22:25:02.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.200+0000 7f42c2ffd700 1 --2- 172.21.15.114:0/3182401550 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f42b806d9e0 0x7f42b806fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:02.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.200+0000 7f42c2ffd700 1 -- 172.21.15.114:0/3182401550 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f42c408ea30 con 0x7f42cc101640 2024-02-16T22:25:02.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.200+0000 7f42d111e700 1 --2- 172.21.15.114:0/3182401550 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f42b806d9e0 0x7f42b806fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:02.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.201+0000 7f42d111e700 1 --2- 172.21.15.114:0/3182401550 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f42b806d9e0 0x7f42b806fea0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f42bc005950 tx=0x7f42bc00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:25:02.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.205+0000 7f42c2ffd700 1 -- 172.21.15.114:0/3182401550 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f42c405bba0 con 0x7f42cc101640 2024-02-16T22:25:02.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.492+0000 7f42d3b83700 1 -- 172.21.15.114:0/3182401550 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f42cc02ced0 con 0x7f42cc101640 2024-02-16T22:25:02.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.493+0000 7f42c2ffd700 1 -- 172.21.15.114:0/3182401550 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7f42c405b730 con 0x7f42cc101640 2024-02-16T22:25:02.495 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:25:02.495 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": { 2024-02-16T22:25:02.495 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:25:02.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:25:02.495 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": { 2024-02-16T22:25:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:25:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:25:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": { 2024-02-16T22:25:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-02-16T22:25:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:25:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-02-16T22:25:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-02-16T22:25:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:25:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: "overall": { 2024-02-16T22:25:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 10, 2024-02-16T22:25:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T22:25:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-02-16T22:25:02.496 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:25:02.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.496+0000 7f42d3b83700 1 -- 172.21.15.114:0/3182401550 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f42b806d9e0 msgr2=0x7f42b806fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:02.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.496+0000 7f42d3b83700 1 --2- 172.21.15.114:0/3182401550 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f42b806d9e0 0x7f42b806fea0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f42bc005950 tx=0x7f42bc00b410 comp rx=0 tx=0).stop 2024-02-16T22:25:02.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.496+0000 7f42d3b83700 1 -- 172.21.15.114:0/3182401550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42cc101640 msgr2=0x7f42cc196300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:02.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.496+0000 7f42d3b83700 1 --2- 172.21.15.114:0/3182401550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42cc101640 0x7f42cc196300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f42c4006010 tx=0x7f42c400bd70 comp rx=0 tx=0).stop 2024-02-16T22:25:02.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.496+0000 7f42d3b83700 1 -- 172.21.15.114:0/3182401550 shutdown_connections 2024-02-16T22:25:02.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.496+0000 7f42d3b83700 1 --2- 172.21.15.114:0/3182401550 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f42b806d9e0 0x7f42b806fea0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:02.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.496+0000 7f42d3b83700 1 --2- 172.21.15.114:0/3182401550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42cc101640 0x7f42cc196300 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:02.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.496+0000 7f42d3b83700 1 --2- 172.21.15.114:0/3182401550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42cc103f70 0x7f42cc196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:02.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.496+0000 7f42d3b83700 1 -- 172.21.15.114:0/3182401550 >> 172.21.15.114:0/3182401550 conn(0x7f42cc0fb030 msgr2=0x7f42cc0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:25:02.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.496+0000 7f42d3b83700 1 -- 172.21.15.114:0/3182401550 shutdown_connections 2024-02-16T22:25:02.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.496+0000 7f42d3b83700 1 -- 172.21.15.114:0/3182401550 wait complete. 2024-02-16T22:25:02.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.603+0000 7fd20bf2a700 1 -- 172.21.15.114:0/3597982990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd204100bd0 msgr2=0x7fd204100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:02.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.603+0000 7fd20bf2a700 1 --2- 172.21.15.114:0/3597982990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd204100bd0 0x7fd204100ff0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009910 tx=0x7fd1fc009c20 comp rx=0 tx=0).stop 2024-02-16T22:25:02.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.603+0000 7fd20bf2a700 1 -- 172.21.15.114:0/3597982990 shutdown_connections 2024-02-16T22:25:02.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.603+0000 7fd20bf2a700 1 --2- 172.21.15.114:0/3597982990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd204101dd0 0x7fd204102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:02.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.603+0000 7fd20bf2a700 1 --2- 172.21.15.114:0/3597982990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd204100bd0 0x7fd204100ff0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:02.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.603+0000 7fd20bf2a700 1 -- 172.21.15.114:0/3597982990 >> 172.21.15.114:0/3597982990 conn(0x7fd2040fc190 msgr2=0x7fd2040fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:25:02.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.603+0000 7fd20bf2a700 1 -- 172.21.15.114:0/3597982990 shutdown_connections 2024-02-16T22:25:02.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.603+0000 7fd20bf2a700 1 -- 172.21.15.114:0/3597982990 wait complete. 2024-02-16T22:25:02.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.603+0000 7fd20bf2a700 1 Processor -- start 2024-02-16T22:25:02.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.603+0000 7fd20bf2a700 1 -- start start 2024-02-16T22:25:02.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.604+0000 7fd20bf2a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd204100bd0 0x7fd204196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:02.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.604+0000 7fd20bf2a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd204101dd0 0x7fd2041968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:02.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.604+0000 7fd20bf2a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd204196ec0 con 0x7fd204100bd0 2024-02-16T22:25:02.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.604+0000 7fd20bf2a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd204197000 con 0x7fd204101dd0 2024-02-16T22:25:02.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.604+0000 7fd209cc6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd204100bd0 0x7fd204196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:02.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.604+0000 7fd209cc6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd204100bd0 0x7fd204196360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47450/0 (socket says 172.21.15.114:47450) 2024-02-16T22:25:02.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.604+0000 7fd209cc6700 1 -- 172.21.15.114:0/213761544 learned_addr learned my addr 172.21.15.114:0/213761544 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:25:02.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.604+0000 7fd2094c5700 1 --2- 172.21.15.114:0/213761544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd204101dd0 0x7fd2041968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:02.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.604+0000 7fd209cc6700 1 -- 172.21.15.114:0/213761544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd204101dd0 msgr2=0x7fd2041968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:02.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.604+0000 7fd209cc6700 1 --2- 172.21.15.114:0/213761544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd204101dd0 0x7fd2041968a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:02.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.604+0000 7fd209cc6700 1 -- 172.21.15.114:0/213761544 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1fc0095f0 con 0x7fd204100bd0 2024-02-16T22:25:02.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.604+0000 7fd209cc6700 1 --2- 172.21.15.114:0/213761544 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd204100bd0 0x7fd204196360 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc00bda0 tx=0x7fd1fc00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:25:02.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.604+0000 7fd202ffd700 1 -- 172.21.15.114:0/213761544 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1fc01d070 con 0x7fd204100bd0 2024-02-16T22:25:02.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.604+0000 7fd202ffd700 1 -- 172.21.15.114:0/213761544 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd1fc00f460 con 0x7fd204100bd0 2024-02-16T22:25:02.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.604+0000 7fd20bf2a700 1 -- 172.21.15.114:0/213761544 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd20419ba50 con 0x7fd204100bd0 2024-02-16T22:25:02.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.604+0000 7fd202ffd700 1 -- 172.21.15.114:0/213761544 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1fc021640 con 0x7fd204100bd0 2024-02-16T22:25:02.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.604+0000 7fd20bf2a700 1 -- 172.21.15.114:0/213761544 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd20419bf40 con 0x7fd204100bd0 2024-02-16T22:25:02.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.605+0000 7fd20bf2a700 1 -- 172.21.15.114:0/213761544 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd204109730 con 0x7fd204100bd0 2024-02-16T22:25:02.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.606+0000 7fd202ffd700 1 -- 172.21.15.114:0/213761544 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 37) v1 ==== 91597+0+0 (secure 0 0 0) 0x7fd1fc00fa00 con 0x7fd204100bd0 2024-02-16T22:25:02.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.606+0000 7fd202ffd700 1 --2- 172.21.15.114:0/213761544 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd1ec06d9e0 0x7fd1ec06fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:02.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.606+0000 7fd202ffd700 1 -- 172.21.15.114:0/213761544 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd1fc08ea20 con 0x7fd204100bd0 2024-02-16T22:25:02.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.606+0000 7fd2094c5700 1 --2- 172.21.15.114:0/213761544 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd1ec06d9e0 0x7fd1ec06fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:02.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.607+0000 7fd2094c5700 1 --2- 172.21.15.114:0/213761544 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd1ec06d9e0 0x7fd1ec06fea0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8005950 tx=0x7fd1f80058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:25:02.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.611+0000 7fd202ffd700 1 -- 172.21.15.114:0/213761544 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd1fc05bb90 con 0x7fd204100bd0 2024-02-16T22:25:02.613 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:02 smithi114 ceph-mon[94452]: from='client.34186 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:25:02.613 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:02 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:02.613 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:02 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:02.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:02 smithi123 ceph-mon[82331]: from='client.34186 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:25:02.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:02 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:02.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:02 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:02.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.852+0000 7fd20bf2a700 1 -- 172.21.15.114:0/213761544 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd20419c580 con 0x7fd204100bd0 2024-02-16T22:25:02.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.853+0000 7fd202ffd700 1 -- 172.21.15.114:0/213761544 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1846 (secure 0 0 0) 0x7fd1fc05b720 con 0x7fd204100bd0 2024-02-16T22:25:02.855 INFO:teuthology.orchestra.run.smithi114.stdout:e14 2024-02-16T22:25:02.855 INFO:teuthology.orchestra.run.smithi114.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:25:02.855 INFO:teuthology.orchestra.run.smithi114.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:25:02.855 INFO:teuthology.orchestra.run.smithi114.stdout:legacy client fscid: 1 2024-02-16T22:25:02.855 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:25:02.855 INFO:teuthology.orchestra.run.smithi114.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:25:02.855 INFO:teuthology.orchestra.run.smithi114.stdout:fs_name cephfs 2024-02-16T22:25:02.855 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 12 2024-02-16T22:25:02.856 INFO:teuthology.orchestra.run.smithi114.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:25:02.856 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-02-16T22:18:02.108825+0000 2024-02-16T22:25:02.856 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-02-16T22:18:16.413371+0000 2024-02-16T22:25:02.856 INFO:teuthology.orchestra.run.smithi114.stdout:tableserver 0 2024-02-16T22:25:02.856 INFO:teuthology.orchestra.run.smithi114.stdout:root 0 2024-02-16T22:25:02.856 INFO:teuthology.orchestra.run.smithi114.stdout:session_timeout 60 2024-02-16T22:25:02.856 INFO:teuthology.orchestra.run.smithi114.stdout:session_autoclose 300 2024-02-16T22:25:02.856 INFO:teuthology.orchestra.run.smithi114.stdout:max_file_size 1099511627776 2024-02-16T22:25:02.856 INFO:teuthology.orchestra.run.smithi114.stdout:max_xattr_size 65536 2024-02-16T22:25:02.856 INFO:teuthology.orchestra.run.smithi114.stdout:required_client_features {} 2024-02-16T22:25:02.856 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure 0 2024-02-16T22:25:02.856 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure_osd_epoch 0 2024-02-16T22:25:02.856 INFO:teuthology.orchestra.run.smithi114.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:25:02.857 INFO:teuthology.orchestra.run.smithi114.stdout:max_mds 1 2024-02-16T22:25:02.857 INFO:teuthology.orchestra.run.smithi114.stdout:in 0 2024-02-16T22:25:02.857 INFO:teuthology.orchestra.run.smithi114.stdout:up {0=14468} 2024-02-16T22:25:02.857 INFO:teuthology.orchestra.run.smithi114.stdout:failed 2024-02-16T22:25:02.857 INFO:teuthology.orchestra.run.smithi114.stdout:damaged 2024-02-16T22:25:02.857 INFO:teuthology.orchestra.run.smithi114.stdout:stopped 2024-02-16T22:25:02.857 INFO:teuthology.orchestra.run.smithi114.stdout:data_pools [3] 2024-02-16T22:25:02.857 INFO:teuthology.orchestra.run.smithi114.stdout:metadata_pool 2 2024-02-16T22:25:02.857 INFO:teuthology.orchestra.run.smithi114.stdout:inline_data enabled 2024-02-16T22:25:02.857 INFO:teuthology.orchestra.run.smithi114.stdout:balancer 2024-02-16T22:25:02.857 INFO:teuthology.orchestra.run.smithi114.stdout:bal_rank_mask -1 2024-02-16T22:25:02.857 INFO:teuthology.orchestra.run.smithi114.stdout:standby_count_wanted 1 2024-02-16T22:25:02.857 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.zzryke{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.114:6826/277953535,v1:172.21.15.114:6827/277953535] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:25:02.857 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:25:02.857 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:25:02.857 INFO:teuthology.orchestra.run.smithi114.stdout:Standby daemons: 2024-02-16T22:25:02.858 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:25:02.858 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.fdxpey{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:25:02.858 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.vhlqbe{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:25:02.858 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.gopcfa{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:25:02.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.855+0000 7fd20bf2a700 1 -- 172.21.15.114:0/213761544 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd1ec06d9e0 msgr2=0x7fd1ec06fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:02.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.855+0000 7fd20bf2a700 1 --2- 172.21.15.114:0/213761544 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd1ec06d9e0 0x7fd1ec06fea0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8005950 tx=0x7fd1f80058e0 comp rx=0 tx=0).stop 2024-02-16T22:25:02.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.855+0000 7fd20bf2a700 1 -- 172.21.15.114:0/213761544 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd204100bd0 msgr2=0x7fd204196360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:02.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.855+0000 7fd20bf2a700 1 --2- 172.21.15.114:0/213761544 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd204100bd0 0x7fd204196360 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc00bda0 tx=0x7fd1fc00be80 comp rx=0 tx=0).stop 2024-02-16T22:25:02.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.855+0000 7fd20bf2a700 1 -- 172.21.15.114:0/213761544 shutdown_connections 2024-02-16T22:25:02.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.855+0000 7fd20bf2a700 1 --2- 172.21.15.114:0/213761544 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd1ec06d9e0 0x7fd1ec06fea0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:02.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.855+0000 7fd20bf2a700 1 --2- 172.21.15.114:0/213761544 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd204100bd0 0x7fd204196360 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:02.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.855+0000 7fd20bf2a700 1 --2- 172.21.15.114:0/213761544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd204101dd0 0x7fd2041968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:02.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.855+0000 7fd20bf2a700 1 -- 172.21.15.114:0/213761544 >> 172.21.15.114:0/213761544 conn(0x7fd2040fc190 msgr2=0x7fd204105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:25:02.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.855+0000 7fd20bf2a700 1 -- 172.21.15.114:0/213761544 shutdown_connections 2024-02-16T22:25:02.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.855+0000 7fd20bf2a700 1 -- 172.21.15.114:0/213761544 wait complete. 2024-02-16T22:25:02.859 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 14 2024-02-16T22:25:02.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.960+0000 7f7881c76700 1 -- 172.21.15.114:0/3385492148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f787c0ff290 msgr2=0x7f787c0ff6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:02.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.960+0000 7f7881c76700 1 --2- 172.21.15.114:0/3385492148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f787c0ff290 0x7f787c0ff6b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f786c009910 tx=0x7f786c009c20 comp rx=0 tx=0).stop 2024-02-16T22:25:02.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.960+0000 7f7881c76700 1 -- 172.21.15.114:0/3385492148 shutdown_connections 2024-02-16T22:25:02.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.960+0000 7f7881c76700 1 --2- 172.21.15.114:0/3385492148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f787c100490 0x7f787c1008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:02.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.960+0000 7f7881c76700 1 --2- 172.21.15.114:0/3385492148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f787c0ff290 0x7f787c0ff6b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:02.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.960+0000 7f7881c76700 1 -- 172.21.15.114:0/3385492148 >> 172.21.15.114:0/3385492148 conn(0x7f787c0fa810 msgr2=0x7f787c0fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:25:02.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.960+0000 7f7881c76700 1 -- 172.21.15.114:0/3385492148 shutdown_connections 2024-02-16T22:25:02.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.960+0000 7f7881c76700 1 -- 172.21.15.114:0/3385492148 wait complete. 2024-02-16T22:25:02.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.961+0000 7f7881c76700 1 Processor -- start 2024-02-16T22:25:02.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.961+0000 7f7881c76700 1 -- start start 2024-02-16T22:25:02.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.961+0000 7f7881c76700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f787c0ff290 0x7f787c104330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:02.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.961+0000 7f7881c76700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f787c100490 0x7f787c102980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:02.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.961+0000 7f7881c76700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c102ec0 con 0x7f787c0ff290 2024-02-16T22:25:02.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.961+0000 7f7881c76700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c103000 con 0x7f787c100490 2024-02-16T22:25:02.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.961+0000 7f787b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f787c0ff290 0x7f787c104330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:02.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.961+0000 7f787b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f787c0ff290 0x7f787c104330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47470/0 (socket says 172.21.15.114:47470) 2024-02-16T22:25:02.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.961+0000 7f787b7fe700 1 -- 172.21.15.114:0/2123985594 learned_addr learned my addr 172.21.15.114:0/2123985594 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:25:02.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.962+0000 7f787affd700 1 --2- 172.21.15.114:0/2123985594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f787c100490 0x7f787c102980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:02.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.962+0000 7f787b7fe700 1 -- 172.21.15.114:0/2123985594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f787c100490 msgr2=0x7f787c102980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:02.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.962+0000 7f787b7fe700 1 --2- 172.21.15.114:0/2123985594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f787c100490 0x7f787c102980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:02.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.962+0000 7f787b7fe700 1 -- 172.21.15.114:0/2123985594 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c0095f0 con 0x7f787c0ff290 2024-02-16T22:25:02.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.962+0000 7f787b7fe700 1 --2- 172.21.15.114:0/2123985594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f787c0ff290 0x7f787c104330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f786c00bda0 tx=0x7f786c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:25:02.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.962+0000 7f7878ff9700 1 -- 172.21.15.114:0/2123985594 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f786c01d070 con 0x7f787c0ff290 2024-02-16T22:25:02.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.962+0000 7f7878ff9700 1 -- 172.21.15.114:0/2123985594 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f786c004c50 con 0x7f787c0ff290 2024-02-16T22:25:02.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.962+0000 7f7881c76700 1 -- 172.21.15.114:0/2123985594 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f787c103280 con 0x7f787c0ff290 2024-02-16T22:25:02.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.962+0000 7f7878ff9700 1 -- 172.21.15.114:0/2123985594 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f786c00f690 con 0x7f787c0ff290 2024-02-16T22:25:02.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.962+0000 7f7881c76700 1 -- 172.21.15.114:0/2123985594 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f787c103770 con 0x7f787c0ff290 2024-02-16T22:25:02.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.963+0000 7f7881c76700 1 -- 172.21.15.114:0/2123985594 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f787c107df0 con 0x7f787c0ff290 2024-02-16T22:25:02.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.964+0000 7f7878ff9700 1 -- 172.21.15.114:0/2123985594 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 37) v1 ==== 91597+0+0 (secure 0 0 0) 0x7f786c004dc0 con 0x7f787c0ff290 2024-02-16T22:25:02.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.964+0000 7f7878ff9700 1 --2- 172.21.15.114:0/2123985594 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f786406d9e0 0x7f786406fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:02.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.964+0000 7f787affd700 1 --2- 172.21.15.114:0/2123985594 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f786406d9e0 0x7f786406fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:02.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.964+0000 7f7878ff9700 1 -- 172.21.15.114:0/2123985594 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f786c08eef0 con 0x7f787c0ff290 2024-02-16T22:25:02.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.964+0000 7f787affd700 1 --2- 172.21.15.114:0/2123985594 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f786406d9e0 0x7f786406fea0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7870005ea0 tx=0x7f7870005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:25:02.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:02.968+0000 7f7878ff9700 1 -- 172.21.15.114:0/2123985594 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f786c05c060 con 0x7f787c0ff290 2024-02-16T22:25:03.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.186+0000 7f7881c76700 1 -- 172.21.15.114:0/2123985594 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f787c19e870 con 0x7f786406d9e0 2024-02-16T22:25:03.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.188+0000 7f7878ff9700 1 -- 172.21.15.114:0/2123985594 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+402 (secure 0 0 0) 0x7f787c19e870 con 0x7f786406d9e0 2024-02-16T22:25:03.190 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:25:03.190 INFO:teuthology.orchestra.run.smithi114.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:25:03.190 INFO:teuthology.orchestra.run.smithi114.stdout: "in_progress": true, 2024-02-16T22:25:03.190 INFO:teuthology.orchestra.run.smithi114.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:25:03.190 INFO:teuthology.orchestra.run.smithi114.stdout: "services_complete": [ 2024-02-16T22:25:03.190 INFO:teuthology.orchestra.run.smithi114.stdout: "crash", 2024-02-16T22:25:03.190 INFO:teuthology.orchestra.run.smithi114.stdout: "mon", 2024-02-16T22:25:03.190 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr" 2024-02-16T22:25:03.190 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-02-16T22:25:03.190 INFO:teuthology.orchestra.run.smithi114.stdout: "progress": "6/23 daemons upgraded", 2024-02-16T22:25:03.190 INFO:teuthology.orchestra.run.smithi114.stdout: "message": "Currently upgrading crash daemons", 2024-02-16T22:25:03.190 INFO:teuthology.orchestra.run.smithi114.stdout: "is_paused": false 2024-02-16T22:25:03.190 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:25:03.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.190+0000 7f7881c76700 1 -- 172.21.15.114:0/2123985594 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f786406d9e0 msgr2=0x7f786406fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:03.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.190+0000 7f7881c76700 1 --2- 172.21.15.114:0/2123985594 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f786406d9e0 0x7f786406fea0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7870005ea0 tx=0x7f7870005e30 comp rx=0 tx=0).stop 2024-02-16T22:25:03.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.190+0000 7f7881c76700 1 -- 172.21.15.114:0/2123985594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f787c0ff290 msgr2=0x7f787c104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:03.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.190+0000 7f7881c76700 1 --2- 172.21.15.114:0/2123985594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f787c0ff290 0x7f787c104330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f786c00bda0 tx=0x7f786c00be80 comp rx=0 tx=0).stop 2024-02-16T22:25:03.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.190+0000 7f7881c76700 1 -- 172.21.15.114:0/2123985594 shutdown_connections 2024-02-16T22:25:03.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.190+0000 7f7881c76700 1 --2- 172.21.15.114:0/2123985594 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f786406d9e0 0x7f786406fea0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:03.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.190+0000 7f7881c76700 1 --2- 172.21.15.114:0/2123985594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f787c0ff290 0x7f787c104330 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:03.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.190+0000 7f7881c76700 1 --2- 172.21.15.114:0/2123985594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f787c100490 0x7f787c102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:03.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.190+0000 7f7881c76700 1 -- 172.21.15.114:0/2123985594 >> 172.21.15.114:0/2123985594 conn(0x7f787c0fa810 msgr2=0x7f787c1016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:25:03.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.190+0000 7f7881c76700 1 -- 172.21.15.114:0/2123985594 shutdown_connections 2024-02-16T22:25:03.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.190+0000 7f7881c76700 1 -- 172.21.15.114:0/2123985594 wait complete. 2024-02-16T22:25:03.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.296+0000 7fe85b0ca700 1 -- 172.21.15.114:0/1950752380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe854100c00 msgr2=0x7fe854101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:03.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.296+0000 7fe85b0ca700 1 --2- 172.21.15.114:0/1950752380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe854100c00 0x7fe854101020 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe848009910 tx=0x7fe848009c20 comp rx=0 tx=0).stop 2024-02-16T22:25:03.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.296+0000 7fe85b0ca700 1 -- 172.21.15.114:0/1950752380 shutdown_connections 2024-02-16T22:25:03.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.296+0000 7fe85b0ca700 1 --2- 172.21.15.114:0/1950752380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe854101e00 0x7fe854102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:03.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.296+0000 7fe85b0ca700 1 --2- 172.21.15.114:0/1950752380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe854100c00 0x7fe854101020 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:03.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.296+0000 7fe85b0ca700 1 -- 172.21.15.114:0/1950752380 >> 172.21.15.114:0/1950752380 conn(0x7fe8540fc180 msgr2=0x7fe8540fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:25:03.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.296+0000 7fe85b0ca700 1 -- 172.21.15.114:0/1950752380 shutdown_connections 2024-02-16T22:25:03.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.296+0000 7fe85b0ca700 1 -- 172.21.15.114:0/1950752380 wait complete. 2024-02-16T22:25:03.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.296+0000 7fe85b0ca700 1 Processor -- start 2024-02-16T22:25:03.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.296+0000 7fe85b0ca700 1 -- start start 2024-02-16T22:25:03.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.296+0000 7fe85b0ca700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe854100c00 0x7fe854074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:03.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.296+0000 7fe85b0ca700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe854101e00 0x7fe854073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:03.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.296+0000 7fe85b0ca700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8540736c0 con 0x7fe854101e00 2024-02-16T22:25:03.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.296+0000 7fe85b0ca700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe854073800 con 0x7fe854100c00 2024-02-16T22:25:03.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.297+0000 7fe853fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe854101e00 0x7fe854073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:03.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.297+0000 7fe853fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe854101e00 0x7fe854073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47480/0 (socket says 172.21.15.114:47480) 2024-02-16T22:25:03.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.297+0000 7fe853fff700 1 -- 172.21.15.114:0/2268976841 learned_addr learned my addr 172.21.15.114:0/2268976841 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:25:03.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.297+0000 7fe858e66700 1 --2- 172.21.15.114:0/2268976841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe854100c00 0x7fe854074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:03.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.297+0000 7fe853fff700 1 -- 172.21.15.114:0/2268976841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe854100c00 msgr2=0x7fe854074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:03.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.297+0000 7fe853fff700 1 --2- 172.21.15.114:0/2268976841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe854100c00 0x7fe854074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:03.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.297+0000 7fe853fff700 1 -- 172.21.15.114:0/2268976841 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8480095f0 con 0x7fe854101e00 2024-02-16T22:25:03.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.297+0000 7fe853fff700 1 --2- 172.21.15.114:0/2268976841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe854101e00 0x7fe854073180 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe84000e920 tx=0x7fe84000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:25:03.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.297+0000 7fe851ffb700 1 -- 172.21.15.114:0/2268976841 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe84000c9d0 con 0x7fe854101e00 2024-02-16T22:25:03.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.297+0000 7fe85b0ca700 1 -- 172.21.15.114:0/2268976841 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe854073ae0 con 0x7fe854101e00 2024-02-16T22:25:03.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.297+0000 7fe851ffb700 1 -- 172.21.15.114:0/2268976841 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe840010430 con 0x7fe854101e00 2024-02-16T22:25:03.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.297+0000 7fe851ffb700 1 -- 172.21.15.114:0/2268976841 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe840018650 con 0x7fe854101e00 2024-02-16T22:25:03.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.298+0000 7fe85b0ca700 1 -- 172.21.15.114:0/2268976841 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe85419e3f0 con 0x7fe854101e00 2024-02-16T22:25:03.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.299+0000 7fe85b0ca700 1 -- 172.21.15.114:0/2268976841 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe854109760 con 0x7fe854101e00 2024-02-16T22:25:03.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.299+0000 7fe851ffb700 1 -- 172.21.15.114:0/2268976841 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 37) v1 ==== 91597+0+0 (secure 0 0 0) 0x7fe8400105a0 con 0x7fe854101e00 2024-02-16T22:25:03.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.299+0000 7fe851ffb700 1 --2- 172.21.15.114:0/2268976841 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe84406d9e0 0x7fe84406fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:03.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.300+0000 7fe851ffb700 1 -- 172.21.15.114:0/2268976841 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe840014070 con 0x7fe854101e00 2024-02-16T22:25:03.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.300+0000 7fe858e66700 1 --2- 172.21.15.114:0/2268976841 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe84406d9e0 0x7fe84406fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:03.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.300+0000 7fe858e66700 1 --2- 172.21.15.114:0/2268976841 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe84406d9e0 0x7fe84406fea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe84800b6f0 tx=0x7fe848005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:25:03.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.304+0000 7fe851ffb700 1 -- 172.21.15.114:0/2268976841 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe840052d20 con 0x7fe854101e00 2024-02-16T22:25:03.536 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:03 smithi114 ceph-mon[94452]: from='client.34190 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:25:03.536 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:03 smithi114 ceph-mon[94452]: pgmap v41: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:25:03.536 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:03 smithi114 ceph-mon[94452]: from='client.44129 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:25:03.536 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:03 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/3182401550' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:25:03.536 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:03 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:03.537 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:03 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:03.537 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:03 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/213761544' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:25:03.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.603+0000 7fe85b0ca700 1 -- 172.21.15.114:0/2268976841 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe854109920 con 0x7fe854101e00 2024-02-16T22:25:03.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.603+0000 7fe851ffb700 1 -- 172.21.15.114:0/2268976841 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fe84005ab30 con 0x7fe854101e00 2024-02-16T22:25:03.605 INFO:teuthology.orchestra.run.smithi114.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:25:03.605 INFO:teuthology.orchestra.run.smithi114.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:25:03.605 INFO:teuthology.orchestra.run.smithi114.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:25:03.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.605+0000 7fe85b0ca700 1 -- 172.21.15.114:0/2268976841 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe84406d9e0 msgr2=0x7fe84406fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:03.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.605+0000 7fe85b0ca700 1 --2- 172.21.15.114:0/2268976841 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe84406d9e0 0x7fe84406fea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe84800b6f0 tx=0x7fe848005940 comp rx=0 tx=0).stop 2024-02-16T22:25:03.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.605+0000 7fe85b0ca700 1 -- 172.21.15.114:0/2268976841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe854101e00 msgr2=0x7fe854073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:03.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.605+0000 7fe85b0ca700 1 --2- 172.21.15.114:0/2268976841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe854101e00 0x7fe854073180 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe84000e920 tx=0x7fe84000ec30 comp rx=0 tx=0).stop 2024-02-16T22:25:03.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.605+0000 7fe85b0ca700 1 -- 172.21.15.114:0/2268976841 shutdown_connections 2024-02-16T22:25:03.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.605+0000 7fe85b0ca700 1 --2- 172.21.15.114:0/2268976841 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe84406d9e0 0x7fe84406fea0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:03.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.605+0000 7fe85b0ca700 1 --2- 172.21.15.114:0/2268976841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe854100c00 0x7fe854074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:03.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.605+0000 7fe85b0ca700 1 --2- 172.21.15.114:0/2268976841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe854101e00 0x7fe854073180 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:03.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.605+0000 7fe85b0ca700 1 -- 172.21.15.114:0/2268976841 >> 172.21.15.114:0/2268976841 conn(0x7fe8540fc180 msgr2=0x7fe854105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:25:03.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.605+0000 7fe85b0ca700 1 -- 172.21.15.114:0/2268976841 shutdown_connections 2024-02-16T22:25:03.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:03.605+0000 7fe85b0ca700 1 -- 172.21.15.114:0/2268976841 wait complete. 2024-02-16T22:25:03.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:03 smithi123 ceph-mon[82331]: from='client.34190 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:25:03.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:03 smithi123 ceph-mon[82331]: pgmap v41: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:25:03.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:03 smithi123 ceph-mon[82331]: from='client.44129 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:25:03.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:03 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/3182401550' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:25:03.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:03 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:03.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:03 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:03.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:03 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/213761544' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:25:04.656 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:04 smithi114 ceph-mon[94452]: from='client.34206 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:25:04.656 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:04 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/2268976841' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:25:04.741 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:04 smithi123 ceph-mon[82331]: from='client.34206 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:25:04.741 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:04 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/2268976841' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:25:05.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:05 smithi114 ceph-mon[94452]: pgmap v42: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:25:05.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:05 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:05.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:05 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:05.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:05 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:25:05.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:05 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:25:05.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:05 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:05.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:05 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:25:05.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:05 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:25:05.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:05 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:25:05.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:05 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:25:05.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:05 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:05.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:05 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi114"}]: dispatch 2024-02-16T22:25:05.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:05 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi114"}]': finished 2024-02-16T22:25:05.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:05 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi123"}]: dispatch 2024-02-16T22:25:05.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:05 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi123"}]': finished 2024-02-16T22:25:05.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:05 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-16T22:25:05.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:05 smithi123 ceph-mon[82331]: pgmap v42: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:25:05.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:05 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:05.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:05 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:05.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:05 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:25:05.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:05 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:25:05.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:05 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:05.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:05 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:25:05.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:05 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:25:05.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:05 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:25:05.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:05 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:25:05.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:05 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:05.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:05 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi114"}]: dispatch 2024-02-16T22:25:05.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:05 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi114"}]': finished 2024-02-16T22:25:05.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:05 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi123"}]: dispatch 2024-02-16T22:25:05.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:05 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi123"}]': finished 2024-02-16T22:25:05.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:05 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-16T22:25:06.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:06 smithi114 ceph-mon[94452]: Upgrade: Setting container_image for all crash 2024-02-16T22:25:06.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:06 smithi114 ceph-mon[94452]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-16T22:25:06.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:06 smithi114 ceph-mon[94452]: Upgrade: osd.0 is safe to restart 2024-02-16T22:25:06.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:06 smithi123 ceph-mon[82331]: Upgrade: Setting container_image for all crash 2024-02-16T22:25:06.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:06 smithi123 ceph-mon[82331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-16T22:25:06.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:06 smithi123 ceph-mon[82331]: Upgrade: osd.0 is safe to restart 2024-02-16T22:25:07.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:07 smithi114 ceph-mon[94452]: pgmap v43: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:25:07.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:07 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:07.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:07 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-16T22:25:07.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:07 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:25:07.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:07 smithi123 ceph-mon[82331]: pgmap v43: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:25:07.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:07 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:07.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:07 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-16T22:25:07.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:07 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:25:08.673 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:08 smithi114 ceph-mon[94452]: Upgrade: Updating osd.0 2024-02-16T22:25:08.674 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:08 smithi114 ceph-mon[94452]: Deploying daemon osd.0 on smithi114 2024-02-16T22:25:08.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:08 smithi123 ceph-mon[82331]: Upgrade: Updating osd.0 2024-02-16T22:25:08.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:08 smithi123 ceph-mon[82331]: Deploying daemon osd.0 on smithi114 2024-02-16T22:25:09.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:09 smithi114 ceph-mon[94452]: pgmap v44: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:25:09.787 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:09 smithi123 ceph-mon[82331]: pgmap v44: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:25:10.451 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:10 smithi114 systemd[1]: Stopping Ceph osd.0 for 1dcbaabe-cd18-11ee-95ba-87774f69a715... 2024-02-16T22:25:10.452 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:10 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0[48477]: 2024-02-16T22:25:10.374+0000 7f6d32b56700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T22:25:10.452 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:10 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0[48477]: 2024-02-16T22:25:10.374+0000 7f6d32b56700 -1 osd.0 48 *** Got signal Terminated *** 2024-02-16T22:25:10.452 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:10 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0[48477]: 2024-02-16T22:25:10.374+0000 7f6d32b56700 -1 osd.0 48 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T22:25:10.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:10 smithi123 ceph-mon[82331]: osd.0 marked itself down and dead 2024-02-16T22:25:10.845 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:10 smithi114 ceph-mon[94452]: osd.0 marked itself down and dead 2024-02-16T22:25:11.201 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:10 smithi114 podman[103080]: 2024-02-16 22:25:10.946625221 +0000 UTC m=+0.674784670 container died 24ca2e9714e0667c8723eead1ffc3e18ae2ae5bcfbdef58a90b38c215a45dc6a (image=quay.io/ceph/ceph@sha256:6176526d5ef4047f148d47c951653dcc6b0de49091fda006d3a088e7a1f404d0, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T22:25:11.585 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:11 smithi114 ceph-mon[94452]: pgmap v45: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:25:11.586 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:11 smithi114 ceph-mon[94452]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T22:25:11.586 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:11 smithi114 ceph-mon[94452]: osdmap e49: 6 total, 5 up, 6 in 2024-02-16T22:25:11.586 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:11 smithi114 podman[103080]: 2024-02-16 22:25:11.324712723 +0000 UTC m=+1.052872172 container cleanup 24ca2e9714e0667c8723eead1ffc3e18ae2ae5bcfbdef58a90b38c215a45dc6a (image=quay.io/ceph/ceph@sha256:6176526d5ef4047f148d47c951653dcc6b0de49091fda006d3a088e7a1f404d0, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, RELEASE=HEAD) 2024-02-16T22:25:11.586 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:11 smithi114 bash[103080]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0 2024-02-16T22:25:11.722 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:11 smithi123 ceph-mon[82331]: pgmap v45: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:25:11.722 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:11 smithi123 ceph-mon[82331]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T22:25:11.722 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:11 smithi123 ceph-mon[82331]: osdmap e49: 6 total, 5 up, 6 in 2024-02-16T22:25:11.932 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:11 smithi114 podman[103096]: 2024-02-16 22:25:11.584713814 +0000 UTC m=+0.634358339 container remove 24ca2e9714e0667c8723eead1ffc3e18ae2ae5bcfbdef58a90b38c215a45dc6a (image=quay.io/ceph/ceph@sha256:6176526d5ef4047f148d47c951653dcc6b0de49091fda006d3a088e7a1f404d0, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, RELEASE=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T22:25:12.198 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:11 smithi114 podman[103190]: 2024-02-16 22:25:11.834479957 +0000 UTC m=+0.017762514 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T22:25:12.198 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:11 smithi114 podman[103190]: 2024-02-16 22:25:11.944121652 +0000 UTC m=+0.127404197 container create 8e5410612a3d58a8cfb662af45072d12b6ec87becf8c794369d578ccfbc0eed8 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD) 2024-02-16T22:25:12.198 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:12 smithi114 podman[103190]: 2024-02-16 22:25:12.089051472 +0000 UTC m=+0.272334026 container init 8e5410612a3d58a8cfb662af45072d12b6ec87becf8c794369d578ccfbc0eed8 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-02-16T22:25:12.198 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:12 smithi114 podman[103190]: 2024-02-16 22:25:12.100999963 +0000 UTC m=+0.284282506 container start 8e5410612a3d58a8cfb662af45072d12b6ec87becf8c794369d578ccfbc0eed8 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-02-16T22:25:12.199 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:12 smithi114 podman[103190]: 2024-02-16 22:25:12.196957692 +0000 UTC m=+0.380240256 container attach 8e5410612a3d58a8cfb662af45072d12b6ec87becf8c794369d578ccfbc0eed8 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-02-16T22:25:12.451 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:12 smithi114 podman[103190]: 2024-02-16 22:25:12.392374343 +0000 UTC m=+0.575656888 container died 8e5410612a3d58a8cfb662af45072d12b6ec87becf8c794369d578ccfbc0eed8 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214) 2024-02-16T22:25:12.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:12 smithi123 ceph-mon[82331]: osdmap e50: 6 total, 5 up, 6 in 2024-02-16T22:25:12.948 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:12 smithi114 ceph-mon[94452]: osdmap e50: 6 total, 5 up, 6 in 2024-02-16T22:25:13.201 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:12 smithi114 podman[103218]: 2024-02-16 22:25:12.992466335 +0000 UTC m=+0.595220909 container remove 8e5410612a3d58a8cfb662af45072d12b6ec87becf8c794369d578ccfbc0eed8 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2) 2024-02-16T22:25:13.202 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:13 smithi114 systemd[1]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@osd.0.service: Deactivated successfully. 2024-02-16T22:25:13.202 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:13 smithi114 systemd[1]: Stopped Ceph osd.0 for 1dcbaabe-cd18-11ee-95ba-87774f69a715. 2024-02-16T22:25:13.202 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:13 smithi114 systemd[1]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@osd.0.service: Consumed 51.400s CPU time. 2024-02-16T22:25:13.478 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:13 smithi114 systemd[1]: Starting Ceph osd.0 for 1dcbaabe-cd18-11ee-95ba-87774f69a715... 2024-02-16T22:25:13.479 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:13 smithi114 podman[103321]: 2024-02-16 22:25:13.435195467 +0000 UTC m=+0.110052487 container create 996a37520fd34ee00421495e0949cfdb3a287390dbdeb3533a7a9ea6ddc55a5f (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0-activate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214) 2024-02-16T22:25:13.479 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:13 smithi114 podman[103321]: 2024-02-16 22:25:13.342025252 +0000 UTC m=+0.016882272 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T22:25:13.739 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:13 smithi114 ceph-mon[94452]: pgmap v48: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/267 objects degraded (19.101%) 2024-02-16T22:25:13.739 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:13 smithi114 ceph-mon[94452]: Health check failed: Degraded data redundancy: 51/267 objects degraded (19.101%), 19 pgs degraded (PG_DEGRADED) 2024-02-16T22:25:13.739 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:13 smithi114 podman[103321]: 2024-02-16 22:25:13.638668318 +0000 UTC m=+0.313525341 container init 996a37520fd34ee00421495e0949cfdb3a287390dbdeb3533a7a9ea6ddc55a5f (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0-activate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-02-16T22:25:13.739 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:13 smithi114 podman[103321]: 2024-02-16 22:25:13.641988148 +0000 UTC m=+0.316845168 container start 996a37520fd34ee00421495e0949cfdb3a287390dbdeb3533a7a9ea6ddc55a5f (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0-activate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T22:25:13.740 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:13 smithi114 podman[103321]: 2024-02-16 22:25:13.738161436 +0000 UTC m=+0.413018469 container attach 996a37520fd34ee00421495e0949cfdb3a287390dbdeb3533a7a9ea6ddc55a5f (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0-activate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240214) 2024-02-16T22:25:13.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:13 smithi123 ceph-mon[82331]: pgmap v48: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/267 objects degraded (19.101%) 2024-02-16T22:25:13.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:13 smithi123 ceph-mon[82331]: Health check failed: Degraded data redundancy: 51/267 objects degraded (19.101%), 19 pgs degraded (PG_DEGRADED) 2024-02-16T22:25:14.452 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:14 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0-activate[103336]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-16T22:25:14.452 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:14 smithi114 bash[103321]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-16T22:25:14.452 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:14 smithi114 bash[103321]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-02-16T22:25:14.452 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:14 smithi114 bash[103321]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-02-16T22:25:14.452 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:14 smithi114 bash[103321]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-02-16T22:25:14.452 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:14 smithi114 bash[103321]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-02-16T22:25:14.452 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:14 smithi114 bash[103321]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-16T22:25:14.452 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:14 smithi114 bash[103321]: --> ceph-volume raw activate successful for osd ID: 0 2024-02-16T22:25:14.452 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:14 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0-activate[103336]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-02-16T22:25:14.452 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:14 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0-activate[103336]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-02-16T22:25:14.452 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:14 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0-activate[103336]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-02-16T22:25:14.453 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:14 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0-activate[103336]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-02-16T22:25:14.453 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:14 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0-activate[103336]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-16T22:25:14.453 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:14 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0-activate[103336]: --> ceph-volume raw activate successful for osd ID: 0 2024-02-16T22:25:14.453 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:14 smithi114 podman[103321]: 2024-02-16 22:25:14.10164861 +0000 UTC m=+0.776505631 container died 996a37520fd34ee00421495e0949cfdb3a287390dbdeb3533a7a9ea6ddc55a5f (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-02-16T22:25:15.059 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:14 smithi114 podman[103466]: 2024-02-16 22:25:14.725637135 +0000 UTC m=+0.618882417 container remove 996a37520fd34ee00421495e0949cfdb3a287390dbdeb3533a7a9ea6ddc55a5f (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0-activate, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-02-16T22:25:15.059 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:14 smithi114 podman[103532]: 2024-02-16 22:25:14.843141786 +0000 UTC m=+0.018350284 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T22:25:15.059 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:14 smithi114 podman[103532]: 2024-02-16 22:25:14.960173511 +0000 UTC m=+0.135382051 container create 1978fdad94a55f95dc3bc9df43b45ad517bc5ac0b5547294892a4d45d98c9dd6 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=) 2024-02-16T22:25:15.313 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:15 smithi114 podman[103532]: 2024-02-16 22:25:15.154844621 +0000 UTC m=+0.330053123 container init 1978fdad94a55f95dc3bc9df43b45ad517bc5ac0b5547294892a4d45d98c9dd6 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True) 2024-02-16T22:25:15.314 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:15 smithi114 podman[103532]: 2024-02-16 22:25:15.158264636 +0000 UTC m=+0.333473142 container start 1978fdad94a55f95dc3bc9df43b45ad517bc5ac0b5547294892a4d45d98c9dd6 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-02-16T22:25:15.314 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:15 smithi114 ceph-osd[103552]: -- 172.21.15.114:0/1014258047 <== mon.0 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55725d2c71e0 con 0x55725d298c00 2024-02-16T22:25:15.314 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:15 smithi114 bash[103532]: 1978fdad94a55f95dc3bc9df43b45ad517bc5ac0b5547294892a4d45d98c9dd6 2024-02-16T22:25:15.314 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:15 smithi114 systemd[1]: Started Ceph osd.0 for 1dcbaabe-cd18-11ee-95ba-87774f69a715. 2024-02-16T22:25:15.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:15 smithi114 ceph-mon[94452]: pgmap v49: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/267 objects degraded (19.101%) 2024-02-16T22:25:15.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:15 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:15.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:15 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:15.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:15 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:25:15.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:15 smithi123 ceph-mon[82331]: pgmap v49: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/267 objects degraded (19.101%) 2024-02-16T22:25:15.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:15 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:15.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:15 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:15.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:15 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:25:16.201 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:15 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0[103547]: 2024-02-16T22:25:15.706+0000 7f35a872a7c0 -1 Falling back to public interface 2024-02-16T22:25:17.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:16 smithi114 ceph-mon[94452]: pgmap v50: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/267 objects degraded (19.101%) 2024-02-16T22:25:17.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:16 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:17.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:16 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:25:17.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:16 smithi123 ceph-mon[82331]: pgmap v50: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/267 objects degraded (19.101%) 2024-02-16T22:25:17.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:16 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:17.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:16 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:25:17.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:17 smithi114 ceph-mon[94452]: mgrmap e38: smithi114.vcullz(active, since 91s), standbys: smithi123.inzgjd 2024-02-16T22:25:18.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:17 smithi123 ceph-mon[82331]: mgrmap e38: smithi114.vcullz(active, since 91s), standbys: smithi123.inzgjd 2024-02-16T22:25:19.173 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:18 smithi114 ceph-mon[94452]: pgmap v51: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 383 B/s wr, 0 op/s; 51/267 objects degraded (19.101%) 2024-02-16T22:25:19.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:18 smithi123 ceph-mon[82331]: pgmap v51: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 383 B/s wr, 0 op/s; 51/267 objects degraded (19.101%) 2024-02-16T22:25:20.160 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:20 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0[103547]: 2024-02-16T22:25:20.149+0000 7f35a872a7c0 -1 osd.0 0 read_superblock omap replica is missing. 2024-02-16T22:25:20.835 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:20 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0[103547]: 2024-02-16T22:25:20.472+0000 7f35a872a7c0 -1 osd.0 48 log_to_monitors true 2024-02-16T22:25:21.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:20 smithi114 ceph-mon[94452]: pgmap v52: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 330 B/s wr, 0 op/s; 51/267 objects degraded (19.101%) 2024-02-16T22:25:21.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:20 smithi114 ceph-mon[94452]: from='osd.0 [v2:172.21.15.114:6802/221411803,v1:172.21.15.114:6803/221411803]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-16T22:25:21.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:20 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:21.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:20 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:21.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:20 smithi123 ceph-mon[82331]: pgmap v52: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 330 B/s wr, 0 op/s; 51/267 objects degraded (19.101%) 2024-02-16T22:25:21.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:20 smithi123 ceph-mon[82331]: from='osd.0 [v2:172.21.15.114:6802/221411803,v1:172.21.15.114:6803/221411803]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-16T22:25:21.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:20 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:21.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:20 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:22.201 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:25:21 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0[103547]: 2024-02-16T22:25:21.772+0000 7f3595b7a700 -1 osd.0 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T22:25:22.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:21 smithi114 ceph-mon[94452]: from='osd.0 [v2:172.21.15.114:6802/221411803,v1:172.21.15.114:6803/221411803]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-16T22:25:22.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:21 smithi114 ceph-mon[94452]: osdmap e51: 6 total, 5 up, 6 in 2024-02-16T22:25:22.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:21 smithi114 ceph-mon[94452]: from='osd.0 [v2:172.21.15.114:6802/221411803,v1:172.21.15.114:6803/221411803]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-02-16T22:25:22.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:21 smithi114 ceph-mon[94452]: from='osd.0 [v2:172.21.15.114:6802/221411803,v1:172.21.15.114:6803/221411803]' entity='osd.0' 2024-02-16T22:25:22.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:21 smithi123 ceph-mon[82331]: from='osd.0 [v2:172.21.15.114:6802/221411803,v1:172.21.15.114:6803/221411803]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-16T22:25:22.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:21 smithi123 ceph-mon[82331]: osdmap e51: 6 total, 5 up, 6 in 2024-02-16T22:25:22.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:21 smithi123 ceph-mon[82331]: from='osd.0 [v2:172.21.15.114:6802/221411803,v1:172.21.15.114:6803/221411803]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-02-16T22:25:22.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:21 smithi123 ceph-mon[82331]: from='osd.0 [v2:172.21.15.114:6802/221411803,v1:172.21.15.114:6803/221411803]' entity='osd.0' 2024-02-16T22:25:23.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:22 smithi114 ceph-mon[94452]: pgmap v54: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 307 B/s wr, 0 op/s; 51/267 objects degraded (19.101%) 2024-02-16T22:25:23.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:22 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:23.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:22 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:23.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:22 smithi114 ceph-mon[94452]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T22:25:23.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:22 smithi114 ceph-mon[94452]: osd.0 [v2:172.21.15.114:6802/221411803,v1:172.21.15.114:6803/221411803] boot 2024-02-16T22:25:23.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:22 smithi114 ceph-mon[94452]: osdmap e52: 6 total, 6 up, 6 in 2024-02-16T22:25:23.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:22 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:25:23.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:22 smithi123 ceph-mon[82331]: pgmap v54: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 307 B/s wr, 0 op/s; 51/267 objects degraded (19.101%) 2024-02-16T22:25:23.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:22 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:23.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:22 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:23.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:22 smithi123 ceph-mon[82331]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T22:25:23.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:22 smithi123 ceph-mon[82331]: osd.0 [v2:172.21.15.114:6802/221411803,v1:172.21.15.114:6803/221411803] boot 2024-02-16T22:25:23.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:22 smithi123 ceph-mon[82331]: osdmap e52: 6 total, 6 up, 6 in 2024-02-16T22:25:23.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:22 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T22:25:24.868 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:24 smithi114 ceph-mon[94452]: pgmap v56: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 383 B/s wr, 0 op/s; 51/267 objects degraded (19.101%) 2024-02-16T22:25:24.869 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:24 smithi114 ceph-mon[94452]: osdmap e53: 6 total, 6 up, 6 in 2024-02-16T22:25:25.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:24 smithi123 ceph-mon[82331]: pgmap v56: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 383 B/s wr, 0 op/s; 51/267 objects degraded (19.101%) 2024-02-16T22:25:25.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:24 smithi123 ceph-mon[82331]: osdmap e53: 6 total, 6 up, 6 in 2024-02-16T22:25:26.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:26 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:26.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:26 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:26.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:26 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:25:26.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:26 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:25:26.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:26 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:26.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:26 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:25:26.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:26 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:25:26.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:26 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:25:26.453 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:26 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:25:26.453 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:26 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T22:25:26.453 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:26 smithi114 ceph-mon[94452]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T22:25:26.453 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:26 smithi114 ceph-mon[94452]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-02-16T22:25:26.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:26 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:26.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:26 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:26.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:26 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:25:26.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:26 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:25:26.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:26 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:26.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:26 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:25:26.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:26 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:25:26.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:26 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:25:26.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:26 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:25:26.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:26 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T22:25:26.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:26 smithi123 ceph-mon[82331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T22:25:26.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:26 smithi123 ceph-mon[82331]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-02-16T22:25:27.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:27 smithi123 ceph-mon[82331]: pgmap v58: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/267 objects degraded (19.101%) 2024-02-16T22:25:27.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:27 smithi114 ceph-mon[94452]: pgmap v58: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/267 objects degraded (19.101%) 2024-02-16T22:25:28.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:28 smithi123 ceph-mon[82331]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 51/267 objects degraded (19.101%), 19 pgs degraded) 2024-02-16T22:25:28.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:28 smithi114 ceph-mon[94452]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 51/267 objects degraded (19.101%), 19 pgs degraded) 2024-02-16T22:25:29.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:29 smithi123 ceph-mon[82331]: pgmap v59: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 40 B/s, 0 objects/s recovering 2024-02-16T22:25:29.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:29 smithi114 ceph-mon[94452]: pgmap v59: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 40 B/s, 0 objects/s recovering 2024-02-16T22:25:31.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:31 smithi114 ceph-mon[94452]: pgmap v60: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 35 B/s, 0 objects/s recovering 2024-02-16T22:25:31.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:31 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:31.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:31 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:25:31.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:31 smithi123 ceph-mon[82331]: pgmap v60: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 35 B/s, 0 objects/s recovering 2024-02-16T22:25:31.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:31 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:31.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:31 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:25:33.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:33 smithi123 ceph-mon[82331]: pgmap v61: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 31 B/s, 0 objects/s recovering 2024-02-16T22:25:33.581 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:33 smithi114 ceph-mon[94452]: pgmap v61: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 31 B/s, 0 objects/s recovering 2024-02-16T22:25:33.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.723+0000 7ff19574d700 1 -- 172.21.15.114:0/329242109 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1900ffab0 msgr2=0x7ff1900ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:33.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.723+0000 7ff19574d700 1 --2- 172.21.15.114:0/329242109 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1900ffab0 0x7ff1900ffed0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff188009910 tx=0x7ff188009c20 comp rx=0 tx=0).stop 2024-02-16T22:25:33.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.723+0000 7ff19574d700 1 -- 172.21.15.114:0/329242109 shutdown_connections 2024-02-16T22:25:33.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.723+0000 7ff19574d700 1 --2- 172.21.15.114:0/329242109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff190100cb0 0x7ff190101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:33.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.723+0000 7ff19574d700 1 --2- 172.21.15.114:0/329242109 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1900ffab0 0x7ff1900ffed0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:33.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.723+0000 7ff19574d700 1 -- 172.21.15.114:0/329242109 >> 172.21.15.114:0/329242109 conn(0x7ff1900fb030 msgr2=0x7ff1900fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:25:33.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.723+0000 7ff19574d700 1 -- 172.21.15.114:0/329242109 shutdown_connections 2024-02-16T22:25:33.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.723+0000 7ff19574d700 1 -- 172.21.15.114:0/329242109 wait complete. 2024-02-16T22:25:33.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.724+0000 7ff19574d700 1 Processor -- start 2024-02-16T22:25:33.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.724+0000 7ff19574d700 1 -- start start 2024-02-16T22:25:33.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.724+0000 7ff19574d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1900ffab0 0x7ff190195240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:33.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.724+0000 7ff19574d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff190100cb0 0x7ff190195780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:33.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.724+0000 7ff19574d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff190195d10 con 0x7ff190100cb0 2024-02-16T22:25:33.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.724+0000 7ff19574d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff190195e50 con 0x7ff1900ffab0 2024-02-16T22:25:33.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.725+0000 7ff18e7fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff190100cb0 0x7ff190195780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:33.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.725+0000 7ff18e7fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff190100cb0 0x7ff190195780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58154/0 (socket says 172.21.15.114:58154) 2024-02-16T22:25:33.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.725+0000 7ff18e7fc700 1 -- 172.21.15.114:0/2304444599 learned_addr learned my addr 172.21.15.114:0/2304444599 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:25:33.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.725+0000 7ff18effd700 1 --2- 172.21.15.114:0/2304444599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1900ffab0 0x7ff190195240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:33.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.725+0000 7ff18e7fc700 1 -- 172.21.15.114:0/2304444599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1900ffab0 msgr2=0x7ff190195240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:33.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.725+0000 7ff18e7fc700 1 --2- 172.21.15.114:0/2304444599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1900ffab0 0x7ff190195240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:33.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.725+0000 7ff18e7fc700 1 -- 172.21.15.114:0/2304444599 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1880095f0 con 0x7ff190100cb0 2024-02-16T22:25:33.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.725+0000 7ff18effd700 1 --2- 172.21.15.114:0/2304444599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1900ffab0 0x7ff190195240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:25:33.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.725+0000 7ff18e7fc700 1 --2- 172.21.15.114:0/2304444599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff190100cb0 0x7ff190195780 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff178007b50 tx=0x7ff178007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:25:33.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.725+0000 7ff187fff700 1 -- 172.21.15.114:0/2304444599 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff178011070 con 0x7ff190100cb0 2024-02-16T22:25:33.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.725+0000 7ff187fff700 1 -- 172.21.15.114:0/2304444599 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff17800c2e0 con 0x7ff190100cb0 2024-02-16T22:25:33.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.725+0000 7ff19574d700 1 -- 172.21.15.114:0/2304444599 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff190068470 con 0x7ff190100cb0 2024-02-16T22:25:33.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.725+0000 7ff187fff700 1 -- 172.21.15.114:0/2304444599 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff17800acf0 con 0x7ff190100cb0 2024-02-16T22:25:33.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.725+0000 7ff19574d700 1 -- 172.21.15.114:0/2304444599 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff190068990 con 0x7ff190100cb0 2024-02-16T22:25:33.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.726+0000 7ff19574d700 1 -- 172.21.15.114:0/2304444599 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff190108610 con 0x7ff190100cb0 2024-02-16T22:25:33.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.727+0000 7ff187fff700 1 -- 172.21.15.114:0/2304444599 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7ff17800e710 con 0x7ff190100cb0 2024-02-16T22:25:33.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.727+0000 7ff187fff700 1 --2- 172.21.15.114:0/2304444599 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff17c06d8a0 0x7ff17c06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:33.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.727+0000 7ff187fff700 1 -- 172.21.15.114:0/2304444599 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff17808e6b0 con 0x7ff190100cb0 2024-02-16T22:25:33.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.727+0000 7ff18effd700 1 --2- 172.21.15.114:0/2304444599 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff17c06d8a0 0x7ff17c06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:33.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.728+0000 7ff18effd700 1 --2- 172.21.15.114:0/2304444599 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff17c06d8a0 0x7ff17c06fd60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff188006010 tx=0x7ff188005ab0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:25:33.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.732+0000 7ff187fff700 1 -- 172.21.15.114:0/2304444599 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff17805bac0 con 0x7ff190100cb0 2024-02-16T22:25:33.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.946+0000 7ff19574d700 1 -- 172.21.15.114:0/2304444599 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff190105600 con 0x7ff17c06d8a0 2024-02-16T22:25:33.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.947+0000 7ff187fff700 1 -- 172.21.15.114:0/2304444599 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7ff190105600 con 0x7ff17c06d8a0 2024-02-16T22:25:33.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.950+0000 7ff19574d700 1 -- 172.21.15.114:0/2304444599 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff17c06d8a0 msgr2=0x7ff17c06fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:33.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.950+0000 7ff19574d700 1 --2- 172.21.15.114:0/2304444599 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff17c06d8a0 0x7ff17c06fd60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff188006010 tx=0x7ff188005ab0 comp rx=0 tx=0).stop 2024-02-16T22:25:33.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.950+0000 7ff19574d700 1 -- 172.21.15.114:0/2304444599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff190100cb0 msgr2=0x7ff190195780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:33.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.950+0000 7ff19574d700 1 --2- 172.21.15.114:0/2304444599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff190100cb0 0x7ff190195780 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff178007b50 tx=0x7ff178007f10 comp rx=0 tx=0).stop 2024-02-16T22:25:33.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.950+0000 7ff19574d700 1 -- 172.21.15.114:0/2304444599 shutdown_connections 2024-02-16T22:25:33.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.950+0000 7ff19574d700 1 --2- 172.21.15.114:0/2304444599 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff17c06d8a0 0x7ff17c06fd60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:33.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.950+0000 7ff19574d700 1 --2- 172.21.15.114:0/2304444599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1900ffab0 0x7ff190195240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:33.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.950+0000 7ff19574d700 1 --2- 172.21.15.114:0/2304444599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff190100cb0 0x7ff190195780 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:33.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.950+0000 7ff19574d700 1 -- 172.21.15.114:0/2304444599 >> 172.21.15.114:0/2304444599 conn(0x7ff1900fb030 msgr2=0x7ff190103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:25:33.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.950+0000 7ff19574d700 1 -- 172.21.15.114:0/2304444599 shutdown_connections 2024-02-16T22:25:33.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:33.950+0000 7ff19574d700 1 -- 172.21.15.114:0/2304444599 wait complete. 2024-02-16T22:25:33.962 INFO:teuthology.orchestra.run.smithi114.stdout:true 2024-02-16T22:25:34.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.054+0000 7f1097a18700 1 -- 172.21.15.114:0/2263418788 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1090101dc0 msgr2=0x7f1090102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:34.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.054+0000 7f1097a18700 1 --2- 172.21.15.114:0/2263418788 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1090101dc0 0x7f1090102220 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1080009910 tx=0x7f1080009c20 comp rx=0 tx=0).stop 2024-02-16T22:25:34.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.054+0000 7f1097a18700 1 -- 172.21.15.114:0/2263418788 shutdown_connections 2024-02-16T22:25:34.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.054+0000 7f1097a18700 1 --2- 172.21.15.114:0/2263418788 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1090101dc0 0x7f1090102220 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:34.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.054+0000 7f1097a18700 1 --2- 172.21.15.114:0/2263418788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1090100bc0 0x7f1090100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:34.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.054+0000 7f1097a18700 1 -- 172.21.15.114:0/2263418788 >> 172.21.15.114:0/2263418788 conn(0x7f10900fc140 msgr2=0x7f10900fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:25:34.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.055+0000 7f1097a18700 1 -- 172.21.15.114:0/2263418788 shutdown_connections 2024-02-16T22:25:34.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.055+0000 7f1097a18700 1 -- 172.21.15.114:0/2263418788 wait complete. 2024-02-16T22:25:34.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.055+0000 7f1097a18700 1 Processor -- start 2024-02-16T22:25:34.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.055+0000 7f1097a18700 1 -- start start 2024-02-16T22:25:34.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.055+0000 7f1097a18700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1090100bc0 0x7f1090074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:34.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.055+0000 7f1097a18700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1090101dc0 0x7f1090073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:34.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.055+0000 7f1097a18700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10900736c0 con 0x7f1090100bc0 2024-02-16T22:25:34.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.055+0000 7f1097a18700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1090073800 con 0x7f1090101dc0 2024-02-16T22:25:34.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.055+0000 7f10957b4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1090100bc0 0x7f1090074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:34.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.055+0000 7f10957b4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1090100bc0 0x7f1090074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58176/0 (socket says 172.21.15.114:58176) 2024-02-16T22:25:34.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.055+0000 7f10957b4700 1 -- 172.21.15.114:0/3861286742 learned_addr learned my addr 172.21.15.114:0/3861286742 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:25:34.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.056+0000 7f1094fb3700 1 --2- 172.21.15.114:0/3861286742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1090101dc0 0x7f1090073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:34.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.056+0000 7f10957b4700 1 -- 172.21.15.114:0/3861286742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1090101dc0 msgr2=0x7f1090073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:34.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.056+0000 7f10957b4700 1 --2- 172.21.15.114:0/3861286742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1090101dc0 0x7f1090073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:34.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.056+0000 7f10957b4700 1 -- 172.21.15.114:0/3861286742 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10800095f0 con 0x7f1090100bc0 2024-02-16T22:25:34.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.056+0000 7f10957b4700 1 --2- 172.21.15.114:0/3861286742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1090100bc0 0x7f1090074b30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f108c00d900 tx=0x7f108c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:25:34.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.056+0000 7f10867fc700 1 -- 172.21.15.114:0/3861286742 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f108c009900 con 0x7f1090100bc0 2024-02-16T22:25:34.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.056+0000 7f1097a18700 1 -- 172.21.15.114:0/3861286742 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1090073ae0 con 0x7f1090100bc0 2024-02-16T22:25:34.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.056+0000 7f10867fc700 1 -- 172.21.15.114:0/3861286742 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f108c010460 con 0x7f1090100bc0 2024-02-16T22:25:34.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.056+0000 7f10867fc700 1 -- 172.21.15.114:0/3861286742 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f108c00f5b0 con 0x7f1090100bc0 2024-02-16T22:25:34.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.056+0000 7f1097a18700 1 -- 172.21.15.114:0/3861286742 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f109019e3f0 con 0x7f1090100bc0 2024-02-16T22:25:34.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.057+0000 7f1097a18700 1 -- 172.21.15.114:0/3861286742 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1090109740 con 0x7f1090100bc0 2024-02-16T22:25:34.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.058+0000 7f10867fc700 1 -- 172.21.15.114:0/3861286742 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f108c0105d0 con 0x7f1090100bc0 2024-02-16T22:25:34.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.058+0000 7f10867fc700 1 --2- 172.21.15.114:0/3861286742 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f107c06d8a0 0x7f107c06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:34.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.058+0000 7f10867fc700 1 -- 172.21.15.114:0/3861286742 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f108c08d040 con 0x7f1090100bc0 2024-02-16T22:25:34.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.058+0000 7f1094fb3700 1 --2- 172.21.15.114:0/3861286742 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f107c06d8a0 0x7f107c06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:34.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.058+0000 7f1094fb3700 1 --2- 172.21.15.114:0/3861286742 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f107c06d8a0 0x7f107c06fd60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f108000b5c0 tx=0x7f1080005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:25:34.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.063+0000 7f10867fc700 1 -- 172.21.15.114:0/3861286742 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f108c05a3d0 con 0x7f1090100bc0 2024-02-16T22:25:34.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.275+0000 7f1097a18700 1 -- 172.21.15.114:0/3861286742 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1090106710 con 0x7f107c06d8a0 2024-02-16T22:25:34.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.276+0000 7f10867fc700 1 -- 172.21.15.114:0/3861286742 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f1090106710 con 0x7f107c06d8a0 2024-02-16T22:25:34.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.278+0000 7f1097a18700 1 -- 172.21.15.114:0/3861286742 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f107c06d8a0 msgr2=0x7f107c06fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:34.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.278+0000 7f1097a18700 1 --2- 172.21.15.114:0/3861286742 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f107c06d8a0 0x7f107c06fd60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f108000b5c0 tx=0x7f1080005ee0 comp rx=0 tx=0).stop 2024-02-16T22:25:34.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.278+0000 7f1097a18700 1 -- 172.21.15.114:0/3861286742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1090100bc0 msgr2=0x7f1090074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:34.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.278+0000 7f1097a18700 1 --2- 172.21.15.114:0/3861286742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1090100bc0 0x7f1090074b30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f108c00d900 tx=0x7f108c00dc10 comp rx=0 tx=0).stop 2024-02-16T22:25:34.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.278+0000 7f1097a18700 1 -- 172.21.15.114:0/3861286742 shutdown_connections 2024-02-16T22:25:34.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.278+0000 7f1097a18700 1 --2- 172.21.15.114:0/3861286742 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f107c06d8a0 0x7f107c06fd60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:34.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.278+0000 7f1097a18700 1 --2- 172.21.15.114:0/3861286742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1090100bc0 0x7f1090074b30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:34.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.278+0000 7f1097a18700 1 --2- 172.21.15.114:0/3861286742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1090101dc0 0x7f1090073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:34.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.278+0000 7f1097a18700 1 -- 172.21.15.114:0/3861286742 >> 172.21.15.114:0/3861286742 conn(0x7f10900fc140 msgr2=0x7f1090104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:25:34.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.278+0000 7f1097a18700 1 -- 172.21.15.114:0/3861286742 shutdown_connections 2024-02-16T22:25:34.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.278+0000 7f1097a18700 1 -- 172.21.15.114:0/3861286742 wait complete. 2024-02-16T22:25:34.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.383+0000 7ff380822700 1 -- 172.21.15.114:0/515828660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff37c104030 msgr2=0x7ff37c106420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:34.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.383+0000 7ff380822700 1 --2- 172.21.15.114:0/515828660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff37c104030 0x7ff37c106420 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff370009910 tx=0x7ff370009c20 comp rx=0 tx=0).stop 2024-02-16T22:25:34.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.383+0000 7ff380822700 1 -- 172.21.15.114:0/515828660 shutdown_connections 2024-02-16T22:25:34.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.383+0000 7ff380822700 1 --2- 172.21.15.114:0/515828660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff37c104030 0x7ff37c106420 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:34.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.383+0000 7ff380822700 1 --2- 172.21.15.114:0/515828660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff37c101700 0x7ff37c103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:34.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.383+0000 7ff380822700 1 -- 172.21.15.114:0/515828660 >> 172.21.15.114:0/515828660 conn(0x7ff37c0fb010 msgr2=0x7ff37c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:25:34.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.383+0000 7ff380822700 1 -- 172.21.15.114:0/515828660 shutdown_connections 2024-02-16T22:25:34.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.383+0000 7ff380822700 1 -- 172.21.15.114:0/515828660 wait complete. 2024-02-16T22:25:34.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.383+0000 7ff380822700 1 Processor -- start 2024-02-16T22:25:34.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.383+0000 7ff380822700 1 -- start start 2024-02-16T22:25:34.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.384+0000 7ff380822700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff37c101700 0x7ff37c101060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:34.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.384+0000 7ff380822700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff37c104030 0x7ff37c0ff6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:34.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.384+0000 7ff380822700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37c0ffbf0 con 0x7ff37c104030 2024-02-16T22:25:34.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.384+0000 7ff380822700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37c0ffd30 con 0x7ff37c101700 2024-02-16T22:25:34.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.384+0000 7ff379d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff37c104030 0x7ff37c0ff6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:34.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.384+0000 7ff379d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff37c104030 0x7ff37c0ff6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58196/0 (socket says 172.21.15.114:58196) 2024-02-16T22:25:34.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.384+0000 7ff379d9b700 1 -- 172.21.15.114:0/126069406 learned_addr learned my addr 172.21.15.114:0/126069406 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:25:34.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.384+0000 7ff37a59c700 1 --2- 172.21.15.114:0/126069406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff37c101700 0x7ff37c101060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:34.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.384+0000 7ff379d9b700 1 -- 172.21.15.114:0/126069406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff37c101700 msgr2=0x7ff37c101060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:34.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.384+0000 7ff379d9b700 1 --2- 172.21.15.114:0/126069406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff37c101700 0x7ff37c101060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:34.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.384+0000 7ff379d9b700 1 -- 172.21.15.114:0/126069406 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3700095f0 con 0x7ff37c104030 2024-02-16T22:25:34.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.384+0000 7ff379d9b700 1 --2- 172.21.15.114:0/126069406 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff37c104030 0x7ff37c0ff6b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff37000bd70 tx=0x7ff37000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:25:34.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.384+0000 7ff36b7fe700 1 -- 172.21.15.114:0/126069406 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff37001d070 con 0x7ff37c104030 2024-02-16T22:25:34.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.384+0000 7ff36b7fe700 1 -- 172.21.15.114:0/126069406 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff370022470 con 0x7ff37c104030 2024-02-16T22:25:34.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.384+0000 7ff36b7fe700 1 -- 172.21.15.114:0/126069406 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff37000f670 con 0x7ff37c104030 2024-02-16T22:25:34.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.384+0000 7ff380822700 1 -- 172.21.15.114:0/126069406 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff37c0fffb0 con 0x7ff37c104030 2024-02-16T22:25:34.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.385+0000 7ff380822700 1 -- 172.21.15.114:0/126069406 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff37c1004a0 con 0x7ff37c104030 2024-02-16T22:25:34.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.385+0000 7ff380822700 1 -- 172.21.15.114:0/126069406 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff37c190520 con 0x7ff37c104030 2024-02-16T22:25:34.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.386+0000 7ff36b7fe700 1 -- 172.21.15.114:0/126069406 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7ff370022a50 con 0x7ff37c104030 2024-02-16T22:25:34.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.386+0000 7ff36b7fe700 1 --2- 172.21.15.114:0/126069406 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff36406d8a0 0x7ff36406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:34.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.386+0000 7ff36b7fe700 1 -- 172.21.15.114:0/126069406 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff37008eb80 con 0x7ff37c104030 2024-02-16T22:25:34.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.386+0000 7ff37a59c700 1 --2- 172.21.15.114:0/126069406 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff36406d8a0 0x7ff36406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:34.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.387+0000 7ff37a59c700 1 --2- 172.21.15.114:0/126069406 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff36406d8a0 0x7ff36406fd60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff36c005fd0 tx=0x7ff36c00a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:25:34.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.391+0000 7ff36b7fe700 1 -- 172.21.15.114:0/126069406 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff37005c160 con 0x7ff37c104030 2024-02-16T22:25:34.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.595+0000 7ff380822700 1 -- 172.21.15.114:0/126069406 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff37c0611d0 con 0x7ff36406d8a0 2024-02-16T22:25:34.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.608+0000 7ff36b7fe700 1 -- 172.21.15.114:0/126069406 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ff37c0611d0 con 0x7ff36406d8a0 2024-02-16T22:25:34.610 INFO:teuthology.orchestra.run.smithi114.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:25:34.610 INFO:teuthology.orchestra.run.smithi114.stdout:alertmanager.smithi114 smithi114 *:9093,9094 running (11m) 13s ago 12m 24.7M - 0.25.0 c8568f914cd2 e7404fea400e 2024-02-16T22:25:34.610 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi114 smithi114 running (13m) 13s ago 13m 8044k - 18.2.1 94eadd990e0a 7fd07162cd96 2024-02-16T22:25:34.610 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi123 smithi123 running (12m) 36s ago 12m 7696k - 18.2.1 94eadd990e0a 7687f5e99621 2024-02-16T22:25:34.610 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi114 smithi114 running (45s) 13s ago 13m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 9b8ae9efe83b 2024-02-16T22:25:34.610 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi123 smithi123 running (40s) 36s ago 12m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 6d97263a0ffa 2024-02-16T22:25:34.610 INFO:teuthology.orchestra.run.smithi114.stdout:grafana.smithi114 smithi114 *:3000 running (11m) 13s ago 12m 88.2M - 9.4.7 954c08fa6188 db1c26c3bc20 2024-02-16T22:25:34.610 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.fdxpey smithi114 running (7m) 13s ago 7m 18.0M - 18.2.1 94eadd990e0a 704b40c144fb 2024-02-16T22:25:34.610 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.zzryke smithi114 running (7m) 13s ago 7m 158M - 18.2.1 94eadd990e0a 1f795d23c9a4 2024-02-16T22:25:34.610 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.gopcfa smithi123 running (7m) 36s ago 7m 17.3M - 18.2.1 94eadd990e0a 166134b7532a 2024-02-16T22:25:34.610 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.vhlqbe smithi123 running (7m) 36s ago 7m 18.8M - 18.2.1 94eadd990e0a 243c8bb96fa8 2024-02-16T22:25:34.611 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi114.vcullz smithi114 *:8443,9283,8765 running (3m) 13s ago 14m 497M - 19.0.0-1450-gf78a58c0 f081f847070a 14ebeb7888c0 2024-02-16T22:25:34.611 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi123.inzgjd smithi123 *:8443,9283,8765 running (2m) 36s ago 11m 449M - 19.0.0-1450-gf78a58c0 f081f847070a f10aab9faf9e 2024-02-16T22:25:34.611 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi114 smithi114 running (2m) 13s ago 14m 43.8M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 58ac06372523 2024-02-16T22:25:34.611 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi123 smithi123 running (88s) 36s ago 11m 34.0M 2048M 19.0.0-1450-gf78a58c0 f081f847070a f76485e0ea24 2024-02-16T22:25:34.611 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi114 smithi114 *:9100 running (13m) 13s ago 13m 15.9M - 1.5.0 0da6a335fe13 744a090e06d0 2024-02-16T22:25:34.611 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi123 smithi123 *:9100 running (12m) 36s ago 12m 16.0M - 1.5.0 0da6a335fe13 3994b8c29b2e 2024-02-16T22:25:34.611 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 smithi114 running (19s) 13s ago 11m 11.9M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 1978fdad94a5 2024-02-16T22:25:34.611 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 smithi114 running (10m) 13s ago 10m 222M 1216M 18.2.1 94eadd990e0a 32bb191b88e7 2024-02-16T22:25:34.611 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 smithi114 running (10m) 13s ago 10m 195M 1216M 18.2.1 94eadd990e0a 6608718d0818 2024-02-16T22:25:34.611 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 smithi123 running (9m) 36s ago 9m 259M 2240M 18.2.1 94eadd990e0a 575db379de3d 2024-02-16T22:25:34.611 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 smithi123 running (9m) 36s ago 9m 220M 2240M 18.2.1 94eadd990e0a 7e266866cd9c 2024-02-16T22:25:34.611 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 smithi123 running (8m) 36s ago 8m 199M 2240M 18.2.1 94eadd990e0a 24e5bef19fda 2024-02-16T22:25:34.611 INFO:teuthology.orchestra.run.smithi114.stdout:prometheus.smithi114 smithi114 *:9095 running (2m) 13s ago 12m 59.5M - 2.43.0 a07b618ecd1d 37aac985cdd7 2024-02-16T22:25:34.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.610+0000 7ff380822700 1 -- 172.21.15.114:0/126069406 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff36406d8a0 msgr2=0x7ff36406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:34.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.610+0000 7ff380822700 1 --2- 172.21.15.114:0/126069406 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff36406d8a0 0x7ff36406fd60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff36c005fd0 tx=0x7ff36c00a560 comp rx=0 tx=0).stop 2024-02-16T22:25:34.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.610+0000 7ff380822700 1 -- 172.21.15.114:0/126069406 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff37c104030 msgr2=0x7ff37c0ff6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:34.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.610+0000 7ff380822700 1 --2- 172.21.15.114:0/126069406 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff37c104030 0x7ff37c0ff6b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff37000bd70 tx=0x7ff37000be50 comp rx=0 tx=0).stop 2024-02-16T22:25:34.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.610+0000 7ff380822700 1 -- 172.21.15.114:0/126069406 shutdown_connections 2024-02-16T22:25:34.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.610+0000 7ff380822700 1 --2- 172.21.15.114:0/126069406 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff36406d8a0 0x7ff36406fd60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:34.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.610+0000 7ff380822700 1 --2- 172.21.15.114:0/126069406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff37c101700 0x7ff37c101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:34.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.610+0000 7ff380822700 1 --2- 172.21.15.114:0/126069406 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff37c104030 0x7ff37c0ff6b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:34.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.610+0000 7ff380822700 1 -- 172.21.15.114:0/126069406 >> 172.21.15.114:0/126069406 conn(0x7ff37c0fb010 msgr2=0x7ff37c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:25:34.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.610+0000 7ff380822700 1 -- 172.21.15.114:0/126069406 shutdown_connections 2024-02-16T22:25:34.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.610+0000 7ff380822700 1 -- 172.21.15.114:0/126069406 wait complete. 2024-02-16T22:25:34.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.719+0000 7f90003e9700 1 -- 172.21.15.114:0/2934542678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ff80fee80 msgr2=0x7f8ff80ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:34.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.719+0000 7f90003e9700 1 --2- 172.21.15.114:0/2934542678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ff80fee80 0x7f8ff80ff2a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4009910 tx=0x7f8ff4009c20 comp rx=0 tx=0).stop 2024-02-16T22:25:34.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.719+0000 7f90003e9700 1 -- 172.21.15.114:0/2934542678 shutdown_connections 2024-02-16T22:25:34.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.719+0000 7f90003e9700 1 --2- 172.21.15.114:0/2934542678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ff8101e60 0x7f8ff81022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:34.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.719+0000 7f90003e9700 1 --2- 172.21.15.114:0/2934542678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ff80fee80 0x7f8ff80ff2a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:34.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.719+0000 7f90003e9700 1 -- 172.21.15.114:0/2934542678 >> 172.21.15.114:0/2934542678 conn(0x7f8ff80fa7c0 msgr2=0x7f8ff80fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:25:34.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.719+0000 7f90003e9700 1 -- 172.21.15.114:0/2934542678 shutdown_connections 2024-02-16T22:25:34.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.719+0000 7f90003e9700 1 -- 172.21.15.114:0/2934542678 wait complete. 2024-02-16T22:25:34.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.720+0000 7f90003e9700 1 Processor -- start 2024-02-16T22:25:34.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.720+0000 7f90003e9700 1 -- start start 2024-02-16T22:25:34.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.720+0000 7f90003e9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ff80fee80 0x7f8ff81017c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:34.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.720+0000 7f90003e9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ff8101e60 0x7f8ff80ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:34.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.720+0000 7f90003e9700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff8100350 con 0x7f8ff80fee80 2024-02-16T22:25:34.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.720+0000 7f90003e9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff8100490 con 0x7f8ff8101e60 2024-02-16T22:25:34.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.720+0000 7f8ffe185700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ff80fee80 0x7f8ff81017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:34.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.720+0000 7f8ffe185700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ff80fee80 0x7f8ff81017c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58214/0 (socket says 172.21.15.114:58214) 2024-02-16T22:25:34.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.720+0000 7f8ffe185700 1 -- 172.21.15.114:0/2929234177 learned_addr learned my addr 172.21.15.114:0/2929234177 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:25:34.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.720+0000 7f8ffd984700 1 --2- 172.21.15.114:0/2929234177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ff8101e60 0x7f8ff80ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:34.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.721+0000 7f8ffe185700 1 -- 172.21.15.114:0/2929234177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ff8101e60 msgr2=0x7f8ff80ffe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:34.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.721+0000 7f8ffe185700 1 --2- 172.21.15.114:0/2929234177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ff8101e60 0x7f8ff80ffe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:34.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.721+0000 7f8ffe185700 1 -- 172.21.15.114:0/2929234177 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff40095f0 con 0x7f8ff80fee80 2024-02-16T22:25:34.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.721+0000 7f8ffd984700 1 --2- 172.21.15.114:0/2929234177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ff8101e60 0x7f8ff80ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:25:34.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.721+0000 7f8ffe185700 1 --2- 172.21.15.114:0/2929234177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ff80fee80 0x7f8ff81017c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8ff400bda0 tx=0x7f8ff400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:25:34.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.721+0000 7f8fef7fe700 1 -- 172.21.15.114:0/2929234177 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ff401d070 con 0x7f8ff80fee80 2024-02-16T22:25:34.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.721+0000 7f8fef7fe700 1 -- 172.21.15.114:0/2929234177 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8ff400f460 con 0x7f8ff80fee80 2024-02-16T22:25:34.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.721+0000 7f90003e9700 1 -- 172.21.15.114:0/2929234177 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff8100710 con 0x7f8ff80fee80 2024-02-16T22:25:34.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.721+0000 7f8fef7fe700 1 -- 172.21.15.114:0/2929234177 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ff4021640 con 0x7f8ff80fee80 2024-02-16T22:25:34.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.721+0000 7f90003e9700 1 -- 172.21.15.114:0/2929234177 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff8100c00 con 0x7f8ff80fee80 2024-02-16T22:25:34.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.722+0000 7f90003e9700 1 -- 172.21.15.114:0/2929234177 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff8109720 con 0x7f8ff80fee80 2024-02-16T22:25:34.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.723+0000 7f8fef7fe700 1 -- 172.21.15.114:0/2929234177 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f8ff400f5d0 con 0x7f8ff80fee80 2024-02-16T22:25:34.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.723+0000 7f8fef7fe700 1 --2- 172.21.15.114:0/2929234177 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f8fe406d8a0 0x7f8fe406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:34.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.723+0000 7f8fef7fe700 1 -- 172.21.15.114:0/2929234177 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f8ff408e800 con 0x7f8ff80fee80 2024-02-16T22:25:34.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.723+0000 7f8ffd984700 1 --2- 172.21.15.114:0/2929234177 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f8fe406d8a0 0x7f8fe406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:34.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.724+0000 7f8ffd984700 1 --2- 172.21.15.114:0/2929234177 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f8fe406d8a0 0x7f8fe406fd60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8005fd0 tx=0x7f8fe8005f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:25:34.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:34.728+0000 7f8fef7fe700 1 -- 172.21.15.114:0/2929234177 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8ff405bb90 con 0x7f8ff80fee80 2024-02-16T22:25:35.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.006+0000 7f90003e9700 1 -- 172.21.15.114:0/2929234177 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8ff819e820 con 0x7f8ff80fee80 2024-02-16T22:25:35.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.006+0000 7f8fef7fe700 1 -- 172.21.15.114:0/2929234177 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f8ff405b720 con 0x7f8ff80fee80 2024-02-16T22:25:35.009 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:25:35.009 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": { 2024-02-16T22:25:35.009 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:25:35.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:25:35.009 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": { 2024-02-16T22:25:35.009 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:25:35.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:25:35.009 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": { 2024-02-16T22:25:35.009 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 5, 2024-02-16T22:25:35.009 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T22:25:35.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:25:35.009 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-02-16T22:25:35.009 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-02-16T22:25:35.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:25:35.010 INFO:teuthology.orchestra.run.smithi114.stdout: "overall": { 2024-02-16T22:25:35.010 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 9, 2024-02-16T22:25:35.010 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 5 2024-02-16T22:25:35.010 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-02-16T22:25:35.010 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:25:35.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.008+0000 7f90003e9700 1 -- 172.21.15.114:0/2929234177 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f8fe406d8a0 msgr2=0x7f8fe406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:35.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.008+0000 7f90003e9700 1 --2- 172.21.15.114:0/2929234177 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f8fe406d8a0 0x7f8fe406fd60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8005fd0 tx=0x7f8fe8005f60 comp rx=0 tx=0).stop 2024-02-16T22:25:35.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.008+0000 7f90003e9700 1 -- 172.21.15.114:0/2929234177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ff80fee80 msgr2=0x7f8ff81017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:35.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.008+0000 7f90003e9700 1 --2- 172.21.15.114:0/2929234177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ff80fee80 0x7f8ff81017c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8ff400bda0 tx=0x7f8ff400be80 comp rx=0 tx=0).stop 2024-02-16T22:25:35.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.008+0000 7f90003e9700 1 -- 172.21.15.114:0/2929234177 shutdown_connections 2024-02-16T22:25:35.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.008+0000 7f90003e9700 1 --2- 172.21.15.114:0/2929234177 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f8fe406d8a0 0x7f8fe406fd60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:35.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.008+0000 7f90003e9700 1 --2- 172.21.15.114:0/2929234177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ff80fee80 0x7f8ff81017c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:35.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.008+0000 7f90003e9700 1 --2- 172.21.15.114:0/2929234177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ff8101e60 0x7f8ff80ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:35.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.008+0000 7f90003e9700 1 -- 172.21.15.114:0/2929234177 >> 172.21.15.114:0/2929234177 conn(0x7f8ff80fa7c0 msgr2=0x7f8ff8105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:25:35.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.008+0000 7f90003e9700 1 -- 172.21.15.114:0/2929234177 shutdown_connections 2024-02-16T22:25:35.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.008+0000 7f90003e9700 1 -- 172.21.15.114:0/2929234177 wait complete. 2024-02-16T22:25:35.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.111+0000 7f2e505e6700 1 -- 172.21.15.114:0/3379891169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e48101dc0 msgr2=0x7f2e48102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:35.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.111+0000 7f2e505e6700 1 --2- 172.21.15.114:0/3379891169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e48101dc0 0x7f2e48102220 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009910 tx=0x7f2e38009c20 comp rx=0 tx=0).stop 2024-02-16T22:25:35.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.112+0000 7f2e505e6700 1 -- 172.21.15.114:0/3379891169 shutdown_connections 2024-02-16T22:25:35.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.112+0000 7f2e505e6700 1 --2- 172.21.15.114:0/3379891169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e48101dc0 0x7f2e48102220 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:35.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.112+0000 7f2e505e6700 1 --2- 172.21.15.114:0/3379891169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e48100bc0 0x7f2e48100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:35.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.112+0000 7f2e505e6700 1 -- 172.21.15.114:0/3379891169 >> 172.21.15.114:0/3379891169 conn(0x7f2e480fc140 msgr2=0x7f2e480fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:25:35.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.112+0000 7f2e505e6700 1 -- 172.21.15.114:0/3379891169 shutdown_connections 2024-02-16T22:25:35.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.112+0000 7f2e505e6700 1 -- 172.21.15.114:0/3379891169 wait complete. 2024-02-16T22:25:35.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.112+0000 7f2e505e6700 1 Processor -- start 2024-02-16T22:25:35.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.112+0000 7f2e505e6700 1 -- start start 2024-02-16T22:25:35.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.112+0000 7f2e505e6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e48100bc0 0x7f2e481963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:35.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.112+0000 7f2e505e6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e48101dc0 0x7f2e48196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:35.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.112+0000 7f2e505e6700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e48196f40 con 0x7f2e48101dc0 2024-02-16T22:25:35.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.112+0000 7f2e505e6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e48197080 con 0x7f2e48100bc0 2024-02-16T22:25:35.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.112+0000 7f2e4db81700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e48101dc0 0x7f2e48196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:35.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.112+0000 7f2e4db81700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e48101dc0 0x7f2e48196920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58236/0 (socket says 172.21.15.114:58236) 2024-02-16T22:25:35.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.112+0000 7f2e4db81700 1 -- 172.21.15.114:0/3059592882 learned_addr learned my addr 172.21.15.114:0/3059592882 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:25:35.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.113+0000 7f2e4e382700 1 --2- 172.21.15.114:0/3059592882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e48100bc0 0x7f2e481963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:35.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.113+0000 7f2e4db81700 1 -- 172.21.15.114:0/3059592882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e48100bc0 msgr2=0x7f2e481963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:35.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.113+0000 7f2e4db81700 1 --2- 172.21.15.114:0/3059592882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e48100bc0 0x7f2e481963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:35.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.113+0000 7f2e4db81700 1 -- 172.21.15.114:0/3059592882 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e380095f0 con 0x7f2e48101dc0 2024-02-16T22:25:35.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.113+0000 7f2e4e382700 1 --2- 172.21.15.114:0/3059592882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e48100bc0 0x7f2e481963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T22:25:35.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.113+0000 7f2e4db81700 1 --2- 172.21.15.114:0/3059592882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e48101dc0 0x7f2e48196920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800bd70 tx=0x7f2e3800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:25:35.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.113+0000 7f2e3f7fe700 1 -- 172.21.15.114:0/3059592882 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e3801d070 con 0x7f2e48101dc0 2024-02-16T22:25:35.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.113+0000 7f2e3f7fe700 1 -- 172.21.15.114:0/3059592882 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2e38022470 con 0x7f2e48101dc0 2024-02-16T22:25:35.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.113+0000 7f2e3f7fe700 1 -- 172.21.15.114:0/3059592882 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e3800f670 con 0x7f2e48101dc0 2024-02-16T22:25:35.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.113+0000 7f2e505e6700 1 -- 172.21.15.114:0/3059592882 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e4819bad0 con 0x7f2e48101dc0 2024-02-16T22:25:35.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.113+0000 7f2e505e6700 1 -- 172.21.15.114:0/3059592882 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e4819c040 con 0x7f2e48101dc0 2024-02-16T22:25:35.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.114+0000 7f2e505e6700 1 -- 172.21.15.114:0/3059592882 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e481097c0 con 0x7f2e48101dc0 2024-02-16T22:25:35.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.115+0000 7f2e3f7fe700 1 -- 172.21.15.114:0/3059592882 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f2e38022a50 con 0x7f2e48101dc0 2024-02-16T22:25:35.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.115+0000 7f2e3f7fe700 1 --2- 172.21.15.114:0/3059592882 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2e3406d8a0 0x7f2e3406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:35.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.115+0000 7f2e3f7fe700 1 -- 172.21.15.114:0/3059592882 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f2e3808ebb0 con 0x7f2e48101dc0 2024-02-16T22:25:35.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.115+0000 7f2e4e382700 1 --2- 172.21.15.114:0/3059592882 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2e3406d8a0 0x7f2e3406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:35.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.115+0000 7f2e4e382700 1 --2- 172.21.15.114:0/3059592882 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2e3406d8a0 0x7f2e3406fd60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2e48101c20 tx=0x7f2e44008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:25:35.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.120+0000 7f2e3f7fe700 1 -- 172.21.15.114:0/3059592882 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2e38095080 con 0x7f2e48101dc0 2024-02-16T22:25:35.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.358+0000 7f2e505e6700 1 -- 172.21.15.114:0/3059592882 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2e4819c6d0 con 0x7f2e48101dc0 2024-02-16T22:25:35.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.358+0000 7f2e3f7fe700 1 -- 172.21.15.114:0/3059592882 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1846 (secure 0 0 0) 0x7f2e38027070 con 0x7f2e48101dc0 2024-02-16T22:25:35.360 INFO:teuthology.orchestra.run.smithi114.stdout:e14 2024-02-16T22:25:35.360 INFO:teuthology.orchestra.run.smithi114.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:25:35.360 INFO:teuthology.orchestra.run.smithi114.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:25:35.361 INFO:teuthology.orchestra.run.smithi114.stdout:legacy client fscid: 1 2024-02-16T22:25:35.361 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:25:35.361 INFO:teuthology.orchestra.run.smithi114.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:25:35.361 INFO:teuthology.orchestra.run.smithi114.stdout:fs_name cephfs 2024-02-16T22:25:35.361 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 12 2024-02-16T22:25:35.361 INFO:teuthology.orchestra.run.smithi114.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:25:35.361 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-02-16T22:18:02.108825+0000 2024-02-16T22:25:35.361 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-02-16T22:18:16.413371+0000 2024-02-16T22:25:35.361 INFO:teuthology.orchestra.run.smithi114.stdout:tableserver 0 2024-02-16T22:25:35.361 INFO:teuthology.orchestra.run.smithi114.stdout:root 0 2024-02-16T22:25:35.361 INFO:teuthology.orchestra.run.smithi114.stdout:session_timeout 60 2024-02-16T22:25:35.361 INFO:teuthology.orchestra.run.smithi114.stdout:session_autoclose 300 2024-02-16T22:25:35.361 INFO:teuthology.orchestra.run.smithi114.stdout:max_file_size 1099511627776 2024-02-16T22:25:35.361 INFO:teuthology.orchestra.run.smithi114.stdout:max_xattr_size 65536 2024-02-16T22:25:35.362 INFO:teuthology.orchestra.run.smithi114.stdout:required_client_features {} 2024-02-16T22:25:35.362 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure 0 2024-02-16T22:25:35.362 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure_osd_epoch 0 2024-02-16T22:25:35.362 INFO:teuthology.orchestra.run.smithi114.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:25:35.362 INFO:teuthology.orchestra.run.smithi114.stdout:max_mds 1 2024-02-16T22:25:35.362 INFO:teuthology.orchestra.run.smithi114.stdout:in 0 2024-02-16T22:25:35.362 INFO:teuthology.orchestra.run.smithi114.stdout:up {0=14468} 2024-02-16T22:25:35.362 INFO:teuthology.orchestra.run.smithi114.stdout:failed 2024-02-16T22:25:35.362 INFO:teuthology.orchestra.run.smithi114.stdout:damaged 2024-02-16T22:25:35.362 INFO:teuthology.orchestra.run.smithi114.stdout:stopped 2024-02-16T22:25:35.362 INFO:teuthology.orchestra.run.smithi114.stdout:data_pools [3] 2024-02-16T22:25:35.362 INFO:teuthology.orchestra.run.smithi114.stdout:metadata_pool 2 2024-02-16T22:25:35.362 INFO:teuthology.orchestra.run.smithi114.stdout:inline_data enabled 2024-02-16T22:25:35.363 INFO:teuthology.orchestra.run.smithi114.stdout:balancer 2024-02-16T22:25:35.363 INFO:teuthology.orchestra.run.smithi114.stdout:bal_rank_mask -1 2024-02-16T22:25:35.363 INFO:teuthology.orchestra.run.smithi114.stdout:standby_count_wanted 1 2024-02-16T22:25:35.363 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.zzryke{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.114:6826/277953535,v1:172.21.15.114:6827/277953535] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:25:35.363 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:25:35.363 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:25:35.363 INFO:teuthology.orchestra.run.smithi114.stdout:Standby daemons: 2024-02-16T22:25:35.363 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:25:35.363 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.fdxpey{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:25:35.363 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.vhlqbe{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:25:35.363 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.gopcfa{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:25:35.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.360+0000 7f2e505e6700 1 -- 172.21.15.114:0/3059592882 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2e3406d8a0 msgr2=0x7f2e3406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:35.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.360+0000 7f2e505e6700 1 --2- 172.21.15.114:0/3059592882 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2e3406d8a0 0x7f2e3406fd60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2e48101c20 tx=0x7f2e44008040 comp rx=0 tx=0).stop 2024-02-16T22:25:35.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.360+0000 7f2e505e6700 1 -- 172.21.15.114:0/3059592882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e48101dc0 msgr2=0x7f2e48196920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:35.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.360+0000 7f2e505e6700 1 --2- 172.21.15.114:0/3059592882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e48101dc0 0x7f2e48196920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800bd70 tx=0x7f2e3800be50 comp rx=0 tx=0).stop 2024-02-16T22:25:35.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.360+0000 7f2e505e6700 1 -- 172.21.15.114:0/3059592882 shutdown_connections 2024-02-16T22:25:35.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.360+0000 7f2e505e6700 1 --2- 172.21.15.114:0/3059592882 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2e3406d8a0 0x7f2e3406fd60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:35.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.360+0000 7f2e505e6700 1 --2- 172.21.15.114:0/3059592882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e48100bc0 0x7f2e481963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:35.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.360+0000 7f2e505e6700 1 --2- 172.21.15.114:0/3059592882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e48101dc0 0x7f2e48196920 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:35.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.360+0000 7f2e505e6700 1 -- 172.21.15.114:0/3059592882 >> 172.21.15.114:0/3059592882 conn(0x7f2e480fc140 msgr2=0x7f2e48104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:25:35.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.360+0000 7f2e505e6700 1 -- 172.21.15.114:0/3059592882 shutdown_connections 2024-02-16T22:25:35.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.360+0000 7f2e505e6700 1 -- 172.21.15.114:0/3059592882 wait complete. 2024-02-16T22:25:35.365 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 14 2024-02-16T22:25:35.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.464+0000 7f89f18ba700 1 -- 172.21.15.114:0/2250462508 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89ec0ffad0 msgr2=0x7f89ec0ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:35.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.464+0000 7f89f18ba700 1 --2- 172.21.15.114:0/2250462508 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89ec0ffad0 0x7f89ec0ffef0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f89e0009910 tx=0x7f89e0009c20 comp rx=0 tx=0).stop 2024-02-16T22:25:35.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.464+0000 7f89f18ba700 1 -- 172.21.15.114:0/2250462508 shutdown_connections 2024-02-16T22:25:35.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.464+0000 7f89f18ba700 1 --2- 172.21.15.114:0/2250462508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89ec100cd0 0x7f89ec101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:35.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.464+0000 7f89f18ba700 1 --2- 172.21.15.114:0/2250462508 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89ec0ffad0 0x7f89ec0ffef0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:35.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.464+0000 7f89f18ba700 1 -- 172.21.15.114:0/2250462508 >> 172.21.15.114:0/2250462508 conn(0x7f89ec0fb050 msgr2=0x7f89ec0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:25:35.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.464+0000 7f89f18ba700 1 -- 172.21.15.114:0/2250462508 shutdown_connections 2024-02-16T22:25:35.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.464+0000 7f89f18ba700 1 -- 172.21.15.114:0/2250462508 wait complete. 2024-02-16T22:25:35.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.464+0000 7f89f18ba700 1 Processor -- start 2024-02-16T22:25:35.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.465+0000 7f89f18ba700 1 -- start start 2024-02-16T22:25:35.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.465+0000 7f89f18ba700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89ec0ffad0 0x7f89ec196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:35.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.465+0000 7f89f18ba700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89ec100cd0 0x7f89ec1968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:35.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.465+0000 7f89f18ba700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89ec196ec0 con 0x7f89ec0ffad0 2024-02-16T22:25:35.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.465+0000 7f89f18ba700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89ec197000 con 0x7f89ec100cd0 2024-02-16T22:25:35.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.465+0000 7f89eaffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89ec0ffad0 0x7f89ec196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:35.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.465+0000 7f89eaffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89ec0ffad0 0x7f89ec196360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58256/0 (socket says 172.21.15.114:58256) 2024-02-16T22:25:35.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.465+0000 7f89eaffd700 1 -- 172.21.15.114:0/2434224250 learned_addr learned my addr 172.21.15.114:0/2434224250 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:25:35.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.465+0000 7f89ea7fc700 1 --2- 172.21.15.114:0/2434224250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89ec100cd0 0x7f89ec1968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:35.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.465+0000 7f89eaffd700 1 -- 172.21.15.114:0/2434224250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89ec100cd0 msgr2=0x7f89ec1968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:35.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.465+0000 7f89eaffd700 1 --2- 172.21.15.114:0/2434224250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89ec100cd0 0x7f89ec1968a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:35.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.465+0000 7f89eaffd700 1 -- 172.21.15.114:0/2434224250 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e00095f0 con 0x7f89ec0ffad0 2024-02-16T22:25:35.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.465+0000 7f89eaffd700 1 --2- 172.21.15.114:0/2434224250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89ec0ffad0 0x7f89ec196360 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f89e0000c00 tx=0x7f89e000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:25:35.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.465+0000 7f89f08b8700 1 -- 172.21.15.114:0/2434224250 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89e001d070 con 0x7f89ec0ffad0 2024-02-16T22:25:35.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.465+0000 7f89f18ba700 1 -- 172.21.15.114:0/2434224250 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89ec19ba50 con 0x7f89ec0ffad0 2024-02-16T22:25:35.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.465+0000 7f89f08b8700 1 -- 172.21.15.114:0/2434224250 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f89e0005dc0 con 0x7f89ec0ffad0 2024-02-16T22:25:35.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.465+0000 7f89f08b8700 1 -- 172.21.15.114:0/2434224250 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89e00226a0 con 0x7f89ec0ffad0 2024-02-16T22:25:35.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.466+0000 7f89f18ba700 1 -- 172.21.15.114:0/2434224250 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89ec19bf40 con 0x7f89ec0ffad0 2024-02-16T22:25:35.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.466+0000 7f89f18ba700 1 -- 172.21.15.114:0/2434224250 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89ec108630 con 0x7f89ec0ffad0 2024-02-16T22:25:35.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.467+0000 7f89f08b8700 1 -- 172.21.15.114:0/2434224250 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f89e0003910 con 0x7f89ec0ffad0 2024-02-16T22:25:35.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.467+0000 7f89f08b8700 1 --2- 172.21.15.114:0/2434224250 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f89d406d8a0 0x7f89d406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:35.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.467+0000 7f89f08b8700 1 -- 172.21.15.114:0/2434224250 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f89e008eaf0 con 0x7f89ec0ffad0 2024-02-16T22:25:35.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.467+0000 7f89ea7fc700 1 --2- 172.21.15.114:0/2434224250 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f89d406d8a0 0x7f89d406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:35.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.468+0000 7f89ea7fc700 1 --2- 172.21.15.114:0/2434224250 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f89d406d8a0 0x7f89d406fd60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f89dc009b30 tx=0x7f89dc009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:25:35.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.472+0000 7f89f08b8700 1 -- 172.21.15.114:0/2434224250 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f89e005be80 con 0x7f89ec0ffad0 2024-02-16T22:25:35.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:35 smithi123 ceph-mon[82331]: pgmap v62: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 28 B/s, 0 objects/s recovering 2024-02-16T22:25:35.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:35 smithi123 ceph-mon[82331]: from='client.34218 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:25:35.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:35 smithi123 ceph-mon[82331]: from='client.34222 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:25:35.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:35 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/2929234177' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:25:35.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.687+0000 7f89f18ba700 1 -- 172.21.15.114:0/2434224250 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f89ec069440 con 0x7f89d406d8a0 2024-02-16T22:25:35.690 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:35 smithi114 ceph-mon[94452]: pgmap v62: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 28 B/s, 0 objects/s recovering 2024-02-16T22:25:35.690 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:35 smithi114 ceph-mon[94452]: from='client.34218 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:25:35.690 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:35 smithi114 ceph-mon[94452]: from='client.34222 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:25:35.690 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:35 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/2929234177' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:25:35.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.689+0000 7f89f08b8700 1 -- 172.21.15.114:0/2434224250 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f89ec069440 con 0x7f89d406d8a0 2024-02-16T22:25:35.691 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:25:35.691 INFO:teuthology.orchestra.run.smithi114.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:25:35.691 INFO:teuthology.orchestra.run.smithi114.stdout: "in_progress": true, 2024-02-16T22:25:35.691 INFO:teuthology.orchestra.run.smithi114.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:25:35.691 INFO:teuthology.orchestra.run.smithi114.stdout: "services_complete": [ 2024-02-16T22:25:35.691 INFO:teuthology.orchestra.run.smithi114.stdout: "crash", 2024-02-16T22:25:35.691 INFO:teuthology.orchestra.run.smithi114.stdout: "mon", 2024-02-16T22:25:35.691 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr" 2024-02-16T22:25:35.691 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-02-16T22:25:35.691 INFO:teuthology.orchestra.run.smithi114.stdout: "progress": "7/23 daemons upgraded", 2024-02-16T22:25:35.691 INFO:teuthology.orchestra.run.smithi114.stdout: "message": "Currently upgrading osd daemons", 2024-02-16T22:25:35.691 INFO:teuthology.orchestra.run.smithi114.stdout: "is_paused": false 2024-02-16T22:25:35.691 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:25:35.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.691+0000 7f89f18ba700 1 -- 172.21.15.114:0/2434224250 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f89d406d8a0 msgr2=0x7f89d406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:35.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.691+0000 7f89f18ba700 1 --2- 172.21.15.114:0/2434224250 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f89d406d8a0 0x7f89d406fd60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f89dc009b30 tx=0x7f89dc009260 comp rx=0 tx=0).stop 2024-02-16T22:25:35.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.691+0000 7f89f18ba700 1 -- 172.21.15.114:0/2434224250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89ec0ffad0 msgr2=0x7f89ec196360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:35.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.691+0000 7f89f18ba700 1 --2- 172.21.15.114:0/2434224250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89ec0ffad0 0x7f89ec196360 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f89e0000c00 tx=0x7f89e000f740 comp rx=0 tx=0).stop 2024-02-16T22:25:35.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.691+0000 7f89f18ba700 1 -- 172.21.15.114:0/2434224250 shutdown_connections 2024-02-16T22:25:35.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.691+0000 7f89f18ba700 1 --2- 172.21.15.114:0/2434224250 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f89d406d8a0 0x7f89d406fd60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:35.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.691+0000 7f89f18ba700 1 --2- 172.21.15.114:0/2434224250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89ec0ffad0 0x7f89ec196360 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:35.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.691+0000 7f89f18ba700 1 --2- 172.21.15.114:0/2434224250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89ec100cd0 0x7f89ec1968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:35.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.691+0000 7f89f18ba700 1 -- 172.21.15.114:0/2434224250 >> 172.21.15.114:0/2434224250 conn(0x7f89ec0fb050 msgr2=0x7f89ec068d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:25:35.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.691+0000 7f89f18ba700 1 -- 172.21.15.114:0/2434224250 shutdown_connections 2024-02-16T22:25:35.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.691+0000 7f89f18ba700 1 -- 172.21.15.114:0/2434224250 wait complete. 2024-02-16T22:25:35.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.794+0000 7f1b29364700 1 -- 172.21.15.114:0/865032464 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b24074d60 msgr2=0x7f1b240731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:35.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.794+0000 7f1b29364700 1 --2- 172.21.15.114:0/865032464 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b24074d60 0x7f1b240731c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1b18009910 tx=0x7f1b18009c20 comp rx=0 tx=0).stop 2024-02-16T22:25:35.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.794+0000 7f1b29364700 1 -- 172.21.15.114:0/865032464 shutdown_connections 2024-02-16T22:25:35.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.794+0000 7f1b29364700 1 --2- 172.21.15.114:0/865032464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b24073790 0x7f1b24073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:35.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.794+0000 7f1b29364700 1 --2- 172.21.15.114:0/865032464 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b24074d60 0x7f1b240731c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:35.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.794+0000 7f1b29364700 1 -- 172.21.15.114:0/865032464 >> 172.21.15.114:0/865032464 conn(0x7f1b240fc140 msgr2=0x7f1b240fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:25:35.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.794+0000 7f1b29364700 1 -- 172.21.15.114:0/865032464 shutdown_connections 2024-02-16T22:25:35.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.794+0000 7f1b29364700 1 -- 172.21.15.114:0/865032464 wait complete. 2024-02-16T22:25:35.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.795+0000 7f1b29364700 1 Processor -- start 2024-02-16T22:25:35.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.795+0000 7f1b29364700 1 -- start start 2024-02-16T22:25:35.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.795+0000 7f1b29364700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b24073790 0x7f1b24198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:35.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.795+0000 7f1b29364700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b24074d60 0x7f1b24198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:35.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.795+0000 7f1b29364700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b241990c0 con 0x7f1b24074d60 2024-02-16T22:25:35.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.795+0000 7f1b29364700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b24199200 con 0x7f1b24073790 2024-02-16T22:25:35.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.795+0000 7f1b227fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b24074d60 0x7f1b24198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:35.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.795+0000 7f1b227fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b24074d60 0x7f1b24198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58276/0 (socket says 172.21.15.114:58276) 2024-02-16T22:25:35.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.795+0000 7f1b227fc700 1 -- 172.21.15.114:0/3890177641 learned_addr learned my addr 172.21.15.114:0/3890177641 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:25:35.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.795+0000 7f1b22ffd700 1 --2- 172.21.15.114:0/3890177641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b24073790 0x7f1b24198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:35.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.796+0000 7f1b227fc700 1 -- 172.21.15.114:0/3890177641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b24073790 msgr2=0x7f1b24198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:35.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.796+0000 7f1b227fc700 1 --2- 172.21.15.114:0/3890177641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b24073790 0x7f1b24198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:35.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.796+0000 7f1b227fc700 1 -- 172.21.15.114:0/3890177641 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b180095f0 con 0x7f1b24074d60 2024-02-16T22:25:35.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.796+0000 7f1b22ffd700 1 --2- 172.21.15.114:0/3890177641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b24073790 0x7f1b24198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T22:25:35.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.796+0000 7f1b227fc700 1 --2- 172.21.15.114:0/3890177641 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b24074d60 0x7f1b24198aa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400c8a0 tx=0x7f1b1400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:25:35.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.796+0000 7f1b0bfff700 1 -- 172.21.15.114:0/3890177641 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b1400cea0 con 0x7f1b24074d60 2024-02-16T22:25:35.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.796+0000 7f1b29364700 1 -- 172.21.15.114:0/3890177641 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b2419dcb0 con 0x7f1b24074d60 2024-02-16T22:25:35.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.796+0000 7f1b0bfff700 1 -- 172.21.15.114:0/3890177641 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1b14004830 con 0x7f1b24074d60 2024-02-16T22:25:35.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.796+0000 7f1b0bfff700 1 -- 172.21.15.114:0/3890177641 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b140055b0 con 0x7f1b24074d60 2024-02-16T22:25:35.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.796+0000 7f1b29364700 1 -- 172.21.15.114:0/3890177641 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b2419e200 con 0x7f1b24074d60 2024-02-16T22:25:35.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.797+0000 7f1b29364700 1 -- 172.21.15.114:0/3890177641 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b2410b910 con 0x7f1b24074d60 2024-02-16T22:25:35.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.797+0000 7f1b0bfff700 1 -- 172.21.15.114:0/3890177641 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f1b14007ad0 con 0x7f1b24074d60 2024-02-16T22:25:35.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.798+0000 7f1b0bfff700 1 --2- 172.21.15.114:0/3890177641 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f1b0c06d8a0 0x7f1b0c06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:25:35.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.798+0000 7f1b0bfff700 1 -- 172.21.15.114:0/3890177641 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f1b1408d200 con 0x7f1b24074d60 2024-02-16T22:25:35.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.798+0000 7f1b22ffd700 1 --2- 172.21.15.114:0/3890177641 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f1b0c06d8a0 0x7f1b0c06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:25:35.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.798+0000 7f1b22ffd700 1 --2- 172.21.15.114:0/3890177641 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f1b0c06d8a0 0x7f1b0c06fd60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1b1800b5c0 tx=0x7f1b18005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:25:35.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:35.802+0000 7f1b0bfff700 1 -- 172.21.15.114:0/3890177641 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1b1405a590 con 0x7f1b24074d60 2024-02-16T22:25:36.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:36.077+0000 7f1b29364700 1 -- 172.21.15.114:0/3890177641 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1b2410bb00 con 0x7f1b24074d60 2024-02-16T22:25:36.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:36.077+0000 7f1b0bfff700 1 -- 172.21.15.114:0/3890177641 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f1b1405a120 con 0x7f1b24074d60 2024-02-16T22:25:36.080 INFO:teuthology.orchestra.run.smithi114.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:25:36.080 INFO:teuthology.orchestra.run.smithi114.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:25:36.080 INFO:teuthology.orchestra.run.smithi114.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:25:36.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:36.080+0000 7f1b29364700 1 -- 172.21.15.114:0/3890177641 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f1b0c06d8a0 msgr2=0x7f1b0c06fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:36.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:36.080+0000 7f1b29364700 1 --2- 172.21.15.114:0/3890177641 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f1b0c06d8a0 0x7f1b0c06fd60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1b1800b5c0 tx=0x7f1b18005e20 comp rx=0 tx=0).stop 2024-02-16T22:25:36.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:36.080+0000 7f1b29364700 1 -- 172.21.15.114:0/3890177641 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b24074d60 msgr2=0x7f1b24198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:25:36.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:36.080+0000 7f1b29364700 1 --2- 172.21.15.114:0/3890177641 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b24074d60 0x7f1b24198aa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400c8a0 tx=0x7f1b1400cc60 comp rx=0 tx=0).stop 2024-02-16T22:25:36.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:36.080+0000 7f1b29364700 1 -- 172.21.15.114:0/3890177641 shutdown_connections 2024-02-16T22:25:36.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:36.080+0000 7f1b29364700 1 --2- 172.21.15.114:0/3890177641 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f1b0c06d8a0 0x7f1b0c06fd60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:36.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:36.080+0000 7f1b29364700 1 --2- 172.21.15.114:0/3890177641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b24073790 0x7f1b24198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:36.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:36.080+0000 7f1b29364700 1 --2- 172.21.15.114:0/3890177641 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b24074d60 0x7f1b24198aa0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:25:36.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:36.080+0000 7f1b29364700 1 -- 172.21.15.114:0/3890177641 >> 172.21.15.114:0/3890177641 conn(0x7f1b240fc140 msgr2=0x7f1b24106a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:25:36.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:36.080+0000 7f1b29364700 1 -- 172.21.15.114:0/3890177641 shutdown_connections 2024-02-16T22:25:36.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:25:36.080+0000 7f1b29364700 1 -- 172.21.15.114:0/3890177641 wait complete. 2024-02-16T22:25:36.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:36 smithi114 ceph-mon[94452]: from='client.34226 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:25:36.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:36 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/3059592882' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:25:36.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:36.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:36 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/3890177641' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:25:36.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:36 smithi123 ceph-mon[82331]: from='client.34226 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:25:36.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:36 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/3059592882' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:25:36.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:36.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:36 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/3890177641' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:25:37.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:37 smithi123 ceph-mon[82331]: from='client.34238 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:25:37.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:37 smithi123 ceph-mon[82331]: pgmap v63: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 23 B/s, 0 objects/s recovering 2024-02-16T22:25:37.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:37 smithi114 ceph-mon[94452]: from='client.34238 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:25:37.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:37 smithi114 ceph-mon[94452]: pgmap v63: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 23 B/s, 0 objects/s recovering 2024-02-16T22:25:39.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:39 smithi123 ceph-mon[82331]: pgmap v64: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 23 B/s, 0 objects/s recovering 2024-02-16T22:25:39.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:39 smithi114 ceph-mon[94452]: pgmap v64: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 23 B/s, 0 objects/s recovering 2024-02-16T22:25:41.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:41 smithi123 ceph-mon[82331]: pgmap v65: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:25:41.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:41 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T22:25:41.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:41 smithi123 ceph-mon[82331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T22:25:41.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:41 smithi123 ceph-mon[82331]: Upgrade: osd.1 is safe to restart 2024-02-16T22:25:41.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:41 smithi114 ceph-mon[94452]: pgmap v65: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:25:41.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:41 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T22:25:41.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:41 smithi114 ceph-mon[94452]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T22:25:41.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:41 smithi114 ceph-mon[94452]: Upgrade: osd.1 is safe to restart 2024-02-16T22:25:43.099 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:42 smithi114 ceph-mon[94452]: pgmap v66: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:25:43.099 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:42 smithi114 ceph-mon[94452]: Upgrade: Updating osd.1 2024-02-16T22:25:43.099 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:42 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:43.099 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:42 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-16T22:25:43.099 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:42 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:25:43.099 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:42 smithi114 ceph-mon[94452]: Deploying daemon osd.1 on smithi114 2024-02-16T22:25:43.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:42 smithi123 ceph-mon[82331]: pgmap v66: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:25:43.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:42 smithi123 ceph-mon[82331]: Upgrade: Updating osd.1 2024-02-16T22:25:43.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:42 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:43.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:42 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-16T22:25:43.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:42 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:25:43.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:42 smithi123 ceph-mon[82331]: Deploying daemon osd.1 on smithi114 2024-02-16T22:25:45.148 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:44 smithi114 ceph-mon[94452]: pgmap v67: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:25:45.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:44 smithi123 ceph-mon[82331]: pgmap v67: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:25:45.422 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:45 smithi114 systemd[1]: Stopping Ceph osd.1 for 1dcbaabe-cd18-11ee-95ba-87774f69a715... 2024-02-16T22:25:45.701 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:45 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1[55938]: 2024-02-16T22:25:45.434+0000 7f8a9f959700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T22:25:45.701 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:45 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1[55938]: 2024-02-16T22:25:45.434+0000 7f8a9f959700 -1 osd.1 53 *** Got signal Terminated *** 2024-02-16T22:25:45.702 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:45 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1[55938]: 2024-02-16T22:25:45.434+0000 7f8a9f959700 -1 osd.1 53 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T22:25:46.193 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:45 smithi114 ceph-mon[94452]: osd.1 marked itself down and dead 2024-02-16T22:25:46.194 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:45 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:25:46.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:45 smithi123 ceph-mon[82331]: osd.1 marked itself down and dead 2024-02-16T22:25:46.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:45 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:25:46.451 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:46 smithi114 podman[108673]: 2024-02-16 22:25:46.292005946 +0000 UTC m=+0.969339076 container died 32bb191b88e77cd946f16d6e4ebac5dbf455f5a258ed75f28ce1b4f68a141b55 (image=quay.io/ceph/ceph@sha256:6176526d5ef4047f148d47c951653dcc6b0de49091fda006d3a088e7a1f404d0, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1, RELEASE=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.29.1, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD) 2024-02-16T22:25:46.888 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:46 smithi114 podman[108673]: 2024-02-16 22:25:46.660490704 +0000 UTC m=+1.337823833 container cleanup 32bb191b88e77cd946f16d6e4ebac5dbf455f5a258ed75f28ce1b4f68a141b55 (image=quay.io/ceph/ceph@sha256:6176526d5ef4047f148d47c951653dcc6b0de49091fda006d3a088e7a1f404d0, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, RELEASE=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-02-16T22:25:46.888 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:46 smithi114 bash[108673]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1 2024-02-16T22:25:47.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:46 smithi114 ceph-mon[94452]: pgmap v68: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:25:47.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:46 smithi114 ceph-mon[94452]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T22:25:47.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:46 smithi114 ceph-mon[94452]: osdmap e54: 6 total, 5 up, 6 in 2024-02-16T22:25:47.202 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:46 smithi114 podman[108690]: 2024-02-16 22:25:46.954317068 +0000 UTC m=+0.659352796 container remove 32bb191b88e77cd946f16d6e4ebac5dbf455f5a258ed75f28ce1b4f68a141b55 (image=quay.io/ceph/ceph@sha256:6176526d5ef4047f148d47c951653dcc6b0de49091fda006d3a088e7a1f404d0, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=HEAD, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-02-16T22:25:47.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:46 smithi123 ceph-mon[82331]: pgmap v68: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:25:47.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:46 smithi123 ceph-mon[82331]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T22:25:47.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:46 smithi123 ceph-mon[82331]: osdmap e54: 6 total, 5 up, 6 in 2024-02-16T22:25:47.525 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:47 smithi114 podman[108785]: 2024-02-16 22:25:47.174844712 +0000 UTC m=+0.018197891 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T22:25:47.526 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:47 smithi114 podman[108785]: 2024-02-16 22:25:47.305104805 +0000 UTC m=+0.148458025 container create fa7d984afaf9a2eeab85b242da0ebb91cfa7db9d51628c6b2ff0b5f597528ce0 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240214) 2024-02-16T22:25:47.526 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:47 smithi114 podman[108785]: 2024-02-16 22:25:47.524239631 +0000 UTC m=+0.367592810 container init fa7d984afaf9a2eeab85b242da0ebb91cfa7db9d51628c6b2ff0b5f597528ce0 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T22:25:47.818 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:47 smithi114 podman[108785]: 2024-02-16 22:25:47.527690284 +0000 UTC m=+0.371043472 container start fa7d984afaf9a2eeab85b242da0ebb91cfa7db9d51628c6b2ff0b5f597528ce0 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5) 2024-02-16T22:25:47.818 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:47 smithi114 podman[108785]: 2024-02-16 22:25:47.607420128 +0000 UTC m=+0.450773325 container attach fa7d984afaf9a2eeab85b242da0ebb91cfa7db9d51628c6b2ff0b5f597528ce0 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-02-16T22:25:48.161 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:47 smithi114 ceph-mon[94452]: osdmap e55: 6 total, 5 up, 6 in 2024-02-16T22:25:48.162 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:47 smithi114 podman[108785]: 2024-02-16 22:25:47.817368059 +0000 UTC m=+0.660721238 container died fa7d984afaf9a2eeab85b242da0ebb91cfa7db9d51628c6b2ff0b5f597528ce0 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T22:25:48.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:47 smithi123 ceph-mon[82331]: osdmap e55: 6 total, 5 up, 6 in 2024-02-16T22:25:48.702 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:48 smithi114 podman[108810]: 2024-02-16 22:25:48.43634198 +0000 UTC m=+0.609232141 container remove fa7d984afaf9a2eeab85b242da0ebb91cfa7db9d51628c6b2ff0b5f597528ce0 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0) 2024-02-16T22:25:48.702 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:48 smithi114 systemd[1]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@osd.1.service: Deactivated successfully. 2024-02-16T22:25:48.702 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:48 smithi114 systemd[1]: Stopped Ceph osd.1 for 1dcbaabe-cd18-11ee-95ba-87774f69a715. 2024-02-16T22:25:48.702 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:48 smithi114 systemd[1]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@osd.1.service: Consumed 55.640s CPU time. 2024-02-16T22:25:48.702 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:48 smithi114 systemd[1]: Starting Ceph osd.1 for 1dcbaabe-cd18-11ee-95ba-87774f69a715... 2024-02-16T22:25:49.191 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:48 smithi114 ceph-mon[94452]: pgmap v71: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:25:49.191 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:48 smithi114 ceph-mon[94452]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-02-16T22:25:49.191 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:48 smithi114 podman[108912]: 2024-02-16 22:25:48.783858916 +0000 UTC m=+0.018159586 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T22:25:49.191 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:48 smithi114 podman[108912]: 2024-02-16 22:25:48.887351121 +0000 UTC m=+0.121651778 container create f93977045d2f4d7212cdfc21186cc27703acb2de328dbbaed8590400957cd114 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1-activate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0) 2024-02-16T22:25:49.191 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:49 smithi114 podman[108912]: 2024-02-16 22:25:49.099571584 +0000 UTC m=+0.333872249 container init f93977045d2f4d7212cdfc21186cc27703acb2de328dbbaed8590400957cd114 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1-activate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-02-16T22:25:49.191 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:49 smithi114 podman[108912]: 2024-02-16 22:25:49.103250605 +0000 UTC m=+0.337551274 container start f93977045d2f4d7212cdfc21186cc27703acb2de328dbbaed8590400957cd114 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1-activate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-02-16T22:25:49.191 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:49 smithi114 podman[108912]: 2024-02-16 22:25:49.189824431 +0000 UTC m=+0.424125104 container attach f93977045d2f4d7212cdfc21186cc27703acb2de328dbbaed8590400957cd114 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1-activate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-02-16T22:25:49.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:48 smithi123 ceph-mon[82331]: pgmap v71: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:25:49.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:48 smithi123 ceph-mon[82331]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-02-16T22:25:49.891 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:49 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1-activate[108927]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-16T22:25:49.891 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:49 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1-activate[108927]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-02-16T22:25:49.891 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:49 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1-activate[108927]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-02-16T22:25:49.892 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:49 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1-activate[108927]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-02-16T22:25:49.892 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:49 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1-activate[108927]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-02-16T22:25:49.892 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:49 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1-activate[108927]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-16T22:25:49.892 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:49 smithi114 bash[108912]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-16T22:25:49.892 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:49 smithi114 bash[108912]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-02-16T22:25:49.892 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:49 smithi114 bash[108912]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-02-16T22:25:49.892 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:49 smithi114 bash[108912]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-02-16T22:25:49.892 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:49 smithi114 bash[108912]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-02-16T22:25:49.892 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:49 smithi114 bash[108912]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-16T22:25:49.892 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:49 smithi114 bash[108912]: --> ceph-volume raw activate successful for osd ID: 1 2024-02-16T22:25:49.892 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:49 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1-activate[108927]: --> ceph-volume raw activate successful for osd ID: 1 2024-02-16T22:25:49.892 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:49 smithi114 podman[108912]: 2024-02-16 22:25:49.57166911 +0000 UTC m=+0.805969770 container died f93977045d2f4d7212cdfc21186cc27703acb2de328dbbaed8590400957cd114 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1-activate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-02-16T22:25:50.452 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:50 smithi114 podman[109067]: 2024-02-16 22:25:50.169222208 +0000 UTC m=+0.592789431 container remove f93977045d2f4d7212cdfc21186cc27703acb2de328dbbaed8590400957cd114 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1-activate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-02-16T22:25:50.452 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:50 smithi114 podman[109132]: 2024-02-16 22:25:50.290496899 +0000 UTC m=+0.018217793 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T22:25:50.452 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:50 smithi114 podman[109132]: 2024-02-16 22:25:50.394989006 +0000 UTC m=+0.122709891 container create 3f90e792898fe87e2fa6e43bce9d0479cade0580bbe38a5b6d81039508338b8b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True) 2024-02-16T22:25:50.903 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:50 smithi114 podman[109132]: 2024-02-16 22:25:50.647870413 +0000 UTC m=+0.375591301 container init 3f90e792898fe87e2fa6e43bce9d0479cade0580bbe38a5b6d81039508338b8b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-02-16T22:25:50.903 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:50 smithi114 podman[109132]: 2024-02-16 22:25:50.65106912 +0000 UTC m=+0.378790011 container start 3f90e792898fe87e2fa6e43bce9d0479cade0580bbe38a5b6d81039508338b8b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214) 2024-02-16T22:25:50.903 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:50 smithi114 ceph-osd[109151]: -- 172.21.15.114:0/4090933760 <== mon.0 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ae1f7551e0 con 0x55ae1f727000 2024-02-16T22:25:50.904 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:50 smithi114 bash[109132]: 3f90e792898fe87e2fa6e43bce9d0479cade0580bbe38a5b6d81039508338b8b 2024-02-16T22:25:50.904 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:50 smithi114 systemd[1]: Started Ceph osd.1 for 1dcbaabe-cd18-11ee-95ba-87774f69a715. 2024-02-16T22:25:51.196 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:50 smithi114 ceph-mon[94452]: pgmap v72: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:25:51.196 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:50 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:51.196 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:50 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:51.196 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:50 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:25:51.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:50 smithi123 ceph-mon[82331]: pgmap v72: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:25:51.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:50 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:51.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:50 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:51.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:50 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:25:51.451 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:51 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1[109147]: 2024-02-16T22:25:51.194+0000 7fe7afc787c0 -1 Falling back to public interface 2024-02-16T22:25:52.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:51 smithi123 ceph-mon[82331]: Health check failed: Degraded data redundancy: 35/267 objects degraded (13.109%), 13 pgs degraded (PG_DEGRADED) 2024-02-16T22:25:52.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:51 smithi123 ceph-mon[82331]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-02-16T22:25:52.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:51 smithi114 ceph-mon[94452]: Health check failed: Degraded data redundancy: 35/267 objects degraded (13.109%), 13 pgs degraded (PG_DEGRADED) 2024-02-16T22:25:52.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:51 smithi114 ceph-mon[94452]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-02-16T22:25:53.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:52 smithi123 ceph-mon[82331]: pgmap v73: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-02-16T22:25:53.366 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:52 smithi114 ceph-mon[94452]: pgmap v73: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-02-16T22:25:55.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:54 smithi114 ceph-mon[94452]: pgmap v74: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-02-16T22:25:55.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:54 smithi123 ceph-mon[82331]: pgmap v74: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-02-16T22:25:55.795 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:55 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1[109147]: 2024-02-16T22:25:55.687+0000 7fe7afc787c0 -1 osd.1 0 read_superblock omap replica is missing. 2024-02-16T22:25:56.451 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:56 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1[109147]: 2024-02-16T22:25:56.087+0000 7fe7afc787c0 -1 osd.1 53 log_to_monitors true 2024-02-16T22:25:57.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:56 smithi123 ceph-mon[82331]: pgmap v75: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-02-16T22:25:57.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:56 smithi123 ceph-mon[82331]: from='osd.1 [v2:172.21.15.114:6810/1152173882,v1:172.21.15.114:6811/1152173882]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-16T22:25:57.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:56 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:57.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:56 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:57.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:56 smithi114 ceph-mon[94452]: pgmap v75: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-02-16T22:25:57.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:56 smithi114 ceph-mon[94452]: from='osd.1 [v2:172.21.15.114:6810/1152173882,v1:172.21.15.114:6811/1152173882]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-16T22:25:57.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:56 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:57.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:56 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:58.179 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:25:57 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1[109147]: 2024-02-16T22:25:57.823+0000 7fe79d0c8700 -1 osd.1 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T22:25:58.179 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:57 smithi114 ceph-mon[94452]: from='osd.1 [v2:172.21.15.114:6810/1152173882,v1:172.21.15.114:6811/1152173882]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-16T22:25:58.180 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:57 smithi114 ceph-mon[94452]: osdmap e56: 6 total, 5 up, 6 in 2024-02-16T22:25:58.180 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:57 smithi114 ceph-mon[94452]: from='osd.1 [v2:172.21.15.114:6810/1152173882,v1:172.21.15.114:6811/1152173882]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-02-16T22:25:58.180 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:57 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:58.180 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:57 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:58.180 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:57 smithi114 ceph-mon[94452]: from='osd.1 [v2:172.21.15.114:6810/1152173882,v1:172.21.15.114:6811/1152173882]' entity='osd.1' 2024-02-16T22:25:58.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:57 smithi123 ceph-mon[82331]: from='osd.1 [v2:172.21.15.114:6810/1152173882,v1:172.21.15.114:6811/1152173882]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-16T22:25:58.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:57 smithi123 ceph-mon[82331]: osdmap e56: 6 total, 5 up, 6 in 2024-02-16T22:25:58.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:57 smithi123 ceph-mon[82331]: from='osd.1 [v2:172.21.15.114:6810/1152173882,v1:172.21.15.114:6811/1152173882]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-02-16T22:25:58.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:57 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:58.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:57 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:25:58.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:57 smithi123 ceph-mon[82331]: from='osd.1 [v2:172.21.15.114:6810/1152173882,v1:172.21.15.114:6811/1152173882]' entity='osd.1' 2024-02-16T22:25:59.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:59 smithi114 ceph-mon[94452]: pgmap v77: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-02-16T22:25:59.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:59 smithi114 ceph-mon[94452]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T22:25:59.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:59 smithi114 ceph-mon[94452]: osd.1 [v2:172.21.15.114:6810/1152173882,v1:172.21.15.114:6811/1152173882] boot 2024-02-16T22:25:59.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:59 smithi114 ceph-mon[94452]: osdmap e57: 6 total, 6 up, 6 in 2024-02-16T22:25:59.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:25:59 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:25:59.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:59 smithi123 ceph-mon[82331]: pgmap v77: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-02-16T22:25:59.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:59 smithi123 ceph-mon[82331]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T22:25:59.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:59 smithi123 ceph-mon[82331]: osd.1 [v2:172.21.15.114:6810/1152173882,v1:172.21.15.114:6811/1152173882] boot 2024-02-16T22:25:59.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:59 smithi123 ceph-mon[82331]: osdmap e57: 6 total, 6 up, 6 in 2024-02-16T22:25:59.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:25:59 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T22:26:01.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:00 smithi114 ceph-mon[94452]: pgmap v79: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-02-16T22:26:01.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:00 smithi114 ceph-mon[94452]: osdmap e58: 6 total, 6 up, 6 in 2024-02-16T22:26:01.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:00 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:01.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:00 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:01.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:00 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:26:01.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:00 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:26:01.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:00 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:01.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:00 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:01.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:00 smithi123 ceph-mon[82331]: pgmap v79: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-02-16T22:26:01.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:00 smithi123 ceph-mon[82331]: osdmap e58: 6 total, 6 up, 6 in 2024-02-16T22:26:01.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:00 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:01.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:00 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:01.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:00 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:26:01.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:00 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:26:01.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:00 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:01.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:00 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:02.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:26:02.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:26:02.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:26:02.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:26:02.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:26:02.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T22:26:02.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:01 smithi114 ceph-mon[94452]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T22:26:02.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:01 smithi114 ceph-mon[94452]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-02-16T22:26:02.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:01 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:26:02.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:01 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:26:02.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:01 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:26:02.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:01 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:26:02.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:01 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:26:02.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:01 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T22:26:02.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:01 smithi123 ceph-mon[82331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T22:26:02.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:01 smithi123 ceph-mon[82331]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-02-16T22:26:03.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:02 smithi114 ceph-mon[94452]: pgmap v81: 65 pgs: 7 active+undersized+degraded, 5 active+undersized, 53 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 18/267 objects degraded (6.742%) 2024-02-16T22:26:03.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:02 smithi114 ceph-mon[94452]: Health check update: Degraded data redundancy: 18/267 objects degraded (6.742%), 7 pgs degraded (PG_DEGRADED) 2024-02-16T22:26:03.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:02 smithi123 ceph-mon[82331]: pgmap v81: 65 pgs: 7 active+undersized+degraded, 5 active+undersized, 53 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 18/267 objects degraded (6.742%) 2024-02-16T22:26:03.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:02 smithi123 ceph-mon[82331]: Health check update: Degraded data redundancy: 18/267 objects degraded (6.742%), 7 pgs degraded (PG_DEGRADED) 2024-02-16T22:26:04.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:03 smithi114 ceph-mon[94452]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/267 objects degraded (6.742%), 7 pgs degraded) 2024-02-16T22:26:04.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:03 smithi123 ceph-mon[82331]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/267 objects degraded (6.742%), 7 pgs degraded) 2024-02-16T22:26:05.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:04 smithi114 ceph-mon[94452]: pgmap v82: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:05.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:04 smithi123 ceph-mon[82331]: pgmap v82: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:06.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.198+0000 7f13f39b7700 1 -- 172.21.15.114:0/3747498407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13ec100cd0 msgr2=0x7f13ec101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:06.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.198+0000 7f13f39b7700 1 --2- 172.21.15.114:0/3747498407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13ec100cd0 0x7f13ec101130 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009910 tx=0x7f13e0009c20 comp rx=0 tx=0).stop 2024-02-16T22:26:06.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.198+0000 7f13f39b7700 1 -- 172.21.15.114:0/3747498407 shutdown_connections 2024-02-16T22:26:06.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.199+0000 7f13f39b7700 1 --2- 172.21.15.114:0/3747498407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13ec100cd0 0x7f13ec101130 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:06.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.199+0000 7f13f39b7700 1 --2- 172.21.15.114:0/3747498407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13ec0ffad0 0x7f13ec0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:06.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.199+0000 7f13f39b7700 1 -- 172.21.15.114:0/3747498407 >> 172.21.15.114:0/3747498407 conn(0x7f13ec0fb050 msgr2=0x7f13ec0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:26:06.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.199+0000 7f13f39b7700 1 -- 172.21.15.114:0/3747498407 shutdown_connections 2024-02-16T22:26:06.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.199+0000 7f13f39b7700 1 -- 172.21.15.114:0/3747498407 wait complete. 2024-02-16T22:26:06.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.199+0000 7f13f39b7700 1 Processor -- start 2024-02-16T22:26:06.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.199+0000 7f13f39b7700 1 -- start start 2024-02-16T22:26:06.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.199+0000 7f13f39b7700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13ec0ffad0 0x7f13ec103a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:06.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.199+0000 7f13f39b7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13ec100cd0 0x7f13ec1020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:06.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.199+0000 7f13f39b7700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec1025f0 con 0x7f13ec0ffad0 2024-02-16T22:26:06.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.199+0000 7f13f39b7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec102730 con 0x7f13ec100cd0 2024-02-16T22:26:06.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.199+0000 7f13f1753700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13ec0ffad0 0x7f13ec103a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:06.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.200+0000 7f13f1753700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13ec0ffad0 0x7f13ec103a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59476/0 (socket says 172.21.15.114:59476) 2024-02-16T22:26:06.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.200+0000 7f13f1753700 1 -- 172.21.15.114:0/3582291030 learned_addr learned my addr 172.21.15.114:0/3582291030 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:26:06.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.200+0000 7f13f0f52700 1 --2- 172.21.15.114:0/3582291030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13ec100cd0 0x7f13ec1020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:06.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.200+0000 7f13f1753700 1 -- 172.21.15.114:0/3582291030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13ec100cd0 msgr2=0x7f13ec1020b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:06.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.200+0000 7f13f1753700 1 --2- 172.21.15.114:0/3582291030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13ec100cd0 0x7f13ec1020b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:06.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.200+0000 7f13f1753700 1 -- 172.21.15.114:0/3582291030 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e00095f0 con 0x7f13ec0ffad0 2024-02-16T22:26:06.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.200+0000 7f13f0f52700 1 --2- 172.21.15.114:0/3582291030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13ec100cd0 0x7f13ec1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:26:06.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.200+0000 7f13f1753700 1 --2- 172.21.15.114:0/3582291030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13ec0ffad0 0x7f13ec103a60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f13dc007e90 tx=0x7f13dc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:26:06.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.200+0000 7f13ea7fc700 1 -- 172.21.15.114:0/3582291030 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13dc017070 con 0x7f13ec0ffad0 2024-02-16T22:26:06.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.200+0000 7f13f39b7700 1 -- 172.21.15.114:0/3582291030 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13ec102a10 con 0x7f13ec0ffad0 2024-02-16T22:26:06.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.200+0000 7f13ea7fc700 1 -- 172.21.15.114:0/3582291030 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f13dc00f040 con 0x7f13ec0ffad0 2024-02-16T22:26:06.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.200+0000 7f13ea7fc700 1 -- 172.21.15.114:0/3582291030 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13dc004880 con 0x7f13ec0ffad0 2024-02-16T22:26:06.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.200+0000 7f13f39b7700 1 -- 172.21.15.114:0/3582291030 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13ec19e3e0 con 0x7f13ec0ffad0 2024-02-16T22:26:06.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.201+0000 7f13f39b7700 1 -- 172.21.15.114:0/3582291030 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13ec108630 con 0x7f13ec0ffad0 2024-02-16T22:26:06.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.202+0000 7f13ea7fc700 1 -- 172.21.15.114:0/3582291030 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f13dc0040b0 con 0x7f13ec0ffad0 2024-02-16T22:26:06.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.202+0000 7f13ea7fc700 1 --2- 172.21.15.114:0/3582291030 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f13d406d850 0x7f13d406fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:06.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.202+0000 7f13ea7fc700 1 -- 172.21.15.114:0/3582291030 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f13dc08cef0 con 0x7f13ec0ffad0 2024-02-16T22:26:06.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.202+0000 7f13f0f52700 1 --2- 172.21.15.114:0/3582291030 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f13d406d850 0x7f13d406fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:06.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.203+0000 7f13f0f52700 1 --2- 172.21.15.114:0/3582291030 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f13d406d850 0x7f13d406fd10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f13e0005b40 tx=0x7f13e0005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:26:06.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.207+0000 7f13ea7fc700 1 -- 172.21.15.114:0/3582291030 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f13dc05a280 con 0x7f13ec0ffad0 2024-02-16T22:26:06.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.422+0000 7f13f39b7700 1 -- 172.21.15.114:0/3582291030 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f13ec0694a0 con 0x7f13d406d850 2024-02-16T22:26:06.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.424+0000 7f13ea7fc700 1 -- 172.21.15.114:0/3582291030 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f13ec0694a0 con 0x7f13d406d850 2024-02-16T22:26:06.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.426+0000 7f13f39b7700 1 -- 172.21.15.114:0/3582291030 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f13d406d850 msgr2=0x7f13d406fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:06.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.426+0000 7f13f39b7700 1 --2- 172.21.15.114:0/3582291030 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f13d406d850 0x7f13d406fd10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f13e0005b40 tx=0x7f13e0005a90 comp rx=0 tx=0).stop 2024-02-16T22:26:06.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.426+0000 7f13f39b7700 1 -- 172.21.15.114:0/3582291030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13ec0ffad0 msgr2=0x7f13ec103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:06.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.426+0000 7f13f39b7700 1 --2- 172.21.15.114:0/3582291030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13ec0ffad0 0x7f13ec103a60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f13dc007e90 tx=0x7f13dc00d3b0 comp rx=0 tx=0).stop 2024-02-16T22:26:06.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.426+0000 7f13f39b7700 1 -- 172.21.15.114:0/3582291030 shutdown_connections 2024-02-16T22:26:06.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.426+0000 7f13f39b7700 1 --2- 172.21.15.114:0/3582291030 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f13d406d850 0x7f13d406fd10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:06.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.426+0000 7f13f39b7700 1 --2- 172.21.15.114:0/3582291030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13ec0ffad0 0x7f13ec103a60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:06.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.426+0000 7f13f39b7700 1 --2- 172.21.15.114:0/3582291030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13ec100cd0 0x7f13ec1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:06.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.426+0000 7f13f39b7700 1 -- 172.21.15.114:0/3582291030 >> 172.21.15.114:0/3582291030 conn(0x7f13ec0fb050 msgr2=0x7f13ec068d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:26:06.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.426+0000 7f13f39b7700 1 -- 172.21.15.114:0/3582291030 shutdown_connections 2024-02-16T22:26:06.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.426+0000 7f13f39b7700 1 -- 172.21.15.114:0/3582291030 wait complete. 2024-02-16T22:26:06.442 INFO:teuthology.orchestra.run.smithi114.stdout:true 2024-02-16T22:26:06.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.534+0000 7f00eaa89700 1 -- 172.21.15.114:0/1747442694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00e4101dc0 msgr2=0x7f00e4102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:06.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.534+0000 7f00eaa89700 1 --2- 172.21.15.114:0/1747442694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00e4101dc0 0x7f00e4102220 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009910 tx=0x7f00d8009c20 comp rx=0 tx=0).stop 2024-02-16T22:26:06.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.534+0000 7f00eaa89700 1 -- 172.21.15.114:0/1747442694 shutdown_connections 2024-02-16T22:26:06.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.534+0000 7f00eaa89700 1 --2- 172.21.15.114:0/1747442694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00e4101dc0 0x7f00e4102220 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:06.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.534+0000 7f00eaa89700 1 --2- 172.21.15.114:0/1747442694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00e4100bc0 0x7f00e4100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:06.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.534+0000 7f00eaa89700 1 -- 172.21.15.114:0/1747442694 >> 172.21.15.114:0/1747442694 conn(0x7f00e40fc140 msgr2=0x7f00e40fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:26:06.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.534+0000 7f00eaa89700 1 -- 172.21.15.114:0/1747442694 shutdown_connections 2024-02-16T22:26:06.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.534+0000 7f00eaa89700 1 -- 172.21.15.114:0/1747442694 wait complete. 2024-02-16T22:26:06.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.535+0000 7f00eaa89700 1 Processor -- start 2024-02-16T22:26:06.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.535+0000 7f00eaa89700 1 -- start start 2024-02-16T22:26:06.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.535+0000 7f00eaa89700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00e4100bc0 0x7f00e4196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:06.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.535+0000 7f00eaa89700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00e4101dc0 0x7f00e41968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:06.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.535+0000 7f00eaa89700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e4196ed0 con 0x7f00e4100bc0 2024-02-16T22:26:06.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.535+0000 7f00eaa89700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e4197010 con 0x7f00e4101dc0 2024-02-16T22:26:06.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.535+0000 7f00e8825700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00e4100bc0 0x7f00e4196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:06.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.535+0000 7f00e8825700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00e4100bc0 0x7f00e4196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59496/0 (socket says 172.21.15.114:59496) 2024-02-16T22:26:06.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.535+0000 7f00e8825700 1 -- 172.21.15.114:0/3329482037 learned_addr learned my addr 172.21.15.114:0/3329482037 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:26:06.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.535+0000 7f00e3fff700 1 --2- 172.21.15.114:0/3329482037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00e4101dc0 0x7f00e41968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:06.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.536+0000 7f00e8825700 1 -- 172.21.15.114:0/3329482037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00e4101dc0 msgr2=0x7f00e41968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:06.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.536+0000 7f00e8825700 1 --2- 172.21.15.114:0/3329482037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00e4101dc0 0x7f00e41968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:06.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.536+0000 7f00e8825700 1 -- 172.21.15.114:0/3329482037 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d80095f0 con 0x7f00e4100bc0 2024-02-16T22:26:06.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.536+0000 7f00e8825700 1 --2- 172.21.15.114:0/3329482037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00e4100bc0 0x7f00e4196370 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00d900 tx=0x7f00dc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:26:06.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.536+0000 7f00e1ffb700 1 -- 172.21.15.114:0/3329482037 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00dc0041f0 con 0x7f00e4100bc0 2024-02-16T22:26:06.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.536+0000 7f00e1ffb700 1 -- 172.21.15.114:0/3329482037 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f00dc004350 con 0x7f00e4100bc0 2024-02-16T22:26:06.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.536+0000 7f00eaa89700 1 -- 172.21.15.114:0/3329482037 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00e419bac0 con 0x7f00e4100bc0 2024-02-16T22:26:06.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.536+0000 7f00e1ffb700 1 -- 172.21.15.114:0/3329482037 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00dc003d20 con 0x7f00e4100bc0 2024-02-16T22:26:06.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.536+0000 7f00eaa89700 1 -- 172.21.15.114:0/3329482037 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e419c010 con 0x7f00e4100bc0 2024-02-16T22:26:06.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.537+0000 7f00eaa89700 1 -- 172.21.15.114:0/3329482037 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00e4109720 con 0x7f00e4100bc0 2024-02-16T22:26:06.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.537+0000 7f00e1ffb700 1 -- 172.21.15.114:0/3329482037 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f00dc009730 con 0x7f00e4100bc0 2024-02-16T22:26:06.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.538+0000 7f00e1ffb700 1 --2- 172.21.15.114:0/3329482037 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f00cc06d8a0 0x7f00cc06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:06.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.538+0000 7f00e1ffb700 1 -- 172.21.15.114:0/3329482037 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f00dc08d350 con 0x7f00e4100bc0 2024-02-16T22:26:06.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.538+0000 7f00e3fff700 1 --2- 172.21.15.114:0/3329482037 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f00cc06d8a0 0x7f00cc06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:06.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.538+0000 7f00e3fff700 1 --2- 172.21.15.114:0/3329482037 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f00cc06d8a0 0x7f00cc06fd60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f00d8006010 tx=0x7f00d800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:26:06.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.543+0000 7f00e1ffb700 1 -- 172.21.15.114:0/3329482037 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f00dc05a6e0 con 0x7f00e4100bc0 2024-02-16T22:26:06.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.755+0000 7f00eaa89700 1 -- 172.21.15.114:0/3329482037 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f00e4106710 con 0x7f00cc06d8a0 2024-02-16T22:26:06.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.756+0000 7f00e1ffb700 1 -- 172.21.15.114:0/3329482037 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f00e4106710 con 0x7f00cc06d8a0 2024-02-16T22:26:06.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.758+0000 7f00eaa89700 1 -- 172.21.15.114:0/3329482037 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f00cc06d8a0 msgr2=0x7f00cc06fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:06.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.758+0000 7f00eaa89700 1 --2- 172.21.15.114:0/3329482037 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f00cc06d8a0 0x7f00cc06fd60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f00d8006010 tx=0x7f00d800b540 comp rx=0 tx=0).stop 2024-02-16T22:26:06.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.758+0000 7f00eaa89700 1 -- 172.21.15.114:0/3329482037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00e4100bc0 msgr2=0x7f00e4196370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:06.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.758+0000 7f00eaa89700 1 --2- 172.21.15.114:0/3329482037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00e4100bc0 0x7f00e4196370 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00d900 tx=0x7f00dc00dc10 comp rx=0 tx=0).stop 2024-02-16T22:26:06.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.758+0000 7f00eaa89700 1 -- 172.21.15.114:0/3329482037 shutdown_connections 2024-02-16T22:26:06.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.758+0000 7f00eaa89700 1 --2- 172.21.15.114:0/3329482037 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f00cc06d8a0 0x7f00cc06fd60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:06.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.758+0000 7f00eaa89700 1 --2- 172.21.15.114:0/3329482037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00e4100bc0 0x7f00e4196370 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:06.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.758+0000 7f00eaa89700 1 --2- 172.21.15.114:0/3329482037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00e4101dc0 0x7f00e41968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:06.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.758+0000 7f00eaa89700 1 -- 172.21.15.114:0/3329482037 >> 172.21.15.114:0/3329482037 conn(0x7f00e40fc140 msgr2=0x7f00e4104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:26:06.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.758+0000 7f00eaa89700 1 -- 172.21.15.114:0/3329482037 shutdown_connections 2024-02-16T22:26:06.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.758+0000 7f00eaa89700 1 -- 172.21.15.114:0/3329482037 wait complete. 2024-02-16T22:26:06.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.863+0000 7f7cf4618700 1 -- 172.21.15.114:0/230353389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cec1010e0 msgr2=0x7f7cec101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:06.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.863+0000 7f7cf4618700 1 --2- 172.21.15.114:0/230353389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cec1010e0 0x7f7cec101500 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8009910 tx=0x7f7ce8009c20 comp rx=0 tx=0).stop 2024-02-16T22:26:06.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.863+0000 7f7cf4618700 1 -- 172.21.15.114:0/230353389 shutdown_connections 2024-02-16T22:26:06.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.863+0000 7f7cf4618700 1 --2- 172.21.15.114:0/230353389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cec0ffd40 0x7f7cec1001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:06.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.863+0000 7f7cf4618700 1 --2- 172.21.15.114:0/230353389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cec1010e0 0x7f7cec101500 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:06.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.863+0000 7f7cf4618700 1 -- 172.21.15.114:0/230353389 >> 172.21.15.114:0/230353389 conn(0x7f7cec0fa810 msgr2=0x7f7cec0fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:26:06.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.863+0000 7f7cf4618700 1 -- 172.21.15.114:0/230353389 shutdown_connections 2024-02-16T22:26:06.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.863+0000 7f7cf4618700 1 -- 172.21.15.114:0/230353389 wait complete. 2024-02-16T22:26:06.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.864+0000 7f7cf4618700 1 Processor -- start 2024-02-16T22:26:06.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.864+0000 7f7cf4618700 1 -- start start 2024-02-16T22:26:06.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.864+0000 7f7cf4618700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cec0ffd40 0x7f7cec196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:06.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.864+0000 7f7cf4618700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cec1010e0 0x7f7cec196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:06.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.864+0000 7f7cf4618700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cec196fa0 con 0x7f7cec1010e0 2024-02-16T22:26:06.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.864+0000 7f7cf4618700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cec1970e0 con 0x7f7cec0ffd40 2024-02-16T22:26:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.864+0000 7f7cf1bb3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cec1010e0 0x7f7cec196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.864+0000 7f7cf1bb3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cec1010e0 0x7f7cec196980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59508/0 (socket says 172.21.15.114:59508) 2024-02-16T22:26:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.864+0000 7f7cf1bb3700 1 -- 172.21.15.114:0/2486806691 learned_addr learned my addr 172.21.15.114:0/2486806691 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:26:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.864+0000 7f7cf23b4700 1 --2- 172.21.15.114:0/2486806691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cec0ffd40 0x7f7cec196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.864+0000 7f7cf1bb3700 1 -- 172.21.15.114:0/2486806691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cec0ffd40 msgr2=0x7f7cec196440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.864+0000 7f7cf1bb3700 1 --2- 172.21.15.114:0/2486806691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cec0ffd40 0x7f7cec196440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.864+0000 7f7cf1bb3700 1 -- 172.21.15.114:0/2486806691 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce80095f0 con 0x7f7cec1010e0 2024-02-16T22:26:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.865+0000 7f7cf1bb3700 1 --2- 172.21.15.114:0/2486806691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cec1010e0 0x7f7cec196980 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00c8f0 tx=0x7f7cdc00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:26:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.865+0000 7f7ce37fe700 1 -- 172.21.15.114:0/2486806691 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cdc0078e0 con 0x7f7cec1010e0 2024-02-16T22:26:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.865+0000 7f7ce37fe700 1 -- 172.21.15.114:0/2486806691 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7cdc00f450 con 0x7f7cec1010e0 2024-02-16T22:26:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.865+0000 7f7cf4618700 1 -- 172.21.15.114:0/2486806691 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cec19bb90 con 0x7f7cec1010e0 2024-02-16T22:26:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.865+0000 7f7ce37fe700 1 -- 172.21.15.114:0/2486806691 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cdc018610 con 0x7f7cec1010e0 2024-02-16T22:26:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.865+0000 7f7cf4618700 1 -- 172.21.15.114:0/2486806691 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cec19c0e0 con 0x7f7cec1010e0 2024-02-16T22:26:06.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.866+0000 7f7cf4618700 1 -- 172.21.15.114:0/2486806691 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cec1086a0 con 0x7f7cec1010e0 2024-02-16T22:26:06.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.866+0000 7f7ce37fe700 1 -- 172.21.15.114:0/2486806691 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f7cdc00f5c0 con 0x7f7cec1010e0 2024-02-16T22:26:06.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.867+0000 7f7ce37fe700 1 --2- 172.21.15.114:0/2486806691 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f7cd806d8a0 0x7f7cd806fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:06.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.867+0000 7f7ce37fe700 1 -- 172.21.15.114:0/2486806691 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f7cdc08d280 con 0x7f7cec1010e0 2024-02-16T22:26:06.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.867+0000 7f7cf23b4700 1 --2- 172.21.15.114:0/2486806691 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f7cd806d8a0 0x7f7cd806fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:06.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.867+0000 7f7cf23b4700 1 --2- 172.21.15.114:0/2486806691 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f7cd806d8a0 0x7f7cd806fd60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8005090 tx=0x7f7ce801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:26:06.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:06.872+0000 7f7ce37fe700 1 -- 172.21.15.114:0/2486806691 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7cdc05a690 con 0x7f7cec1010e0 2024-02-16T22:26:07.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.076+0000 7f7cf4618700 1 -- 172.21.15.114:0/2486806691 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7cec0ff340 con 0x7f7cd806d8a0 2024-02-16T22:26:07.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.087+0000 7f7ce37fe700 1 -- 172.21.15.114:0/2486806691 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f7cec0ff340 con 0x7f7cd806d8a0 2024-02-16T22:26:07.089 INFO:teuthology.orchestra.run.smithi114.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:26:07.089 INFO:teuthology.orchestra.run.smithi114.stdout:alertmanager.smithi114 smithi114 *:9093,9094 running (12m) 10s ago 13m 24.7M - 0.25.0 c8568f914cd2 e7404fea400e 2024-02-16T22:26:07.089 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi114 smithi114 running (13m) 10s ago 13m 8266k - 18.2.1 94eadd990e0a 7fd07162cd96 2024-02-16T22:26:07.089 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi123 smithi123 running (12m) 69s ago 12m 7696k - 18.2.1 94eadd990e0a 7687f5e99621 2024-02-16T22:26:07.090 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi114 smithi114 running (77s) 10s ago 13m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 9b8ae9efe83b 2024-02-16T22:26:07.090 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi123 smithi123 running (72s) 69s ago 12m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 6d97263a0ffa 2024-02-16T22:26:07.090 INFO:teuthology.orchestra.run.smithi114.stdout:grafana.smithi114 smithi114 *:3000 running (11m) 10s ago 13m 88.4M - 9.4.7 954c08fa6188 db1c26c3bc20 2024-02-16T22:26:07.090 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.fdxpey smithi114 running (7m) 10s ago 7m 18.2M - 18.2.1 94eadd990e0a 704b40c144fb 2024-02-16T22:26:07.090 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.zzryke smithi114 running (8m) 10s ago 8m 157M - 18.2.1 94eadd990e0a 1f795d23c9a4 2024-02-16T22:26:07.090 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.gopcfa smithi123 running (7m) 69s ago 7m 17.3M - 18.2.1 94eadd990e0a 166134b7532a 2024-02-16T22:26:07.090 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.vhlqbe smithi123 running (7m) 69s ago 7m 18.8M - 18.2.1 94eadd990e0a 243c8bb96fa8 2024-02-16T22:26:07.090 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi114.vcullz smithi114 *:8443,9283,8765 running (4m) 10s ago 14m 499M - 19.0.0-1450-gf78a58c0 f081f847070a 14ebeb7888c0 2024-02-16T22:26:07.090 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi123.inzgjd smithi123 *:8443,9283,8765 running (2m) 69s ago 12m 449M - 19.0.0-1450-gf78a58c0 f081f847070a f10aab9faf9e 2024-02-16T22:26:07.090 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi114 smithi114 running (2m) 10s ago 15m 46.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 58ac06372523 2024-02-16T22:26:07.090 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi123 smithi123 running (2m) 69s ago 12m 34.0M 2048M 19.0.0-1450-gf78a58c0 f081f847070a f76485e0ea24 2024-02-16T22:26:07.090 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi114 smithi114 *:9100 running (13m) 10s ago 13m 15.5M - 1.5.0 0da6a335fe13 744a090e06d0 2024-02-16T22:26:07.090 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi123 smithi123 *:9100 running (12m) 69s ago 12m 16.0M - 1.5.0 0da6a335fe13 3994b8c29b2e 2024-02-16T22:26:07.090 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 smithi114 running (52s) 10s ago 11m 89.8M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 1978fdad94a5 2024-02-16T22:26:07.090 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 smithi114 running (16s) 10s ago 11m 12.7M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 3f90e792898f 2024-02-16T22:26:07.091 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 smithi114 running (10m) 10s ago 10m 195M 1216M 18.2.1 94eadd990e0a 6608718d0818 2024-02-16T22:26:07.091 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 smithi123 running (10m) 69s ago 10m 259M 2240M 18.2.1 94eadd990e0a 575db379de3d 2024-02-16T22:26:07.091 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 smithi123 running (9m) 69s ago 9m 220M 2240M 18.2.1 94eadd990e0a 7e266866cd9c 2024-02-16T22:26:07.091 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 smithi123 running (9m) 69s ago 9m 199M 2240M 18.2.1 94eadd990e0a 24e5bef19fda 2024-02-16T22:26:07.091 INFO:teuthology.orchestra.run.smithi114.stdout:prometheus.smithi114 smithi114 *:9095 running (2m) 10s ago 13m 59.5M - 2.43.0 a07b618ecd1d 37aac985cdd7 2024-02-16T22:26:07.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.089+0000 7f7cf4618700 1 -- 172.21.15.114:0/2486806691 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f7cd806d8a0 msgr2=0x7f7cd806fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:07.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.089+0000 7f7cf4618700 1 --2- 172.21.15.114:0/2486806691 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f7cd806d8a0 0x7f7cd806fd60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8005090 tx=0x7f7ce801a040 comp rx=0 tx=0).stop 2024-02-16T22:26:07.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.089+0000 7f7cf4618700 1 -- 172.21.15.114:0/2486806691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cec1010e0 msgr2=0x7f7cec196980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:07.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.089+0000 7f7cf4618700 1 --2- 172.21.15.114:0/2486806691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cec1010e0 0x7f7cec196980 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00c8f0 tx=0x7f7cdc00cc00 comp rx=0 tx=0).stop 2024-02-16T22:26:07.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.089+0000 7f7cf4618700 1 -- 172.21.15.114:0/2486806691 shutdown_connections 2024-02-16T22:26:07.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.089+0000 7f7cf4618700 1 --2- 172.21.15.114:0/2486806691 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f7cd806d8a0 0x7f7cd806fd60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:07.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.089+0000 7f7cf4618700 1 --2- 172.21.15.114:0/2486806691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cec0ffd40 0x7f7cec196440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:07.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.089+0000 7f7cf4618700 1 --2- 172.21.15.114:0/2486806691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cec1010e0 0x7f7cec196980 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:07.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.089+0000 7f7cf4618700 1 -- 172.21.15.114:0/2486806691 >> 172.21.15.114:0/2486806691 conn(0x7f7cec0fa810 msgr2=0x7f7cec0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:26:07.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.090+0000 7f7cf4618700 1 -- 172.21.15.114:0/2486806691 shutdown_connections 2024-02-16T22:26:07.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.090+0000 7f7cf4618700 1 -- 172.21.15.114:0/2486806691 wait complete. 2024-02-16T22:26:07.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.191+0000 7f318b283700 1 -- 172.21.15.114:0/4136902690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31840ffad0 msgr2=0x7f31840ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:07.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.191+0000 7f318b283700 1 --2- 172.21.15.114:0/4136902690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31840ffad0 0x7f31840ffef0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f317c009910 tx=0x7f317c009c20 comp rx=0 tx=0).stop 2024-02-16T22:26:07.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.191+0000 7f318b283700 1 -- 172.21.15.114:0/4136902690 shutdown_connections 2024-02-16T22:26:07.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.191+0000 7f318b283700 1 --2- 172.21.15.114:0/4136902690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3184100cd0 0x7f3184101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:07.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.191+0000 7f318b283700 1 --2- 172.21.15.114:0/4136902690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31840ffad0 0x7f31840ffef0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:07.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.191+0000 7f318b283700 1 -- 172.21.15.114:0/4136902690 >> 172.21.15.114:0/4136902690 conn(0x7f31840fb050 msgr2=0x7f31840fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:26:07.194 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:06 smithi114 ceph-mon[94452]: pgmap v83: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:07.194 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:06 smithi114 ceph-mon[94452]: from='client.34250 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:26:07.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.191+0000 7f318b283700 1 -- 172.21.15.114:0/4136902690 shutdown_connections 2024-02-16T22:26:07.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.191+0000 7f318b283700 1 -- 172.21.15.114:0/4136902690 wait complete. 2024-02-16T22:26:07.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.192+0000 7f318b283700 1 Processor -- start 2024-02-16T22:26:07.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.192+0000 7f318b283700 1 -- start start 2024-02-16T22:26:07.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.192+0000 7f318b283700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31840ffad0 0x7f31841963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:07.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.192+0000 7f318b283700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3184100cd0 0x7f31841968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:07.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.192+0000 7f318b283700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3184196f10 con 0x7f31840ffad0 2024-02-16T22:26:07.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.192+0000 7f318b283700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3184197050 con 0x7f3184100cd0 2024-02-16T22:26:07.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.192+0000 7f318901f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31840ffad0 0x7f31841963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:07.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.192+0000 7f318901f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31840ffad0 0x7f31841963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59514/0 (socket says 172.21.15.114:59514) 2024-02-16T22:26:07.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.192+0000 7f318901f700 1 -- 172.21.15.114:0/4031394842 learned_addr learned my addr 172.21.15.114:0/4031394842 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:26:07.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.192+0000 7f318881e700 1 --2- 172.21.15.114:0/4031394842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3184100cd0 0x7f31841968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:07.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.193+0000 7f318901f700 1 -- 172.21.15.114:0/4031394842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3184100cd0 msgr2=0x7f31841968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:07.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.193+0000 7f318901f700 1 --2- 172.21.15.114:0/4031394842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3184100cd0 0x7f31841968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:07.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.193+0000 7f318901f700 1 -- 172.21.15.114:0/4031394842 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317c0095f0 con 0x7f31840ffad0 2024-02-16T22:26:07.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.193+0000 7f318881e700 1 --2- 172.21.15.114:0/4031394842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3184100cd0 0x7f31841968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:26:07.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.193+0000 7f318901f700 1 --2- 172.21.15.114:0/4031394842 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31840ffad0 0x7f31841963b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f317c00bd70 tx=0x7f317c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:26:07.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.193+0000 7f317a7fc700 1 -- 172.21.15.114:0/4031394842 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f317c01d070 con 0x7f31840ffad0 2024-02-16T22:26:07.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.193+0000 7f317a7fc700 1 -- 172.21.15.114:0/4031394842 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f317c00f460 con 0x7f31840ffad0 2024-02-16T22:26:07.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.193+0000 7f318b283700 1 -- 172.21.15.114:0/4031394842 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f318419baa0 con 0x7f31840ffad0 2024-02-16T22:26:07.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.193+0000 7f317a7fc700 1 -- 172.21.15.114:0/4031394842 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f317c021640 con 0x7f31840ffad0 2024-02-16T22:26:07.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.193+0000 7f318b283700 1 -- 172.21.15.114:0/4031394842 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f318419c010 con 0x7f31840ffad0 2024-02-16T22:26:07.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.194+0000 7f318b283700 1 -- 172.21.15.114:0/4031394842 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3184108630 con 0x7f31840ffad0 2024-02-16T22:26:07.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.194+0000 7f317a7fc700 1 -- 172.21.15.114:0/4031394842 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f317c00fa90 con 0x7f31840ffad0 2024-02-16T22:26:07.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.195+0000 7f317a7fc700 1 --2- 172.21.15.114:0/4031394842 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f317006d8a0 0x7f317006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:07.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.195+0000 7f317a7fc700 1 -- 172.21.15.114:0/4031394842 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f317c08e640 con 0x7f31840ffad0 2024-02-16T22:26:07.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.195+0000 7f318881e700 1 --2- 172.21.15.114:0/4031394842 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f317006d8a0 0x7f317006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:07.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.195+0000 7f318881e700 1 --2- 172.21.15.114:0/4031394842 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f317006d8a0 0x7f317006fd60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3174005950 tx=0x7f31740058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:26:07.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.199+0000 7f317a7fc700 1 -- 172.21.15.114:0/4031394842 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f317c05ba50 con 0x7f31840ffad0 2024-02-16T22:26:07.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:06 smithi123 ceph-mon[82331]: pgmap v83: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:07.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:06 smithi123 ceph-mon[82331]: from='client.34250 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:26:07.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.474+0000 7f318b283700 1 -- 172.21.15.114:0/4031394842 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f318419ca80 con 0x7f31840ffad0 2024-02-16T22:26:07.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.475+0000 7f317a7fc700 1 -- 172.21.15.114:0/4031394842 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f317c05b5e0 con 0x7f31840ffad0 2024-02-16T22:26:07.477 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:26:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": { 2024-02-16T22:26:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:26:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:26:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": { 2024-02-16T22:26:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:26:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:26:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": { 2024-02-16T22:26:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4, 2024-02-16T22:26:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:26:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:26:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-02-16T22:26:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-02-16T22:26:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:26:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: "overall": { 2024-02-16T22:26:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 8, 2024-02-16T22:26:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T22:26:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-02-16T22:26:07.478 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:26:07.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.477+0000 7f318b283700 1 -- 172.21.15.114:0/4031394842 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f317006d8a0 msgr2=0x7f317006fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:07.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.477+0000 7f318b283700 1 --2- 172.21.15.114:0/4031394842 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f317006d8a0 0x7f317006fd60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3174005950 tx=0x7f31740058e0 comp rx=0 tx=0).stop 2024-02-16T22:26:07.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.477+0000 7f318b283700 1 -- 172.21.15.114:0/4031394842 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31840ffad0 msgr2=0x7f31841963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:07.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.477+0000 7f318b283700 1 --2- 172.21.15.114:0/4031394842 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31840ffad0 0x7f31841963b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f317c00bd70 tx=0x7f317c00be50 comp rx=0 tx=0).stop 2024-02-16T22:26:07.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.477+0000 7f318b283700 1 -- 172.21.15.114:0/4031394842 shutdown_connections 2024-02-16T22:26:07.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.477+0000 7f318b283700 1 --2- 172.21.15.114:0/4031394842 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f317006d8a0 0x7f317006fd60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:07.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.477+0000 7f318b283700 1 --2- 172.21.15.114:0/4031394842 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31840ffad0 0x7f31841963b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:07.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.477+0000 7f318b283700 1 --2- 172.21.15.114:0/4031394842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3184100cd0 0x7f31841968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:07.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.477+0000 7f318b283700 1 -- 172.21.15.114:0/4031394842 >> 172.21.15.114:0/4031394842 conn(0x7f31840fb050 msgr2=0x7f3184068df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:26:07.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.477+0000 7f318b283700 1 -- 172.21.15.114:0/4031394842 shutdown_connections 2024-02-16T22:26:07.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.477+0000 7f318b283700 1 -- 172.21.15.114:0/4031394842 wait complete. 2024-02-16T22:26:07.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.580+0000 7f6a534d0700 1 -- 172.21.15.114:0/2622572398 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a4c0ffac0 msgr2=0x7f6a4c0ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:07.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.580+0000 7f6a534d0700 1 --2- 172.21.15.114:0/2622572398 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a4c0ffac0 0x7f6a4c0ffee0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009910 tx=0x7f6a48009c20 comp rx=0 tx=0).stop 2024-02-16T22:26:07.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.580+0000 7f6a534d0700 1 -- 172.21.15.114:0/2622572398 shutdown_connections 2024-02-16T22:26:07.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.580+0000 7f6a534d0700 1 --2- 172.21.15.114:0/2622572398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a4c100cc0 0x7f6a4c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:07.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.580+0000 7f6a534d0700 1 --2- 172.21.15.114:0/2622572398 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a4c0ffac0 0x7f6a4c0ffee0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:07.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.580+0000 7f6a534d0700 1 -- 172.21.15.114:0/2622572398 >> 172.21.15.114:0/2622572398 conn(0x7f6a4c0fb060 msgr2=0x7f6a4c0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:26:07.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.580+0000 7f6a534d0700 1 -- 172.21.15.114:0/2622572398 shutdown_connections 2024-02-16T22:26:07.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.580+0000 7f6a534d0700 1 -- 172.21.15.114:0/2622572398 wait complete. 2024-02-16T22:26:07.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.581+0000 7f6a534d0700 1 Processor -- start 2024-02-16T22:26:07.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.581+0000 7f6a534d0700 1 -- start start 2024-02-16T22:26:07.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.581+0000 7f6a534d0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a4c0ffac0 0x7f6a4c196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:07.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.581+0000 7f6a534d0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a4c100cc0 0x7f6a4c196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:07.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.581+0000 7f6a534d0700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c196e90 con 0x7f6a4c100cc0 2024-02-16T22:26:07.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.581+0000 7f6a534d0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c196fd0 con 0x7f6a4c0ffac0 2024-02-16T22:26:07.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.581+0000 7f6a50a6b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a4c100cc0 0x7f6a4c196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:07.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.581+0000 7f6a50a6b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a4c100cc0 0x7f6a4c196870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59542/0 (socket says 172.21.15.114:59542) 2024-02-16T22:26:07.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.581+0000 7f6a50a6b700 1 -- 172.21.15.114:0/508865547 learned_addr learned my addr 172.21.15.114:0/508865547 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:26:07.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.581+0000 7f6a5126c700 1 --2- 172.21.15.114:0/508865547 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a4c0ffac0 0x7f6a4c196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:07.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.582+0000 7f6a50a6b700 1 -- 172.21.15.114:0/508865547 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a4c0ffac0 msgr2=0x7f6a4c196330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:07.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.582+0000 7f6a50a6b700 1 --2- 172.21.15.114:0/508865547 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a4c0ffac0 0x7f6a4c196330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:07.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.582+0000 7f6a50a6b700 1 -- 172.21.15.114:0/508865547 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a480095f0 con 0x7f6a4c100cc0 2024-02-16T22:26:07.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.582+0000 7f6a50a6b700 1 --2- 172.21.15.114:0/508865547 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a4c100cc0 0x7f6a4c196870 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00c8a0 tx=0x7f6a3c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:26:07.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.582+0000 7f6a427fc700 1 -- 172.21.15.114:0/508865547 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a3c0078e0 con 0x7f6a4c100cc0 2024-02-16T22:26:07.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.582+0000 7f6a427fc700 1 -- 172.21.15.114:0/508865547 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6a3c00f450 con 0x7f6a4c100cc0 2024-02-16T22:26:07.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.582+0000 7f6a534d0700 1 -- 172.21.15.114:0/508865547 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a4c19ba80 con 0x7f6a4c100cc0 2024-02-16T22:26:07.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.582+0000 7f6a427fc700 1 -- 172.21.15.114:0/508865547 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a3c00e5e0 con 0x7f6a4c100cc0 2024-02-16T22:26:07.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.582+0000 7f6a534d0700 1 -- 172.21.15.114:0/508865547 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a4c19bfd0 con 0x7f6a4c100cc0 2024-02-16T22:26:07.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.583+0000 7f6a534d0700 1 -- 172.21.15.114:0/508865547 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a4c108620 con 0x7f6a4c100cc0 2024-02-16T22:26:07.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.583+0000 7f6a427fc700 1 -- 172.21.15.114:0/508865547 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f6a3c00f5c0 con 0x7f6a4c100cc0 2024-02-16T22:26:07.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.584+0000 7f6a427fc700 1 --2- 172.21.15.114:0/508865547 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6a3806d8a0 0x7f6a3806fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:07.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.584+0000 7f6a427fc700 1 -- 172.21.15.114:0/508865547 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f6a3c08d530 con 0x7f6a4c100cc0 2024-02-16T22:26:07.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.584+0000 7f6a5126c700 1 --2- 172.21.15.114:0/508865547 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6a3806d8a0 0x7f6a3806fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:07.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.584+0000 7f6a5126c700 1 --2- 172.21.15.114:0/508865547 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6a3806d8a0 0x7f6a3806fd60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6a48006010 tx=0x7f6a48005280 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:26:07.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.588+0000 7f6a427fc700 1 -- 172.21.15.114:0/508865547 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6a3c05a820 con 0x7f6a4c100cc0 2024-02-16T22:26:07.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.826+0000 7f6a534d0700 1 -- 172.21.15.114:0/508865547 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6a4c1087e0 con 0x7f6a4c100cc0 2024-02-16T22:26:07.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.826+0000 7f6a427fc700 1 -- 172.21.15.114:0/508865547 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1846 (secure 0 0 0) 0x7f6a3c05a3b0 con 0x7f6a4c100cc0 2024-02-16T22:26:07.829 INFO:teuthology.orchestra.run.smithi114.stdout:e14 2024-02-16T22:26:07.829 INFO:teuthology.orchestra.run.smithi114.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:26:07.829 INFO:teuthology.orchestra.run.smithi114.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:26:07.829 INFO:teuthology.orchestra.run.smithi114.stdout:legacy client fscid: 1 2024-02-16T22:26:07.829 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:26:07.829 INFO:teuthology.orchestra.run.smithi114.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:26:07.829 INFO:teuthology.orchestra.run.smithi114.stdout:fs_name cephfs 2024-02-16T22:26:07.829 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 12 2024-02-16T22:26:07.829 INFO:teuthology.orchestra.run.smithi114.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:26:07.829 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-02-16T22:18:02.108825+0000 2024-02-16T22:26:07.829 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-02-16T22:18:16.413371+0000 2024-02-16T22:26:07.830 INFO:teuthology.orchestra.run.smithi114.stdout:tableserver 0 2024-02-16T22:26:07.830 INFO:teuthology.orchestra.run.smithi114.stdout:root 0 2024-02-16T22:26:07.830 INFO:teuthology.orchestra.run.smithi114.stdout:session_timeout 60 2024-02-16T22:26:07.830 INFO:teuthology.orchestra.run.smithi114.stdout:session_autoclose 300 2024-02-16T22:26:07.830 INFO:teuthology.orchestra.run.smithi114.stdout:max_file_size 1099511627776 2024-02-16T22:26:07.830 INFO:teuthology.orchestra.run.smithi114.stdout:max_xattr_size 65536 2024-02-16T22:26:07.830 INFO:teuthology.orchestra.run.smithi114.stdout:required_client_features {} 2024-02-16T22:26:07.830 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure 0 2024-02-16T22:26:07.830 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure_osd_epoch 0 2024-02-16T22:26:07.830 INFO:teuthology.orchestra.run.smithi114.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:26:07.830 INFO:teuthology.orchestra.run.smithi114.stdout:max_mds 1 2024-02-16T22:26:07.830 INFO:teuthology.orchestra.run.smithi114.stdout:in 0 2024-02-16T22:26:07.830 INFO:teuthology.orchestra.run.smithi114.stdout:up {0=14468} 2024-02-16T22:26:07.830 INFO:teuthology.orchestra.run.smithi114.stdout:failed 2024-02-16T22:26:07.831 INFO:teuthology.orchestra.run.smithi114.stdout:damaged 2024-02-16T22:26:07.831 INFO:teuthology.orchestra.run.smithi114.stdout:stopped 2024-02-16T22:26:07.831 INFO:teuthology.orchestra.run.smithi114.stdout:data_pools [3] 2024-02-16T22:26:07.831 INFO:teuthology.orchestra.run.smithi114.stdout:metadata_pool 2 2024-02-16T22:26:07.831 INFO:teuthology.orchestra.run.smithi114.stdout:inline_data enabled 2024-02-16T22:26:07.831 INFO:teuthology.orchestra.run.smithi114.stdout:balancer 2024-02-16T22:26:07.831 INFO:teuthology.orchestra.run.smithi114.stdout:bal_rank_mask -1 2024-02-16T22:26:07.831 INFO:teuthology.orchestra.run.smithi114.stdout:standby_count_wanted 1 2024-02-16T22:26:07.831 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.zzryke{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.114:6826/277953535,v1:172.21.15.114:6827/277953535] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:26:07.831 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:26:07.831 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:26:07.831 INFO:teuthology.orchestra.run.smithi114.stdout:Standby daemons: 2024-02-16T22:26:07.831 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:26:07.831 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.fdxpey{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:26:07.832 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.vhlqbe{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:26:07.832 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.gopcfa{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:26:07.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.829+0000 7f6a534d0700 1 -- 172.21.15.114:0/508865547 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6a3806d8a0 msgr2=0x7f6a3806fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:07.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.829+0000 7f6a534d0700 1 --2- 172.21.15.114:0/508865547 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6a3806d8a0 0x7f6a3806fd60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6a48006010 tx=0x7f6a48005280 comp rx=0 tx=0).stop 2024-02-16T22:26:07.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.829+0000 7f6a534d0700 1 -- 172.21.15.114:0/508865547 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a4c100cc0 msgr2=0x7f6a4c196870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:07.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.829+0000 7f6a534d0700 1 --2- 172.21.15.114:0/508865547 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a4c100cc0 0x7f6a4c196870 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00c8a0 tx=0x7f6a3c00cbb0 comp rx=0 tx=0).stop 2024-02-16T22:26:07.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.829+0000 7f6a534d0700 1 -- 172.21.15.114:0/508865547 shutdown_connections 2024-02-16T22:26:07.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.829+0000 7f6a534d0700 1 --2- 172.21.15.114:0/508865547 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6a3806d8a0 0x7f6a3806fd60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:07.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.829+0000 7f6a534d0700 1 --2- 172.21.15.114:0/508865547 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a4c0ffac0 0x7f6a4c196330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:07.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.829+0000 7f6a534d0700 1 --2- 172.21.15.114:0/508865547 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a4c100cc0 0x7f6a4c196870 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:07.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.829+0000 7f6a534d0700 1 -- 172.21.15.114:0/508865547 >> 172.21.15.114:0/508865547 conn(0x7f6a4c0fb060 msgr2=0x7f6a4c068db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:26:07.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.829+0000 7f6a534d0700 1 -- 172.21.15.114:0/508865547 shutdown_connections 2024-02-16T22:26:07.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.829+0000 7f6a534d0700 1 -- 172.21.15.114:0/508865547 wait complete. 2024-02-16T22:26:07.833 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 14 2024-02-16T22:26:07.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.937+0000 7f62907c6700 1 -- 172.21.15.114:0/1195675154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6288100cb0 msgr2=0x7f6288101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:07.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.937+0000 7f62907c6700 1 --2- 172.21.15.114:0/1195675154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6288100cb0 0x7f6288101110 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6278009910 tx=0x7f6278009c20 comp rx=0 tx=0).stop 2024-02-16T22:26:07.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.937+0000 7f62907c6700 1 -- 172.21.15.114:0/1195675154 shutdown_connections 2024-02-16T22:26:07.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.937+0000 7f62907c6700 1 --2- 172.21.15.114:0/1195675154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6288100cb0 0x7f6288101110 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:07.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.937+0000 7f62907c6700 1 --2- 172.21.15.114:0/1195675154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62880ffab0 0x7f62880ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:07.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.937+0000 7f62907c6700 1 -- 172.21.15.114:0/1195675154 >> 172.21.15.114:0/1195675154 conn(0x7f62880fb030 msgr2=0x7f62880fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:26:07.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.937+0000 7f62907c6700 1 -- 172.21.15.114:0/1195675154 shutdown_connections 2024-02-16T22:26:07.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.937+0000 7f62907c6700 1 -- 172.21.15.114:0/1195675154 wait complete. 2024-02-16T22:26:07.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.937+0000 7f62907c6700 1 Processor -- start 2024-02-16T22:26:07.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.937+0000 7f62907c6700 1 -- start start 2024-02-16T22:26:07.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.938+0000 7f62907c6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62880ffab0 0x7f6288196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:07.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.938+0000 7f62907c6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6288100cb0 0x7f6288196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:07.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.938+0000 7f62907c6700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6288196ea0 con 0x7f6288100cb0 2024-02-16T22:26:07.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.938+0000 7f62907c6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6288196fe0 con 0x7f62880ffab0 2024-02-16T22:26:07.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.938+0000 7f628dd61700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6288100cb0 0x7f6288196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:07.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.938+0000 7f628dd61700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6288100cb0 0x7f6288196880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59554/0 (socket says 172.21.15.114:59554) 2024-02-16T22:26:07.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.938+0000 7f628dd61700 1 -- 172.21.15.114:0/4100790401 learned_addr learned my addr 172.21.15.114:0/4100790401 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:26:07.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.938+0000 7f628e562700 1 --2- 172.21.15.114:0/4100790401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62880ffab0 0x7f6288196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:07.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.938+0000 7f628dd61700 1 -- 172.21.15.114:0/4100790401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62880ffab0 msgr2=0x7f6288196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:07.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.938+0000 7f628dd61700 1 --2- 172.21.15.114:0/4100790401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62880ffab0 0x7f6288196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:07.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.938+0000 7f628dd61700 1 -- 172.21.15.114:0/4100790401 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62780095f0 con 0x7f6288100cb0 2024-02-16T22:26:07.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.938+0000 7f628dd61700 1 --2- 172.21.15.114:0/4100790401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6288100cb0 0x7f6288196880 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6278006010 tx=0x7f627800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:26:07.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.938+0000 7f627f7fe700 1 -- 172.21.15.114:0/4100790401 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f627801d070 con 0x7f6288100cb0 2024-02-16T22:26:07.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.938+0000 7f627f7fe700 1 -- 172.21.15.114:0/4100790401 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6278022470 con 0x7f6288100cb0 2024-02-16T22:26:07.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.938+0000 7f62907c6700 1 -- 172.21.15.114:0/4100790401 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f628819ba30 con 0x7f6288100cb0 2024-02-16T22:26:07.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.938+0000 7f627f7fe700 1 -- 172.21.15.114:0/4100790401 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f627800f670 con 0x7f6288100cb0 2024-02-16T22:26:07.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.938+0000 7f62907c6700 1 -- 172.21.15.114:0/4100790401 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f628819bfa0 con 0x7f6288100cb0 2024-02-16T22:26:07.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.939+0000 7f62907c6700 1 -- 172.21.15.114:0/4100790401 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6288108610 con 0x7f6288100cb0 2024-02-16T22:26:07.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.940+0000 7f627f7fe700 1 -- 172.21.15.114:0/4100790401 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f6278004af0 con 0x7f6288100cb0 2024-02-16T22:26:07.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.940+0000 7f627f7fe700 1 --2- 172.21.15.114:0/4100790401 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f627406d8a0 0x7f627406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:07.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.940+0000 7f627f7fe700 1 -- 172.21.15.114:0/4100790401 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f627808ed10 con 0x7f6288100cb0 2024-02-16T22:26:07.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.940+0000 7f628e562700 1 --2- 172.21.15.114:0/4100790401 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f627406d8a0 0x7f627406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:07.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.941+0000 7f628e562700 1 --2- 172.21.15.114:0/4100790401 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f627406d8a0 0x7f627406fd60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6284005fd0 tx=0x7f6284005dd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:26:07.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:07.945+0000 7f627f7fe700 1 -- 172.21.15.114:0/4100790401 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f627805c2f0 con 0x7f6288100cb0 2024-02-16T22:26:08.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.156+0000 7f62907c6700 1 -- 172.21.15.114:0/4100790401 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f628819c6a0 con 0x7f627406d8a0 2024-02-16T22:26:08.158 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:07 smithi114 ceph-mon[94452]: from='client.34254 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:26:08.158 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:07 smithi114 ceph-mon[94452]: from='client.34258 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:26:08.158 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:07 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/4031394842' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:26:08.158 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:07 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/508865547' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:26:08.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.157+0000 7f627f7fe700 1 -- 172.21.15.114:0/4100790401 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f628819c6a0 con 0x7f627406d8a0 2024-02-16T22:26:08.159 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:26:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:26:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "in_progress": true, 2024-02-16T22:26:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:26:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "services_complete": [ 2024-02-16T22:26:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "crash", 2024-02-16T22:26:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "mon", 2024-02-16T22:26:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr" 2024-02-16T22:26:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-02-16T22:26:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "progress": "8/23 daemons upgraded", 2024-02-16T22:26:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "message": "Currently upgrading osd daemons", 2024-02-16T22:26:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "is_paused": false 2024-02-16T22:26:08.160 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:26:08.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.159+0000 7f62907c6700 1 -- 172.21.15.114:0/4100790401 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f627406d8a0 msgr2=0x7f627406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:08.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.159+0000 7f62907c6700 1 --2- 172.21.15.114:0/4100790401 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f627406d8a0 0x7f627406fd60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6284005fd0 tx=0x7f6284005dd0 comp rx=0 tx=0).stop 2024-02-16T22:26:08.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.159+0000 7f62907c6700 1 -- 172.21.15.114:0/4100790401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6288100cb0 msgr2=0x7f6288196880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:08.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.159+0000 7f62907c6700 1 --2- 172.21.15.114:0/4100790401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6288100cb0 0x7f6288196880 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6278006010 tx=0x7f627800be20 comp rx=0 tx=0).stop 2024-02-16T22:26:08.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.159+0000 7f62907c6700 1 -- 172.21.15.114:0/4100790401 shutdown_connections 2024-02-16T22:26:08.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.159+0000 7f62907c6700 1 --2- 172.21.15.114:0/4100790401 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f627406d8a0 0x7f627406fd60 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:08.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.159+0000 7f62907c6700 1 --2- 172.21.15.114:0/4100790401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62880ffab0 0x7f6288196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:08.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.159+0000 7f62907c6700 1 --2- 172.21.15.114:0/4100790401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6288100cb0 0x7f6288196880 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:08.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.159+0000 7f62907c6700 1 -- 172.21.15.114:0/4100790401 >> 172.21.15.114:0/4100790401 conn(0x7f62880fb030 msgr2=0x7f6288068410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:26:08.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.160+0000 7f62907c6700 1 -- 172.21.15.114:0/4100790401 shutdown_connections 2024-02-16T22:26:08.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.160+0000 7f62907c6700 1 -- 172.21.15.114:0/4100790401 wait complete. 2024-02-16T22:26:08.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.263+0000 7ff8ae925700 1 -- 172.21.15.114:0/1137478807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8a8101da0 msgr2=0x7ff8a8104230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:08.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.263+0000 7ff8ae925700 1 --2- 172.21.15.114:0/1137478807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8a8101da0 0x7ff8a8104230 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009910 tx=0x7ff89c009c20 comp rx=0 tx=0).stop 2024-02-16T22:26:08.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.263+0000 7ff8ae925700 1 -- 172.21.15.114:0/1137478807 shutdown_connections 2024-02-16T22:26:08.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.263+0000 7ff8ae925700 1 --2- 172.21.15.114:0/1137478807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8a8101da0 0x7ff8a8104230 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:08.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.263+0000 7ff8ae925700 1 --2- 172.21.15.114:0/1137478807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8a80ff430 0x7ff8a8101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:08.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.263+0000 7ff8ae925700 1 -- 172.21.15.114:0/1137478807 >> 172.21.15.114:0/1137478807 conn(0x7ff8a80fb030 msgr2=0x7ff8a80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:26:08.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.264+0000 7ff8ae925700 1 -- 172.21.15.114:0/1137478807 shutdown_connections 2024-02-16T22:26:08.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.264+0000 7ff8ae925700 1 -- 172.21.15.114:0/1137478807 wait complete. 2024-02-16T22:26:08.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.264+0000 7ff8ae925700 1 Processor -- start 2024-02-16T22:26:08.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.264+0000 7ff8ae925700 1 -- start start 2024-02-16T22:26:08.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.264+0000 7ff8ae925700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8a80ff430 0x7ff8a819c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:08.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.264+0000 7ff8ae925700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8a8101da0 0x7ff8a819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:08.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.264+0000 7ff8ae925700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a819d490 con 0x7ff8a8101da0 2024-02-16T22:26:08.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.264+0000 7ff8ae925700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a819d5d0 con 0x7ff8a80ff430 2024-02-16T22:26:08.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.265+0000 7ff8a77fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8a8101da0 0x7ff8a819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:08.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.265+0000 7ff8a77fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8a8101da0 0x7ff8a819ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59566/0 (socket says 172.21.15.114:59566) 2024-02-16T22:26:08.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.265+0000 7ff8a77fe700 1 -- 172.21.15.114:0/304080409 learned_addr learned my addr 172.21.15.114:0/304080409 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:26:08.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.265+0000 7ff8a7fff700 1 --2- 172.21.15.114:0/304080409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8a80ff430 0x7ff8a819c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:08.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.265+0000 7ff8a77fe700 1 -- 172.21.15.114:0/304080409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8a80ff430 msgr2=0x7ff8a819c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:08.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.265+0000 7ff8a77fe700 1 --2- 172.21.15.114:0/304080409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8a80ff430 0x7ff8a819c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:08.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.265+0000 7ff8a77fe700 1 -- 172.21.15.114:0/304080409 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff89c0095f0 con 0x7ff8a8101da0 2024-02-16T22:26:08.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.265+0000 7ff8a77fe700 1 --2- 172.21.15.114:0/304080409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8a8101da0 0x7ff8a819ce70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00b5c0 tx=0x7ff89c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:26:08.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.265+0000 7ff8a57fa700 1 -- 172.21.15.114:0/304080409 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff89c01d070 con 0x7ff8a8101da0 2024-02-16T22:26:08.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.265+0000 7ff8a57fa700 1 -- 172.21.15.114:0/304080409 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff89c00f460 con 0x7ff8a8101da0 2024-02-16T22:26:08.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.265+0000 7ff8ae925700 1 -- 172.21.15.114:0/304080409 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8a81a2020 con 0x7ff8a8101da0 2024-02-16T22:26:08.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.265+0000 7ff8a57fa700 1 -- 172.21.15.114:0/304080409 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff89c021640 con 0x7ff8a8101da0 2024-02-16T22:26:08.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.265+0000 7ff8ae925700 1 -- 172.21.15.114:0/304080409 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a81a2510 con 0x7ff8a8101da0 2024-02-16T22:26:08.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.266+0000 7ff8ae925700 1 -- 172.21.15.114:0/304080409 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8a8196b10 con 0x7ff8a8101da0 2024-02-16T22:26:08.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.267+0000 7ff8a57fa700 1 -- 172.21.15.114:0/304080409 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7ff89c00fa60 con 0x7ff8a8101da0 2024-02-16T22:26:08.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.267+0000 7ff8a57fa700 1 --2- 172.21.15.114:0/304080409 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff89006d8a0 0x7ff89006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:08.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.267+0000 7ff8a57fa700 1 -- 172.21.15.114:0/304080409 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff89c08e6e0 con 0x7ff8a8101da0 2024-02-16T22:26:08.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.267+0000 7ff8a7fff700 1 --2- 172.21.15.114:0/304080409 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff89006d8a0 0x7ff89006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:08.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.267+0000 7ff8a7fff700 1 --2- 172.21.15.114:0/304080409 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff89006d8a0 0x7ff89006fd60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff898005950 tx=0x7ff89800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:26:08.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.272+0000 7ff8a57fa700 1 -- 172.21.15.114:0/304080409 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff89c05baf0 con 0x7ff8a8101da0 2024-02-16T22:26:08.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:07 smithi123 ceph-mon[82331]: from='client.34254 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:26:08.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:07 smithi123 ceph-mon[82331]: from='client.34258 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:26:08.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:07 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/4031394842' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:26:08.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:07 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/508865547' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:26:08.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.547+0000 7ff8ae925700 1 -- 172.21.15.114:0/304080409 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff8a802cc70 con 0x7ff8a8101da0 2024-02-16T22:26:08.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.547+0000 7ff8a57fa700 1 -- 172.21.15.114:0/304080409 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7ff89c05b680 con 0x7ff8a8101da0 2024-02-16T22:26:08.550 INFO:teuthology.orchestra.run.smithi114.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:26:08.550 INFO:teuthology.orchestra.run.smithi114.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:26:08.550 INFO:teuthology.orchestra.run.smithi114.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:26:08.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.549+0000 7ff8ae925700 1 -- 172.21.15.114:0/304080409 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff89006d8a0 msgr2=0x7ff89006fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:08.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.549+0000 7ff8ae925700 1 --2- 172.21.15.114:0/304080409 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff89006d8a0 0x7ff89006fd60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff898005950 tx=0x7ff89800b410 comp rx=0 tx=0).stop 2024-02-16T22:26:08.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.549+0000 7ff8ae925700 1 -- 172.21.15.114:0/304080409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8a8101da0 msgr2=0x7ff8a819ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:08.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.549+0000 7ff8ae925700 1 --2- 172.21.15.114:0/304080409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8a8101da0 0x7ff8a819ce70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00b5c0 tx=0x7ff89c00bd70 comp rx=0 tx=0).stop 2024-02-16T22:26:08.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.550+0000 7ff8ae925700 1 -- 172.21.15.114:0/304080409 shutdown_connections 2024-02-16T22:26:08.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.550+0000 7ff8ae925700 1 --2- 172.21.15.114:0/304080409 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff89006d8a0 0x7ff89006fd60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:08.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.550+0000 7ff8ae925700 1 --2- 172.21.15.114:0/304080409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8a80ff430 0x7ff8a819c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:08.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.550+0000 7ff8ae925700 1 --2- 172.21.15.114:0/304080409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8a8101da0 0x7ff8a819ce70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:08.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.550+0000 7ff8ae925700 1 -- 172.21.15.114:0/304080409 >> 172.21.15.114:0/304080409 conn(0x7ff8a80fb030 msgr2=0x7ff8a80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:26:08.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.550+0000 7ff8ae925700 1 -- 172.21.15.114:0/304080409 shutdown_connections 2024-02-16T22:26:08.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:08.550+0000 7ff8ae925700 1 -- 172.21.15.114:0/304080409 wait complete. 2024-02-16T22:26:09.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:08 smithi114 ceph-mon[94452]: pgmap v84: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:09.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:08 smithi114 ceph-mon[94452]: from='client.34270 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:26:09.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:08 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/304080409' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:26:09.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:08 smithi123 ceph-mon[82331]: pgmap v84: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:09.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:08 smithi123 ceph-mon[82331]: from='client.34270 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:26:09.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:08 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/304080409' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:26:11.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:10 smithi114 ceph-mon[94452]: pgmap v85: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:11.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:10 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:11.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:10 smithi123 ceph-mon[82331]: pgmap v85: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:11.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:10 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:13.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:12 smithi114 ceph-mon[94452]: pgmap v86: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:13.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:12 smithi123 ceph-mon[82331]: pgmap v86: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:15.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:14 smithi114 ceph-mon[94452]: pgmap v87: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:15.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:14 smithi123 ceph-mon[82331]: pgmap v87: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:17.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:16 smithi114 ceph-mon[94452]: pgmap v88: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:17.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:16 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:17.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:16 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:26:17.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:16 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T22:26:17.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:16 smithi114 ceph-mon[94452]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T22:26:17.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:16 smithi114 ceph-mon[94452]: Upgrade: osd.2 is safe to restart 2024-02-16T22:26:17.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:16 smithi123 ceph-mon[82331]: pgmap v88: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:17.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:16 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:17.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:16 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:26:17.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:16 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T22:26:17.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:16 smithi123 ceph-mon[82331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T22:26:17.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:16 smithi123 ceph-mon[82331]: Upgrade: osd.2 is safe to restart 2024-02-16T22:26:18.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:18 smithi114 ceph-mon[94452]: Upgrade: Updating osd.2 2024-02-16T22:26:18.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:18.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-16T22:26:18.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:26:18.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:18 smithi114 ceph-mon[94452]: Deploying daemon osd.2 on smithi114 2024-02-16T22:26:18.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:18 smithi123 ceph-mon[82331]: Upgrade: Updating osd.2 2024-02-16T22:26:18.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:18.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-16T22:26:18.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:26:18.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:18 smithi123 ceph-mon[82331]: Deploying daemon osd.2 on smithi114 2024-02-16T22:26:19.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:19 smithi123 ceph-mon[82331]: pgmap v89: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:19.879 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:19 smithi114 ceph-mon[94452]: pgmap v89: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:21.202 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:20 smithi114 systemd[1]: Stopping Ceph osd.2 for 1dcbaabe-cd18-11ee-95ba-87774f69a715... 2024-02-16T22:26:21.202 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:21 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2[63791]: 2024-02-16T22:26:21.103+0000 7fed1168b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T22:26:21.202 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:21 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2[63791]: 2024-02-16T22:26:21.103+0000 7fed1168b700 -1 osd.2 58 *** Got signal Terminated *** 2024-02-16T22:26:21.202 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:21 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2[63791]: 2024-02-16T22:26:21.103+0000 7fed1168b700 -1 osd.2 58 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T22:26:21.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:21 smithi123 ceph-mon[82331]: pgmap v90: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:21.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:21 smithi123 ceph-mon[82331]: osd.2 marked itself down and dead 2024-02-16T22:26:21.880 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:21 smithi114 ceph-mon[94452]: pgmap v90: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:21.881 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:21 smithi114 ceph-mon[94452]: osd.2 marked itself down and dead 2024-02-16T22:26:22.181 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:21 smithi114 podman[114124]: 2024-02-16 22:26:21.879554747 +0000 UTC m=+0.873287788 container died 6608718d081804dcd0493709f71ce7b9a8abbaaccdd3224dac1d37835a9ce5f5 (image=quay.io/ceph/ceph@sha256:6176526d5ef4047f148d47c951653dcc6b0de49091fda006d3a088e7a1f404d0, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=-18.2.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, RELEASE=HEAD) 2024-02-16T22:26:22.451 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:22 smithi114 podman[114124]: 2024-02-16 22:26:22.27886477 +0000 UTC m=+1.272597801 container cleanup 6608718d081804dcd0493709f71ce7b9a8abbaaccdd3224dac1d37835a9ce5f5 (image=quay.io/ceph/ceph@sha256:6176526d5ef4047f148d47c951653dcc6b0de49091fda006d3a088e7a1f404d0, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.29.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD, org.label-schema.build-date=20240214, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.1, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-02-16T22:26:22.452 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:22 smithi114 bash[114124]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2 2024-02-16T22:26:22.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:22 smithi123 ceph-mon[82331]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T22:26:22.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:22 smithi123 ceph-mon[82331]: osdmap e59: 6 total, 5 up, 6 in 2024-02-16T22:26:22.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:22 smithi123 ceph-mon[82331]: osdmap e60: 6 total, 5 up, 6 in 2024-02-16T22:26:22.815 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:22 smithi114 podman[114143]: 2024-02-16 22:26:22.513574517 +0000 UTC m=+0.634429886 container remove 6608718d081804dcd0493709f71ce7b9a8abbaaccdd3224dac1d37835a9ce5f5 (image=quay.io/ceph/ceph@sha256:6176526d5ef4047f148d47c951653dcc6b0de49091fda006d3a088e7a1f404d0, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, RELEASE=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-02-16T22:26:22.815 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:22 smithi114 ceph-mon[94452]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T22:26:22.815 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:22 smithi114 ceph-mon[94452]: osdmap e59: 6 total, 5 up, 6 in 2024-02-16T22:26:22.815 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:22 smithi114 ceph-mon[94452]: osdmap e60: 6 total, 5 up, 6 in 2024-02-16T22:26:23.121 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:22 smithi114 podman[114232]: 2024-02-16 22:26:22.814243282 +0000 UTC m=+0.114510110 container create 0d031560a30cc4eadec293a3d2bf27c12577907ee0a3ded35daee3d0f9dcf21b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-02-16T22:26:23.121 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:22 smithi114 podman[114232]: 2024-02-16 22:26:22.718106863 +0000 UTC m=+0.018373689 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T22:26:23.121 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:23 smithi114 podman[114232]: 2024-02-16 22:26:23.019165942 +0000 UTC m=+0.319432768 container init 0d031560a30cc4eadec293a3d2bf27c12577907ee0a3ded35daee3d0f9dcf21b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS) 2024-02-16T22:26:23.121 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:23 smithi114 podman[114232]: 2024-02-16 22:26:23.022190728 +0000 UTC m=+0.322457558 container start 0d031560a30cc4eadec293a3d2bf27c12577907ee0a3ded35daee3d0f9dcf21b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T22:26:23.121 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:23 smithi114 podman[114232]: 2024-02-16 22:26:23.119112707 +0000 UTC m=+0.419379545 container attach 0d031560a30cc4eadec293a3d2bf27c12577907ee0a3ded35daee3d0f9dcf21b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2-deactivate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-02-16T22:26:23.451 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:23 smithi114 podman[114232]: 2024-02-16 22:26:23.320621992 +0000 UTC m=+0.620888808 container died 0d031560a30cc4eadec293a3d2bf27c12577907ee0a3ded35daee3d0f9dcf21b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD) 2024-02-16T22:26:23.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:23 smithi123 ceph-mon[82331]: pgmap v92: 65 pgs: 12 peering, 7 stale+active+clean, 46 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:23.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:23 smithi123 ceph-mon[82331]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-02-16T22:26:23.895 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:23 smithi114 ceph-mon[94452]: pgmap v92: 65 pgs: 12 peering, 7 stale+active+clean, 46 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:23.895 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:23 smithi114 ceph-mon[94452]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-02-16T22:26:24.155 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:24 smithi114 podman[114257]: 2024-02-16 22:26:24.139895923 +0000 UTC m=+0.812464760 container remove 0d031560a30cc4eadec293a3d2bf27c12577907ee0a3ded35daee3d0f9dcf21b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T22:26:24.452 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:24 smithi114 systemd[1]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@osd.2.service: Deactivated successfully. 2024-02-16T22:26:24.452 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:24 smithi114 systemd[1]: Stopped Ceph osd.2 for 1dcbaabe-cd18-11ee-95ba-87774f69a715. 2024-02-16T22:26:24.452 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:24 smithi114 systemd[1]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@osd.2.service: Consumed 47.336s CPU time. 2024-02-16T22:26:24.452 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:24 smithi114 systemd[1]: Starting Ceph osd.2 for 1dcbaabe-cd18-11ee-95ba-87774f69a715... 2024-02-16T22:26:24.731 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:24 smithi114 podman[114364]: 2024-02-16 22:26:24.491838232 +0000 UTC m=+0.018895110 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T22:26:24.731 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:24 smithi114 podman[114364]: 2024-02-16 22:26:24.607921642 +0000 UTC m=+0.134978515 container create 8c80594a2507477fd21111029619afa5f2a2a78ed1914410bcc5138c40ae88cb (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2-activate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-02-16T22:26:25.201 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:24 smithi114 podman[114364]: 2024-02-16 22:26:24.8197511 +0000 UTC m=+0.346807974 container init 8c80594a2507477fd21111029619afa5f2a2a78ed1914410bcc5138c40ae88cb (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2-activate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-02-16T22:26:25.202 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:24 smithi114 podman[114364]: 2024-02-16 22:26:24.82288274 +0000 UTC m=+0.349939615 container start 8c80594a2507477fd21111029619afa5f2a2a78ed1914410bcc5138c40ae88cb (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214) 2024-02-16T22:26:25.202 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:24 smithi114 podman[114364]: 2024-02-16 22:26:24.918927763 +0000 UTC m=+0.445984637 container attach 8c80594a2507477fd21111029619afa5f2a2a78ed1914410bcc5138c40ae88cb (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2-activate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5) 2024-02-16T22:26:25.536 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:25 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2-activate[114380]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-16T22:26:25.536 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:25 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2-activate[114380]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-02-16T22:26:25.536 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:25 smithi114 bash[114364]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-16T22:26:25.536 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:25 smithi114 bash[114364]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-02-16T22:26:25.536 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:25 smithi114 bash[114364]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-02-16T22:26:25.537 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:25 smithi114 bash[114364]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-02-16T22:26:25.537 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:25 smithi114 bash[114364]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-02-16T22:26:25.537 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:25 smithi114 bash[114364]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-16T22:26:25.537 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:25 smithi114 bash[114364]: --> ceph-volume raw activate successful for osd ID: 2 2024-02-16T22:26:25.537 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:25 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2-activate[114380]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-02-16T22:26:25.537 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:25 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2-activate[114380]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-02-16T22:26:25.537 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:25 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2-activate[114380]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-02-16T22:26:25.537 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:25 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2-activate[114380]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-16T22:26:25.537 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:25 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2-activate[114380]: --> ceph-volume raw activate successful for osd ID: 2 2024-02-16T22:26:25.537 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:25 smithi114 podman[114364]: 2024-02-16 22:26:25.287369788 +0000 UTC m=+0.814426662 container died 8c80594a2507477fd21111029619afa5f2a2a78ed1914410bcc5138c40ae88cb (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2-activate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0) 2024-02-16T22:26:25.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:25 smithi123 ceph-mon[82331]: pgmap v94: 65 pgs: 19 peering, 3 stale+active+clean, 43 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:25.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:25 smithi114 ceph-mon[94452]: pgmap v94: 65 pgs: 19 peering, 3 stale+active+clean, 43 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:26.314 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:25 smithi114 podman[114515]: 2024-02-16 22:26:25.964488747 +0000 UTC m=+0.671767004 container remove 8c80594a2507477fd21111029619afa5f2a2a78ed1914410bcc5138c40ae88cb (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2-activate, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-02-16T22:26:26.314 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:26 smithi114 podman[114581]: 2024-02-16 22:26:26.083054379 +0000 UTC m=+0.017255972 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T22:26:26.314 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:26 smithi114 podman[114581]: 2024-02-16 22:26:26.20715389 +0000 UTC m=+0.141355483 container create c663dfa0518bdf5185229b25afe3ed746bacd0994e5af9ff5caf921541aa4609 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-02-16T22:26:26.612 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:26 smithi114 podman[114581]: 2024-02-16 22:26:26.401751833 +0000 UTC m=+0.335953431 container init c663dfa0518bdf5185229b25afe3ed746bacd0994e5af9ff5caf921541aa4609 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5) 2024-02-16T22:26:26.612 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:26 smithi114 podman[114581]: 2024-02-16 22:26:26.404992934 +0000 UTC m=+0.339194528 container start c663dfa0518bdf5185229b25afe3ed746bacd0994e5af9ff5caf921541aa4609 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T22:26:26.612 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:26 smithi114 ceph-osd[114600]: -- 172.21.15.114:0/2758286106 <== mon.0 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d715eb71e0 con 0x55d715e88c00 2024-02-16T22:26:26.950 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:26 smithi114 bash[114581]: c663dfa0518bdf5185229b25afe3ed746bacd0994e5af9ff5caf921541aa4609 2024-02-16T22:26:26.950 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:26 smithi114 systemd[1]: Started Ceph osd.2 for 1dcbaabe-cd18-11ee-95ba-87774f69a715. 2024-02-16T22:26:27.201 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:26 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2[114596]: 2024-02-16T22:26:26.948+0000 7fb1ccab57c0 -1 Falling back to public interface 2024-02-16T22:26:27.757 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:27 smithi114 ceph-mon[94452]: pgmap v95: 65 pgs: 19 peering, 3 stale+active+clean, 43 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:27.757 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:27 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:27.757 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:27 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:27.758 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:27 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:26:27.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:27 smithi123 ceph-mon[82331]: pgmap v95: 65 pgs: 19 peering, 3 stale+active+clean, 43 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:27.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:27 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:27.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:27 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:27.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:27 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:26:28.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:28 smithi114 ceph-mon[94452]: pgmap v96: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-02-16T22:26:28.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:28 smithi114 ceph-mon[94452]: Health check failed: Degraded data redundancy: 35/267 objects degraded (13.109%), 12 pgs degraded (PG_DEGRADED) 2024-02-16T22:26:28.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:28 smithi114 ceph-mon[94452]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 6 pgs peering) 2024-02-16T22:26:29.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:28 smithi123 ceph-mon[82331]: pgmap v96: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-02-16T22:26:29.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:28 smithi123 ceph-mon[82331]: Health check failed: Degraded data redundancy: 35/267 objects degraded (13.109%), 12 pgs degraded (PG_DEGRADED) 2024-02-16T22:26:29.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:28 smithi123 ceph-mon[82331]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 6 pgs peering) 2024-02-16T22:26:31.106 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:30 smithi114 ceph-mon[94452]: pgmap v97: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-02-16T22:26:31.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:30 smithi123 ceph-mon[82331]: pgmap v97: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-02-16T22:26:31.385 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:31 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2[114596]: 2024-02-16T22:26:31.104+0000 7fb1ccab57c0 -1 osd.2 0 read_superblock omap replica is missing. 2024-02-16T22:26:31.663 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:31 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2[114596]: 2024-02-16T22:26:31.383+0000 7fb1ccab57c0 -1 osd.2 58 log_to_monitors true 2024-02-16T22:26:31.951 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:26:31 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2[114596]: 2024-02-16T22:26:31.857+0000 7fb1bdf0d700 -1 osd.2 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T22:26:31.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:31 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:31.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:31 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:26:31.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:31 smithi114 ceph-mon[94452]: from='osd.2 [v2:172.21.15.114:6818/3638870673,v1:172.21.15.114:6819/3638870673]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-16T22:26:32.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:31 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:32.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:31 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:26:32.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:31 smithi123 ceph-mon[82331]: from='osd.2 [v2:172.21.15.114:6818/3638870673,v1:172.21.15.114:6819/3638870673]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-16T22:26:33.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:32 smithi114 ceph-mon[94452]: pgmap v98: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-02-16T22:26:33.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:32 smithi114 ceph-mon[94452]: from='osd.2 [v2:172.21.15.114:6818/3638870673,v1:172.21.15.114:6819/3638870673]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-16T22:26:33.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:32 smithi114 ceph-mon[94452]: osdmap e61: 6 total, 5 up, 6 in 2024-02-16T22:26:33.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:32 smithi114 ceph-mon[94452]: from='osd.2 [v2:172.21.15.114:6818/3638870673,v1:172.21.15.114:6819/3638870673]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-02-16T22:26:33.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:32 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:33.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:32 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:33.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:32 smithi123 ceph-mon[82331]: pgmap v98: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-02-16T22:26:33.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:32 smithi123 ceph-mon[82331]: from='osd.2 [v2:172.21.15.114:6818/3638870673,v1:172.21.15.114:6819/3638870673]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-16T22:26:33.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:32 smithi123 ceph-mon[82331]: osdmap e61: 6 total, 5 up, 6 in 2024-02-16T22:26:33.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:32 smithi123 ceph-mon[82331]: from='osd.2 [v2:172.21.15.114:6818/3638870673,v1:172.21.15.114:6819/3638870673]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-02-16T22:26:33.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:32 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:33.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:32 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:34.099 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:33 smithi114 ceph-mon[94452]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T22:26:34.100 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:33 smithi114 ceph-mon[94452]: osd.2 [v2:172.21.15.114:6818/3638870673,v1:172.21.15.114:6819/3638870673] boot 2024-02-16T22:26:34.100 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:33 smithi114 ceph-mon[94452]: osdmap e62: 6 total, 6 up, 6 in 2024-02-16T22:26:34.100 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:33 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:26:34.100 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:33 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:34.100 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:33 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:34.100 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:33 smithi114 ceph-mon[94452]: osdmap e63: 6 total, 6 up, 6 in 2024-02-16T22:26:34.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:33 smithi123 ceph-mon[82331]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T22:26:34.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:33 smithi123 ceph-mon[82331]: osd.2 [v2:172.21.15.114:6818/3638870673,v1:172.21.15.114:6819/3638870673] boot 2024-02-16T22:26:34.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:33 smithi123 ceph-mon[82331]: osdmap e62: 6 total, 6 up, 6 in 2024-02-16T22:26:34.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:33 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T22:26:34.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:33 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:34.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:33 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:34.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:33 smithi123 ceph-mon[82331]: osdmap e63: 6 total, 6 up, 6 in 2024-02-16T22:26:35.372 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:35 smithi114 ceph-mon[94452]: pgmap v101: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-02-16T22:26:35.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:35 smithi123 ceph-mon[82331]: pgmap v101: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-02-16T22:26:37.279 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:37 smithi114 ceph-mon[94452]: pgmap v103: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-02-16T22:26:37.279 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:37 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:37.279 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:37 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:37.279 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:37 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:26:37.279 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:37 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:26:37.279 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:37 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:37.280 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:37 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:26:37.280 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:37 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:26:37.280 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:37 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:26:37.280 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:37 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:26:37.280 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:37 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T22:26:37.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:37 smithi123 ceph-mon[82331]: pgmap v103: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-02-16T22:26:37.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:37 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:37.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:37 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:37.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:37 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:26:37.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:37 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:26:37.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:37 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:37.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:37 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:26:37.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:37 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:26:37.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:37 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:26:37.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:37 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:26:37.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:37 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T22:26:38.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:38 smithi114 ceph-mon[94452]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T22:26:38.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:38 smithi114 ceph-mon[94452]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-02-16T22:26:38.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:38 smithi123 ceph-mon[82331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T22:26:38.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:38 smithi123 ceph-mon[82331]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-02-16T22:26:38.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.669+0000 7f110064e700 1 -- 172.21.15.114:0/3453412849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10f8100bc0 msgr2=0x7f10f8100fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:38.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.669+0000 7f110064e700 1 --2- 172.21.15.114:0/3453412849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10f8100bc0 0x7f10f8100fe0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f10f4009910 tx=0x7f10f4009c20 comp rx=0 tx=0).stop 2024-02-16T22:26:38.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.669+0000 7f110064e700 1 -- 172.21.15.114:0/3453412849 shutdown_connections 2024-02-16T22:26:38.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.669+0000 7f110064e700 1 --2- 172.21.15.114:0/3453412849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10f8101dc0 0x7f10f8102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:38.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.669+0000 7f110064e700 1 --2- 172.21.15.114:0/3453412849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10f8100bc0 0x7f10f8100fe0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:38.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.669+0000 7f110064e700 1 -- 172.21.15.114:0/3453412849 >> 172.21.15.114:0/3453412849 conn(0x7f10f80fc140 msgr2=0x7f10f80fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:26:38.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.669+0000 7f110064e700 1 -- 172.21.15.114:0/3453412849 shutdown_connections 2024-02-16T22:26:38.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.669+0000 7f110064e700 1 -- 172.21.15.114:0/3453412849 wait complete. 2024-02-16T22:26:38.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.669+0000 7f110064e700 1 Processor -- start 2024-02-16T22:26:38.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.669+0000 7f110064e700 1 -- start start 2024-02-16T22:26:38.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.670+0000 7f110064e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10f8100bc0 0x7f10f8196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:38.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.670+0000 7f110064e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10f8101dc0 0x7f10f81968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:38.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.670+0000 7f110064e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f8196ef0 con 0x7f10f8100bc0 2024-02-16T22:26:38.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.670+0000 7f110064e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f8197030 con 0x7f10f8101dc0 2024-02-16T22:26:38.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.670+0000 7f10fe3ea700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10f8100bc0 0x7f10f8196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:38.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.670+0000 7f10fe3ea700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10f8100bc0 0x7f10f8196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33860/0 (socket says 172.21.15.114:33860) 2024-02-16T22:26:38.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.670+0000 7f10fe3ea700 1 -- 172.21.15.114:0/2521782893 learned_addr learned my addr 172.21.15.114:0/2521782893 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:26:38.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.670+0000 7f10fe3ea700 1 -- 172.21.15.114:0/2521782893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10f8101dc0 msgr2=0x7f10f81968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:38.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.670+0000 7f10fdbe9700 1 --2- 172.21.15.114:0/2521782893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10f8101dc0 0x7f10f81968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:38.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.670+0000 7f10fe3ea700 1 --2- 172.21.15.114:0/2521782893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10f8101dc0 0x7f10f81968d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:38.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.670+0000 7f10fe3ea700 1 -- 172.21.15.114:0/2521782893 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f40095f0 con 0x7f10f8100bc0 2024-02-16T22:26:38.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.670+0000 7f10fe3ea700 1 --2- 172.21.15.114:0/2521782893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10f8100bc0 0x7f10f8196390 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f10f400bda0 tx=0x7f10f400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:26:38.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.670+0000 7f10ef7fe700 1 -- 172.21.15.114:0/2521782893 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10f401d070 con 0x7f10f8100bc0 2024-02-16T22:26:38.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.671+0000 7f10ef7fe700 1 -- 172.21.15.114:0/2521782893 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f10f4004c50 con 0x7f10f8100bc0 2024-02-16T22:26:38.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.671+0000 7f110064e700 1 -- 172.21.15.114:0/2521782893 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f819ba80 con 0x7f10f8100bc0 2024-02-16T22:26:38.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.671+0000 7f10ef7fe700 1 -- 172.21.15.114:0/2521782893 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10f400f690 con 0x7f10f8100bc0 2024-02-16T22:26:38.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.671+0000 7f110064e700 1 -- 172.21.15.114:0/2521782893 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f819bf70 con 0x7f10f8100bc0 2024-02-16T22:26:38.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.672+0000 7f110064e700 1 -- 172.21.15.114:0/2521782893 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f8109720 con 0x7f10f8100bc0 2024-02-16T22:26:38.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.672+0000 7f10ef7fe700 1 -- 172.21.15.114:0/2521782893 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f10f40229b0 con 0x7f10f8100bc0 2024-02-16T22:26:38.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.672+0000 7f10ef7fe700 1 --2- 172.21.15.114:0/2521782893 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f10e406d8a0 0x7f10e406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:38.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.673+0000 7f10ef7fe700 1 -- 172.21.15.114:0/2521782893 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f10f408ec60 con 0x7f10f8100bc0 2024-02-16T22:26:38.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.673+0000 7f10fdbe9700 1 --2- 172.21.15.114:0/2521782893 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f10e406d8a0 0x7f10e406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:38.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.673+0000 7f10fdbe9700 1 --2- 172.21.15.114:0/2521782893 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f10e406d8a0 0x7f10e406fd60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f10e8005950 tx=0x7f10e8005cf0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:26:38.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.677+0000 7f10ef7fe700 1 -- 172.21.15.114:0/2521782893 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f10f405c070 con 0x7f10f8100bc0 2024-02-16T22:26:38.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.889+0000 7f110064e700 1 -- 172.21.15.114:0/2521782893 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f10f8106710 con 0x7f10e406d8a0 2024-02-16T22:26:38.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.891+0000 7f10ef7fe700 1 -- 172.21.15.114:0/2521782893 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f10f8106710 con 0x7f10e406d8a0 2024-02-16T22:26:38.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.893+0000 7f110064e700 1 -- 172.21.15.114:0/2521782893 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f10e406d8a0 msgr2=0x7f10e406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:38.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.893+0000 7f110064e700 1 --2- 172.21.15.114:0/2521782893 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f10e406d8a0 0x7f10e406fd60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f10e8005950 tx=0x7f10e8005cf0 comp rx=0 tx=0).stop 2024-02-16T22:26:38.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.893+0000 7f110064e700 1 -- 172.21.15.114:0/2521782893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10f8100bc0 msgr2=0x7f10f8196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:38.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.893+0000 7f110064e700 1 --2- 172.21.15.114:0/2521782893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10f8100bc0 0x7f10f8196390 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f10f400bda0 tx=0x7f10f400be80 comp rx=0 tx=0).stop 2024-02-16T22:26:38.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.893+0000 7f110064e700 1 -- 172.21.15.114:0/2521782893 shutdown_connections 2024-02-16T22:26:38.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.893+0000 7f110064e700 1 --2- 172.21.15.114:0/2521782893 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f10e406d8a0 0x7f10e406fd60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:38.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.893+0000 7f110064e700 1 --2- 172.21.15.114:0/2521782893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10f8100bc0 0x7f10f8196390 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:38.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.893+0000 7f110064e700 1 --2- 172.21.15.114:0/2521782893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10f8101dc0 0x7f10f81968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:38.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.893+0000 7f110064e700 1 -- 172.21.15.114:0/2521782893 >> 172.21.15.114:0/2521782893 conn(0x7f10f80fc140 msgr2=0x7f10f8104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:26:38.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.893+0000 7f110064e700 1 -- 172.21.15.114:0/2521782893 shutdown_connections 2024-02-16T22:26:38.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.893+0000 7f110064e700 1 -- 172.21.15.114:0/2521782893 wait complete. 2024-02-16T22:26:38.905 INFO:teuthology.orchestra.run.smithi114.stdout:true 2024-02-16T22:26:39.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.999+0000 7f92cd3ed700 1 -- 172.21.15.114:0/3309638503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92c80ff4b0 msgr2=0x7f92c80ff8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:39.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.999+0000 7f92cd3ed700 1 --2- 172.21.15.114:0/3309638503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92c80ff4b0 0x7f92c80ff8d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f92b8009910 tx=0x7f92b8009c20 comp rx=0 tx=0).stop 2024-02-16T22:26:39.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.999+0000 7f92cd3ed700 1 -- 172.21.15.114:0/3309638503 shutdown_connections 2024-02-16T22:26:39.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.999+0000 7f92cd3ed700 1 --2- 172.21.15.114:0/3309638503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92c80ffe10 0x7f92c8100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:39.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.999+0000 7f92cd3ed700 1 --2- 172.21.15.114:0/3309638503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92c80ff4b0 0x7f92c80ff8d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:39.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.999+0000 7f92cd3ed700 1 -- 172.21.15.114:0/3309638503 >> 172.21.15.114:0/3309638503 conn(0x7f92c80fb030 msgr2=0x7f92c80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:26:39.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.999+0000 7f92cd3ed700 1 -- 172.21.15.114:0/3309638503 shutdown_connections 2024-02-16T22:26:39.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.999+0000 7f92cd3ed700 1 -- 172.21.15.114:0/3309638503 wait complete. 2024-02-16T22:26:39.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.999+0000 7f92cd3ed700 1 Processor -- start 2024-02-16T22:26:39.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:38.999+0000 7f92cd3ed700 1 -- start start 2024-02-16T22:26:39.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.000+0000 7f92cd3ed700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92c80ff4b0 0x7f92c8198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:39.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.000+0000 7f92cd3ed700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92c80ffe10 0x7f92c8198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:39.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.000+0000 7f92cd3ed700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c8198fe0 con 0x7f92c80ff4b0 2024-02-16T22:26:39.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.000+0000 7f92cd3ed700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c8199120 con 0x7f92c80ffe10 2024-02-16T22:26:39.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.000+0000 7f92c6ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92c80ff4b0 0x7f92c8198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:39.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.000+0000 7f92c6ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92c80ff4b0 0x7f92c8198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33882/0 (socket says 172.21.15.114:33882) 2024-02-16T22:26:39.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.000+0000 7f92c6ffd700 1 -- 172.21.15.114:0/4074188368 learned_addr learned my addr 172.21.15.114:0/4074188368 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:26:39.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.000+0000 7f92c6ffd700 1 -- 172.21.15.114:0/4074188368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92c80ffe10 msgr2=0x7f92c8198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:39.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.000+0000 7f92c6ffd700 1 --2- 172.21.15.114:0/4074188368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92c80ffe10 0x7f92c8198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:39.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.000+0000 7f92c6ffd700 1 -- 172.21.15.114:0/4074188368 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b80095f0 con 0x7f92c80ff4b0 2024-02-16T22:26:39.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.000+0000 7f92c6ffd700 1 --2- 172.21.15.114:0/4074188368 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92c80ff4b0 0x7f92c8198510 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f92b800bd70 tx=0x7f92b800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:26:39.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.000+0000 7f92affff700 1 -- 172.21.15.114:0/4074188368 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92b801d070 con 0x7f92c80ff4b0 2024-02-16T22:26:39.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.000+0000 7f92affff700 1 -- 172.21.15.114:0/4074188368 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f92b8022470 con 0x7f92c80ff4b0 2024-02-16T22:26:39.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.000+0000 7f92cd3ed700 1 -- 172.21.15.114:0/4074188368 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c819db80 con 0x7f92c80ff4b0 2024-02-16T22:26:39.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.000+0000 7f92affff700 1 -- 172.21.15.114:0/4074188368 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92b800f670 con 0x7f92c80ff4b0 2024-02-16T22:26:39.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.000+0000 7f92cd3ed700 1 -- 172.21.15.114:0/4074188368 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c819e040 con 0x7f92c80ff4b0 2024-02-16T22:26:39.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.001+0000 7f92cd3ed700 1 -- 172.21.15.114:0/4074188368 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c810b910 con 0x7f92c80ff4b0 2024-02-16T22:26:39.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.002+0000 7f92affff700 1 -- 172.21.15.114:0/4074188368 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f92b8022a50 con 0x7f92c80ff4b0 2024-02-16T22:26:39.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.002+0000 7f92affff700 1 --2- 172.21.15.114:0/4074188368 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f92b006d8a0 0x7f92b006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:39.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.002+0000 7f92affff700 1 -- 172.21.15.114:0/4074188368 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f92b808eb60 con 0x7f92c80ff4b0 2024-02-16T22:26:39.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.002+0000 7f92c67fc700 1 --2- 172.21.15.114:0/4074188368 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f92b006d8a0 0x7f92b006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:39.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.003+0000 7f92c67fc700 1 --2- 172.21.15.114:0/4074188368 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f92b006d8a0 0x7f92b006fd60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f92bc005fd0 tx=0x7f92bc005d00 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:26:39.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.007+0000 7f92affff700 1 -- 172.21.15.114:0/4074188368 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f92b805c0c0 con 0x7f92c80ff4b0 2024-02-16T22:26:39.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.220+0000 7f92cd3ed700 1 -- 172.21.15.114:0/4074188368 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f92c8109200 con 0x7f92b006d8a0 2024-02-16T22:26:39.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.221+0000 7f92affff700 1 -- 172.21.15.114:0/4074188368 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f92c8109200 con 0x7f92b006d8a0 2024-02-16T22:26:39.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.223+0000 7f92cd3ed700 1 -- 172.21.15.114:0/4074188368 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f92b006d8a0 msgr2=0x7f92b006fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:39.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.223+0000 7f92cd3ed700 1 --2- 172.21.15.114:0/4074188368 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f92b006d8a0 0x7f92b006fd60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f92bc005fd0 tx=0x7f92bc005d00 comp rx=0 tx=0).stop 2024-02-16T22:26:39.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.223+0000 7f92cd3ed700 1 -- 172.21.15.114:0/4074188368 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92c80ff4b0 msgr2=0x7f92c8198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:39.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.223+0000 7f92cd3ed700 1 --2- 172.21.15.114:0/4074188368 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92c80ff4b0 0x7f92c8198510 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f92b800bd70 tx=0x7f92b800be50 comp rx=0 tx=0).stop 2024-02-16T22:26:39.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.223+0000 7f92cd3ed700 1 -- 172.21.15.114:0/4074188368 shutdown_connections 2024-02-16T22:26:39.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.223+0000 7f92cd3ed700 1 --2- 172.21.15.114:0/4074188368 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f92b006d8a0 0x7f92b006fd60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:39.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.224+0000 7f92cd3ed700 1 --2- 172.21.15.114:0/4074188368 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92c80ff4b0 0x7f92c8198510 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:39.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.224+0000 7f92cd3ed700 1 --2- 172.21.15.114:0/4074188368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92c80ffe10 0x7f92c8198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:39.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.224+0000 7f92cd3ed700 1 -- 172.21.15.114:0/4074188368 >> 172.21.15.114:0/4074188368 conn(0x7f92c80fb030 msgr2=0x7f92c8107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:26:39.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.224+0000 7f92cd3ed700 1 -- 172.21.15.114:0/4074188368 shutdown_connections 2024-02-16T22:26:39.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.224+0000 7f92cd3ed700 1 -- 172.21.15.114:0/4074188368 wait complete. 2024-02-16T22:26:39.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.328+0000 7f43d28ea700 1 -- 172.21.15.114:0/352708090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43cc0ffd90 msgr2=0x7f43cc1001f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:39.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.328+0000 7f43d28ea700 1 --2- 172.21.15.114:0/352708090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43cc0ffd90 0x7f43cc1001f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f43c0009910 tx=0x7f43c0009c20 comp rx=0 tx=0).stop 2024-02-16T22:26:39.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.328+0000 7f43d28ea700 1 -- 172.21.15.114:0/352708090 shutdown_connections 2024-02-16T22:26:39.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.328+0000 7f43d28ea700 1 --2- 172.21.15.114:0/352708090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43cc0ffd90 0x7f43cc1001f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:39.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.328+0000 7f43d28ea700 1 --2- 172.21.15.114:0/352708090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43cc101050 0x7f43cc101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:39.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.328+0000 7f43d28ea700 1 -- 172.21.15.114:0/352708090 >> 172.21.15.114:0/352708090 conn(0x7f43cc0fa7c0 msgr2=0x7f43cc0fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:26:39.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.328+0000 7f43d28ea700 1 -- 172.21.15.114:0/352708090 shutdown_connections 2024-02-16T22:26:39.331 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:39 smithi114 ceph-mon[94452]: pgmap v104: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:39.331 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:39 smithi114 ceph-mon[94452]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 35/267 objects degraded (13.109%), 12 pgs degraded) 2024-02-16T22:26:39.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.328+0000 7f43d28ea700 1 -- 172.21.15.114:0/352708090 wait complete. 2024-02-16T22:26:39.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.328+0000 7f43d28ea700 1 Processor -- start 2024-02-16T22:26:39.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.328+0000 7f43d28ea700 1 -- start start 2024-02-16T22:26:39.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.329+0000 7f43d28ea700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43cc0ffd90 0x7f43cc196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:39.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.329+0000 7f43d28ea700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43cc101050 0x7f43cc196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:39.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.329+0000 7f43d28ea700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc196fb0 con 0x7f43cc0ffd90 2024-02-16T22:26:39.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.329+0000 7f43d28ea700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc1970f0 con 0x7f43cc101050 2024-02-16T22:26:39.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.329+0000 7f43cbfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43cc0ffd90 0x7f43cc196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:39.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.329+0000 7f43cbfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43cc0ffd90 0x7f43cc196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33904/0 (socket says 172.21.15.114:33904) 2024-02-16T22:26:39.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.329+0000 7f43cbfff700 1 -- 172.21.15.114:0/3021016157 learned_addr learned my addr 172.21.15.114:0/3021016157 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:26:39.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.329+0000 7f43cb7fe700 1 --2- 172.21.15.114:0/3021016157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43cc101050 0x7f43cc196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:39.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.329+0000 7f43cbfff700 1 -- 172.21.15.114:0/3021016157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43cc101050 msgr2=0x7f43cc196990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:39.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.329+0000 7f43cbfff700 1 --2- 172.21.15.114:0/3021016157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43cc101050 0x7f43cc196990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:39.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.329+0000 7f43cbfff700 1 -- 172.21.15.114:0/3021016157 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c00095f0 con 0x7f43cc0ffd90 2024-02-16T22:26:39.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.329+0000 7f43cbfff700 1 --2- 172.21.15.114:0/3021016157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43cc0ffd90 0x7f43cc196450 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00b480 tx=0x7f43bc00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:26:39.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.329+0000 7f43c97fa700 1 -- 172.21.15.114:0/3021016157 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43bc011800 con 0x7f43cc0ffd90 2024-02-16T22:26:39.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.329+0000 7f43c97fa700 1 -- 172.21.15.114:0/3021016157 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f43bc011e10 con 0x7f43cc0ffd90 2024-02-16T22:26:39.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.329+0000 7f43d28ea700 1 -- 172.21.15.114:0/3021016157 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43cc19bba0 con 0x7f43cc0ffd90 2024-02-16T22:26:39.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.329+0000 7f43c97fa700 1 -- 172.21.15.114:0/3021016157 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43bc010ea0 con 0x7f43cc0ffd90 2024-02-16T22:26:39.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.330+0000 7f43d28ea700 1 -- 172.21.15.114:0/3021016157 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43cc19c0f0 con 0x7f43cc0ffd90 2024-02-16T22:26:39.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.331+0000 7f43d28ea700 1 -- 172.21.15.114:0/3021016157 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43cc1086d0 con 0x7f43cc0ffd90 2024-02-16T22:26:39.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.331+0000 7f43c97fa700 1 -- 172.21.15.114:0/3021016157 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f43bc010380 con 0x7f43cc0ffd90 2024-02-16T22:26:39.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.332+0000 7f43c97fa700 1 --2- 172.21.15.114:0/3021016157 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f43b406d8a0 0x7f43b406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:39.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.332+0000 7f43c97fa700 1 -- 172.21.15.114:0/3021016157 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f43bc08cff0 con 0x7f43cc0ffd90 2024-02-16T22:26:39.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.332+0000 7f43cb7fe700 1 --2- 172.21.15.114:0/3021016157 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f43b406d8a0 0x7f43b406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:39.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.332+0000 7f43cb7fe700 1 --2- 172.21.15.114:0/3021016157 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f43b406d8a0 0x7f43b406fd60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f43c0006010 tx=0x7f43c000b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:26:39.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.336+0000 7f43c97fa700 1 -- 172.21.15.114:0/3021016157 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f43bc05a380 con 0x7f43cc0ffd90 2024-02-16T22:26:39.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.538+0000 7f43d28ea700 1 -- 172.21.15.114:0/3021016157 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f43cc0ff390 con 0x7f43b406d8a0 2024-02-16T22:26:39.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.549+0000 7f43c97fa700 1 -- 172.21.15.114:0/3021016157 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f43cc0ff390 con 0x7f43b406d8a0 2024-02-16T22:26:39.551 INFO:teuthology.orchestra.run.smithi114.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:26:39.551 INFO:teuthology.orchestra.run.smithi114.stdout:alertmanager.smithi114 smithi114 *:9093,9094 running (12m) 7s ago 14m 24.0M - 0.25.0 c8568f914cd2 e7404fea400e 2024-02-16T22:26:39.551 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi114 smithi114 running (14m) 7s ago 14m 8506k - 18.2.1 94eadd990e0a 7fd07162cd96 2024-02-16T22:26:39.551 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi123 smithi123 running (13m) 101s ago 13m 7696k - 18.2.1 94eadd990e0a 7687f5e99621 2024-02-16T22:26:39.551 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi114 smithi114 running (110s) 7s ago 14m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 9b8ae9efe83b 2024-02-16T22:26:39.551 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi123 smithi123 running (105s) 101s ago 13m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 6d97263a0ffa 2024-02-16T22:26:39.551 INFO:teuthology.orchestra.run.smithi114.stdout:grafana.smithi114 smithi114 *:3000 running (12m) 7s ago 13m 88.4M - 9.4.7 954c08fa6188 db1c26c3bc20 2024-02-16T22:26:39.551 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.fdxpey smithi114 running (8m) 7s ago 8m 18.5M - 18.2.1 94eadd990e0a 704b40c144fb 2024-02-16T22:26:39.551 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.zzryke smithi114 running (8m) 7s ago 8m 157M - 18.2.1 94eadd990e0a 1f795d23c9a4 2024-02-16T22:26:39.551 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.gopcfa smithi123 running (8m) 101s ago 8m 17.3M - 18.2.1 94eadd990e0a 166134b7532a 2024-02-16T22:26:39.551 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.vhlqbe smithi123 running (8m) 101s ago 8m 18.8M - 18.2.1 94eadd990e0a 243c8bb96fa8 2024-02-16T22:26:39.551 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi114.vcullz smithi114 *:8443,9283,8765 running (4m) 7s ago 15m 500M - 19.0.0-1450-gf78a58c0 f081f847070a 14ebeb7888c0 2024-02-16T22:26:39.552 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi123.inzgjd smithi123 *:8443,9283,8765 running (3m) 101s ago 13m 449M - 19.0.0-1450-gf78a58c0 f081f847070a f10aab9faf9e 2024-02-16T22:26:39.552 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi114 smithi114 running (3m) 7s ago 15m 51.5M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 58ac06372523 2024-02-16T22:26:39.552 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi123 smithi123 running (2m) 101s ago 12m 34.0M 2048M 19.0.0-1450-gf78a58c0 f081f847070a f76485e0ea24 2024-02-16T22:26:39.552 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi114 smithi114 *:9100 running (14m) 7s ago 14m 15.9M - 1.5.0 0da6a335fe13 744a090e06d0 2024-02-16T22:26:39.552 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi123 smithi123 *:9100 running (13m) 101s ago 13m 16.0M - 1.5.0 0da6a335fe13 3994b8c29b2e 2024-02-16T22:26:39.552 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 smithi114 running (84s) 7s ago 12m 91.2M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 1978fdad94a5 2024-02-16T22:26:39.552 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 smithi114 running (49s) 7s ago 11m 90.2M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 3f90e792898f 2024-02-16T22:26:39.552 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 smithi114 running (13s) 7s ago 11m 12.3M 1216M 19.0.0-1450-gf78a58c0 f081f847070a c663dfa0518b 2024-02-16T22:26:39.552 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 smithi123 running (10m) 101s ago 10m 259M 2240M 18.2.1 94eadd990e0a 575db379de3d 2024-02-16T22:26:39.552 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 smithi123 running (10m) 101s ago 10m 220M 2240M 18.2.1 94eadd990e0a 7e266866cd9c 2024-02-16T22:26:39.552 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 smithi123 running (9m) 101s ago 9m 199M 2240M 18.2.1 94eadd990e0a 24e5bef19fda 2024-02-16T22:26:39.552 INFO:teuthology.orchestra.run.smithi114.stdout:prometheus.smithi114 smithi114 *:9095 running (3m) 7s ago 13m 57.8M - 2.43.0 a07b618ecd1d 37aac985cdd7 2024-02-16T22:26:39.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:39 smithi123 ceph-mon[82331]: pgmap v104: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:39.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:39 smithi123 ceph-mon[82331]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 35/267 objects degraded (13.109%), 12 pgs degraded) 2024-02-16T22:26:39.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.551+0000 7f43d28ea700 1 -- 172.21.15.114:0/3021016157 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f43b406d8a0 msgr2=0x7f43b406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:39.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.551+0000 7f43d28ea700 1 --2- 172.21.15.114:0/3021016157 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f43b406d8a0 0x7f43b406fd60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f43c0006010 tx=0x7f43c000b540 comp rx=0 tx=0).stop 2024-02-16T22:26:39.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.551+0000 7f43d28ea700 1 -- 172.21.15.114:0/3021016157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43cc0ffd90 msgr2=0x7f43cc196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:39.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.551+0000 7f43d28ea700 1 --2- 172.21.15.114:0/3021016157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43cc0ffd90 0x7f43cc196450 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00b480 tx=0x7f43bc00b790 comp rx=0 tx=0).stop 2024-02-16T22:26:39.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.551+0000 7f43d28ea700 1 -- 172.21.15.114:0/3021016157 shutdown_connections 2024-02-16T22:26:39.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.551+0000 7f43d28ea700 1 --2- 172.21.15.114:0/3021016157 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f43b406d8a0 0x7f43b406fd60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:39.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.551+0000 7f43d28ea700 1 --2- 172.21.15.114:0/3021016157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43cc0ffd90 0x7f43cc196450 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:39.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.551+0000 7f43d28ea700 1 --2- 172.21.15.114:0/3021016157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43cc101050 0x7f43cc196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:39.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.551+0000 7f43d28ea700 1 -- 172.21.15.114:0/3021016157 >> 172.21.15.114:0/3021016157 conn(0x7f43cc0fa7c0 msgr2=0x7f43cc0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:26:39.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.551+0000 7f43d28ea700 1 -- 172.21.15.114:0/3021016157 shutdown_connections 2024-02-16T22:26:39.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.551+0000 7f43d28ea700 1 -- 172.21.15.114:0/3021016157 wait complete. 2024-02-16T22:26:39.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.653+0000 7fe48daf6700 1 -- 172.21.15.114:0/3491082182 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4880fe940 msgr2=0x7fe4880fed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:39.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.653+0000 7fe48daf6700 1 --2- 172.21.15.114:0/3491082182 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4880fe940 0x7fe4880fed60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe47800b0d0 tx=0x7fe47800b3e0 comp rx=0 tx=0).stop 2024-02-16T22:26:39.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.653+0000 7fe48daf6700 1 -- 172.21.15.114:0/3491082182 shutdown_connections 2024-02-16T22:26:39.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.653+0000 7fe48daf6700 1 --2- 172.21.15.114:0/3491082182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4880ff2a0 0x7fe4881062b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:39.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.653+0000 7fe48daf6700 1 --2- 172.21.15.114:0/3491082182 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4880fe940 0x7fe4880fed60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:39.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.653+0000 7fe48daf6700 1 -- 172.21.15.114:0/3491082182 >> 172.21.15.114:0/3491082182 conn(0x7fe4880fa380 msgr2=0x7fe4880fc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:26:39.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.653+0000 7fe48daf6700 1 -- 172.21.15.114:0/3491082182 shutdown_connections 2024-02-16T22:26:39.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.653+0000 7fe48daf6700 1 -- 172.21.15.114:0/3491082182 wait complete. 2024-02-16T22:26:39.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.653+0000 7fe48daf6700 1 Processor -- start 2024-02-16T22:26:39.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.653+0000 7fe48daf6700 1 -- start start 2024-02-16T22:26:39.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.654+0000 7fe48daf6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4880fe940 0x7fe4881963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:39.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.654+0000 7fe48daf6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4880ff2a0 0x7fe488196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:39.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.654+0000 7fe48daf6700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe488196f30 con 0x7fe4880ff2a0 2024-02-16T22:26:39.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.654+0000 7fe48daf6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe488197070 con 0x7fe4880fe940 2024-02-16T22:26:39.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.654+0000 7fe486ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4880ff2a0 0x7fe488196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.654+0000 7fe486ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4880ff2a0 0x7fe488196910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33924/0 (socket says 172.21.15.114:33924) 2024-02-16T22:26:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.654+0000 7fe486ffd700 1 -- 172.21.15.114:0/181861907 learned_addr learned my addr 172.21.15.114:0/181861907 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:26:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.654+0000 7fe4877fe700 1 --2- 172.21.15.114:0/181861907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4880fe940 0x7fe4881963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.654+0000 7fe486ffd700 1 -- 172.21.15.114:0/181861907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4880fe940 msgr2=0x7fe4881963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.654+0000 7fe486ffd700 1 --2- 172.21.15.114:0/181861907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4880fe940 0x7fe4881963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.654+0000 7fe486ffd700 1 -- 172.21.15.114:0/181861907 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe478009d00 con 0x7fe4880ff2a0 2024-02-16T22:26:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.654+0000 7fe486ffd700 1 --2- 172.21.15.114:0/181861907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4880ff2a0 0x7fe488196910 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00c840 tx=0x7fe47c00cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:26:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.654+0000 7fe484ff9700 1 -- 172.21.15.114:0/181861907 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe47c0041f0 con 0x7fe4880ff2a0 2024-02-16T22:26:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.654+0000 7fe484ff9700 1 -- 172.21.15.114:0/181861907 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe47c004ce0 con 0x7fe4880ff2a0 2024-02-16T22:26:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.654+0000 7fe48daf6700 1 -- 172.21.15.114:0/181861907 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe48819bb20 con 0x7fe4880ff2a0 2024-02-16T22:26:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.655+0000 7fe484ff9700 1 -- 172.21.15.114:0/181861907 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe47c0076d0 con 0x7fe4880ff2a0 2024-02-16T22:26:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.655+0000 7fe48daf6700 1 -- 172.21.15.114:0/181861907 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe48819c070 con 0x7fe4880ff2a0 2024-02-16T22:26:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.656+0000 7fe48daf6700 1 -- 172.21.15.114:0/181861907 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe488190590 con 0x7fe4880ff2a0 2024-02-16T22:26:39.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.656+0000 7fe484ff9700 1 -- 172.21.15.114:0/181861907 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fe47c004350 con 0x7fe4880ff2a0 2024-02-16T22:26:39.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.657+0000 7fe484ff9700 1 --2- 172.21.15.114:0/181861907 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe47006d8a0 0x7fe47006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:39.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.657+0000 7fe484ff9700 1 -- 172.21.15.114:0/181861907 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe47c08cf70 con 0x7fe4880ff2a0 2024-02-16T22:26:39.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.657+0000 7fe4877fe700 1 --2- 172.21.15.114:0/181861907 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe47006d8a0 0x7fe47006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:39.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.657+0000 7fe4877fe700 1 --2- 172.21.15.114:0/181861907 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe47006d8a0 0x7fe47006fd60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe478015040 tx=0x7fe478006000 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:26:39.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.661+0000 7fe484ff9700 1 -- 172.21.15.114:0/181861907 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe47c0c0d20 con 0x7fe4880ff2a0 2024-02-16T22:26:39.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.939+0000 7fe48daf6700 1 -- 172.21.15.114:0/181861907 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe48802cc70 con 0x7fe4880ff2a0 2024-02-16T22:26:39.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.940+0000 7fe484ff9700 1 -- 172.21.15.114:0/181861907 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7fe47c01d070 con 0x7fe4880ff2a0 2024-02-16T22:26:39.943 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:26:39.943 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": { 2024-02-16T22:26:39.943 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:26:39.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:26:39.943 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": { 2024-02-16T22:26:39.943 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:26:39.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:26:39.943 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": { 2024-02-16T22:26:39.943 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 3, 2024-02-16T22:26:39.944 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-16T22:26:39.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:26:39.944 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-02-16T22:26:39.944 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-02-16T22:26:39.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:26:39.944 INFO:teuthology.orchestra.run.smithi114.stdout: "overall": { 2024-02-16T22:26:39.944 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 7, 2024-02-16T22:26:39.944 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 7 2024-02-16T22:26:39.944 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-02-16T22:26:39.944 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:26:39.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.942+0000 7fe48daf6700 1 -- 172.21.15.114:0/181861907 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe47006d8a0 msgr2=0x7fe47006fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:39.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.942+0000 7fe48daf6700 1 --2- 172.21.15.114:0/181861907 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe47006d8a0 0x7fe47006fd60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe478015040 tx=0x7fe478006000 comp rx=0 tx=0).stop 2024-02-16T22:26:39.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.942+0000 7fe48daf6700 1 -- 172.21.15.114:0/181861907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4880ff2a0 msgr2=0x7fe488196910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:39.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.942+0000 7fe48daf6700 1 --2- 172.21.15.114:0/181861907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4880ff2a0 0x7fe488196910 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00c840 tx=0x7fe47c00cb50 comp rx=0 tx=0).stop 2024-02-16T22:26:39.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.942+0000 7fe48daf6700 1 -- 172.21.15.114:0/181861907 shutdown_connections 2024-02-16T22:26:39.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.942+0000 7fe48daf6700 1 --2- 172.21.15.114:0/181861907 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe47006d8a0 0x7fe47006fd60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:39.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.942+0000 7fe48daf6700 1 --2- 172.21.15.114:0/181861907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4880fe940 0x7fe4881963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:39.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.942+0000 7fe48daf6700 1 --2- 172.21.15.114:0/181861907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe4880ff2a0 0x7fe488196910 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:39.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.942+0000 7fe48daf6700 1 -- 172.21.15.114:0/181861907 >> 172.21.15.114:0/181861907 conn(0x7fe4880fa380 msgr2=0x7fe488104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:26:39.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.942+0000 7fe48daf6700 1 -- 172.21.15.114:0/181861907 shutdown_connections 2024-02-16T22:26:39.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:39.942+0000 7fe48daf6700 1 -- 172.21.15.114:0/181861907 wait complete. 2024-02-16T22:26:40.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.045+0000 7f246161b700 1 -- 172.21.15.114:0/4131796800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f245c1016d0 msgr2=0x7f245c103ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:40.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.045+0000 7f246161b700 1 --2- 172.21.15.114:0/4131796800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f245c1016d0 0x7f245c103ac0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f244c009910 tx=0x7f244c009c20 comp rx=0 tx=0).stop 2024-02-16T22:26:40.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.045+0000 7f246161b700 1 -- 172.21.15.114:0/4131796800 shutdown_connections 2024-02-16T22:26:40.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.045+0000 7f246161b700 1 --2- 172.21.15.114:0/4131796800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f245c104000 0x7f245c1063f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:40.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.045+0000 7f246161b700 1 --2- 172.21.15.114:0/4131796800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f245c1016d0 0x7f245c103ac0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:40.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.045+0000 7f246161b700 1 -- 172.21.15.114:0/4131796800 >> 172.21.15.114:0/4131796800 conn(0x7f245c0fb060 msgr2=0x7f245c0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:26:40.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.045+0000 7f246161b700 1 -- 172.21.15.114:0/4131796800 shutdown_connections 2024-02-16T22:26:40.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.045+0000 7f246161b700 1 -- 172.21.15.114:0/4131796800 wait complete. 2024-02-16T22:26:40.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.045+0000 7f246161b700 1 Processor -- start 2024-02-16T22:26:40.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.045+0000 7f246161b700 1 -- start start 2024-02-16T22:26:40.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.046+0000 7f246161b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f245c1016d0 0x7f245c101030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:40.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.046+0000 7f246161b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f245c104000 0x7f245c0ff680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:40.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.046+0000 7f246161b700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245c0ffbc0 con 0x7f245c1016d0 2024-02-16T22:26:40.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.046+0000 7f246161b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245c0ffd00 con 0x7f245c104000 2024-02-16T22:26:40.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.046+0000 7f245affd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f245c1016d0 0x7f245c101030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:40.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.046+0000 7f245affd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f245c1016d0 0x7f245c101030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33932/0 (socket says 172.21.15.114:33932) 2024-02-16T22:26:40.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.046+0000 7f245affd700 1 -- 172.21.15.114:0/2761633868 learned_addr learned my addr 172.21.15.114:0/2761633868 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:26:40.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.046+0000 7f24527fc700 1 --2- 172.21.15.114:0/2761633868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f245c104000 0x7f245c0ff680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:40.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.046+0000 7f245affd700 1 -- 172.21.15.114:0/2761633868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f245c104000 msgr2=0x7f245c0ff680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:40.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.046+0000 7f245affd700 1 --2- 172.21.15.114:0/2761633868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f245c104000 0x7f245c0ff680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:40.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.046+0000 7f245affd700 1 -- 172.21.15.114:0/2761633868 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244c0095f0 con 0x7f245c1016d0 2024-02-16T22:26:40.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.046+0000 7f245affd700 1 --2- 172.21.15.114:0/2761633868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f245c1016d0 0x7f245c101030 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f244c000c00 tx=0x7f244c00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:26:40.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.046+0000 7f2458ff9700 1 -- 172.21.15.114:0/2761633868 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f244c01d070 con 0x7f245c1016d0 2024-02-16T22:26:40.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.046+0000 7f246161b700 1 -- 172.21.15.114:0/2761633868 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f245c0fff80 con 0x7f245c1016d0 2024-02-16T22:26:40.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.046+0000 7f2458ff9700 1 -- 172.21.15.114:0/2761633868 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f244c005dc0 con 0x7f245c1016d0 2024-02-16T22:26:40.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.046+0000 7f2458ff9700 1 -- 172.21.15.114:0/2761633868 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f244c0226a0 con 0x7f245c1016d0 2024-02-16T22:26:40.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.047+0000 7f246161b700 1 -- 172.21.15.114:0/2761633868 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f245c100470 con 0x7f245c1016d0 2024-02-16T22:26:40.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.047+0000 7f246161b700 1 -- 172.21.15.114:0/2761633868 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f245c190530 con 0x7f245c1016d0 2024-02-16T22:26:40.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.048+0000 7f2458ff9700 1 -- 172.21.15.114:0/2761633868 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f244c003680 con 0x7f245c1016d0 2024-02-16T22:26:40.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.048+0000 7f2458ff9700 1 --2- 172.21.15.114:0/2761633868 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f244806d8a0 0x7f244806fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:40.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.048+0000 7f2458ff9700 1 -- 172.21.15.114:0/2761633868 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f244c08ea50 con 0x7f245c1016d0 2024-02-16T22:26:40.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.048+0000 7f24527fc700 1 --2- 172.21.15.114:0/2761633868 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f244806d8a0 0x7f244806fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:40.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.049+0000 7f24527fc700 1 --2- 172.21.15.114:0/2761633868 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f244806d8a0 0x7f244806fd60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2444009bf0 tx=0x7f2444009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:26:40.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.053+0000 7f2458ff9700 1 -- 172.21.15.114:0/2761633868 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f244c05bde0 con 0x7f245c1016d0 2024-02-16T22:26:40.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:40 smithi114 ceph-mon[94452]: from='client.34282 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:26:40.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:40 smithi114 ceph-mon[94452]: from='client.34286 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:26:40.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:40 smithi114 ceph-mon[94452]: from='client.34290 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:26:40.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:40 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/181861907' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:26:40.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.294+0000 7f246161b700 1 -- 172.21.15.114:0/2761633868 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f245c02cc70 con 0x7f245c1016d0 2024-02-16T22:26:40.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.295+0000 7f2458ff9700 1 -- 172.21.15.114:0/2761633868 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1846 (secure 0 0 0) 0x7f244c003930 con 0x7f245c1016d0 2024-02-16T22:26:40.297 INFO:teuthology.orchestra.run.smithi114.stdout:e14 2024-02-16T22:26:40.297 INFO:teuthology.orchestra.run.smithi114.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:26:40.297 INFO:teuthology.orchestra.run.smithi114.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:26:40.297 INFO:teuthology.orchestra.run.smithi114.stdout:legacy client fscid: 1 2024-02-16T22:26:40.297 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:26:40.297 INFO:teuthology.orchestra.run.smithi114.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:26:40.298 INFO:teuthology.orchestra.run.smithi114.stdout:fs_name cephfs 2024-02-16T22:26:40.298 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 12 2024-02-16T22:26:40.298 INFO:teuthology.orchestra.run.smithi114.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:26:40.298 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-02-16T22:18:02.108825+0000 2024-02-16T22:26:40.298 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-02-16T22:18:16.413371+0000 2024-02-16T22:26:40.298 INFO:teuthology.orchestra.run.smithi114.stdout:tableserver 0 2024-02-16T22:26:40.298 INFO:teuthology.orchestra.run.smithi114.stdout:root 0 2024-02-16T22:26:40.298 INFO:teuthology.orchestra.run.smithi114.stdout:session_timeout 60 2024-02-16T22:26:40.298 INFO:teuthology.orchestra.run.smithi114.stdout:session_autoclose 300 2024-02-16T22:26:40.298 INFO:teuthology.orchestra.run.smithi114.stdout:max_file_size 1099511627776 2024-02-16T22:26:40.298 INFO:teuthology.orchestra.run.smithi114.stdout:max_xattr_size 65536 2024-02-16T22:26:40.298 INFO:teuthology.orchestra.run.smithi114.stdout:required_client_features {} 2024-02-16T22:26:40.298 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure 0 2024-02-16T22:26:40.298 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure_osd_epoch 0 2024-02-16T22:26:40.299 INFO:teuthology.orchestra.run.smithi114.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:26:40.299 INFO:teuthology.orchestra.run.smithi114.stdout:max_mds 1 2024-02-16T22:26:40.299 INFO:teuthology.orchestra.run.smithi114.stdout:in 0 2024-02-16T22:26:40.299 INFO:teuthology.orchestra.run.smithi114.stdout:up {0=14468} 2024-02-16T22:26:40.299 INFO:teuthology.orchestra.run.smithi114.stdout:failed 2024-02-16T22:26:40.299 INFO:teuthology.orchestra.run.smithi114.stdout:damaged 2024-02-16T22:26:40.299 INFO:teuthology.orchestra.run.smithi114.stdout:stopped 2024-02-16T22:26:40.299 INFO:teuthology.orchestra.run.smithi114.stdout:data_pools [3] 2024-02-16T22:26:40.299 INFO:teuthology.orchestra.run.smithi114.stdout:metadata_pool 2 2024-02-16T22:26:40.299 INFO:teuthology.orchestra.run.smithi114.stdout:inline_data enabled 2024-02-16T22:26:40.299 INFO:teuthology.orchestra.run.smithi114.stdout:balancer 2024-02-16T22:26:40.299 INFO:teuthology.orchestra.run.smithi114.stdout:bal_rank_mask -1 2024-02-16T22:26:40.299 INFO:teuthology.orchestra.run.smithi114.stdout:standby_count_wanted 1 2024-02-16T22:26:40.300 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.zzryke{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.114:6826/277953535,v1:172.21.15.114:6827/277953535] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:26:40.300 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:26:40.300 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:26:40.300 INFO:teuthology.orchestra.run.smithi114.stdout:Standby daemons: 2024-02-16T22:26:40.300 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:26:40.300 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.fdxpey{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:26:40.300 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.vhlqbe{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:26:40.300 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.gopcfa{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:26:40.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.297+0000 7f246161b700 1 -- 172.21.15.114:0/2761633868 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f244806d8a0 msgr2=0x7f244806fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:40.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.297+0000 7f246161b700 1 --2- 172.21.15.114:0/2761633868 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f244806d8a0 0x7f244806fd60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2444009bf0 tx=0x7f2444009260 comp rx=0 tx=0).stop 2024-02-16T22:26:40.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.297+0000 7f246161b700 1 -- 172.21.15.114:0/2761633868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f245c1016d0 msgr2=0x7f245c101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:40.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.297+0000 7f246161b700 1 --2- 172.21.15.114:0/2761633868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f245c1016d0 0x7f245c101030 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f244c000c00 tx=0x7f244c00f740 comp rx=0 tx=0).stop 2024-02-16T22:26:40.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.297+0000 7f246161b700 1 -- 172.21.15.114:0/2761633868 shutdown_connections 2024-02-16T22:26:40.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.297+0000 7f246161b700 1 --2- 172.21.15.114:0/2761633868 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f244806d8a0 0x7f244806fd60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:40.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.297+0000 7f246161b700 1 --2- 172.21.15.114:0/2761633868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f245c1016d0 0x7f245c101030 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:40.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.297+0000 7f246161b700 1 --2- 172.21.15.114:0/2761633868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f245c104000 0x7f245c0ff680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:40.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.297+0000 7f246161b700 1 -- 172.21.15.114:0/2761633868 >> 172.21.15.114:0/2761633868 conn(0x7f245c0fb060 msgr2=0x7f245c0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:26:40.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.297+0000 7f246161b700 1 -- 172.21.15.114:0/2761633868 shutdown_connections 2024-02-16T22:26:40.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.297+0000 7f246161b700 1 -- 172.21.15.114:0/2761633868 wait complete. 2024-02-16T22:26:40.302 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 14 2024-02-16T22:26:40.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.402+0000 7fbc06fb7700 1 -- 172.21.15.114:0/3094195345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc00101dd0 msgr2=0x7fbc00102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:40.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.402+0000 7fbc06fb7700 1 --2- 172.21.15.114:0/3094195345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc00101dd0 0x7fbc00102230 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009910 tx=0x7fbbec009c20 comp rx=0 tx=0).stop 2024-02-16T22:26:40.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.402+0000 7fbc06fb7700 1 -- 172.21.15.114:0/3094195345 shutdown_connections 2024-02-16T22:26:40.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.402+0000 7fbc06fb7700 1 --2- 172.21.15.114:0/3094195345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc00101dd0 0x7fbc00102230 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:40.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.402+0000 7fbc06fb7700 1 --2- 172.21.15.114:0/3094195345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc00100bd0 0x7fbc00100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:40.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.402+0000 7fbc06fb7700 1 -- 172.21.15.114:0/3094195345 >> 172.21.15.114:0/3094195345 conn(0x7fbc000fc190 msgr2=0x7fbc000fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:26:40.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.402+0000 7fbc06fb7700 1 -- 172.21.15.114:0/3094195345 shutdown_connections 2024-02-16T22:26:40.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.402+0000 7fbc06fb7700 1 -- 172.21.15.114:0/3094195345 wait complete. 2024-02-16T22:26:40.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.403+0000 7fbc06fb7700 1 Processor -- start 2024-02-16T22:26:40.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.403+0000 7fbc06fb7700 1 -- start start 2024-02-16T22:26:40.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.403+0000 7fbc06fb7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc00100bd0 0x7fbc00074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:40.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.403+0000 7fbc06fb7700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc00101dd0 0x7fbc00073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:40.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.403+0000 7fbc06fb7700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc000736c0 con 0x7fbc00101dd0 2024-02-16T22:26:40.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.403+0000 7fbc06fb7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc00073800 con 0x7fbc00100bd0 2024-02-16T22:26:40.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.403+0000 7fbbfffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc00101dd0 0x7fbc00073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:40.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.403+0000 7fbbfffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc00101dd0 0x7fbc00073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33944/0 (socket says 172.21.15.114:33944) 2024-02-16T22:26:40.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.403+0000 7fbbfffff700 1 -- 172.21.15.114:0/2227034544 learned_addr learned my addr 172.21.15.114:0/2227034544 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:26:40.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.403+0000 7fbc04d53700 1 --2- 172.21.15.114:0/2227034544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc00100bd0 0x7fbc00074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:40.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.403+0000 7fbbfffff700 1 -- 172.21.15.114:0/2227034544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc00100bd0 msgr2=0x7fbc00074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:40.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.403+0000 7fbbfffff700 1 --2- 172.21.15.114:0/2227034544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc00100bd0 0x7fbc00074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:40.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.403+0000 7fbbfffff700 1 -- 172.21.15.114:0/2227034544 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbec0095f0 con 0x7fbc00101dd0 2024-02-16T22:26:40.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.403+0000 7fbc04d53700 1 --2- 172.21.15.114:0/2227034544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc00100bd0 0x7fbc00074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T22:26:40.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.403+0000 7fbbfffff700 1 --2- 172.21.15.114:0/2227034544 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc00101dd0 0x7fbc00073180 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00bd70 tx=0x7fbbec00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:26:40.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.404+0000 7fbbfdffb700 1 -- 172.21.15.114:0/2227034544 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbec01d070 con 0x7fbc00101dd0 2024-02-16T22:26:40.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.404+0000 7fbc06fb7700 1 -- 172.21.15.114:0/2227034544 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc00073a80 con 0x7fbc00101dd0 2024-02-16T22:26:40.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.404+0000 7fbbfdffb700 1 -- 172.21.15.114:0/2227034544 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbbec022470 con 0x7fbc00101dd0 2024-02-16T22:26:40.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.404+0000 7fbbfdffb700 1 -- 172.21.15.114:0/2227034544 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbec00f670 con 0x7fbc00101dd0 2024-02-16T22:26:40.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.404+0000 7fbc06fb7700 1 -- 172.21.15.114:0/2227034544 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc00073f70 con 0x7fbc00101dd0 2024-02-16T22:26:40.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.405+0000 7fbc06fb7700 1 -- 172.21.15.114:0/2227034544 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc00109730 con 0x7fbc00101dd0 2024-02-16T22:26:40.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.405+0000 7fbbfdffb700 1 -- 172.21.15.114:0/2227034544 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fbbec022a50 con 0x7fbc00101dd0 2024-02-16T22:26:40.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.406+0000 7fbbfdffb700 1 --2- 172.21.15.114:0/2227034544 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fbbf006d8a0 0x7fbbf006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:40.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.406+0000 7fbbfdffb700 1 -- 172.21.15.114:0/2227034544 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fbbec08eb60 con 0x7fbc00101dd0 2024-02-16T22:26:40.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.406+0000 7fbc04d53700 1 --2- 172.21.15.114:0/2227034544 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fbbf006d8a0 0x7fbbf006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:40.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.406+0000 7fbc04d53700 1 --2- 172.21.15.114:0/2227034544 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fbbf006d8a0 0x7fbbf006fd60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4005fd0 tx=0x7fbbf4005e40 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:26:40.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.410+0000 7fbbfdffb700 1 -- 172.21.15.114:0/2227034544 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbbec05c0c0 con 0x7fbc00101dd0 2024-02-16T22:26:40.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:40 smithi123 ceph-mon[82331]: from='client.34282 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:26:40.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:40 smithi123 ceph-mon[82331]: from='client.34286 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:26:40.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:40 smithi123 ceph-mon[82331]: from='client.34290 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:26:40.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:40 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/181861907' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:26:40.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.623+0000 7fbc06fb7700 1 -- 172.21.15.114:0/2227034544 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc00106720 con 0x7fbbf006d8a0 2024-02-16T22:26:40.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.624+0000 7fbbfdffb700 1 -- 172.21.15.114:0/2227034544 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fbc00106720 con 0x7fbbf006d8a0 2024-02-16T22:26:40.626 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:26:40.626 INFO:teuthology.orchestra.run.smithi114.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:26:40.626 INFO:teuthology.orchestra.run.smithi114.stdout: "in_progress": true, 2024-02-16T22:26:40.626 INFO:teuthology.orchestra.run.smithi114.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:26:40.626 INFO:teuthology.orchestra.run.smithi114.stdout: "services_complete": [ 2024-02-16T22:26:40.626 INFO:teuthology.orchestra.run.smithi114.stdout: "crash", 2024-02-16T22:26:40.626 INFO:teuthology.orchestra.run.smithi114.stdout: "mon", 2024-02-16T22:26:40.627 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr" 2024-02-16T22:26:40.627 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-02-16T22:26:40.627 INFO:teuthology.orchestra.run.smithi114.stdout: "progress": "9/23 daemons upgraded", 2024-02-16T22:26:40.627 INFO:teuthology.orchestra.run.smithi114.stdout: "message": "Currently upgrading osd daemons", 2024-02-16T22:26:40.627 INFO:teuthology.orchestra.run.smithi114.stdout: "is_paused": false 2024-02-16T22:26:40.627 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:26:40.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.626+0000 7fbc06fb7700 1 -- 172.21.15.114:0/2227034544 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fbbf006d8a0 msgr2=0x7fbbf006fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:40.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.626+0000 7fbc06fb7700 1 --2- 172.21.15.114:0/2227034544 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fbbf006d8a0 0x7fbbf006fd60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4005fd0 tx=0x7fbbf4005e40 comp rx=0 tx=0).stop 2024-02-16T22:26:40.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.626+0000 7fbc06fb7700 1 -- 172.21.15.114:0/2227034544 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc00101dd0 msgr2=0x7fbc00073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:40.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.626+0000 7fbc06fb7700 1 --2- 172.21.15.114:0/2227034544 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc00101dd0 0x7fbc00073180 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00bd70 tx=0x7fbbec00be50 comp rx=0 tx=0).stop 2024-02-16T22:26:40.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.626+0000 7fbc06fb7700 1 -- 172.21.15.114:0/2227034544 shutdown_connections 2024-02-16T22:26:40.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.626+0000 7fbc06fb7700 1 --2- 172.21.15.114:0/2227034544 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fbbf006d8a0 0x7fbbf006fd60 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:40.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.626+0000 7fbc06fb7700 1 --2- 172.21.15.114:0/2227034544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc00100bd0 0x7fbc00074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:40.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.626+0000 7fbc06fb7700 1 --2- 172.21.15.114:0/2227034544 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc00101dd0 0x7fbc00073180 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:40.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.626+0000 7fbc06fb7700 1 -- 172.21.15.114:0/2227034544 >> 172.21.15.114:0/2227034544 conn(0x7fbc000fc190 msgr2=0x7fbc00105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:26:40.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.627+0000 7fbc06fb7700 1 -- 172.21.15.114:0/2227034544 shutdown_connections 2024-02-16T22:26:40.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.627+0000 7fbc06fb7700 1 -- 172.21.15.114:0/2227034544 wait complete. 2024-02-16T22:26:40.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.729+0000 7f9f98e80700 1 -- 172.21.15.114:0/664945252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f94100be0 msgr2=0x7f9f94101000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:40.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.729+0000 7f9f98e80700 1 --2- 172.21.15.114:0/664945252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f94100be0 0x7f9f94101000 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9f84009910 tx=0x7f9f84009c20 comp rx=0 tx=0).stop 2024-02-16T22:26:40.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.729+0000 7f9f98e80700 1 -- 172.21.15.114:0/664945252 shutdown_connections 2024-02-16T22:26:40.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.729+0000 7f9f98e80700 1 --2- 172.21.15.114:0/664945252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f94101de0 0x7f9f94102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:40.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.729+0000 7f9f98e80700 1 --2- 172.21.15.114:0/664945252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f94100be0 0x7f9f94101000 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:40.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.729+0000 7f9f98e80700 1 -- 172.21.15.114:0/664945252 >> 172.21.15.114:0/664945252 conn(0x7f9f940fc140 msgr2=0x7f9f940fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:26:40.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.729+0000 7f9f98e80700 1 -- 172.21.15.114:0/664945252 shutdown_connections 2024-02-16T22:26:40.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.729+0000 7f9f98e80700 1 -- 172.21.15.114:0/664945252 wait complete. 2024-02-16T22:26:40.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.730+0000 7f9f98e80700 1 Processor -- start 2024-02-16T22:26:40.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.730+0000 7f9f98e80700 1 -- start start 2024-02-16T22:26:40.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.730+0000 7f9f98e80700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f94100be0 0x7f9f94198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:40.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.730+0000 7f9f98e80700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f94101de0 0x7f9f94198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:40.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.730+0000 7f9f98e80700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f94199160 con 0x7f9f94100be0 2024-02-16T22:26:40.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.730+0000 7f9f98e80700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f941992a0 con 0x7f9f94101de0 2024-02-16T22:26:40.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.731+0000 7f9f9259c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f94100be0 0x7f9f94198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:40.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.731+0000 7f9f9259c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f94100be0 0x7f9f94198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33968/0 (socket says 172.21.15.114:33968) 2024-02-16T22:26:40.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.731+0000 7f9f9259c700 1 -- 172.21.15.114:0/242024840 learned_addr learned my addr 172.21.15.114:0/242024840 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:26:40.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.731+0000 7f9f91d9b700 1 --2- 172.21.15.114:0/242024840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f94101de0 0x7f9f94198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:40.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.731+0000 7f9f9259c700 1 -- 172.21.15.114:0/242024840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f94101de0 msgr2=0x7f9f94198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:40.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.731+0000 7f9f9259c700 1 --2- 172.21.15.114:0/242024840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f94101de0 0x7f9f94198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:40.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.731+0000 7f9f9259c700 1 -- 172.21.15.114:0/242024840 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f840095f0 con 0x7f9f94100be0 2024-02-16T22:26:40.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.731+0000 7f9f9259c700 1 --2- 172.21.15.114:0/242024840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f94100be0 0x7f9f94198600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9f8400bda0 tx=0x7f9f8400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:26:40.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.731+0000 7f9f837fe700 1 -- 172.21.15.114:0/242024840 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f8401d070 con 0x7f9f94100be0 2024-02-16T22:26:40.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.731+0000 7f9f98e80700 1 -- 172.21.15.114:0/242024840 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f9419dcf0 con 0x7f9f94100be0 2024-02-16T22:26:40.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.731+0000 7f9f837fe700 1 -- 172.21.15.114:0/242024840 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9f84004c50 con 0x7f9f94100be0 2024-02-16T22:26:40.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.731+0000 7f9f837fe700 1 -- 172.21.15.114:0/242024840 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f8400f690 con 0x7f9f94100be0 2024-02-16T22:26:40.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.731+0000 7f9f98e80700 1 -- 172.21.15.114:0/242024840 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f9419e210 con 0x7f9f94100be0 2024-02-16T22:26:40.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.732+0000 7f9f98e80700 1 -- 172.21.15.114:0/242024840 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f94105ec0 con 0x7f9f94100be0 2024-02-16T22:26:40.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.733+0000 7f9f837fe700 1 -- 172.21.15.114:0/242024840 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f9f840229b0 con 0x7f9f94100be0 2024-02-16T22:26:40.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.733+0000 7f9f837fe700 1 --2- 172.21.15.114:0/242024840 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f9f7c06d8a0 0x7f9f7c06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:26:40.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.733+0000 7f9f837fe700 1 -- 172.21.15.114:0/242024840 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f9f8408ebb0 con 0x7f9f94100be0 2024-02-16T22:26:40.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.733+0000 7f9f91d9b700 1 --2- 172.21.15.114:0/242024840 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f9f7c06d8a0 0x7f9f7c06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:26:40.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.734+0000 7f9f91d9b700 1 --2- 172.21.15.114:0/242024840 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f9f7c06d8a0 0x7f9f7c06fd60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9f88006fd0 tx=0x7f9f88008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:26:40.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:40.738+0000 7f9f837fe700 1 -- 172.21.15.114:0/242024840 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9f84095080 con 0x7f9f94100be0 2024-02-16T22:26:41.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:41.014+0000 7f9f98e80700 1 -- 172.21.15.114:0/242024840 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9f94061a40 con 0x7f9f94100be0 2024-02-16T22:26:41.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:41.014+0000 7f9f837fe700 1 -- 172.21.15.114:0/242024840 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f9f840270e0 con 0x7f9f94100be0 2024-02-16T22:26:41.017 INFO:teuthology.orchestra.run.smithi114.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:26:41.017 INFO:teuthology.orchestra.run.smithi114.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:26:41.017 INFO:teuthology.orchestra.run.smithi114.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:26:41.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:41.016+0000 7f9f98e80700 1 -- 172.21.15.114:0/242024840 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f9f7c06d8a0 msgr2=0x7f9f7c06fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:41.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:41.016+0000 7f9f98e80700 1 --2- 172.21.15.114:0/242024840 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f9f7c06d8a0 0x7f9f7c06fd60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9f88006fd0 tx=0x7f9f88008040 comp rx=0 tx=0).stop 2024-02-16T22:26:41.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:41.016+0000 7f9f98e80700 1 -- 172.21.15.114:0/242024840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f94100be0 msgr2=0x7f9f94198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:26:41.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:41.016+0000 7f9f98e80700 1 --2- 172.21.15.114:0/242024840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f94100be0 0x7f9f94198600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9f8400bda0 tx=0x7f9f8400be80 comp rx=0 tx=0).stop 2024-02-16T22:26:41.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:41.016+0000 7f9f98e80700 1 -- 172.21.15.114:0/242024840 shutdown_connections 2024-02-16T22:26:41.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:41.016+0000 7f9f98e80700 1 --2- 172.21.15.114:0/242024840 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f9f7c06d8a0 0x7f9f7c06fd60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:41.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:41.016+0000 7f9f98e80700 1 --2- 172.21.15.114:0/242024840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f94100be0 0x7f9f94198600 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:41.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:41.016+0000 7f9f98e80700 1 --2- 172.21.15.114:0/242024840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f94101de0 0x7f9f94198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:26:41.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:41.016+0000 7f9f98e80700 1 -- 172.21.15.114:0/242024840 >> 172.21.15.114:0/242024840 conn(0x7f9f940fc140 msgr2=0x7f9f94103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:26:41.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:41.017+0000 7f9f98e80700 1 -- 172.21.15.114:0/242024840 shutdown_connections 2024-02-16T22:26:41.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:26:41.017+0000 7f9f98e80700 1 -- 172.21.15.114:0/242024840 wait complete. 2024-02-16T22:26:41.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:41 smithi114 ceph-mon[94452]: pgmap v105: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:41.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:41 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/2761633868' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:26:41.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:41 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/242024840' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:26:41.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:41 smithi123 ceph-mon[82331]: pgmap v105: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:41.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:41 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/2761633868' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:26:41.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:41 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/242024840' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:26:42.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:42 smithi114 ceph-mon[94452]: from='client.34302 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:26:42.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:42 smithi123 ceph-mon[82331]: from='client.34302 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:26:43.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:43 smithi114 ceph-mon[94452]: pgmap v106: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:43.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:43 smithi123 ceph-mon[82331]: pgmap v106: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:45.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:45 smithi114 ceph-mon[94452]: pgmap v107: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:45.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:45 smithi123 ceph-mon[82331]: pgmap v107: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:47.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:46 smithi114 ceph-mon[94452]: pgmap v108: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:47.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:47.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:26:47.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:47.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:46 smithi123 ceph-mon[82331]: pgmap v108: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:47.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:46 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:47.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:46 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:26:47.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:46 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:49.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:48 smithi114 ceph-mon[94452]: pgmap v109: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:49.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:48 smithi123 ceph-mon[82331]: pgmap v109: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:51.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:50 smithi123 ceph-mon[82331]: pgmap v110: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:51.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:50 smithi114 ceph-mon[94452]: pgmap v110: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:52.915 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:52 smithi123 ceph-mon[82331]: pgmap v111: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:52.915 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:52 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T22:26:52.915 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:52 smithi123 ceph-mon[82331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T22:26:52.916 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:52 smithi123 ceph-mon[82331]: Upgrade: osd.3 is safe to restart 2024-02-16T22:26:53.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:52 smithi114 ceph-mon[94452]: pgmap v111: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:53.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:52 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T22:26:53.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:52 smithi114 ceph-mon[94452]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T22:26:53.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:52 smithi114 ceph-mon[94452]: Upgrade: osd.3 is safe to restart 2024-02-16T22:26:54.992 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:54 smithi123 ceph-mon[82331]: Upgrade: Updating osd.3 2024-02-16T22:26:54.992 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:54 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:54.992 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:54 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-16T22:26:54.992 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:54 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:26:54.992 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:54 smithi123 ceph-mon[82331]: Deploying daemon osd.3 on smithi123 2024-02-16T22:26:54.993 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:54 smithi123 ceph-mon[82331]: pgmap v112: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:55.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:54 smithi114 ceph-mon[94452]: Upgrade: Updating osd.3 2024-02-16T22:26:55.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:54 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:26:55.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:54 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-16T22:26:55.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:54 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:26:55.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:54 smithi114 ceph-mon[94452]: Deploying daemon osd.3 on smithi123 2024-02-16T22:26:55.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:54 smithi114 ceph-mon[94452]: pgmap v112: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:57.102 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:56 smithi123 ceph-mon[82331]: pgmap v113: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:57.103 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:26:56 smithi123 systemd[1]: Stopping Ceph osd.3 for 1dcbaabe-cd18-11ee-95ba-87774f69a715... 2024-02-16T22:26:57.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:56 smithi114 ceph-mon[94452]: pgmap v113: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:57.552 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:26:57 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3[43094]: 2024-02-16T22:26:57.099+0000 7f3ece151700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T22:26:57.552 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:26:57 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3[43094]: 2024-02-16T22:26:57.099+0000 7f3ece151700 -1 osd.3 63 *** Got signal Terminated *** 2024-02-16T22:26:57.552 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:26:57 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3[43094]: 2024-02-16T22:26:57.099+0000 7f3ece151700 -1 osd.3 63 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T22:26:58.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:57 smithi114 ceph-mon[94452]: osd.3 marked itself down and dead 2024-02-16T22:26:58.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:57 smithi123 ceph-mon[82331]: osd.3 marked itself down and dead 2024-02-16T22:26:58.643 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:26:58 smithi123 podman[91588]: 2024-02-16 22:26:58.375389457 +0000 UTC m=+1.379802946 container died 575db379de3d34679b0cc700f4cbba4f70232a38cf53fcd3267bf191f98a15ac (image=quay.io/ceph/ceph@sha256:6176526d5ef4047f148d47c951653dcc6b0de49091fda006d3a088e7a1f404d0, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3, RELEASE=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS) 2024-02-16T22:26:59.011 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:58 smithi123 ceph-mon[82331]: pgmap v114: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:59.011 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:58 smithi123 ceph-mon[82331]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T22:26:59.011 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:58 smithi123 ceph-mon[82331]: osdmap e64: 6 total, 5 up, 6 in 2024-02-16T22:26:59.011 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:26:58 smithi123 podman[91588]: 2024-02-16 22:26:58.74240457 +0000 UTC m=+1.746818059 container cleanup 575db379de3d34679b0cc700f4cbba4f70232a38cf53fcd3267bf191f98a15ac (image=quay.io/ceph/ceph@sha256:6176526d5ef4047f148d47c951653dcc6b0de49091fda006d3a088e7a1f404d0, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-02-16T22:26:59.011 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:26:58 smithi123 bash[91588]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3 2024-02-16T22:26:59.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:58 smithi114 ceph-mon[94452]: pgmap v114: 65 pgs: 65 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:26:59.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:58 smithi114 ceph-mon[94452]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T22:26:59.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:58 smithi114 ceph-mon[94452]: osdmap e64: 6 total, 5 up, 6 in 2024-02-16T22:26:59.302 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:26:59 smithi123 podman[91616]: 2024-02-16 22:26:59.010105448 +0000 UTC m=+0.636805471 container remove 575db379de3d34679b0cc700f4cbba4f70232a38cf53fcd3267bf191f98a15ac (image=quay.io/ceph/ceph@sha256:6176526d5ef4047f148d47c951653dcc6b0de49091fda006d3a088e7a1f404d0, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3, RELEASE=HEAD, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214) 2024-02-16T22:26:59.622 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:26:59 smithi123 podman[91708]: 2024-02-16 22:26:59.261906697 +0000 UTC m=+0.018632071 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T22:26:59.622 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:26:59 smithi123 podman[91708]: 2024-02-16 22:26:59.419280142 +0000 UTC m=+0.176005509 container create 01d6217ac85d8a6df39f24cc139b233d26e9ab4f669e25e5b7265502efbc0774 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux ) 2024-02-16T22:26:59.623 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:26:59 smithi123 podman[91708]: 2024-02-16 22:26:59.621616141 +0000 UTC m=+0.378341508 container init 01d6217ac85d8a6df39f24cc139b233d26e9ab4f669e25e5b7265502efbc0774 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3-deactivate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240214) 2024-02-16T22:26:59.914 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:26:59 smithi123 ceph-mon[82331]: osdmap e65: 6 total, 5 up, 6 in 2024-02-16T22:26:59.915 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:26:59 smithi123 podman[91708]: 2024-02-16 22:26:59.624796621 +0000 UTC m=+0.381521992 container start 01d6217ac85d8a6df39f24cc139b233d26e9ab4f669e25e5b7265502efbc0774 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T22:26:59.915 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:26:59 smithi123 podman[91708]: 2024-02-16 22:26:59.713120474 +0000 UTC m=+0.469845879 container attach 01d6217ac85d8a6df39f24cc139b233d26e9ab4f669e25e5b7265502efbc0774 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS) 2024-02-16T22:27:00.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:26:59 smithi114 ceph-mon[94452]: osdmap e65: 6 total, 5 up, 6 in 2024-02-16T22:27:00.209 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:26:59 smithi123 podman[91708]: 2024-02-16 22:26:59.913578842 +0000 UTC m=+0.670304206 container died 01d6217ac85d8a6df39f24cc139b233d26e9ab4f669e25e5b7265502efbc0774 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-02-16T22:27:00.802 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:00 smithi123 podman[91734]: 2024-02-16 22:27:00.466418592 +0000 UTC m=+0.547906430 container remove 01d6217ac85d8a6df39f24cc139b233d26e9ab4f669e25e5b7265502efbc0774 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-02-16T22:27:00.802 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:00 smithi123 systemd[1]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@osd.3.service: Deactivated successfully. 2024-02-16T22:27:00.802 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:00 smithi123 systemd[1]: Stopped Ceph osd.3 for 1dcbaabe-cd18-11ee-95ba-87774f69a715. 2024-02-16T22:27:00.802 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:00 smithi123 systemd[1]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@osd.3.service: Consumed 1min 1.738s CPU time. 2024-02-16T22:27:00.803 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:00 smithi123 systemd[1]: Starting Ceph osd.3 for 1dcbaabe-cd18-11ee-95ba-87774f69a715... 2024-02-16T22:27:01.058 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:00 smithi123 ceph-mon[82331]: pgmap v117: 65 pgs: 16 stale+active+clean, 49 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:01.058 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:00 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:01.058 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:00 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:27:01.058 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:00 smithi123 podman[91835]: 2024-02-16 22:27:00.806710644 +0000 UTC m=+0.018171536 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T22:27:01.059 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:00 smithi123 podman[91835]: 2024-02-16 22:27:00.925989485 +0000 UTC m=+0.137450377 container create fa9720fb0ab035e7e86eef2a0790b35a5b491ae6e548cd4aa6573a9859a6b02b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3-activate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-02-16T22:27:01.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:00 smithi114 ceph-mon[94452]: pgmap v117: 65 pgs: 16 stale+active+clean, 49 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:01.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:00 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:01.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:00 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:27:01.552 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:01 smithi123 podman[91835]: 2024-02-16 22:27:01.153525664 +0000 UTC m=+0.364986554 container init fa9720fb0ab035e7e86eef2a0790b35a5b491ae6e548cd4aa6573a9859a6b02b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3-activate, org.label-schema.build-date=20240214, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-02-16T22:27:01.552 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:01 smithi123 podman[91835]: 2024-02-16 22:27:01.156612078 +0000 UTC m=+0.368072978 container start fa9720fb0ab035e7e86eef2a0790b35a5b491ae6e548cd4aa6573a9859a6b02b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3-activate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-02-16T22:27:01.552 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:01 smithi123 podman[91835]: 2024-02-16 22:27:01.236660255 +0000 UTC m=+0.448121155 container attach fa9720fb0ab035e7e86eef2a0790b35a5b491ae6e548cd4aa6573a9859a6b02b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3-activate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T22:27:01.900 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:01 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3-activate[91850]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-16T22:27:01.900 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:01 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3-activate[91850]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-02-16T22:27:01.900 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:01 smithi123 bash[91835]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-16T22:27:01.900 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:01 smithi123 bash[91835]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-02-16T22:27:01.900 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:01 smithi123 bash[91835]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-02-16T22:27:01.900 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:01 smithi123 bash[91835]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-02-16T22:27:01.900 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:01 smithi123 bash[91835]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-02-16T22:27:01.901 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:01 smithi123 bash[91835]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-16T22:27:01.901 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:01 smithi123 bash[91835]: --> ceph-volume raw activate successful for osd ID: 3 2024-02-16T22:27:01.901 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:01 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3-activate[91850]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-02-16T22:27:01.901 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:01 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3-activate[91850]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-02-16T22:27:01.901 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:01 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3-activate[91850]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-02-16T22:27:01.901 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:01 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3-activate[91850]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-16T22:27:01.901 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:01 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3-activate[91850]: --> ceph-volume raw activate successful for osd ID: 3 2024-02-16T22:27:01.901 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:01 smithi123 podman[91835]: 2024-02-16 22:27:01.622165926 +0000 UTC m=+0.833626825 container died fa9720fb0ab035e7e86eef2a0790b35a5b491ae6e548cd4aa6573a9859a6b02b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3-activate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-02-16T22:27:02.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:01 smithi114 ceph-mon[94452]: Health check failed: Degraded data redundancy: 40/267 objects degraded (14.981%), 11 pgs degraded (PG_DEGRADED) 2024-02-16T22:27:02.277 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:01 smithi123 ceph-mon[82331]: Health check failed: Degraded data redundancy: 40/267 objects degraded (14.981%), 11 pgs degraded (PG_DEGRADED) 2024-02-16T22:27:02.552 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:02 smithi123 podman[91976]: 2024-02-16 22:27:02.276395117 +0000 UTC m=+0.648965691 container remove fa9720fb0ab035e7e86eef2a0790b35a5b491ae6e548cd4aa6573a9859a6b02b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3-activate, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T22:27:02.553 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:02 smithi123 podman[92045]: 2024-02-16 22:27:02.39477915 +0000 UTC m=+0.018689187 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T22:27:02.553 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:02 smithi123 podman[92045]: 2024-02-16 22:27:02.499434921 +0000 UTC m=+0.123344992 container create b921448ba169e5c06eb928ef5deb1d1e36a264f84fef4b24355587700c38e996 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True) 2024-02-16T22:27:02.903 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:02 smithi123 podman[92045]: 2024-02-16 22:27:02.693390174 +0000 UTC m=+0.317300200 container init b921448ba169e5c06eb928ef5deb1d1e36a264f84fef4b24355587700c38e996 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5) 2024-02-16T22:27:02.903 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:02 smithi123 podman[92045]: 2024-02-16 22:27:02.696212471 +0000 UTC m=+0.320122498 container start b921448ba169e5c06eb928ef5deb1d1e36a264f84fef4b24355587700c38e996 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214) 2024-02-16T22:27:02.903 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:02 smithi123 bash[92045]: b921448ba169e5c06eb928ef5deb1d1e36a264f84fef4b24355587700c38e996 2024-02-16T22:27:02.903 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:02 smithi123 systemd[1]: Started Ceph osd.3 for 1dcbaabe-cd18-11ee-95ba-87774f69a715. 2024-02-16T22:27:03.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:02 smithi114 ceph-mon[94452]: pgmap v118: 65 pgs: 11 active+undersized+degraded, 3 stale+active+clean, 14 active+undersized, 37 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 40/267 objects degraded (14.981%) 2024-02-16T22:27:03.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:02 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:03.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:02 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:03.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:02 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:27:03.239 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:02 smithi123 ceph-mon[82331]: pgmap v118: 65 pgs: 11 active+undersized+degraded, 3 stale+active+clean, 14 active+undersized, 37 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 40/267 objects degraded (14.981%) 2024-02-16T22:27:03.239 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:02 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:03.239 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:02 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:03.240 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:02 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:27:03.552 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:03 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3[92060]: 2024-02-16T22:27:03.236+0000 7f33c2b137c0 -1 Falling back to public interface 2024-02-16T22:27:05.196 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:04 smithi123 ceph-mon[82331]: pgmap v119: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 59/267 objects degraded (22.097%) 2024-02-16T22:27:05.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:04 smithi114 ceph-mon[94452]: pgmap v119: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 59/267 objects degraded (22.097%) 2024-02-16T22:27:07.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:06 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:07.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:06 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:07.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:06 smithi123 ceph-mon[82331]: pgmap v120: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 59/267 objects degraded (22.097%) 2024-02-16T22:27:07.057 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:06 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:07.057 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:06 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:07.057 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:06 smithi114 ceph-mon[94452]: pgmap v120: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 59/267 objects degraded (22.097%) 2024-02-16T22:27:08.052 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:07 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3[92060]: 2024-02-16T22:27:07.789+0000 7f33c2b137c0 -1 osd.3 0 read_superblock omap replica is missing. 2024-02-16T22:27:08.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:07 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:08.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:07 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:08.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:07 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:08.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:07 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:08.463 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:08 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3[92060]: 2024-02-16T22:27:08.162+0000 7f33c2b137c0 -1 osd.3 63 log_to_monitors true 2024-02-16T22:27:09.052 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:27:08 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3[92060]: 2024-02-16T22:27:08.889+0000 7f33b3f6b700 -1 osd.3 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T22:27:09.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:08 smithi123 ceph-mon[82331]: pgmap v121: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 59/267 objects degraded (22.097%) 2024-02-16T22:27:09.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:08 smithi123 ceph-mon[82331]: from='osd.3 [v2:172.21.15.123:6800/3100755210,v1:172.21.15.123:6801/3100755210]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T22:27:09.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:08 smithi123 ceph-mon[82331]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T22:27:09.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:08 smithi114 ceph-mon[94452]: pgmap v121: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 59/267 objects degraded (22.097%) 2024-02-16T22:27:09.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:08 smithi114 ceph-mon[94452]: from='osd.3 [v2:172.21.15.123:6800/3100755210,v1:172.21.15.123:6801/3100755210]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T22:27:09.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:08 smithi114 ceph-mon[94452]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T22:27:10.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:09 smithi114 ceph-mon[94452]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-16T22:27:10.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:09 smithi114 ceph-mon[94452]: from='osd.3 [v2:172.21.15.123:6800/3100755210,v1:172.21.15.123:6801/3100755210]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-02-16T22:27:10.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:09 smithi114 ceph-mon[94452]: osdmap e66: 6 total, 5 up, 6 in 2024-02-16T22:27:10.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:09 smithi114 ceph-mon[94452]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-02-16T22:27:10.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:09 smithi123 ceph-mon[82331]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-16T22:27:10.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:09 smithi123 ceph-mon[82331]: from='osd.3 [v2:172.21.15.123:6800/3100755210,v1:172.21.15.123:6801/3100755210]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-02-16T22:27:10.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:09 smithi123 ceph-mon[82331]: osdmap e66: 6 total, 5 up, 6 in 2024-02-16T22:27:10.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:09 smithi123 ceph-mon[82331]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-02-16T22:27:11.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.129+0000 7fd4d8b84700 1 -- 172.21.15.114:0/2703892113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4d4100cb0 msgr2=0x7fd4d4101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:11.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.129+0000 7fd4d8b84700 1 --2- 172.21.15.114:0/2703892113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4d4100cb0 0x7fd4d4101110 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4009910 tx=0x7fd4c4009c20 comp rx=0 tx=0).stop 2024-02-16T22:27:11.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.129+0000 7fd4d8b84700 1 -- 172.21.15.114:0/2703892113 shutdown_connections 2024-02-16T22:27:11.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.129+0000 7fd4d8b84700 1 --2- 172.21.15.114:0/2703892113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4d4100cb0 0x7fd4d4101110 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:11.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.129+0000 7fd4d8b84700 1 --2- 172.21.15.114:0/2703892113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4d40ffab0 0x7fd4d40ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:11.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.129+0000 7fd4d8b84700 1 -- 172.21.15.114:0/2703892113 >> 172.21.15.114:0/2703892113 conn(0x7fd4d40fb030 msgr2=0x7fd4d40fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:27:11.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.129+0000 7fd4d8b84700 1 -- 172.21.15.114:0/2703892113 shutdown_connections 2024-02-16T22:27:11.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.129+0000 7fd4d8b84700 1 -- 172.21.15.114:0/2703892113 wait complete. 2024-02-16T22:27:11.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.130+0000 7fd4d8b84700 1 Processor -- start 2024-02-16T22:27:11.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.130+0000 7fd4d8b84700 1 -- start start 2024-02-16T22:27:11.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.130+0000 7fd4d8b84700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4d40ffab0 0x7fd4d4103a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:11.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.130+0000 7fd4d8b84700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4d4100cb0 0x7fd4d4102090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:11.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.130+0000 7fd4d8b84700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d41025d0 con 0x7fd4d40ffab0 2024-02-16T22:27:11.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.130+0000 7fd4d8b84700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d4102710 con 0x7fd4d4100cb0 2024-02-16T22:27:11.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.131+0000 7fd4d259c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4d40ffab0 0x7fd4d4103a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:11.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.131+0000 7fd4d259c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4d40ffab0 0x7fd4d4103a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39974/0 (socket says 172.21.15.114:39974) 2024-02-16T22:27:11.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.131+0000 7fd4d259c700 1 -- 172.21.15.114:0/3961724151 learned_addr learned my addr 172.21.15.114:0/3961724151 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:27:11.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.131+0000 7fd4d1d9b700 1 --2- 172.21.15.114:0/3961724151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4d4100cb0 0x7fd4d4102090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:11.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.131+0000 7fd4d259c700 1 -- 172.21.15.114:0/3961724151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4d4100cb0 msgr2=0x7fd4d4102090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:11.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.131+0000 7fd4d259c700 1 --2- 172.21.15.114:0/3961724151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4d4100cb0 0x7fd4d4102090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:11.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.131+0000 7fd4d259c700 1 -- 172.21.15.114:0/3961724151 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c40095f0 con 0x7fd4d40ffab0 2024-02-16T22:27:11.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.131+0000 7fd4d1d9b700 1 --2- 172.21.15.114:0/3961724151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4d4100cb0 0x7fd4d4102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:27:11.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.131+0000 7fd4d259c700 1 --2- 172.21.15.114:0/3961724151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4d40ffab0 0x7fd4d4103a40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd4c800d900 tx=0x7fd4c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:27:11.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.131+0000 7fd4c37fe700 1 -- 172.21.15.114:0/3961724151 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4c80041f0 con 0x7fd4d40ffab0 2024-02-16T22:27:11.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.131+0000 7fd4d8b84700 1 -- 172.21.15.114:0/3961724151 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d41029f0 con 0x7fd4d40ffab0 2024-02-16T22:27:11.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.131+0000 7fd4d8b84700 1 -- 172.21.15.114:0/3961724151 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d419e3d0 con 0x7fd4d40ffab0 2024-02-16T22:27:11.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.131+0000 7fd4c37fe700 1 -- 172.21.15.114:0/3961724151 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd4c8004350 con 0x7fd4d40ffab0 2024-02-16T22:27:11.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.131+0000 7fd4c37fe700 1 -- 172.21.15.114:0/3961724151 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4c8003da0 con 0x7fd4d40ffab0 2024-02-16T22:27:11.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.132+0000 7fd4d8b84700 1 -- 172.21.15.114:0/3961724151 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d4108610 con 0x7fd4d40ffab0 2024-02-16T22:27:11.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.133+0000 7fd4c37fe700 1 -- 172.21.15.114:0/3961724151 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fd4c8009730 con 0x7fd4d40ffab0 2024-02-16T22:27:11.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.133+0000 7fd4c37fe700 1 --2- 172.21.15.114:0/3961724151 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd4bc06d8a0 0x7fd4bc06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:11.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.133+0000 7fd4c37fe700 1 -- 172.21.15.114:0/3961724151 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd4c808d400 con 0x7fd4d40ffab0 2024-02-16T22:27:11.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.133+0000 7fd4d1d9b700 1 --2- 172.21.15.114:0/3961724151 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd4bc06d8a0 0x7fd4bc06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:11.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.134+0000 7fd4d1d9b700 1 --2- 172.21.15.114:0/3961724151 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd4bc06d8a0 0x7fd4bc06fd60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4005b40 tx=0x7fd4c4005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:27:11.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.138+0000 7fd4c37fe700 1 -- 172.21.15.114:0/3961724151 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd4c805a790 con 0x7fd4d40ffab0 2024-02-16T22:27:11.140 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:10 smithi114 ceph-mon[94452]: pgmap v123: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 59/267 objects degraded (22.097%) 2024-02-16T22:27:11.140 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:10 smithi114 ceph-mon[94452]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T22:27:11.140 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:10 smithi114 ceph-mon[94452]: osd.3 [v2:172.21.15.123:6800/3100755210,v1:172.21.15.123:6801/3100755210] boot 2024-02-16T22:27:11.141 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:10 smithi114 ceph-mon[94452]: osdmap e67: 6 total, 6 up, 6 in 2024-02-16T22:27:11.141 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:10 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:27:11.141 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:10 smithi114 ceph-mon[94452]: Health check update: Degraded data redundancy: 59/267 objects degraded (22.097%), 18 pgs degraded (PG_DEGRADED) 2024-02-16T22:27:11.141 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:10 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:11.141 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:10 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:11.141 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:10 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:27:11.141 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:10 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:27:11.141 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:10 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:11.141 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:10 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:27:11.141 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:10 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:27:11.141 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:10 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:27:11.142 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:10 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:27:11.142 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:10 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T22:27:11.142 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:10 smithi114 ceph-mon[94452]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T22:27:11.142 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:10 smithi114 ceph-mon[94452]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-02-16T22:27:11.142 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:10 smithi114 ceph-mon[94452]: osdmap e68: 6 total, 6 up, 6 in 2024-02-16T22:27:11.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:10 smithi123 ceph-mon[82331]: pgmap v123: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 59/267 objects degraded (22.097%) 2024-02-16T22:27:11.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:10 smithi123 ceph-mon[82331]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T22:27:11.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:10 smithi123 ceph-mon[82331]: osd.3 [v2:172.21.15.123:6800/3100755210,v1:172.21.15.123:6801/3100755210] boot 2024-02-16T22:27:11.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:10 smithi123 ceph-mon[82331]: osdmap e67: 6 total, 6 up, 6 in 2024-02-16T22:27:11.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:10 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T22:27:11.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:10 smithi123 ceph-mon[82331]: Health check update: Degraded data redundancy: 59/267 objects degraded (22.097%), 18 pgs degraded (PG_DEGRADED) 2024-02-16T22:27:11.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:10 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:11.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:10 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:11.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:10 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:27:11.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:10 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:27:11.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:10 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:11.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:10 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:27:11.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:10 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:27:11.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:10 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:27:11.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:10 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:27:11.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:10 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T22:27:11.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:10 smithi123 ceph-mon[82331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T22:27:11.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:10 smithi123 ceph-mon[82331]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-02-16T22:27:11.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:10 smithi123 ceph-mon[82331]: osdmap e68: 6 total, 6 up, 6 in 2024-02-16T22:27:11.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.347+0000 7fd4d8b84700 1 -- 172.21.15.114:0/3961724151 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4d419e980 con 0x7fd4bc06d8a0 2024-02-16T22:27:11.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.348+0000 7fd4c37fe700 1 -- 172.21.15.114:0/3961724151 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fd4d419e980 con 0x7fd4bc06d8a0 2024-02-16T22:27:11.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.350+0000 7fd4d8b84700 1 -- 172.21.15.114:0/3961724151 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd4bc06d8a0 msgr2=0x7fd4bc06fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:11.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.350+0000 7fd4d8b84700 1 --2- 172.21.15.114:0/3961724151 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd4bc06d8a0 0x7fd4bc06fd60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4005b40 tx=0x7fd4c4005a90 comp rx=0 tx=0).stop 2024-02-16T22:27:11.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.350+0000 7fd4d8b84700 1 -- 172.21.15.114:0/3961724151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4d40ffab0 msgr2=0x7fd4d4103a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:11.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.350+0000 7fd4d8b84700 1 --2- 172.21.15.114:0/3961724151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4d40ffab0 0x7fd4d4103a40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd4c800d900 tx=0x7fd4c800dcc0 comp rx=0 tx=0).stop 2024-02-16T22:27:11.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.350+0000 7fd4d8b84700 1 -- 172.21.15.114:0/3961724151 shutdown_connections 2024-02-16T22:27:11.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.350+0000 7fd4d8b84700 1 --2- 172.21.15.114:0/3961724151 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd4bc06d8a0 0x7fd4bc06fd60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:11.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.350+0000 7fd4d8b84700 1 --2- 172.21.15.114:0/3961724151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4d40ffab0 0x7fd4d4103a40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:11.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.350+0000 7fd4d8b84700 1 --2- 172.21.15.114:0/3961724151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4d4100cb0 0x7fd4d4102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:11.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.351+0000 7fd4d8b84700 1 -- 172.21.15.114:0/3961724151 >> 172.21.15.114:0/3961724151 conn(0x7fd4d40fb030 msgr2=0x7fd4d4068410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:27:11.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.351+0000 7fd4d8b84700 1 -- 172.21.15.114:0/3961724151 shutdown_connections 2024-02-16T22:27:11.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.351+0000 7fd4d8b84700 1 -- 172.21.15.114:0/3961724151 wait complete. 2024-02-16T22:27:11.363 INFO:teuthology.orchestra.run.smithi114.stdout:true 2024-02-16T22:27:11.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.454+0000 7fbe47172700 1 -- 172.21.15.114:0/1087296278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe400ffab0 msgr2=0x7fbe400ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:11.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.454+0000 7fbe47172700 1 --2- 172.21.15.114:0/1087296278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe400ffab0 0x7fbe400ffed0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009910 tx=0x7fbe34009c20 comp rx=0 tx=0).stop 2024-02-16T22:27:11.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.454+0000 7fbe47172700 1 -- 172.21.15.114:0/1087296278 shutdown_connections 2024-02-16T22:27:11.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.454+0000 7fbe47172700 1 --2- 172.21.15.114:0/1087296278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe40100cb0 0x7fbe40101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:11.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.454+0000 7fbe47172700 1 --2- 172.21.15.114:0/1087296278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe400ffab0 0x7fbe400ffed0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:11.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.454+0000 7fbe47172700 1 -- 172.21.15.114:0/1087296278 >> 172.21.15.114:0/1087296278 conn(0x7fbe400fb030 msgr2=0x7fbe400fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:27:11.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.454+0000 7fbe47172700 1 -- 172.21.15.114:0/1087296278 shutdown_connections 2024-02-16T22:27:11.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.454+0000 7fbe47172700 1 -- 172.21.15.114:0/1087296278 wait complete. 2024-02-16T22:27:11.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.455+0000 7fbe47172700 1 Processor -- start 2024-02-16T22:27:11.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.455+0000 7fbe47172700 1 -- start start 2024-02-16T22:27:11.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.455+0000 7fbe47172700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe400ffab0 0x7fbe40068f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:11.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.455+0000 7fbe47172700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe40100cb0 0x7fbe4019b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:11.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.455+0000 7fbe47172700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4019b670 con 0x7fbe400ffab0 2024-02-16T22:27:11.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.455+0000 7fbe47172700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4019b7b0 con 0x7fbe40100cb0 2024-02-16T22:27:11.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.455+0000 7fbe44f0e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe400ffab0 0x7fbe40068f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:11.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.455+0000 7fbe44f0e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe400ffab0 0x7fbe40068f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59552/0 (socket says 172.21.15.114:59552) 2024-02-16T22:27:11.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.455+0000 7fbe44f0e700 1 -- 172.21.15.114:0/1272294809 learned_addr learned my addr 172.21.15.114:0/1272294809 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:27:11.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.455+0000 7fbe3ffff700 1 --2- 172.21.15.114:0/1272294809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe40100cb0 0x7fbe4019b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:11.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.456+0000 7fbe44f0e700 1 -- 172.21.15.114:0/1272294809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe40100cb0 msgr2=0x7fbe4019b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:11.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.456+0000 7fbe44f0e700 1 --2- 172.21.15.114:0/1272294809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe40100cb0 0x7fbe4019b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:11.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.456+0000 7fbe44f0e700 1 -- 172.21.15.114:0/1272294809 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe340095f0 con 0x7fbe400ffab0 2024-02-16T22:27:11.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.456+0000 7fbe44f0e700 1 --2- 172.21.15.114:0/1272294809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe400ffab0 0x7fbe40068f70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400bda0 tx=0x7fbe3400bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:27:11.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.456+0000 7fbe3dffb700 1 -- 172.21.15.114:0/1272294809 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe3401d070 con 0x7fbe400ffab0 2024-02-16T22:27:11.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.456+0000 7fbe3dffb700 1 -- 172.21.15.114:0/1272294809 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbe3400f460 con 0x7fbe400ffab0 2024-02-16T22:27:11.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.456+0000 7fbe47172700 1 -- 172.21.15.114:0/1272294809 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe4019ba30 con 0x7fbe400ffab0 2024-02-16T22:27:11.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.456+0000 7fbe3dffb700 1 -- 172.21.15.114:0/1272294809 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe34021620 con 0x7fbe400ffab0 2024-02-16T22:27:11.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.456+0000 7fbe47172700 1 -- 172.21.15.114:0/1272294809 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe4019bf20 con 0x7fbe400ffab0 2024-02-16T22:27:11.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.457+0000 7fbe47172700 1 -- 172.21.15.114:0/1272294809 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe40108610 con 0x7fbe400ffab0 2024-02-16T22:27:11.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.458+0000 7fbe3dffb700 1 -- 172.21.15.114:0/1272294809 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fbe3400f5d0 con 0x7fbe400ffab0 2024-02-16T22:27:11.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.458+0000 7fbe3dffb700 1 --2- 172.21.15.114:0/1272294809 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fbe3006d8a0 0x7fbe3006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:11.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.458+0000 7fbe3dffb700 1 -- 172.21.15.114:0/1272294809 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fbe3408e770 con 0x7fbe400ffab0 2024-02-16T22:27:11.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.458+0000 7fbe3ffff700 1 --2- 172.21.15.114:0/1272294809 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fbe3006d8a0 0x7fbe3006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:11.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.459+0000 7fbe3ffff700 1 --2- 172.21.15.114:0/1272294809 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fbe3006d8a0 0x7fbe3006fd60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbe400694f0 tx=0x7fbe2c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:27:11.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.463+0000 7fbe3dffb700 1 -- 172.21.15.114:0/1272294809 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbe3405bb00 con 0x7fbe400ffab0 2024-02-16T22:27:11.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.672+0000 7fbe47172700 1 -- 172.21.15.114:0/1272294809 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe40105600 con 0x7fbe3006d8a0 2024-02-16T22:27:11.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.673+0000 7fbe3dffb700 1 -- 172.21.15.114:0/1272294809 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fbe40105600 con 0x7fbe3006d8a0 2024-02-16T22:27:11.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.675+0000 7fbe47172700 1 -- 172.21.15.114:0/1272294809 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fbe3006d8a0 msgr2=0x7fbe3006fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:11.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.675+0000 7fbe47172700 1 --2- 172.21.15.114:0/1272294809 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fbe3006d8a0 0x7fbe3006fd60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbe400694f0 tx=0x7fbe2c00b410 comp rx=0 tx=0).stop 2024-02-16T22:27:11.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.675+0000 7fbe47172700 1 -- 172.21.15.114:0/1272294809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe400ffab0 msgr2=0x7fbe40068f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:11.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.675+0000 7fbe47172700 1 --2- 172.21.15.114:0/1272294809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe400ffab0 0x7fbe40068f70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400bda0 tx=0x7fbe3400bdd0 comp rx=0 tx=0).stop 2024-02-16T22:27:11.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.675+0000 7fbe47172700 1 -- 172.21.15.114:0/1272294809 shutdown_connections 2024-02-16T22:27:11.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.675+0000 7fbe47172700 1 --2- 172.21.15.114:0/1272294809 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fbe3006d8a0 0x7fbe3006fd60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:11.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.675+0000 7fbe47172700 1 --2- 172.21.15.114:0/1272294809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe400ffab0 0x7fbe40068f70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:11.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.675+0000 7fbe47172700 1 --2- 172.21.15.114:0/1272294809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe40100cb0 0x7fbe4019b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:11.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.675+0000 7fbe47172700 1 -- 172.21.15.114:0/1272294809 >> 172.21.15.114:0/1272294809 conn(0x7fbe400fb030 msgr2=0x7fbe40103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:27:11.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.675+0000 7fbe47172700 1 -- 172.21.15.114:0/1272294809 shutdown_connections 2024-02-16T22:27:11.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.676+0000 7fbe47172700 1 -- 172.21.15.114:0/1272294809 wait complete. 2024-02-16T22:27:11.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.779+0000 7fce72ecd700 1 -- 172.21.15.114:0/4115622165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce6c074dc0 msgr2=0x7fce6c073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:11.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.779+0000 7fce72ecd700 1 --2- 172.21.15.114:0/4115622165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce6c074dc0 0x7fce6c073220 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fce60009910 tx=0x7fce60009c20 comp rx=0 tx=0).stop 2024-02-16T22:27:11.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.779+0000 7fce72ecd700 1 -- 172.21.15.114:0/4115622165 shutdown_connections 2024-02-16T22:27:11.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.779+0000 7fce72ecd700 1 --2- 172.21.15.114:0/4115622165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce6c0737f0 0x7fce6c073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:11.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.779+0000 7fce72ecd700 1 --2- 172.21.15.114:0/4115622165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce6c074dc0 0x7fce6c073220 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:11.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.779+0000 7fce72ecd700 1 -- 172.21.15.114:0/4115622165 >> 172.21.15.114:0/4115622165 conn(0x7fce6c0fc140 msgr2=0x7fce6c0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:27:11.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.779+0000 7fce72ecd700 1 -- 172.21.15.114:0/4115622165 shutdown_connections 2024-02-16T22:27:11.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.779+0000 7fce72ecd700 1 -- 172.21.15.114:0/4115622165 wait complete. 2024-02-16T22:27:11.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.779+0000 7fce72ecd700 1 Processor -- start 2024-02-16T22:27:11.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.779+0000 7fce72ecd700 1 -- start start 2024-02-16T22:27:11.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.780+0000 7fce72ecd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce6c0737f0 0x7fce6c19c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:11.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.780+0000 7fce72ecd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce6c074dc0 0x7fce6c19cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:11.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.780+0000 7fce72ecd700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c19d4c0 con 0x7fce6c074dc0 2024-02-16T22:27:11.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.780+0000 7fce72ecd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c19d600 con 0x7fce6c0737f0 2024-02-16T22:27:11.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.780+0000 7fce6bfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce6c074dc0 0x7fce6c19cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:11.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.780+0000 7fce6bfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce6c074dc0 0x7fce6c19cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59574/0 (socket says 172.21.15.114:59574) 2024-02-16T22:27:11.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.780+0000 7fce6bfff700 1 -- 172.21.15.114:0/4236990880 learned_addr learned my addr 172.21.15.114:0/4236990880 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:27:11.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.780+0000 7fce70c69700 1 --2- 172.21.15.114:0/4236990880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce6c0737f0 0x7fce6c19c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:11.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.780+0000 7fce6bfff700 1 -- 172.21.15.114:0/4236990880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce6c0737f0 msgr2=0x7fce6c19c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:11.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.780+0000 7fce6bfff700 1 --2- 172.21.15.114:0/4236990880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce6c0737f0 0x7fce6c19c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:11.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.780+0000 7fce6bfff700 1 -- 172.21.15.114:0/4236990880 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce600095f0 con 0x7fce6c074dc0 2024-02-16T22:27:11.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.780+0000 7fce6bfff700 1 --2- 172.21.15.114:0/4236990880 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce6c074dc0 0x7fce6c19cea0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fce5800e8f0 tx=0x7fce5800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:27:11.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.780+0000 7fce69ffb700 1 -- 172.21.15.114:0/4236990880 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce58009790 con 0x7fce6c074dc0 2024-02-16T22:27:11.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.781+0000 7fce69ffb700 1 -- 172.21.15.114:0/4236990880 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fce5800c810 con 0x7fce6c074dc0 2024-02-16T22:27:11.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.781+0000 7fce72ecd700 1 -- 172.21.15.114:0/4236990880 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce6c1a20b0 con 0x7fce6c074dc0 2024-02-16T22:27:11.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.781+0000 7fce69ffb700 1 -- 172.21.15.114:0/4236990880 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce58003d20 con 0x7fce6c074dc0 2024-02-16T22:27:11.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.781+0000 7fce72ecd700 1 -- 172.21.15.114:0/4236990880 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce6c1a2600 con 0x7fce6c074dc0 2024-02-16T22:27:11.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.782+0000 7fce72ecd700 1 -- 172.21.15.114:0/4236990880 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce6c10fd30 con 0x7fce6c074dc0 2024-02-16T22:27:11.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.782+0000 7fce69ffb700 1 -- 172.21.15.114:0/4236990880 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fce580041f0 con 0x7fce6c074dc0 2024-02-16T22:27:11.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.783+0000 7fce69ffb700 1 --2- 172.21.15.114:0/4236990880 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fce5c06d8a0 0x7fce5c06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:11.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.783+0000 7fce69ffb700 1 -- 172.21.15.114:0/4236990880 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fce58014070 con 0x7fce6c074dc0 2024-02-16T22:27:11.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.783+0000 7fce70c69700 1 --2- 172.21.15.114:0/4236990880 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fce5c06d8a0 0x7fce5c06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:11.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.783+0000 7fce70c69700 1 --2- 172.21.15.114:0/4236990880 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fce5c06d8a0 0x7fce5c06fd60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fce60009fd0 tx=0x7fce60005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:27:11.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.789+0000 7fce69ffb700 1 -- 172.21.15.114:0/4236990880 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fce5805aac0 con 0x7fce6c074dc0 2024-02-16T22:27:11.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:11.993+0000 7fce72ecd700 1 -- 172.21.15.114:0/4236990880 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fce6c103c70 con 0x7fce5c06d8a0 2024-02-16T22:27:12.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.003+0000 7fce69ffb700 1 -- 172.21.15.114:0/4236990880 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fce6c103c70 con 0x7fce5c06d8a0 2024-02-16T22:27:12.006 INFO:teuthology.orchestra.run.smithi114.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:27:12.006 INFO:teuthology.orchestra.run.smithi114.stdout:alertmanager.smithi114 smithi114 *:9093,9094 running (13m) 39s ago 14m 24.0M - 0.25.0 c8568f914cd2 e7404fea400e 2024-02-16T22:27:12.006 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi114 smithi114 running (14m) 39s ago 14m 8506k - 18.2.1 94eadd990e0a 7fd07162cd96 2024-02-16T22:27:12.006 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi123 smithi123 running (13m) 6s ago 13m 7864k - 18.2.1 94eadd990e0a 7687f5e99621 2024-02-16T22:27:12.006 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi114 smithi114 running (2m) 39s ago 14m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 9b8ae9efe83b 2024-02-16T22:27:12.006 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi123 smithi123 running (2m) 6s ago 13m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 6d97263a0ffa 2024-02-16T22:27:12.006 INFO:teuthology.orchestra.run.smithi114.stdout:grafana.smithi114 smithi114 *:3000 running (12m) 39s ago 14m 88.4M - 9.4.7 954c08fa6188 db1c26c3bc20 2024-02-16T22:27:12.006 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.fdxpey smithi114 running (9m) 39s ago 9m 18.5M - 18.2.1 94eadd990e0a 704b40c144fb 2024-02-16T22:27:12.006 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.zzryke smithi114 running (9m) 39s ago 9m 157M - 18.2.1 94eadd990e0a 1f795d23c9a4 2024-02-16T22:27:12.006 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.gopcfa smithi123 running (8m) 6s ago 8m 18.1M - 18.2.1 94eadd990e0a 166134b7532a 2024-02-16T22:27:12.006 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.vhlqbe smithi123 running (9m) 6s ago 9m 19.8M - 18.2.1 94eadd990e0a 243c8bb96fa8 2024-02-16T22:27:12.007 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi114.vcullz smithi114 *:8443,9283,8765 running (5m) 39s ago 16m 500M - 19.0.0-1450-gf78a58c0 f081f847070a 14ebeb7888c0 2024-02-16T22:27:12.007 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi123.inzgjd smithi123 *:8443,9283,8765 running (3m) 6s ago 13m 450M - 19.0.0-1450-gf78a58c0 f081f847070a f10aab9faf9e 2024-02-16T22:27:12.007 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi114 smithi114 running (3m) 39s ago 16m 51.5M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 58ac06372523 2024-02-16T22:27:12.007 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi123 smithi123 running (3m) 6s ago 13m 35.6M 2048M 19.0.0-1450-gf78a58c0 f081f847070a f76485e0ea24 2024-02-16T22:27:12.007 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi114 smithi114 *:9100 running (14m) 39s ago 14m 15.9M - 1.5.0 0da6a335fe13 744a090e06d0 2024-02-16T22:27:12.007 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi123 smithi123 *:9100 running (13m) 6s ago 13m 15.0M - 1.5.0 0da6a335fe13 3994b8c29b2e 2024-02-16T22:27:12.007 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 smithi114 running (117s) 39s ago 12m 91.2M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 1978fdad94a5 2024-02-16T22:27:12.007 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 smithi114 running (81s) 39s ago 12m 90.2M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 3f90e792898f 2024-02-16T22:27:12.007 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 smithi114 running (45s) 39s ago 11m 12.3M 1216M 19.0.0-1450-gf78a58c0 f081f847070a c663dfa0518b 2024-02-16T22:27:12.007 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 smithi123 running (9s) 6s ago 11m 15.1M 2240M 19.0.0-1450-gf78a58c0 f081f847070a b921448ba169 2024-02-16T22:27:12.007 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 smithi123 running (10m) 6s ago 10m 221M 2240M 18.2.1 94eadd990e0a 7e266866cd9c 2024-02-16T22:27:12.007 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 smithi123 running (10m) 6s ago 10m 200M 2240M 18.2.1 94eadd990e0a 24e5bef19fda 2024-02-16T22:27:12.007 INFO:teuthology.orchestra.run.smithi114.stdout:prometheus.smithi114 smithi114 *:9095 running (4m) 39s ago 14m 57.8M - 2.43.0 a07b618ecd1d 37aac985cdd7 2024-02-16T22:27:12.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.005+0000 7fce72ecd700 1 -- 172.21.15.114:0/4236990880 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fce5c06d8a0 msgr2=0x7fce5c06fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:12.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.005+0000 7fce72ecd700 1 --2- 172.21.15.114:0/4236990880 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fce5c06d8a0 0x7fce5c06fd60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fce60009fd0 tx=0x7fce60005a90 comp rx=0 tx=0).stop 2024-02-16T22:27:12.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.005+0000 7fce72ecd700 1 -- 172.21.15.114:0/4236990880 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce6c074dc0 msgr2=0x7fce6c19cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:12.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.005+0000 7fce72ecd700 1 --2- 172.21.15.114:0/4236990880 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce6c074dc0 0x7fce6c19cea0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fce5800e8f0 tx=0x7fce5800ec00 comp rx=0 tx=0).stop 2024-02-16T22:27:12.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.005+0000 7fce72ecd700 1 -- 172.21.15.114:0/4236990880 shutdown_connections 2024-02-16T22:27:12.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.005+0000 7fce72ecd700 1 --2- 172.21.15.114:0/4236990880 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fce5c06d8a0 0x7fce5c06fd60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:12.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.005+0000 7fce72ecd700 1 --2- 172.21.15.114:0/4236990880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce6c0737f0 0x7fce6c19c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:12.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.005+0000 7fce72ecd700 1 --2- 172.21.15.114:0/4236990880 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce6c074dc0 0x7fce6c19cea0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:12.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.005+0000 7fce72ecd700 1 -- 172.21.15.114:0/4236990880 >> 172.21.15.114:0/4236990880 conn(0x7fce6c0fc140 msgr2=0x7fce6c102550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:27:12.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.006+0000 7fce72ecd700 1 -- 172.21.15.114:0/4236990880 shutdown_connections 2024-02-16T22:27:12.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.006+0000 7fce72ecd700 1 -- 172.21.15.114:0/4236990880 wait complete. 2024-02-16T22:27:12.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.107+0000 7f83da88e700 1 -- 172.21.15.114:0/4211281602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83d4100480 msgr2=0x7f83d41008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:12.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.107+0000 7f83da88e700 1 --2- 172.21.15.114:0/4211281602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83d4100480 0x7f83d41008e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009910 tx=0x7f83c8009c20 comp rx=0 tx=0).stop 2024-02-16T22:27:12.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.108+0000 7f83da88e700 1 -- 172.21.15.114:0/4211281602 shutdown_connections 2024-02-16T22:27:12.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.108+0000 7f83da88e700 1 --2- 172.21.15.114:0/4211281602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83d4100480 0x7f83d41008e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:12.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.108+0000 7f83da88e700 1 --2- 172.21.15.114:0/4211281602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83d40ff280 0x7f83d40ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:12.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.108+0000 7f83da88e700 1 -- 172.21.15.114:0/4211281602 >> 172.21.15.114:0/4211281602 conn(0x7f83d40fa800 msgr2=0x7f83d40fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:27:12.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.108+0000 7f83da88e700 1 -- 172.21.15.114:0/4211281602 shutdown_connections 2024-02-16T22:27:12.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.108+0000 7f83da88e700 1 -- 172.21.15.114:0/4211281602 wait complete. 2024-02-16T22:27:12.111 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:11 smithi114 ceph-mon[94452]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:27:12.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.108+0000 7f83da88e700 1 Processor -- start 2024-02-16T22:27:12.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.108+0000 7f83da88e700 1 -- start start 2024-02-16T22:27:12.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.109+0000 7f83da88e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83d40ff280 0x7f83d4195270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:12.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.109+0000 7f83da88e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83d4100480 0x7f83d41957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:12.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.109+0000 7f83da88e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d4195dd0 con 0x7f83d40ff280 2024-02-16T22:27:12.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.109+0000 7f83da88e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d4195f10 con 0x7f83d4100480 2024-02-16T22:27:12.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.109+0000 7f83d3fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83d40ff280 0x7f83d4195270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:12.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.109+0000 7f83d3fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83d40ff280 0x7f83d4195270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59588/0 (socket says 172.21.15.114:59588) 2024-02-16T22:27:12.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.109+0000 7f83d3fff700 1 -- 172.21.15.114:0/1430142122 learned_addr learned my addr 172.21.15.114:0/1430142122 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:27:12.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.109+0000 7f83d37fe700 1 --2- 172.21.15.114:0/1430142122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83d4100480 0x7f83d41957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:12.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.109+0000 7f83d3fff700 1 -- 172.21.15.114:0/1430142122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83d4100480 msgr2=0x7f83d41957b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:12.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.109+0000 7f83d3fff700 1 --2- 172.21.15.114:0/1430142122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83d4100480 0x7f83d41957b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:12.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.109+0000 7f83d3fff700 1 -- 172.21.15.114:0/1430142122 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c80095f0 con 0x7f83d40ff280 2024-02-16T22:27:12.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.109+0000 7f83d3fff700 1 --2- 172.21.15.114:0/1430142122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83d40ff280 0x7f83d4195270 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f83c400ea80 tx=0x7f83c400ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:27:12.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.109+0000 7f83d17fa700 1 -- 172.21.15.114:0/1430142122 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83c4013070 con 0x7f83d40ff280 2024-02-16T22:27:12.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.109+0000 7f83da88e700 1 -- 172.21.15.114:0/1430142122 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d4101700 con 0x7f83d40ff280 2024-02-16T22:27:12.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.109+0000 7f83d17fa700 1 -- 172.21.15.114:0/1430142122 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f83c400b940 con 0x7f83d40ff280 2024-02-16T22:27:12.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.109+0000 7f83d17fa700 1 -- 172.21.15.114:0/1430142122 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83c4003cc0 con 0x7f83d40ff280 2024-02-16T22:27:12.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.110+0000 7f83da88e700 1 -- 172.21.15.114:0/1430142122 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d4101c50 con 0x7f83d40ff280 2024-02-16T22:27:12.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.111+0000 7f83d17fa700 1 -- 172.21.15.114:0/1430142122 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f83c40041f0 con 0x7f83d40ff280 2024-02-16T22:27:12.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.111+0000 7f83da88e700 1 -- 172.21.15.114:0/1430142122 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83d4107de0 con 0x7f83d40ff280 2024-02-16T22:27:12.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.111+0000 7f83d17fa700 1 --2- 172.21.15.114:0/1430142122 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f83bc06d8a0 0x7f83bc06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:12.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.112+0000 7f83d17fa700 1 -- 172.21.15.114:0/1430142122 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f83c408ed90 con 0x7f83d40ff280 2024-02-16T22:27:12.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.112+0000 7f83d37fe700 1 --2- 172.21.15.114:0/1430142122 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f83bc06d8a0 0x7f83bc06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:12.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.112+0000 7f83d37fe700 1 --2- 172.21.15.114:0/1430142122 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f83bc06d8a0 0x7f83bc06fd60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009fd0 tx=0x7f83c8005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:27:12.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.119+0000 7f83d17fa700 1 -- 172.21.15.114:0/1430142122 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f83c405c120 con 0x7f83d40ff280 2024-02-16T22:27:12.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:11 smithi123 ceph-mon[82331]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:27:12.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.406+0000 7f83da88e700 1 -- 172.21.15.114:0/1430142122 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f83d4102290 con 0x7f83d40ff280 2024-02-16T22:27:12.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.406+0000 7f83d17fa700 1 -- 172.21.15.114:0/1430142122 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f83c405bcb0 con 0x7f83d40ff280 2024-02-16T22:27:12.408 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:27:12.408 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": { 2024-02-16T22:27:12.409 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:27:12.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:27:12.409 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": { 2024-02-16T22:27:12.409 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:27:12.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:27:12.409 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": { 2024-02-16T22:27:12.409 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-02-16T22:27:12.409 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T22:27:12.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:27:12.409 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-02-16T22:27:12.409 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-02-16T22:27:12.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:27:12.409 INFO:teuthology.orchestra.run.smithi114.stdout: "overall": { 2024-02-16T22:27:12.410 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6, 2024-02-16T22:27:12.410 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 8 2024-02-16T22:27:12.410 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-02-16T22:27:12.410 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:27:12.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.408+0000 7f83da88e700 1 -- 172.21.15.114:0/1430142122 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f83bc06d8a0 msgr2=0x7f83bc06fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:12.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.408+0000 7f83da88e700 1 --2- 172.21.15.114:0/1430142122 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f83bc06d8a0 0x7f83bc06fd60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009fd0 tx=0x7f83c8005940 comp rx=0 tx=0).stop 2024-02-16T22:27:12.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.408+0000 7f83da88e700 1 -- 172.21.15.114:0/1430142122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83d40ff280 msgr2=0x7f83d4195270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:12.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.408+0000 7f83da88e700 1 --2- 172.21.15.114:0/1430142122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83d40ff280 0x7f83d4195270 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f83c400ea80 tx=0x7f83c400ed90 comp rx=0 tx=0).stop 2024-02-16T22:27:12.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.408+0000 7f83da88e700 1 -- 172.21.15.114:0/1430142122 shutdown_connections 2024-02-16T22:27:12.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.408+0000 7f83da88e700 1 --2- 172.21.15.114:0/1430142122 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f83bc06d8a0 0x7f83bc06fd60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:12.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.408+0000 7f83da88e700 1 --2- 172.21.15.114:0/1430142122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83d40ff280 0x7f83d4195270 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:12.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.408+0000 7f83da88e700 1 --2- 172.21.15.114:0/1430142122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83d4100480 0x7f83d41957b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:12.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.408+0000 7f83da88e700 1 -- 172.21.15.114:0/1430142122 >> 172.21.15.114:0/1430142122 conn(0x7f83d40fa800 msgr2=0x7f83d40fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:27:12.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.408+0000 7f83da88e700 1 -- 172.21.15.114:0/1430142122 shutdown_connections 2024-02-16T22:27:12.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.408+0000 7f83da88e700 1 -- 172.21.15.114:0/1430142122 wait complete. 2024-02-16T22:27:12.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.512+0000 7f2e2b1ad700 1 -- 172.21.15.114:0/1908285226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e240ffab0 msgr2=0x7f2e240ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:12.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.512+0000 7f2e2b1ad700 1 --2- 172.21.15.114:0/1908285226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e240ffab0 0x7f2e240ffed0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009910 tx=0x7f2e18009c20 comp rx=0 tx=0).stop 2024-02-16T22:27:12.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.512+0000 7f2e2b1ad700 1 -- 172.21.15.114:0/1908285226 shutdown_connections 2024-02-16T22:27:12.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.512+0000 7f2e2b1ad700 1 --2- 172.21.15.114:0/1908285226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e24100cb0 0x7f2e24101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:12.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.512+0000 7f2e2b1ad700 1 --2- 172.21.15.114:0/1908285226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e240ffab0 0x7f2e240ffed0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:12.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.512+0000 7f2e2b1ad700 1 -- 172.21.15.114:0/1908285226 >> 172.21.15.114:0/1908285226 conn(0x7f2e240fb030 msgr2=0x7f2e240fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:27:12.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.512+0000 7f2e2b1ad700 1 -- 172.21.15.114:0/1908285226 shutdown_connections 2024-02-16T22:27:12.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.512+0000 7f2e2b1ad700 1 -- 172.21.15.114:0/1908285226 wait complete. 2024-02-16T22:27:12.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.512+0000 7f2e2b1ad700 1 Processor -- start 2024-02-16T22:27:12.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.513+0000 7f2e2b1ad700 1 -- start start 2024-02-16T22:27:12.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.513+0000 7f2e2b1ad700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e240ffab0 0x7f2e24103a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:12.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.513+0000 7f2e2b1ad700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e24100cb0 0x7f2e24102090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:12.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.513+0000 7f2e2b1ad700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e241025d0 con 0x7f2e24100cb0 2024-02-16T22:27:12.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.513+0000 7f2e2b1ad700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e24102710 con 0x7f2e240ffab0 2024-02-16T22:27:12.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.513+0000 7f2e23fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e24100cb0 0x7f2e24102090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:12.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.513+0000 7f2e23fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e24100cb0 0x7f2e24102090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59604/0 (socket says 172.21.15.114:59604) 2024-02-16T22:27:12.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.513+0000 7f2e23fff700 1 -- 172.21.15.114:0/3118522685 learned_addr learned my addr 172.21.15.114:0/3118522685 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:27:12.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.513+0000 7f2e28f49700 1 --2- 172.21.15.114:0/3118522685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e240ffab0 0x7f2e24103a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:12.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.513+0000 7f2e23fff700 1 -- 172.21.15.114:0/3118522685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e240ffab0 msgr2=0x7f2e24103a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:12.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.513+0000 7f2e23fff700 1 --2- 172.21.15.114:0/3118522685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e240ffab0 0x7f2e24103a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:12.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.513+0000 7f2e23fff700 1 -- 172.21.15.114:0/3118522685 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e180095f0 con 0x7f2e24100cb0 2024-02-16T22:27:12.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.513+0000 7f2e23fff700 1 --2- 172.21.15.114:0/3118522685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e24100cb0 0x7f2e24102090 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000d900 tx=0x7f2e1000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:27:12.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.513+0000 7f2e21ffb700 1 -- 172.21.15.114:0/3118522685 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e100041f0 con 0x7f2e24100cb0 2024-02-16T22:27:12.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.513+0000 7f2e21ffb700 1 -- 172.21.15.114:0/3118522685 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2e10004ce0 con 0x7f2e24100cb0 2024-02-16T22:27:12.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.513+0000 7f2e2b1ad700 1 -- 172.21.15.114:0/3118522685 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e241029f0 con 0x7f2e24100cb0 2024-02-16T22:27:12.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.513+0000 7f2e21ffb700 1 -- 172.21.15.114:0/3118522685 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e1000b750 con 0x7f2e24100cb0 2024-02-16T22:27:12.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.514+0000 7f2e2b1ad700 1 -- 172.21.15.114:0/3118522685 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e2419e380 con 0x7f2e24100cb0 2024-02-16T22:27:12.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.515+0000 7f2e2b1ad700 1 -- 172.21.15.114:0/3118522685 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e24108610 con 0x7f2e24100cb0 2024-02-16T22:27:12.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.515+0000 7f2e21ffb700 1 -- 172.21.15.114:0/3118522685 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f2e10004e50 con 0x7f2e24100cb0 2024-02-16T22:27:12.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.516+0000 7f2e21ffb700 1 --2- 172.21.15.114:0/3118522685 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2e1406d8a0 0x7f2e1406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:12.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.516+0000 7f2e21ffb700 1 -- 172.21.15.114:0/3118522685 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f2e1008d300 con 0x7f2e24100cb0 2024-02-16T22:27:12.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.516+0000 7f2e28f49700 1 --2- 172.21.15.114:0/3118522685 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2e1406d8a0 0x7f2e1406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:12.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.516+0000 7f2e28f49700 1 --2- 172.21.15.114:0/3118522685 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2e1406d8a0 0x7f2e1406fd60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2e18006010 tx=0x7f2e18005ec0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:27:12.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.520+0000 7f2e21ffb700 1 -- 172.21.15.114:0/3118522685 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2e1005a710 con 0x7f2e24100cb0 2024-02-16T22:27:12.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.758+0000 7f2e2b1ad700 1 -- 172.21.15.114:0/3118522685 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2e241087d0 con 0x7f2e24100cb0 2024-02-16T22:27:12.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.758+0000 7f2e21ffb700 1 -- 172.21.15.114:0/3118522685 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1846 (secure 0 0 0) 0x7f2e10016750 con 0x7f2e24100cb0 2024-02-16T22:27:12.760 INFO:teuthology.orchestra.run.smithi114.stdout:e14 2024-02-16T22:27:12.760 INFO:teuthology.orchestra.run.smithi114.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:27:12.761 INFO:teuthology.orchestra.run.smithi114.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:27:12.761 INFO:teuthology.orchestra.run.smithi114.stdout:legacy client fscid: 1 2024-02-16T22:27:12.761 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:27:12.761 INFO:teuthology.orchestra.run.smithi114.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:27:12.761 INFO:teuthology.orchestra.run.smithi114.stdout:fs_name cephfs 2024-02-16T22:27:12.761 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 12 2024-02-16T22:27:12.761 INFO:teuthology.orchestra.run.smithi114.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:27:12.761 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-02-16T22:18:02.108825+0000 2024-02-16T22:27:12.761 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-02-16T22:18:16.413371+0000 2024-02-16T22:27:12.761 INFO:teuthology.orchestra.run.smithi114.stdout:tableserver 0 2024-02-16T22:27:12.761 INFO:teuthology.orchestra.run.smithi114.stdout:root 0 2024-02-16T22:27:12.761 INFO:teuthology.orchestra.run.smithi114.stdout:session_timeout 60 2024-02-16T22:27:12.762 INFO:teuthology.orchestra.run.smithi114.stdout:session_autoclose 300 2024-02-16T22:27:12.762 INFO:teuthology.orchestra.run.smithi114.stdout:max_file_size 1099511627776 2024-02-16T22:27:12.762 INFO:teuthology.orchestra.run.smithi114.stdout:max_xattr_size 65536 2024-02-16T22:27:12.762 INFO:teuthology.orchestra.run.smithi114.stdout:required_client_features {} 2024-02-16T22:27:12.762 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure 0 2024-02-16T22:27:12.762 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure_osd_epoch 0 2024-02-16T22:27:12.762 INFO:teuthology.orchestra.run.smithi114.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:27:12.762 INFO:teuthology.orchestra.run.smithi114.stdout:max_mds 1 2024-02-16T22:27:12.762 INFO:teuthology.orchestra.run.smithi114.stdout:in 0 2024-02-16T22:27:12.762 INFO:teuthology.orchestra.run.smithi114.stdout:up {0=14468} 2024-02-16T22:27:12.762 INFO:teuthology.orchestra.run.smithi114.stdout:failed 2024-02-16T22:27:12.762 INFO:teuthology.orchestra.run.smithi114.stdout:damaged 2024-02-16T22:27:12.762 INFO:teuthology.orchestra.run.smithi114.stdout:stopped 2024-02-16T22:27:12.762 INFO:teuthology.orchestra.run.smithi114.stdout:data_pools [3] 2024-02-16T22:27:12.763 INFO:teuthology.orchestra.run.smithi114.stdout:metadata_pool 2 2024-02-16T22:27:12.763 INFO:teuthology.orchestra.run.smithi114.stdout:inline_data enabled 2024-02-16T22:27:12.763 INFO:teuthology.orchestra.run.smithi114.stdout:balancer 2024-02-16T22:27:12.763 INFO:teuthology.orchestra.run.smithi114.stdout:bal_rank_mask -1 2024-02-16T22:27:12.763 INFO:teuthology.orchestra.run.smithi114.stdout:standby_count_wanted 1 2024-02-16T22:27:12.763 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.zzryke{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.114:6826/277953535,v1:172.21.15.114:6827/277953535] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:27:12.763 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:27:12.763 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:27:12.763 INFO:teuthology.orchestra.run.smithi114.stdout:Standby daemons: 2024-02-16T22:27:12.763 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:27:12.763 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.fdxpey{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:27:12.763 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.vhlqbe{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:27:12.763 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.gopcfa{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:27:12.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.760+0000 7f2e2b1ad700 1 -- 172.21.15.114:0/3118522685 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2e1406d8a0 msgr2=0x7f2e1406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:12.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.760+0000 7f2e2b1ad700 1 --2- 172.21.15.114:0/3118522685 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2e1406d8a0 0x7f2e1406fd60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2e18006010 tx=0x7f2e18005ec0 comp rx=0 tx=0).stop 2024-02-16T22:27:12.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.760+0000 7f2e2b1ad700 1 -- 172.21.15.114:0/3118522685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e24100cb0 msgr2=0x7f2e24102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:12.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.760+0000 7f2e2b1ad700 1 --2- 172.21.15.114:0/3118522685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e24100cb0 0x7f2e24102090 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000d900 tx=0x7f2e1000dc10 comp rx=0 tx=0).stop 2024-02-16T22:27:12.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.760+0000 7f2e2b1ad700 1 -- 172.21.15.114:0/3118522685 shutdown_connections 2024-02-16T22:27:12.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.760+0000 7f2e2b1ad700 1 --2- 172.21.15.114:0/3118522685 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2e1406d8a0 0x7f2e1406fd60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:12.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.760+0000 7f2e2b1ad700 1 --2- 172.21.15.114:0/3118522685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e240ffab0 0x7f2e24103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:12.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.760+0000 7f2e2b1ad700 1 --2- 172.21.15.114:0/3118522685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e24100cb0 0x7f2e24102090 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:12.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.760+0000 7f2e2b1ad700 1 -- 172.21.15.114:0/3118522685 >> 172.21.15.114:0/3118522685 conn(0x7f2e240fb030 msgr2=0x7f2e240683d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:27:12.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.760+0000 7f2e2b1ad700 1 -- 172.21.15.114:0/3118522685 shutdown_connections 2024-02-16T22:27:12.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.760+0000 7f2e2b1ad700 1 -- 172.21.15.114:0/3118522685 wait complete. 2024-02-16T22:27:12.765 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 14 2024-02-16T22:27:12.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.864+0000 7f3132ceb700 1 -- 172.21.15.114:0/3268182556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f312c0ff450 msgr2=0x7f312c101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:12.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.864+0000 7f3132ceb700 1 --2- 172.21.15.114:0/3268182556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f312c0ff450 0x7f312c101880 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f312800b0a0 tx=0x7f312800b3b0 comp rx=0 tx=0).stop 2024-02-16T22:27:12.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.864+0000 7f3132ceb700 1 -- 172.21.15.114:0/3268182556 shutdown_connections 2024-02-16T22:27:12.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.864+0000 7f3132ceb700 1 --2- 172.21.15.114:0/3268182556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f312c101dc0 0x7f312c104250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:12.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.864+0000 7f3132ceb700 1 --2- 172.21.15.114:0/3268182556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f312c0ff450 0x7f312c101880 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:12.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.864+0000 7f3132ceb700 1 -- 172.21.15.114:0/3268182556 >> 172.21.15.114:0/3268182556 conn(0x7f312c0fb050 msgr2=0x7f312c0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:27:12.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.864+0000 7f3132ceb700 1 -- 172.21.15.114:0/3268182556 shutdown_connections 2024-02-16T22:27:12.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.864+0000 7f3132ceb700 1 -- 172.21.15.114:0/3268182556 wait complete. 2024-02-16T22:27:12.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.864+0000 7f3132ceb700 1 Processor -- start 2024-02-16T22:27:12.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.864+0000 7f3132ceb700 1 -- start start 2024-02-16T22:27:12.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.865+0000 7f3132ceb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f312c0ff450 0x7f312c19c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:12.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.865+0000 7f3132ceb700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f312c101dc0 0x7f312c19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:12.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.865+0000 7f3132ceb700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312c19d440 con 0x7f312c101dc0 2024-02-16T22:27:12.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.865+0000 7f3132ceb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312c19d580 con 0x7f312c0ff450 2024-02-16T22:27:12.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.865+0000 7f3123fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f312c101dc0 0x7f312c19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:12.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.865+0000 7f3123fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f312c101dc0 0x7f312c19ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59622/0 (socket says 172.21.15.114:59622) 2024-02-16T22:27:12.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.865+0000 7f3123fff700 1 -- 172.21.15.114:0/3128076502 learned_addr learned my addr 172.21.15.114:0/3128076502 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:27:12.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.865+0000 7f3130a87700 1 --2- 172.21.15.114:0/3128076502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f312c0ff450 0x7f312c19c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:12.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.865+0000 7f3123fff700 1 -- 172.21.15.114:0/3128076502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f312c0ff450 msgr2=0x7f312c19c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:12.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.865+0000 7f3123fff700 1 --2- 172.21.15.114:0/3128076502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f312c0ff450 0x7f312c19c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:12.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.865+0000 7f3123fff700 1 -- 172.21.15.114:0/3128076502 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3128009d00 con 0x7f312c101dc0 2024-02-16T22:27:12.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.865+0000 7f3123fff700 1 --2- 172.21.15.114:0/3128076502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f312c101dc0 0x7f312c19ceb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f311c00c930 tx=0x7f311c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:27:12.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.865+0000 7f3121ffb700 1 -- 172.21.15.114:0/3128076502 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f311c007ad0 con 0x7f312c101dc0 2024-02-16T22:27:12.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.865+0000 7f3121ffb700 1 -- 172.21.15.114:0/3128076502 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f311c00ce80 con 0x7f312c101dc0 2024-02-16T22:27:12.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.865+0000 7f3121ffb700 1 -- 172.21.15.114:0/3128076502 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f311c018630 con 0x7f312c101dc0 2024-02-16T22:27:12.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.865+0000 7f3132ceb700 1 -- 172.21.15.114:0/3128076502 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f312c1a2040 con 0x7f312c101dc0 2024-02-16T22:27:12.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.866+0000 7f3132ceb700 1 -- 172.21.15.114:0/3128076502 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f312c1a2590 con 0x7f312c101dc0 2024-02-16T22:27:12.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.867+0000 7f3132ceb700 1 -- 172.21.15.114:0/3128076502 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f312c196b30 con 0x7f312c101dc0 2024-02-16T22:27:12.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.867+0000 7f3121ffb700 1 -- 172.21.15.114:0/3128076502 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f311c007c30 con 0x7f312c101dc0 2024-02-16T22:27:12.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.867+0000 7f3121ffb700 1 --2- 172.21.15.114:0/3128076502 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f311406d8a0 0x7f311406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:12.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.867+0000 7f3121ffb700 1 -- 172.21.15.114:0/3128076502 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f311c08d460 con 0x7f312c101dc0 2024-02-16T22:27:12.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.867+0000 7f3130a87700 1 --2- 172.21.15.114:0/3128076502 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f311406d8a0 0x7f311406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:12.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.868+0000 7f3130a87700 1 --2- 172.21.15.114:0/3128076502 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f311406d8a0 0x7f311406fd60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3128000f80 tx=0x7f31280060c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:27:12.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:12.872+0000 7f3121ffb700 1 -- 172.21.15.114:0/3128076502 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f311c05a7f0 con 0x7f312c101dc0 2024-02-16T22:27:13.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.087+0000 7f3132ceb700 1 -- 172.21.15.114:0/3128076502 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f312c0611d0 con 0x7f311406d8a0 2024-02-16T22:27:13.089 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:12 smithi114 ceph-mon[94452]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:27:13.090 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:12 smithi114 ceph-mon[94452]: pgmap v126: 65 pgs: 4 peering, 15 active+undersized+degraded, 11 active+undersized, 35 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-02-16T22:27:13.090 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:12 smithi114 ceph-mon[94452]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:27:13.090 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:12 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/1430142122' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:27:13.090 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:12 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/3118522685' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:27:13.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.088+0000 7f3121ffb700 1 -- 172.21.15.114:0/3128076502 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f312c0611d0 con 0x7f311406d8a0 2024-02-16T22:27:13.090 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:27:13.090 INFO:teuthology.orchestra.run.smithi114.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:27:13.091 INFO:teuthology.orchestra.run.smithi114.stdout: "in_progress": true, 2024-02-16T22:27:13.091 INFO:teuthology.orchestra.run.smithi114.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:27:13.091 INFO:teuthology.orchestra.run.smithi114.stdout: "services_complete": [ 2024-02-16T22:27:13.091 INFO:teuthology.orchestra.run.smithi114.stdout: "crash", 2024-02-16T22:27:13.091 INFO:teuthology.orchestra.run.smithi114.stdout: "mon", 2024-02-16T22:27:13.091 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr" 2024-02-16T22:27:13.091 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-02-16T22:27:13.091 INFO:teuthology.orchestra.run.smithi114.stdout: "progress": "10/23 daemons upgraded", 2024-02-16T22:27:13.091 INFO:teuthology.orchestra.run.smithi114.stdout: "message": "Currently upgrading osd daemons", 2024-02-16T22:27:13.091 INFO:teuthology.orchestra.run.smithi114.stdout: "is_paused": false 2024-02-16T22:27:13.092 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:27:13.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.090+0000 7f3132ceb700 1 -- 172.21.15.114:0/3128076502 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f311406d8a0 msgr2=0x7f311406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:13.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.090+0000 7f3132ceb700 1 --2- 172.21.15.114:0/3128076502 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f311406d8a0 0x7f311406fd60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3128000f80 tx=0x7f31280060c0 comp rx=0 tx=0).stop 2024-02-16T22:27:13.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.090+0000 7f3132ceb700 1 -- 172.21.15.114:0/3128076502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f312c101dc0 msgr2=0x7f312c19ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:13.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.090+0000 7f3132ceb700 1 --2- 172.21.15.114:0/3128076502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f312c101dc0 0x7f312c19ceb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f311c00c930 tx=0x7f311c00cc40 comp rx=0 tx=0).stop 2024-02-16T22:27:13.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.090+0000 7f3132ceb700 1 -- 172.21.15.114:0/3128076502 shutdown_connections 2024-02-16T22:27:13.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.090+0000 7f3132ceb700 1 --2- 172.21.15.114:0/3128076502 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f311406d8a0 0x7f311406fd60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:13.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.090+0000 7f3132ceb700 1 --2- 172.21.15.114:0/3128076502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f312c0ff450 0x7f312c19c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:13.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.090+0000 7f3132ceb700 1 --2- 172.21.15.114:0/3128076502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f312c101dc0 0x7f312c19ceb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:13.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.090+0000 7f3132ceb700 1 -- 172.21.15.114:0/3128076502 >> 172.21.15.114:0/3128076502 conn(0x7f312c0fb050 msgr2=0x7f312c0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:27:13.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.090+0000 7f3132ceb700 1 -- 172.21.15.114:0/3128076502 shutdown_connections 2024-02-16T22:27:13.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.090+0000 7f3132ceb700 1 -- 172.21.15.114:0/3128076502 wait complete. 2024-02-16T22:27:13.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.193+0000 7f2ca08c3700 1 -- 172.21.15.114:0/2049085653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c9c100be0 msgr2=0x7f2c9c101000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:13.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.193+0000 7f2ca08c3700 1 --2- 172.21.15.114:0/2049085653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c9c100be0 0x7f2c9c101000 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009910 tx=0x7f2c90009c20 comp rx=0 tx=0).stop 2024-02-16T22:27:13.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.193+0000 7f2ca08c3700 1 -- 172.21.15.114:0/2049085653 shutdown_connections 2024-02-16T22:27:13.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.193+0000 7f2ca08c3700 1 --2- 172.21.15.114:0/2049085653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c9c101de0 0x7f2c9c102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:13.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.193+0000 7f2ca08c3700 1 --2- 172.21.15.114:0/2049085653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c9c100be0 0x7f2c9c101000 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:13.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.193+0000 7f2ca08c3700 1 -- 172.21.15.114:0/2049085653 >> 172.21.15.114:0/2049085653 conn(0x7f2c9c0fc140 msgr2=0x7f2c9c0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:27:13.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.193+0000 7f2ca08c3700 1 -- 172.21.15.114:0/2049085653 shutdown_connections 2024-02-16T22:27:13.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.193+0000 7f2ca08c3700 1 -- 172.21.15.114:0/2049085653 wait complete. 2024-02-16T22:27:13.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.194+0000 7f2ca08c3700 1 Processor -- start 2024-02-16T22:27:13.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.194+0000 7f2ca08c3700 1 -- start start 2024-02-16T22:27:13.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.194+0000 7f2ca08c3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c9c100be0 0x7f2c9c196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:13.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.194+0000 7f2ca08c3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c9c101de0 0x7f2c9c196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:13.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.194+0000 7f2ca08c3700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c196f70 con 0x7f2c9c101de0 2024-02-16T22:27:13.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.194+0000 7f2ca08c3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c1970b0 con 0x7f2c9c100be0 2024-02-16T22:27:13.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.194+0000 7f2c99d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c9c101de0 0x7f2c9c196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:13.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.194+0000 7f2c99d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c9c101de0 0x7f2c9c196950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59630/0 (socket says 172.21.15.114:59630) 2024-02-16T22:27:13.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.194+0000 7f2c99d9b700 1 -- 172.21.15.114:0/4092856041 learned_addr learned my addr 172.21.15.114:0/4092856041 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:27:13.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.194+0000 7f2c9a59c700 1 --2- 172.21.15.114:0/4092856041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c9c100be0 0x7f2c9c196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:13.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.194+0000 7f2c99d9b700 1 -- 172.21.15.114:0/4092856041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c9c100be0 msgr2=0x7f2c9c196410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:13.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.194+0000 7f2c99d9b700 1 --2- 172.21.15.114:0/4092856041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c9c100be0 0x7f2c9c196410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:13.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.194+0000 7f2c99d9b700 1 -- 172.21.15.114:0/4092856041 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c900095f0 con 0x7f2c9c101de0 2024-02-16T22:27:13.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.195+0000 7f2c99d9b700 1 --2- 172.21.15.114:0/4092856041 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c9c101de0 0x7f2c9c196950 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00b4b0 tx=0x7f2c8c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:27:13.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.195+0000 7f2c8b7fe700 1 -- 172.21.15.114:0/4092856041 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c8c011800 con 0x7f2c9c101de0 2024-02-16T22:27:13.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.195+0000 7f2c8b7fe700 1 -- 172.21.15.114:0/4092856041 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2c8c011e10 con 0x7f2c9c101de0 2024-02-16T22:27:13.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.195+0000 7f2ca08c3700 1 -- 172.21.15.114:0/4092856041 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c9c19bb60 con 0x7f2c9c101de0 2024-02-16T22:27:13.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.195+0000 7f2c8b7fe700 1 -- 172.21.15.114:0/4092856041 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c8c00f2e0 con 0x7f2c9c101de0 2024-02-16T22:27:13.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.195+0000 7f2ca08c3700 1 -- 172.21.15.114:0/4092856041 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c9c19c130 con 0x7f2c9c101de0 2024-02-16T22:27:13.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.196+0000 7f2ca08c3700 1 -- 172.21.15.114:0/4092856041 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c9c1097c0 con 0x7f2c9c101de0 2024-02-16T22:27:13.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.196+0000 7f2c8b7fe700 1 -- 172.21.15.114:0/4092856041 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f2c8c011960 con 0x7f2c9c101de0 2024-02-16T22:27:13.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.197+0000 7f2c8b7fe700 1 --2- 172.21.15.114:0/4092856041 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2c8406d8a0 0x7f2c8406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:13.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.197+0000 7f2c8b7fe700 1 -- 172.21.15.114:0/4092856041 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f2c8c08d040 con 0x7f2c9c101de0 2024-02-16T22:27:13.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.197+0000 7f2c9a59c700 1 --2- 172.21.15.114:0/4092856041 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2c8406d8a0 0x7f2c8406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:13.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.197+0000 7f2c9a59c700 1 --2- 172.21.15.114:0/4092856041 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2c8406d8a0 0x7f2c8406fd60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2c90006010 tx=0x7f2c90005280 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:27:13.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.201+0000 7f2c8b7fe700 1 -- 172.21.15.114:0/4092856041 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2c8c05a3d0 con 0x7f2c9c101de0 2024-02-16T22:27:13.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:12 smithi123 ceph-mon[82331]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:27:13.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:12 smithi123 ceph-mon[82331]: pgmap v126: 65 pgs: 4 peering, 15 active+undersized+degraded, 11 active+undersized, 35 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-02-16T22:27:13.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:12 smithi123 ceph-mon[82331]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:27:13.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:12 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/1430142122' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:27:13.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:12 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/3118522685' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:27:13.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.474+0000 7f2ca08c3700 1 -- 172.21.15.114:0/4092856041 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2c9c1099b0 con 0x7f2c9c101de0 2024-02-16T22:27:13.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.475+0000 7f2c8b7fe700 1 -- 172.21.15.114:0/4092856041 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1204 (secure 0 0 0) 0x7f2c8c056030 con 0x7f2c9c101de0 2024-02-16T22:27:13.477 INFO:teuthology.orchestra.run.smithi114.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; Degraded data redundancy: 47/267 objects degraded (17.603%), 15 pgs degraded 2024-02-16T22:27:13.477 INFO:teuthology.orchestra.run.smithi114.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:27:13.477 INFO:teuthology.orchestra.run.smithi114.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:27:13.477 INFO:teuthology.orchestra.run.smithi114.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 47/267 objects degraded (17.603%), 15 pgs degraded 2024-02-16T22:27:13.477 INFO:teuthology.orchestra.run.smithi114.stdout: pg 1.0 is active+undersized+degraded, acting [0,1] 2024-02-16T22:27:13.477 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.0 is active+undersized+degraded, acting [1,0] 2024-02-16T22:27:13.477 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.5 is active+undersized+degraded, acting [0,4] 2024-02-16T22:27:13.477 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.6 is active+undersized+degraded, acting [1,4] 2024-02-16T22:27:13.478 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.7 is active+undersized+degraded, acting [4,2] 2024-02-16T22:27:13.478 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.8 is active+undersized+degraded, acting [5,0] 2024-02-16T22:27:13.478 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.a is active+undersized+degraded, acting [1,4] 2024-02-16T22:27:13.478 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.b is active+undersized+degraded, acting [4,5] 2024-02-16T22:27:13.478 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.c is active+undersized+degraded, acting [2,0] 2024-02-16T22:27:13.478 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.d is active+undersized+degraded, acting [1,2] 2024-02-16T22:27:13.478 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.e is active+undersized+degraded, acting [2,0] 2024-02-16T22:27:13.478 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.14 is active+undersized+degraded, acting [4,5] 2024-02-16T22:27:13.478 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.15 is active+undersized+degraded, acting [1,0] 2024-02-16T22:27:13.478 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.1a is active+undersized+degraded, acting [4,5] 2024-02-16T22:27:13.478 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.1d is active+undersized+degraded, acting [5,0] 2024-02-16T22:27:13.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.477+0000 7f2ca08c3700 1 -- 172.21.15.114:0/4092856041 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2c8406d8a0 msgr2=0x7f2c8406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:13.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.477+0000 7f2ca08c3700 1 --2- 172.21.15.114:0/4092856041 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2c8406d8a0 0x7f2c8406fd60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2c90006010 tx=0x7f2c90005280 comp rx=0 tx=0).stop 2024-02-16T22:27:13.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.477+0000 7f2ca08c3700 1 -- 172.21.15.114:0/4092856041 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c9c101de0 msgr2=0x7f2c9c196950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:13.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.477+0000 7f2ca08c3700 1 --2- 172.21.15.114:0/4092856041 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c9c101de0 0x7f2c9c196950 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00b4b0 tx=0x7f2c8c00b7c0 comp rx=0 tx=0).stop 2024-02-16T22:27:13.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.477+0000 7f2ca08c3700 1 -- 172.21.15.114:0/4092856041 shutdown_connections 2024-02-16T22:27:13.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.477+0000 7f2ca08c3700 1 --2- 172.21.15.114:0/4092856041 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2c8406d8a0 0x7f2c8406fd60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:13.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.477+0000 7f2ca08c3700 1 --2- 172.21.15.114:0/4092856041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c9c100be0 0x7f2c9c196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:13.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.477+0000 7f2ca08c3700 1 --2- 172.21.15.114:0/4092856041 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c9c101de0 0x7f2c9c196950 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:13.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.477+0000 7f2ca08c3700 1 -- 172.21.15.114:0/4092856041 >> 172.21.15.114:0/4092856041 conn(0x7f2c9c0fc140 msgr2=0x7f2c9c105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:27:13.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.477+0000 7f2ca08c3700 1 -- 172.21.15.114:0/4092856041 shutdown_connections 2024-02-16T22:27:13.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:13.477+0000 7f2ca08c3700 1 -- 172.21.15.114:0/4092856041 wait complete. 2024-02-16T22:27:14.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:13 smithi114 ceph-mon[94452]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:27:14.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:13 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/4092856041' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:27:14.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:13 smithi123 ceph-mon[82331]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:27:14.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:13 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/4092856041' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:27:15.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:14 smithi114 ceph-mon[94452]: pgmap v127: 65 pgs: 4 peering, 9 active+undersized+degraded, 7 active+undersized, 45 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 29/267 objects degraded (10.861%) 2024-02-16T22:27:15.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:14 smithi123 ceph-mon[82331]: pgmap v127: 65 pgs: 4 peering, 9 active+undersized+degraded, 7 active+undersized, 45 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 29/267 objects degraded (10.861%) 2024-02-16T22:27:16.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:15 smithi114 ceph-mon[94452]: Health check update: Degraded data redundancy: 29/267 objects degraded (10.861%), 9 pgs degraded (PG_DEGRADED) 2024-02-16T22:27:16.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:15 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:16.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:15 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:27:16.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:15 smithi123 ceph-mon[82331]: Health check update: Degraded data redundancy: 29/267 objects degraded (10.861%), 9 pgs degraded (PG_DEGRADED) 2024-02-16T22:27:16.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:15 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:16.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:15 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:27:17.179 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:16 smithi114 ceph-mon[94452]: pgmap v128: 65 pgs: 4 peering, 61 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:17.179 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:16 smithi114 ceph-mon[94452]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 29/267 objects degraded (10.861%), 9 pgs degraded) 2024-02-16T22:27:17.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:16 smithi123 ceph-mon[82331]: pgmap v128: 65 pgs: 4 peering, 61 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:17.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:16 smithi123 ceph-mon[82331]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 29/267 objects degraded (10.861%), 9 pgs degraded) 2024-02-16T22:27:19.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:18 smithi114 ceph-mon[94452]: pgmap v129: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:19.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:18 smithi123 ceph-mon[82331]: pgmap v129: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:21.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:20 smithi114 ceph-mon[94452]: pgmap v130: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:21.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:20 smithi123 ceph-mon[82331]: pgmap v130: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:23.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:22 smithi114 ceph-mon[94452]: pgmap v131: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:23.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:22 smithi123 ceph-mon[82331]: pgmap v131: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:25.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:24 smithi114 ceph-mon[94452]: pgmap v132: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:25.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:24 smithi123 ceph-mon[82331]: pgmap v132: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:26.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:25 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T22:27:26.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:25 smithi114 ceph-mon[94452]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T22:27:26.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:25 smithi114 ceph-mon[94452]: Upgrade: osd.4 is safe to restart 2024-02-16T22:27:26.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:25 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:26.240 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:25 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T22:27:26.241 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:25 smithi123 ceph-mon[82331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T22:27:26.241 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:25 smithi123 ceph-mon[82331]: Upgrade: osd.4 is safe to restart 2024-02-16T22:27:26.241 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:25 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:27.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:26 smithi123 ceph-mon[82331]: pgmap v133: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:27.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:26 smithi114 ceph-mon[94452]: pgmap v133: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:28.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:28 smithi123 ceph-mon[82331]: Upgrade: Updating osd.4 2024-02-16T22:27:28.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:28 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:28.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:28 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-16T22:27:28.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:28 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:27:28.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:28 smithi123 ceph-mon[82331]: Deploying daemon osd.4 on smithi123 2024-02-16T22:27:28.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:28 smithi114 ceph-mon[94452]: Upgrade: Updating osd.4 2024-02-16T22:27:28.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:28 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:28.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:28 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-16T22:27:28.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:28 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:27:28.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:28 smithi114 ceph-mon[94452]: Deploying daemon osd.4 on smithi123 2024-02-16T22:27:29.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:29 smithi123 ceph-mon[82331]: pgmap v134: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:29.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:29 smithi114 ceph-mon[94452]: pgmap v134: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:30.718 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:30 smithi123 systemd[1]: Stopping Ceph osd.4 for 1dcbaabe-cd18-11ee-95ba-87774f69a715... 2024-02-16T22:27:31.052 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:30 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4[49978]: 2024-02-16T22:27:30.715+0000 7fd5fb709700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T22:27:31.052 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:30 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4[49978]: 2024-02-16T22:27:30.715+0000 7fd5fb709700 -1 osd.4 68 *** Got signal Terminated *** 2024-02-16T22:27:31.052 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:30 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4[49978]: 2024-02-16T22:27:30.715+0000 7fd5fb709700 -1 osd.4 68 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T22:27:31.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:31 smithi114 ceph-mon[94452]: pgmap v135: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:31.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:31 smithi114 ceph-mon[94452]: osd.4 marked itself down and dead 2024-02-16T22:27:31.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:31 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:31.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:31 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:27:31.503 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:31 smithi123 ceph-mon[82331]: pgmap v135: 65 pgs: 65 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:31.503 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:31 smithi123 ceph-mon[82331]: osd.4 marked itself down and dead 2024-02-16T22:27:31.503 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:31 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:31.503 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:31 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:27:31.784 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:31 smithi123 podman[96964]: 2024-02-16 22:27:31.502500746 +0000 UTC m=+0.893400780 container died 7e266866cd9c9de4302e0a373343c54ef434c563c814518817adfb79b5a9c822 (image=quay.io/ceph/ceph@sha256:6176526d5ef4047f148d47c951653dcc6b0de49091fda006d3a088e7a1f404d0, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4, io.buildah.version=1.29.1, GIT_CLEAN=True, RELEASE=HEAD, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-02-16T22:27:32.052 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:31 smithi123 podman[96964]: 2024-02-16 22:27:31.890970314 +0000 UTC m=+1.281870391 container cleanup 7e266866cd9c9de4302e0a373343c54ef434c563c814518817adfb79b5a9c822 (image=quay.io/ceph/ceph@sha256:6176526d5ef4047f148d47c951653dcc6b0de49091fda006d3a088e7a1f404d0, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, RELEASE=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-02-16T22:27:32.052 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:31 smithi123 bash[96964]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4 2024-02-16T22:27:32.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:32 smithi114 ceph-mon[94452]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T22:27:32.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:32 smithi114 ceph-mon[94452]: osdmap e69: 6 total, 5 up, 6 in 2024-02-16T22:27:32.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:32 smithi114 ceph-mon[94452]: osdmap e70: 6 total, 5 up, 6 in 2024-02-16T22:27:32.500 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:32 smithi123 ceph-mon[82331]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T22:27:32.500 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:32 smithi123 ceph-mon[82331]: osdmap e69: 6 total, 5 up, 6 in 2024-02-16T22:27:32.500 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:32 smithi123 ceph-mon[82331]: osdmap e70: 6 total, 5 up, 6 in 2024-02-16T22:27:32.501 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:32 smithi123 podman[96982]: 2024-02-16 22:27:32.142114928 +0000 UTC m=+0.635058110 container remove 7e266866cd9c9de4302e0a373343c54ef434c563c814518817adfb79b5a9c822 (image=quay.io/ceph/ceph@sha256:6176526d5ef4047f148d47c951653dcc6b0de49091fda006d3a088e7a1f404d0, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.1) 2024-02-16T22:27:32.788 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:32 smithi123 podman[97076]: 2024-02-16 22:27:32.402128412 +0000 UTC m=+0.018586692 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T22:27:32.788 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:32 smithi123 podman[97076]: 2024-02-16 22:27:32.526303322 +0000 UTC m=+0.142761603 container create e284f2154a2c1f5f583d28331c96da31bc5cda8a8a6ece4f82f27f91969e2fef (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T22:27:32.788 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:32 smithi123 podman[97076]: 2024-02-16 22:27:32.711937682 +0000 UTC m=+0.328395964 container init e284f2154a2c1f5f583d28331c96da31bc5cda8a8a6ece4f82f27f91969e2fef (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214) 2024-02-16T22:27:32.788 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:32 smithi123 podman[97076]: 2024-02-16 22:27:32.715029939 +0000 UTC m=+0.331488225 container start e284f2154a2c1f5f583d28331c96da31bc5cda8a8a6ece4f82f27f91969e2fef (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4-deactivate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T22:27:32.788 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:32 smithi123 podman[97076]: 2024-02-16 22:27:32.786831124 +0000 UTC m=+0.403289421 container attach e284f2154a2c1f5f583d28331c96da31bc5cda8a8a6ece4f82f27f91969e2fef (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-02-16T22:27:33.052 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:33 smithi123 podman[97076]: 2024-02-16 22:27:33.009388029 +0000 UTC m=+0.625846311 container died e284f2154a2c1f5f583d28331c96da31bc5cda8a8a6ece4f82f27f91969e2fef (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T22:27:33.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:33 smithi114 ceph-mon[94452]: pgmap v137: 65 pgs: 5 peering, 5 stale+active+clean, 55 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:33.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:33 smithi114 ceph-mon[94452]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-16T22:27:33.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:33 smithi123 ceph-mon[82331]: pgmap v137: 65 pgs: 5 peering, 5 stale+active+clean, 55 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:33.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:33 smithi123 ceph-mon[82331]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-16T22:27:34.021 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:33 smithi123 podman[97103]: 2024-02-16 22:27:33.570374752 +0000 UTC m=+0.552740657 container remove e284f2154a2c1f5f583d28331c96da31bc5cda8a8a6ece4f82f27f91969e2fef (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240214) 2024-02-16T22:27:34.021 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:33 smithi123 systemd[1]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@osd.4.service: Deactivated successfully. 2024-02-16T22:27:34.022 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:33 smithi123 systemd[1]: Stopped Ceph osd.4 for 1dcbaabe-cd18-11ee-95ba-87774f69a715. 2024-02-16T22:27:34.022 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:33 smithi123 systemd[1]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@osd.4.service: Consumed 50.397s CPU time. 2024-02-16T22:27:34.022 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:33 smithi123 systemd[1]: Starting Ceph osd.4 for 1dcbaabe-cd18-11ee-95ba-87774f69a715... 2024-02-16T22:27:34.274 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:34 smithi123 podman[97205]: 2024-02-16 22:27:33.923006723 +0000 UTC m=+0.017543886 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T22:27:34.274 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:34 smithi123 podman[97205]: 2024-02-16 22:27:34.033077142 +0000 UTC m=+0.127614295 container create 194799bd25c0dafa06b4fb5ccf949fde70398ef780b6d41511dd23a74c8b2347 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4-activate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-02-16T22:27:34.274 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:34 smithi123 podman[97205]: 2024-02-16 22:27:34.269522316 +0000 UTC m=+0.364059483 container init 194799bd25c0dafa06b4fb5ccf949fde70398ef780b6d41511dd23a74c8b2347 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4-activate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T22:27:34.274 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:34 smithi123 podman[97205]: 2024-02-16 22:27:34.272880533 +0000 UTC m=+0.367417696 container start 194799bd25c0dafa06b4fb5ccf949fde70398ef780b6d41511dd23a74c8b2347 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4-activate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T22:27:34.552 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:34 smithi123 podman[97205]: 2024-02-16 22:27:34.368871711 +0000 UTC m=+0.463408880 container attach 194799bd25c0dafa06b4fb5ccf949fde70398ef780b6d41511dd23a74c8b2347 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4-activate, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T22:27:35.023 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:34 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4-activate[97219]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-16T22:27:35.023 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:34 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4-activate[97219]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-02-16T22:27:35.023 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:34 smithi123 bash[97205]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-16T22:27:35.023 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:34 smithi123 bash[97205]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-02-16T22:27:35.023 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:34 smithi123 bash[97205]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-02-16T22:27:35.023 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:34 smithi123 bash[97205]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-02-16T22:27:35.023 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:34 smithi123 bash[97205]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-02-16T22:27:35.023 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:34 smithi123 bash[97205]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-16T22:27:35.023 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:34 smithi123 bash[97205]: --> ceph-volume raw activate successful for osd ID: 4 2024-02-16T22:27:35.023 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:34 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4-activate[97219]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-02-16T22:27:35.024 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:34 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4-activate[97219]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-02-16T22:27:35.024 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:34 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4-activate[97219]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-02-16T22:27:35.024 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:34 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4-activate[97219]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-16T22:27:35.024 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:34 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4-activate[97219]: --> ceph-volume raw activate successful for osd ID: 4 2024-02-16T22:27:35.024 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:34 smithi123 podman[97205]: 2024-02-16 22:27:34.740125277 +0000 UTC m=+0.834662440 container died 194799bd25c0dafa06b4fb5ccf949fde70398ef780b6d41511dd23a74c8b2347 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4-activate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-02-16T22:27:35.298 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:35 smithi123 ceph-mon[82331]: pgmap v139: 65 pgs: 15 peering, 3 stale+active+clean, 47 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:35.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:35 smithi114 ceph-mon[94452]: pgmap v139: 65 pgs: 15 peering, 3 stale+active+clean, 47 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:35.552 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:35 smithi123 podman[97356]: 2024-02-16 22:27:35.29703837 +0000 UTC m=+0.552119776 container remove 194799bd25c0dafa06b4fb5ccf949fde70398ef780b6d41511dd23a74c8b2347 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4-activate, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS) 2024-02-16T22:27:35.552 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:35 smithi123 podman[97417]: 2024-02-16 22:27:35.411483453 +0000 UTC m=+0.018028184 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T22:27:35.553 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:35 smithi123 podman[97417]: 2024-02-16 22:27:35.514384051 +0000 UTC m=+0.120928786 container create 54c2926d56b18cbda791e1f15b658ed259633a5b9fcaecf2fa72a1a4790b4958 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-02-16T22:27:36.052 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:35 smithi123 podman[97417]: 2024-02-16 22:27:35.725331558 +0000 UTC m=+0.331876290 container init 54c2926d56b18cbda791e1f15b658ed259633a5b9fcaecf2fa72a1a4790b4958 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T22:27:36.052 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:35 smithi123 podman[97417]: 2024-02-16 22:27:35.728416284 +0000 UTC m=+0.334961016 container start 54c2926d56b18cbda791e1f15b658ed259633a5b9fcaecf2fa72a1a4790b4958 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T22:27:36.052 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:35 smithi123 ceph-osd[97436]: -- 172.21.15.123:0/3098296229 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x565107ffb1e0 con 0x565107fccc00 2024-02-16T22:27:36.052 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:35 smithi123 bash[97417]: 54c2926d56b18cbda791e1f15b658ed259633a5b9fcaecf2fa72a1a4790b4958 2024-02-16T22:27:36.053 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:35 smithi123 systemd[1]: Started Ceph osd.4 for 1dcbaabe-cd18-11ee-95ba-87774f69a715. 2024-02-16T22:27:36.552 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:36 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4[97432]: 2024-02-16T22:27:36.273+0000 7f7484b507c0 -1 Falling back to public interface 2024-02-16T22:27:37.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:36 smithi123 ceph-mon[82331]: pgmap v140: 65 pgs: 5 active+undersized+degraded, 15 peering, 2 stale+active+clean, 6 active+undersized, 37 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 22/267 objects degraded (8.240%) 2024-02-16T22:27:37.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:37.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:37.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:27:37.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:36 smithi123 ceph-mon[82331]: Health check failed: Degraded data redundancy: 22/267 objects degraded (8.240%), 5 pgs degraded (PG_DEGRADED) 2024-02-16T22:27:37.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:36 smithi114 ceph-mon[94452]: pgmap v140: 65 pgs: 5 active+undersized+degraded, 15 peering, 2 stale+active+clean, 6 active+undersized, 37 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 22/267 objects degraded (8.240%) 2024-02-16T22:27:37.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:37.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:37.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:27:37.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:36 smithi114 ceph-mon[94452]: Health check failed: Degraded data redundancy: 22/267 objects degraded (8.240%), 5 pgs degraded (PG_DEGRADED) 2024-02-16T22:27:38.193 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:37 smithi123 ceph-mon[82331]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-02-16T22:27:38.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:37 smithi114 ceph-mon[94452]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-02-16T22:27:39.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:38 smithi114 ceph-mon[94452]: pgmap v141: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 46/267 objects degraded (17.228%) 2024-02-16T22:27:39.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:38 smithi123 ceph-mon[82331]: pgmap v141: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 46/267 objects degraded (17.228%) 2024-02-16T22:27:40.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:39 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:40.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:39 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:40.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:39 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:40.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:39 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:40.908 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:40 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4[97432]: 2024-02-16T22:27:40.825+0000 7f7484b507c0 -1 osd.4 0 read_superblock omap replica is missing. 2024-02-16T22:27:41.180 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:41 smithi123 ceph-mon[82331]: pgmap v142: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 46/267 objects degraded (17.228%) 2024-02-16T22:27:41.180 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:41 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:41.180 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:41 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:41.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:41 smithi114 ceph-mon[94452]: pgmap v142: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 46/267 objects degraded (17.228%) 2024-02-16T22:27:41.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:41 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:41.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:41 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:41.552 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:41 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4[97432]: 2024-02-16T22:27:41.185+0000 7f7484b507c0 -1 osd.4 68 log_to_monitors true 2024-02-16T22:27:42.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:42 smithi123 ceph-mon[82331]: from='osd.4 [v2:172.21.15.123:6808/75423100,v1:172.21.15.123:6809/75423100]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T22:27:42.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:42 smithi123 ceph-mon[82331]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T22:27:42.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:42 smithi114 ceph-mon[94452]: from='osd.4 [v2:172.21.15.123:6808/75423100,v1:172.21.15.123:6809/75423100]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T22:27:42.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:42 smithi114 ceph-mon[94452]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T22:27:43.297 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:27:43 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4[97432]: 2024-02-16T22:27:43.033+0000 7f7471fa0700 -1 osd.4 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T22:27:43.298 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:43 smithi123 ceph-mon[82331]: pgmap v143: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 46/267 objects degraded (17.228%) 2024-02-16T22:27:43.298 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:43 smithi123 ceph-mon[82331]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-16T22:27:43.298 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:43 smithi123 ceph-mon[82331]: osdmap e71: 6 total, 5 up, 6 in 2024-02-16T22:27:43.298 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:43 smithi123 ceph-mon[82331]: from='osd.4 [v2:172.21.15.123:6808/75423100,v1:172.21.15.123:6809/75423100]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-02-16T22:27:43.298 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:43 smithi123 ceph-mon[82331]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-02-16T22:27:43.298 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:43 smithi123 ceph-mon[82331]: from='osd.4 ' entity='osd.4' 2024-02-16T22:27:43.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:43 smithi114 ceph-mon[94452]: pgmap v143: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 46/267 objects degraded (17.228%) 2024-02-16T22:27:43.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:43 smithi114 ceph-mon[94452]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-16T22:27:43.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:43 smithi114 ceph-mon[94452]: osdmap e71: 6 total, 5 up, 6 in 2024-02-16T22:27:43.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:43 smithi114 ceph-mon[94452]: from='osd.4 [v2:172.21.15.123:6808/75423100,v1:172.21.15.123:6809/75423100]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-02-16T22:27:43.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:43 smithi114 ceph-mon[94452]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-02-16T22:27:43.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:43 smithi114 ceph-mon[94452]: from='osd.4 ' entity='osd.4' 2024-02-16T22:27:43.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.594+0000 7fa150c51700 1 -- 172.21.15.114:0/3521679765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa14c101db0 msgr2=0x7fa14c102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:43.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.594+0000 7fa150c51700 1 --2- 172.21.15.114:0/3521679765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa14c101db0 0x7fa14c102210 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa140009910 tx=0x7fa140009c20 comp rx=0 tx=0).stop 2024-02-16T22:27:43.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.594+0000 7fa150c51700 1 -- 172.21.15.114:0/3521679765 shutdown_connections 2024-02-16T22:27:43.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.594+0000 7fa150c51700 1 --2- 172.21.15.114:0/3521679765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa14c101db0 0x7fa14c102210 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:43.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.594+0000 7fa150c51700 1 --2- 172.21.15.114:0/3521679765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa14c100bb0 0x7fa14c100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:43.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.594+0000 7fa150c51700 1 -- 172.21.15.114:0/3521679765 >> 172.21.15.114:0/3521679765 conn(0x7fa14c0fc110 msgr2=0x7fa14c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:27:43.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.594+0000 7fa150c51700 1 -- 172.21.15.114:0/3521679765 shutdown_connections 2024-02-16T22:27:43.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.594+0000 7fa150c51700 1 -- 172.21.15.114:0/3521679765 wait complete. 2024-02-16T22:27:43.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.595+0000 7fa150c51700 1 Processor -- start 2024-02-16T22:27:43.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.595+0000 7fa150c51700 1 -- start start 2024-02-16T22:27:43.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.595+0000 7fa150c51700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa14c100bb0 0x7fa14c074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:43.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.595+0000 7fa150c51700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa14c101db0 0x7fa14c073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:43.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.595+0000 7fa150c51700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c0736c0 con 0x7fa14c101db0 2024-02-16T22:27:43.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.595+0000 7fa150c51700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c073800 con 0x7fa14c100bb0 2024-02-16T22:27:43.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.595+0000 7fa149d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa14c101db0 0x7fa14c073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:43.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.595+0000 7fa149d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa14c101db0 0x7fa14c073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48660/0 (socket says 172.21.15.114:48660) 2024-02-16T22:27:43.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.595+0000 7fa149d9b700 1 -- 172.21.15.114:0/405649697 learned_addr learned my addr 172.21.15.114:0/405649697 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:27:43.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.595+0000 7fa14a59c700 1 --2- 172.21.15.114:0/405649697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa14c100bb0 0x7fa14c074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:43.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.595+0000 7fa149d9b700 1 -- 172.21.15.114:0/405649697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa14c100bb0 msgr2=0x7fa14c074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:43.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.595+0000 7fa149d9b700 1 --2- 172.21.15.114:0/405649697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa14c100bb0 0x7fa14c074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:43.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.595+0000 7fa149d9b700 1 -- 172.21.15.114:0/405649697 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1400095f0 con 0x7fa14c101db0 2024-02-16T22:27:43.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.596+0000 7fa149d9b700 1 --2- 172.21.15.114:0/405649697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa14c101db0 0x7fa14c073180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa140006010 tx=0x7fa14000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:27:43.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.596+0000 7fa13b7fe700 1 -- 172.21.15.114:0/405649697 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa14001d070 con 0x7fa14c101db0 2024-02-16T22:27:43.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.596+0000 7fa13b7fe700 1 -- 172.21.15.114:0/405649697 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa14000f460 con 0x7fa14c101db0 2024-02-16T22:27:43.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.596+0000 7fa150c51700 1 -- 172.21.15.114:0/405649697 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa14c073a80 con 0x7fa14c101db0 2024-02-16T22:27:43.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.596+0000 7fa13b7fe700 1 -- 172.21.15.114:0/405649697 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa140021640 con 0x7fa14c101db0 2024-02-16T22:27:43.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.596+0000 7fa150c51700 1 -- 172.21.15.114:0/405649697 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa14c073f70 con 0x7fa14c101db0 2024-02-16T22:27:43.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.597+0000 7fa150c51700 1 -- 172.21.15.114:0/405649697 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa14c109710 con 0x7fa14c101db0 2024-02-16T22:27:43.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.597+0000 7fa13b7fe700 1 -- 172.21.15.114:0/405649697 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fa14000fab0 con 0x7fa14c101db0 2024-02-16T22:27:43.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.598+0000 7fa13b7fe700 1 --2- 172.21.15.114:0/405649697 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fa13406d8a0 0x7fa13406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:43.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.598+0000 7fa13b7fe700 1 -- 172.21.15.114:0/405649697 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fa14008e7b0 con 0x7fa14c101db0 2024-02-16T22:27:43.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.598+0000 7fa14a59c700 1 --2- 172.21.15.114:0/405649697 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fa13406d8a0 0x7fa13406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:43.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.598+0000 7fa14a59c700 1 --2- 172.21.15.114:0/405649697 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fa13406d8a0 0x7fa13406fd60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa14c101c10 tx=0x7fa13c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:27:43.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.603+0000 7fa13b7fe700 1 -- 172.21.15.114:0/405649697 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa14005bb40 con 0x7fa14c101db0 2024-02-16T22:27:43.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.837+0000 7fa150c51700 1 -- 172.21.15.114:0/405649697 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa14c106700 con 0x7fa13406d8a0 2024-02-16T22:27:43.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.839+0000 7fa13b7fe700 1 -- 172.21.15.114:0/405649697 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fa14c106700 con 0x7fa13406d8a0 2024-02-16T22:27:43.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.841+0000 7fa150c51700 1 -- 172.21.15.114:0/405649697 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fa13406d8a0 msgr2=0x7fa13406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:43.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.841+0000 7fa150c51700 1 --2- 172.21.15.114:0/405649697 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fa13406d8a0 0x7fa13406fd60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa14c101c10 tx=0x7fa13c00b410 comp rx=0 tx=0).stop 2024-02-16T22:27:43.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.841+0000 7fa150c51700 1 -- 172.21.15.114:0/405649697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa14c101db0 msgr2=0x7fa14c073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:43.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.841+0000 7fa150c51700 1 --2- 172.21.15.114:0/405649697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa14c101db0 0x7fa14c073180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa140006010 tx=0x7fa14000bd70 comp rx=0 tx=0).stop 2024-02-16T22:27:43.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.841+0000 7fa150c51700 1 -- 172.21.15.114:0/405649697 shutdown_connections 2024-02-16T22:27:43.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.841+0000 7fa150c51700 1 --2- 172.21.15.114:0/405649697 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fa13406d8a0 0x7fa13406fd60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:43.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.841+0000 7fa150c51700 1 --2- 172.21.15.114:0/405649697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa14c100bb0 0x7fa14c074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:43.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.841+0000 7fa150c51700 1 --2- 172.21.15.114:0/405649697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa14c101db0 0x7fa14c073180 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:43.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.841+0000 7fa150c51700 1 -- 172.21.15.114:0/405649697 >> 172.21.15.114:0/405649697 conn(0x7fa14c0fc110 msgr2=0x7fa14c104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:27:43.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.841+0000 7fa150c51700 1 -- 172.21.15.114:0/405649697 shutdown_connections 2024-02-16T22:27:43.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.841+0000 7fa150c51700 1 -- 172.21.15.114:0/405649697 wait complete. 2024-02-16T22:27:43.853 INFO:teuthology.orchestra.run.smithi114.stdout:true 2024-02-16T22:27:43.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.944+0000 7f24f703d700 1 -- 172.21.15.114:0/2861999664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24f0100cc0 msgr2=0x7f24f0101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:43.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.944+0000 7f24f703d700 1 --2- 172.21.15.114:0/2861999664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24f0100cc0 0x7f24f0101120 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f24dc009910 tx=0x7f24dc009c20 comp rx=0 tx=0).stop 2024-02-16T22:27:43.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.944+0000 7f24f703d700 1 -- 172.21.15.114:0/2861999664 shutdown_connections 2024-02-16T22:27:43.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.945+0000 7f24f703d700 1 --2- 172.21.15.114:0/2861999664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24f0100cc0 0x7f24f0101120 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:43.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.945+0000 7f24f703d700 1 --2- 172.21.15.114:0/2861999664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24f00ffac0 0x7f24f00ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:43.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.945+0000 7f24f703d700 1 -- 172.21.15.114:0/2861999664 >> 172.21.15.114:0/2861999664 conn(0x7f24f00fb060 msgr2=0x7f24f00fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:27:43.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.945+0000 7f24f703d700 1 -- 172.21.15.114:0/2861999664 shutdown_connections 2024-02-16T22:27:43.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.945+0000 7f24f703d700 1 -- 172.21.15.114:0/2861999664 wait complete. 2024-02-16T22:27:43.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.945+0000 7f24f703d700 1 Processor -- start 2024-02-16T22:27:43.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.945+0000 7f24f703d700 1 -- start start 2024-02-16T22:27:43.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.945+0000 7f24f703d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24f00ffac0 0x7f24f0196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:43.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.945+0000 7f24f703d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24f0100cc0 0x7f24f01968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:43.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.945+0000 7f24f703d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f0196ed0 con 0x7f24f00ffac0 2024-02-16T22:27:43.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.945+0000 7f24f703d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f0197010 con 0x7f24f0100cc0 2024-02-16T22:27:43.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.945+0000 7f24f4dd9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24f00ffac0 0x7f24f0196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:43.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.945+0000 7f24f4dd9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24f00ffac0 0x7f24f0196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48674/0 (socket says 172.21.15.114:48674) 2024-02-16T22:27:43.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.945+0000 7f24f4dd9700 1 -- 172.21.15.114:0/1418031414 learned_addr learned my addr 172.21.15.114:0/1418031414 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:27:43.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.946+0000 7f24effff700 1 --2- 172.21.15.114:0/1418031414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24f0100cc0 0x7f24f01968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:43.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.946+0000 7f24f4dd9700 1 -- 172.21.15.114:0/1418031414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24f0100cc0 msgr2=0x7f24f01968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:43.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.946+0000 7f24f4dd9700 1 --2- 172.21.15.114:0/1418031414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24f0100cc0 0x7f24f01968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:43.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.946+0000 7f24f4dd9700 1 -- 172.21.15.114:0/1418031414 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24dc0095f0 con 0x7f24f00ffac0 2024-02-16T22:27:43.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.946+0000 7f24f4dd9700 1 --2- 172.21.15.114:0/1418031414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24f00ffac0 0x7f24f0196370 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f24e400e8f0 tx=0x7f24e400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:27:43.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.946+0000 7f24edffb700 1 -- 172.21.15.114:0/1418031414 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24e4009790 con 0x7f24f00ffac0 2024-02-16T22:27:43.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.946+0000 7f24f703d700 1 -- 172.21.15.114:0/1418031414 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f019bac0 con 0x7f24f00ffac0 2024-02-16T22:27:43.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.946+0000 7f24edffb700 1 -- 172.21.15.114:0/1418031414 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f24e400c810 con 0x7f24f00ffac0 2024-02-16T22:27:43.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.946+0000 7f24edffb700 1 -- 172.21.15.114:0/1418031414 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24e4003d20 con 0x7f24f00ffac0 2024-02-16T22:27:43.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.946+0000 7f24f703d700 1 -- 172.21.15.114:0/1418031414 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f019c010 con 0x7f24f00ffac0 2024-02-16T22:27:43.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.947+0000 7f24f703d700 1 -- 172.21.15.114:0/1418031414 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f0108620 con 0x7f24f00ffac0 2024-02-16T22:27:43.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.948+0000 7f24edffb700 1 -- 172.21.15.114:0/1418031414 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f24e40041f0 con 0x7f24f00ffac0 2024-02-16T22:27:43.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.948+0000 7f24edffb700 1 --2- 172.21.15.114:0/1418031414 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f24e006d8a0 0x7f24e006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:43.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.948+0000 7f24effff700 1 --2- 172.21.15.114:0/1418031414 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f24e006d8a0 0x7f24e006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:43.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.948+0000 7f24edffb700 1 -- 172.21.15.114:0/1418031414 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f24e4014070 con 0x7f24f00ffac0 2024-02-16T22:27:43.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.949+0000 7f24effff700 1 --2- 172.21.15.114:0/1418031414 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f24e006d8a0 0x7f24e006fd60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f24dc009fd0 tx=0x7f24dc005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:27:43.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:43.953+0000 7f24edffb700 1 -- 172.21.15.114:0/1418031414 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f24e405ab00 con 0x7f24f00ffac0 2024-02-16T22:27:44.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.174+0000 7f24f703d700 1 -- 172.21.15.114:0/1418031414 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f24f00694c0 con 0x7f24e006d8a0 2024-02-16T22:27:44.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.175+0000 7f24edffb700 1 -- 172.21.15.114:0/1418031414 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f24f00694c0 con 0x7f24e006d8a0 2024-02-16T22:27:44.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.178+0000 7f24f703d700 1 -- 172.21.15.114:0/1418031414 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f24e006d8a0 msgr2=0x7f24e006fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:44.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.178+0000 7f24f703d700 1 --2- 172.21.15.114:0/1418031414 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f24e006d8a0 0x7f24e006fd60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f24dc009fd0 tx=0x7f24dc005fd0 comp rx=0 tx=0).stop 2024-02-16T22:27:44.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.178+0000 7f24f703d700 1 -- 172.21.15.114:0/1418031414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24f00ffac0 msgr2=0x7f24f0196370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:44.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.178+0000 7f24f703d700 1 --2- 172.21.15.114:0/1418031414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24f00ffac0 0x7f24f0196370 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f24e400e8f0 tx=0x7f24e400ec00 comp rx=0 tx=0).stop 2024-02-16T22:27:44.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.178+0000 7f24f703d700 1 -- 172.21.15.114:0/1418031414 shutdown_connections 2024-02-16T22:27:44.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.178+0000 7f24f703d700 1 --2- 172.21.15.114:0/1418031414 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f24e006d8a0 0x7f24e006fd60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:44.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.178+0000 7f24f703d700 1 --2- 172.21.15.114:0/1418031414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24f00ffac0 0x7f24f0196370 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:44.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.178+0000 7f24f703d700 1 --2- 172.21.15.114:0/1418031414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24f0100cc0 0x7f24f01968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:44.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.178+0000 7f24f703d700 1 -- 172.21.15.114:0/1418031414 >> 172.21.15.114:0/1418031414 conn(0x7f24f00fb060 msgr2=0x7f24f0068db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:27:44.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.178+0000 7f24f703d700 1 -- 172.21.15.114:0/1418031414 shutdown_connections 2024-02-16T22:27:44.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.178+0000 7f24f703d700 1 -- 172.21.15.114:0/1418031414 wait complete. 2024-02-16T22:27:44.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.280+0000 7fad274de700 1 -- 172.21.15.114:0/1737583556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad200fec30 msgr2=0x7fad200ff090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:44.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.280+0000 7fad274de700 1 --2- 172.21.15.114:0/1737583556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad200fec30 0x7fad200ff090 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fad10009910 tx=0x7fad10009c20 comp rx=0 tx=0).stop 2024-02-16T22:27:44.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.280+0000 7fad274de700 1 -- 172.21.15.114:0/1737583556 shutdown_connections 2024-02-16T22:27:44.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.280+0000 7fad274de700 1 --2- 172.21.15.114:0/1737583556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad200fec30 0x7fad200ff090 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:44.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.280+0000 7fad274de700 1 --2- 172.21.15.114:0/1737583556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad201021b0 0x7fad201025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:44.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.280+0000 7fad274de700 1 -- 172.21.15.114:0/1737583556 >> 172.21.15.114:0/1737583556 conn(0x7fad200fa7f0 msgr2=0x7fad200fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:27:44.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.281+0000 7fad274de700 1 -- 172.21.15.114:0/1737583556 shutdown_connections 2024-02-16T22:27:44.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.281+0000 7fad274de700 1 -- 172.21.15.114:0/1737583556 wait complete. 2024-02-16T22:27:44.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.281+0000 7fad274de700 1 Processor -- start 2024-02-16T22:27:44.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.281+0000 7fad274de700 1 -- start start 2024-02-16T22:27:44.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.281+0000 7fad274de700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad200fec30 0x7fad201963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:44.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.281+0000 7fad274de700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad201021b0 0x7fad201968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:44.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.281+0000 7fad274de700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad20196f10 con 0x7fad200fec30 2024-02-16T22:27:44.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.281+0000 7fad274de700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad20197050 con 0x7fad201021b0 2024-02-16T22:27:44.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.281+0000 7fad2527a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad200fec30 0x7fad201963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:44.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.281+0000 7fad2527a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad200fec30 0x7fad201963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48694/0 (socket says 172.21.15.114:48694) 2024-02-16T22:27:44.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.281+0000 7fad2527a700 1 -- 172.21.15.114:0/249362242 learned_addr learned my addr 172.21.15.114:0/249362242 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:27:44.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.281+0000 7fad24a79700 1 --2- 172.21.15.114:0/249362242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad201021b0 0x7fad201968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:44.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.282+0000 7fad2527a700 1 -- 172.21.15.114:0/249362242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad201021b0 msgr2=0x7fad201968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:44.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.282+0000 7fad2527a700 1 --2- 172.21.15.114:0/249362242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad201021b0 0x7fad201968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:44.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.282+0000 7fad2527a700 1 -- 172.21.15.114:0/249362242 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad100095f0 con 0x7fad200fec30 2024-02-16T22:27:44.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.282+0000 7fad2527a700 1 --2- 172.21.15.114:0/249362242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad200fec30 0x7fad201963b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fad1c00d900 tx=0x7fad1c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:27:44.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.282+0000 7fad167fc700 1 -- 172.21.15.114:0/249362242 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad1c0041f0 con 0x7fad200fec30 2024-02-16T22:27:44.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.282+0000 7fad167fc700 1 -- 172.21.15.114:0/249362242 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fad1c004350 con 0x7fad200fec30 2024-02-16T22:27:44.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.282+0000 7fad274de700 1 -- 172.21.15.114:0/249362242 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad2019bb00 con 0x7fad200fec30 2024-02-16T22:27:44.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.282+0000 7fad167fc700 1 -- 172.21.15.114:0/249362242 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad1c003d20 con 0x7fad200fec30 2024-02-16T22:27:44.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.282+0000 7fad274de700 1 -- 172.21.15.114:0/249362242 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad2019c050 con 0x7fad200fec30 2024-02-16T22:27:44.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.283+0000 7fad274de700 1 -- 172.21.15.114:0/249362242 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad20109740 con 0x7fad200fec30 2024-02-16T22:27:44.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.284+0000 7fad167fc700 1 -- 172.21.15.114:0/249362242 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fad1c009730 con 0x7fad200fec30 2024-02-16T22:27:44.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.284+0000 7fad167fc700 1 --2- 172.21.15.114:0/249362242 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fad0c06d8a0 0x7fad0c06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:44.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.284+0000 7fad167fc700 1 -- 172.21.15.114:0/249362242 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fad1c08d2c0 con 0x7fad200fec30 2024-02-16T22:27:44.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.284+0000 7fad24a79700 1 --2- 172.21.15.114:0/249362242 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fad0c06d8a0 0x7fad0c06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:44.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.284+0000 7fad24a79700 1 --2- 172.21.15.114:0/249362242 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fad0c06d8a0 0x7fad0c06fd60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fad10006010 tx=0x7fad1000b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:27:44.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.289+0000 7fad167fc700 1 -- 172.21.15.114:0/249362242 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fad1c05a6d0 con 0x7fad200fec30 2024-02-16T22:27:44.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.490+0000 7fad274de700 1 -- 172.21.15.114:0/249362242 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fad201068a0 con 0x7fad0c06d8a0 2024-02-16T22:27:44.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.501+0000 7fad167fc700 1 -- 172.21.15.114:0/249362242 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fad201068a0 con 0x7fad0c06d8a0 2024-02-16T22:27:44.503 INFO:teuthology.orchestra.run.smithi114.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:27:44.503 INFO:teuthology.orchestra.run.smithi114.stdout:alertmanager.smithi114 smithi114 *:9093,9094 running (13m) 72s ago 15m 24.0M - 0.25.0 c8568f914cd2 e7404fea400e 2024-02-16T22:27:44.503 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi114 smithi114 running (15m) 72s ago 15m 8506k - 18.2.1 94eadd990e0a 7fd07162cd96 2024-02-16T22:27:44.504 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi123 smithi123 running (14m) 5s ago 14m 8120k - 18.2.1 94eadd990e0a 7687f5e99621 2024-02-16T22:27:44.504 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi114 smithi114 running (2m) 72s ago 15m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 9b8ae9efe83b 2024-02-16T22:27:44.504 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi123 smithi123 running (2m) 5s ago 14m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 6d97263a0ffa 2024-02-16T22:27:44.504 INFO:teuthology.orchestra.run.smithi114.stdout:grafana.smithi114 smithi114 *:3000 running (13m) 72s ago 14m 88.4M - 9.4.7 954c08fa6188 db1c26c3bc20 2024-02-16T22:27:44.504 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.fdxpey smithi114 running (9m) 72s ago 9m 18.5M - 18.2.1 94eadd990e0a 704b40c144fb 2024-02-16T22:27:44.504 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.zzryke smithi114 running (9m) 72s ago 9m 157M - 18.2.1 94eadd990e0a 1f795d23c9a4 2024-02-16T22:27:44.504 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.gopcfa smithi123 running (9m) 5s ago 9m 18.4M - 18.2.1 94eadd990e0a 166134b7532a 2024-02-16T22:27:44.504 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.vhlqbe smithi123 running (9m) 5s ago 9m 20.0M - 18.2.1 94eadd990e0a 243c8bb96fa8 2024-02-16T22:27:44.504 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi114.vcullz smithi114 *:8443,9283,8765 running (5m) 72s ago 16m 500M - 19.0.0-1450-gf78a58c0 f081f847070a 14ebeb7888c0 2024-02-16T22:27:44.504 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi123.inzgjd smithi123 *:8443,9283,8765 running (4m) 5s ago 14m 450M - 19.0.0-1450-gf78a58c0 f081f847070a f10aab9faf9e 2024-02-16T22:27:44.504 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi114 smithi114 running (4m) 72s ago 16m 51.5M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 58ac06372523 2024-02-16T22:27:44.504 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi123 smithi123 running (3m) 5s ago 14m 36.6M 2048M 19.0.0-1450-gf78a58c0 f081f847070a f76485e0ea24 2024-02-16T22:27:44.505 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi114 smithi114 *:9100 running (15m) 72s ago 15m 15.9M - 1.5.0 0da6a335fe13 744a090e06d0 2024-02-16T22:27:44.505 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi123 smithi123 *:9100 running (14m) 5s ago 14m 15.5M - 1.5.0 0da6a335fe13 3994b8c29b2e 2024-02-16T22:27:44.505 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 smithi114 running (2m) 72s ago 13m 91.2M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 1978fdad94a5 2024-02-16T22:27:44.505 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 smithi114 running (114s) 72s ago 12m 90.2M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 3f90e792898f 2024-02-16T22:27:44.505 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 smithi114 running (78s) 72s ago 12m 12.3M 1216M 19.0.0-1450-gf78a58c0 f081f847070a c663dfa0518b 2024-02-16T22:27:44.505 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 smithi123 running (42s) 5s ago 11m 99.6M 2240M 19.0.0-1450-gf78a58c0 f081f847070a b921448ba169 2024-02-16T22:27:44.505 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 smithi123 running (9s) 5s ago 11m 15.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 54c2926d56b1 2024-02-16T22:27:44.505 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 smithi123 running (11m) 5s ago 11m 202M 2240M 18.2.1 94eadd990e0a 24e5bef19fda 2024-02-16T22:27:44.505 INFO:teuthology.orchestra.run.smithi114.stdout:prometheus.smithi114 smithi114 *:9095 running (4m) 72s ago 14m 57.8M - 2.43.0 a07b618ecd1d 37aac985cdd7 2024-02-16T22:27:44.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.503+0000 7fad274de700 1 -- 172.21.15.114:0/249362242 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fad0c06d8a0 msgr2=0x7fad0c06fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:44.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.503+0000 7fad274de700 1 --2- 172.21.15.114:0/249362242 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fad0c06d8a0 0x7fad0c06fd60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fad10006010 tx=0x7fad1000b540 comp rx=0 tx=0).stop 2024-02-16T22:27:44.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.503+0000 7fad274de700 1 -- 172.21.15.114:0/249362242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad200fec30 msgr2=0x7fad201963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:44.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.503+0000 7fad274de700 1 --2- 172.21.15.114:0/249362242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad200fec30 0x7fad201963b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fad1c00d900 tx=0x7fad1c00dc10 comp rx=0 tx=0).stop 2024-02-16T22:27:44.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.503+0000 7fad274de700 1 -- 172.21.15.114:0/249362242 shutdown_connections 2024-02-16T22:27:44.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.503+0000 7fad274de700 1 --2- 172.21.15.114:0/249362242 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fad0c06d8a0 0x7fad0c06fd60 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:44.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.503+0000 7fad274de700 1 --2- 172.21.15.114:0/249362242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad200fec30 0x7fad201963b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:44.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.503+0000 7fad274de700 1 --2- 172.21.15.114:0/249362242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad201021b0 0x7fad201968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:44.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.503+0000 7fad274de700 1 -- 172.21.15.114:0/249362242 >> 172.21.15.114:0/249362242 conn(0x7fad200fa7f0 msgr2=0x7fad20105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:27:44.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.503+0000 7fad274de700 1 -- 172.21.15.114:0/249362242 shutdown_connections 2024-02-16T22:27:44.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.504+0000 7fad274de700 1 -- 172.21.15.114:0/249362242 wait complete. 2024-02-16T22:27:44.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.605+0000 7f56807f8700 1 -- 172.21.15.114:0/3963934111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5678103f90 msgr2=0x7f5678106380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:44.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.605+0000 7f56807f8700 1 --2- 172.21.15.114:0/3963934111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5678103f90 0x7f5678106380 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5668009910 tx=0x7f5668009c20 comp rx=0 tx=0).stop 2024-02-16T22:27:44.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.605+0000 7f56807f8700 1 -- 172.21.15.114:0/3963934111 shutdown_connections 2024-02-16T22:27:44.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.605+0000 7f56807f8700 1 --2- 172.21.15.114:0/3963934111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5678103f90 0x7f5678106380 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:44.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.605+0000 7f56807f8700 1 --2- 172.21.15.114:0/3963934111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5678101660 0x7f5678103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:44.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.605+0000 7f56807f8700 1 -- 172.21.15.114:0/3963934111 >> 172.21.15.114:0/3963934111 conn(0x7f56780fb050 msgr2=0x7f56780fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:27:44.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.605+0000 7f56807f8700 1 -- 172.21.15.114:0/3963934111 shutdown_connections 2024-02-16T22:27:44.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.605+0000 7f56807f8700 1 -- 172.21.15.114:0/3963934111 wait complete. 2024-02-16T22:27:44.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.606+0000 7f56807f8700 1 Processor -- start 2024-02-16T22:27:44.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.606+0000 7f56807f8700 1 -- start start 2024-02-16T22:27:44.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.606+0000 7f56807f8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5678101660 0x7f5678196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:44.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.606+0000 7f56807f8700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5678103f90 0x7f56781968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:44.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.606+0000 7f56807f8700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5678196ed0 con 0x7f5678103f90 2024-02-16T22:27:44.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.606+0000 7f56807f8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5678197010 con 0x7f5678101660 2024-02-16T22:27:44.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.606+0000 7f567dd93700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5678103f90 0x7f56781968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:44.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.606+0000 7f567e594700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5678101660 0x7f5678196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:44.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.606+0000 7f567dd93700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5678103f90 0x7f56781968b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48710/0 (socket says 172.21.15.114:48710) 2024-02-16T22:27:44.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.606+0000 7f567dd93700 1 -- 172.21.15.114:0/3271826832 learned_addr learned my addr 172.21.15.114:0/3271826832 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:27:44.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.607+0000 7f567dd93700 1 -- 172.21.15.114:0/3271826832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5678101660 msgr2=0x7f5678196370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:44.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.607+0000 7f567dd93700 1 --2- 172.21.15.114:0/3271826832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5678101660 0x7f5678196370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:44.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.607+0000 7f567dd93700 1 -- 172.21.15.114:0/3271826832 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56680095f0 con 0x7f5678103f90 2024-02-16T22:27:44.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.607+0000 7f567dd93700 1 --2- 172.21.15.114:0/3271826832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5678103f90 0x7f56781968b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f566800bd70 tx=0x7f566800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:27:44.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.607+0000 7f566f7fe700 1 -- 172.21.15.114:0/3271826832 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f566801d070 con 0x7f5678103f90 2024-02-16T22:27:44.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.607+0000 7f566f7fe700 1 -- 172.21.15.114:0/3271826832 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5668005dc0 con 0x7f5678103f90 2024-02-16T22:27:44.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.607+0000 7f56807f8700 1 -- 172.21.15.114:0/3271826832 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f567819ba60 con 0x7f5678103f90 2024-02-16T22:27:44.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.607+0000 7f566f7fe700 1 -- 172.21.15.114:0/3271826832 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5668021410 con 0x7f5678103f90 2024-02-16T22:27:44.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.607+0000 7f56807f8700 1 -- 172.21.15.114:0/3271826832 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f567819bf50 con 0x7f5678103f90 2024-02-16T22:27:44.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.608+0000 7f56807f8700 1 -- 172.21.15.114:0/3271826832 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5678190560 con 0x7f5678103f90 2024-02-16T22:27:44.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.608+0000 7f566f7fe700 1 -- 172.21.15.114:0/3271826832 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f566800f7d0 con 0x7f5678103f90 2024-02-16T22:27:44.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.609+0000 7f566f7fe700 1 --2- 172.21.15.114:0/3271826832 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f566406d8a0 0x7f566406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:44.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.609+0000 7f566f7fe700 1 -- 172.21.15.114:0/3271826832 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f566808e7c0 con 0x7f5678103f90 2024-02-16T22:27:44.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.609+0000 7f567e594700 1 --2- 172.21.15.114:0/3271826832 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f566406d8a0 0x7f566406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:44.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.609+0000 7f567e594700 1 --2- 172.21.15.114:0/3271826832 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f566406d8a0 0x7f566406fd60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5674004630 tx=0x7f567400b3c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:27:44.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.613+0000 7f566f7fe700 1 -- 172.21.15.114:0/3271826832 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f566805bb50 con 0x7f5678103f90 2024-02-16T22:27:44.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.889+0000 7f56807f8700 1 -- 172.21.15.114:0/3271826832 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f567802cc70 con 0x7f5678103f90 2024-02-16T22:27:44.892 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:44 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:44.892 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:44 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:44.892 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:44 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:27:44.892 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:44 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:27:44.892 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:44 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:44.892 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:44 smithi114 ceph-mon[94452]: pgmap v145: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 46/267 objects degraded (17.228%) 2024-02-16T22:27:44.892 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:44 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:27:44.892 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:44 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:27:44.892 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:44 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:27:44.892 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:44 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:27:44.892 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:44 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T22:27:44.892 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:44 smithi114 ceph-mon[94452]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T22:27:44.892 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:44 smithi114 ceph-mon[94452]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-02-16T22:27:44.893 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:44 smithi114 ceph-mon[94452]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:27:44.893 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:44 smithi114 ceph-mon[94452]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T22:27:44.893 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:44 smithi114 ceph-mon[94452]: osd.4 [v2:172.21.15.123:6808/75423100,v1:172.21.15.123:6809/75423100] boot 2024-02-16T22:27:44.893 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:44 smithi114 ceph-mon[94452]: osdmap e72: 6 total, 6 up, 6 in 2024-02-16T22:27:44.893 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:44 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:27:44.893 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:44 smithi114 ceph-mon[94452]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:27:44.893 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:44 smithi114 ceph-mon[94452]: from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:27:44.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.890+0000 7f566f7fe700 1 -- 172.21.15.114:0/3271826832 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f566800f460 con 0x7f5678103f90 2024-02-16T22:27:44.893 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:27:44.894 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": { 2024-02-16T22:27:44.894 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:27:44.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:27:44.894 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": { 2024-02-16T22:27:44.894 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:27:44.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:27:44.894 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": { 2024-02-16T22:27:44.894 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-02-16T22:27:44.894 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 5 2024-02-16T22:27:44.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:27:44.894 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-02-16T22:27:44.894 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-02-16T22:27:44.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:27:44.894 INFO:teuthology.orchestra.run.smithi114.stdout: "overall": { 2024-02-16T22:27:44.895 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 5, 2024-02-16T22:27:44.895 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 9 2024-02-16T22:27:44.895 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-02-16T22:27:44.895 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:27:44.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.893+0000 7f56807f8700 1 -- 172.21.15.114:0/3271826832 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f566406d8a0 msgr2=0x7f566406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:44.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.893+0000 7f56807f8700 1 --2- 172.21.15.114:0/3271826832 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f566406d8a0 0x7f566406fd60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5674004630 tx=0x7f567400b3c0 comp rx=0 tx=0).stop 2024-02-16T22:27:44.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.893+0000 7f56807f8700 1 -- 172.21.15.114:0/3271826832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5678103f90 msgr2=0x7f56781968b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:44.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.893+0000 7f56807f8700 1 --2- 172.21.15.114:0/3271826832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5678103f90 0x7f56781968b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f566800bd70 tx=0x7f566800bda0 comp rx=0 tx=0).stop 2024-02-16T22:27:44.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.893+0000 7f56807f8700 1 -- 172.21.15.114:0/3271826832 shutdown_connections 2024-02-16T22:27:44.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.893+0000 7f56807f8700 1 --2- 172.21.15.114:0/3271826832 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f566406d8a0 0x7f566406fd60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:44.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.893+0000 7f56807f8700 1 --2- 172.21.15.114:0/3271826832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5678101660 0x7f5678196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:44.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.893+0000 7f56807f8700 1 --2- 172.21.15.114:0/3271826832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5678103f90 0x7f56781968b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:44.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.893+0000 7f56807f8700 1 -- 172.21.15.114:0/3271826832 >> 172.21.15.114:0/3271826832 conn(0x7f56780fb050 msgr2=0x7f56780fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:27:44.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.893+0000 7f56807f8700 1 -- 172.21.15.114:0/3271826832 shutdown_connections 2024-02-16T22:27:44.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.893+0000 7f56807f8700 1 -- 172.21.15.114:0/3271826832 wait complete. 2024-02-16T22:27:45.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.997+0000 7f76daf93700 1 -- 172.21.15.114:0/1640738910 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76d40ffab0 msgr2=0x7f76d40ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:45.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.997+0000 7f76daf93700 1 --2- 172.21.15.114:0/1640738910 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76d40ffab0 0x7f76d40ffed0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f76c8009910 tx=0x7f76c8009c20 comp rx=0 tx=0).stop 2024-02-16T22:27:45.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.997+0000 7f76daf93700 1 -- 172.21.15.114:0/1640738910 shutdown_connections 2024-02-16T22:27:45.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.997+0000 7f76daf93700 1 --2- 172.21.15.114:0/1640738910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76d4100cb0 0x7f76d4101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:45.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.997+0000 7f76daf93700 1 --2- 172.21.15.114:0/1640738910 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76d40ffab0 0x7f76d40ffed0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:45.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.997+0000 7f76daf93700 1 -- 172.21.15.114:0/1640738910 >> 172.21.15.114:0/1640738910 conn(0x7f76d40fb030 msgr2=0x7f76d40fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:27:45.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.997+0000 7f76daf93700 1 -- 172.21.15.114:0/1640738910 shutdown_connections 2024-02-16T22:27:45.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.997+0000 7f76daf93700 1 -- 172.21.15.114:0/1640738910 wait complete. 2024-02-16T22:27:45.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.997+0000 7f76daf93700 1 Processor -- start 2024-02-16T22:27:45.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.997+0000 7f76daf93700 1 -- start start 2024-02-16T22:27:45.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.998+0000 7f76daf93700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76d40ffab0 0x7f76d4196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:45.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.998+0000 7f76daf93700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76d4100cb0 0x7f76d4196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:45.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.998+0000 7f76daf93700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d4196e80 con 0x7f76d4100cb0 2024-02-16T22:27:45.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.998+0000 7f76daf93700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d4196fc0 con 0x7f76d40ffab0 2024-02-16T22:27:45.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.998+0000 7f76d3fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76d4100cb0 0x7f76d4196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:45.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.998+0000 7f76d3fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76d4100cb0 0x7f76d4196860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48732/0 (socket says 172.21.15.114:48732) 2024-02-16T22:27:45.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.998+0000 7f76d3fff700 1 -- 172.21.15.114:0/3042595820 learned_addr learned my addr 172.21.15.114:0/3042595820 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:27:45.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.998+0000 7f76d8d2f700 1 --2- 172.21.15.114:0/3042595820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76d40ffab0 0x7f76d4196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:45.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.998+0000 7f76d3fff700 1 -- 172.21.15.114:0/3042595820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76d40ffab0 msgr2=0x7f76d4196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:45.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.998+0000 7f76d3fff700 1 --2- 172.21.15.114:0/3042595820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76d40ffab0 0x7f76d4196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:45.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.998+0000 7f76d3fff700 1 -- 172.21.15.114:0/3042595820 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c80095f0 con 0x7f76d4100cb0 2024-02-16T22:27:45.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.998+0000 7f76d3fff700 1 --2- 172.21.15.114:0/3042595820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76d4100cb0 0x7f76d4196860 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f76c000d8d0 tx=0x7f76c000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:27:45.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.998+0000 7f76d1ffb700 1 -- 172.21.15.114:0/3042595820 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76c00098a0 con 0x7f76d4100cb0 2024-02-16T22:27:45.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.998+0000 7f76d1ffb700 1 -- 172.21.15.114:0/3042595820 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f76c0010460 con 0x7f76d4100cb0 2024-02-16T22:27:45.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.998+0000 7f76daf93700 1 -- 172.21.15.114:0/3042595820 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d419ba70 con 0x7f76d4100cb0 2024-02-16T22:27:45.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.998+0000 7f76d1ffb700 1 -- 172.21.15.114:0/3042595820 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76c000f5f0 con 0x7f76d4100cb0 2024-02-16T22:27:45.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.999+0000 7f76daf93700 1 -- 172.21.15.114:0/3042595820 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d419bfc0 con 0x7f76d4100cb0 2024-02-16T22:27:45.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:44.999+0000 7f76daf93700 1 -- 172.21.15.114:0/3042595820 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76d4108760 con 0x7f76d4100cb0 2024-02-16T22:27:45.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.000+0000 7f76d1ffb700 1 -- 172.21.15.114:0/3042595820 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f76c0009a00 con 0x7f76d4100cb0 2024-02-16T22:27:45.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.000+0000 7f76d1ffb700 1 --2- 172.21.15.114:0/3042595820 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f76c406d8a0 0x7f76c406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:45.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.000+0000 7f76d1ffb700 1 -- 172.21.15.114:0/3042595820 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f76c008d2b0 con 0x7f76d4100cb0 2024-02-16T22:27:45.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.000+0000 7f76d8d2f700 1 --2- 172.21.15.114:0/3042595820 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f76c406d8a0 0x7f76c406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:45.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.001+0000 7f76d8d2f700 1 --2- 172.21.15.114:0/3042595820 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f76c406d8a0 0x7f76c406fd60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f76c8006010 tx=0x7f76c800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:27:45.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.005+0000 7f76d1ffb700 1 -- 172.21.15.114:0/3042595820 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f76c005a6c0 con 0x7f76d4100cb0 2024-02-16T22:27:45.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:44 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:45.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:44 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:45.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:44 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:27:45.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:44 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:27:45.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:44 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:45.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:44 smithi123 ceph-mon[82331]: pgmap v145: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 46/267 objects degraded (17.228%) 2024-02-16T22:27:45.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:44 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:27:45.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:44 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:27:45.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:44 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:27:45.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:44 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:27:45.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:44 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T22:27:45.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:44 smithi123 ceph-mon[82331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T22:27:45.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:44 smithi123 ceph-mon[82331]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-02-16T22:27:45.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:44 smithi123 ceph-mon[82331]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:27:45.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:44 smithi123 ceph-mon[82331]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T22:27:45.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:44 smithi123 ceph-mon[82331]: osd.4 [v2:172.21.15.123:6808/75423100,v1:172.21.15.123:6809/75423100] boot 2024-02-16T22:27:45.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:44 smithi123 ceph-mon[82331]: osdmap e72: 6 total, 6 up, 6 in 2024-02-16T22:27:45.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:44 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T22:27:45.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:44 smithi123 ceph-mon[82331]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:27:45.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:44 smithi123 ceph-mon[82331]: from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:27:45.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.261+0000 7f76daf93700 1 -- 172.21.15.114:0/3042595820 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f76d419c690 con 0x7f76d4100cb0 2024-02-16T22:27:45.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.262+0000 7f76d1ffb700 1 -- 172.21.15.114:0/3042595820 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1846 (secure 0 0 0) 0x7f76c0020750 con 0x7f76d4100cb0 2024-02-16T22:27:45.264 INFO:teuthology.orchestra.run.smithi114.stdout:e14 2024-02-16T22:27:45.264 INFO:teuthology.orchestra.run.smithi114.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:27:45.264 INFO:teuthology.orchestra.run.smithi114.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:27:45.264 INFO:teuthology.orchestra.run.smithi114.stdout:legacy client fscid: 1 2024-02-16T22:27:45.264 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:27:45.264 INFO:teuthology.orchestra.run.smithi114.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:27:45.264 INFO:teuthology.orchestra.run.smithi114.stdout:fs_name cephfs 2024-02-16T22:27:45.264 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 12 2024-02-16T22:27:45.265 INFO:teuthology.orchestra.run.smithi114.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:27:45.265 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-02-16T22:18:02.108825+0000 2024-02-16T22:27:45.265 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-02-16T22:18:16.413371+0000 2024-02-16T22:27:45.265 INFO:teuthology.orchestra.run.smithi114.stdout:tableserver 0 2024-02-16T22:27:45.265 INFO:teuthology.orchestra.run.smithi114.stdout:root 0 2024-02-16T22:27:45.265 INFO:teuthology.orchestra.run.smithi114.stdout:session_timeout 60 2024-02-16T22:27:45.265 INFO:teuthology.orchestra.run.smithi114.stdout:session_autoclose 300 2024-02-16T22:27:45.265 INFO:teuthology.orchestra.run.smithi114.stdout:max_file_size 1099511627776 2024-02-16T22:27:45.265 INFO:teuthology.orchestra.run.smithi114.stdout:max_xattr_size 65536 2024-02-16T22:27:45.265 INFO:teuthology.orchestra.run.smithi114.stdout:required_client_features {} 2024-02-16T22:27:45.265 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure 0 2024-02-16T22:27:45.265 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure_osd_epoch 0 2024-02-16T22:27:45.265 INFO:teuthology.orchestra.run.smithi114.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:27:45.265 INFO:teuthology.orchestra.run.smithi114.stdout:max_mds 1 2024-02-16T22:27:45.266 INFO:teuthology.orchestra.run.smithi114.stdout:in 0 2024-02-16T22:27:45.266 INFO:teuthology.orchestra.run.smithi114.stdout:up {0=14468} 2024-02-16T22:27:45.266 INFO:teuthology.orchestra.run.smithi114.stdout:failed 2024-02-16T22:27:45.266 INFO:teuthology.orchestra.run.smithi114.stdout:damaged 2024-02-16T22:27:45.266 INFO:teuthology.orchestra.run.smithi114.stdout:stopped 2024-02-16T22:27:45.266 INFO:teuthology.orchestra.run.smithi114.stdout:data_pools [3] 2024-02-16T22:27:45.266 INFO:teuthology.orchestra.run.smithi114.stdout:metadata_pool 2 2024-02-16T22:27:45.266 INFO:teuthology.orchestra.run.smithi114.stdout:inline_data enabled 2024-02-16T22:27:45.266 INFO:teuthology.orchestra.run.smithi114.stdout:balancer 2024-02-16T22:27:45.266 INFO:teuthology.orchestra.run.smithi114.stdout:bal_rank_mask -1 2024-02-16T22:27:45.267 INFO:teuthology.orchestra.run.smithi114.stdout:standby_count_wanted 1 2024-02-16T22:27:45.267 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.zzryke{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.114:6826/277953535,v1:172.21.15.114:6827/277953535] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:27:45.267 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:27:45.267 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:27:45.267 INFO:teuthology.orchestra.run.smithi114.stdout:Standby daemons: 2024-02-16T22:27:45.267 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:27:45.267 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.fdxpey{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:27:45.267 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.vhlqbe{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:27:45.267 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.gopcfa{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:27:45.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.264+0000 7f76daf93700 1 -- 172.21.15.114:0/3042595820 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f76c406d8a0 msgr2=0x7f76c406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:45.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.264+0000 7f76daf93700 1 --2- 172.21.15.114:0/3042595820 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f76c406d8a0 0x7f76c406fd60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f76c8006010 tx=0x7f76c800b540 comp rx=0 tx=0).stop 2024-02-16T22:27:45.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.264+0000 7f76daf93700 1 -- 172.21.15.114:0/3042595820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76d4100cb0 msgr2=0x7f76d4196860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:45.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.264+0000 7f76daf93700 1 --2- 172.21.15.114:0/3042595820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76d4100cb0 0x7f76d4196860 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f76c000d8d0 tx=0x7f76c000dbe0 comp rx=0 tx=0).stop 2024-02-16T22:27:45.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.264+0000 7f76daf93700 1 -- 172.21.15.114:0/3042595820 shutdown_connections 2024-02-16T22:27:45.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.264+0000 7f76daf93700 1 --2- 172.21.15.114:0/3042595820 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f76c406d8a0 0x7f76c406fd60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:45.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.264+0000 7f76daf93700 1 --2- 172.21.15.114:0/3042595820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76d40ffab0 0x7f76d4196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:45.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.264+0000 7f76daf93700 1 --2- 172.21.15.114:0/3042595820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76d4100cb0 0x7f76d4196860 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:45.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.264+0000 7f76daf93700 1 -- 172.21.15.114:0/3042595820 >> 172.21.15.114:0/3042595820 conn(0x7f76d40fb030 msgr2=0x7f76d40683d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:27:45.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.264+0000 7f76daf93700 1 -- 172.21.15.114:0/3042595820 shutdown_connections 2024-02-16T22:27:45.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.264+0000 7f76daf93700 1 -- 172.21.15.114:0/3042595820 wait complete. 2024-02-16T22:27:45.269 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 14 2024-02-16T22:27:45.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.368+0000 7fd935a9b700 1 -- 172.21.15.114:0/2113798076 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9300ffd30 msgr2=0x7fd9301001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:45.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.368+0000 7fd935a9b700 1 --2- 172.21.15.114:0/2113798076 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9300ffd30 0x7fd9301001b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd920009910 tx=0x7fd920009c20 comp rx=0 tx=0).stop 2024-02-16T22:27:45.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.368+0000 7fd935a9b700 1 -- 172.21.15.114:0/2113798076 shutdown_connections 2024-02-16T22:27:45.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.368+0000 7fd935a9b700 1 --2- 172.21.15.114:0/2113798076 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9300ffd30 0x7fd9301001b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:45.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.368+0000 7fd935a9b700 1 --2- 172.21.15.114:0/2113798076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9301010d0 0x7fd9301014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:45.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.368+0000 7fd935a9b700 1 -- 172.21.15.114:0/2113798076 >> 172.21.15.114:0/2113798076 conn(0x7fd9300fa7c0 msgr2=0x7fd9300fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:27:45.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.368+0000 7fd935a9b700 1 -- 172.21.15.114:0/2113798076 shutdown_connections 2024-02-16T22:27:45.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.368+0000 7fd935a9b700 1 -- 172.21.15.114:0/2113798076 wait complete. 2024-02-16T22:27:45.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.369+0000 7fd935a9b700 1 Processor -- start 2024-02-16T22:27:45.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.369+0000 7fd935a9b700 1 -- start start 2024-02-16T22:27:45.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.369+0000 7fd935a9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9300ffd30 0x7fd930195240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:45.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.369+0000 7fd935a9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9301010d0 0x7fd930195780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:45.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.369+0000 7fd935a9b700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd930195da0 con 0x7fd9300ffd30 2024-02-16T22:27:45.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.369+0000 7fd935a9b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd930195ee0 con 0x7fd9301010d0 2024-02-16T22:27:45.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.369+0000 7fd92f7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9300ffd30 0x7fd930195240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:45.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.369+0000 7fd92f7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9300ffd30 0x7fd930195240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48752/0 (socket says 172.21.15.114:48752) 2024-02-16T22:27:45.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.369+0000 7fd92f7fe700 1 -- 172.21.15.114:0/3731444831 learned_addr learned my addr 172.21.15.114:0/3731444831 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:27:45.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.369+0000 7fd92effd700 1 --2- 172.21.15.114:0/3731444831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9301010d0 0x7fd930195780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:45.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.369+0000 7fd92f7fe700 1 -- 172.21.15.114:0/3731444831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9301010d0 msgr2=0x7fd930195780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:45.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.369+0000 7fd92f7fe700 1 --2- 172.21.15.114:0/3731444831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9301010d0 0x7fd930195780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:45.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.369+0000 7fd92f7fe700 1 -- 172.21.15.114:0/3731444831 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9200095f0 con 0x7fd9300ffd30 2024-02-16T22:27:45.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.370+0000 7fd92f7fe700 1 --2- 172.21.15.114:0/3731444831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9300ffd30 0x7fd930195240 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd92400d900 tx=0x7fd92400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:27:45.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.370+0000 7fd92cff9700 1 -- 172.21.15.114:0/3731444831 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9240041f0 con 0x7fd9300ffd30 2024-02-16T22:27:45.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.370+0000 7fd92cff9700 1 -- 172.21.15.114:0/3731444831 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd924004350 con 0x7fd9300ffd30 2024-02-16T22:27:45.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.370+0000 7fd935a9b700 1 -- 172.21.15.114:0/3731444831 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd93019a990 con 0x7fd9300ffd30 2024-02-16T22:27:45.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.370+0000 7fd92cff9700 1 -- 172.21.15.114:0/3731444831 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd924003da0 con 0x7fd9300ffd30 2024-02-16T22:27:45.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.370+0000 7fd935a9b700 1 -- 172.21.15.114:0/3731444831 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd93019aee0 con 0x7fd9300ffd30 2024-02-16T22:27:45.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.371+0000 7fd935a9b700 1 -- 172.21.15.114:0/3731444831 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd930108640 con 0x7fd9300ffd30 2024-02-16T22:27:45.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.371+0000 7fd92cff9700 1 -- 172.21.15.114:0/3731444831 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fd924009730 con 0x7fd9300ffd30 2024-02-16T22:27:45.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.372+0000 7fd92cff9700 1 --2- 172.21.15.114:0/3731444831 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd91806d8a0 0x7fd91806fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:45.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.372+0000 7fd92cff9700 1 -- 172.21.15.114:0/3731444831 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd92408d400 con 0x7fd9300ffd30 2024-02-16T22:27:45.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.372+0000 7fd92effd700 1 --2- 172.21.15.114:0/3731444831 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd91806d8a0 0x7fd91806fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:45.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.372+0000 7fd92effd700 1 --2- 172.21.15.114:0/3731444831 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd91806d8a0 0x7fd91806fd60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd920005b40 tx=0x7fd920005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:27:45.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.377+0000 7fd92cff9700 1 -- 172.21.15.114:0/3731444831 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd92405a790 con 0x7fd9300ffd30 2024-02-16T22:27:45.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.588+0000 7fd935a9b700 1 -- 172.21.15.114:0/3731444831 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd930105690 con 0x7fd91806d8a0 2024-02-16T22:27:45.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.589+0000 7fd92cff9700 1 -- 172.21.15.114:0/3731444831 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fd930105690 con 0x7fd91806d8a0 2024-02-16T22:27:45.591 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:27:45.591 INFO:teuthology.orchestra.run.smithi114.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:27:45.591 INFO:teuthology.orchestra.run.smithi114.stdout: "in_progress": true, 2024-02-16T22:27:45.591 INFO:teuthology.orchestra.run.smithi114.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:27:45.591 INFO:teuthology.orchestra.run.smithi114.stdout: "services_complete": [ 2024-02-16T22:27:45.591 INFO:teuthology.orchestra.run.smithi114.stdout: "crash", 2024-02-16T22:27:45.591 INFO:teuthology.orchestra.run.smithi114.stdout: "mon", 2024-02-16T22:27:45.591 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr" 2024-02-16T22:27:45.591 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-02-16T22:27:45.591 INFO:teuthology.orchestra.run.smithi114.stdout: "progress": "11/23 daemons upgraded", 2024-02-16T22:27:45.592 INFO:teuthology.orchestra.run.smithi114.stdout: "message": "Currently upgrading osd daemons", 2024-02-16T22:27:45.592 INFO:teuthology.orchestra.run.smithi114.stdout: "is_paused": false 2024-02-16T22:27:45.592 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:27:45.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.591+0000 7fd935a9b700 1 -- 172.21.15.114:0/3731444831 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd91806d8a0 msgr2=0x7fd91806fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:45.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.591+0000 7fd935a9b700 1 --2- 172.21.15.114:0/3731444831 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd91806d8a0 0x7fd91806fd60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd920005b40 tx=0x7fd920005a90 comp rx=0 tx=0).stop 2024-02-16T22:27:45.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.591+0000 7fd935a9b700 1 -- 172.21.15.114:0/3731444831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9300ffd30 msgr2=0x7fd930195240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:45.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.591+0000 7fd935a9b700 1 --2- 172.21.15.114:0/3731444831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9300ffd30 0x7fd930195240 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd92400d900 tx=0x7fd92400dcc0 comp rx=0 tx=0).stop 2024-02-16T22:27:45.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.591+0000 7fd935a9b700 1 -- 172.21.15.114:0/3731444831 shutdown_connections 2024-02-16T22:27:45.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.591+0000 7fd935a9b700 1 --2- 172.21.15.114:0/3731444831 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd91806d8a0 0x7fd91806fd60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:45.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.591+0000 7fd935a9b700 1 --2- 172.21.15.114:0/3731444831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9300ffd30 0x7fd930195240 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:45.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.591+0000 7fd935a9b700 1 --2- 172.21.15.114:0/3731444831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9301010d0 0x7fd930195780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:45.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.591+0000 7fd935a9b700 1 -- 172.21.15.114:0/3731444831 >> 172.21.15.114:0/3731444831 conn(0x7fd9300fa7c0 msgr2=0x7fd930103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:27:45.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.591+0000 7fd935a9b700 1 -- 172.21.15.114:0/3731444831 shutdown_connections 2024-02-16T22:27:45.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.591+0000 7fd935a9b700 1 -- 172.21.15.114:0/3731444831 wait complete. 2024-02-16T22:27:45.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.695+0000 7f3e26316700 1 -- 172.21.15.114:0/877405501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e201016b0 msgr2=0x7f3e20103aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:45.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.695+0000 7f3e26316700 1 --2- 172.21.15.114:0/877405501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e201016b0 0x7f3e20103aa0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3e10009910 tx=0x7f3e10009c20 comp rx=0 tx=0).stop 2024-02-16T22:27:45.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.695+0000 7f3e26316700 1 -- 172.21.15.114:0/877405501 shutdown_connections 2024-02-16T22:27:45.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.695+0000 7f3e26316700 1 --2- 172.21.15.114:0/877405501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e20103fe0 0x7f3e201063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:45.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.695+0000 7f3e26316700 1 --2- 172.21.15.114:0/877405501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e201016b0 0x7f3e20103aa0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:45.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.695+0000 7f3e26316700 1 -- 172.21.15.114:0/877405501 >> 172.21.15.114:0/877405501 conn(0x7f3e200fb060 msgr2=0x7f3e200fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:27:45.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.695+0000 7f3e26316700 1 -- 172.21.15.114:0/877405501 shutdown_connections 2024-02-16T22:27:45.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.695+0000 7f3e26316700 1 -- 172.21.15.114:0/877405501 wait complete. 2024-02-16T22:27:45.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.696+0000 7f3e26316700 1 Processor -- start 2024-02-16T22:27:45.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.696+0000 7f3e26316700 1 -- start start 2024-02-16T22:27:45.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.696+0000 7f3e26316700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e201016b0 0x7f3e201963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:45.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.696+0000 7f3e26316700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e20103fe0 0x7f3e20196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:45.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.696+0000 7f3e26316700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e20196f40 con 0x7f3e20103fe0 2024-02-16T22:27:45.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.696+0000 7f3e26316700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e20197080 con 0x7f3e201016b0 2024-02-16T22:27:45.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.696+0000 7f3e1f7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e20103fe0 0x7f3e20196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:45.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.696+0000 7f3e1f7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e20103fe0 0x7f3e20196920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48762/0 (socket says 172.21.15.114:48762) 2024-02-16T22:27:45.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.696+0000 7f3e1f7fe700 1 -- 172.21.15.114:0/3126586646 learned_addr learned my addr 172.21.15.114:0/3126586646 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:27:45.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.696+0000 7f3e1ffff700 1 --2- 172.21.15.114:0/3126586646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e201016b0 0x7f3e201963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:45.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.696+0000 7f3e1f7fe700 1 -- 172.21.15.114:0/3126586646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e201016b0 msgr2=0x7f3e201963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:45.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.696+0000 7f3e1f7fe700 1 --2- 172.21.15.114:0/3126586646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e201016b0 0x7f3e201963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:45.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.696+0000 7f3e1f7fe700 1 -- 172.21.15.114:0/3126586646 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e100095f0 con 0x7f3e20103fe0 2024-02-16T22:27:45.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.697+0000 7f3e1f7fe700 1 --2- 172.21.15.114:0/3126586646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e20103fe0 0x7f3e20196920 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400b4b0 tx=0x7f3e1400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:27:45.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.697+0000 7f3e1d7fa700 1 -- 172.21.15.114:0/3126586646 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e14011800 con 0x7f3e20103fe0 2024-02-16T22:27:45.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.697+0000 7f3e1d7fa700 1 -- 172.21.15.114:0/3126586646 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3e14011e10 con 0x7f3e20103fe0 2024-02-16T22:27:45.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.697+0000 7f3e1d7fa700 1 -- 172.21.15.114:0/3126586646 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e1400f2e0 con 0x7f3e20103fe0 2024-02-16T22:27:45.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.697+0000 7f3e26316700 1 -- 172.21.15.114:0/3126586646 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e2019bb30 con 0x7f3e20103fe0 2024-02-16T22:27:45.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.697+0000 7f3e26316700 1 -- 172.21.15.114:0/3126586646 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e2019c0b0 con 0x7f3e20103fe0 2024-02-16T22:27:45.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.698+0000 7f3e26316700 1 -- 172.21.15.114:0/3126586646 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e201905b0 con 0x7f3e20103fe0 2024-02-16T22:27:45.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.698+0000 7f3e1d7fa700 1 -- 172.21.15.114:0/3126586646 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f3e14010380 con 0x7f3e20103fe0 2024-02-16T22:27:45.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.699+0000 7f3e1d7fa700 1 --2- 172.21.15.114:0/3126586646 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f3e0806d8a0 0x7f3e0806fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:27:45.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.699+0000 7f3e1d7fa700 1 -- 172.21.15.114:0/3126586646 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f3e1408d0e0 con 0x7f3e20103fe0 2024-02-16T22:27:45.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.699+0000 7f3e1ffff700 1 --2- 172.21.15.114:0/3126586646 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f3e0806d8a0 0x7f3e0806fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:27:45.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.699+0000 7f3e1ffff700 1 --2- 172.21.15.114:0/3126586646 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f3e0806d8a0 0x7f3e0806fd60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3e10006010 tx=0x7f3e1000b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:27:45.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.703+0000 7f3e1d7fa700 1 -- 172.21.15.114:0/3126586646 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3e1405a470 con 0x7f3e20103fe0 2024-02-16T22:27:45.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:45 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/3271826832' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:27:45.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:45 smithi114 ceph-mon[94452]: osdmap e73: 6 total, 6 up, 6 in 2024-02-16T22:27:45.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:45 smithi114 ceph-mon[94452]: Health check update: Degraded data redundancy: 46/267 objects degraded (17.228%), 15 pgs degraded (PG_DEGRADED) 2024-02-16T22:27:45.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:45 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/3042595820' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:27:45.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.985+0000 7f3e26316700 1 -- 172.21.15.114:0/3126586646 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3e2002cc70 con 0x7f3e20103fe0 2024-02-16T22:27:45.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.985+0000 7f3e1d7fa700 1 -- 172.21.15.114:0/3126586646 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1207 (secure 0 0 0) 0x7f3e1405a000 con 0x7f3e20103fe0 2024-02-16T22:27:45.987 INFO:teuthology.orchestra.run.smithi114.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; Degraded data redundancy: 46/267 objects degraded (17.228%), 15 pgs degraded 2024-02-16T22:27:45.987 INFO:teuthology.orchestra.run.smithi114.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:27:45.987 INFO:teuthology.orchestra.run.smithi114.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:27:45.987 INFO:teuthology.orchestra.run.smithi114.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 46/267 objects degraded (17.228%), 15 pgs degraded 2024-02-16T22:27:45.987 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.4 is active+undersized+degraded, acting [1,0] 2024-02-16T22:27:45.987 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.5 is active+undersized+degraded, acting [3,0] 2024-02-16T22:27:45.987 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.6 is active+undersized+degraded, acting [1,3] 2024-02-16T22:27:45.987 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.7 is active+undersized+degraded, acting [3,2] 2024-02-16T22:27:45.988 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.9 is active+undersized+degraded, acting [1,0] 2024-02-16T22:27:45.988 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.a is active+undersized+degraded, acting [1,3] 2024-02-16T22:27:45.988 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.b is active+undersized+degraded, acting [3,5] 2024-02-16T22:27:45.988 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.f is active+undersized+degraded, acting [0,5] 2024-02-16T22:27:45.988 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.13 is active+undersized+degraded, acting [0,2] 2024-02-16T22:27:45.988 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.14 is active+undersized+degraded, acting [3,5] 2024-02-16T22:27:45.988 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.18 is active+undersized+degraded, acting [5,3] 2024-02-16T22:27:45.988 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.19 is active+undersized+degraded, acting [0,2] 2024-02-16T22:27:45.988 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.1a is active+undersized+degraded, acting [3,5] 2024-02-16T22:27:45.988 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.1c is active+undersized+degraded, acting [5,2] 2024-02-16T22:27:45.988 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.1f is active+undersized+degraded, acting [0,3] 2024-02-16T22:27:45.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.987+0000 7f3e26316700 1 -- 172.21.15.114:0/3126586646 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f3e0806d8a0 msgr2=0x7f3e0806fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:45.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.987+0000 7f3e26316700 1 --2- 172.21.15.114:0/3126586646 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f3e0806d8a0 0x7f3e0806fd60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3e10006010 tx=0x7f3e1000b540 comp rx=0 tx=0).stop 2024-02-16T22:27:45.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.987+0000 7f3e26316700 1 -- 172.21.15.114:0/3126586646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e20103fe0 msgr2=0x7f3e20196920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:27:45.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.987+0000 7f3e26316700 1 --2- 172.21.15.114:0/3126586646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e20103fe0 0x7f3e20196920 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400b4b0 tx=0x7f3e1400b870 comp rx=0 tx=0).stop 2024-02-16T22:27:45.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.987+0000 7f3e26316700 1 -- 172.21.15.114:0/3126586646 shutdown_connections 2024-02-16T22:27:45.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.987+0000 7f3e26316700 1 --2- 172.21.15.114:0/3126586646 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f3e0806d8a0 0x7f3e0806fd60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:45.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.987+0000 7f3e26316700 1 --2- 172.21.15.114:0/3126586646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e201016b0 0x7f3e201963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:45.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.987+0000 7f3e26316700 1 --2- 172.21.15.114:0/3126586646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e20103fe0 0x7f3e20196920 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:27:45.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.987+0000 7f3e26316700 1 -- 172.21.15.114:0/3126586646 >> 172.21.15.114:0/3126586646 conn(0x7f3e200fb060 msgr2=0x7f3e200fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:27:45.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.987+0000 7f3e26316700 1 -- 172.21.15.114:0/3126586646 shutdown_connections 2024-02-16T22:27:45.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:27:45.987+0000 7f3e26316700 1 -- 172.21.15.114:0/3126586646 wait complete. 2024-02-16T22:27:46.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:45 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/3271826832' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:27:46.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:45 smithi123 ceph-mon[82331]: osdmap e73: 6 total, 6 up, 6 in 2024-02-16T22:27:46.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:45 smithi123 ceph-mon[82331]: Health check update: Degraded data redundancy: 46/267 objects degraded (17.228%), 15 pgs degraded (PG_DEGRADED) 2024-02-16T22:27:46.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:45 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/3042595820' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:27:47.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:46 smithi114 ceph-mon[94452]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:27:47.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:46 smithi114 ceph-mon[94452]: pgmap v148: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 46/267 objects degraded (17.228%) 2024-02-16T22:27:47.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:47.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:27:47.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:46 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/3126586646' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:27:47.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:46 smithi123 ceph-mon[82331]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:27:47.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:46 smithi123 ceph-mon[82331]: pgmap v148: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 46/267 objects degraded (17.228%) 2024-02-16T22:27:47.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:46 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:47.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:46 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:27:47.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:46 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/3126586646' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:27:49.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:48 smithi114 ceph-mon[94452]: pgmap v149: 65 pgs: 5 active+undersized+degraded, 5 active+undersized, 55 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 22/267 objects degraded (8.240%) 2024-02-16T22:27:49.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:48 smithi123 ceph-mon[82331]: pgmap v149: 65 pgs: 5 active+undersized+degraded, 5 active+undersized, 55 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 22/267 objects degraded (8.240%) 2024-02-16T22:27:50.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:49 smithi114 ceph-mon[94452]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 22/267 objects degraded (8.240%), 5 pgs degraded) 2024-02-16T22:27:50.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:49 smithi123 ceph-mon[82331]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 22/267 objects degraded (8.240%), 5 pgs degraded) 2024-02-16T22:27:51.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:50 smithi114 ceph-mon[94452]: pgmap v150: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:51.281 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:50 smithi123 ceph-mon[82331]: pgmap v150: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:53.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:52 smithi114 ceph-mon[94452]: pgmap v151: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:53.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:52 smithi123 ceph-mon[82331]: pgmap v151: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:55.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:54 smithi114 ceph-mon[94452]: pgmap v152: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:55.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:54 smithi123 ceph-mon[82331]: pgmap v152: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:57.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:56 smithi114 ceph-mon[94452]: pgmap v153: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:57.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:56 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:57.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:56 smithi123 ceph-mon[82331]: pgmap v153: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:57.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:56 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:27:59.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:58 smithi114 ceph-mon[94452]: pgmap v154: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:59.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:58 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T22:27:59.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:58 smithi123 ceph-mon[82331]: pgmap v154: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:27:59.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:58 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T22:28:00.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:59 smithi123 ceph-mon[82331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T22:28:00.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:27:59 smithi123 ceph-mon[82331]: Upgrade: osd.5 is safe to restart 2024-02-16T22:28:00.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:59 smithi114 ceph-mon[94452]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T22:28:00.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:27:59 smithi114 ceph-mon[94452]: Upgrade: osd.5 is safe to restart 2024-02-16T22:28:01.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:01 smithi123 ceph-mon[82331]: pgmap v155: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:28:01.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:01 smithi123 ceph-mon[82331]: Upgrade: Updating osd.5 2024-02-16T22:28:01.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:01 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:01.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:01 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-16T22:28:01.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:01 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:28:01.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:01 smithi123 ceph-mon[82331]: Deploying daemon osd.5 on smithi123 2024-02-16T22:28:01.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:01 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:01.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:01 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:28:01.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:01 smithi114 ceph-mon[94452]: pgmap v155: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:28:01.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:01 smithi114 ceph-mon[94452]: Upgrade: Updating osd.5 2024-02-16T22:28:01.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:01.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-16T22:28:01.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:28:01.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:01 smithi114 ceph-mon[94452]: Deploying daemon osd.5 on smithi123 2024-02-16T22:28:01.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:01.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:28:03.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:03 smithi123 ceph-mon[82331]: pgmap v156: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:28:03.582 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:03 smithi114 ceph-mon[94452]: pgmap v156: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:28:03.947 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:03 smithi123 systemd[1]: Stopping Ceph osd.5 for 1dcbaabe-cd18-11ee-95ba-87774f69a715... 2024-02-16T22:28:04.296 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:03 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[56928]: 2024-02-16T22:28:03.971+0000 7f3d45791700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T22:28:04.296 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:03 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[56928]: 2024-02-16T22:28:03.971+0000 7f3d45791700 -1 osd.5 73 *** Got signal Terminated *** 2024-02-16T22:28:04.296 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:03 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[56928]: 2024-02-16T22:28:03.971+0000 7f3d45791700 -1 osd.5 73 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T22:28:04.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:04 smithi123 ceph-mon[82331]: osd.5 marked itself down and dead 2024-02-16T22:28:04.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:04 smithi114 ceph-mon[94452]: osd.5 marked itself down and dead 2024-02-16T22:28:05.098 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:04 smithi123 podman[102332]: 2024-02-16 22:28:04.806024136 +0000 UTC m=+0.963672817 container died 24e5bef19fda078e25d6d487956086f3f0601004a3f681e6fb502374b5e7841b (image=quay.io/ceph/ceph@sha256:6176526d5ef4047f148d47c951653dcc6b0de49091fda006d3a088e7a1f404d0, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5, GIT_CLEAN=True, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214) 2024-02-16T22:28:05.473 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:05 smithi123 ceph-mon[82331]: pgmap v157: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:28:05.474 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:05 smithi123 ceph-mon[82331]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T22:28:05.474 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:05 smithi123 ceph-mon[82331]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-02-16T22:28:05.474 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:05 smithi123 ceph-mon[82331]: osdmap e74: 6 total, 5 up, 6 in 2024-02-16T22:28:05.474 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:05 smithi123 podman[102332]: 2024-02-16 22:28:05.212757589 +0000 UTC m=+1.370406266 container cleanup 24e5bef19fda078e25d6d487956086f3f0601004a3f681e6fb502374b5e7841b (image=quay.io/ceph/ceph@sha256:6176526d5ef4047f148d47c951653dcc6b0de49091fda006d3a088e7a1f404d0, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-02-16T22:28:05.474 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:05 smithi123 bash[102332]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5 2024-02-16T22:28:05.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:05 smithi114 ceph-mon[94452]: pgmap v157: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:28:05.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:05 smithi114 ceph-mon[94452]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T22:28:05.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:05 smithi114 ceph-mon[94452]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-02-16T22:28:05.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:05 smithi114 ceph-mon[94452]: osdmap e74: 6 total, 5 up, 6 in 2024-02-16T22:28:05.802 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:05 smithi123 podman[102361]: 2024-02-16 22:28:05.472547788 +0000 UTC m=+0.665147984 container remove 24e5bef19fda078e25d6d487956086f3f0601004a3f681e6fb502374b5e7841b (image=quay.io/ceph/ceph@sha256:6176526d5ef4047f148d47c951653dcc6b0de49091fda006d3a088e7a1f404d0, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5, ceph=True, RELEASE=HEAD, GIT_CLEAN=True, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-02-16T22:28:06.160 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:05 smithi123 podman[102453]: 2024-02-16 22:28:05.782260336 +0000 UTC m=+0.017719347 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T22:28:06.160 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:05 smithi123 podman[102453]: 2024-02-16 22:28:05.881705501 +0000 UTC m=+0.117164511 container create 2dabf2e89794b0abf8c31ee77809f20d1823436739e3e5f077b37ba3dd440491 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5-deactivate, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214) 2024-02-16T22:28:06.160 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:06 smithi123 podman[102453]: 2024-02-16 22:28:06.084110614 +0000 UTC m=+0.319569625 container init 2dabf2e89794b0abf8c31ee77809f20d1823436739e3e5f077b37ba3dd440491 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T22:28:06.160 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:06 smithi123 podman[102453]: 2024-02-16 22:28:06.087217005 +0000 UTC m=+0.322676020 container start 2dabf2e89794b0abf8c31ee77809f20d1823436739e3e5f077b37ba3dd440491 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-02-16T22:28:06.160 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:06 smithi123 podman[102453]: 2024-02-16 22:28:06.159017035 +0000 UTC m=+0.394476064 container attach 2dabf2e89794b0abf8c31ee77809f20d1823436739e3e5f077b37ba3dd440491 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-02-16T22:28:06.431 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:06 smithi123 ceph-mon[82331]: osdmap e75: 6 total, 5 up, 6 in 2024-02-16T22:28:06.432 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:06 smithi123 podman[102453]: 2024-02-16 22:28:06.38228209 +0000 UTC m=+0.617741113 container died 2dabf2e89794b0abf8c31ee77809f20d1823436739e3e5f077b37ba3dd440491 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5-deactivate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214) 2024-02-16T22:28:06.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:06 smithi114 ceph-mon[94452]: osdmap e75: 6 total, 5 up, 6 in 2024-02-16T22:28:07.302 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:07 smithi123 podman[102479]: 2024-02-16 22:28:07.029434537 +0000 UTC m=+0.641791084 container remove 2dabf2e89794b0abf8c31ee77809f20d1823436739e3e5f077b37ba3dd440491 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-02-16T22:28:07.302 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:07 smithi123 systemd[1]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@osd.5.service: Deactivated successfully. 2024-02-16T22:28:07.302 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:07 smithi123 systemd[1]: Stopped Ceph osd.5 for 1dcbaabe-cd18-11ee-95ba-87774f69a715. 2024-02-16T22:28:07.303 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:07 smithi123 systemd[1]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@osd.5.service: Consumed 50.831s CPU time. 2024-02-16T22:28:07.303 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:07 smithi123 systemd[1]: Starting Ceph osd.5 for 1dcbaabe-cd18-11ee-95ba-87774f69a715... 2024-02-16T22:28:07.563 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:07 smithi123 ceph-mon[82331]: pgmap v160: 65 pgs: 13 stale+active+clean, 52 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:28:07.563 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:07 smithi123 podman[102581]: 2024-02-16 22:28:07.362492653 +0000 UTC m=+0.017608238 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T22:28:07.563 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:07 smithi123 podman[102581]: 2024-02-16 22:28:07.464068857 +0000 UTC m=+0.119184438 container create 8ba99a0a123be1249df14ecaead15fa05f39379dcedbf3ee8ac371401a056654 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T22:28:07.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:07 smithi114 ceph-mon[94452]: pgmap v160: 65 pgs: 13 stale+active+clean, 52 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:28:08.052 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:07 smithi123 podman[102581]: 2024-02-16 22:28:07.649827081 +0000 UTC m=+0.304942673 container init 8ba99a0a123be1249df14ecaead15fa05f39379dcedbf3ee8ac371401a056654 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5-activate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214) 2024-02-16T22:28:08.052 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:07 smithi123 podman[102581]: 2024-02-16 22:28:07.653175734 +0000 UTC m=+0.308291325 container start 8ba99a0a123be1249df14ecaead15fa05f39379dcedbf3ee8ac371401a056654 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5-activate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214) 2024-02-16T22:28:08.052 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:07 smithi123 podman[102581]: 2024-02-16 22:28:07.724381939 +0000 UTC m=+0.379497528 container attach 8ba99a0a123be1249df14ecaead15fa05f39379dcedbf3ee8ac371401a056654 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5-activate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux ) 2024-02-16T22:28:08.364 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:08 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5-activate[102596]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-16T22:28:08.364 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:08 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5-activate[102596]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-02-16T22:28:08.364 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:08 smithi123 bash[102581]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-16T22:28:08.365 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:08 smithi123 bash[102581]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-02-16T22:28:08.365 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:08 smithi123 bash[102581]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-02-16T22:28:08.365 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:08 smithi123 bash[102581]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-02-16T22:28:08.365 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:08 smithi123 bash[102581]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-02-16T22:28:08.365 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:08 smithi123 bash[102581]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-16T22:28:08.365 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:08 smithi123 bash[102581]: --> ceph-volume raw activate successful for osd ID: 5 2024-02-16T22:28:08.365 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:08 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5-activate[102596]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-02-16T22:28:08.365 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:08 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5-activate[102596]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-02-16T22:28:08.365 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:08 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5-activate[102596]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-02-16T22:28:08.365 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:08 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5-activate[102596]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-16T22:28:08.365 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:08 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5-activate[102596]: --> ceph-volume raw activate successful for osd ID: 5 2024-02-16T22:28:08.365 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:08 smithi123 podman[102581]: 2024-02-16 22:28:08.118996414 +0000 UTC m=+0.774111999 container died 8ba99a0a123be1249df14ecaead15fa05f39379dcedbf3ee8ac371401a056654 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5-activate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , ceph=True) 2024-02-16T22:28:08.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:08 smithi114 ceph-mon[94452]: Health check failed: Degraded data redundancy: 17/267 objects degraded (6.367%), 7 pgs degraded (PG_DEGRADED) 2024-02-16T22:28:08.720 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:08 smithi123 ceph-mon[82331]: Health check failed: Degraded data redundancy: 17/267 objects degraded (6.367%), 7 pgs degraded (PG_DEGRADED) 2024-02-16T22:28:09.052 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:08 smithi123 podman[102717]: 2024-02-16 22:28:08.719600331 +0000 UTC m=+0.590498418 container remove 8ba99a0a123be1249df14ecaead15fa05f39379dcedbf3ee8ac371401a056654 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5-activate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-02-16T22:28:09.052 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:08 smithi123 podman[102780]: 2024-02-16 22:28:08.838390546 +0000 UTC m=+0.017826635 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T22:28:09.052 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:08 smithi123 podman[102780]: 2024-02-16 22:28:08.937339133 +0000 UTC m=+0.116775221 container create dd99bf03474900061451ba59068106233278253dd5aaf375fca8ac3f5245ef90 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-02-16T22:28:09.365 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:09 smithi123 podman[102780]: 2024-02-16 22:28:09.131052966 +0000 UTC m=+0.310489055 container init dd99bf03474900061451ba59068106233278253dd5aaf375fca8ac3f5245ef90 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-02-16T22:28:09.365 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:09 smithi123 podman[102780]: 2024-02-16 22:28:09.134025715 +0000 UTC m=+0.313461814 container start dd99bf03474900061451ba59068106233278253dd5aaf375fca8ac3f5245ef90 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T22:28:09.366 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:09 smithi123 ceph-osd[102799]: -- 172.21.15.123:0/2104198631 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ecc77331e0 con 0x55ecc7704c00 2024-02-16T22:28:09.366 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:09 smithi123 bash[102780]: dd99bf03474900061451ba59068106233278253dd5aaf375fca8ac3f5245ef90 2024-02-16T22:28:09.366 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:09 smithi123 systemd[1]: Started Ceph osd.5 for 1dcbaabe-cd18-11ee-95ba-87774f69a715. 2024-02-16T22:28:09.677 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:09 smithi123 ceph-mon[82331]: pgmap v161: 65 pgs: 7 active+undersized+degraded, 3 stale+active+clean, 11 active+undersized, 44 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 17/267 objects degraded (6.367%) 2024-02-16T22:28:09.677 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:09 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:09.677 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:09 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:09.677 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:09 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:28:09.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:09 smithi114 ceph-mon[94452]: pgmap v161: 65 pgs: 7 active+undersized+degraded, 3 stale+active+clean, 11 active+undersized, 44 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 17/267 objects degraded (6.367%) 2024-02-16T22:28:09.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:09 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:09.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:09 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:09.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:09 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:28:10.052 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:09 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:28:09.675+0000 7f01817607c0 -1 Falling back to public interface 2024-02-16T22:28:11.649 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:11 smithi123 ceph-mon[82331]: pgmap v162: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 41/267 objects degraded (15.356%) 2024-02-16T22:28:11.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:11 smithi114 ceph-mon[94452]: pgmap v162: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 41/267 objects degraded (15.356%) 2024-02-16T22:28:13.581 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:13 smithi114 ceph-mon[94452]: pgmap v163: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 41/267 objects degraded (15.356%) 2024-02-16T22:28:13.582 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:13 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:13.582 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:13 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:13.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:13 smithi123 ceph-mon[82331]: pgmap v163: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 41/267 objects degraded (15.356%) 2024-02-16T22:28:13.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:13 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:13.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:13 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:14.351 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:14 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:28:14.069+0000 7f01817607c0 -1 osd.5 0 read_superblock omap replica is missing. 2024-02-16T22:28:14.352 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:14 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:28:14.349+0000 7f01817607c0 -1 osd.5 73 log_to_monitors true 2024-02-16T22:28:14.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:14 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:14.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:14 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:14.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:14 smithi114 ceph-mon[94452]: from='osd.5 [v2:172.21.15.123:6816/4210417260,v1:172.21.15.123:6817/4210417260]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T22:28:14.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:14 smithi114 ceph-mon[94452]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T22:28:14.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:14 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:14.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:14 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:14.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:14 smithi123 ceph-mon[82331]: from='osd.5 [v2:172.21.15.123:6816/4210417260,v1:172.21.15.123:6817/4210417260]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T22:28:14.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:14 smithi123 ceph-mon[82331]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T22:28:15.374 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:28:15 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:28:15.349+0000 7f016ebb0700 -1 osd.5 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T22:28:15.627 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:15 smithi123 ceph-mon[82331]: pgmap v164: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 41/267 objects degraded (15.356%) 2024-02-16T22:28:15.627 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:15 smithi123 ceph-mon[82331]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-16T22:28:15.627 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:15 smithi123 ceph-mon[82331]: from='osd.5 [v2:172.21.15.123:6816/4210417260,v1:172.21.15.123:6817/4210417260]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-02-16T22:28:15.627 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:15 smithi123 ceph-mon[82331]: osdmap e76: 6 total, 5 up, 6 in 2024-02-16T22:28:15.627 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:15 smithi123 ceph-mon[82331]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-02-16T22:28:15.627 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:15 smithi123 ceph-mon[82331]: Health check update: Degraded data redundancy: 41/267 objects degraded (15.356%), 13 pgs degraded (PG_DEGRADED) 2024-02-16T22:28:15.627 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:15 smithi123 ceph-mon[82331]: from='osd.5 ' entity='osd.5' 2024-02-16T22:28:15.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:15 smithi114 ceph-mon[94452]: pgmap v164: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 258 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 41/267 objects degraded (15.356%) 2024-02-16T22:28:15.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:15 smithi114 ceph-mon[94452]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-16T22:28:15.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:15 smithi114 ceph-mon[94452]: from='osd.5 [v2:172.21.15.123:6816/4210417260,v1:172.21.15.123:6817/4210417260]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-02-16T22:28:15.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:15 smithi114 ceph-mon[94452]: osdmap e76: 6 total, 5 up, 6 in 2024-02-16T22:28:15.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:15 smithi114 ceph-mon[94452]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-02-16T22:28:15.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:15 smithi114 ceph-mon[94452]: Health check update: Degraded data redundancy: 41/267 objects degraded (15.356%), 13 pgs degraded (PG_DEGRADED) 2024-02-16T22:28:15.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:15 smithi114 ceph-mon[94452]: from='osd.5 ' entity='osd.5' 2024-02-16T22:28:16.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.104+0000 7f5cda87a700 1 -- 172.21.15.114:0/3229228513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cd40ffaf0 msgr2=0x7f5cd40fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:16.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.104+0000 7f5cda87a700 1 --2- 172.21.15.114:0/3229228513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cd40ffaf0 0x7f5cd40fff10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4009910 tx=0x7f5cc4009c20 comp rx=0 tx=0).stop 2024-02-16T22:28:16.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.104+0000 7f5cda87a700 1 -- 172.21.15.114:0/3229228513 shutdown_connections 2024-02-16T22:28:16.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.104+0000 7f5cda87a700 1 --2- 172.21.15.114:0/3229228513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cd4100cf0 0x7f5cd4101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:16.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.104+0000 7f5cda87a700 1 --2- 172.21.15.114:0/3229228513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cd40ffaf0 0x7f5cd40fff10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:16.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.104+0000 7f5cda87a700 1 -- 172.21.15.114:0/3229228513 >> 172.21.15.114:0/3229228513 conn(0x7f5cd40fb070 msgr2=0x7f5cd40fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:28:16.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.104+0000 7f5cda87a700 1 -- 172.21.15.114:0/3229228513 shutdown_connections 2024-02-16T22:28:16.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.104+0000 7f5cda87a700 1 -- 172.21.15.114:0/3229228513 wait complete. 2024-02-16T22:28:16.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.104+0000 7f5cda87a700 1 Processor -- start 2024-02-16T22:28:16.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.104+0000 7f5cda87a700 1 -- start start 2024-02-16T22:28:16.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.105+0000 7f5cda87a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cd40ffaf0 0x7f5cd41953d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:16.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.105+0000 7f5cda87a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cd4100cf0 0x7f5cd4195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:16.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.105+0000 7f5cda87a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd4195f30 con 0x7f5cd40ffaf0 2024-02-16T22:28:16.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.105+0000 7f5cda87a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd4196070 con 0x7f5cd4100cf0 2024-02-16T22:28:16.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.105+0000 7f5cd3fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cd40ffaf0 0x7f5cd41953d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:16.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.105+0000 7f5cd3fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cd40ffaf0 0x7f5cd41953d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50766/0 (socket says 172.21.15.114:50766) 2024-02-16T22:28:16.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.105+0000 7f5cd3fff700 1 -- 172.21.15.114:0/3640187726 learned_addr learned my addr 172.21.15.114:0/3640187726 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:28:16.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.105+0000 7f5cd37fe700 1 --2- 172.21.15.114:0/3640187726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cd4100cf0 0x7f5cd4195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:16.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.105+0000 7f5cd3fff700 1 -- 172.21.15.114:0/3640187726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cd4100cf0 msgr2=0x7f5cd4195910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:16.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.105+0000 7f5cd3fff700 1 --2- 172.21.15.114:0/3640187726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cd4100cf0 0x7f5cd4195910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:16.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.105+0000 7f5cd3fff700 1 -- 172.21.15.114:0/3640187726 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc40095f0 con 0x7f5cd40ffaf0 2024-02-16T22:28:16.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.105+0000 7f5cd3fff700 1 --2- 172.21.15.114:0/3640187726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cd40ffaf0 0x7f5cd41953d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400bd70 tx=0x7f5cc400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:28:16.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.106+0000 7f5cd17fa700 1 -- 172.21.15.114:0/3640187726 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cc401f070 con 0x7f5cd40ffaf0 2024-02-16T22:28:16.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.106+0000 7f5cd17fa700 1 -- 172.21.15.114:0/3640187726 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5cc4019610 con 0x7f5cd40ffaf0 2024-02-16T22:28:16.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.106+0000 7f5cda87a700 1 -- 172.21.15.114:0/3640187726 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd4068d90 con 0x7f5cd40ffaf0 2024-02-16T22:28:16.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.106+0000 7f5cd17fa700 1 -- 172.21.15.114:0/3640187726 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cc400f690 con 0x7f5cd40ffaf0 2024-02-16T22:28:16.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.106+0000 7f5cda87a700 1 -- 172.21.15.114:0/3640187726 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd4069300 con 0x7f5cd40ffaf0 2024-02-16T22:28:16.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.107+0000 7f5cda87a700 1 -- 172.21.15.114:0/3640187726 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd41086d0 con 0x7f5cd40ffaf0 2024-02-16T22:28:16.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.107+0000 7f5cd17fa700 1 -- 172.21.15.114:0/3640187726 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f5cc4019780 con 0x7f5cd40ffaf0 2024-02-16T22:28:16.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.108+0000 7f5cd17fa700 1 --2- 172.21.15.114:0/3640187726 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f5cbc06d8a0 0x7f5cbc06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:16.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.108+0000 7f5cd37fe700 1 --2- 172.21.15.114:0/3640187726 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f5cbc06d8a0 0x7f5cbc06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:16.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.108+0000 7f5cd17fa700 1 -- 172.21.15.114:0/3640187726 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f5cc408e520 con 0x7f5cd40ffaf0 2024-02-16T22:28:16.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.108+0000 7f5cd37fe700 1 --2- 172.21.15.114:0/3640187726 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f5cbc06d8a0 0x7f5cbc06fd60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009a10 tx=0x7f5cc8009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:28:16.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.112+0000 7f5cd17fa700 1 -- 172.21.15.114:0/3640187726 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5cc405b930 con 0x7f5cd40ffaf0 2024-02-16T22:28:16.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.324+0000 7f5cda87a700 1 -- 172.21.15.114:0/3640187726 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5cd4105640 con 0x7f5cbc06d8a0 2024-02-16T22:28:16.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.325+0000 7f5cd17fa700 1 -- 172.21.15.114:0/3640187726 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5cd4105640 con 0x7f5cbc06d8a0 2024-02-16T22:28:16.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.327+0000 7f5cda87a700 1 -- 172.21.15.114:0/3640187726 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f5cbc06d8a0 msgr2=0x7f5cbc06fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:16.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.327+0000 7f5cda87a700 1 --2- 172.21.15.114:0/3640187726 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f5cbc06d8a0 0x7f5cbc06fd60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009a10 tx=0x7f5cc8009190 comp rx=0 tx=0).stop 2024-02-16T22:28:16.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.327+0000 7f5cda87a700 1 -- 172.21.15.114:0/3640187726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cd40ffaf0 msgr2=0x7f5cd41953d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:16.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.327+0000 7f5cda87a700 1 --2- 172.21.15.114:0/3640187726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cd40ffaf0 0x7f5cd41953d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400bd70 tx=0x7f5cc400bda0 comp rx=0 tx=0).stop 2024-02-16T22:28:16.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.327+0000 7f5cda87a700 1 -- 172.21.15.114:0/3640187726 shutdown_connections 2024-02-16T22:28:16.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.327+0000 7f5cda87a700 1 --2- 172.21.15.114:0/3640187726 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f5cbc06d8a0 0x7f5cbc06fd60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:16.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.327+0000 7f5cda87a700 1 --2- 172.21.15.114:0/3640187726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cd40ffaf0 0x7f5cd41953d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:16.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.327+0000 7f5cda87a700 1 --2- 172.21.15.114:0/3640187726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cd4100cf0 0x7f5cd4195910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:16.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.327+0000 7f5cda87a700 1 -- 172.21.15.114:0/3640187726 >> 172.21.15.114:0/3640187726 conn(0x7f5cd40fb070 msgr2=0x7f5cd4103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:28:16.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.327+0000 7f5cda87a700 1 -- 172.21.15.114:0/3640187726 shutdown_connections 2024-02-16T22:28:16.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.327+0000 7f5cda87a700 1 -- 172.21.15.114:0/3640187726 wait complete. 2024-02-16T22:28:16.340 INFO:teuthology.orchestra.run.smithi114.stdout:true 2024-02-16T22:28:16.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.439+0000 7fad7d472700 1 -- 172.21.15.114:0/2297952112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad780ffab0 msgr2=0x7fad780ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:16.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.439+0000 7fad7d472700 1 --2- 172.21.15.114:0/2297952112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad780ffab0 0x7fad780ffed0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fad68009910 tx=0x7fad68009c20 comp rx=0 tx=0).stop 2024-02-16T22:28:16.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.439+0000 7fad7d472700 1 -- 172.21.15.114:0/2297952112 shutdown_connections 2024-02-16T22:28:16.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.439+0000 7fad7d472700 1 --2- 172.21.15.114:0/2297952112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad78100cb0 0x7fad78101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:16.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.439+0000 7fad7d472700 1 --2- 172.21.15.114:0/2297952112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad780ffab0 0x7fad780ffed0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:16.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.439+0000 7fad7d472700 1 -- 172.21.15.114:0/2297952112 >> 172.21.15.114:0/2297952112 conn(0x7fad780fb030 msgr2=0x7fad780fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:28:16.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.439+0000 7fad7d472700 1 -- 172.21.15.114:0/2297952112 shutdown_connections 2024-02-16T22:28:16.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.439+0000 7fad7d472700 1 -- 172.21.15.114:0/2297952112 wait complete. 2024-02-16T22:28:16.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.440+0000 7fad7d472700 1 Processor -- start 2024-02-16T22:28:16.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.440+0000 7fad7d472700 1 -- start start 2024-02-16T22:28:16.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.440+0000 7fad7d472700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad780ffab0 0x7fad781952e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:16.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.440+0000 7fad7d472700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad78100cb0 0x7fad78195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:16.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.440+0000 7fad7d472700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad78195e40 con 0x7fad780ffab0 2024-02-16T22:28:16.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.440+0000 7fad7d472700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad78195f80 con 0x7fad78100cb0 2024-02-16T22:28:16.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.440+0000 7fad76ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad780ffab0 0x7fad781952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:16.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.440+0000 7fad76ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad780ffab0 0x7fad781952e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50774/0 (socket says 172.21.15.114:50774) 2024-02-16T22:28:16.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.440+0000 7fad76ffd700 1 -- 172.21.15.114:0/3623405840 learned_addr learned my addr 172.21.15.114:0/3623405840 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:28:16.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.441+0000 7fad767fc700 1 --2- 172.21.15.114:0/3623405840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad78100cb0 0x7fad78195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:16.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.441+0000 7fad76ffd700 1 -- 172.21.15.114:0/3623405840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad78100cb0 msgr2=0x7fad78195820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:16.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.441+0000 7fad76ffd700 1 --2- 172.21.15.114:0/3623405840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad78100cb0 0x7fad78195820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:16.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.441+0000 7fad76ffd700 1 -- 172.21.15.114:0/3623405840 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad680095f0 con 0x7fad780ffab0 2024-02-16T22:28:16.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.441+0000 7fad76ffd700 1 --2- 172.21.15.114:0/3623405840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad780ffab0 0x7fad781952e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fad6800bda0 tx=0x7fad6800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:28:16.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.441+0000 7fad5ffff700 1 -- 172.21.15.114:0/3623405840 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad6801d070 con 0x7fad780ffab0 2024-02-16T22:28:16.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.441+0000 7fad7d472700 1 -- 172.21.15.114:0/3623405840 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad7819a9d0 con 0x7fad780ffab0 2024-02-16T22:28:16.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.441+0000 7fad5ffff700 1 -- 172.21.15.114:0/3623405840 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fad68004c50 con 0x7fad780ffab0 2024-02-16T22:28:16.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.441+0000 7fad5ffff700 1 -- 172.21.15.114:0/3623405840 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad6800f690 con 0x7fad780ffab0 2024-02-16T22:28:16.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.441+0000 7fad7d472700 1 -- 172.21.15.114:0/3623405840 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad7819af40 con 0x7fad780ffab0 2024-02-16T22:28:16.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.442+0000 7fad7d472700 1 -- 172.21.15.114:0/3623405840 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad78108690 con 0x7fad780ffab0 2024-02-16T22:28:16.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.443+0000 7fad5ffff700 1 -- 172.21.15.114:0/3623405840 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fad68022a20 con 0x7fad780ffab0 2024-02-16T22:28:16.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.443+0000 7fad5ffff700 1 --2- 172.21.15.114:0/3623405840 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fad6006d8a0 0x7fad6006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:16.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.443+0000 7fad5ffff700 1 -- 172.21.15.114:0/3623405840 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fad6808ed30 con 0x7fad780ffab0 2024-02-16T22:28:16.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.443+0000 7fad767fc700 1 --2- 172.21.15.114:0/3623405840 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fad6006d8a0 0x7fad6006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:16.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.444+0000 7fad767fc700 1 --2- 172.21.15.114:0/3623405840 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fad6006d8a0 0x7fad6006fd60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fad6c006fd0 tx=0x7fad6c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:28:16.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.448+0000 7fad5ffff700 1 -- 172.21.15.114:0/3623405840 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fad6805c310 con 0x7fad780ffab0 2024-02-16T22:28:16.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.661+0000 7fad7d472700 1 -- 172.21.15.114:0/3623405840 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fad78105600 con 0x7fad6006d8a0 2024-02-16T22:28:16.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.662+0000 7fad5ffff700 1 -- 172.21.15.114:0/3623405840 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fad78105600 con 0x7fad6006d8a0 2024-02-16T22:28:16.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.664+0000 7fad7d472700 1 -- 172.21.15.114:0/3623405840 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fad6006d8a0 msgr2=0x7fad6006fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:16.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.664+0000 7fad7d472700 1 --2- 172.21.15.114:0/3623405840 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fad6006d8a0 0x7fad6006fd60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fad6c006fd0 tx=0x7fad6c008040 comp rx=0 tx=0).stop 2024-02-16T22:28:16.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.664+0000 7fad7d472700 1 -- 172.21.15.114:0/3623405840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad780ffab0 msgr2=0x7fad781952e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:16.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.664+0000 7fad7d472700 1 --2- 172.21.15.114:0/3623405840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad780ffab0 0x7fad781952e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fad6800bda0 tx=0x7fad6800be80 comp rx=0 tx=0).stop 2024-02-16T22:28:16.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.664+0000 7fad7d472700 1 -- 172.21.15.114:0/3623405840 shutdown_connections 2024-02-16T22:28:16.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.664+0000 7fad7d472700 1 --2- 172.21.15.114:0/3623405840 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fad6006d8a0 0x7fad6006fd60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:16.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.664+0000 7fad7d472700 1 --2- 172.21.15.114:0/3623405840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad780ffab0 0x7fad781952e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:16.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.664+0000 7fad7d472700 1 --2- 172.21.15.114:0/3623405840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad78100cb0 0x7fad78195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:16.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.664+0000 7fad7d472700 1 -- 172.21.15.114:0/3623405840 >> 172.21.15.114:0/3623405840 conn(0x7fad780fb030 msgr2=0x7fad78103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:28:16.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.665+0000 7fad7d472700 1 -- 172.21.15.114:0/3623405840 shutdown_connections 2024-02-16T22:28:16.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.665+0000 7fad7d472700 1 -- 172.21.15.114:0/3623405840 wait complete. 2024-02-16T22:28:16.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.768+0000 7ff5c2f8d700 1 -- 172.21.15.114:0/3114566762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5bc101eb0 msgr2=0x7ff5bc102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:16.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.768+0000 7ff5c2f8d700 1 --2- 172.21.15.114:0/3114566762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5bc101eb0 0x7ff5bc102310 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8009910 tx=0x7ff5a8009c20 comp rx=0 tx=0).stop 2024-02-16T22:28:16.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.768+0000 7ff5c2f8d700 1 -- 172.21.15.114:0/3114566762 shutdown_connections 2024-02-16T22:28:16.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.768+0000 7ff5c2f8d700 1 --2- 172.21.15.114:0/3114566762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5bc101eb0 0x7ff5bc102310 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:16.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.769+0000 7ff5c2f8d700 1 --2- 172.21.15.114:0/3114566762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5bc0feed0 0x7ff5bc0ff2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:16.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.769+0000 7ff5c2f8d700 1 -- 172.21.15.114:0/3114566762 >> 172.21.15.114:0/3114566762 conn(0x7ff5bc0fa810 msgr2=0x7ff5bc0fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:28:16.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.769+0000 7ff5c2f8d700 1 -- 172.21.15.114:0/3114566762 shutdown_connections 2024-02-16T22:28:16.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.769+0000 7ff5c2f8d700 1 -- 172.21.15.114:0/3114566762 wait complete. 2024-02-16T22:28:16.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.769+0000 7ff5c2f8d700 1 Processor -- start 2024-02-16T22:28:16.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.769+0000 7ff5c2f8d700 1 -- start start 2024-02-16T22:28:16.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.769+0000 7ff5c2f8d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5bc0feed0 0x7ff5bc1963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:16.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.769+0000 7ff5c2f8d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5bc101eb0 0x7ff5bc196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:16.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.769+0000 7ff5c2f8d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc196f50 con 0x7ff5bc0feed0 2024-02-16T22:28:16.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.769+0000 7ff5c2f8d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc197090 con 0x7ff5bc101eb0 2024-02-16T22:28:16.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.769+0000 7ff5c0d29700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5bc0feed0 0x7ff5bc1963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:16.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.769+0000 7ff5c0d29700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5bc0feed0 0x7ff5bc1963f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50806/0 (socket says 172.21.15.114:50806) 2024-02-16T22:28:16.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.769+0000 7ff5c0d29700 1 -- 172.21.15.114:0/928003724 learned_addr learned my addr 172.21.15.114:0/928003724 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:28:16.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.769+0000 7ff5bbfff700 1 --2- 172.21.15.114:0/928003724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5bc101eb0 0x7ff5bc196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:16.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.770+0000 7ff5c0d29700 1 -- 172.21.15.114:0/928003724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5bc101eb0 msgr2=0x7ff5bc196930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:16.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.770+0000 7ff5c0d29700 1 --2- 172.21.15.114:0/928003724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5bc101eb0 0x7ff5bc196930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:16.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.770+0000 7ff5c0d29700 1 -- 172.21.15.114:0/928003724 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a80095f0 con 0x7ff5bc0feed0 2024-02-16T22:28:16.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.770+0000 7ff5bbfff700 1 --2- 172.21.15.114:0/928003724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5bc101eb0 0x7ff5bc196930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:28:16.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.770+0000 7ff5c0d29700 1 --2- 172.21.15.114:0/928003724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5bc0feed0 0x7ff5bc1963f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff5b000e8f0 tx=0x7ff5b000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:28:16.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.770+0000 7ff5b9ffb700 1 -- 172.21.15.114:0/928003724 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5b0009790 con 0x7ff5bc0feed0 2024-02-16T22:28:16.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.770+0000 7ff5b9ffb700 1 -- 172.21.15.114:0/928003724 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff5b000c810 con 0x7ff5bc0feed0 2024-02-16T22:28:16.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.770+0000 7ff5b9ffb700 1 -- 172.21.15.114:0/928003724 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5b0003d20 con 0x7ff5bc0feed0 2024-02-16T22:28:16.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.770+0000 7ff5c2f8d700 1 -- 172.21.15.114:0/928003724 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5bc19bb40 con 0x7ff5bc0feed0 2024-02-16T22:28:16.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.770+0000 7ff5c2f8d700 1 -- 172.21.15.114:0/928003724 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5bc19c090 con 0x7ff5bc0feed0 2024-02-16T22:28:16.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.771+0000 7ff5c2f8d700 1 -- 172.21.15.114:0/928003724 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5bc109770 con 0x7ff5bc0feed0 2024-02-16T22:28:16.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.771+0000 7ff5b9ffb700 1 -- 172.21.15.114:0/928003724 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7ff5b00041f0 con 0x7ff5bc0feed0 2024-02-16T22:28:16.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.772+0000 7ff5b9ffb700 1 --2- 172.21.15.114:0/928003724 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff5ac06d8a0 0x7ff5ac06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:16.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.772+0000 7ff5b9ffb700 1 -- 172.21.15.114:0/928003724 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff5b0014070 con 0x7ff5bc0feed0 2024-02-16T22:28:16.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.772+0000 7ff5bbfff700 1 --2- 172.21.15.114:0/928003724 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff5ac06d8a0 0x7ff5ac06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:16.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.772+0000 7ff5bbfff700 1 --2- 172.21.15.114:0/928003724 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff5ac06d8a0 0x7ff5ac06fd60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80059d0 tx=0x7ff5a8005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:28:16.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.776+0000 7ff5b9ffb700 1 -- 172.21.15.114:0/928003724 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff5b005aac0 con 0x7ff5bc0feed0 2024-02-16T22:28:16.981 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:16 smithi123 ceph-mon[82331]: pgmap v166: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 41/267 objects degraded (15.356%) 2024-02-16T22:28:16.982 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:16 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:16.982 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:16 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:28:16.982 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:16 smithi123 ceph-mon[82331]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:28:16.982 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:16 smithi123 ceph-mon[82331]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T22:28:16.982 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:16 smithi123 ceph-mon[82331]: osd.5 [v2:172.21.15.123:6816/4210417260,v1:172.21.15.123:6817/4210417260] boot 2024-02-16T22:28:16.982 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:16 smithi123 ceph-mon[82331]: osdmap e77: 6 total, 6 up, 6 in 2024-02-16T22:28:16.982 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:16 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:28:16.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.982+0000 7ff5c2f8d700 1 -- 172.21.15.114:0/928003724 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5bc106800 con 0x7ff5ac06d8a0 2024-02-16T22:28:16.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.993+0000 7ff5b9ffb700 1 -- 172.21.15.114:0/928003724 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7ff5bc106800 con 0x7ff5ac06d8a0 2024-02-16T22:28:16.995 INFO:teuthology.orchestra.run.smithi114.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:28:16.995 INFO:teuthology.orchestra.run.smithi114.stdout:alertmanager.smithi114 smithi114 *:9093,9094 running (14m) 104s ago 15m 24.0M - 0.25.0 c8568f914cd2 e7404fea400e 2024-02-16T22:28:16.995 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi114 smithi114 running (15m) 104s ago 15m 8506k - 18.2.1 94eadd990e0a 7fd07162cd96 2024-02-16T22:28:16.995 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi123 smithi123 running (14m) 4s ago 14m 8342k - 18.2.1 94eadd990e0a 7687f5e99621 2024-02-16T22:28:16.995 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi114 smithi114 running (3m) 104s ago 15m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 9b8ae9efe83b 2024-02-16T22:28:16.995 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi123 smithi123 running (3m) 4s ago 14m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 6d97263a0ffa 2024-02-16T22:28:16.995 INFO:teuthology.orchestra.run.smithi114.stdout:grafana.smithi114 smithi114 *:3000 running (14m) 104s ago 15m 88.4M - 9.4.7 954c08fa6188 db1c26c3bc20 2024-02-16T22:28:16.995 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.fdxpey smithi114 running (10m) 104s ago 10m 18.5M - 18.2.1 94eadd990e0a 704b40c144fb 2024-02-16T22:28:16.996 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.zzryke smithi114 running (10m) 104s ago 10m 157M - 18.2.1 94eadd990e0a 1f795d23c9a4 2024-02-16T22:28:16.996 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.gopcfa smithi123 running (10m) 4s ago 10m 18.6M - 18.2.1 94eadd990e0a 166134b7532a 2024-02-16T22:28:16.996 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.vhlqbe smithi123 running (10m) 4s ago 10m 20.3M - 18.2.1 94eadd990e0a 243c8bb96fa8 2024-02-16T22:28:16.996 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi114.vcullz smithi114 *:8443,9283,8765 running (6m) 104s ago 17m 500M - 19.0.0-1450-gf78a58c0 f081f847070a 14ebeb7888c0 2024-02-16T22:28:16.996 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi123.inzgjd smithi123 *:8443,9283,8765 running (5m) 4s ago 14m 450M - 19.0.0-1450-gf78a58c0 f081f847070a f10aab9faf9e 2024-02-16T22:28:16.996 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi114 smithi114 running (4m) 104s ago 17m 51.5M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 58ac06372523 2024-02-16T22:28:16.996 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi123 smithi123 running (4m) 4s ago 14m 37.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a f76485e0ea24 2024-02-16T22:28:16.996 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi114 smithi114 *:9100 running (15m) 104s ago 15m 15.9M - 1.5.0 0da6a335fe13 744a090e06d0 2024-02-16T22:28:16.996 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi123 smithi123 *:9100 running (14m) 4s ago 14m 14.9M - 1.5.0 0da6a335fe13 3994b8c29b2e 2024-02-16T22:28:16.996 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 smithi114 running (3m) 104s ago 13m 91.2M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 1978fdad94a5 2024-02-16T22:28:16.996 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 smithi114 running (2m) 104s ago 13m 90.2M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 3f90e792898f 2024-02-16T22:28:16.996 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 smithi114 running (110s) 104s ago 12m 12.3M 1216M 19.0.0-1450-gf78a58c0 f081f847070a c663dfa0518b 2024-02-16T22:28:16.996 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 smithi123 running (74s) 4s ago 12m 99.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a b921448ba169 2024-02-16T22:28:16.997 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 smithi123 running (41s) 4s ago 12m 97.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 54c2926d56b1 2024-02-16T22:28:16.997 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 smithi123 running (8s) 4s ago 11m 12.6M 2240M 19.0.0-1450-gf78a58c0 f081f847070a dd99bf034749 2024-02-16T22:28:16.997 INFO:teuthology.orchestra.run.smithi114.stdout:prometheus.smithi114 smithi114 *:9095 running (5m) 104s ago 15m 57.8M - 2.43.0 a07b618ecd1d 37aac985cdd7 2024-02-16T22:28:16.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.995+0000 7ff5c2f8d700 1 -- 172.21.15.114:0/928003724 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff5ac06d8a0 msgr2=0x7ff5ac06fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:16.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.995+0000 7ff5c2f8d700 1 --2- 172.21.15.114:0/928003724 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff5ac06d8a0 0x7ff5ac06fd60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80059d0 tx=0x7ff5a8005940 comp rx=0 tx=0).stop 2024-02-16T22:28:16.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.995+0000 7ff5c2f8d700 1 -- 172.21.15.114:0/928003724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5bc0feed0 msgr2=0x7ff5bc1963f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:16.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.995+0000 7ff5c2f8d700 1 --2- 172.21.15.114:0/928003724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5bc0feed0 0x7ff5bc1963f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff5b000e8f0 tx=0x7ff5b000ec00 comp rx=0 tx=0).stop 2024-02-16T22:28:16.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.995+0000 7ff5c2f8d700 1 -- 172.21.15.114:0/928003724 shutdown_connections 2024-02-16T22:28:16.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.995+0000 7ff5c2f8d700 1 --2- 172.21.15.114:0/928003724 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff5ac06d8a0 0x7ff5ac06fd60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:16.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.995+0000 7ff5c2f8d700 1 --2- 172.21.15.114:0/928003724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5bc0feed0 0x7ff5bc1963f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:16.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.995+0000 7ff5c2f8d700 1 --2- 172.21.15.114:0/928003724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5bc101eb0 0x7ff5bc196930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:16.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.995+0000 7ff5c2f8d700 1 -- 172.21.15.114:0/928003724 >> 172.21.15.114:0/928003724 conn(0x7ff5bc0fa810 msgr2=0x7ff5bc1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:28:16.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.995+0000 7ff5c2f8d700 1 -- 172.21.15.114:0/928003724 shutdown_connections 2024-02-16T22:28:16.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:16.995+0000 7ff5c2f8d700 1 -- 172.21.15.114:0/928003724 wait complete. 2024-02-16T22:28:17.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.111+0000 7f1166e8f700 1 -- 172.21.15.114:0/408721182 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1160100cd0 msgr2=0x7f1160101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:17.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.111+0000 7f1166e8f700 1 --2- 172.21.15.114:0/408721182 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1160100cd0 0x7f1160101130 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f114c009910 tx=0x7f114c009c20 comp rx=0 tx=0).stop 2024-02-16T22:28:17.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.111+0000 7f1166e8f700 1 -- 172.21.15.114:0/408721182 shutdown_connections 2024-02-16T22:28:17.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.111+0000 7f1166e8f700 1 --2- 172.21.15.114:0/408721182 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1160100cd0 0x7f1160101130 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:17.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.111+0000 7f1166e8f700 1 --2- 172.21.15.114:0/408721182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11600ffad0 0x7f11600ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:17.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.111+0000 7f1166e8f700 1 -- 172.21.15.114:0/408721182 >> 172.21.15.114:0/408721182 conn(0x7f11600fb050 msgr2=0x7f11600fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:28:17.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.111+0000 7f1166e8f700 1 -- 172.21.15.114:0/408721182 shutdown_connections 2024-02-16T22:28:17.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.111+0000 7f1166e8f700 1 -- 172.21.15.114:0/408721182 wait complete. 2024-02-16T22:28:17.115 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:16 smithi114 ceph-mon[94452]: pgmap v166: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 41/267 objects degraded (15.356%) 2024-02-16T22:28:17.116 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:16 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:17.116 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:16 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:28:17.116 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:16 smithi114 ceph-mon[94452]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:28:17.116 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:16 smithi114 ceph-mon[94452]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T22:28:17.116 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:16 smithi114 ceph-mon[94452]: osd.5 [v2:172.21.15.123:6816/4210417260,v1:172.21.15.123:6817/4210417260] boot 2024-02-16T22:28:17.116 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:16 smithi114 ceph-mon[94452]: osdmap e77: 6 total, 6 up, 6 in 2024-02-16T22:28:17.117 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:16 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T22:28:17.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.112+0000 7f1166e8f700 1 Processor -- start 2024-02-16T22:28:17.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.112+0000 7f1166e8f700 1 -- start start 2024-02-16T22:28:17.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.112+0000 7f1166e8f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11600ffad0 0x7f1160069890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:17.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.112+0000 7f1166e8f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1160100cd0 0x7f116019b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:17.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.112+0000 7f1166e8f700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116019b6b0 con 0x7f1160100cd0 2024-02-16T22:28:17.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.112+0000 7f1166e8f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116019b7f0 con 0x7f11600ffad0 2024-02-16T22:28:17.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.112+0000 7f115ffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1160100cd0 0x7f116019b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:17.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.112+0000 7f115ffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1160100cd0 0x7f116019b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50834/0 (socket says 172.21.15.114:50834) 2024-02-16T22:28:17.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.112+0000 7f115ffff700 1 -- 172.21.15.114:0/3414876170 learned_addr learned my addr 172.21.15.114:0/3414876170 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:28:17.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.112+0000 7f1164c2b700 1 --2- 172.21.15.114:0/3414876170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11600ffad0 0x7f1160069890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:17.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.112+0000 7f115ffff700 1 -- 172.21.15.114:0/3414876170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11600ffad0 msgr2=0x7f1160069890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:17.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.112+0000 7f115ffff700 1 --2- 172.21.15.114:0/3414876170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11600ffad0 0x7f1160069890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:17.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.112+0000 7f115ffff700 1 -- 172.21.15.114:0/3414876170 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114c0095f0 con 0x7f1160100cd0 2024-02-16T22:28:17.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.112+0000 7f115ffff700 1 --2- 172.21.15.114:0/3414876170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1160100cd0 0x7f116019b0e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f114c000c00 tx=0x7f114c00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:28:17.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.113+0000 7f115dffb700 1 -- 172.21.15.114:0/3414876170 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f114c01d070 con 0x7f1160100cd0 2024-02-16T22:28:17.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.113+0000 7f115dffb700 1 -- 172.21.15.114:0/3414876170 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f114c005dc0 con 0x7f1160100cd0 2024-02-16T22:28:17.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.113+0000 7f1166e8f700 1 -- 172.21.15.114:0/3414876170 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f116019ba70 con 0x7f1160100cd0 2024-02-16T22:28:17.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.113+0000 7f115dffb700 1 -- 172.21.15.114:0/3414876170 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f114c0226a0 con 0x7f1160100cd0 2024-02-16T22:28:17.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.113+0000 7f1166e8f700 1 -- 172.21.15.114:0/3414876170 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f116019bf60 con 0x7f1160100cd0 2024-02-16T22:28:17.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.114+0000 7f1166e8f700 1 -- 172.21.15.114:0/3414876170 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1160108630 con 0x7f1160100cd0 2024-02-16T22:28:17.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.115+0000 7f115dffb700 1 -- 172.21.15.114:0/3414876170 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f114c003680 con 0x7f1160100cd0 2024-02-16T22:28:17.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.115+0000 7f115dffb700 1 --2- 172.21.15.114:0/3414876170 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f115006d8a0 0x7f115006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:17.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.115+0000 7f115dffb700 1 -- 172.21.15.114:0/3414876170 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f114c08ea30 con 0x7f1160100cd0 2024-02-16T22:28:17.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.115+0000 7f1164c2b700 1 --2- 172.21.15.114:0/3414876170 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f115006d8a0 0x7f115006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:17.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.116+0000 7f1164c2b700 1 --2- 172.21.15.114:0/3414876170 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f115006d8a0 0x7f115006fd60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1160100b30 tx=0x7f1154009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:28:17.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.122+0000 7f115dffb700 1 -- 172.21.15.114:0/3414876170 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f114c05bdc0 con 0x7f1160100cd0 2024-02-16T22:28:17.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.425+0000 7f1166e8f700 1 -- 172.21.15.114:0/3414876170 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f116019c5a0 con 0x7f1160100cd0 2024-02-16T22:28:17.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.426+0000 7f115dffb700 1 -- 172.21.15.114:0/3414876170 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+701 (secure 0 0 0) 0x7f114c003930 con 0x7f1160100cd0 2024-02-16T22:28:17.428 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:28:17.428 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": { 2024-02-16T22:28:17.428 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:28:17.428 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:28:17.428 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": { 2024-02-16T22:28:17.428 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:28:17.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:28:17.429 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": { 2024-02-16T22:28:17.429 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T22:28:17.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:28:17.429 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-02-16T22:28:17.429 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-02-16T22:28:17.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:28:17.429 INFO:teuthology.orchestra.run.smithi114.stdout: "overall": { 2024-02-16T22:28:17.429 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4, 2024-02-16T22:28:17.429 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 10 2024-02-16T22:28:17.429 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-02-16T22:28:17.429 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:28:17.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.428+0000 7f1166e8f700 1 -- 172.21.15.114:0/3414876170 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f115006d8a0 msgr2=0x7f115006fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:17.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.428+0000 7f1166e8f700 1 --2- 172.21.15.114:0/3414876170 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f115006d8a0 0x7f115006fd60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1160100b30 tx=0x7f1154009260 comp rx=0 tx=0).stop 2024-02-16T22:28:17.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.428+0000 7f1166e8f700 1 -- 172.21.15.114:0/3414876170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1160100cd0 msgr2=0x7f116019b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:17.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.428+0000 7f1166e8f700 1 --2- 172.21.15.114:0/3414876170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1160100cd0 0x7f116019b0e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f114c000c00 tx=0x7f114c00f740 comp rx=0 tx=0).stop 2024-02-16T22:28:17.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.428+0000 7f1166e8f700 1 -- 172.21.15.114:0/3414876170 shutdown_connections 2024-02-16T22:28:17.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.428+0000 7f1166e8f700 1 --2- 172.21.15.114:0/3414876170 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f115006d8a0 0x7f115006fd60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:17.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.428+0000 7f1166e8f700 1 --2- 172.21.15.114:0/3414876170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11600ffad0 0x7f1160069890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:17.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.428+0000 7f1166e8f700 1 --2- 172.21.15.114:0/3414876170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1160100cd0 0x7f116019b0e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:17.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.428+0000 7f1166e8f700 1 -- 172.21.15.114:0/3414876170 >> 172.21.15.114:0/3414876170 conn(0x7f11600fb050 msgr2=0x7f1160103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:28:17.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.428+0000 7f1166e8f700 1 -- 172.21.15.114:0/3414876170 shutdown_connections 2024-02-16T22:28:17.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.428+0000 7f1166e8f700 1 -- 172.21.15.114:0/3414876170 wait complete. 2024-02-16T22:28:17.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.537+0000 7f37723b0700 1 -- 172.21.15.114:0/101151632 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f376c0ffab0 msgr2=0x7f376c0ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:17.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.537+0000 7f37723b0700 1 --2- 172.21.15.114:0/101151632 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f376c0ffab0 0x7f376c0ffed0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3760009910 tx=0x7f3760009c20 comp rx=0 tx=0).stop 2024-02-16T22:28:17.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.537+0000 7f37723b0700 1 -- 172.21.15.114:0/101151632 shutdown_connections 2024-02-16T22:28:17.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.537+0000 7f37723b0700 1 --2- 172.21.15.114:0/101151632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f376c100cb0 0x7f376c101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:17.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.537+0000 7f37723b0700 1 --2- 172.21.15.114:0/101151632 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f376c0ffab0 0x7f376c0ffed0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:17.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.537+0000 7f37723b0700 1 -- 172.21.15.114:0/101151632 >> 172.21.15.114:0/101151632 conn(0x7f376c0fb030 msgr2=0x7f376c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:28:17.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.537+0000 7f37723b0700 1 -- 172.21.15.114:0/101151632 shutdown_connections 2024-02-16T22:28:17.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.537+0000 7f37723b0700 1 -- 172.21.15.114:0/101151632 wait complete. 2024-02-16T22:28:17.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.537+0000 7f37723b0700 1 Processor -- start 2024-02-16T22:28:17.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.537+0000 7f37723b0700 1 -- start start 2024-02-16T22:28:17.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.538+0000 7f37723b0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f376c0ffab0 0x7f376c068f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:17.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.538+0000 7f37723b0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f376c100cb0 0x7f376c19d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:17.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.538+0000 7f37723b0700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376c196fb0 con 0x7f376c0ffab0 2024-02-16T22:28:17.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.538+0000 7f37723b0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376c19da80 con 0x7f376c100cb0 2024-02-16T22:28:17.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.538+0000 7f376bfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f376c0ffab0 0x7f376c068f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:17.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.538+0000 7f376bfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f376c0ffab0 0x7f376c068f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50844/0 (socket says 172.21.15.114:50844) 2024-02-16T22:28:17.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.538+0000 7f376bfff700 1 -- 172.21.15.114:0/4196787566 learned_addr learned my addr 172.21.15.114:0/4196787566 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:28:17.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.538+0000 7f376b7fe700 1 --2- 172.21.15.114:0/4196787566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f376c100cb0 0x7f376c19d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:17.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.538+0000 7f376bfff700 1 -- 172.21.15.114:0/4196787566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f376c100cb0 msgr2=0x7f376c19d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:17.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.538+0000 7f376bfff700 1 --2- 172.21.15.114:0/4196787566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f376c100cb0 0x7f376c19d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:17.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.538+0000 7f376bfff700 1 -- 172.21.15.114:0/4196787566 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37600095f0 con 0x7f376c0ffab0 2024-02-16T22:28:17.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.538+0000 7f376bfff700 1 --2- 172.21.15.114:0/4196787566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f376c0ffab0 0x7f376c068f70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f376000bdd0 tx=0x7f376000beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:28:17.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.538+0000 7f37697fa700 1 -- 172.21.15.114:0/4196787566 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f376001d070 con 0x7f376c0ffab0 2024-02-16T22:28:17.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.538+0000 7f37723b0700 1 -- 172.21.15.114:0/4196787566 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f376c19dcd0 con 0x7f376c0ffab0 2024-02-16T22:28:17.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.539+0000 7f37697fa700 1 -- 172.21.15.114:0/4196787566 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3760004c50 con 0x7f376c0ffab0 2024-02-16T22:28:17.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.539+0000 7f37723b0700 1 -- 172.21.15.114:0/4196787566 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f376c19e240 con 0x7f376c0ffab0 2024-02-16T22:28:17.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.539+0000 7f37697fa700 1 -- 172.21.15.114:0/4196787566 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3760021640 con 0x7f376c0ffab0 2024-02-16T22:28:17.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.540+0000 7f37723b0700 1 -- 172.21.15.114:0/4196787566 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f376c108690 con 0x7f376c0ffab0 2024-02-16T22:28:17.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.540+0000 7f37697fa700 1 -- 172.21.15.114:0/4196787566 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f376000fa10 con 0x7f376c0ffab0 2024-02-16T22:28:17.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.541+0000 7f37697fa700 1 --2- 172.21.15.114:0/4196787566 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f375406d8a0 0x7f375406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:17.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.541+0000 7f37697fa700 1 -- 172.21.15.114:0/4196787566 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f376008e8f0 con 0x7f376c0ffab0 2024-02-16T22:28:17.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.541+0000 7f376b7fe700 1 --2- 172.21.15.114:0/4196787566 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f375406d8a0 0x7f375406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:17.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.541+0000 7f376b7fe700 1 --2- 172.21.15.114:0/4196787566 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f375406d8a0 0x7f375406fd60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f375c006fd0 tx=0x7f375c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:28:17.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.545+0000 7f37697fa700 1 -- 172.21.15.114:0/4196787566 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f376005bc80 con 0x7f376c0ffab0 2024-02-16T22:28:17.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.812+0000 7f37723b0700 1 -- 172.21.15.114:0/4196787566 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f376c19e960 con 0x7f376c0ffab0 2024-02-16T22:28:17.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.812+0000 7f37697fa700 1 -- 172.21.15.114:0/4196787566 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1846 (secure 0 0 0) 0x7f376005b810 con 0x7f376c0ffab0 2024-02-16T22:28:17.815 INFO:teuthology.orchestra.run.smithi114.stdout:e14 2024-02-16T22:28:17.815 INFO:teuthology.orchestra.run.smithi114.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:28:17.815 INFO:teuthology.orchestra.run.smithi114.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:28:17.815 INFO:teuthology.orchestra.run.smithi114.stdout:legacy client fscid: 1 2024-02-16T22:28:17.815 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:28:17.815 INFO:teuthology.orchestra.run.smithi114.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:28:17.815 INFO:teuthology.orchestra.run.smithi114.stdout:fs_name cephfs 2024-02-16T22:28:17.815 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 12 2024-02-16T22:28:17.816 INFO:teuthology.orchestra.run.smithi114.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:28:17.816 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-02-16T22:18:02.108825+0000 2024-02-16T22:28:17.816 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-02-16T22:18:16.413371+0000 2024-02-16T22:28:17.816 INFO:teuthology.orchestra.run.smithi114.stdout:tableserver 0 2024-02-16T22:28:17.816 INFO:teuthology.orchestra.run.smithi114.stdout:root 0 2024-02-16T22:28:17.816 INFO:teuthology.orchestra.run.smithi114.stdout:session_timeout 60 2024-02-16T22:28:17.816 INFO:teuthology.orchestra.run.smithi114.stdout:session_autoclose 300 2024-02-16T22:28:17.816 INFO:teuthology.orchestra.run.smithi114.stdout:max_file_size 1099511627776 2024-02-16T22:28:17.816 INFO:teuthology.orchestra.run.smithi114.stdout:max_xattr_size 65536 2024-02-16T22:28:17.816 INFO:teuthology.orchestra.run.smithi114.stdout:required_client_features {} 2024-02-16T22:28:17.816 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure 0 2024-02-16T22:28:17.816 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure_osd_epoch 0 2024-02-16T22:28:17.816 INFO:teuthology.orchestra.run.smithi114.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:28:17.817 INFO:teuthology.orchestra.run.smithi114.stdout:max_mds 1 2024-02-16T22:28:17.817 INFO:teuthology.orchestra.run.smithi114.stdout:in 0 2024-02-16T22:28:17.817 INFO:teuthology.orchestra.run.smithi114.stdout:up {0=14468} 2024-02-16T22:28:17.817 INFO:teuthology.orchestra.run.smithi114.stdout:failed 2024-02-16T22:28:17.817 INFO:teuthology.orchestra.run.smithi114.stdout:damaged 2024-02-16T22:28:17.817 INFO:teuthology.orchestra.run.smithi114.stdout:stopped 2024-02-16T22:28:17.817 INFO:teuthology.orchestra.run.smithi114.stdout:data_pools [3] 2024-02-16T22:28:17.817 INFO:teuthology.orchestra.run.smithi114.stdout:metadata_pool 2 2024-02-16T22:28:17.817 INFO:teuthology.orchestra.run.smithi114.stdout:inline_data enabled 2024-02-16T22:28:17.817 INFO:teuthology.orchestra.run.smithi114.stdout:balancer 2024-02-16T22:28:17.817 INFO:teuthology.orchestra.run.smithi114.stdout:bal_rank_mask -1 2024-02-16T22:28:17.817 INFO:teuthology.orchestra.run.smithi114.stdout:standby_count_wanted 1 2024-02-16T22:28:17.817 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.zzryke{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.114:6826/277953535,v1:172.21.15.114:6827/277953535] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:28:17.817 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:28:17.818 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:28:17.818 INFO:teuthology.orchestra.run.smithi114.stdout:Standby daemons: 2024-02-16T22:28:17.818 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:28:17.818 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.fdxpey{-1:14484} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:28:17.818 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.vhlqbe{-1:24269} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:28:17.818 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.gopcfa{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:28:17.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.815+0000 7f37723b0700 1 -- 172.21.15.114:0/4196787566 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f375406d8a0 msgr2=0x7f375406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:17.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.815+0000 7f37723b0700 1 --2- 172.21.15.114:0/4196787566 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f375406d8a0 0x7f375406fd60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f375c006fd0 tx=0x7f375c008040 comp rx=0 tx=0).stop 2024-02-16T22:28:17.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.815+0000 7f37723b0700 1 -- 172.21.15.114:0/4196787566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f376c0ffab0 msgr2=0x7f376c068f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:17.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.815+0000 7f37723b0700 1 --2- 172.21.15.114:0/4196787566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f376c0ffab0 0x7f376c068f70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f376000bdd0 tx=0x7f376000beb0 comp rx=0 tx=0).stop 2024-02-16T22:28:17.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.816+0000 7f37723b0700 1 -- 172.21.15.114:0/4196787566 shutdown_connections 2024-02-16T22:28:17.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.816+0000 7f37723b0700 1 --2- 172.21.15.114:0/4196787566 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f375406d8a0 0x7f375406fd60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:17.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.816+0000 7f37723b0700 1 --2- 172.21.15.114:0/4196787566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f376c0ffab0 0x7f376c068f70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:17.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.816+0000 7f37723b0700 1 --2- 172.21.15.114:0/4196787566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f376c100cb0 0x7f376c19d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:17.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.816+0000 7f37723b0700 1 -- 172.21.15.114:0/4196787566 >> 172.21.15.114:0/4196787566 conn(0x7f376c0fb030 msgr2=0x7f376c103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:28:17.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.816+0000 7f37723b0700 1 -- 172.21.15.114:0/4196787566 shutdown_connections 2024-02-16T22:28:17.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.816+0000 7f37723b0700 1 -- 172.21.15.114:0/4196787566 wait complete. 2024-02-16T22:28:17.819 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 14 2024-02-16T22:28:17.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.946+0000 7f6f14dc8700 1 -- 172.21.15.114:0/3761044534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f101016e0 msgr2=0x7f6f10103ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:17.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.946+0000 7f6f14dc8700 1 --2- 172.21.15.114:0/3761044534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f101016e0 0x7f6f10103ad0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6f00009910 tx=0x7f6f00009c20 comp rx=0 tx=0).stop 2024-02-16T22:28:17.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.947+0000 7f6f14dc8700 1 -- 172.21.15.114:0/3761044534 shutdown_connections 2024-02-16T22:28:17.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.947+0000 7f6f14dc8700 1 --2- 172.21.15.114:0/3761044534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f10104010 0x7f6f10106400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:17.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.947+0000 7f6f14dc8700 1 --2- 172.21.15.114:0/3761044534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f101016e0 0x7f6f10103ad0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:17.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.947+0000 7f6f14dc8700 1 -- 172.21.15.114:0/3761044534 >> 172.21.15.114:0/3761044534 conn(0x7f6f100fb050 msgr2=0x7f6f100fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:28:17.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.947+0000 7f6f14dc8700 1 -- 172.21.15.114:0/3761044534 shutdown_connections 2024-02-16T22:28:17.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.947+0000 7f6f14dc8700 1 -- 172.21.15.114:0/3761044534 wait complete. 2024-02-16T22:28:17.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.947+0000 7f6f14dc8700 1 Processor -- start 2024-02-16T22:28:17.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.947+0000 7f6f14dc8700 1 -- start start 2024-02-16T22:28:17.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.947+0000 7f6f14dc8700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f101016e0 0x7f6f101963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:17.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.948+0000 7f6f14dc8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f10104010 0x7f6f101968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:17.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.948+0000 7f6f14dc8700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f10196f00 con 0x7f6f101016e0 2024-02-16T22:28:17.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.948+0000 7f6f14dc8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f10197040 con 0x7f6f10104010 2024-02-16T22:28:17.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.948+0000 7f6f0e59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f101016e0 0x7f6f101963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:17.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.948+0000 7f6f0e59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f101016e0 0x7f6f101963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50866/0 (socket says 172.21.15.114:50866) 2024-02-16T22:28:17.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.948+0000 7f6f0e59c700 1 -- 172.21.15.114:0/1653804016 learned_addr learned my addr 172.21.15.114:0/1653804016 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:28:17.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.948+0000 7f6f07fff700 1 --2- 172.21.15.114:0/1653804016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f10104010 0x7f6f101968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:17.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.948+0000 7f6f0e59c700 1 -- 172.21.15.114:0/1653804016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f10104010 msgr2=0x7f6f101968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:17.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.948+0000 7f6f0e59c700 1 --2- 172.21.15.114:0/1653804016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f10104010 0x7f6f101968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:17.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.948+0000 7f6f0e59c700 1 -- 172.21.15.114:0/1653804016 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f000095f0 con 0x7f6f101016e0 2024-02-16T22:28:17.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.948+0000 7f6f0e59c700 1 --2- 172.21.15.114:0/1653804016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f101016e0 0x7f6f101963a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000bd70 tx=0x7f6f0000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:28:17.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.948+0000 7f6f077fe700 1 -- 172.21.15.114:0/1653804016 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f0001d070 con 0x7f6f101016e0 2024-02-16T22:28:17.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.948+0000 7f6f077fe700 1 -- 172.21.15.114:0/1653804016 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6f00005dc0 con 0x7f6f101016e0 2024-02-16T22:28:17.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.948+0000 7f6f077fe700 1 -- 172.21.15.114:0/1653804016 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f00021410 con 0x7f6f101016e0 2024-02-16T22:28:17.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.948+0000 7f6f14dc8700 1 -- 172.21.15.114:0/1653804016 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f1019ba40 con 0x7f6f101016e0 2024-02-16T22:28:17.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.948+0000 7f6f14dc8700 1 -- 172.21.15.114:0/1653804016 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f1019bee0 con 0x7f6f101016e0 2024-02-16T22:28:17.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.949+0000 7f6f14dc8700 1 -- 172.21.15.114:0/1653804016 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f10190550 con 0x7f6f101016e0 2024-02-16T22:28:17.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.950+0000 7f6f077fe700 1 -- 172.21.15.114:0/1653804016 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f6f0000f7d0 con 0x7f6f101016e0 2024-02-16T22:28:17.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.950+0000 7f6f077fe700 1 --2- 172.21.15.114:0/1653804016 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6efc06d8a0 0x7f6efc06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:17.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.950+0000 7f6f077fe700 1 -- 172.21.15.114:0/1653804016 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f6f0008e820 con 0x7f6f101016e0 2024-02-16T22:28:17.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.950+0000 7f6f07fff700 1 --2- 172.21.15.114:0/1653804016 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6efc06d8a0 0x7f6efc06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:17.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.951+0000 7f6f07fff700 1 --2- 172.21.15.114:0/1653804016 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6efc06d8a0 0x7f6efc06fd60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8004630 tx=0x7f6ef800b3c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:28:17.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:17.955+0000 7f6f077fe700 1 -- 172.21.15.114:0/1653804016 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6f0005bbb0 con 0x7f6f101016e0 2024-02-16T22:28:18.104 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:28:18.104 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: from='client.34384 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:28:18.104 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:18.104 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:18.104 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:28:18.104 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:28:18.104 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:18.105 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:28:18.105 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:28:18.105 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:28:18.105 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:28:18.105 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:28:18.105 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: Upgrade: Setting container_image for all osd 2024-02-16T22:28:18.105 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:18.105 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-02-16T22:28:18.105 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-02-16T22:28:18.105 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-02-16T22:28:18.105 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-02-16T22:28:18.105 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-02-16T22:28:18.105 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-02-16T22:28:18.105 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-02-16T22:28:18.106 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-02-16T22:28:18.106 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-02-16T22:28:18.106 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-02-16T22:28:18.106 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-02-16T22:28:18.106 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-02-16T22:28:18.106 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: Upgrade: Setting require_osd_release to 19 squid 2024-02-16T22:28:18.106 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-02-16T22:28:18.106 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-02-16T22:28:18.106 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-02-16T22:28:18.106 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: osdmap e78: 6 total, 6 up, 6 in 2024-02-16T22:28:18.106 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi114.zzryke"]}]: dispatch 2024-02-16T22:28:18.106 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: Upgrade: It appears safe to stop mds.cephfs.smithi114.zzryke 2024-02-16T22:28:18.106 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/3414876170' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:28:18.106 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:18 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/4196787566' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:28:18.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.188+0000 7f6f14dc8700 1 -- 172.21.15.114:0/1653804016 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f100611d0 con 0x7f6efc06d8a0 2024-02-16T22:28:18.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.190+0000 7f6f077fe700 1 -- 172.21.15.114:0/1653804016 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6f100611d0 con 0x7f6efc06d8a0 2024-02-16T22:28:18.192 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:28:18.192 INFO:teuthology.orchestra.run.smithi114.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:28:18.192 INFO:teuthology.orchestra.run.smithi114.stdout: "in_progress": true, 2024-02-16T22:28:18.192 INFO:teuthology.orchestra.run.smithi114.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:28:18.192 INFO:teuthology.orchestra.run.smithi114.stdout: "services_complete": [ 2024-02-16T22:28:18.192 INFO:teuthology.orchestra.run.smithi114.stdout: "osd", 2024-02-16T22:28:18.193 INFO:teuthology.orchestra.run.smithi114.stdout: "crash", 2024-02-16T22:28:18.193 INFO:teuthology.orchestra.run.smithi114.stdout: "mon", 2024-02-16T22:28:18.193 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr" 2024-02-16T22:28:18.193 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-02-16T22:28:18.193 INFO:teuthology.orchestra.run.smithi114.stdout: "progress": "12/23 daemons upgraded", 2024-02-16T22:28:18.193 INFO:teuthology.orchestra.run.smithi114.stdout: "message": "Currently upgrading mds daemons", 2024-02-16T22:28:18.193 INFO:teuthology.orchestra.run.smithi114.stdout: "is_paused": false 2024-02-16T22:28:18.193 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:28:18.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.192+0000 7f6f14dc8700 1 -- 172.21.15.114:0/1653804016 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6efc06d8a0 msgr2=0x7f6efc06fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:18.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.192+0000 7f6f14dc8700 1 --2- 172.21.15.114:0/1653804016 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6efc06d8a0 0x7f6efc06fd60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8004630 tx=0x7f6ef800b3c0 comp rx=0 tx=0).stop 2024-02-16T22:28:18.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.192+0000 7f6f14dc8700 1 -- 172.21.15.114:0/1653804016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f101016e0 msgr2=0x7f6f101963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:18.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.192+0000 7f6f14dc8700 1 --2- 172.21.15.114:0/1653804016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f101016e0 0x7f6f101963a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000bd70 tx=0x7f6f0000bda0 comp rx=0 tx=0).stop 2024-02-16T22:28:18.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.192+0000 7f6f14dc8700 1 -- 172.21.15.114:0/1653804016 shutdown_connections 2024-02-16T22:28:18.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.192+0000 7f6f14dc8700 1 --2- 172.21.15.114:0/1653804016 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6efc06d8a0 0x7f6efc06fd60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:18.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.192+0000 7f6f14dc8700 1 --2- 172.21.15.114:0/1653804016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f101016e0 0x7f6f101963a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:18.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.192+0000 7f6f14dc8700 1 --2- 172.21.15.114:0/1653804016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f10104010 0x7f6f101968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:18.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.192+0000 7f6f14dc8700 1 -- 172.21.15.114:0/1653804016 >> 172.21.15.114:0/1653804016 conn(0x7f6f100fb050 msgr2=0x7f6f100fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:28:18.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.192+0000 7f6f14dc8700 1 -- 172.21.15.114:0/1653804016 shutdown_connections 2024-02-16T22:28:18.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.192+0000 7f6f14dc8700 1 -- 172.21.15.114:0/1653804016 wait complete. 2024-02-16T22:28:18.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:28:18.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: from='client.34384 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:28:18.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:18.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:18.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:28:18.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:28:18.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:18.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:28:18.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:28:18.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:28:18.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:28:18.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:28:18.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: Upgrade: Setting container_image for all osd 2024-02-16T22:28:18.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:18.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-02-16T22:28:18.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-02-16T22:28:18.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-02-16T22:28:18.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-02-16T22:28:18.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-02-16T22:28:18.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-02-16T22:28:18.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-02-16T22:28:18.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-02-16T22:28:18.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-02-16T22:28:18.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-02-16T22:28:18.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-02-16T22:28:18.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-02-16T22:28:18.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: Upgrade: Setting require_osd_release to 19 squid 2024-02-16T22:28:18.305 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-02-16T22:28:18.305 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-02-16T22:28:18.305 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-02-16T22:28:18.305 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: osdmap e78: 6 total, 6 up, 6 in 2024-02-16T22:28:18.305 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi114.zzryke"]}]: dispatch 2024-02-16T22:28:18.305 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: Upgrade: It appears safe to stop mds.cephfs.smithi114.zzryke 2024-02-16T22:28:18.305 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/3414876170' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:28:18.305 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:18 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/4196787566' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:28:18.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.301+0000 7f8fee3f4700 1 -- 172.21.15.114:0/3499852536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fe80ffab0 msgr2=0x7f8fe80ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:18.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.301+0000 7f8fee3f4700 1 --2- 172.21.15.114:0/3499852536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fe80ffab0 0x7f8fe80ffed0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8009910 tx=0x7f8fd8009c20 comp rx=0 tx=0).stop 2024-02-16T22:28:18.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.301+0000 7f8fee3f4700 1 -- 172.21.15.114:0/3499852536 shutdown_connections 2024-02-16T22:28:18.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.301+0000 7f8fee3f4700 1 --2- 172.21.15.114:0/3499852536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fe8100cb0 0x7f8fe8101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:18.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.301+0000 7f8fee3f4700 1 --2- 172.21.15.114:0/3499852536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fe80ffab0 0x7f8fe80ffed0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:18.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.301+0000 7f8fee3f4700 1 -- 172.21.15.114:0/3499852536 >> 172.21.15.114:0/3499852536 conn(0x7f8fe80fb030 msgr2=0x7f8fe80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:28:18.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.301+0000 7f8fee3f4700 1 -- 172.21.15.114:0/3499852536 shutdown_connections 2024-02-16T22:28:18.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.301+0000 7f8fee3f4700 1 -- 172.21.15.114:0/3499852536 wait complete. 2024-02-16T22:28:18.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.301+0000 7f8fee3f4700 1 Processor -- start 2024-02-16T22:28:18.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.301+0000 7f8fee3f4700 1 -- start start 2024-02-16T22:28:18.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.302+0000 7f8fee3f4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fe80ffab0 0x7f8fe8196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:18.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.302+0000 7f8fee3f4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fe8100cb0 0x7f8fe8196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:18.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.302+0000 7f8fee3f4700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe8196e80 con 0x7f8fe8100cb0 2024-02-16T22:28:18.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.302+0000 7f8fee3f4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe8196fc0 con 0x7f8fe80ffab0 2024-02-16T22:28:18.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.302+0000 7f8fe77fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fe8100cb0 0x7f8fe8196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:18.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.302+0000 7f8fe77fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fe8100cb0 0x7f8fe8196860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50896/0 (socket says 172.21.15.114:50896) 2024-02-16T22:28:18.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.302+0000 7f8fe77fe700 1 -- 172.21.15.114:0/3071800823 learned_addr learned my addr 172.21.15.114:0/3071800823 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:28:18.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.302+0000 7f8fe7fff700 1 --2- 172.21.15.114:0/3071800823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fe80ffab0 0x7f8fe8196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:18.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.302+0000 7f8fe77fe700 1 -- 172.21.15.114:0/3071800823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fe80ffab0 msgr2=0x7f8fe8196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:18.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.302+0000 7f8fe77fe700 1 --2- 172.21.15.114:0/3071800823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fe80ffab0 0x7f8fe8196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:18.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.302+0000 7f8fe77fe700 1 -- 172.21.15.114:0/3071800823 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd80095f0 con 0x7f8fe8100cb0 2024-02-16T22:28:18.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.302+0000 7f8fe77fe700 1 --2- 172.21.15.114:0/3071800823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fe8100cb0 0x7f8fe8196860 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc00d8d0 tx=0x7f8fdc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:28:18.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.302+0000 7f8fe57fa700 1 -- 172.21.15.114:0/3071800823 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fdc009940 con 0x7f8fe8100cb0 2024-02-16T22:28:18.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.302+0000 7f8fee3f4700 1 -- 172.21.15.114:0/3071800823 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fe819ba70 con 0x7f8fe8100cb0 2024-02-16T22:28:18.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.302+0000 7f8fe57fa700 1 -- 172.21.15.114:0/3071800823 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8fdc010460 con 0x7f8fe8100cb0 2024-02-16T22:28:18.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.302+0000 7f8fe57fa700 1 -- 172.21.15.114:0/3071800823 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fdc00f5f0 con 0x7f8fe8100cb0 2024-02-16T22:28:18.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.303+0000 7f8fee3f4700 1 -- 172.21.15.114:0/3071800823 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fe819bfc0 con 0x7f8fe8100cb0 2024-02-16T22:28:18.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.303+0000 7f8fee3f4700 1 -- 172.21.15.114:0/3071800823 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fe8108610 con 0x7f8fe8100cb0 2024-02-16T22:28:18.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.304+0000 7f8fe57fa700 1 -- 172.21.15.114:0/3071800823 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f8fdc0105d0 con 0x7f8fe8100cb0 2024-02-16T22:28:18.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.304+0000 7f8fe57fa700 1 --2- 172.21.15.114:0/3071800823 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f8fd006d8a0 0x7f8fd006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:18.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.304+0000 7f8fe57fa700 1 -- 172.21.15.114:0/3071800823 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f8fdc08d470 con 0x7f8fe8100cb0 2024-02-16T22:28:18.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.304+0000 7f8fe7fff700 1 --2- 172.21.15.114:0/3071800823 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f8fd006d8a0 0x7f8fd006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:18.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.305+0000 7f8fe7fff700 1 --2- 172.21.15.114:0/3071800823 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f8fd006d8a0 0x7f8fd006fd60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8006010 tx=0x7f8fd8005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:28:18.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.309+0000 7f8fe57fa700 1 -- 172.21.15.114:0/3071800823 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8fdc05a800 con 0x7f8fe8100cb0 2024-02-16T22:28:18.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.588+0000 7f8fee3f4700 1 -- 172.21.15.114:0/3071800823 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8fe819c630 con 0x7f8fe8100cb0 2024-02-16T22:28:18.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.589+0000 7f8fe57fa700 1 -- 172.21.15.114:0/3071800823 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+873 (secure 0 0 0) 0x7f8fdc05a390 con 0x7f8fe8100cb0 2024-02-16T22:28:18.591 INFO:teuthology.orchestra.run.smithi114.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; Degraded data redundancy: 32/267 objects degraded (11.985%), 9 pgs degraded 2024-02-16T22:28:18.591 INFO:teuthology.orchestra.run.smithi114.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:28:18.591 INFO:teuthology.orchestra.run.smithi114.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:28:18.591 INFO:teuthology.orchestra.run.smithi114.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 32/267 objects degraded (11.985%), 9 pgs degraded 2024-02-16T22:28:18.591 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.2 is active+undersized+degraded, acting [1,0] 2024-02-16T22:28:18.591 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.3 is active+undersized+degraded, acting [2,1] 2024-02-16T22:28:18.592 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.8 is active+undersized+degraded, acting [3,0] 2024-02-16T22:28:18.592 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.b is active+undersized+degraded, acting [3,4] 2024-02-16T22:28:18.592 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.14 is active+undersized+degraded, acting [3,4] 2024-02-16T22:28:18.592 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.16 is active+undersized+degraded, acting [3,2] 2024-02-16T22:28:18.595 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.18 is active+undersized+degraded, acting [4,3] 2024-02-16T22:28:18.595 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.1a is active+undersized+degraded, acting [3,4] 2024-02-16T22:28:18.595 INFO:teuthology.orchestra.run.smithi114.stdout: pg 2.1d is active+undersized+degraded, acting [3,0] 2024-02-16T22:28:18.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.591+0000 7f8fee3f4700 1 -- 172.21.15.114:0/3071800823 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f8fd006d8a0 msgr2=0x7f8fd006fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:18.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.591+0000 7f8fee3f4700 1 --2- 172.21.15.114:0/3071800823 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f8fd006d8a0 0x7f8fd006fd60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8006010 tx=0x7f8fd8005a90 comp rx=0 tx=0).stop 2024-02-16T22:28:18.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.591+0000 7f8fee3f4700 1 -- 172.21.15.114:0/3071800823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fe8100cb0 msgr2=0x7f8fe8196860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:18.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.591+0000 7f8fee3f4700 1 --2- 172.21.15.114:0/3071800823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fe8100cb0 0x7f8fe8196860 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc00d8d0 tx=0x7f8fdc00dc90 comp rx=0 tx=0).stop 2024-02-16T22:28:18.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.591+0000 7f8fee3f4700 1 -- 172.21.15.114:0/3071800823 shutdown_connections 2024-02-16T22:28:18.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.591+0000 7f8fee3f4700 1 --2- 172.21.15.114:0/3071800823 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f8fd006d8a0 0x7f8fd006fd60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:18.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.591+0000 7f8fee3f4700 1 --2- 172.21.15.114:0/3071800823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fe80ffab0 0x7f8fe8196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:18.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.591+0000 7f8fee3f4700 1 --2- 172.21.15.114:0/3071800823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fe8100cb0 0x7f8fe8196860 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:18.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.591+0000 7f8fee3f4700 1 -- 172.21.15.114:0/3071800823 >> 172.21.15.114:0/3071800823 conn(0x7f8fe80fb030 msgr2=0x7f8fe80683d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:28:18.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.591+0000 7f8fee3f4700 1 -- 172.21.15.114:0/3071800823 shutdown_connections 2024-02-16T22:28:18.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:18.591+0000 7f8fee3f4700 1 -- 172.21.15.114:0/3071800823 wait complete. 2024-02-16T22:28:19.681 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:19 smithi114 ceph-mon[94452]: pgmap v169: 65 pgs: 7 peering, 9 active+undersized+degraded, 12 active+undersized, 37 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 32/267 objects degraded (11.985%) 2024-02-16T22:28:19.681 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:19 smithi114 ceph-mon[94452]: from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:28:19.681 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:19 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/3071800823' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:28:19.681 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:19 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:19.681 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:19 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi114.zzryke", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:28:19.682 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:19 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:28:19.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:19 smithi123 ceph-mon[82331]: pgmap v169: 65 pgs: 7 peering, 9 active+undersized+degraded, 12 active+undersized, 37 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 32/267 objects degraded (11.985%) 2024-02-16T22:28:19.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:19 smithi123 ceph-mon[82331]: from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:28:19.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:19 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/3071800823' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:28:19.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:19 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:19.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:19 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi114.zzryke", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:28:19.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:19 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:28:20.607 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:20 smithi114 ceph-mon[94452]: Upgrade: Updating mds.cephfs.smithi114.zzryke 2024-02-16T22:28:20.608 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:20 smithi114 ceph-mon[94452]: Deploying daemon mds.cephfs.smithi114.zzryke on smithi114 2024-02-16T22:28:20.608 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:20 smithi114 ceph-mon[94452]: Health check update: Degraded data redundancy: 32/267 objects degraded (11.985%), 9 pgs degraded (PG_DEGRADED) 2024-02-16T22:28:20.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:20 smithi123 ceph-mon[82331]: Upgrade: Updating mds.cephfs.smithi114.zzryke 2024-02-16T22:28:20.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:20 smithi123 ceph-mon[82331]: Deploying daemon mds.cephfs.smithi114.zzryke on smithi114 2024-02-16T22:28:20.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:20 smithi123 ceph-mon[82331]: Health check update: Degraded data redundancy: 32/267 objects degraded (11.985%), 9 pgs degraded (PG_DEGRADED) 2024-02-16T22:28:21.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:21 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi114[94448]: 2024-02-16T22:28:21.202+0000 7f1ab68f5700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T22:28:21.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:21 smithi114 ceph-mon[94452]: pgmap v170: 65 pgs: 7 peering, 1 active+undersized, 57 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:28:21.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:21 smithi114 ceph-mon[94452]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 32/267 objects degraded (11.985%), 9 pgs degraded) 2024-02-16T22:28:21.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:21 smithi114 ceph-mon[94452]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-16T22:28:21.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:21 smithi114 ceph-mon[94452]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T22:28:21.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:21 smithi114 ceph-mon[94452]: osdmap e79: 6 total, 6 up, 6 in 2024-02-16T22:28:21.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:21 smithi114 ceph-mon[94452]: Standby daemon mds.cephfs.smithi114.fdxpey assigned to filesystem cephfs as rank 0 2024-02-16T22:28:21.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:21 smithi114 ceph-mon[94452]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T22:28:21.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:21 smithi114 ceph-mon[94452]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-02-16T22:28:21.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:21 smithi114 ceph-mon[94452]: fsmap cephfs:1/1 {0=cephfs.smithi114.fdxpey=up:replay} 2 up:standby 2024-02-16T22:28:21.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:21 smithi123 ceph-mon[82331]: pgmap v170: 65 pgs: 7 peering, 1 active+undersized, 57 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:28:21.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:21 smithi123 ceph-mon[82331]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 32/267 objects degraded (11.985%), 9 pgs degraded) 2024-02-16T22:28:21.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:21 smithi123 ceph-mon[82331]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-16T22:28:21.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:21 smithi123 ceph-mon[82331]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T22:28:21.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:21 smithi123 ceph-mon[82331]: osdmap e79: 6 total, 6 up, 6 in 2024-02-16T22:28:21.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:21 smithi123 ceph-mon[82331]: Standby daemon mds.cephfs.smithi114.fdxpey assigned to filesystem cephfs as rank 0 2024-02-16T22:28:21.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:21 smithi123 ceph-mon[82331]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T22:28:21.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:21 smithi123 ceph-mon[82331]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-02-16T22:28:21.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:21 smithi123 ceph-mon[82331]: fsmap cephfs:1/1 {0=cephfs.smithi114.fdxpey=up:replay} 2 up:standby 2024-02-16T22:28:22.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:22 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi114.zzryke"}]: dispatch 2024-02-16T22:28:22.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:22 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi114.zzryke"}]: dispatch 2024-02-16T22:28:23.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:23 smithi114 ceph-mon[94452]: pgmap v172: 65 pgs: 7 peering, 58 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:28:23.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:23 smithi123 ceph-mon[82331]: pgmap v172: 65 pgs: 7 peering, 58 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:28:25.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:25 smithi114 ceph-mon[94452]: pgmap v173: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.7 MiB/s rd, 1 op/s 2024-02-16T22:28:25.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:25 smithi123 ceph-mon[82331]: pgmap v173: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.7 MiB/s rd, 1 op/s 2024-02-16T22:28:27.647 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:27 smithi114 ceph-mon[94452]: pgmap v174: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 4 op/s 2024-02-16T22:28:27.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:27 smithi123 ceph-mon[82331]: pgmap v174: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 4 op/s 2024-02-16T22:28:29.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:29 smithi114 ceph-mon[94452]: pgmap v175: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-02-16T22:28:29.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:29 smithi123 ceph-mon[82331]: pgmap v175: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-02-16T22:28:30.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:30 smithi114 ceph-mon[94452]: mds.? [v2:172.21.15.114:6826/2017800151,v1:172.21.15.114:6827/2017800151] up:boot 2024-02-16T22:28:30.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:30 smithi114 ceph-mon[94452]: fsmap cephfs:1/1 {0=cephfs.smithi114.fdxpey=up:replay} 3 up:standby 2024-02-16T22:28:30.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:30 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi114.zzryke"}]: dispatch 2024-02-16T22:28:30.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:30 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:30.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:30 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:30.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:30 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:28:30.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:30 smithi123 ceph-mon[82331]: mds.? [v2:172.21.15.114:6826/2017800151,v1:172.21.15.114:6827/2017800151] up:boot 2024-02-16T22:28:30.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:30 smithi123 ceph-mon[82331]: fsmap cephfs:1/1 {0=cephfs.smithi114.fdxpey=up:replay} 3 up:standby 2024-02-16T22:28:30.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:30 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi114.zzryke"}]: dispatch 2024-02-16T22:28:30.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:30 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:30.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:30 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:30.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:30 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:28:31.671 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:31 smithi123 ceph-mon[82331]: pgmap v176: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-02-16T22:28:31.671 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:31 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:31.671 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:31 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:28:31.671 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:31 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:31.697 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:31 smithi114 ceph-mon[94452]: pgmap v176: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-02-16T22:28:31.698 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:31 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:31.698 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:31 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:28:31.698 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:31 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:33.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:33 smithi114 ceph-mon[94452]: pgmap v177: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-02-16T22:28:33.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:33 smithi114 ceph-mon[94452]: reconnect by client.24321 192.168.0.1:0/2666785881 after 0.000999979 2024-02-16T22:28:33.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:33 smithi114 ceph-mon[94452]: reconnect by client.14536 192.168.0.1:0/21264991 after 0.000999979 2024-02-16T22:28:33.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:33 smithi114 ceph-mon[94452]: mds.? [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] up:reconnect 2024-02-16T22:28:33.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:33 smithi114 ceph-mon[94452]: fsmap cephfs:1/1 {0=cephfs.smithi114.fdxpey=up:reconnect} 3 up:standby 2024-02-16T22:28:33.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:33 smithi123 ceph-mon[82331]: pgmap v177: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-02-16T22:28:33.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:33 smithi123 ceph-mon[82331]: reconnect by client.24321 192.168.0.1:0/2666785881 after 0.000999979 2024-02-16T22:28:33.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:33 smithi123 ceph-mon[82331]: reconnect by client.14536 192.168.0.1:0/21264991 after 0.000999979 2024-02-16T22:28:33.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:33 smithi123 ceph-mon[82331]: mds.? [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] up:reconnect 2024-02-16T22:28:33.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:33 smithi123 ceph-mon[82331]: fsmap cephfs:1/1 {0=cephfs.smithi114.fdxpey=up:reconnect} 3 up:standby 2024-02-16T22:28:34.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:34 smithi114 ceph-mon[94452]: mds.? [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] up:rejoin 2024-02-16T22:28:34.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:34 smithi114 ceph-mon[94452]: fsmap cephfs:1/1 {0=cephfs.smithi114.fdxpey=up:rejoin} 3 up:standby 2024-02-16T22:28:34.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:34 smithi114 ceph-mon[94452]: daemon mds.cephfs.smithi114.fdxpey is now active in filesystem cephfs as rank 0 2024-02-16T22:28:34.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:34 smithi123 ceph-mon[82331]: mds.? [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] up:rejoin 2024-02-16T22:28:34.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:34 smithi123 ceph-mon[82331]: fsmap cephfs:1/1 {0=cephfs.smithi114.fdxpey=up:rejoin} 3 up:standby 2024-02-16T22:28:34.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:34 smithi123 ceph-mon[82331]: daemon mds.cephfs.smithi114.fdxpey is now active in filesystem cephfs as rank 0 2024-02-16T22:28:35.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:35 smithi114 ceph-mon[94452]: pgmap v178: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-02-16T22:28:35.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:35 smithi114 ceph-mon[94452]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-16T22:28:35.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:35 smithi114 ceph-mon[94452]: mds.? [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] up:active 2024-02-16T22:28:35.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:35 smithi114 ceph-mon[94452]: fsmap cephfs:1 {0=cephfs.smithi114.fdxpey=up:active} 3 up:standby 2024-02-16T22:28:35.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:35 smithi123 ceph-mon[82331]: pgmap v178: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-02-16T22:28:35.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:35 smithi123 ceph-mon[82331]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-16T22:28:35.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:35 smithi123 ceph-mon[82331]: mds.? [v2:172.21.15.114:6828/865692293,v1:172.21.15.114:6829/865692293] up:active 2024-02-16T22:28:35.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:35 smithi123 ceph-mon[82331]: fsmap cephfs:1 {0=cephfs.smithi114.fdxpey=up:active} 3 up:standby 2024-02-16T22:28:37.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:37 smithi114 ceph-mon[94452]: pgmap v179: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 85 B/s wr, 5 op/s 2024-02-16T22:28:37.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:37 smithi123 ceph-mon[82331]: pgmap v179: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 85 B/s wr, 5 op/s 2024-02-16T22:28:39.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:39 smithi114 ceph-mon[94452]: pgmap v180: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 13 MiB/s rd, 682 B/s wr, 6 op/s 2024-02-16T22:28:39.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:39 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:39.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:39 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:39.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:39 smithi123 ceph-mon[82331]: pgmap v180: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 13 MiB/s rd, 682 B/s wr, 6 op/s 2024-02-16T22:28:39.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:39 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:39.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:39 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:41.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:41 smithi114 ceph-mon[94452]: pgmap v181: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 7.9 MiB/s rd, 682 B/s wr, 6 op/s 2024-02-16T22:28:41.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:41 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:41.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:41 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:41.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:41 smithi123 ceph-mon[82331]: pgmap v181: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 7.9 MiB/s rd, 682 B/s wr, 6 op/s 2024-02-16T22:28:41.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:41 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:41.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:41 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:43.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:43 smithi114 ceph-mon[94452]: pgmap v182: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 3.5 MiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-16T22:28:43.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:43 smithi123 ceph-mon[82331]: pgmap v182: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 3.5 MiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-16T22:28:45.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:44 smithi114 ceph-mon[94452]: pgmap v183: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 3.3 MiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-16T22:28:45.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:44 smithi114 ceph-mon[94452]: Detected new or changed devices on smithi114 2024-02-16T22:28:45.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:44 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:45.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:44 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:45.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:44 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:28:45.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:44 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:28:45.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:44 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:45.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:44 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:28:45.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:44 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:28:45.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:44 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:28:45.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:44 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:28:45.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:44 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:28:45.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:44 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi114.fdxpey"]}]: dispatch 2024-02-16T22:28:45.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:44 smithi114 ceph-mon[94452]: Upgrade: It appears safe to stop mds.cephfs.smithi114.fdxpey 2024-02-16T22:28:45.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:44 smithi123 ceph-mon[82331]: pgmap v183: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 3.3 MiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-16T22:28:45.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:44 smithi123 ceph-mon[82331]: Detected new or changed devices on smithi114 2024-02-16T22:28:45.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:44 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:45.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:44 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:45.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:44 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:28:45.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:44 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:28:45.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:44 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:45.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:44 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:28:45.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:44 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:28:45.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:44 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:28:45.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:44 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:28:45.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:44 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:28:45.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:44 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi114.fdxpey"]}]: dispatch 2024-02-16T22:28:45.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:44 smithi123 ceph-mon[82331]: Upgrade: It appears safe to stop mds.cephfs.smithi114.fdxpey 2024-02-16T22:28:46.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:46 smithi114 ceph-mon[94452]: Upgrade: Updating mds.cephfs.smithi114.fdxpey 2024-02-16T22:28:46.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:46.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi114.fdxpey", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:28:46.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:28:46.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:46 smithi114 ceph-mon[94452]: Deploying daemon mds.cephfs.smithi114.fdxpey on smithi114 2024-02-16T22:28:46.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:46.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:28:46.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:46 smithi123 ceph-mon[82331]: Upgrade: Updating mds.cephfs.smithi114.fdxpey 2024-02-16T22:28:46.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:46 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:46.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:46 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi114.fdxpey", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:28:46.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:46 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:28:46.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:46 smithi123 ceph-mon[82331]: Deploying daemon mds.cephfs.smithi114.fdxpey on smithi114 2024-02-16T22:28:46.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:46 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:46.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:46 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:28:47.639 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:47 smithi114 ceph-mon[94452]: pgmap v184: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 345 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-16T22:28:47.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:47 smithi123 ceph-mon[82331]: pgmap v184: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 345 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-16T22:28:47.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:47 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi114[94448]: 2024-02-16T22:28:47.637+0000 7f1ab68f5700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T22:28:48.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.704+0000 7f47bf5b6700 1 -- 172.21.15.114:0/279081222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47b8104010 msgr2=0x7f47b8106400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:48.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.704+0000 7f47bf5b6700 1 --2- 172.21.15.114:0/279081222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47b8104010 0x7f47b8106400 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f47a8009910 tx=0x7f47a8009c20 comp rx=0 tx=0).stop 2024-02-16T22:28:48.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.704+0000 7f47bf5b6700 1 -- 172.21.15.114:0/279081222 shutdown_connections 2024-02-16T22:28:48.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.704+0000 7f47bf5b6700 1 --2- 172.21.15.114:0/279081222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47b8104010 0x7f47b8106400 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:48.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.704+0000 7f47bf5b6700 1 --2- 172.21.15.114:0/279081222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47b81016e0 0x7f47b8103ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:48.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.704+0000 7f47bf5b6700 1 -- 172.21.15.114:0/279081222 >> 172.21.15.114:0/279081222 conn(0x7f47b80fb050 msgr2=0x7f47b80fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:28:48.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.704+0000 7f47bf5b6700 1 -- 172.21.15.114:0/279081222 shutdown_connections 2024-02-16T22:28:48.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.704+0000 7f47bf5b6700 1 -- 172.21.15.114:0/279081222 wait complete. 2024-02-16T22:28:48.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.704+0000 7f47bf5b6700 1 Processor -- start 2024-02-16T22:28:48.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.705+0000 7f47bf5b6700 1 -- start start 2024-02-16T22:28:48.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.705+0000 7f47bf5b6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47b81016e0 0x7f47b8196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:48.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.705+0000 7f47bf5b6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47b8104010 0x7f47b81968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:48.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.705+0000 7f47bf5b6700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b8196ee0 con 0x7f47b81016e0 2024-02-16T22:28:48.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.705+0000 7f47bf5b6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b8197020 con 0x7f47b8104010 2024-02-16T22:28:48.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.705+0000 7f47bd352700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47b81016e0 0x7f47b8196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:48.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.705+0000 7f47bd352700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47b81016e0 0x7f47b8196380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40508/0 (socket says 172.21.15.114:40508) 2024-02-16T22:28:48.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.705+0000 7f47bd352700 1 -- 172.21.15.114:0/4010309104 learned_addr learned my addr 172.21.15.114:0/4010309104 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:28:48.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.705+0000 7f47bcb51700 1 --2- 172.21.15.114:0/4010309104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47b8104010 0x7f47b81968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:48.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.705+0000 7f47bd352700 1 -- 172.21.15.114:0/4010309104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47b8104010 msgr2=0x7f47b81968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:48.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.705+0000 7f47bd352700 1 --2- 172.21.15.114:0/4010309104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47b8104010 0x7f47b81968c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:48.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.705+0000 7f47bd352700 1 -- 172.21.15.114:0/4010309104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a80095f0 con 0x7f47b81016e0 2024-02-16T22:28:48.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.705+0000 7f47bd352700 1 --2- 172.21.15.114:0/4010309104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47b81016e0 0x7f47b8196380 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f47b00060b0 tx=0x7f47b000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:28:48.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.705+0000 7f47ae7fc700 1 -- 172.21.15.114:0/4010309104 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47b000dda0 con 0x7f47b81016e0 2024-02-16T22:28:48.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.705+0000 7f47ae7fc700 1 -- 172.21.15.114:0/4010309104 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f47b000f040 con 0x7f47b81016e0 2024-02-16T22:28:48.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.705+0000 7f47bf5b6700 1 -- 172.21.15.114:0/4010309104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b819bad0 con 0x7f47b81016e0 2024-02-16T22:28:48.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.705+0000 7f47ae7fc700 1 -- 172.21.15.114:0/4010309104 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47b00146f0 con 0x7f47b81016e0 2024-02-16T22:28:48.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.706+0000 7f47bf5b6700 1 -- 172.21.15.114:0/4010309104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b819c020 con 0x7f47b81016e0 2024-02-16T22:28:48.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.706+0000 7f47bf5b6700 1 -- 172.21.15.114:0/4010309104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b8190540 con 0x7f47b81016e0 2024-02-16T22:28:48.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.707+0000 7f47ae7fc700 1 -- 172.21.15.114:0/4010309104 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f47b0015400 con 0x7f47b81016e0 2024-02-16T22:28:48.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.707+0000 7f47ae7fc700 1 --2- 172.21.15.114:0/4010309104 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f47a406d8a0 0x7f47a406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:48.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.707+0000 7f47ae7fc700 1 -- 172.21.15.114:0/4010309104 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f47b008cf30 con 0x7f47b81016e0 2024-02-16T22:28:48.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.707+0000 7f47bcb51700 1 --2- 172.21.15.114:0/4010309104 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f47a406d8a0 0x7f47a406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:48.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.708+0000 7f47bcb51700 1 --2- 172.21.15.114:0/4010309104 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f47a406d8a0 0x7f47a406fd60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f47a8006010 tx=0x7f47a800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:28:48.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.712+0000 7f47ae7fc700 1 -- 172.21.15.114:0/4010309104 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f47b00c0d20 con 0x7f47b81016e0 2024-02-16T22:28:48.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:48 smithi123 ceph-mon[82331]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-16T22:28:48.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:48 smithi123 ceph-mon[82331]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T22:28:48.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:48 smithi123 ceph-mon[82331]: osdmap e80: 6 total, 6 up, 6 in 2024-02-16T22:28:48.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:48 smithi123 ceph-mon[82331]: Standby daemon mds.cephfs.smithi123.vhlqbe assigned to filesystem cephfs as rank 0 2024-02-16T22:28:48.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:48 smithi123 ceph-mon[82331]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T22:28:48.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:48 smithi123 ceph-mon[82331]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-02-16T22:28:48.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:48 smithi123 ceph-mon[82331]: fsmap cephfs:1/1 {0=cephfs.smithi123.vhlqbe=up:replay} 2 up:standby 2024-02-16T22:28:48.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.926+0000 7f47bf5b6700 1 -- 172.21.15.114:0/4010309104 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f47b80611d0 con 0x7f47a406d8a0 2024-02-16T22:28:48.929 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:48 smithi114 ceph-mon[94452]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-16T22:28:48.929 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:48 smithi114 ceph-mon[94452]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T22:28:48.929 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:48 smithi114 ceph-mon[94452]: osdmap e80: 6 total, 6 up, 6 in 2024-02-16T22:28:48.929 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:48 smithi114 ceph-mon[94452]: Standby daemon mds.cephfs.smithi123.vhlqbe assigned to filesystem cephfs as rank 0 2024-02-16T22:28:48.929 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:48 smithi114 ceph-mon[94452]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T22:28:48.929 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:48 smithi114 ceph-mon[94452]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-02-16T22:28:48.929 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:48 smithi114 ceph-mon[94452]: fsmap cephfs:1/1 {0=cephfs.smithi123.vhlqbe=up:replay} 2 up:standby 2024-02-16T22:28:48.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.928+0000 7f47ae7fc700 1 -- 172.21.15.114:0/4010309104 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f47b80611d0 con 0x7f47a406d8a0 2024-02-16T22:28:48.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.930+0000 7f47bf5b6700 1 -- 172.21.15.114:0/4010309104 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f47a406d8a0 msgr2=0x7f47a406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:48.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.930+0000 7f47bf5b6700 1 --2- 172.21.15.114:0/4010309104 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f47a406d8a0 0x7f47a406fd60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f47a8006010 tx=0x7f47a800b540 comp rx=0 tx=0).stop 2024-02-16T22:28:48.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.930+0000 7f47bf5b6700 1 -- 172.21.15.114:0/4010309104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47b81016e0 msgr2=0x7f47b8196380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:48.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.930+0000 7f47bf5b6700 1 --2- 172.21.15.114:0/4010309104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47b81016e0 0x7f47b8196380 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f47b00060b0 tx=0x7f47b000d3b0 comp rx=0 tx=0).stop 2024-02-16T22:28:48.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.930+0000 7f47bf5b6700 1 -- 172.21.15.114:0/4010309104 shutdown_connections 2024-02-16T22:28:48.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.930+0000 7f47bf5b6700 1 --2- 172.21.15.114:0/4010309104 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f47a406d8a0 0x7f47a406fd60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:48.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.930+0000 7f47bf5b6700 1 --2- 172.21.15.114:0/4010309104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47b81016e0 0x7f47b8196380 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:48.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.930+0000 7f47bf5b6700 1 --2- 172.21.15.114:0/4010309104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47b8104010 0x7f47b81968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:48.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.930+0000 7f47bf5b6700 1 -- 172.21.15.114:0/4010309104 >> 172.21.15.114:0/4010309104 conn(0x7f47b80fb050 msgr2=0x7f47b80fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:28:48.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.930+0000 7f47bf5b6700 1 -- 172.21.15.114:0/4010309104 shutdown_connections 2024-02-16T22:28:48.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:48.930+0000 7f47bf5b6700 1 -- 172.21.15.114:0/4010309104 wait complete. 2024-02-16T22:28:48.942 INFO:teuthology.orchestra.run.smithi114.stdout:true 2024-02-16T22:28:49.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.035+0000 7f911a905700 1 -- 172.21.15.114:0/4143008043 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9114101dc0 msgr2=0x7f9114102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:49.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.035+0000 7f911a905700 1 --2- 172.21.15.114:0/4143008043 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9114101dc0 0x7f9114102220 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f90fc009910 tx=0x7f90fc009c20 comp rx=0 tx=0).stop 2024-02-16T22:28:49.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.035+0000 7f911a905700 1 -- 172.21.15.114:0/4143008043 shutdown_connections 2024-02-16T22:28:49.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.035+0000 7f911a905700 1 --2- 172.21.15.114:0/4143008043 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9114101dc0 0x7f9114102220 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:49.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.035+0000 7f911a905700 1 --2- 172.21.15.114:0/4143008043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9114100bc0 0x7f9114100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:49.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.035+0000 7f911a905700 1 -- 172.21.15.114:0/4143008043 >> 172.21.15.114:0/4143008043 conn(0x7f91140fc140 msgr2=0x7f91140fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:28:49.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.036+0000 7f911a905700 1 -- 172.21.15.114:0/4143008043 shutdown_connections 2024-02-16T22:28:49.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.036+0000 7f911a905700 1 -- 172.21.15.114:0/4143008043 wait complete. 2024-02-16T22:28:49.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.036+0000 7f911a905700 1 Processor -- start 2024-02-16T22:28:49.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.036+0000 7f911a905700 1 -- start start 2024-02-16T22:28:49.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.036+0000 7f911a905700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9114100bc0 0x7f9114196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:49.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.036+0000 7f911a905700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9114101dc0 0x7f91141968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:49.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.036+0000 7f911a905700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9114196ed0 con 0x7f9114100bc0 2024-02-16T22:28:49.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.036+0000 7f911a905700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9114197010 con 0x7f9114101dc0 2024-02-16T22:28:49.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.037+0000 7f9113fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9114100bc0 0x7f9114196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:49.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.037+0000 7f9113fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9114100bc0 0x7f9114196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40526/0 (socket says 172.21.15.114:40526) 2024-02-16T22:28:49.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.037+0000 7f9113fff700 1 -- 172.21.15.114:0/4124914751 learned_addr learned my addr 172.21.15.114:0/4124914751 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:28:49.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.037+0000 7f91137fe700 1 --2- 172.21.15.114:0/4124914751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9114101dc0 0x7f91141968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:49.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.037+0000 7f9113fff700 1 -- 172.21.15.114:0/4124914751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9114101dc0 msgr2=0x7f91141968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:49.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.037+0000 7f9113fff700 1 --2- 172.21.15.114:0/4124914751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9114101dc0 0x7f91141968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:49.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.037+0000 7f9113fff700 1 -- 172.21.15.114:0/4124914751 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90fc0095f0 con 0x7f9114100bc0 2024-02-16T22:28:49.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.037+0000 7f9113fff700 1 --2- 172.21.15.114:0/4124914751 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9114100bc0 0x7f9114196370 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f910400e8f0 tx=0x7f910400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:28:49.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.037+0000 7f91117fa700 1 -- 172.21.15.114:0/4124914751 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9104009790 con 0x7f9114100bc0 2024-02-16T22:28:49.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.037+0000 7f91117fa700 1 -- 172.21.15.114:0/4124914751 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f910400c810 con 0x7f9114100bc0 2024-02-16T22:28:49.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.037+0000 7f911a905700 1 -- 172.21.15.114:0/4124914751 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f911419bac0 con 0x7f9114100bc0 2024-02-16T22:28:49.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.037+0000 7f91117fa700 1 -- 172.21.15.114:0/4124914751 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9104003d20 con 0x7f9114100bc0 2024-02-16T22:28:49.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.037+0000 7f911a905700 1 -- 172.21.15.114:0/4124914751 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f911419c010 con 0x7f9114100bc0 2024-02-16T22:28:49.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.038+0000 7f911a905700 1 -- 172.21.15.114:0/4124914751 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9114109720 con 0x7f9114100bc0 2024-02-16T22:28:49.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.039+0000 7f91117fa700 1 -- 172.21.15.114:0/4124914751 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f91040041f0 con 0x7f9114100bc0 2024-02-16T22:28:49.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.039+0000 7f91117fa700 1 --2- 172.21.15.114:0/4124914751 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f910006d8a0 0x7f910006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:49.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.039+0000 7f91117fa700 1 -- 172.21.15.114:0/4124914751 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9104014070 con 0x7f9114100bc0 2024-02-16T22:28:49.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.040+0000 7f91137fe700 1 --2- 172.21.15.114:0/4124914751 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f910006d8a0 0x7f910006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:49.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.040+0000 7f91137fe700 1 --2- 172.21.15.114:0/4124914751 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f910006d8a0 0x7f910006fd60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f90fc009fd0 tx=0x7f90fc005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:28:49.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.045+0000 7f91117fa700 1 -- 172.21.15.114:0/4124914751 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f910405ab10 con 0x7f9114100bc0 2024-02-16T22:28:49.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.257+0000 7f911a905700 1 -- 172.21.15.114:0/4124914751 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9114106710 con 0x7f910006d8a0 2024-02-16T22:28:49.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.259+0000 7f91117fa700 1 -- 172.21.15.114:0/4124914751 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9114106710 con 0x7f910006d8a0 2024-02-16T22:28:49.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.261+0000 7f911a905700 1 -- 172.21.15.114:0/4124914751 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f910006d8a0 msgr2=0x7f910006fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:49.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.261+0000 7f911a905700 1 --2- 172.21.15.114:0/4124914751 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f910006d8a0 0x7f910006fd60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f90fc009fd0 tx=0x7f90fc005940 comp rx=0 tx=0).stop 2024-02-16T22:28:49.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.261+0000 7f911a905700 1 -- 172.21.15.114:0/4124914751 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9114100bc0 msgr2=0x7f9114196370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:49.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.261+0000 7f911a905700 1 --2- 172.21.15.114:0/4124914751 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9114100bc0 0x7f9114196370 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f910400e8f0 tx=0x7f910400ec00 comp rx=0 tx=0).stop 2024-02-16T22:28:49.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.261+0000 7f911a905700 1 -- 172.21.15.114:0/4124914751 shutdown_connections 2024-02-16T22:28:49.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.261+0000 7f911a905700 1 --2- 172.21.15.114:0/4124914751 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f910006d8a0 0x7f910006fd60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:49.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.261+0000 7f911a905700 1 --2- 172.21.15.114:0/4124914751 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9114100bc0 0x7f9114196370 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:49.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.261+0000 7f911a905700 1 --2- 172.21.15.114:0/4124914751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9114101dc0 0x7f91141968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:49.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.261+0000 7f911a905700 1 -- 172.21.15.114:0/4124914751 >> 172.21.15.114:0/4124914751 conn(0x7f91140fc140 msgr2=0x7f9114104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:28:49.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.261+0000 7f911a905700 1 -- 172.21.15.114:0/4124914751 shutdown_connections 2024-02-16T22:28:49.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.261+0000 7f911a905700 1 -- 172.21.15.114:0/4124914751 wait complete. 2024-02-16T22:28:49.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.364+0000 7fc4c78dd700 1 -- 172.21.15.114:0/1626881536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c0100bb0 msgr2=0x7fc4c0100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:49.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.364+0000 7fc4c78dd700 1 --2- 172.21.15.114:0/1626881536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c0100bb0 0x7fc4c0100fd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009910 tx=0x7fc4bc009c20 comp rx=0 tx=0).stop 2024-02-16T22:28:49.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.364+0000 7fc4c78dd700 1 -- 172.21.15.114:0/1626881536 shutdown_connections 2024-02-16T22:28:49.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.364+0000 7fc4c78dd700 1 --2- 172.21.15.114:0/1626881536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4c0101db0 0x7fc4c0102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:49.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.364+0000 7fc4c78dd700 1 --2- 172.21.15.114:0/1626881536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c0100bb0 0x7fc4c0100fd0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:49.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.364+0000 7fc4c78dd700 1 -- 172.21.15.114:0/1626881536 >> 172.21.15.114:0/1626881536 conn(0x7fc4c00fc110 msgr2=0x7fc4c00fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:28:49.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.364+0000 7fc4c78dd700 1 -- 172.21.15.114:0/1626881536 shutdown_connections 2024-02-16T22:28:49.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.364+0000 7fc4c78dd700 1 -- 172.21.15.114:0/1626881536 wait complete. 2024-02-16T22:28:49.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.364+0000 7fc4c78dd700 1 Processor -- start 2024-02-16T22:28:49.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.364+0000 7fc4c78dd700 1 -- start start 2024-02-16T22:28:49.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.365+0000 7fc4c78dd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4c0100bb0 0x7fc4c0198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:49.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.365+0000 7fc4c78dd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c0101db0 0x7fc4c0198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:49.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.365+0000 7fc4c78dd700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c01990f0 con 0x7fc4c0101db0 2024-02-16T22:28:49.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.365+0000 7fc4c78dd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c0199230 con 0x7fc4c0100bb0 2024-02-16T22:28:49.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.365+0000 7fc4c4e78700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c0101db0 0x7fc4c0198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:49.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.365+0000 7fc4c4e78700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c0101db0 0x7fc4c0198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40554/0 (socket says 172.21.15.114:40554) 2024-02-16T22:28:49.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.365+0000 7fc4c4e78700 1 -- 172.21.15.114:0/1296985728 learned_addr learned my addr 172.21.15.114:0/1296985728 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:28:49.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.365+0000 7fc4c5679700 1 --2- 172.21.15.114:0/1296985728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4c0100bb0 0x7fc4c0198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:49.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.365+0000 7fc4c4e78700 1 -- 172.21.15.114:0/1296985728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4c0100bb0 msgr2=0x7fc4c0198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:49.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.365+0000 7fc4c4e78700 1 --2- 172.21.15.114:0/1296985728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4c0100bb0 0x7fc4c0198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:49.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.365+0000 7fc4c4e78700 1 -- 172.21.15.114:0/1296985728 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4bc0095f0 con 0x7fc4c0101db0 2024-02-16T22:28:49.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.365+0000 7fc4c4e78700 1 --2- 172.21.15.114:0/1296985728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c0101db0 0x7fc4c0198ad0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc4b000e920 tx=0x7fc4b000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:28:49.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.365+0000 7fc4b67fc700 1 -- 172.21.15.114:0/1296985728 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4b000c9d0 con 0x7fc4c0101db0 2024-02-16T22:28:49.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.365+0000 7fc4c78dd700 1 -- 172.21.15.114:0/1296985728 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c019dce0 con 0x7fc4c0101db0 2024-02-16T22:28:49.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.365+0000 7fc4b67fc700 1 -- 172.21.15.114:0/1296985728 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc4b0010430 con 0x7fc4c0101db0 2024-02-16T22:28:49.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.365+0000 7fc4b67fc700 1 -- 172.21.15.114:0/1296985728 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4b0018650 con 0x7fc4c0101db0 2024-02-16T22:28:49.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.366+0000 7fc4c78dd700 1 -- 172.21.15.114:0/1296985728 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c019e230 con 0x7fc4c0101db0 2024-02-16T22:28:49.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.366+0000 7fc4c78dd700 1 -- 172.21.15.114:0/1296985728 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c0105e90 con 0x7fc4c0101db0 2024-02-16T22:28:49.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.367+0000 7fc4b67fc700 1 -- 172.21.15.114:0/1296985728 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fc4b00105a0 con 0x7fc4c0101db0 2024-02-16T22:28:49.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.368+0000 7fc4b67fc700 1 --2- 172.21.15.114:0/1296985728 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fc4ac06d8a0 0x7fc4ac06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:49.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.368+0000 7fc4c5679700 1 --2- 172.21.15.114:0/1296985728 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fc4ac06d8a0 0x7fc4ac06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:49.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.368+0000 7fc4b67fc700 1 -- 172.21.15.114:0/1296985728 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc4b0014070 con 0x7fc4c0101db0 2024-02-16T22:28:49.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.368+0000 7fc4c5679700 1 --2- 172.21.15.114:0/1296985728 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fc4ac06d8a0 0x7fc4ac06fd60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc00b6f0 tx=0x7fc4bc005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:28:49.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.372+0000 7fc4b67fc700 1 -- 172.21.15.114:0/1296985728 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc4b0052d20 con 0x7fc4c0101db0 2024-02-16T22:28:49.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.580+0000 7fc4c78dd700 1 -- 172.21.15.114:0/1296985728 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4c00611d0 con 0x7fc4ac06d8a0 2024-02-16T22:28:49.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.590+0000 7fc4b67fc700 1 -- 172.21.15.114:0/1296985728 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fc4c00611d0 con 0x7fc4ac06d8a0 2024-02-16T22:28:49.593 INFO:teuthology.orchestra.run.smithi114.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:28:49.593 INFO:teuthology.orchestra.run.smithi114.stdout:alertmanager.smithi114 smithi114 *:9093,9094 running (14m) 10s ago 16m 24.3M - 0.25.0 c8568f914cd2 e7404fea400e 2024-02-16T22:28:49.593 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi114 smithi114 running (16m) 10s ago 16m 8896k - 18.2.1 94eadd990e0a 7fd07162cd96 2024-02-16T22:28:49.593 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi123 smithi123 running (15m) 37s ago 15m 8342k - 18.2.1 94eadd990e0a 7687f5e99621 2024-02-16T22:28:49.593 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi114 smithi114 running (4m) 10s ago 16m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 9b8ae9efe83b 2024-02-16T22:28:49.593 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi123 smithi123 running (3m) 37s ago 15m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 6d97263a0ffa 2024-02-16T22:28:49.593 INFO:teuthology.orchestra.run.smithi114.stdout:grafana.smithi114 smithi114 *:3000 running (14m) 10s ago 15m 87.2M - 9.4.7 954c08fa6188 db1c26c3bc20 2024-02-16T22:28:49.593 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.fdxpey smithi114 running (10m) 10s ago 10m 249M - 18.2.1 94eadd990e0a 704b40c144fb 2024-02-16T22:28:49.593 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.zzryke smithi114 running (20s) 10s ago 10m 11.4M - 19.0.0-1450-gf78a58c0 f081f847070a 63fa2af7290e 2024-02-16T22:28:49.593 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.gopcfa smithi123 running (10m) 37s ago 10m 18.6M - 18.2.1 94eadd990e0a 166134b7532a 2024-02-16T22:28:49.593 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.vhlqbe smithi123 running (10m) 37s ago 10m 20.3M - 18.2.1 94eadd990e0a 243c8bb96fa8 2024-02-16T22:28:49.593 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi114.vcullz smithi114 *:8443,9283,8765 running (6m) 10s ago 17m 507M - 19.0.0-1450-gf78a58c0 f081f847070a 14ebeb7888c0 2024-02-16T22:28:49.593 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi123.inzgjd smithi123 *:8443,9283,8765 running (5m) 37s ago 15m 450M - 19.0.0-1450-gf78a58c0 f081f847070a f10aab9faf9e 2024-02-16T22:28:49.594 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi114 smithi114 running (5m) 10s ago 17m 50.4M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 58ac06372523 2024-02-16T22:28:49.594 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi123 smithi123 running (4m) 37s ago 15m 37.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a f76485e0ea24 2024-02-16T22:28:49.594 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi114 smithi114 *:9100 running (16m) 10s ago 16m 15.7M - 1.5.0 0da6a335fe13 744a090e06d0 2024-02-16T22:28:49.594 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi123 smithi123 *:9100 running (15m) 37s ago 15m 14.9M - 1.5.0 0da6a335fe13 3994b8c29b2e 2024-02-16T22:28:49.594 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 smithi114 running (3m) 10s ago 14m 96.2M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 1978fdad94a5 2024-02-16T22:28:49.594 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 smithi114 running (2m) 10s ago 13m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 3f90e792898f 2024-02-16T22:28:49.594 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 smithi114 running (2m) 10s ago 13m 87.3M 1216M 19.0.0-1450-gf78a58c0 f081f847070a c663dfa0518b 2024-02-16T22:28:49.595 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 smithi123 running (107s) 37s ago 13m 99.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a b921448ba169 2024-02-16T22:28:49.595 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 smithi123 running (74s) 37s ago 12m 97.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 54c2926d56b1 2024-02-16T22:28:49.595 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 smithi123 running (40s) 37s ago 12m 12.6M 2240M 19.0.0-1450-gf78a58c0 f081f847070a dd99bf034749 2024-02-16T22:28:49.595 INFO:teuthology.orchestra.run.smithi114.stdout:prometheus.smithi114 smithi114 *:9095 running (5m) 10s ago 15m 63.8M - 2.43.0 a07b618ecd1d 37aac985cdd7 2024-02-16T22:28:49.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.593+0000 7fc4c78dd700 1 -- 172.21.15.114:0/1296985728 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fc4ac06d8a0 msgr2=0x7fc4ac06fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:49.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.593+0000 7fc4c78dd700 1 --2- 172.21.15.114:0/1296985728 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fc4ac06d8a0 0x7fc4ac06fd60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc00b6f0 tx=0x7fc4bc005940 comp rx=0 tx=0).stop 2024-02-16T22:28:49.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.593+0000 7fc4c78dd700 1 -- 172.21.15.114:0/1296985728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c0101db0 msgr2=0x7fc4c0198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:49.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.593+0000 7fc4c78dd700 1 --2- 172.21.15.114:0/1296985728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c0101db0 0x7fc4c0198ad0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc4b000e920 tx=0x7fc4b000ec30 comp rx=0 tx=0).stop 2024-02-16T22:28:49.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.593+0000 7fc4c78dd700 1 -- 172.21.15.114:0/1296985728 shutdown_connections 2024-02-16T22:28:49.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.593+0000 7fc4c78dd700 1 --2- 172.21.15.114:0/1296985728 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fc4ac06d8a0 0x7fc4ac06fd60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:49.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.593+0000 7fc4c78dd700 1 --2- 172.21.15.114:0/1296985728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4c0100bb0 0x7fc4c0198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:49.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.593+0000 7fc4c78dd700 1 --2- 172.21.15.114:0/1296985728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c0101db0 0x7fc4c0198ad0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:49.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.593+0000 7fc4c78dd700 1 -- 172.21.15.114:0/1296985728 >> 172.21.15.114:0/1296985728 conn(0x7fc4c00fc110 msgr2=0x7fc4c0102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:28:49.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.593+0000 7fc4c78dd700 1 -- 172.21.15.114:0/1296985728 shutdown_connections 2024-02-16T22:28:49.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.593+0000 7fc4c78dd700 1 -- 172.21.15.114:0/1296985728 wait complete. 2024-02-16T22:28:49.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.694+0000 7f327b62c700 1 -- 172.21.15.114:0/13681652 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3274101670 msgr2=0x7f3274103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:49.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.694+0000 7f327b62c700 1 --2- 172.21.15.114:0/13681652 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3274101670 0x7f3274103a60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3270009910 tx=0x7f3270009c20 comp rx=0 tx=0).stop 2024-02-16T22:28:49.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.694+0000 7f327b62c700 1 -- 172.21.15.114:0/13681652 shutdown_connections 2024-02-16T22:28:49.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.694+0000 7f327b62c700 1 --2- 172.21.15.114:0/13681652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3274103fa0 0x7f3274106390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:49.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.694+0000 7f327b62c700 1 --2- 172.21.15.114:0/13681652 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3274101670 0x7f3274103a60 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:49.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.695+0000 7f327b62c700 1 -- 172.21.15.114:0/13681652 >> 172.21.15.114:0/13681652 conn(0x7f32740fb080 msgr2=0x7f32740fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:28:49.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.695+0000 7f327b62c700 1 -- 172.21.15.114:0/13681652 shutdown_connections 2024-02-16T22:28:49.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.695+0000 7f327b62c700 1 -- 172.21.15.114:0/13681652 wait complete. 2024-02-16T22:28:49.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.695+0000 7f327b62c700 1 Processor -- start 2024-02-16T22:28:49.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.695+0000 7f327b62c700 1 -- start start 2024-02-16T22:28:49.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.695+0000 7f327b62c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3274101670 0x7f3274196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:49.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.695+0000 7f327b62c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3274103fa0 0x7f3274196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:49.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.695+0000 7f327b62c700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3274196e70 con 0x7f3274103fa0 2024-02-16T22:28:49.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.695+0000 7f327b62c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3274196fb0 con 0x7f3274101670 2024-02-16T22:28:49.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.696+0000 7f3278bc7700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3274103fa0 0x7f3274196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:49.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.696+0000 7f3278bc7700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3274103fa0 0x7f3274196850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40574/0 (socket says 172.21.15.114:40574) 2024-02-16T22:28:49.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.696+0000 7f32793c8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3274101670 0x7f3274196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:49.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.696+0000 7f3278bc7700 1 -- 172.21.15.114:0/1146731340 learned_addr learned my addr 172.21.15.114:0/1146731340 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:28:49.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.696+0000 7f3278bc7700 1 -- 172.21.15.114:0/1146731340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3274101670 msgr2=0x7f3274196310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:49.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.696+0000 7f3278bc7700 1 --2- 172.21.15.114:0/1146731340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3274101670 0x7f3274196310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:49.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.696+0000 7f3278bc7700 1 -- 172.21.15.114:0/1146731340 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32700095f0 con 0x7f3274103fa0 2024-02-16T22:28:49.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.696+0000 7f3278bc7700 1 --2- 172.21.15.114:0/1146731340 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3274103fa0 0x7f3274196850 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f326400d8d0 tx=0x7f326400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:28:49.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.696+0000 7f326a7fc700 1 -- 172.21.15.114:0/1146731340 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32640098a0 con 0x7f3274103fa0 2024-02-16T22:28:49.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.696+0000 7f326a7fc700 1 -- 172.21.15.114:0/1146731340 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3264010460 con 0x7f3274103fa0 2024-02-16T22:28:49.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.696+0000 7f327b62c700 1 -- 172.21.15.114:0/1146731340 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f327419ba60 con 0x7f3274103fa0 2024-02-16T22:28:49.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.696+0000 7f326a7fc700 1 -- 172.21.15.114:0/1146731340 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f326400f5f0 con 0x7f3274103fa0 2024-02-16T22:28:49.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.696+0000 7f327b62c700 1 -- 172.21.15.114:0/1146731340 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f327419bfb0 con 0x7f3274103fa0 2024-02-16T22:28:49.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.697+0000 7f327b62c700 1 -- 172.21.15.114:0/1146731340 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3274190550 con 0x7f3274103fa0 2024-02-16T22:28:49.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.697+0000 7f326a7fc700 1 -- 172.21.15.114:0/1146731340 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f3264009a00 con 0x7f3274103fa0 2024-02-16T22:28:49.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.698+0000 7f326a7fc700 1 --2- 172.21.15.114:0/1146731340 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f326006d8a0 0x7f326006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:49.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.698+0000 7f32793c8700 1 --2- 172.21.15.114:0/1146731340 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f326006d8a0 0x7f326006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:49.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.698+0000 7f326a7fc700 1 -- 172.21.15.114:0/1146731340 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f326408d360 con 0x7f3274103fa0 2024-02-16T22:28:49.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.699+0000 7f32793c8700 1 --2- 172.21.15.114:0/1146731340 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f326006d8a0 0x7f326006fd60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3270006010 tx=0x7f327000b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:28:49.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.703+0000 7f326a7fc700 1 -- 172.21.15.114:0/1146731340 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f326405a6c0 con 0x7f3274103fa0 2024-02-16T22:28:49.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:49 smithi123 ceph-mon[82331]: pgmap v186: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 921 B/s rd, 5.2 KiB/s wr, 3 op/s 2024-02-16T22:28:49.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:49 smithi114 ceph-mon[94452]: pgmap v186: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 921 B/s rd, 5.2 KiB/s wr, 3 op/s 2024-02-16T22:28:49.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.976+0000 7f327b62c700 1 -- 172.21.15.114:0/1146731340 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f327402d090 con 0x7f3274103fa0 2024-02-16T22:28:49.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.977+0000 7f326a7fc700 1 -- 172.21.15.114:0/1146731340 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f326405a250 con 0x7f3274103fa0 2024-02-16T22:28:49.979 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:28:49.979 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": { 2024-02-16T22:28:49.979 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:28:49.979 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:28:49.979 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": { 2024-02-16T22:28:49.979 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:28:49.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:28:49.980 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": { 2024-02-16T22:28:49.980 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T22:28:49.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:28:49.980 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-02-16T22:28:49.980 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-02-16T22:28:49.980 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T22:28:49.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:28:49.980 INFO:teuthology.orchestra.run.smithi114.stdout: "overall": { 2024-02-16T22:28:49.980 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-02-16T22:28:49.980 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 11 2024-02-16T22:28:49.981 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-02-16T22:28:49.981 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:28:49.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.979+0000 7f327b62c700 1 -- 172.21.15.114:0/1146731340 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f326006d8a0 msgr2=0x7f326006fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:49.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.979+0000 7f327b62c700 1 --2- 172.21.15.114:0/1146731340 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f326006d8a0 0x7f326006fd60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3270006010 tx=0x7f327000b540 comp rx=0 tx=0).stop 2024-02-16T22:28:49.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.979+0000 7f327b62c700 1 -- 172.21.15.114:0/1146731340 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3274103fa0 msgr2=0x7f3274196850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:49.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.979+0000 7f327b62c700 1 --2- 172.21.15.114:0/1146731340 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3274103fa0 0x7f3274196850 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f326400d8d0 tx=0x7f326400dbe0 comp rx=0 tx=0).stop 2024-02-16T22:28:49.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.979+0000 7f327b62c700 1 -- 172.21.15.114:0/1146731340 shutdown_connections 2024-02-16T22:28:49.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.979+0000 7f327b62c700 1 --2- 172.21.15.114:0/1146731340 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f326006d8a0 0x7f326006fd60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:49.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.979+0000 7f327b62c700 1 --2- 172.21.15.114:0/1146731340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3274101670 0x7f3274196310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:49.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.979+0000 7f327b62c700 1 --2- 172.21.15.114:0/1146731340 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3274103fa0 0x7f3274196850 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:49.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.979+0000 7f327b62c700 1 -- 172.21.15.114:0/1146731340 >> 172.21.15.114:0/1146731340 conn(0x7f32740fb080 msgr2=0x7f32740fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:28:49.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.979+0000 7f327b62c700 1 -- 172.21.15.114:0/1146731340 shutdown_connections 2024-02-16T22:28:49.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:49.979+0000 7f327b62c700 1 -- 172.21.15.114:0/1146731340 wait complete. 2024-02-16T22:28:50.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.083+0000 7f9bf6dfa700 1 -- 172.21.15.114:0/328642025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bf0103fe0 msgr2=0x7f9bf01063d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:50.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.083+0000 7f9bf6dfa700 1 --2- 172.21.15.114:0/328642025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bf0103fe0 0x7f9bf01063d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc009910 tx=0x7f9bdc009c20 comp rx=0 tx=0).stop 2024-02-16T22:28:50.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.083+0000 7f9bf6dfa700 1 -- 172.21.15.114:0/328642025 shutdown_connections 2024-02-16T22:28:50.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.083+0000 7f9bf6dfa700 1 --2- 172.21.15.114:0/328642025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bf0103fe0 0x7f9bf01063d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:50.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.083+0000 7f9bf6dfa700 1 --2- 172.21.15.114:0/328642025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bf01016b0 0x7f9bf0103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:50.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.083+0000 7f9bf6dfa700 1 -- 172.21.15.114:0/328642025 >> 172.21.15.114:0/328642025 conn(0x7f9bf00fb060 msgr2=0x7f9bf00fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:28:50.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.083+0000 7f9bf6dfa700 1 -- 172.21.15.114:0/328642025 shutdown_connections 2024-02-16T22:28:50.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.083+0000 7f9bf6dfa700 1 -- 172.21.15.114:0/328642025 wait complete. 2024-02-16T22:28:50.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.084+0000 7f9bf6dfa700 1 Processor -- start 2024-02-16T22:28:50.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.084+0000 7f9bf6dfa700 1 -- start start 2024-02-16T22:28:50.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.084+0000 7f9bf6dfa700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bf01016b0 0x7f9bf0198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:50.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.084+0000 7f9bf6dfa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bf0103fe0 0x7f9bf0198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:50.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.084+0000 7f9bf6dfa700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf01990a0 con 0x7f9bf01016b0 2024-02-16T22:28:50.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.084+0000 7f9bf6dfa700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf01991e0 con 0x7f9bf0103fe0 2024-02-16T22:28:50.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.084+0000 7f9bf4b96700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bf01016b0 0x7f9bf0198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:50.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.084+0000 7f9bf4b96700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bf01016b0 0x7f9bf0198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40602/0 (socket says 172.21.15.114:40602) 2024-02-16T22:28:50.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.084+0000 7f9bf4b96700 1 -- 172.21.15.114:0/781864392 learned_addr learned my addr 172.21.15.114:0/781864392 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:28:50.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.084+0000 7f9beffff700 1 --2- 172.21.15.114:0/781864392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bf0103fe0 0x7f9bf0198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:50.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.084+0000 7f9bf4b96700 1 -- 172.21.15.114:0/781864392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bf0103fe0 msgr2=0x7f9bf0198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:50.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.084+0000 7f9bf4b96700 1 --2- 172.21.15.114:0/781864392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bf0103fe0 0x7f9bf0198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:50.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.084+0000 7f9bf4b96700 1 -- 172.21.15.114:0/781864392 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bdc0095f0 con 0x7f9bf01016b0 2024-02-16T22:28:50.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.084+0000 7f9bf4b96700 1 --2- 172.21.15.114:0/781864392 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bf01016b0 0x7f9bf0198540 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9be400e920 tx=0x7f9be400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:28:50.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.085+0000 7f9bedffb700 1 -- 172.21.15.114:0/781864392 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9be400c9d0 con 0x7f9bf01016b0 2024-02-16T22:28:50.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.085+0000 7f9bedffb700 1 -- 172.21.15.114:0/781864392 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9be4010430 con 0x7f9bf01016b0 2024-02-16T22:28:50.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.085+0000 7f9bf6dfa700 1 -- 172.21.15.114:0/781864392 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf019dc90 con 0x7f9bf01016b0 2024-02-16T22:28:50.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.085+0000 7f9bedffb700 1 -- 172.21.15.114:0/781864392 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9be4018650 con 0x7f9bf01016b0 2024-02-16T22:28:50.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.085+0000 7f9bf6dfa700 1 -- 172.21.15.114:0/781864392 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf019e190 con 0x7f9bf01016b0 2024-02-16T22:28:50.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.086+0000 7f9bf6dfa700 1 -- 172.21.15.114:0/781864392 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf00fcc30 con 0x7f9bf01016b0 2024-02-16T22:28:50.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.086+0000 7f9bedffb700 1 -- 172.21.15.114:0/781864392 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f9be40105a0 con 0x7f9bf01016b0 2024-02-16T22:28:50.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.087+0000 7f9bedffb700 1 --2- 172.21.15.114:0/781864392 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f9be006d8a0 0x7f9be006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:50.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.087+0000 7f9bedffb700 1 -- 172.21.15.114:0/781864392 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9be4014070 con 0x7f9bf01016b0 2024-02-16T22:28:50.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.087+0000 7f9beffff700 1 --2- 172.21.15.114:0/781864392 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f9be006d8a0 0x7f9be006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:50.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.087+0000 7f9beffff700 1 --2- 172.21.15.114:0/781864392 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f9be006d8a0 0x7f9be006fd60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc009fd0 tx=0x7f9bdc005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:28:50.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.092+0000 7f9bedffb700 1 -- 172.21.15.114:0/781864392 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9be4052d20 con 0x7f9bf01016b0 2024-02-16T22:28:50.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.332+0000 7f9bf6dfa700 1 -- 172.21.15.114:0/781864392 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9bf0061ae0 con 0x7f9bf01016b0 2024-02-16T22:28:50.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.332+0000 7f9bedffb700 1 -- 172.21.15.114:0/781864392 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1673 (secure 0 0 0) 0x7f9be405aa80 con 0x7f9bf01016b0 2024-02-16T22:28:50.334 INFO:teuthology.orchestra.run.smithi114.stdout:e22 2024-02-16T22:28:50.335 INFO:teuthology.orchestra.run.smithi114.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:28:50.335 INFO:teuthology.orchestra.run.smithi114.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:28:50.335 INFO:teuthology.orchestra.run.smithi114.stdout:legacy client fscid: 1 2024-02-16T22:28:50.335 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:28:50.335 INFO:teuthology.orchestra.run.smithi114.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:28:50.335 INFO:teuthology.orchestra.run.smithi114.stdout:fs_name cephfs 2024-02-16T22:28:50.335 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 22 2024-02-16T22:28:50.335 INFO:teuthology.orchestra.run.smithi114.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:28:50.335 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-02-16T22:18:02.108825+0000 2024-02-16T22:28:50.335 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-02-16T22:28:47.642938+0000 2024-02-16T22:28:50.335 INFO:teuthology.orchestra.run.smithi114.stdout:tableserver 0 2024-02-16T22:28:50.335 INFO:teuthology.orchestra.run.smithi114.stdout:root 0 2024-02-16T22:28:50.335 INFO:teuthology.orchestra.run.smithi114.stdout:session_timeout 60 2024-02-16T22:28:50.336 INFO:teuthology.orchestra.run.smithi114.stdout:session_autoclose 300 2024-02-16T22:28:50.336 INFO:teuthology.orchestra.run.smithi114.stdout:max_file_size 1099511627776 2024-02-16T22:28:50.336 INFO:teuthology.orchestra.run.smithi114.stdout:max_xattr_size 65536 2024-02-16T22:28:50.336 INFO:teuthology.orchestra.run.smithi114.stdout:required_client_features {} 2024-02-16T22:28:50.336 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure 0 2024-02-16T22:28:50.336 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure_osd_epoch 80 2024-02-16T22:28:50.336 INFO:teuthology.orchestra.run.smithi114.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:28:50.336 INFO:teuthology.orchestra.run.smithi114.stdout:max_mds 1 2024-02-16T22:28:50.336 INFO:teuthology.orchestra.run.smithi114.stdout:in 0 2024-02-16T22:28:50.336 INFO:teuthology.orchestra.run.smithi114.stdout:up {0=24269} 2024-02-16T22:28:50.336 INFO:teuthology.orchestra.run.smithi114.stdout:failed 2024-02-16T22:28:50.336 INFO:teuthology.orchestra.run.smithi114.stdout:damaged 2024-02-16T22:28:50.337 INFO:teuthology.orchestra.run.smithi114.stdout:stopped 2024-02-16T22:28:50.337 INFO:teuthology.orchestra.run.smithi114.stdout:data_pools [3] 2024-02-16T22:28:50.337 INFO:teuthology.orchestra.run.smithi114.stdout:metadata_pool 2 2024-02-16T22:28:50.337 INFO:teuthology.orchestra.run.smithi114.stdout:inline_data enabled 2024-02-16T22:28:50.337 INFO:teuthology.orchestra.run.smithi114.stdout:balancer 2024-02-16T22:28:50.337 INFO:teuthology.orchestra.run.smithi114.stdout:bal_rank_mask -1 2024-02-16T22:28:50.337 INFO:teuthology.orchestra.run.smithi114.stdout:standby_count_wanted 1 2024-02-16T22:28:50.337 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.vhlqbe{0:24269} state up:replay seq 3 join_fscid=1 addr [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:28:50.337 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:28:50.337 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:28:50.337 INFO:teuthology.orchestra.run.smithi114.stdout:Standby daemons: 2024-02-16T22:28:50.337 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:28:50.337 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.gopcfa{-1:24285} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:28:50.338 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.zzryke{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.114:6826/2017800151,v1:172.21.15.114:6827/2017800151] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T22:28:50.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.335+0000 7f9bf6dfa700 1 -- 172.21.15.114:0/781864392 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f9be006d8a0 msgr2=0x7f9be006fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:50.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.335+0000 7f9bf6dfa700 1 --2- 172.21.15.114:0/781864392 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f9be006d8a0 0x7f9be006fd60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc009fd0 tx=0x7f9bdc005940 comp rx=0 tx=0).stop 2024-02-16T22:28:50.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.335+0000 7f9bf6dfa700 1 -- 172.21.15.114:0/781864392 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bf01016b0 msgr2=0x7f9bf0198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:50.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.335+0000 7f9bf6dfa700 1 --2- 172.21.15.114:0/781864392 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bf01016b0 0x7f9bf0198540 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9be400e920 tx=0x7f9be400ec30 comp rx=0 tx=0).stop 2024-02-16T22:28:50.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.335+0000 7f9bf6dfa700 1 -- 172.21.15.114:0/781864392 shutdown_connections 2024-02-16T22:28:50.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.335+0000 7f9bf6dfa700 1 --2- 172.21.15.114:0/781864392 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f9be006d8a0 0x7f9be006fd60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:50.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.335+0000 7f9bf6dfa700 1 --2- 172.21.15.114:0/781864392 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bf01016b0 0x7f9bf0198540 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:50.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.335+0000 7f9bf6dfa700 1 --2- 172.21.15.114:0/781864392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bf0103fe0 0x7f9bf0198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:50.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.335+0000 7f9bf6dfa700 1 -- 172.21.15.114:0/781864392 >> 172.21.15.114:0/781864392 conn(0x7f9bf00fb060 msgr2=0x7f9bf00ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:28:50.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.335+0000 7f9bf6dfa700 1 -- 172.21.15.114:0/781864392 shutdown_connections 2024-02-16T22:28:50.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.335+0000 7f9bf6dfa700 1 -- 172.21.15.114:0/781864392 wait complete. 2024-02-16T22:28:50.339 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 22 2024-02-16T22:28:50.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.440+0000 7ff62bb8a700 1 -- 172.21.15.114:0/1560742194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff624074dc0 msgr2=0x7ff624073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:50.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.440+0000 7ff62bb8a700 1 --2- 172.21.15.114:0/1560742194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff624074dc0 0x7ff624073220 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff620009910 tx=0x7ff620009c20 comp rx=0 tx=0).stop 2024-02-16T22:28:50.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.440+0000 7ff62bb8a700 1 -- 172.21.15.114:0/1560742194 shutdown_connections 2024-02-16T22:28:50.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.440+0000 7ff62bb8a700 1 --2- 172.21.15.114:0/1560742194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6240737f0 0x7ff624073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:50.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.440+0000 7ff62bb8a700 1 --2- 172.21.15.114:0/1560742194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff624074dc0 0x7ff624073220 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:50.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.441+0000 7ff62bb8a700 1 -- 172.21.15.114:0/1560742194 >> 172.21.15.114:0/1560742194 conn(0x7ff6240fc140 msgr2=0x7ff6240fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:28:50.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.441+0000 7ff62bb8a700 1 -- 172.21.15.114:0/1560742194 shutdown_connections 2024-02-16T22:28:50.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.441+0000 7ff62bb8a700 1 -- 172.21.15.114:0/1560742194 wait complete. 2024-02-16T22:28:50.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.441+0000 7ff62bb8a700 1 Processor -- start 2024-02-16T22:28:50.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.441+0000 7ff62bb8a700 1 -- start start 2024-02-16T22:28:50.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.441+0000 7ff62bb8a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6240737f0 0x7ff62419c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:50.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.441+0000 7ff62bb8a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff624074dc0 0x7ff62419cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:50.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.441+0000 7ff62bb8a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62419d4c0 con 0x7ff624074dc0 2024-02-16T22:28:50.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.441+0000 7ff62bb8a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62419d600 con 0x7ff6240737f0 2024-02-16T22:28:50.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.441+0000 7ff629125700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff624074dc0 0x7ff62419cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:50.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.441+0000 7ff629125700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff624074dc0 0x7ff62419cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40612/0 (socket says 172.21.15.114:40612) 2024-02-16T22:28:50.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.441+0000 7ff629125700 1 -- 172.21.15.114:0/1679930383 learned_addr learned my addr 172.21.15.114:0/1679930383 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:28:50.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.441+0000 7ff629926700 1 --2- 172.21.15.114:0/1679930383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6240737f0 0x7ff62419c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:50.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.442+0000 7ff629125700 1 -- 172.21.15.114:0/1679930383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6240737f0 msgr2=0x7ff62419c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:50.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.442+0000 7ff629125700 1 --2- 172.21.15.114:0/1679930383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6240737f0 0x7ff62419c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:50.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.442+0000 7ff629125700 1 -- 172.21.15.114:0/1679930383 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6200095f0 con 0x7ff624074dc0 2024-02-16T22:28:50.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.442+0000 7ff629125700 1 --2- 172.21.15.114:0/1679930383 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff624074dc0 0x7ff62419cea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff61400d900 tx=0x7ff61400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:28:50.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.442+0000 7ff61affd700 1 -- 172.21.15.114:0/1679930383 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6140041f0 con 0x7ff624074dc0 2024-02-16T22:28:50.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.442+0000 7ff62bb8a700 1 -- 172.21.15.114:0/1679930383 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6241a20b0 con 0x7ff624074dc0 2024-02-16T22:28:50.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.442+0000 7ff61affd700 1 -- 172.21.15.114:0/1679930383 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff614004350 con 0x7ff624074dc0 2024-02-16T22:28:50.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.442+0000 7ff61affd700 1 -- 172.21.15.114:0/1679930383 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff614003d20 con 0x7ff624074dc0 2024-02-16T22:28:50.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.442+0000 7ff62bb8a700 1 -- 172.21.15.114:0/1679930383 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6241a2600 con 0x7ff624074dc0 2024-02-16T22:28:50.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.443+0000 7ff62bb8a700 1 -- 172.21.15.114:0/1679930383 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff62410fd30 con 0x7ff624074dc0 2024-02-16T22:28:50.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.444+0000 7ff61affd700 1 -- 172.21.15.114:0/1679930383 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7ff614009730 con 0x7ff624074dc0 2024-02-16T22:28:50.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.444+0000 7ff61affd700 1 --2- 172.21.15.114:0/1679930383 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff61006d8a0 0x7ff61006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:50.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.444+0000 7ff61affd700 1 -- 172.21.15.114:0/1679930383 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff61408d400 con 0x7ff624074dc0 2024-02-16T22:28:50.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.444+0000 7ff629926700 1 --2- 172.21.15.114:0/1679930383 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff61006d8a0 0x7ff61006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:50.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.445+0000 7ff629926700 1 --2- 172.21.15.114:0/1679930383 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff61006d8a0 0x7ff61006fd60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff620006010 tx=0x7ff620005ec0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:28:50.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.449+0000 7ff61affd700 1 -- 172.21.15.114:0/1679930383 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff61405a6e0 con 0x7ff624074dc0 2024-02-16T22:28:50.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.661+0000 7ff62bb8a700 1 -- 172.21.15.114:0/1679930383 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff624103c70 con 0x7ff61006d8a0 2024-02-16T22:28:50.664 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:50 smithi114 ceph-mon[94452]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:28:50.664 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:50 smithi114 ceph-mon[94452]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:28:50.664 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:50 smithi114 ceph-mon[94452]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:28:50.664 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:50 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/1146731340' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:28:50.664 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:50 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/781864392' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:28:50.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.662+0000 7ff61affd700 1 -- 172.21.15.114:0/1679930383 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff624103c70 con 0x7ff61006d8a0 2024-02-16T22:28:50.665 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:28:50.665 INFO:teuthology.orchestra.run.smithi114.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:28:50.665 INFO:teuthology.orchestra.run.smithi114.stdout: "in_progress": true, 2024-02-16T22:28:50.665 INFO:teuthology.orchestra.run.smithi114.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:28:50.665 INFO:teuthology.orchestra.run.smithi114.stdout: "services_complete": [ 2024-02-16T22:28:50.665 INFO:teuthology.orchestra.run.smithi114.stdout: "osd", 2024-02-16T22:28:50.665 INFO:teuthology.orchestra.run.smithi114.stdout: "crash", 2024-02-16T22:28:50.665 INFO:teuthology.orchestra.run.smithi114.stdout: "mon", 2024-02-16T22:28:50.665 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr" 2024-02-16T22:28:50.665 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-02-16T22:28:50.665 INFO:teuthology.orchestra.run.smithi114.stdout: "progress": "13/23 daemons upgraded", 2024-02-16T22:28:50.665 INFO:teuthology.orchestra.run.smithi114.stdout: "message": "Currently upgrading mds daemons", 2024-02-16T22:28:50.665 INFO:teuthology.orchestra.run.smithi114.stdout: "is_paused": false 2024-02-16T22:28:50.666 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:28:50.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.664+0000 7ff62bb8a700 1 -- 172.21.15.114:0/1679930383 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff61006d8a0 msgr2=0x7ff61006fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:50.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.664+0000 7ff62bb8a700 1 --2- 172.21.15.114:0/1679930383 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff61006d8a0 0x7ff61006fd60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff620006010 tx=0x7ff620005ec0 comp rx=0 tx=0).stop 2024-02-16T22:28:50.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.664+0000 7ff62bb8a700 1 -- 172.21.15.114:0/1679930383 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff624074dc0 msgr2=0x7ff62419cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:50.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.664+0000 7ff62bb8a700 1 --2- 172.21.15.114:0/1679930383 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff624074dc0 0x7ff62419cea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff61400d900 tx=0x7ff61400dc10 comp rx=0 tx=0).stop 2024-02-16T22:28:50.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.664+0000 7ff62bb8a700 1 -- 172.21.15.114:0/1679930383 shutdown_connections 2024-02-16T22:28:50.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.664+0000 7ff62bb8a700 1 --2- 172.21.15.114:0/1679930383 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff61006d8a0 0x7ff61006fd60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:50.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.664+0000 7ff62bb8a700 1 --2- 172.21.15.114:0/1679930383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6240737f0 0x7ff62419c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:50.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.664+0000 7ff62bb8a700 1 --2- 172.21.15.114:0/1679930383 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff624074dc0 0x7ff62419cea0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:50.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.664+0000 7ff62bb8a700 1 -- 172.21.15.114:0/1679930383 >> 172.21.15.114:0/1679930383 conn(0x7ff6240fc140 msgr2=0x7ff624102550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:28:50.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.665+0000 7ff62bb8a700 1 -- 172.21.15.114:0/1679930383 shutdown_connections 2024-02-16T22:28:50.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.665+0000 7ff62bb8a700 1 -- 172.21.15.114:0/1679930383 wait complete. 2024-02-16T22:28:50.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.767+0000 7fd5af99a700 1 -- 172.21.15.114:0/3048383377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5a8101eb0 msgr2=0x7fd5a8102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:50.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.767+0000 7fd5af99a700 1 --2- 172.21.15.114:0/3048383377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5a8101eb0 0x7fd5a8102310 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd598009910 tx=0x7fd598009c20 comp rx=0 tx=0).stop 2024-02-16T22:28:50.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.767+0000 7fd5af99a700 1 -- 172.21.15.114:0/3048383377 shutdown_connections 2024-02-16T22:28:50.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.767+0000 7fd5af99a700 1 --2- 172.21.15.114:0/3048383377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5a8101eb0 0x7fd5a8102310 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:50.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.767+0000 7fd5af99a700 1 --2- 172.21.15.114:0/3048383377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5a80feed0 0x7fd5a80ff2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:50.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.767+0000 7fd5af99a700 1 -- 172.21.15.114:0/3048383377 >> 172.21.15.114:0/3048383377 conn(0x7fd5a80fa810 msgr2=0x7fd5a80fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:28:50.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.767+0000 7fd5af99a700 1 -- 172.21.15.114:0/3048383377 shutdown_connections 2024-02-16T22:28:50.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.767+0000 7fd5af99a700 1 -- 172.21.15.114:0/3048383377 wait complete. 2024-02-16T22:28:50.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.767+0000 7fd5af99a700 1 Processor -- start 2024-02-16T22:28:50.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.768+0000 7fd5af99a700 1 -- start start 2024-02-16T22:28:50.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.768+0000 7fd5af99a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5a80feed0 0x7fd5a81963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:50.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.768+0000 7fd5af99a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5a8101eb0 0x7fd5a8196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:50.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.768+0000 7fd5af99a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a8196f40 con 0x7fd5a8101eb0 2024-02-16T22:28:50.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.768+0000 7fd5af99a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a8197080 con 0x7fd5a80feed0 2024-02-16T22:28:50.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.768+0000 7fd5acf35700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5a8101eb0 0x7fd5a8196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:50.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.768+0000 7fd5acf35700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5a8101eb0 0x7fd5a8196920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40628/0 (socket says 172.21.15.114:40628) 2024-02-16T22:28:50.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.768+0000 7fd5acf35700 1 -- 172.21.15.114:0/1841264744 learned_addr learned my addr 172.21.15.114:0/1841264744 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:28:50.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.768+0000 7fd5ad736700 1 --2- 172.21.15.114:0/1841264744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5a80feed0 0x7fd5a81963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:50.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.768+0000 7fd5acf35700 1 -- 172.21.15.114:0/1841264744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5a80feed0 msgr2=0x7fd5a81963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:50.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.768+0000 7fd5acf35700 1 --2- 172.21.15.114:0/1841264744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5a80feed0 0x7fd5a81963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:50.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.768+0000 7fd5acf35700 1 -- 172.21.15.114:0/1841264744 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5980095f0 con 0x7fd5a8101eb0 2024-02-16T22:28:50.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.768+0000 7fd5acf35700 1 --2- 172.21.15.114:0/1841264744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5a8101eb0 0x7fd5a8196920 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd59800b5c0 tx=0x7fd59800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:28:50.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.768+0000 7fd59e7fc700 1 -- 172.21.15.114:0/1841264744 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd59801d070 con 0x7fd5a8101eb0 2024-02-16T22:28:50.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.768+0000 7fd59e7fc700 1 -- 172.21.15.114:0/1841264744 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd59800f460 con 0x7fd5a8101eb0 2024-02-16T22:28:50.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.768+0000 7fd5af99a700 1 -- 172.21.15.114:0/1841264744 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a819bad0 con 0x7fd5a8101eb0 2024-02-16T22:28:50.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.769+0000 7fd59e7fc700 1 -- 172.21.15.114:0/1841264744 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd598017630 con 0x7fd5a8101eb0 2024-02-16T22:28:50.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.769+0000 7fd5af99a700 1 -- 172.21.15.114:0/1841264744 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a819bfc0 con 0x7fd5a8101eb0 2024-02-16T22:28:50.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.769+0000 7fd5af99a700 1 -- 172.21.15.114:0/1841264744 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5a8109770 con 0x7fd5a8101eb0 2024-02-16T22:28:50.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.770+0000 7fd59e7fc700 1 -- 172.21.15.114:0/1841264744 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fd59800fa00 con 0x7fd5a8101eb0 2024-02-16T22:28:50.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.770+0000 7fd59e7fc700 1 --2- 172.21.15.114:0/1841264744 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd59406d8a0 0x7fd59406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:28:50.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.770+0000 7fd59e7fc700 1 -- 172.21.15.114:0/1841264744 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd59808ea00 con 0x7fd5a8101eb0 2024-02-16T22:28:50.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.770+0000 7fd5ad736700 1 --2- 172.21.15.114:0/1841264744 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd59406d8a0 0x7fd59406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:28:50.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.771+0000 7fd5ad736700 1 --2- 172.21.15.114:0/1841264744 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd59406d8a0 0x7fd59406fd60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8068bb0 tx=0x7fd5a400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:28:50.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:50.775+0000 7fd59e7fc700 1 -- 172.21.15.114:0/1841264744 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd59805bce0 con 0x7fd5a8101eb0 2024-02-16T22:28:50.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:50 smithi123 ceph-mon[82331]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:28:50.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:50 smithi123 ceph-mon[82331]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:28:50.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:50 smithi123 ceph-mon[82331]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:28:50.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:50 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/1146731340' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:28:50.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:50 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/781864392' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:28:51.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:51.054+0000 7fd5af99a700 1 -- 172.21.15.114:0/1841264744 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd5a8109930 con 0x7fd5a8101eb0 2024-02-16T22:28:51.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:51.055+0000 7fd59e7fc700 1 -- 172.21.15.114:0/1841264744 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+297 (secure 0 0 0) 0x7fd59805b870 con 0x7fd5a8101eb0 2024-02-16T22:28:51.057 INFO:teuthology.orchestra.run.smithi114.stdout:HEALTH_WARN 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data 2024-02-16T22:28:51.057 INFO:teuthology.orchestra.run.smithi114.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-16T22:28:51.057 INFO:teuthology.orchestra.run.smithi114.stdout: fs cephfs is degraded 2024-02-16T22:28:51.057 INFO:teuthology.orchestra.run.smithi114.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:28:51.057 INFO:teuthology.orchestra.run.smithi114.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:28:51.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:51.057+0000 7fd5af99a700 1 -- 172.21.15.114:0/1841264744 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd59406d8a0 msgr2=0x7fd59406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:51.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:51.057+0000 7fd5af99a700 1 --2- 172.21.15.114:0/1841264744 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd59406d8a0 0x7fd59406fd60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8068bb0 tx=0x7fd5a400b410 comp rx=0 tx=0).stop 2024-02-16T22:28:51.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:51.057+0000 7fd5af99a700 1 -- 172.21.15.114:0/1841264744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5a8101eb0 msgr2=0x7fd5a8196920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:28:51.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:51.057+0000 7fd5af99a700 1 --2- 172.21.15.114:0/1841264744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5a8101eb0 0x7fd5a8196920 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd59800b5c0 tx=0x7fd59800bd70 comp rx=0 tx=0).stop 2024-02-16T22:28:51.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:51.057+0000 7fd5af99a700 1 -- 172.21.15.114:0/1841264744 shutdown_connections 2024-02-16T22:28:51.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:51.057+0000 7fd5af99a700 1 --2- 172.21.15.114:0/1841264744 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd59406d8a0 0x7fd59406fd60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:51.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:51.057+0000 7fd5af99a700 1 --2- 172.21.15.114:0/1841264744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5a80feed0 0x7fd5a81963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:51.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:51.057+0000 7fd5af99a700 1 --2- 172.21.15.114:0/1841264744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5a8101eb0 0x7fd5a8196920 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:28:51.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:51.057+0000 7fd5af99a700 1 -- 172.21.15.114:0/1841264744 >> 172.21.15.114:0/1841264744 conn(0x7fd5a80fa810 msgr2=0x7fd5a81050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:28:51.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:51.057+0000 7fd5af99a700 1 -- 172.21.15.114:0/1841264744 shutdown_connections 2024-02-16T22:28:51.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:28:51.057+0000 7fd5af99a700 1 -- 172.21.15.114:0/1841264744 wait complete. 2024-02-16T22:28:51.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:51 smithi123 ceph-mon[82331]: pgmap v187: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 4.8 MiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-02-16T22:28:51.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:51 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/1841264744' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:28:51.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:51 smithi114 ceph-mon[94452]: pgmap v187: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 4.8 MiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-02-16T22:28:51.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:51 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/1841264744' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:28:52.817 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:52 smithi123 ceph-mon[82331]: from='client.34428 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:28:52.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:52 smithi114 ceph-mon[94452]: from='client.34428 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:28:53.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:53 smithi114 ceph-mon[94452]: pgmap v188: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 5.6 MiB/s rd, 2 op/s 2024-02-16T22:28:54.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:53 smithi123 ceph-mon[82331]: pgmap v188: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 5.6 MiB/s rd, 2 op/s 2024-02-16T22:28:55.870 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:55 smithi114 ceph-mon[94452]: pgmap v189: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-02-16T22:28:56.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:55 smithi123 ceph-mon[82331]: pgmap v189: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-02-16T22:28:57.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:57 smithi114 ceph-mon[94452]: pgmap v190: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-02-16T22:28:57.455 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:57 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:57.455 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:57 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:57.455 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:57 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:28:57.455 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:57 smithi114 ceph-mon[94452]: mds.? [v2:172.21.15.114:6828/3702914193,v1:172.21.15.114:6829/3702914193] up:boot 2024-02-16T22:28:57.455 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:57 smithi114 ceph-mon[94452]: fsmap cephfs:1/1 {0=cephfs.smithi123.vhlqbe=up:replay} 3 up:standby 2024-02-16T22:28:57.455 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:57 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi114.fdxpey"}]: dispatch 2024-02-16T22:28:57.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:57 smithi123 ceph-mon[82331]: pgmap v190: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-02-16T22:28:57.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:57 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:57.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:57 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:28:57.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:57 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:28:57.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:57 smithi123 ceph-mon[82331]: mds.? [v2:172.21.15.114:6828/3702914193,v1:172.21.15.114:6829/3702914193] up:boot 2024-02-16T22:28:57.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:57 smithi123 ceph-mon[82331]: fsmap cephfs:1/1 {0=cephfs.smithi123.vhlqbe=up:replay} 3 up:standby 2024-02-16T22:28:57.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:57 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi114.fdxpey"}]: dispatch 2024-02-16T22:28:58.695 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:58 smithi114 ceph-mon[94452]: reconnect by client.24321 192.168.0.1:0/2666785881 after 0 2024-02-16T22:28:58.695 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:58 smithi114 ceph-mon[94452]: reconnect by client.14536 192.168.0.1:0/21264991 after 0.000999978 2024-02-16T22:28:58.695 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:58 smithi114 ceph-mon[94452]: mds.? [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] up:reconnect 2024-02-16T22:28:58.695 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:58 smithi114 ceph-mon[94452]: fsmap cephfs:1/1 {0=cephfs.smithi123.vhlqbe=up:reconnect} 3 up:standby 2024-02-16T22:28:59.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:58 smithi123 ceph-mon[82331]: reconnect by client.24321 192.168.0.1:0/2666785881 after 0 2024-02-16T22:28:59.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:58 smithi123 ceph-mon[82331]: reconnect by client.14536 192.168.0.1:0/21264991 after 0.000999978 2024-02-16T22:28:59.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:58 smithi123 ceph-mon[82331]: mds.? [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] up:reconnect 2024-02-16T22:28:59.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:58 smithi123 ceph-mon[82331]: fsmap cephfs:1/1 {0=cephfs.smithi123.vhlqbe=up:reconnect} 3 up:standby 2024-02-16T22:28:59.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:59 smithi114 ceph-mon[94452]: pgmap v191: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-02-16T22:28:59.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:59 smithi114 ceph-mon[94452]: mds.? [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] up:rejoin 2024-02-16T22:28:59.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:59 smithi114 ceph-mon[94452]: fsmap cephfs:1/1 {0=cephfs.smithi123.vhlqbe=up:rejoin} 3 up:standby 2024-02-16T22:28:59.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:28:59 smithi114 ceph-mon[94452]: daemon mds.cephfs.smithi123.vhlqbe is now active in filesystem cephfs as rank 0 2024-02-16T22:29:00.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:59 smithi123 ceph-mon[82331]: pgmap v191: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-02-16T22:29:00.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:59 smithi123 ceph-mon[82331]: mds.? [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] up:rejoin 2024-02-16T22:29:00.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:59 smithi123 ceph-mon[82331]: fsmap cephfs:1/1 {0=cephfs.smithi123.vhlqbe=up:rejoin} 3 up:standby 2024-02-16T22:29:00.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:28:59 smithi123 ceph-mon[82331]: daemon mds.cephfs.smithi123.vhlqbe is now active in filesystem cephfs as rank 0 2024-02-16T22:29:00.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:00 smithi114 ceph-mon[94452]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-16T22:29:00.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:00 smithi114 ceph-mon[94452]: mds.? [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] up:active 2024-02-16T22:29:00.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:00 smithi114 ceph-mon[94452]: fsmap cephfs:1 {0=cephfs.smithi123.vhlqbe=up:active} 3 up:standby 2024-02-16T22:29:01.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:00 smithi123 ceph-mon[82331]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-16T22:29:01.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:00 smithi123 ceph-mon[82331]: mds.? [v2:172.21.15.123:6824/418014612,v1:172.21.15.123:6825/418014612] up:active 2024-02-16T22:29:01.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:00 smithi123 ceph-mon[82331]: fsmap cephfs:1 {0=cephfs.smithi123.vhlqbe=up:active} 3 up:standby 2024-02-16T22:29:01.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:01 smithi114 ceph-mon[94452]: pgmap v192: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 85 B/s wr, 6 op/s 2024-02-16T22:29:01.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:29:01.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:01.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:02.005 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:01 smithi123 ceph-mon[82331]: pgmap v192: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 85 B/s wr, 6 op/s 2024-02-16T22:29:02.006 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:01 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:29:02.006 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:01 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:02.006 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:01 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:03.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:03 smithi114 ceph-mon[94452]: pgmap v193: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 170 B/s wr, 5 op/s 2024-02-16T22:29:03.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:03 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:03.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:03 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:04.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:03 smithi123 ceph-mon[82331]: pgmap v193: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 170 B/s wr, 5 op/s 2024-02-16T22:29:04.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:03 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:04.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:03 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:05.943 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:05 smithi114 ceph-mon[94452]: pgmap v194: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 170 B/s wr, 7 op/s 2024-02-16T22:29:06.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:05 smithi123 ceph-mon[82331]: pgmap v194: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 170 B/s wr, 7 op/s 2024-02-16T22:29:07.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:06 smithi123 ceph-mon[82331]: pgmap v195: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 170 B/s wr, 5 op/s 2024-02-16T22:29:07.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:06 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:07.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:06 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:07.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:06 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:29:07.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:06 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:29:07.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:06 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:07.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:06 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:29:07.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:06 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:07.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:06 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:07.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:06 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:07.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:06 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:07.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:06 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi123.vhlqbe"]}]: dispatch 2024-02-16T22:29:07.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:06 smithi123 ceph-mon[82331]: Upgrade: It appears safe to stop mds.cephfs.smithi123.vhlqbe 2024-02-16T22:29:07.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:06 smithi114 ceph-mon[94452]: pgmap v195: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 170 B/s wr, 5 op/s 2024-02-16T22:29:07.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:06 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:07.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:06 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:07.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:06 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:29:07.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:06 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:29:07.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:06 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:07.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:06 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:29:07.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:06 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:07.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:06 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:07.453 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:06 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:07.453 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:06 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:07.453 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:06 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi123.vhlqbe"]}]: dispatch 2024-02-16T22:29:07.453 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:06 smithi114 ceph-mon[94452]: Upgrade: It appears safe to stop mds.cephfs.smithi123.vhlqbe 2024-02-16T22:29:09.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:08 smithi123 ceph-mon[82331]: pgmap v196: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 5.2 MiB/s rd, 4.1 KiB/s wr, 6 op/s 2024-02-16T22:29:09.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:08 smithi123 ceph-mon[82331]: Upgrade: Updating mds.cephfs.smithi123.vhlqbe 2024-02-16T22:29:09.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:08 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:09.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:08 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi123.vhlqbe", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:29:09.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:08 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:29:09.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:08 smithi123 ceph-mon[82331]: Deploying daemon mds.cephfs.smithi123.vhlqbe on smithi123 2024-02-16T22:29:09.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:08 smithi114 ceph-mon[94452]: pgmap v196: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 5.2 MiB/s rd, 4.1 KiB/s wr, 6 op/s 2024-02-16T22:29:09.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:08 smithi114 ceph-mon[94452]: Upgrade: Updating mds.cephfs.smithi123.vhlqbe 2024-02-16T22:29:09.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:08 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:09.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:08 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi123.vhlqbe", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:29:09.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:08 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:29:09.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:08 smithi114 ceph-mon[94452]: Deploying daemon mds.cephfs.smithi123.vhlqbe on smithi123 2024-02-16T22:29:10.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:09 smithi123 ceph-mon[82331]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-16T22:29:10.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:09 smithi123 ceph-mon[82331]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T22:29:10.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:09 smithi123 ceph-mon[82331]: osdmap e81: 6 total, 6 up, 6 in 2024-02-16T22:29:10.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:09 smithi123 ceph-mon[82331]: Standby daemon mds.cephfs.smithi123.gopcfa assigned to filesystem cephfs as rank 0 2024-02-16T22:29:10.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:09 smithi123 ceph-mon[82331]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T22:29:10.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:09 smithi123 ceph-mon[82331]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-02-16T22:29:10.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:09 smithi123 ceph-mon[82331]: fsmap cephfs:1/1 {0=cephfs.smithi123.gopcfa=up:replay} 2 up:standby 2024-02-16T22:29:10.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:09 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi114[94448]: 2024-02-16T22:29:09.777+0000 7f1ab68f5700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T22:29:10.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:09 smithi114 ceph-mon[94452]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-16T22:29:10.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:09 smithi114 ceph-mon[94452]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T22:29:10.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:09 smithi114 ceph-mon[94452]: osdmap e81: 6 total, 6 up, 6 in 2024-02-16T22:29:10.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:09 smithi114 ceph-mon[94452]: Standby daemon mds.cephfs.smithi123.gopcfa assigned to filesystem cephfs as rank 0 2024-02-16T22:29:10.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:09 smithi114 ceph-mon[94452]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T22:29:10.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:09 smithi114 ceph-mon[94452]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-02-16T22:29:10.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:09 smithi114 ceph-mon[94452]: fsmap cephfs:1/1 {0=cephfs.smithi123.gopcfa=up:replay} 2 up:standby 2024-02-16T22:29:11.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:10 smithi114 ceph-mon[94452]: pgmap v198: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 4.0 KiB/s rd, 4.8 KiB/s wr, 6 op/s 2024-02-16T22:29:11.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:10 smithi123 ceph-mon[82331]: pgmap v198: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 4.0 KiB/s rd, 4.8 KiB/s wr, 6 op/s 2024-02-16T22:29:12.137 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:11 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi123.vhlqbe"}]: dispatch 2024-02-16T22:29:12.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:11 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi123.vhlqbe"}]: dispatch 2024-02-16T22:29:13.103 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:12 smithi114 ceph-mon[94452]: pgmap v199: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 413 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-02-16T22:29:13.161 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:12 smithi123 ceph-mon[82331]: pgmap v199: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 413 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-02-16T22:29:15.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:14 smithi114 ceph-mon[94452]: pgmap v200: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 4.4 MiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-02-16T22:29:15.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:14 smithi123 ceph-mon[82331]: pgmap v200: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 4.4 MiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-02-16T22:29:17.199 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:16 smithi123 ceph-mon[82331]: pgmap v201: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-02-16T22:29:17.200 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:16 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:17.200 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:16 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:29:17.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:16 smithi114 ceph-mon[94452]: pgmap v201: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-02-16T22:29:17.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:16 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:17.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:16 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:29:19.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:18 smithi114 ceph-mon[94452]: pgmap v202: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 6 op/s 2024-02-16T22:29:19.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:18 smithi123 ceph-mon[82331]: pgmap v202: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 6 op/s 2024-02-16T22:29:21.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.175+0000 7f8325e1f700 1 -- 172.21.15.114:0/1303963676 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83200ffab0 msgr2=0x7f83200ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:21.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.175+0000 7f8325e1f700 1 --2- 172.21.15.114:0/1303963676 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83200ffab0 0x7f83200ffed0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8310009910 tx=0x7f8310009c20 comp rx=0 tx=0).stop 2024-02-16T22:29:21.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.175+0000 7f8325e1f700 1 -- 172.21.15.114:0/1303963676 shutdown_connections 2024-02-16T22:29:21.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.175+0000 7f8325e1f700 1 --2- 172.21.15.114:0/1303963676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8320100cb0 0x7f8320101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:21.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.175+0000 7f8325e1f700 1 --2- 172.21.15.114:0/1303963676 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83200ffab0 0x7f83200ffed0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:21.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.175+0000 7f8325e1f700 1 -- 172.21.15.114:0/1303963676 >> 172.21.15.114:0/1303963676 conn(0x7f83200fb030 msgr2=0x7f83200fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:29:21.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.175+0000 7f8325e1f700 1 -- 172.21.15.114:0/1303963676 shutdown_connections 2024-02-16T22:29:21.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.175+0000 7f8325e1f700 1 -- 172.21.15.114:0/1303963676 wait complete. 2024-02-16T22:29:21.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.175+0000 7f8325e1f700 1 Processor -- start 2024-02-16T22:29:21.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.176+0000 7f8325e1f700 1 -- start start 2024-02-16T22:29:21.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.176+0000 7f8325e1f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83200ffab0 0x7f8320103a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:21.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.176+0000 7f8325e1f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8320100cb0 0x7f8320102090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:21.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.176+0000 7f8325e1f700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83201025d0 con 0x7f8320100cb0 2024-02-16T22:29:21.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.176+0000 7f8325e1f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8320102710 con 0x7f83200ffab0 2024-02-16T22:29:21.181 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:20 smithi114 ceph-mon[94452]: pgmap v203: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 7 op/s 2024-02-16T22:29:21.181 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:20 smithi114 ceph-mon[94452]: reconnect by client.24321 192.168.0.1:0/2666785881 after 0.000999977 2024-02-16T22:29:21.181 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:20 smithi114 ceph-mon[94452]: reconnect by client.14536 192.168.0.1:0/21264991 after 0.000999977 2024-02-16T22:29:21.181 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:20 smithi114 ceph-mon[94452]: mds.? [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] up:reconnect 2024-02-16T22:29:21.181 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:20 smithi114 ceph-mon[94452]: fsmap cephfs:1/1 {0=cephfs.smithi123.gopcfa=up:reconnect} 2 up:standby 2024-02-16T22:29:21.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.176+0000 7f831effd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8320100cb0 0x7f8320102090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:21.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.176+0000 7f831effd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8320100cb0 0x7f8320102090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43052/0 (socket says 172.21.15.114:43052) 2024-02-16T22:29:21.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.176+0000 7f831effd700 1 -- 172.21.15.114:0/3772212455 learned_addr learned my addr 172.21.15.114:0/3772212455 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:29:21.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.176+0000 7f831f7fe700 1 --2- 172.21.15.114:0/3772212455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83200ffab0 0x7f8320103a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:21.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.176+0000 7f831effd700 1 -- 172.21.15.114:0/3772212455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83200ffab0 msgr2=0x7f8320103a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:21.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.176+0000 7f831effd700 1 --2- 172.21.15.114:0/3772212455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83200ffab0 0x7f8320103a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:21.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.176+0000 7f831effd700 1 -- 172.21.15.114:0/3772212455 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83100095f0 con 0x7f8320100cb0 2024-02-16T22:29:21.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.176+0000 7f831f7fe700 1 --2- 172.21.15.114:0/3772212455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83200ffab0 0x7f8320103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:29:21.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.176+0000 7f831effd700 1 --2- 172.21.15.114:0/3772212455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8320100cb0 0x7f8320102090 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f831400c8a0 tx=0x7f831400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:29:21.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.176+0000 7f831cff9700 1 -- 172.21.15.114:0/3772212455 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f831400cea0 con 0x7f8320100cb0 2024-02-16T22:29:21.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.177+0000 7f831cff9700 1 -- 172.21.15.114:0/3772212455 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8314004830 con 0x7f8320100cb0 2024-02-16T22:29:21.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.177+0000 7f8325e1f700 1 -- 172.21.15.114:0/3772212455 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83201029f0 con 0x7f8320100cb0 2024-02-16T22:29:21.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.177+0000 7f831cff9700 1 -- 172.21.15.114:0/3772212455 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83140055b0 con 0x7f8320100cb0 2024-02-16T22:29:21.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.177+0000 7f8325e1f700 1 -- 172.21.15.114:0/3772212455 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832019e380 con 0x7f8320100cb0 2024-02-16T22:29:21.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.178+0000 7f8325e1f700 1 -- 172.21.15.114:0/3772212455 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8320108610 con 0x7f8320100cb0 2024-02-16T22:29:21.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.178+0000 7f831cff9700 1 -- 172.21.15.114:0/3772212455 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f8314007ad0 con 0x7f8320100cb0 2024-02-16T22:29:21.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.179+0000 7f831cff9700 1 --2- 172.21.15.114:0/3772212455 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f830806d8a0 0x7f830806fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:21.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.179+0000 7f831cff9700 1 -- 172.21.15.114:0/3772212455 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f831408d2a0 con 0x7f8320100cb0 2024-02-16T22:29:21.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.179+0000 7f831f7fe700 1 --2- 172.21.15.114:0/3772212455 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f830806d8a0 0x7f830806fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:21.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.179+0000 7f831f7fe700 1 --2- 172.21.15.114:0/3772212455 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f830806d8a0 0x7f830806fd60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8310006010 tx=0x7f8310005920 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:29:21.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.183+0000 7f831cff9700 1 -- 172.21.15.114:0/3772212455 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f831405a530 con 0x7f8320100cb0 2024-02-16T22:29:21.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:20 smithi123 ceph-mon[82331]: pgmap v203: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 7 op/s 2024-02-16T22:29:21.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:20 smithi123 ceph-mon[82331]: reconnect by client.24321 192.168.0.1:0/2666785881 after 0.000999977 2024-02-16T22:29:21.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:20 smithi123 ceph-mon[82331]: reconnect by client.14536 192.168.0.1:0/21264991 after 0.000999977 2024-02-16T22:29:21.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:20 smithi123 ceph-mon[82331]: mds.? [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] up:reconnect 2024-02-16T22:29:21.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:20 smithi123 ceph-mon[82331]: fsmap cephfs:1/1 {0=cephfs.smithi123.gopcfa=up:reconnect} 2 up:standby 2024-02-16T22:29:21.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.395+0000 7f8325e1f700 1 -- 172.21.15.114:0/3772212455 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f832019e930 con 0x7f830806d8a0 2024-02-16T22:29:21.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.397+0000 7f831cff9700 1 -- 172.21.15.114:0/3772212455 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f832019e930 con 0x7f830806d8a0 2024-02-16T22:29:21.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.399+0000 7f8325e1f700 1 -- 172.21.15.114:0/3772212455 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f830806d8a0 msgr2=0x7f830806fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:21.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.399+0000 7f8325e1f700 1 --2- 172.21.15.114:0/3772212455 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f830806d8a0 0x7f830806fd60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8310006010 tx=0x7f8310005920 comp rx=0 tx=0).stop 2024-02-16T22:29:21.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.399+0000 7f8325e1f700 1 -- 172.21.15.114:0/3772212455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8320100cb0 msgr2=0x7f8320102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:21.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.399+0000 7f8325e1f700 1 --2- 172.21.15.114:0/3772212455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8320100cb0 0x7f8320102090 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f831400c8a0 tx=0x7f831400cc60 comp rx=0 tx=0).stop 2024-02-16T22:29:21.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.399+0000 7f8325e1f700 1 -- 172.21.15.114:0/3772212455 shutdown_connections 2024-02-16T22:29:21.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.399+0000 7f8325e1f700 1 --2- 172.21.15.114:0/3772212455 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f830806d8a0 0x7f830806fd60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:21.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.399+0000 7f8325e1f700 1 --2- 172.21.15.114:0/3772212455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83200ffab0 0x7f8320103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:21.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.399+0000 7f8325e1f700 1 --2- 172.21.15.114:0/3772212455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8320100cb0 0x7f8320102090 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:21.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.399+0000 7f8325e1f700 1 -- 172.21.15.114:0/3772212455 >> 172.21.15.114:0/3772212455 conn(0x7f83200fb030 msgr2=0x7f83200683d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:29:21.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.399+0000 7f8325e1f700 1 -- 172.21.15.114:0/3772212455 shutdown_connections 2024-02-16T22:29:21.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.399+0000 7f8325e1f700 1 -- 172.21.15.114:0/3772212455 wait complete. 2024-02-16T22:29:21.411 INFO:teuthology.orchestra.run.smithi114.stdout:true 2024-02-16T22:29:21.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.503+0000 7f242ccc5700 1 -- 172.21.15.114:0/4185413528 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2428103f90 msgr2=0x7f2428106380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:21.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.503+0000 7f242ccc5700 1 --2- 172.21.15.114:0/4185413528 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2428103f90 0x7f2428106380 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2410009910 tx=0x7f2410009c20 comp rx=0 tx=0).stop 2024-02-16T22:29:21.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.503+0000 7f242ccc5700 1 -- 172.21.15.114:0/4185413528 shutdown_connections 2024-02-16T22:29:21.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.503+0000 7f242ccc5700 1 --2- 172.21.15.114:0/4185413528 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2428103f90 0x7f2428106380 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:21.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.503+0000 7f242ccc5700 1 --2- 172.21.15.114:0/4185413528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2428101660 0x7f2428103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:21.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.503+0000 7f242ccc5700 1 -- 172.21.15.114:0/4185413528 >> 172.21.15.114:0/4185413528 conn(0x7f24280fb050 msgr2=0x7f24280fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:29:21.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.503+0000 7f242ccc5700 1 -- 172.21.15.114:0/4185413528 shutdown_connections 2024-02-16T22:29:21.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.503+0000 7f242ccc5700 1 -- 172.21.15.114:0/4185413528 wait complete. 2024-02-16T22:29:21.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.504+0000 7f242ccc5700 1 Processor -- start 2024-02-16T22:29:21.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.504+0000 7f242ccc5700 1 -- start start 2024-02-16T22:29:21.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.504+0000 7f242ccc5700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2428101660 0x7f2428100fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:21.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.504+0000 7f242ccc5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2428103f90 0x7f24280ff610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:21.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.504+0000 7f242ccc5700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24280ffb50 con 0x7f2428101660 2024-02-16T22:29:21.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.504+0000 7f242ccc5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24280ffc90 con 0x7f2428103f90 2024-02-16T22:29:21.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.504+0000 7f242659c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2428101660 0x7f2428100fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:21.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.504+0000 7f242659c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2428101660 0x7f2428100fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47926/0 (socket says 172.21.15.114:47926) 2024-02-16T22:29:21.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.504+0000 7f242659c700 1 -- 172.21.15.114:0/744906082 learned_addr learned my addr 172.21.15.114:0/744906082 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:29:21.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.504+0000 7f241dd9b700 1 --2- 172.21.15.114:0/744906082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2428103f90 0x7f24280ff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:21.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.505+0000 7f242659c700 1 -- 172.21.15.114:0/744906082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2428103f90 msgr2=0x7f24280ff610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:21.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.505+0000 7f242659c700 1 --2- 172.21.15.114:0/744906082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2428103f90 0x7f24280ff610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:21.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.505+0000 7f242659c700 1 -- 172.21.15.114:0/744906082 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24100095f0 con 0x7f2428101660 2024-02-16T22:29:21.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.505+0000 7f242659c700 1 --2- 172.21.15.114:0/744906082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2428101660 0x7f2428100fc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f241800c8a0 tx=0x7f241800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:29:21.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.505+0000 7f241ffff700 1 -- 172.21.15.114:0/744906082 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24180078e0 con 0x7f2428101660 2024-02-16T22:29:21.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.505+0000 7f241ffff700 1 -- 172.21.15.114:0/744906082 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f241800f450 con 0x7f2428101660 2024-02-16T22:29:21.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.505+0000 7f242ccc5700 1 -- 172.21.15.114:0/744906082 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24280fff70 con 0x7f2428101660 2024-02-16T22:29:21.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.505+0000 7f241ffff700 1 -- 172.21.15.114:0/744906082 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f241800e5e0 con 0x7f2428101660 2024-02-16T22:29:21.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.505+0000 7f242ccc5700 1 -- 172.21.15.114:0/744906082 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f242819e3c0 con 0x7f2428101660 2024-02-16T22:29:21.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.506+0000 7f242ccc5700 1 -- 172.21.15.114:0/744906082 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2428190540 con 0x7f2428101660 2024-02-16T22:29:21.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.507+0000 7f241ffff700 1 -- 172.21.15.114:0/744906082 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f241800f5c0 con 0x7f2428101660 2024-02-16T22:29:21.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.507+0000 7f241ffff700 1 --2- 172.21.15.114:0/744906082 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f241406d8a0 0x7f241406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:21.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.507+0000 7f241ffff700 1 -- 172.21.15.114:0/744906082 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f241808d420 con 0x7f2428101660 2024-02-16T22:29:21.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.507+0000 7f241dd9b700 1 --2- 172.21.15.114:0/744906082 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f241406d8a0 0x7f241406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:21.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.507+0000 7f241dd9b700 1 --2- 172.21.15.114:0/744906082 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f241406d8a0 0x7f241406fd60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2410006010 tx=0x7f241000b560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:29:21.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.512+0000 7f241ffff700 1 -- 172.21.15.114:0/744906082 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f241805a730 con 0x7f2428101660 2024-02-16T22:29:21.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.723+0000 7f242ccc5700 1 -- 172.21.15.114:0/744906082 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f24280611d0 con 0x7f241406d8a0 2024-02-16T22:29:21.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.724+0000 7f241ffff700 1 -- 172.21.15.114:0/744906082 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f24280611d0 con 0x7f241406d8a0 2024-02-16T22:29:21.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.727+0000 7f242ccc5700 1 -- 172.21.15.114:0/744906082 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f241406d8a0 msgr2=0x7f241406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:21.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.727+0000 7f242ccc5700 1 --2- 172.21.15.114:0/744906082 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f241406d8a0 0x7f241406fd60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2410006010 tx=0x7f241000b560 comp rx=0 tx=0).stop 2024-02-16T22:29:21.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.727+0000 7f242ccc5700 1 -- 172.21.15.114:0/744906082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2428101660 msgr2=0x7f2428100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:21.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.727+0000 7f242ccc5700 1 --2- 172.21.15.114:0/744906082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2428101660 0x7f2428100fc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f241800c8a0 tx=0x7f241800cbb0 comp rx=0 tx=0).stop 2024-02-16T22:29:21.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.727+0000 7f242ccc5700 1 -- 172.21.15.114:0/744906082 shutdown_connections 2024-02-16T22:29:21.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.727+0000 7f242ccc5700 1 --2- 172.21.15.114:0/744906082 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f241406d8a0 0x7f241406fd60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:21.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.727+0000 7f242ccc5700 1 --2- 172.21.15.114:0/744906082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2428101660 0x7f2428100fc0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:21.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.727+0000 7f242ccc5700 1 --2- 172.21.15.114:0/744906082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2428103f90 0x7f24280ff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:21.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.727+0000 7f242ccc5700 1 -- 172.21.15.114:0/744906082 >> 172.21.15.114:0/744906082 conn(0x7f24280fb050 msgr2=0x7f24280fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:29:21.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.727+0000 7f242ccc5700 1 -- 172.21.15.114:0/744906082 shutdown_connections 2024-02-16T22:29:21.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.727+0000 7f242ccc5700 1 -- 172.21.15.114:0/744906082 wait complete. 2024-02-16T22:29:21.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.830+0000 7f31e7377700 1 -- 172.21.15.114:0/381073735 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31e0101dc0 msgr2=0x7f31e0102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:21.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.830+0000 7f31e7377700 1 --2- 172.21.15.114:0/381073735 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31e0101dc0 0x7f31e0102220 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f31d0009910 tx=0x7f31d0009c20 comp rx=0 tx=0).stop 2024-02-16T22:29:21.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.830+0000 7f31e7377700 1 -- 172.21.15.114:0/381073735 shutdown_connections 2024-02-16T22:29:21.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.830+0000 7f31e7377700 1 --2- 172.21.15.114:0/381073735 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31e0101dc0 0x7f31e0102220 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:21.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.830+0000 7f31e7377700 1 --2- 172.21.15.114:0/381073735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31e0100bc0 0x7f31e0100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:21.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.830+0000 7f31e7377700 1 -- 172.21.15.114:0/381073735 >> 172.21.15.114:0/381073735 conn(0x7f31e00fc140 msgr2=0x7f31e00fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:29:21.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.831+0000 7f31e7377700 1 -- 172.21.15.114:0/381073735 shutdown_connections 2024-02-16T22:29:21.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.831+0000 7f31e7377700 1 -- 172.21.15.114:0/381073735 wait complete. 2024-02-16T22:29:21.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.831+0000 7f31e7377700 1 Processor -- start 2024-02-16T22:29:21.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.831+0000 7f31e7377700 1 -- start start 2024-02-16T22:29:21.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.831+0000 7f31e7377700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31e0100bc0 0x7f31e0074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:21.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.831+0000 7f31e7377700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31e0101dc0 0x7f31e0073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:21.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.831+0000 7f31e7377700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e00736c0 con 0x7f31e0101dc0 2024-02-16T22:29:21.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.831+0000 7f31e7377700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e0073800 con 0x7f31e0100bc0 2024-02-16T22:29:21.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.832+0000 7f31e4912700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31e0101dc0 0x7f31e0073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:21.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.832+0000 7f31e4912700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31e0101dc0 0x7f31e0073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47946/0 (socket says 172.21.15.114:47946) 2024-02-16T22:29:21.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.832+0000 7f31e4912700 1 -- 172.21.15.114:0/1629266833 learned_addr learned my addr 172.21.15.114:0/1629266833 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:29:21.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.832+0000 7f31e5113700 1 --2- 172.21.15.114:0/1629266833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31e0100bc0 0x7f31e0074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:21.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.832+0000 7f31e4912700 1 -- 172.21.15.114:0/1629266833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31e0100bc0 msgr2=0x7f31e0074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:21.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.832+0000 7f31e4912700 1 --2- 172.21.15.114:0/1629266833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31e0100bc0 0x7f31e0074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:21.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.832+0000 7f31e4912700 1 -- 172.21.15.114:0/1629266833 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d00095f0 con 0x7f31e0101dc0 2024-02-16T22:29:21.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.832+0000 7f31e4912700 1 --2- 172.21.15.114:0/1629266833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31e0101dc0 0x7f31e0073180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f31d0005b40 tx=0x7f31d000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:29:21.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.832+0000 7f31d67fc700 1 -- 172.21.15.114:0/1629266833 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31d001d070 con 0x7f31e0101dc0 2024-02-16T22:29:21.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.832+0000 7f31d67fc700 1 -- 172.21.15.114:0/1629266833 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f31d000f460 con 0x7f31e0101dc0 2024-02-16T22:29:21.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.832+0000 7f31e7377700 1 -- 172.21.15.114:0/1629266833 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31e0073a80 con 0x7f31e0101dc0 2024-02-16T22:29:21.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.832+0000 7f31d67fc700 1 -- 172.21.15.114:0/1629266833 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31d0021640 con 0x7f31e0101dc0 2024-02-16T22:29:21.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.832+0000 7f31e7377700 1 -- 172.21.15.114:0/1629266833 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31e0073f70 con 0x7f31e0101dc0 2024-02-16T22:29:21.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.833+0000 7f31e7377700 1 -- 172.21.15.114:0/1629266833 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31e0109720 con 0x7f31e0101dc0 2024-02-16T22:29:21.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.834+0000 7f31d67fc700 1 -- 172.21.15.114:0/1629266833 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f31d000fa90 con 0x7f31e0101dc0 2024-02-16T22:29:21.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.834+0000 7f31d67fc700 1 --2- 172.21.15.114:0/1629266833 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f31cc06d8a0 0x7f31cc06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:21.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.834+0000 7f31d67fc700 1 -- 172.21.15.114:0/1629266833 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f31d008e880 con 0x7f31e0101dc0 2024-02-16T22:29:21.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.834+0000 7f31e5113700 1 --2- 172.21.15.114:0/1629266833 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f31cc06d8a0 0x7f31cc06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:21.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.835+0000 7f31e5113700 1 --2- 172.21.15.114:0/1629266833 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f31cc06d8a0 0x7f31cc06fd60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f31dc005950 tx=0x7f31dc0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:29:21.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:21.839+0000 7f31d67fc700 1 -- 172.21.15.114:0/1629266833 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f31d005bb90 con 0x7f31e0101dc0 2024-02-16T22:29:22.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.042+0000 7f31e7377700 1 -- 172.21.15.114:0/1629266833 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f31e0106710 con 0x7f31cc06d8a0 2024-02-16T22:29:22.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.053+0000 7f31d67fc700 1 -- 172.21.15.114:0/1629266833 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f31e0106710 con 0x7f31cc06d8a0 2024-02-16T22:29:22.055 INFO:teuthology.orchestra.run.smithi114.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:29:22.055 INFO:teuthology.orchestra.run.smithi114.stdout:alertmanager.smithi114 smithi114 *:9093,9094 running (15m) 20s ago 16m 24.3M - 0.25.0 c8568f914cd2 e7404fea400e 2024-02-16T22:29:22.055 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi114 smithi114 running (17m) 20s ago 17m 9017k - 18.2.1 94eadd990e0a 7fd07162cd96 2024-02-16T22:29:22.056 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi123 smithi123 running (15m) 69s ago 15m 8342k - 18.2.1 94eadd990e0a 7687f5e99621 2024-02-16T22:29:22.056 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi114 smithi114 running (4m) 20s ago 16m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 9b8ae9efe83b 2024-02-16T22:29:22.056 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi123 smithi123 running (4m) 69s ago 15m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 6d97263a0ffa 2024-02-16T22:29:22.056 INFO:teuthology.orchestra.run.smithi114.stdout:grafana.smithi114 smithi114 *:3000 running (15m) 20s ago 16m 88.3M - 9.4.7 954c08fa6188 db1c26c3bc20 2024-02-16T22:29:22.056 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.fdxpey smithi114 running (26s) 20s ago 11m 12.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2984ae8a955a 2024-02-16T22:29:22.056 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.zzryke smithi114 running (53s) 20s ago 11m 12.4M - 19.0.0-1450-gf78a58c0 f081f847070a 63fa2af7290e 2024-02-16T22:29:22.056 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.gopcfa smithi123 running (11m) 69s ago 11m 18.6M - 18.2.1 94eadd990e0a 166134b7532a 2024-02-16T22:29:22.056 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.vhlqbe smithi123 running (11m) 69s ago 11m 20.3M - 18.2.1 94eadd990e0a 243c8bb96fa8 2024-02-16T22:29:22.056 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi114.vcullz smithi114 *:8443,9283,8765 running (7m) 20s ago 18m 507M - 19.0.0-1450-gf78a58c0 f081f847070a 14ebeb7888c0 2024-02-16T22:29:22.056 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi123.inzgjd smithi123 *:8443,9283,8765 running (6m) 69s ago 15m 450M - 19.0.0-1450-gf78a58c0 f081f847070a f10aab9faf9e 2024-02-16T22:29:22.056 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi114 smithi114 running (5m) 20s ago 18m 52.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 58ac06372523 2024-02-16T22:29:22.056 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi123 smithi123 running (5m) 69s ago 15m 37.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a f76485e0ea24 2024-02-16T22:29:22.056 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi114 smithi114 *:9100 running (16m) 20s ago 16m 15.7M - 1.5.0 0da6a335fe13 744a090e06d0 2024-02-16T22:29:22.057 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi123 smithi123 *:9100 running (15m) 69s ago 15m 14.9M - 1.5.0 0da6a335fe13 3994b8c29b2e 2024-02-16T22:29:22.057 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 smithi114 running (4m) 20s ago 14m 98.3M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 1978fdad94a5 2024-02-16T22:29:22.057 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 smithi114 running (3m) 20s ago 14m 104M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 3f90e792898f 2024-02-16T22:29:22.057 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 smithi114 running (2m) 20s ago 13m 89.1M 1216M 19.0.0-1450-gf78a58c0 f081f847070a c663dfa0518b 2024-02-16T22:29:22.057 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 smithi123 running (2m) 69s ago 13m 99.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a b921448ba169 2024-02-16T22:29:22.057 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 smithi123 running (106s) 69s ago 13m 97.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 54c2926d56b1 2024-02-16T22:29:22.057 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 smithi123 running (73s) 69s ago 12m 12.6M 2240M 19.0.0-1450-gf78a58c0 f081f847070a dd99bf034749 2024-02-16T22:29:22.057 INFO:teuthology.orchestra.run.smithi114.stdout:prometheus.smithi114 smithi114 *:9095 running (6m) 20s ago 16m 63.8M - 2.43.0 a07b618ecd1d 37aac985cdd7 2024-02-16T22:29:22.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.055+0000 7f31e7377700 1 -- 172.21.15.114:0/1629266833 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f31cc06d8a0 msgr2=0x7f31cc06fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:22.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.055+0000 7f31e7377700 1 --2- 172.21.15.114:0/1629266833 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f31cc06d8a0 0x7f31cc06fd60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f31dc005950 tx=0x7f31dc0058e0 comp rx=0 tx=0).stop 2024-02-16T22:29:22.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.055+0000 7f31e7377700 1 -- 172.21.15.114:0/1629266833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31e0101dc0 msgr2=0x7f31e0073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:22.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.055+0000 7f31e7377700 1 --2- 172.21.15.114:0/1629266833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31e0101dc0 0x7f31e0073180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f31d0005b40 tx=0x7f31d000be20 comp rx=0 tx=0).stop 2024-02-16T22:29:22.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.055+0000 7f31e7377700 1 -- 172.21.15.114:0/1629266833 shutdown_connections 2024-02-16T22:29:22.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.055+0000 7f31e7377700 1 --2- 172.21.15.114:0/1629266833 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f31cc06d8a0 0x7f31cc06fd60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:22.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.055+0000 7f31e7377700 1 --2- 172.21.15.114:0/1629266833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31e0100bc0 0x7f31e0074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:22.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.055+0000 7f31e7377700 1 --2- 172.21.15.114:0/1629266833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31e0101dc0 0x7f31e0073180 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:22.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.055+0000 7f31e7377700 1 -- 172.21.15.114:0/1629266833 >> 172.21.15.114:0/1629266833 conn(0x7f31e00fc140 msgr2=0x7f31e0104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:29:22.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.056+0000 7f31e7377700 1 -- 172.21.15.114:0/1629266833 shutdown_connections 2024-02-16T22:29:22.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.056+0000 7f31e7377700 1 -- 172.21.15.114:0/1629266833 wait complete. 2024-02-16T22:29:22.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.157+0000 7f084ff25700 1 -- 172.21.15.114:0/1253766023 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0848100bc0 msgr2=0x7f0848100fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:22.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.157+0000 7f084ff25700 1 --2- 172.21.15.114:0/1253766023 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0848100bc0 0x7f0848100fe0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0840009910 tx=0x7f0840009c20 comp rx=0 tx=0).stop 2024-02-16T22:29:22.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.157+0000 7f084ff25700 1 -- 172.21.15.114:0/1253766023 shutdown_connections 2024-02-16T22:29:22.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.157+0000 7f084ff25700 1 --2- 172.21.15.114:0/1253766023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0848101dc0 0x7f0848102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:22.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.157+0000 7f084ff25700 1 --2- 172.21.15.114:0/1253766023 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0848100bc0 0x7f0848100fe0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:22.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.157+0000 7f084ff25700 1 -- 172.21.15.114:0/1253766023 >> 172.21.15.114:0/1253766023 conn(0x7f08480fc140 msgr2=0x7f08480fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:29:22.160 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:21 smithi114 ceph-mon[94452]: mds.? [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] up:rejoin 2024-02-16T22:29:22.160 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:21 smithi114 ceph-mon[94452]: fsmap cephfs:1/1 {0=cephfs.smithi123.gopcfa=up:rejoin} 2 up:standby 2024-02-16T22:29:22.160 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:21 smithi114 ceph-mon[94452]: daemon mds.cephfs.smithi123.gopcfa is now active in filesystem cephfs as rank 0 2024-02-16T22:29:22.160 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:21 smithi114 ceph-mon[94452]: from='client.34440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:29:22.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.157+0000 7f084ff25700 1 -- 172.21.15.114:0/1253766023 shutdown_connections 2024-02-16T22:29:22.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.157+0000 7f084ff25700 1 -- 172.21.15.114:0/1253766023 wait complete. 2024-02-16T22:29:22.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.158+0000 7f084ff25700 1 Processor -- start 2024-02-16T22:29:22.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.158+0000 7f084ff25700 1 -- start start 2024-02-16T22:29:22.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.158+0000 7f084ff25700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0848100bc0 0x7f0848074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:22.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.158+0000 7f084ff25700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0848101dc0 0x7f0848073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:22.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.158+0000 7f084ff25700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08480736c0 con 0x7f0848101dc0 2024-02-16T22:29:22.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.158+0000 7f084ff25700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0848073800 con 0x7f0848100bc0 2024-02-16T22:29:22.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.158+0000 7f084d4c0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0848101dc0 0x7f0848073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:22.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.158+0000 7f084d4c0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0848101dc0 0x7f0848073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47962/0 (socket says 172.21.15.114:47962) 2024-02-16T22:29:22.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.158+0000 7f084d4c0700 1 -- 172.21.15.114:0/499222326 learned_addr learned my addr 172.21.15.114:0/499222326 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:29:22.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.158+0000 7f084dcc1700 1 --2- 172.21.15.114:0/499222326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0848100bc0 0x7f0848074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:22.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.158+0000 7f084d4c0700 1 -- 172.21.15.114:0/499222326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0848100bc0 msgr2=0x7f0848074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:22.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.158+0000 7f084d4c0700 1 --2- 172.21.15.114:0/499222326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0848100bc0 0x7f0848074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:22.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.158+0000 7f084d4c0700 1 -- 172.21.15.114:0/499222326 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08400095f0 con 0x7f0848101dc0 2024-02-16T22:29:22.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.158+0000 7f084d4c0700 1 --2- 172.21.15.114:0/499222326 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0848101dc0 0x7f0848073180 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f083c00d8d0 tx=0x7f083c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:29:22.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.159+0000 7f0846ffd700 1 -- 172.21.15.114:0/499222326 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f083c0098a0 con 0x7f0848101dc0 2024-02-16T22:29:22.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.159+0000 7f0846ffd700 1 -- 172.21.15.114:0/499222326 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f083c010460 con 0x7f0848101dc0 2024-02-16T22:29:22.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.159+0000 7f084ff25700 1 -- 172.21.15.114:0/499222326 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0848073ae0 con 0x7f0848101dc0 2024-02-16T22:29:22.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.159+0000 7f0846ffd700 1 -- 172.21.15.114:0/499222326 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f083c00f5f0 con 0x7f0848101dc0 2024-02-16T22:29:22.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.159+0000 7f084ff25700 1 -- 172.21.15.114:0/499222326 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f084819e3f0 con 0x7f0848101dc0 2024-02-16T22:29:22.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.160+0000 7f084ff25700 1 -- 172.21.15.114:0/499222326 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0848109720 con 0x7f0848101dc0 2024-02-16T22:29:22.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.160+0000 7f0846ffd700 1 -- 172.21.15.114:0/499222326 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f083c009a00 con 0x7f0848101dc0 2024-02-16T22:29:22.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.161+0000 7f0846ffd700 1 --2- 172.21.15.114:0/499222326 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f083006d8a0 0x7f083006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:22.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.161+0000 7f0846ffd700 1 -- 172.21.15.114:0/499222326 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f083c08d390 con 0x7f0848101dc0 2024-02-16T22:29:22.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.161+0000 7f084dcc1700 1 --2- 172.21.15.114:0/499222326 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f083006d8a0 0x7f083006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:22.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.161+0000 7f084dcc1700 1 --2- 172.21.15.114:0/499222326 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f083006d8a0 0x7f083006fd60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0840006010 tx=0x7f084000b560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:29:22.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.165+0000 7f0846ffd700 1 -- 172.21.15.114:0/499222326 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f083c05a6a0 con 0x7f0848101dc0 2024-02-16T22:29:22.277 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:21 smithi123 ceph-mon[82331]: mds.? [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] up:rejoin 2024-02-16T22:29:22.278 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:21 smithi123 ceph-mon[82331]: fsmap cephfs:1/1 {0=cephfs.smithi123.gopcfa=up:rejoin} 2 up:standby 2024-02-16T22:29:22.278 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:21 smithi123 ceph-mon[82331]: daemon mds.cephfs.smithi123.gopcfa is now active in filesystem cephfs as rank 0 2024-02-16T22:29:22.278 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:21 smithi123 ceph-mon[82331]: from='client.34440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:29:22.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.442+0000 7f084ff25700 1 -- 172.21.15.114:0/499222326 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f08481098e0 con 0x7f0848101dc0 2024-02-16T22:29:22.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.442+0000 7f0846ffd700 1 -- 172.21.15.114:0/499222326 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f083c05a230 con 0x7f0848101dc0 2024-02-16T22:29:22.443 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:29:22.443 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": { 2024-02-16T22:29:22.444 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:29:22.444 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:29:22.444 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": { 2024-02-16T22:29:22.444 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:29:22.444 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:29:22.444 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": { 2024-02-16T22:29:22.444 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T22:29:22.444 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:29:22.444 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-02-16T22:29:22.444 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-02-16T22:29:22.444 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:29:22.444 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:29:22.444 INFO:teuthology.orchestra.run.smithi114.stdout: "overall": { 2024-02-16T22:29:22.445 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-02-16T22:29:22.445 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 12 2024-02-16T22:29:22.445 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-02-16T22:29:22.445 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:29:22.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.444+0000 7f084ff25700 1 -- 172.21.15.114:0/499222326 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f083006d8a0 msgr2=0x7f083006fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:22.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.444+0000 7f084ff25700 1 --2- 172.21.15.114:0/499222326 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f083006d8a0 0x7f083006fd60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0840006010 tx=0x7f084000b560 comp rx=0 tx=0).stop 2024-02-16T22:29:22.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.444+0000 7f084ff25700 1 -- 172.21.15.114:0/499222326 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0848101dc0 msgr2=0x7f0848073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:22.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.444+0000 7f084ff25700 1 --2- 172.21.15.114:0/499222326 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0848101dc0 0x7f0848073180 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f083c00d8d0 tx=0x7f083c00dbe0 comp rx=0 tx=0).stop 2024-02-16T22:29:22.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.444+0000 7f084ff25700 1 -- 172.21.15.114:0/499222326 shutdown_connections 2024-02-16T22:29:22.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.444+0000 7f084ff25700 1 --2- 172.21.15.114:0/499222326 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f083006d8a0 0x7f083006fd60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:22.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.444+0000 7f084ff25700 1 --2- 172.21.15.114:0/499222326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0848100bc0 0x7f0848074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:22.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.444+0000 7f084ff25700 1 --2- 172.21.15.114:0/499222326 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0848101dc0 0x7f0848073180 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:22.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.444+0000 7f084ff25700 1 -- 172.21.15.114:0/499222326 >> 172.21.15.114:0/499222326 conn(0x7f08480fc140 msgr2=0x7f0848104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:29:22.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.445+0000 7f084ff25700 1 -- 172.21.15.114:0/499222326 shutdown_connections 2024-02-16T22:29:22.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.445+0000 7f084ff25700 1 -- 172.21.15.114:0/499222326 wait complete. 2024-02-16T22:29:22.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.547+0000 7f960caf0700 1 -- 172.21.15.114:0/2636496158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9608101e00 msgr2=0x7f9608102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:22.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.547+0000 7f960caf0700 1 --2- 172.21.15.114:0/2636496158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9608101e00 0x7f9608102260 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f95fc009910 tx=0x7f95fc009c20 comp rx=0 tx=0).stop 2024-02-16T22:29:22.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.547+0000 7f960caf0700 1 -- 172.21.15.114:0/2636496158 shutdown_connections 2024-02-16T22:29:22.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.547+0000 7f960caf0700 1 --2- 172.21.15.114:0/2636496158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9608101e00 0x7f9608102260 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:22.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.547+0000 7f960caf0700 1 --2- 172.21.15.114:0/2636496158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9608100c00 0x7f9608101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:22.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.547+0000 7f960caf0700 1 -- 172.21.15.114:0/2636496158 >> 172.21.15.114:0/2636496158 conn(0x7f96080fc180 msgr2=0x7f96080fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:29:22.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.547+0000 7f960caf0700 1 -- 172.21.15.114:0/2636496158 shutdown_connections 2024-02-16T22:29:22.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.547+0000 7f960caf0700 1 -- 172.21.15.114:0/2636496158 wait complete. 2024-02-16T22:29:22.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.548+0000 7f960caf0700 1 Processor -- start 2024-02-16T22:29:22.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.548+0000 7f960caf0700 1 -- start start 2024-02-16T22:29:22.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.548+0000 7f960caf0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9608100c00 0x7f9608074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:22.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.548+0000 7f960caf0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9608101e00 0x7f9608073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:22.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.548+0000 7f960caf0700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96080736c0 con 0x7f9608100c00 2024-02-16T22:29:22.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.548+0000 7f960caf0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9608073800 con 0x7f9608101e00 2024-02-16T22:29:22.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.548+0000 7f960659c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9608100c00 0x7f9608074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:22.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.548+0000 7f960659c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9608100c00 0x7f9608074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47972/0 (socket says 172.21.15.114:47972) 2024-02-16T22:29:22.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.548+0000 7f960659c700 1 -- 172.21.15.114:0/3002021730 learned_addr learned my addr 172.21.15.114:0/3002021730 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:29:22.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.549+0000 7f9605d9b700 1 --2- 172.21.15.114:0/3002021730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9608101e00 0x7f9608073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:22.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.549+0000 7f960659c700 1 -- 172.21.15.114:0/3002021730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9608101e00 msgr2=0x7f9608073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:22.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.549+0000 7f960659c700 1 --2- 172.21.15.114:0/3002021730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9608101e00 0x7f9608073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:22.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.549+0000 7f960659c700 1 -- 172.21.15.114:0/3002021730 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95fc0095f0 con 0x7f9608100c00 2024-02-16T22:29:22.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.549+0000 7f960659c700 1 --2- 172.21.15.114:0/3002021730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9608100c00 0x7f9608074b30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f95f800b480 tx=0x7f95f800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:29:22.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.549+0000 7f95f77fe700 1 -- 172.21.15.114:0/3002021730 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95f80118c0 con 0x7f9608100c00 2024-02-16T22:29:22.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.549+0000 7f960caf0700 1 -- 172.21.15.114:0/3002021730 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9608073ae0 con 0x7f9608100c00 2024-02-16T22:29:22.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.549+0000 7f95f77fe700 1 -- 172.21.15.114:0/3002021730 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f95f8010380 con 0x7f9608100c00 2024-02-16T22:29:22.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.549+0000 7f95f77fe700 1 -- 172.21.15.114:0/3002021730 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95f800f4f0 con 0x7f9608100c00 2024-02-16T22:29:22.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.549+0000 7f960caf0700 1 -- 172.21.15.114:0/3002021730 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f960819e500 con 0x7f9608100c00 2024-02-16T22:29:22.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.550+0000 7f960caf0700 1 -- 172.21.15.114:0/3002021730 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9608109800 con 0x7f9608100c00 2024-02-16T22:29:22.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.551+0000 7f95f77fe700 1 -- 172.21.15.114:0/3002021730 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f95f80104f0 con 0x7f9608100c00 2024-02-16T22:29:22.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.551+0000 7f95f77fe700 1 --2- 172.21.15.114:0/3002021730 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f95f006d8a0 0x7f95f006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:22.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.551+0000 7f95f77fe700 1 -- 172.21.15.114:0/3002021730 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f95f808d1f0 con 0x7f9608100c00 2024-02-16T22:29:22.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.551+0000 7f9605d9b700 1 --2- 172.21.15.114:0/3002021730 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f95f006d8a0 0x7f95f006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:22.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.551+0000 7f9605d9b700 1 --2- 172.21.15.114:0/3002021730 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f95f006d8a0 0x7f95f006fd60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00b5c0 tx=0x7f95fc0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:29:22.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.556+0000 7f95f77fe700 1 -- 172.21.15.114:0/3002021730 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f95f805a480 con 0x7f9608100c00 2024-02-16T22:29:22.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.790+0000 7f960caf0700 1 -- 172.21.15.114:0/3002021730 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f96081099f0 con 0x7f9608100c00 2024-02-16T22:29:22.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.790+0000 7f95f77fe700 1 -- 172.21.15.114:0/3002021730 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1677 (secure 0 0 0) 0x7f95f805a010 con 0x7f9608100c00 2024-02-16T22:29:22.791 INFO:teuthology.orchestra.run.smithi114.stdout:e31 2024-02-16T22:29:22.791 INFO:teuthology.orchestra.run.smithi114.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:29:22.791 INFO:teuthology.orchestra.run.smithi114.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:29:22.792 INFO:teuthology.orchestra.run.smithi114.stdout:legacy client fscid: 1 2024-02-16T22:29:22.792 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:29:22.792 INFO:teuthology.orchestra.run.smithi114.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:29:22.792 INFO:teuthology.orchestra.run.smithi114.stdout:fs_name cephfs 2024-02-16T22:29:22.792 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 31 2024-02-16T22:29:22.792 INFO:teuthology.orchestra.run.smithi114.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:29:22.792 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-02-16T22:18:02.108825+0000 2024-02-16T22:29:22.792 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-02-16T22:29:20.909186+0000 2024-02-16T22:29:22.792 INFO:teuthology.orchestra.run.smithi114.stdout:tableserver 0 2024-02-16T22:29:22.792 INFO:teuthology.orchestra.run.smithi114.stdout:root 0 2024-02-16T22:29:22.793 INFO:teuthology.orchestra.run.smithi114.stdout:session_timeout 60 2024-02-16T22:29:22.793 INFO:teuthology.orchestra.run.smithi114.stdout:session_autoclose 300 2024-02-16T22:29:22.793 INFO:teuthology.orchestra.run.smithi114.stdout:max_file_size 1099511627776 2024-02-16T22:29:22.793 INFO:teuthology.orchestra.run.smithi114.stdout:max_xattr_size 65536 2024-02-16T22:29:22.793 INFO:teuthology.orchestra.run.smithi114.stdout:required_client_features {} 2024-02-16T22:29:22.793 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure 0 2024-02-16T22:29:22.793 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure_osd_epoch 81 2024-02-16T22:29:22.793 INFO:teuthology.orchestra.run.smithi114.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:29:22.793 INFO:teuthology.orchestra.run.smithi114.stdout:max_mds 1 2024-02-16T22:29:22.793 INFO:teuthology.orchestra.run.smithi114.stdout:in 0 2024-02-16T22:29:22.793 INFO:teuthology.orchestra.run.smithi114.stdout:up {0=24285} 2024-02-16T22:29:22.794 INFO:teuthology.orchestra.run.smithi114.stdout:failed 2024-02-16T22:29:22.794 INFO:teuthology.orchestra.run.smithi114.stdout:damaged 2024-02-16T22:29:22.794 INFO:teuthology.orchestra.run.smithi114.stdout:stopped 2024-02-16T22:29:22.794 INFO:teuthology.orchestra.run.smithi114.stdout:data_pools [3] 2024-02-16T22:29:22.794 INFO:teuthology.orchestra.run.smithi114.stdout:metadata_pool 2 2024-02-16T22:29:22.794 INFO:teuthology.orchestra.run.smithi114.stdout:inline_data enabled 2024-02-16T22:29:22.794 INFO:teuthology.orchestra.run.smithi114.stdout:balancer 2024-02-16T22:29:22.794 INFO:teuthology.orchestra.run.smithi114.stdout:bal_rank_mask -1 2024-02-16T22:29:22.794 INFO:teuthology.orchestra.run.smithi114.stdout:standby_count_wanted 1 2024-02-16T22:29:22.794 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.gopcfa{0:24285} state up:active seq 170 join_fscid=1 addr [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T22:29:22.794 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:29:22.794 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:29:22.795 INFO:teuthology.orchestra.run.smithi114.stdout:Standby daemons: 2024-02-16T22:29:22.795 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:29:22.795 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.zzryke{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.114:6826/2017800151,v1:172.21.15.114:6827/2017800151] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T22:29:22.795 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.fdxpey{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.114:6828/3702914193,v1:172.21.15.114:6829/3702914193] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T22:29:22.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.792+0000 7f960caf0700 1 -- 172.21.15.114:0/3002021730 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f95f006d8a0 msgr2=0x7f95f006fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:22.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.792+0000 7f960caf0700 1 --2- 172.21.15.114:0/3002021730 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f95f006d8a0 0x7f95f006fd60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00b5c0 tx=0x7f95fc0058e0 comp rx=0 tx=0).stop 2024-02-16T22:29:22.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.792+0000 7f960caf0700 1 -- 172.21.15.114:0/3002021730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9608100c00 msgr2=0x7f9608074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:22.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.792+0000 7f960caf0700 1 --2- 172.21.15.114:0/3002021730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9608100c00 0x7f9608074b30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f95f800b480 tx=0x7f95f800b840 comp rx=0 tx=0).stop 2024-02-16T22:29:22.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.792+0000 7f960caf0700 1 -- 172.21.15.114:0/3002021730 shutdown_connections 2024-02-16T22:29:22.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.792+0000 7f960caf0700 1 --2- 172.21.15.114:0/3002021730 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f95f006d8a0 0x7f95f006fd60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:22.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.792+0000 7f960caf0700 1 --2- 172.21.15.114:0/3002021730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9608100c00 0x7f9608074b30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:22.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.792+0000 7f960caf0700 1 --2- 172.21.15.114:0/3002021730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9608101e00 0x7f9608073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:22.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.792+0000 7f960caf0700 1 -- 172.21.15.114:0/3002021730 >> 172.21.15.114:0/3002021730 conn(0x7f96080fc180 msgr2=0x7f9608105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:29:22.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.792+0000 7f960caf0700 1 -- 172.21.15.114:0/3002021730 shutdown_connections 2024-02-16T22:29:22.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.792+0000 7f960caf0700 1 -- 172.21.15.114:0/3002021730 wait complete. 2024-02-16T22:29:22.797 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 31 2024-02-16T22:29:22.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.898+0000 7f199949c700 1 -- 172.21.15.114:0/2497294177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19940ffad0 msgr2=0x7f19940ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:22.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.898+0000 7f199949c700 1 --2- 172.21.15.114:0/2497294177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19940ffad0 0x7f19940ffef0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1984009910 tx=0x7f1984009c20 comp rx=0 tx=0).stop 2024-02-16T22:29:22.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.898+0000 7f199949c700 1 -- 172.21.15.114:0/2497294177 shutdown_connections 2024-02-16T22:29:22.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.898+0000 7f199949c700 1 --2- 172.21.15.114:0/2497294177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1994100cd0 0x7f1994101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:22.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.898+0000 7f199949c700 1 --2- 172.21.15.114:0/2497294177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19940ffad0 0x7f19940ffef0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:22.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.898+0000 7f199949c700 1 -- 172.21.15.114:0/2497294177 >> 172.21.15.114:0/2497294177 conn(0x7f19940fb050 msgr2=0x7f19940fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:29:22.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.898+0000 7f199949c700 1 -- 172.21.15.114:0/2497294177 shutdown_connections 2024-02-16T22:29:22.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.898+0000 7f199949c700 1 -- 172.21.15.114:0/2497294177 wait complete. 2024-02-16T22:29:22.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.899+0000 7f199949c700 1 Processor -- start 2024-02-16T22:29:22.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.899+0000 7f199949c700 1 -- start start 2024-02-16T22:29:22.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.899+0000 7f199949c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19940ffad0 0x7f1994103a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:22.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.899+0000 7f199949c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1994100cd0 0x7f19941020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:22.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.899+0000 7f199949c700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19941025f0 con 0x7f1994100cd0 2024-02-16T22:29:22.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.899+0000 7f199949c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1994102730 con 0x7f19940ffad0 2024-02-16T22:29:22.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.899+0000 7f19927fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1994100cd0 0x7f19941020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:22.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.899+0000 7f19927fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1994100cd0 0x7f19941020b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47986/0 (socket says 172.21.15.114:47986) 2024-02-16T22:29:22.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.899+0000 7f19927fc700 1 -- 172.21.15.114:0/3590189188 learned_addr learned my addr 172.21.15.114:0/3590189188 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:29:22.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.899+0000 7f1992ffd700 1 --2- 172.21.15.114:0/3590189188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19940ffad0 0x7f1994103a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:22.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.899+0000 7f19927fc700 1 -- 172.21.15.114:0/3590189188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19940ffad0 msgr2=0x7f1994103a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:22.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.899+0000 7f19927fc700 1 --2- 172.21.15.114:0/3590189188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19940ffad0 0x7f1994103a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:22.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.899+0000 7f19927fc700 1 -- 172.21.15.114:0/3590189188 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19840095f0 con 0x7f1994100cd0 2024-02-16T22:29:22.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.899+0000 7f1992ffd700 1 --2- 172.21.15.114:0/3590189188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19940ffad0 0x7f1994103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:29:22.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.900+0000 7f19927fc700 1 --2- 172.21.15.114:0/3590189188 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1994100cd0 0x7f19941020b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f198800c8a0 tx=0x7f198800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:29:22.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.900+0000 7f197bfff700 1 -- 172.21.15.114:0/3590189188 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f198800cea0 con 0x7f1994100cd0 2024-02-16T22:29:22.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.900+0000 7f197bfff700 1 -- 172.21.15.114:0/3590189188 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1988004830 con 0x7f1994100cd0 2024-02-16T22:29:22.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.900+0000 7f199949c700 1 -- 172.21.15.114:0/3590189188 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1994102a10 con 0x7f1994100cd0 2024-02-16T22:29:22.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.900+0000 7f197bfff700 1 -- 172.21.15.114:0/3590189188 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19880055b0 con 0x7f1994100cd0 2024-02-16T22:29:22.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.900+0000 7f199949c700 1 -- 172.21.15.114:0/3590189188 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f199419e430 con 0x7f1994100cd0 2024-02-16T22:29:22.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.901+0000 7f199949c700 1 -- 172.21.15.114:0/3590189188 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1994108630 con 0x7f1994100cd0 2024-02-16T22:29:22.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.901+0000 7f197bfff700 1 -- 172.21.15.114:0/3590189188 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f1988007ad0 con 0x7f1994100cd0 2024-02-16T22:29:22.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.902+0000 7f197bfff700 1 --2- 172.21.15.114:0/3590189188 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f197c06d8a0 0x7f197c06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:22.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.902+0000 7f197bfff700 1 -- 172.21.15.114:0/3590189188 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f198808d2a0 con 0x7f1994100cd0 2024-02-16T22:29:22.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.902+0000 7f1992ffd700 1 --2- 172.21.15.114:0/3590189188 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f197c06d8a0 0x7f197c06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:22.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.902+0000 7f1992ffd700 1 --2- 172.21.15.114:0/3590189188 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f197c06d8a0 0x7f197c06fd60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1984006010 tx=0x7f1984005920 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:29:22.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:22.906+0000 7f197bfff700 1 -- 172.21.15.114:0/3590189188 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f198805a530 con 0x7f1994100cd0 2024-02-16T22:29:23.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:22 smithi123 ceph-mon[82331]: from='client.34444 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:29:23.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:22 smithi123 ceph-mon[82331]: pgmap v204: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 85 B/s wr, 6 op/s 2024-02-16T22:29:23.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:22 smithi123 ceph-mon[82331]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-16T22:29:23.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:22 smithi123 ceph-mon[82331]: mds.? [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] up:active 2024-02-16T22:29:23.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:22 smithi123 ceph-mon[82331]: fsmap cephfs:1 {0=cephfs.smithi123.gopcfa=up:active} 2 up:standby 2024-02-16T22:29:23.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:22 smithi123 ceph-mon[82331]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:29:23.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:22 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/499222326' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:23.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:22 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/3002021730' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:29:23.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.120+0000 7f199949c700 1 -- 172.21.15.114:0/3590189188 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f199419e9e0 con 0x7f197c06d8a0 2024-02-16T22:29:23.122 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:22 smithi114 ceph-mon[94452]: from='client.34444 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:29:23.122 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:22 smithi114 ceph-mon[94452]: pgmap v204: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 85 B/s wr, 6 op/s 2024-02-16T22:29:23.122 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:22 smithi114 ceph-mon[94452]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-16T22:29:23.122 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:22 smithi114 ceph-mon[94452]: mds.? [v2:172.21.15.123:6826/2812517585,v1:172.21.15.123:6827/2812517585] up:active 2024-02-16T22:29:23.122 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:22 smithi114 ceph-mon[94452]: fsmap cephfs:1 {0=cephfs.smithi123.gopcfa=up:active} 2 up:standby 2024-02-16T22:29:23.123 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:22 smithi114 ceph-mon[94452]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:29:23.123 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:22 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/499222326' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:23.123 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:22 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/3002021730' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:29:23.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.122+0000 7f197bfff700 1 -- 172.21.15.114:0/3590189188 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f199419e9e0 con 0x7f197c06d8a0 2024-02-16T22:29:23.123 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:29:23.124 INFO:teuthology.orchestra.run.smithi114.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:29:23.124 INFO:teuthology.orchestra.run.smithi114.stdout: "in_progress": true, 2024-02-16T22:29:23.124 INFO:teuthology.orchestra.run.smithi114.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:29:23.124 INFO:teuthology.orchestra.run.smithi114.stdout: "services_complete": [ 2024-02-16T22:29:23.124 INFO:teuthology.orchestra.run.smithi114.stdout: "osd", 2024-02-16T22:29:23.124 INFO:teuthology.orchestra.run.smithi114.stdout: "crash", 2024-02-16T22:29:23.124 INFO:teuthology.orchestra.run.smithi114.stdout: "mon", 2024-02-16T22:29:23.124 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr" 2024-02-16T22:29:23.124 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-02-16T22:29:23.124 INFO:teuthology.orchestra.run.smithi114.stdout: "progress": "14/23 daemons upgraded", 2024-02-16T22:29:23.124 INFO:teuthology.orchestra.run.smithi114.stdout: "message": "Currently upgrading mds daemons", 2024-02-16T22:29:23.125 INFO:teuthology.orchestra.run.smithi114.stdout: "is_paused": false 2024-02-16T22:29:23.125 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:29:23.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.124+0000 7f199949c700 1 -- 172.21.15.114:0/3590189188 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f197c06d8a0 msgr2=0x7f197c06fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:23.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.124+0000 7f199949c700 1 --2- 172.21.15.114:0/3590189188 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f197c06d8a0 0x7f197c06fd60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1984006010 tx=0x7f1984005920 comp rx=0 tx=0).stop 2024-02-16T22:29:23.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.124+0000 7f199949c700 1 -- 172.21.15.114:0/3590189188 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1994100cd0 msgr2=0x7f19941020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:23.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.124+0000 7f199949c700 1 --2- 172.21.15.114:0/3590189188 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1994100cd0 0x7f19941020b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f198800c8a0 tx=0x7f198800cc60 comp rx=0 tx=0).stop 2024-02-16T22:29:23.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.124+0000 7f199949c700 1 -- 172.21.15.114:0/3590189188 shutdown_connections 2024-02-16T22:29:23.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.124+0000 7f199949c700 1 --2- 172.21.15.114:0/3590189188 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f197c06d8a0 0x7f197c06fd60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:23.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.124+0000 7f199949c700 1 --2- 172.21.15.114:0/3590189188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19940ffad0 0x7f1994103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:23.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.124+0000 7f199949c700 1 --2- 172.21.15.114:0/3590189188 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1994100cd0 0x7f19941020b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:23.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.124+0000 7f199949c700 1 -- 172.21.15.114:0/3590189188 >> 172.21.15.114:0/3590189188 conn(0x7f19940fb050 msgr2=0x7f1994068dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:29:23.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.125+0000 7f199949c700 1 -- 172.21.15.114:0/3590189188 shutdown_connections 2024-02-16T22:29:23.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.125+0000 7f199949c700 1 -- 172.21.15.114:0/3590189188 wait complete. 2024-02-16T22:29:23.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.228+0000 7f54872f9700 1 -- 172.21.15.114:0/2675946601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f54800ffad0 msgr2=0x7f54800ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:23.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.228+0000 7f54872f9700 1 --2- 172.21.15.114:0/2675946601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f54800ffad0 0x7f54800ffef0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f547c009910 tx=0x7f547c009c20 comp rx=0 tx=0).stop 2024-02-16T22:29:23.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.228+0000 7f54872f9700 1 -- 172.21.15.114:0/2675946601 shutdown_connections 2024-02-16T22:29:23.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.228+0000 7f54872f9700 1 --2- 172.21.15.114:0/2675946601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5480100cd0 0x7f5480101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:23.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.228+0000 7f54872f9700 1 --2- 172.21.15.114:0/2675946601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f54800ffad0 0x7f54800ffef0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:23.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.228+0000 7f54872f9700 1 -- 172.21.15.114:0/2675946601 >> 172.21.15.114:0/2675946601 conn(0x7f54800fb050 msgr2=0x7f54800fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:29:23.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.228+0000 7f54872f9700 1 -- 172.21.15.114:0/2675946601 shutdown_connections 2024-02-16T22:29:23.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.228+0000 7f54872f9700 1 -- 172.21.15.114:0/2675946601 wait complete. 2024-02-16T22:29:23.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.229+0000 7f54872f9700 1 Processor -- start 2024-02-16T22:29:23.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.229+0000 7f54872f9700 1 -- start start 2024-02-16T22:29:23.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.229+0000 7f54872f9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54800ffad0 0x7f54801952b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:23.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.229+0000 7f54872f9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5480100cd0 0x7f54801957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:23.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.229+0000 7f54872f9700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5480195e10 con 0x7f5480100cd0 2024-02-16T22:29:23.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.229+0000 7f54872f9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5480195f50 con 0x7f54800ffad0 2024-02-16T22:29:23.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.229+0000 7f5484894700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5480100cd0 0x7f54801957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:23.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.229+0000 7f5484894700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5480100cd0 0x7f54801957f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47992/0 (socket says 172.21.15.114:47992) 2024-02-16T22:29:23.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.229+0000 7f5484894700 1 -- 172.21.15.114:0/2248060012 learned_addr learned my addr 172.21.15.114:0/2248060012 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:29:23.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.229+0000 7f5485095700 1 --2- 172.21.15.114:0/2248060012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54800ffad0 0x7f54801952b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:23.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.229+0000 7f5484894700 1 -- 172.21.15.114:0/2248060012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54800ffad0 msgr2=0x7f54801952b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:23.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.229+0000 7f5484894700 1 --2- 172.21.15.114:0/2248060012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54800ffad0 0x7f54801952b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:23.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.229+0000 7f5484894700 1 -- 172.21.15.114:0/2248060012 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547c0095f0 con 0x7f5480100cd0 2024-02-16T22:29:23.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.229+0000 7f5485095700 1 --2- 172.21.15.114:0/2248060012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54800ffad0 0x7f54801952b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:29:23.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.230+0000 7f5484894700 1 --2- 172.21.15.114:0/2248060012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5480100cd0 0x7f54801957f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f547000c8a0 tx=0x7f547000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:29:23.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.230+0000 7f54767fc700 1 -- 172.21.15.114:0/2248060012 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f547000cea0 con 0x7f5480100cd0 2024-02-16T22:29:23.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.230+0000 7f54767fc700 1 -- 172.21.15.114:0/2248060012 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5470004830 con 0x7f5480100cd0 2024-02-16T22:29:23.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.230+0000 7f54872f9700 1 -- 172.21.15.114:0/2248060012 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f548019aa00 con 0x7f5480100cd0 2024-02-16T22:29:23.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.230+0000 7f54767fc700 1 -- 172.21.15.114:0/2248060012 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54700055b0 con 0x7f5480100cd0 2024-02-16T22:29:23.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.230+0000 7f54872f9700 1 -- 172.21.15.114:0/2248060012 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548019af50 con 0x7f5480100cd0 2024-02-16T22:29:23.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.231+0000 7f54872f9700 1 -- 172.21.15.114:0/2248060012 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5480108630 con 0x7f5480100cd0 2024-02-16T22:29:23.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.231+0000 7f54767fc700 1 -- 172.21.15.114:0/2248060012 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f5470007ad0 con 0x7f5480100cd0 2024-02-16T22:29:23.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.232+0000 7f54767fc700 1 --2- 172.21.15.114:0/2248060012 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f546c06d8a0 0x7f546c06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:23.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.232+0000 7f54767fc700 1 -- 172.21.15.114:0/2248060012 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f547008d2a0 con 0x7f5480100cd0 2024-02-16T22:29:23.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.232+0000 7f5485095700 1 --2- 172.21.15.114:0/2248060012 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f546c06d8a0 0x7f546c06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:23.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.232+0000 7f5485095700 1 --2- 172.21.15.114:0/2248060012 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f546c06d8a0 0x7f546c06fd60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f547c006010 tx=0x7f547c005ab0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:29:23.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.236+0000 7f54767fc700 1 -- 172.21.15.114:0/2248060012 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f547005a530 con 0x7f5480100cd0 2024-02-16T22:29:23.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.515+0000 7f54872f9700 1 -- 172.21.15.114:0/2248060012 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f54801087f0 con 0x7f5480100cd0 2024-02-16T22:29:23.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.515+0000 7f54767fc700 1 -- 172.21.15.114:0/2248060012 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f547005a0c0 con 0x7f5480100cd0 2024-02-16T22:29:23.517 INFO:teuthology.orchestra.run.smithi114.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:29:23.517 INFO:teuthology.orchestra.run.smithi114.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:29:23.517 INFO:teuthology.orchestra.run.smithi114.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:29:23.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.517+0000 7f54872f9700 1 -- 172.21.15.114:0/2248060012 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f546c06d8a0 msgr2=0x7f546c06fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:23.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.517+0000 7f54872f9700 1 --2- 172.21.15.114:0/2248060012 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f546c06d8a0 0x7f546c06fd60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f547c006010 tx=0x7f547c005ab0 comp rx=0 tx=0).stop 2024-02-16T22:29:23.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.517+0000 7f54872f9700 1 -- 172.21.15.114:0/2248060012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5480100cd0 msgr2=0x7f54801957f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:23.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.517+0000 7f54872f9700 1 --2- 172.21.15.114:0/2248060012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5480100cd0 0x7f54801957f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f547000c8a0 tx=0x7f547000cc60 comp rx=0 tx=0).stop 2024-02-16T22:29:23.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.517+0000 7f54872f9700 1 -- 172.21.15.114:0/2248060012 shutdown_connections 2024-02-16T22:29:23.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.517+0000 7f54872f9700 1 --2- 172.21.15.114:0/2248060012 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f546c06d8a0 0x7f546c06fd60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:23.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.518+0000 7f54872f9700 1 --2- 172.21.15.114:0/2248060012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54800ffad0 0x7f54801952b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:23.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.518+0000 7f54872f9700 1 --2- 172.21.15.114:0/2248060012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5480100cd0 0x7f54801957f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:23.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.518+0000 7f54872f9700 1 -- 172.21.15.114:0/2248060012 >> 172.21.15.114:0/2248060012 conn(0x7f54800fb050 msgr2=0x7f5480103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:29:23.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.518+0000 7f54872f9700 1 -- 172.21.15.114:0/2248060012 shutdown_connections 2024-02-16T22:29:23.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:23.518+0000 7f54872f9700 1 -- 172.21.15.114:0/2248060012 wait complete. 2024-02-16T22:29:24.145 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:23 smithi114 ceph-mon[94452]: from='client.34460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:29:24.146 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:23 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/2248060012' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:29:24.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:23 smithi123 ceph-mon[82331]: from='client.34460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:29:24.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:23 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/2248060012' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:29:25.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:24 smithi114 ceph-mon[94452]: pgmap v205: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 85 B/s wr, 7 op/s 2024-02-16T22:29:25.263 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:24 smithi123 ceph-mon[82331]: pgmap v205: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 85 B/s wr, 7 op/s 2024-02-16T22:29:27.333 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:27 smithi123 ceph-mon[82331]: pgmap v206: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 170 B/s wr, 7 op/s 2024-02-16T22:29:27.333 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:27 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:27.333 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:27 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:27.333 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:27 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:29:27.333 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:27 smithi123 ceph-mon[82331]: mds.? [v2:172.21.15.123:6824/1425065679,v1:172.21.15.123:6825/1425065679] up:boot 2024-02-16T22:29:27.333 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:27 smithi123 ceph-mon[82331]: fsmap cephfs:1 {0=cephfs.smithi123.gopcfa=up:active} 3 up:standby 2024-02-16T22:29:27.334 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:27 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi123.vhlqbe"}]: dispatch 2024-02-16T22:29:27.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:27 smithi114 ceph-mon[94452]: pgmap v206: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 170 B/s wr, 7 op/s 2024-02-16T22:29:27.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:27 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:27.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:27 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:27.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:27 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:29:27.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:27 smithi114 ceph-mon[94452]: mds.? [v2:172.21.15.123:6824/1425065679,v1:172.21.15.123:6825/1425065679] up:boot 2024-02-16T22:29:27.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:27 smithi114 ceph-mon[94452]: fsmap cephfs:1 {0=cephfs.smithi123.gopcfa=up:active} 3 up:standby 2024-02-16T22:29:27.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:27 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi123.vhlqbe"}]: dispatch 2024-02-16T22:29:29.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:29 smithi114 ceph-mon[94452]: pgmap v207: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 9.5 MiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-02-16T22:29:29.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:29 smithi123 ceph-mon[82331]: pgmap v207: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 9.5 MiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-02-16T22:29:30.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:30 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:30.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:30 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:30.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:30 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:30.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:30 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:31.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:31 smithi123 ceph-mon[82331]: pgmap v208: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 3.2 MiB/s rd, 4.1 KiB/s wr, 5 op/s 2024-02-16T22:29:31.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:31 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:31.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:31 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:31.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:31 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:31.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:31 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:29:31.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:31 smithi114 ceph-mon[94452]: pgmap v208: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 3.2 MiB/s rd, 4.1 KiB/s wr, 5 op/s 2024-02-16T22:29:31.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:31 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:31.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:31 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:31.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:31 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:31.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:31 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:29:33.416 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:33 smithi123 ceph-mon[82331]: pgmap v209: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 206 KiB/s rd, 4.1 KiB/s wr, 5 op/s 2024-02-16T22:29:33.581 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:33 smithi114 ceph-mon[94452]: pgmap v209: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 206 KiB/s rd, 4.1 KiB/s wr, 5 op/s 2024-02-16T22:29:35.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:34 smithi114 ceph-mon[94452]: pgmap v210: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 4.0 KiB/s wr, 5 op/s 2024-02-16T22:29:35.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:34 smithi114 ceph-mon[94452]: Detected new or changed devices on smithi123 2024-02-16T22:29:35.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:34 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:35.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:34 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:35.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:34 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:29:35.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:34 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:29:35.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:34 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:35.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:34 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:29:35.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:34 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:35.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:34 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:35.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:34 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:35.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:34 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:35.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:34 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi123.gopcfa"]}]: dispatch 2024-02-16T22:29:35.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:34 smithi114 ceph-mon[94452]: Upgrade: It appears safe to stop mds.cephfs.smithi123.gopcfa 2024-02-16T22:29:35.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:34 smithi123 ceph-mon[82331]: pgmap v210: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 4.0 KiB/s wr, 5 op/s 2024-02-16T22:29:35.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:34 smithi123 ceph-mon[82331]: Detected new or changed devices on smithi123 2024-02-16T22:29:35.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:34 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:35.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:34 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:35.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:34 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:29:35.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:34 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:29:35.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:34 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:35.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:34 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:29:35.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:34 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:35.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:34 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:35.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:34 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:35.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:34 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:35.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:34 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi123.gopcfa"]}]: dispatch 2024-02-16T22:29:35.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:34 smithi123 ceph-mon[82331]: Upgrade: It appears safe to stop mds.cephfs.smithi123.gopcfa 2024-02-16T22:29:36.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:36 smithi123 ceph-mon[82331]: Upgrade: Updating mds.cephfs.smithi123.gopcfa 2024-02-16T22:29:36.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:36.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi123.gopcfa", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:29:36.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:29:36.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:36 smithi123 ceph-mon[82331]: Deploying daemon mds.cephfs.smithi123.gopcfa on smithi123 2024-02-16T22:29:36.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:36 smithi123 ceph-mon[82331]: pgmap v211: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 4.0 KiB/s wr, 3 op/s 2024-02-16T22:29:36.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:36 smithi114 ceph-mon[94452]: Upgrade: Updating mds.cephfs.smithi123.gopcfa 2024-02-16T22:29:36.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:36.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi123.gopcfa", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T22:29:36.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:29:36.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:36 smithi114 ceph-mon[94452]: Deploying daemon mds.cephfs.smithi123.gopcfa on smithi123 2024-02-16T22:29:36.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:36 smithi114 ceph-mon[94452]: pgmap v211: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 4.0 KiB/s wr, 3 op/s 2024-02-16T22:29:38.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:37 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi114[94448]: 2024-02-16T22:29:37.875+0000 7f1ab68f5700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T22:29:39.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:38 smithi114 ceph-mon[94452]: pgmap v212: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 3.9 KiB/s wr, 2 op/s 2024-02-16T22:29:39.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:38 smithi114 ceph-mon[94452]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-16T22:29:39.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:38 smithi114 ceph-mon[94452]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T22:29:39.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:38 smithi114 ceph-mon[94452]: osdmap e82: 6 total, 6 up, 6 in 2024-02-16T22:29:39.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:38 smithi114 ceph-mon[94452]: Standby daemon mds.cephfs.smithi114.zzryke assigned to filesystem cephfs as rank 0 2024-02-16T22:29:39.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:38 smithi114 ceph-mon[94452]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T22:29:39.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:38 smithi114 ceph-mon[94452]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-02-16T22:29:39.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:38 smithi114 ceph-mon[94452]: fsmap cephfs:1/1 {0=cephfs.smithi114.zzryke=up:replay} 2 up:standby 2024-02-16T22:29:39.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:38 smithi123 ceph-mon[82331]: pgmap v212: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 3.9 KiB/s wr, 2 op/s 2024-02-16T22:29:39.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:38 smithi123 ceph-mon[82331]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-16T22:29:39.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:38 smithi123 ceph-mon[82331]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T22:29:39.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:38 smithi123 ceph-mon[82331]: osdmap e82: 6 total, 6 up, 6 in 2024-02-16T22:29:39.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:38 smithi123 ceph-mon[82331]: Standby daemon mds.cephfs.smithi114.zzryke assigned to filesystem cephfs as rank 0 2024-02-16T22:29:39.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:38 smithi123 ceph-mon[82331]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T22:29:39.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:38 smithi123 ceph-mon[82331]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-02-16T22:29:39.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:38 smithi123 ceph-mon[82331]: fsmap cephfs:1/1 {0=cephfs.smithi114.zzryke=up:replay} 2 up:standby 2024-02-16T22:29:41.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:40 smithi114 ceph-mon[94452]: pgmap v214: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 4.8 MiB/s rd, 1 op/s 2024-02-16T22:29:41.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:40 smithi123 ceph-mon[82331]: pgmap v214: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 4.8 MiB/s rd, 1 op/s 2024-02-16T22:29:43.071 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:42 smithi123 ceph-mon[82331]: pgmap v215: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 5.6 MiB/s rd, 2 op/s 2024-02-16T22:29:43.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:42 smithi114 ceph-mon[94452]: pgmap v215: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 5.6 MiB/s rd, 2 op/s 2024-02-16T22:29:45.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:44 smithi114 ceph-mon[94452]: pgmap v216: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-02-16T22:29:45.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:44 smithi123 ceph-mon[82331]: pgmap v216: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-02-16T22:29:47.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:46 smithi114 ceph-mon[94452]: pgmap v217: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 5 op/s 2024-02-16T22:29:47.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:47.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:29:47.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:46 smithi123 ceph-mon[82331]: pgmap v217: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 5 op/s 2024-02-16T22:29:47.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:46 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:47.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:46 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:29:49.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:48 smithi114 ceph-mon[94452]: pgmap v218: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 7 op/s 2024-02-16T22:29:49.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:48 smithi114 ceph-mon[94452]: reconnect by client.14536 192.168.0.1:0/21264991 after 0 2024-02-16T22:29:49.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:48 smithi114 ceph-mon[94452]: reconnect by client.24321 192.168.0.1:0/2666785881 after 0 2024-02-16T22:29:49.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:48 smithi114 ceph-mon[94452]: mds.? [v2:172.21.15.114:6826/2017800151,v1:172.21.15.114:6827/2017800151] up:reconnect 2024-02-16T22:29:49.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:48 smithi114 ceph-mon[94452]: fsmap cephfs:1/1 {0=cephfs.smithi114.zzryke=up:reconnect} 2 up:standby 2024-02-16T22:29:49.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:48 smithi123 ceph-mon[82331]: pgmap v218: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 7 op/s 2024-02-16T22:29:49.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:48 smithi123 ceph-mon[82331]: reconnect by client.14536 192.168.0.1:0/21264991 after 0 2024-02-16T22:29:49.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:48 smithi123 ceph-mon[82331]: reconnect by client.24321 192.168.0.1:0/2666785881 after 0 2024-02-16T22:29:49.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:48 smithi123 ceph-mon[82331]: mds.? [v2:172.21.15.114:6826/2017800151,v1:172.21.15.114:6827/2017800151] up:reconnect 2024-02-16T22:29:49.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:48 smithi123 ceph-mon[82331]: fsmap cephfs:1/1 {0=cephfs.smithi114.zzryke=up:reconnect} 2 up:standby 2024-02-16T22:29:50.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:49 smithi123 ceph-mon[82331]: mds.? [v2:172.21.15.114:6826/2017800151,v1:172.21.15.114:6827/2017800151] up:rejoin 2024-02-16T22:29:50.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:49 smithi123 ceph-mon[82331]: mds.? [v2:172.21.15.123:6826/3128771108,v1:172.21.15.123:6827/3128771108] up:boot 2024-02-16T22:29:50.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:49 smithi123 ceph-mon[82331]: fsmap cephfs:1/1 {0=cephfs.smithi114.zzryke=up:rejoin} 3 up:standby 2024-02-16T22:29:50.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:49 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi123.gopcfa"}]: dispatch 2024-02-16T22:29:50.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:49 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:50.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:49 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:50.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:49 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:29:50.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:49 smithi123 ceph-mon[82331]: daemon mds.cephfs.smithi114.zzryke is now active in filesystem cephfs as rank 0 2024-02-16T22:29:50.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:49 smithi114 ceph-mon[94452]: mds.? [v2:172.21.15.114:6826/2017800151,v1:172.21.15.114:6827/2017800151] up:rejoin 2024-02-16T22:29:50.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:49 smithi114 ceph-mon[94452]: mds.? [v2:172.21.15.123:6826/3128771108,v1:172.21.15.123:6827/3128771108] up:boot 2024-02-16T22:29:50.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:49 smithi114 ceph-mon[94452]: fsmap cephfs:1/1 {0=cephfs.smithi114.zzryke=up:rejoin} 3 up:standby 2024-02-16T22:29:50.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:49 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi123.gopcfa"}]: dispatch 2024-02-16T22:29:50.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:49 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:50.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:49 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:50.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:49 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:29:50.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:49 smithi114 ceph-mon[94452]: daemon mds.cephfs.smithi114.zzryke is now active in filesystem cephfs as rank 0 2024-02-16T22:29:51.154 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:50 smithi123 ceph-mon[82331]: pgmap v219: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 85 B/s wr, 6 op/s 2024-02-16T22:29:51.154 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:50 smithi123 ceph-mon[82331]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-16T22:29:51.155 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:50 smithi123 ceph-mon[82331]: mds.? [v2:172.21.15.114:6826/2017800151,v1:172.21.15.114:6827/2017800151] up:active 2024-02-16T22:29:51.155 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:50 smithi123 ceph-mon[82331]: fsmap cephfs:1 {0=cephfs.smithi114.zzryke=up:active} 3 up:standby 2024-02-16T22:29:51.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:50 smithi114 ceph-mon[94452]: pgmap v219: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 85 B/s wr, 6 op/s 2024-02-16T22:29:51.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:50 smithi114 ceph-mon[94452]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-16T22:29:51.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:50 smithi114 ceph-mon[94452]: mds.? [v2:172.21.15.114:6826/2017800151,v1:172.21.15.114:6827/2017800151] up:active 2024-02-16T22:29:51.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:50 smithi114 ceph-mon[94452]: fsmap cephfs:1 {0=cephfs.smithi114.zzryke=up:active} 3 up:standby 2024-02-16T22:29:53.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:52 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:53.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:52 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:53.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:52 smithi123 ceph-mon[82331]: pgmap v220: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 170 B/s wr, 5 op/s 2024-02-16T22:29:53.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:52 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:53.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:52 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:53.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:52 smithi114 ceph-mon[94452]: pgmap v220: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 170 B/s wr, 5 op/s 2024-02-16T22:29:53.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.635+0000 7fe1d52a9700 1 -- 172.21.15.114:0/1327227682 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1d00ffad0 msgr2=0x7fe1d00ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:53.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.635+0000 7fe1d52a9700 1 --2- 172.21.15.114:0/1327227682 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1d00ffad0 0x7fe1d00ffef0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0009910 tx=0x7fe1c0009c20 comp rx=0 tx=0).stop 2024-02-16T22:29:53.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.635+0000 7fe1d52a9700 1 -- 172.21.15.114:0/1327227682 shutdown_connections 2024-02-16T22:29:53.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.635+0000 7fe1d52a9700 1 --2- 172.21.15.114:0/1327227682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1d0100cd0 0x7fe1d0101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:53.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.635+0000 7fe1d52a9700 1 --2- 172.21.15.114:0/1327227682 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1d00ffad0 0x7fe1d00ffef0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:53.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.635+0000 7fe1d52a9700 1 -- 172.21.15.114:0/1327227682 >> 172.21.15.114:0/1327227682 conn(0x7fe1d00fb050 msgr2=0x7fe1d00fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:29:53.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.635+0000 7fe1d52a9700 1 -- 172.21.15.114:0/1327227682 shutdown_connections 2024-02-16T22:29:53.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.635+0000 7fe1d52a9700 1 -- 172.21.15.114:0/1327227682 wait complete. 2024-02-16T22:29:53.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.635+0000 7fe1d52a9700 1 Processor -- start 2024-02-16T22:29:53.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.635+0000 7fe1d52a9700 1 -- start start 2024-02-16T22:29:53.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.636+0000 7fe1d52a9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1d00ffad0 0x7fe1d01963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:53.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.636+0000 7fe1d52a9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1d0100cd0 0x7fe1d01968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:53.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.636+0000 7fe1d52a9700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d0196f10 con 0x7fe1d0100cd0 2024-02-16T22:29:53.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.636+0000 7fe1d52a9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d0197050 con 0x7fe1d00ffad0 2024-02-16T22:29:53.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.636+0000 7fe1c67fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1d0100cd0 0x7fe1d01968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:53.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.636+0000 7fe1c67fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1d0100cd0 0x7fe1d01968f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59816/0 (socket says 172.21.15.114:59816) 2024-02-16T22:29:53.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.636+0000 7fe1c67fc700 1 -- 172.21.15.114:0/3710258790 learned_addr learned my addr 172.21.15.114:0/3710258790 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:29:53.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.636+0000 7fe1ceffd700 1 --2- 172.21.15.114:0/3710258790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1d00ffad0 0x7fe1d01963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:53.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.636+0000 7fe1c67fc700 1 -- 172.21.15.114:0/3710258790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1d00ffad0 msgr2=0x7fe1d01963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:53.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.636+0000 7fe1c67fc700 1 --2- 172.21.15.114:0/3710258790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1d00ffad0 0x7fe1d01963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:53.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.636+0000 7fe1c67fc700 1 -- 172.21.15.114:0/3710258790 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c00095f0 con 0x7fe1d0100cd0 2024-02-16T22:29:53.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.636+0000 7fe1c67fc700 1 --2- 172.21.15.114:0/3710258790 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1d0100cd0 0x7fe1d01968f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800d8d0 tx=0x7fe1b800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:29:53.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.636+0000 7fe1ccff9700 1 -- 172.21.15.114:0/3710258790 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1b80098a0 con 0x7fe1d0100cd0 2024-02-16T22:29:53.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.636+0000 7fe1d52a9700 1 -- 172.21.15.114:0/3710258790 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d019bb00 con 0x7fe1d0100cd0 2024-02-16T22:29:53.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.636+0000 7fe1ccff9700 1 -- 172.21.15.114:0/3710258790 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe1b8010460 con 0x7fe1d0100cd0 2024-02-16T22:29:53.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.636+0000 7fe1ccff9700 1 -- 172.21.15.114:0/3710258790 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1b800f5f0 con 0x7fe1d0100cd0 2024-02-16T22:29:53.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.637+0000 7fe1d52a9700 1 -- 172.21.15.114:0/3710258790 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d019c050 con 0x7fe1d0100cd0 2024-02-16T22:29:53.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.637+0000 7fe1d52a9700 1 -- 172.21.15.114:0/3710258790 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1d0108630 con 0x7fe1d0100cd0 2024-02-16T22:29:53.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.638+0000 7fe1ccff9700 1 -- 172.21.15.114:0/3710258790 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fe1b8009a00 con 0x7fe1d0100cd0 2024-02-16T22:29:53.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.638+0000 7fe1ccff9700 1 --2- 172.21.15.114:0/3710258790 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe1bc06d8a0 0x7fe1bc06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:53.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.638+0000 7fe1ccff9700 1 -- 172.21.15.114:0/3710258790 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7fe1b808d470 con 0x7fe1d0100cd0 2024-02-16T22:29:53.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.638+0000 7fe1ceffd700 1 --2- 172.21.15.114:0/3710258790 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe1bc06d8a0 0x7fe1bc06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:53.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.639+0000 7fe1ceffd700 1 --2- 172.21.15.114:0/3710258790 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe1bc06d8a0 0x7fe1bc06fd60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0006010 tx=0x7fe1c0005ec0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:29:53.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.643+0000 7fe1ccff9700 1 -- 172.21.15.114:0/3710258790 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe1b805a6a0 con 0x7fe1d0100cd0 2024-02-16T22:29:53.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.856+0000 7fe1d52a9700 1 -- 172.21.15.114:0/3710258790 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1d00694a0 con 0x7fe1bc06d8a0 2024-02-16T22:29:53.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.857+0000 7fe1ccff9700 1 -- 172.21.15.114:0/3710258790 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe1d00694a0 con 0x7fe1bc06d8a0 2024-02-16T22:29:53.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.860+0000 7fe1d52a9700 1 -- 172.21.15.114:0/3710258790 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe1bc06d8a0 msgr2=0x7fe1bc06fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:53.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.860+0000 7fe1d52a9700 1 --2- 172.21.15.114:0/3710258790 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe1bc06d8a0 0x7fe1bc06fd60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0006010 tx=0x7fe1c0005ec0 comp rx=0 tx=0).stop 2024-02-16T22:29:53.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.860+0000 7fe1d52a9700 1 -- 172.21.15.114:0/3710258790 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1d0100cd0 msgr2=0x7fe1d01968f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:53.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.860+0000 7fe1d52a9700 1 --2- 172.21.15.114:0/3710258790 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1d0100cd0 0x7fe1d01968f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800d8d0 tx=0x7fe1b800dbe0 comp rx=0 tx=0).stop 2024-02-16T22:29:53.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.860+0000 7fe1d52a9700 1 -- 172.21.15.114:0/3710258790 shutdown_connections 2024-02-16T22:29:53.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.860+0000 7fe1d52a9700 1 --2- 172.21.15.114:0/3710258790 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe1bc06d8a0 0x7fe1bc06fd60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:53.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.860+0000 7fe1d52a9700 1 --2- 172.21.15.114:0/3710258790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1d00ffad0 0x7fe1d01963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:53.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.860+0000 7fe1d52a9700 1 --2- 172.21.15.114:0/3710258790 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1d0100cd0 0x7fe1d01968f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:53.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.860+0000 7fe1d52a9700 1 -- 172.21.15.114:0/3710258790 >> 172.21.15.114:0/3710258790 conn(0x7fe1d00fb050 msgr2=0x7fe1d0068d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:29:53.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.860+0000 7fe1d52a9700 1 -- 172.21.15.114:0/3710258790 shutdown_connections 2024-02-16T22:29:53.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.860+0000 7fe1d52a9700 1 -- 172.21.15.114:0/3710258790 wait complete. 2024-02-16T22:29:53.873 INFO:teuthology.orchestra.run.smithi114.stdout:true 2024-02-16T22:29:53.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.968+0000 7f01bff95700 1 -- 172.21.15.114:0/1988173577 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01b8100cb0 msgr2=0x7f01b8101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:53.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.968+0000 7f01bff95700 1 --2- 172.21.15.114:0/1988173577 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01b8100cb0 0x7f01b8101110 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f01a8009910 tx=0x7f01a8009c20 comp rx=0 tx=0).stop 2024-02-16T22:29:53.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.968+0000 7f01bff95700 1 -- 172.21.15.114:0/1988173577 shutdown_connections 2024-02-16T22:29:53.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.968+0000 7f01bff95700 1 --2- 172.21.15.114:0/1988173577 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01b8100cb0 0x7f01b8101110 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:53.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.968+0000 7f01bff95700 1 --2- 172.21.15.114:0/1988173577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01b80ffab0 0x7f01b80ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:53.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.968+0000 7f01bff95700 1 -- 172.21.15.114:0/1988173577 >> 172.21.15.114:0/1988173577 conn(0x7f01b80fb030 msgr2=0x7f01b80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:29:53.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.968+0000 7f01bff95700 1 -- 172.21.15.114:0/1988173577 shutdown_connections 2024-02-16T22:29:53.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.968+0000 7f01bff95700 1 -- 172.21.15.114:0/1988173577 wait complete. 2024-02-16T22:29:53.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.969+0000 7f01bff95700 1 Processor -- start 2024-02-16T22:29:53.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.969+0000 7f01bff95700 1 -- start start 2024-02-16T22:29:53.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.969+0000 7f01bff95700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01b80ffab0 0x7f01b8068f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:53.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.969+0000 7f01bff95700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01b8100cb0 0x7f01b819b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:53.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.969+0000 7f01bff95700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b819b600 con 0x7f01b80ffab0 2024-02-16T22:29:53.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.969+0000 7f01bff95700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b819b740 con 0x7f01b8100cb0 2024-02-16T22:29:53.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.969+0000 7f01bdd31700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01b80ffab0 0x7f01b8068f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:53.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.969+0000 7f01bdd31700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01b80ffab0 0x7f01b8068f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59836/0 (socket says 172.21.15.114:59836) 2024-02-16T22:29:53.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.969+0000 7f01bdd31700 1 -- 172.21.15.114:0/3554986052 learned_addr learned my addr 172.21.15.114:0/3554986052 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:29:53.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.969+0000 7f01bd530700 1 --2- 172.21.15.114:0/3554986052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01b8100cb0 0x7f01b819b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:53.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.970+0000 7f01bdd31700 1 -- 172.21.15.114:0/3554986052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01b8100cb0 msgr2=0x7f01b819b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:53.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.970+0000 7f01bdd31700 1 --2- 172.21.15.114:0/3554986052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01b8100cb0 0x7f01b819b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:53.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.970+0000 7f01bdd31700 1 -- 172.21.15.114:0/3554986052 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a80095f0 con 0x7f01b80ffab0 2024-02-16T22:29:53.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.970+0000 7f01bdd31700 1 --2- 172.21.15.114:0/3554986052 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01b80ffab0 0x7f01b8068f70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f01b400c910 tx=0x7f01b400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:29:53.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.970+0000 7f01aeffd700 1 -- 172.21.15.114:0/3554986052 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01b40041f0 con 0x7f01b80ffab0 2024-02-16T22:29:53.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.970+0000 7f01aeffd700 1 -- 172.21.15.114:0/3554986052 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f01b400ce60 con 0x7f01b80ffab0 2024-02-16T22:29:53.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.970+0000 7f01bff95700 1 -- 172.21.15.114:0/3554986052 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b819ba20 con 0x7f01b80ffab0 2024-02-16T22:29:53.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.970+0000 7f01aeffd700 1 -- 172.21.15.114:0/3554986052 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01b4003d20 con 0x7f01b80ffab0 2024-02-16T22:29:53.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.970+0000 7f01bff95700 1 -- 172.21.15.114:0/3554986052 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b819bf70 con 0x7f01b80ffab0 2024-02-16T22:29:53.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.971+0000 7f01bff95700 1 -- 172.21.15.114:0/3554986052 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b8108610 con 0x7f01b80ffab0 2024-02-16T22:29:53.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.972+0000 7f01aeffd700 1 -- 172.21.15.114:0/3554986052 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f01b4007900 con 0x7f01b80ffab0 2024-02-16T22:29:53.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.972+0000 7f01aeffd700 1 --2- 172.21.15.114:0/3554986052 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f01a406d8a0 0x7f01a406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:53.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.973+0000 7f01aeffd700 1 -- 172.21.15.114:0/3554986052 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f01b408d200 con 0x7f01b80ffab0 2024-02-16T22:29:53.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.973+0000 7f01bd530700 1 --2- 172.21.15.114:0/3554986052 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f01a406d8a0 0x7f01a406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:53.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.973+0000 7f01bd530700 1 --2- 172.21.15.114:0/3554986052 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f01a406d8a0 0x7f01a406fd60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f01b80694f0 tx=0x7f01a800b560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:29:53.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:53.978+0000 7f01aeffd700 1 -- 172.21.15.114:0/3554986052 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f01b405a4b0 con 0x7f01b80ffab0 2024-02-16T22:29:54.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:53 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:54.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:53 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:54.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:53 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:54.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:53 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:54.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.200+0000 7f01bff95700 1 -- 172.21.15.114:0/3554986052 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f01b8105600 con 0x7f01a406d8a0 2024-02-16T22:29:54.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.201+0000 7f01aeffd700 1 -- 172.21.15.114:0/3554986052 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f01b8105600 con 0x7f01a406d8a0 2024-02-16T22:29:54.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.203+0000 7f01bff95700 1 -- 172.21.15.114:0/3554986052 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f01a406d8a0 msgr2=0x7f01a406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:54.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.203+0000 7f01bff95700 1 --2- 172.21.15.114:0/3554986052 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f01a406d8a0 0x7f01a406fd60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f01b80694f0 tx=0x7f01a800b560 comp rx=0 tx=0).stop 2024-02-16T22:29:54.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.203+0000 7f01bff95700 1 -- 172.21.15.114:0/3554986052 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01b80ffab0 msgr2=0x7f01b8068f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:54.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.203+0000 7f01bff95700 1 --2- 172.21.15.114:0/3554986052 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01b80ffab0 0x7f01b8068f70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f01b400c910 tx=0x7f01b400cc20 comp rx=0 tx=0).stop 2024-02-16T22:29:54.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.203+0000 7f01bff95700 1 -- 172.21.15.114:0/3554986052 shutdown_connections 2024-02-16T22:29:54.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.203+0000 7f01bff95700 1 --2- 172.21.15.114:0/3554986052 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f01a406d8a0 0x7f01a406fd60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:54.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.203+0000 7f01bff95700 1 --2- 172.21.15.114:0/3554986052 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01b80ffab0 0x7f01b8068f70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:54.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.203+0000 7f01bff95700 1 --2- 172.21.15.114:0/3554986052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01b8100cb0 0x7f01b819b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:54.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.203+0000 7f01bff95700 1 -- 172.21.15.114:0/3554986052 >> 172.21.15.114:0/3554986052 conn(0x7f01b80fb030 msgr2=0x7f01b8103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:29:54.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.204+0000 7f01bff95700 1 -- 172.21.15.114:0/3554986052 shutdown_connections 2024-02-16T22:29:54.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.204+0000 7f01bff95700 1 -- 172.21.15.114:0/3554986052 wait complete. 2024-02-16T22:29:54.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.314+0000 7f8382573700 1 -- 172.21.15.114:0/3067654683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f837c0fec70 msgr2=0x7f837c0ff0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:54.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.314+0000 7f8382573700 1 --2- 172.21.15.114:0/3067654683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f837c0fec70 0x7f837c0ff0d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8370009910 tx=0x7f8370009c20 comp rx=0 tx=0).stop 2024-02-16T22:29:54.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.314+0000 7f8382573700 1 -- 172.21.15.114:0/3067654683 shutdown_connections 2024-02-16T22:29:54.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.314+0000 7f8382573700 1 --2- 172.21.15.114:0/3067654683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f837c0fec70 0x7f837c0ff0d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:54.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.314+0000 7f8382573700 1 --2- 172.21.15.114:0/3067654683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f837c1021f0 0x7f837c102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:54.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.314+0000 7f8382573700 1 -- 172.21.15.114:0/3067654683 >> 172.21.15.114:0/3067654683 conn(0x7f837c0fa810 msgr2=0x7f837c0fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:29:54.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.314+0000 7f8382573700 1 -- 172.21.15.114:0/3067654683 shutdown_connections 2024-02-16T22:29:54.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.314+0000 7f8382573700 1 -- 172.21.15.114:0/3067654683 wait complete. 2024-02-16T22:29:54.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.314+0000 7f8382573700 1 Processor -- start 2024-02-16T22:29:54.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.315+0000 7f8382573700 1 -- start start 2024-02-16T22:29:54.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.315+0000 7f8382573700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f837c0fec70 0x7f837c1018f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:54.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.315+0000 7f8382573700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f837c1021f0 0x7f837c0fff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:54.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.315+0000 7f8382573700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837c100480 con 0x7f837c1021f0 2024-02-16T22:29:54.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.315+0000 7f8382573700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837c1005c0 con 0x7f837c0fec70 2024-02-16T22:29:54.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.315+0000 7f837b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f837c1021f0 0x7f837c0fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:54.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.315+0000 7f837b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f837c1021f0 0x7f837c0fff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59844/0 (socket says 172.21.15.114:59844) 2024-02-16T22:29:54.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.315+0000 7f837b7fe700 1 -- 172.21.15.114:0/2872629590 learned_addr learned my addr 172.21.15.114:0/2872629590 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:29:54.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.315+0000 7f837b7fe700 1 -- 172.21.15.114:0/2872629590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f837c0fec70 msgr2=0x7f837c1018f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:54.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.315+0000 7f837bfff700 1 --2- 172.21.15.114:0/2872629590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f837c0fec70 0x7f837c1018f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:54.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.315+0000 7f837b7fe700 1 --2- 172.21.15.114:0/2872629590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f837c0fec70 0x7f837c1018f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:54.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.315+0000 7f837b7fe700 1 -- 172.21.15.114:0/2872629590 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83700095f0 con 0x7f837c1021f0 2024-02-16T22:29:54.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.315+0000 7f837bfff700 1 --2- 172.21.15.114:0/2872629590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f837c0fec70 0x7f837c1018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T22:29:54.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.315+0000 7f837b7fe700 1 --2- 172.21.15.114:0/2872629590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f837c1021f0 0x7f837c0fff40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8370006010 tx=0x7f83700056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:29:54.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.316+0000 7f83797fa700 1 -- 172.21.15.114:0/2872629590 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f837001d070 con 0x7f837c1021f0 2024-02-16T22:29:54.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.316+0000 7f8382573700 1 -- 172.21.15.114:0/2872629590 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f837c100840 con 0x7f837c1021f0 2024-02-16T22:29:54.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.316+0000 7f83797fa700 1 -- 172.21.15.114:0/2872629590 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f837000f460 con 0x7f837c1021f0 2024-02-16T22:29:54.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.316+0000 7f83797fa700 1 -- 172.21.15.114:0/2872629590 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8370021640 con 0x7f837c1021f0 2024-02-16T22:29:54.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.316+0000 7f8382573700 1 -- 172.21.15.114:0/2872629590 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f837c100d30 con 0x7f837c1021f0 2024-02-16T22:29:54.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.317+0000 7f8382573700 1 -- 172.21.15.114:0/2872629590 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f837c109820 con 0x7f837c1021f0 2024-02-16T22:29:54.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.317+0000 7f83797fa700 1 -- 172.21.15.114:0/2872629590 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f837000f5d0 con 0x7f837c1021f0 2024-02-16T22:29:54.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.318+0000 7f83797fa700 1 --2- 172.21.15.114:0/2872629590 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f836406d8a0 0x7f836406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:54.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.318+0000 7f83797fa700 1 -- 172.21.15.114:0/2872629590 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f837008ea30 con 0x7f837c1021f0 2024-02-16T22:29:54.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.318+0000 7f837bfff700 1 --2- 172.21.15.114:0/2872629590 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f836406d8a0 0x7f836406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:54.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.318+0000 7f837bfff700 1 --2- 172.21.15.114:0/2872629590 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f836406d8a0 0x7f836406fd60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f837c068ad0 tx=0x7f836c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:29:54.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.322+0000 7f83797fa700 1 -- 172.21.15.114:0/2872629590 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f837005bc60 con 0x7f837c1021f0 2024-02-16T22:29:54.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.524+0000 7f8382573700 1 -- 172.21.15.114:0/2872629590 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f837c1068e0 con 0x7f836406d8a0 2024-02-16T22:29:54.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.537+0000 7f83797fa700 1 -- 172.21.15.114:0/2872629590 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f837c1068e0 con 0x7f836406d8a0 2024-02-16T22:29:54.539 INFO:teuthology.orchestra.run.smithi114.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:29:54.539 INFO:teuthology.orchestra.run.smithi114.stdout:alertmanager.smithi114 smithi114 *:9093,9094 running (15m) 52s ago 17m 24.3M - 0.25.0 c8568f914cd2 e7404fea400e 2024-02-16T22:29:54.539 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi114 smithi114 running (17m) 52s ago 17m 9017k - 18.2.1 94eadd990e0a 7fd07162cd96 2024-02-16T22:29:54.539 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi123 smithi123 running (16m) 2s ago 16m 8929k - 18.2.1 94eadd990e0a 7687f5e99621 2024-02-16T22:29:54.539 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi114 smithi114 running (5m) 52s ago 17m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 9b8ae9efe83b 2024-02-16T22:29:54.539 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi123 smithi123 running (5m) 2s ago 16m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 6d97263a0ffa 2024-02-16T22:29:54.539 INFO:teuthology.orchestra.run.smithi114.stdout:grafana.smithi114 smithi114 *:3000 running (15m) 52s ago 17m 88.3M - 9.4.7 954c08fa6188 db1c26c3bc20 2024-02-16T22:29:54.539 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.fdxpey smithi114 running (58s) 52s ago 11m 12.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2984ae8a955a 2024-02-16T22:29:54.539 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.zzryke smithi114 running (85s) 52s ago 11m 12.4M - 19.0.0-1450-gf78a58c0 f081f847070a 63fa2af7290e 2024-02-16T22:29:54.540 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.gopcfa smithi123 running (6s) 2s ago 11m 13.8M - 19.0.0-1450-gf78a58c0 f081f847070a ec6612a62e2a 2024-02-16T22:29:54.540 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.vhlqbe smithi123 running (28s) 2s ago 11m 15.7M - 19.0.0-1450-gf78a58c0 f081f847070a be790c6287e7 2024-02-16T22:29:54.540 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi114.vcullz smithi114 *:8443,9283,8765 running (7m) 52s ago 18m 507M - 19.0.0-1450-gf78a58c0 f081f847070a 14ebeb7888c0 2024-02-16T22:29:54.540 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi123.inzgjd smithi123 *:8443,9283,8765 running (6m) 2s ago 16m 451M - 19.0.0-1450-gf78a58c0 f081f847070a f10aab9faf9e 2024-02-16T22:29:54.540 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi114 smithi114 running (6m) 52s ago 18m 52.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 58ac06372523 2024-02-16T22:29:54.540 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi123 smithi123 running (5m) 2s ago 16m 41.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a f76485e0ea24 2024-02-16T22:29:54.540 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi114 smithi114 *:9100 running (17m) 52s ago 17m 15.7M - 1.5.0 0da6a335fe13 744a090e06d0 2024-02-16T22:29:54.540 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi123 smithi123 *:9100 running (16m) 2s ago 16m 15.1M - 1.5.0 0da6a335fe13 3994b8c29b2e 2024-02-16T22:29:54.540 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 smithi114 running (4m) 52s ago 15m 98.3M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 1978fdad94a5 2024-02-16T22:29:54.540 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 smithi114 running (4m) 52s ago 14m 104M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 3f90e792898f 2024-02-16T22:29:54.541 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 smithi114 running (3m) 52s ago 14m 89.1M 1216M 19.0.0-1450-gf78a58c0 f081f847070a c663dfa0518b 2024-02-16T22:29:54.541 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 smithi123 running (2m) 2s ago 14m 113M 2240M 19.0.0-1450-gf78a58c0 f081f847070a b921448ba169 2024-02-16T22:29:54.541 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 smithi123 running (2m) 2s ago 13m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 54c2926d56b1 2024-02-16T22:29:54.541 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 smithi123 running (105s) 2s ago 13m 93.7M 2240M 19.0.0-1450-gf78a58c0 f081f847070a dd99bf034749 2024-02-16T22:29:54.541 INFO:teuthology.orchestra.run.smithi114.stdout:prometheus.smithi114 smithi114 *:9095 running (6m) 52s ago 16m 63.8M - 2.43.0 a07b618ecd1d 37aac985cdd7 2024-02-16T22:29:54.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.539+0000 7f8382573700 1 -- 172.21.15.114:0/2872629590 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f836406d8a0 msgr2=0x7f836406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:54.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.539+0000 7f8382573700 1 --2- 172.21.15.114:0/2872629590 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f836406d8a0 0x7f836406fd60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f837c068ad0 tx=0x7f836c008040 comp rx=0 tx=0).stop 2024-02-16T22:29:54.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.539+0000 7f8382573700 1 -- 172.21.15.114:0/2872629590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f837c1021f0 msgr2=0x7f837c0fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:54.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.539+0000 7f8382573700 1 --2- 172.21.15.114:0/2872629590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f837c1021f0 0x7f837c0fff40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8370006010 tx=0x7f83700056a0 comp rx=0 tx=0).stop 2024-02-16T22:29:54.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.539+0000 7f8382573700 1 -- 172.21.15.114:0/2872629590 shutdown_connections 2024-02-16T22:29:54.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.539+0000 7f8382573700 1 --2- 172.21.15.114:0/2872629590 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f836406d8a0 0x7f836406fd60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:54.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.539+0000 7f8382573700 1 --2- 172.21.15.114:0/2872629590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f837c0fec70 0x7f837c1018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:54.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.539+0000 7f8382573700 1 --2- 172.21.15.114:0/2872629590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f837c1021f0 0x7f837c0fff40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:54.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.539+0000 7f8382573700 1 -- 172.21.15.114:0/2872629590 >> 172.21.15.114:0/2872629590 conn(0x7f837c0fa810 msgr2=0x7f837c1051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:29:54.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.539+0000 7f8382573700 1 -- 172.21.15.114:0/2872629590 shutdown_connections 2024-02-16T22:29:54.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.539+0000 7f8382573700 1 -- 172.21.15.114:0/2872629590 wait complete. 2024-02-16T22:29:54.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.642+0000 7fd46077b700 1 -- 172.21.15.114:0/682568252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4581016d0 msgr2=0x7fd458103ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:54.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.642+0000 7fd46077b700 1 --2- 172.21.15.114:0/682568252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4581016d0 0x7fd458103ac0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd454009910 tx=0x7fd454009c20 comp rx=0 tx=0).stop 2024-02-16T22:29:54.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.642+0000 7fd46077b700 1 -- 172.21.15.114:0/682568252 shutdown_connections 2024-02-16T22:29:54.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.642+0000 7fd46077b700 1 --2- 172.21.15.114:0/682568252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd458104000 0x7fd4581063f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:54.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.642+0000 7fd46077b700 1 --2- 172.21.15.114:0/682568252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4581016d0 0x7fd458103ac0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:54.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.642+0000 7fd46077b700 1 -- 172.21.15.114:0/682568252 >> 172.21.15.114:0/682568252 conn(0x7fd4580fb060 msgr2=0x7fd4580fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:29:54.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.642+0000 7fd46077b700 1 -- 172.21.15.114:0/682568252 shutdown_connections 2024-02-16T22:29:54.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.642+0000 7fd46077b700 1 -- 172.21.15.114:0/682568252 wait complete. 2024-02-16T22:29:54.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.642+0000 7fd46077b700 1 Processor -- start 2024-02-16T22:29:54.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.642+0000 7fd46077b700 1 -- start start 2024-02-16T22:29:54.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.642+0000 7fd46077b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4581016d0 0x7fd458196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:54.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.642+0000 7fd46077b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd458104000 0x7fd458196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:54.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.642+0000 7fd46077b700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd458196e90 con 0x7fd4581016d0 2024-02-16T22:29:54.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.642+0000 7fd46077b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd458196fd0 con 0x7fd458104000 2024-02-16T22:29:54.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.643+0000 7fd45e517700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4581016d0 0x7fd458196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:54.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.643+0000 7fd45e517700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4581016d0 0x7fd458196330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59872/0 (socket says 172.21.15.114:59872) 2024-02-16T22:29:54.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.643+0000 7fd45e517700 1 -- 172.21.15.114:0/1419276434 learned_addr learned my addr 172.21.15.114:0/1419276434 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:29:54.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.643+0000 7fd45dd16700 1 --2- 172.21.15.114:0/1419276434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd458104000 0x7fd458196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:54.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.643+0000 7fd45e517700 1 -- 172.21.15.114:0/1419276434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd458104000 msgr2=0x7fd458196870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:54.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.643+0000 7fd45e517700 1 --2- 172.21.15.114:0/1419276434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd458104000 0x7fd458196870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:54.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.643+0000 7fd45e517700 1 -- 172.21.15.114:0/1419276434 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4540095f0 con 0x7fd4581016d0 2024-02-16T22:29:54.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.643+0000 7fd45dd16700 1 --2- 172.21.15.114:0/1419276434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd458104000 0x7fd458196870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:29:54.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.643+0000 7fd45e517700 1 --2- 172.21.15.114:0/1419276434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4581016d0 0x7fd458196330 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd454000c00 tx=0x7fd45400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:29:54.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.643+0000 7fd44f7fe700 1 -- 172.21.15.114:0/1419276434 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd45401d070 con 0x7fd4581016d0 2024-02-16T22:29:54.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.643+0000 7fd46077b700 1 -- 172.21.15.114:0/1419276434 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd45819ba20 con 0x7fd4581016d0 2024-02-16T22:29:54.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.643+0000 7fd44f7fe700 1 -- 172.21.15.114:0/1419276434 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd454005dc0 con 0x7fd4581016d0 2024-02-16T22:29:54.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.643+0000 7fd44f7fe700 1 -- 172.21.15.114:0/1419276434 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4540226a0 con 0x7fd4581016d0 2024-02-16T22:29:54.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.643+0000 7fd46077b700 1 -- 172.21.15.114:0/1419276434 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd45819bf10 con 0x7fd4581016d0 2024-02-16T22:29:54.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.644+0000 7fd46077b700 1 -- 172.21.15.114:0/1419276434 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd458190530 con 0x7fd4581016d0 2024-02-16T22:29:54.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.645+0000 7fd44f7fe700 1 -- 172.21.15.114:0/1419276434 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fd4540038f0 con 0x7fd4581016d0 2024-02-16T22:29:54.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.645+0000 7fd44f7fe700 1 --2- 172.21.15.114:0/1419276434 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd44406d8a0 0x7fd44406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:54.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.645+0000 7fd44f7fe700 1 -- 172.21.15.114:0/1419276434 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7fd45408eb30 con 0x7fd4581016d0 2024-02-16T22:29:54.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.645+0000 7fd45dd16700 1 --2- 172.21.15.114:0/1419276434 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd44406d8a0 0x7fd44406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:54.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.646+0000 7fd45dd16700 1 --2- 172.21.15.114:0/1419276434 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd44406d8a0 0x7fd44406fd60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd448011490 tx=0x7fd448009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:29:54.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.650+0000 7fd44f7fe700 1 -- 172.21.15.114:0/1419276434 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd45405bd60 con 0x7fd4581016d0 2024-02-16T22:29:54.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.923+0000 7fd46077b700 1 -- 172.21.15.114:0/1419276434 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd45802d090 con 0x7fd4581016d0 2024-02-16T22:29:54.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.924+0000 7fd44f7fe700 1 -- 172.21.15.114:0/1419276434 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fd454003680 con 0x7fd4581016d0 2024-02-16T22:29:54.926 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:29:54.926 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": { 2024-02-16T22:29:54.926 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:29:54.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:29:54.926 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": { 2024-02-16T22:29:54.926 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:29:54.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:29:54.926 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": { 2024-02-16T22:29:54.926 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T22:29:54.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:29:54.926 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-02-16T22:29:54.927 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T22:29:54.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:29:54.927 INFO:teuthology.orchestra.run.smithi114.stdout: "overall": { 2024-02-16T22:29:54.927 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-16T22:29:54.927 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-02-16T22:29:54.927 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:29:54.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.926+0000 7fd46077b700 1 -- 172.21.15.114:0/1419276434 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd44406d8a0 msgr2=0x7fd44406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:54.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.926+0000 7fd46077b700 1 --2- 172.21.15.114:0/1419276434 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd44406d8a0 0x7fd44406fd60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd448011490 tx=0x7fd448009260 comp rx=0 tx=0).stop 2024-02-16T22:29:54.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.926+0000 7fd46077b700 1 -- 172.21.15.114:0/1419276434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4581016d0 msgr2=0x7fd458196330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:54.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.926+0000 7fd46077b700 1 --2- 172.21.15.114:0/1419276434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4581016d0 0x7fd458196330 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd454000c00 tx=0x7fd45400f690 comp rx=0 tx=0).stop 2024-02-16T22:29:54.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.926+0000 7fd46077b700 1 -- 172.21.15.114:0/1419276434 shutdown_connections 2024-02-16T22:29:54.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.926+0000 7fd46077b700 1 --2- 172.21.15.114:0/1419276434 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd44406d8a0 0x7fd44406fd60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:54.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.926+0000 7fd46077b700 1 --2- 172.21.15.114:0/1419276434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4581016d0 0x7fd458196330 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:54.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.926+0000 7fd46077b700 1 --2- 172.21.15.114:0/1419276434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd458104000 0x7fd458196870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:54.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.926+0000 7fd46077b700 1 -- 172.21.15.114:0/1419276434 >> 172.21.15.114:0/1419276434 conn(0x7fd4580fb060 msgr2=0x7fd4580fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:29:54.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.926+0000 7fd46077b700 1 -- 172.21.15.114:0/1419276434 shutdown_connections 2024-02-16T22:29:54.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:54.926+0000 7fd46077b700 1 -- 172.21.15.114:0/1419276434 wait complete. 2024-02-16T22:29:55.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.028+0000 7f0c40828700 1 -- 172.21.15.114:0/697544999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c3c103fa0 msgr2=0x7f0c3c106390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:55.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.028+0000 7f0c40828700 1 --2- 172.21.15.114:0/697544999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c3c103fa0 0x7f0c3c106390 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009910 tx=0x7f0c30009c20 comp rx=0 tx=0).stop 2024-02-16T22:29:55.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.029+0000 7f0c40828700 1 -- 172.21.15.114:0/697544999 shutdown_connections 2024-02-16T22:29:55.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.029+0000 7f0c40828700 1 --2- 172.21.15.114:0/697544999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c3c103fa0 0x7f0c3c106390 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:55.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.029+0000 7f0c40828700 1 --2- 172.21.15.114:0/697544999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c3c101670 0x7f0c3c103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:55.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.029+0000 7f0c40828700 1 -- 172.21.15.114:0/697544999 >> 172.21.15.114:0/697544999 conn(0x7f0c3c0fb080 msgr2=0x7f0c3c0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:29:55.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.029+0000 7f0c40828700 1 -- 172.21.15.114:0/697544999 shutdown_connections 2024-02-16T22:29:55.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.029+0000 7f0c40828700 1 -- 172.21.15.114:0/697544999 wait complete. 2024-02-16T22:29:55.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.029+0000 7f0c40828700 1 Processor -- start 2024-02-16T22:29:55.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.029+0000 7f0c40828700 1 -- start start 2024-02-16T22:29:55.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.029+0000 7f0c40828700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c3c101670 0x7f0c3c198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:55.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.029+0000 7f0c40828700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c3c103fa0 0x7f0c3c198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:55.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.029+0000 7f0c40828700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c1990a0 con 0x7f0c3c101670 2024-02-16T22:29:55.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.029+0000 7f0c40828700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c1991e0 con 0x7f0c3c103fa0 2024-02-16T22:29:55.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.030+0000 7f0c3a59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c3c101670 0x7f0c3c198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:55.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.030+0000 7f0c3a59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c3c101670 0x7f0c3c198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59888/0 (socket says 172.21.15.114:59888) 2024-02-16T22:29:55.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.030+0000 7f0c3a59c700 1 -- 172.21.15.114:0/341183425 learned_addr learned my addr 172.21.15.114:0/341183425 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:29:55.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.030+0000 7f0c39d9b700 1 --2- 172.21.15.114:0/341183425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c3c103fa0 0x7f0c3c198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:55.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.030+0000 7f0c3a59c700 1 -- 172.21.15.114:0/341183425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c3c103fa0 msgr2=0x7f0c3c198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:55.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.030+0000 7f0c3a59c700 1 --2- 172.21.15.114:0/341183425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c3c103fa0 0x7f0c3c198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:55.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.030+0000 7f0c3a59c700 1 -- 172.21.15.114:0/341183425 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c300095f0 con 0x7f0c3c101670 2024-02-16T22:29:55.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.030+0000 7f0c3a59c700 1 --2- 172.21.15.114:0/341183425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c3c101670 0x7f0c3c198540 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00d8d0 tx=0x7f0c2c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:29:55.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.030+0000 7f0c2b7fe700 1 -- 172.21.15.114:0/341183425 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c2c0098a0 con 0x7f0c3c101670 2024-02-16T22:29:55.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.030+0000 7f0c2b7fe700 1 -- 172.21.15.114:0/341183425 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0c2c010460 con 0x7f0c3c101670 2024-02-16T22:29:55.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.030+0000 7f0c40828700 1 -- 172.21.15.114:0/341183425 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c3c19dc90 con 0x7f0c3c101670 2024-02-16T22:29:55.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.030+0000 7f0c2b7fe700 1 -- 172.21.15.114:0/341183425 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c2c00f5f0 con 0x7f0c3c101670 2024-02-16T22:29:55.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.030+0000 7f0c40828700 1 -- 172.21.15.114:0/341183425 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c3c19e190 con 0x7f0c3c101670 2024-02-16T22:29:55.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.031+0000 7f0c40828700 1 -- 172.21.15.114:0/341183425 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c3c0fcc50 con 0x7f0c3c101670 2024-02-16T22:29:55.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.032+0000 7f0c2b7fe700 1 -- 172.21.15.114:0/341183425 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f0c2c009a00 con 0x7f0c3c101670 2024-02-16T22:29:55.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.032+0000 7f0c2b7fe700 1 --2- 172.21.15.114:0/341183425 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f0c2406d8a0 0x7f0c2406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:55.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.032+0000 7f0c2b7fe700 1 -- 172.21.15.114:0/341183425 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f0c2c08d470 con 0x7f0c3c101670 2024-02-16T22:29:55.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.032+0000 7f0c39d9b700 1 --2- 172.21.15.114:0/341183425 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f0c2406d8a0 0x7f0c2406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:55.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.032+0000 7f0c39d9b700 1 --2- 172.21.15.114:0/341183425 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f0c2406d8a0 0x7f0c2406fd60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0c30006010 tx=0x7f0c3000b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:29:55.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.037+0000 7f0c2b7fe700 1 -- 172.21.15.114:0/341183425 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0c2c05a6a0 con 0x7f0c3c101670 2024-02-16T22:29:55.049 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:54 smithi123 ceph-mon[82331]: pgmap v221: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 170 B/s wr, 7 op/s 2024-02-16T22:29:55.049 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:54 smithi123 ceph-mon[82331]: from='client.34472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:29:55.049 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:54 smithi123 ceph-mon[82331]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:29:55.049 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:54 smithi123 ceph-mon[82331]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:29:55.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:54 smithi114 ceph-mon[94452]: pgmap v221: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 170 B/s wr, 7 op/s 2024-02-16T22:29:55.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:54 smithi114 ceph-mon[94452]: from='client.34472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:29:55.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:54 smithi114 ceph-mon[94452]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:29:55.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:54 smithi114 ceph-mon[94452]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:29:55.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.273+0000 7f0c40828700 1 -- 172.21.15.114:0/341183425 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0c3c061aa0 con 0x7f0c3c101670 2024-02-16T22:29:55.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.274+0000 7f0c2b7fe700 1 -- 172.21.15.114:0/341183425 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f0c2c05a230 con 0x7f0c3c101670 2024-02-16T22:29:55.276 INFO:teuthology.orchestra.run.smithi114.stdout:e37 2024-02-16T22:29:55.276 INFO:teuthology.orchestra.run.smithi114.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:29:55.276 INFO:teuthology.orchestra.run.smithi114.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:29:55.276 INFO:teuthology.orchestra.run.smithi114.stdout:legacy client fscid: 1 2024-02-16T22:29:55.276 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:29:55.276 INFO:teuthology.orchestra.run.smithi114.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:29:55.276 INFO:teuthology.orchestra.run.smithi114.stdout:fs_name cephfs 2024-02-16T22:29:55.276 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 37 2024-02-16T22:29:55.276 INFO:teuthology.orchestra.run.smithi114.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:29:55.276 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-02-16T22:18:02.108825+0000 2024-02-16T22:29:55.276 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-02-16T22:29:48.901292+0000 2024-02-16T22:29:55.276 INFO:teuthology.orchestra.run.smithi114.stdout:tableserver 0 2024-02-16T22:29:55.276 INFO:teuthology.orchestra.run.smithi114.stdout:root 0 2024-02-16T22:29:55.276 INFO:teuthology.orchestra.run.smithi114.stdout:session_timeout 60 2024-02-16T22:29:55.276 INFO:teuthology.orchestra.run.smithi114.stdout:session_autoclose 300 2024-02-16T22:29:55.277 INFO:teuthology.orchestra.run.smithi114.stdout:max_file_size 1099511627776 2024-02-16T22:29:55.277 INFO:teuthology.orchestra.run.smithi114.stdout:max_xattr_size 65536 2024-02-16T22:29:55.277 INFO:teuthology.orchestra.run.smithi114.stdout:required_client_features {} 2024-02-16T22:29:55.277 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure 0 2024-02-16T22:29:55.277 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure_osd_epoch 82 2024-02-16T22:29:55.277 INFO:teuthology.orchestra.run.smithi114.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-16T22:29:55.277 INFO:teuthology.orchestra.run.smithi114.stdout:max_mds 1 2024-02-16T22:29:55.277 INFO:teuthology.orchestra.run.smithi114.stdout:in 0 2024-02-16T22:29:55.277 INFO:teuthology.orchestra.run.smithi114.stdout:up {0=34404} 2024-02-16T22:29:55.277 INFO:teuthology.orchestra.run.smithi114.stdout:failed 2024-02-16T22:29:55.277 INFO:teuthology.orchestra.run.smithi114.stdout:damaged 2024-02-16T22:29:55.277 INFO:teuthology.orchestra.run.smithi114.stdout:stopped 2024-02-16T22:29:55.277 INFO:teuthology.orchestra.run.smithi114.stdout:data_pools [3] 2024-02-16T22:29:55.277 INFO:teuthology.orchestra.run.smithi114.stdout:metadata_pool 2 2024-02-16T22:29:55.278 INFO:teuthology.orchestra.run.smithi114.stdout:inline_data enabled 2024-02-16T22:29:55.278 INFO:teuthology.orchestra.run.smithi114.stdout:balancer 2024-02-16T22:29:55.278 INFO:teuthology.orchestra.run.smithi114.stdout:bal_rank_mask -1 2024-02-16T22:29:55.278 INFO:teuthology.orchestra.run.smithi114.stdout:standby_count_wanted 1 2024-02-16T22:29:55.278 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.zzryke{0:34404} state up:active seq 23 join_fscid=1 addr [v2:172.21.15.114:6826/2017800151,v1:172.21.15.114:6827/2017800151] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T22:29:55.278 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:29:55.278 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:29:55.278 INFO:teuthology.orchestra.run.smithi114.stdout:Standby daemons: 2024-02-16T22:29:55.278 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:29:55.278 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.fdxpey{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.114:6828/3702914193,v1:172.21.15.114:6829/3702914193] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T22:29:55.278 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.vhlqbe{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.123:6824/1425065679,v1:172.21.15.123:6825/1425065679] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T22:29:55.278 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.gopcfa{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.123:6826/3128771108,v1:172.21.15.123:6827/3128771108] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T22:29:55.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.276+0000 7f0c40828700 1 -- 172.21.15.114:0/341183425 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f0c2406d8a0 msgr2=0x7f0c2406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:55.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.276+0000 7f0c40828700 1 --2- 172.21.15.114:0/341183425 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f0c2406d8a0 0x7f0c2406fd60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0c30006010 tx=0x7f0c3000b540 comp rx=0 tx=0).stop 2024-02-16T22:29:55.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.276+0000 7f0c40828700 1 -- 172.21.15.114:0/341183425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c3c101670 msgr2=0x7f0c3c198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:55.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.276+0000 7f0c40828700 1 --2- 172.21.15.114:0/341183425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c3c101670 0x7f0c3c198540 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00d8d0 tx=0x7f0c2c00dbe0 comp rx=0 tx=0).stop 2024-02-16T22:29:55.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.277+0000 7f0c40828700 1 -- 172.21.15.114:0/341183425 shutdown_connections 2024-02-16T22:29:55.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.277+0000 7f0c40828700 1 --2- 172.21.15.114:0/341183425 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f0c2406d8a0 0x7f0c2406fd60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:55.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.277+0000 7f0c40828700 1 --2- 172.21.15.114:0/341183425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c3c101670 0x7f0c3c198540 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:55.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.277+0000 7f0c40828700 1 --2- 172.21.15.114:0/341183425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c3c103fa0 0x7f0c3c198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:55.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.277+0000 7f0c40828700 1 -- 172.21.15.114:0/341183425 >> 172.21.15.114:0/341183425 conn(0x7f0c3c0fb080 msgr2=0x7f0c3c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:29:55.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.277+0000 7f0c40828700 1 -- 172.21.15.114:0/341183425 shutdown_connections 2024-02-16T22:29:55.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.277+0000 7f0c40828700 1 -- 172.21.15.114:0/341183425 wait complete. 2024-02-16T22:29:55.280 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 37 2024-02-16T22:29:55.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.379+0000 7f4e40f0a700 1 -- 172.21.15.114:0/1052690852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e3c100460 msgr2=0x7f4e3c1008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:55.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.379+0000 7f4e40f0a700 1 --2- 172.21.15.114:0/1052690852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e3c100460 0x7f4e3c1008e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4e24009910 tx=0x7f4e24009c20 comp rx=0 tx=0).stop 2024-02-16T22:29:55.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.379+0000 7f4e40f0a700 1 -- 172.21.15.114:0/1052690852 shutdown_connections 2024-02-16T22:29:55.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.379+0000 7f4e40f0a700 1 --2- 172.21.15.114:0/1052690852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e3c100460 0x7f4e3c1008e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:55.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.379+0000 7f4e40f0a700 1 --2- 172.21.15.114:0/1052690852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e3c0ff3a0 0x7f4e3c0ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:55.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.379+0000 7f4e40f0a700 1 -- 172.21.15.114:0/1052690852 >> 172.21.15.114:0/1052690852 conn(0x7f4e3c0fa380 msgr2=0x7f4e3c0fc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:29:55.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.379+0000 7f4e40f0a700 1 -- 172.21.15.114:0/1052690852 shutdown_connections 2024-02-16T22:29:55.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.379+0000 7f4e40f0a700 1 -- 172.21.15.114:0/1052690852 wait complete. 2024-02-16T22:29:55.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.380+0000 7f4e40f0a700 1 Processor -- start 2024-02-16T22:29:55.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.380+0000 7f4e40f0a700 1 -- start start 2024-02-16T22:29:55.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.380+0000 7f4e40f0a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e3c0ff3a0 0x7f4e3c102240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:55.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.380+0000 7f4e40f0a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e3c100460 0x7f4e3c19b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:55.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.380+0000 7f4e40f0a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c19b6d0 con 0x7f4e3c0ff3a0 2024-02-16T22:29:55.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.380+0000 7f4e40f0a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c19b810 con 0x7f4e3c100460 2024-02-16T22:29:55.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.380+0000 7f4e3a59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e3c0ff3a0 0x7f4e3c102240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:55.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.380+0000 7f4e3a59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e3c0ff3a0 0x7f4e3c102240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59900/0 (socket says 172.21.15.114:59900) 2024-02-16T22:29:55.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.380+0000 7f4e3a59c700 1 -- 172.21.15.114:0/4002033586 learned_addr learned my addr 172.21.15.114:0/4002033586 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:29:55.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.381+0000 7f4e39d9b700 1 --2- 172.21.15.114:0/4002033586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e3c100460 0x7f4e3c19b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:55.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.381+0000 7f4e3a59c700 1 -- 172.21.15.114:0/4002033586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e3c100460 msgr2=0x7f4e3c19b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:55.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.381+0000 7f4e3a59c700 1 --2- 172.21.15.114:0/4002033586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e3c100460 0x7f4e3c19b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:55.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.381+0000 7f4e3a59c700 1 -- 172.21.15.114:0/4002033586 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e240095f0 con 0x7f4e3c0ff3a0 2024-02-16T22:29:55.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.381+0000 7f4e3a59c700 1 --2- 172.21.15.114:0/4002033586 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e3c0ff3a0 0x7f4e3c102240 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c00b4b0 tx=0x7f4e2c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:29:55.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.381+0000 7f4e337fe700 1 -- 172.21.15.114:0/4002033586 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e2c011800 con 0x7f4e3c0ff3a0 2024-02-16T22:29:55.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.381+0000 7f4e40f0a700 1 -- 172.21.15.114:0/4002033586 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e3c19baf0 con 0x7f4e3c0ff3a0 2024-02-16T22:29:55.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.381+0000 7f4e337fe700 1 -- 172.21.15.114:0/4002033586 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4e2c011e10 con 0x7f4e3c0ff3a0 2024-02-16T22:29:55.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.381+0000 7f4e337fe700 1 -- 172.21.15.114:0/4002033586 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e2c00f2e0 con 0x7f4e3c0ff3a0 2024-02-16T22:29:55.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.381+0000 7f4e40f0a700 1 -- 172.21.15.114:0/4002033586 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e3c19c040 con 0x7f4e3c0ff3a0 2024-02-16T22:29:55.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.382+0000 7f4e40f0a700 1 -- 172.21.15.114:0/4002033586 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e3c107e10 con 0x7f4e3c0ff3a0 2024-02-16T22:29:55.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.383+0000 7f4e337fe700 1 -- 172.21.15.114:0/4002033586 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f4e2c010380 con 0x7f4e3c0ff3a0 2024-02-16T22:29:55.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.383+0000 7f4e337fe700 1 --2- 172.21.15.114:0/4002033586 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f4e2806d8a0 0x7f4e2806fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:55.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.383+0000 7f4e337fe700 1 -- 172.21.15.114:0/4002033586 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f4e2c08d190 con 0x7f4e3c0ff3a0 2024-02-16T22:29:55.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.383+0000 7f4e39d9b700 1 --2- 172.21.15.114:0/4002033586 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f4e2806d8a0 0x7f4e2806fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:55.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.384+0000 7f4e39d9b700 1 --2- 172.21.15.114:0/4002033586 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f4e2806d8a0 0x7f4e2806fd60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4e24006010 tx=0x7f4e2400b560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:29:55.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.388+0000 7f4e337fe700 1 -- 172.21.15.114:0/4002033586 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4e2c05a3c0 con 0x7f4e3c0ff3a0 2024-02-16T22:29:55.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.600+0000 7f4e40f0a700 1 -- 172.21.15.114:0/4002033586 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e3c1047a0 con 0x7f4e2806d8a0 2024-02-16T22:29:55.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.602+0000 7f4e337fe700 1 -- 172.21.15.114:0/4002033586 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4e3c1047a0 con 0x7f4e2806d8a0 2024-02-16T22:29:55.604 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:29:55.604 INFO:teuthology.orchestra.run.smithi114.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:29:55.604 INFO:teuthology.orchestra.run.smithi114.stdout: "in_progress": true, 2024-02-16T22:29:55.604 INFO:teuthology.orchestra.run.smithi114.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:29:55.604 INFO:teuthology.orchestra.run.smithi114.stdout: "services_complete": [ 2024-02-16T22:29:55.604 INFO:teuthology.orchestra.run.smithi114.stdout: "osd", 2024-02-16T22:29:55.604 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr", 2024-02-16T22:29:55.605 INFO:teuthology.orchestra.run.smithi114.stdout: "mon", 2024-02-16T22:29:55.605 INFO:teuthology.orchestra.run.smithi114.stdout: "mds", 2024-02-16T22:29:55.605 INFO:teuthology.orchestra.run.smithi114.stdout: "crash" 2024-02-16T22:29:55.605 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-02-16T22:29:55.605 INFO:teuthology.orchestra.run.smithi114.stdout: "progress": "16/23 daemons upgraded", 2024-02-16T22:29:55.605 INFO:teuthology.orchestra.run.smithi114.stdout: "message": "Currently upgrading mds daemons", 2024-02-16T22:29:55.605 INFO:teuthology.orchestra.run.smithi114.stdout: "is_paused": false 2024-02-16T22:29:55.605 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:29:55.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.604+0000 7f4e40f0a700 1 -- 172.21.15.114:0/4002033586 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f4e2806d8a0 msgr2=0x7f4e2806fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:55.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.605+0000 7f4e40f0a700 1 --2- 172.21.15.114:0/4002033586 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f4e2806d8a0 0x7f4e2806fd60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4e24006010 tx=0x7f4e2400b560 comp rx=0 tx=0).stop 2024-02-16T22:29:55.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.605+0000 7f4e40f0a700 1 -- 172.21.15.114:0/4002033586 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e3c0ff3a0 msgr2=0x7f4e3c102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:55.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.605+0000 7f4e40f0a700 1 --2- 172.21.15.114:0/4002033586 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e3c0ff3a0 0x7f4e3c102240 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c00b4b0 tx=0x7f4e2c00b7c0 comp rx=0 tx=0).stop 2024-02-16T22:29:55.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.605+0000 7f4e40f0a700 1 -- 172.21.15.114:0/4002033586 shutdown_connections 2024-02-16T22:29:55.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.605+0000 7f4e40f0a700 1 --2- 172.21.15.114:0/4002033586 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f4e2806d8a0 0x7f4e2806fd60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:55.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.605+0000 7f4e40f0a700 1 --2- 172.21.15.114:0/4002033586 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e3c0ff3a0 0x7f4e3c102240 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:55.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.605+0000 7f4e40f0a700 1 --2- 172.21.15.114:0/4002033586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e3c100460 0x7f4e3c19b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:55.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.605+0000 7f4e40f0a700 1 -- 172.21.15.114:0/4002033586 >> 172.21.15.114:0/4002033586 conn(0x7f4e3c0fa380 msgr2=0x7f4e3c0fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:29:55.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.605+0000 7f4e40f0a700 1 -- 172.21.15.114:0/4002033586 shutdown_connections 2024-02-16T22:29:55.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.605+0000 7f4e40f0a700 1 -- 172.21.15.114:0/4002033586 wait complete. 2024-02-16T22:29:55.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.707+0000 7fb604617700 1 -- 172.21.15.114:0/3022341821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5fc101e00 msgr2=0x7fb5fc102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:55.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.707+0000 7fb604617700 1 --2- 172.21.15.114:0/3022341821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5fc101e00 0x7fb5fc102260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec009910 tx=0x7fb5ec009c20 comp rx=0 tx=0).stop 2024-02-16T22:29:55.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.707+0000 7fb604617700 1 -- 172.21.15.114:0/3022341821 shutdown_connections 2024-02-16T22:29:55.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.707+0000 7fb604617700 1 --2- 172.21.15.114:0/3022341821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5fc101e00 0x7fb5fc102260 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:55.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.707+0000 7fb604617700 1 --2- 172.21.15.114:0/3022341821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5fc100c00 0x7fb5fc101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:55.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.707+0000 7fb604617700 1 -- 172.21.15.114:0/3022341821 >> 172.21.15.114:0/3022341821 conn(0x7fb5fc0fc180 msgr2=0x7fb5fc0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:29:55.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.707+0000 7fb604617700 1 -- 172.21.15.114:0/3022341821 shutdown_connections 2024-02-16T22:29:55.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.707+0000 7fb604617700 1 -- 172.21.15.114:0/3022341821 wait complete. 2024-02-16T22:29:55.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.707+0000 7fb604617700 1 Processor -- start 2024-02-16T22:29:55.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.708+0000 7fb604617700 1 -- start start 2024-02-16T22:29:55.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.708+0000 7fb604617700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5fc100c00 0x7fb5fc196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:55.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.708+0000 7fb604617700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5fc101e00 0x7fb5fc1968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:55.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.708+0000 7fb604617700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc196ee0 con 0x7fb5fc100c00 2024-02-16T22:29:55.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.708+0000 7fb604617700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc197020 con 0x7fb5fc101e00 2024-02-16T22:29:55.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.708+0000 7fb6023b3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5fc100c00 0x7fb5fc196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:55.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.708+0000 7fb6023b3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5fc100c00 0x7fb5fc196380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59906/0 (socket says 172.21.15.114:59906) 2024-02-16T22:29:55.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.708+0000 7fb6023b3700 1 -- 172.21.15.114:0/1898751827 learned_addr learned my addr 172.21.15.114:0/1898751827 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:29:55.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.708+0000 7fb601bb2700 1 --2- 172.21.15.114:0/1898751827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5fc101e00 0x7fb5fc1968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:55.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.708+0000 7fb6023b3700 1 -- 172.21.15.114:0/1898751827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5fc101e00 msgr2=0x7fb5fc1968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:55.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.708+0000 7fb6023b3700 1 --2- 172.21.15.114:0/1898751827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5fc101e00 0x7fb5fc1968c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:55.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.708+0000 7fb6023b3700 1 -- 172.21.15.114:0/1898751827 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ec0095f0 con 0x7fb5fc100c00 2024-02-16T22:29:55.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.708+0000 7fb6023b3700 1 --2- 172.21.15.114:0/1898751827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5fc100c00 0x7fb5fc196380 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb5f800d900 tx=0x7fb5f800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:29:55.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.708+0000 7fb5f37fe700 1 -- 172.21.15.114:0/1898751827 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5f800b920 con 0x7fb5fc100c00 2024-02-16T22:29:55.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.708+0000 7fb5f37fe700 1 -- 172.21.15.114:0/1898751827 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb5f8010460 con 0x7fb5fc100c00 2024-02-16T22:29:55.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.708+0000 7fb5f37fe700 1 -- 172.21.15.114:0/1898751827 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5f800f590 con 0x7fb5fc100c00 2024-02-16T22:29:55.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.708+0000 7fb604617700 1 -- 172.21.15.114:0/1898751827 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5fc19bad0 con 0x7fb5fc100c00 2024-02-16T22:29:55.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.709+0000 7fb604617700 1 -- 172.21.15.114:0/1898751827 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5fc19c020 con 0x7fb5fc100c00 2024-02-16T22:29:55.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.709+0000 7fb604617700 1 -- 172.21.15.114:0/1898751827 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5fc109760 con 0x7fb5fc100c00 2024-02-16T22:29:55.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.710+0000 7fb5f37fe700 1 -- 172.21.15.114:0/1898751827 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fb5f80105d0 con 0x7fb5fc100c00 2024-02-16T22:29:55.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.710+0000 7fb5f37fe700 1 --2- 172.21.15.114:0/1898751827 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb5e806d8a0 0x7fb5e806fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:29:55.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.710+0000 7fb5f37fe700 1 -- 172.21.15.114:0/1898751827 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7fb5f808d5c0 con 0x7fb5fc100c00 2024-02-16T22:29:55.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.710+0000 7fb601bb2700 1 --2- 172.21.15.114:0/1898751827 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb5e806d8a0 0x7fb5e806fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:29:55.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.711+0000 7fb601bb2700 1 --2- 172.21.15.114:0/1898751827 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb5e806d8a0 0x7fb5e806fd60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec006010 tx=0x7fb5ec00b560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:29:55.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.715+0000 7fb5f37fe700 1 -- 172.21.15.114:0/1898751827 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb5f805a7f0 con 0x7fb5fc100c00 2024-02-16T22:29:55.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.990+0000 7fb604617700 1 -- 172.21.15.114:0/1898751827 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb5fc109950 con 0x7fb5fc100c00 2024-02-16T22:29:55.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.990+0000 7fb5f37fe700 1 -- 172.21.15.114:0/1898751827 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fb5f805a380 con 0x7fb5fc100c00 2024-02-16T22:29:55.992 INFO:teuthology.orchestra.run.smithi114.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:29:55.992 INFO:teuthology.orchestra.run.smithi114.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:29:55.992 INFO:teuthology.orchestra.run.smithi114.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:29:55.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.992+0000 7fb604617700 1 -- 172.21.15.114:0/1898751827 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb5e806d8a0 msgr2=0x7fb5e806fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:55.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.992+0000 7fb604617700 1 --2- 172.21.15.114:0/1898751827 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb5e806d8a0 0x7fb5e806fd60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec006010 tx=0x7fb5ec00b560 comp rx=0 tx=0).stop 2024-02-16T22:29:55.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.992+0000 7fb604617700 1 -- 172.21.15.114:0/1898751827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5fc100c00 msgr2=0x7fb5fc196380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:29:55.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.992+0000 7fb604617700 1 --2- 172.21.15.114:0/1898751827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5fc100c00 0x7fb5fc196380 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb5f800d900 tx=0x7fb5f800dc10 comp rx=0 tx=0).stop 2024-02-16T22:29:55.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.992+0000 7fb604617700 1 -- 172.21.15.114:0/1898751827 shutdown_connections 2024-02-16T22:29:55.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.992+0000 7fb604617700 1 --2- 172.21.15.114:0/1898751827 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb5e806d8a0 0x7fb5e806fd60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:55.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.992+0000 7fb604617700 1 --2- 172.21.15.114:0/1898751827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5fc100c00 0x7fb5fc196380 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:55.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.992+0000 7fb604617700 1 --2- 172.21.15.114:0/1898751827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5fc101e00 0x7fb5fc1968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:29:55.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.992+0000 7fb604617700 1 -- 172.21.15.114:0/1898751827 >> 172.21.15.114:0/1898751827 conn(0x7fb5fc0fc180 msgr2=0x7fb5fc105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:29:55.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.992+0000 7fb604617700 1 -- 172.21.15.114:0/1898751827 shutdown_connections 2024-02-16T22:29:55.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:29:55.992+0000 7fb604617700 1 -- 172.21.15.114:0/1898751827 wait complete. 2024-02-16T22:29:56.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:55 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/1419276434' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:56.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:55 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/341183425' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:29:56.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:55 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/1419276434' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:56.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:55 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/341183425' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:29:57.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:56 smithi114 ceph-mon[94452]: from='client.34492 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:29:57.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:56 smithi114 ceph-mon[94452]: pgmap v222: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 170 B/s wr, 6 op/s 2024-02-16T22:29:57.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:56 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/1898751827' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:29:57.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:56 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:57.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:56 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:57.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:56 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:29:57.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:56 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:29:57.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:56 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:57.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:56 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:29:57.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:56 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:57.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:56 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:57.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:56 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:57.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:56 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:57.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:56 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:57.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:56 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:57.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:56 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi114.fdxpey"}]: dispatch 2024-02-16T22:29:57.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:56 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi114.fdxpey"}]': finished 2024-02-16T22:29:57.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:56 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi114.zzryke"}]: dispatch 2024-02-16T22:29:57.203 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:56 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi114.zzryke"}]': finished 2024-02-16T22:29:57.204 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:56 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi123.gopcfa"}]: dispatch 2024-02-16T22:29:57.204 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:56 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi123.gopcfa"}]': finished 2024-02-16T22:29:57.204 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:56 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi123.vhlqbe"}]: dispatch 2024-02-16T22:29:57.204 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:56 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi123.vhlqbe"}]': finished 2024-02-16T22:29:57.204 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:56 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:57.204 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:56 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:57.204 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:56 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:57.204 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:56 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:57.204 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:56 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:57.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:56 smithi123 ceph-mon[82331]: from='client.34492 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:29:57.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:56 smithi123 ceph-mon[82331]: pgmap v222: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 170 B/s wr, 6 op/s 2024-02-16T22:29:57.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:56 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/1898751827' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:29:57.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:56 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:57.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:56 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:57.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:56 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:29:57.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:56 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:29:57.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:56 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:57.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:56 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:29:57.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:56 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:57.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:56 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:57.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:56 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:57.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:56 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:57.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:56 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:57.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:56 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:57.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:56 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi114.fdxpey"}]: dispatch 2024-02-16T22:29:57.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:56 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi114.fdxpey"}]': finished 2024-02-16T22:29:57.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:56 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi114.zzryke"}]: dispatch 2024-02-16T22:29:57.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:56 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi114.zzryke"}]': finished 2024-02-16T22:29:57.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:56 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi123.gopcfa"}]: dispatch 2024-02-16T22:29:57.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:56 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi123.gopcfa"}]': finished 2024-02-16T22:29:57.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:56 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi123.vhlqbe"}]: dispatch 2024-02-16T22:29:57.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:56 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi123.vhlqbe"}]': finished 2024-02-16T22:29:57.304 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:56 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:57.305 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:56 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:57.305 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:56 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:57.305 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:56 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:57.305 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:56 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:29:58.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:57 smithi123 ceph-mon[82331]: Upgrade: Setting container_image for all mds 2024-02-16T22:29:58.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:57 smithi123 ceph-mon[82331]: Upgrade: Setting container_image for all rgw 2024-02-16T22:29:58.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:57 smithi123 ceph-mon[82331]: Upgrade: Setting container_image for all rbd-mirror 2024-02-16T22:29:58.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:57 smithi114 ceph-mon[94452]: Upgrade: Setting container_image for all mds 2024-02-16T22:29:58.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:57 smithi114 ceph-mon[94452]: Upgrade: Setting container_image for all rgw 2024-02-16T22:29:58.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:57 smithi114 ceph-mon[94452]: Upgrade: Setting container_image for all rbd-mirror 2024-02-16T22:29:59.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:59 smithi114 ceph-mon[94452]: pgmap v223: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 5.5 MiB/s rd, 4.1 KiB/s wr, 6 op/s 2024-02-16T22:29:59.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:59 smithi114 ceph-mon[94452]: Upgrade: Updating ceph-exporter.smithi114 (1/2) 2024-02-16T22:29:59.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:59 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:59.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:59 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi114", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T22:29:59.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:59 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:29:59.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:29:59 smithi114 ceph-mon[94452]: Deploying daemon ceph-exporter.smithi114 on smithi114 2024-02-16T22:29:59.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:59 smithi123 ceph-mon[82331]: pgmap v223: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 5.5 MiB/s rd, 4.1 KiB/s wr, 6 op/s 2024-02-16T22:29:59.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:59 smithi123 ceph-mon[82331]: Upgrade: Updating ceph-exporter.smithi114 (1/2) 2024-02-16T22:29:59.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:59 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:29:59.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:59 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi114", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T22:29:59.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:59 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:29:59.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:29:59 smithi123 ceph-mon[82331]: Deploying daemon ceph-exporter.smithi114 on smithi114 2024-02-16T22:30:00.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:00 smithi123 ceph-mon[82331]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:30:00.584 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:00 smithi114 ceph-mon[94452]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:30:01.411 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:01 smithi114 ceph-mon[94452]: pgmap v224: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 4.1 KiB/s wr, 5 op/s 2024-02-16T22:30:01.412 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:01.412 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:30:01.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:01 smithi123 ceph-mon[82331]: pgmap v224: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 4.1 KiB/s wr, 5 op/s 2024-02-16T22:30:01.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:01 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:01.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:01 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:30:03.524 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:03 smithi114 ceph-mon[94452]: pgmap v225: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 4.0 KiB/s wr, 5 op/s 2024-02-16T22:30:03.524 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:03 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:03.524 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:03 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:03.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:03 smithi123 ceph-mon[82331]: pgmap v225: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 4.0 KiB/s wr, 5 op/s 2024-02-16T22:30:03.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:03 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:03.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:03 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:05.285 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:05 smithi123 ceph-mon[82331]: pgmap v226: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 3.9 KiB/s wr, 4 op/s 2024-02-16T22:30:05.285 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:05 smithi123 ceph-mon[82331]: Upgrade: Updating ceph-exporter.smithi123 (2/2) 2024-02-16T22:30:05.285 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:05 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:05.285 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:05 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi123", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T22:30:05.285 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:05 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:30:05.285 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:05 smithi123 ceph-mon[82331]: Deploying daemon ceph-exporter.smithi123 on smithi123 2024-02-16T22:30:05.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:05 smithi114 ceph-mon[94452]: pgmap v226: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 3.9 KiB/s wr, 4 op/s 2024-02-16T22:30:05.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:05 smithi114 ceph-mon[94452]: Upgrade: Updating ceph-exporter.smithi123 (2/2) 2024-02-16T22:30:05.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:05 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:05.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:05 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi123", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-16T22:30:05.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:05 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:30:05.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:05 smithi114 ceph-mon[94452]: Deploying daemon ceph-exporter.smithi123 on smithi123 2024-02-16T22:30:07.199 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:07 smithi123 ceph-mon[82331]: pgmap v227: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 3.9 KiB/s wr, 2 op/s 2024-02-16T22:30:07.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:07 smithi114 ceph-mon[94452]: pgmap v227: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 3.9 KiB/s wr, 2 op/s 2024-02-16T22:30:08.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:08 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:08.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:08 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:08.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:08 smithi114 ceph-mon[94452]: pgmap v228: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 3.9 KiB/s wr, 1 op/s 2024-02-16T22:30:08.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:08 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:30:09.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:08 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:09.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:08 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:09.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:08 smithi123 ceph-mon[82331]: pgmap v228: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 3.9 KiB/s wr, 1 op/s 2024-02-16T22:30:09.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:08 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:30:11.443 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:11 smithi114 ceph-mon[94452]: pgmap v229: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:11.443 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:11 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:11.443 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:11 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:11.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:11 smithi123 ceph-mon[82331]: pgmap v229: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:11.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:11 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:11.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:11 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:12.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:12 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:12.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:12 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:12.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:12 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:12.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:12 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:13.279 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:13 smithi123 ceph-mon[82331]: pgmap v230: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:13.279 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:13 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:13.279 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:13 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:13.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:13 smithi114 ceph-mon[94452]: pgmap v230: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:13.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:13 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:13.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:13 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:14.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:14 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:14.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:14 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:14.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:14 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:14.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:14 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:15.349 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:15 smithi114 ceph-mon[94452]: pgmap v231: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:15.349 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:15 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:15.349 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:15 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:15.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:15 smithi123 ceph-mon[82331]: pgmap v231: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:15.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:15 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:15.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:15 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:16.394 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:16 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:30:16.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:16 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:30:17.472 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:17 smithi114 ceph-mon[94452]: pgmap v232: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:17.473 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:17 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:17.473 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:17 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:17.473 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:17 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:30:17.473 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:17 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:30:17.473 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:17 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:17.473 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:17 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:30:17.473 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:17 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:17.473 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:17 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:17.473 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:17 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:17.473 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:17 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:17.473 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:17 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:17.473 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:17 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:17.473 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:17 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:17.474 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:17 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:17.474 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:17 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:17.474 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:17 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:17.474 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:17 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi114"}]: dispatch 2024-02-16T22:30:17.474 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:17 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi114"}]': finished 2024-02-16T22:30:17.474 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:17 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi123"}]: dispatch 2024-02-16T22:30:17.474 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:17 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi123"}]': finished 2024-02-16T22:30:17.474 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:17 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:17.474 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:17 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:17.474 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:17 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:17.474 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:17 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:17.474 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:17 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:17.474 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:17 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:17.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:17 smithi123 ceph-mon[82331]: pgmap v232: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:17.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:17 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:17.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:17 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:17.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:17 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:30:17.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:17 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:30:17.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:17 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:17.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:17 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:30:17.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:17 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:17.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:17 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:17.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:17 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:17.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:17 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:17.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:17 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:17.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:17 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:17.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:17 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:17.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:17 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:17.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:17 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:17.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:17 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:17.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:17 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi114"}]: dispatch 2024-02-16T22:30:17.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:17 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi114"}]': finished 2024-02-16T22:30:17.553 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:17 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi123"}]: dispatch 2024-02-16T22:30:17.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:17 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi123"}]': finished 2024-02-16T22:30:17.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:17 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:17.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:17 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:17.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:17 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:17.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:17 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:17.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:17 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:17.554 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:17 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:18.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:18 smithi123 ceph-mon[82331]: Upgrade: Setting container_image for all ceph-exporter 2024-02-16T22:30:18.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:18 smithi123 ceph-mon[82331]: Upgrade: Setting container_image for all iscsi 2024-02-16T22:30:18.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:18 smithi123 ceph-mon[82331]: Upgrade: Setting container_image for all nfs 2024-02-16T22:30:18.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:18 smithi123 ceph-mon[82331]: Upgrade: Setting container_image for all nvmeof 2024-02-16T22:30:18.628 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:18 smithi114 ceph-mon[94452]: Upgrade: Setting container_image for all ceph-exporter 2024-02-16T22:30:18.628 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:18 smithi114 ceph-mon[94452]: Upgrade: Setting container_image for all iscsi 2024-02-16T22:30:18.628 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:18 smithi114 ceph-mon[94452]: Upgrade: Setting container_image for all nfs 2024-02-16T22:30:18.628 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:18 smithi114 ceph-mon[94452]: Upgrade: Setting container_image for all nvmeof 2024-02-16T22:30:19.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:19 smithi114 ceph-mon[94452]: pgmap v233: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:19.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:19 smithi123 ceph-mon[82331]: pgmap v233: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:20.502 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:20 smithi114 ceph-mon[94452]: Upgrade: Updating node-exporter.smithi114 (1/2) 2024-02-16T22:30:20.502 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:20 smithi114 ceph-mon[94452]: Deploying daemon node-exporter.smithi114 on smithi114 2024-02-16T22:30:20.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:20 smithi123 ceph-mon[82331]: Upgrade: Updating node-exporter.smithi114 (1/2) 2024-02-16T22:30:20.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:20 smithi123 ceph-mon[82331]: Deploying daemon node-exporter.smithi114 on smithi114 2024-02-16T22:30:21.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:21 smithi123 ceph-mon[82331]: pgmap v234: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:21.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:21 smithi114 ceph-mon[94452]: pgmap v234: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:22.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:22 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:22.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:22 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:22.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:22 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:22.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:22 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:23.644 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:23 smithi123 ceph-mon[82331]: pgmap v235: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:23.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:23 smithi114 ceph-mon[94452]: pgmap v235: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:24.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:24 smithi123 ceph-mon[82331]: Upgrade: Updating node-exporter.smithi123 (2/2) 2024-02-16T22:30:24.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:24 smithi123 ceph-mon[82331]: Deploying daemon node-exporter.smithi123 on smithi123 2024-02-16T22:30:24.704 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:24 smithi114 ceph-mon[94452]: Upgrade: Updating node-exporter.smithi123 (2/2) 2024-02-16T22:30:24.705 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:24 smithi114 ceph-mon[94452]: Deploying daemon node-exporter.smithi123 on smithi123 2024-02-16T22:30:25.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:25 smithi114 ceph-mon[94452]: pgmap v236: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:25.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:25 smithi123 ceph-mon[82331]: pgmap v236: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:26.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.104+0000 7f8a99c6d700 1 -- 172.21.15.114:0/301443039 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a940ffd20 msgr2=0x7f8a941001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:26.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.104+0000 7f8a99c6d700 1 --2- 172.21.15.114:0/301443039 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a940ffd20 0x7f8a941001a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8a88009910 tx=0x7f8a88009c20 comp rx=0 tx=0).stop 2024-02-16T22:30:26.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.104+0000 7f8a99c6d700 1 -- 172.21.15.114:0/301443039 shutdown_connections 2024-02-16T22:30:26.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.104+0000 7f8a99c6d700 1 --2- 172.21.15.114:0/301443039 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a940ffd20 0x7f8a941001a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:26.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.104+0000 7f8a99c6d700 1 --2- 172.21.15.114:0/301443039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a94101150 0x7f8a94101570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:26.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.104+0000 7f8a99c6d700 1 -- 172.21.15.114:0/301443039 >> 172.21.15.114:0/301443039 conn(0x7f8a940fa7f0 msgr2=0x7f8a940fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:30:26.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.105+0000 7f8a99c6d700 1 -- 172.21.15.114:0/301443039 shutdown_connections 2024-02-16T22:30:26.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.105+0000 7f8a99c6d700 1 -- 172.21.15.114:0/301443039 wait complete. 2024-02-16T22:30:26.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.106+0000 7f8a99c6d700 1 Processor -- start 2024-02-16T22:30:26.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.106+0000 7f8a99c6d700 1 -- start start 2024-02-16T22:30:26.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.106+0000 7f8a99c6d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a940ffd20 0x7f8a94196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:26.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.106+0000 7f8a99c6d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a94101150 0x7f8a941968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:26.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.106+0000 7f8a99c6d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a94196ee0 con 0x7f8a94101150 2024-02-16T22:30:26.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.106+0000 7f8a99c6d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a94197020 con 0x7f8a940ffd20 2024-02-16T22:30:26.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.107+0000 7f8a92ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a94101150 0x7f8a941968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:26.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.107+0000 7f8a92ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a94101150 0x7f8a941968c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53934/0 (socket says 172.21.15.114:53934) 2024-02-16T22:30:26.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.107+0000 7f8a92ffd700 1 -- 172.21.15.114:0/3831488317 learned_addr learned my addr 172.21.15.114:0/3831488317 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:30:26.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.107+0000 7f8a937fe700 1 --2- 172.21.15.114:0/3831488317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a940ffd20 0x7f8a94196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:26.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.107+0000 7f8a92ffd700 1 -- 172.21.15.114:0/3831488317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a940ffd20 msgr2=0x7f8a94196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:26.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.107+0000 7f8a92ffd700 1 --2- 172.21.15.114:0/3831488317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a940ffd20 0x7f8a94196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:26.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.107+0000 7f8a92ffd700 1 -- 172.21.15.114:0/3831488317 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a880095f0 con 0x7f8a94101150 2024-02-16T22:30:26.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.107+0000 7f8a92ffd700 1 --2- 172.21.15.114:0/3831488317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a94101150 0x7f8a941968c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8a88000c00 tx=0x7f8a8800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:30:26.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.107+0000 7f8a90ff9700 1 -- 172.21.15.114:0/3831488317 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a8801d070 con 0x7f8a94101150 2024-02-16T22:30:26.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.107+0000 7f8a99c6d700 1 -- 172.21.15.114:0/3831488317 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a9419ba70 con 0x7f8a94101150 2024-02-16T22:30:26.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.107+0000 7f8a90ff9700 1 -- 172.21.15.114:0/3831488317 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8a88005dc0 con 0x7f8a94101150 2024-02-16T22:30:26.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.107+0000 7f8a90ff9700 1 -- 172.21.15.114:0/3831488317 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a880226a0 con 0x7f8a94101150 2024-02-16T22:30:26.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.108+0000 7f8a99c6d700 1 -- 172.21.15.114:0/3831488317 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a9419bf60 con 0x7f8a94101150 2024-02-16T22:30:26.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.109+0000 7f8a99c6d700 1 -- 172.21.15.114:0/3831488317 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a94108700 con 0x7f8a94101150 2024-02-16T22:30:26.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.109+0000 7f8a90ff9700 1 -- 172.21.15.114:0/3831488317 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f8a88022800 con 0x7f8a94101150 2024-02-16T22:30:26.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.109+0000 7f8a90ff9700 1 --2- 172.21.15.114:0/3831488317 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f8a7c06d8a0 0x7f8a7c06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:26.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.109+0000 7f8a90ff9700 1 -- 172.21.15.114:0/3831488317 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f8a8808ea30 con 0x7f8a94101150 2024-02-16T22:30:26.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.109+0000 7f8a937fe700 1 --2- 172.21.15.114:0/3831488317 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f8a7c06d8a0 0x7f8a7c06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:26.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.110+0000 7f8a937fe700 1 --2- 172.21.15.114:0/3831488317 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f8a7c06d8a0 0x7f8a7c06fd60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8a84009c60 tx=0x7f8a84009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:30:26.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.114+0000 7f8a90ff9700 1 -- 172.21.15.114:0/3831488317 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8a8805bce0 con 0x7f8a94101150 2024-02-16T22:30:26.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.346+0000 7f8a99c6d700 1 -- 172.21.15.114:0/3831488317 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a940ff320 con 0x7f8a7c06d8a0 2024-02-16T22:30:26.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.348+0000 7f8a90ff9700 1 -- 172.21.15.114:0/3831488317 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7f8a940ff320 con 0x7f8a7c06d8a0 2024-02-16T22:30:26.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.350+0000 7f8a99c6d700 1 -- 172.21.15.114:0/3831488317 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f8a7c06d8a0 msgr2=0x7f8a7c06fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:26.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.350+0000 7f8a99c6d700 1 --2- 172.21.15.114:0/3831488317 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f8a7c06d8a0 0x7f8a7c06fd60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8a84009c60 tx=0x7f8a84009260 comp rx=0 tx=0).stop 2024-02-16T22:30:26.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.350+0000 7f8a99c6d700 1 -- 172.21.15.114:0/3831488317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a94101150 msgr2=0x7f8a941968c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:26.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.350+0000 7f8a99c6d700 1 --2- 172.21.15.114:0/3831488317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a94101150 0x7f8a941968c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8a88000c00 tx=0x7f8a8800f690 comp rx=0 tx=0).stop 2024-02-16T22:30:26.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.350+0000 7f8a99c6d700 1 -- 172.21.15.114:0/3831488317 shutdown_connections 2024-02-16T22:30:26.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.350+0000 7f8a99c6d700 1 --2- 172.21.15.114:0/3831488317 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f8a7c06d8a0 0x7f8a7c06fd60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:26.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.350+0000 7f8a99c6d700 1 --2- 172.21.15.114:0/3831488317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a940ffd20 0x7f8a94196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:26.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.350+0000 7f8a99c6d700 1 --2- 172.21.15.114:0/3831488317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a94101150 0x7f8a941968c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:26.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.350+0000 7f8a99c6d700 1 -- 172.21.15.114:0/3831488317 >> 172.21.15.114:0/3831488317 conn(0x7f8a940fa7f0 msgr2=0x7f8a940fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:30:26.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.350+0000 7f8a99c6d700 1 -- 172.21.15.114:0/3831488317 shutdown_connections 2024-02-16T22:30:26.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.350+0000 7f8a99c6d700 1 -- 172.21.15.114:0/3831488317 wait complete. 2024-02-16T22:30:26.362 INFO:teuthology.orchestra.run.smithi114.stdout:true 2024-02-16T22:30:26.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.455+0000 7fccf9640700 1 -- 172.21.15.114:0/82163173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccf4100cd0 msgr2=0x7fccf4101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:26.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.455+0000 7fccf9640700 1 --2- 172.21.15.114:0/82163173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccf4100cd0 0x7fccf4101130 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009910 tx=0x7fcce8009c20 comp rx=0 tx=0).stop 2024-02-16T22:30:26.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.455+0000 7fccf9640700 1 -- 172.21.15.114:0/82163173 shutdown_connections 2024-02-16T22:30:26.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.455+0000 7fccf9640700 1 --2- 172.21.15.114:0/82163173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccf4100cd0 0x7fccf4101130 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:26.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.455+0000 7fccf9640700 1 --2- 172.21.15.114:0/82163173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccf40ffad0 0x7fccf40ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:26.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.455+0000 7fccf9640700 1 -- 172.21.15.114:0/82163173 >> 172.21.15.114:0/82163173 conn(0x7fccf40fb050 msgr2=0x7fccf40fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:30:26.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.455+0000 7fccf9640700 1 -- 172.21.15.114:0/82163173 shutdown_connections 2024-02-16T22:30:26.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.455+0000 7fccf9640700 1 -- 172.21.15.114:0/82163173 wait complete. 2024-02-16T22:30:26.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.456+0000 7fccf9640700 1 Processor -- start 2024-02-16T22:30:26.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.456+0000 7fccf9640700 1 -- start start 2024-02-16T22:30:26.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.456+0000 7fccf9640700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccf40ffad0 0x7fccf4196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:26.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.456+0000 7fccf9640700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccf4100cd0 0x7fccf41968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:26.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.456+0000 7fccf9640700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf4196e50 con 0x7fccf4100cd0 2024-02-16T22:30:26.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.456+0000 7fccf9640700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf4196f90 con 0x7fccf40ffad0 2024-02-16T22:30:26.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.456+0000 7fccf27fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccf4100cd0 0x7fccf41968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:26.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.456+0000 7fccf27fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccf4100cd0 0x7fccf41968c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53950/0 (socket says 172.21.15.114:53950) 2024-02-16T22:30:26.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.456+0000 7fccf27fc700 1 -- 172.21.15.114:0/14567149 learned_addr learned my addr 172.21.15.114:0/14567149 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:30:26.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.456+0000 7fccf2ffd700 1 --2- 172.21.15.114:0/14567149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccf40ffad0 0x7fccf4196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:26.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.457+0000 7fccf27fc700 1 -- 172.21.15.114:0/14567149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccf40ffad0 msgr2=0x7fccf4196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:26.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.457+0000 7fccf27fc700 1 --2- 172.21.15.114:0/14567149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccf40ffad0 0x7fccf4196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:26.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.457+0000 7fccf27fc700 1 -- 172.21.15.114:0/14567149 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce80095f0 con 0x7fccf4100cd0 2024-02-16T22:30:26.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.457+0000 7fccf27fc700 1 --2- 172.21.15.114:0/14567149 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccf4100cd0 0x7fccf41968c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcce800bd70 tx=0x7fcce800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:30:26.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.457+0000 7fccdbfff700 1 -- 172.21.15.114:0/14567149 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcce801d070 con 0x7fccf4100cd0 2024-02-16T22:30:26.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.457+0000 7fccf9640700 1 -- 172.21.15.114:0/14567149 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf419b9f0 con 0x7fccf4100cd0 2024-02-16T22:30:26.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.457+0000 7fccdbfff700 1 -- 172.21.15.114:0/14567149 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcce8005940 con 0x7fccf4100cd0 2024-02-16T22:30:26.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.457+0000 7fccdbfff700 1 -- 172.21.15.114:0/14567149 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcce8021430 con 0x7fccf4100cd0 2024-02-16T22:30:26.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.457+0000 7fccf9640700 1 -- 172.21.15.114:0/14567149 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf419beb0 con 0x7fccf4100cd0 2024-02-16T22:30:26.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.458+0000 7fccf9640700 1 -- 172.21.15.114:0/14567149 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccf4108630 con 0x7fccf4100cd0 2024-02-16T22:30:26.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.459+0000 7fccdbfff700 1 -- 172.21.15.114:0/14567149 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fcce800f860 con 0x7fccf4100cd0 2024-02-16T22:30:26.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.459+0000 7fccdbfff700 1 --2- 172.21.15.114:0/14567149 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fccdc06d8a0 0x7fccdc06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:26.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.459+0000 7fccf2ffd700 1 --2- 172.21.15.114:0/14567149 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fccdc06d8a0 0x7fccdc06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:26.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.459+0000 7fccdbfff700 1 -- 172.21.15.114:0/14567149 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7fcce808e9d0 con 0x7fccf4100cd0 2024-02-16T22:30:26.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.459+0000 7fccf2ffd700 1 --2- 172.21.15.114:0/14567149 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fccdc06d8a0 0x7fccdc06fd60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fccf4100b30 tx=0x7fcce400a3b0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:30:26.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.464+0000 7fccdbfff700 1 -- 172.21.15.114:0/14567149 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcce805bc00 con 0x7fccf4100cd0 2024-02-16T22:30:26.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.690+0000 7fccf9640700 1 -- 172.21.15.114:0/14567149 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fccf40694a0 con 0x7fccdc06d8a0 2024-02-16T22:30:26.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.691+0000 7fccdbfff700 1 -- 172.21.15.114:0/14567149 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7fccf40694a0 con 0x7fccdc06d8a0 2024-02-16T22:30:26.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.693+0000 7fccf9640700 1 -- 172.21.15.114:0/14567149 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fccdc06d8a0 msgr2=0x7fccdc06fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:26.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.693+0000 7fccf9640700 1 --2- 172.21.15.114:0/14567149 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fccdc06d8a0 0x7fccdc06fd60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fccf4100b30 tx=0x7fcce400a3b0 comp rx=0 tx=0).stop 2024-02-16T22:30:26.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.693+0000 7fccf9640700 1 -- 172.21.15.114:0/14567149 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccf4100cd0 msgr2=0x7fccf41968c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:26.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.693+0000 7fccf9640700 1 --2- 172.21.15.114:0/14567149 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccf4100cd0 0x7fccf41968c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcce800bd70 tx=0x7fcce800bda0 comp rx=0 tx=0).stop 2024-02-16T22:30:26.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.693+0000 7fccf9640700 1 -- 172.21.15.114:0/14567149 shutdown_connections 2024-02-16T22:30:26.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.693+0000 7fccf9640700 1 --2- 172.21.15.114:0/14567149 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fccdc06d8a0 0x7fccdc06fd60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:26.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.693+0000 7fccf9640700 1 --2- 172.21.15.114:0/14567149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccf40ffad0 0x7fccf4196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:26.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.693+0000 7fccf9640700 1 --2- 172.21.15.114:0/14567149 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccf4100cd0 0x7fccf41968c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:26.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.693+0000 7fccf9640700 1 -- 172.21.15.114:0/14567149 >> 172.21.15.114:0/14567149 conn(0x7fccf40fb050 msgr2=0x7fccf4068d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:30:26.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.693+0000 7fccf9640700 1 -- 172.21.15.114:0/14567149 shutdown_connections 2024-02-16T22:30:26.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.693+0000 7fccf9640700 1 -- 172.21.15.114:0/14567149 wait complete. 2024-02-16T22:30:26.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.798+0000 7fb57f126700 1 -- 172.21.15.114:0/1395749591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb578101e00 msgr2=0x7fb578102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:26.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.798+0000 7fb57f126700 1 --2- 172.21.15.114:0/1395749591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb578101e00 0x7fb578102260 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009910 tx=0x7fb56c009c20 comp rx=0 tx=0).stop 2024-02-16T22:30:26.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.798+0000 7fb57f126700 1 -- 172.21.15.114:0/1395749591 shutdown_connections 2024-02-16T22:30:26.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.798+0000 7fb57f126700 1 --2- 172.21.15.114:0/1395749591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb578101e00 0x7fb578102260 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:26.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.798+0000 7fb57f126700 1 --2- 172.21.15.114:0/1395749591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb578100c00 0x7fb578101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:26.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.798+0000 7fb57f126700 1 -- 172.21.15.114:0/1395749591 >> 172.21.15.114:0/1395749591 conn(0x7fb5780fc180 msgr2=0x7fb5780fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:30:26.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.798+0000 7fb57f126700 1 -- 172.21.15.114:0/1395749591 shutdown_connections 2024-02-16T22:30:26.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.798+0000 7fb57f126700 1 -- 172.21.15.114:0/1395749591 wait complete. 2024-02-16T22:30:26.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.798+0000 7fb57f126700 1 Processor -- start 2024-02-16T22:30:26.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.798+0000 7fb57f126700 1 -- start start 2024-02-16T22:30:26.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.799+0000 7fb57f126700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb578100c00 0x7fb578074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:26.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.799+0000 7fb57f126700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb578101e00 0x7fb578073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:26.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.799+0000 7fb57f126700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5780736c0 con 0x7fb578100c00 2024-02-16T22:30:26.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.799+0000 7fb57f126700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb578073800 con 0x7fb578101e00 2024-02-16T22:30:26.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.799+0000 7fb57cec2700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb578100c00 0x7fb578074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:26.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.799+0000 7fb57cec2700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb578100c00 0x7fb578074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53962/0 (socket says 172.21.15.114:53962) 2024-02-16T22:30:26.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.799+0000 7fb57cec2700 1 -- 172.21.15.114:0/1884436049 learned_addr learned my addr 172.21.15.114:0/1884436049 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:30:26.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.799+0000 7fb577fff700 1 --2- 172.21.15.114:0/1884436049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb578101e00 0x7fb578073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:26.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.799+0000 7fb57cec2700 1 -- 172.21.15.114:0/1884436049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb578101e00 msgr2=0x7fb578073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:26.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.799+0000 7fb57cec2700 1 --2- 172.21.15.114:0/1884436049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb578101e00 0x7fb578073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:26.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.799+0000 7fb57cec2700 1 -- 172.21.15.114:0/1884436049 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56c0095f0 con 0x7fb578100c00 2024-02-16T22:30:26.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.799+0000 7fb57cec2700 1 --2- 172.21.15.114:0/1884436049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb578100c00 0x7fb578074b30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb57000ada0 tx=0x7fb57000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:30:26.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.799+0000 7fb575ffb700 1 -- 172.21.15.114:0/1884436049 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb57000f070 con 0x7fb578100c00 2024-02-16T22:30:26.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.799+0000 7fb57f126700 1 -- 172.21.15.114:0/1884436049 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb578073ae0 con 0x7fb578100c00 2024-02-16T22:30:26.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.799+0000 7fb575ffb700 1 -- 172.21.15.114:0/1884436049 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb57000ae90 con 0x7fb578100c00 2024-02-16T22:30:26.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.799+0000 7fb575ffb700 1 -- 172.21.15.114:0/1884436049 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5700083f0 con 0x7fb578100c00 2024-02-16T22:30:26.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.800+0000 7fb57f126700 1 -- 172.21.15.114:0/1884436049 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb57819e3e0 con 0x7fb578100c00 2024-02-16T22:30:26.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:26 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:26.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:26 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:26.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:26 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:30:26.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:26 smithi123 ceph-mon[82331]: pgmap v237: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:26.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:26 smithi123 ceph-mon[82331]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:30:26.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.801+0000 7fb57f126700 1 -- 172.21.15.114:0/1884436049 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb578109760 con 0x7fb578100c00 2024-02-16T22:30:26.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.801+0000 7fb575ffb700 1 -- 172.21.15.114:0/1884436049 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fb57000cad0 con 0x7fb578100c00 2024-02-16T22:30:26.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.801+0000 7fb575ffb700 1 --2- 172.21.15.114:0/1884436049 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb56006d8a0 0x7fb56006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:26.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.801+0000 7fb575ffb700 1 -- 172.21.15.114:0/1884436049 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7fb57008cf60 con 0x7fb578100c00 2024-02-16T22:30:26.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.801+0000 7fb577fff700 1 --2- 172.21.15.114:0/1884436049 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb56006d8a0 0x7fb56006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:26.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.802+0000 7fb577fff700 1 --2- 172.21.15.114:0/1884436049 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb56006d8a0 0x7fb56006fd60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb56c006010 tx=0x7fb56c00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:30:26.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:26.806+0000 7fb575ffb700 1 -- 172.21.15.114:0/1884436049 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb5700c0d20 con 0x7fb578100c00 2024-02-16T22:30:26.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:26 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:26.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:26 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:26.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:26 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:30:26.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:26 smithi114 ceph-mon[94452]: pgmap v237: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:26.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:26 smithi114 ceph-mon[94452]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:30:27.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.040+0000 7fb57f126700 1 -- 172.21.15.114:0/1884436049 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb578106750 con 0x7fb56006d8a0 2024-02-16T22:30:27.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.051+0000 7fb575ffb700 1 -- 172.21.15.114:0/1884436049 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fb578106750 con 0x7fb56006d8a0 2024-02-16T22:30:27.053 INFO:teuthology.orchestra.run.smithi114.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:30:27.053 INFO:teuthology.orchestra.run.smithi114.stdout:alertmanager.smithi114 smithi114 *:9093,9094 running (16m) 14s ago 17m 24.4M - 0.25.0 c8568f914cd2 e7404fea400e 2024-02-16T22:30:27.053 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi114 smithi114 running (25s) 14s ago 18m 7646k - 19.0.0-1450-gf78a58c0 f081f847070a 1f621c33f6d4 2024-02-16T22:30:27.053 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi123 smithi123 running (19s) 16s ago 17m 6924k - 19.0.0-1450-gf78a58c0 f081f847070a 2f0b3c7f313b 2024-02-16T22:30:27.053 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi114 smithi114 running (5m) 14s ago 18m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 9b8ae9efe83b 2024-02-16T22:30:27.053 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi123 smithi123 running (5m) 16s ago 16m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 6d97263a0ffa 2024-02-16T22:30:27.054 INFO:teuthology.orchestra.run.smithi114.stdout:grafana.smithi114 smithi114 *:3000 running (16m) 14s ago 17m 88.9M - 9.4.7 954c08fa6188 db1c26c3bc20 2024-02-16T22:30:27.054 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.fdxpey smithi114 running (91s) 14s ago 12m 13.7M - 19.0.0-1450-gf78a58c0 f081f847070a 2984ae8a955a 2024-02-16T22:30:27.054 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.zzryke smithi114 running (118s) 14s ago 12m 84.9M - 19.0.0-1450-gf78a58c0 f081f847070a 63fa2af7290e 2024-02-16T22:30:27.054 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.gopcfa smithi123 running (38s) 16s ago 12m 14.7M - 19.0.0-1450-gf78a58c0 f081f847070a ec6612a62e2a 2024-02-16T22:30:27.054 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.vhlqbe smithi123 running (61s) 16s ago 12m 15.8M - 19.0.0-1450-gf78a58c0 f081f847070a be790c6287e7 2024-02-16T22:30:27.054 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi114.vcullz smithi114 *:8443,9283,8765 running (8m) 14s ago 19m 509M - 19.0.0-1450-gf78a58c0 f081f847070a 14ebeb7888c0 2024-02-16T22:30:27.054 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi123.inzgjd smithi123 *:8443,9283,8765 running (7m) 16s ago 16m 451M - 19.0.0-1450-gf78a58c0 f081f847070a f10aab9faf9e 2024-02-16T22:30:27.054 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi114 smithi114 running (6m) 14s ago 19m 55.2M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 58ac06372523 2024-02-16T22:30:27.054 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi123 smithi123 running (6m) 16s ago 16m 41.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a f76485e0ea24 2024-02-16T22:30:27.054 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi114 smithi114 *:9100 starting - - - - 2024-02-16T22:30:27.054 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi123 smithi123 *:9100 starting - - - - 2024-02-16T22:30:27.054 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 smithi114 running (5m) 14s ago 15m 100M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 1978fdad94a5 2024-02-16T22:30:27.054 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 smithi114 running (4m) 14s ago 15m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 3f90e792898f 2024-02-16T22:30:27.054 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 smithi114 running (4m) 14s ago 15m 91.3M 1216M 19.0.0-1450-gf78a58c0 f081f847070a c663dfa0518b 2024-02-16T22:30:27.054 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 smithi123 running (3m) 16s ago 14m 113M 2240M 19.0.0-1450-gf78a58c0 f081f847070a b921448ba169 2024-02-16T22:30:27.054 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 smithi123 running (2m) 16s ago 14m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 54c2926d56b1 2024-02-16T22:30:27.055 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 smithi123 running (2m) 16s ago 13m 94.9M 2240M 19.0.0-1450-gf78a58c0 f081f847070a dd99bf034749 2024-02-16T22:30:27.055 INFO:teuthology.orchestra.run.smithi114.stdout:prometheus.smithi114 smithi114 *:9095 running (7m) 14s ago 17m 66.0M - 2.43.0 a07b618ecd1d 37aac985cdd7 2024-02-16T22:30:27.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.054+0000 7fb57f126700 1 -- 172.21.15.114:0/1884436049 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb56006d8a0 msgr2=0x7fb56006fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:27.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.054+0000 7fb57f126700 1 --2- 172.21.15.114:0/1884436049 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb56006d8a0 0x7fb56006fd60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb56c006010 tx=0x7fb56c00b540 comp rx=0 tx=0).stop 2024-02-16T22:30:27.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.054+0000 7fb57f126700 1 -- 172.21.15.114:0/1884436049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb578100c00 msgr2=0x7fb578074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:27.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.054+0000 7fb57f126700 1 --2- 172.21.15.114:0/1884436049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb578100c00 0x7fb578074b30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb57000ada0 tx=0x7fb57000c330 comp rx=0 tx=0).stop 2024-02-16T22:30:27.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.054+0000 7fb57f126700 1 -- 172.21.15.114:0/1884436049 shutdown_connections 2024-02-16T22:30:27.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.054+0000 7fb57f126700 1 --2- 172.21.15.114:0/1884436049 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb56006d8a0 0x7fb56006fd60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:27.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.054+0000 7fb57f126700 1 --2- 172.21.15.114:0/1884436049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb578100c00 0x7fb578074b30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:27.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.054+0000 7fb57f126700 1 --2- 172.21.15.114:0/1884436049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb578101e00 0x7fb578073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:27.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.054+0000 7fb57f126700 1 -- 172.21.15.114:0/1884436049 >> 172.21.15.114:0/1884436049 conn(0x7fb5780fc180 msgr2=0x7fb578105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:30:27.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.054+0000 7fb57f126700 1 -- 172.21.15.114:0/1884436049 shutdown_connections 2024-02-16T22:30:27.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.054+0000 7fb57f126700 1 -- 172.21.15.114:0/1884436049 wait complete. 2024-02-16T22:30:27.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.174+0000 7f3a416aa700 1 -- 172.21.15.114:0/3265302296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a3c073770 msgr2=0x7f3a3c073bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:27.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.174+0000 7f3a416aa700 1 --2- 172.21.15.114:0/3265302296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a3c073770 0x7f3a3c073bf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3a30009910 tx=0x7f3a30009c20 comp rx=0 tx=0).stop 2024-02-16T22:30:27.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.174+0000 7f3a416aa700 1 -- 172.21.15.114:0/3265302296 shutdown_connections 2024-02-16T22:30:27.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.174+0000 7f3a416aa700 1 --2- 172.21.15.114:0/3265302296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a3c073770 0x7f3a3c073bf0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:27.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.174+0000 7f3a416aa700 1 --2- 172.21.15.114:0/3265302296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a3c074d40 0x7f3a3c0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:27.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.174+0000 7f3a416aa700 1 -- 172.21.15.114:0/3265302296 >> 172.21.15.114:0/3265302296 conn(0x7f3a3c0fc180 msgr2=0x7f3a3c0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:30:27.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.174+0000 7f3a416aa700 1 -- 172.21.15.114:0/3265302296 shutdown_connections 2024-02-16T22:30:27.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.174+0000 7f3a416aa700 1 -- 172.21.15.114:0/3265302296 wait complete. 2024-02-16T22:30:27.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.174+0000 7f3a416aa700 1 Processor -- start 2024-02-16T22:30:27.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.175+0000 7f3a416aa700 1 -- start start 2024-02-16T22:30:27.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.175+0000 7f3a416aa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a3c073770 0x7f3a3c198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:27.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.175+0000 7f3a416aa700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a3c074d40 0x7f3a3c198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:27.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.175+0000 7f3a416aa700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c1990b0 con 0x7f3a3c074d40 2024-02-16T22:30:27.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.175+0000 7f3a416aa700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c1991f0 con 0x7f3a3c073770 2024-02-16T22:30:27.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.175+0000 7f3a3a7fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a3c074d40 0x7f3a3c198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:27.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.175+0000 7f3a3a7fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a3c074d40 0x7f3a3c198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53970/0 (socket says 172.21.15.114:53970) 2024-02-16T22:30:27.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.175+0000 7f3a3a7fc700 1 -- 172.21.15.114:0/834123850 learned_addr learned my addr 172.21.15.114:0/834123850 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:30:27.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.175+0000 7f3a3affd700 1 --2- 172.21.15.114:0/834123850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a3c073770 0x7f3a3c198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:27.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.175+0000 7f3a3a7fc700 1 -- 172.21.15.114:0/834123850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a3c073770 msgr2=0x7f3a3c198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:27.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.175+0000 7f3a3a7fc700 1 --2- 172.21.15.114:0/834123850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a3c073770 0x7f3a3c198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:27.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.175+0000 7f3a3a7fc700 1 -- 172.21.15.114:0/834123850 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a300095f0 con 0x7f3a3c074d40 2024-02-16T22:30:27.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.175+0000 7f3a3affd700 1 --2- 172.21.15.114:0/834123850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a3c073770 0x7f3a3c198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:30:27.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.175+0000 7f3a3a7fc700 1 --2- 172.21.15.114:0/834123850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a3c074d40 0x7f3a3c198a90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000bd70 tx=0x7f3a3000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:30:27.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.175+0000 7f3a23fff700 1 -- 172.21.15.114:0/834123850 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a3001d070 con 0x7f3a3c074d40 2024-02-16T22:30:27.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.175+0000 7f3a416aa700 1 -- 172.21.15.114:0/834123850 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a3c19dc40 con 0x7f3a3c074d40 2024-02-16T22:30:27.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.175+0000 7f3a23fff700 1 -- 172.21.15.114:0/834123850 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3a30022470 con 0x7f3a3c074d40 2024-02-16T22:30:27.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.176+0000 7f3a23fff700 1 -- 172.21.15.114:0/834123850 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a3000f670 con 0x7f3a3c074d40 2024-02-16T22:30:27.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.176+0000 7f3a416aa700 1 -- 172.21.15.114:0/834123850 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a3c19e130 con 0x7f3a3c074d40 2024-02-16T22:30:27.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.176+0000 7f3a416aa700 1 -- 172.21.15.114:0/834123850 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a3c10b950 con 0x7f3a3c074d40 2024-02-16T22:30:27.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.177+0000 7f3a23fff700 1 -- 172.21.15.114:0/834123850 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f3a30022a50 con 0x7f3a3c074d40 2024-02-16T22:30:27.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.178+0000 7f3a23fff700 1 --2- 172.21.15.114:0/834123850 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f3a2406d8a0 0x7f3a2406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:27.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.178+0000 7f3a23fff700 1 -- 172.21.15.114:0/834123850 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f3a3008f040 con 0x7f3a3c074d40 2024-02-16T22:30:27.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.178+0000 7f3a3affd700 1 --2- 172.21.15.114:0/834123850 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f3a2406d8a0 0x7f3a2406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:27.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.178+0000 7f3a3affd700 1 --2- 172.21.15.114:0/834123850 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f3a2406d8a0 0x7f3a2406fd60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3a34009dd0 tx=0x7f3a34009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:30:27.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.182+0000 7f3a23fff700 1 -- 172.21.15.114:0/834123850 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3a3005c3a0 con 0x7f3a3c074d40 2024-02-16T22:30:27.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.476+0000 7f3a416aa700 1 -- 172.21.15.114:0/834123850 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3a3c19e740 con 0x7f3a3c074d40 2024-02-16T22:30:27.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.476+0000 7f3a23fff700 1 -- 172.21.15.114:0/834123850 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f3a30031280 con 0x7f3a3c074d40 2024-02-16T22:30:27.478 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:30:27.478 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": { 2024-02-16T22:30:27.478 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:30:27.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:30:27.479 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": { 2024-02-16T22:30:27.479 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:30:27.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:30:27.479 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": { 2024-02-16T22:30:27.479 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T22:30:27.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:30:27.479 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-02-16T22:30:27.479 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T22:30:27.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:30:27.479 INFO:teuthology.orchestra.run.smithi114.stdout: "overall": { 2024-02-16T22:30:27.479 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-16T22:30:27.479 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-02-16T22:30:27.479 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:30:27.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.479+0000 7f3a416aa700 1 -- 172.21.15.114:0/834123850 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f3a2406d8a0 msgr2=0x7f3a2406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:27.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.479+0000 7f3a416aa700 1 --2- 172.21.15.114:0/834123850 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f3a2406d8a0 0x7f3a2406fd60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3a34009dd0 tx=0x7f3a34009040 comp rx=0 tx=0).stop 2024-02-16T22:30:27.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.479+0000 7f3a416aa700 1 -- 172.21.15.114:0/834123850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a3c074d40 msgr2=0x7f3a3c198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:27.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.479+0000 7f3a416aa700 1 --2- 172.21.15.114:0/834123850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a3c074d40 0x7f3a3c198a90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000bd70 tx=0x7f3a3000be50 comp rx=0 tx=0).stop 2024-02-16T22:30:27.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.479+0000 7f3a416aa700 1 -- 172.21.15.114:0/834123850 shutdown_connections 2024-02-16T22:30:27.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.479+0000 7f3a416aa700 1 --2- 172.21.15.114:0/834123850 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f3a2406d8a0 0x7f3a2406fd60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:27.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.479+0000 7f3a416aa700 1 --2- 172.21.15.114:0/834123850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a3c073770 0x7f3a3c198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:27.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.479+0000 7f3a416aa700 1 --2- 172.21.15.114:0/834123850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a3c074d40 0x7f3a3c198a90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:27.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.479+0000 7f3a416aa700 1 -- 172.21.15.114:0/834123850 >> 172.21.15.114:0/834123850 conn(0x7f3a3c0fc180 msgr2=0x7f3a3c106a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:30:27.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.479+0000 7f3a416aa700 1 -- 172.21.15.114:0/834123850 shutdown_connections 2024-02-16T22:30:27.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.479+0000 7f3a416aa700 1 -- 172.21.15.114:0/834123850 wait complete. 2024-02-16T22:30:27.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.584+0000 7ff4570df700 1 -- 172.21.15.114:0/1524423364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff450073770 msgr2=0x7ff450073bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:27.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.584+0000 7ff4570df700 1 --2- 172.21.15.114:0/1524423364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff450073770 0x7ff450073bf0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff440009910 tx=0x7ff440009c20 comp rx=0 tx=0).stop 2024-02-16T22:30:27.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.584+0000 7ff4570df700 1 -- 172.21.15.114:0/1524423364 shutdown_connections 2024-02-16T22:30:27.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.584+0000 7ff4570df700 1 --2- 172.21.15.114:0/1524423364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff450073770 0x7ff450073bf0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:27.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.584+0000 7ff4570df700 1 --2- 172.21.15.114:0/1524423364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff450074d40 0x7ff4500731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:27.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.584+0000 7ff4570df700 1 -- 172.21.15.114:0/1524423364 >> 172.21.15.114:0/1524423364 conn(0x7ff4500fc140 msgr2=0x7ff4500fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:30:27.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.584+0000 7ff4570df700 1 -- 172.21.15.114:0/1524423364 shutdown_connections 2024-02-16T22:30:27.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.584+0000 7ff4570df700 1 -- 172.21.15.114:0/1524423364 wait complete. 2024-02-16T22:30:27.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.585+0000 7ff4570df700 1 Processor -- start 2024-02-16T22:30:27.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.585+0000 7ff4570df700 1 -- start start 2024-02-16T22:30:27.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.585+0000 7ff4570df700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff450073770 0x7ff450198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:27.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.585+0000 7ff4570df700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff450074d40 0x7ff450198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:27.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.585+0000 7ff4570df700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff450199090 con 0x7ff450073770 2024-02-16T22:30:27.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.585+0000 7ff4570df700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4501991d0 con 0x7ff450074d40 2024-02-16T22:30:27.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.585+0000 7ff454e7b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff450073770 0x7ff450198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:27.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.585+0000 7ff454e7b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff450073770 0x7ff450198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53976/0 (socket says 172.21.15.114:53976) 2024-02-16T22:30:27.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.585+0000 7ff454e7b700 1 -- 172.21.15.114:0/679482125 learned_addr learned my addr 172.21.15.114:0/679482125 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:30:27.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.585+0000 7ff44ffff700 1 --2- 172.21.15.114:0/679482125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff450074d40 0x7ff450198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:27.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.585+0000 7ff454e7b700 1 -- 172.21.15.114:0/679482125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff450074d40 msgr2=0x7ff450198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:27.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.585+0000 7ff454e7b700 1 --2- 172.21.15.114:0/679482125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff450074d40 0x7ff450198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:27.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.585+0000 7ff454e7b700 1 -- 172.21.15.114:0/679482125 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4400095f0 con 0x7ff450073770 2024-02-16T22:30:27.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.585+0000 7ff454e7b700 1 --2- 172.21.15.114:0/679482125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff450073770 0x7ff450198530 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff448007e90 tx=0x7ff44800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:30:27.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.586+0000 7ff44dffb700 1 -- 172.21.15.114:0/679482125 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff44800de60 con 0x7ff450073770 2024-02-16T22:30:27.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.586+0000 7ff4570df700 1 -- 172.21.15.114:0/679482125 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff45019dc80 con 0x7ff450073770 2024-02-16T22:30:27.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.586+0000 7ff44dffb700 1 -- 172.21.15.114:0/679482125 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff44800f040 con 0x7ff450073770 2024-02-16T22:30:27.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.586+0000 7ff44dffb700 1 -- 172.21.15.114:0/679482125 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff448004520 con 0x7ff450073770 2024-02-16T22:30:27.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.586+0000 7ff4570df700 1 -- 172.21.15.114:0/679482125 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff45019e180 con 0x7ff450073770 2024-02-16T22:30:27.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.587+0000 7ff4570df700 1 -- 172.21.15.114:0/679482125 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff45010b910 con 0x7ff450073770 2024-02-16T22:30:27.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.587+0000 7ff44dffb700 1 -- 172.21.15.114:0/679482125 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7ff4480046e0 con 0x7ff450073770 2024-02-16T22:30:27.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.588+0000 7ff44dffb700 1 --2- 172.21.15.114:0/679482125 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff43c06d8a0 0x7ff43c06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:27.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.588+0000 7ff44dffb700 1 -- 172.21.15.114:0/679482125 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7ff44808d290 con 0x7ff450073770 2024-02-16T22:30:27.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.588+0000 7ff44ffff700 1 --2- 172.21.15.114:0/679482125 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff43c06d8a0 0x7ff43c06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:27.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.588+0000 7ff44ffff700 1 --2- 172.21.15.114:0/679482125 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff43c06d8a0 0x7ff43c06fd60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff44000b5c0 tx=0x7ff440005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:30:27.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.594+0000 7ff44dffb700 1 -- 172.21.15.114:0/679482125 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff44805a4c0 con 0x7ff450073770 2024-02-16T22:30:27.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.838+0000 7ff4570df700 1 -- 172.21.15.114:0/679482125 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff45019e790 con 0x7ff450073770 2024-02-16T22:30:27.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.839+0000 7ff44dffb700 1 -- 172.21.15.114:0/679482125 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1876 (secure 0 0 0) 0x7ff448004ad0 con 0x7ff450073770 2024-02-16T22:30:27.841 INFO:teuthology.orchestra.run.smithi114.stdout:e37 2024-02-16T22:30:27.841 INFO:teuthology.orchestra.run.smithi114.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:30:27.841 INFO:teuthology.orchestra.run.smithi114.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:30:27.841 INFO:teuthology.orchestra.run.smithi114.stdout:legacy client fscid: 1 2024-02-16T22:30:27.842 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:30:27.842 INFO:teuthology.orchestra.run.smithi114.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:30:27.842 INFO:teuthology.orchestra.run.smithi114.stdout:fs_name cephfs 2024-02-16T22:30:27.842 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 37 2024-02-16T22:30:27.842 INFO:teuthology.orchestra.run.smithi114.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:30:27.842 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-02-16T22:18:02.108825+0000 2024-02-16T22:30:27.842 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-02-16T22:29:48.901292+0000 2024-02-16T22:30:27.842 INFO:teuthology.orchestra.run.smithi114.stdout:tableserver 0 2024-02-16T22:30:27.842 INFO:teuthology.orchestra.run.smithi114.stdout:root 0 2024-02-16T22:30:27.842 INFO:teuthology.orchestra.run.smithi114.stdout:session_timeout 60 2024-02-16T22:30:27.842 INFO:teuthology.orchestra.run.smithi114.stdout:session_autoclose 300 2024-02-16T22:30:27.842 INFO:teuthology.orchestra.run.smithi114.stdout:max_file_size 1099511627776 2024-02-16T22:30:27.842 INFO:teuthology.orchestra.run.smithi114.stdout:max_xattr_size 65536 2024-02-16T22:30:27.842 INFO:teuthology.orchestra.run.smithi114.stdout:required_client_features {} 2024-02-16T22:30:27.842 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure 0 2024-02-16T22:30:27.843 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure_osd_epoch 82 2024-02-16T22:30:27.843 INFO:teuthology.orchestra.run.smithi114.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-16T22:30:27.843 INFO:teuthology.orchestra.run.smithi114.stdout:max_mds 1 2024-02-16T22:30:27.843 INFO:teuthology.orchestra.run.smithi114.stdout:in 0 2024-02-16T22:30:27.843 INFO:teuthology.orchestra.run.smithi114.stdout:up {0=34404} 2024-02-16T22:30:27.843 INFO:teuthology.orchestra.run.smithi114.stdout:failed 2024-02-16T22:30:27.843 INFO:teuthology.orchestra.run.smithi114.stdout:damaged 2024-02-16T22:30:27.843 INFO:teuthology.orchestra.run.smithi114.stdout:stopped 2024-02-16T22:30:27.843 INFO:teuthology.orchestra.run.smithi114.stdout:data_pools [3] 2024-02-16T22:30:27.843 INFO:teuthology.orchestra.run.smithi114.stdout:metadata_pool 2 2024-02-16T22:30:27.843 INFO:teuthology.orchestra.run.smithi114.stdout:inline_data enabled 2024-02-16T22:30:27.843 INFO:teuthology.orchestra.run.smithi114.stdout:balancer 2024-02-16T22:30:27.843 INFO:teuthology.orchestra.run.smithi114.stdout:bal_rank_mask -1 2024-02-16T22:30:27.843 INFO:teuthology.orchestra.run.smithi114.stdout:standby_count_wanted 1 2024-02-16T22:30:27.844 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.zzryke{0:34404} state up:active seq 23 join_fscid=1 addr [v2:172.21.15.114:6826/2017800151,v1:172.21.15.114:6827/2017800151] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T22:30:27.844 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:30:27.844 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:30:27.844 INFO:teuthology.orchestra.run.smithi114.stdout:Standby daemons: 2024-02-16T22:30:27.844 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:30:27.844 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.fdxpey{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.114:6828/3702914193,v1:172.21.15.114:6829/3702914193] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T22:30:27.844 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.vhlqbe{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.123:6824/1425065679,v1:172.21.15.123:6825/1425065679] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T22:30:27.844 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.gopcfa{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.123:6826/3128771108,v1:172.21.15.123:6827/3128771108] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T22:30:27.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.842+0000 7ff4570df700 1 -- 172.21.15.114:0/679482125 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff43c06d8a0 msgr2=0x7ff43c06fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:27.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.842+0000 7ff4570df700 1 --2- 172.21.15.114:0/679482125 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff43c06d8a0 0x7ff43c06fd60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff44000b5c0 tx=0x7ff440005ee0 comp rx=0 tx=0).stop 2024-02-16T22:30:27.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.842+0000 7ff4570df700 1 -- 172.21.15.114:0/679482125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff450073770 msgr2=0x7ff450198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:27.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.842+0000 7ff4570df700 1 --2- 172.21.15.114:0/679482125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff450073770 0x7ff450198530 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff448007e90 tx=0x7ff44800d3b0 comp rx=0 tx=0).stop 2024-02-16T22:30:27.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.842+0000 7ff4570df700 1 -- 172.21.15.114:0/679482125 shutdown_connections 2024-02-16T22:30:27.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.842+0000 7ff4570df700 1 --2- 172.21.15.114:0/679482125 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff43c06d8a0 0x7ff43c06fd60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:27.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.842+0000 7ff4570df700 1 --2- 172.21.15.114:0/679482125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff450073770 0x7ff450198530 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:27.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.842+0000 7ff4570df700 1 --2- 172.21.15.114:0/679482125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff450074d40 0x7ff450198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:27.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.842+0000 7ff4570df700 1 -- 172.21.15.114:0/679482125 >> 172.21.15.114:0/679482125 conn(0x7ff4500fc140 msgr2=0x7ff450106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:30:27.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.842+0000 7ff4570df700 1 -- 172.21.15.114:0/679482125 shutdown_connections 2024-02-16T22:30:27.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.842+0000 7ff4570df700 1 -- 172.21.15.114:0/679482125 wait complete. 2024-02-16T22:30:27.846 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 37 2024-02-16T22:30:27.909 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:27 smithi123 ceph-mon[82331]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:30:27.909 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:27 smithi123 ceph-mon[82331]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:30:27.909 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:27 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/834123850' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:27.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:27 smithi114 ceph-mon[94452]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:30:27.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:27 smithi114 ceph-mon[94452]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:30:27.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:27 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/834123850' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:27.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.957+0000 7f72db33d700 1 -- 172.21.15.114:0/2543882207 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72d4100cb0 msgr2=0x7f72d4101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:27.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.957+0000 7f72db33d700 1 --2- 172.21.15.114:0/2543882207 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72d4100cb0 0x7f72d4101110 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f72c4009910 tx=0x7f72c4009c20 comp rx=0 tx=0).stop 2024-02-16T22:30:27.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.957+0000 7f72db33d700 1 -- 172.21.15.114:0/2543882207 shutdown_connections 2024-02-16T22:30:27.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.957+0000 7f72db33d700 1 --2- 172.21.15.114:0/2543882207 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72d4100cb0 0x7f72d4101110 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:27.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.957+0000 7f72db33d700 1 --2- 172.21.15.114:0/2543882207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72d40ffab0 0x7f72d40ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:27.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.957+0000 7f72db33d700 1 -- 172.21.15.114:0/2543882207 >> 172.21.15.114:0/2543882207 conn(0x7f72d40fb030 msgr2=0x7f72d40fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:30:27.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.957+0000 7f72db33d700 1 -- 172.21.15.114:0/2543882207 shutdown_connections 2024-02-16T22:30:27.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.957+0000 7f72db33d700 1 -- 172.21.15.114:0/2543882207 wait complete. 2024-02-16T22:30:27.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.958+0000 7f72db33d700 1 Processor -- start 2024-02-16T22:30:27.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.958+0000 7f72db33d700 1 -- start start 2024-02-16T22:30:27.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.958+0000 7f72db33d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72d40ffab0 0x7f72d4068f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:27.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.958+0000 7f72db33d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72d4100cb0 0x7f72d419b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:27.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.958+0000 7f72db33d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d419b690 con 0x7f72d40ffab0 2024-02-16T22:30:27.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.958+0000 7f72db33d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d419b7d0 con 0x7f72d4100cb0 2024-02-16T22:30:27.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.958+0000 7f72d90d9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72d40ffab0 0x7f72d4068f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:27.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.958+0000 7f72d90d9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72d40ffab0 0x7f72d4068f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53994/0 (socket says 172.21.15.114:53994) 2024-02-16T22:30:27.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.958+0000 7f72d90d9700 1 -- 172.21.15.114:0/815251893 learned_addr learned my addr 172.21.15.114:0/815251893 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:30:27.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.958+0000 7f72d88d8700 1 --2- 172.21.15.114:0/815251893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72d4100cb0 0x7f72d419b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:27.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.958+0000 7f72d90d9700 1 -- 172.21.15.114:0/815251893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72d4100cb0 msgr2=0x7f72d419b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:27.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.958+0000 7f72d90d9700 1 --2- 172.21.15.114:0/815251893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72d4100cb0 0x7f72d419b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:27.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.958+0000 7f72d90d9700 1 -- 172.21.15.114:0/815251893 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c40095f0 con 0x7f72d40ffab0 2024-02-16T22:30:27.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.958+0000 7f72d90d9700 1 --2- 172.21.15.114:0/815251893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72d40ffab0 0x7f72d4068f70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f72d000e8f0 tx=0x7f72d000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:30:27.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.959+0000 7f72ca7fc700 1 -- 172.21.15.114:0/815251893 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72d0009790 con 0x7f72d40ffab0 2024-02-16T22:30:27.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.959+0000 7f72ca7fc700 1 -- 172.21.15.114:0/815251893 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f72d000c810 con 0x7f72d40ffab0 2024-02-16T22:30:27.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.959+0000 7f72ca7fc700 1 -- 172.21.15.114:0/815251893 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72d0003d20 con 0x7f72d40ffab0 2024-02-16T22:30:27.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.959+0000 7f72db33d700 1 -- 172.21.15.114:0/815251893 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d419bab0 con 0x7f72d40ffab0 2024-02-16T22:30:27.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.959+0000 7f72db33d700 1 -- 172.21.15.114:0/815251893 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d419bfd0 con 0x7f72d40ffab0 2024-02-16T22:30:27.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.960+0000 7f72db33d700 1 -- 172.21.15.114:0/815251893 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d4108610 con 0x7f72d40ffab0 2024-02-16T22:30:27.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.960+0000 7f72ca7fc700 1 -- 172.21.15.114:0/815251893 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f72d00041f0 con 0x7f72d40ffab0 2024-02-16T22:30:27.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.960+0000 7f72ca7fc700 1 --2- 172.21.15.114:0/815251893 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f72c006d8a0 0x7f72c006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:27.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.961+0000 7f72ca7fc700 1 -- 172.21.15.114:0/815251893 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f72d0014070 con 0x7f72d40ffab0 2024-02-16T22:30:27.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.961+0000 7f72d88d8700 1 --2- 172.21.15.114:0/815251893 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f72c006d8a0 0x7f72c006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:27.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.961+0000 7f72d88d8700 1 --2- 172.21.15.114:0/815251893 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f72c006d8a0 0x7f72c006fd60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f72c4009fd0 tx=0x7f72c4005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:30:27.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:27.965+0000 7f72ca7fc700 1 -- 172.21.15.114:0/815251893 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f72d005ab00 con 0x7f72d40ffab0 2024-02-16T22:30:28.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.188+0000 7f72db33d700 1 -- 172.21.15.114:0/815251893 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f72d4105600 con 0x7f72c006d8a0 2024-02-16T22:30:28.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.189+0000 7f72ca7fc700 1 -- 172.21.15.114:0/815251893 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7f72d4105600 con 0x7f72c006d8a0 2024-02-16T22:30:28.191 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:30:28.191 INFO:teuthology.orchestra.run.smithi114.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:30:28.191 INFO:teuthology.orchestra.run.smithi114.stdout: "in_progress": true, 2024-02-16T22:30:28.191 INFO:teuthology.orchestra.run.smithi114.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:30:28.191 INFO:teuthology.orchestra.run.smithi114.stdout: "services_complete": [ 2024-02-16T22:30:28.192 INFO:teuthology.orchestra.run.smithi114.stdout: "osd", 2024-02-16T22:30:28.192 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph-exporter", 2024-02-16T22:30:28.192 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr", 2024-02-16T22:30:28.192 INFO:teuthology.orchestra.run.smithi114.stdout: "mon", 2024-02-16T22:30:28.192 INFO:teuthology.orchestra.run.smithi114.stdout: "mds", 2024-02-16T22:30:28.192 INFO:teuthology.orchestra.run.smithi114.stdout: "crash" 2024-02-16T22:30:28.192 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-02-16T22:30:28.192 INFO:teuthology.orchestra.run.smithi114.stdout: "progress": "18/23 daemons upgraded", 2024-02-16T22:30:28.192 INFO:teuthology.orchestra.run.smithi114.stdout: "message": "Currently upgrading node-exporter daemons", 2024-02-16T22:30:28.192 INFO:teuthology.orchestra.run.smithi114.stdout: "is_paused": false 2024-02-16T22:30:28.192 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:30:28.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.191+0000 7f72db33d700 1 -- 172.21.15.114:0/815251893 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f72c006d8a0 msgr2=0x7f72c006fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:28.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.191+0000 7f72db33d700 1 --2- 172.21.15.114:0/815251893 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f72c006d8a0 0x7f72c006fd60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f72c4009fd0 tx=0x7f72c4005940 comp rx=0 tx=0).stop 2024-02-16T22:30:28.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.191+0000 7f72db33d700 1 -- 172.21.15.114:0/815251893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72d40ffab0 msgr2=0x7f72d4068f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:28.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.191+0000 7f72db33d700 1 --2- 172.21.15.114:0/815251893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72d40ffab0 0x7f72d4068f70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f72d000e8f0 tx=0x7f72d000ec00 comp rx=0 tx=0).stop 2024-02-16T22:30:28.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.191+0000 7f72db33d700 1 -- 172.21.15.114:0/815251893 shutdown_connections 2024-02-16T22:30:28.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.191+0000 7f72db33d700 1 --2- 172.21.15.114:0/815251893 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f72c006d8a0 0x7f72c006fd60 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:28.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.191+0000 7f72db33d700 1 --2- 172.21.15.114:0/815251893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72d40ffab0 0x7f72d4068f70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:28.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.191+0000 7f72db33d700 1 --2- 172.21.15.114:0/815251893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72d4100cb0 0x7f72d419b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:28.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.191+0000 7f72db33d700 1 -- 172.21.15.114:0/815251893 >> 172.21.15.114:0/815251893 conn(0x7f72d40fb030 msgr2=0x7f72d4103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:30:28.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.191+0000 7f72db33d700 1 -- 172.21.15.114:0/815251893 shutdown_connections 2024-02-16T22:30:28.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.191+0000 7f72db33d700 1 -- 172.21.15.114:0/815251893 wait complete. 2024-02-16T22:30:28.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.295+0000 7f51fb4dd700 1 -- 172.21.15.114:0/2548106496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51f4103f80 msgr2=0x7f51f4106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:28.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.295+0000 7f51fb4dd700 1 --2- 172.21.15.114:0/2548106496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51f4103f80 0x7f51f4106370 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f51e4009910 tx=0x7f51e4009c20 comp rx=0 tx=0).stop 2024-02-16T22:30:28.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.295+0000 7f51fb4dd700 1 -- 172.21.15.114:0/2548106496 shutdown_connections 2024-02-16T22:30:28.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.295+0000 7f51fb4dd700 1 --2- 172.21.15.114:0/2548106496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51f4103f80 0x7f51f4106370 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:28.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.295+0000 7f51fb4dd700 1 --2- 172.21.15.114:0/2548106496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51f4101650 0x7f51f4103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:28.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.295+0000 7f51fb4dd700 1 -- 172.21.15.114:0/2548106496 >> 172.21.15.114:0/2548106496 conn(0x7f51f40fb060 msgr2=0x7f51f40fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:30:28.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.296+0000 7f51fb4dd700 1 -- 172.21.15.114:0/2548106496 shutdown_connections 2024-02-16T22:30:28.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.296+0000 7f51fb4dd700 1 -- 172.21.15.114:0/2548106496 wait complete. 2024-02-16T22:30:28.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.296+0000 7f51fb4dd700 1 Processor -- start 2024-02-16T22:30:28.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.296+0000 7f51fb4dd700 1 -- start start 2024-02-16T22:30:28.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.296+0000 7f51fb4dd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51f4101650 0x7f51f4100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:28.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.296+0000 7f51fb4dd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51f4103f80 0x7f51f40ff600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:28.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.296+0000 7f51fb4dd700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f40ffb40 con 0x7f51f4101650 2024-02-16T22:30:28.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.296+0000 7f51fb4dd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f40ffc80 con 0x7f51f4103f80 2024-02-16T22:30:28.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.296+0000 7f51f9279700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51f4101650 0x7f51f4100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:28.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.296+0000 7f51f9279700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51f4101650 0x7f51f4100fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:54004/0 (socket says 172.21.15.114:54004) 2024-02-16T22:30:28.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.296+0000 7f51f9279700 1 -- 172.21.15.114:0/1354033934 learned_addr learned my addr 172.21.15.114:0/1354033934 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:30:28.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.297+0000 7f51f8a78700 1 --2- 172.21.15.114:0/1354033934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51f4103f80 0x7f51f40ff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:28.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.297+0000 7f51f9279700 1 -- 172.21.15.114:0/1354033934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51f4103f80 msgr2=0x7f51f40ff600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:28.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.297+0000 7f51f9279700 1 --2- 172.21.15.114:0/1354033934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51f4103f80 0x7f51f40ff600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:28.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.297+0000 7f51f9279700 1 -- 172.21.15.114:0/1354033934 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e40095f0 con 0x7f51f4101650 2024-02-16T22:30:28.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.297+0000 7f51f9279700 1 --2- 172.21.15.114:0/1354033934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51f4101650 0x7f51f4100fb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f51f000e920 tx=0x7f51f000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:30:28.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.297+0000 7f51ea7fc700 1 -- 172.21.15.114:0/1354033934 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51f000c9d0 con 0x7f51f4101650 2024-02-16T22:30:28.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.297+0000 7f51ea7fc700 1 -- 172.21.15.114:0/1354033934 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f51f0010430 con 0x7f51f4101650 2024-02-16T22:30:28.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.297+0000 7f51fb4dd700 1 -- 172.21.15.114:0/1354033934 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f40fff60 con 0x7f51f4101650 2024-02-16T22:30:28.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.297+0000 7f51ea7fc700 1 -- 172.21.15.114:0/1354033934 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51f0018650 con 0x7f51f4101650 2024-02-16T22:30:28.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.297+0000 7f51fb4dd700 1 -- 172.21.15.114:0/1354033934 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f419e3b0 con 0x7f51f4101650 2024-02-16T22:30:28.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.298+0000 7f51fb4dd700 1 -- 172.21.15.114:0/1354033934 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f4190530 con 0x7f51f4101650 2024-02-16T22:30:28.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.298+0000 7f51ea7fc700 1 -- 172.21.15.114:0/1354033934 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f51f00105a0 con 0x7f51f4101650 2024-02-16T22:30:28.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.299+0000 7f51ea7fc700 1 --2- 172.21.15.114:0/1354033934 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f51e006d8a0 0x7f51e006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:28.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.299+0000 7f51ea7fc700 1 -- 172.21.15.114:0/1354033934 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f51f0014070 con 0x7f51f4101650 2024-02-16T22:30:28.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.299+0000 7f51f8a78700 1 --2- 172.21.15.114:0/1354033934 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f51e006d8a0 0x7f51e006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:28.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.299+0000 7f51f8a78700 1 --2- 172.21.15.114:0/1354033934 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f51e006d8a0 0x7f51e006fd60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f51e4009fd0 tx=0x7f51e4005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:30:28.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.304+0000 7f51ea7fc700 1 -- 172.21.15.114:0/1354033934 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f51f0052d20 con 0x7f51f4101650 2024-02-16T22:30:28.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.590+0000 7f51fb4dd700 1 -- 172.21.15.114:0/1354033934 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f51f40619a0 con 0x7f51f4101650 2024-02-16T22:30:28.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.590+0000 7f51ea7fc700 1 -- 172.21.15.114:0/1354033934 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f51f005ab10 con 0x7f51f4101650 2024-02-16T22:30:28.592 INFO:teuthology.orchestra.run.smithi114.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:30:28.593 INFO:teuthology.orchestra.run.smithi114.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:30:28.593 INFO:teuthology.orchestra.run.smithi114.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:30:28.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.593+0000 7f51fb4dd700 1 -- 172.21.15.114:0/1354033934 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f51e006d8a0 msgr2=0x7f51e006fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:28.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.593+0000 7f51fb4dd700 1 --2- 172.21.15.114:0/1354033934 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f51e006d8a0 0x7f51e006fd60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f51e4009fd0 tx=0x7f51e4005940 comp rx=0 tx=0).stop 2024-02-16T22:30:28.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.593+0000 7f51fb4dd700 1 -- 172.21.15.114:0/1354033934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51f4101650 msgr2=0x7f51f4100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:28.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.593+0000 7f51fb4dd700 1 --2- 172.21.15.114:0/1354033934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51f4101650 0x7f51f4100fb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f51f000e920 tx=0x7f51f000ec30 comp rx=0 tx=0).stop 2024-02-16T22:30:28.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.593+0000 7f51fb4dd700 1 -- 172.21.15.114:0/1354033934 shutdown_connections 2024-02-16T22:30:28.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.593+0000 7f51fb4dd700 1 --2- 172.21.15.114:0/1354033934 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f51e006d8a0 0x7f51e006fd60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:28.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.593+0000 7f51fb4dd700 1 --2- 172.21.15.114:0/1354033934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51f4101650 0x7f51f4100fb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:28.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.593+0000 7f51fb4dd700 1 --2- 172.21.15.114:0/1354033934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51f4103f80 0x7f51f40ff600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:28.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.593+0000 7f51fb4dd700 1 -- 172.21.15.114:0/1354033934 >> 172.21.15.114:0/1354033934 conn(0x7f51f40fb060 msgr2=0x7f51f40fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:30:28.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.593+0000 7f51fb4dd700 1 -- 172.21.15.114:0/1354033934 shutdown_connections 2024-02-16T22:30:28.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:28.593+0000 7f51fb4dd700 1 -- 172.21.15.114:0/1354033934 wait complete. 2024-02-16T22:30:28.901 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:28 smithi114 ceph-mon[94452]: pgmap v238: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:28.901 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:28 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/679482125' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:30:28.901 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:28 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:28.901 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:28 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:28.901 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:28 smithi114 ceph-mon[94452]: from='client.34522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:30:28.901 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:28 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/1354033934' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:30:29.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:28 smithi123 ceph-mon[82331]: pgmap v238: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:29.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:28 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/679482125' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:30:29.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:28 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:29.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:28 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:29.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:28 smithi123 ceph-mon[82331]: from='client.34522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:30:29.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:28 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/1354033934' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:30:30.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:30 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:30.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:30 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:30.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:30 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:30.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:30 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:31.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:31 smithi123 ceph-mon[82331]: pgmap v239: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:31.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:31 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:31.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:31 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:31.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:31 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:30:31.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:31 smithi114 ceph-mon[94452]: pgmap v239: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:31.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:31 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:31.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:31 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:31.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:31 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:30:32.530 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:32 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:32.530 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:32 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:32.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:32 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:32.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:32 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:33.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:33 smithi114 ceph-mon[94452]: pgmap v240: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:33.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:33 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:33.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:33 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:34.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:33 smithi123 ceph-mon[82331]: pgmap v240: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:34.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:33 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:34.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:33 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:35.741 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:35 smithi114 ceph-mon[94452]: pgmap v241: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:35.741 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:35 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:35.741 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:35 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:35.742 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:35 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:30:35.742 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:35 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:30:35.742 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:35 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:35.742 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:35 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:30:35.742 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:35 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:35.742 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:35 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:35.742 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:35 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:35.742 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:35 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:35.742 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:35 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:35.742 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:35 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:35.742 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:35 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:35.742 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:35 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:35.742 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:35 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:35.742 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:35 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:35.742 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:35 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:35.743 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:35 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:35.743 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:35 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:36.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:35 smithi123 ceph-mon[82331]: pgmap v241: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:36.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:35 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:36.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:35 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:36.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:35 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:30:36.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:35 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:30:36.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:35 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:36.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:35 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:30:36.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:35 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:36.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:35 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:36.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:35 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:36.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:35 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:36.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:35 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:36.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:35 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:36.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:35 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:36.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:35 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:36.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:35 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:36.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:35 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:36.053 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:35 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:36.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:35 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:36.054 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:35 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:37.791 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:37 smithi114 ceph-mon[94452]: pgmap v242: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:37.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:37 smithi123 ceph-mon[82331]: pgmap v242: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:38.794 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:38 smithi114 ceph-mon[94452]: Upgrade: Updating prometheus.smithi114 2024-02-16T22:30:38.795 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:38 smithi114 ceph-mon[94452]: Deploying daemon prometheus.smithi114 on smithi114 2024-02-16T22:30:39.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:38 smithi123 ceph-mon[82331]: Upgrade: Updating prometheus.smithi114 2024-02-16T22:30:39.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:38 smithi123 ceph-mon[82331]: Deploying daemon prometheus.smithi114 on smithi114 2024-02-16T22:30:39.890 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:39 smithi114 ceph-mon[94452]: pgmap v243: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:40.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:39 smithi123 ceph-mon[82331]: pgmap v243: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:41.794 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:41 smithi114 ceph-mon[94452]: pgmap v244: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:42.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:41 smithi123 ceph-mon[82331]: pgmap v244: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:43.303 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:43 smithi114 ceph-mon[94452]: pgmap v245: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:43.304 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:43 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:43.304 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:43 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:43.304 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:43 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:30:43.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:43 smithi123 ceph-mon[82331]: pgmap v245: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:43.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:43 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:43.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:43 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:43.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:43 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:30:45.377 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:45 smithi114 ceph-mon[94452]: pgmap v246: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:45.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:45 smithi123 ceph-mon[82331]: pgmap v246: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:46.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:30:46.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:46 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:30:47.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:47 smithi114 ceph-mon[94452]: pgmap v247: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:47.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:47 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:47.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:47 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:47.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:47 smithi123 ceph-mon[82331]: pgmap v247: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:47.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:47 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:47.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:47 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:49.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:48 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:49.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:48 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:49.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:48 smithi123 ceph-mon[82331]: pgmap v248: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:49.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:48 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:49.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:48 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:49.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:48 smithi114 ceph-mon[94452]: pgmap v248: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:50.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:50 smithi114 ceph-mon[94452]: pgmap v249: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:51.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:50 smithi123 ceph-mon[82331]: pgmap v249: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:52.651 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:52 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:52.651 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:52 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:52.651 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:52 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:30:52.651 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:52 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:30:52.651 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:52 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:52.651 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:52 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T22:30:52.652 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:52 smithi114 ceph-mon[94452]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T22:30:52.652 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:52 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:30:52.652 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:52 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:52.652 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:52 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:52.652 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:52 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:52.652 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:52 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:52.652 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:52 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:52.652 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:52 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:52.652 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:52 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:52.652 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:52 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:52.652 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:52 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:52.653 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:52 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:52.653 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:52 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:52.653 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:52 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:52.653 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:52 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:52.653 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:52 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:52.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:52 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:52.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:52 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:52.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:52 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:30:52.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:52 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:30:52.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:52 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:52.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:52 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T22:30:52.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:52 smithi123 ceph-mon[82331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T22:30:52.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:52 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:30:52.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:52 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:52.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:52 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:52.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:52 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:52.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:52 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:52.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:52 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:52.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:52 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:52.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:52 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:52.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:52 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:52.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:52 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:52.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:52 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:52.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:52 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:52.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:52 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:52.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:52 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:52.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:52 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:30:53.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:53 smithi123 ceph-mon[82331]: pgmap v250: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:53.653 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:53 smithi114 ceph-mon[94452]: pgmap v250: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:54.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:54 smithi123 ceph-mon[82331]: Upgrade: Updating alertmanager.smithi114 2024-02-16T22:30:54.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:54 smithi123 ceph-mon[82331]: Deploying daemon alertmanager.smithi114 on smithi114 2024-02-16T22:30:54.695 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:54 smithi114 ceph-mon[94452]: Upgrade: Updating alertmanager.smithi114 2024-02-16T22:30:54.695 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:54 smithi114 ceph-mon[94452]: Deploying daemon alertmanager.smithi114 on smithi114 2024-02-16T22:30:55.508 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:55 smithi114 ceph-mon[94452]: pgmap v251: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:55.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:55 smithi123 ceph-mon[82331]: pgmap v251: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:57.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:57 smithi114 ceph-mon[94452]: pgmap v252: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:57.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:57 smithi123 ceph-mon[82331]: pgmap v252: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:58.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.714+0000 7fe366fd7700 1 -- 172.21.15.114:0/1088642632 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe360101db0 msgr2=0x7fe360102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:58.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.714+0000 7fe366fd7700 1 --2- 172.21.15.114:0/1088642632 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe360101db0 0x7fe360102210 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe354009910 tx=0x7fe354009c20 comp rx=0 tx=0).stop 2024-02-16T22:30:58.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.714+0000 7fe366fd7700 1 -- 172.21.15.114:0/1088642632 shutdown_connections 2024-02-16T22:30:58.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.714+0000 7fe366fd7700 1 --2- 172.21.15.114:0/1088642632 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe360101db0 0x7fe360102210 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:58.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.714+0000 7fe366fd7700 1 --2- 172.21.15.114:0/1088642632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe360100bb0 0x7fe360100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:58.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.714+0000 7fe366fd7700 1 -- 172.21.15.114:0/1088642632 >> 172.21.15.114:0/1088642632 conn(0x7fe3600fc110 msgr2=0x7fe3600fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:30:58.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.714+0000 7fe366fd7700 1 -- 172.21.15.114:0/1088642632 shutdown_connections 2024-02-16T22:30:58.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.715+0000 7fe366fd7700 1 -- 172.21.15.114:0/1088642632 wait complete. 2024-02-16T22:30:58.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.715+0000 7fe366fd7700 1 Processor -- start 2024-02-16T22:30:58.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.715+0000 7fe366fd7700 1 -- start start 2024-02-16T22:30:58.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.715+0000 7fe366fd7700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe360100bb0 0x7fe3601985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:58.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.715+0000 7fe366fd7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe360101db0 0x7fe360198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:58.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.715+0000 7fe366fd7700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe360199150 con 0x7fe360100bb0 2024-02-16T22:30:58.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.715+0000 7fe366fd7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe360199290 con 0x7fe360101db0 2024-02-16T22:30:58.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.715+0000 7fe364d73700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe360100bb0 0x7fe3601985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:58.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.715+0000 7fe364d73700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe360100bb0 0x7fe3601985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46168/0 (socket says 172.21.15.114:46168) 2024-02-16T22:30:58.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.715+0000 7fe364d73700 1 -- 172.21.15.114:0/4095597265 learned_addr learned my addr 172.21.15.114:0/4095597265 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:30:58.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.715+0000 7fe35ffff700 1 --2- 172.21.15.114:0/4095597265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe360101db0 0x7fe360198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:58.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.716+0000 7fe364d73700 1 -- 172.21.15.114:0/4095597265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe360101db0 msgr2=0x7fe360198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:58.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.716+0000 7fe364d73700 1 --2- 172.21.15.114:0/4095597265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe360101db0 0x7fe360198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:58.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.716+0000 7fe364d73700 1 -- 172.21.15.114:0/4095597265 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3540095f0 con 0x7fe360100bb0 2024-02-16T22:30:58.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.716+0000 7fe364d73700 1 --2- 172.21.15.114:0/4095597265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe360100bb0 0x7fe3601985f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe35800b9c0 tx=0x7fe35800bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:30:58.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.716+0000 7fe35dffb700 1 -- 172.21.15.114:0/4095597265 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe35800d8b0 con 0x7fe360100bb0 2024-02-16T22:30:58.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.716+0000 7fe366fd7700 1 -- 172.21.15.114:0/4095597265 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36019dd40 con 0x7fe360100bb0 2024-02-16T22:30:58.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.716+0000 7fe35dffb700 1 -- 172.21.15.114:0/4095597265 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe358015420 con 0x7fe360100bb0 2024-02-16T22:30:58.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.716+0000 7fe35dffb700 1 -- 172.21.15.114:0/4095597265 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe35800c610 con 0x7fe360100bb0 2024-02-16T22:30:58.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.716+0000 7fe366fd7700 1 -- 172.21.15.114:0/4095597265 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36019e2b0 con 0x7fe360100bb0 2024-02-16T22:30:58.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.717+0000 7fe366fd7700 1 -- 172.21.15.114:0/4095597265 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe360105e90 con 0x7fe360100bb0 2024-02-16T22:30:58.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.718+0000 7fe35dffb700 1 -- 172.21.15.114:0/4095597265 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fe358015590 con 0x7fe360100bb0 2024-02-16T22:30:58.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.718+0000 7fe35dffb700 1 --2- 172.21.15.114:0/4095597265 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe34806d8a0 0x7fe34806fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:58.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.718+0000 7fe35dffb700 1 -- 172.21.15.114:0/4095597265 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7fe358011070 con 0x7fe360100bb0 2024-02-16T22:30:58.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.718+0000 7fe35ffff700 1 --2- 172.21.15.114:0/4095597265 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe34806d8a0 0x7fe34806fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:58.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.719+0000 7fe35ffff700 1 --2- 172.21.15.114:0/4095597265 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe34806d8a0 0x7fe34806fd60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe35400b5c0 tx=0x7fe3540052a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:30:58.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.723+0000 7fe35dffb700 1 -- 172.21.15.114:0/4095597265 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe35804fd20 con 0x7fe360100bb0 2024-02-16T22:30:58.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.944+0000 7fe366fd7700 1 -- 172.21.15.114:0/4095597265 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3600611d0 con 0x7fe34806d8a0 2024-02-16T22:30:58.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.946+0000 7fe35dffb700 1 -- 172.21.15.114:0/4095597265 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+465 (secure 0 0 0) 0x7fe3600611d0 con 0x7fe34806d8a0 2024-02-16T22:30:58.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.948+0000 7fe366fd7700 1 -- 172.21.15.114:0/4095597265 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe34806d8a0 msgr2=0x7fe34806fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:58.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.948+0000 7fe366fd7700 1 --2- 172.21.15.114:0/4095597265 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe34806d8a0 0x7fe34806fd60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe35400b5c0 tx=0x7fe3540052a0 comp rx=0 tx=0).stop 2024-02-16T22:30:58.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.948+0000 7fe366fd7700 1 -- 172.21.15.114:0/4095597265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe360100bb0 msgr2=0x7fe3601985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:58.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.948+0000 7fe366fd7700 1 --2- 172.21.15.114:0/4095597265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe360100bb0 0x7fe3601985f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe35800b9c0 tx=0x7fe35800bcd0 comp rx=0 tx=0).stop 2024-02-16T22:30:58.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.948+0000 7fe366fd7700 1 -- 172.21.15.114:0/4095597265 shutdown_connections 2024-02-16T22:30:58.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.948+0000 7fe366fd7700 1 --2- 172.21.15.114:0/4095597265 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe34806d8a0 0x7fe34806fd60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:58.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.948+0000 7fe366fd7700 1 --2- 172.21.15.114:0/4095597265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe360100bb0 0x7fe3601985f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:58.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.948+0000 7fe366fd7700 1 --2- 172.21.15.114:0/4095597265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe360101db0 0x7fe360198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:58.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.948+0000 7fe366fd7700 1 -- 172.21.15.114:0/4095597265 >> 172.21.15.114:0/4095597265 conn(0x7fe3600fc110 msgr2=0x7fe360102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:30:58.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.948+0000 7fe366fd7700 1 -- 172.21.15.114:0/4095597265 shutdown_connections 2024-02-16T22:30:58.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:58.948+0000 7fe366fd7700 1 -- 172.21.15.114:0/4095597265 wait complete. 2024-02-16T22:30:58.960 INFO:teuthology.orchestra.run.smithi114.stdout:true 2024-02-16T22:30:59.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.057+0000 7f6e94fdf700 1 -- 172.21.15.114:0/46675655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e900ffab0 msgr2=0x7f6e900ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:59.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.057+0000 7f6e94fdf700 1 --2- 172.21.15.114:0/46675655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e900ffab0 0x7f6e900ffed0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009910 tx=0x7f6e80009c20 comp rx=0 tx=0).stop 2024-02-16T22:30:59.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.057+0000 7f6e94fdf700 1 -- 172.21.15.114:0/46675655 shutdown_connections 2024-02-16T22:30:59.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.057+0000 7f6e94fdf700 1 --2- 172.21.15.114:0/46675655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e90100cb0 0x7f6e90101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:59.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.057+0000 7f6e94fdf700 1 --2- 172.21.15.114:0/46675655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e900ffab0 0x7f6e900ffed0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:59.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.057+0000 7f6e94fdf700 1 -- 172.21.15.114:0/46675655 >> 172.21.15.114:0/46675655 conn(0x7f6e900fb030 msgr2=0x7f6e900fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:30:59.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.057+0000 7f6e94fdf700 1 -- 172.21.15.114:0/46675655 shutdown_connections 2024-02-16T22:30:59.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.057+0000 7f6e94fdf700 1 -- 172.21.15.114:0/46675655 wait complete. 2024-02-16T22:30:59.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.058+0000 7f6e94fdf700 1 Processor -- start 2024-02-16T22:30:59.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.058+0000 7f6e94fdf700 1 -- start start 2024-02-16T22:30:59.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.058+0000 7f6e94fdf700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e900ffab0 0x7f6e90196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:59.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.058+0000 7f6e94fdf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e90100cb0 0x7f6e901968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:59.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.058+0000 7f6e94fdf700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e90196ec0 con 0x7f6e900ffab0 2024-02-16T22:30:59.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.058+0000 7f6e94fdf700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e90197000 con 0x7f6e90100cb0 2024-02-16T22:30:59.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.058+0000 7f6e8e59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e900ffab0 0x7f6e90196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:59.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.058+0000 7f6e8e59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e900ffab0 0x7f6e90196360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46184/0 (socket says 172.21.15.114:46184) 2024-02-16T22:30:59.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.058+0000 7f6e8e59c700 1 -- 172.21.15.114:0/1195960717 learned_addr learned my addr 172.21.15.114:0/1195960717 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:30:59.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.058+0000 7f6e8dd9b700 1 --2- 172.21.15.114:0/1195960717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e90100cb0 0x7f6e901968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:59.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.059+0000 7f6e8e59c700 1 -- 172.21.15.114:0/1195960717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e90100cb0 msgr2=0x7f6e901968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:59.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.059+0000 7f6e8e59c700 1 --2- 172.21.15.114:0/1195960717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e90100cb0 0x7f6e901968a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:59.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.059+0000 7f6e8e59c700 1 -- 172.21.15.114:0/1195960717 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e800095f0 con 0x7f6e900ffab0 2024-02-16T22:30:59.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.059+0000 7f6e8e59c700 1 --2- 172.21.15.114:0/1195960717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e900ffab0 0x7f6e90196360 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6e8000bda0 tx=0x7f6e8000bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:30:59.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.059+0000 7f6e7f7fe700 1 -- 172.21.15.114:0/1195960717 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e8001d070 con 0x7f6e900ffab0 2024-02-16T22:30:59.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.059+0000 7f6e7f7fe700 1 -- 172.21.15.114:0/1195960717 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6e80022470 con 0x7f6e900ffab0 2024-02-16T22:30:59.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.059+0000 7f6e94fdf700 1 -- 172.21.15.114:0/1195960717 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e9019ba50 con 0x7f6e900ffab0 2024-02-16T22:30:59.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.059+0000 7f6e7f7fe700 1 -- 172.21.15.114:0/1195960717 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e8000f650 con 0x7f6e900ffab0 2024-02-16T22:30:59.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.059+0000 7f6e94fdf700 1 -- 172.21.15.114:0/1195960717 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e9019bf40 con 0x7f6e900ffab0 2024-02-16T22:30:59.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.060+0000 7f6e94fdf700 1 -- 172.21.15.114:0/1195960717 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e90108610 con 0x7f6e900ffab0 2024-02-16T22:30:59.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.060+0000 7f6e7f7fe700 1 -- 172.21.15.114:0/1195960717 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f6e80004af0 con 0x7f6e900ffab0 2024-02-16T22:30:59.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.061+0000 7f6e7f7fe700 1 --2- 172.21.15.114:0/1195960717 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6e7806d8a0 0x7f6e7806fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:59.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.061+0000 7f6e7f7fe700 1 -- 172.21.15.114:0/1195960717 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f6e8008ec50 con 0x7f6e900ffab0 2024-02-16T22:30:59.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.061+0000 7f6e8dd9b700 1 --2- 172.21.15.114:0/1195960717 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6e7806d8a0 0x7f6e7806fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:59.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.061+0000 7f6e8dd9b700 1 --2- 172.21.15.114:0/1195960717 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6e7806d8a0 0x7f6e7806fd60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6e84005950 tx=0x7f6e8400a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:30:59.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.066+0000 7f6e7f7fe700 1 -- 172.21.15.114:0/1195960717 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6e8005c050 con 0x7f6e900ffab0 2024-02-16T22:30:59.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.295+0000 7f6e94fdf700 1 -- 172.21.15.114:0/1195960717 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e9019c640 con 0x7f6e7806d8a0 2024-02-16T22:30:59.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.297+0000 7f6e7f7fe700 1 -- 172.21.15.114:0/1195960717 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+465 (secure 0 0 0) 0x7f6e9019c640 con 0x7f6e7806d8a0 2024-02-16T22:30:59.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.300+0000 7f6e94fdf700 1 -- 172.21.15.114:0/1195960717 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6e7806d8a0 msgr2=0x7f6e7806fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:59.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.300+0000 7f6e94fdf700 1 --2- 172.21.15.114:0/1195960717 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6e7806d8a0 0x7f6e7806fd60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6e84005950 tx=0x7f6e8400a300 comp rx=0 tx=0).stop 2024-02-16T22:30:59.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.300+0000 7f6e94fdf700 1 -- 172.21.15.114:0/1195960717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e900ffab0 msgr2=0x7f6e90196360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:59.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.300+0000 7f6e94fdf700 1 --2- 172.21.15.114:0/1195960717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e900ffab0 0x7f6e90196360 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6e8000bda0 tx=0x7f6e8000bdd0 comp rx=0 tx=0).stop 2024-02-16T22:30:59.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.300+0000 7f6e94fdf700 1 -- 172.21.15.114:0/1195960717 shutdown_connections 2024-02-16T22:30:59.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.300+0000 7f6e94fdf700 1 --2- 172.21.15.114:0/1195960717 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6e7806d8a0 0x7f6e7806fd60 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:59.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.300+0000 7f6e94fdf700 1 --2- 172.21.15.114:0/1195960717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e900ffab0 0x7f6e90196360 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:59.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.300+0000 7f6e94fdf700 1 --2- 172.21.15.114:0/1195960717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e90100cb0 0x7f6e901968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:59.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.300+0000 7f6e94fdf700 1 -- 172.21.15.114:0/1195960717 >> 172.21.15.114:0/1195960717 conn(0x7f6e900fb030 msgr2=0x7f6e90068410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:30:59.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.300+0000 7f6e94fdf700 1 -- 172.21.15.114:0/1195960717 shutdown_connections 2024-02-16T22:30:59.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.300+0000 7f6e94fdf700 1 -- 172.21.15.114:0/1195960717 wait complete. 2024-02-16T22:30:59.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.412+0000 7fe17ed5e700 1 -- 172.21.15.114:0/2969625839 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1780ffab0 msgr2=0x7fe1780ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:59.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.412+0000 7fe17ed5e700 1 --2- 172.21.15.114:0/2969625839 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1780ffab0 0x7fe1780ffed0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe16c009910 tx=0x7fe16c009c20 comp rx=0 tx=0).stop 2024-02-16T22:30:59.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.412+0000 7fe17ed5e700 1 -- 172.21.15.114:0/2969625839 shutdown_connections 2024-02-16T22:30:59.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.412+0000 7fe17ed5e700 1 --2- 172.21.15.114:0/2969625839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe178100cb0 0x7fe178101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:59.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.412+0000 7fe17ed5e700 1 --2- 172.21.15.114:0/2969625839 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1780ffab0 0x7fe1780ffed0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:59.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.412+0000 7fe17ed5e700 1 -- 172.21.15.114:0/2969625839 >> 172.21.15.114:0/2969625839 conn(0x7fe1780fb030 msgr2=0x7fe1780fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:30:59.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.412+0000 7fe17ed5e700 1 -- 172.21.15.114:0/2969625839 shutdown_connections 2024-02-16T22:30:59.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.412+0000 7fe17ed5e700 1 -- 172.21.15.114:0/2969625839 wait complete. 2024-02-16T22:30:59.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.413+0000 7fe17ed5e700 1 Processor -- start 2024-02-16T22:30:59.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.413+0000 7fe17ed5e700 1 -- start start 2024-02-16T22:30:59.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.413+0000 7fe17ed5e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1780ffab0 0x7fe178068f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:59.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.413+0000 7fe17ed5e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe178100cb0 0x7fe17819b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:59.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.413+0000 7fe17ed5e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17819b670 con 0x7fe1780ffab0 2024-02-16T22:30:59.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.413+0000 7fe17ed5e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17819b7b0 con 0x7fe178100cb0 2024-02-16T22:30:59.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.413+0000 7fe17cafa700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1780ffab0 0x7fe178068f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:59.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.413+0000 7fe17cafa700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1780ffab0 0x7fe178068f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46208/0 (socket says 172.21.15.114:46208) 2024-02-16T22:30:59.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.413+0000 7fe17cafa700 1 -- 172.21.15.114:0/2883338112 learned_addr learned my addr 172.21.15.114:0/2883338112 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:30:59.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.413+0000 7fe177fff700 1 --2- 172.21.15.114:0/2883338112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe178100cb0 0x7fe17819b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:59.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.414+0000 7fe17cafa700 1 -- 172.21.15.114:0/2883338112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe178100cb0 msgr2=0x7fe17819b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:59.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.414+0000 7fe17cafa700 1 --2- 172.21.15.114:0/2883338112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe178100cb0 0x7fe17819b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:59.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.414+0000 7fe17cafa700 1 -- 172.21.15.114:0/2883338112 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16c0095f0 con 0x7fe1780ffab0 2024-02-16T22:30:59.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.414+0000 7fe17cafa700 1 --2- 172.21.15.114:0/2883338112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1780ffab0 0x7fe178068f70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe16c00bda0 tx=0x7fe16c00bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:30:59.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.414+0000 7fe175ffb700 1 -- 172.21.15.114:0/2883338112 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe16c01d070 con 0x7fe1780ffab0 2024-02-16T22:30:59.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.414+0000 7fe175ffb700 1 -- 172.21.15.114:0/2883338112 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe16c022470 con 0x7fe1780ffab0 2024-02-16T22:30:59.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.414+0000 7fe17ed5e700 1 -- 172.21.15.114:0/2883338112 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe17819ba30 con 0x7fe1780ffab0 2024-02-16T22:30:59.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.414+0000 7fe175ffb700 1 -- 172.21.15.114:0/2883338112 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe16c00f670 con 0x7fe1780ffab0 2024-02-16T22:30:59.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.414+0000 7fe17ed5e700 1 -- 172.21.15.114:0/2883338112 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe17819bf20 con 0x7fe1780ffab0 2024-02-16T22:30:59.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.416+0000 7fe17ed5e700 1 -- 172.21.15.114:0/2883338112 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe178108610 con 0x7fe1780ffab0 2024-02-16T22:30:59.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.416+0000 7fe175ffb700 1 -- 172.21.15.114:0/2883338112 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fe16c004af0 con 0x7fe1780ffab0 2024-02-16T22:30:59.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.416+0000 7fe175ffb700 1 --2- 172.21.15.114:0/2883338112 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe16806d8a0 0x7fe16806fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:59.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.417+0000 7fe175ffb700 1 -- 172.21.15.114:0/2883338112 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7fe16c08eb50 con 0x7fe1780ffab0 2024-02-16T22:30:59.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.417+0000 7fe177fff700 1 --2- 172.21.15.114:0/2883338112 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe16806d8a0 0x7fe16806fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:59.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.417+0000 7fe177fff700 1 --2- 172.21.15.114:0/2883338112 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe16806d8a0 0x7fe16806fd60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe1780694f0 tx=0x7fe16400a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:30:59.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.421+0000 7fe175ffb700 1 -- 172.21.15.114:0/2883338112 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe16c05bfd0 con 0x7fe1780ffab0 2024-02-16T22:30:59.628 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:59 smithi114 ceph-mon[94452]: pgmap v253: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:59.628 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:59 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:59.628 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:59 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:59.628 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:30:59 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:30:59.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.634+0000 7fe17ed5e700 1 -- 172.21.15.114:0/2883338112 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe178105600 con 0x7fe16806d8a0 2024-02-16T22:30:59.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.645+0000 7fe175ffb700 1 -- 172.21.15.114:0/2883338112 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe178105600 con 0x7fe16806d8a0 2024-02-16T22:30:59.646 INFO:teuthology.orchestra.run.smithi114.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:30:59.646 INFO:teuthology.orchestra.run.smithi114.stdout:alertmanager.smithi114 smithi114 *:9093,9094 starting - - - - 2024-02-16T22:30:59.646 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi114 smithi114 running (57s) 13s ago 18m 7746k - 19.0.0-1450-gf78a58c0 f081f847070a 1f621c33f6d4 2024-02-16T22:30:59.646 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi123 smithi123 running (52s) 31s ago 17m 7482k - 19.0.0-1450-gf78a58c0 f081f847070a 2f0b3c7f313b 2024-02-16T22:30:59.646 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi114 smithi114 running (6m) 13s ago 18m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 9b8ae9efe83b 2024-02-16T22:30:59.647 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi123 smithi123 running (6m) 31s ago 17m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 6d97263a0ffa 2024-02-16T22:30:59.647 INFO:teuthology.orchestra.run.smithi114.stdout:grafana.smithi114 smithi114 *:3000 running (16m) 13s ago 18m 89.1M - 9.4.7 954c08fa6188 db1c26c3bc20 2024-02-16T22:30:59.647 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.fdxpey smithi114 running (2m) 13s ago 12m 13.9M - 19.0.0-1450-gf78a58c0 f081f847070a 2984ae8a955a 2024-02-16T22:30:59.647 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.zzryke smithi114 running (2m) 13s ago 12m 84.9M - 19.0.0-1450-gf78a58c0 f081f847070a 63fa2af7290e 2024-02-16T22:30:59.647 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.gopcfa smithi123 running (71s) 31s ago 12m 14.9M - 19.0.0-1450-gf78a58c0 f081f847070a ec6612a62e2a 2024-02-16T22:30:59.647 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.vhlqbe smithi123 running (94s) 31s ago 12m 15.9M - 19.0.0-1450-gf78a58c0 f081f847070a be790c6287e7 2024-02-16T22:30:59.647 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi114.vcullz smithi114 *:8443,9283,8765 running (8m) 13s ago 19m 512M - 19.0.0-1450-gf78a58c0 f081f847070a 14ebeb7888c0 2024-02-16T22:30:59.647 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi123.inzgjd smithi123 *:8443,9283,8765 running (7m) 31s ago 17m 451M - 19.0.0-1450-gf78a58c0 f081f847070a f10aab9faf9e 2024-02-16T22:30:59.647 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi114 smithi114 running (7m) 13s ago 19m 56.8M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 58ac06372523 2024-02-16T22:30:59.647 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi123 smithi123 running (6m) 31s ago 17m 43.0M 2048M 19.0.0-1450-gf78a58c0 f081f847070a f76485e0ea24 2024-02-16T22:30:59.647 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi114 smithi114 *:9100 running (38s) 13s ago 18m 17.5M - 1.5.0 0da6a335fe13 ee1aa5ff6491 2024-02-16T22:30:59.648 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi123 smithi123 *:9100 running (34s) 31s ago 17m 3657k - 1.5.0 0da6a335fe13 a5e2e31d4643 2024-02-16T22:30:59.648 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 smithi114 running (5m) 13s ago 16m 100M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 1978fdad94a5 2024-02-16T22:30:59.648 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 smithi114 running (5m) 13s ago 16m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 3f90e792898f 2024-02-16T22:30:59.648 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 smithi114 running (4m) 13s ago 15m 91.5M 1216M 19.0.0-1450-gf78a58c0 f081f847070a c663dfa0518b 2024-02-16T22:30:59.648 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 smithi123 running (3m) 31s ago 15m 113M 2240M 19.0.0-1450-gf78a58c0 f081f847070a b921448ba169 2024-02-16T22:30:59.648 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 smithi123 running (3m) 31s ago 14m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 54c2926d56b1 2024-02-16T22:30:59.648 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 smithi123 running (2m) 31s ago 14m 93.6M 2240M 19.0.0-1450-gf78a58c0 f081f847070a dd99bf034749 2024-02-16T22:30:59.648 INFO:teuthology.orchestra.run.smithi114.stdout:prometheus.smithi114 smithi114 *:9095 running (18s) 13s ago 17m 39.8M - 2.43.0 a07b618ecd1d a6b9ebc86282 2024-02-16T22:30:59.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.648+0000 7fe17ed5e700 1 -- 172.21.15.114:0/2883338112 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe16806d8a0 msgr2=0x7fe16806fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:59.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.648+0000 7fe17ed5e700 1 --2- 172.21.15.114:0/2883338112 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe16806d8a0 0x7fe16806fd60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe1780694f0 tx=0x7fe16400a300 comp rx=0 tx=0).stop 2024-02-16T22:30:59.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.648+0000 7fe17ed5e700 1 -- 172.21.15.114:0/2883338112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1780ffab0 msgr2=0x7fe178068f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:59.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.648+0000 7fe17ed5e700 1 --2- 172.21.15.114:0/2883338112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1780ffab0 0x7fe178068f70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe16c00bda0 tx=0x7fe16c00bdd0 comp rx=0 tx=0).stop 2024-02-16T22:30:59.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.648+0000 7fe17ed5e700 1 -- 172.21.15.114:0/2883338112 shutdown_connections 2024-02-16T22:30:59.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.648+0000 7fe17ed5e700 1 --2- 172.21.15.114:0/2883338112 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe16806d8a0 0x7fe16806fd60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:59.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.648+0000 7fe17ed5e700 1 --2- 172.21.15.114:0/2883338112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1780ffab0 0x7fe178068f70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:59.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.648+0000 7fe17ed5e700 1 --2- 172.21.15.114:0/2883338112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe178100cb0 0x7fe17819b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:59.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.648+0000 7fe17ed5e700 1 -- 172.21.15.114:0/2883338112 >> 172.21.15.114:0/2883338112 conn(0x7fe1780fb030 msgr2=0x7fe178103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:30:59.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.648+0000 7fe17ed5e700 1 -- 172.21.15.114:0/2883338112 shutdown_connections 2024-02-16T22:30:59.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.648+0000 7fe17ed5e700 1 -- 172.21.15.114:0/2883338112 wait complete. 2024-02-16T22:30:59.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.755+0000 7efccf2ab700 1 -- 172.21.15.114:0/2136794440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcc8101e00 msgr2=0x7efcc8102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:59.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.755+0000 7efccf2ab700 1 --2- 172.21.15.114:0/2136794440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcc8101e00 0x7efcc8102260 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efcb8009910 tx=0x7efcb8009c20 comp rx=0 tx=0).stop 2024-02-16T22:30:59.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.755+0000 7efccf2ab700 1 -- 172.21.15.114:0/2136794440 shutdown_connections 2024-02-16T22:30:59.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.755+0000 7efccf2ab700 1 --2- 172.21.15.114:0/2136794440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcc8101e00 0x7efcc8102260 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:59.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.755+0000 7efccf2ab700 1 --2- 172.21.15.114:0/2136794440 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcc8100c00 0x7efcc8101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:59.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.755+0000 7efccf2ab700 1 -- 172.21.15.114:0/2136794440 >> 172.21.15.114:0/2136794440 conn(0x7efcc80fc180 msgr2=0x7efcc80fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:30:59.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.755+0000 7efccf2ab700 1 -- 172.21.15.114:0/2136794440 shutdown_connections 2024-02-16T22:30:59.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.755+0000 7efccf2ab700 1 -- 172.21.15.114:0/2136794440 wait complete. 2024-02-16T22:30:59.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.756+0000 7efccf2ab700 1 Processor -- start 2024-02-16T22:30:59.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.756+0000 7efccf2ab700 1 -- start start 2024-02-16T22:30:59.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.756+0000 7efccf2ab700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcc8100c00 0x7efcc8198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:59.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.756+0000 7efccf2ab700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcc8101e00 0x7efcc8198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:59.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.756+0000 7efccf2ab700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc8199170 con 0x7efcc8100c00 2024-02-16T22:30:59.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.756+0000 7efccf2ab700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc81992b0 con 0x7efcc8101e00 2024-02-16T22:30:59.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.756+0000 7efccd047700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcc8100c00 0x7efcc8198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:59.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.756+0000 7efccd047700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcc8100c00 0x7efcc8198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46220/0 (socket says 172.21.15.114:46220) 2024-02-16T22:30:59.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.756+0000 7efccd047700 1 -- 172.21.15.114:0/3794869183 learned_addr learned my addr 172.21.15.114:0/3794869183 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:30:59.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.757+0000 7efccc846700 1 --2- 172.21.15.114:0/3794869183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcc8101e00 0x7efcc8198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:59.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.757+0000 7efccd047700 1 -- 172.21.15.114:0/3794869183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcc8101e00 msgr2=0x7efcc8198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:30:59.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.757+0000 7efccd047700 1 --2- 172.21.15.114:0/3794869183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcc8101e00 0x7efcc8198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:30:59.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.757+0000 7efccd047700 1 -- 172.21.15.114:0/3794869183 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb80095f0 con 0x7efcc8100c00 2024-02-16T22:30:59.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.757+0000 7efccc846700 1 --2- 172.21.15.114:0/3794869183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcc8101e00 0x7efcc8198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:30:59.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.757+0000 7efccd047700 1 --2- 172.21.15.114:0/3794869183 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcc8100c00 0x7efcc8198610 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efcc400b480 tx=0x7efcc400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:30:59.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.757+0000 7efcbe7fc700 1 -- 172.21.15.114:0/3794869183 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efcc40118c0 con 0x7efcc8100c00 2024-02-16T22:30:59.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.757+0000 7efccf2ab700 1 -- 172.21.15.114:0/3794869183 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc819dd60 con 0x7efcc8100c00 2024-02-16T22:30:59.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.757+0000 7efcbe7fc700 1 -- 172.21.15.114:0/3794869183 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efcc4010380 con 0x7efcc8100c00 2024-02-16T22:30:59.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.757+0000 7efcbe7fc700 1 -- 172.21.15.114:0/3794869183 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efcc400f4f0 con 0x7efcc8100c00 2024-02-16T22:30:59.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.757+0000 7efccf2ab700 1 -- 172.21.15.114:0/3794869183 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc819e2d0 con 0x7efcc8100c00 2024-02-16T22:30:59.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.758+0000 7efccf2ab700 1 -- 172.21.15.114:0/3794869183 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcc8105ee0 con 0x7efcc8100c00 2024-02-16T22:30:59.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.759+0000 7efcbe7fc700 1 -- 172.21.15.114:0/3794869183 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7efcc40104f0 con 0x7efcc8100c00 2024-02-16T22:30:59.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.759+0000 7efcbe7fc700 1 --2- 172.21.15.114:0/3794869183 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7efcb406d8a0 0x7efcb406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:30:59.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.759+0000 7efcbe7fc700 1 -- 172.21.15.114:0/3794869183 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7efcc408d220 con 0x7efcc8100c00 2024-02-16T22:30:59.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.759+0000 7efccc846700 1 --2- 172.21.15.114:0/3794869183 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7efcb406d8a0 0x7efcb406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:30:59.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.759+0000 7efccc846700 1 --2- 172.21.15.114:0/3794869183 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7efcb406d8a0 0x7efcb406fd60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efcb8005b40 tx=0x7efcb8005ab0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:30:59.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:30:59.764+0000 7efcbe7fc700 1 -- 172.21.15.114:0/3794869183 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efcc405a450 con 0x7efcc8100c00 2024-02-16T22:30:59.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:59 smithi123 ceph-mon[82331]: pgmap v253: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:30:59.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:59 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:59.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:59 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:30:59.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:30:59 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:31:00.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.044+0000 7efccf2ab700 1 -- 172.21.15.114:0/3794869183 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7efcc802d090 con 0x7efcc8100c00 2024-02-16T22:31:00.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.044+0000 7efcbe7fc700 1 -- 172.21.15.114:0/3794869183 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7efcc4059fe0 con 0x7efcc8100c00 2024-02-16T22:31:00.046 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:31:00.046 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": { 2024-02-16T22:31:00.046 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:31:00.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:31:00.046 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": { 2024-02-16T22:31:00.046 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:31:00.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:31:00.046 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": { 2024-02-16T22:31:00.046 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T22:31:00.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:31:00.046 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-02-16T22:31:00.046 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T22:31:00.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:31:00.046 INFO:teuthology.orchestra.run.smithi114.stdout: "overall": { 2024-02-16T22:31:00.047 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-16T22:31:00.047 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-02-16T22:31:00.047 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:31:00.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.047+0000 7efccf2ab700 1 -- 172.21.15.114:0/3794869183 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7efcb406d8a0 msgr2=0x7efcb406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:00.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.047+0000 7efccf2ab700 1 --2- 172.21.15.114:0/3794869183 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7efcb406d8a0 0x7efcb406fd60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efcb8005b40 tx=0x7efcb8005ab0 comp rx=0 tx=0).stop 2024-02-16T22:31:00.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.047+0000 7efccf2ab700 1 -- 172.21.15.114:0/3794869183 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcc8100c00 msgr2=0x7efcc8198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:00.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.047+0000 7efccf2ab700 1 --2- 172.21.15.114:0/3794869183 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcc8100c00 0x7efcc8198610 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efcc400b480 tx=0x7efcc400b840 comp rx=0 tx=0).stop 2024-02-16T22:31:00.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.047+0000 7efccf2ab700 1 -- 172.21.15.114:0/3794869183 shutdown_connections 2024-02-16T22:31:00.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.047+0000 7efccf2ab700 1 --2- 172.21.15.114:0/3794869183 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7efcb406d8a0 0x7efcb406fd60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:00.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.047+0000 7efccf2ab700 1 --2- 172.21.15.114:0/3794869183 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcc8100c00 0x7efcc8198610 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:00.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.047+0000 7efccf2ab700 1 --2- 172.21.15.114:0/3794869183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcc8101e00 0x7efcc8198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:00.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.047+0000 7efccf2ab700 1 -- 172.21.15.114:0/3794869183 >> 172.21.15.114:0/3794869183 conn(0x7efcc80fc180 msgr2=0x7efcc8103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:31:00.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.047+0000 7efccf2ab700 1 -- 172.21.15.114:0/3794869183 shutdown_connections 2024-02-16T22:31:00.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.047+0000 7efccf2ab700 1 -- 172.21.15.114:0/3794869183 wait complete. 2024-02-16T22:31:00.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.151+0000 7ff2178b2700 1 -- 172.21.15.114:0/545369045 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2100fec70 msgr2=0x7ff2100ff0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:00.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.151+0000 7ff2178b2700 1 --2- 172.21.15.114:0/545369045 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2100fec70 0x7ff2100ff0d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff20c009910 tx=0x7ff20c009c20 comp rx=0 tx=0).stop 2024-02-16T22:31:00.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.151+0000 7ff2178b2700 1 -- 172.21.15.114:0/545369045 shutdown_connections 2024-02-16T22:31:00.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.151+0000 7ff2178b2700 1 --2- 172.21.15.114:0/545369045 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2100fec70 0x7ff2100ff0d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:00.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.151+0000 7ff2178b2700 1 --2- 172.21.15.114:0/545369045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2101021f0 0x7ff210102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:00.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.151+0000 7ff2178b2700 1 -- 172.21.15.114:0/545369045 >> 172.21.15.114:0/545369045 conn(0x7ff2100fa810 msgr2=0x7ff2100fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:31:00.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.151+0000 7ff2178b2700 1 -- 172.21.15.114:0/545369045 shutdown_connections 2024-02-16T22:31:00.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.151+0000 7ff2178b2700 1 -- 172.21.15.114:0/545369045 wait complete. 2024-02-16T22:31:00.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.152+0000 7ff2178b2700 1 Processor -- start 2024-02-16T22:31:00.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.152+0000 7ff2178b2700 1 -- start start 2024-02-16T22:31:00.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.152+0000 7ff2178b2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2100fec70 0x7ff210196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:31:00.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.152+0000 7ff2178b2700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2101021f0 0x7ff2101969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:31:00.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.152+0000 7ff2178b2700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff210196fc0 con 0x7ff2101021f0 2024-02-16T22:31:00.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.152+0000 7ff2178b2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff210197100 con 0x7ff2100fec70 2024-02-16T22:31:00.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.152+0000 7ff214e4d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2101021f0 0x7ff2101969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:31:00.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.152+0000 7ff214e4d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2101021f0 0x7ff2101969a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46240/0 (socket says 172.21.15.114:46240) 2024-02-16T22:31:00.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.152+0000 7ff214e4d700 1 -- 172.21.15.114:0/2134350227 learned_addr learned my addr 172.21.15.114:0/2134350227 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:31:00.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.152+0000 7ff21564e700 1 --2- 172.21.15.114:0/2134350227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2100fec70 0x7ff210196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:31:00.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.153+0000 7ff214e4d700 1 -- 172.21.15.114:0/2134350227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2100fec70 msgr2=0x7ff210196460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:00.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.153+0000 7ff214e4d700 1 --2- 172.21.15.114:0/2134350227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2100fec70 0x7ff210196460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:00.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.153+0000 7ff214e4d700 1 -- 172.21.15.114:0/2134350227 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff20c0095f0 con 0x7ff2101021f0 2024-02-16T22:31:00.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.153+0000 7ff214e4d700 1 --2- 172.21.15.114:0/2134350227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2101021f0 0x7ff2101969a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff20c006010 tx=0x7ff20c0055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:31:00.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.153+0000 7ff2067fc700 1 -- 172.21.15.114:0/2134350227 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff20c01d070 con 0x7ff2101021f0 2024-02-16T22:31:00.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.153+0000 7ff2067fc700 1 -- 172.21.15.114:0/2134350227 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff20c00f460 con 0x7ff2101021f0 2024-02-16T22:31:00.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.153+0000 7ff2178b2700 1 -- 172.21.15.114:0/2134350227 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff21019bb50 con 0x7ff2101021f0 2024-02-16T22:31:00.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.153+0000 7ff2067fc700 1 -- 172.21.15.114:0/2134350227 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff20c021640 con 0x7ff2101021f0 2024-02-16T22:31:00.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.153+0000 7ff2178b2700 1 -- 172.21.15.114:0/2134350227 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff21019c0c0 con 0x7ff2101021f0 2024-02-16T22:31:00.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.154+0000 7ff2178b2700 1 -- 172.21.15.114:0/2134350227 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff210109780 con 0x7ff2101021f0 2024-02-16T22:31:00.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.154+0000 7ff2067fc700 1 -- 172.21.15.114:0/2134350227 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7ff20c00f5d0 con 0x7ff2101021f0 2024-02-16T22:31:00.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.155+0000 7ff2067fc700 1 --2- 172.21.15.114:0/2134350227 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff1fc06d8a0 0x7ff1fc06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:31:00.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.155+0000 7ff2067fc700 1 -- 172.21.15.114:0/2134350227 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7ff20c08e870 con 0x7ff2101021f0 2024-02-16T22:31:00.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.155+0000 7ff21564e700 1 --2- 172.21.15.114:0/2134350227 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff1fc06d8a0 0x7ff1fc06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:31:00.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.155+0000 7ff21564e700 1 --2- 172.21.15.114:0/2134350227 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff1fc06d8a0 0x7ff1fc06fd60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff210068ad0 tx=0x7ff20000a380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:31:00.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.160+0000 7ff2067fc700 1 -- 172.21.15.114:0/2134350227 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff20c05baa0 con 0x7ff2101021f0 2024-02-16T22:31:00.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.408+0000 7ff2178b2700 1 -- 172.21.15.114:0/2134350227 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff21019c790 con 0x7ff2101021f0 2024-02-16T22:31:00.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.408+0000 7ff2067fc700 1 -- 172.21.15.114:0/2134350227 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1876 (secure 0 0 0) 0x7ff20c05b630 con 0x7ff2101021f0 2024-02-16T22:31:00.411 INFO:teuthology.orchestra.run.smithi114.stdout:e37 2024-02-16T22:31:00.411 INFO:teuthology.orchestra.run.smithi114.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:31:00.411 INFO:teuthology.orchestra.run.smithi114.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:31:00.411 INFO:teuthology.orchestra.run.smithi114.stdout:legacy client fscid: 1 2024-02-16T22:31:00.411 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:31:00.411 INFO:teuthology.orchestra.run.smithi114.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:31:00.411 INFO:teuthology.orchestra.run.smithi114.stdout:fs_name cephfs 2024-02-16T22:31:00.411 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 37 2024-02-16T22:31:00.411 INFO:teuthology.orchestra.run.smithi114.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:31:00.411 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-02-16T22:18:02.108825+0000 2024-02-16T22:31:00.411 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-02-16T22:29:48.901292+0000 2024-02-16T22:31:00.411 INFO:teuthology.orchestra.run.smithi114.stdout:tableserver 0 2024-02-16T22:31:00.411 INFO:teuthology.orchestra.run.smithi114.stdout:root 0 2024-02-16T22:31:00.411 INFO:teuthology.orchestra.run.smithi114.stdout:session_timeout 60 2024-02-16T22:31:00.411 INFO:teuthology.orchestra.run.smithi114.stdout:session_autoclose 300 2024-02-16T22:31:00.412 INFO:teuthology.orchestra.run.smithi114.stdout:max_file_size 1099511627776 2024-02-16T22:31:00.412 INFO:teuthology.orchestra.run.smithi114.stdout:max_xattr_size 65536 2024-02-16T22:31:00.412 INFO:teuthology.orchestra.run.smithi114.stdout:required_client_features {} 2024-02-16T22:31:00.412 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure 0 2024-02-16T22:31:00.412 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure_osd_epoch 82 2024-02-16T22:31:00.412 INFO:teuthology.orchestra.run.smithi114.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-16T22:31:00.412 INFO:teuthology.orchestra.run.smithi114.stdout:max_mds 1 2024-02-16T22:31:00.412 INFO:teuthology.orchestra.run.smithi114.stdout:in 0 2024-02-16T22:31:00.412 INFO:teuthology.orchestra.run.smithi114.stdout:up {0=34404} 2024-02-16T22:31:00.412 INFO:teuthology.orchestra.run.smithi114.stdout:failed 2024-02-16T22:31:00.412 INFO:teuthology.orchestra.run.smithi114.stdout:damaged 2024-02-16T22:31:00.412 INFO:teuthology.orchestra.run.smithi114.stdout:stopped 2024-02-16T22:31:00.412 INFO:teuthology.orchestra.run.smithi114.stdout:data_pools [3] 2024-02-16T22:31:00.413 INFO:teuthology.orchestra.run.smithi114.stdout:metadata_pool 2 2024-02-16T22:31:00.413 INFO:teuthology.orchestra.run.smithi114.stdout:inline_data enabled 2024-02-16T22:31:00.413 INFO:teuthology.orchestra.run.smithi114.stdout:balancer 2024-02-16T22:31:00.413 INFO:teuthology.orchestra.run.smithi114.stdout:bal_rank_mask -1 2024-02-16T22:31:00.413 INFO:teuthology.orchestra.run.smithi114.stdout:standby_count_wanted 1 2024-02-16T22:31:00.413 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.zzryke{0:34404} state up:active seq 23 join_fscid=1 addr [v2:172.21.15.114:6826/2017800151,v1:172.21.15.114:6827/2017800151] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T22:31:00.413 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:31:00.413 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:31:00.413 INFO:teuthology.orchestra.run.smithi114.stdout:Standby daemons: 2024-02-16T22:31:00.413 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:31:00.413 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.fdxpey{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.114:6828/3702914193,v1:172.21.15.114:6829/3702914193] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T22:31:00.413 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.vhlqbe{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.123:6824/1425065679,v1:172.21.15.123:6825/1425065679] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T22:31:00.413 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.gopcfa{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.123:6826/3128771108,v1:172.21.15.123:6827/3128771108] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T22:31:00.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.411+0000 7ff2178b2700 1 -- 172.21.15.114:0/2134350227 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff1fc06d8a0 msgr2=0x7ff1fc06fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:00.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.411+0000 7ff2178b2700 1 --2- 172.21.15.114:0/2134350227 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff1fc06d8a0 0x7ff1fc06fd60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff210068ad0 tx=0x7ff20000a380 comp rx=0 tx=0).stop 2024-02-16T22:31:00.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.411+0000 7ff2178b2700 1 -- 172.21.15.114:0/2134350227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2101021f0 msgr2=0x7ff2101969a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:00.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.411+0000 7ff2178b2700 1 --2- 172.21.15.114:0/2134350227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2101021f0 0x7ff2101969a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff20c006010 tx=0x7ff20c0055f0 comp rx=0 tx=0).stop 2024-02-16T22:31:00.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.411+0000 7ff2178b2700 1 -- 172.21.15.114:0/2134350227 shutdown_connections 2024-02-16T22:31:00.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.411+0000 7ff2178b2700 1 --2- 172.21.15.114:0/2134350227 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff1fc06d8a0 0x7ff1fc06fd60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:00.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.411+0000 7ff2178b2700 1 --2- 172.21.15.114:0/2134350227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2100fec70 0x7ff210196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:00.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.411+0000 7ff2178b2700 1 --2- 172.21.15.114:0/2134350227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2101021f0 0x7ff2101969a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:00.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.411+0000 7ff2178b2700 1 -- 172.21.15.114:0/2134350227 >> 172.21.15.114:0/2134350227 conn(0x7ff2100fa810 msgr2=0x7ff2101051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:31:00.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.412+0000 7ff2178b2700 1 -- 172.21.15.114:0/2134350227 shutdown_connections 2024-02-16T22:31:00.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.412+0000 7ff2178b2700 1 -- 172.21.15.114:0/2134350227 wait complete. 2024-02-16T22:31:00.415 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 37 2024-02-16T22:31:00.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.527+0000 7f6e18594700 1 -- 172.21.15.114:0/3115564744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e10103f70 msgr2=0x7f6e10106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:00.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.527+0000 7f6e18594700 1 --2- 172.21.15.114:0/3115564744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e10103f70 0x7f6e10106360 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c009910 tx=0x7f6e0c009c20 comp rx=0 tx=0).stop 2024-02-16T22:31:00.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.527+0000 7f6e18594700 1 -- 172.21.15.114:0/3115564744 shutdown_connections 2024-02-16T22:31:00.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.527+0000 7f6e18594700 1 --2- 172.21.15.114:0/3115564744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e10103f70 0x7f6e10106360 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:00.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.527+0000 7f6e18594700 1 --2- 172.21.15.114:0/3115564744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e10101640 0x7f6e10103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:00.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.527+0000 7f6e18594700 1 -- 172.21.15.114:0/3115564744 >> 172.21.15.114:0/3115564744 conn(0x7f6e100fb030 msgr2=0x7f6e100fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:31:00.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.527+0000 7f6e18594700 1 -- 172.21.15.114:0/3115564744 shutdown_connections 2024-02-16T22:31:00.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.527+0000 7f6e18594700 1 -- 172.21.15.114:0/3115564744 wait complete. 2024-02-16T22:31:00.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.528+0000 7f6e18594700 1 Processor -- start 2024-02-16T22:31:00.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.528+0000 7f6e18594700 1 -- start start 2024-02-16T22:31:00.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.528+0000 7f6e18594700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e10101640 0x7f6e10196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:31:00.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.528+0000 7f6e18594700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e10103f70 0x7f6e10196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:31:00.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.528+0000 7f6e18594700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e10196e90 con 0x7f6e10101640 2024-02-16T22:31:00.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.528+0000 7f6e18594700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e10196fd0 con 0x7f6e10103f70 2024-02-16T22:31:00.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.528+0000 7f6e16330700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e10101640 0x7f6e10196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:31:00.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.528+0000 7f6e16330700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e10101640 0x7f6e10196330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46260/0 (socket says 172.21.15.114:46260) 2024-02-16T22:31:00.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.528+0000 7f6e16330700 1 -- 172.21.15.114:0/2958146531 learned_addr learned my addr 172.21.15.114:0/2958146531 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:31:00.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.528+0000 7f6e15b2f700 1 --2- 172.21.15.114:0/2958146531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e10103f70 0x7f6e10196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:31:00.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.528+0000 7f6e16330700 1 -- 172.21.15.114:0/2958146531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e10103f70 msgr2=0x7f6e10196870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:00.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.528+0000 7f6e16330700 1 --2- 172.21.15.114:0/2958146531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e10103f70 0x7f6e10196870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:00.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.529+0000 7f6e16330700 1 -- 172.21.15.114:0/2958146531 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0c0095f0 con 0x7f6e10101640 2024-02-16T22:31:00.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.529+0000 7f6e16330700 1 --2- 172.21.15.114:0/2958146531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e10101640 0x7f6e10196330 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000d8d0 tx=0x7f6e0000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:31:00.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.529+0000 7f6e077fe700 1 -- 172.21.15.114:0/2958146531 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e000098a0 con 0x7f6e10101640 2024-02-16T22:31:00.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.529+0000 7f6e077fe700 1 -- 172.21.15.114:0/2958146531 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6e00010460 con 0x7f6e10101640 2024-02-16T22:31:00.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.529+0000 7f6e18594700 1 -- 172.21.15.114:0/2958146531 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e1019ba80 con 0x7f6e10101640 2024-02-16T22:31:00.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.529+0000 7f6e077fe700 1 -- 172.21.15.114:0/2958146531 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e0000f5f0 con 0x7f6e10101640 2024-02-16T22:31:00.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.529+0000 7f6e18594700 1 -- 172.21.15.114:0/2958146531 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e1019bfd0 con 0x7f6e10101640 2024-02-16T22:31:00.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.530+0000 7f6e18594700 1 -- 172.21.15.114:0/2958146531 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e10190520 con 0x7f6e10101640 2024-02-16T22:31:00.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.530+0000 7f6e077fe700 1 -- 172.21.15.114:0/2958146531 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f6e00009a00 con 0x7f6e10101640 2024-02-16T22:31:00.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.531+0000 7f6e077fe700 1 --2- 172.21.15.114:0/2958146531 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6dfc06d8a0 0x7f6dfc06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:31:00.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.531+0000 7f6e15b2f700 1 --2- 172.21.15.114:0/2958146531 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6dfc06d8a0 0x7f6dfc06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:31:00.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.531+0000 7f6e077fe700 1 -- 172.21.15.114:0/2958146531 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f6e0008d3f0 con 0x7f6e10101640 2024-02-16T22:31:00.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.531+0000 7f6e15b2f700 1 --2- 172.21.15.114:0/2958146531 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6dfc06d8a0 0x7f6dfc06fd60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c006010 tx=0x7f6e0c00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:31:00.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.535+0000 7f6e077fe700 1 -- 172.21.15.114:0/2958146531 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6e0005a6a0 con 0x7f6e10101640 2024-02-16T22:31:00.635 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:00 smithi114 ceph-mon[94452]: from='client.34530 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:31:00.635 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:00 smithi114 ceph-mon[94452]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:31:00.635 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:00 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/3794869183' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:00.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.751+0000 7f6e18594700 1 -- 172.21.15.114:0/2958146531 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e100611d0 con 0x7f6dfc06d8a0 2024-02-16T22:31:00.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.752+0000 7f6e077fe700 1 -- 172.21.15.114:0/2958146531 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+465 (secure 0 0 0) 0x7f6e100611d0 con 0x7f6dfc06d8a0 2024-02-16T22:31:00.754 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:31:00.754 INFO:teuthology.orchestra.run.smithi114.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:31:00.754 INFO:teuthology.orchestra.run.smithi114.stdout: "in_progress": true, 2024-02-16T22:31:00.754 INFO:teuthology.orchestra.run.smithi114.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:31:00.754 INFO:teuthology.orchestra.run.smithi114.stdout: "services_complete": [ 2024-02-16T22:31:00.755 INFO:teuthology.orchestra.run.smithi114.stdout: "osd", 2024-02-16T22:31:00.755 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph-exporter", 2024-02-16T22:31:00.755 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr", 2024-02-16T22:31:00.755 INFO:teuthology.orchestra.run.smithi114.stdout: "mon", 2024-02-16T22:31:00.755 INFO:teuthology.orchestra.run.smithi114.stdout: "mds", 2024-02-16T22:31:00.755 INFO:teuthology.orchestra.run.smithi114.stdout: "crash" 2024-02-16T22:31:00.755 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-02-16T22:31:00.755 INFO:teuthology.orchestra.run.smithi114.stdout: "progress": "18/23 daemons upgraded", 2024-02-16T22:31:00.755 INFO:teuthology.orchestra.run.smithi114.stdout: "message": "Currently upgrading alertmanager daemons", 2024-02-16T22:31:00.755 INFO:teuthology.orchestra.run.smithi114.stdout: "is_paused": false 2024-02-16T22:31:00.755 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:31:00.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.755+0000 7f6e18594700 1 -- 172.21.15.114:0/2958146531 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6dfc06d8a0 msgr2=0x7f6dfc06fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:00.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.755+0000 7f6e18594700 1 --2- 172.21.15.114:0/2958146531 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6dfc06d8a0 0x7f6dfc06fd60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c006010 tx=0x7f6e0c00b540 comp rx=0 tx=0).stop 2024-02-16T22:31:00.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.755+0000 7f6e18594700 1 -- 172.21.15.114:0/2958146531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e10101640 msgr2=0x7f6e10196330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:00.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.755+0000 7f6e18594700 1 --2- 172.21.15.114:0/2958146531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e10101640 0x7f6e10196330 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000d8d0 tx=0x7f6e0000dbe0 comp rx=0 tx=0).stop 2024-02-16T22:31:00.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.755+0000 7f6e18594700 1 -- 172.21.15.114:0/2958146531 shutdown_connections 2024-02-16T22:31:00.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.755+0000 7f6e18594700 1 --2- 172.21.15.114:0/2958146531 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6dfc06d8a0 0x7f6dfc06fd60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:00.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.755+0000 7f6e18594700 1 --2- 172.21.15.114:0/2958146531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e10101640 0x7f6e10196330 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:00.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.755+0000 7f6e18594700 1 --2- 172.21.15.114:0/2958146531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e10103f70 0x7f6e10196870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:00.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.755+0000 7f6e18594700 1 -- 172.21.15.114:0/2958146531 >> 172.21.15.114:0/2958146531 conn(0x7f6e100fb030 msgr2=0x7f6e100fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:31:00.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.755+0000 7f6e18594700 1 -- 172.21.15.114:0/2958146531 shutdown_connections 2024-02-16T22:31:00.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.755+0000 7f6e18594700 1 -- 172.21.15.114:0/2958146531 wait complete. 2024-02-16T22:31:00.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:00 smithi123 ceph-mon[82331]: from='client.34530 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:31:00.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:00 smithi123 ceph-mon[82331]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:31:00.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:00 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/3794869183' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:00.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.860+0000 7f51f2716700 1 -- 172.21.15.114:0/1841531942 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51ec100cc0 msgr2=0x7f51ec101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:00.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.860+0000 7f51f2716700 1 --2- 172.21.15.114:0/1841531942 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51ec100cc0 0x7f51ec101120 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f51dc009910 tx=0x7f51dc009c20 comp rx=0 tx=0).stop 2024-02-16T22:31:00.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.860+0000 7f51f2716700 1 -- 172.21.15.114:0/1841531942 shutdown_connections 2024-02-16T22:31:00.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.860+0000 7f51f2716700 1 --2- 172.21.15.114:0/1841531942 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51ec100cc0 0x7f51ec101120 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:00.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.860+0000 7f51f2716700 1 --2- 172.21.15.114:0/1841531942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51ec0ffac0 0x7f51ec0ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:00.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.860+0000 7f51f2716700 1 -- 172.21.15.114:0/1841531942 >> 172.21.15.114:0/1841531942 conn(0x7f51ec0fb060 msgr2=0x7f51ec0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:31:00.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.860+0000 7f51f2716700 1 -- 172.21.15.114:0/1841531942 shutdown_connections 2024-02-16T22:31:00.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.860+0000 7f51f2716700 1 -- 172.21.15.114:0/1841531942 wait complete. 2024-02-16T22:31:00.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.861+0000 7f51f2716700 1 Processor -- start 2024-02-16T22:31:00.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.861+0000 7f51f2716700 1 -- start start 2024-02-16T22:31:00.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.861+0000 7f51f2716700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51ec0ffac0 0x7f51ec196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:31:00.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.861+0000 7f51f2716700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51ec100cc0 0x7f51ec1968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:31:00.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.861+0000 7f51f2716700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec196ed0 con 0x7f51ec0ffac0 2024-02-16T22:31:00.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.861+0000 7f51f2716700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec197010 con 0x7f51ec100cc0 2024-02-16T22:31:00.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.861+0000 7f51ebfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51ec0ffac0 0x7f51ec196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:31:00.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.861+0000 7f51ebfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51ec0ffac0 0x7f51ec196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46282/0 (socket says 172.21.15.114:46282) 2024-02-16T22:31:00.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.862+0000 7f51ebfff700 1 -- 172.21.15.114:0/2218942046 learned_addr learned my addr 172.21.15.114:0/2218942046 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:31:00.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.862+0000 7f51eb7fe700 1 --2- 172.21.15.114:0/2218942046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51ec100cc0 0x7f51ec1968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:31:00.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.862+0000 7f51ebfff700 1 -- 172.21.15.114:0/2218942046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51ec100cc0 msgr2=0x7f51ec1968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:00.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.862+0000 7f51ebfff700 1 --2- 172.21.15.114:0/2218942046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51ec100cc0 0x7f51ec1968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:00.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.862+0000 7f51ebfff700 1 -- 172.21.15.114:0/2218942046 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc0095f0 con 0x7f51ec0ffac0 2024-02-16T22:31:00.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.862+0000 7f51ebfff700 1 --2- 172.21.15.114:0/2218942046 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51ec0ffac0 0x7f51ec196370 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f51e000d900 tx=0x7f51e000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:31:00.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.862+0000 7f51e97fa700 1 -- 172.21.15.114:0/2218942046 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51e00041f0 con 0x7f51ec0ffac0 2024-02-16T22:31:00.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.862+0000 7f51e97fa700 1 -- 172.21.15.114:0/2218942046 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f51e0004350 con 0x7f51ec0ffac0 2024-02-16T22:31:00.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.862+0000 7f51e97fa700 1 -- 172.21.15.114:0/2218942046 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51e0003d20 con 0x7f51ec0ffac0 2024-02-16T22:31:00.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.862+0000 7f51f2716700 1 -- 172.21.15.114:0/2218942046 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ec19bac0 con 0x7f51ec0ffac0 2024-02-16T22:31:00.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.862+0000 7f51f2716700 1 -- 172.21.15.114:0/2218942046 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51ec19c010 con 0x7f51ec0ffac0 2024-02-16T22:31:00.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.863+0000 7f51f2716700 1 -- 172.21.15.114:0/2218942046 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51ec108620 con 0x7f51ec0ffac0 2024-02-16T22:31:00.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.864+0000 7f51e97fa700 1 -- 172.21.15.114:0/2218942046 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f51e0009730 con 0x7f51ec0ffac0 2024-02-16T22:31:00.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.864+0000 7f51e97fa700 1 --2- 172.21.15.114:0/2218942046 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f51d406d8a0 0x7f51d406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:31:00.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.864+0000 7f51e97fa700 1 -- 172.21.15.114:0/2218942046 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f51e008d4b0 con 0x7f51ec0ffac0 2024-02-16T22:31:00.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.864+0000 7f51eb7fe700 1 --2- 172.21.15.114:0/2218942046 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f51d406d8a0 0x7f51d406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:31:00.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.865+0000 7f51eb7fe700 1 --2- 172.21.15.114:0/2218942046 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f51d406d8a0 0x7f51d406fd60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00b5c0 tx=0x7f51dc005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:31:00.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:00.869+0000 7f51e97fa700 1 -- 172.21.15.114:0/2218942046 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f51e005a6e0 con 0x7f51ec0ffac0 2024-02-16T22:31:01.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:01.150+0000 7f51f2716700 1 -- 172.21.15.114:0/2218942046 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f51ec108810 con 0x7f51ec0ffac0 2024-02-16T22:31:01.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:01.151+0000 7f51e97fa700 1 -- 172.21.15.114:0/2218942046 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f51e0003e80 con 0x7f51ec0ffac0 2024-02-16T22:31:01.153 INFO:teuthology.orchestra.run.smithi114.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:31:01.153 INFO:teuthology.orchestra.run.smithi114.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:31:01.153 INFO:teuthology.orchestra.run.smithi114.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:31:01.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:01.152+0000 7f51f2716700 1 -- 172.21.15.114:0/2218942046 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f51d406d8a0 msgr2=0x7f51d406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:01.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:01.153+0000 7f51f2716700 1 --2- 172.21.15.114:0/2218942046 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f51d406d8a0 0x7f51d406fd60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00b5c0 tx=0x7f51dc005ee0 comp rx=0 tx=0).stop 2024-02-16T22:31:01.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:01.153+0000 7f51f2716700 1 -- 172.21.15.114:0/2218942046 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51ec0ffac0 msgr2=0x7f51ec196370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:01.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:01.153+0000 7f51f2716700 1 --2- 172.21.15.114:0/2218942046 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51ec0ffac0 0x7f51ec196370 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f51e000d900 tx=0x7f51e000dc10 comp rx=0 tx=0).stop 2024-02-16T22:31:01.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:01.153+0000 7f51f2716700 1 -- 172.21.15.114:0/2218942046 shutdown_connections 2024-02-16T22:31:01.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:01.153+0000 7f51f2716700 1 --2- 172.21.15.114:0/2218942046 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f51d406d8a0 0x7f51d406fd60 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:01.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:01.153+0000 7f51f2716700 1 --2- 172.21.15.114:0/2218942046 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51ec0ffac0 0x7f51ec196370 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:01.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:01.153+0000 7f51f2716700 1 --2- 172.21.15.114:0/2218942046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51ec100cc0 0x7f51ec1968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:01.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:01.153+0000 7f51f2716700 1 -- 172.21.15.114:0/2218942046 >> 172.21.15.114:0/2218942046 conn(0x7f51ec0fb060 msgr2=0x7f51ec068db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:31:01.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:01.153+0000 7f51f2716700 1 -- 172.21.15.114:0/2218942046 shutdown_connections 2024-02-16T22:31:01.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:01.153+0000 7f51f2716700 1 -- 172.21.15.114:0/2218942046 wait complete. 2024-02-16T22:31:01.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:01 smithi114 ceph-mon[94452]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:31:01.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:01 smithi114 ceph-mon[94452]: pgmap v254: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:01.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:01 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/2134350227' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:31:01.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:31:01.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:01 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/2218942046' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:31:01.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:01 smithi123 ceph-mon[82331]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:31:01.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:01 smithi123 ceph-mon[82331]: pgmap v254: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:01.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:01 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/2134350227' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:31:01.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:01 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:31:01.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:01 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/2218942046' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:31:02.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:02 smithi114 ceph-mon[94452]: from='client.34550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:31:02.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:02 smithi123 ceph-mon[82331]: from='client.34550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:31:03.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:03 smithi123 ceph-mon[82331]: pgmap v255: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:03.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:03 smithi114 ceph-mon[94452]: pgmap v255: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:04.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:04 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:04.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:04 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:05.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:04 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:05.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:04 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:05.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:05 smithi114 ceph-mon[94452]: pgmap v256: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:05.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:05 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:05.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:05 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:06.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:05 smithi123 ceph-mon[82331]: pgmap v256: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:06.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:05 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:06.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:05 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:07.713 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:07 smithi114 ceph-mon[94452]: pgmap v257: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:08.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:07 smithi123 ceph-mon[82331]: pgmap v257: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:09.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:09 smithi114 ceph-mon[94452]: pgmap v258: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:09.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:09 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:09.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:09 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:09.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:09 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:31:09.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:09 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:31:09.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:09 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:09.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:09 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T22:31:09.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:09 smithi114 ceph-mon[94452]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T22:31:09.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:09 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:31:09.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:09 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:09.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:09 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:09.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:09 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:09.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:09 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:09.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:09 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:09.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:09 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:09.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:09 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:09.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:09 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:09.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:09 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:09.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:09 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:09.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:09 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:09.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:09 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:09.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:09 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:09.704 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:09 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:09.704 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:09 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:09.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:09 smithi123 ceph-mon[82331]: pgmap v258: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:09.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:09 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:09.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:09 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:09.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:09 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:31:09.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:09 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:31:09.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:09 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:09.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:09 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T22:31:09.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:09 smithi123 ceph-mon[82331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T22:31:09.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:09 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:31:09.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:09 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:09.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:09 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:09.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:09 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:09.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:09 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:09.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:09 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:09.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:09 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:09.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:09 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:09.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:09 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:09.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:09 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:09.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:09 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:09.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:09 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:09.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:09 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:09.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:09 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:09.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:09 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:09.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:09 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:11.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:11 smithi114 ceph-mon[94452]: pgmap v259: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:11.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:11 smithi114 ceph-mon[94452]: Upgrade: Updating grafana.smithi114 2024-02-16T22:31:11.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:11 smithi114 ceph-mon[94452]: Deploying daemon grafana.smithi114 on smithi114 2024-02-16T22:31:11.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:11 smithi123 ceph-mon[82331]: pgmap v259: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:11.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:11 smithi123 ceph-mon[82331]: Upgrade: Updating grafana.smithi114 2024-02-16T22:31:11.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:11 smithi123 ceph-mon[82331]: Deploying daemon grafana.smithi114 on smithi114 2024-02-16T22:31:13.581 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:13 smithi114 ceph-mon[94452]: pgmap v260: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:13.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:13 smithi123 ceph-mon[82331]: pgmap v260: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:15.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:15 smithi114 ceph-mon[94452]: pgmap v261: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:15.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:15 smithi123 ceph-mon[82331]: pgmap v261: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:16.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:16 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:31:16.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:16 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:31:17.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:17 smithi114 ceph-mon[94452]: pgmap v262: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:17.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:17 smithi123 ceph-mon[82331]: pgmap v262: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:19.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:19 smithi114 ceph-mon[94452]: pgmap v263: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:19.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:19 smithi123 ceph-mon[82331]: pgmap v263: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:21.686 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:21 smithi114 ceph-mon[94452]: pgmap v264: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:21.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:21 smithi123 ceph-mon[82331]: pgmap v264: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:23.673 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:23 smithi114 ceph-mon[94452]: pgmap v265: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:23.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:23 smithi123 ceph-mon[82331]: pgmap v265: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:25.674 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:25 smithi114 ceph-mon[94452]: pgmap v266: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:25.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:25 smithi123 ceph-mon[82331]: pgmap v266: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:26.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:26 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:26.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:26 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:26.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:26 smithi114 ceph-mon[94452]: pgmap v267: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:26.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:26 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:31:27.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:26 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:27.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:26 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:27.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:26 smithi123 ceph-mon[82331]: pgmap v267: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:27.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:26 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:31:29.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:28 smithi114 ceph-mon[94452]: pgmap v268: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:29.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:28 smithi123 ceph-mon[82331]: pgmap v268: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:31.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.262+0000 7f4adcb55700 1 -- 172.21.15.114:0/3839038957 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ad80ffab0 msgr2=0x7f4ad80ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:31.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.262+0000 7f4adcb55700 1 --2- 172.21.15.114:0/3839038957 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ad80ffab0 0x7f4ad80ffed0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8009910 tx=0x7f4ac8009c20 comp rx=0 tx=0).stop 2024-02-16T22:31:31.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.262+0000 7f4adcb55700 1 -- 172.21.15.114:0/3839038957 shutdown_connections 2024-02-16T22:31:31.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.262+0000 7f4adcb55700 1 --2- 172.21.15.114:0/3839038957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ad8100cb0 0x7f4ad8101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:31.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.262+0000 7f4adcb55700 1 --2- 172.21.15.114:0/3839038957 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ad80ffab0 0x7f4ad80ffed0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:31.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.262+0000 7f4adcb55700 1 -- 172.21.15.114:0/3839038957 >> 172.21.15.114:0/3839038957 conn(0x7f4ad80fb050 msgr2=0x7f4ad80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:31:31.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.262+0000 7f4adcb55700 1 -- 172.21.15.114:0/3839038957 shutdown_connections 2024-02-16T22:31:31.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.262+0000 7f4adcb55700 1 -- 172.21.15.114:0/3839038957 wait complete. 2024-02-16T22:31:31.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.263+0000 7f4adcb55700 1 Processor -- start 2024-02-16T22:31:31.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.263+0000 7f4adcb55700 1 -- start start 2024-02-16T22:31:31.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.263+0000 7f4adcb55700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ad80ffab0 0x7f4ad81952e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:31:31.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.263+0000 7f4adcb55700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ad8100cb0 0x7f4ad8195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:31:31.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.263+0000 7f4adcb55700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad8195e40 con 0x7f4ad8100cb0 2024-02-16T22:31:31.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.263+0000 7f4adcb55700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad8195f80 con 0x7f4ad80ffab0 2024-02-16T22:31:31.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.263+0000 7f4ad5d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ad8100cb0 0x7f4ad8195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:31:31.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.263+0000 7f4ad5d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ad8100cb0 0x7f4ad8195820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33052/0 (socket says 172.21.15.114:33052) 2024-02-16T22:31:31.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.264+0000 7f4ad5d9b700 1 -- 172.21.15.114:0/2807870433 learned_addr learned my addr 172.21.15.114:0/2807870433 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:31:31.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.264+0000 7f4ad659c700 1 --2- 172.21.15.114:0/2807870433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ad80ffab0 0x7f4ad81952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:31:31.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.264+0000 7f4ad5d9b700 1 -- 172.21.15.114:0/2807870433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ad80ffab0 msgr2=0x7f4ad81952e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:31.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.264+0000 7f4ad5d9b700 1 --2- 172.21.15.114:0/2807870433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ad80ffab0 0x7f4ad81952e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:31.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.264+0000 7f4ad5d9b700 1 -- 172.21.15.114:0/2807870433 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac80095f0 con 0x7f4ad8100cb0 2024-02-16T22:31:31.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.264+0000 7f4ad5d9b700 1 --2- 172.21.15.114:0/2807870433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ad8100cb0 0x7f4ad8195820 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4acc00d960 tx=0x7f4acc00dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:31:31.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.264+0000 7f4ac77fe700 1 -- 172.21.15.114:0/2807870433 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4acc00ec20 con 0x7f4ad8100cb0 2024-02-16T22:31:31.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.264+0000 7f4ac77fe700 1 -- 172.21.15.114:0/2807870433 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4acc00ed80 con 0x7f4ad8100cb0 2024-02-16T22:31:31.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.264+0000 7f4adcb55700 1 -- 172.21.15.114:0/2807870433 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad8068d00 con 0x7f4ad8100cb0 2024-02-16T22:31:31.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.264+0000 7f4ac77fe700 1 -- 172.21.15.114:0/2807870433 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4acc0147b0 con 0x7f4ad8100cb0 2024-02-16T22:31:31.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.264+0000 7f4adcb55700 1 -- 172.21.15.114:0/2807870433 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ad80692d0 con 0x7f4ad8100cb0 2024-02-16T22:31:31.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.265+0000 7f4adcb55700 1 -- 172.21.15.114:0/2807870433 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ad8108690 con 0x7f4ad8100cb0 2024-02-16T22:31:31.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.266+0000 7f4ac77fe700 1 -- 172.21.15.114:0/2807870433 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f4acc014950 con 0x7f4ad8100cb0 2024-02-16T22:31:31.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.266+0000 7f4ac77fe700 1 --2- 172.21.15.114:0/2807870433 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f4ac006d8a0 0x7f4ac006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:31:31.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.266+0000 7f4ac77fe700 1 -- 172.21.15.114:0/2807870433 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f4acc08efb0 con 0x7f4ad8100cb0 2024-02-16T22:31:31.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.266+0000 7f4ad659c700 1 --2- 172.21.15.114:0/2807870433 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f4ac006d8a0 0x7f4ac006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:31:31.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.267+0000 7f4ad659c700 1 --2- 172.21.15.114:0/2807870433 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f4ac006d8a0 0x7f4ac006fd60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8006010 tx=0x7f4ac8005920 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:31:31.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.271+0000 7f4ac77fe700 1 -- 172.21.15.114:0/2807870433 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4acc05c290 con 0x7f4ad8100cb0 2024-02-16T22:31:31.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.493+0000 7f4adcb55700 1 -- 172.21.15.114:0/2807870433 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ad8105600 con 0x7f4ac006d8a0 2024-02-16T22:31:31.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.495+0000 7f4ac77fe700 1 -- 172.21.15.114:0/2807870433 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+460 (secure 0 0 0) 0x7f4ad8105600 con 0x7f4ac006d8a0 2024-02-16T22:31:31.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.497+0000 7f4adcb55700 1 -- 172.21.15.114:0/2807870433 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f4ac006d8a0 msgr2=0x7f4ac006fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:31.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.497+0000 7f4adcb55700 1 --2- 172.21.15.114:0/2807870433 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f4ac006d8a0 0x7f4ac006fd60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8006010 tx=0x7f4ac8005920 comp rx=0 tx=0).stop 2024-02-16T22:31:31.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.497+0000 7f4adcb55700 1 -- 172.21.15.114:0/2807870433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ad8100cb0 msgr2=0x7f4ad8195820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:31.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.497+0000 7f4adcb55700 1 --2- 172.21.15.114:0/2807870433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ad8100cb0 0x7f4ad8195820 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4acc00d960 tx=0x7f4acc00dd20 comp rx=0 tx=0).stop 2024-02-16T22:31:31.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.497+0000 7f4adcb55700 1 -- 172.21.15.114:0/2807870433 shutdown_connections 2024-02-16T22:31:31.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.497+0000 7f4adcb55700 1 --2- 172.21.15.114:0/2807870433 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f4ac006d8a0 0x7f4ac006fd60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:31.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.497+0000 7f4adcb55700 1 --2- 172.21.15.114:0/2807870433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ad80ffab0 0x7f4ad81952e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:31.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.497+0000 7f4adcb55700 1 --2- 172.21.15.114:0/2807870433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ad8100cb0 0x7f4ad8195820 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:31.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.497+0000 7f4adcb55700 1 -- 172.21.15.114:0/2807870433 >> 172.21.15.114:0/2807870433 conn(0x7f4ad80fb050 msgr2=0x7f4ad8103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:31:31.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.497+0000 7f4adcb55700 1 -- 172.21.15.114:0/2807870433 shutdown_connections 2024-02-16T22:31:31.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.497+0000 7f4adcb55700 1 -- 172.21.15.114:0/2807870433 wait complete. 2024-02-16T22:31:31.510 INFO:teuthology.orchestra.run.smithi114.stdout:true 2024-02-16T22:31:31.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.605+0000 7fa198f31700 1 -- 172.21.15.114:0/4257122475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1940ffab0 msgr2=0x7fa1940ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:31.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.605+0000 7fa198f31700 1 --2- 172.21.15.114:0/4257122475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1940ffab0 0x7fa1940ffed0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa188009910 tx=0x7fa188009c20 comp rx=0 tx=0).stop 2024-02-16T22:31:31.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.605+0000 7fa198f31700 1 -- 172.21.15.114:0/4257122475 shutdown_connections 2024-02-16T22:31:31.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.605+0000 7fa198f31700 1 --2- 172.21.15.114:0/4257122475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa194100cb0 0x7fa194101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:31.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.605+0000 7fa198f31700 1 --2- 172.21.15.114:0/4257122475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1940ffab0 0x7fa1940ffed0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:31.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.605+0000 7fa198f31700 1 -- 172.21.15.114:0/4257122475 >> 172.21.15.114:0/4257122475 conn(0x7fa1940fb030 msgr2=0x7fa1940fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:31:31.608 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:31 smithi114 ceph-mon[94452]: pgmap v269: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:31.608 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:31 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:31.608 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:31 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:31.608 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:31 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:31:31.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.606+0000 7fa198f31700 1 -- 172.21.15.114:0/4257122475 shutdown_connections 2024-02-16T22:31:31.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.606+0000 7fa198f31700 1 -- 172.21.15.114:0/4257122475 wait complete. 2024-02-16T22:31:31.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.606+0000 7fa198f31700 1 Processor -- start 2024-02-16T22:31:31.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.606+0000 7fa198f31700 1 -- start start 2024-02-16T22:31:31.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.606+0000 7fa198f31700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1940ffab0 0x7fa194103a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:31:31.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.607+0000 7fa198f31700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa194100cb0 0x7fa194102090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:31:31.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.607+0000 7fa198f31700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1941025d0 con 0x7fa194100cb0 2024-02-16T22:31:31.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.607+0000 7fa198f31700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa194102710 con 0x7fa1940ffab0 2024-02-16T22:31:31.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.607+0000 7fa191d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa194100cb0 0x7fa194102090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:31:31.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.607+0000 7fa191d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa194100cb0 0x7fa194102090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36344/0 (socket says 172.21.15.114:36344) 2024-02-16T22:31:31.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.607+0000 7fa191d9b700 1 -- 172.21.15.114:0/122095247 learned_addr learned my addr 172.21.15.114:0/122095247 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:31:31.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.607+0000 7fa19259c700 1 --2- 172.21.15.114:0/122095247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1940ffab0 0x7fa194103a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:31:31.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.607+0000 7fa191d9b700 1 -- 172.21.15.114:0/122095247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1940ffab0 msgr2=0x7fa194103a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:31.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.607+0000 7fa191d9b700 1 --2- 172.21.15.114:0/122095247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1940ffab0 0x7fa194103a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:31.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.607+0000 7fa191d9b700 1 -- 172.21.15.114:0/122095247 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1880095f0 con 0x7fa194100cb0 2024-02-16T22:31:31.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.607+0000 7fa19259c700 1 --2- 172.21.15.114:0/122095247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1940ffab0 0x7fa194103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:31:31.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.607+0000 7fa191d9b700 1 --2- 172.21.15.114:0/122095247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa194100cb0 0x7fa194102090 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa18400c930 tx=0x7fa18400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:31:31.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.607+0000 7fa1837fe700 1 -- 172.21.15.114:0/122095247 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa184007ad0 con 0x7fa194100cb0 2024-02-16T22:31:31.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.607+0000 7fa1837fe700 1 -- 172.21.15.114:0/122095247 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa184007c30 con 0x7fa194100cb0 2024-02-16T22:31:31.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.607+0000 7fa198f31700 1 -- 172.21.15.114:0/122095247 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1941029f0 con 0x7fa194100cb0 2024-02-16T22:31:31.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.607+0000 7fa1837fe700 1 -- 172.21.15.114:0/122095247 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa184018630 con 0x7fa194100cb0 2024-02-16T22:31:31.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.607+0000 7fa198f31700 1 -- 172.21.15.114:0/122095247 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa19419e380 con 0x7fa194100cb0 2024-02-16T22:31:31.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.608+0000 7fa198f31700 1 -- 172.21.15.114:0/122095247 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa194108610 con 0x7fa194100cb0 2024-02-16T22:31:31.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.609+0000 7fa1837fe700 1 -- 172.21.15.114:0/122095247 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fa18400f450 con 0x7fa194100cb0 2024-02-16T22:31:31.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.609+0000 7fa1837fe700 1 --2- 172.21.15.114:0/122095247 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fa17c06d8a0 0x7fa17c06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:31:31.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.609+0000 7fa1837fe700 1 -- 172.21.15.114:0/122095247 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7fa18408d6f0 con 0x7fa194100cb0 2024-02-16T22:31:31.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.609+0000 7fa19259c700 1 --2- 172.21.15.114:0/122095247 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fa17c06d8a0 0x7fa17c06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:31:31.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.610+0000 7fa19259c700 1 --2- 172.21.15.114:0/122095247 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fa17c06d8a0 0x7fa17c06fd60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa188006010 tx=0x7fa18800b560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:31:31.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.614+0000 7fa1837fe700 1 -- 172.21.15.114:0/122095247 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa18405a9a0 con 0x7fa194100cb0 2024-02-16T22:31:31.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:31 smithi123 ceph-mon[82331]: pgmap v269: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:31.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:31 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:31.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:31 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:31.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:31 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:31:31.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.834+0000 7fa198f31700 1 -- 172.21.15.114:0/122095247 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa194068ae0 con 0x7fa17c06d8a0 2024-02-16T22:31:31.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.835+0000 7fa1837fe700 1 -- 172.21.15.114:0/122095247 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+460 (secure 0 0 0) 0x7fa194068ae0 con 0x7fa17c06d8a0 2024-02-16T22:31:31.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.838+0000 7fa198f31700 1 -- 172.21.15.114:0/122095247 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fa17c06d8a0 msgr2=0x7fa17c06fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:31.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.838+0000 7fa198f31700 1 --2- 172.21.15.114:0/122095247 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fa17c06d8a0 0x7fa17c06fd60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa188006010 tx=0x7fa18800b560 comp rx=0 tx=0).stop 2024-02-16T22:31:31.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.838+0000 7fa198f31700 1 -- 172.21.15.114:0/122095247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa194100cb0 msgr2=0x7fa194102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:31.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.838+0000 7fa198f31700 1 --2- 172.21.15.114:0/122095247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa194100cb0 0x7fa194102090 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa18400c930 tx=0x7fa18400ccf0 comp rx=0 tx=0).stop 2024-02-16T22:31:31.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.838+0000 7fa198f31700 1 -- 172.21.15.114:0/122095247 shutdown_connections 2024-02-16T22:31:31.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.838+0000 7fa198f31700 1 --2- 172.21.15.114:0/122095247 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fa17c06d8a0 0x7fa17c06fd60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:31.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.838+0000 7fa198f31700 1 --2- 172.21.15.114:0/122095247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1940ffab0 0x7fa194103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:31.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.838+0000 7fa198f31700 1 --2- 172.21.15.114:0/122095247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa194100cb0 0x7fa194102090 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:31.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.838+0000 7fa198f31700 1 -- 172.21.15.114:0/122095247 >> 172.21.15.114:0/122095247 conn(0x7fa1940fb030 msgr2=0x7fa1940683d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:31:31.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.838+0000 7fa198f31700 1 -- 172.21.15.114:0/122095247 shutdown_connections 2024-02-16T22:31:31.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.838+0000 7fa198f31700 1 -- 172.21.15.114:0/122095247 wait complete. 2024-02-16T22:31:31.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.964+0000 7f6cc5fe1700 1 -- 172.21.15.114:0/2504119970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cc00ffab0 msgr2=0x7f6cc00ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:31.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.964+0000 7f6cc5fe1700 1 --2- 172.21.15.114:0/2504119970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cc00ffab0 0x7f6cc00ffed0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0009910 tx=0x7f6cb0009c20 comp rx=0 tx=0).stop 2024-02-16T22:31:31.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.964+0000 7f6cc5fe1700 1 -- 172.21.15.114:0/2504119970 shutdown_connections 2024-02-16T22:31:31.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.964+0000 7f6cc5fe1700 1 --2- 172.21.15.114:0/2504119970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cc0100cb0 0x7f6cc0101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:31.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.964+0000 7f6cc5fe1700 1 --2- 172.21.15.114:0/2504119970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cc00ffab0 0x7f6cc00ffed0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:31.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.964+0000 7f6cc5fe1700 1 -- 172.21.15.114:0/2504119970 >> 172.21.15.114:0/2504119970 conn(0x7f6cc00fb030 msgr2=0x7f6cc00fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:31:31.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.965+0000 7f6cc5fe1700 1 -- 172.21.15.114:0/2504119970 shutdown_connections 2024-02-16T22:31:31.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.965+0000 7f6cc5fe1700 1 -- 172.21.15.114:0/2504119970 wait complete. 2024-02-16T22:31:31.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.965+0000 7f6cc5fe1700 1 Processor -- start 2024-02-16T22:31:31.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.965+0000 7f6cc5fe1700 1 -- start start 2024-02-16T22:31:31.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.965+0000 7f6cc5fe1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cc00ffab0 0x7f6cc0196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:31:31.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.965+0000 7f6cc5fe1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cc0100cb0 0x7f6cc01968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:31:31.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.965+0000 7f6cc5fe1700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc0196ec0 con 0x7f6cc00ffab0 2024-02-16T22:31:31.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.965+0000 7f6cc5fe1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc0197000 con 0x7f6cc0100cb0 2024-02-16T22:31:31.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.966+0000 7f6cbf7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cc00ffab0 0x7f6cc0196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:31:31.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.966+0000 7f6cbeffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cc0100cb0 0x7f6cc01968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:31:31.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.966+0000 7f6cbf7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cc00ffab0 0x7f6cc0196360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36364/0 (socket says 172.21.15.114:36364) 2024-02-16T22:31:31.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.966+0000 7f6cbf7fe700 1 -- 172.21.15.114:0/361619995 learned_addr learned my addr 172.21.15.114:0/361619995 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:31:31.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.966+0000 7f6cbf7fe700 1 -- 172.21.15.114:0/361619995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cc0100cb0 msgr2=0x7f6cc01968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:31.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.966+0000 7f6cbf7fe700 1 --2- 172.21.15.114:0/361619995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cc0100cb0 0x7f6cc01968a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:31.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.966+0000 7f6cbf7fe700 1 -- 172.21.15.114:0/361619995 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb00095f0 con 0x7f6cc00ffab0 2024-02-16T22:31:31.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.966+0000 7f6cbf7fe700 1 --2- 172.21.15.114:0/361619995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cc00ffab0 0x7f6cc0196360 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000bda0 tx=0x7f6cb000bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:31:31.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.966+0000 7f6cbcff9700 1 -- 172.21.15.114:0/361619995 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cb001d070 con 0x7f6cc00ffab0 2024-02-16T22:31:31.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.966+0000 7f6cbcff9700 1 -- 172.21.15.114:0/361619995 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6cb000f460 con 0x7f6cc00ffab0 2024-02-16T22:31:31.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.966+0000 7f6cc5fe1700 1 -- 172.21.15.114:0/361619995 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cc019ba50 con 0x7f6cc00ffab0 2024-02-16T22:31:31.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.966+0000 7f6cbcff9700 1 -- 172.21.15.114:0/361619995 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cb0021640 con 0x7f6cc00ffab0 2024-02-16T22:31:31.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.966+0000 7f6cc5fe1700 1 -- 172.21.15.114:0/361619995 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc019bf40 con 0x7f6cc00ffab0 2024-02-16T22:31:31.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.967+0000 7f6cc5fe1700 1 -- 172.21.15.114:0/361619995 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cc0108610 con 0x7f6cc00ffab0 2024-02-16T22:31:31.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.968+0000 7f6cbcff9700 1 -- 172.21.15.114:0/361619995 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f6cb002b440 con 0x7f6cc00ffab0 2024-02-16T22:31:31.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.968+0000 7f6cbcff9700 1 --2- 172.21.15.114:0/361619995 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6ca806d8a0 0x7f6ca806fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:31:31.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.968+0000 7f6cbcff9700 1 -- 172.21.15.114:0/361619995 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f6cb008e900 con 0x7f6cc00ffab0 2024-02-16T22:31:31.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.968+0000 7f6cbeffd700 1 --2- 172.21.15.114:0/361619995 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6ca806d8a0 0x7f6ca806fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:31:31.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.969+0000 7f6cbeffd700 1 --2- 172.21.15.114:0/361619995 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6ca806d8a0 0x7f6ca806fd60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4005950 tx=0x7f6cb400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:31:31.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:31.973+0000 7f6cbcff9700 1 -- 172.21.15.114:0/361619995 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6cb005bb30 con 0x7f6cc00ffab0 2024-02-16T22:31:32.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.180+0000 7f6cc5fe1700 1 -- 172.21.15.114:0/361619995 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6cc019c690 con 0x7f6ca806d8a0 2024-02-16T22:31:32.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.192+0000 7f6cbcff9700 1 -- 172.21.15.114:0/361619995 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f6cc019c690 con 0x7f6ca806d8a0 2024-02-16T22:31:32.194 INFO:teuthology.orchestra.run.smithi114.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:31:32.194 INFO:teuthology.orchestra.run.smithi114.stdout:alertmanager.smithi114 smithi114 *:9093,9094 running (34s) 1s ago 18m 15.0M - 0.25.0 c8568f914cd2 a46f553e8ce0 2024-02-16T22:31:32.194 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi114 smithi114 running (90s) 1s ago 19m 7826k - 19.0.0-1450-gf78a58c0 f081f847070a 1f621c33f6d4 2024-02-16T22:31:32.194 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi123 smithi123 running (84s) 64s ago 18m 7482k - 19.0.0-1450-gf78a58c0 f081f847070a 2f0b3c7f313b 2024-02-16T22:31:32.194 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi114 smithi114 running (6m) 1s ago 19m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 9b8ae9efe83b 2024-02-16T22:31:32.194 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi123 smithi123 running (6m) 64s ago 18m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 6d97263a0ffa 2024-02-16T22:31:32.194 INFO:teuthology.orchestra.run.smithi114.stdout:grafana.smithi114 smithi114 *:3000 running (6s) 1s ago 18m 57.4M - 9.4.12 a0ea5e3c46d9 32e00d2cbcbb 2024-02-16T22:31:32.195 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.fdxpey smithi114 running (2m) 1s ago 13m 14.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2984ae8a955a 2024-02-16T22:31:32.195 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.zzryke smithi114 running (3m) 1s ago 13m 84.6M - 19.0.0-1450-gf78a58c0 f081f847070a 63fa2af7290e 2024-02-16T22:31:32.195 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.gopcfa smithi123 running (103s) 64s ago 13m 14.9M - 19.0.0-1450-gf78a58c0 f081f847070a ec6612a62e2a 2024-02-16T22:31:32.195 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.vhlqbe smithi123 running (2m) 64s ago 13m 15.9M - 19.0.0-1450-gf78a58c0 f081f847070a be790c6287e7 2024-02-16T22:31:32.195 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi114.vcullz smithi114 *:8443,9283,8765 running (9m) 1s ago 20m 512M - 19.0.0-1450-gf78a58c0 f081f847070a 14ebeb7888c0 2024-02-16T22:31:32.195 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi123.inzgjd smithi123 *:8443,9283,8765 running (8m) 64s ago 17m 451M - 19.0.0-1450-gf78a58c0 f081f847070a f10aab9faf9e 2024-02-16T22:31:32.195 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi114 smithi114 running (7m) 1s ago 20m 58.4M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 58ac06372523 2024-02-16T22:31:32.195 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi123 smithi123 running (7m) 64s ago 17m 43.0M 2048M 19.0.0-1450-gf78a58c0 f081f847070a f76485e0ea24 2024-02-16T22:31:32.195 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi114 smithi114 *:9100 running (71s) 1s ago 19m 17.1M - 1.5.0 0da6a335fe13 ee1aa5ff6491 2024-02-16T22:31:32.195 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi123 smithi123 *:9100 running (67s) 64s ago 17m 3657k - 1.5.0 0da6a335fe13 a5e2e31d4643 2024-02-16T22:31:32.195 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 smithi114 running (6m) 1s ago 17m 100M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 1978fdad94a5 2024-02-16T22:31:32.195 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 smithi114 running (5m) 1s ago 16m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 3f90e792898f 2024-02-16T22:31:32.195 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 smithi114 running (5m) 1s ago 16m 91.7M 1216M 19.0.0-1450-gf78a58c0 f081f847070a c663dfa0518b 2024-02-16T22:31:32.195 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 smithi123 running (4m) 64s ago 15m 113M 2240M 19.0.0-1450-gf78a58c0 f081f847070a b921448ba169 2024-02-16T22:31:32.196 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 smithi123 running (3m) 64s ago 15m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 54c2926d56b1 2024-02-16T22:31:32.196 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 smithi123 running (3m) 64s ago 14m 93.6M 2240M 19.0.0-1450-gf78a58c0 f081f847070a dd99bf034749 2024-02-16T22:31:32.196 INFO:teuthology.orchestra.run.smithi114.stdout:prometheus.smithi114 smithi114 *:9095 running (50s) 1s ago 18m 50.1M - 2.43.0 a07b618ecd1d a6b9ebc86282 2024-02-16T22:31:32.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.194+0000 7f6cc5fe1700 1 -- 172.21.15.114:0/361619995 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6ca806d8a0 msgr2=0x7f6ca806fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:32.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.194+0000 7f6cc5fe1700 1 --2- 172.21.15.114:0/361619995 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6ca806d8a0 0x7f6ca806fd60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4005950 tx=0x7f6cb400b410 comp rx=0 tx=0).stop 2024-02-16T22:31:32.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.195+0000 7f6cc5fe1700 1 -- 172.21.15.114:0/361619995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cc00ffab0 msgr2=0x7f6cc0196360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:32.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.195+0000 7f6cc5fe1700 1 --2- 172.21.15.114:0/361619995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cc00ffab0 0x7f6cc0196360 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000bda0 tx=0x7f6cb000bdd0 comp rx=0 tx=0).stop 2024-02-16T22:31:32.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.195+0000 7f6cc5fe1700 1 -- 172.21.15.114:0/361619995 shutdown_connections 2024-02-16T22:31:32.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.195+0000 7f6cc5fe1700 1 --2- 172.21.15.114:0/361619995 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6ca806d8a0 0x7f6ca806fd60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:32.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.195+0000 7f6cc5fe1700 1 --2- 172.21.15.114:0/361619995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cc00ffab0 0x7f6cc0196360 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:32.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.195+0000 7f6cc5fe1700 1 --2- 172.21.15.114:0/361619995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cc0100cb0 0x7f6cc01968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:32.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.195+0000 7f6cc5fe1700 1 -- 172.21.15.114:0/361619995 >> 172.21.15.114:0/361619995 conn(0x7f6cc00fb030 msgr2=0x7f6cc0068410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:31:32.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.195+0000 7f6cc5fe1700 1 -- 172.21.15.114:0/361619995 shutdown_connections 2024-02-16T22:31:32.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.195+0000 7f6cc5fe1700 1 -- 172.21.15.114:0/361619995 wait complete. 2024-02-16T22:31:32.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.303+0000 7fa2d4553700 1 -- 172.21.15.114:0/4212026412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2cc100cf0 msgr2=0x7fa2cc101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:32.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.303+0000 7fa2d4553700 1 --2- 172.21.15.114:0/4212026412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2cc100cf0 0x7fa2cc101150 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009910 tx=0x7fa2bc009c20 comp rx=0 tx=0).stop 2024-02-16T22:31:32.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.304+0000 7fa2d4553700 1 -- 172.21.15.114:0/4212026412 shutdown_connections 2024-02-16T22:31:32.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.304+0000 7fa2d4553700 1 --2- 172.21.15.114:0/4212026412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2cc100cf0 0x7fa2cc101150 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:32.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.304+0000 7fa2d4553700 1 --2- 172.21.15.114:0/4212026412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2cc0ffaf0 0x7fa2cc0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:32.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.304+0000 7fa2d4553700 1 -- 172.21.15.114:0/4212026412 >> 172.21.15.114:0/4212026412 conn(0x7fa2cc0fb070 msgr2=0x7fa2cc0fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:31:32.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.304+0000 7fa2d4553700 1 -- 172.21.15.114:0/4212026412 shutdown_connections 2024-02-16T22:31:32.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.304+0000 7fa2d4553700 1 -- 172.21.15.114:0/4212026412 wait complete. 2024-02-16T22:31:32.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.304+0000 7fa2d4553700 1 Processor -- start 2024-02-16T22:31:32.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.304+0000 7fa2d4553700 1 -- start start 2024-02-16T22:31:32.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.304+0000 7fa2d4553700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2cc0ffaf0 0x7fa2cc1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:31:32.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.304+0000 7fa2d4553700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2cc100cf0 0x7fa2cc198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:31:32.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.304+0000 7fa2d4553700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc1990b0 con 0x7fa2cc0ffaf0 2024-02-16T22:31:32.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.304+0000 7fa2d4553700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc1991f0 con 0x7fa2cc100cf0 2024-02-16T22:31:32.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.305+0000 7fa2d22ef700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2cc0ffaf0 0x7fa2cc1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:31:32.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.305+0000 7fa2d22ef700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2cc0ffaf0 0x7fa2cc1985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36376/0 (socket says 172.21.15.114:36376) 2024-02-16T22:31:32.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.305+0000 7fa2d22ef700 1 -- 172.21.15.114:0/3959986731 learned_addr learned my addr 172.21.15.114:0/3959986731 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:31:32.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.305+0000 7fa2d22ef700 1 -- 172.21.15.114:0/3959986731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2cc100cf0 msgr2=0x7fa2cc198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:32.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.305+0000 7fa2d22ef700 1 --2- 172.21.15.114:0/3959986731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2cc100cf0 0x7fa2cc198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:32.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.305+0000 7fa2d22ef700 1 -- 172.21.15.114:0/3959986731 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2bc0095f0 con 0x7fa2cc0ffaf0 2024-02-16T22:31:32.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.305+0000 7fa2d22ef700 1 --2- 172.21.15.114:0/3959986731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2cc0ffaf0 0x7fa2cc1985e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800c910 tx=0x7fa2c800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:31:32.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.305+0000 7fa2c37fe700 1 -- 172.21.15.114:0/3959986731 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2c80041f0 con 0x7fa2cc0ffaf0 2024-02-16T22:31:32.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.305+0000 7fa2c37fe700 1 -- 172.21.15.114:0/3959986731 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa2c8007db0 con 0x7fa2cc0ffaf0 2024-02-16T22:31:32.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.305+0000 7fa2d4553700 1 -- 172.21.15.114:0/3959986731 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2cc19dc60 con 0x7fa2cc0ffaf0 2024-02-16T22:31:32.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.305+0000 7fa2c37fe700 1 -- 172.21.15.114:0/3959986731 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2c8003d60 con 0x7fa2cc0ffaf0 2024-02-16T22:31:32.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.305+0000 7fa2d4553700 1 -- 172.21.15.114:0/3959986731 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2cc19e180 con 0x7fa2cc0ffaf0 2024-02-16T22:31:32.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.306+0000 7fa2d4553700 1 -- 172.21.15.114:0/3959986731 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2cc108650 con 0x7fa2cc0ffaf0 2024-02-16T22:31:32.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.307+0000 7fa2c37fe700 1 -- 172.21.15.114:0/3959986731 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fa2c8007900 con 0x7fa2cc0ffaf0 2024-02-16T22:31:32.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.307+0000 7fa2c37fe700 1 --2- 172.21.15.114:0/3959986731 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fa2b806d8a0 0x7fa2b806fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:31:32.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.307+0000 7fa2c37fe700 1 -- 172.21.15.114:0/3959986731 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7fa2c808d2e0 con 0x7fa2cc0ffaf0 2024-02-16T22:31:32.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.307+0000 7fa2d1aee700 1 --2- 172.21.15.114:0/3959986731 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fa2b806d8a0 0x7fa2b806fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:31:32.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.307+0000 7fa2d1aee700 1 --2- 172.21.15.114:0/3959986731 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fa2b806d8a0 0x7fa2b806fd60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00b5c0 tx=0x7fa2bc005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:31:32.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.312+0000 7fa2c37fe700 1 -- 172.21.15.114:0/3959986731 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa2c805a510 con 0x7fa2cc0ffaf0 2024-02-16T22:31:32.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.594+0000 7fa2d4553700 1 -- 172.21.15.114:0/3959986731 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa2cc108810 con 0x7fa2cc0ffaf0 2024-02-16T22:31:32.597 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:32 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:32.597 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:32 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:32.597 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:32 smithi114 ceph-mon[94452]: from='client.34558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:31:32.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.595+0000 7fa2c37fe700 1 -- 172.21.15.114:0/3959986731 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fa2c805a0a0 con 0x7fa2cc0ffaf0 2024-02-16T22:31:32.597 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:31:32.597 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": { 2024-02-16T22:31:32.598 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:31:32.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:31:32.598 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": { 2024-02-16T22:31:32.598 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:31:32.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:31:32.598 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": { 2024-02-16T22:31:32.598 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T22:31:32.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:31:32.598 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-02-16T22:31:32.598 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T22:31:32.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:31:32.598 INFO:teuthology.orchestra.run.smithi114.stdout: "overall": { 2024-02-16T22:31:32.598 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-16T22:31:32.598 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-02-16T22:31:32.599 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:31:32.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.597+0000 7fa2d4553700 1 -- 172.21.15.114:0/3959986731 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fa2b806d8a0 msgr2=0x7fa2b806fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:32.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.597+0000 7fa2d4553700 1 --2- 172.21.15.114:0/3959986731 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fa2b806d8a0 0x7fa2b806fd60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00b5c0 tx=0x7fa2bc005fd0 comp rx=0 tx=0).stop 2024-02-16T22:31:32.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.597+0000 7fa2d4553700 1 -- 172.21.15.114:0/3959986731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2cc0ffaf0 msgr2=0x7fa2cc1985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:32.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.597+0000 7fa2d4553700 1 --2- 172.21.15.114:0/3959986731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2cc0ffaf0 0x7fa2cc1985e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800c910 tx=0x7fa2c800ccd0 comp rx=0 tx=0).stop 2024-02-16T22:31:32.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.597+0000 7fa2d4553700 1 -- 172.21.15.114:0/3959986731 shutdown_connections 2024-02-16T22:31:32.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.597+0000 7fa2d4553700 1 --2- 172.21.15.114:0/3959986731 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fa2b806d8a0 0x7fa2b806fd60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:32.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.597+0000 7fa2d4553700 1 --2- 172.21.15.114:0/3959986731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2cc0ffaf0 0x7fa2cc1985e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:32.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.597+0000 7fa2d4553700 1 --2- 172.21.15.114:0/3959986731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2cc100cf0 0x7fa2cc198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:32.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.597+0000 7fa2d4553700 1 -- 172.21.15.114:0/3959986731 >> 172.21.15.114:0/3959986731 conn(0x7fa2cc0fb070 msgr2=0x7fa2cc068d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:31:32.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.597+0000 7fa2d4553700 1 -- 172.21.15.114:0/3959986731 shutdown_connections 2024-02-16T22:31:32.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.598+0000 7fa2d4553700 1 -- 172.21.15.114:0/3959986731 wait complete. 2024-02-16T22:31:32.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.707+0000 7fe13d248700 1 -- 172.21.15.114:0/1762054204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1380ffab0 msgr2=0x7fe1380ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:32.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.707+0000 7fe13d248700 1 --2- 172.21.15.114:0/1762054204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1380ffab0 0x7fe1380ffed0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe128009910 tx=0x7fe128009c20 comp rx=0 tx=0).stop 2024-02-16T22:31:32.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.707+0000 7fe13d248700 1 -- 172.21.15.114:0/1762054204 shutdown_connections 2024-02-16T22:31:32.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.707+0000 7fe13d248700 1 --2- 172.21.15.114:0/1762054204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe138100cb0 0x7fe138101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:32.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.707+0000 7fe13d248700 1 --2- 172.21.15.114:0/1762054204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1380ffab0 0x7fe1380ffed0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:32.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.707+0000 7fe13d248700 1 -- 172.21.15.114:0/1762054204 >> 172.21.15.114:0/1762054204 conn(0x7fe1380fb050 msgr2=0x7fe1380fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:31:32.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.707+0000 7fe13d248700 1 -- 172.21.15.114:0/1762054204 shutdown_connections 2024-02-16T22:31:32.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.707+0000 7fe13d248700 1 -- 172.21.15.114:0/1762054204 wait complete. 2024-02-16T22:31:32.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.708+0000 7fe13d248700 1 Processor -- start 2024-02-16T22:31:32.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.708+0000 7fe13d248700 1 -- start start 2024-02-16T22:31:32.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.709+0000 7fe13d248700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1380ffab0 0x7fe138196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:31:32.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.709+0000 7fe13d248700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe138100cb0 0x7fe1381968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:31:32.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.709+0000 7fe13d248700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe138196ed0 con 0x7fe1380ffab0 2024-02-16T22:31:32.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.709+0000 7fe13d248700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe138197010 con 0x7fe138100cb0 2024-02-16T22:31:32.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.709+0000 7fe136d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1380ffab0 0x7fe138196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:31:32.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.709+0000 7fe136d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1380ffab0 0x7fe138196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36398/0 (socket says 172.21.15.114:36398) 2024-02-16T22:31:32.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.709+0000 7fe136d9d700 1 -- 172.21.15.114:0/2586543066 learned_addr learned my addr 172.21.15.114:0/2586543066 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:31:32.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.709+0000 7fe13659c700 1 --2- 172.21.15.114:0/2586543066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe138100cb0 0x7fe1381968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:31:32.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.709+0000 7fe136d9d700 1 -- 172.21.15.114:0/2586543066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe138100cb0 msgr2=0x7fe1381968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:32.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.709+0000 7fe136d9d700 1 --2- 172.21.15.114:0/2586543066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe138100cb0 0x7fe1381968b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:32.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.709+0000 7fe136d9d700 1 -- 172.21.15.114:0/2586543066 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1280095f0 con 0x7fe1380ffab0 2024-02-16T22:31:32.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.709+0000 7fe136d9d700 1 --2- 172.21.15.114:0/2586543066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1380ffab0 0x7fe138196370 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe12800bda0 tx=0x7fe12800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:31:32.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.709+0000 7fe11ffff700 1 -- 172.21.15.114:0/2586543066 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe12801d070 con 0x7fe1380ffab0 2024-02-16T22:31:32.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.709+0000 7fe11ffff700 1 -- 172.21.15.114:0/2586543066 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe128004c50 con 0x7fe1380ffab0 2024-02-16T22:31:32.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.709+0000 7fe13d248700 1 -- 172.21.15.114:0/2586543066 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe13819ba60 con 0x7fe1380ffab0 2024-02-16T22:31:32.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.709+0000 7fe11ffff700 1 -- 172.21.15.114:0/2586543066 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe12800f690 con 0x7fe1380ffab0 2024-02-16T22:31:32.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.710+0000 7fe13d248700 1 -- 172.21.15.114:0/2586543066 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe13819bf50 con 0x7fe1380ffab0 2024-02-16T22:31:32.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.710+0000 7fe13d248700 1 -- 172.21.15.114:0/2586543066 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe138108610 con 0x7fe1380ffab0 2024-02-16T22:31:32.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.711+0000 7fe11ffff700 1 -- 172.21.15.114:0/2586543066 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fe1280229b0 con 0x7fe1380ffab0 2024-02-16T22:31:32.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.711+0000 7fe11ffff700 1 --2- 172.21.15.114:0/2586543066 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe12006d8a0 0x7fe12006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:31:32.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.711+0000 7fe11ffff700 1 -- 172.21.15.114:0/2586543066 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7fe12808ed10 con 0x7fe1380ffab0 2024-02-16T22:31:32.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.711+0000 7fe13659c700 1 --2- 172.21.15.114:0/2586543066 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe12006d8a0 0x7fe12006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:31:32.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.712+0000 7fe13659c700 1 --2- 172.21.15.114:0/2586543066 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe12006d8a0 0x7fe12006fd60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe12c005950 tx=0x7fe12c005cf0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:31:32.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.716+0000 7fe11ffff700 1 -- 172.21.15.114:0/2586543066 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe12805bff0 con 0x7fe1380ffab0 2024-02-16T22:31:32.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:32 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:32.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:32 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:32.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:32 smithi123 ceph-mon[82331]: from='client.34558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:31:32.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.952+0000 7fe13d248700 1 -- 172.21.15.114:0/2586543066 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe13819c990 con 0x7fe1380ffab0 2024-02-16T22:31:32.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.953+0000 7fe11ffff700 1 -- 172.21.15.114:0/2586543066 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fe128027770 con 0x7fe1380ffab0 2024-02-16T22:31:32.955 INFO:teuthology.orchestra.run.smithi114.stdout:e37 2024-02-16T22:31:32.955 INFO:teuthology.orchestra.run.smithi114.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:31:32.955 INFO:teuthology.orchestra.run.smithi114.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:31:32.955 INFO:teuthology.orchestra.run.smithi114.stdout:legacy client fscid: 1 2024-02-16T22:31:32.955 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:31:32.955 INFO:teuthology.orchestra.run.smithi114.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:31:32.955 INFO:teuthology.orchestra.run.smithi114.stdout:fs_name cephfs 2024-02-16T22:31:32.955 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 37 2024-02-16T22:31:32.955 INFO:teuthology.orchestra.run.smithi114.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:31:32.955 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-02-16T22:18:02.108825+0000 2024-02-16T22:31:32.955 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-02-16T22:29:48.901292+0000 2024-02-16T22:31:32.955 INFO:teuthology.orchestra.run.smithi114.stdout:tableserver 0 2024-02-16T22:31:32.955 INFO:teuthology.orchestra.run.smithi114.stdout:root 0 2024-02-16T22:31:32.956 INFO:teuthology.orchestra.run.smithi114.stdout:session_timeout 60 2024-02-16T22:31:32.956 INFO:teuthology.orchestra.run.smithi114.stdout:session_autoclose 300 2024-02-16T22:31:32.956 INFO:teuthology.orchestra.run.smithi114.stdout:max_file_size 1099511627776 2024-02-16T22:31:32.956 INFO:teuthology.orchestra.run.smithi114.stdout:max_xattr_size 65536 2024-02-16T22:31:32.956 INFO:teuthology.orchestra.run.smithi114.stdout:required_client_features {} 2024-02-16T22:31:32.956 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure 0 2024-02-16T22:31:32.956 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure_osd_epoch 82 2024-02-16T22:31:32.956 INFO:teuthology.orchestra.run.smithi114.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-16T22:31:32.956 INFO:teuthology.orchestra.run.smithi114.stdout:max_mds 1 2024-02-16T22:31:32.956 INFO:teuthology.orchestra.run.smithi114.stdout:in 0 2024-02-16T22:31:32.956 INFO:teuthology.orchestra.run.smithi114.stdout:up {0=34404} 2024-02-16T22:31:32.956 INFO:teuthology.orchestra.run.smithi114.stdout:failed 2024-02-16T22:31:32.957 INFO:teuthology.orchestra.run.smithi114.stdout:damaged 2024-02-16T22:31:32.957 INFO:teuthology.orchestra.run.smithi114.stdout:stopped 2024-02-16T22:31:32.957 INFO:teuthology.orchestra.run.smithi114.stdout:data_pools [3] 2024-02-16T22:31:32.957 INFO:teuthology.orchestra.run.smithi114.stdout:metadata_pool 2 2024-02-16T22:31:32.957 INFO:teuthology.orchestra.run.smithi114.stdout:inline_data enabled 2024-02-16T22:31:32.957 INFO:teuthology.orchestra.run.smithi114.stdout:balancer 2024-02-16T22:31:32.957 INFO:teuthology.orchestra.run.smithi114.stdout:bal_rank_mask -1 2024-02-16T22:31:32.957 INFO:teuthology.orchestra.run.smithi114.stdout:standby_count_wanted 1 2024-02-16T22:31:32.957 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.zzryke{0:34404} state up:active seq 23 join_fscid=1 addr [v2:172.21.15.114:6826/2017800151,v1:172.21.15.114:6827/2017800151] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T22:31:32.957 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:31:32.957 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:31:32.957 INFO:teuthology.orchestra.run.smithi114.stdout:Standby daemons: 2024-02-16T22:31:32.957 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:31:32.957 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.fdxpey{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.114:6828/3702914193,v1:172.21.15.114:6829/3702914193] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T22:31:32.957 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.vhlqbe{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.123:6824/1425065679,v1:172.21.15.123:6825/1425065679] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T22:31:32.958 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.gopcfa{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.123:6826/3128771108,v1:172.21.15.123:6827/3128771108] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T22:31:32.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.954+0000 7fe13d248700 1 -- 172.21.15.114:0/2586543066 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe12006d8a0 msgr2=0x7fe12006fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:32.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.954+0000 7fe13d248700 1 --2- 172.21.15.114:0/2586543066 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe12006d8a0 0x7fe12006fd60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe12c005950 tx=0x7fe12c005cf0 comp rx=0 tx=0).stop 2024-02-16T22:31:32.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.954+0000 7fe13d248700 1 -- 172.21.15.114:0/2586543066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1380ffab0 msgr2=0x7fe138196370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:32.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.954+0000 7fe13d248700 1 --2- 172.21.15.114:0/2586543066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1380ffab0 0x7fe138196370 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe12800bda0 tx=0x7fe12800be80 comp rx=0 tx=0).stop 2024-02-16T22:31:32.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.955+0000 7fe13d248700 1 -- 172.21.15.114:0/2586543066 shutdown_connections 2024-02-16T22:31:32.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.955+0000 7fe13d248700 1 --2- 172.21.15.114:0/2586543066 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe12006d8a0 0x7fe12006fd60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:32.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.955+0000 7fe13d248700 1 --2- 172.21.15.114:0/2586543066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1380ffab0 0x7fe138196370 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:32.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.955+0000 7fe13d248700 1 --2- 172.21.15.114:0/2586543066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe138100cb0 0x7fe1381968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:32.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.955+0000 7fe13d248700 1 -- 172.21.15.114:0/2586543066 >> 172.21.15.114:0/2586543066 conn(0x7fe1380fb050 msgr2=0x7fe138068c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:31:32.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.955+0000 7fe13d248700 1 -- 172.21.15.114:0/2586543066 shutdown_connections 2024-02-16T22:31:32.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:32.955+0000 7fe13d248700 1 -- 172.21.15.114:0/2586543066 wait complete. 2024-02-16T22:31:32.959 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 37 2024-02-16T22:31:33.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.060+0000 7f7cb7533700 1 -- 172.21.15.114:0/1106163746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cb00ffab0 msgr2=0x7f7cb00ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:33.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.060+0000 7f7cb7533700 1 --2- 172.21.15.114:0/1106163746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cb00ffab0 0x7f7cb00ffed0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009910 tx=0x7f7ca0009c20 comp rx=0 tx=0).stop 2024-02-16T22:31:33.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.060+0000 7f7cb7533700 1 -- 172.21.15.114:0/1106163746 shutdown_connections 2024-02-16T22:31:33.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.060+0000 7f7cb7533700 1 --2- 172.21.15.114:0/1106163746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb0100cb0 0x7f7cb0101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:33.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.060+0000 7f7cb7533700 1 --2- 172.21.15.114:0/1106163746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cb00ffab0 0x7f7cb00ffed0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:33.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.060+0000 7f7cb7533700 1 -- 172.21.15.114:0/1106163746 >> 172.21.15.114:0/1106163746 conn(0x7f7cb00fb030 msgr2=0x7f7cb00fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:31:33.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.060+0000 7f7cb7533700 1 -- 172.21.15.114:0/1106163746 shutdown_connections 2024-02-16T22:31:33.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.060+0000 7f7cb7533700 1 -- 172.21.15.114:0/1106163746 wait complete. 2024-02-16T22:31:33.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.061+0000 7f7cb7533700 1 Processor -- start 2024-02-16T22:31:33.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.061+0000 7f7cb7533700 1 -- start start 2024-02-16T22:31:33.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.061+0000 7f7cb7533700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb00ffab0 0x7f7cb01952e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:31:33.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.061+0000 7f7cb7533700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cb0100cb0 0x7f7cb0195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:31:33.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.061+0000 7f7cb7533700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb0195e40 con 0x7f7cb0100cb0 2024-02-16T22:31:33.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.061+0000 7f7cb7533700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb0195f80 con 0x7f7cb00ffab0 2024-02-16T22:31:33.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.061+0000 7f7cb4ace700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cb0100cb0 0x7f7cb0195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:31:33.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.061+0000 7f7cb4ace700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cb0100cb0 0x7f7cb0195820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36420/0 (socket says 172.21.15.114:36420) 2024-02-16T22:31:33.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.061+0000 7f7cb4ace700 1 -- 172.21.15.114:0/302240447 learned_addr learned my addr 172.21.15.114:0/302240447 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:31:33.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.061+0000 7f7cb52cf700 1 --2- 172.21.15.114:0/302240447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb00ffab0 0x7f7cb01952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:31:33.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.061+0000 7f7cb4ace700 1 -- 172.21.15.114:0/302240447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb00ffab0 msgr2=0x7f7cb01952e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:33.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.061+0000 7f7cb4ace700 1 --2- 172.21.15.114:0/302240447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb00ffab0 0x7f7cb01952e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:33.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.061+0000 7f7cb4ace700 1 -- 172.21.15.114:0/302240447 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca00095f0 con 0x7f7cb0100cb0 2024-02-16T22:31:33.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.062+0000 7f7cb4ace700 1 --2- 172.21.15.114:0/302240447 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cb0100cb0 0x7f7cb0195820 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7cac00b4b0 tx=0x7f7cac00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:31:33.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.062+0000 7f7ca67fc700 1 -- 172.21.15.114:0/302240447 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cac011800 con 0x7f7cb0100cb0 2024-02-16T22:31:33.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.062+0000 7f7ca67fc700 1 -- 172.21.15.114:0/302240447 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7cac011e10 con 0x7f7cb0100cb0 2024-02-16T22:31:33.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.062+0000 7f7cb7533700 1 -- 172.21.15.114:0/302240447 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb019aa30 con 0x7f7cb0100cb0 2024-02-16T22:31:33.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.062+0000 7f7cb7533700 1 -- 172.21.15.114:0/302240447 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb019b000 con 0x7f7cb0100cb0 2024-02-16T22:31:33.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.063+0000 7f7ca67fc700 1 -- 172.21.15.114:0/302240447 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cac00f2e0 con 0x7f7cb0100cb0 2024-02-16T22:31:33.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.063+0000 7f7cb7533700 1 -- 172.21.15.114:0/302240447 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb0108690 con 0x7f7cb0100cb0 2024-02-16T22:31:33.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.064+0000 7f7ca67fc700 1 -- 172.21.15.114:0/302240447 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f7cac010380 con 0x7f7cb0100cb0 2024-02-16T22:31:33.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.064+0000 7f7ca67fc700 1 --2- 172.21.15.114:0/302240447 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f7c9c06d8a0 0x7f7c9c06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:31:33.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.064+0000 7f7ca67fc700 1 -- 172.21.15.114:0/302240447 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f7cac08d130 con 0x7f7cb0100cb0 2024-02-16T22:31:33.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.064+0000 7f7cb52cf700 1 --2- 172.21.15.114:0/302240447 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f7c9c06d8a0 0x7f7c9c06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:31:33.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.065+0000 7f7cb52cf700 1 --2- 172.21.15.114:0/302240447 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f7c9c06d8a0 0x7f7c9c06fd60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7ca00059d0 tx=0x7f7ca0005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:31:33.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.069+0000 7f7ca67fc700 1 -- 172.21.15.114:0/302240447 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7cac05a3e0 con 0x7f7cb0100cb0 2024-02-16T22:31:33.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.286+0000 7f7cb7533700 1 -- 172.21.15.114:0/302240447 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7cb0105600 con 0x7f7c9c06d8a0 2024-02-16T22:31:33.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.288+0000 7f7ca67fc700 1 -- 172.21.15.114:0/302240447 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+460 (secure 0 0 0) 0x7f7cb0105600 con 0x7f7c9c06d8a0 2024-02-16T22:31:33.290 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:31:33.290 INFO:teuthology.orchestra.run.smithi114.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T22:31:33.290 INFO:teuthology.orchestra.run.smithi114.stdout: "in_progress": true, 2024-02-16T22:31:33.290 INFO:teuthology.orchestra.run.smithi114.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T22:31:33.290 INFO:teuthology.orchestra.run.smithi114.stdout: "services_complete": [ 2024-02-16T22:31:33.290 INFO:teuthology.orchestra.run.smithi114.stdout: "osd", 2024-02-16T22:31:33.290 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph-exporter", 2024-02-16T22:31:33.290 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr", 2024-02-16T22:31:33.290 INFO:teuthology.orchestra.run.smithi114.stdout: "mon", 2024-02-16T22:31:33.290 INFO:teuthology.orchestra.run.smithi114.stdout: "mds", 2024-02-16T22:31:33.290 INFO:teuthology.orchestra.run.smithi114.stdout: "crash" 2024-02-16T22:31:33.291 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-02-16T22:31:33.291 INFO:teuthology.orchestra.run.smithi114.stdout: "progress": "18/23 daemons upgraded", 2024-02-16T22:31:33.291 INFO:teuthology.orchestra.run.smithi114.stdout: "message": "Currently upgrading grafana daemons", 2024-02-16T22:31:33.291 INFO:teuthology.orchestra.run.smithi114.stdout: "is_paused": false 2024-02-16T22:31:33.291 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:31:33.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.290+0000 7f7cb7533700 1 -- 172.21.15.114:0/302240447 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f7c9c06d8a0 msgr2=0x7f7c9c06fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:33.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.290+0000 7f7cb7533700 1 --2- 172.21.15.114:0/302240447 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f7c9c06d8a0 0x7f7c9c06fd60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7ca00059d0 tx=0x7f7ca0005960 comp rx=0 tx=0).stop 2024-02-16T22:31:33.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.290+0000 7f7cb7533700 1 -- 172.21.15.114:0/302240447 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cb0100cb0 msgr2=0x7f7cb0195820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:33.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.290+0000 7f7cb7533700 1 --2- 172.21.15.114:0/302240447 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cb0100cb0 0x7f7cb0195820 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7cac00b4b0 tx=0x7f7cac00b870 comp rx=0 tx=0).stop 2024-02-16T22:31:33.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.290+0000 7f7cb7533700 1 -- 172.21.15.114:0/302240447 shutdown_connections 2024-02-16T22:31:33.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.290+0000 7f7cb7533700 1 --2- 172.21.15.114:0/302240447 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f7c9c06d8a0 0x7f7c9c06fd60 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:33.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.290+0000 7f7cb7533700 1 --2- 172.21.15.114:0/302240447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb00ffab0 0x7f7cb01952e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:33.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.290+0000 7f7cb7533700 1 --2- 172.21.15.114:0/302240447 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cb0100cb0 0x7f7cb0195820 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:33.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.290+0000 7f7cb7533700 1 -- 172.21.15.114:0/302240447 >> 172.21.15.114:0/302240447 conn(0x7f7cb00fb030 msgr2=0x7f7cb0103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:31:33.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.290+0000 7f7cb7533700 1 -- 172.21.15.114:0/302240447 shutdown_connections 2024-02-16T22:31:33.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.290+0000 7f7cb7533700 1 -- 172.21.15.114:0/302240447 wait complete. 2024-02-16T22:31:33.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.397+0000 7fdc3272b700 1 -- 172.21.15.114:0/2427726112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc2c101dc0 msgr2=0x7fdc2c102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:33.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.397+0000 7fdc3272b700 1 --2- 172.21.15.114:0/2427726112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc2c101dc0 0x7fdc2c102220 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdc20009910 tx=0x7fdc20009c20 comp rx=0 tx=0).stop 2024-02-16T22:31:33.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.397+0000 7fdc3272b700 1 -- 172.21.15.114:0/2427726112 shutdown_connections 2024-02-16T22:31:33.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.397+0000 7fdc3272b700 1 --2- 172.21.15.114:0/2427726112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc2c101dc0 0x7fdc2c102220 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:33.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.397+0000 7fdc3272b700 1 --2- 172.21.15.114:0/2427726112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc2c100bc0 0x7fdc2c100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:33.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.397+0000 7fdc3272b700 1 -- 172.21.15.114:0/2427726112 >> 172.21.15.114:0/2427726112 conn(0x7fdc2c0fc140 msgr2=0x7fdc2c0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:31:33.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.397+0000 7fdc3272b700 1 -- 172.21.15.114:0/2427726112 shutdown_connections 2024-02-16T22:31:33.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.397+0000 7fdc3272b700 1 -- 172.21.15.114:0/2427726112 wait complete. 2024-02-16T22:31:33.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.397+0000 7fdc3272b700 1 Processor -- start 2024-02-16T22:31:33.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.397+0000 7fdc3272b700 1 -- start start 2024-02-16T22:31:33.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.398+0000 7fdc3272b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc2c100bc0 0x7fdc2c196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:31:33.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.398+0000 7fdc3272b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc2c101dc0 0x7fdc2c1968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:31:33.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.398+0000 7fdc3272b700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c196ed0 con 0x7fdc2c101dc0 2024-02-16T22:31:33.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.398+0000 7fdc3272b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c197010 con 0x7fdc2c100bc0 2024-02-16T22:31:33.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.398+0000 7fdc2b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc2c101dc0 0x7fdc2c1968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:31:33.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.398+0000 7fdc2b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc2c101dc0 0x7fdc2c1968b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36450/0 (socket says 172.21.15.114:36450) 2024-02-16T22:31:33.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.398+0000 7fdc2b7fe700 1 -- 172.21.15.114:0/1080188791 learned_addr learned my addr 172.21.15.114:0/1080188791 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:31:33.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.398+0000 7fdc2bfff700 1 --2- 172.21.15.114:0/1080188791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc2c100bc0 0x7fdc2c196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:31:33.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.398+0000 7fdc2b7fe700 1 -- 172.21.15.114:0/1080188791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc2c100bc0 msgr2=0x7fdc2c196370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:33.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.398+0000 7fdc2b7fe700 1 --2- 172.21.15.114:0/1080188791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc2c100bc0 0x7fdc2c196370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:33.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.398+0000 7fdc2b7fe700 1 -- 172.21.15.114:0/1080188791 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc200095f0 con 0x7fdc2c101dc0 2024-02-16T22:31:33.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.398+0000 7fdc2b7fe700 1 --2- 172.21.15.114:0/1080188791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc2c101dc0 0x7fdc2c1968b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdc20005b40 tx=0x7fdc2000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:31:33.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.398+0000 7fdc297fa700 1 -- 172.21.15.114:0/1080188791 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc2001d070 con 0x7fdc2c101dc0 2024-02-16T22:31:33.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.398+0000 7fdc3272b700 1 -- 172.21.15.114:0/1080188791 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc2c19ba60 con 0x7fdc2c101dc0 2024-02-16T22:31:33.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.398+0000 7fdc297fa700 1 -- 172.21.15.114:0/1080188791 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdc2000f460 con 0x7fdc2c101dc0 2024-02-16T22:31:33.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.398+0000 7fdc297fa700 1 -- 172.21.15.114:0/1080188791 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc20021620 con 0x7fdc2c101dc0 2024-02-16T22:31:33.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.399+0000 7fdc3272b700 1 -- 172.21.15.114:0/1080188791 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc2c19bf50 con 0x7fdc2c101dc0 2024-02-16T22:31:33.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.399+0000 7fdc3272b700 1 -- 172.21.15.114:0/1080188791 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc2c109720 con 0x7fdc2c101dc0 2024-02-16T22:31:33.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.400+0000 7fdc297fa700 1 -- 172.21.15.114:0/1080188791 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fdc2000fa40 con 0x7fdc2c101dc0 2024-02-16T22:31:33.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.400+0000 7fdc297fa700 1 --2- 172.21.15.114:0/1080188791 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fdc1406d8a0 0x7fdc1406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:31:33.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.400+0000 7fdc297fa700 1 -- 172.21.15.114:0/1080188791 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7fdc2008e8e0 con 0x7fdc2c101dc0 2024-02-16T22:31:33.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.401+0000 7fdc2bfff700 1 --2- 172.21.15.114:0/1080188791 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fdc1406d8a0 0x7fdc1406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:31:33.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.401+0000 7fdc2bfff700 1 --2- 172.21.15.114:0/1080188791 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fdc1406d8a0 0x7fdc1406fd60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c101c20 tx=0x7fdc1c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:31:33.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.405+0000 7fdc297fa700 1 -- 172.21.15.114:0/1080188791 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdc2005bb10 con 0x7fdc2c101dc0 2024-02-16T22:31:33.531 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:33 smithi114 ceph-mon[94452]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:31:33.531 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:33 smithi114 ceph-mon[94452]: pgmap v270: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:33.531 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:33 smithi114 ceph-mon[94452]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:31:33.531 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:33 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/3959986731' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:33.531 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:33 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/2586543066' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:31:33.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.692+0000 7fdc3272b700 1 -- 172.21.15.114:0/1080188791 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fdc2c19c650 con 0x7fdc2c101dc0 2024-02-16T22:31:33.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.692+0000 7fdc297fa700 1 -- 172.21.15.114:0/1080188791 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fdc2005b6a0 con 0x7fdc2c101dc0 2024-02-16T22:31:33.695 INFO:teuthology.orchestra.run.smithi114.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:31:33.695 INFO:teuthology.orchestra.run.smithi114.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:31:33.695 INFO:teuthology.orchestra.run.smithi114.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:31:33.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.694+0000 7fdc3272b700 1 -- 172.21.15.114:0/1080188791 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fdc1406d8a0 msgr2=0x7fdc1406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:33.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.694+0000 7fdc3272b700 1 --2- 172.21.15.114:0/1080188791 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fdc1406d8a0 0x7fdc1406fd60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c101c20 tx=0x7fdc1c00b410 comp rx=0 tx=0).stop 2024-02-16T22:31:33.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.694+0000 7fdc3272b700 1 -- 172.21.15.114:0/1080188791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc2c101dc0 msgr2=0x7fdc2c1968b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:31:33.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.694+0000 7fdc3272b700 1 --2- 172.21.15.114:0/1080188791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc2c101dc0 0x7fdc2c1968b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdc20005b40 tx=0x7fdc2000bd70 comp rx=0 tx=0).stop 2024-02-16T22:31:33.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.694+0000 7fdc3272b700 1 -- 172.21.15.114:0/1080188791 shutdown_connections 2024-02-16T22:31:33.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.694+0000 7fdc3272b700 1 --2- 172.21.15.114:0/1080188791 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fdc1406d8a0 0x7fdc1406fd60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:33.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.694+0000 7fdc3272b700 1 --2- 172.21.15.114:0/1080188791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc2c100bc0 0x7fdc2c196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:33.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.694+0000 7fdc3272b700 1 --2- 172.21.15.114:0/1080188791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc2c101dc0 0x7fdc2c1968b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:31:33.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.694+0000 7fdc3272b700 1 -- 172.21.15.114:0/1080188791 >> 172.21.15.114:0/1080188791 conn(0x7fdc2c0fc140 msgr2=0x7fdc2c104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:31:33.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.694+0000 7fdc3272b700 1 -- 172.21.15.114:0/1080188791 shutdown_connections 2024-02-16T22:31:33.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:31:33.694+0000 7fdc3272b700 1 -- 172.21.15.114:0/1080188791 wait complete. 2024-02-16T22:31:33.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:33 smithi123 ceph-mon[82331]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:31:33.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:33 smithi123 ceph-mon[82331]: pgmap v270: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:33.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:33 smithi123 ceph-mon[82331]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:31:33.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:33 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/3959986731' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:33.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:33 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/2586543066' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:31:34.694 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:34 smithi114 ceph-mon[94452]: from='client.34578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:31:34.695 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:34 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/1080188791' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:31:34.801 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:34 smithi123 ceph-mon[82331]: from='client.34578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:31:34.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:34 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/1080188791' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:31:35.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:35 smithi114 ceph-mon[94452]: pgmap v271: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:35.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:35 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:35.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:35 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:35.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:35 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:31:35.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:35 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:31:35.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:35 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:35.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:35 smithi123 ceph-mon[82331]: pgmap v271: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:35.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:35 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:35.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:35 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:35.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:35 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:31:35.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:35 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:31:35.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:35 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:36.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T22:31:36.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T22:31:36.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:31:36.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.703 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.704 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.704 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.704 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.704 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.704 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.704 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.704 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: Upgrade: Finalizing container_image settings 2024-02-16T22:31:36.704 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:36.704 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-02-16T22:31:36.704 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-02-16T22:31:36.704 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T22:31:36.704 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-02-16T22:31:36.704 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-02-16T22:31:36.705 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-02-16T22:31:36.705 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-02-16T22:31:36.705 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-02-16T22:31:36.705 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-02-16T22:31:36.705 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-02-16T22:31:36.705 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-02-16T22:31:36.705 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-02-16T22:31:36.705 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-02-16T22:31:36.705 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-02-16T22:31:36.705 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T22:31:36.705 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-02-16T22:31:36.705 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-02-16T22:31:36.705 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-02-16T22:31:36.705 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-02-16T22:31:36.706 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-02-16T22:31:36.706 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-02-16T22:31:36.706 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-02-16T22:31:36.706 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-02-16T22:31:36.706 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T22:31:36.706 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T22:31:36.706 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T22:31:36.706 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T22:31:36.706 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T22:31:36.706 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T22:31:36.706 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: Upgrade: Complete! 2024-02-16T22:31:36.706 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-02-16T22:31:36.706 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-02-16T22:31:36.706 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:31:36.707 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:36.707 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:31:36.707 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:31:36.707 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:36.707 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:31:36.707 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:31:36.707 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:31:36.707 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:36.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T22:31:36.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T22:31:36.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:31:36.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.803 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:31:36.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: Upgrade: Finalizing container_image settings 2024-02-16T22:31:36.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:36.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-02-16T22:31:36.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-02-16T22:31:36.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T22:31:36.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-02-16T22:31:36.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-02-16T22:31:36.804 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-02-16T22:31:36.805 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-02-16T22:31:36.805 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-02-16T22:31:36.805 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-02-16T22:31:36.805 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-02-16T22:31:36.805 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-02-16T22:31:36.805 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-02-16T22:31:36.805 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-02-16T22:31:36.805 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-02-16T22:31:36.805 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T22:31:36.805 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-02-16T22:31:36.805 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-02-16T22:31:36.805 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-02-16T22:31:36.805 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-02-16T22:31:36.805 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-02-16T22:31:36.805 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-02-16T22:31:36.806 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-02-16T22:31:36.806 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-02-16T22:31:36.806 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T22:31:36.806 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T22:31:36.806 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T22:31:36.806 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T22:31:36.806 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T22:31:36.806 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T22:31:36.806 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: Upgrade: Complete! 2024-02-16T22:31:36.806 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-02-16T22:31:36.806 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-02-16T22:31:36.806 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:31:36.806 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:36.806 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:31:36.807 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:31:36.807 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:36.807 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:31:36.807 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:31:36.807 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:31:36.807 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:31:37.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:37 smithi114 ceph-mon[94452]: pgmap v272: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:38.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:37 smithi123 ceph-mon[82331]: pgmap v272: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:39.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:39 smithi114 ceph-mon[94452]: pgmap v273: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:40.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:39 smithi123 ceph-mon[82331]: pgmap v273: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:41.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:41 smithi114 ceph-mon[94452]: pgmap v274: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:42.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:41 smithi123 ceph-mon[82331]: pgmap v274: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:43.727 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:43 smithi114 ceph-mon[94452]: pgmap v275: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:43.854 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:43 smithi123 ceph-mon[82331]: pgmap v275: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:45.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:45 smithi114 ceph-mon[94452]: pgmap v276: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:46.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:45 smithi123 ceph-mon[82331]: pgmap v276: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:46.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:31:47.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:46 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:31:47.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:47 smithi114 ceph-mon[94452]: pgmap v277: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:48.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:47 smithi123 ceph-mon[82331]: pgmap v277: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:49.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:49 smithi114 ceph-mon[94452]: pgmap v278: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:50.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:49 smithi123 ceph-mon[82331]: pgmap v278: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:51.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:51 smithi114 ceph-mon[94452]: pgmap v279: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:52.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:51 smithi123 ceph-mon[82331]: pgmap v279: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:53.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:53 smithi114 ceph-mon[94452]: pgmap v280: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:54.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:53 smithi123 ceph-mon[82331]: pgmap v280: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:55.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:55 smithi114 ceph-mon[94452]: pgmap v281: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:56.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:55 smithi123 ceph-mon[82331]: pgmap v281: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:57.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:57 smithi114 ceph-mon[94452]: pgmap v282: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:58.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:57 smithi123 ceph-mon[82331]: pgmap v282: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:31:59.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:31:59 smithi114 ceph-mon[94452]: pgmap v283: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:00.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:31:59 smithi123 ceph-mon[82331]: pgmap v283: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:01.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:01 smithi114 ceph-mon[94452]: pgmap v284: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:01.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:32:02.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:01 smithi123 ceph-mon[82331]: pgmap v284: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:02.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:01 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:32:03.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:03 smithi114 ceph-mon[94452]: pgmap v285: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:03.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.811+0000 7f522223f700 1 -- 172.21.15.114:0/4132607743 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f521c0feed0 msgr2=0x7f521c0ff2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:32:03.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.811+0000 7f522223f700 1 --2- 172.21.15.114:0/4132607743 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f521c0feed0 0x7f521c0ff2f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5210009910 tx=0x7f5210009c20 comp rx=0 tx=0).stop 2024-02-16T22:32:03.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.811+0000 7f522223f700 1 -- 172.21.15.114:0/4132607743 shutdown_connections 2024-02-16T22:32:03.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.811+0000 7f522223f700 1 --2- 172.21.15.114:0/4132607743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f521c101eb0 0x7f521c102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:32:03.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.811+0000 7f522223f700 1 --2- 172.21.15.114:0/4132607743 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f521c0feed0 0x7f521c0ff2f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:32:03.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.811+0000 7f522223f700 1 -- 172.21.15.114:0/4132607743 >> 172.21.15.114:0/4132607743 conn(0x7f521c0fa810 msgr2=0x7f521c0fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:32:03.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.811+0000 7f522223f700 1 -- 172.21.15.114:0/4132607743 shutdown_connections 2024-02-16T22:32:03.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.811+0000 7f522223f700 1 -- 172.21.15.114:0/4132607743 wait complete. 2024-02-16T22:32:03.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.812+0000 7f522223f700 1 Processor -- start 2024-02-16T22:32:03.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.812+0000 7f522223f700 1 -- start start 2024-02-16T22:32:03.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.812+0000 7f522223f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f521c0feed0 0x7f521c196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:32:03.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.812+0000 7f522223f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f521c101eb0 0x7f521c196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:32:03.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.812+0000 7f522223f700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c196fa0 con 0x7f521c101eb0 2024-02-16T22:32:03.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.812+0000 7f522223f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c1970e0 con 0x7f521c0feed0 2024-02-16T22:32:03.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.813+0000 7f521affd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f521c101eb0 0x7f521c196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:32:03.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.813+0000 7f521affd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f521c101eb0 0x7f521c196980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39872/0 (socket says 172.21.15.114:39872) 2024-02-16T22:32:03.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.813+0000 7f521affd700 1 -- 172.21.15.114:0/1859649586 learned_addr learned my addr 172.21.15.114:0/1859649586 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:32:03.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.813+0000 7f521b7fe700 1 --2- 172.21.15.114:0/1859649586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f521c0feed0 0x7f521c196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:32:03.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.813+0000 7f521affd700 1 -- 172.21.15.114:0/1859649586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f521c0feed0 msgr2=0x7f521c196440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:32:03.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.813+0000 7f521affd700 1 --2- 172.21.15.114:0/1859649586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f521c0feed0 0x7f521c196440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:32:03.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.813+0000 7f521affd700 1 -- 172.21.15.114:0/1859649586 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52100095f0 con 0x7f521c101eb0 2024-02-16T22:32:03.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.813+0000 7f521b7fe700 1 --2- 172.21.15.114:0/1859649586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f521c0feed0 0x7f521c196440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:32:03.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.813+0000 7f521affd700 1 --2- 172.21.15.114:0/1859649586 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f521c101eb0 0x7f521c196980 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f520c00c8a0 tx=0x7f520c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:32:03.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.813+0000 7f5218ff9700 1 -- 172.21.15.114:0/1859649586 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f520c00cea0 con 0x7f521c101eb0 2024-02-16T22:32:03.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.813+0000 7f5218ff9700 1 -- 172.21.15.114:0/1859649586 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f520c004830 con 0x7f521c101eb0 2024-02-16T22:32:03.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.813+0000 7f522223f700 1 -- 172.21.15.114:0/1859649586 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f521c19bb90 con 0x7f521c101eb0 2024-02-16T22:32:03.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.813+0000 7f5218ff9700 1 -- 172.21.15.114:0/1859649586 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f520c0055b0 con 0x7f521c101eb0 2024-02-16T22:32:03.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.813+0000 7f522223f700 1 -- 172.21.15.114:0/1859649586 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f521c19c160 con 0x7f521c101eb0 2024-02-16T22:32:03.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.814+0000 7f522223f700 1 -- 172.21.15.114:0/1859649586 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f521c109770 con 0x7f521c101eb0 2024-02-16T22:32:03.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.815+0000 7f5218ff9700 1 -- 172.21.15.114:0/1859649586 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f520c0049a0 con 0x7f521c101eb0 2024-02-16T22:32:03.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.815+0000 7f5218ff9700 1 --2- 172.21.15.114:0/1859649586 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f520406d8a0 0x7f520406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:32:03.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.815+0000 7f5218ff9700 1 -- 172.21.15.114:0/1859649586 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f520c08d1b0 con 0x7f521c101eb0 2024-02-16T22:32:03.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.815+0000 7f521b7fe700 1 --2- 172.21.15.114:0/1859649586 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f520406d8a0 0x7f520406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:32:03.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.816+0000 7f521b7fe700 1 --2- 172.21.15.114:0/1859649586 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f520406d8a0 0x7f520406fd60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5210005090 tx=0x7f521001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:32:03.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:03.820+0000 7f5218ff9700 1 -- 172.21.15.114:0/1859649586 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f520c05a460 con 0x7f521c101eb0 2024-02-16T22:32:04.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:04.031+0000 7f522223f700 1 -- 172.21.15.114:0/1859649586 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f521c106800 con 0x7f520406d8a0 2024-02-16T22:32:04.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:04.032+0000 7f5218ff9700 1 -- 172.21.15.114:0/1859649586 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f521c106800 con 0x7f520406d8a0 2024-02-16T22:32:04.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:04.034+0000 7f522223f700 1 -- 172.21.15.114:0/1859649586 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f520406d8a0 msgr2=0x7f520406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:32:04.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:04.034+0000 7f522223f700 1 --2- 172.21.15.114:0/1859649586 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f520406d8a0 0x7f520406fd60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5210005090 tx=0x7f521001a040 comp rx=0 tx=0).stop 2024-02-16T22:32:04.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:04.034+0000 7f522223f700 1 -- 172.21.15.114:0/1859649586 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f521c101eb0 msgr2=0x7f521c196980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:32:04.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:04.034+0000 7f522223f700 1 --2- 172.21.15.114:0/1859649586 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f521c101eb0 0x7f521c196980 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f520c00c8a0 tx=0x7f520c00cc60 comp rx=0 tx=0).stop 2024-02-16T22:32:04.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:04.034+0000 7f522223f700 1 -- 172.21.15.114:0/1859649586 shutdown_connections 2024-02-16T22:32:04.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:04.034+0000 7f522223f700 1 --2- 172.21.15.114:0/1859649586 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f520406d8a0 0x7f520406fd60 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:32:04.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:04.034+0000 7f522223f700 1 --2- 172.21.15.114:0/1859649586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f521c0feed0 0x7f521c196440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:32:04.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:04.034+0000 7f522223f700 1 --2- 172.21.15.114:0/1859649586 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f521c101eb0 0x7f521c196980 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:32:04.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:04.034+0000 7f522223f700 1 -- 172.21.15.114:0/1859649586 >> 172.21.15.114:0/1859649586 conn(0x7f521c0fa810 msgr2=0x7f521c1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:32:04.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:04.035+0000 7f522223f700 1 -- 172.21.15.114:0/1859649586 shutdown_connections 2024-02-16T22:32:04.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:04.035+0000 7f522223f700 1 -- 172.21.15.114:0/1859649586 wait complete. 2024-02-16T22:32:04.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:03 smithi123 ceph-mon[82331]: pgmap v285: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:04.655 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph orch ps' 2024-02-16T22:32:04.867 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:32:05.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:05 smithi114 ceph-mon[94452]: pgmap v286: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:05.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:05 smithi114 ceph-mon[94452]: from='client.34586 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:32:06.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:05 smithi123 ceph-mon[82331]: pgmap v286: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:06.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:05 smithi123 ceph-mon[82331]: from='client.34586 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:32:06.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.530+0000 7fa95fc52700 1 -- 172.21.15.114:0/2310793563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa958100da0 msgr2=0x7fa958101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:32:06.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.530+0000 7fa95fc52700 1 --2- 172.21.15.114:0/2310793563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa958100da0 0x7fa958101180 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa95400b0a0 tx=0x7fa95400b3b0 comp rx=0 tx=0).stop 2024-02-16T22:32:06.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.530+0000 7fa95fc52700 1 -- 172.21.15.114:0/2310793563 shutdown_connections 2024-02-16T22:32:06.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.530+0000 7fa95fc52700 1 --2- 172.21.15.114:0/2310793563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa958101750 0x7fa9581057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:32:06.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.530+0000 7fa95fc52700 1 --2- 172.21.15.114:0/2310793563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa958100da0 0x7fa958101180 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:32:06.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.530+0000 7fa95fc52700 1 -- 172.21.15.114:0/2310793563 >> 172.21.15.114:0/2310793563 conn(0x7fa9580fc630 msgr2=0x7fa9580fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:32:06.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.530+0000 7fa95fc52700 1 -- 172.21.15.114:0/2310793563 shutdown_connections 2024-02-16T22:32:06.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.530+0000 7fa95fc52700 1 -- 172.21.15.114:0/2310793563 wait complete. 2024-02-16T22:32:06.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.531+0000 7fa95fc52700 1 Processor -- start 2024-02-16T22:32:06.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.531+0000 7fa95fc52700 1 -- start start 2024-02-16T22:32:06.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.531+0000 7fa95fc52700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa958100da0 0x7fa95819c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:32:06.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.531+0000 7fa95fc52700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa958101750 0x7fa95819cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:32:06.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.531+0000 7fa95fc52700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95819d170 con 0x7fa958101750 2024-02-16T22:32:06.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.531+0000 7fa95fc52700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa958196620 con 0x7fa958100da0 2024-02-16T22:32:06.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.531+0000 7fa95d1ed700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa958101750 0x7fa95819cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:32:06.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.531+0000 7fa95d1ed700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa958101750 0x7fa95819cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39888/0 (socket says 172.21.15.114:39888) 2024-02-16T22:32:06.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.531+0000 7fa95d1ed700 1 -- 172.21.15.114:0/1554346393 learned_addr learned my addr 172.21.15.114:0/1554346393 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:32:06.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.531+0000 7fa95d9ee700 1 --2- 172.21.15.114:0/1554346393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa958100da0 0x7fa95819c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:32:06.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.532+0000 7fa95d1ed700 1 -- 172.21.15.114:0/1554346393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa958100da0 msgr2=0x7fa95819c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:32:06.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.532+0000 7fa95d1ed700 1 --2- 172.21.15.114:0/1554346393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa958100da0 0x7fa95819c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:32:06.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.532+0000 7fa95d1ed700 1 -- 172.21.15.114:0/1554346393 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa954009d00 con 0x7fa958101750 2024-02-16T22:32:06.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.532+0000 7fa95d1ed700 1 --2- 172.21.15.114:0/1554346393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa958101750 0x7fa95819cae0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa94800e8f0 tx=0x7fa94800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:32:06.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.532+0000 7fa94effd700 1 -- 172.21.15.114:0/1554346393 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa948009790 con 0x7fa958101750 2024-02-16T22:32:06.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.532+0000 7fa94effd700 1 -- 172.21.15.114:0/1554346393 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa94800c810 con 0x7fa958101750 2024-02-16T22:32:06.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.532+0000 7fa95fc52700 1 -- 172.21.15.114:0/1554346393 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa958196900 con 0x7fa958101750 2024-02-16T22:32:06.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.532+0000 7fa94effd700 1 -- 172.21.15.114:0/1554346393 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa948005580 con 0x7fa958101750 2024-02-16T22:32:06.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.532+0000 7fa95fc52700 1 -- 172.21.15.114:0/1554346393 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa958196e50 con 0x7fa958101750 2024-02-16T22:32:06.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.533+0000 7fa95fc52700 1 -- 172.21.15.114:0/1554346393 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa958109140 con 0x7fa958101750 2024-02-16T22:32:06.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.534+0000 7fa94effd700 1 -- 172.21.15.114:0/1554346393 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fa94800c980 con 0x7fa958101750 2024-02-16T22:32:06.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.534+0000 7fa94effd700 1 --2- 172.21.15.114:0/1554346393 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fa94406d8a0 0x7fa94406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:32:06.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.534+0000 7fa95d9ee700 1 --2- 172.21.15.114:0/1554346393 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fa94406d8a0 0x7fa94406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:32:06.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.534+0000 7fa94effd700 1 -- 172.21.15.114:0/1554346393 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7fa948014070 con 0x7fa958101750 2024-02-16T22:32:06.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.535+0000 7fa95d9ee700 1 --2- 172.21.15.114:0/1554346393 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fa94406d8a0 0x7fa94406fd60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa954000f80 tx=0x7fa9540060c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:32:06.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.539+0000 7fa94effd700 1 -- 172.21.15.114:0/1554346393 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa94805aac0 con 0x7fa958101750 2024-02-16T22:32:06.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.745+0000 7fa95fc52700 1 -- 172.21.15.114:0/1554346393 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa958197130 con 0x7fa94406d8a0 2024-02-16T22:32:06.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.756+0000 7fa94effd700 1 -- 172.21.15.114:0/1554346393 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fa958197130 con 0x7fa94406d8a0 2024-02-16T22:32:06.758 INFO:teuthology.orchestra.run.smithi114.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:32:06.758 INFO:teuthology.orchestra.run.smithi114.stdout:alertmanager.smithi114 smithi114 *:9093,9094 running (68s) 36s ago 19m 15.0M - 0.25.0 c8568f914cd2 a46f553e8ce0 2024-02-16T22:32:06.758 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi114 smithi114 running (2m) 36s ago 19m 7826k - 19.0.0-1450-gf78a58c0 f081f847070a 1f621c33f6d4 2024-02-16T22:32:06.758 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi123 smithi123 running (119s) 98s ago 18m 7482k - 19.0.0-1450-gf78a58c0 f081f847070a 2f0b3c7f313b 2024-02-16T22:32:06.758 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi114 smithi114 running (7m) 36s ago 19m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 9b8ae9efe83b 2024-02-16T22:32:06.758 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi123 smithi123 running (7m) 98s ago 18m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 6d97263a0ffa 2024-02-16T22:32:06.758 INFO:teuthology.orchestra.run.smithi114.stdout:grafana.smithi114 smithi114 *:3000 running (41s) 36s ago 19m 57.4M - 9.4.12 a0ea5e3c46d9 32e00d2cbcbb 2024-02-16T22:32:06.758 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.fdxpey smithi114 running (3m) 36s ago 13m 14.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2984ae8a955a 2024-02-16T22:32:06.759 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.zzryke smithi114 running (3m) 36s ago 14m 84.6M - 19.0.0-1450-gf78a58c0 f081f847070a 63fa2af7290e 2024-02-16T22:32:06.759 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.gopcfa smithi123 running (2m) 98s ago 13m 14.9M - 19.0.0-1450-gf78a58c0 f081f847070a ec6612a62e2a 2024-02-16T22:32:06.759 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.vhlqbe smithi123 running (2m) 98s ago 13m 15.9M - 19.0.0-1450-gf78a58c0 f081f847070a be790c6287e7 2024-02-16T22:32:06.759 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi114.vcullz smithi114 *:8443,9283,8765 running (10m) 36s ago 20m 512M - 19.0.0-1450-gf78a58c0 f081f847070a 14ebeb7888c0 2024-02-16T22:32:06.759 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi123.inzgjd smithi123 *:8443,9283,8765 running (8m) 98s ago 18m 451M - 19.0.0-1450-gf78a58c0 f081f847070a f10aab9faf9e 2024-02-16T22:32:06.759 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi114 smithi114 running (8m) 36s ago 21m 58.4M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 58ac06372523 2024-02-16T22:32:06.759 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi123 smithi123 running (8m) 98s ago 18m 43.0M 2048M 19.0.0-1450-gf78a58c0 f081f847070a f76485e0ea24 2024-02-16T22:32:06.759 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi114 smithi114 *:9100 running (105s) 36s ago 19m 17.1M - 1.5.0 0da6a335fe13 ee1aa5ff6491 2024-02-16T22:32:06.759 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi123 smithi123 *:9100 running (101s) 98s ago 18m 3657k - 1.5.0 0da6a335fe13 a5e2e31d4643 2024-02-16T22:32:06.759 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 smithi114 running (6m) 36s ago 17m 100M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 1978fdad94a5 2024-02-16T22:32:06.759 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 smithi114 running (6m) 36s ago 17m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 3f90e792898f 2024-02-16T22:32:06.759 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 smithi114 running (5m) 36s ago 16m 91.7M 1216M 19.0.0-1450-gf78a58c0 f081f847070a c663dfa0518b 2024-02-16T22:32:06.759 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 smithi123 running (5m) 98s ago 16m 113M 2240M 19.0.0-1450-gf78a58c0 f081f847070a b921448ba169 2024-02-16T22:32:06.759 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 smithi123 running (4m) 98s ago 15m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 54c2926d56b1 2024-02-16T22:32:06.760 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 smithi123 running (3m) 98s ago 15m 93.6M 2240M 19.0.0-1450-gf78a58c0 f081f847070a dd99bf034749 2024-02-16T22:32:06.760 INFO:teuthology.orchestra.run.smithi114.stdout:prometheus.smithi114 smithi114 *:9095 running (85s) 36s ago 19m 50.1M - 2.43.0 a07b618ecd1d a6b9ebc86282 2024-02-16T22:32:06.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.758+0000 7fa95fc52700 1 -- 172.21.15.114:0/1554346393 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fa94406d8a0 msgr2=0x7fa94406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:32:06.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.758+0000 7fa95fc52700 1 --2- 172.21.15.114:0/1554346393 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fa94406d8a0 0x7fa94406fd60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa954000f80 tx=0x7fa9540060c0 comp rx=0 tx=0).stop 2024-02-16T22:32:06.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.758+0000 7fa95fc52700 1 -- 172.21.15.114:0/1554346393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa958101750 msgr2=0x7fa95819cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:32:06.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.758+0000 7fa95fc52700 1 --2- 172.21.15.114:0/1554346393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa958101750 0x7fa95819cae0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa94800e8f0 tx=0x7fa94800ec00 comp rx=0 tx=0).stop 2024-02-16T22:32:06.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.758+0000 7fa95fc52700 1 -- 172.21.15.114:0/1554346393 shutdown_connections 2024-02-16T22:32:06.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.758+0000 7fa95fc52700 1 --2- 172.21.15.114:0/1554346393 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fa94406d8a0 0x7fa94406fd60 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:32:06.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.758+0000 7fa95fc52700 1 --2- 172.21.15.114:0/1554346393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa958100da0 0x7fa95819c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:32:06.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.758+0000 7fa95fc52700 1 --2- 172.21.15.114:0/1554346393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa958101750 0x7fa95819cae0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:32:06.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.758+0000 7fa95fc52700 1 -- 172.21.15.114:0/1554346393 >> 172.21.15.114:0/1554346393 conn(0x7fa9580fc630 msgr2=0x7fa9580fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:32:06.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.758+0000 7fa95fc52700 1 -- 172.21.15.114:0/1554346393 shutdown_connections 2024-02-16T22:32:06.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:06.758+0000 7fa95fc52700 1 -- 172.21.15.114:0/1554346393 wait complete. 2024-02-16T22:32:06.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:06 smithi114 ceph-mon[94452]: pgmap v287: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:07.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:06 smithi123 ceph-mon[82331]: pgmap v287: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:07.404 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph orch upgrade status' 2024-02-16T22:32:07.620 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:32:07.892 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:07 smithi114 ceph-mon[94452]: from='client.34590 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:32:08.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:07 smithi123 ceph-mon[82331]: from='client.34590 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:32:08.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:08 smithi114 ceph-mon[94452]: pgmap v288: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:08.903 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:08 smithi123 ceph-mon[82331]: pgmap v288: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:09.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.243+0000 7f0730ef5700 1 -- 172.21.15.114:0/3246371952 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f072c0ff510 msgr2=0x7f072c0ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:32:09.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.243+0000 7f0730ef5700 1 --2- 172.21.15.114:0/3246371952 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f072c0ff510 0x7f072c0ff8f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f071c009910 tx=0x7f071c009c20 comp rx=0 tx=0).stop 2024-02-16T22:32:09.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.243+0000 7f0730ef5700 1 -- 172.21.15.114:0/3246371952 shutdown_connections 2024-02-16T22:32:09.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.243+0000 7f0730ef5700 1 --2- 172.21.15.114:0/3246371952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f072c0ffec0 0x7f072c105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:32:09.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.243+0000 7f0730ef5700 1 --2- 172.21.15.114:0/3246371952 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f072c0ff510 0x7f072c0ff8f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:32:09.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.243+0000 7f0730ef5700 1 -- 172.21.15.114:0/3246371952 >> 172.21.15.114:0/3246371952 conn(0x7f072c074d80 msgr2=0x7f072c075190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:32:09.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.243+0000 7f0730ef5700 1 -- 172.21.15.114:0/3246371952 shutdown_connections 2024-02-16T22:32:09.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.243+0000 7f0730ef5700 1 -- 172.21.15.114:0/3246371952 wait complete. 2024-02-16T22:32:09.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.244+0000 7f0730ef5700 1 Processor -- start 2024-02-16T22:32:09.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.244+0000 7f0730ef5700 1 -- start start 2024-02-16T22:32:09.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.244+0000 7f0730ef5700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f072c0ff510 0x7f072c19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:32:09.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.244+0000 7f0730ef5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f072c0ffec0 0x7f072c19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:32:09.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.244+0000 7f0730ef5700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c19d040 con 0x7f072c0ff510 2024-02-16T22:32:09.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.244+0000 7f0730ef5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c196580 con 0x7f072c0ffec0 2024-02-16T22:32:09.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.244+0000 7f072a59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f072c0ff510 0x7f072c19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:32:09.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.244+0000 7f072a59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f072c0ff510 0x7f072c19c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39910/0 (socket says 172.21.15.114:39910) 2024-02-16T22:32:09.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.244+0000 7f072a59c700 1 -- 172.21.15.114:0/1047720041 learned_addr learned my addr 172.21.15.114:0/1047720041 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:32:09.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.244+0000 7f0729d9b700 1 --2- 172.21.15.114:0/1047720041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f072c0ffec0 0x7f072c19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:32:09.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.244+0000 7f072a59c700 1 -- 172.21.15.114:0/1047720041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f072c0ffec0 msgr2=0x7f072c19ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:32:09.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.244+0000 7f072a59c700 1 --2- 172.21.15.114:0/1047720041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f072c0ffec0 0x7f072c19ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:32:09.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.244+0000 7f072a59c700 1 -- 172.21.15.114:0/1047720041 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071c0095f0 con 0x7f072c0ff510 2024-02-16T22:32:09.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.244+0000 7f072a59c700 1 --2- 172.21.15.114:0/1047720041 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f072c0ff510 0x7f072c19c500 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f071c00bda0 tx=0x7f071c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:32:09.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.245+0000 7f071b7fe700 1 -- 172.21.15.114:0/1047720041 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f071c01d070 con 0x7f072c0ff510 2024-02-16T22:32:09.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.245+0000 7f071b7fe700 1 -- 172.21.15.114:0/1047720041 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f071c004fb0 con 0x7f072c0ff510 2024-02-16T22:32:09.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.245+0000 7f0730ef5700 1 -- 172.21.15.114:0/1047720041 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f072c196800 con 0x7f072c0ff510 2024-02-16T22:32:09.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.245+0000 7f071b7fe700 1 -- 172.21.15.114:0/1047720041 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f071c00f690 con 0x7f072c0ff510 2024-02-16T22:32:09.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.245+0000 7f0730ef5700 1 -- 172.21.15.114:0/1047720041 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f072c196cf0 con 0x7f072c0ff510 2024-02-16T22:32:09.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.246+0000 7f0730ef5700 1 -- 172.21.15.114:0/1047720041 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f072c109180 con 0x7f072c0ff510 2024-02-16T22:32:09.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.246+0000 7f071b7fe700 1 -- 172.21.15.114:0/1047720041 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f071c00f7f0 con 0x7f072c0ff510 2024-02-16T22:32:09.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.246+0000 7f071b7fe700 1 --2- 172.21.15.114:0/1047720041 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f071406d8a0 0x7f071406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:32:09.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.247+0000 7f071b7fe700 1 -- 172.21.15.114:0/1047720041 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f071c08ee20 con 0x7f072c0ff510 2024-02-16T22:32:09.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.247+0000 7f0729d9b700 1 --2- 172.21.15.114:0/1047720041 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f071406d8a0 0x7f071406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:32:09.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.247+0000 7f0729d9b700 1 --2- 172.21.15.114:0/1047720041 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f071406d8a0 0x7f071406fd60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0720005ea0 tx=0x7f0720005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:32:09.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.251+0000 7f071b7fe700 1 -- 172.21.15.114:0/1047720041 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f071c05c0c0 con 0x7f072c0ff510 2024-02-16T22:32:09.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.464+0000 7f0730ef5700 1 -- 172.21.15.114:0/1047720041 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f072c196fd0 con 0x7f071406d8a0 2024-02-16T22:32:09.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.465+0000 7f071b7fe700 1 -- 172.21.15.114:0/1047720041 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f072c196fd0 con 0x7f071406d8a0 2024-02-16T22:32:09.467 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:32:09.467 INFO:teuthology.orchestra.run.smithi114.stdout: "target_image": null, 2024-02-16T22:32:09.467 INFO:teuthology.orchestra.run.smithi114.stdout: "in_progress": false, 2024-02-16T22:32:09.467 INFO:teuthology.orchestra.run.smithi114.stdout: "which": "", 2024-02-16T22:32:09.467 INFO:teuthology.orchestra.run.smithi114.stdout: "services_complete": [], 2024-02-16T22:32:09.467 INFO:teuthology.orchestra.run.smithi114.stdout: "progress": null, 2024-02-16T22:32:09.467 INFO:teuthology.orchestra.run.smithi114.stdout: "message": "", 2024-02-16T22:32:09.467 INFO:teuthology.orchestra.run.smithi114.stdout: "is_paused": false 2024-02-16T22:32:09.467 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:32:09.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.467+0000 7f0730ef5700 1 -- 172.21.15.114:0/1047720041 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f071406d8a0 msgr2=0x7f071406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:32:09.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.467+0000 7f0730ef5700 1 --2- 172.21.15.114:0/1047720041 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f071406d8a0 0x7f071406fd60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0720005ea0 tx=0x7f0720005e30 comp rx=0 tx=0).stop 2024-02-16T22:32:09.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.467+0000 7f0730ef5700 1 -- 172.21.15.114:0/1047720041 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f072c0ff510 msgr2=0x7f072c19c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:32:09.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.467+0000 7f0730ef5700 1 --2- 172.21.15.114:0/1047720041 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f072c0ff510 0x7f072c19c500 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f071c00bda0 tx=0x7f071c00be80 comp rx=0 tx=0).stop 2024-02-16T22:32:09.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.467+0000 7f0730ef5700 1 -- 172.21.15.114:0/1047720041 shutdown_connections 2024-02-16T22:32:09.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.467+0000 7f0730ef5700 1 --2- 172.21.15.114:0/1047720041 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f071406d8a0 0x7f071406fd60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:32:09.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.467+0000 7f0730ef5700 1 --2- 172.21.15.114:0/1047720041 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f072c0ff510 0x7f072c19c500 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:32:09.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.467+0000 7f0730ef5700 1 --2- 172.21.15.114:0/1047720041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f072c0ffec0 0x7f072c19ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:32:09.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.467+0000 7f0730ef5700 1 -- 172.21.15.114:0/1047720041 >> 172.21.15.114:0/1047720041 conn(0x7f072c074d80 msgr2=0x7f072c0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:32:09.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.467+0000 7f0730ef5700 1 -- 172.21.15.114:0/1047720041 shutdown_connections 2024-02-16T22:32:09.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:09.467+0000 7f0730ef5700 1 -- 172.21.15.114:0/1047720041 wait complete. 2024-02-16T22:32:09.763 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:09 smithi114 ceph-mon[94452]: from='client.34594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:32:10.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:09 smithi123 ceph-mon[82331]: from='client.34594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:32:10.086 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph health detail' 2024-02-16T22:32:10.295 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:32:10.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:10 smithi114 ceph-mon[94452]: pgmap v289: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:11.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:10 smithi123 ceph-mon[82331]: pgmap v289: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:11.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.932+0000 7ff7258b2700 1 -- 172.21.15.114:0/2563215909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff720105a40 msgr2=0x7ff720105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:32:11.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.932+0000 7ff7258b2700 1 --2- 172.21.15.114:0/2563215909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff720105a40 0x7ff720105e20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff714009910 tx=0x7ff714009c20 comp rx=0 tx=0).stop 2024-02-16T22:32:11.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.932+0000 7ff7258b2700 1 -- 172.21.15.114:0/2563215909 shutdown_connections 2024-02-16T22:32:11.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.932+0000 7ff7258b2700 1 --2- 172.21.15.114:0/2563215909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7200ffa40 0x7ff7200ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:32:11.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.932+0000 7ff7258b2700 1 --2- 172.21.15.114:0/2563215909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff720105a40 0x7ff720105e20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:32:11.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.932+0000 7ff7258b2700 1 -- 172.21.15.114:0/2563215909 >> 172.21.15.114:0/2563215909 conn(0x7ff7200fb570 msgr2=0x7ff7200fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:32:11.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.932+0000 7ff7258b2700 1 -- 172.21.15.114:0/2563215909 shutdown_connections 2024-02-16T22:32:11.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.932+0000 7ff7258b2700 1 -- 172.21.15.114:0/2563215909 wait complete. 2024-02-16T22:32:11.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.933+0000 7ff7258b2700 1 Processor -- start 2024-02-16T22:32:11.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.933+0000 7ff7258b2700 1 -- start start 2024-02-16T22:32:11.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.933+0000 7ff7258b2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7200ffa40 0x7ff720197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:32:11.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.933+0000 7ff7258b2700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff720105a40 0x7ff720197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:32:11.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.933+0000 7ff7258b2700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff720198180 con 0x7ff720105a40 2024-02-16T22:32:11.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.933+0000 7ff7258b2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7201982c0 con 0x7ff7200ffa40 2024-02-16T22:32:11.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.933+0000 7ff71e7fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff720105a40 0x7ff720197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:32:11.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.933+0000 7ff71e7fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff720105a40 0x7ff720197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52062/0 (socket says 172.21.15.114:52062) 2024-02-16T22:32:11.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.934+0000 7ff71e7fc700 1 -- 172.21.15.114:0/422851253 learned_addr learned my addr 172.21.15.114:0/422851253 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:32:11.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.934+0000 7ff71effd700 1 --2- 172.21.15.114:0/422851253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7200ffa40 0x7ff720197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:32:11.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.934+0000 7ff71e7fc700 1 -- 172.21.15.114:0/422851253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7200ffa40 msgr2=0x7ff720197560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:32:11.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.934+0000 7ff71e7fc700 1 --2- 172.21.15.114:0/422851253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7200ffa40 0x7ff720197560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:32:11.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.934+0000 7ff71e7fc700 1 -- 172.21.15.114:0/422851253 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7140095f0 con 0x7ff720105a40 2024-02-16T22:32:11.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.934+0000 7ff71e7fc700 1 --2- 172.21.15.114:0/422851253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff720105a40 0x7ff720197aa0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff71000d8d0 tx=0x7ff71000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:32:11.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.934+0000 7ff7248b0700 1 -- 172.21.15.114:0/422851253 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7100098a0 con 0x7ff720105a40 2024-02-16T22:32:11.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.934+0000 7ff7248b0700 1 -- 172.21.15.114:0/422851253 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff710010460 con 0x7ff720105a40 2024-02-16T22:32:11.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.934+0000 7ff7258b2700 1 -- 172.21.15.114:0/422851253 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7201a05d0 con 0x7ff720105a40 2024-02-16T22:32:11.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.934+0000 7ff7248b0700 1 -- 172.21.15.114:0/422851253 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff71000f5f0 con 0x7ff720105a40 2024-02-16T22:32:11.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.934+0000 7ff7258b2700 1 -- 172.21.15.114:0/422851253 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7201a0af0 con 0x7ff720105a40 2024-02-16T22:32:11.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.935+0000 7ff7258b2700 1 -- 172.21.15.114:0/422851253 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff720108040 con 0x7ff720105a40 2024-02-16T22:32:11.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.936+0000 7ff7248b0700 1 -- 172.21.15.114:0/422851253 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7ff710009a00 con 0x7ff720105a40 2024-02-16T22:32:11.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.936+0000 7ff7248b0700 1 --2- 172.21.15.114:0/422851253 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff70806d8a0 0x7ff70806fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:32:11.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.937+0000 7ff7248b0700 1 -- 172.21.15.114:0/422851253 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7ff71008d3f0 con 0x7ff720105a40 2024-02-16T22:32:11.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.937+0000 7ff71effd700 1 --2- 172.21.15.114:0/422851253 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff70806d8a0 0x7ff70806fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:32:11.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.937+0000 7ff71effd700 1 --2- 172.21.15.114:0/422851253 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff70806d8a0 0x7ff70806fd60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff714006010 tx=0x7ff71401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:32:11.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:11.941+0000 7ff7248b0700 1 -- 172.21.15.114:0/422851253 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff710056b70 con 0x7ff720105a40 2024-02-16T22:32:12.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:12.215+0000 7ff7258b2700 1 -- 172.21.15.114:0/422851253 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff72004ea90 con 0x7ff720105a40 2024-02-16T22:32:12.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:12.216+0000 7ff7248b0700 1 -- 172.21.15.114:0/422851253 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7ff71005a230 con 0x7ff720105a40 2024-02-16T22:32:12.218 INFO:teuthology.orchestra.run.smithi114.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-02-16T22:32:12.218 INFO:teuthology.orchestra.run.smithi114.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-02-16T22:32:12.218 INFO:teuthology.orchestra.run.smithi114.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-02-16T22:32:12.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:12.218+0000 7ff7258b2700 1 -- 172.21.15.114:0/422851253 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff70806d8a0 msgr2=0x7ff70806fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:32:12.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:12.218+0000 7ff7258b2700 1 --2- 172.21.15.114:0/422851253 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff70806d8a0 0x7ff70806fd60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff714006010 tx=0x7ff71401a040 comp rx=0 tx=0).stop 2024-02-16T22:32:12.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:12.218+0000 7ff7258b2700 1 -- 172.21.15.114:0/422851253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff720105a40 msgr2=0x7ff720197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:32:12.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:12.218+0000 7ff7258b2700 1 --2- 172.21.15.114:0/422851253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff720105a40 0x7ff720197aa0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff71000d8d0 tx=0x7ff71000dbe0 comp rx=0 tx=0).stop 2024-02-16T22:32:12.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:12.218+0000 7ff7258b2700 1 -- 172.21.15.114:0/422851253 shutdown_connections 2024-02-16T22:32:12.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:12.218+0000 7ff7258b2700 1 --2- 172.21.15.114:0/422851253 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff70806d8a0 0x7ff70806fd60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:32:12.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:12.218+0000 7ff7258b2700 1 --2- 172.21.15.114:0/422851253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7200ffa40 0x7ff720197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:32:12.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:12.218+0000 7ff7258b2700 1 --2- 172.21.15.114:0/422851253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff720105a40 0x7ff720197aa0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:32:12.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:12.218+0000 7ff7258b2700 1 -- 172.21.15.114:0/422851253 >> 172.21.15.114:0/422851253 conn(0x7ff7200fb570 msgr2=0x7ff7200fbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:32:12.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:12.218+0000 7ff7258b2700 1 -- 172.21.15.114:0/422851253 shutdown_connections 2024-02-16T22:32:12.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:12.218+0000 7ff7258b2700 1 -- 172.21.15.114:0/422851253 wait complete. 2024-02-16T22:32:12.809 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph versions' 2024-02-16T22:32:13.017 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:32:13.130 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:12 smithi114 ceph-mon[94452]: pgmap v290: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:13.130 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:12 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/422851253' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:32:13.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:12 smithi123 ceph-mon[82331]: pgmap v290: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:13.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:12 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/422851253' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T22:32:14.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.675+0000 7f18c9a9a700 1 -- 172.21.15.114:0/317102528 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f18c4101d40 msgr2=0x7f18c4102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:32:14.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.675+0000 7f18c9a9a700 1 --2- 172.21.15.114:0/317102528 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f18c4101d40 0x7f18c4102120 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009910 tx=0x7f18b4009c20 comp rx=0 tx=0).stop 2024-02-16T22:32:14.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.675+0000 7f18c9a9a700 1 -- 172.21.15.114:0/317102528 shutdown_connections 2024-02-16T22:32:14.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.675+0000 7f18c9a9a700 1 --2- 172.21.15.114:0/317102528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18c4102660 0x7f18c410ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:32:14.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.675+0000 7f18c9a9a700 1 --2- 172.21.15.114:0/317102528 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f18c4101d40 0x7f18c4102120 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:32:14.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.675+0000 7f18c9a9a700 1 -- 172.21.15.114:0/317102528 >> 172.21.15.114:0/317102528 conn(0x7f18c40fb570 msgr2=0x7f18c40fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:32:14.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.675+0000 7f18c9a9a700 1 -- 172.21.15.114:0/317102528 shutdown_connections 2024-02-16T22:32:14.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.675+0000 7f18c9a9a700 1 -- 172.21.15.114:0/317102528 wait complete. 2024-02-16T22:32:14.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.675+0000 7f18c9a9a700 1 Processor -- start 2024-02-16T22:32:14.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.676+0000 7f18c9a9a700 1 -- start start 2024-02-16T22:32:14.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.676+0000 7f18c9a9a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f18c4101d40 0x7f18c419c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:32:14.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.676+0000 7f18c9a9a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18c4102660 0x7f18c419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:32:14.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.676+0000 7f18c9a9a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c419d2a0 con 0x7f18c4101d40 2024-02-16T22:32:14.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.676+0000 7f18c9a9a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c4196750 con 0x7f18c4102660 2024-02-16T22:32:14.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.676+0000 7f18c37fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f18c4101d40 0x7f18c419c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:32:14.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.676+0000 7f18c37fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f18c4101d40 0x7f18c419c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52088/0 (socket says 172.21.15.114:52088) 2024-02-16T22:32:14.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.676+0000 7f18c37fe700 1 -- 172.21.15.114:0/2228769448 learned_addr learned my addr 172.21.15.114:0/2228769448 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:32:14.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.676+0000 7f18baffd700 1 --2- 172.21.15.114:0/2228769448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18c4102660 0x7f18c419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:32:14.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.676+0000 7f18c37fe700 1 -- 172.21.15.114:0/2228769448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18c4102660 msgr2=0x7f18c419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:32:14.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.676+0000 7f18c37fe700 1 --2- 172.21.15.114:0/2228769448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18c4102660 0x7f18c419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:32:14.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.676+0000 7f18c37fe700 1 -- 172.21.15.114:0/2228769448 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b40095f0 con 0x7f18c4101d40 2024-02-16T22:32:14.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.676+0000 7f18c37fe700 1 --2- 172.21.15.114:0/2228769448 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f18c4101d40 0x7f18c419c6d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f18b4006010 tx=0x7f18b400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:32:14.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.677+0000 7f18c17fa700 1 -- 172.21.15.114:0/2228769448 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18b401d070 con 0x7f18c4101d40 2024-02-16T22:32:14.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.677+0000 7f18c9a9a700 1 -- 172.21.15.114:0/2228769448 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c41969d0 con 0x7f18c4101d40 2024-02-16T22:32:14.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.677+0000 7f18c17fa700 1 -- 172.21.15.114:0/2228769448 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f18b4004fb0 con 0x7f18c4101d40 2024-02-16T22:32:14.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.677+0000 7f18c17fa700 1 -- 172.21.15.114:0/2228769448 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18b4022ea0 con 0x7f18c4101d40 2024-02-16T22:32:14.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.677+0000 7f18c9a9a700 1 -- 172.21.15.114:0/2228769448 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c4196ec0 con 0x7f18c4101d40 2024-02-16T22:32:14.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.678+0000 7f18c9a9a700 1 -- 172.21.15.114:0/2228769448 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18c4108250 con 0x7f18c4101d40 2024-02-16T22:32:14.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.678+0000 7f18c17fa700 1 -- 172.21.15.114:0/2228769448 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f18b4022470 con 0x7f18c4101d40 2024-02-16T22:32:14.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.679+0000 7f18c17fa700 1 --2- 172.21.15.114:0/2228769448 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f18b006d8a0 0x7f18b006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:32:14.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.679+0000 7f18c17fa700 1 -- 172.21.15.114:0/2228769448 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f18b408ec60 con 0x7f18c4101d40 2024-02-16T22:32:14.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.679+0000 7f18baffd700 1 --2- 172.21.15.114:0/2228769448 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f18b006d8a0 0x7f18b006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:32:14.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.679+0000 7f18baffd700 1 --2- 172.21.15.114:0/2228769448 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f18b006d8a0 0x7f18b006fd60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f18c4197bc0 tx=0x7f18ac008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:32:14.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.683+0000 7f18c17fa700 1 -- 172.21.15.114:0/2228769448 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f18b405c060 con 0x7f18c4101d40 2024-02-16T22:32:14.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:14 smithi114 ceph-mon[94452]: pgmap v291: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:14.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.960+0000 7f18c9a9a700 1 -- 172.21.15.114:0/2228769448 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f18c404ea90 con 0x7f18c4101d40 2024-02-16T22:32:14.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.960+0000 7f18c17fa700 1 -- 172.21.15.114:0/2228769448 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f18b4027280 con 0x7f18c4101d40 2024-02-16T22:32:14.962 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:32:14.962 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": { 2024-02-16T22:32:14.962 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:32:14.962 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:32:14.962 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": { 2024-02-16T22:32:14.962 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:32:14.962 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:32:14.963 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": { 2024-02-16T22:32:14.963 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T22:32:14.963 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:32:14.963 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-02-16T22:32:14.963 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T22:32:14.963 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:32:14.963 INFO:teuthology.orchestra.run.smithi114.stdout: "overall": { 2024-02-16T22:32:14.963 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-16T22:32:14.963 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-02-16T22:32:14.963 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:32:14.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.963+0000 7f18c9a9a700 1 -- 172.21.15.114:0/2228769448 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f18b006d8a0 msgr2=0x7f18b006fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:32:14.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.963+0000 7f18c9a9a700 1 --2- 172.21.15.114:0/2228769448 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f18b006d8a0 0x7f18b006fd60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f18c4197bc0 tx=0x7f18ac008040 comp rx=0 tx=0).stop 2024-02-16T22:32:14.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.963+0000 7f18c9a9a700 1 -- 172.21.15.114:0/2228769448 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f18c4101d40 msgr2=0x7f18c419c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:32:14.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.963+0000 7f18c9a9a700 1 --2- 172.21.15.114:0/2228769448 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f18c4101d40 0x7f18c419c6d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f18b4006010 tx=0x7f18b400be20 comp rx=0 tx=0).stop 2024-02-16T22:32:14.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.963+0000 7f18c9a9a700 1 -- 172.21.15.114:0/2228769448 shutdown_connections 2024-02-16T22:32:14.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.963+0000 7f18c9a9a700 1 --2- 172.21.15.114:0/2228769448 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f18b006d8a0 0x7f18b006fd60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:32:14.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.963+0000 7f18c9a9a700 1 --2- 172.21.15.114:0/2228769448 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f18c4101d40 0x7f18c419c6d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:32:14.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.963+0000 7f18c9a9a700 1 --2- 172.21.15.114:0/2228769448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18c4102660 0x7f18c419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:32:14.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.963+0000 7f18c9a9a700 1 -- 172.21.15.114:0/2228769448 >> 172.21.15.114:0/2228769448 conn(0x7f18c40fb570 msgr2=0x7f18c4105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:32:14.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.963+0000 7f18c9a9a700 1 -- 172.21.15.114:0/2228769448 shutdown_connections 2024-02-16T22:32:14.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:32:14.963+0000 7f18c9a9a700 1 -- 172.21.15.114:0/2228769448 wait complete. 2024-02-16T22:32:15.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:14 smithi123 ceph-mon[82331]: pgmap v291: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:15.648 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-02-16T22:32:15.857 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:32:16.171 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:15 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/2228769448' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:32:16.172 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:15 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:32:16.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:15 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/2228769448' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:32:16.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:15 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:32:17.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:16 smithi114 ceph-mon[94452]: pgmap v292: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:17.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:16 smithi123 ceph-mon[82331]: pgmap v292: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:17.422 INFO:teuthology.orchestra.run.smithi114.stdout:wait for servicemap items w/ changing names to refresh 2024-02-16T22:32:18.120 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'sleep 60' 2024-02-16T22:32:18.336 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:32:19.134 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:18 smithi114 ceph-mon[94452]: pgmap v293: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:19.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:18 smithi123 ceph-mon[82331]: pgmap v293: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:21.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:20 smithi114 ceph-mon[94452]: pgmap v294: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:21.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:20 smithi123 ceph-mon[82331]: pgmap v294: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:23.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:22 smithi114 ceph-mon[94452]: pgmap v295: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:23.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:22 smithi123 ceph-mon[82331]: pgmap v295: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:25.202 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:24 smithi114 ceph-mon[94452]: pgmap v296: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:25.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:24 smithi123 ceph-mon[82331]: pgmap v296: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:27.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:26 smithi114 ceph-mon[94452]: pgmap v297: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:27.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:26 smithi123 ceph-mon[82331]: pgmap v297: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:29.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:28 smithi114 ceph-mon[94452]: pgmap v298: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:29.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:28 smithi123 ceph-mon[82331]: pgmap v298: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:31.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:30 smithi114 ceph-mon[94452]: pgmap v299: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:31.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:30 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:32:31.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:30 smithi123 ceph-mon[82331]: pgmap v299: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:31.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:30 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:32:33.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:32 smithi114 ceph-mon[94452]: pgmap v300: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:33.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:32 smithi123 ceph-mon[82331]: pgmap v300: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:35.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:34 smithi114 ceph-mon[94452]: pgmap v301: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:35.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:34 smithi123 ceph-mon[82331]: pgmap v301: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:37.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:36 smithi114 ceph-mon[94452]: pgmap v302: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:37.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:32:37.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:32:37.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:36 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:32:37.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:36 smithi123 ceph-mon[82331]: pgmap v302: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:37.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:32:37.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:32:37.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:36 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:32:38.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:37 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:32:38.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:37 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:32:38.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:37 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:32:38.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:37 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:32:38.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:37 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:32:38.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:37 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:32:39.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:39 smithi123 ceph-mon[82331]: pgmap v303: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:39.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:39 smithi114 ceph-mon[94452]: pgmap v303: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:41.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:41 smithi123 ceph-mon[82331]: pgmap v304: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:41.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:41 smithi114 ceph-mon[94452]: pgmap v304: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:43.798 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:43 smithi114 ceph-mon[94452]: pgmap v305: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:43.801 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:43 smithi123 ceph-mon[82331]: pgmap v305: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:45.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:45 smithi123 ceph-mon[82331]: pgmap v306: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:45.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:45 smithi114 ceph-mon[94452]: pgmap v306: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:46.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:46 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:32:46.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:32:47.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:47 smithi123 ceph-mon[82331]: pgmap v307: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:47.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:47 smithi114 ceph-mon[94452]: pgmap v307: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:49.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:49 smithi123 ceph-mon[82331]: pgmap v308: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:49.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:49 smithi114 ceph-mon[94452]: pgmap v308: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:51.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:51 smithi123 ceph-mon[82331]: pgmap v309: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:51.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:51 smithi114 ceph-mon[94452]: pgmap v309: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:53.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:53 smithi123 ceph-mon[82331]: pgmap v310: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:53.904 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:53 smithi114 ceph-mon[94452]: pgmap v310: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:55.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:55 smithi114 ceph-mon[94452]: pgmap v311: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:55.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:55 smithi123 ceph-mon[82331]: pgmap v311: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:57.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:57 smithi123 ceph-mon[82331]: pgmap v312: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:57.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:57 smithi114 ceph-mon[94452]: pgmap v312: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:59.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:32:59 smithi123 ceph-mon[82331]: pgmap v313: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:32:59.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:32:59 smithi114 ceph-mon[94452]: pgmap v313: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:01.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:01 smithi123 ceph-mon[82331]: pgmap v314: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:01.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:01 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:33:01.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:01 smithi114 ceph-mon[94452]: pgmap v314: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:01.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:33:03.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:03 smithi123 ceph-mon[82331]: pgmap v315: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:03.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:03 smithi114 ceph-mon[94452]: pgmap v315: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:05.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:05 smithi123 ceph-mon[82331]: pgmap v316: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:05.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:05 smithi114 ceph-mon[94452]: pgmap v316: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:07.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:07 smithi114 ceph-mon[94452]: pgmap v317: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:08.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:07 smithi123 ceph-mon[82331]: pgmap v317: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:09.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:09 smithi123 ceph-mon[82331]: pgmap v318: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:09.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:09 smithi114 ceph-mon[94452]: pgmap v318: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:11.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:11 smithi114 ceph-mon[94452]: pgmap v319: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:12.051 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:11 smithi123 ceph-mon[82331]: pgmap v319: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:13.856 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:13 smithi123 ceph-mon[82331]: pgmap v320: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:13.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:13 smithi114 ceph-mon[94452]: pgmap v320: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:15.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:15 smithi114 ceph-mon[94452]: pgmap v321: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:16.051 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:15 smithi123 ceph-mon[82331]: pgmap v321: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:16.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:16 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:33:17.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:16 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:33:17.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:17 smithi114 ceph-mon[94452]: pgmap v322: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:18.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:17 smithi123 ceph-mon[82331]: pgmap v322: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:19.843 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:19 smithi114 ceph-mon[94452]: pgmap v323: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:20.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:19 smithi123 ceph-mon[82331]: pgmap v323: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:20.578 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph orch ps' 2024-02-16T22:33:20.797 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:33:21.857 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:21 smithi114 ceph-mon[94452]: pgmap v324: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:22.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:21 smithi123 ceph-mon[82331]: pgmap v324: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:22.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.495+0000 7f6bdcba8700 1 -- 172.21.15.114:0/3384553971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bd80ff400 msgr2=0x7f6bd80ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:22.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.495+0000 7f6bdcba8700 1 --2- 172.21.15.114:0/3384553971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bd80ff400 0x7f6bd80ff7e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8009910 tx=0x7f6bc8009c20 comp rx=0 tx=0).stop 2024-02-16T22:33:22.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.495+0000 7f6bdcba8700 1 -- 172.21.15.114:0/3384553971 shutdown_connections 2024-02-16T22:33:22.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.495+0000 7f6bdcba8700 1 --2- 172.21.15.114:0/3384553971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd80ffdb0 0x7f6bd8103e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:22.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.495+0000 7f6bdcba8700 1 --2- 172.21.15.114:0/3384553971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bd80ff400 0x7f6bd80ff7e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:22.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.495+0000 7f6bdcba8700 1 -- 172.21.15.114:0/3384553971 >> 172.21.15.114:0/3384553971 conn(0x7f6bd8075e30 msgr2=0x7f6bd80fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:22.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.495+0000 7f6bdcba8700 1 -- 172.21.15.114:0/3384553971 shutdown_connections 2024-02-16T22:33:22.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.495+0000 7f6bdcba8700 1 -- 172.21.15.114:0/3384553971 wait complete. 2024-02-16T22:33:22.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.496+0000 7f6bdcba8700 1 Processor -- start 2024-02-16T22:33:22.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.496+0000 7f6bdcba8700 1 -- start start 2024-02-16T22:33:22.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.496+0000 7f6bdcba8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd80ff400 0x7f6bd8197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:22.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.496+0000 7f6bdcba8700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bd80ffdb0 0x7f6bd8197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:22.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.496+0000 7f6bdcba8700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd81980e0 con 0x7f6bd80ffdb0 2024-02-16T22:33:22.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.496+0000 7f6bdcba8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd8198250 con 0x7f6bd80ff400 2024-02-16T22:33:22.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.496+0000 7f6bd5d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bd80ffdb0 0x7f6bd8197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:22.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.496+0000 7f6bd5d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bd80ffdb0 0x7f6bd8197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60272/0 (socket says 172.21.15.114:60272) 2024-02-16T22:33:22.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.496+0000 7f6bd5d9b700 1 -- 172.21.15.114:0/880155441 learned_addr learned my addr 172.21.15.114:0/880155441 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:22.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.496+0000 7f6bd659c700 1 --2- 172.21.15.114:0/880155441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd80ff400 0x7f6bd8197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:22.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.496+0000 7f6bd5d9b700 1 -- 172.21.15.114:0/880155441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd80ff400 msgr2=0x7f6bd8197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:22.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.496+0000 7f6bd5d9b700 1 --2- 172.21.15.114:0/880155441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd80ff400 0x7f6bd8197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:22.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.496+0000 7f6bd5d9b700 1 -- 172.21.15.114:0/880155441 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc80095f0 con 0x7f6bd80ffdb0 2024-02-16T22:33:22.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.496+0000 7f6bd659c700 1 --2- 172.21.15.114:0/880155441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd80ff400 0x7f6bd8197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:33:22.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.496+0000 7f6bd5d9b700 1 --2- 172.21.15.114:0/880155441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bd80ffdb0 0x7f6bd8197ac0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc00d8d0 tx=0x7f6bcc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:22.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.497+0000 7f6bc77fe700 1 -- 172.21.15.114:0/880155441 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bcc0041f0 con 0x7f6bd80ffdb0 2024-02-16T22:33:22.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.497+0000 7f6bc77fe700 1 -- 172.21.15.114:0/880155441 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6bcc007620 con 0x7f6bd80ffdb0 2024-02-16T22:33:22.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.497+0000 7f6bdcba8700 1 -- 172.21.15.114:0/880155441 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd81a05f0 con 0x7f6bd80ffdb0 2024-02-16T22:33:22.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.497+0000 7f6bc77fe700 1 -- 172.21.15.114:0/880155441 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bcc003c50 con 0x7f6bd80ffdb0 2024-02-16T22:33:22.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.497+0000 7f6bdcba8700 1 -- 172.21.15.114:0/880155441 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd81a0ae0 con 0x7f6bd80ffdb0 2024-02-16T22:33:22.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.498+0000 7f6bdcba8700 1 -- 172.21.15.114:0/880155441 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bd804ea90 con 0x7f6bd80ffdb0 2024-02-16T22:33:22.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.498+0000 7f6bc77fe700 1 -- 172.21.15.114:0/880155441 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f6bcc00b750 con 0x7f6bd80ffdb0 2024-02-16T22:33:22.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.498+0000 7f6bc77fe700 1 --2- 172.21.15.114:0/880155441 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6bc006d8a0 0x7f6bc006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:22.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.499+0000 7f6bc77fe700 1 -- 172.21.15.114:0/880155441 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f6bcc08d1a0 con 0x7f6bd80ffdb0 2024-02-16T22:33:22.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.499+0000 7f6bd659c700 1 --2- 172.21.15.114:0/880155441 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6bc006d8a0 0x7f6bc006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:22.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.499+0000 7f6bd659c700 1 --2- 172.21.15.114:0/880155441 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6bc006d8a0 0x7f6bc006fd60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8005b40 tx=0x7f6bc801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:22.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.503+0000 7f6bc77fe700 1 -- 172.21.15.114:0/880155441 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6bcc0568a0 con 0x7f6bd80ffdb0 2024-02-16T22:33:22.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.704+0000 7f6bdcba8700 1 -- 172.21.15.114:0/880155441 --> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6bd80fd990 con 0x7f6bc006d8a0 2024-02-16T22:33:22.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.717+0000 7f6bc77fe700 1 -- 172.21.15.114:0/880155441 <== mgr.34104 v2:172.21.15.114:6800/1665442292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f6bd80fd990 con 0x7f6bc006d8a0 2024-02-16T22:33:22.718 INFO:teuthology.orchestra.run.smithi114.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T22:33:22.718 INFO:teuthology.orchestra.run.smithi114.stdout:alertmanager.smithi114 smithi114 *:9093,9094 running (2m) 112s ago 20m 15.0M - 0.25.0 c8568f914cd2 a46f553e8ce0 2024-02-16T22:33:22.718 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi114 smithi114 running (3m) 112s ago 21m 7826k - 19.0.0-1450-gf78a58c0 f081f847070a 1f621c33f6d4 2024-02-16T22:33:22.718 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-exporter.smithi123 smithi123 running (3m) 2m ago 19m 7482k - 19.0.0-1450-gf78a58c0 f081f847070a 2f0b3c7f313b 2024-02-16T22:33:22.718 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi114 smithi114 running (8m) 112s ago 21m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 9b8ae9efe83b 2024-02-16T22:33:22.718 INFO:teuthology.orchestra.run.smithi114.stdout:crash.smithi123 smithi123 running (8m) 2m ago 19m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 6d97263a0ffa 2024-02-16T22:33:22.718 INFO:teuthology.orchestra.run.smithi114.stdout:grafana.smithi114 smithi114 *:3000 running (117s) 112s ago 20m 57.4M - 9.4.12 a0ea5e3c46d9 32e00d2cbcbb 2024-02-16T22:33:22.718 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.fdxpey smithi114 running (4m) 112s ago 15m 14.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2984ae8a955a 2024-02-16T22:33:22.718 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi114.zzryke smithi114 running (4m) 112s ago 15m 84.6M - 19.0.0-1450-gf78a58c0 f081f847070a 63fa2af7290e 2024-02-16T22:33:22.718 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.gopcfa smithi123 running (3m) 2m ago 15m 14.9M - 19.0.0-1450-gf78a58c0 f081f847070a ec6612a62e2a 2024-02-16T22:33:22.718 INFO:teuthology.orchestra.run.smithi114.stdout:mds.cephfs.smithi123.vhlqbe smithi123 running (3m) 2m ago 15m 15.9M - 19.0.0-1450-gf78a58c0 f081f847070a be790c6287e7 2024-02-16T22:33:22.719 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi114.vcullz smithi114 *:8443,9283,8765 running (11m) 112s ago 22m 512M - 19.0.0-1450-gf78a58c0 f081f847070a 14ebeb7888c0 2024-02-16T22:33:22.719 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.smithi123.inzgjd smithi123 *:8443,9283,8765 running (10m) 2m ago 19m 451M - 19.0.0-1450-gf78a58c0 f081f847070a f10aab9faf9e 2024-02-16T22:33:22.719 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi114 smithi114 running (9m) 112s ago 22m 58.4M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 58ac06372523 2024-02-16T22:33:22.719 INFO:teuthology.orchestra.run.smithi114.stdout:mon.smithi123 smithi123 running (9m) 2m ago 19m 43.0M 2048M 19.0.0-1450-gf78a58c0 f081f847070a f76485e0ea24 2024-02-16T22:33:22.719 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi114 smithi114 *:9100 running (3m) 112s ago 20m 17.1M - 1.5.0 0da6a335fe13 ee1aa5ff6491 2024-02-16T22:33:22.719 INFO:teuthology.orchestra.run.smithi114.stdout:node-exporter.smithi123 smithi123 *:9100 running (2m) 2m ago 19m 3657k - 1.5.0 0da6a335fe13 a5e2e31d4643 2024-02-16T22:33:22.719 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 smithi114 running (8m) 112s ago 18m 100M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 1978fdad94a5 2024-02-16T22:33:22.719 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 smithi114 running (7m) 112s ago 18m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 3f90e792898f 2024-02-16T22:33:22.719 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 smithi114 running (6m) 112s ago 17m 91.7M 1216M 19.0.0-1450-gf78a58c0 f081f847070a c663dfa0518b 2024-02-16T22:33:22.719 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 smithi123 running (6m) 2m ago 17m 113M 2240M 19.0.0-1450-gf78a58c0 f081f847070a b921448ba169 2024-02-16T22:33:22.719 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 smithi123 running (5m) 2m ago 17m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 54c2926d56b1 2024-02-16T22:33:22.719 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 smithi123 running (5m) 2m ago 16m 93.6M 2240M 19.0.0-1450-gf78a58c0 f081f847070a dd99bf034749 2024-02-16T22:33:22.719 INFO:teuthology.orchestra.run.smithi114.stdout:prometheus.smithi114 smithi114 *:9095 running (2m) 112s ago 20m 50.1M - 2.43.0 a07b618ecd1d a6b9ebc86282 2024-02-16T22:33:22.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.719+0000 7f6bdcba8700 1 -- 172.21.15.114:0/880155441 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6bc006d8a0 msgr2=0x7f6bc006fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:22.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.719+0000 7f6bdcba8700 1 --2- 172.21.15.114:0/880155441 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6bc006d8a0 0x7f6bc006fd60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8005b40 tx=0x7f6bc801a040 comp rx=0 tx=0).stop 2024-02-16T22:33:22.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.719+0000 7f6bdcba8700 1 -- 172.21.15.114:0/880155441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bd80ffdb0 msgr2=0x7f6bd8197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:22.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.719+0000 7f6bdcba8700 1 --2- 172.21.15.114:0/880155441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bd80ffdb0 0x7f6bd8197ac0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc00d8d0 tx=0x7f6bcc00dc90 comp rx=0 tx=0).stop 2024-02-16T22:33:22.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.719+0000 7f6bdcba8700 1 -- 172.21.15.114:0/880155441 shutdown_connections 2024-02-16T22:33:22.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.719+0000 7f6bdcba8700 1 --2- 172.21.15.114:0/880155441 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f6bc006d8a0 0x7f6bc006fd60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:22.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.719+0000 7f6bdcba8700 1 --2- 172.21.15.114:0/880155441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd80ff400 0x7f6bd8197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:22.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.719+0000 7f6bdcba8700 1 --2- 172.21.15.114:0/880155441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bd80ffdb0 0x7f6bd8197ac0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:22.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.719+0000 7f6bdcba8700 1 -- 172.21.15.114:0/880155441 >> 172.21.15.114:0/880155441 conn(0x7f6bd8075e30 msgr2=0x7f6bd80fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:22.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.719+0000 7f6bdcba8700 1 -- 172.21.15.114:0/880155441 shutdown_connections 2024-02-16T22:33:22.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:22.719+0000 7f6bdcba8700 1 -- 172.21.15.114:0/880155441 wait complete. 2024-02-16T22:33:23.329 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph versions' 2024-02-16T22:33:23.539 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:33:23.893 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:23 smithi114 ceph-mon[94452]: pgmap v325: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:24.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:23 smithi123 ceph-mon[82331]: pgmap v325: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:24.897 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:24 smithi114 ceph-mon[94452]: from='client.34606 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:33:25.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:24 smithi123 ceph-mon[82331]: from='client.34606 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T22:33:25.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.140+0000 7ff341893700 1 -- 172.21.15.114:0/3057711902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff33c100620 msgr2=0x7ff33c100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:25.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.140+0000 7ff341893700 1 --2- 172.21.15.114:0/3057711902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff33c100620 0x7ff33c100a00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff32c009910 tx=0x7ff32c009c20 comp rx=0 tx=0).stop 2024-02-16T22:33:25.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.140+0000 7ff341893700 1 -- 172.21.15.114:0/3057711902 shutdown_connections 2024-02-16T22:33:25.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.140+0000 7ff341893700 1 --2- 172.21.15.114:0/3057711902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff33c100fd0 0x7ff33c1046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:25.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.140+0000 7ff341893700 1 --2- 172.21.15.114:0/3057711902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff33c100620 0x7ff33c100a00 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:25.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.140+0000 7ff341893700 1 -- 172.21.15.114:0/3057711902 >> 172.21.15.114:0/3057711902 conn(0x7ff33c074d80 msgr2=0x7ff33c075190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:25.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.140+0000 7ff341893700 1 -- 172.21.15.114:0/3057711902 shutdown_connections 2024-02-16T22:33:25.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.140+0000 7ff341893700 1 -- 172.21.15.114:0/3057711902 wait complete. 2024-02-16T22:33:25.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.140+0000 7ff341893700 1 Processor -- start 2024-02-16T22:33:25.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.140+0000 7ff341893700 1 -- start start 2024-02-16T22:33:25.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.141+0000 7ff341893700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff33c100620 0x7ff33c101780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:25.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.141+0000 7ff341893700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff33c100fd0 0x7ff33c101cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:25.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.141+0000 7ff341893700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c102200 con 0x7ff33c100620 2024-02-16T22:33:25.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.141+0000 7ff341893700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c102340 con 0x7ff33c100fd0 2024-02-16T22:33:25.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.141+0000 7ff33affd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff33c100620 0x7ff33c101780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:25.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.141+0000 7ff33affd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff33c100620 0x7ff33c101780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60284/0 (socket says 172.21.15.114:60284) 2024-02-16T22:33:25.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.141+0000 7ff33affd700 1 -- 172.21.15.114:0/4231423033 learned_addr learned my addr 172.21.15.114:0/4231423033 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:25.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.141+0000 7ff33affd700 1 -- 172.21.15.114:0/4231423033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff33c100fd0 msgr2=0x7ff33c101cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:25.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.141+0000 7ff33affd700 1 --2- 172.21.15.114:0/4231423033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff33c100fd0 0x7ff33c101cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:25.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.141+0000 7ff33affd700 1 -- 172.21.15.114:0/4231423033 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32c0095f0 con 0x7ff33c100620 2024-02-16T22:33:25.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.141+0000 7ff33affd700 1 --2- 172.21.15.114:0/4231423033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff33c100620 0x7ff33c101780 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff32c006010 tx=0x7ff32c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:25.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.141+0000 7ff340891700 1 -- 172.21.15.114:0/4231423033 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff32c01d070 con 0x7ff33c100620 2024-02-16T22:33:25.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.141+0000 7ff340891700 1 -- 172.21.15.114:0/4231423033 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff32c00f460 con 0x7ff33c100620 2024-02-16T22:33:25.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.141+0000 7ff341893700 1 -- 172.21.15.114:0/4231423033 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff33c1025c0 con 0x7ff33c100620 2024-02-16T22:33:25.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.141+0000 7ff340891700 1 -- 172.21.15.114:0/4231423033 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff32c017630 con 0x7ff33c100620 2024-02-16T22:33:25.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.141+0000 7ff341893700 1 -- 172.21.15.114:0/4231423033 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33c1a2c00 con 0x7ff33c100620 2024-02-16T22:33:25.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.142+0000 7ff341893700 1 -- 172.21.15.114:0/4231423033 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff33c109060 con 0x7ff33c100620 2024-02-16T22:33:25.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.143+0000 7ff340891700 1 -- 172.21.15.114:0/4231423033 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7ff32c00f5d0 con 0x7ff33c100620 2024-02-16T22:33:25.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.143+0000 7ff340891700 1 --2- 172.21.15.114:0/4231423033 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff32406d8a0 0x7ff32406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:25.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.144+0000 7ff33a7fc700 1 --2- 172.21.15.114:0/4231423033 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff32406d8a0 0x7ff32406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:25.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.144+0000 7ff340891700 1 -- 172.21.15.114:0/4231423033 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7ff32c08eaa0 con 0x7ff33c100620 2024-02-16T22:33:25.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.144+0000 7ff33a7fc700 1 --2- 172.21.15.114:0/4231423033 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff32406d8a0 0x7ff32406fd60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff330005950 tx=0x7ff3300058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:25.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.148+0000 7ff340891700 1 -- 172.21.15.114:0/4231423033 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff32c0581a0 con 0x7ff33c100620 2024-02-16T22:33:25.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.422+0000 7ff341893700 1 -- 172.21.15.114:0/4231423033 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff33c02cc70 con 0x7ff33c100620 2024-02-16T22:33:25.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.423+0000 7ff340891700 1 -- 172.21.15.114:0/4231423033 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7ff32c05b860 con 0x7ff33c100620 2024-02-16T22:33:25.425 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-02-16T22:33:25.425 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": { 2024-02-16T22:33:25.425 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:33:25.425 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:33:25.425 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": { 2024-02-16T22:33:25.425 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T22:33:25.425 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:33:25.425 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": { 2024-02-16T22:33:25.425 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T22:33:25.425 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:33:25.426 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-02-16T22:33:25.426 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T22:33:25.426 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-02-16T22:33:25.426 INFO:teuthology.orchestra.run.smithi114.stdout: "overall": { 2024-02-16T22:33:25.426 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-16T22:33:25.426 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-02-16T22:33:25.426 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-02-16T22:33:25.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.425+0000 7ff341893700 1 -- 172.21.15.114:0/4231423033 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff32406d8a0 msgr2=0x7ff32406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:25.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.425+0000 7ff341893700 1 --2- 172.21.15.114:0/4231423033 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff32406d8a0 0x7ff32406fd60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff330005950 tx=0x7ff3300058e0 comp rx=0 tx=0).stop 2024-02-16T22:33:25.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.425+0000 7ff341893700 1 -- 172.21.15.114:0/4231423033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff33c100620 msgr2=0x7ff33c101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:25.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.425+0000 7ff341893700 1 --2- 172.21.15.114:0/4231423033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff33c100620 0x7ff33c101780 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff32c006010 tx=0x7ff32c00be20 comp rx=0 tx=0).stop 2024-02-16T22:33:25.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.425+0000 7ff341893700 1 -- 172.21.15.114:0/4231423033 shutdown_connections 2024-02-16T22:33:25.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.425+0000 7ff341893700 1 --2- 172.21.15.114:0/4231423033 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff32406d8a0 0x7ff32406fd60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:25.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.425+0000 7ff341893700 1 --2- 172.21.15.114:0/4231423033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff33c100620 0x7ff33c101780 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:25.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.425+0000 7ff341893700 1 --2- 172.21.15.114:0/4231423033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff33c100fd0 0x7ff33c101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:25.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.425+0000 7ff341893700 1 -- 172.21.15.114:0/4231423033 >> 172.21.15.114:0/4231423033 conn(0x7ff33c074d80 msgr2=0x7ff33c103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:25.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.425+0000 7ff341893700 1 -- 172.21.15.114:0/4231423033 shutdown_connections 2024-02-16T22:33:25.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:25.425+0000 7ff341893700 1 -- 172.21.15.114:0/4231423033 wait complete. 2024-02-16T22:33:25.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:25 smithi114 ceph-mon[94452]: pgmap v326: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:25.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:25 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/4231423033' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:33:26.027 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-02-16T22:33:26.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:25 smithi123 ceph-mon[82331]: pgmap v326: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:26.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:25 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/4231423033' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:33:26.238 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:33:27.859 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:27 smithi114 ceph-mon[94452]: pgmap v327: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:27.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.857+0000 7f5f71ebc700 1 -- 172.21.15.114:0/3991468269 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f6c100dd0 msgr2=0x7f5f6c1011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:27.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.857+0000 7f5f71ebc700 1 --2- 172.21.15.114:0/3991468269 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f6c100dd0 0x7f5f6c1011b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5f60009910 tx=0x7f5f60009c20 comp rx=0 tx=0).stop 2024-02-16T22:33:27.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.857+0000 7f5f71ebc700 1 -- 172.21.15.114:0/3991468269 shutdown_connections 2024-02-16T22:33:27.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.857+0000 7f5f71ebc700 1 --2- 172.21.15.114:0/3991468269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f6c101780 0x7f5f6c1057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:27.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.857+0000 7f5f71ebc700 1 --2- 172.21.15.114:0/3991468269 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f6c100dd0 0x7f5f6c1011b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:27.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.857+0000 7f5f71ebc700 1 -- 172.21.15.114:0/3991468269 >> 172.21.15.114:0/3991468269 conn(0x7f5f6c0fc640 msgr2=0x7f5f6c0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:27.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.857+0000 7f5f71ebc700 1 -- 172.21.15.114:0/3991468269 shutdown_connections 2024-02-16T22:33:27.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.857+0000 7f5f71ebc700 1 -- 172.21.15.114:0/3991468269 wait complete. 2024-02-16T22:33:27.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.857+0000 7f5f71ebc700 1 Processor -- start 2024-02-16T22:33:27.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.858+0000 7f5f71ebc700 1 -- start start 2024-02-16T22:33:27.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.858+0000 7f5f71ebc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f6c100dd0 0x7f5f6c1934d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:27.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.858+0000 7f5f71ebc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f6c101780 0x7f5f6c193a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:27.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.858+0000 7f5f71ebc700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c1975d0 con 0x7f5f6c101780 2024-02-16T22:33:27.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.858+0000 7f5f71ebc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c193f50 con 0x7f5f6c100dd0 2024-02-16T22:33:27.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.858+0000 7f5f6affd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f6c101780 0x7f5f6c193a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:27.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.858+0000 7f5f6affd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f6c101780 0x7f5f6c193a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60290/0 (socket says 172.21.15.114:60290) 2024-02-16T22:33:27.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.858+0000 7f5f6affd700 1 -- 172.21.15.114:0/70917442 learned_addr learned my addr 172.21.15.114:0/70917442 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:27.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.858+0000 7f5f6b7fe700 1 --2- 172.21.15.114:0/70917442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f6c100dd0 0x7f5f6c1934d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:27.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.858+0000 7f5f6affd700 1 -- 172.21.15.114:0/70917442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f6c100dd0 msgr2=0x7f5f6c1934d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:27.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.858+0000 7f5f6affd700 1 --2- 172.21.15.114:0/70917442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f6c100dd0 0x7f5f6c1934d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:27.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.859+0000 7f5f6affd700 1 -- 172.21.15.114:0/70917442 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f600095f0 con 0x7f5f6c101780 2024-02-16T22:33:27.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.859+0000 7f5f6affd700 1 --2- 172.21.15.114:0/70917442 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f6c101780 0x7f5f6c193a10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c00c960 tx=0x7f5f5c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:27.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.859+0000 7f5f68ff9700 1 -- 172.21.15.114:0/70917442 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f5c007a70 con 0x7f5f6c101780 2024-02-16T22:33:27.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.859+0000 7f5f71ebc700 1 -- 172.21.15.114:0/70917442 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f6c194230 con 0x7f5f6c101780 2024-02-16T22:33:27.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.859+0000 7f5f68ff9700 1 -- 172.21.15.114:0/70917442 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f5c007bd0 con 0x7f5f6c101780 2024-02-16T22:33:27.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.859+0000 7f5f68ff9700 1 -- 172.21.15.114:0/70917442 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f5c0186c0 con 0x7f5f6c101780 2024-02-16T22:33:27.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.859+0000 7f5f71ebc700 1 -- 172.21.15.114:0/70917442 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f6c1a70a0 con 0x7f5f6c101780 2024-02-16T22:33:27.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.860+0000 7f5f71ebc700 1 -- 172.21.15.114:0/70917442 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f6c104f50 con 0x7f5f6c101780 2024-02-16T22:33:27.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.860+0000 7f5f68ff9700 1 -- 172.21.15.114:0/70917442 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f5f5c00faa0 con 0x7f5f6c101780 2024-02-16T22:33:27.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.861+0000 7f5f68ff9700 1 --2- 172.21.15.114:0/70917442 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f5f5406d8a0 0x7f5f5406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:27.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.861+0000 7f5f68ff9700 1 -- 172.21.15.114:0/70917442 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f5f5c08d610 con 0x7f5f6c101780 2024-02-16T22:33:27.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.861+0000 7f5f6b7fe700 1 --2- 172.21.15.114:0/70917442 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f5f5406d8a0 0x7f5f5406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:27.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.861+0000 7f5f6b7fe700 1 --2- 172.21.15.114:0/70917442 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f5f5406d8a0 0x7f5f5406fd60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000b5c0 tx=0x7f5f6001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:27.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:27.865+0000 7f5f68ff9700 1 -- 172.21.15.114:0/70917442 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5f5c05a840 con 0x7f5f6c101780 2024-02-16T22:33:28.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:27 smithi123 ceph-mon[82331]: pgmap v327: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:28.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:28.140+0000 7f5f71ebc700 1 -- 172.21.15.114:0/70917442 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5f6c02cd10 con 0x7f5f6c101780 2024-02-16T22:33:28.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:28.140+0000 7f5f68ff9700 1 -- 172.21.15.114:0/70917442 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f5f5c05a3d0 con 0x7f5f6c101780 2024-02-16T22:33:28.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:28.142+0000 7f5f71ebc700 1 -- 172.21.15.114:0/70917442 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f5f5406d8a0 msgr2=0x7f5f5406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:28.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:28.142+0000 7f5f71ebc700 1 --2- 172.21.15.114:0/70917442 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f5f5406d8a0 0x7f5f5406fd60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000b5c0 tx=0x7f5f6001a040 comp rx=0 tx=0).stop 2024-02-16T22:33:28.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:28.142+0000 7f5f71ebc700 1 -- 172.21.15.114:0/70917442 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f6c101780 msgr2=0x7f5f6c193a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:28.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:28.142+0000 7f5f71ebc700 1 --2- 172.21.15.114:0/70917442 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f6c101780 0x7f5f6c193a10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c00c960 tx=0x7f5f5c00cd20 comp rx=0 tx=0).stop 2024-02-16T22:33:28.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:28.142+0000 7f5f71ebc700 1 -- 172.21.15.114:0/70917442 shutdown_connections 2024-02-16T22:33:28.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:28.142+0000 7f5f71ebc700 1 --2- 172.21.15.114:0/70917442 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f5f5406d8a0 0x7f5f5406fd60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:28.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:28.142+0000 7f5f71ebc700 1 --2- 172.21.15.114:0/70917442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f6c100dd0 0x7f5f6c1934d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:28.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:28.142+0000 7f5f71ebc700 1 --2- 172.21.15.114:0/70917442 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f6c101780 0x7f5f6c193a10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:28.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:28.142+0000 7f5f71ebc700 1 -- 172.21.15.114:0/70917442 >> 172.21.15.114:0/70917442 conn(0x7f5f6c0fc640 msgr2=0x7f5f6c0fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:28.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:28.142+0000 7f5f71ebc700 1 -- 172.21.15.114:0/70917442 shutdown_connections 2024-02-16T22:33:28.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:28.142+0000 7f5f71ebc700 1 -- 172.21.15.114:0/70917442 wait complete. 2024-02-16T22:33:28.154 INFO:teuthology.orchestra.run.smithi114.stdout:true 2024-02-16T22:33:28.773 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:28 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/70917442' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:33:28.775 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-02-16T22:33:28.981 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:33:29.051 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:28 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/70917442' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:33:29.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:29 smithi114 ceph-mon[94452]: pgmap v328: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:30.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:29 smithi123 ceph-mon[82331]: pgmap v328: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:30.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.544+0000 7f0112912700 1 -- 172.21.15.114:0/2565223637 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f010c1026c0 msgr2=0x7f010c10abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:30.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.544+0000 7f0112912700 1 --2- 172.21.15.114:0/2565223637 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f010c1026c0 0x7f010c10abb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f00fc009910 tx=0x7f00fc009c20 comp rx=0 tx=0).stop 2024-02-16T22:33:30.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.544+0000 7f0112912700 1 -- 172.21.15.114:0/2565223637 shutdown_connections 2024-02-16T22:33:30.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.544+0000 7f0112912700 1 --2- 172.21.15.114:0/2565223637 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f010c1026c0 0x7f010c10abb0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:30.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.544+0000 7f0112912700 1 --2- 172.21.15.114:0/2565223637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f010c101da0 0x7f010c102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:30.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.544+0000 7f0112912700 1 -- 172.21.15.114:0/2565223637 >> 172.21.15.114:0/2565223637 conn(0x7f010c0fb570 msgr2=0x7f010c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:30.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.544+0000 7f0112912700 1 -- 172.21.15.114:0/2565223637 shutdown_connections 2024-02-16T22:33:30.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.544+0000 7f0112912700 1 -- 172.21.15.114:0/2565223637 wait complete. 2024-02-16T22:33:30.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.545+0000 7f0112912700 1 Processor -- start 2024-02-16T22:33:30.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.545+0000 7f0112912700 1 -- start start 2024-02-16T22:33:30.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.545+0000 7f0112912700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f010c101da0 0x7f010c19c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:30.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.545+0000 7f0112912700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f010c1026c0 0x7f010c19ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:30.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.545+0000 7f0112912700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c19d110 con 0x7f010c1026c0 2024-02-16T22:33:30.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.545+0000 7f0112912700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c1965c0 con 0x7f010c101da0 2024-02-16T22:33:30.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.545+0000 7f010b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f010c1026c0 0x7f010c19ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:30.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.545+0000 7f010b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f010c1026c0 0x7f010c19ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60306/0 (socket says 172.21.15.114:60306) 2024-02-16T22:33:30.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.545+0000 7f010b7fe700 1 -- 172.21.15.114:0/486382104 learned_addr learned my addr 172.21.15.114:0/486382104 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:30.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.545+0000 7f010bfff700 1 --2- 172.21.15.114:0/486382104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f010c101da0 0x7f010c19c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:30.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.546+0000 7f010b7fe700 1 -- 172.21.15.114:0/486382104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f010c101da0 msgr2=0x7f010c19c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:30.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.546+0000 7f010b7fe700 1 --2- 172.21.15.114:0/486382104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f010c101da0 0x7f010c19c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:30.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.546+0000 7f010b7fe700 1 -- 172.21.15.114:0/486382104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00fc0095f0 con 0x7f010c1026c0 2024-02-16T22:33:30.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.546+0000 7f010b7fe700 1 --2- 172.21.15.114:0/486382104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f010c1026c0 0x7f010c19ca80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0092e0 tx=0x7f00fc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:30.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.546+0000 7f01097fa700 1 -- 172.21.15.114:0/486382104 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00fc01d070 con 0x7f010c1026c0 2024-02-16T22:33:30.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.546+0000 7f01097fa700 1 -- 172.21.15.114:0/486382104 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f00fc00f460 con 0x7f010c1026c0 2024-02-16T22:33:30.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.546+0000 7f0112912700 1 -- 172.21.15.114:0/486382104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f010c196840 con 0x7f010c1026c0 2024-02-16T22:33:30.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.546+0000 7f01097fa700 1 -- 172.21.15.114:0/486382104 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00fc017630 con 0x7f010c1026c0 2024-02-16T22:33:30.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.546+0000 7f0112912700 1 -- 172.21.15.114:0/486382104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010c196d30 con 0x7f010c1026c0 2024-02-16T22:33:30.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.547+0000 7f0112912700 1 -- 172.21.15.114:0/486382104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f010c1082b0 con 0x7f010c1026c0 2024-02-16T22:33:30.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.547+0000 7f01097fa700 1 -- 172.21.15.114:0/486382104 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f00fc00f5d0 con 0x7f010c1026c0 2024-02-16T22:33:30.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.548+0000 7f01097fa700 1 --2- 172.21.15.114:0/486382104 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f00f806d8a0 0x7f00f806fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:30.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.548+0000 7f01097fa700 1 -- 172.21.15.114:0/486382104 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f00fc08eb30 con 0x7f010c1026c0 2024-02-16T22:33:30.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.548+0000 7f010bfff700 1 --2- 172.21.15.114:0/486382104 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f00f806d8a0 0x7f00f806fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:30.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.548+0000 7f010bfff700 1 --2- 172.21.15.114:0/486382104 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f00f806d8a0 0x7f00f806fd60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f00f4005fd0 tx=0x7f00f400a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:30.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.552+0000 7f01097fa700 1 -- 172.21.15.114:0/486382104 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f00fc05bd60 con 0x7f010c1026c0 2024-02-16T22:33:30.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.825+0000 7f0112912700 1 -- 172.21.15.114:0/486382104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f010c04ea90 con 0x7f010c1026c0 2024-02-16T22:33:30.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.825+0000 7f01097fa700 1 -- 172.21.15.114:0/486382104 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f00fc05b8f0 con 0x7f010c1026c0 2024-02-16T22:33:30.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.828+0000 7f0112912700 1 -- 172.21.15.114:0/486382104 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f00f806d8a0 msgr2=0x7f00f806fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:30.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.828+0000 7f0112912700 1 --2- 172.21.15.114:0/486382104 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f00f806d8a0 0x7f00f806fd60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f00f4005fd0 tx=0x7f00f400a560 comp rx=0 tx=0).stop 2024-02-16T22:33:30.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.828+0000 7f0112912700 1 -- 172.21.15.114:0/486382104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f010c1026c0 msgr2=0x7f010c19ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:30.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.828+0000 7f0112912700 1 --2- 172.21.15.114:0/486382104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f010c1026c0 0x7f010c19ca80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0092e0 tx=0x7f00fc00be20 comp rx=0 tx=0).stop 2024-02-16T22:33:30.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.828+0000 7f0112912700 1 -- 172.21.15.114:0/486382104 shutdown_connections 2024-02-16T22:33:30.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.828+0000 7f0112912700 1 --2- 172.21.15.114:0/486382104 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f00f806d8a0 0x7f00f806fd60 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:30.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.828+0000 7f0112912700 1 --2- 172.21.15.114:0/486382104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f010c101da0 0x7f010c19c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:30.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.828+0000 7f0112912700 1 --2- 172.21.15.114:0/486382104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f010c1026c0 0x7f010c19ca80 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:30.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.828+0000 7f0112912700 1 -- 172.21.15.114:0/486382104 >> 172.21.15.114:0/486382104 conn(0x7f010c0fb570 msgr2=0x7f010c1053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:30.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.828+0000 7f0112912700 1 -- 172.21.15.114:0/486382104 shutdown_connections 2024-02-16T22:33:30.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:30.828+0000 7f0112912700 1 -- 172.21.15.114:0/486382104 wait complete. 2024-02-16T22:33:30.840 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)" 2024-02-16T22:33:31.441 DEBUG:teuthology.parallel:result is None 2024-02-16T22:33:31.442 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T22:33:31.454 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi114.front.sepia.ceph.com 2024-02-16T22:33:31.455 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-16T22:33:31.664 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/mon.smithi114/config 2024-02-16T22:33:31.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:31 smithi114 ceph-mon[94452]: pgmap v329: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:31.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:31 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:33:31.702 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:31 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/486382104' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:33:32.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:31 smithi123 ceph-mon[82331]: pgmap v329: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:32.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:31 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:33:32.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:31 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/486382104' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T22:33:33.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.445+0000 7f2ec83b5700 1 -- 172.21.15.114:0/1834680041 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ec00ff530 msgr2=0x7f2ec00ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:33.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.445+0000 7f2ec83b5700 1 --2- 172.21.15.114:0/1834680041 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ec00ff530 0x7f2ec00ff910 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2eb000b0a0 tx=0x7f2eb000b3b0 comp rx=0 tx=0).stop 2024-02-16T22:33:33.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.445+0000 7f2ec83b5700 1 -- 172.21.15.114:0/1834680041 shutdown_connections 2024-02-16T22:33:33.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.445+0000 7f2ec83b5700 1 --2- 172.21.15.114:0/1834680041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ec00ffee0 0x7f2ec0105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:33.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.445+0000 7f2ec83b5700 1 --2- 172.21.15.114:0/1834680041 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ec00ff530 0x7f2ec00ff910 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:33.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.445+0000 7f2ec83b5700 1 -- 172.21.15.114:0/1834680041 >> 172.21.15.114:0/1834680041 conn(0x7f2ec0074df0 msgr2=0x7f2ec0075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:33.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.445+0000 7f2ec83b5700 1 -- 172.21.15.114:0/1834680041 shutdown_connections 2024-02-16T22:33:33.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.445+0000 7f2ec83b5700 1 -- 172.21.15.114:0/1834680041 wait complete. 2024-02-16T22:33:33.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.445+0000 7f2ec83b5700 1 Processor -- start 2024-02-16T22:33:33.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.445+0000 7f2ec83b5700 1 -- start start 2024-02-16T22:33:33.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.445+0000 7f2ec83b5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ec00ff530 0x7f2ec01004b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:33.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.446+0000 7f2ec83b5700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ec00ffee0 0x7f2ec01009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:33.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.446+0000 7f2ec83b5700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec0100f30 con 0x7f2ec00ffee0 2024-02-16T22:33:33.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.446+0000 7f2ec83b5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec0101070 con 0x7f2ec00ff530 2024-02-16T22:33:33.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.446+0000 7f2ec5950700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ec00ffee0 0x7f2ec01009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:33.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.446+0000 7f2ec5950700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ec00ffee0 0x7f2ec01009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47872/0 (socket says 172.21.15.114:47872) 2024-02-16T22:33:33.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.446+0000 7f2ec5950700 1 -- 172.21.15.114:0/1464009587 learned_addr learned my addr 172.21.15.114:0/1464009587 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:33.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.446+0000 7f2ec6151700 1 --2- 172.21.15.114:0/1464009587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ec00ff530 0x7f2ec01004b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:33.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.446+0000 7f2ec5950700 1 -- 172.21.15.114:0/1464009587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ec00ff530 msgr2=0x7f2ec01004b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:33.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.446+0000 7f2ec5950700 1 --2- 172.21.15.114:0/1464009587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ec00ff530 0x7f2ec01004b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:33.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.446+0000 7f2ec5950700 1 -- 172.21.15.114:0/1464009587 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb0009d00 con 0x7f2ec00ffee0 2024-02-16T22:33:33.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.446+0000 7f2ec5950700 1 --2- 172.21.15.114:0/1464009587 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ec00ffee0 0x7f2ec01009f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400c960 tx=0x7f2eb400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:33.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.446+0000 7f2ebf7fe700 1 -- 172.21.15.114:0/1464009587 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2eb4007a70 con 0x7f2ec00ffee0 2024-02-16T22:33:33.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.446+0000 7f2ebf7fe700 1 -- 172.21.15.114:0/1464009587 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2eb4007bd0 con 0x7f2ec00ffee0 2024-02-16T22:33:33.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.446+0000 7f2ebf7fe700 1 -- 172.21.15.114:0/1464009587 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2eb40186c0 con 0x7f2ec00ffee0 2024-02-16T22:33:33.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.446+0000 7f2ec83b5700 1 -- 172.21.15.114:0/1464009587 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ec01a2990 con 0x7f2ec00ffee0 2024-02-16T22:33:33.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.446+0000 7f2ec83b5700 1 -- 172.21.15.114:0/1464009587 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ec01a2cf0 con 0x7f2ec00ffee0 2024-02-16T22:33:33.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.447+0000 7f2ec83b5700 1 -- 172.21.15.114:0/1464009587 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ec01091e0 con 0x7f2ec00ffee0 2024-02-16T22:33:33.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.448+0000 7f2ebf7fe700 1 -- 172.21.15.114:0/1464009587 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f2eb400faa0 con 0x7f2ec00ffee0 2024-02-16T22:33:33.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.448+0000 7f2ebf7fe700 1 --2- 172.21.15.114:0/1464009587 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2ea806d8a0 0x7f2ea806fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:33.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.448+0000 7f2ebf7fe700 1 -- 172.21.15.114:0/1464009587 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f2eb408d580 con 0x7f2ec00ffee0 2024-02-16T22:33:33.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.448+0000 7f2ec6151700 1 --2- 172.21.15.114:0/1464009587 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2ea806d8a0 0x7f2ea806fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:33.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.449+0000 7f2ec6151700 1 --2- 172.21.15.114:0/1464009587 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2ea806d8a0 0x7f2ea806fd60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2eb000b830 tx=0x7f2eb00097c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:33.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.453+0000 7f2ebf7fe700 1 -- 172.21.15.114:0/1464009587 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2eb405a830 con 0x7f2ec00ffee0 2024-02-16T22:33:33.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.693+0000 7f2ec83b5700 1 -- 172.21.15.114:0/1464009587 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2ec004ea90 con 0x7f2ec00ffee0 2024-02-16T22:33:33.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.694+0000 7f2ebf7fe700 1 -- 172.21.15.114:0/1464009587 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f2eb405a3c0 con 0x7f2ec00ffee0 2024-02-16T22:33:33.696 INFO:teuthology.orchestra.run.smithi114.stdout:e37 2024-02-16T22:33:33.696 INFO:teuthology.orchestra.run.smithi114.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T22:33:33.696 INFO:teuthology.orchestra.run.smithi114.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T22:33:33.696 INFO:teuthology.orchestra.run.smithi114.stdout:legacy client fscid: 1 2024-02-16T22:33:33.696 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:33.696 INFO:teuthology.orchestra.run.smithi114.stdout:Filesystem 'cephfs' (1) 2024-02-16T22:33:33.696 INFO:teuthology.orchestra.run.smithi114.stdout:fs_name cephfs 2024-02-16T22:33:33.696 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 37 2024-02-16T22:33:33.696 INFO:teuthology.orchestra.run.smithi114.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T22:33:33.696 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-02-16T22:18:02.108825+0000 2024-02-16T22:33:33.696 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-02-16T22:29:48.901292+0000 2024-02-16T22:33:33.696 INFO:teuthology.orchestra.run.smithi114.stdout:tableserver 0 2024-02-16T22:33:33.697 INFO:teuthology.orchestra.run.smithi114.stdout:root 0 2024-02-16T22:33:33.697 INFO:teuthology.orchestra.run.smithi114.stdout:session_timeout 60 2024-02-16T22:33:33.697 INFO:teuthology.orchestra.run.smithi114.stdout:session_autoclose 300 2024-02-16T22:33:33.697 INFO:teuthology.orchestra.run.smithi114.stdout:max_file_size 1099511627776 2024-02-16T22:33:33.697 INFO:teuthology.orchestra.run.smithi114.stdout:max_xattr_size 65536 2024-02-16T22:33:33.697 INFO:teuthology.orchestra.run.smithi114.stdout:required_client_features {} 2024-02-16T22:33:33.697 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure 0 2024-02-16T22:33:33.697 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure_osd_epoch 82 2024-02-16T22:33:33.697 INFO:teuthology.orchestra.run.smithi114.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-16T22:33:33.697 INFO:teuthology.orchestra.run.smithi114.stdout:max_mds 1 2024-02-16T22:33:33.697 INFO:teuthology.orchestra.run.smithi114.stdout:in 0 2024-02-16T22:33:33.697 INFO:teuthology.orchestra.run.smithi114.stdout:up {0=34404} 2024-02-16T22:33:33.697 INFO:teuthology.orchestra.run.smithi114.stdout:failed 2024-02-16T22:33:33.697 INFO:teuthology.orchestra.run.smithi114.stdout:damaged 2024-02-16T22:33:33.697 INFO:teuthology.orchestra.run.smithi114.stdout:stopped 2024-02-16T22:33:33.698 INFO:teuthology.orchestra.run.smithi114.stdout:data_pools [3] 2024-02-16T22:33:33.698 INFO:teuthology.orchestra.run.smithi114.stdout:metadata_pool 2 2024-02-16T22:33:33.698 INFO:teuthology.orchestra.run.smithi114.stdout:inline_data enabled 2024-02-16T22:33:33.698 INFO:teuthology.orchestra.run.smithi114.stdout:balancer 2024-02-16T22:33:33.698 INFO:teuthology.orchestra.run.smithi114.stdout:bal_rank_mask -1 2024-02-16T22:33:33.698 INFO:teuthology.orchestra.run.smithi114.stdout:standby_count_wanted 1 2024-02-16T22:33:33.698 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.zzryke{0:34404} state up:active seq 23 join_fscid=1 addr [v2:172.21.15.114:6826/2017800151,v1:172.21.15.114:6827/2017800151] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T22:33:33.698 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:33.698 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:33.698 INFO:teuthology.orchestra.run.smithi114.stdout:Standby daemons: 2024-02-16T22:33:33.698 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:33.698 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi114.fdxpey{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.114:6828/3702914193,v1:172.21.15.114:6829/3702914193] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T22:33:33.698 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.vhlqbe{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.123:6824/1425065679,v1:172.21.15.123:6825/1425065679] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T22:33:33.698 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.cephfs.smithi123.gopcfa{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.123:6826/3128771108,v1:172.21.15.123:6827/3128771108] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T22:33:33.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.696+0000 7f2ec83b5700 1 -- 172.21.15.114:0/1464009587 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2ea806d8a0 msgr2=0x7f2ea806fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:33.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.696+0000 7f2ec83b5700 1 --2- 172.21.15.114:0/1464009587 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2ea806d8a0 0x7f2ea806fd60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2eb000b830 tx=0x7f2eb00097c0 comp rx=0 tx=0).stop 2024-02-16T22:33:33.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.696+0000 7f2ec83b5700 1 -- 172.21.15.114:0/1464009587 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ec00ffee0 msgr2=0x7f2ec01009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:33.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.696+0000 7f2ec83b5700 1 --2- 172.21.15.114:0/1464009587 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ec00ffee0 0x7f2ec01009f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400c960 tx=0x7f2eb400cd20 comp rx=0 tx=0).stop 2024-02-16T22:33:33.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.696+0000 7f2ec83b5700 1 -- 172.21.15.114:0/1464009587 shutdown_connections 2024-02-16T22:33:33.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.696+0000 7f2ec83b5700 1 --2- 172.21.15.114:0/1464009587 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2ea806d8a0 0x7f2ea806fd60 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:33.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.696+0000 7f2ec83b5700 1 --2- 172.21.15.114:0/1464009587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ec00ff530 0x7f2ec01004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:33.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.696+0000 7f2ec83b5700 1 --2- 172.21.15.114:0/1464009587 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ec00ffee0 0x7f2ec01009f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:33.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.696+0000 7f2ec83b5700 1 -- 172.21.15.114:0/1464009587 >> 172.21.15.114:0/1464009587 conn(0x7f2ec0074df0 msgr2=0x7f2ec00feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:33.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.696+0000 7f2ec83b5700 1 -- 172.21.15.114:0/1464009587 shutdown_connections 2024-02-16T22:33:33.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:33.696+0000 7f2ec83b5700 1 -- 172.21.15.114:0/1464009587 wait complete. 2024-02-16T22:33:33.700 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 37 2024-02-16T22:33:33.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:33 smithi114 ceph-mon[94452]: pgmap v330: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:34.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:33 smithi123 ceph-mon[82331]: pgmap v330: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:34.313 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-02-16T22:33:34.324 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-16T22:33:34.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.428+0000 7fd1a19cb640 1 -- 172.21.15.114:0/370182176 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd19c0ff1a0 msgr2=0x7fd19c0ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:34.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.428+0000 7fd1a19cb640 1 --2- 172.21.15.114:0/370182176 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd19c0ff1a0 0x7fd19c0ff580 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd18c009880 tx=0x7fd18c02bce0 comp rx=0 tx=0).stop 2024-02-16T22:33:34.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.428+0000 7fd1a19cb640 1 -- 172.21.15.114:0/370182176 shutdown_connections 2024-02-16T22:33:34.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.428+0000 7fd1a19cb640 1 --2- 172.21.15.114:0/370182176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd19c0ffb50 0x7fd19c103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:34.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.428+0000 7fd1a19cb640 1 --2- 172.21.15.114:0/370182176 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd19c0ff1a0 0x7fd19c0ff580 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:34.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.428+0000 7fd1a19cb640 1 -- 172.21.15.114:0/370182176 >> 172.21.15.114:0/370182176 conn(0x7fd19c0faa20 msgr2=0x7fd19c0fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:34.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.428+0000 7fd1a19cb640 1 -- 172.21.15.114:0/370182176 shutdown_connections 2024-02-16T22:33:34.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.428+0000 7fd1a19cb640 1 -- 172.21.15.114:0/370182176 wait complete. 2024-02-16T22:33:34.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.429+0000 7fd1a19cb640 1 Processor -- start 2024-02-16T22:33:34.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.429+0000 7fd1a19cb640 1 -- start start 2024-02-16T22:33:34.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.429+0000 7fd1a19cb640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd19c0ff1a0 0x7fd19c196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:34.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.429+0000 7fd1a19cb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd19c197030 0x7fd19c197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:34.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.429+0000 7fd1a19cb640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c19fb60 con 0x7fd19c197030 2024-02-16T22:33:34.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.429+0000 7fd1a19cb640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c19fca0 con 0x7fd19c0ff1a0 2024-02-16T22:33:34.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.429+0000 7fd193fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd19c197030 0x7fd19c197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:34.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.429+0000 7fd193fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd19c197030 0x7fd19c197490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47888/0 (socket says 172.21.15.114:47888) 2024-02-16T22:33:34.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.429+0000 7fd193fff640 1 -- 172.21.15.114:0/2536631179 learned_addr learned my addr 172.21.15.114:0/2536631179 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:34.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.429+0000 7fd19affd640 1 --2- 172.21.15.114:0/2536631179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd19c0ff1a0 0x7fd19c196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:34.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.429+0000 7fd193fff640 1 -- 172.21.15.114:0/2536631179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd19c0ff1a0 msgr2=0x7fd19c196af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:34.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.429+0000 7fd193fff640 1 --2- 172.21.15.114:0/2536631179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd19c0ff1a0 0x7fd19c196af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:34.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.429+0000 7fd193fff640 1 -- 172.21.15.114:0/2536631179 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18c009560 con 0x7fd19c197030 2024-02-16T22:33:34.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.430+0000 7fd193fff640 1 --2- 172.21.15.114:0/2536631179 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd19c197030 0x7fd19c197490 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd18400d720 tx=0x7fd18400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:34.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.430+0000 7fd1a09c9640 1 -- 172.21.15.114:0/2536631179 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1840099f0 con 0x7fd19c197030 2024-02-16T22:33:34.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.430+0000 7fd1a19cb640 1 -- 172.21.15.114:0/2536631179 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd19c19ff00 con 0x7fd19c197030 2024-02-16T22:33:34.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.430+0000 7fd1a09c9640 1 -- 172.21.15.114:0/2536631179 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd184009b50 con 0x7fd19c197030 2024-02-16T22:33:34.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.430+0000 7fd1a09c9640 1 -- 172.21.15.114:0/2536631179 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd18400f5b0 con 0x7fd19c197030 2024-02-16T22:33:34.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.430+0000 7fd1a19cb640 1 -- 172.21.15.114:0/2536631179 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd19c1a03e0 con 0x7fd19c197030 2024-02-16T22:33:34.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.431+0000 7fd1a19cb640 1 -- 172.21.15.114:0/2536631179 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd19c04e040 con 0x7fd19c197030 2024-02-16T22:33:34.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.431+0000 7fd1a09c9640 1 -- 172.21.15.114:0/2536631179 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fd184009cc0 con 0x7fd19c197030 2024-02-16T22:33:34.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.431+0000 7fd1a09c9640 1 --2- 172.21.15.114:0/2536631179 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd18806d8d0 0x7fd18806fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:34.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.432+0000 7fd1a09c9640 1 -- 172.21.15.114:0/2536631179 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7fd18408d6d0 con 0x7fd19c197030 2024-02-16T22:33:34.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.432+0000 7fd19affd640 1 --2- 172.21.15.114:0/2536631179 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd18806d8d0 0x7fd18806fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:34.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.432+0000 7fd19affd640 1 --2- 172.21.15.114:0/2536631179 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd18806d8d0 0x7fd18806fd90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd18c006010 tx=0x7fd18c005dd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:34.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.436+0000 7fd1a09c9640 1 -- 172.21.15.114:0/2536631179 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd184056dd0 con 0x7fd19c197030 2024-02-16T22:33:34.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.612+0000 7fd1a19cb640 1 -- 172.21.15.114:0/2536631179 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd19c1074a0 con 0x7fd19c197030 2024-02-16T22:33:34.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.613+0000 7fd1a09c9640 1 -- 172.21.15.114:0/2536631179 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+17080 (secure 0 0 0) 0x7fd18405a490 con 0x7fd19c197030 2024-02-16T22:33:34.615 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:34.616 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":82,"fsid":"1dcbaabe-cd18-11ee-95ba-87774f69a715","created":"2024-02-16T22:11:02.186164+0000","modified":"2024-02-16T22:29:37.876619+0000","last_up_change":"2024-02-16T22:28:16.339897+0000","last_in_change":"2024-02-16T22:16:28.224489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-16T22:15:31.749346+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-16T22:18:01.042512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-16T22:18:01.336617+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3a72ccf1-3f93-4c06-99c3-092a7f9a732d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":48,"up_from":52,"up_thru":77,"down_at":49,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":221411803},{"type":"v1","addr":"172.21.15.114:6803","nonce":221411803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":221411803},{"type":"v1","addr":"172.21.15.114:6805","nonce":221411803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":221411803},{"type":"v1","addr":"172.21.15.114:6809","nonce":221411803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":221411803},{"type":"v1","addr":"172.21.15.114:6807","nonce":221411803}]},"public_addr":"172.21.15.114:6803/221411803","cluster_addr":"172.21.15.114:6805/221411803","heartbeat_back_addr":"172.21.15.114:6809/221411803","heartbeat_front_addr":"172.21.15.114:6807/221411803","state":["exists","up"]},{"osd":1,"uuid":"1e8829dd-cac2-462f-aca1-86bcfb475a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":53,"up_from":57,"up_thru":77,"down_at":54,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1152173882},{"type":"v1","addr":"172.21.15.114:6811","nonce":1152173882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1152173882},{"type":"v1","addr":"172.21.15.114:6813","nonce":1152173882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1152173882},{"type":"v1","addr":"172.21.15.114:6817","nonce":1152173882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1152173882},{"type":"v1","addr":"172.21.15.114:6815","nonce":1152173882}]},"public_addr":"172.21.15.114:6811/1152173882","cluster_addr":"172.21.15.114:6813/1152173882","heartbeat_back_addr":"172.21.15.114:6817/1152173882","heartbeat_front_addr":"172.21.15.114:6815/1152173882","state":["exists","up"]},{"osd":2,"uuid":"44f9fce8-5ea8-4741-9936-6720df91b628","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":58,"up_from":62,"up_thru":77,"down_at":59,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":3638870673},{"type":"v1","addr":"172.21.15.114:6819","nonce":3638870673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":3638870673},{"type":"v1","addr":"172.21.15.114:6821","nonce":3638870673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":3638870673},{"type":"v1","addr":"172.21.15.114:6825","nonce":3638870673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":3638870673},{"type":"v1","addr":"172.21.15.114:6823","nonce":3638870673}]},"public_addr":"172.21.15.114:6819/3638870673","cluster_addr":"172.21.15.114:6821/3638870673","heartbeat_back_addr":"172.21.15.114:6825/3638870673","heartbeat_front_addr":"172.21.15.114:6823/3638870673","state":["exists","up"]},{"osd":3,"uuid":"b6eb4941-bef4-4722-9050-cd6c041f9fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":63,"up_from":67,"up_thru":77,"down_at":64,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3100755210},{"type":"v1","addr":"172.21.15.123:6801","nonce":3100755210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3100755210},{"type":"v1","addr":"172.21.15.123:6803","nonce":3100755210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3100755210},{"type":"v1","addr":"172.21.15.123:6807","nonce":3100755210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3100755210},{"type":"v1","addr":"172.21.15.123:6805","nonce":3100755210}]},"public_addr":"172.21.15.123:6801/3100755210","cluster_addr":"172.21.15.123:6803/3100755210","heartbeat_back_addr":"172.21.15.123:6807/3100755210","heartbeat_front_addr":"172.21.15.123:6805/3100755210","state":["exists","up"]},{"osd":4,"uuid":"e9caa348-26f4-4ed5-a64e-59231fc48546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":68,"up_from":72,"up_thru":77,"down_at":69,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":75423100},{"type":"v1","addr":"172.21.15.123:6809","nonce":75423100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":75423100},{"type":"v1","addr":"172.21.15.123:6811","nonce":75423100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":75423100},{"type":"v1","addr":"172.21.15.123:6815","nonce":75423100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":75423100},{"type":"v1","addr":"172.21.15.123:6813","nonce":75423100}]},"public_addr":"172.21.15.123:6809/75423100","cluster_addr":"172.21.15.123:6811/75423100","heartbeat_back_addr":"172.21.15.123:6815/75423100","heartbeat_front_addr":"172.21.15.123:6813/75423100","state":["exists","up"]},{"osd":5,"uuid":"79d81af1-2361-430d-bc76-35ef74aa6bad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":73,"up_from":77,"up_thru":77,"down_at":74,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":4210417260},{"type":"v1","addr":"172.21.15.123:6817","nonce":4210417260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4210417260},{"type":"v1","addr":"172.21.15.123:6819","nonce":4210417260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4210417260},{"type":"v1","addr":"172.21.15.123:6823","nonce":4210417260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4210417260},{"type":"v1","addr":"172.21.15.123:6821","nonce":4210417260}]},"public_addr":"172.21.15.123:6817/4210417260","cluster_addr":"172.21.15.123:6819/4210417260","heartbeat_back_addr":"172.21.15.123:6823/4210417260","heartbeat_front_addr":"172.21.15.123:6821/4210417260","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:14:32.818023+0000","dead_epoch":48},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:15:00.597924+0000","dead_epoch":53},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:15:29.574022+0000","dead_epoch":58},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:15:53.159458+0000","dead_epoch":63},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:16:20.058254+0000","dead_epoch":68},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:16:45.789725+0000","dead_epoch":73}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6827/2812517585":"2024-02-17T22:29:37.876591+0000","172.21.15.123:6826/2812517585":"2024-02-17T22:29:37.876591+0000","172.21.15.123:6825/418014612":"2024-02-17T22:29:09.777831+0000","172.21.15.123:6824/418014612":"2024-02-17T22:29:09.777831+0000","172.21.15.114:6827/277953535":"2024-02-17T22:28:21.203522+0000","172.21.15.114:6826/277953535":"2024-02-17T22:28:21.203522+0000","172.21.15.114:0/297794530":"2024-02-17T22:23:45.719861+0000","172.21.15.114:6800/2055574123":"2024-02-17T22:11:27.861007+0000","172.21.15.114:6829/865692293":"2024-02-17T22:28:47.638133+0000","172.21.15.114:0/4031283053":"2024-02-17T22:21:10.405873+0000","172.21.15.114:0/1034649687":"2024-02-17T22:13:11.405214+0000","172.21.15.114:0/3991553908":"2024-02-17T22:21:10.405873+0000","172.21.15.114:0/744140874":"2024-02-17T22:12:00.492971+0000","172.21.15.114:6828/865692293":"2024-02-17T22:28:47.638133+0000","172.21.15.114:6800/3153243195":"2024-02-17T22:21:10.405873+0000","172.21.15.114:0/3211326148":"2024-02-17T22:13:11.405214+0000","172.21.15.123:6829/1539029039":"2024-02-17T22:22:31.925771+0000","172.21.15.114:6801/3153243195":"2024-02-17T22:21:10.405873+0000","172.21.15.114:6801/2055574123":"2024-02-17T22:11:27.861007+0000","172.21.15.123:6828/1539029039":"2024-02-17T22:22:31.925771+0000","172.21.15.114:0/2393113501":"2024-02-17T22:21:10.405873+0000","172.21.15.114:6800/2648536185":"2024-02-17T22:13:11.405214+0000","172.21.15.114:0/693887539":"2024-02-17T22:21:10.405873+0000","172.21.15.114:6800/625910309":"2024-02-17T22:12:00.492971+0000","172.21.15.123:0/2858796357":"2024-02-17T22:22:31.925771+0000","172.21.15.114:0/4029631970":"2024-02-17T22:11:27.861007+0000","172.21.15.114:0/2206519078":"2024-02-17T22:23:45.719861+0000","172.21.15.114:6801/2648536185":"2024-02-17T22:13:11.405214+0000","172.21.15.123:0/1942138241":"2024-02-17T22:22:31.925771+0000","172.21.15.123:0/216490899":"2024-02-17T22:22:31.925771+0000","172.21.15.114:0/3971798486":"2024-02-17T22:11:27.861007+0000","172.21.15.114:0/499892900":"2024-02-17T22:12:00.492971+0000","172.21.15.114:0/1349416361":"2024-02-17T22:12:00.492971+0000","172.21.15.123:0/2896457931":"2024-02-17T22:22:31.925771+0000","172.21.15.123:0/3641186660":"2024-02-17T22:22:31.925771+0000","172.21.15.114:0/125196455":"2024-02-17T22:23:45.719861+0000","172.21.15.114:6801/625910309":"2024-02-17T22:12:00.492971+0000","172.21.15.114:0/2762476246":"2024-02-17T22:13:11.405214+0000","172.21.15.123:0/4134230222":"2024-02-17T22:22:31.925771+0000","172.21.15.114:6800/3051780151":"2024-02-17T22:23:45.719861+0000","172.21.15.114:0/489142132":"2024-02-17T22:23:45.719861+0000","172.21.15.114:6801/3051780151":"2024-02-17T22:23:45.719861+0000","172.21.15.114:0/1088872575":"2024-02-17T22:23:45.719861+0000","172.21.15.114:0/2825621999":"2024-02-17T22:11:27.861007+0000","172.21.15.114:0/1250197670":"2024-02-17T22:23:45.719861+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-16T22:33:34.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.615+0000 7fd1a19cb640 1 -- 172.21.15.114:0/2536631179 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd18806d8d0 msgr2=0x7fd18806fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:34.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.615+0000 7fd1a19cb640 1 --2- 172.21.15.114:0/2536631179 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd18806d8d0 0x7fd18806fd90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd18c006010 tx=0x7fd18c005dd0 comp rx=0 tx=0).stop 2024-02-16T22:33:34.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.615+0000 7fd1a19cb640 1 -- 172.21.15.114:0/2536631179 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd19c197030 msgr2=0x7fd19c197490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:34.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.615+0000 7fd1a19cb640 1 --2- 172.21.15.114:0/2536631179 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd19c197030 0x7fd19c197490 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd18400d720 tx=0x7fd18400dbe0 comp rx=0 tx=0).stop 2024-02-16T22:33:34.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.615+0000 7fd1a19cb640 1 -- 172.21.15.114:0/2536631179 shutdown_connections 2024-02-16T22:33:34.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.615+0000 7fd1a19cb640 1 --2- 172.21.15.114:0/2536631179 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd18806d8d0 0x7fd18806fd90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:34.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.615+0000 7fd1a19cb640 1 --2- 172.21.15.114:0/2536631179 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd19c197030 0x7fd19c197490 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:34.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.615+0000 7fd1a19cb640 1 --2- 172.21.15.114:0/2536631179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd19c0ff1a0 0x7fd19c196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:34.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.615+0000 7fd1a19cb640 1 -- 172.21.15.114:0/2536631179 >> 172.21.15.114:0/2536631179 conn(0x7fd19c0faa20 msgr2=0x7fd19c0fc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:34.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.615+0000 7fd1a19cb640 1 -- 172.21.15.114:0/2536631179 shutdown_connections 2024-02-16T22:33:34.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.615+0000 7fd1a19cb640 1 -- 172.21.15.114:0/2536631179 wait complete. 2024-02-16T22:33:34.629 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-16T22:15:31.749346+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-16T22:18:01.042512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-16T22:18:01.336617+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-16T22:33:34.629 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-16T22:33:34.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.731+0000 7fd09b8f7640 1 -- 172.21.15.114:0/1697705895 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0940ff180 msgr2=0x7fd0940ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:34.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.731+0000 7fd09b8f7640 1 --2- 172.21.15.114:0/1697705895 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0940ff180 0x7fd0940ff560 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd090009880 tx=0x7fd09002bce0 comp rx=0 tx=0).stop 2024-02-16T22:33:34.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.731+0000 7fd09b8f7640 1 -- 172.21.15.114:0/1697705895 shutdown_connections 2024-02-16T22:33:34.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.731+0000 7fd09b8f7640 1 --2- 172.21.15.114:0/1697705895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0940ffb30 0x7fd094103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:34.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.731+0000 7fd09b8f7640 1 --2- 172.21.15.114:0/1697705895 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0940ff180 0x7fd0940ff560 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:34.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.731+0000 7fd09b8f7640 1 -- 172.21.15.114:0/1697705895 >> 172.21.15.114:0/1697705895 conn(0x7fd0940faa00 msgr2=0x7fd0940fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:34.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.731+0000 7fd09b8f7640 1 -- 172.21.15.114:0/1697705895 shutdown_connections 2024-02-16T22:33:34.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.731+0000 7fd09b8f7640 1 -- 172.21.15.114:0/1697705895 wait complete. 2024-02-16T22:33:34.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.732+0000 7fd09b8f7640 1 Processor -- start 2024-02-16T22:33:34.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.732+0000 7fd09b8f7640 1 -- start start 2024-02-16T22:33:34.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.732+0000 7fd09b8f7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0940ff180 0x7fd094100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:34.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.732+0000 7fd09b8f7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd094101150 0x7fd0941015b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:34.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.732+0000 7fd09b8f7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0941a1f50 con 0x7fd0940ff180 2024-02-16T22:33:34.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.732+0000 7fd09b8f7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0941a2090 con 0x7fd094101150 2024-02-16T22:33:34.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.732+0000 7fd099693640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0940ff180 0x7fd094100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:34.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.732+0000 7fd099693640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0940ff180 0x7fd094100c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47904/0 (socket says 172.21.15.114:47904) 2024-02-16T22:33:34.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.732+0000 7fd099693640 1 -- 172.21.15.114:0/2733041474 learned_addr learned my addr 172.21.15.114:0/2733041474 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:34.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.732+0000 7fd099693640 1 -- 172.21.15.114:0/2733041474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd094101150 msgr2=0x7fd0941015b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:34.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.732+0000 7fd099693640 1 --2- 172.21.15.114:0/2733041474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd094101150 0x7fd0941015b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:34.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.732+0000 7fd099693640 1 -- 172.21.15.114:0/2733041474 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd090009560 con 0x7fd0940ff180 2024-02-16T22:33:34.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.732+0000 7fd099693640 1 --2- 172.21.15.114:0/2733041474 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0940ff180 0x7fd094100c10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd090009430 tx=0x7fd090004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:34.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.732+0000 7fd08a7fc640 1 -- 172.21.15.114:0/2733041474 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0900035f0 con 0x7fd0940ff180 2024-02-16T22:33:34.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.732+0000 7fd08a7fc640 1 -- 172.21.15.114:0/2733041474 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd090003c10 con 0x7fd0940ff180 2024-02-16T22:33:34.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.732+0000 7fd09b8f7640 1 -- 172.21.15.114:0/2733041474 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0941a21d0 con 0x7fd0940ff180 2024-02-16T22:33:34.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.732+0000 7fd08a7fc640 1 -- 172.21.15.114:0/2733041474 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd090005670 con 0x7fd0940ff180 2024-02-16T22:33:34.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.732+0000 7fd09b8f7640 1 -- 172.21.15.114:0/2733041474 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0941a2680 con 0x7fd0940ff180 2024-02-16T22:33:34.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.733+0000 7fd09b8f7640 1 -- 172.21.15.114:0/2733041474 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd09404e040 con 0x7fd0940ff180 2024-02-16T22:33:34.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.734+0000 7fd08a7fc640 1 -- 172.21.15.114:0/2733041474 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fd090004dd0 con 0x7fd0940ff180 2024-02-16T22:33:34.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.734+0000 7fd08a7fc640 1 --2- 172.21.15.114:0/2733041474 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd08006d810 0x7fd08006fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:34.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.734+0000 7fd08a7fc640 1 -- 172.21.15.114:0/2733041474 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7fd0900af930 con 0x7fd0940ff180 2024-02-16T22:33:34.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.734+0000 7fd098e92640 1 --2- 172.21.15.114:0/2733041474 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd08006d810 0x7fd08006fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:34.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.735+0000 7fd098e92640 1 --2- 172.21.15.114:0/2733041474 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd08006d810 0x7fd08006fcd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd084005fd0 tx=0x7fd084005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:34.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.739+0000 7fd08a7fc640 1 -- 172.21.15.114:0/2733041474 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd0900790b0 con 0x7fd0940ff180 2024-02-16T22:33:34.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.911+0000 7fd09b8f7640 1 -- 172.21.15.114:0/2733041474 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd0940ffb30 con 0x7fd0940ff180 2024-02-16T22:33:34.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.912+0000 7fd08a7fc640 1 -- 172.21.15.114:0/2733041474 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd0900ade60 con 0x7fd0940ff180 2024-02-16T22:33:34.914 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2024-02-16T22:33:34.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.914+0000 7fd09b8f7640 1 -- 172.21.15.114:0/2733041474 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd08006d810 msgr2=0x7fd08006fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:34.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.914+0000 7fd09b8f7640 1 --2- 172.21.15.114:0/2733041474 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd08006d810 0x7fd08006fcd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd084005fd0 tx=0x7fd084005850 comp rx=0 tx=0).stop 2024-02-16T22:33:34.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.914+0000 7fd09b8f7640 1 -- 172.21.15.114:0/2733041474 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0940ff180 msgr2=0x7fd094100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:34.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.914+0000 7fd09b8f7640 1 --2- 172.21.15.114:0/2733041474 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0940ff180 0x7fd094100c10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd090009430 tx=0x7fd090004040 comp rx=0 tx=0).stop 2024-02-16T22:33:34.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.914+0000 7fd09b8f7640 1 -- 172.21.15.114:0/2733041474 shutdown_connections 2024-02-16T22:33:34.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.914+0000 7fd09b8f7640 1 --2- 172.21.15.114:0/2733041474 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd08006d810 0x7fd08006fcd0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:34.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.914+0000 7fd09b8f7640 1 --2- 172.21.15.114:0/2733041474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd094101150 0x7fd0941015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:34.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.914+0000 7fd09b8f7640 1 --2- 172.21.15.114:0/2733041474 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0940ff180 0x7fd094100c10 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:34.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.914+0000 7fd09b8f7640 1 -- 172.21.15.114:0/2733041474 >> 172.21.15.114:0/2733041474 conn(0x7fd0940faa00 msgr2=0x7fd094103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:34.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.914+0000 7fd09b8f7640 1 -- 172.21.15.114:0/2733041474 shutdown_connections 2024-02-16T22:33:34.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:34.914+0000 7fd09b8f7640 1 -- 172.21.15.114:0/2733041474 wait complete. 2024-02-16T22:33:34.926 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:34 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/1464009587' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:33:34.927 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-16T22:33:35.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.029+0000 7fd5fe5f0640 1 -- 172.21.15.114:0/3180467928 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5f80ff260 msgr2=0x7fd5f80ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:35.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.029+0000 7fd5fe5f0640 1 --2- 172.21.15.114:0/3180467928 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5f80ff260 0x7fd5f80ff640 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8009880 tx=0x7fd5e802bce0 comp rx=0 tx=0).stop 2024-02-16T22:33:35.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.029+0000 7fd5fe5f0640 1 -- 172.21.15.114:0/3180467928 shutdown_connections 2024-02-16T22:33:35.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.029+0000 7fd5fe5f0640 1 --2- 172.21.15.114:0/3180467928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5f80ffc10 0x7fd5f8103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:35.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.029+0000 7fd5fe5f0640 1 --2- 172.21.15.114:0/3180467928 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5f80ff260 0x7fd5f80ff640 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:35.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.029+0000 7fd5fe5f0640 1 -- 172.21.15.114:0/3180467928 >> 172.21.15.114:0/3180467928 conn(0x7fd5f8074a50 msgr2=0x7fd5f8074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:35.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.029+0000 7fd5fe5f0640 1 -- 172.21.15.114:0/3180467928 shutdown_connections 2024-02-16T22:33:35.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.029+0000 7fd5fe5f0640 1 -- 172.21.15.114:0/3180467928 wait complete. 2024-02-16T22:33:35.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.030+0000 7fd5fe5f0640 1 Processor -- start 2024-02-16T22:33:35.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.030+0000 7fd5fe5f0640 1 -- start start 2024-02-16T22:33:35.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.030+0000 7fd5fe5f0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5f80ff260 0x7fd5f8196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:35.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.030+0000 7fd5fe5f0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5f81972a0 0x7fd5f8197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:35.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.030+0000 7fd5fe5f0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f819fdd0 con 0x7fd5f81972a0 2024-02-16T22:33:35.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.030+0000 7fd5fe5f0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f819ff10 con 0x7fd5f80ff260 2024-02-16T22:33:35.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.030+0000 7fd5f77fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5f81972a0 0x7fd5f8197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:35.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.031+0000 7fd5f77fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5f81972a0 0x7fd5f8197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47934/0 (socket says 172.21.15.114:47934) 2024-02-16T22:33:35.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.031+0000 7fd5f77fe640 1 -- 172.21.15.114:0/430771862 learned_addr learned my addr 172.21.15.114:0/430771862 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:35.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.031+0000 7fd5f7fff640 1 --2- 172.21.15.114:0/430771862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5f80ff260 0x7fd5f8196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:35.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.031+0000 7fd5f77fe640 1 -- 172.21.15.114:0/430771862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5f80ff260 msgr2=0x7fd5f8196d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:35.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.031+0000 7fd5f77fe640 1 --2- 172.21.15.114:0/430771862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5f80ff260 0x7fd5f8196d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:35.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.031+0000 7fd5f77fe640 1 -- 172.21.15.114:0/430771862 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e8009560 con 0x7fd5f81972a0 2024-02-16T22:33:35.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.031+0000 7fd5f77fe640 1 --2- 172.21.15.114:0/430771862 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5f81972a0 0x7fd5f8197700 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00b400 tx=0x7fd5ec00b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:35.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.031+0000 7fd5f57fa640 1 -- 172.21.15.114:0/430771862 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5ec011800 con 0x7fd5f81972a0 2024-02-16T22:33:35.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.031+0000 7fd5f57fa640 1 -- 172.21.15.114:0/430771862 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd5ec011e20 con 0x7fd5f81972a0 2024-02-16T22:33:35.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.031+0000 7fd5fe5f0640 1 -- 172.21.15.114:0/430771862 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f81a0170 con 0x7fd5f81972a0 2024-02-16T22:33:35.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.031+0000 7fd5f57fa640 1 -- 172.21.15.114:0/430771862 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5ec00f2e0 con 0x7fd5f81972a0 2024-02-16T22:33:35.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.031+0000 7fd5fe5f0640 1 -- 172.21.15.114:0/430771862 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f81a0680 con 0x7fd5f81972a0 2024-02-16T22:33:35.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.032+0000 7fd5fe5f0640 1 -- 172.21.15.114:0/430771862 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5f804e040 con 0x7fd5f81972a0 2024-02-16T22:33:35.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.032+0000 7fd5f57fa640 1 -- 172.21.15.114:0/430771862 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fd5ec010380 con 0x7fd5f81972a0 2024-02-16T22:33:35.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.033+0000 7fd5f57fa640 1 --2- 172.21.15.114:0/430771862 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd5e006d810 0x7fd5e006fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:35.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.033+0000 7fd5f7fff640 1 --2- 172.21.15.114:0/430771862 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd5e006d810 0x7fd5e006fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:35.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.033+0000 7fd5f57fa640 1 -- 172.21.15.114:0/430771862 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7fd5ec08d1d0 con 0x7fd5f81972a0 2024-02-16T22:33:35.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.033+0000 7fd5f7fff640 1 --2- 172.21.15.114:0/430771862 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd5e006d810 0x7fd5e006fcd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8006010 tx=0x7fd5e8005b50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:35.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.038+0000 7fd5f57fa640 1 -- 172.21.15.114:0/430771862 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd5ec0568d0 con 0x7fd5f81972a0 2024-02-16T22:33:35.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:34 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/1464009587' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T22:33:35.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.210+0000 7fd5fe5f0640 1 -- 172.21.15.114:0/430771862 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fd5f8107480 con 0x7fd5f81972a0 2024-02-16T22:33:35.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.210+0000 7fd5f57fa640 1 -- 172.21.15.114:0/430771862 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v82) v1 ==== 107+0+11 (secure 0 0 0) 0x7fd5ec059f90 con 0x7fd5f81972a0 2024-02-16T22:33:35.212 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-02-16T22:33:35.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.212+0000 7fd5fe5f0640 1 -- 172.21.15.114:0/430771862 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd5e006d810 msgr2=0x7fd5e006fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:35.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.212+0000 7fd5fe5f0640 1 --2- 172.21.15.114:0/430771862 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd5e006d810 0x7fd5e006fcd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8006010 tx=0x7fd5e8005b50 comp rx=0 tx=0).stop 2024-02-16T22:33:35.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.212+0000 7fd5fe5f0640 1 -- 172.21.15.114:0/430771862 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5f81972a0 msgr2=0x7fd5f8197700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:35.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.212+0000 7fd5fe5f0640 1 --2- 172.21.15.114:0/430771862 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5f81972a0 0x7fd5f8197700 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00b400 tx=0x7fd5ec00b8c0 comp rx=0 tx=0).stop 2024-02-16T22:33:35.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.212+0000 7fd5fe5f0640 1 -- 172.21.15.114:0/430771862 shutdown_connections 2024-02-16T22:33:35.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.212+0000 7fd5fe5f0640 1 --2- 172.21.15.114:0/430771862 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fd5e006d810 0x7fd5e006fcd0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:35.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.212+0000 7fd5fe5f0640 1 --2- 172.21.15.114:0/430771862 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5f81972a0 0x7fd5f8197700 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:35.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.212+0000 7fd5fe5f0640 1 --2- 172.21.15.114:0/430771862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5f80ff260 0x7fd5f8196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:35.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.212+0000 7fd5fe5f0640 1 -- 172.21.15.114:0/430771862 >> 172.21.15.114:0/430771862 conn(0x7fd5f8074a50 msgr2=0x7fd5f80fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:35.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.212+0000 7fd5fe5f0640 1 -- 172.21.15.114:0/430771862 shutdown_connections 2024-02-16T22:33:35.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.212+0000 7fd5fe5f0640 1 -- 172.21.15.114:0/430771862 wait complete. 2024-02-16T22:33:35.225 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-16T22:33:35.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.327+0000 7f22fe101640 1 -- 172.21.15.114:0/459808548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22f8101b00 msgr2=0x7f22f8109fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:35.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.327+0000 7f22fe101640 1 --2- 172.21.15.114:0/459808548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22f8101b00 0x7f22f8109fe0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f22ec0098b0 tx=0x7f22ec02bcd0 comp rx=0 tx=0).stop 2024-02-16T22:33:35.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.327+0000 7f22fe101640 1 -- 172.21.15.114:0/459808548 shutdown_connections 2024-02-16T22:33:35.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.327+0000 7f22fe101640 1 --2- 172.21.15.114:0/459808548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22f8101b00 0x7f22f8109fe0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:35.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.327+0000 7f22fe101640 1 --2- 172.21.15.114:0/459808548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22f8101150 0x7f22f8101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:35.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.327+0000 7f22fe101640 1 -- 172.21.15.114:0/459808548 >> 172.21.15.114:0/459808548 conn(0x7f22f80faa40 msgr2=0x7f22f80fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:35.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.327+0000 7f22fe101640 1 -- 172.21.15.114:0/459808548 shutdown_connections 2024-02-16T22:33:35.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.327+0000 7f22fe101640 1 -- 172.21.15.114:0/459808548 wait complete. 2024-02-16T22:33:35.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.327+0000 7f22fe101640 1 Processor -- start 2024-02-16T22:33:35.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.328+0000 7f22fe101640 1 -- start start 2024-02-16T22:33:35.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.328+0000 7f22fe101640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22f8101150 0x7f22f8195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:35.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.328+0000 7f22fe101640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22f8101b00 0x7f22f81962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:35.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.328+0000 7f22fe101640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f81968b0 con 0x7f22f8101b00 2024-02-16T22:33:35.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.328+0000 7f22fe101640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f8196a20 con 0x7f22f8101150 2024-02-16T22:33:35.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.328+0000 7f22f6ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22f8101b00 0x7f22f81962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:35.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.328+0000 7f22f6ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22f8101b00 0x7f22f81962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47948/0 (socket says 172.21.15.114:47948) 2024-02-16T22:33:35.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.328+0000 7f22f6ffd640 1 -- 172.21.15.114:0/1129444274 learned_addr learned my addr 172.21.15.114:0/1129444274 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:35.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.328+0000 7f22f6ffd640 1 -- 172.21.15.114:0/1129444274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22f8101150 msgr2=0x7f22f8195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:35.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.328+0000 7f22f6ffd640 1 --2- 172.21.15.114:0/1129444274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22f8101150 0x7f22f8195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:35.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.328+0000 7f22f6ffd640 1 -- 172.21.15.114:0/1129444274 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ec009560 con 0x7f22f8101b00 2024-02-16T22:33:35.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.328+0000 7f22f6ffd640 1 --2- 172.21.15.114:0/1129444274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22f8101b00 0x7f22f81962b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f22ec009880 tx=0x7f22ec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:35.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.329+0000 7f22f4ff9640 1 -- 172.21.15.114:0/1129444274 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22ec0037c0 con 0x7f22f8101b00 2024-02-16T22:33:35.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.329+0000 7f22fe101640 1 -- 172.21.15.114:0/1129444274 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22f819a800 con 0x7f22f8101b00 2024-02-16T22:33:35.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.329+0000 7f22f4ff9640 1 -- 172.21.15.114:0/1129444274 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f22ec003de0 con 0x7f22f8101b00 2024-02-16T22:33:35.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.329+0000 7f22f4ff9640 1 -- 172.21.15.114:0/1129444274 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22ec034440 con 0x7f22f8101b00 2024-02-16T22:33:35.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.329+0000 7f22fe101640 1 -- 172.21.15.114:0/1129444274 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f819ad00 con 0x7f22f8101b00 2024-02-16T22:33:35.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.330+0000 7f22fe101640 1 -- 172.21.15.114:0/1129444274 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22f804e040 con 0x7f22f8101b00 2024-02-16T22:33:35.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.330+0000 7f22f4ff9640 1 -- 172.21.15.114:0/1129444274 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f22ec004fd0 con 0x7f22f8101b00 2024-02-16T22:33:35.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.330+0000 7f22f4ff9640 1 --2- 172.21.15.114:0/1129444274 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f22e006d810 0x7f22e006fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:35.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.330+0000 7f22f4ff9640 1 -- 172.21.15.114:0/1129444274 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f22ec0afb00 con 0x7f22f8101b00 2024-02-16T22:33:35.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.330+0000 7f22f77fe640 1 --2- 172.21.15.114:0/1129444274 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f22e006d810 0x7f22e006fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:35.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.331+0000 7f22f77fe640 1 --2- 172.21.15.114:0/1129444274 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f22e006d810 0x7f22e006fcd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f22e800a990 tx=0x7f22e8005c70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:35.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.335+0000 7f22f4ff9640 1 -- 172.21.15.114:0/1129444274 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f22ec079200 con 0x7f22f8101b00 2024-02-16T22:33:35.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.506+0000 7f22fe101640 1 -- 172.21.15.114:0/1129444274 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f22f8107650 con 0x7f22f8101b00 2024-02-16T22:33:35.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.506+0000 7f22f4ff9640 1 -- 172.21.15.114:0/1129444274 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v82) v1 ==== 107+0+11 (secure 0 0 0) 0x7f22ec07c8c0 con 0x7f22f8101b00 2024-02-16T22:33:35.508 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-02-16T22:33:35.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.509+0000 7f22fe101640 1 -- 172.21.15.114:0/1129444274 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f22e006d810 msgr2=0x7f22e006fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:35.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.509+0000 7f22fe101640 1 --2- 172.21.15.114:0/1129444274 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f22e006d810 0x7f22e006fcd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f22e800a990 tx=0x7f22e8005c70 comp rx=0 tx=0).stop 2024-02-16T22:33:35.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.509+0000 7f22fe101640 1 -- 172.21.15.114:0/1129444274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22f8101b00 msgr2=0x7f22f81962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:35.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.509+0000 7f22fe101640 1 --2- 172.21.15.114:0/1129444274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22f8101b00 0x7f22f81962b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f22ec009880 tx=0x7f22ec004270 comp rx=0 tx=0).stop 2024-02-16T22:33:35.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.509+0000 7f22fe101640 1 -- 172.21.15.114:0/1129444274 shutdown_connections 2024-02-16T22:33:35.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.509+0000 7f22fe101640 1 --2- 172.21.15.114:0/1129444274 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f22e006d810 0x7f22e006fcd0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:35.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.509+0000 7f22fe101640 1 --2- 172.21.15.114:0/1129444274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22f8101b00 0x7f22f81962b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:35.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.509+0000 7f22fe101640 1 --2- 172.21.15.114:0/1129444274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22f8101150 0x7f22f8195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:35.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.509+0000 7f22fe101640 1 -- 172.21.15.114:0/1129444274 >> 172.21.15.114:0/1129444274 conn(0x7f22f80faa40 msgr2=0x7f22f8104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:35.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.509+0000 7f22fe101640 1 -- 172.21.15.114:0/1129444274 shutdown_connections 2024-02-16T22:33:35.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.509+0000 7f22fe101640 1 -- 172.21.15.114:0/1129444274 wait complete. 2024-02-16T22:33:35.522 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-16T22:33:35.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.621+0000 7f265c550640 1 -- 172.21.15.114:0/1512475706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26540ffb30 msgr2=0x7f2654103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:35.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.621+0000 7f265c550640 1 --2- 172.21.15.114:0/1512475706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26540ffb30 0x7f2654103b70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2644009880 tx=0x7f264402bce0 comp rx=0 tx=0).stop 2024-02-16T22:33:35.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.621+0000 7f265c550640 1 -- 172.21.15.114:0/1512475706 shutdown_connections 2024-02-16T22:33:35.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.621+0000 7f265c550640 1 --2- 172.21.15.114:0/1512475706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26540ffb30 0x7f2654103b70 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:35.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.621+0000 7f265c550640 1 --2- 172.21.15.114:0/1512475706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26540ff180 0x7f26540ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:35.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.621+0000 7f265c550640 1 -- 172.21.15.114:0/1512475706 >> 172.21.15.114:0/1512475706 conn(0x7f26540fa9e0 msgr2=0x7f26540fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:35.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.621+0000 7f265c550640 1 -- 172.21.15.114:0/1512475706 shutdown_connections 2024-02-16T22:33:35.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.621+0000 7f265c550640 1 -- 172.21.15.114:0/1512475706 wait complete. 2024-02-16T22:33:35.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.622+0000 7f265c550640 1 Processor -- start 2024-02-16T22:33:35.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.622+0000 7f265c550640 1 -- start start 2024-02-16T22:33:35.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.622+0000 7f265c550640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26540ff180 0x7f2654195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:35.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.622+0000 7f265c550640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2654196370 0x7f265419a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:35.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.622+0000 7f265c550640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2654196970 con 0x7f2654196370 2024-02-16T22:33:35.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.622+0000 7f265c550640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2654196ae0 con 0x7f26540ff180 2024-02-16T22:33:35.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.622+0000 7f2659aeb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2654196370 0x7f265419a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:35.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.622+0000 7f2659aeb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2654196370 0x7f265419a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47970/0 (socket says 172.21.15.114:47970) 2024-02-16T22:33:35.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.622+0000 7f2659aeb640 1 -- 172.21.15.114:0/1263229626 learned_addr learned my addr 172.21.15.114:0/1263229626 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:35.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.622+0000 7f265a2ec640 1 --2- 172.21.15.114:0/1263229626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26540ff180 0x7f2654195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:35.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.623+0000 7f2659aeb640 1 -- 172.21.15.114:0/1263229626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26540ff180 msgr2=0x7f2654195e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:35.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.623+0000 7f2659aeb640 1 --2- 172.21.15.114:0/1263229626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26540ff180 0x7f2654195e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:35.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.623+0000 7f2659aeb640 1 -- 172.21.15.114:0/1263229626 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2644009560 con 0x7f2654196370 2024-02-16T22:33:35.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.623+0000 7f2659aeb640 1 --2- 172.21.15.114:0/1263229626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2654196370 0x7f265419a7d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2644036040 tx=0x7f2644004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:35.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.623+0000 7f264b7fe640 1 -- 172.21.15.114:0/1263229626 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26440037c0 con 0x7f2654196370 2024-02-16T22:33:35.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.623+0000 7f265c550640 1 -- 172.21.15.114:0/1263229626 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f265419ad70 con 0x7f2654196370 2024-02-16T22:33:35.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.623+0000 7f264b7fe640 1 -- 172.21.15.114:0/1263229626 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2644004aa0 con 0x7f2654196370 2024-02-16T22:33:35.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.623+0000 7f264b7fe640 1 -- 172.21.15.114:0/1263229626 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2644034440 con 0x7f2654196370 2024-02-16T22:33:35.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.623+0000 7f265c550640 1 -- 172.21.15.114:0/1263229626 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f265419b2d0 con 0x7f2654196370 2024-02-16T22:33:35.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.624+0000 7f265c550640 1 -- 172.21.15.114:0/1263229626 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f265404e040 con 0x7f2654196370 2024-02-16T22:33:35.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.625+0000 7f264b7fe640 1 -- 172.21.15.114:0/1263229626 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f2644004f60 con 0x7f2654196370 2024-02-16T22:33:35.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.625+0000 7f264b7fe640 1 --2- 172.21.15.114:0/1263229626 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f264006d8d0 0x7f264006fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:35.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.625+0000 7f264b7fe640 1 -- 172.21.15.114:0/1263229626 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f26440af9f0 con 0x7f2654196370 2024-02-16T22:33:35.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.625+0000 7f265a2ec640 1 --2- 172.21.15.114:0/1263229626 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f264006d8d0 0x7f264006fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:35.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.625+0000 7f265a2ec640 1 --2- 172.21.15.114:0/1263229626 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f264006d8d0 0x7f264006fd90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2650009990 tx=0x7f2650009110 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:35.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.630+0000 7f264b7fe640 1 -- 172.21.15.114:0/1263229626 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f26440b1e60 con 0x7f2654196370 2024-02-16T22:33:35.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.841+0000 7f265c550640 1 -- 172.21.15.114:0/1263229626 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26540009c0 con 0x7f2654196370 2024-02-16T22:33:35.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.841+0000 7f264b7fe640 1 -- 172.21.15.114:0/1263229626 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+5008 (secure 0 0 0) 0x7f26540009c0 con 0x7f2654196370 2024-02-16T22:33:35.843 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:35.843 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi114.fdxpey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6829/3702914193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3702914193},{"type":"v1","addr":"172.21.15.114:6829","nonce":3702914193}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":44331,"name":"cephfs.smithi123.vhlqbe","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6825/1425065679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1425065679},{"type":"v1","addr":"172.21.15.123:6825","nonce":1425065679}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":44335,"name":"cephfs.smithi123.gopcfa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6827/3128771108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3128771108},{"type":"v1","addr":"172.21.15.123:6827","nonce":3128771108}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:18:02.108825+0000","modified":"2024-02-16T22:29:48.901292+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi114.zzryke","rank":0,"incarnation":34,"state":"up:active","state_seq":23,"addr":"172.21.15.114:6827/2017800151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2017800151},{"type":"v1","addr":"172.21.15.114:6827","nonce":2017800151}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T22:33:35.843 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 37 2024-02-16T22:33:35.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.843+0000 7f265c550640 1 -- 172.21.15.114:0/1263229626 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f264006d8d0 msgr2=0x7f264006fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:35.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.843+0000 7f265c550640 1 --2- 172.21.15.114:0/1263229626 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f264006d8d0 0x7f264006fd90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2650009990 tx=0x7f2650009110 comp rx=0 tx=0).stop 2024-02-16T22:33:35.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.843+0000 7f265c550640 1 -- 172.21.15.114:0/1263229626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2654196370 msgr2=0x7f265419a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:35.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.843+0000 7f265c550640 1 --2- 172.21.15.114:0/1263229626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2654196370 0x7f265419a7d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2644036040 tx=0x7f2644004270 comp rx=0 tx=0).stop 2024-02-16T22:33:35.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.843+0000 7f265c550640 1 -- 172.21.15.114:0/1263229626 shutdown_connections 2024-02-16T22:33:35.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.843+0000 7f265c550640 1 --2- 172.21.15.114:0/1263229626 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f264006d8d0 0x7f264006fd90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:35.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.843+0000 7f265c550640 1 --2- 172.21.15.114:0/1263229626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2654196370 0x7f265419a7d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:35.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.843+0000 7f265c550640 1 --2- 172.21.15.114:0/1263229626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26540ff180 0x7f2654195e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:35.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.843+0000 7f265c550640 1 -- 172.21.15.114:0/1263229626 >> 172.21.15.114:0/1263229626 conn(0x7f26540fa9e0 msgr2=0x7f2654103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:35.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.843+0000 7f265c550640 1 -- 172.21.15.114:0/1263229626 shutdown_connections 2024-02-16T22:33:35.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.843+0000 7f265c550640 1 -- 172.21.15.114:0/1263229626 wait complete. 2024-02-16T22:33:35.855 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:35 smithi114 ceph-mon[94452]: pgmap v331: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:35.855 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:35 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/2536631179' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T22:33:35.855 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:35 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/2733041474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T22:33:35.855 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:35 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/430771862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T22:33:35.855 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:35 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/1129444274' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T22:33:35.856 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 12, 'max_mds': 1, 'flags': 18} 2024-02-16T22:33:35.856 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 13 2024-02-16T22:33:35.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.956+0000 7fc406d3c640 1 -- 172.21.15.114:0/1072004053 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4000ff260 msgr2=0x7fc4000ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:35.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.956+0000 7fc406d3c640 1 --2- 172.21.15.114:0/1072004053 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4000ff260 0x7fc4000ff640 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4009880 tx=0x7fc3f402bce0 comp rx=0 tx=0).stop 2024-02-16T22:33:35.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.957+0000 7fc406d3c640 1 -- 172.21.15.114:0/1072004053 shutdown_connections 2024-02-16T22:33:35.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.957+0000 7fc406d3c640 1 --2- 172.21.15.114:0/1072004053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4000ffc10 0x7fc400103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:35.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.957+0000 7fc406d3c640 1 --2- 172.21.15.114:0/1072004053 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4000ff260 0x7fc4000ff640 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:35.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.957+0000 7fc406d3c640 1 -- 172.21.15.114:0/1072004053 >> 172.21.15.114:0/1072004053 conn(0x7fc400074a50 msgr2=0x7fc400074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:35.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.957+0000 7fc406d3c640 1 -- 172.21.15.114:0/1072004053 shutdown_connections 2024-02-16T22:33:35.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.957+0000 7fc406d3c640 1 -- 172.21.15.114:0/1072004053 wait complete. 2024-02-16T22:33:35.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.957+0000 7fc406d3c640 1 Processor -- start 2024-02-16T22:33:35.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.957+0000 7fc406d3c640 1 -- start start 2024-02-16T22:33:35.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.957+0000 7fc406d3c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4000ff260 0x7fc4001949e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:35.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.957+0000 7fc406d3c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc400194f20 0x7fc4001953a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:35.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.957+0000 7fc406d3c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4000682f0 con 0x7fc400194f20 2024-02-16T22:33:35.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.957+0000 7fc406d3c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4001a2050 con 0x7fc4000ff260 2024-02-16T22:33:35.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.957+0000 7fc3fffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc400194f20 0x7fc4001953a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:35.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.957+0000 7fc3fffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc400194f20 0x7fc4001953a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47974/0 (socket says 172.21.15.114:47974) 2024-02-16T22:33:35.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.957+0000 7fc3fffff640 1 -- 172.21.15.114:0/1132511251 learned_addr learned my addr 172.21.15.114:0/1132511251 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:35.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.958+0000 7fc404ad8640 1 --2- 172.21.15.114:0/1132511251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4000ff260 0x7fc4001949e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:35.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.958+0000 7fc3fffff640 1 -- 172.21.15.114:0/1132511251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4000ff260 msgr2=0x7fc4001949e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:35.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.958+0000 7fc3fffff640 1 --2- 172.21.15.114:0/1132511251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4000ff260 0x7fc4001949e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:35.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.958+0000 7fc3fffff640 1 -- 172.21.15.114:0/1132511251 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f4009560 con 0x7fc400194f20 2024-02-16T22:33:35.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.958+0000 7fc3fffff640 1 --2- 172.21.15.114:0/1132511251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc400194f20 0x7fc4001953a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec005fd0 tx=0x7fc3ec00dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:35.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.958+0000 7fc3fdffb640 1 -- 172.21.15.114:0/1132511251 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3ec009c00 con 0x7fc400194f20 2024-02-16T22:33:35.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.958+0000 7fc406d3c640 1 -- 172.21.15.114:0/1132511251 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4001a21f0 con 0x7fc400194f20 2024-02-16T22:33:35.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.958+0000 7fc406d3c640 1 -- 172.21.15.114:0/1132511251 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4001a2700 con 0x7fc400194f20 2024-02-16T22:33:35.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.958+0000 7fc3fdffb640 1 -- 172.21.15.114:0/1132511251 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3ec00de90 con 0x7fc400194f20 2024-02-16T22:33:35.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.958+0000 7fc3fdffb640 1 -- 172.21.15.114:0/1132511251 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3ec003e20 con 0x7fc400194f20 2024-02-16T22:33:35.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.959+0000 7fc406d3c640 1 -- 172.21.15.114:0/1132511251 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3e4005280 con 0x7fc400194f20 2024-02-16T22:33:35.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.960+0000 7fc3fdffb640 1 -- 172.21.15.114:0/1132511251 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fc3ec0035f0 con 0x7fc400194f20 2024-02-16T22:33:35.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.960+0000 7fc3fdffb640 1 --2- 172.21.15.114:0/1132511251 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fc3f006d7c0 0x7fc3f006fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:35.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.960+0000 7fc3fdffb640 1 -- 172.21.15.114:0/1132511251 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7fc3ec08d6a0 con 0x7fc400194f20 2024-02-16T22:33:35.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.960+0000 7fc404ad8640 1 --2- 172.21.15.114:0/1132511251 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fc3f006d7c0 0x7fc3f006fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:35.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.961+0000 7fc404ad8640 1 --2- 172.21.15.114:0/1132511251 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fc3f006d7c0 0x7fc3f006fc80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4036040 tx=0x7fc3f4005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:35.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:35.965+0000 7fc3fdffb640 1 -- 172.21.15.114:0/1132511251 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc3ec056da0 con 0x7fc400194f20 2024-02-16T22:33:36.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:35 smithi123 ceph-mon[82331]: pgmap v331: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:36.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:35 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/2536631179' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T22:33:36.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:35 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/2733041474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T22:33:36.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:35 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/430771862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T22:33:36.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:35 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/1129444274' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T22:33:36.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.175+0000 7fc406d3c640 1 -- 172.21.15.114:0/1132511251 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 13, "format": "json"} v 0) v1 -- 0x7fc3e40050f0 con 0x7fc400194f20 2024-02-16T22:33:36.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.176+0000 7fc3fdffb640 1 -- 172.21.15.114:0/1132511251 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 13, "format": "json"}]=0 dumped fsmap epoch 13 v37) v1 ==== 107+0+4827 (secure 0 0 0) 0x7fc3ec05a460 con 0x7fc400194f20 2024-02-16T22:33:36.178 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:36.178 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14484,"name":"cephfs.smithi114.fdxpey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.114:6829/865692293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":865692293},{"type":"v1","addr":"172.21.15.114:6829","nonce":865692293}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24269,"name":"cephfs.smithi123.vhlqbe","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.123:6825/418014612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":418014612},{"type":"v1","addr":"172.21.15.123:6825","nonce":418014612}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24285,"name":"cephfs.smithi123.gopcfa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6827/2812517585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2812517585},{"type":"v1","addr":"172.21.15.123:6827","nonce":2812517585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:18:02.108825+0000","modified":"2024-02-16T22:18:16.413371+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14468":{"gid":14468,"name":"cephfs.smithi114.zzryke","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6827/277953535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":277953535},{"type":"v1","addr":"172.21.15.114:6827","nonce":277953535}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T22:33:36.179 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 13 2024-02-16T22:33:36.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.178+0000 7fc406d3c640 1 -- 172.21.15.114:0/1132511251 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fc3f006d7c0 msgr2=0x7fc3f006fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:36.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.178+0000 7fc406d3c640 1 --2- 172.21.15.114:0/1132511251 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fc3f006d7c0 0x7fc3f006fc80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4036040 tx=0x7fc3f4005fd0 comp rx=0 tx=0).stop 2024-02-16T22:33:36.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.178+0000 7fc406d3c640 1 -- 172.21.15.114:0/1132511251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc400194f20 msgr2=0x7fc4001953a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:36.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.178+0000 7fc406d3c640 1 --2- 172.21.15.114:0/1132511251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc400194f20 0x7fc4001953a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec005fd0 tx=0x7fc3ec00dd60 comp rx=0 tx=0).stop 2024-02-16T22:33:36.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.178+0000 7fc406d3c640 1 -- 172.21.15.114:0/1132511251 shutdown_connections 2024-02-16T22:33:36.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.178+0000 7fc406d3c640 1 --2- 172.21.15.114:0/1132511251 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fc3f006d7c0 0x7fc3f006fc80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:36.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.178+0000 7fc406d3c640 1 --2- 172.21.15.114:0/1132511251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc400194f20 0x7fc4001953a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:36.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.178+0000 7fc406d3c640 1 --2- 172.21.15.114:0/1132511251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4000ff260 0x7fc4001949e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:36.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.178+0000 7fc406d3c640 1 -- 172.21.15.114:0/1132511251 >> 172.21.15.114:0/1132511251 conn(0x7fc400074a50 msgr2=0x7fc4000fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:36.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.178+0000 7fc406d3c640 1 -- 172.21.15.114:0/1132511251 shutdown_connections 2024-02-16T22:33:36.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.178+0000 7fc406d3c640 1 -- 172.21.15.114:0/1132511251 wait complete. 2024-02-16T22:33:36.191 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 14 2024-02-16T22:33:36.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.291+0000 7ff6fc8a4640 1 -- 172.21.15.114:0/835790145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6f80ffb30 msgr2=0x7ff6f8103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:36.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.291+0000 7ff6fc8a4640 1 --2- 172.21.15.114:0/835790145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6f80ffb30 0x7ff6f8103b70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0009880 tx=0x7ff6e002bce0 comp rx=0 tx=0).stop 2024-02-16T22:33:36.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.291+0000 7ff6fc8a4640 1 -- 172.21.15.114:0/835790145 shutdown_connections 2024-02-16T22:33:36.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.291+0000 7ff6fc8a4640 1 --2- 172.21.15.114:0/835790145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6f80ffb30 0x7ff6f8103b70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:36.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.291+0000 7ff6fc8a4640 1 --2- 172.21.15.114:0/835790145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6f80ff180 0x7ff6f80ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:36.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.291+0000 7ff6fc8a4640 1 -- 172.21.15.114:0/835790145 >> 172.21.15.114:0/835790145 conn(0x7ff6f80faa00 msgr2=0x7ff6f80fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:36.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.291+0000 7ff6fc8a4640 1 -- 172.21.15.114:0/835790145 shutdown_connections 2024-02-16T22:33:36.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.291+0000 7ff6fc8a4640 1 -- 172.21.15.114:0/835790145 wait complete. 2024-02-16T22:33:36.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.292+0000 7ff6fc8a4640 1 Processor -- start 2024-02-16T22:33:36.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.292+0000 7ff6fc8a4640 1 -- start start 2024-02-16T22:33:36.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.292+0000 7ff6fc8a4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6f80ff180 0x7ff6f8196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:36.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.292+0000 7ff6fc8a4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6f8197100 0x7ff6f8197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:36.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.292+0000 7ff6fc8a4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f819fcc0 con 0x7ff6f8197100 2024-02-16T22:33:36.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.292+0000 7ff6fc8a4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f819fe00 con 0x7ff6f80ff180 2024-02-16T22:33:36.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.293+0000 7ff6edd9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6f8197100 0x7ff6f8197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:36.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.293+0000 7ff6edd9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6f8197100 0x7ff6f8197560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47994/0 (socket says 172.21.15.114:47994) 2024-02-16T22:33:36.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.293+0000 7ff6edd9b640 1 -- 172.21.15.114:0/2719845178 learned_addr learned my addr 172.21.15.114:0/2719845178 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:36.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.293+0000 7ff6f659c640 1 --2- 172.21.15.114:0/2719845178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6f80ff180 0x7ff6f8196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:36.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.293+0000 7ff6edd9b640 1 -- 172.21.15.114:0/2719845178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6f80ff180 msgr2=0x7ff6f8196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:36.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.293+0000 7ff6edd9b640 1 --2- 172.21.15.114:0/2719845178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6f80ff180 0x7ff6f8196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:36.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.293+0000 7ff6edd9b640 1 -- 172.21.15.114:0/2719845178 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e0009560 con 0x7ff6f8197100 2024-02-16T22:33:36.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.293+0000 7ff6f659c640 1 --2- 172.21.15.114:0/2719845178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6f80ff180 0x7ff6f8196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T22:33:36.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.293+0000 7ff6edd9b640 1 --2- 172.21.15.114:0/2719845178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6f8197100 0x7ff6f8197560 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0036040 tx=0x7ff6e002c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:36.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.293+0000 7ff6effff640 1 -- 172.21.15.114:0/2719845178 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6e00035f0 con 0x7ff6f8197100 2024-02-16T22:33:36.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.293+0000 7ff6fc8a4640 1 -- 172.21.15.114:0/2719845178 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f819ffa0 con 0x7ff6f8197100 2024-02-16T22:33:36.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.293+0000 7ff6effff640 1 -- 172.21.15.114:0/2719845178 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff6e0003c10 con 0x7ff6f8197100 2024-02-16T22:33:36.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.293+0000 7ff6effff640 1 -- 172.21.15.114:0/2719845178 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6e0005630 con 0x7ff6f8197100 2024-02-16T22:33:36.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.293+0000 7ff6fc8a4640 1 -- 172.21.15.114:0/2719845178 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f81a0470 con 0x7ff6f8197100 2024-02-16T22:33:36.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.294+0000 7ff6fc8a4640 1 -- 172.21.15.114:0/2719845178 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6f804e040 con 0x7ff6f8197100 2024-02-16T22:33:36.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.295+0000 7ff6effff640 1 -- 172.21.15.114:0/2719845178 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7ff6e0003750 con 0x7ff6f8197100 2024-02-16T22:33:36.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.295+0000 7ff6effff640 1 --2- 172.21.15.114:0/2719845178 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff6e406d810 0x7ff6e406fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:36.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.295+0000 7ff6effff640 1 -- 172.21.15.114:0/2719845178 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7ff6e00af900 con 0x7ff6f8197100 2024-02-16T22:33:36.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.295+0000 7ff6f659c640 1 --2- 172.21.15.114:0/2719845178 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff6e406d810 0x7ff6e406fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:36.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.295+0000 7ff6f659c640 1 --2- 172.21.15.114:0/2719845178 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff6e406d810 0x7ff6e406fcd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8009b50 tx=0x7ff6e8005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:36.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.300+0000 7ff6effff640 1 -- 172.21.15.114:0/2719845178 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff6e0079000 con 0x7ff6f8197100 2024-02-16T22:33:36.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.509+0000 7ff6fc8a4640 1 -- 172.21.15.114:0/2719845178 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 14, "format": "json"} v 0) v1 -- 0x7ff6f8107480 con 0x7ff6f8197100 2024-02-16T22:33:36.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.509+0000 7ff6effff640 1 -- 172.21.15.114:0/2719845178 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 14, "format": "json"}]=0 dumped fsmap epoch 14 v37) v1 ==== 107+0+4826 (secure 0 0 0) 0x7ff6e007c6c0 con 0x7ff6f8197100 2024-02-16T22:33:36.511 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:36.511 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14484,"name":"cephfs.smithi114.fdxpey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.114:6829/865692293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":865692293},{"type":"v1","addr":"172.21.15.114:6829","nonce":865692293}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24269,"name":"cephfs.smithi123.vhlqbe","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.123:6825/418014612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":418014612},{"type":"v1","addr":"172.21.15.123:6825","nonce":418014612}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24285,"name":"cephfs.smithi123.gopcfa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.123:6827/2812517585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2812517585},{"type":"v1","addr":"172.21.15.123:6827","nonce":2812517585}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:18:02.108825+0000","modified":"2024-02-16T22:18:16.413371+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14468":{"gid":14468,"name":"cephfs.smithi114.zzryke","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6827/277953535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":277953535},{"type":"v1","addr":"172.21.15.114:6827","nonce":277953535}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T22:33:36.512 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 14 2024-02-16T22:33:36.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.511+0000 7ff6fc8a4640 1 -- 172.21.15.114:0/2719845178 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff6e406d810 msgr2=0x7ff6e406fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:36.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.511+0000 7ff6fc8a4640 1 --2- 172.21.15.114:0/2719845178 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff6e406d810 0x7ff6e406fcd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8009b50 tx=0x7ff6e8005f90 comp rx=0 tx=0).stop 2024-02-16T22:33:36.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.511+0000 7ff6fc8a4640 1 -- 172.21.15.114:0/2719845178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6f8197100 msgr2=0x7ff6f8197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:36.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.511+0000 7ff6fc8a4640 1 --2- 172.21.15.114:0/2719845178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6f8197100 0x7ff6f8197560 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0036040 tx=0x7ff6e002c750 comp rx=0 tx=0).stop 2024-02-16T22:33:36.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.511+0000 7ff6fc8a4640 1 -- 172.21.15.114:0/2719845178 shutdown_connections 2024-02-16T22:33:36.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.511+0000 7ff6fc8a4640 1 --2- 172.21.15.114:0/2719845178 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff6e406d810 0x7ff6e406fcd0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:36.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.511+0000 7ff6fc8a4640 1 --2- 172.21.15.114:0/2719845178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6f8197100 0x7ff6f8197560 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:36.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.511+0000 7ff6fc8a4640 1 --2- 172.21.15.114:0/2719845178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6f80ff180 0x7ff6f8196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:36.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.511+0000 7ff6fc8a4640 1 -- 172.21.15.114:0/2719845178 >> 172.21.15.114:0/2719845178 conn(0x7ff6f80faa00 msgr2=0x7ff6f80fc150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:36.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.512+0000 7ff6fc8a4640 1 -- 172.21.15.114:0/2719845178 shutdown_connections 2024-02-16T22:33:36.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.512+0000 7ff6fc8a4640 1 -- 172.21.15.114:0/2719845178 wait complete. 2024-02-16T22:33:36.525 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 15 2024-02-16T22:33:36.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.625+0000 7fac5e03a640 1 -- 172.21.15.114:0/2413635164 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac580ff260 msgr2=0x7fac580ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:36.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.625+0000 7fac5e03a640 1 --2- 172.21.15.114:0/2413635164 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac580ff260 0x7fac580ff640 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fac48009880 tx=0x7fac4802bce0 comp rx=0 tx=0).stop 2024-02-16T22:33:36.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.625+0000 7fac5e03a640 1 -- 172.21.15.114:0/2413635164 shutdown_connections 2024-02-16T22:33:36.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.625+0000 7fac5e03a640 1 --2- 172.21.15.114:0/2413635164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac580ffc10 0x7fac58103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:36.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.625+0000 7fac5e03a640 1 --2- 172.21.15.114:0/2413635164 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac580ff260 0x7fac580ff640 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:36.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.625+0000 7fac5e03a640 1 -- 172.21.15.114:0/2413635164 >> 172.21.15.114:0/2413635164 conn(0x7fac58074a50 msgr2=0x7fac58074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:36.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.626+0000 7fac5e03a640 1 -- 172.21.15.114:0/2413635164 shutdown_connections 2024-02-16T22:33:36.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.626+0000 7fac5e03a640 1 -- 172.21.15.114:0/2413635164 wait complete. 2024-02-16T22:33:36.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.626+0000 7fac5e03a640 1 Processor -- start 2024-02-16T22:33:36.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.626+0000 7fac5e03a640 1 -- start start 2024-02-16T22:33:36.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.626+0000 7fac5e03a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac580ff260 0x7fac58196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:36.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.626+0000 7fac5e03a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac58197130 0x7fac58197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:36.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.626+0000 7fac5e03a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5819fc60 con 0x7fac580ff260 2024-02-16T22:33:36.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.626+0000 7fac5e03a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5819fda0 con 0x7fac58197130 2024-02-16T22:33:36.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.627+0000 7fac577fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac580ff260 0x7fac58196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:36.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.627+0000 7fac577fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac580ff260 0x7fac58196bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48022/0 (socket says 172.21.15.114:48022) 2024-02-16T22:33:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.627+0000 7fac577fe640 1 -- 172.21.15.114:0/1049402032 learned_addr learned my addr 172.21.15.114:0/1049402032 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.627+0000 7fac4effd640 1 --2- 172.21.15.114:0/1049402032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac58197130 0x7fac58197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.627+0000 7fac577fe640 1 -- 172.21.15.114:0/1049402032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac58197130 msgr2=0x7fac58197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.627+0000 7fac577fe640 1 --2- 172.21.15.114:0/1049402032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac58197130 0x7fac58197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.627+0000 7fac577fe640 1 -- 172.21.15.114:0/1049402032 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac48009560 con 0x7fac580ff260 2024-02-16T22:33:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.627+0000 7fac4effd640 1 --2- 172.21.15.114:0/1049402032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac58197130 0x7fac58197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T22:33:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.627+0000 7fac577fe640 1 --2- 172.21.15.114:0/1049402032 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac580ff260 0x7fac58196bf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fac48009430 tx=0x7fac48004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.627+0000 7fac557fa640 1 -- 172.21.15.114:0/1049402032 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac480035f0 con 0x7fac580ff260 2024-02-16T22:33:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.627+0000 7fac557fa640 1 -- 172.21.15.114:0/1049402032 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac48003c10 con 0x7fac580ff260 2024-02-16T22:33:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.627+0000 7fac5e03a640 1 -- 172.21.15.114:0/1049402032 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac5819ffa0 con 0x7fac580ff260 2024-02-16T22:33:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.627+0000 7fac557fa640 1 -- 172.21.15.114:0/1049402032 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac48034440 con 0x7fac580ff260 2024-02-16T22:33:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.627+0000 7fac5e03a640 1 -- 172.21.15.114:0/1049402032 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac581a03d0 con 0x7fac580ff260 2024-02-16T22:33:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.628+0000 7fac5e03a640 1 -- 172.21.15.114:0/1049402032 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac5804e040 con 0x7fac580ff260 2024-02-16T22:33:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.628+0000 7fac557fa640 1 -- 172.21.15.114:0/1049402032 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fac48004fd0 con 0x7fac580ff260 2024-02-16T22:33:36.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.629+0000 7fac557fa640 1 --2- 172.21.15.114:0/1049402032 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fac4406d810 0x7fac4406fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:36.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.629+0000 7fac557fa640 1 -- 172.21.15.114:0/1049402032 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7fac480afa90 con 0x7fac580ff260 2024-02-16T22:33:36.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.629+0000 7fac4effd640 1 --2- 172.21.15.114:0/1049402032 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fac4406d810 0x7fac4406fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:36.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.629+0000 7fac4effd640 1 --2- 172.21.15.114:0/1049402032 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fac4406d810 0x7fac4406fcd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fac400058c0 tx=0x7fac40005d30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:36.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.633+0000 7fac557fa640 1 -- 172.21.15.114:0/1049402032 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fac48079190 con 0x7fac580ff260 2024-02-16T22:33:36.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.843+0000 7fac5e03a640 1 -- 172.21.15.114:0/1049402032 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 15, "format": "json"} v 0) v1 -- 0x7fac580ffcb0 con 0x7fac580ff260 2024-02-16T22:33:36.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.843+0000 7fac557fa640 1 -- 172.21.15.114:0/1049402032 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 15, "format": "json"}]=0 dumped fsmap epoch 15 v37) v1 ==== 107+0+4038 (secure 0 0 0) 0x7fac4807c850 con 0x7fac580ff260 2024-02-16T22:33:36.845 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:36.845 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14484,"name":"cephfs.smithi114.fdxpey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.114:6829/865692293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":865692293},{"type":"v1","addr":"172.21.15.114:6829","nonce":865692293}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24269,"name":"cephfs.smithi123.vhlqbe","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.123:6825/418014612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":418014612},{"type":"v1","addr":"172.21.15.123:6825","nonce":418014612}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24285,"name":"cephfs.smithi123.gopcfa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.123:6827/2812517585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2812517585},{"type":"v1","addr":"172.21.15.123:6827","nonce":2812517585}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:18:02.108825+0000","modified":"2024-02-16T22:28:21.203539+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T22:33:36.846 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 15 2024-02-16T22:33:36.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.845+0000 7fac5e03a640 1 -- 172.21.15.114:0/1049402032 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fac4406d810 msgr2=0x7fac4406fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:36.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.845+0000 7fac5e03a640 1 --2- 172.21.15.114:0/1049402032 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fac4406d810 0x7fac4406fcd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fac400058c0 tx=0x7fac40005d30 comp rx=0 tx=0).stop 2024-02-16T22:33:36.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.845+0000 7fac5e03a640 1 -- 172.21.15.114:0/1049402032 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac580ff260 msgr2=0x7fac58196bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:36.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.845+0000 7fac5e03a640 1 --2- 172.21.15.114:0/1049402032 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac580ff260 0x7fac58196bf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fac48009430 tx=0x7fac48004040 comp rx=0 tx=0).stop 2024-02-16T22:33:36.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.845+0000 7fac5e03a640 1 -- 172.21.15.114:0/1049402032 shutdown_connections 2024-02-16T22:33:36.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.845+0000 7fac5e03a640 1 --2- 172.21.15.114:0/1049402032 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fac4406d810 0x7fac4406fcd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:36.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.845+0000 7fac5e03a640 1 --2- 172.21.15.114:0/1049402032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac58197130 0x7fac58197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:36.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.845+0000 7fac5e03a640 1 --2- 172.21.15.114:0/1049402032 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac580ff260 0x7fac58196bf0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:36.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.845+0000 7fac5e03a640 1 -- 172.21.15.114:0/1049402032 >> 172.21.15.114:0/1049402032 conn(0x7fac58074a50 msgr2=0x7fac580fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:36.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.845+0000 7fac5e03a640 1 -- 172.21.15.114:0/1049402032 shutdown_connections 2024-02-16T22:33:36.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.845+0000 7fac5e03a640 1 -- 172.21.15.114:0/1049402032 wait complete. 2024-02-16T22:33:36.857 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:36 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/1263229626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-16T22:33:36.857 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:36 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/1132511251' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 13, "format": "json"}]: dispatch 2024-02-16T22:33:36.857 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:36 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/2719845178' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 14, "format": "json"}]: dispatch 2024-02-16T22:33:36.858 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 16 2024-02-16T22:33:36.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.958+0000 7f10a81dd640 1 -- 172.21.15.114:0/951529930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10a0100c50 msgr2=0x7f10a0104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:36.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.958+0000 7f10a81dd640 1 --2- 172.21.15.114:0/951529930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10a0100c50 0x7f10a0104c90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1090009880 tx=0x7f109002bce0 comp rx=0 tx=0).stop 2024-02-16T22:33:36.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.958+0000 7f10a81dd640 1 -- 172.21.15.114:0/951529930 shutdown_connections 2024-02-16T22:33:36.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.958+0000 7f10a81dd640 1 --2- 172.21.15.114:0/951529930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10a0100c50 0x7f10a0104c90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:36.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.958+0000 7f10a81dd640 1 --2- 172.21.15.114:0/951529930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10a01002a0 0x7f10a0100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:36.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.958+0000 7f10a81dd640 1 -- 172.21.15.114:0/951529930 >> 172.21.15.114:0/951529930 conn(0x7f10a00fbb20 msgr2=0x7f10a00fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:36.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.958+0000 7f10a81dd640 1 -- 172.21.15.114:0/951529930 shutdown_connections 2024-02-16T22:33:36.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.958+0000 7f10a81dd640 1 -- 172.21.15.114:0/951529930 wait complete. 2024-02-16T22:33:36.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.959+0000 7f10a81dd640 1 Processor -- start 2024-02-16T22:33:36.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.959+0000 7f10a81dd640 1 -- start start 2024-02-16T22:33:36.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.959+0000 7f10a81dd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10a01002a0 0x7f10a0195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:36.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.959+0000 7f10a81dd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10a01962c0 0x7f10a019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:36.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.959+0000 7f10a81dd640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a01968c0 con 0x7f10a01002a0 2024-02-16T22:33:36.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.959+0000 7f10a81dd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a0196a30 con 0x7f10a01962c0 2024-02-16T22:33:36.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.959+0000 7f10a5f79640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10a01002a0 0x7f10a0195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:36.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.959+0000 7f10a5f79640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10a01002a0 0x7f10a0195d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48036/0 (socket says 172.21.15.114:48036) 2024-02-16T22:33:36.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.959+0000 7f10a5f79640 1 -- 172.21.15.114:0/2566502104 learned_addr learned my addr 172.21.15.114:0/2566502104 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:36.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.959+0000 7f10a5778640 1 --2- 172.21.15.114:0/2566502104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10a01962c0 0x7f10a019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:36.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.959+0000 7f10a5f79640 1 -- 172.21.15.114:0/2566502104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10a01962c0 msgr2=0x7f10a019a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:36.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.959+0000 7f10a5f79640 1 --2- 172.21.15.114:0/2566502104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10a01962c0 0x7f10a019a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:36.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.959+0000 7f10a5f79640 1 -- 172.21.15.114:0/2566502104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1090009560 con 0x7f10a01002a0 2024-02-16T22:33:36.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.959+0000 7f10a5f79640 1 --2- 172.21.15.114:0/2566502104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10a01002a0 0x7f10a0195d80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f109800ea30 tx=0x7f109800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:36.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.959+0000 7f1096ffd640 1 -- 172.21.15.114:0/2566502104 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f109800cdb0 con 0x7f10a01002a0 2024-02-16T22:33:36.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.959+0000 7f1096ffd640 1 -- 172.21.15.114:0/2566502104 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1098005020 con 0x7f10a01002a0 2024-02-16T22:33:36.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.959+0000 7f10a81dd640 1 -- 172.21.15.114:0/2566502104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a019ad20 con 0x7f10a01002a0 2024-02-16T22:33:36.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.959+0000 7f1096ffd640 1 -- 172.21.15.114:0/2566502104 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10980116a0 con 0x7f10a01002a0 2024-02-16T22:33:36.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.959+0000 7f10a81dd640 1 -- 172.21.15.114:0/2566502104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a019b280 con 0x7f10a01002a0 2024-02-16T22:33:36.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.960+0000 7f10a81dd640 1 -- 172.21.15.114:0/2566502104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a004e040 con 0x7f10a01002a0 2024-02-16T22:33:36.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.961+0000 7f1096ffd640 1 -- 172.21.15.114:0/2566502104 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f1098005190 con 0x7f10a01002a0 2024-02-16T22:33:36.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.961+0000 7f1096ffd640 1 --2- 172.21.15.114:0/2566502104 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f108c06d810 0x7f108c06fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:36.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.962+0000 7f10a5778640 1 --2- 172.21.15.114:0/2566502104 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f108c06d810 0x7f108c06fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:36.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.962+0000 7f1096ffd640 1 -- 172.21.15.114:0/2566502104 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f109808d920 con 0x7f10a01002a0 2024-02-16T22:33:36.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.962+0000 7f10a5778640 1 --2- 172.21.15.114:0/2566502104 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f108c06d810 0x7f108c06fcd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1090000c00 tx=0x7f1090005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:36.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:36.966+0000 7f1096ffd640 1 -- 172.21.15.114:0/2566502104 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1098057150 con 0x7f10a01002a0 2024-02-16T22:33:37.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:36 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/1263229626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-16T22:33:37.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:36 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/1132511251' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 13, "format": "json"}]: dispatch 2024-02-16T22:33:37.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:36 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/2719845178' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 14, "format": "json"}]: dispatch 2024-02-16T22:33:37.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.179+0000 7f10a81dd640 1 -- 172.21.15.114:0/2566502104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 16, "format": "json"} v 0) v1 -- 0x7f10a01970d0 con 0x7f10a01002a0 2024-02-16T22:33:37.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.179+0000 7f1096ffd640 1 -- 172.21.15.114:0/2566502104 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 16, "format": "json"}]=0 dumped fsmap epoch 16 v37) v1 ==== 107+0+4049 (secure 0 0 0) 0x7f109805a810 con 0x7f10a01002a0 2024-02-16T22:33:37.181 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:37.181 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24269,"name":"cephfs.smithi123.vhlqbe","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.123:6825/418014612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":418014612},{"type":"v1","addr":"172.21.15.123:6825","nonce":418014612}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24285,"name":"cephfs.smithi123.gopcfa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.123:6827/2812517585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2812517585},{"type":"v1","addr":"172.21.15.123:6827","nonce":2812517585}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:18:02.108825+0000","modified":"2024-02-16T22:28:21.207983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14484":{"gid":14484,"name":"cephfs.smithi114.fdxpey","rank":0,"incarnation":16,"state":"up:replay","state_seq":2,"addr":"172.21.15.114:6829/865692293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":865692293},{"type":"v1","addr":"172.21.15.114:6829","nonce":865692293}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T22:33:37.181 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 16 2024-02-16T22:33:37.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.181+0000 7f10a81dd640 1 -- 172.21.15.114:0/2566502104 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f108c06d810 msgr2=0x7f108c06fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:37.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.181+0000 7f10a81dd640 1 --2- 172.21.15.114:0/2566502104 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f108c06d810 0x7f108c06fcd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1090000c00 tx=0x7f1090005fd0 comp rx=0 tx=0).stop 2024-02-16T22:33:37.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.181+0000 7f10a81dd640 1 -- 172.21.15.114:0/2566502104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10a01002a0 msgr2=0x7f10a0195d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:37.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.181+0000 7f10a81dd640 1 --2- 172.21.15.114:0/2566502104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10a01002a0 0x7f10a0195d80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f109800ea30 tx=0x7f109800eef0 comp rx=0 tx=0).stop 2024-02-16T22:33:37.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.181+0000 7f10a81dd640 1 -- 172.21.15.114:0/2566502104 shutdown_connections 2024-02-16T22:33:37.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.181+0000 7f10a81dd640 1 --2- 172.21.15.114:0/2566502104 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f108c06d810 0x7f108c06fcd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:37.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.181+0000 7f10a81dd640 1 --2- 172.21.15.114:0/2566502104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10a01962c0 0x7f10a019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:37.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.181+0000 7f10a81dd640 1 --2- 172.21.15.114:0/2566502104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10a01002a0 0x7f10a0195d80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:37.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.181+0000 7f10a81dd640 1 -- 172.21.15.114:0/2566502104 >> 172.21.15.114:0/2566502104 conn(0x7f10a00fbb20 msgr2=0x7f10a00fd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:37.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.181+0000 7f10a81dd640 1 -- 172.21.15.114:0/2566502104 shutdown_connections 2024-02-16T22:33:37.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.181+0000 7f10a81dd640 1 -- 172.21.15.114:0/2566502104 wait complete. 2024-02-16T22:33:37.194 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 17 2024-02-16T22:33:37.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.295+0000 7fa0bd69d640 1 -- 172.21.15.114:0/1522236253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0b8100c20 msgr2=0x7fa0b8104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:37.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.295+0000 7fa0bd69d640 1 --2- 172.21.15.114:0/1522236253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0b8100c20 0x7fa0b8104c60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8009880 tx=0x7fa0a802bce0 comp rx=0 tx=0).stop 2024-02-16T22:33:37.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.295+0000 7fa0bd69d640 1 -- 172.21.15.114:0/1522236253 shutdown_connections 2024-02-16T22:33:37.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.295+0000 7fa0bd69d640 1 --2- 172.21.15.114:0/1522236253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0b8100c20 0x7fa0b8104c60 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:37.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.295+0000 7fa0bd69d640 1 --2- 172.21.15.114:0/1522236253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0b8100270 0x7fa0b8100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:37.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.295+0000 7fa0bd69d640 1 -- 172.21.15.114:0/1522236253 >> 172.21.15.114:0/1522236253 conn(0x7fa0b80fbaf0 msgr2=0x7fa0b80fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:37.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.295+0000 7fa0bd69d640 1 -- 172.21.15.114:0/1522236253 shutdown_connections 2024-02-16T22:33:37.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.295+0000 7fa0bd69d640 1 -- 172.21.15.114:0/1522236253 wait complete. 2024-02-16T22:33:37.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.295+0000 7fa0bd69d640 1 Processor -- start 2024-02-16T22:33:37.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.295+0000 7fa0bd69d640 1 -- start start 2024-02-16T22:33:37.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.296+0000 7fa0bd69d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0b8100270 0x7fa0b8195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:37.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.296+0000 7fa0bd69d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0b8196330 0x7fa0b819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:37.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.296+0000 7fa0bd69d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b8196930 con 0x7fa0b8100270 2024-02-16T22:33:37.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.296+0000 7fa0bd69d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b8196aa0 con 0x7fa0b8196330 2024-02-16T22:33:37.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.296+0000 7fa0b6ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0b8100270 0x7fa0b8195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:37.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.296+0000 7fa0b6ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0b8100270 0x7fa0b8195df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48046/0 (socket says 172.21.15.114:48046) 2024-02-16T22:33:37.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.296+0000 7fa0b6ffd640 1 -- 172.21.15.114:0/3542102038 learned_addr learned my addr 172.21.15.114:0/3542102038 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:37.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.296+0000 7fa0b6ffd640 1 -- 172.21.15.114:0/3542102038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0b8196330 msgr2=0x7fa0b819a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:37.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.296+0000 7fa0b6ffd640 1 --2- 172.21.15.114:0/3542102038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0b8196330 0x7fa0b819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:37.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.296+0000 7fa0b6ffd640 1 -- 172.21.15.114:0/3542102038 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a8009560 con 0x7fa0b8100270 2024-02-16T22:33:37.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.296+0000 7fa0b6ffd640 1 --2- 172.21.15.114:0/3542102038 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0b8100270 0x7fa0b8195df0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00c840 tx=0x7fa0ac00cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:37.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.296+0000 7fa09ffff640 1 -- 172.21.15.114:0/3542102038 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0ac0041f0 con 0x7fa0b8100270 2024-02-16T22:33:37.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.296+0000 7fa09ffff640 1 -- 172.21.15.114:0/3542102038 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa0ac007e70 con 0x7fa0b8100270 2024-02-16T22:33:37.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.296+0000 7fa09ffff640 1 -- 172.21.15.114:0/3542102038 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0ac0049d0 con 0x7fa0b8100270 2024-02-16T22:33:37.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.296+0000 7fa0bd69d640 1 -- 172.21.15.114:0/3542102038 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b819ad90 con 0x7fa0b8100270 2024-02-16T22:33:37.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.296+0000 7fa0bd69d640 1 -- 172.21.15.114:0/3542102038 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b819b2f0 con 0x7fa0b8100270 2024-02-16T22:33:37.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.297+0000 7fa0bd69d640 1 -- 172.21.15.114:0/3542102038 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b804e040 con 0x7fa0b8100270 2024-02-16T22:33:37.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.298+0000 7fa09ffff640 1 -- 172.21.15.114:0/3542102038 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fa0ac0079b0 con 0x7fa0b8100270 2024-02-16T22:33:37.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.298+0000 7fa09ffff640 1 --2- 172.21.15.114:0/3542102038 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fa0a006d810 0x7fa0a006fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:37.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.298+0000 7fa09ffff640 1 -- 172.21.15.114:0/3542102038 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7fa0ac08d3b0 con 0x7fa0b8100270 2024-02-16T22:33:37.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.298+0000 7fa0b67fc640 1 --2- 172.21.15.114:0/3542102038 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fa0a006d810 0x7fa0a006fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:37.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.299+0000 7fa0b67fc640 1 --2- 172.21.15.114:0/3542102038 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fa0a006d810 0x7fa0a006fcd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8036040 tx=0x7fa0a8000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:37.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.303+0000 7fa09ffff640 1 -- 172.21.15.114:0/3542102038 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa0ac056b30 con 0x7fa0b8100270 2024-02-16T22:33:37.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.513+0000 7fa0bd69d640 1 -- 172.21.15.114:0/3542102038 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 17, "format": "json"} v 0) v1 -- 0x7fa0b8197170 con 0x7fa0b8100270 2024-02-16T22:33:37.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.514+0000 7fa09ffff640 1 -- 172.21.15.114:0/3542102038 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 17, "format": "json"}]=0 dumped fsmap epoch 17 v37) v1 ==== 107+0+4865 (secure 0 0 0) 0x7fa0ac05a1f0 con 0x7fa0b8100270 2024-02-16T22:33:37.516 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:37.516 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24269,"name":"cephfs.smithi123.vhlqbe","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.123:6825/418014612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":418014612},{"type":"v1","addr":"172.21.15.123:6825","nonce":418014612}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24285,"name":"cephfs.smithi123.gopcfa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.123:6827/2812517585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2812517585},{"type":"v1","addr":"172.21.15.123:6827","nonce":2812517585}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34404,"name":"cephfs.smithi114.zzryke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6827/2017800151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2017800151},{"type":"v1","addr":"172.21.15.114:6827","nonce":2017800151}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:18:02.108825+0000","modified":"2024-02-16T22:28:21.207983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14484":{"gid":14484,"name":"cephfs.smithi114.fdxpey","rank":0,"incarnation":16,"state":"up:replay","state_seq":2,"addr":"172.21.15.114:6829/865692293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":865692293},{"type":"v1","addr":"172.21.15.114:6829","nonce":865692293}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T22:33:37.516 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 17 2024-02-16T22:33:37.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.516+0000 7fa0bd69d640 1 -- 172.21.15.114:0/3542102038 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fa0a006d810 msgr2=0x7fa0a006fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:37.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.516+0000 7fa0bd69d640 1 --2- 172.21.15.114:0/3542102038 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fa0a006d810 0x7fa0a006fcd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8036040 tx=0x7fa0a8000bc0 comp rx=0 tx=0).stop 2024-02-16T22:33:37.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.516+0000 7fa0bd69d640 1 -- 172.21.15.114:0/3542102038 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0b8100270 msgr2=0x7fa0b8195df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:37.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.516+0000 7fa0bd69d640 1 --2- 172.21.15.114:0/3542102038 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0b8100270 0x7fa0b8195df0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00c840 tx=0x7fa0ac00cd00 comp rx=0 tx=0).stop 2024-02-16T22:33:37.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.516+0000 7fa0bd69d640 1 -- 172.21.15.114:0/3542102038 shutdown_connections 2024-02-16T22:33:37.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.516+0000 7fa0bd69d640 1 --2- 172.21.15.114:0/3542102038 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fa0a006d810 0x7fa0a006fcd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:37.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.516+0000 7fa0bd69d640 1 --2- 172.21.15.114:0/3542102038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0b8196330 0x7fa0b819a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:37.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.516+0000 7fa0bd69d640 1 --2- 172.21.15.114:0/3542102038 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0b8100270 0x7fa0b8195df0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:37.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.516+0000 7fa0bd69d640 1 -- 172.21.15.114:0/3542102038 >> 172.21.15.114:0/3542102038 conn(0x7fa0b80fbaf0 msgr2=0x7fa0b80fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:37.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.516+0000 7fa0bd69d640 1 -- 172.21.15.114:0/3542102038 shutdown_connections 2024-02-16T22:33:37.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.516+0000 7fa0bd69d640 1 -- 172.21.15.114:0/3542102038 wait complete. 2024-02-16T22:33:37.529 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-02-16T22:33:37.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.630+0000 7fb20ed8b640 1 -- 172.21.15.114:0/269054535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2080ffc10 msgr2=0x7fb208103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:37.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.631+0000 7fb20ed8b640 1 --2- 172.21.15.114:0/269054535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2080ffc10 0x7fb208103b70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009880 tx=0x7fb1f802bce0 comp rx=0 tx=0).stop 2024-02-16T22:33:37.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.631+0000 7fb20ed8b640 1 -- 172.21.15.114:0/269054535 shutdown_connections 2024-02-16T22:33:37.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.631+0000 7fb20ed8b640 1 --2- 172.21.15.114:0/269054535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2080ffc10 0x7fb208103b70 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:37.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.631+0000 7fb20ed8b640 1 --2- 172.21.15.114:0/269054535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2080ff260 0x7fb2080ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:37.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.631+0000 7fb20ed8b640 1 -- 172.21.15.114:0/269054535 >> 172.21.15.114:0/269054535 conn(0x7fb208074a50 msgr2=0x7fb208074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:37.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.631+0000 7fb20ed8b640 1 -- 172.21.15.114:0/269054535 shutdown_connections 2024-02-16T22:33:37.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.631+0000 7fb20ed8b640 1 -- 172.21.15.114:0/269054535 wait complete. 2024-02-16T22:33:37.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.631+0000 7fb20ed8b640 1 Processor -- start 2024-02-16T22:33:37.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.631+0000 7fb20ed8b640 1 -- start start 2024-02-16T22:33:37.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.631+0000 7fb20ed8b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2080ff260 0x7fb208196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:37.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.631+0000 7fb20ed8b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2081970b0 0x7fb208197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:37.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.631+0000 7fb20ed8b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20819fbe0 con 0x7fb2081970b0 2024-02-16T22:33:37.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.631+0000 7fb20ed8b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20819fd20 con 0x7fb2080ff260 2024-02-16T22:33:37.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.632+0000 7fb1fffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2081970b0 0x7fb208197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:37.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.632+0000 7fb1fffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2081970b0 0x7fb208197510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48066/0 (socket says 172.21.15.114:48066) 2024-02-16T22:33:37.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.632+0000 7fb1fffff640 1 -- 172.21.15.114:0/3927067341 learned_addr learned my addr 172.21.15.114:0/3927067341 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:37.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.632+0000 7fb20cb27640 1 --2- 172.21.15.114:0/3927067341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2080ff260 0x7fb208196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:37.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.632+0000 7fb1fffff640 1 -- 172.21.15.114:0/3927067341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2080ff260 msgr2=0x7fb208196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:37.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.632+0000 7fb1fffff640 1 --2- 172.21.15.114:0/3927067341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2080ff260 0x7fb208196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:37.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.632+0000 7fb1fffff640 1 -- 172.21.15.114:0/3927067341 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f8009560 con 0x7fb2081970b0 2024-02-16T22:33:37.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.632+0000 7fb1fffff640 1 --2- 172.21.15.114:0/3927067341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2081970b0 0x7fb208197510 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8036040 tx=0x7fb1f802c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:37.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.632+0000 7fb1fdffb640 1 -- 172.21.15.114:0/3927067341 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1f80035f0 con 0x7fb2081970b0 2024-02-16T22:33:37.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.632+0000 7fb1fdffb640 1 -- 172.21.15.114:0/3927067341 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb1f8003c10 con 0x7fb2081970b0 2024-02-16T22:33:37.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.632+0000 7fb20ed8b640 1 -- 172.21.15.114:0/3927067341 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb20819ff20 con 0x7fb2081970b0 2024-02-16T22:33:37.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.632+0000 7fb1fdffb640 1 -- 172.21.15.114:0/3927067341 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1f8034440 con 0x7fb2081970b0 2024-02-16T22:33:37.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.633+0000 7fb20ed8b640 1 -- 172.21.15.114:0/3927067341 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2081a03a0 con 0x7fb2081970b0 2024-02-16T22:33:37.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.634+0000 7fb20ed8b640 1 -- 172.21.15.114:0/3927067341 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb20804e040 con 0x7fb2081970b0 2024-02-16T22:33:37.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.634+0000 7fb1fdffb640 1 -- 172.21.15.114:0/3927067341 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fb1f8003750 con 0x7fb2081970b0 2024-02-16T22:33:37.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.635+0000 7fb1fdffb640 1 --2- 172.21.15.114:0/3927067341 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb1f006d810 0x7fb1f006fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:37.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.635+0000 7fb1fdffb640 1 -- 172.21.15.114:0/3927067341 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7fb1f80af930 con 0x7fb2081970b0 2024-02-16T22:33:37.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.635+0000 7fb20cb27640 1 --2- 172.21.15.114:0/3927067341 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb1f006d810 0x7fb1f006fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:37.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.635+0000 7fb20cb27640 1 --2- 172.21.15.114:0/3927067341 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb1f006d810 0x7fb1f006fcd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb204005d00 tx=0x7fb20400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:37.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.639+0000 7fb1fdffb640 1 -- 172.21.15.114:0/3927067341 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb1f80790e0 con 0x7fb2081970b0 2024-02-16T22:33:37.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.865+0000 7fb20ed8b640 1 -- 172.21.15.114:0/3927067341 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7fb208107480 con 0x7fb2081970b0 2024-02-16T22:33:37.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.866+0000 7fb1fdffb640 1 -- 172.21.15.114:0/3927067341 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v37) v1 ==== 107+0+4870 (secure 0 0 0) 0x7fb1f807c7a0 con 0x7fb2081970b0 2024-02-16T22:33:37.868 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:37.868 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24269,"name":"cephfs.smithi123.vhlqbe","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.123:6825/418014612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":418014612},{"type":"v1","addr":"172.21.15.123:6825","nonce":418014612}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24285,"name":"cephfs.smithi123.gopcfa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.123:6827/2812517585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2812517585},{"type":"v1","addr":"172.21.15.123:6827","nonce":2812517585}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34404,"name":"cephfs.smithi114.zzryke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6827/2017800151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2017800151},{"type":"v1","addr":"172.21.15.114:6827","nonce":2017800151}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:18:02.108825+0000","modified":"2024-02-16T22:28:31.423900+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14484":{"gid":14484,"name":"cephfs.smithi114.fdxpey","rank":0,"incarnation":16,"state":"up:reconnect","state_seq":157,"addr":"172.21.15.114:6829/865692293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":865692293},{"type":"v1","addr":"172.21.15.114:6829","nonce":865692293}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T22:33:37.868 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 18 2024-02-16T22:33:37.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.868+0000 7fb20ed8b640 1 -- 172.21.15.114:0/3927067341 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb1f006d810 msgr2=0x7fb1f006fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:37.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.868+0000 7fb20ed8b640 1 --2- 172.21.15.114:0/3927067341 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb1f006d810 0x7fb1f006fcd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb204005d00 tx=0x7fb20400b410 comp rx=0 tx=0).stop 2024-02-16T22:33:37.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.868+0000 7fb20ed8b640 1 -- 172.21.15.114:0/3927067341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2081970b0 msgr2=0x7fb208197510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:37.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.868+0000 7fb20ed8b640 1 --2- 172.21.15.114:0/3927067341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2081970b0 0x7fb208197510 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8036040 tx=0x7fb1f802c850 comp rx=0 tx=0).stop 2024-02-16T22:33:37.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.868+0000 7fb20ed8b640 1 -- 172.21.15.114:0/3927067341 shutdown_connections 2024-02-16T22:33:37.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.868+0000 7fb20ed8b640 1 --2- 172.21.15.114:0/3927067341 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb1f006d810 0x7fb1f006fcd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:37.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.868+0000 7fb20ed8b640 1 --2- 172.21.15.114:0/3927067341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2081970b0 0x7fb208197510 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:37.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.868+0000 7fb20ed8b640 1 --2- 172.21.15.114:0/3927067341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2080ff260 0x7fb208196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:37.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.869+0000 7fb20ed8b640 1 -- 172.21.15.114:0/3927067341 >> 172.21.15.114:0/3927067341 conn(0x7fb208074a50 msgr2=0x7fb2080fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:37.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.869+0000 7fb20ed8b640 1 -- 172.21.15.114:0/3927067341 shutdown_connections 2024-02-16T22:33:37.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.869+0000 7fb20ed8b640 1 -- 172.21.15.114:0/3927067341 wait complete. 2024-02-16T22:33:37.880 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:37 smithi114 ceph-mon[94452]: pgmap v332: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:37.881 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:37 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/1049402032' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 15, "format": "json"}]: dispatch 2024-02-16T22:33:37.881 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:37 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/2566502104' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-02-16T22:33:37.881 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:37 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/3542102038' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-02-16T22:33:37.881 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:37 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:33:37.882 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-02-16T22:33:37.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.984+0000 7f7ba849d640 1 -- 172.21.15.114:0/2348348148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ba00ffc10 msgr2=0x7f7ba0103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:37.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.984+0000 7f7ba849d640 1 --2- 172.21.15.114:0/2348348148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ba00ffc10 0x7f7ba0103b70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009880 tx=0x7f7b9002bce0 comp rx=0 tx=0).stop 2024-02-16T22:33:37.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.984+0000 7f7ba849d640 1 -- 172.21.15.114:0/2348348148 shutdown_connections 2024-02-16T22:33:37.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.984+0000 7f7ba849d640 1 --2- 172.21.15.114:0/2348348148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ba00ffc10 0x7f7ba0103b70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:37.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.984+0000 7f7ba849d640 1 --2- 172.21.15.114:0/2348348148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ba00ff260 0x7f7ba00ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:37.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.984+0000 7f7ba849d640 1 -- 172.21.15.114:0/2348348148 >> 172.21.15.114:0/2348348148 conn(0x7f7ba0074a50 msgr2=0x7f7ba0074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:37.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.984+0000 7f7ba849d640 1 -- 172.21.15.114:0/2348348148 shutdown_connections 2024-02-16T22:33:37.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.984+0000 7f7ba849d640 1 -- 172.21.15.114:0/2348348148 wait complete. 2024-02-16T22:33:37.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.985+0000 7f7ba849d640 1 Processor -- start 2024-02-16T22:33:37.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.985+0000 7f7ba849d640 1 -- start start 2024-02-16T22:33:37.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.985+0000 7f7ba849d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ba00ff260 0x7f7ba0196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:37.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.985+0000 7f7ba849d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ba01970f0 0x7f7ba0197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:37.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.985+0000 7f7ba849d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba019fc20 con 0x7f7ba01970f0 2024-02-16T22:33:37.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.985+0000 7f7ba849d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba019fd60 con 0x7f7ba00ff260 2024-02-16T22:33:37.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.985+0000 7f7ba5a38640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ba01970f0 0x7f7ba0197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:37.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.985+0000 7f7ba5a38640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ba01970f0 0x7f7ba0197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48092/0 (socket says 172.21.15.114:48092) 2024-02-16T22:33:37.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.985+0000 7f7ba5a38640 1 -- 172.21.15.114:0/1517008909 learned_addr learned my addr 172.21.15.114:0/1517008909 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:37.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.985+0000 7f7ba6239640 1 --2- 172.21.15.114:0/1517008909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ba00ff260 0x7f7ba0196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:37.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.986+0000 7f7ba5a38640 1 -- 172.21.15.114:0/1517008909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ba00ff260 msgr2=0x7f7ba0196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:37.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.986+0000 7f7ba5a38640 1 --2- 172.21.15.114:0/1517008909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ba00ff260 0x7f7ba0196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:37.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.986+0000 7f7ba5a38640 1 -- 172.21.15.114:0/1517008909 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b90009560 con 0x7f7ba01970f0 2024-02-16T22:33:37.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.986+0000 7f7ba5a38640 1 --2- 172.21.15.114:0/1517008909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ba01970f0 0x7f7ba0197550 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7b90036040 tx=0x7f7b9002c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:37.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.986+0000 7f7b977fe640 1 -- 172.21.15.114:0/1517008909 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b900035f0 con 0x7f7ba01970f0 2024-02-16T22:33:37.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.986+0000 7f7b977fe640 1 -- 172.21.15.114:0/1517008909 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b90003c10 con 0x7f7ba01970f0 2024-02-16T22:33:37.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.986+0000 7f7ba849d640 1 -- 172.21.15.114:0/1517008909 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba019ff60 con 0x7f7ba01970f0 2024-02-16T22:33:37.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.986+0000 7f7b977fe640 1 -- 172.21.15.114:0/1517008909 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b90005630 con 0x7f7ba01970f0 2024-02-16T22:33:37.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.986+0000 7f7ba849d640 1 -- 172.21.15.114:0/1517008909 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba01a03e0 con 0x7f7ba01970f0 2024-02-16T22:33:37.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.987+0000 7f7ba849d640 1 -- 172.21.15.114:0/1517008909 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba004e040 con 0x7f7ba01970f0 2024-02-16T22:33:37.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.987+0000 7f7b977fe640 1 -- 172.21.15.114:0/1517008909 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f7b90003750 con 0x7f7ba01970f0 2024-02-16T22:33:37.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.988+0000 7f7b977fe640 1 --2- 172.21.15.114:0/1517008909 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f7b8c06d810 0x7f7b8c06fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:37.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.988+0000 7f7b977fe640 1 -- 172.21.15.114:0/1517008909 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f7b900af8a0 con 0x7f7ba01970f0 2024-02-16T22:33:37.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.988+0000 7f7ba6239640 1 --2- 172.21.15.114:0/1517008909 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f7b8c06d810 0x7f7b8c06fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:37.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.988+0000 7f7ba6239640 1 --2- 172.21.15.114:0/1517008909 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f7b8c06d810 0x7f7b8c06fcd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c009bd0 tx=0x7f7b9c005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:37.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:37.993+0000 7f7b977fe640 1 -- 172.21.15.114:0/1517008909 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7b90079050 con 0x7f7ba01970f0 2024-02-16T22:33:38.051 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:37 smithi123 ceph-mon[82331]: pgmap v332: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:38.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:37 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/1049402032' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 15, "format": "json"}]: dispatch 2024-02-16T22:33:38.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:37 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/2566502104' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-02-16T22:33:38.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:37 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/3542102038' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-02-16T22:33:38.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:37 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:33:38.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.225+0000 7f7ba849d640 1 -- 172.21.15.114:0/1517008909 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f7ba0107480 con 0x7f7ba01970f0 2024-02-16T22:33:38.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.226+0000 7f7b977fe640 1 -- 172.21.15.114:0/1517008909 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v37) v1 ==== 107+0+4867 (secure 0 0 0) 0x7f7b9007c710 con 0x7f7ba01970f0 2024-02-16T22:33:38.228 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:38.228 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24269,"name":"cephfs.smithi123.vhlqbe","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.123:6825/418014612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":418014612},{"type":"v1","addr":"172.21.15.123:6825","nonce":418014612}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24285,"name":"cephfs.smithi123.gopcfa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.123:6827/2812517585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2812517585},{"type":"v1","addr":"172.21.15.123:6827","nonce":2812517585}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34404,"name":"cephfs.smithi114.zzryke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6827/2017800151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2017800151},{"type":"v1","addr":"172.21.15.114:6827","nonce":2017800151}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:18:02.108825+0000","modified":"2024-02-16T22:28:32.387205+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14484":{"gid":14484,"name":"cephfs.smithi114.fdxpey","rank":0,"incarnation":16,"state":"up:rejoin","state_seq":158,"addr":"172.21.15.114:6829/865692293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":865692293},{"type":"v1","addr":"172.21.15.114:6829","nonce":865692293}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T22:33:38.228 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 19 2024-02-16T22:33:38.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.228+0000 7f7ba849d640 1 -- 172.21.15.114:0/1517008909 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f7b8c06d810 msgr2=0x7f7b8c06fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:38.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.228+0000 7f7ba849d640 1 --2- 172.21.15.114:0/1517008909 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f7b8c06d810 0x7f7b8c06fcd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c009bd0 tx=0x7f7b9c005f90 comp rx=0 tx=0).stop 2024-02-16T22:33:38.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.228+0000 7f7ba849d640 1 -- 172.21.15.114:0/1517008909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ba01970f0 msgr2=0x7f7ba0197550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:38.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.228+0000 7f7ba849d640 1 --2- 172.21.15.114:0/1517008909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ba01970f0 0x7f7ba0197550 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7b90036040 tx=0x7f7b9002c750 comp rx=0 tx=0).stop 2024-02-16T22:33:38.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.228+0000 7f7ba849d640 1 -- 172.21.15.114:0/1517008909 shutdown_connections 2024-02-16T22:33:38.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.228+0000 7f7ba849d640 1 --2- 172.21.15.114:0/1517008909 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f7b8c06d810 0x7f7b8c06fcd0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:38.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.228+0000 7f7ba849d640 1 --2- 172.21.15.114:0/1517008909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ba01970f0 0x7f7ba0197550 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:38.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.228+0000 7f7ba849d640 1 --2- 172.21.15.114:0/1517008909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ba00ff260 0x7f7ba0196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:38.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.228+0000 7f7ba849d640 1 -- 172.21.15.114:0/1517008909 >> 172.21.15.114:0/1517008909 conn(0x7f7ba0074a50 msgr2=0x7f7ba00fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:38.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.228+0000 7f7ba849d640 1 -- 172.21.15.114:0/1517008909 shutdown_connections 2024-02-16T22:33:38.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.228+0000 7f7ba849d640 1 -- 172.21.15.114:0/1517008909 wait complete. 2024-02-16T22:33:38.241 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-02-16T22:33:38.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.342+0000 7f2efceb6640 1 -- 172.21.15.114:0/4032171409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ef8067f70 msgr2=0x7f2ef8104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:38.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.342+0000 7f2efceb6640 1 --2- 172.21.15.114:0/4032171409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ef8067f70 0x7f2ef8104c70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2eec0097e0 tx=0x7f2eec02bc20 comp rx=0 tx=0).stop 2024-02-16T22:33:38.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.343+0000 7f2efceb6640 1 -- 172.21.15.114:0/4032171409 shutdown_connections 2024-02-16T22:33:38.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.343+0000 7f2efceb6640 1 --2- 172.21.15.114:0/4032171409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ef8067f70 0x7f2ef8104c70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:38.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.343+0000 7f2efceb6640 1 --2- 172.21.15.114:0/4032171409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ef80675c0 0x7f2ef80679a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:38.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.343+0000 7f2efceb6640 1 -- 172.21.15.114:0/4032171409 >> 172.21.15.114:0/4032171409 conn(0x7f2ef8074a50 msgr2=0x7f2ef8074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:38.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.343+0000 7f2efceb6640 1 -- 172.21.15.114:0/4032171409 shutdown_connections 2024-02-16T22:33:38.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.343+0000 7f2efceb6640 1 -- 172.21.15.114:0/4032171409 wait complete. 2024-02-16T22:33:38.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.343+0000 7f2efceb6640 1 Processor -- start 2024-02-16T22:33:38.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.343+0000 7f2efceb6640 1 -- start start 2024-02-16T22:33:38.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.344+0000 7f2efceb6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ef80675c0 0x7f2ef8195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:38.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.344+0000 7f2efceb6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ef81962a0 0x7f2ef819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:38.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.344+0000 7f2efceb6640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef81968a0 con 0x7f2ef81962a0 2024-02-16T22:33:38.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.344+0000 7f2efceb6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef8196a10 con 0x7f2ef80675c0 2024-02-16T22:33:38.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.344+0000 7f2ef5d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ef81962a0 0x7f2ef819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:38.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.344+0000 7f2ef5d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ef81962a0 0x7f2ef819a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48102/0 (socket says 172.21.15.114:48102) 2024-02-16T22:33:38.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.344+0000 7f2ef5d9b640 1 -- 172.21.15.114:0/2325995341 learned_addr learned my addr 172.21.15.114:0/2325995341 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:38.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.344+0000 7f2ef659c640 1 --2- 172.21.15.114:0/2325995341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ef80675c0 0x7f2ef8195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:38.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.344+0000 7f2ef5d9b640 1 -- 172.21.15.114:0/2325995341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ef80675c0 msgr2=0x7f2ef8195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:38.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.344+0000 7f2ef5d9b640 1 --2- 172.21.15.114:0/2325995341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ef80675c0 0x7f2ef8195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:38.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.344+0000 7f2ef5d9b640 1 -- 172.21.15.114:0/2325995341 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee8009560 con 0x7f2ef81962a0 2024-02-16T22:33:38.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.344+0000 7f2ef5d9b640 1 --2- 172.21.15.114:0/2325995341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ef81962a0 0x7f2ef819a700 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2eec035fd0 tx=0x7f2eec034440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:38.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.344+0000 7f2ee77fe640 1 -- 172.21.15.114:0/2325995341 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2eec0349c0 con 0x7f2ef81962a0 2024-02-16T22:33:38.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.344+0000 7f2ee77fe640 1 -- 172.21.15.114:0/2325995341 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2eec02ca80 con 0x7f2ef81962a0 2024-02-16T22:33:38.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.344+0000 7f2efceb6640 1 -- 172.21.15.114:0/2325995341 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eec009490 con 0x7f2ef81962a0 2024-02-16T22:33:38.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.344+0000 7f2ee77fe640 1 -- 172.21.15.114:0/2325995341 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2eec0035f0 con 0x7f2ef81962a0 2024-02-16T22:33:38.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.344+0000 7f2efceb6640 1 -- 172.21.15.114:0/2325995341 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef819b070 con 0x7f2ef81962a0 2024-02-16T22:33:38.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.345+0000 7f2efceb6640 1 -- 172.21.15.114:0/2325995341 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef804e040 con 0x7f2ef81962a0 2024-02-16T22:33:38.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.346+0000 7f2ee77fe640 1 -- 172.21.15.114:0/2325995341 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f2eec03b040 con 0x7f2ef81962a0 2024-02-16T22:33:38.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.346+0000 7f2ee77fe640 1 --2- 172.21.15.114:0/2325995341 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2ee006d810 0x7f2ee006fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:38.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.346+0000 7f2ee77fe640 1 -- 172.21.15.114:0/2325995341 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f2eec0af340 con 0x7f2ef81962a0 2024-02-16T22:33:38.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.346+0000 7f2ef659c640 1 --2- 172.21.15.114:0/2325995341 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2ee006d810 0x7f2ee006fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:38.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.346+0000 7f2ef659c640 1 --2- 172.21.15.114:0/2325995341 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2ee006d810 0x7f2ee006fcd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8009530 tx=0x7f2ee8009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:38.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.351+0000 7f2ee77fe640 1 -- 172.21.15.114:0/2325995341 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2eec078af0 con 0x7f2ef81962a0 2024-02-16T22:33:38.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.560+0000 7f2efceb6640 1 -- 172.21.15.114:0/2325995341 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f2ef8108580 con 0x7f2ef81962a0 2024-02-16T22:33:38.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.560+0000 7f2ee77fe640 1 -- 172.21.15.114:0/2325995341 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v37) v1 ==== 107+0+4867 (secure 0 0 0) 0x7f2eec07c1b0 con 0x7f2ef81962a0 2024-02-16T22:33:38.562 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:38.562 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24269,"name":"cephfs.smithi123.vhlqbe","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.123:6825/418014612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":418014612},{"type":"v1","addr":"172.21.15.123:6825","nonce":418014612}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24285,"name":"cephfs.smithi123.gopcfa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.123:6827/2812517585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2812517585},{"type":"v1","addr":"172.21.15.123:6827","nonce":2812517585}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34404,"name":"cephfs.smithi114.zzryke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6827/2017800151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2017800151},{"type":"v1","addr":"172.21.15.114:6827","nonce":2017800151}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:18:02.108825+0000","modified":"2024-02-16T22:28:33.459692+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14484":{"gid":14484,"name":"cephfs.smithi114.fdxpey","rank":0,"incarnation":16,"state":"up:active","state_seq":159,"addr":"172.21.15.114:6829/865692293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":865692293},{"type":"v1","addr":"172.21.15.114:6829","nonce":865692293}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T22:33:38.562 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 20 2024-02-16T22:33:38.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.562+0000 7f2efceb6640 1 -- 172.21.15.114:0/2325995341 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2ee006d810 msgr2=0x7f2ee006fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:38.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.562+0000 7f2efceb6640 1 --2- 172.21.15.114:0/2325995341 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2ee006d810 0x7f2ee006fcd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8009530 tx=0x7f2ee8009260 comp rx=0 tx=0).stop 2024-02-16T22:33:38.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.562+0000 7f2efceb6640 1 -- 172.21.15.114:0/2325995341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ef81962a0 msgr2=0x7f2ef819a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:38.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.562+0000 7f2efceb6640 1 --2- 172.21.15.114:0/2325995341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ef81962a0 0x7f2ef819a700 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2eec035fd0 tx=0x7f2eec034440 comp rx=0 tx=0).stop 2024-02-16T22:33:38.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.562+0000 7f2efceb6640 1 -- 172.21.15.114:0/2325995341 shutdown_connections 2024-02-16T22:33:38.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.562+0000 7f2efceb6640 1 --2- 172.21.15.114:0/2325995341 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2ee006d810 0x7f2ee006fcd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:38.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.562+0000 7f2efceb6640 1 --2- 172.21.15.114:0/2325995341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ef81962a0 0x7f2ef819a700 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:38.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.562+0000 7f2efceb6640 1 --2- 172.21.15.114:0/2325995341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ef80675c0 0x7f2ef8195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:38.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.562+0000 7f2efceb6640 1 -- 172.21.15.114:0/2325995341 >> 172.21.15.114:0/2325995341 conn(0x7f2ef8074a50 msgr2=0x7f2ef80fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:38.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.562+0000 7f2efceb6640 1 -- 172.21.15.114:0/2325995341 shutdown_connections 2024-02-16T22:33:38.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.562+0000 7f2efceb6640 1 -- 172.21.15.114:0/2325995341 wait complete. 2024-02-16T22:33:38.575 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-02-16T22:33:38.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.677+0000 7f37f0bdc640 1 -- 172.21.15.114:0/944062865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37ec0ffc10 msgr2=0x7f37ec103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:38.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.677+0000 7f37f0bdc640 1 --2- 172.21.15.114:0/944062865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37ec0ffc10 0x7f37ec103b70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009880 tx=0x7f37e002bce0 comp rx=0 tx=0).stop 2024-02-16T22:33:38.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.678+0000 7f37f0bdc640 1 -- 172.21.15.114:0/944062865 shutdown_connections 2024-02-16T22:33:38.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.678+0000 7f37f0bdc640 1 --2- 172.21.15.114:0/944062865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37ec0ffc10 0x7f37ec103b70 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:38.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.678+0000 7f37f0bdc640 1 --2- 172.21.15.114:0/944062865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37ec0ff260 0x7f37ec0ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:38.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.678+0000 7f37f0bdc640 1 -- 172.21.15.114:0/944062865 >> 172.21.15.114:0/944062865 conn(0x7f37ec074a50 msgr2=0x7f37ec074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:38.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.678+0000 7f37f0bdc640 1 -- 172.21.15.114:0/944062865 shutdown_connections 2024-02-16T22:33:38.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.678+0000 7f37f0bdc640 1 -- 172.21.15.114:0/944062865 wait complete. 2024-02-16T22:33:38.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.678+0000 7f37f0bdc640 1 Processor -- start 2024-02-16T22:33:38.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.678+0000 7f37f0bdc640 1 -- start start 2024-02-16T22:33:38.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.678+0000 7f37f0bdc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37ec0ff260 0x7f37ec196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:38.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.679+0000 7f37f0bdc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37ec1970f0 0x7f37ec197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:38.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.679+0000 7f37f0bdc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec19fc20 con 0x7f37ec1970f0 2024-02-16T22:33:38.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.679+0000 7f37f0bdc640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec19fd60 con 0x7f37ec0ff260 2024-02-16T22:33:38.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.679+0000 7f37e9d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37ec1970f0 0x7f37ec197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:38.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.679+0000 7f37e9d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37ec1970f0 0x7f37ec197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48126/0 (socket says 172.21.15.114:48126) 2024-02-16T22:33:38.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.679+0000 7f37e9d9b640 1 -- 172.21.15.114:0/4276614696 learned_addr learned my addr 172.21.15.114:0/4276614696 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:38.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.679+0000 7f37ea59c640 1 --2- 172.21.15.114:0/4276614696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37ec0ff260 0x7f37ec196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:38.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.679+0000 7f37e9d9b640 1 -- 172.21.15.114:0/4276614696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37ec0ff260 msgr2=0x7f37ec196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:38.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.679+0000 7f37e9d9b640 1 --2- 172.21.15.114:0/4276614696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37ec0ff260 0x7f37ec196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:38.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.679+0000 7f37e9d9b640 1 -- 172.21.15.114:0/4276614696 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e0009560 con 0x7f37ec1970f0 2024-02-16T22:33:38.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.679+0000 7f37e9d9b640 1 --2- 172.21.15.114:0/4276614696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37ec1970f0 0x7f37ec197550 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f37e0036040 tx=0x7f37e002c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:38.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.679+0000 7f37db7fe640 1 -- 172.21.15.114:0/4276614696 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37e00035f0 con 0x7f37ec1970f0 2024-02-16T22:33:38.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.679+0000 7f37f0bdc640 1 -- 172.21.15.114:0/4276614696 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ec19ff60 con 0x7f37ec1970f0 2024-02-16T22:33:38.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.679+0000 7f37db7fe640 1 -- 172.21.15.114:0/4276614696 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f37e0003c10 con 0x7f37ec1970f0 2024-02-16T22:33:38.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.679+0000 7f37db7fe640 1 -- 172.21.15.114:0/4276614696 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37e0034440 con 0x7f37ec1970f0 2024-02-16T22:33:38.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.679+0000 7f37f0bdc640 1 -- 172.21.15.114:0/4276614696 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ec1a03e0 con 0x7f37ec1970f0 2024-02-16T22:33:38.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.680+0000 7f37f0bdc640 1 -- 172.21.15.114:0/4276614696 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37ec04e040 con 0x7f37ec1970f0 2024-02-16T22:33:38.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.681+0000 7f37db7fe640 1 -- 172.21.15.114:0/4276614696 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f37e0003750 con 0x7f37ec1970f0 2024-02-16T22:33:38.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.681+0000 7f37db7fe640 1 --2- 172.21.15.114:0/4276614696 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f37d406d810 0x7f37d406fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:38.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.681+0000 7f37db7fe640 1 -- 172.21.15.114:0/4276614696 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f37e00af8f0 con 0x7f37ec1970f0 2024-02-16T22:33:38.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.681+0000 7f37ea59c640 1 --2- 172.21.15.114:0/4276614696 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f37d406d810 0x7f37d406fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:38.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.681+0000 7f37ea59c640 1 --2- 172.21.15.114:0/4276614696 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f37d406d810 0x7f37d406fcd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f37dc005fd0 tx=0x7f37dc009680 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:38.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.686+0000 7f37db7fe640 1 -- 172.21.15.114:0/4276614696 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f37e00790a0 con 0x7f37ec1970f0 2024-02-16T22:33:38.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.901+0000 7f37f0bdc640 1 -- 172.21.15.114:0/4276614696 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f37ec107480 con 0x7f37ec1970f0 2024-02-16T22:33:38.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.902+0000 7f37db7fe640 1 -- 172.21.15.114:0/4276614696 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v37) v1 ==== 107+0+4075 (secure 0 0 0) 0x7f37e007c760 con 0x7f37ec1970f0 2024-02-16T22:33:38.904 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:38.904 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24269,"name":"cephfs.smithi123.vhlqbe","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.123:6825/418014612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":418014612},{"type":"v1","addr":"172.21.15.123:6825","nonce":418014612}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24285,"name":"cephfs.smithi123.gopcfa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.123:6827/2812517585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2812517585},{"type":"v1","addr":"172.21.15.123:6827","nonce":2812517585}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34404,"name":"cephfs.smithi114.zzryke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6827/2017800151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2017800151},{"type":"v1","addr":"172.21.15.114:6827","nonce":2017800151}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:18:02.108825+0000","modified":"2024-02-16T22:28:47.638149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T22:33:38.904 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 21 2024-02-16T22:33:38.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.904+0000 7f37f0bdc640 1 -- 172.21.15.114:0/4276614696 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f37d406d810 msgr2=0x7f37d406fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:38.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.904+0000 7f37f0bdc640 1 --2- 172.21.15.114:0/4276614696 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f37d406d810 0x7f37d406fcd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f37dc005fd0 tx=0x7f37dc009680 comp rx=0 tx=0).stop 2024-02-16T22:33:38.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.904+0000 7f37f0bdc640 1 -- 172.21.15.114:0/4276614696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37ec1970f0 msgr2=0x7f37ec197550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:38.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.904+0000 7f37f0bdc640 1 --2- 172.21.15.114:0/4276614696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37ec1970f0 0x7f37ec197550 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f37e0036040 tx=0x7f37e002c850 comp rx=0 tx=0).stop 2024-02-16T22:33:38.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.904+0000 7f37f0bdc640 1 -- 172.21.15.114:0/4276614696 shutdown_connections 2024-02-16T22:33:38.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.904+0000 7f37f0bdc640 1 --2- 172.21.15.114:0/4276614696 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f37d406d810 0x7f37d406fcd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:38.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.904+0000 7f37f0bdc640 1 --2- 172.21.15.114:0/4276614696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37ec1970f0 0x7f37ec197550 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:38.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.904+0000 7f37f0bdc640 1 --2- 172.21.15.114:0/4276614696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37ec0ff260 0x7f37ec196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:38.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.904+0000 7f37f0bdc640 1 -- 172.21.15.114:0/4276614696 >> 172.21.15.114:0/4276614696 conn(0x7f37ec074a50 msgr2=0x7f37ec0fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:38.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.904+0000 7f37f0bdc640 1 -- 172.21.15.114:0/4276614696 shutdown_connections 2024-02-16T22:33:38.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:38.904+0000 7f37f0bdc640 1 -- 172.21.15.114:0/4276614696 wait complete. 2024-02-16T22:33:38.916 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:38 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/3927067341' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-02-16T22:33:38.916 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:38 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:33:38.916 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:38 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:33:38.916 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:38 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/1517008909' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-02-16T22:33:38.916 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:38 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/2325995341' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-02-16T22:33:38.917 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-02-16T22:33:39.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.016+0000 7fb334cd2640 1 -- 172.21.15.114:0/769968987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3300ffc10 msgr2=0x7fb330103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:39.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.016+0000 7fb334cd2640 1 --2- 172.21.15.114:0/769968987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3300ffc10 0x7fb330103b70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb320009880 tx=0x7fb32002bce0 comp rx=0 tx=0).stop 2024-02-16T22:33:39.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.017+0000 7fb334cd2640 1 -- 172.21.15.114:0/769968987 shutdown_connections 2024-02-16T22:33:39.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.017+0000 7fb334cd2640 1 --2- 172.21.15.114:0/769968987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3300ffc10 0x7fb330103b70 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:39.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.017+0000 7fb334cd2640 1 --2- 172.21.15.114:0/769968987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3300ff260 0x7fb3300ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:39.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.017+0000 7fb334cd2640 1 -- 172.21.15.114:0/769968987 >> 172.21.15.114:0/769968987 conn(0x7fb330074a50 msgr2=0x7fb330074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:39.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.017+0000 7fb334cd2640 1 -- 172.21.15.114:0/769968987 shutdown_connections 2024-02-16T22:33:39.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.017+0000 7fb334cd2640 1 -- 172.21.15.114:0/769968987 wait complete. 2024-02-16T22:33:39.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.017+0000 7fb334cd2640 1 Processor -- start 2024-02-16T22:33:39.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.017+0000 7fb334cd2640 1 -- start start 2024-02-16T22:33:39.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.017+0000 7fb334cd2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3300ff260 0x7fb330196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:39.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.017+0000 7fb334cd2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3301970f0 0x7fb330197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:39.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.017+0000 7fb334cd2640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33019fc20 con 0x7fb3301970f0 2024-02-16T22:33:39.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.017+0000 7fb334cd2640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33019fd60 con 0x7fb3300ff260 2024-02-16T22:33:39.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.018+0000 7fb32dd9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3301970f0 0x7fb330197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:39.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.018+0000 7fb32dd9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3301970f0 0x7fb330197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48138/0 (socket says 172.21.15.114:48138) 2024-02-16T22:33:39.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.018+0000 7fb32dd9b640 1 -- 172.21.15.114:0/67649949 learned_addr learned my addr 172.21.15.114:0/67649949 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:39.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.018+0000 7fb32e59c640 1 --2- 172.21.15.114:0/67649949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3300ff260 0x7fb330196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:39.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.018+0000 7fb32dd9b640 1 -- 172.21.15.114:0/67649949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3300ff260 msgr2=0x7fb330196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:39.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.018+0000 7fb32dd9b640 1 --2- 172.21.15.114:0/67649949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3300ff260 0x7fb330196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:39.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.018+0000 7fb32dd9b640 1 -- 172.21.15.114:0/67649949 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb320009560 con 0x7fb3301970f0 2024-02-16T22:33:39.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.018+0000 7fb32e59c640 1 --2- 172.21.15.114:0/67649949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3300ff260 0x7fb330196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T22:33:39.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.018+0000 7fb32dd9b640 1 --2- 172.21.15.114:0/67649949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3301970f0 0x7fb330197550 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb320036040 tx=0x7fb32002c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:39.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.018+0000 7fb31f7fe640 1 -- 172.21.15.114:0/67649949 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3200035f0 con 0x7fb3301970f0 2024-02-16T22:33:39.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.018+0000 7fb31f7fe640 1 -- 172.21.15.114:0/67649949 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb320003c10 con 0x7fb3301970f0 2024-02-16T22:33:39.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.018+0000 7fb334cd2640 1 -- 172.21.15.114:0/67649949 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb33019ff60 con 0x7fb3301970f0 2024-02-16T22:33:39.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.018+0000 7fb31f7fe640 1 -- 172.21.15.114:0/67649949 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb320005630 con 0x7fb3301970f0 2024-02-16T22:33:39.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.018+0000 7fb334cd2640 1 -- 172.21.15.114:0/67649949 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3301a03e0 con 0x7fb3301970f0 2024-02-16T22:33:39.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.019+0000 7fb334cd2640 1 -- 172.21.15.114:0/67649949 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb33004e040 con 0x7fb3301970f0 2024-02-16T22:33:39.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.020+0000 7fb31f7fe640 1 -- 172.21.15.114:0/67649949 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fb320003750 con 0x7fb3301970f0 2024-02-16T22:33:39.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.020+0000 7fb31f7fe640 1 --2- 172.21.15.114:0/67649949 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb31806d810 0x7fb31806fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:39.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.020+0000 7fb31f7fe640 1 -- 172.21.15.114:0/67649949 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7fb3200af7f0 con 0x7fb3301970f0 2024-02-16T22:33:39.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.020+0000 7fb32e59c640 1 --2- 172.21.15.114:0/67649949 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb31806d810 0x7fb31806fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:39.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.020+0000 7fb32e59c640 1 --2- 172.21.15.114:0/67649949 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb31806d810 0x7fb31806fcd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb324009fc0 tx=0x7fb324005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:39.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.025+0000 7fb31f7fe640 1 -- 172.21.15.114:0/67649949 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb320079020 con 0x7fb3301970f0 2024-02-16T22:33:39.051 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:38 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/3927067341' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-02-16T22:33:39.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:38 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:33:39.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:38 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:33:39.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:38 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/1517008909' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-02-16T22:33:39.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:38 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/2325995341' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-02-16T22:33:39.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.234+0000 7fb334cd2640 1 -- 172.21.15.114:0/67649949 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7fb330107480 con 0x7fb3301970f0 2024-02-16T22:33:39.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.235+0000 7fb31f7fe640 1 -- 172.21.15.114:0/67649949 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v37) v1 ==== 107+0+4086 (secure 0 0 0) 0x7fb32007c6e0 con 0x7fb3301970f0 2024-02-16T22:33:39.237 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:39.237 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24285,"name":"cephfs.smithi123.gopcfa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.123:6827/2812517585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2812517585},{"type":"v1","addr":"172.21.15.123:6827","nonce":2812517585}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34404,"name":"cephfs.smithi114.zzryke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6827/2017800151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2017800151},{"type":"v1","addr":"172.21.15.114:6827","nonce":2017800151}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:18:02.108825+0000","modified":"2024-02-16T22:28:47.642938+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24269":{"gid":24269,"name":"cephfs.smithi123.vhlqbe","rank":0,"incarnation":22,"state":"up:replay","state_seq":3,"addr":"172.21.15.123:6825/418014612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":418014612},{"type":"v1","addr":"172.21.15.123:6825","nonce":418014612}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T22:33:39.237 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 22 2024-02-16T22:33:39.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.237+0000 7fb334cd2640 1 -- 172.21.15.114:0/67649949 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb31806d810 msgr2=0x7fb31806fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:39.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.237+0000 7fb334cd2640 1 --2- 172.21.15.114:0/67649949 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb31806d810 0x7fb31806fcd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb324009fc0 tx=0x7fb324005f90 comp rx=0 tx=0).stop 2024-02-16T22:33:39.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.237+0000 7fb334cd2640 1 -- 172.21.15.114:0/67649949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3301970f0 msgr2=0x7fb330197550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:39.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.237+0000 7fb334cd2640 1 --2- 172.21.15.114:0/67649949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3301970f0 0x7fb330197550 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb320036040 tx=0x7fb32002c750 comp rx=0 tx=0).stop 2024-02-16T22:33:39.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.237+0000 7fb334cd2640 1 -- 172.21.15.114:0/67649949 shutdown_connections 2024-02-16T22:33:39.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.237+0000 7fb334cd2640 1 --2- 172.21.15.114:0/67649949 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fb31806d810 0x7fb31806fcd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:39.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.237+0000 7fb334cd2640 1 --2- 172.21.15.114:0/67649949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3301970f0 0x7fb330197550 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:39.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.237+0000 7fb334cd2640 1 --2- 172.21.15.114:0/67649949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3300ff260 0x7fb330196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:39.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.237+0000 7fb334cd2640 1 -- 172.21.15.114:0/67649949 >> 172.21.15.114:0/67649949 conn(0x7fb330074a50 msgr2=0x7fb3300fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:39.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.237+0000 7fb334cd2640 1 -- 172.21.15.114:0/67649949 shutdown_connections 2024-02-16T22:33:39.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.237+0000 7fb334cd2640 1 -- 172.21.15.114:0/67649949 wait complete. 2024-02-16T22:33:39.250 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-02-16T22:33:39.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.350+0000 7f62a0803640 1 -- 172.21.15.114:0/412217434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f629c101920 msgr2=0x7f629c109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:39.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.350+0000 7f62a0803640 1 --2- 172.21.15.114:0/412217434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f629c101920 0x7f629c109e00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6284009880 tx=0x7f628402bd70 comp rx=0 tx=0).stop 2024-02-16T22:33:39.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.350+0000 7f62a0803640 1 -- 172.21.15.114:0/412217434 shutdown_connections 2024-02-16T22:33:39.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.350+0000 7f62a0803640 1 --2- 172.21.15.114:0/412217434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f629c101920 0x7f629c109e00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:39.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.350+0000 7f62a0803640 1 --2- 172.21.15.114:0/412217434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f629c101000 0x7f629c1013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:39.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.350+0000 7f62a0803640 1 -- 172.21.15.114:0/412217434 >> 172.21.15.114:0/412217434 conn(0x7f629c0fa7e0 msgr2=0x7f629c0fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:39.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.350+0000 7f62a0803640 1 -- 172.21.15.114:0/412217434 shutdown_connections 2024-02-16T22:33:39.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.351+0000 7f62a0803640 1 -- 172.21.15.114:0/412217434 wait complete. 2024-02-16T22:33:39.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.351+0000 7f62a0803640 1 Processor -- start 2024-02-16T22:33:39.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.351+0000 7f62a0803640 1 -- start start 2024-02-16T22:33:39.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.352+0000 7f62a0803640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f629c101000 0x7f629c0fec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:39.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.352+0000 7f62a0803640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f629c101920 0x7f629c0ff190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:39.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.352+0000 7f62a0803640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629c0ff6d0 con 0x7f629c101920 2024-02-16T22:33:39.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.352+0000 7f62a0803640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629c0ff840 con 0x7f629c101000 2024-02-16T22:33:39.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.352+0000 7f629a59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f629c101920 0x7f629c0ff190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:39.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.352+0000 7f629a59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f629c101920 0x7f629c0ff190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48162/0 (socket says 172.21.15.114:48162) 2024-02-16T22:33:39.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.352+0000 7f629a59c640 1 -- 172.21.15.114:0/3217677651 learned_addr learned my addr 172.21.15.114:0/3217677651 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:39.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.352+0000 7f629ad9d640 1 --2- 172.21.15.114:0/3217677651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f629c101000 0x7f629c0fec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:39.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.352+0000 7f629a59c640 1 -- 172.21.15.114:0/3217677651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f629c101000 msgr2=0x7f629c0fec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:39.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.352+0000 7f629a59c640 1 --2- 172.21.15.114:0/3217677651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f629c101000 0x7f629c0fec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:39.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.352+0000 7f629a59c640 1 -- 172.21.15.114:0/3217677651 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6284009560 con 0x7f629c101920 2024-02-16T22:33:39.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.352+0000 7f629ad9d640 1 --2- 172.21.15.114:0/3217677651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f629c101000 0x7f629c0fec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T22:33:39.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.352+0000 7f629a59c640 1 --2- 172.21.15.114:0/3217677651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f629c101920 0x7f629c0ff190 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6284009230 tx=0x7f628402c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:39.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.352+0000 7f6293fff640 1 -- 172.21.15.114:0/3217677651 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62840044a0 con 0x7f629c101920 2024-02-16T22:33:39.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.352+0000 7f6293fff640 1 -- 172.21.15.114:0/3217677651 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6284003ab0 con 0x7f629c101920 2024-02-16T22:33:39.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.352+0000 7f62a0803640 1 -- 172.21.15.114:0/3217677651 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f629c0ffac0 con 0x7f629c101920 2024-02-16T22:33:39.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.352+0000 7f6293fff640 1 -- 172.21.15.114:0/3217677651 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6284005600 con 0x7f629c101920 2024-02-16T22:33:39.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.352+0000 7f62a0803640 1 -- 172.21.15.114:0/3217677651 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f629c1a2030 con 0x7f629c101920 2024-02-16T22:33:39.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.353+0000 7f62a0803640 1 -- 172.21.15.114:0/3217677651 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f629c04e040 con 0x7f629c101920 2024-02-16T22:33:39.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.354+0000 7f6293fff640 1 -- 172.21.15.114:0/3217677651 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f6284004dd0 con 0x7f629c101920 2024-02-16T22:33:39.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.354+0000 7f6293fff640 1 --2- 172.21.15.114:0/3217677651 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f628806d810 0x7f628806fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:39.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.354+0000 7f6293fff640 1 -- 172.21.15.114:0/3217677651 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f62840aff50 con 0x7f629c101920 2024-02-16T22:33:39.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.354+0000 7f629ad9d640 1 --2- 172.21.15.114:0/3217677651 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f628806d810 0x7f628806fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:39.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.355+0000 7f629ad9d640 1 --2- 172.21.15.114:0/3217677651 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f628806d810 0x7f628806fcd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f628c009b40 tx=0x7f628c005e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:39.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.359+0000 7f6293fff640 1 -- 172.21.15.114:0/3217677651 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f62840e3d20 con 0x7f629c101920 2024-02-16T22:33:39.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.581+0000 7f62a0803640 1 -- 172.21.15.114:0/3217677651 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f629c0ffa60 con 0x7f629c101920 2024-02-16T22:33:39.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.582+0000 7f6293fff640 1 -- 172.21.15.114:0/3217677651 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v37) v1 ==== 107+0+4902 (secure 0 0 0) 0x7f6284043090 con 0x7f629c101920 2024-02-16T22:33:39.584 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:39.584 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24285,"name":"cephfs.smithi123.gopcfa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.123:6827/2812517585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2812517585},{"type":"v1","addr":"172.21.15.123:6827","nonce":2812517585}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34404,"name":"cephfs.smithi114.zzryke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6827/2017800151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2017800151},{"type":"v1","addr":"172.21.15.114:6827","nonce":2017800151}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34436,"name":"cephfs.smithi114.fdxpey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6829/3702914193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3702914193},{"type":"v1","addr":"172.21.15.114:6829","nonce":3702914193}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:18:02.108825+0000","modified":"2024-02-16T22:28:47.642938+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24269":{"gid":24269,"name":"cephfs.smithi123.vhlqbe","rank":0,"incarnation":22,"state":"up:replay","state_seq":3,"addr":"172.21.15.123:6825/418014612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":418014612},{"type":"v1","addr":"172.21.15.123:6825","nonce":418014612}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T22:33:39.584 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 23 2024-02-16T22:33:39.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.584+0000 7f62a0803640 1 -- 172.21.15.114:0/3217677651 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f628806d810 msgr2=0x7f628806fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:39.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.584+0000 7f62a0803640 1 --2- 172.21.15.114:0/3217677651 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f628806d810 0x7f628806fcd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f628c009b40 tx=0x7f628c005e50 comp rx=0 tx=0).stop 2024-02-16T22:33:39.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.584+0000 7f62a0803640 1 -- 172.21.15.114:0/3217677651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f629c101920 msgr2=0x7f629c0ff190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:39.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.584+0000 7f62a0803640 1 --2- 172.21.15.114:0/3217677651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f629c101920 0x7f629c0ff190 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6284009230 tx=0x7f628402c7a0 comp rx=0 tx=0).stop 2024-02-16T22:33:39.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.584+0000 7f62a0803640 1 -- 172.21.15.114:0/3217677651 shutdown_connections 2024-02-16T22:33:39.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.584+0000 7f62a0803640 1 --2- 172.21.15.114:0/3217677651 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f628806d810 0x7f628806fcd0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:39.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.584+0000 7f62a0803640 1 --2- 172.21.15.114:0/3217677651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f629c101920 0x7f629c0ff190 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:39.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.584+0000 7f62a0803640 1 --2- 172.21.15.114:0/3217677651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f629c101000 0x7f629c0fec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:39.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.584+0000 7f62a0803640 1 -- 172.21.15.114:0/3217677651 >> 172.21.15.114:0/3217677651 conn(0x7f629c0fa7e0 msgr2=0x7f629c104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:39.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.584+0000 7f62a0803640 1 -- 172.21.15.114:0/3217677651 shutdown_connections 2024-02-16T22:33:39.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.584+0000 7f62a0803640 1 -- 172.21.15.114:0/3217677651 wait complete. 2024-02-16T22:33:39.597 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-02-16T22:33:39.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.697+0000 7f0af588f640 1 -- 172.21.15.114:0/213896201 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0af00ff260 msgr2=0x7f0af00ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:39.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.697+0000 7f0af588f640 1 --2- 172.21.15.114:0/213896201 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0af00ff260 0x7f0af00ff640 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0009c20 tx=0x7f0ae002c1b0 comp rx=0 tx=0).stop 2024-02-16T22:33:39.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.697+0000 7f0af588f640 1 -- 172.21.15.114:0/213896201 shutdown_connections 2024-02-16T22:33:39.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.697+0000 7f0af588f640 1 --2- 172.21.15.114:0/213896201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0af00ffc10 0x7f0af0103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:39.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.697+0000 7f0af588f640 1 --2- 172.21.15.114:0/213896201 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0af00ff260 0x7f0af00ff640 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:39.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.697+0000 7f0af588f640 1 -- 172.21.15.114:0/213896201 >> 172.21.15.114:0/213896201 conn(0x7f0af0074a50 msgr2=0x7f0af0074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:39.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.697+0000 7f0af588f640 1 -- 172.21.15.114:0/213896201 shutdown_connections 2024-02-16T22:33:39.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.697+0000 7f0af588f640 1 -- 172.21.15.114:0/213896201 wait complete. 2024-02-16T22:33:39.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.698+0000 7f0af588f640 1 Processor -- start 2024-02-16T22:33:39.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.698+0000 7f0af588f640 1 -- start start 2024-02-16T22:33:39.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.698+0000 7f0af588f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0af00ff260 0x7f0af0195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:39.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.698+0000 7f0af588f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0af0196370 0x7f0af019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:39.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.698+0000 7f0af588f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af0196970 con 0x7f0af0196370 2024-02-16T22:33:39.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.698+0000 7f0af588f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af0196ae0 con 0x7f0af00ff260 2024-02-16T22:33:39.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.698+0000 7f0aee7fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0af0196370 0x7f0af019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:39.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.698+0000 7f0aee7fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0af0196370 0x7f0af019a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48174/0 (socket says 172.21.15.114:48174) 2024-02-16T22:33:39.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.698+0000 7f0aee7fc640 1 -- 172.21.15.114:0/2876053257 learned_addr learned my addr 172.21.15.114:0/2876053257 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:39.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.699+0000 7f0aeeffd640 1 --2- 172.21.15.114:0/2876053257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0af00ff260 0x7f0af0195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:39.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.699+0000 7f0aee7fc640 1 -- 172.21.15.114:0/2876053257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0af00ff260 msgr2=0x7f0af0195e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:39.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.699+0000 7f0aee7fc640 1 --2- 172.21.15.114:0/2876053257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0af00ff260 0x7f0af0195e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:39.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.699+0000 7f0aee7fc640 1 -- 172.21.15.114:0/2876053257 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae0009900 con 0x7f0af0196370 2024-02-16T22:33:39.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.699+0000 7f0aee7fc640 1 --2- 172.21.15.114:0/2876053257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0af0196370 0x7f0af019a7d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0ad800e9a0 tx=0x7f0ad800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:39.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.699+0000 7f0af488d640 1 -- 172.21.15.114:0/2876053257 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ad800cc90 con 0x7f0af0196370 2024-02-16T22:33:39.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.699+0000 7f0af488d640 1 -- 172.21.15.114:0/2876053257 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ad800cdf0 con 0x7f0af0196370 2024-02-16T22:33:39.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.699+0000 7f0af588f640 1 -- 172.21.15.114:0/2876053257 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af019add0 con 0x7f0af0196370 2024-02-16T22:33:39.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.699+0000 7f0af488d640 1 -- 172.21.15.114:0/2876053257 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ad8010430 con 0x7f0af0196370 2024-02-16T22:33:39.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.699+0000 7f0af588f640 1 -- 172.21.15.114:0/2876053257 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af019b330 con 0x7f0af0196370 2024-02-16T22:33:39.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.700+0000 7f0af588f640 1 -- 172.21.15.114:0/2876053257 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af019b610 con 0x7f0af0196370 2024-02-16T22:33:39.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.700+0000 7f0af488d640 1 -- 172.21.15.114:0/2876053257 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f0ad8005020 con 0x7f0af0196370 2024-02-16T22:33:39.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.701+0000 7f0af488d640 1 --2- 172.21.15.114:0/2876053257 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f0adc06d810 0x7f0adc06fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:39.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.701+0000 7f0af488d640 1 -- 172.21.15.114:0/2876053257 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f0ad8014070 con 0x7f0af0196370 2024-02-16T22:33:39.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.701+0000 7f0aeeffd640 1 --2- 172.21.15.114:0/2876053257 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f0adc06d810 0x7f0adc06fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:39.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.701+0000 7f0aeeffd640 1 --2- 172.21.15.114:0/2876053257 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f0adc06d810 0x7f0adc06fcd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0033000 tx=0x7f0ae0004a60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:39.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.705+0000 7f0af488d640 1 -- 172.21.15.114:0/2876053257 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0ad80572a0 con 0x7f0af0196370 2024-02-16T22:33:39.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.914+0000 7f0af588f640 1 -- 172.21.15.114:0/2876053257 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f0af004e040 con 0x7f0af0196370 2024-02-16T22:33:39.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.915+0000 7f0af488d640 1 -- 172.21.15.114:0/2876053257 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v37) v1 ==== 107+0+4907 (secure 0 0 0) 0x7f0ad805a960 con 0x7f0af0196370 2024-02-16T22:33:39.917 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:39.917 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24285,"name":"cephfs.smithi123.gopcfa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.123:6827/2812517585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2812517585},{"type":"v1","addr":"172.21.15.123:6827","nonce":2812517585}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34404,"name":"cephfs.smithi114.zzryke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6827/2017800151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2017800151},{"type":"v1","addr":"172.21.15.114:6827","nonce":2017800151}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34436,"name":"cephfs.smithi114.fdxpey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6829/3702914193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3702914193},{"type":"v1","addr":"172.21.15.114:6829","nonce":3702914193}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:18:02.108825+0000","modified":"2024-02-16T22:28:57.159935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24269":{"gid":24269,"name":"cephfs.smithi123.vhlqbe","rank":0,"incarnation":22,"state":"up:reconnect","state_seq":164,"addr":"172.21.15.123:6825/418014612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":418014612},{"type":"v1","addr":"172.21.15.123:6825","nonce":418014612}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T22:33:39.917 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 24 2024-02-16T22:33:39.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.917+0000 7f0af588f640 1 -- 172.21.15.114:0/2876053257 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f0adc06d810 msgr2=0x7f0adc06fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:39.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.917+0000 7f0af588f640 1 --2- 172.21.15.114:0/2876053257 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f0adc06d810 0x7f0adc06fcd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0033000 tx=0x7f0ae0004a60 comp rx=0 tx=0).stop 2024-02-16T22:33:39.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.917+0000 7f0af588f640 1 -- 172.21.15.114:0/2876053257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0af0196370 msgr2=0x7f0af019a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:39.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.917+0000 7f0af588f640 1 --2- 172.21.15.114:0/2876053257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0af0196370 0x7f0af019a7d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0ad800e9a0 tx=0x7f0ad800ee60 comp rx=0 tx=0).stop 2024-02-16T22:33:39.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.917+0000 7f0af588f640 1 -- 172.21.15.114:0/2876053257 shutdown_connections 2024-02-16T22:33:39.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.917+0000 7f0af588f640 1 --2- 172.21.15.114:0/2876053257 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f0adc06d810 0x7f0adc06fcd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:39.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.917+0000 7f0af588f640 1 --2- 172.21.15.114:0/2876053257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0af0196370 0x7f0af019a7d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:39.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.917+0000 7f0af588f640 1 --2- 172.21.15.114:0/2876053257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0af00ff260 0x7f0af0195e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:39.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.917+0000 7f0af588f640 1 -- 172.21.15.114:0/2876053257 >> 172.21.15.114:0/2876053257 conn(0x7f0af0074a50 msgr2=0x7f0af01033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:39.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.917+0000 7f0af588f640 1 -- 172.21.15.114:0/2876053257 shutdown_connections 2024-02-16T22:33:39.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:39.917+0000 7f0af588f640 1 -- 172.21.15.114:0/2876053257 wait complete. 2024-02-16T22:33:39.929 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:39 smithi114 ceph-mon[94452]: pgmap v333: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:39.929 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:39 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:33:39.929 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:39 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:33:39.929 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:39 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:33:39.929 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:39 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/4276614696' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-02-16T22:33:39.929 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:39 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/67649949' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-02-16T22:33:39.929 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:39 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/3217677651' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-02-16T22:33:39.930 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-02-16T22:33:40.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.031+0000 7fc1d56ca640 1 -- 172.21.15.114:0/2255976681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1d00ff180 msgr2=0x7fc1d00ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:40.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.031+0000 7fc1d56ca640 1 --2- 172.21.15.114:0/2255976681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1d00ff180 0x7fc1d00ff560 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4009880 tx=0x7fc1c402bce0 comp rx=0 tx=0).stop 2024-02-16T22:33:40.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.031+0000 7fc1d56ca640 1 -- 172.21.15.114:0/2255976681 shutdown_connections 2024-02-16T22:33:40.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.031+0000 7fc1d56ca640 1 --2- 172.21.15.114:0/2255976681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1d00ffb30 0x7fc1d0103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:40.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.031+0000 7fc1d56ca640 1 --2- 172.21.15.114:0/2255976681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1d00ff180 0x7fc1d00ff560 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:40.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.031+0000 7fc1d56ca640 1 -- 172.21.15.114:0/2255976681 >> 172.21.15.114:0/2255976681 conn(0x7fc1d00faa00 msgr2=0x7fc1d00fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:40.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.031+0000 7fc1d56ca640 1 -- 172.21.15.114:0/2255976681 shutdown_connections 2024-02-16T22:33:40.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.031+0000 7fc1d56ca640 1 -- 172.21.15.114:0/2255976681 wait complete. 2024-02-16T22:33:40.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.031+0000 7fc1d56ca640 1 Processor -- start 2024-02-16T22:33:40.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.031+0000 7fc1d56ca640 1 -- start start 2024-02-16T22:33:40.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.032+0000 7fc1d56ca640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1d00ff180 0x7fc1d0197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:40.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.032+0000 7fc1d56ca640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1d01984f0 0x7fc1d019c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:40.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.032+0000 7fc1d56ca640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d0198a60 con 0x7fc1d00ff180 2024-02-16T22:33:40.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.032+0000 7fc1d56ca640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d0198bd0 con 0x7fc1d01984f0 2024-02-16T22:33:40.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.032+0000 7fc1ceffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1d00ff180 0x7fc1d0197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:40.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.032+0000 7fc1ceffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1d00ff180 0x7fc1d0197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48194/0 (socket says 172.21.15.114:48194) 2024-02-16T22:33:40.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.032+0000 7fc1ceffd640 1 -- 172.21.15.114:0/3387455447 learned_addr learned my addr 172.21.15.114:0/3387455447 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:40.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.032+0000 7fc1ce7fc640 1 --2- 172.21.15.114:0/3387455447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1d01984f0 0x7fc1d019c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:40.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.032+0000 7fc1ceffd640 1 -- 172.21.15.114:0/3387455447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1d01984f0 msgr2=0x7fc1d019c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:40.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.032+0000 7fc1ceffd640 1 --2- 172.21.15.114:0/3387455447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1d01984f0 0x7fc1d019c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:40.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.032+0000 7fc1ceffd640 1 -- 172.21.15.114:0/3387455447 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c4009560 con 0x7fc1d00ff180 2024-02-16T22:33:40.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.032+0000 7fc1ceffd640 1 --2- 172.21.15.114:0/3387455447 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1d00ff180 0x7fc1d0197fb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4009230 tx=0x7fc1c4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:40.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.032+0000 7fc1b7fff640 1 -- 172.21.15.114:0/3387455447 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1c40035f0 con 0x7fc1d00ff180 2024-02-16T22:33:40.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.032+0000 7fc1d56ca640 1 -- 172.21.15.114:0/3387455447 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1d019cef0 con 0x7fc1d00ff180 2024-02-16T22:33:40.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.032+0000 7fc1b7fff640 1 -- 172.21.15.114:0/3387455447 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc1c4003c10 con 0x7fc1d00ff180 2024-02-16T22:33:40.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.032+0000 7fc1b7fff640 1 -- 172.21.15.114:0/3387455447 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1c4034440 con 0x7fc1d00ff180 2024-02-16T22:33:40.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.032+0000 7fc1d56ca640 1 -- 172.21.15.114:0/3387455447 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1d019d400 con 0x7fc1d00ff180 2024-02-16T22:33:40.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.033+0000 7fc1d56ca640 1 -- 172.21.15.114:0/3387455447 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1d004e040 con 0x7fc1d00ff180 2024-02-16T22:33:40.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.034+0000 7fc1b7fff640 1 -- 172.21.15.114:0/3387455447 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fc1c4004fd0 con 0x7fc1d00ff180 2024-02-16T22:33:40.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.034+0000 7fc1b7fff640 1 --2- 172.21.15.114:0/3387455447 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fc1b806d810 0x7fc1b806fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:40.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.034+0000 7fc1ce7fc640 1 --2- 172.21.15.114:0/3387455447 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fc1b806d810 0x7fc1b806fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:40.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.034+0000 7fc1b7fff640 1 -- 172.21.15.114:0/3387455447 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7fc1c40afa70 con 0x7fc1d00ff180 2024-02-16T22:33:40.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.035+0000 7fc1ce7fc640 1 --2- 172.21.15.114:0/3387455447 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fc1b806d810 0x7fc1b806fcd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0005e10 tx=0x7fc1c0005da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:40.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.039+0000 7fc1b7fff640 1 -- 172.21.15.114:0/3387455447 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc1c4079170 con 0x7fc1d00ff180 2024-02-16T22:33:40.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:39 smithi123 ceph-mon[82331]: pgmap v333: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:40.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:39 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:33:40.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:39 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:33:40.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:39 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:33:40.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:39 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/4276614696' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-02-16T22:33:40.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:39 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/67649949' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-02-16T22:33:40.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:39 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/3217677651' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-02-16T22:33:40.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.252+0000 7fc1d56ca640 1 -- 172.21.15.114:0/3387455447 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7fc1d019d740 con 0x7fc1d00ff180 2024-02-16T22:33:40.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.253+0000 7fc1b7fff640 1 -- 172.21.15.114:0/3387455447 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v37) v1 ==== 107+0+4904 (secure 0 0 0) 0x7fc1c407c830 con 0x7fc1d00ff180 2024-02-16T22:33:40.255 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:40.255 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24285,"name":"cephfs.smithi123.gopcfa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.123:6827/2812517585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2812517585},{"type":"v1","addr":"172.21.15.123:6827","nonce":2812517585}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34404,"name":"cephfs.smithi114.zzryke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6827/2017800151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2017800151},{"type":"v1","addr":"172.21.15.114:6827","nonce":2017800151}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34436,"name":"cephfs.smithi114.fdxpey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6829/3702914193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3702914193},{"type":"v1","addr":"172.21.15.114:6829","nonce":3702914193}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:18:02.108825+0000","modified":"2024-02-16T22:28:57.568172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24269":{"gid":24269,"name":"cephfs.smithi123.vhlqbe","rank":0,"incarnation":22,"state":"up:rejoin","state_seq":165,"addr":"172.21.15.123:6825/418014612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":418014612},{"type":"v1","addr":"172.21.15.123:6825","nonce":418014612}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T22:33:40.255 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 25 2024-02-16T22:33:40.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.255+0000 7fc1d56ca640 1 -- 172.21.15.114:0/3387455447 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fc1b806d810 msgr2=0x7fc1b806fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:40.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.255+0000 7fc1d56ca640 1 --2- 172.21.15.114:0/3387455447 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fc1b806d810 0x7fc1b806fcd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0005e10 tx=0x7fc1c0005da0 comp rx=0 tx=0).stop 2024-02-16T22:33:40.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.255+0000 7fc1d56ca640 1 -- 172.21.15.114:0/3387455447 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1d00ff180 msgr2=0x7fc1d0197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:40.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.255+0000 7fc1d56ca640 1 --2- 172.21.15.114:0/3387455447 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1d00ff180 0x7fc1d0197fb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4009230 tx=0x7fc1c4004040 comp rx=0 tx=0).stop 2024-02-16T22:33:40.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.255+0000 7fc1d56ca640 1 -- 172.21.15.114:0/3387455447 shutdown_connections 2024-02-16T22:33:40.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.255+0000 7fc1d56ca640 1 --2- 172.21.15.114:0/3387455447 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fc1b806d810 0x7fc1b806fcd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:40.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.255+0000 7fc1d56ca640 1 --2- 172.21.15.114:0/3387455447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1d01984f0 0x7fc1d019c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:40.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.255+0000 7fc1d56ca640 1 --2- 172.21.15.114:0/3387455447 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1d00ff180 0x7fc1d0197fb0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:40.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.255+0000 7fc1d56ca640 1 -- 172.21.15.114:0/3387455447 >> 172.21.15.114:0/3387455447 conn(0x7fc1d00faa00 msgr2=0x7fc1d0068c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:40.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.255+0000 7fc1d56ca640 1 -- 172.21.15.114:0/3387455447 shutdown_connections 2024-02-16T22:33:40.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.255+0000 7fc1d56ca640 1 -- 172.21.15.114:0/3387455447 wait complete. 2024-02-16T22:33:40.268 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-02-16T22:33:40.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.368+0000 7ff44b40b640 1 -- 172.21.15.114:0/1873770666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff444067f70 msgr2=0x7ff444104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:40.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.368+0000 7ff44b40b640 1 --2- 172.21.15.114:0/1873770666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff444067f70 0x7ff444104c70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff440009c20 tx=0x7ff44002c1b0 comp rx=0 tx=0).stop 2024-02-16T22:33:40.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.368+0000 7ff44b40b640 1 -- 172.21.15.114:0/1873770666 shutdown_connections 2024-02-16T22:33:40.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.368+0000 7ff44b40b640 1 --2- 172.21.15.114:0/1873770666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff444067f70 0x7ff444104c70 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:40.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.368+0000 7ff44b40b640 1 --2- 172.21.15.114:0/1873770666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4440675c0 0x7ff4440679a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:40.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.368+0000 7ff44b40b640 1 -- 172.21.15.114:0/1873770666 >> 172.21.15.114:0/1873770666 conn(0x7ff444074a50 msgr2=0x7ff444074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:40.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.369+0000 7ff44b40b640 1 -- 172.21.15.114:0/1873770666 shutdown_connections 2024-02-16T22:33:40.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.369+0000 7ff44b40b640 1 -- 172.21.15.114:0/1873770666 wait complete. 2024-02-16T22:33:40.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.369+0000 7ff44b40b640 1 Processor -- start 2024-02-16T22:33:40.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.369+0000 7ff44b40b640 1 -- start start 2024-02-16T22:33:40.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.369+0000 7ff44b40b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4440675c0 0x7ff444195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:40.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.369+0000 7ff44b40b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4441962c0 0x7ff44419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:40.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.369+0000 7ff44b40b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4441968c0 con 0x7ff4440675c0 2024-02-16T22:33:40.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.369+0000 7ff44b40b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff444196a30 con 0x7ff4441962c0 2024-02-16T22:33:40.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.369+0000 7ff4491a7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4440675c0 0x7ff444195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:40.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.369+0000 7ff4491a7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4440675c0 0x7ff444195d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48216/0 (socket says 172.21.15.114:48216) 2024-02-16T22:33:40.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.369+0000 7ff4491a7640 1 -- 172.21.15.114:0/1150441414 learned_addr learned my addr 172.21.15.114:0/1150441414 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:40.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.370+0000 7ff4489a6640 1 --2- 172.21.15.114:0/1150441414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4441962c0 0x7ff44419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:40.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.370+0000 7ff4491a7640 1 -- 172.21.15.114:0/1150441414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4441962c0 msgr2=0x7ff44419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:40.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.370+0000 7ff4491a7640 1 --2- 172.21.15.114:0/1150441414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4441962c0 0x7ff44419a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:40.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.370+0000 7ff4491a7640 1 -- 172.21.15.114:0/1150441414 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff440009900 con 0x7ff4440675c0 2024-02-16T22:33:40.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.370+0000 7ff4489a6640 1 --2- 172.21.15.114:0/1150441414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4441962c0 0x7ff44419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T22:33:40.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.370+0000 7ff4491a7640 1 --2- 172.21.15.114:0/1150441414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4440675c0 0x7ff444195d80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff43c00c920 tx=0x7ff43c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:40.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.370+0000 7ff43a7fc640 1 -- 172.21.15.114:0/1150441414 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff43c007b80 con 0x7ff4440675c0 2024-02-16T22:33:40.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.370+0000 7ff43a7fc640 1 -- 172.21.15.114:0/1150441414 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff43c0054e0 con 0x7ff4440675c0 2024-02-16T22:33:40.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.370+0000 7ff44b40b640 1 -- 172.21.15.114:0/1150441414 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff44419ad20 con 0x7ff4440675c0 2024-02-16T22:33:40.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.370+0000 7ff43a7fc640 1 -- 172.21.15.114:0/1150441414 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff43c00f450 con 0x7ff4440675c0 2024-02-16T22:33:40.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.370+0000 7ff44b40b640 1 -- 172.21.15.114:0/1150441414 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff44419b280 con 0x7ff4440675c0 2024-02-16T22:33:40.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.371+0000 7ff44b40b640 1 -- 172.21.15.114:0/1150441414 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff44419b560 con 0x7ff4440675c0 2024-02-16T22:33:40.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.371+0000 7ff43a7fc640 1 -- 172.21.15.114:0/1150441414 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7ff43c004f60 con 0x7ff4440675c0 2024-02-16T22:33:40.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.372+0000 7ff43a7fc640 1 --2- 172.21.15.114:0/1150441414 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff42c06d810 0x7ff42c06fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:40.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.372+0000 7ff43a7fc640 1 -- 172.21.15.114:0/1150441414 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7ff43c08d470 con 0x7ff4440675c0 2024-02-16T22:33:40.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.372+0000 7ff4489a6640 1 --2- 172.21.15.114:0/1150441414 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff42c06d810 0x7ff42c06fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:40.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.372+0000 7ff4489a6640 1 --2- 172.21.15.114:0/1150441414 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff42c06d810 0x7ff42c06fcd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff440009430 tx=0x7ff440009360 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:40.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.376+0000 7ff43a7fc640 1 -- 172.21.15.114:0/1150441414 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff43c056b70 con 0x7ff4440675c0 2024-02-16T22:33:40.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.583+0000 7ff44b40b640 1 -- 172.21.15.114:0/1150441414 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7ff4441970d0 con 0x7ff4440675c0 2024-02-16T22:33:40.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.584+0000 7ff43a7fc640 1 -- 172.21.15.114:0/1150441414 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v37) v1 ==== 107+0+4904 (secure 0 0 0) 0x7ff43c05a230 con 0x7ff4440675c0 2024-02-16T22:33:40.586 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:40.586 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24285,"name":"cephfs.smithi123.gopcfa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.123:6827/2812517585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2812517585},{"type":"v1","addr":"172.21.15.123:6827","nonce":2812517585}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34404,"name":"cephfs.smithi114.zzryke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6827/2017800151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2017800151},{"type":"v1","addr":"172.21.15.114:6827","nonce":2017800151}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34436,"name":"cephfs.smithi114.fdxpey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6829/3702914193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3702914193},{"type":"v1","addr":"172.21.15.114:6829","nonce":3702914193}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:18:02.108825+0000","modified":"2024-02-16T22:28:58.656002+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24269":{"gid":24269,"name":"cephfs.smithi123.vhlqbe","rank":0,"incarnation":22,"state":"up:active","state_seq":166,"addr":"172.21.15.123:6825/418014612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":418014612},{"type":"v1","addr":"172.21.15.123:6825","nonce":418014612}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T22:33:40.586 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 26 2024-02-16T22:33:40.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.586+0000 7ff44b40b640 1 -- 172.21.15.114:0/1150441414 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff42c06d810 msgr2=0x7ff42c06fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:40.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.586+0000 7ff44b40b640 1 --2- 172.21.15.114:0/1150441414 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff42c06d810 0x7ff42c06fcd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff440009430 tx=0x7ff440009360 comp rx=0 tx=0).stop 2024-02-16T22:33:40.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.586+0000 7ff44b40b640 1 -- 172.21.15.114:0/1150441414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4440675c0 msgr2=0x7ff444195d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:40.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.586+0000 7ff44b40b640 1 --2- 172.21.15.114:0/1150441414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4440675c0 0x7ff444195d80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff43c00c920 tx=0x7ff43c00cde0 comp rx=0 tx=0).stop 2024-02-16T22:33:40.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.586+0000 7ff44b40b640 1 -- 172.21.15.114:0/1150441414 shutdown_connections 2024-02-16T22:33:40.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.586+0000 7ff44b40b640 1 --2- 172.21.15.114:0/1150441414 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff42c06d810 0x7ff42c06fcd0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:40.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.586+0000 7ff44b40b640 1 --2- 172.21.15.114:0/1150441414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4441962c0 0x7ff44419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:40.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.586+0000 7ff44b40b640 1 --2- 172.21.15.114:0/1150441414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4440675c0 0x7ff444195d80 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:40.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.586+0000 7ff44b40b640 1 -- 172.21.15.114:0/1150441414 >> 172.21.15.114:0/1150441414 conn(0x7ff444074a50 msgr2=0x7ff4440fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:40.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.586+0000 7ff44b40b640 1 -- 172.21.15.114:0/1150441414 shutdown_connections 2024-02-16T22:33:40.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.586+0000 7ff44b40b640 1 -- 172.21.15.114:0/1150441414 wait complete. 2024-02-16T22:33:40.599 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-02-16T22:33:40.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.699+0000 7fe077f6e640 1 -- 172.21.15.114:0/1632095639 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0700675c0 msgr2=0x7fe0700679a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:40.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.699+0000 7fe077f6e640 1 --2- 172.21.15.114:0/1632095639 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0700675c0 0x7fe0700679a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe06c009c20 tx=0x7fe06c02c1b0 comp rx=0 tx=0).stop 2024-02-16T22:33:40.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.699+0000 7fe077f6e640 1 -- 172.21.15.114:0/1632095639 shutdown_connections 2024-02-16T22:33:40.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.699+0000 7fe077f6e640 1 --2- 172.21.15.114:0/1632095639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe070067f70 0x7fe070104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:40.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.699+0000 7fe077f6e640 1 --2- 172.21.15.114:0/1632095639 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0700675c0 0x7fe0700679a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:40.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.699+0000 7fe077f6e640 1 -- 172.21.15.114:0/1632095639 >> 172.21.15.114:0/1632095639 conn(0x7fe070074a50 msgr2=0x7fe070074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:40.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.700+0000 7fe077f6e640 1 -- 172.21.15.114:0/1632095639 shutdown_connections 2024-02-16T22:33:40.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.700+0000 7fe077f6e640 1 -- 172.21.15.114:0/1632095639 wait complete. 2024-02-16T22:33:40.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.700+0000 7fe077f6e640 1 Processor -- start 2024-02-16T22:33:40.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.700+0000 7fe077f6e640 1 -- start start 2024-02-16T22:33:40.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.700+0000 7fe077f6e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0700675c0 0x7fe0700ff0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:40.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.700+0000 7fe077f6e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0700ff600 0x7fe0700ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:40.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.700+0000 7fe077f6e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0701a2060 con 0x7fe0700675c0 2024-02-16T22:33:40.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.700+0000 7fe077f6e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0701a21a0 con 0x7fe0700ff600 2024-02-16T22:33:40.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.700+0000 7fe075d0a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0700675c0 0x7fe0700ff0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:40.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.700+0000 7fe075d0a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0700675c0 0x7fe0700ff0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48222/0 (socket says 172.21.15.114:48222) 2024-02-16T22:33:40.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.700+0000 7fe075d0a640 1 -- 172.21.15.114:0/1821162980 learned_addr learned my addr 172.21.15.114:0/1821162980 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:40.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.701+0000 7fe075d0a640 1 -- 172.21.15.114:0/1821162980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0700ff600 msgr2=0x7fe0700ffa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:40.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.701+0000 7fe075d0a640 1 --2- 172.21.15.114:0/1821162980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0700ff600 0x7fe0700ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:40.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.701+0000 7fe075d0a640 1 -- 172.21.15.114:0/1821162980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe06c009900 con 0x7fe0700675c0 2024-02-16T22:33:40.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.701+0000 7fe075d0a640 1 --2- 172.21.15.114:0/1821162980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0700675c0 0x7fe0700ff0c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe06c006010 tx=0x7fe06c032b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:40.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.701+0000 7fe066ffd640 1 -- 172.21.15.114:0/1821162980 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe06c034820 con 0x7fe0700675c0 2024-02-16T22:33:40.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.701+0000 7fe077f6e640 1 -- 172.21.15.114:0/1821162980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0701a22e0 con 0x7fe0700675c0 2024-02-16T22:33:40.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.701+0000 7fe066ffd640 1 -- 172.21.15.114:0/1821162980 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe06c003e30 con 0x7fe0700675c0 2024-02-16T22:33:40.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.701+0000 7fe066ffd640 1 -- 172.21.15.114:0/1821162980 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe06c03f440 con 0x7fe0700675c0 2024-02-16T22:33:40.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.701+0000 7fe077f6e640 1 -- 172.21.15.114:0/1821162980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0701a2790 con 0x7fe0700675c0 2024-02-16T22:33:40.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.702+0000 7fe077f6e640 1 -- 172.21.15.114:0/1821162980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe07004e040 con 0x7fe0700675c0 2024-02-16T22:33:40.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.702+0000 7fe066ffd640 1 -- 172.21.15.114:0/1821162980 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fe06c03b040 con 0x7fe0700675c0 2024-02-16T22:33:40.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.703+0000 7fe066ffd640 1 --2- 172.21.15.114:0/1821162980 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe05c06d810 0x7fe05c06fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:40.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.703+0000 7fe066ffd640 1 -- 172.21.15.114:0/1821162980 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7fe06c0b0470 con 0x7fe0700675c0 2024-02-16T22:33:40.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.703+0000 7fe075509640 1 --2- 172.21.15.114:0/1821162980 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe05c06d810 0x7fe05c06fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:40.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.703+0000 7fe075509640 1 --2- 172.21.15.114:0/1821162980 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe05c06d810 0x7fe05c06fcd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe070072e40 tx=0x7fe060006c90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:40.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.707+0000 7fe066ffd640 1 -- 172.21.15.114:0/1821162980 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe06c079b70 con 0x7fe0700675c0 2024-02-16T22:33:40.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.917+0000 7fe077f6e640 1 -- 172.21.15.114:0/1821162980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7fe070108580 con 0x7fe0700675c0 2024-02-16T22:33:40.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.918+0000 7fe066ffd640 1 -- 172.21.15.114:0/1821162980 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v37) v1 ==== 107+0+4112 (secure 0 0 0) 0x7fe06c07d230 con 0x7fe0700675c0 2024-02-16T22:33:40.920 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:40.920 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24285,"name":"cephfs.smithi123.gopcfa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.123:6827/2812517585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2812517585},{"type":"v1","addr":"172.21.15.123:6827","nonce":2812517585}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34404,"name":"cephfs.smithi114.zzryke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6827/2017800151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2017800151},{"type":"v1","addr":"172.21.15.114:6827","nonce":2017800151}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34436,"name":"cephfs.smithi114.fdxpey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6829/3702914193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3702914193},{"type":"v1","addr":"172.21.15.114:6829","nonce":3702914193}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:18:02.108825+0000","modified":"2024-02-16T22:29:09.777848+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T22:33:40.920 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 27 2024-02-16T22:33:40.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.920+0000 7fe077f6e640 1 -- 172.21.15.114:0/1821162980 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe05c06d810 msgr2=0x7fe05c06fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:40.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.920+0000 7fe077f6e640 1 --2- 172.21.15.114:0/1821162980 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe05c06d810 0x7fe05c06fcd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe070072e40 tx=0x7fe060006c90 comp rx=0 tx=0).stop 2024-02-16T22:33:40.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.920+0000 7fe077f6e640 1 -- 172.21.15.114:0/1821162980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0700675c0 msgr2=0x7fe0700ff0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:40.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.920+0000 7fe077f6e640 1 --2- 172.21.15.114:0/1821162980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0700675c0 0x7fe0700ff0c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe06c006010 tx=0x7fe06c032b60 comp rx=0 tx=0).stop 2024-02-16T22:33:40.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.920+0000 7fe077f6e640 1 -- 172.21.15.114:0/1821162980 shutdown_connections 2024-02-16T22:33:40.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.920+0000 7fe077f6e640 1 --2- 172.21.15.114:0/1821162980 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe05c06d810 0x7fe05c06fcd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:40.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.920+0000 7fe077f6e640 1 --2- 172.21.15.114:0/1821162980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0700ff600 0x7fe0700ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:40.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.920+0000 7fe077f6e640 1 --2- 172.21.15.114:0/1821162980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0700675c0 0x7fe0700ff0c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:40.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.920+0000 7fe077f6e640 1 -- 172.21.15.114:0/1821162980 >> 172.21.15.114:0/1821162980 conn(0x7fe070074a50 msgr2=0x7fe0700fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:40.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.920+0000 7fe077f6e640 1 -- 172.21.15.114:0/1821162980 shutdown_connections 2024-02-16T22:33:40.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:40.920+0000 7fe077f6e640 1 -- 172.21.15.114:0/1821162980 wait complete. 2024-02-16T22:33:40.932 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:40 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/2876053257' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-02-16T22:33:40.932 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:40 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/3387455447' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-02-16T22:33:40.932 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:40 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/1150441414' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-02-16T22:33:40.933 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-02-16T22:33:41.164 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:40 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/2876053257' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-02-16T22:33:41.164 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:40 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/3387455447' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-02-16T22:33:41.164 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:40 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/1150441414' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-02-16T22:33:41.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.265+0000 7feb0dab6640 1 -- 172.21.15.114:0/2623781389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb080675c0 msgr2=0x7feb080679a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:41.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.265+0000 7feb0dab6640 1 --2- 172.21.15.114:0/2623781389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb080675c0 0x7feb080679a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7feaf8009880 tx=0x7feaf802bce0 comp rx=0 tx=0).stop 2024-02-16T22:33:41.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.265+0000 7feb0dab6640 1 -- 172.21.15.114:0/2623781389 shutdown_connections 2024-02-16T22:33:41.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.265+0000 7feb0dab6640 1 --2- 172.21.15.114:0/2623781389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb08067f70 0x7feb08104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:41.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.265+0000 7feb0dab6640 1 --2- 172.21.15.114:0/2623781389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb080675c0 0x7feb080679a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:41.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.265+0000 7feb0dab6640 1 -- 172.21.15.114:0/2623781389 >> 172.21.15.114:0/2623781389 conn(0x7feb08074a50 msgr2=0x7feb08074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:41.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.265+0000 7feb0dab6640 1 -- 172.21.15.114:0/2623781389 shutdown_connections 2024-02-16T22:33:41.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.265+0000 7feb0dab6640 1 -- 172.21.15.114:0/2623781389 wait complete. 2024-02-16T22:33:41.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.265+0000 7feb0dab6640 1 Processor -- start 2024-02-16T22:33:41.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.265+0000 7feb0dab6640 1 -- start start 2024-02-16T22:33:41.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.266+0000 7feb0dab6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb080675c0 0x7feb08195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:41.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.266+0000 7feb0dab6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb081962d0 0x7feb0819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:41.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.266+0000 7feb0dab6640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb08196840 con 0x7feb081962d0 2024-02-16T22:33:41.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.266+0000 7feb0dab6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb081969b0 con 0x7feb080675c0 2024-02-16T22:33:41.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.266+0000 7feafeffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb081962d0 0x7feb0819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:41.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.266+0000 7feafeffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb081962d0 0x7feb0819a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48242/0 (socket says 172.21.15.114:48242) 2024-02-16T22:33:41.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.266+0000 7feafeffd640 1 -- 172.21.15.114:0/485765752 learned_addr learned my addr 172.21.15.114:0/485765752 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:41.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.266+0000 7feb077fe640 1 --2- 172.21.15.114:0/485765752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb080675c0 0x7feb08195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:41.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.266+0000 7feafeffd640 1 -- 172.21.15.114:0/485765752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb080675c0 msgr2=0x7feb08195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:41.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.266+0000 7feafeffd640 1 --2- 172.21.15.114:0/485765752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb080675c0 0x7feb08195d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:41.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.266+0000 7feafeffd640 1 -- 172.21.15.114:0/485765752 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf8009560 con 0x7feb081962d0 2024-02-16T22:33:41.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.266+0000 7feafeffd640 1 --2- 172.21.15.114:0/485765752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb081962d0 0x7feb0819a730 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7feaf000c8f0 tx=0x7feaf000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:41.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.266+0000 7feb057fa640 1 -- 172.21.15.114:0/485765752 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feaf0004410 con 0x7feb081962d0 2024-02-16T22:33:41.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.266+0000 7feb057fa640 1 -- 172.21.15.114:0/485765752 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feaf0007c60 con 0x7feb081962d0 2024-02-16T22:33:41.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.266+0000 7feb057fa640 1 -- 172.21.15.114:0/485765752 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feaf000f450 con 0x7feb081962d0 2024-02-16T22:33:41.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.266+0000 7feb0dab6640 1 -- 172.21.15.114:0/485765752 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0819acd0 con 0x7feb081962d0 2024-02-16T22:33:41.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.266+0000 7feb0dab6640 1 -- 172.21.15.114:0/485765752 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0819b1e0 con 0x7feb081962d0 2024-02-16T22:33:41.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.267+0000 7feb0dab6640 1 -- 172.21.15.114:0/485765752 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0804e040 con 0x7feb081962d0 2024-02-16T22:33:41.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.268+0000 7feb057fa640 1 -- 172.21.15.114:0/485765752 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7feaf00077a0 con 0x7feb081962d0 2024-02-16T22:33:41.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.268+0000 7feb057fa640 1 --2- 172.21.15.114:0/485765752 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7feae806d810 0x7feae806fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:41.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.268+0000 7feb077fe640 1 --2- 172.21.15.114:0/485765752 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7feae806d810 0x7feae806fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:41.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.268+0000 7feb057fa640 1 -- 172.21.15.114:0/485765752 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7feaf008d620 con 0x7feb081962d0 2024-02-16T22:33:41.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.269+0000 7feb077fe640 1 --2- 172.21.15.114:0/485765752 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7feae806d810 0x7feae806fcd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7feaf8006010 tx=0x7feaf8000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:41.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.273+0000 7feb057fa640 1 -- 172.21.15.114:0/485765752 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7feaf0056d20 con 0x7feb081962d0 2024-02-16T22:33:41.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.481+0000 7feb0dab6640 1 -- 172.21.15.114:0/485765752 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7feb081970e0 con 0x7feb081962d0 2024-02-16T22:33:41.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.482+0000 7feb057fa640 1 -- 172.21.15.114:0/485765752 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v37) v1 ==== 107+0+4123 (secure 0 0 0) 0x7feaf005a3e0 con 0x7feb081962d0 2024-02-16T22:33:41.484 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:41.484 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi114.zzryke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6827/2017800151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2017800151},{"type":"v1","addr":"172.21.15.114:6827","nonce":2017800151}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34436,"name":"cephfs.smithi114.fdxpey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6829/3702914193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3702914193},{"type":"v1","addr":"172.21.15.114:6829","nonce":3702914193}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:18:02.108825+0000","modified":"2024-02-16T22:29:09.785972+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24285":{"gid":24285,"name":"cephfs.smithi123.gopcfa","rank":0,"incarnation":28,"state":"up:replay","state_seq":2,"addr":"172.21.15.123:6827/2812517585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2812517585},{"type":"v1","addr":"172.21.15.123:6827","nonce":2812517585}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T22:33:41.484 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2024-02-16T22:33:41.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.484+0000 7feb0dab6640 1 -- 172.21.15.114:0/485765752 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7feae806d810 msgr2=0x7feae806fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:41.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.484+0000 7feb0dab6640 1 --2- 172.21.15.114:0/485765752 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7feae806d810 0x7feae806fcd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7feaf8006010 tx=0x7feaf8000bc0 comp rx=0 tx=0).stop 2024-02-16T22:33:41.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.484+0000 7feb0dab6640 1 -- 172.21.15.114:0/485765752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb081962d0 msgr2=0x7feb0819a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:41.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.484+0000 7feb0dab6640 1 --2- 172.21.15.114:0/485765752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb081962d0 0x7feb0819a730 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7feaf000c8f0 tx=0x7feaf000cdb0 comp rx=0 tx=0).stop 2024-02-16T22:33:41.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.484+0000 7feb0dab6640 1 -- 172.21.15.114:0/485765752 shutdown_connections 2024-02-16T22:33:41.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.484+0000 7feb0dab6640 1 --2- 172.21.15.114:0/485765752 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7feae806d810 0x7feae806fcd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:41.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.484+0000 7feb0dab6640 1 --2- 172.21.15.114:0/485765752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb081962d0 0x7feb0819a730 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:41.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.484+0000 7feb0dab6640 1 --2- 172.21.15.114:0/485765752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb080675c0 0x7feb08195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:41.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.484+0000 7feb0dab6640 1 -- 172.21.15.114:0/485765752 >> 172.21.15.114:0/485765752 conn(0x7feb08074a50 msgr2=0x7feb080fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:41.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.484+0000 7feb0dab6640 1 -- 172.21.15.114:0/485765752 shutdown_connections 2024-02-16T22:33:41.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.484+0000 7feb0dab6640 1 -- 172.21.15.114:0/485765752 wait complete. 2024-02-16T22:33:41.497 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-02-16T22:33:41.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.596+0000 7f4023a9d640 1 -- 172.21.15.114:0/3548990558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f401c0675c0 msgr2=0x7f401c0679a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:41.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.596+0000 7f4023a9d640 1 --2- 172.21.15.114:0/3548990558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f401c0675c0 0x7f401c0679a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4014009c20 tx=0x7f401402c1b0 comp rx=0 tx=0).stop 2024-02-16T22:33:41.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.596+0000 7f4023a9d640 1 -- 172.21.15.114:0/3548990558 shutdown_connections 2024-02-16T22:33:41.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.596+0000 7f4023a9d640 1 --2- 172.21.15.114:0/3548990558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f401c067f70 0x7f401c104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:41.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.596+0000 7f4023a9d640 1 --2- 172.21.15.114:0/3548990558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f401c0675c0 0x7f401c0679a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:41.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.596+0000 7f4023a9d640 1 -- 172.21.15.114:0/3548990558 >> 172.21.15.114:0/3548990558 conn(0x7f401c074a50 msgr2=0x7f401c074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:41.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.596+0000 7f4023a9d640 1 -- 172.21.15.114:0/3548990558 shutdown_connections 2024-02-16T22:33:41.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.596+0000 7f4023a9d640 1 -- 172.21.15.114:0/3548990558 wait complete. 2024-02-16T22:33:41.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.597+0000 7f4023a9d640 1 Processor -- start 2024-02-16T22:33:41.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.597+0000 7f4023a9d640 1 -- start start 2024-02-16T22:33:41.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.597+0000 7f4023a9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f401c0675c0 0x7f401c195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:41.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.597+0000 7f4023a9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f401c1962d0 0x7f401c19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:41.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.597+0000 7f4023a9d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c1968d0 con 0x7f401c0675c0 2024-02-16T22:33:41.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.597+0000 7f4023a9d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c196a40 con 0x7f401c1962d0 2024-02-16T22:33:41.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.597+0000 7f4021839640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f401c0675c0 0x7f401c195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:41.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.597+0000 7f4021839640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f401c0675c0 0x7f401c195d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48894/0 (socket says 172.21.15.114:48894) 2024-02-16T22:33:41.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.597+0000 7f4021839640 1 -- 172.21.15.114:0/3454895572 learned_addr learned my addr 172.21.15.114:0/3454895572 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:41.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.597+0000 7f4021038640 1 --2- 172.21.15.114:0/3454895572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f401c1962d0 0x7f401c19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:41.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.598+0000 7f4021839640 1 -- 172.21.15.114:0/3454895572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f401c1962d0 msgr2=0x7f401c19a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:41.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.598+0000 7f4021839640 1 --2- 172.21.15.114:0/3454895572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f401c1962d0 0x7f401c19a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:41.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.598+0000 7f4021839640 1 -- 172.21.15.114:0/3454895572 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4014009900 con 0x7f401c0675c0 2024-02-16T22:33:41.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.598+0000 7f4021038640 1 --2- 172.21.15.114:0/3454895572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f401c1962d0 0x7f401c19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:33:41.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.598+0000 7f4021839640 1 --2- 172.21.15.114:0/3454895572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f401c0675c0 0x7f401c195d90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4014006010 tx=0x7f4014032b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:41.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.598+0000 7f401affd640 1 -- 172.21.15.114:0/3454895572 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4014034820 con 0x7f401c0675c0 2024-02-16T22:33:41.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.598+0000 7f401affd640 1 -- 172.21.15.114:0/3454895572 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4014003e30 con 0x7f401c0675c0 2024-02-16T22:33:41.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.598+0000 7f4023a9d640 1 -- 172.21.15.114:0/3454895572 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f401c19acd0 con 0x7f401c0675c0 2024-02-16T22:33:41.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.598+0000 7f401affd640 1 -- 172.21.15.114:0/3454895572 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4014004830 con 0x7f401c0675c0 2024-02-16T22:33:41.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.598+0000 7f4023a9d640 1 -- 172.21.15.114:0/3454895572 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f401c19b230 con 0x7f401c0675c0 2024-02-16T22:33:41.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.599+0000 7f4023a9d640 1 -- 172.21.15.114:0/3454895572 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f401c04e040 con 0x7f401c0675c0 2024-02-16T22:33:41.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.599+0000 7f401affd640 1 -- 172.21.15.114:0/3454895572 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f401403b040 con 0x7f401c0675c0 2024-02-16T22:33:41.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.600+0000 7f401affd640 1 --2- 172.21.15.114:0/3454895572 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f400406d810 0x7f400406fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:41.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.600+0000 7f401affd640 1 -- 172.21.15.114:0/3454895572 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f40140b0300 con 0x7f401c0675c0 2024-02-16T22:33:41.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.600+0000 7f4021038640 1 --2- 172.21.15.114:0/3454895572 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f400406d810 0x7f400406fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:41.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.600+0000 7f4021038640 1 --2- 172.21.15.114:0/3454895572 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f400406d810 0x7f400406fcd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4010005fd0 tx=0x7f401000b5f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:41.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.604+0000 7f401affd640 1 -- 172.21.15.114:0/3454895572 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4014079a80 con 0x7f401c0675c0 2024-02-16T22:33:41.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.815+0000 7f4023a9d640 1 -- 172.21.15.114:0/3454895572 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f401c19b4a0 con 0x7f401c0675c0 2024-02-16T22:33:41.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.815+0000 7f401affd640 1 -- 172.21.15.114:0/3454895572 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v37) v1 ==== 107+0+4128 (secure 0 0 0) 0x7f401407d140 con 0x7f401c0675c0 2024-02-16T22:33:41.817 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:41.817 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi114.zzryke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6827/2017800151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2017800151},{"type":"v1","addr":"172.21.15.114:6827","nonce":2017800151}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34436,"name":"cephfs.smithi114.fdxpey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6829/3702914193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3702914193},{"type":"v1","addr":"172.21.15.114:6829","nonce":3702914193}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:18:02.108825+0000","modified":"2024-02-16T22:29:18.843614+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24285":{"gid":24285,"name":"cephfs.smithi123.gopcfa","rank":0,"incarnation":28,"state":"up:reconnect","state_seq":168,"addr":"172.21.15.123:6827/2812517585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2812517585},{"type":"v1","addr":"172.21.15.123:6827","nonce":2812517585}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T22:33:41.817 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 29 2024-02-16T22:33:41.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.817+0000 7f4023a9d640 1 -- 172.21.15.114:0/3454895572 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f400406d810 msgr2=0x7f400406fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:41.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.817+0000 7f4023a9d640 1 --2- 172.21.15.114:0/3454895572 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f400406d810 0x7f400406fcd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4010005fd0 tx=0x7f401000b5f0 comp rx=0 tx=0).stop 2024-02-16T22:33:41.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.817+0000 7f4023a9d640 1 -- 172.21.15.114:0/3454895572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f401c0675c0 msgr2=0x7f401c195d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:41.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.817+0000 7f4023a9d640 1 --2- 172.21.15.114:0/3454895572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f401c0675c0 0x7f401c195d90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4014006010 tx=0x7f4014032b60 comp rx=0 tx=0).stop 2024-02-16T22:33:41.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.817+0000 7f4023a9d640 1 -- 172.21.15.114:0/3454895572 shutdown_connections 2024-02-16T22:33:41.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.817+0000 7f4023a9d640 1 --2- 172.21.15.114:0/3454895572 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f400406d810 0x7f400406fcd0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:41.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.817+0000 7f4023a9d640 1 --2- 172.21.15.114:0/3454895572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f401c1962d0 0x7f401c19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:41.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.817+0000 7f4023a9d640 1 --2- 172.21.15.114:0/3454895572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f401c0675c0 0x7f401c195d90 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:41.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.817+0000 7f4023a9d640 1 -- 172.21.15.114:0/3454895572 >> 172.21.15.114:0/3454895572 conn(0x7f401c074a50 msgr2=0x7f401c0fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:41.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.817+0000 7f4023a9d640 1 -- 172.21.15.114:0/3454895572 shutdown_connections 2024-02-16T22:33:41.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.817+0000 7f4023a9d640 1 -- 172.21.15.114:0/3454895572 wait complete. 2024-02-16T22:33:41.829 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:41 smithi114 ceph-mon[94452]: pgmap v334: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:41.829 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:41 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/1821162980' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-02-16T22:33:41.829 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:41 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/485765752' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-02-16T22:33:41.830 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-02-16T22:33:41.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.931+0000 7f2b6d8ca640 1 -- 172.21.15.114:0/2899684716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b680675c0 msgr2=0x7f2b680679a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:41.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.931+0000 7f2b6d8ca640 1 --2- 172.21.15.114:0/2899684716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b680675c0 0x7f2b680679a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2b58009880 tx=0x7f2b5802bce0 comp rx=0 tx=0).stop 2024-02-16T22:33:41.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.931+0000 7f2b6d8ca640 1 -- 172.21.15.114:0/2899684716 shutdown_connections 2024-02-16T22:33:41.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.931+0000 7f2b6d8ca640 1 --2- 172.21.15.114:0/2899684716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b68067f70 0x7f2b68104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:41.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.931+0000 7f2b6d8ca640 1 --2- 172.21.15.114:0/2899684716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b680675c0 0x7f2b680679a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:41.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.931+0000 7f2b6d8ca640 1 -- 172.21.15.114:0/2899684716 >> 172.21.15.114:0/2899684716 conn(0x7f2b68074a50 msgr2=0x7f2b68074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:41.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.931+0000 7f2b6d8ca640 1 -- 172.21.15.114:0/2899684716 shutdown_connections 2024-02-16T22:33:41.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.931+0000 7f2b6d8ca640 1 -- 172.21.15.114:0/2899684716 wait complete. 2024-02-16T22:33:41.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.931+0000 7f2b6d8ca640 1 Processor -- start 2024-02-16T22:33:41.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.931+0000 7f2b6d8ca640 1 -- start start 2024-02-16T22:33:41.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.932+0000 7f2b6d8ca640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b680675c0 0x7f2b68195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:41.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.932+0000 7f2b6d8ca640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b681962f0 0x7f2b6819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:41.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.932+0000 7f2b6d8ca640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b681968f0 con 0x7f2b681962f0 2024-02-16T22:33:41.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.932+0000 7f2b6d8ca640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b68196a60 con 0x7f2b680675c0 2024-02-16T22:33:41.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.932+0000 7f2b5ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b681962f0 0x7f2b6819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:41.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.932+0000 7f2b5ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b681962f0 0x7f2b6819a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48914/0 (socket says 172.21.15.114:48914) 2024-02-16T22:33:41.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.932+0000 7f2b5ffff640 1 -- 172.21.15.114:0/3357722734 learned_addr learned my addr 172.21.15.114:0/3357722734 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:41.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.932+0000 7f2b66ffd640 1 --2- 172.21.15.114:0/3357722734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b680675c0 0x7f2b68195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:41.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.932+0000 7f2b5ffff640 1 -- 172.21.15.114:0/3357722734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b680675c0 msgr2=0x7f2b68195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:41.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.932+0000 7f2b5ffff640 1 --2- 172.21.15.114:0/3357722734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b680675c0 0x7f2b68195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:41.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.932+0000 7f2b5ffff640 1 -- 172.21.15.114:0/3357722734 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b58009560 con 0x7f2b681962f0 2024-02-16T22:33:41.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.932+0000 7f2b5ffff640 1 --2- 172.21.15.114:0/3357722734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b681962f0 0x7f2b6819a750 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2b5000d6f0 tx=0x7f2b5000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:41.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.932+0000 7f2b6c8c8640 1 -- 172.21.15.114:0/3357722734 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b500051f0 con 0x7f2b681962f0 2024-02-16T22:33:41.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.932+0000 7f2b6c8c8640 1 -- 172.21.15.114:0/3357722734 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b50009ce0 con 0x7f2b681962f0 2024-02-16T22:33:41.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.932+0000 7f2b6d8ca640 1 -- 172.21.15.114:0/3357722734 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b6819ad50 con 0x7f2b681962f0 2024-02-16T22:33:41.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.932+0000 7f2b6c8c8640 1 -- 172.21.15.114:0/3357722734 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b50004e80 con 0x7f2b681962f0 2024-02-16T22:33:41.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.932+0000 7f2b6d8ca640 1 -- 172.21.15.114:0/3357722734 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b6819b2b0 con 0x7f2b681962f0 2024-02-16T22:33:41.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.933+0000 7f2b6d8ca640 1 -- 172.21.15.114:0/3357722734 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b6819b580 con 0x7f2b681962f0 2024-02-16T22:33:41.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.934+0000 7f2b6c8c8640 1 -- 172.21.15.114:0/3357722734 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f2b500046c0 con 0x7f2b681962f0 2024-02-16T22:33:41.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.934+0000 7f2b6c8c8640 1 --2- 172.21.15.114:0/3357722734 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2b5406d810 0x7f2b5406fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:41.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.934+0000 7f2b6c8c8640 1 -- 172.21.15.114:0/3357722734 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f2b5008cf50 con 0x7f2b681962f0 2024-02-16T22:33:41.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.934+0000 7f2b66ffd640 1 --2- 172.21.15.114:0/3357722734 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2b5406d810 0x7f2b5406fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:41.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.934+0000 7f2b66ffd640 1 --2- 172.21.15.114:0/3357722734 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2b5406d810 0x7f2b5406fcd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2b58006010 tx=0x7f2b58000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:41.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:41.938+0000 7f2b6c8c8640 1 -- 172.21.15.114:0/3357722734 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2b500c0d20 con 0x7f2b681962f0 2024-02-16T22:33:42.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:41 smithi123 ceph-mon[82331]: pgmap v334: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:42.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:41 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/1821162980' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-02-16T22:33:42.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:41 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/485765752' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-02-16T22:33:42.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.148+0000 7f2b6d8ca640 1 -- 172.21.15.114:0/3357722734 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7f2b68197160 con 0x7f2b681962f0 2024-02-16T22:33:42.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.149+0000 7f2b6c8c8640 1 -- 172.21.15.114:0/3357722734 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v37) v1 ==== 107+0+4125 (secure 0 0 0) 0x7f2b50020090 con 0x7f2b681962f0 2024-02-16T22:33:42.151 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:42.151 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi114.zzryke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6827/2017800151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2017800151},{"type":"v1","addr":"172.21.15.114:6827","nonce":2017800151}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34436,"name":"cephfs.smithi114.fdxpey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6829/3702914193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3702914193},{"type":"v1","addr":"172.21.15.114:6829","nonce":3702914193}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:18:02.108825+0000","modified":"2024-02-16T22:29:19.833488+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24285":{"gid":24285,"name":"cephfs.smithi123.gopcfa","rank":0,"incarnation":28,"state":"up:rejoin","state_seq":169,"addr":"172.21.15.123:6827/2812517585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2812517585},{"type":"v1","addr":"172.21.15.123:6827","nonce":2812517585}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T22:33:42.151 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 30 2024-02-16T22:33:42.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.151+0000 7f2b6d8ca640 1 -- 172.21.15.114:0/3357722734 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2b5406d810 msgr2=0x7f2b5406fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:42.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.151+0000 7f2b6d8ca640 1 --2- 172.21.15.114:0/3357722734 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2b5406d810 0x7f2b5406fcd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2b58006010 tx=0x7f2b58000bc0 comp rx=0 tx=0).stop 2024-02-16T22:33:42.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.151+0000 7f2b6d8ca640 1 -- 172.21.15.114:0/3357722734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b681962f0 msgr2=0x7f2b6819a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:42.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.151+0000 7f2b6d8ca640 1 --2- 172.21.15.114:0/3357722734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b681962f0 0x7f2b6819a750 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2b5000d6f0 tx=0x7f2b5000dbb0 comp rx=0 tx=0).stop 2024-02-16T22:33:42.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.151+0000 7f2b6d8ca640 1 -- 172.21.15.114:0/3357722734 shutdown_connections 2024-02-16T22:33:42.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.151+0000 7f2b6d8ca640 1 --2- 172.21.15.114:0/3357722734 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f2b5406d810 0x7f2b5406fcd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:42.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.151+0000 7f2b6d8ca640 1 --2- 172.21.15.114:0/3357722734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b681962f0 0x7f2b6819a750 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:42.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.151+0000 7f2b6d8ca640 1 --2- 172.21.15.114:0/3357722734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b680675c0 0x7f2b68195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:42.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.151+0000 7f2b6d8ca640 1 -- 172.21.15.114:0/3357722734 >> 172.21.15.114:0/3357722734 conn(0x7f2b68074a50 msgr2=0x7f2b680fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:42.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.151+0000 7f2b6d8ca640 1 -- 172.21.15.114:0/3357722734 shutdown_connections 2024-02-16T22:33:42.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.151+0000 7f2b6d8ca640 1 -- 172.21.15.114:0/3357722734 wait complete. 2024-02-16T22:33:42.164 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-02-16T22:33:42.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.264+0000 7f3278516640 1 -- 172.21.15.114:0/2249034677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f32700ffc10 msgr2=0x7f3270103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:42.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.264+0000 7f3278516640 1 --2- 172.21.15.114:0/2249034677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f32700ffc10 0x7f3270103b70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f326c009880 tx=0x7f326c02bce0 comp rx=0 tx=0).stop 2024-02-16T22:33:42.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.264+0000 7f3278516640 1 -- 172.21.15.114:0/2249034677 shutdown_connections 2024-02-16T22:33:42.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.264+0000 7f3278516640 1 --2- 172.21.15.114:0/2249034677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f32700ffc10 0x7f3270103b70 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:42.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.264+0000 7f3278516640 1 --2- 172.21.15.114:0/2249034677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32700ff260 0x7f32700ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:42.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.264+0000 7f3278516640 1 -- 172.21.15.114:0/2249034677 >> 172.21.15.114:0/2249034677 conn(0x7f3270074a50 msgr2=0x7f3270074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:42.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.264+0000 7f3278516640 1 -- 172.21.15.114:0/2249034677 shutdown_connections 2024-02-16T22:33:42.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.264+0000 7f3278516640 1 -- 172.21.15.114:0/2249034677 wait complete. 2024-02-16T22:33:42.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.264+0000 7f3278516640 1 Processor -- start 2024-02-16T22:33:42.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.264+0000 7f3278516640 1 -- start start 2024-02-16T22:33:42.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.264+0000 7f3278516640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32700ff260 0x7f3270100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:42.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.264+0000 7f3278516640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3270101150 0x7f32701015b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:42.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.264+0000 7f3278516640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32701a1f30 con 0x7f3270101150 2024-02-16T22:33:42.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.264+0000 7f3278516640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32701a2070 con 0x7f32700ff260 2024-02-16T22:33:42.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.265+0000 7f3275ab1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3270101150 0x7f32701015b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:42.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.265+0000 7f3275ab1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3270101150 0x7f32701015b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48940/0 (socket says 172.21.15.114:48940) 2024-02-16T22:33:42.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.265+0000 7f3275ab1640 1 -- 172.21.15.114:0/1636574978 learned_addr learned my addr 172.21.15.114:0/1636574978 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:42.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.265+0000 7f32762b2640 1 --2- 172.21.15.114:0/1636574978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32700ff260 0x7f3270100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:42.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.265+0000 7f3275ab1640 1 -- 172.21.15.114:0/1636574978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32700ff260 msgr2=0x7f3270100c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:42.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.265+0000 7f3275ab1640 1 --2- 172.21.15.114:0/1636574978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32700ff260 0x7f3270100c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:42.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.265+0000 7f3275ab1640 1 -- 172.21.15.114:0/1636574978 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326c009560 con 0x7f3270101150 2024-02-16T22:33:42.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.265+0000 7f32762b2640 1 --2- 172.21.15.114:0/1636574978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32700ff260 0x7f3270100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:33:42.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.265+0000 7f3275ab1640 1 --2- 172.21.15.114:0/1636574978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3270101150 0x7f32701015b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f326c036040 tx=0x7f326c02c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:42.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.265+0000 7f32677fe640 1 -- 172.21.15.114:0/1636574978 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f326c0035f0 con 0x7f3270101150 2024-02-16T22:33:42.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.265+0000 7f32677fe640 1 -- 172.21.15.114:0/1636574978 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f326c003c10 con 0x7f3270101150 2024-02-16T22:33:42.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.265+0000 7f3278516640 1 -- 172.21.15.114:0/1636574978 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32701a21b0 con 0x7f3270101150 2024-02-16T22:33:42.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.265+0000 7f32677fe640 1 -- 172.21.15.114:0/1636574978 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f326c034440 con 0x7f3270101150 2024-02-16T22:33:42.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.265+0000 7f3278516640 1 -- 172.21.15.114:0/1636574978 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32701a2660 con 0x7f3270101150 2024-02-16T22:33:42.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.266+0000 7f3278516640 1 -- 172.21.15.114:0/1636574978 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f327004e040 con 0x7f3270101150 2024-02-16T22:33:42.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.267+0000 7f32677fe640 1 -- 172.21.15.114:0/1636574978 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f326c003750 con 0x7f3270101150 2024-02-16T22:33:42.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.267+0000 7f32677fe640 1 --2- 172.21.15.114:0/1636574978 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f325c06d810 0x7f325c06fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:42.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.267+0000 7f32677fe640 1 -- 172.21.15.114:0/1636574978 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f326c0af9d0 con 0x7f3270101150 2024-02-16T22:33:42.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.267+0000 7f32762b2640 1 --2- 172.21.15.114:0/1636574978 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f325c06d810 0x7f325c06fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:42.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.267+0000 7f32762b2640 1 --2- 172.21.15.114:0/1636574978 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f325c06d810 0x7f325c06fcd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f32600058c0 tx=0x7f3260005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:42.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.271+0000 7f32677fe640 1 -- 172.21.15.114:0/1636574978 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f326c079130 con 0x7f3270101150 2024-02-16T22:33:42.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.480+0000 7f3278516640 1 -- 172.21.15.114:0/1636574978 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f3270102140 con 0x7f3270101150 2024-02-16T22:33:42.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.481+0000 7f32677fe640 1 -- 172.21.15.114:0/1636574978 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v37) v1 ==== 107+0+4125 (secure 0 0 0) 0x7f326c07c7f0 con 0x7f3270101150 2024-02-16T22:33:42.483 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:42.483 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi114.zzryke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6827/2017800151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2017800151},{"type":"v1","addr":"172.21.15.114:6827","nonce":2017800151}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34436,"name":"cephfs.smithi114.fdxpey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6829/3702914193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3702914193},{"type":"v1","addr":"172.21.15.114:6829","nonce":3702914193}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:18:02.108825+0000","modified":"2024-02-16T22:29:20.909186+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24285":{"gid":24285,"name":"cephfs.smithi123.gopcfa","rank":0,"incarnation":28,"state":"up:active","state_seq":170,"addr":"172.21.15.123:6827/2812517585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2812517585},{"type":"v1","addr":"172.21.15.123:6827","nonce":2812517585}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T22:33:42.484 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 31 2024-02-16T22:33:42.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.483+0000 7f3278516640 1 -- 172.21.15.114:0/1636574978 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f325c06d810 msgr2=0x7f325c06fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:42.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.483+0000 7f3278516640 1 --2- 172.21.15.114:0/1636574978 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f325c06d810 0x7f325c06fcd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f32600058c0 tx=0x7f3260005850 comp rx=0 tx=0).stop 2024-02-16T22:33:42.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.483+0000 7f3278516640 1 -- 172.21.15.114:0/1636574978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3270101150 msgr2=0x7f32701015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:42.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.483+0000 7f3278516640 1 --2- 172.21.15.114:0/1636574978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3270101150 0x7f32701015b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f326c036040 tx=0x7f326c02c850 comp rx=0 tx=0).stop 2024-02-16T22:33:42.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.483+0000 7f3278516640 1 -- 172.21.15.114:0/1636574978 shutdown_connections 2024-02-16T22:33:42.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.483+0000 7f3278516640 1 --2- 172.21.15.114:0/1636574978 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f325c06d810 0x7f325c06fcd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:42.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.483+0000 7f3278516640 1 --2- 172.21.15.114:0/1636574978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3270101150 0x7f32701015b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:42.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.483+0000 7f3278516640 1 --2- 172.21.15.114:0/1636574978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32700ff260 0x7f3270100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:42.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.483+0000 7f3278516640 1 -- 172.21.15.114:0/1636574978 >> 172.21.15.114:0/1636574978 conn(0x7f3270074a50 msgr2=0x7f3270103320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:42.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.483+0000 7f3278516640 1 -- 172.21.15.114:0/1636574978 shutdown_connections 2024-02-16T22:33:42.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.483+0000 7f3278516640 1 -- 172.21.15.114:0/1636574978 wait complete. 2024-02-16T22:33:42.496 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-02-16T22:33:42.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.596+0000 7ff8f2cb4640 1 -- 172.21.15.114:0/2588046799 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8ec0ff180 msgr2=0x7ff8ec0ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:42.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.596+0000 7ff8f2cb4640 1 --2- 172.21.15.114:0/2588046799 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8ec0ff180 0x7ff8ec0ff560 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0009880 tx=0x7ff8e002bce0 comp rx=0 tx=0).stop 2024-02-16T22:33:42.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.596+0000 7ff8f2cb4640 1 -- 172.21.15.114:0/2588046799 shutdown_connections 2024-02-16T22:33:42.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.596+0000 7ff8f2cb4640 1 --2- 172.21.15.114:0/2588046799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8ec0ffb30 0x7ff8ec103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:42.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.596+0000 7ff8f2cb4640 1 --2- 172.21.15.114:0/2588046799 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8ec0ff180 0x7ff8ec0ff560 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:42.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.596+0000 7ff8f2cb4640 1 -- 172.21.15.114:0/2588046799 >> 172.21.15.114:0/2588046799 conn(0x7ff8ec0faa00 msgr2=0x7ff8ec0fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:42.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.596+0000 7ff8f2cb4640 1 -- 172.21.15.114:0/2588046799 shutdown_connections 2024-02-16T22:33:42.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.596+0000 7ff8f2cb4640 1 -- 172.21.15.114:0/2588046799 wait complete. 2024-02-16T22:33:42.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.596+0000 7ff8f2cb4640 1 Processor -- start 2024-02-16T22:33:42.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.597+0000 7ff8f2cb4640 1 -- start start 2024-02-16T22:33:42.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.597+0000 7ff8f2cb4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8ec0ff180 0x7ff8ec196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:42.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.597+0000 7ff8f2cb4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8ec197240 0x7ff8ec1976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:42.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.597+0000 7ff8f2cb4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ec19fd70 con 0x7ff8ec0ff180 2024-02-16T22:33:42.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.597+0000 7ff8f2cb4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ec19feb0 con 0x7ff8ec197240 2024-02-16T22:33:42.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.597+0000 7ff8f0a50640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8ec0ff180 0x7ff8ec196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:42.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.597+0000 7ff8f0a50640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8ec0ff180 0x7ff8ec196d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48954/0 (socket says 172.21.15.114:48954) 2024-02-16T22:33:42.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.597+0000 7ff8f0a50640 1 -- 172.21.15.114:0/711038550 learned_addr learned my addr 172.21.15.114:0/711038550 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:42.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.597+0000 7ff8f0a50640 1 -- 172.21.15.114:0/711038550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8ec197240 msgr2=0x7ff8ec1976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:42.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.597+0000 7ff8f0a50640 1 --2- 172.21.15.114:0/711038550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8ec197240 0x7ff8ec1976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:42.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.597+0000 7ff8f0a50640 1 -- 172.21.15.114:0/711038550 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e0009560 con 0x7ff8ec0ff180 2024-02-16T22:33:42.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.597+0000 7ff8f0a50640 1 --2- 172.21.15.114:0/711038550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8ec0ff180 0x7ff8ec196d00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0009430 tx=0x7ff8e0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:42.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.597+0000 7ff8e9ffb640 1 -- 172.21.15.114:0/711038550 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8e00035f0 con 0x7ff8ec0ff180 2024-02-16T22:33:42.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.597+0000 7ff8f2cb4640 1 -- 172.21.15.114:0/711038550 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8ec1a00b0 con 0x7ff8ec0ff180 2024-02-16T22:33:42.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.597+0000 7ff8e9ffb640 1 -- 172.21.15.114:0/711038550 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8e0003c10 con 0x7ff8ec0ff180 2024-02-16T22:33:42.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.597+0000 7ff8e9ffb640 1 -- 172.21.15.114:0/711038550 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8e0034440 con 0x7ff8ec0ff180 2024-02-16T22:33:42.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.598+0000 7ff8f2cb4640 1 -- 172.21.15.114:0/711038550 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8ec1a0560 con 0x7ff8ec0ff180 2024-02-16T22:33:42.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.598+0000 7ff8f2cb4640 1 -- 172.21.15.114:0/711038550 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8ec04e040 con 0x7ff8ec0ff180 2024-02-16T22:33:42.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.599+0000 7ff8e9ffb640 1 -- 172.21.15.114:0/711038550 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7ff8e0004fd0 con 0x7ff8ec0ff180 2024-02-16T22:33:42.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.599+0000 7ff8e9ffb640 1 --2- 172.21.15.114:0/711038550 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff8dc06d810 0x7ff8dc06fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:42.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.600+0000 7ff8ebfff640 1 --2- 172.21.15.114:0/711038550 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff8dc06d810 0x7ff8dc06fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:42.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.600+0000 7ff8e9ffb640 1 -- 172.21.15.114:0/711038550 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7ff8e00afa90 con 0x7ff8ec0ff180 2024-02-16T22:33:42.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.600+0000 7ff8ebfff640 1 --2- 172.21.15.114:0/711038550 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff8dc06d810 0x7ff8dc06fcd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec069810 tx=0x7ff8d8006c90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:42.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.604+0000 7ff8e9ffb640 1 -- 172.21.15.114:0/711038550 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff8e0079190 con 0x7ff8ec0ff180 2024-02-16T22:33:42.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.816+0000 7ff8f2cb4640 1 -- 172.21.15.114:0/711038550 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7ff8ec107480 con 0x7ff8ec0ff180 2024-02-16T22:33:42.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.816+0000 7ff8e9ffb640 1 -- 172.21.15.114:0/711038550 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v37) v1 ==== 107+0+4941 (secure 0 0 0) 0x7ff8e007c850 con 0x7ff8ec0ff180 2024-02-16T22:33:42.818 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:42.818 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi114.zzryke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6827/2017800151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2017800151},{"type":"v1","addr":"172.21.15.114:6827","nonce":2017800151}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34436,"name":"cephfs.smithi114.fdxpey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6829/3702914193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3702914193},{"type":"v1","addr":"172.21.15.114:6829","nonce":3702914193}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":44331,"name":"cephfs.smithi123.vhlqbe","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6825/1425065679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1425065679},{"type":"v1","addr":"172.21.15.123:6825","nonce":1425065679}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:18:02.108825+0000","modified":"2024-02-16T22:29:20.909186+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24285":{"gid":24285,"name":"cephfs.smithi123.gopcfa","rank":0,"incarnation":28,"state":"up:active","state_seq":170,"addr":"172.21.15.123:6827/2812517585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2812517585},{"type":"v1","addr":"172.21.15.123:6827","nonce":2812517585}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T22:33:42.818 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 32 2024-02-16T22:33:42.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.818+0000 7ff8f2cb4640 1 -- 172.21.15.114:0/711038550 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff8dc06d810 msgr2=0x7ff8dc06fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:42.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.818+0000 7ff8f2cb4640 1 --2- 172.21.15.114:0/711038550 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff8dc06d810 0x7ff8dc06fcd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec069810 tx=0x7ff8d8006c90 comp rx=0 tx=0).stop 2024-02-16T22:33:42.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.818+0000 7ff8f2cb4640 1 -- 172.21.15.114:0/711038550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8ec0ff180 msgr2=0x7ff8ec196d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:42.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.818+0000 7ff8f2cb4640 1 --2- 172.21.15.114:0/711038550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8ec0ff180 0x7ff8ec196d00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0009430 tx=0x7ff8e0004040 comp rx=0 tx=0).stop 2024-02-16T22:33:42.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.818+0000 7ff8f2cb4640 1 -- 172.21.15.114:0/711038550 shutdown_connections 2024-02-16T22:33:42.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.818+0000 7ff8f2cb4640 1 --2- 172.21.15.114:0/711038550 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7ff8dc06d810 0x7ff8dc06fcd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:42.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.818+0000 7ff8f2cb4640 1 --2- 172.21.15.114:0/711038550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8ec197240 0x7ff8ec1976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:42.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.818+0000 7ff8f2cb4640 1 --2- 172.21.15.114:0/711038550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8ec0ff180 0x7ff8ec196d00 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:42.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.818+0000 7ff8f2cb4640 1 -- 172.21.15.114:0/711038550 >> 172.21.15.114:0/711038550 conn(0x7ff8ec0faa00 msgr2=0x7ff8ec0fc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:42.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.818+0000 7ff8f2cb4640 1 -- 172.21.15.114:0/711038550 shutdown_connections 2024-02-16T22:33:42.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.818+0000 7ff8f2cb4640 1 -- 172.21.15.114:0/711038550 wait complete. 2024-02-16T22:33:42.830 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:42 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/3454895572' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-02-16T22:33:42.830 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:42 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/3357722734' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-02-16T22:33:42.830 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:42 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/1636574978' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-02-16T22:33:42.831 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-02-16T22:33:42.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.932+0000 7f0c4f73e640 1 -- 172.21.15.114:0/3449424429 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c48101150 msgr2=0x7f0c48101530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:42.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.932+0000 7f0c4f73e640 1 --2- 172.21.15.114:0/3449424429 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c48101150 0x7f0c48101530 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0c440098b0 tx=0x7f0c4402bcd0 comp rx=0 tx=0).stop 2024-02-16T22:33:42.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.932+0000 7f0c4f73e640 1 -- 172.21.15.114:0/3449424429 shutdown_connections 2024-02-16T22:33:42.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.932+0000 7f0c4f73e640 1 --2- 172.21.15.114:0/3449424429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c48101b00 0x7f0c48109fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:42.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.932+0000 7f0c4f73e640 1 --2- 172.21.15.114:0/3449424429 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c48101150 0x7f0c48101530 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:42.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.932+0000 7f0c4f73e640 1 -- 172.21.15.114:0/3449424429 >> 172.21.15.114:0/3449424429 conn(0x7f0c480faa00 msgr2=0x7f0c480fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:42.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.932+0000 7f0c4f73e640 1 -- 172.21.15.114:0/3449424429 shutdown_connections 2024-02-16T22:33:42.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.932+0000 7f0c4f73e640 1 -- 172.21.15.114:0/3449424429 wait complete. 2024-02-16T22:33:42.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.933+0000 7f0c4f73e640 1 Processor -- start 2024-02-16T22:33:42.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.933+0000 7f0c4f73e640 1 -- start start 2024-02-16T22:33:42.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.933+0000 7f0c4f73e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c48101150 0x7f0c48195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:42.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.933+0000 7f0c4f73e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c48101b00 0x7f0c481963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:42.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.933+0000 7f0c4f73e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c48196a50 con 0x7f0c48101150 2024-02-16T22:33:42.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.933+0000 7f0c4f73e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4819a820 con 0x7f0c48101b00 2024-02-16T22:33:42.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.933+0000 7f0c4d4da640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c48101150 0x7f0c48195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:42.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.933+0000 7f0c4d4da640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c48101150 0x7f0c48195e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48976/0 (socket says 172.21.15.114:48976) 2024-02-16T22:33:42.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.933+0000 7f0c4d4da640 1 -- 172.21.15.114:0/2248031026 learned_addr learned my addr 172.21.15.114:0/2248031026 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:42.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.933+0000 7f0c4ccd9640 1 --2- 172.21.15.114:0/2248031026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c48101b00 0x7f0c481963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:42.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.933+0000 7f0c4d4da640 1 -- 172.21.15.114:0/2248031026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c48101b00 msgr2=0x7f0c481963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:42.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.933+0000 7f0c4d4da640 1 --2- 172.21.15.114:0/2248031026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c48101b00 0x7f0c481963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:42.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.933+0000 7f0c4d4da640 1 -- 172.21.15.114:0/2248031026 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c44009560 con 0x7f0c48101150 2024-02-16T22:33:42.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.933+0000 7f0c4d4da640 1 --2- 172.21.15.114:0/2248031026 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c48101150 0x7f0c48195e80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0c44009880 tx=0x7f0c44004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:42.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.934+0000 7f0c3e7fc640 1 -- 172.21.15.114:0/2248031026 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c440037c0 con 0x7f0c48101150 2024-02-16T22:33:42.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.934+0000 7f0c3e7fc640 1 -- 172.21.15.114:0/2248031026 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c44003de0 con 0x7f0c48101150 2024-02-16T22:33:42.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.934+0000 7f0c4f73e640 1 -- 172.21.15.114:0/2248031026 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c4819aaa0 con 0x7f0c48101150 2024-02-16T22:33:42.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.934+0000 7f0c3e7fc640 1 -- 172.21.15.114:0/2248031026 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c44034440 con 0x7f0c48101150 2024-02-16T22:33:42.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.934+0000 7f0c4f73e640 1 -- 172.21.15.114:0/2248031026 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c4819afa0 con 0x7f0c48101150 2024-02-16T22:33:42.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.935+0000 7f0c4f73e640 1 -- 172.21.15.114:0/2248031026 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c4804e040 con 0x7f0c48101150 2024-02-16T22:33:42.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.935+0000 7f0c3e7fc640 1 -- 172.21.15.114:0/2248031026 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f0c44004fd0 con 0x7f0c48101150 2024-02-16T22:33:42.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.935+0000 7f0c3e7fc640 1 --2- 172.21.15.114:0/2248031026 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f0c3406d810 0x7f0c3406fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:42.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.936+0000 7f0c3e7fc640 1 -- 172.21.15.114:0/2248031026 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f0c440afb20 con 0x7f0c48101150 2024-02-16T22:33:42.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.936+0000 7f0c4ccd9640 1 --2- 172.21.15.114:0/2248031026 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f0c3406d810 0x7f0c3406fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:42.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.936+0000 7f0c4ccd9640 1 --2- 172.21.15.114:0/2248031026 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f0c3406d810 0x7f0c3406fcd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0c48197490 tx=0x7f0c38008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:42.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:42.940+0000 7f0c3e7fc640 1 -- 172.21.15.114:0/2248031026 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0c44079220 con 0x7f0c48101150 2024-02-16T22:33:43.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:42 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/3454895572' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-02-16T22:33:43.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:42 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/3357722734' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-02-16T22:33:43.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:42 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/1636574978' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-02-16T22:33:43.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.146+0000 7f0c4f73e640 1 -- 172.21.15.114:0/2248031026 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f0c48107650 con 0x7f0c48101150 2024-02-16T22:33:43.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.147+0000 7f0c3e7fc640 1 -- 172.21.15.114:0/2248031026 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v37) v1 ==== 107+0+4146 (secure 0 0 0) 0x7f0c4407c8e0 con 0x7f0c48101150 2024-02-16T22:33:43.149 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:43.149 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi114.zzryke","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6827/2017800151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2017800151},{"type":"v1","addr":"172.21.15.114:6827","nonce":2017800151}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":34436,"name":"cephfs.smithi114.fdxpey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6829/3702914193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3702914193},{"type":"v1","addr":"172.21.15.114:6829","nonce":3702914193}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":44331,"name":"cephfs.smithi123.vhlqbe","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6825/1425065679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1425065679},{"type":"v1","addr":"172.21.15.123:6825","nonce":1425065679}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:18:02.108825+0000","modified":"2024-02-16T22:29:37.876605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T22:33:43.149 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 33 2024-02-16T22:33:43.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.148+0000 7f0c4f73e640 1 -- 172.21.15.114:0/2248031026 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f0c3406d810 msgr2=0x7f0c3406fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:43.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.148+0000 7f0c4f73e640 1 --2- 172.21.15.114:0/2248031026 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f0c3406d810 0x7f0c3406fcd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0c48197490 tx=0x7f0c38008040 comp rx=0 tx=0).stop 2024-02-16T22:33:43.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.148+0000 7f0c4f73e640 1 -- 172.21.15.114:0/2248031026 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c48101150 msgr2=0x7f0c48195e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:43.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.148+0000 7f0c4f73e640 1 --2- 172.21.15.114:0/2248031026 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c48101150 0x7f0c48195e80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0c44009880 tx=0x7f0c44004270 comp rx=0 tx=0).stop 2024-02-16T22:33:43.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.149+0000 7f0c4f73e640 1 -- 172.21.15.114:0/2248031026 shutdown_connections 2024-02-16T22:33:43.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.149+0000 7f0c4f73e640 1 --2- 172.21.15.114:0/2248031026 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f0c3406d810 0x7f0c3406fcd0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:43.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.149+0000 7f0c4f73e640 1 --2- 172.21.15.114:0/2248031026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c48101b00 0x7f0c481963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:43.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.149+0000 7f0c4f73e640 1 --2- 172.21.15.114:0/2248031026 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c48101150 0x7f0c48195e80 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:43.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.149+0000 7f0c4f73e640 1 -- 172.21.15.114:0/2248031026 >> 172.21.15.114:0/2248031026 conn(0x7f0c480faa00 msgr2=0x7f0c48104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:43.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.149+0000 7f0c4f73e640 1 -- 172.21.15.114:0/2248031026 shutdown_connections 2024-02-16T22:33:43.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.149+0000 7f0c4f73e640 1 -- 172.21.15.114:0/2248031026 wait complete. 2024-02-16T22:33:43.161 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-02-16T22:33:43.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.264+0000 7f1f5aa5f640 1 -- 172.21.15.114:0/3124910471 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f54100c80 msgr2=0x7f1f54104cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:43.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.264+0000 7f1f5aa5f640 1 --2- 172.21.15.114:0/3124910471 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f54100c80 0x7f1f54104cc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1f48009880 tx=0x7f1f4802bce0 comp rx=0 tx=0).stop 2024-02-16T22:33:43.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.264+0000 7f1f5aa5f640 1 -- 172.21.15.114:0/3124910471 shutdown_connections 2024-02-16T22:33:43.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.264+0000 7f1f5aa5f640 1 --2- 172.21.15.114:0/3124910471 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f54100c80 0x7f1f54104cc0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:43.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.264+0000 7f1f5aa5f640 1 --2- 172.21.15.114:0/3124910471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f541002d0 0x7f1f541006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:43.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.264+0000 7f1f5aa5f640 1 -- 172.21.15.114:0/3124910471 >> 172.21.15.114:0/3124910471 conn(0x7f1f540fbb50 msgr2=0x7f1f540fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:43.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.264+0000 7f1f5aa5f640 1 -- 172.21.15.114:0/3124910471 shutdown_connections 2024-02-16T22:33:43.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.264+0000 7f1f5aa5f640 1 -- 172.21.15.114:0/3124910471 wait complete. 2024-02-16T22:33:43.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.265+0000 7f1f5aa5f640 1 Processor -- start 2024-02-16T22:33:43.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.265+0000 7f1f5aa5f640 1 -- start start 2024-02-16T22:33:43.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.265+0000 7f1f5aa5f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f541002d0 0x7f1f54195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:43.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.265+0000 7f1f5aa5f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f54196390 0x7f1f5419a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:43.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.265+0000 7f1f5aa5f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f54196990 con 0x7f1f54196390 2024-02-16T22:33:43.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.265+0000 7f1f5aa5f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f54196b00 con 0x7f1f541002d0 2024-02-16T22:33:43.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.265+0000 7f1f537fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f54196390 0x7f1f5419a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:43.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.265+0000 7f1f537fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f54196390 0x7f1f5419a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48994/0 (socket says 172.21.15.114:48994) 2024-02-16T22:33:43.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.265+0000 7f1f537fe640 1 -- 172.21.15.114:0/2007018747 learned_addr learned my addr 172.21.15.114:0/2007018747 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:43.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.265+0000 7f1f53fff640 1 --2- 172.21.15.114:0/2007018747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f541002d0 0x7f1f54195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:43.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.265+0000 7f1f537fe640 1 -- 172.21.15.114:0/2007018747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f541002d0 msgr2=0x7f1f54195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:43.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.265+0000 7f1f537fe640 1 --2- 172.21.15.114:0/2007018747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f541002d0 0x7f1f54195e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:43.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.265+0000 7f1f537fe640 1 -- 172.21.15.114:0/2007018747 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f48009560 con 0x7f1f54196390 2024-02-16T22:33:43.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.266+0000 7f1f537fe640 1 --2- 172.21.15.114:0/2007018747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f54196390 0x7f1f5419a7f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1f48036040 tx=0x7f1f4802c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:43.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.266+0000 7f1f517fa640 1 -- 172.21.15.114:0/2007018747 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f480035f0 con 0x7f1f54196390 2024-02-16T22:33:43.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.266+0000 7f1f5aa5f640 1 -- 172.21.15.114:0/2007018747 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f5419ad90 con 0x7f1f54196390 2024-02-16T22:33:43.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.266+0000 7f1f517fa640 1 -- 172.21.15.114:0/2007018747 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f48003c10 con 0x7f1f54196390 2024-02-16T22:33:43.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.266+0000 7f1f517fa640 1 -- 172.21.15.114:0/2007018747 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f48034440 con 0x7f1f54196390 2024-02-16T22:33:43.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.266+0000 7f1f5aa5f640 1 -- 172.21.15.114:0/2007018747 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f5419b2f0 con 0x7f1f54196390 2024-02-16T22:33:43.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.267+0000 7f1f5aa5f640 1 -- 172.21.15.114:0/2007018747 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f5404e040 con 0x7f1f54196390 2024-02-16T22:33:43.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.267+0000 7f1f517fa640 1 -- 172.21.15.114:0/2007018747 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f1f48003750 con 0x7f1f54196390 2024-02-16T22:33:43.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.268+0000 7f1f517fa640 1 --2- 172.21.15.114:0/2007018747 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f1f3c06d810 0x7f1f3c06fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:43.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.268+0000 7f1f517fa640 1 -- 172.21.15.114:0/2007018747 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f1f480af9f0 con 0x7f1f54196390 2024-02-16T22:33:43.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.268+0000 7f1f53fff640 1 --2- 172.21.15.114:0/2007018747 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f1f3c06d810 0x7f1f3c06fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:43.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.268+0000 7f1f53fff640 1 --2- 172.21.15.114:0/2007018747 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f1f3c06d810 0x7f1f3c06fcd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1f44005fd0 tx=0x7f1f44005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:43.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.272+0000 7f1f517fa640 1 -- 172.21.15.114:0/2007018747 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1f480b1e60 con 0x7f1f54196390 2024-02-16T22:33:43.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.486+0000 7f1f5aa5f640 1 -- 172.21.15.114:0/2007018747 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f1f541971a0 con 0x7f1f54196390 2024-02-16T22:33:43.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.487+0000 7f1f517fa640 1 -- 172.21.15.114:0/2007018747 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v37) v1 ==== 107+0+4191 (secure 0 0 0) 0x7f1f480790f0 con 0x7f1f54196390 2024-02-16T22:33:43.489 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:43.489 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi114.fdxpey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6829/3702914193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3702914193},{"type":"v1","addr":"172.21.15.114:6829","nonce":3702914193}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":44331,"name":"cephfs.smithi123.vhlqbe","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6825/1425065679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1425065679},{"type":"v1","addr":"172.21.15.123:6825","nonce":1425065679}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:18:02.108825+0000","modified":"2024-02-16T22:29:37.881594+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi114.zzryke","rank":0,"incarnation":34,"state":"up:replay","state_seq":1,"addr":"172.21.15.114:6827/2017800151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2017800151},{"type":"v1","addr":"172.21.15.114:6827","nonce":2017800151}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T22:33:43.490 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 34 2024-02-16T22:33:43.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.489+0000 7f1f5aa5f640 1 -- 172.21.15.114:0/2007018747 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f1f3c06d810 msgr2=0x7f1f3c06fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:43.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.489+0000 7f1f5aa5f640 1 --2- 172.21.15.114:0/2007018747 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f1f3c06d810 0x7f1f3c06fcd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1f44005fd0 tx=0x7f1f44005850 comp rx=0 tx=0).stop 2024-02-16T22:33:43.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.489+0000 7f1f5aa5f640 1 -- 172.21.15.114:0/2007018747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f54196390 msgr2=0x7f1f5419a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:43.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.489+0000 7f1f5aa5f640 1 --2- 172.21.15.114:0/2007018747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f54196390 0x7f1f5419a7f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1f48036040 tx=0x7f1f4802c850 comp rx=0 tx=0).stop 2024-02-16T22:33:43.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.489+0000 7f1f5aa5f640 1 -- 172.21.15.114:0/2007018747 shutdown_connections 2024-02-16T22:33:43.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.489+0000 7f1f5aa5f640 1 --2- 172.21.15.114:0/2007018747 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f1f3c06d810 0x7f1f3c06fcd0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:43.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.489+0000 7f1f5aa5f640 1 --2- 172.21.15.114:0/2007018747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1f54196390 0x7f1f5419a7f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:43.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.489+0000 7f1f5aa5f640 1 --2- 172.21.15.114:0/2007018747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f541002d0 0x7f1f54195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:43.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.490+0000 7f1f5aa5f640 1 -- 172.21.15.114:0/2007018747 >> 172.21.15.114:0/2007018747 conn(0x7f1f540fbb50 msgr2=0x7f1f540fd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:43.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.490+0000 7f1f5aa5f640 1 -- 172.21.15.114:0/2007018747 shutdown_connections 2024-02-16T22:33:43.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.490+0000 7f1f5aa5f640 1 -- 172.21.15.114:0/2007018747 wait complete. 2024-02-16T22:33:43.502 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-02-16T22:33:43.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.608+0000 7f65febdc640 1 -- 172.21.15.114:0/3717917635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65f8100c60 msgr2=0x7f65f8104ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:43.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.608+0000 7f65febdc640 1 --2- 172.21.15.114:0/3717917635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65f8100c60 0x7f65f8104ca0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f65e4009880 tx=0x7f65e402bce0 comp rx=0 tx=0).stop 2024-02-16T22:33:43.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.608+0000 7f65febdc640 1 -- 172.21.15.114:0/3717917635 shutdown_connections 2024-02-16T22:33:43.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.608+0000 7f65febdc640 1 --2- 172.21.15.114:0/3717917635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65f8100c60 0x7f65f8104ca0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:43.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.608+0000 7f65febdc640 1 --2- 172.21.15.114:0/3717917635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65f81002b0 0x7f65f8100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:43.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.608+0000 7f65febdc640 1 -- 172.21.15.114:0/3717917635 >> 172.21.15.114:0/3717917635 conn(0x7f65f80fbb30 msgr2=0x7f65f80fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:43.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.608+0000 7f65febdc640 1 -- 172.21.15.114:0/3717917635 shutdown_connections 2024-02-16T22:33:43.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.608+0000 7f65febdc640 1 -- 172.21.15.114:0/3717917635 wait complete. 2024-02-16T22:33:43.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.608+0000 7f65febdc640 1 Processor -- start 2024-02-16T22:33:43.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.608+0000 7f65febdc640 1 -- start start 2024-02-16T22:33:43.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.609+0000 7f65febdc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65f81002b0 0x7f65f8195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:43.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.609+0000 7f65febdc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65f81962b0 0x7f65f819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:43.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.609+0000 7f65febdc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f81968b0 con 0x7f65f81962b0 2024-02-16T22:33:43.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.609+0000 7f65febdc640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f8196a20 con 0x7f65f81002b0 2024-02-16T22:33:43.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.609+0000 7f65f7fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65f81962b0 0x7f65f819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:43.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.609+0000 7f65f7fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65f81962b0 0x7f65f819a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49014/0 (socket says 172.21.15.114:49014) 2024-02-16T22:33:43.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.609+0000 7f65f7fff640 1 -- 172.21.15.114:0/2479680295 learned_addr learned my addr 172.21.15.114:0/2479680295 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:43.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.609+0000 7f65fc978640 1 --2- 172.21.15.114:0/2479680295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65f81002b0 0x7f65f8195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:43.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.609+0000 7f65f7fff640 1 -- 172.21.15.114:0/2479680295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65f81002b0 msgr2=0x7f65f8195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:43.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.609+0000 7f65f7fff640 1 --2- 172.21.15.114:0/2479680295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65f81002b0 0x7f65f8195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:43.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.609+0000 7f65f7fff640 1 -- 172.21.15.114:0/2479680295 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e4009560 con 0x7f65f81962b0 2024-02-16T22:33:43.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.609+0000 7f65f7fff640 1 --2- 172.21.15.114:0/2479680295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65f81962b0 0x7f65f819a710 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f65e4036040 tx=0x7f65e402c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:43.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.609+0000 7f65f5ffb640 1 -- 172.21.15.114:0/2479680295 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65e40035f0 con 0x7f65f81962b0 2024-02-16T22:33:43.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.609+0000 7f65febdc640 1 -- 172.21.15.114:0/2479680295 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f819acb0 con 0x7f65f81962b0 2024-02-16T22:33:43.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.609+0000 7f65f5ffb640 1 -- 172.21.15.114:0/2479680295 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65e4003c10 con 0x7f65f81962b0 2024-02-16T22:33:43.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.609+0000 7f65f5ffb640 1 -- 172.21.15.114:0/2479680295 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65e4034440 con 0x7f65f81962b0 2024-02-16T22:33:43.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.609+0000 7f65febdc640 1 -- 172.21.15.114:0/2479680295 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f819b210 con 0x7f65f81962b0 2024-02-16T22:33:43.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.610+0000 7f65febdc640 1 -- 172.21.15.114:0/2479680295 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f804e040 con 0x7f65f81962b0 2024-02-16T22:33:43.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.611+0000 7f65f5ffb640 1 -- 172.21.15.114:0/2479680295 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f65e4003750 con 0x7f65f81962b0 2024-02-16T22:33:43.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.611+0000 7f65f5ffb640 1 --2- 172.21.15.114:0/2479680295 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f65e806d810 0x7f65e806fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:43.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.612+0000 7f65f5ffb640 1 -- 172.21.15.114:0/2479680295 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f65e40af930 con 0x7f65f81962b0 2024-02-16T22:33:43.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.612+0000 7f65fc978640 1 --2- 172.21.15.114:0/2479680295 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f65e806d810 0x7f65e806fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:43.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.612+0000 7f65fc978640 1 --2- 172.21.15.114:0/2479680295 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f65e806d810 0x7f65e806fcd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f65ec005d00 tx=0x7f65ec00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:43.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.620+0000 7f65f5ffb640 1 -- 172.21.15.114:0/2479680295 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f65e40790e0 con 0x7f65f81962b0 2024-02-16T22:33:43.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:43 smithi114 ceph-mon[94452]: pgmap v335: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:43.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:43 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/711038550' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-02-16T22:33:43.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:43 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/2248031026' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-02-16T22:33:43.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:43 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/2007018747' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-02-16T22:33:43.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.840+0000 7f65febdc640 1 -- 172.21.15.114:0/2479680295 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7f65f819b4f0 con 0x7f65f81962b0 2024-02-16T22:33:43.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.840+0000 7f65f5ffb640 1 -- 172.21.15.114:0/2479680295 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v37) v1 ==== 107+0+4195 (secure 0 0 0) 0x7f65e407c7a0 con 0x7f65f81962b0 2024-02-16T22:33:43.842 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:43.842 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi114.fdxpey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6829/3702914193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3702914193},{"type":"v1","addr":"172.21.15.114:6829","nonce":3702914193}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":44331,"name":"cephfs.smithi123.vhlqbe","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6825/1425065679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1425065679},{"type":"v1","addr":"172.21.15.123:6825","nonce":1425065679}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:18:02.108825+0000","modified":"2024-02-16T22:29:47.021413+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi114.zzryke","rank":0,"incarnation":34,"state":"up:reconnect","state_seq":21,"addr":"172.21.15.114:6827/2017800151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2017800151},{"type":"v1","addr":"172.21.15.114:6827","nonce":2017800151}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T22:33:43.843 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 35 2024-02-16T22:33:43.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.842+0000 7f65febdc640 1 -- 172.21.15.114:0/2479680295 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f65e806d810 msgr2=0x7f65e806fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:43.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.842+0000 7f65febdc640 1 --2- 172.21.15.114:0/2479680295 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f65e806d810 0x7f65e806fcd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f65ec005d00 tx=0x7f65ec00b410 comp rx=0 tx=0).stop 2024-02-16T22:33:43.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.842+0000 7f65febdc640 1 -- 172.21.15.114:0/2479680295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65f81962b0 msgr2=0x7f65f819a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:43.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.842+0000 7f65febdc640 1 --2- 172.21.15.114:0/2479680295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65f81962b0 0x7f65f819a710 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f65e4036040 tx=0x7f65e402c850 comp rx=0 tx=0).stop 2024-02-16T22:33:43.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.843+0000 7f65febdc640 1 -- 172.21.15.114:0/2479680295 shutdown_connections 2024-02-16T22:33:43.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.843+0000 7f65febdc640 1 --2- 172.21.15.114:0/2479680295 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f65e806d810 0x7f65e806fcd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:43.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.843+0000 7f65febdc640 1 --2- 172.21.15.114:0/2479680295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65f81962b0 0x7f65f819a710 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:43.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.843+0000 7f65febdc640 1 --2- 172.21.15.114:0/2479680295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65f81002b0 0x7f65f8195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:43.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.843+0000 7f65febdc640 1 -- 172.21.15.114:0/2479680295 >> 172.21.15.114:0/2479680295 conn(0x7f65f80fbb30 msgr2=0x7f65f80fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:43.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.843+0000 7f65febdc640 1 -- 172.21.15.114:0/2479680295 shutdown_connections 2024-02-16T22:33:43.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.843+0000 7f65febdc640 1 -- 172.21.15.114:0/2479680295 wait complete. 2024-02-16T22:33:43.855 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 36 2024-02-16T22:33:43.953 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:43 smithi123 ceph-mon[82331]: pgmap v335: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:43.953 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:43 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/711038550' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-02-16T22:33:43.953 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:43 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/2248031026' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-02-16T22:33:43.953 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:43 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/2007018747' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-02-16T22:33:43.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.958+0000 7fe31f651640 1 -- 172.21.15.114:0/1710375347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe3180ffc10 msgr2=0x7fe318103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:43.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.958+0000 7fe31f651640 1 --2- 172.21.15.114:0/1710375347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe3180ffc10 0x7fe318103b70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe308009880 tx=0x7fe30802bce0 comp rx=0 tx=0).stop 2024-02-16T22:33:43.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.958+0000 7fe31f651640 1 -- 172.21.15.114:0/1710375347 shutdown_connections 2024-02-16T22:33:43.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.958+0000 7fe31f651640 1 --2- 172.21.15.114:0/1710375347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe3180ffc10 0x7fe318103b70 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:43.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.958+0000 7fe31f651640 1 --2- 172.21.15.114:0/1710375347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3180ff260 0x7fe3180ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:43.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.958+0000 7fe31f651640 1 -- 172.21.15.114:0/1710375347 >> 172.21.15.114:0/1710375347 conn(0x7fe318074a50 msgr2=0x7fe318074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:43.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.958+0000 7fe31f651640 1 -- 172.21.15.114:0/1710375347 shutdown_connections 2024-02-16T22:33:43.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.958+0000 7fe31f651640 1 -- 172.21.15.114:0/1710375347 wait complete. 2024-02-16T22:33:43.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.958+0000 7fe31f651640 1 Processor -- start 2024-02-16T22:33:43.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.959+0000 7fe31f651640 1 -- start start 2024-02-16T22:33:43.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.959+0000 7fe31f651640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3180ff260 0x7fe318195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:43.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.959+0000 7fe31f651640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe3181962b0 0x7fe31819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:43.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.959+0000 7fe31f651640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3181968b0 con 0x7fe3181962b0 2024-02-16T22:33:43.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.959+0000 7fe31f651640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe318196a20 con 0x7fe3180ff260 2024-02-16T22:33:43.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.959+0000 7fe31cbec640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe3181962b0 0x7fe31819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:43.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.959+0000 7fe31cbec640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe3181962b0 0x7fe31819a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49020/0 (socket says 172.21.15.114:49020) 2024-02-16T22:33:43.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.959+0000 7fe31cbec640 1 -- 172.21.15.114:0/3935732404 learned_addr learned my addr 172.21.15.114:0/3935732404 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:43.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.959+0000 7fe31d3ed640 1 --2- 172.21.15.114:0/3935732404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3180ff260 0x7fe318195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:43.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.959+0000 7fe31cbec640 1 -- 172.21.15.114:0/3935732404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3180ff260 msgr2=0x7fe318195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:43.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.959+0000 7fe31cbec640 1 --2- 172.21.15.114:0/3935732404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3180ff260 0x7fe318195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:43.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.959+0000 7fe31cbec640 1 -- 172.21.15.114:0/3935732404 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe308009560 con 0x7fe3181962b0 2024-02-16T22:33:43.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.959+0000 7fe31cbec640 1 --2- 172.21.15.114:0/3935732404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe3181962b0 0x7fe31819a710 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe308036040 tx=0x7fe308004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:43.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.960+0000 7fe30e7fc640 1 -- 172.21.15.114:0/3935732404 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3080037c0 con 0x7fe3181962b0 2024-02-16T22:33:43.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.960+0000 7fe31f651640 1 -- 172.21.15.114:0/3935732404 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe31819acb0 con 0x7fe3181962b0 2024-02-16T22:33:43.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.960+0000 7fe30e7fc640 1 -- 172.21.15.114:0/3935732404 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe308003de0 con 0x7fe3181962b0 2024-02-16T22:33:43.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.960+0000 7fe30e7fc640 1 -- 172.21.15.114:0/3935732404 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe308034440 con 0x7fe3181962b0 2024-02-16T22:33:43.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.960+0000 7fe31f651640 1 -- 172.21.15.114:0/3935732404 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31819b210 con 0x7fe3181962b0 2024-02-16T22:33:43.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.960+0000 7fe31f651640 1 -- 172.21.15.114:0/3935732404 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe31804e040 con 0x7fe3181962b0 2024-02-16T22:33:43.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.962+0000 7fe30e7fc640 1 -- 172.21.15.114:0/3935732404 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fe308004fb0 con 0x7fe3181962b0 2024-02-16T22:33:43.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.962+0000 7fe30e7fc640 1 --2- 172.21.15.114:0/3935732404 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe30406d810 0x7fe30406fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:43.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.962+0000 7fe30e7fc640 1 -- 172.21.15.114:0/3935732404 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7fe3080afa20 con 0x7fe3181962b0 2024-02-16T22:33:43.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.962+0000 7fe31d3ed640 1 --2- 172.21.15.114:0/3935732404 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe30406d810 0x7fe30406fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:43.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.962+0000 7fe31d3ed640 1 --2- 172.21.15.114:0/3935732404 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe30406d810 0x7fe30406fcd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe3140058c0 tx=0x7fe31400a250 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:43.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:43.970+0000 7fe30e7fc640 1 -- 172.21.15.114:0/3935732404 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe3080791d0 con 0x7fe3181962b0 2024-02-16T22:33:44.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.188+0000 7fe31f651640 1 -- 172.21.15.114:0/3935732404 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7fe31819b4f0 con 0x7fe3181962b0 2024-02-16T22:33:44.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.188+0000 7fe30e7fc640 1 -- 172.21.15.114:0/3935732404 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v37) v1 ==== 107+0+5008 (secure 0 0 0) 0x7fe30807c890 con 0x7fe3181962b0 2024-02-16T22:33:44.190 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:44.190 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi114.fdxpey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6829/3702914193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3702914193},{"type":"v1","addr":"172.21.15.114:6829","nonce":3702914193}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":44331,"name":"cephfs.smithi123.vhlqbe","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6825/1425065679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1425065679},{"type":"v1","addr":"172.21.15.123:6825","nonce":1425065679}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":44335,"name":"cephfs.smithi123.gopcfa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6827/3128771108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3128771108},{"type":"v1","addr":"172.21.15.123:6827","nonce":3128771108}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T22:18:02.108825+0000","modified":"2024-02-16T22:29:47.831148+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi114.zzryke","rank":0,"incarnation":34,"state":"up:rejoin","state_seq":22,"addr":"172.21.15.114:6827/2017800151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2017800151},{"type":"v1","addr":"172.21.15.114:6827","nonce":2017800151}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T22:33:44.191 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 36 2024-02-16T22:33:44.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.190+0000 7fe31f651640 1 -- 172.21.15.114:0/3935732404 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe30406d810 msgr2=0x7fe30406fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:44.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.190+0000 7fe31f651640 1 --2- 172.21.15.114:0/3935732404 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe30406d810 0x7fe30406fcd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe3140058c0 tx=0x7fe31400a250 comp rx=0 tx=0).stop 2024-02-16T22:33:44.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.190+0000 7fe31f651640 1 -- 172.21.15.114:0/3935732404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe3181962b0 msgr2=0x7fe31819a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:44.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.190+0000 7fe31f651640 1 --2- 172.21.15.114:0/3935732404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe3181962b0 0x7fe31819a710 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe308036040 tx=0x7fe308004270 comp rx=0 tx=0).stop 2024-02-16T22:33:44.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.191+0000 7fe31f651640 1 -- 172.21.15.114:0/3935732404 shutdown_connections 2024-02-16T22:33:44.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.191+0000 7fe31f651640 1 --2- 172.21.15.114:0/3935732404 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe30406d810 0x7fe30406fcd0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:44.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.191+0000 7fe31f651640 1 --2- 172.21.15.114:0/3935732404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe3181962b0 0x7fe31819a710 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:44.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.191+0000 7fe31f651640 1 --2- 172.21.15.114:0/3935732404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3180ff260 0x7fe318195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:44.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.191+0000 7fe31f651640 1 -- 172.21.15.114:0/3935732404 >> 172.21.15.114:0/3935732404 conn(0x7fe318074a50 msgr2=0x7fe318103320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:44.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.191+0000 7fe31f651640 1 -- 172.21.15.114:0/3935732404 shutdown_connections 2024-02-16T22:33:44.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.191+0000 7fe31f651640 1 -- 172.21.15.114:0/3935732404 wait complete. 2024-02-16T22:33:44.203 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-02-16T22:33:44.214 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-02-16T22:33:44.215 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-02-16T22:33:44.215 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-16T22:33:44.231 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-02-16T22:33:44.231 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-16T22:33:44.287 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-16T22:33:44.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.428+0000 7f99e7339640 1 -- 172.21.15.114:0/2874824147 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99e00fefe0 msgr2=0x7f99e00ff440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:44.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.428+0000 7f99e7339640 1 --2- 172.21.15.114:0/2874824147 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99e00fefe0 0x7f99e00ff440 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f99d0009880 tx=0x7f99d002bce0 comp rx=0 tx=0).stop 2024-02-16T22:33:44.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.428+0000 7f99e7339640 1 -- 172.21.15.114:0/2874824147 shutdown_connections 2024-02-16T22:33:44.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.428+0000 7f99e7339640 1 --2- 172.21.15.114:0/2874824147 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99e00fefe0 0x7f99e00ff440 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:44.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.428+0000 7f99e7339640 1 --2- 172.21.15.114:0/2874824147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99e0103f20 0x7f99e0104300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:44.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.428+0000 7f99e7339640 1 -- 172.21.15.114:0/2874824147 >> 172.21.15.114:0/2874824147 conn(0x7f99e0075b40 msgr2=0x7f99e0075f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:44.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.429+0000 7f99e7339640 1 -- 172.21.15.114:0/2874824147 shutdown_connections 2024-02-16T22:33:44.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.429+0000 7f99e7339640 1 -- 172.21.15.114:0/2874824147 wait complete. 2024-02-16T22:33:44.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.429+0000 7f99e7339640 1 Processor -- start 2024-02-16T22:33:44.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.429+0000 7f99e7339640 1 -- start start 2024-02-16T22:33:44.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.429+0000 7f99e7339640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99e00fefe0 0x7f99e0196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:44.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.429+0000 7f99e7339640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99e0103f20 0x7f99e0197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:44.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.429+0000 7f99e7339640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e0197650 con 0x7f99e00fefe0 2024-02-16T22:33:44.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.429+0000 7f99e7339640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e01977c0 con 0x7f99e0103f20 2024-02-16T22:33:44.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.429+0000 7f99e50d5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99e00fefe0 0x7f99e0196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:44.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.429+0000 7f99e50d5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99e00fefe0 0x7f99e0196b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49036/0 (socket says 172.21.15.114:49036) 2024-02-16T22:33:44.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.429+0000 7f99e50d5640 1 -- 172.21.15.114:0/723242259 learned_addr learned my addr 172.21.15.114:0/723242259 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:44.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.430+0000 7f99e48d4640 1 --2- 172.21.15.114:0/723242259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99e0103f20 0x7f99e0197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:44.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.430+0000 7f99e50d5640 1 -- 172.21.15.114:0/723242259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99e0103f20 msgr2=0x7f99e0197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:44.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.430+0000 7f99e50d5640 1 --2- 172.21.15.114:0/723242259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99e0103f20 0x7f99e0197080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:44.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.430+0000 7f99e50d5640 1 -- 172.21.15.114:0/723242259 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d0009560 con 0x7f99e00fefe0 2024-02-16T22:33:44.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.430+0000 7f99e50d5640 1 --2- 172.21.15.114:0/723242259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99e00fefe0 0x7f99e0196b40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f99dc00d6f0 tx=0x7f99dc00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:44.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.430+0000 7f99d67fc640 1 -- 172.21.15.114:0/723242259 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99dc009a30 con 0x7f99e00fefe0 2024-02-16T22:33:44.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.430+0000 7f99e7339640 1 -- 172.21.15.114:0/723242259 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e019fbb0 con 0x7f99e00fefe0 2024-02-16T22:33:44.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.430+0000 7f99d67fc640 1 -- 172.21.15.114:0/723242259 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f99dc0054e0 con 0x7f99e00fefe0 2024-02-16T22:33:44.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.430+0000 7f99d67fc640 1 -- 172.21.15.114:0/723242259 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99dc010460 con 0x7f99e00fefe0 2024-02-16T22:33:44.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.430+0000 7f99e7339640 1 -- 172.21.15.114:0/723242259 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e01a0060 con 0x7f99e00fefe0 2024-02-16T22:33:44.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.431+0000 7f99e7339640 1 -- 172.21.15.114:0/723242259 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e004e040 con 0x7f99e00fefe0 2024-02-16T22:33:44.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.432+0000 7f99d67fc640 1 -- 172.21.15.114:0/723242259 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f99dc009b90 con 0x7f99e00fefe0 2024-02-16T22:33:44.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.432+0000 7f99d67fc640 1 --2- 172.21.15.114:0/723242259 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f99cc06d8d0 0x7f99cc06fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:44.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.432+0000 7f99d67fc640 1 -- 172.21.15.114:0/723242259 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f99dc08d140 con 0x7f99e00fefe0 2024-02-16T22:33:44.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.432+0000 7f99e48d4640 1 --2- 172.21.15.114:0/723242259 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f99cc06d8d0 0x7f99cc06fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:44.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.432+0000 7f99e48d4640 1 --2- 172.21.15.114:0/723242259 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f99cc06d8d0 0x7f99cc06fd90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f99e0067c00 tx=0x7f99d0000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:44.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.436+0000 7f99d67fc640 1 -- 172.21.15.114:0/723242259 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f99dc056840 con 0x7f99e00fefe0 2024-02-16T22:33:44.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.614+0000 7f99e7339640 1 -- 172.21.15.114:0/723242259 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99e010c4e0 con 0x7f99e00fefe0 2024-02-16T22:33:44.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.615+0000 7f99d67fc640 1 -- 172.21.15.114:0/723242259 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+17080 (secure 0 0 0) 0x7f99dc059f00 con 0x7f99e00fefe0 2024-02-16T22:33:44.617 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:44.617 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":82,"fsid":"1dcbaabe-cd18-11ee-95ba-87774f69a715","created":"2024-02-16T22:11:02.186164+0000","modified":"2024-02-16T22:29:37.876619+0000","last_up_change":"2024-02-16T22:28:16.339897+0000","last_in_change":"2024-02-16T22:16:28.224489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-16T22:15:31.749346+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-16T22:18:01.042512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-16T22:18:01.336617+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3a72ccf1-3f93-4c06-99c3-092a7f9a732d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":48,"up_from":52,"up_thru":77,"down_at":49,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":221411803},{"type":"v1","addr":"172.21.15.114:6803","nonce":221411803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":221411803},{"type":"v1","addr":"172.21.15.114:6805","nonce":221411803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":221411803},{"type":"v1","addr":"172.21.15.114:6809","nonce":221411803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":221411803},{"type":"v1","addr":"172.21.15.114:6807","nonce":221411803}]},"public_addr":"172.21.15.114:6803/221411803","cluster_addr":"172.21.15.114:6805/221411803","heartbeat_back_addr":"172.21.15.114:6809/221411803","heartbeat_front_addr":"172.21.15.114:6807/221411803","state":["exists","up"]},{"osd":1,"uuid":"1e8829dd-cac2-462f-aca1-86bcfb475a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":53,"up_from":57,"up_thru":77,"down_at":54,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1152173882},{"type":"v1","addr":"172.21.15.114:6811","nonce":1152173882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1152173882},{"type":"v1","addr":"172.21.15.114:6813","nonce":1152173882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1152173882},{"type":"v1","addr":"172.21.15.114:6817","nonce":1152173882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1152173882},{"type":"v1","addr":"172.21.15.114:6815","nonce":1152173882}]},"public_addr":"172.21.15.114:6811/1152173882","cluster_addr":"172.21.15.114:6813/1152173882","heartbeat_back_addr":"172.21.15.114:6817/1152173882","heartbeat_front_addr":"172.21.15.114:6815/1152173882","state":["exists","up"]},{"osd":2,"uuid":"44f9fce8-5ea8-4741-9936-6720df91b628","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":58,"up_from":62,"up_thru":77,"down_at":59,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":3638870673},{"type":"v1","addr":"172.21.15.114:6819","nonce":3638870673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":3638870673},{"type":"v1","addr":"172.21.15.114:6821","nonce":3638870673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":3638870673},{"type":"v1","addr":"172.21.15.114:6825","nonce":3638870673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":3638870673},{"type":"v1","addr":"172.21.15.114:6823","nonce":3638870673}]},"public_addr":"172.21.15.114:6819/3638870673","cluster_addr":"172.21.15.114:6821/3638870673","heartbeat_back_addr":"172.21.15.114:6825/3638870673","heartbeat_front_addr":"172.21.15.114:6823/3638870673","state":["exists","up"]},{"osd":3,"uuid":"b6eb4941-bef4-4722-9050-cd6c041f9fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":63,"up_from":67,"up_thru":77,"down_at":64,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3100755210},{"type":"v1","addr":"172.21.15.123:6801","nonce":3100755210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3100755210},{"type":"v1","addr":"172.21.15.123:6803","nonce":3100755210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3100755210},{"type":"v1","addr":"172.21.15.123:6807","nonce":3100755210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3100755210},{"type":"v1","addr":"172.21.15.123:6805","nonce":3100755210}]},"public_addr":"172.21.15.123:6801/3100755210","cluster_addr":"172.21.15.123:6803/3100755210","heartbeat_back_addr":"172.21.15.123:6807/3100755210","heartbeat_front_addr":"172.21.15.123:6805/3100755210","state":["exists","up"]},{"osd":4,"uuid":"e9caa348-26f4-4ed5-a64e-59231fc48546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":68,"up_from":72,"up_thru":77,"down_at":69,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":75423100},{"type":"v1","addr":"172.21.15.123:6809","nonce":75423100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":75423100},{"type":"v1","addr":"172.21.15.123:6811","nonce":75423100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":75423100},{"type":"v1","addr":"172.21.15.123:6815","nonce":75423100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":75423100},{"type":"v1","addr":"172.21.15.123:6813","nonce":75423100}]},"public_addr":"172.21.15.123:6809/75423100","cluster_addr":"172.21.15.123:6811/75423100","heartbeat_back_addr":"172.21.15.123:6815/75423100","heartbeat_front_addr":"172.21.15.123:6813/75423100","state":["exists","up"]},{"osd":5,"uuid":"79d81af1-2361-430d-bc76-35ef74aa6bad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":73,"up_from":77,"up_thru":77,"down_at":74,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":4210417260},{"type":"v1","addr":"172.21.15.123:6817","nonce":4210417260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4210417260},{"type":"v1","addr":"172.21.15.123:6819","nonce":4210417260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4210417260},{"type":"v1","addr":"172.21.15.123:6823","nonce":4210417260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4210417260},{"type":"v1","addr":"172.21.15.123:6821","nonce":4210417260}]},"public_addr":"172.21.15.123:6817/4210417260","cluster_addr":"172.21.15.123:6819/4210417260","heartbeat_back_addr":"172.21.15.123:6823/4210417260","heartbeat_front_addr":"172.21.15.123:6821/4210417260","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:14:32.818023+0000","dead_epoch":48},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:15:00.597924+0000","dead_epoch":53},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:15:29.574022+0000","dead_epoch":58},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:15:53.159458+0000","dead_epoch":63},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:16:20.058254+0000","dead_epoch":68},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:16:45.789725+0000","dead_epoch":73}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6827/2812517585":"2024-02-17T22:29:37.876591+0000","172.21.15.123:6826/2812517585":"2024-02-17T22:29:37.876591+0000","172.21.15.123:6825/418014612":"2024-02-17T22:29:09.777831+0000","172.21.15.123:6824/418014612":"2024-02-17T22:29:09.777831+0000","172.21.15.114:6827/277953535":"2024-02-17T22:28:21.203522+0000","172.21.15.114:6826/277953535":"2024-02-17T22:28:21.203522+0000","172.21.15.114:0/297794530":"2024-02-17T22:23:45.719861+0000","172.21.15.114:6800/2055574123":"2024-02-17T22:11:27.861007+0000","172.21.15.114:6829/865692293":"2024-02-17T22:28:47.638133+0000","172.21.15.114:0/4031283053":"2024-02-17T22:21:10.405873+0000","172.21.15.114:0/1034649687":"2024-02-17T22:13:11.405214+0000","172.21.15.114:0/3991553908":"2024-02-17T22:21:10.405873+0000","172.21.15.114:0/744140874":"2024-02-17T22:12:00.492971+0000","172.21.15.114:6828/865692293":"2024-02-17T22:28:47.638133+0000","172.21.15.114:6800/3153243195":"2024-02-17T22:21:10.405873+0000","172.21.15.114:0/3211326148":"2024-02-17T22:13:11.405214+0000","172.21.15.123:6829/1539029039":"2024-02-17T22:22:31.925771+0000","172.21.15.114:6801/3153243195":"2024-02-17T22:21:10.405873+0000","172.21.15.114:6801/2055574123":"2024-02-17T22:11:27.861007+0000","172.21.15.123:6828/1539029039":"2024-02-17T22:22:31.925771+0000","172.21.15.114:0/2393113501":"2024-02-17T22:21:10.405873+0000","172.21.15.114:6800/2648536185":"2024-02-17T22:13:11.405214+0000","172.21.15.114:0/693887539":"2024-02-17T22:21:10.405873+0000","172.21.15.114:6800/625910309":"2024-02-17T22:12:00.492971+0000","172.21.15.123:0/2858796357":"2024-02-17T22:22:31.925771+0000","172.21.15.114:0/4029631970":"2024-02-17T22:11:27.861007+0000","172.21.15.114:0/2206519078":"2024-02-17T22:23:45.719861+0000","172.21.15.114:6801/2648536185":"2024-02-17T22:13:11.405214+0000","172.21.15.123:0/1942138241":"2024-02-17T22:22:31.925771+0000","172.21.15.123:0/216490899":"2024-02-17T22:22:31.925771+0000","172.21.15.114:0/3971798486":"2024-02-17T22:11:27.861007+0000","172.21.15.114:0/499892900":"2024-02-17T22:12:00.492971+0000","172.21.15.114:0/1349416361":"2024-02-17T22:12:00.492971+0000","172.21.15.123:0/2896457931":"2024-02-17T22:22:31.925771+0000","172.21.15.123:0/3641186660":"2024-02-17T22:22:31.925771+0000","172.21.15.114:0/125196455":"2024-02-17T22:23:45.719861+0000","172.21.15.114:6801/625910309":"2024-02-17T22:12:00.492971+0000","172.21.15.114:0/2762476246":"2024-02-17T22:13:11.405214+0000","172.21.15.123:0/4134230222":"2024-02-17T22:22:31.925771+0000","172.21.15.114:6800/3051780151":"2024-02-17T22:23:45.719861+0000","172.21.15.114:0/489142132":"2024-02-17T22:23:45.719861+0000","172.21.15.114:6801/3051780151":"2024-02-17T22:23:45.719861+0000","172.21.15.114:0/1088872575":"2024-02-17T22:23:45.719861+0000","172.21.15.114:0/2825621999":"2024-02-17T22:11:27.861007+0000","172.21.15.114:0/1250197670":"2024-02-17T22:23:45.719861+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-16T22:33:44.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.617+0000 7f99e7339640 1 -- 172.21.15.114:0/723242259 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f99cc06d8d0 msgr2=0x7f99cc06fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:44.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.617+0000 7f99e7339640 1 --2- 172.21.15.114:0/723242259 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f99cc06d8d0 0x7f99cc06fd90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f99e0067c00 tx=0x7f99d0000bc0 comp rx=0 tx=0).stop 2024-02-16T22:33:44.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.617+0000 7f99e7339640 1 -- 172.21.15.114:0/723242259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99e00fefe0 msgr2=0x7f99e0196b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:44.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.617+0000 7f99e7339640 1 --2- 172.21.15.114:0/723242259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99e00fefe0 0x7f99e0196b40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f99dc00d6f0 tx=0x7f99dc00dbb0 comp rx=0 tx=0).stop 2024-02-16T22:33:44.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.617+0000 7f99e7339640 1 -- 172.21.15.114:0/723242259 shutdown_connections 2024-02-16T22:33:44.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.617+0000 7f99e7339640 1 --2- 172.21.15.114:0/723242259 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f99cc06d8d0 0x7f99cc06fd90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:44.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.617+0000 7f99e7339640 1 --2- 172.21.15.114:0/723242259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99e0103f20 0x7f99e0197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:44.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.617+0000 7f99e7339640 1 --2- 172.21.15.114:0/723242259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99e00fefe0 0x7f99e0196b40 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:44.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.617+0000 7f99e7339640 1 -- 172.21.15.114:0/723242259 >> 172.21.15.114:0/723242259 conn(0x7f99e0075b40 msgr2=0x7f99e00fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:44.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.617+0000 7f99e7339640 1 -- 172.21.15.114:0/723242259 shutdown_connections 2024-02-16T22:33:44.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.617+0000 7f99e7339640 1 -- 172.21.15.114:0/723242259 wait complete. 2024-02-16T22:33:44.630 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-16T22:15:31.749346+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-16T22:18:01.042512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-16T22:18:01.336617+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-16T22:33:44.631 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-16T22:33:44.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.734+0000 7f31a1270640 1 -- 172.21.15.114:0/2270338051 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f319c101150 msgr2=0x7f319c101530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:44.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.734+0000 7f31a1270640 1 --2- 172.21.15.114:0/2270338051 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f319c101150 0x7f319c101530 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f318c0098b0 tx=0x7f318c02bcd0 comp rx=0 tx=0).stop 2024-02-16T22:33:44.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.734+0000 7f31a1270640 1 -- 172.21.15.114:0/2270338051 shutdown_connections 2024-02-16T22:33:44.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.734+0000 7f31a1270640 1 --2- 172.21.15.114:0/2270338051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f319c101b00 0x7f319c109fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:44.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.734+0000 7f31a1270640 1 --2- 172.21.15.114:0/2270338051 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f319c101150 0x7f319c101530 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:44.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.734+0000 7f31a1270640 1 -- 172.21.15.114:0/2270338051 >> 172.21.15.114:0/2270338051 conn(0x7f319c0faa40 msgr2=0x7f319c0fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:44.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.734+0000 7f31a1270640 1 -- 172.21.15.114:0/2270338051 shutdown_connections 2024-02-16T22:33:44.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.734+0000 7f31a1270640 1 -- 172.21.15.114:0/2270338051 wait complete. 2024-02-16T22:33:44.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.735+0000 7f31a1270640 1 Processor -- start 2024-02-16T22:33:44.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.735+0000 7f31a1270640 1 -- start start 2024-02-16T22:33:44.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.735+0000 7f31a1270640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f319c101150 0x7f319c195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:44.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.735+0000 7f31a1270640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f319c101b00 0x7f319c1963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:44.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.735+0000 7f31a1270640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319c196a60 con 0x7f319c101b00 2024-02-16T22:33:44.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.735+0000 7f31a1270640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319c19a830 con 0x7f319c101150 2024-02-16T22:33:44.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.735+0000 7f319a7fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f319c101b00 0x7f319c1963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:44.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.735+0000 7f319a7fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f319c101b00 0x7f319c1963d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49046/0 (socket says 172.21.15.114:49046) 2024-02-16T22:33:44.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.735+0000 7f319a7fc640 1 -- 172.21.15.114:0/2064688614 learned_addr learned my addr 172.21.15.114:0/2064688614 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:44.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.735+0000 7f319a7fc640 1 -- 172.21.15.114:0/2064688614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f319c101150 msgr2=0x7f319c195e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:44.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.735+0000 7f319affd640 1 --2- 172.21.15.114:0/2064688614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f319c101150 0x7f319c195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:44.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.735+0000 7f319a7fc640 1 --2- 172.21.15.114:0/2064688614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f319c101150 0x7f319c195e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:44.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.735+0000 7f319a7fc640 1 -- 172.21.15.114:0/2064688614 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318c009560 con 0x7f319c101b00 2024-02-16T22:33:44.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.735+0000 7f319a7fc640 1 --2- 172.21.15.114:0/2064688614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f319c101b00 0x7f319c1963d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f319000b420 tx=0x7f319000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:44.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.736+0000 7f3183fff640 1 -- 172.21.15.114:0/2064688614 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3190011800 con 0x7f319c101b00 2024-02-16T22:33:44.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.736+0000 7f31a1270640 1 -- 172.21.15.114:0/2064688614 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f319c19ab10 con 0x7f319c101b00 2024-02-16T22:33:44.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.736+0000 7f3183fff640 1 -- 172.21.15.114:0/2064688614 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3190011e20 con 0x7f319c101b00 2024-02-16T22:33:44.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.736+0000 7f3183fff640 1 -- 172.21.15.114:0/2064688614 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f319000f2e0 con 0x7f319c101b00 2024-02-16T22:33:44.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.736+0000 7f31a1270640 1 -- 172.21.15.114:0/2064688614 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f319c19b070 con 0x7f319c101b00 2024-02-16T22:33:44.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.737+0000 7f31a1270640 1 -- 172.21.15.114:0/2064688614 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f319c04e040 con 0x7f319c101b00 2024-02-16T22:33:44.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.737+0000 7f3183fff640 1 -- 172.21.15.114:0/2064688614 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f3190010380 con 0x7f319c101b00 2024-02-16T22:33:44.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.738+0000 7f3183fff640 1 --2- 172.21.15.114:0/2064688614 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f318406d810 0x7f318406fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:44.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.738+0000 7f3183fff640 1 -- 172.21.15.114:0/2064688614 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f319008cf70 con 0x7f319c101b00 2024-02-16T22:33:44.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.738+0000 7f319affd640 1 --2- 172.21.15.114:0/2064688614 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f318406d810 0x7f318406fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:44.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.738+0000 7f319affd640 1 --2- 172.21.15.114:0/2064688614 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f318406d810 0x7f318406fcd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f318c036040 tx=0x7f318c005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:44.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.742+0000 7f3183fff640 1 -- 172.21.15.114:0/2064688614 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f31900c0d20 con 0x7f319c101b00 2024-02-16T22:33:44.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.915+0000 7f31a1270640 1 -- 172.21.15.114:0/2064688614 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f319c10d310 con 0x7f319c101b00 2024-02-16T22:33:44.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.915+0000 7f3183fff640 1 -- 172.21.15.114:0/2064688614 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3190056670 con 0x7f319c101b00 2024-02-16T22:33:44.917 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2024-02-16T22:33:44.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.917+0000 7f31a1270640 1 -- 172.21.15.114:0/2064688614 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f318406d810 msgr2=0x7f318406fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:44.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.917+0000 7f31a1270640 1 --2- 172.21.15.114:0/2064688614 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f318406d810 0x7f318406fcd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f318c036040 tx=0x7f318c005fd0 comp rx=0 tx=0).stop 2024-02-16T22:33:44.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.917+0000 7f31a1270640 1 -- 172.21.15.114:0/2064688614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f319c101b00 msgr2=0x7f319c1963d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:44.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.917+0000 7f31a1270640 1 --2- 172.21.15.114:0/2064688614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f319c101b00 0x7f319c1963d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f319000b420 tx=0x7f319000b8e0 comp rx=0 tx=0).stop 2024-02-16T22:33:44.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.917+0000 7f31a1270640 1 -- 172.21.15.114:0/2064688614 shutdown_connections 2024-02-16T22:33:44.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.917+0000 7f31a1270640 1 --2- 172.21.15.114:0/2064688614 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f318406d810 0x7f318406fcd0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:44.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.917+0000 7f31a1270640 1 --2- 172.21.15.114:0/2064688614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f319c101b00 0x7f319c1963d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:44.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.917+0000 7f31a1270640 1 --2- 172.21.15.114:0/2064688614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f319c101150 0x7f319c195e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:44.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.917+0000 7f31a1270640 1 -- 172.21.15.114:0/2064688614 >> 172.21.15.114:0/2064688614 conn(0x7f319c0faa40 msgr2=0x7f319c104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:44.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.917+0000 7f31a1270640 1 -- 172.21.15.114:0/2064688614 shutdown_connections 2024-02-16T22:33:44.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:44.917+0000 7f31a1270640 1 -- 172.21.15.114:0/2064688614 wait complete. 2024-02-16T22:33:44.928 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:44 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/2479680295' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-02-16T22:33:44.928 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:44 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/3935732404' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-02-16T22:33:44.928 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:44 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/723242259' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T22:33:44.929 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-16T22:33:45.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.032+0000 7fe8008d5640 1 -- 172.21.15.114:0/792810639 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7fc0ff260 msgr2=0x7fe7fc0ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:45.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.032+0000 7fe8008d5640 1 --2- 172.21.15.114:0/792810639 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7fc0ff260 0x7fe7fc0ff640 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009880 tx=0x7fe7f002bce0 comp rx=0 tx=0).stop 2024-02-16T22:33:45.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.032+0000 7fe8008d5640 1 -- 172.21.15.114:0/792810639 shutdown_connections 2024-02-16T22:33:45.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.032+0000 7fe8008d5640 1 --2- 172.21.15.114:0/792810639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7fc0ffc10 0x7fe7fc103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:45.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.032+0000 7fe8008d5640 1 --2- 172.21.15.114:0/792810639 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7fc0ff260 0x7fe7fc0ff640 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:45.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.032+0000 7fe8008d5640 1 -- 172.21.15.114:0/792810639 >> 172.21.15.114:0/792810639 conn(0x7fe7fc074a50 msgr2=0x7fe7fc074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:45.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.032+0000 7fe8008d5640 1 -- 172.21.15.114:0/792810639 shutdown_connections 2024-02-16T22:33:45.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.032+0000 7fe8008d5640 1 -- 172.21.15.114:0/792810639 wait complete. 2024-02-16T22:33:45.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.032+0000 7fe8008d5640 1 Processor -- start 2024-02-16T22:33:45.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.032+0000 7fe8008d5640 1 -- start start 2024-02-16T22:33:45.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.033+0000 7fe8008d5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7fc0ff260 0x7fe7fc196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:45.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.033+0000 7fe8008d5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7fc197150 0x7fe7fc1975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:45.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.033+0000 7fe8008d5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc19fc80 con 0x7fe7fc0ff260 2024-02-16T22:33:45.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.033+0000 7fe8008d5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc19fdc0 con 0x7fe7fc197150 2024-02-16T22:33:45.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.033+0000 7fe7fa59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7fc0ff260 0x7fe7fc196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:45.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.033+0000 7fe7fa59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7fc0ff260 0x7fe7fc196c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49078/0 (socket says 172.21.15.114:49078) 2024-02-16T22:33:45.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.033+0000 7fe7fa59c640 1 -- 172.21.15.114:0/1118403647 learned_addr learned my addr 172.21.15.114:0/1118403647 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:45.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.033+0000 7fe7fa59c640 1 -- 172.21.15.114:0/1118403647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7fc197150 msgr2=0x7fe7fc1975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:45.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.033+0000 7fe7fa59c640 1 --2- 172.21.15.114:0/1118403647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7fc197150 0x7fe7fc1975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:45.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.033+0000 7fe7fa59c640 1 -- 172.21.15.114:0/1118403647 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f0009560 con 0x7fe7fc0ff260 2024-02-16T22:33:45.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.033+0000 7fe7fa59c640 1 --2- 172.21.15.114:0/1118403647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7fc0ff260 0x7fe7fc196c10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009430 tx=0x7fe7f0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:45.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.033+0000 7fe7eb7fe640 1 -- 172.21.15.114:0/1118403647 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7f00035f0 con 0x7fe7fc0ff260 2024-02-16T22:33:45.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.033+0000 7fe7eb7fe640 1 -- 172.21.15.114:0/1118403647 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe7f0003c10 con 0x7fe7fc0ff260 2024-02-16T22:33:45.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.033+0000 7fe8008d5640 1 -- 172.21.15.114:0/1118403647 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7fc19ffc0 con 0x7fe7fc0ff260 2024-02-16T22:33:45.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.033+0000 7fe7eb7fe640 1 -- 172.21.15.114:0/1118403647 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7f0005670 con 0x7fe7fc0ff260 2024-02-16T22:33:45.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.033+0000 7fe8008d5640 1 -- 172.21.15.114:0/1118403647 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7fc1a0440 con 0x7fe7fc0ff260 2024-02-16T22:33:45.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.034+0000 7fe8008d5640 1 -- 172.21.15.114:0/1118403647 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7fc04e040 con 0x7fe7fc0ff260 2024-02-16T22:33:45.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.035+0000 7fe7eb7fe640 1 -- 172.21.15.114:0/1118403647 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fe7f0004dd0 con 0x7fe7fc0ff260 2024-02-16T22:33:45.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.035+0000 7fe7eb7fe640 1 --2- 172.21.15.114:0/1118403647 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe7e406d7c0 0x7fe7e406fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:45.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.036+0000 7fe7f9d9b640 1 --2- 172.21.15.114:0/1118403647 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe7e406d7c0 0x7fe7e406fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:45.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.036+0000 7fe7eb7fe640 1 -- 172.21.15.114:0/1118403647 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7fe7f00af9e0 con 0x7fe7fc0ff260 2024-02-16T22:33:45.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.036+0000 7fe7f9d9b640 1 --2- 172.21.15.114:0/1118403647 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe7e406d7c0 0x7fe7e406fc80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec005fd0 tx=0x7fe7ec005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:45.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.040+0000 7fe7eb7fe640 1 -- 172.21.15.114:0/1118403647 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe7f00790e0 con 0x7fe7fc0ff260 2024-02-16T22:33:45.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:44 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/2479680295' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-02-16T22:33:45.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:44 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/3935732404' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-02-16T22:33:45.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:44 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/723242259' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T22:33:45.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.214+0000 7fe8008d5640 1 -- 172.21.15.114:0/1118403647 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fe7fc107480 con 0x7fe7fc0ff260 2024-02-16T22:33:45.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.215+0000 7fe7eb7fe640 1 -- 172.21.15.114:0/1118403647 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v82) v1 ==== 107+0+11 (secure 0 0 0) 0x7fe7f007c7a0 con 0x7fe7fc0ff260 2024-02-16T22:33:45.217 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-02-16T22:33:45.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.217+0000 7fe8008d5640 1 -- 172.21.15.114:0/1118403647 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe7e406d7c0 msgr2=0x7fe7e406fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:45.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.217+0000 7fe8008d5640 1 --2- 172.21.15.114:0/1118403647 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe7e406d7c0 0x7fe7e406fc80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec005fd0 tx=0x7fe7ec005850 comp rx=0 tx=0).stop 2024-02-16T22:33:45.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.217+0000 7fe8008d5640 1 -- 172.21.15.114:0/1118403647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7fc0ff260 msgr2=0x7fe7fc196c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:45.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.217+0000 7fe8008d5640 1 --2- 172.21.15.114:0/1118403647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7fc0ff260 0x7fe7fc196c10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009430 tx=0x7fe7f0004040 comp rx=0 tx=0).stop 2024-02-16T22:33:45.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.217+0000 7fe8008d5640 1 -- 172.21.15.114:0/1118403647 shutdown_connections 2024-02-16T22:33:45.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.217+0000 7fe8008d5640 1 --2- 172.21.15.114:0/1118403647 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe7e406d7c0 0x7fe7e406fc80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:45.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.217+0000 7fe8008d5640 1 --2- 172.21.15.114:0/1118403647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7fc197150 0x7fe7fc1975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:45.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.217+0000 7fe8008d5640 1 --2- 172.21.15.114:0/1118403647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7fc0ff260 0x7fe7fc196c10 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:45.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.217+0000 7fe8008d5640 1 -- 172.21.15.114:0/1118403647 >> 172.21.15.114:0/1118403647 conn(0x7fe7fc074a50 msgr2=0x7fe7fc0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:45.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.217+0000 7fe8008d5640 1 -- 172.21.15.114:0/1118403647 shutdown_connections 2024-02-16T22:33:45.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.217+0000 7fe8008d5640 1 -- 172.21.15.114:0/1118403647 wait complete. 2024-02-16T22:33:45.229 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-16T22:33:45.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.332+0000 7f018da70640 1 -- 172.21.15.114:0/3946986487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0188067f70 msgr2=0x7f0188104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:45.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.332+0000 7f018da70640 1 --2- 172.21.15.114:0/3946986487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0188067f70 0x7f0188104c70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0170009880 tx=0x7f017002bce0 comp rx=0 tx=0).stop 2024-02-16T22:33:45.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.332+0000 7f018da70640 1 -- 172.21.15.114:0/3946986487 shutdown_connections 2024-02-16T22:33:45.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.332+0000 7f018da70640 1 --2- 172.21.15.114:0/3946986487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0188067f70 0x7f0188104c70 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:45.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.332+0000 7f018da70640 1 --2- 172.21.15.114:0/3946986487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01880675c0 0x7f01880679a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:45.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.332+0000 7f018da70640 1 -- 172.21.15.114:0/3946986487 >> 172.21.15.114:0/3946986487 conn(0x7f0188074a50 msgr2=0x7f0188074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:45.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.332+0000 7f018da70640 1 -- 172.21.15.114:0/3946986487 shutdown_connections 2024-02-16T22:33:45.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.332+0000 7f018da70640 1 -- 172.21.15.114:0/3946986487 wait complete. 2024-02-16T22:33:45.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.332+0000 7f018da70640 1 Processor -- start 2024-02-16T22:33:45.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.332+0000 7f018da70640 1 -- start start 2024-02-16T22:33:45.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.333+0000 7f018da70640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01880675c0 0x7f0188195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:45.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.333+0000 7f018da70640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01881962c0 0x7f018819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:45.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.333+0000 7f018da70640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01881968c0 con 0x7f01880675c0 2024-02-16T22:33:45.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.333+0000 7f018da70640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0188196a30 con 0x7f01881962c0 2024-02-16T22:33:45.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.333+0000 7f01877fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01880675c0 0x7f0188195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:45.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.333+0000 7f01877fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01880675c0 0x7f0188195d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49098/0 (socket says 172.21.15.114:49098) 2024-02-16T22:33:45.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.333+0000 7f01877fe640 1 -- 172.21.15.114:0/4170974520 learned_addr learned my addr 172.21.15.114:0/4170974520 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:45.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.333+0000 7f0186ffd640 1 --2- 172.21.15.114:0/4170974520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01881962c0 0x7f018819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:45.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.333+0000 7f01877fe640 1 -- 172.21.15.114:0/4170974520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01881962c0 msgr2=0x7f018819a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:45.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.333+0000 7f01877fe640 1 --2- 172.21.15.114:0/4170974520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01881962c0 0x7f018819a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:45.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.333+0000 7f01877fe640 1 -- 172.21.15.114:0/4170974520 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0170009560 con 0x7f01880675c0 2024-02-16T22:33:45.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.333+0000 7f01877fe640 1 --2- 172.21.15.114:0/4170974520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01880675c0 0x7f0188195d80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f017800ea30 tx=0x7f017800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:45.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.333+0000 7f0184ff9640 1 -- 172.21.15.114:0/4170974520 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f017800cdb0 con 0x7f01880675c0 2024-02-16T22:33:45.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.333+0000 7f0184ff9640 1 -- 172.21.15.114:0/4170974520 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0178005020 con 0x7f01880675c0 2024-02-16T22:33:45.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.333+0000 7f018da70640 1 -- 172.21.15.114:0/4170974520 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f018819ad20 con 0x7f01880675c0 2024-02-16T22:33:45.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.333+0000 7f0184ff9640 1 -- 172.21.15.114:0/4170974520 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01780116a0 con 0x7f01880675c0 2024-02-16T22:33:45.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.333+0000 7f018da70640 1 -- 172.21.15.114:0/4170974520 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f018819b280 con 0x7f01880675c0 2024-02-16T22:33:45.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.334+0000 7f018da70640 1 -- 172.21.15.114:0/4170974520 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f018819b560 con 0x7f01880675c0 2024-02-16T22:33:45.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.335+0000 7f0184ff9640 1 -- 172.21.15.114:0/4170974520 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f0178005190 con 0x7f01880675c0 2024-02-16T22:33:45.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.335+0000 7f0184ff9640 1 --2- 172.21.15.114:0/4170974520 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f017406d810 0x7f017406fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:45.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.335+0000 7f0186ffd640 1 --2- 172.21.15.114:0/4170974520 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f017406d810 0x7f017406fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:45.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.335+0000 7f0184ff9640 1 -- 172.21.15.114:0/4170974520 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f017808da90 con 0x7f01880675c0 2024-02-16T22:33:45.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.336+0000 7f0186ffd640 1 --2- 172.21.15.114:0/4170974520 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f017406d810 0x7f017406fcd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0170006010 tx=0x7f0170000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:45.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.340+0000 7f0184ff9640 1 -- 172.21.15.114:0/4170974520 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0178057190 con 0x7f01880675c0 2024-02-16T22:33:45.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.513+0000 7f018da70640 1 -- 172.21.15.114:0/4170974520 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f018804e040 con 0x7f01880675c0 2024-02-16T22:33:45.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.514+0000 7f0184ff9640 1 -- 172.21.15.114:0/4170974520 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v82) v1 ==== 107+0+11 (secure 0 0 0) 0x7f017805a850 con 0x7f01880675c0 2024-02-16T22:33:45.516 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-02-16T22:33:45.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.516+0000 7f018da70640 1 -- 172.21.15.114:0/4170974520 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f017406d810 msgr2=0x7f017406fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:45.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.516+0000 7f018da70640 1 --2- 172.21.15.114:0/4170974520 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f017406d810 0x7f017406fcd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0170006010 tx=0x7f0170000bc0 comp rx=0 tx=0).stop 2024-02-16T22:33:45.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.516+0000 7f018da70640 1 -- 172.21.15.114:0/4170974520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01880675c0 msgr2=0x7f0188195d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:45.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.516+0000 7f018da70640 1 --2- 172.21.15.114:0/4170974520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01880675c0 0x7f0188195d80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f017800ea30 tx=0x7f017800eef0 comp rx=0 tx=0).stop 2024-02-16T22:33:45.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.516+0000 7f018da70640 1 -- 172.21.15.114:0/4170974520 shutdown_connections 2024-02-16T22:33:45.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.516+0000 7f018da70640 1 --2- 172.21.15.114:0/4170974520 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f017406d810 0x7f017406fcd0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:45.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.516+0000 7f018da70640 1 --2- 172.21.15.114:0/4170974520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01881962c0 0x7f018819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:45.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.516+0000 7f018da70640 1 --2- 172.21.15.114:0/4170974520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01880675c0 0x7f0188195d80 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:45.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.516+0000 7f018da70640 1 -- 172.21.15.114:0/4170974520 >> 172.21.15.114:0/4170974520 conn(0x7f0188074a50 msgr2=0x7f01880fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:45.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.516+0000 7f018da70640 1 -- 172.21.15.114:0/4170974520 shutdown_connections 2024-02-16T22:33:45.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.516+0000 7f018da70640 1 -- 172.21.15.114:0/4170974520 wait complete. 2024-02-16T22:33:45.528 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-16T22:33:45.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.629+0000 7f68583cb640 1 -- 172.21.15.114:0/1240124691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6850100270 msgr2=0x7f6850100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:45.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.629+0000 7f68583cb640 1 --2- 172.21.15.114:0/1240124691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6850100270 0x7f6850100650 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f684c009880 tx=0x7f684c02bce0 comp rx=0 tx=0).stop 2024-02-16T22:33:45.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.629+0000 7f68583cb640 1 -- 172.21.15.114:0/1240124691 shutdown_connections 2024-02-16T22:33:45.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.629+0000 7f68583cb640 1 --2- 172.21.15.114:0/1240124691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6850100c20 0x7f6850104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:45.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.629+0000 7f68583cb640 1 --2- 172.21.15.114:0/1240124691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6850100270 0x7f6850100650 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:45.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.629+0000 7f68583cb640 1 -- 172.21.15.114:0/1240124691 >> 172.21.15.114:0/1240124691 conn(0x7f68500fbad0 msgr2=0x7f68500fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:45.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.629+0000 7f68583cb640 1 -- 172.21.15.114:0/1240124691 shutdown_connections 2024-02-16T22:33:45.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.629+0000 7f68583cb640 1 -- 172.21.15.114:0/1240124691 wait complete. 2024-02-16T22:33:45.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.630+0000 7f68583cb640 1 Processor -- start 2024-02-16T22:33:45.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.630+0000 7f68583cb640 1 -- start start 2024-02-16T22:33:45.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.630+0000 7f68583cb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6850100270 0x7f6850195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:45.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.630+0000 7f68583cb640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6850196200 0x7f68501995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:45.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.630+0000 7f68583cb640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6850196800 con 0x7f6850100270 2024-02-16T22:33:45.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.630+0000 7f68583cb640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6850196940 con 0x7f6850196200 2024-02-16T22:33:45.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.630+0000 7f6856167640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6850100270 0x7f6850195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:45.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.630+0000 7f6856167640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6850100270 0x7f6850195cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49114/0 (socket says 172.21.15.114:49114) 2024-02-16T22:33:45.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.630+0000 7f6856167640 1 -- 172.21.15.114:0/368162642 learned_addr learned my addr 172.21.15.114:0/368162642 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:45.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.630+0000 7f6855966640 1 --2- 172.21.15.114:0/368162642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6850196200 0x7f68501995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:45.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.630+0000 7f6856167640 1 -- 172.21.15.114:0/368162642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6850196200 msgr2=0x7f68501995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:45.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.630+0000 7f6856167640 1 --2- 172.21.15.114:0/368162642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6850196200 0x7f68501995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:45.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.630+0000 7f6856167640 1 -- 172.21.15.114:0/368162642 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c009560 con 0x7f6850100270 2024-02-16T22:33:45.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.630+0000 7f6856167640 1 --2- 172.21.15.114:0/368162642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6850100270 0x7f6850195cc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f684c009230 tx=0x7f684c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:45.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.630+0000 7f68477fe640 1 -- 172.21.15.114:0/368162642 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f684c0035f0 con 0x7f6850100270 2024-02-16T22:33:45.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.630+0000 7f68477fe640 1 -- 172.21.15.114:0/368162642 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f684c003c10 con 0x7f6850100270 2024-02-16T22:33:45.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.630+0000 7f68583cb640 1 -- 172.21.15.114:0/368162642 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6850199b50 con 0x7f6850100270 2024-02-16T22:33:45.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.631+0000 7f68477fe640 1 -- 172.21.15.114:0/368162642 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f684c005670 con 0x7f6850100270 2024-02-16T22:33:45.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.631+0000 7f68583cb640 1 -- 172.21.15.114:0/368162642 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685019a0b0 con 0x7f6850100270 2024-02-16T22:33:45.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.631+0000 7f68583cb640 1 -- 172.21.15.114:0/368162642 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685004e040 con 0x7f6850100270 2024-02-16T22:33:45.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.632+0000 7f68477fe640 1 -- 172.21.15.114:0/368162642 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f684c004dd0 con 0x7f6850100270 2024-02-16T22:33:45.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.632+0000 7f68477fe640 1 --2- 172.21.15.114:0/368162642 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f683c06d870 0x7f683c06fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:45.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.632+0000 7f68477fe640 1 -- 172.21.15.114:0/368162642 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f684c0af9c0 con 0x7f6850100270 2024-02-16T22:33:45.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.632+0000 7f6855966640 1 --2- 172.21.15.114:0/368162642 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f683c06d870 0x7f683c06fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:45.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.633+0000 7f6855966640 1 --2- 172.21.15.114:0/368162642 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f683c06d870 0x7f683c06fd30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6840005fd0 tx=0x7f6840005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:45.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.637+0000 7f68477fe640 1 -- 172.21.15.114:0/368162642 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f684c0790c0 con 0x7f6850100270 2024-02-16T22:33:45.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.822+0000 7f68583cb640 1 -- 172.21.15.114:0/368162642 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6850108570 con 0x7f6850100270 2024-02-16T22:33:45.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.823+0000 7f68477fe640 1 -- 172.21.15.114:0/368162642 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 45 entries v82) v1 ==== 81+0+2700 (secure 0 0 0) 0x7f684c07c780 con 0x7f6850100270 2024-02-16T22:33:45.825 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.123:6827/2812517585 2024-02-17T22:29:37.876591+0000 2024-02-16T22:33:45.825 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.123:6826/2812517585 2024-02-17T22:29:37.876591+0000 2024-02-16T22:33:45.825 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.123:6825/418014612 2024-02-17T22:29:09.777831+0000 2024-02-16T22:33:45.825 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.123:6824/418014612 2024-02-17T22:29:09.777831+0000 2024-02-16T22:33:45.825 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6827/277953535 2024-02-17T22:28:21.203522+0000 2024-02-16T22:33:45.825 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6826/277953535 2024-02-17T22:28:21.203522+0000 2024-02-16T22:33:45.825 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/297794530 2024-02-17T22:23:45.719861+0000 2024-02-16T22:33:45.825 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6800/2055574123 2024-02-17T22:11:27.861007+0000 2024-02-16T22:33:45.825 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6829/865692293 2024-02-17T22:28:47.638133+0000 2024-02-16T22:33:45.825 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/4031283053 2024-02-17T22:21:10.405873+0000 2024-02-16T22:33:45.826 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/1034649687 2024-02-17T22:13:11.405214+0000 2024-02-16T22:33:45.826 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/3991553908 2024-02-17T22:21:10.405873+0000 2024-02-16T22:33:45.826 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/744140874 2024-02-17T22:12:00.492971+0000 2024-02-16T22:33:45.826 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6828/865692293 2024-02-17T22:28:47.638133+0000 2024-02-16T22:33:45.826 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6800/3153243195 2024-02-17T22:21:10.405873+0000 2024-02-16T22:33:45.826 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/3211326148 2024-02-17T22:13:11.405214+0000 2024-02-16T22:33:45.826 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.123:6829/1539029039 2024-02-17T22:22:31.925771+0000 2024-02-16T22:33:45.826 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6801/3153243195 2024-02-17T22:21:10.405873+0000 2024-02-16T22:33:45.826 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6801/2055574123 2024-02-17T22:11:27.861007+0000 2024-02-16T22:33:45.826 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.123:6828/1539029039 2024-02-17T22:22:31.925771+0000 2024-02-16T22:33:45.826 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2393113501 2024-02-17T22:21:10.405873+0000 2024-02-16T22:33:45.826 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6800/2648536185 2024-02-17T22:13:11.405214+0000 2024-02-16T22:33:45.826 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/693887539 2024-02-17T22:21:10.405873+0000 2024-02-16T22:33:45.826 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6800/625910309 2024-02-17T22:12:00.492971+0000 2024-02-16T22:33:45.826 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.123:0/2858796357 2024-02-17T22:22:31.925771+0000 2024-02-16T22:33:45.826 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/4029631970 2024-02-17T22:11:27.861007+0000 2024-02-16T22:33:45.827 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2206519078 2024-02-17T22:23:45.719861+0000 2024-02-16T22:33:45.827 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6801/2648536185 2024-02-17T22:13:11.405214+0000 2024-02-16T22:33:45.827 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.123:0/1942138241 2024-02-17T22:22:31.925771+0000 2024-02-16T22:33:45.827 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.123:0/216490899 2024-02-17T22:22:31.925771+0000 2024-02-16T22:33:45.827 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/3971798486 2024-02-17T22:11:27.861007+0000 2024-02-16T22:33:45.827 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/499892900 2024-02-17T22:12:00.492971+0000 2024-02-16T22:33:45.827 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/1349416361 2024-02-17T22:12:00.492971+0000 2024-02-16T22:33:45.827 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.123:0/2896457931 2024-02-17T22:22:31.925771+0000 2024-02-16T22:33:45.827 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.123:0/3641186660 2024-02-17T22:22:31.925771+0000 2024-02-16T22:33:45.827 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/125196455 2024-02-17T22:23:45.719861+0000 2024-02-16T22:33:45.827 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6801/625910309 2024-02-17T22:12:00.492971+0000 2024-02-16T22:33:45.827 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2762476246 2024-02-17T22:13:11.405214+0000 2024-02-16T22:33:45.827 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.123:0/4134230222 2024-02-17T22:22:31.925771+0000 2024-02-16T22:33:45.827 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6800/3051780151 2024-02-17T22:23:45.719861+0000 2024-02-16T22:33:45.827 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/489142132 2024-02-17T22:23:45.719861+0000 2024-02-16T22:33:45.827 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6801/3051780151 2024-02-17T22:23:45.719861+0000 2024-02-16T22:33:45.828 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/1088872575 2024-02-17T22:23:45.719861+0000 2024-02-16T22:33:45.828 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2825621999 2024-02-17T22:11:27.861007+0000 2024-02-16T22:33:45.828 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/1250197670 2024-02-17T22:23:45.719861+0000 2024-02-16T22:33:45.828 INFO:teuthology.orchestra.run.smithi114.stderr:listed 45 entries 2024-02-16T22:33:45.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.825+0000 7f68583cb640 1 -- 172.21.15.114:0/368162642 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f683c06d870 msgr2=0x7f683c06fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:45.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.825+0000 7f68583cb640 1 --2- 172.21.15.114:0/368162642 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f683c06d870 0x7f683c06fd30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6840005fd0 tx=0x7f6840005850 comp rx=0 tx=0).stop 2024-02-16T22:33:45.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.825+0000 7f68583cb640 1 -- 172.21.15.114:0/368162642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6850100270 msgr2=0x7f6850195cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:45.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.825+0000 7f68583cb640 1 --2- 172.21.15.114:0/368162642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6850100270 0x7f6850195cc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f684c009230 tx=0x7f684c004040 comp rx=0 tx=0).stop 2024-02-16T22:33:45.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.825+0000 7f68583cb640 1 -- 172.21.15.114:0/368162642 shutdown_connections 2024-02-16T22:33:45.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.825+0000 7f68583cb640 1 --2- 172.21.15.114:0/368162642 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f683c06d870 0x7f683c06fd30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:45.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.825+0000 7f68583cb640 1 --2- 172.21.15.114:0/368162642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6850196200 0x7f68501995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:45.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.825+0000 7f68583cb640 1 --2- 172.21.15.114:0/368162642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6850100270 0x7f6850195cc0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:45.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.825+0000 7f68583cb640 1 -- 172.21.15.114:0/368162642 >> 172.21.15.114:0/368162642 conn(0x7f68500fbad0 msgr2=0x7f68500fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:45.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.825+0000 7f68583cb640 1 -- 172.21.15.114:0/368162642 shutdown_connections 2024-02-16T22:33:45.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.825+0000 7f68583cb640 1 -- 172.21.15.114:0/368162642 wait complete. 2024-02-16T22:33:45.837 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:45 smithi114 ceph-mon[94452]: pgmap v336: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:45.837 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:45 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/2064688614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T22:33:45.837 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:45 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/1118403647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T22:33:45.837 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:45 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/4170974520' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T22:33:45.838 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-02-16T22:33:45.838 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-16T22:33:45.854 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-16T22:33:45.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.993+0000 7f818423a640 1 -- 172.21.15.114:0/1513570076 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f817c0ff180 msgr2=0x7f817c0ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:45.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.993+0000 7f818423a640 1 --2- 172.21.15.114:0/1513570076 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f817c0ff180 0x7f817c0ff560 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8178009c20 tx=0x7f817802c1b0 comp rx=0 tx=0).stop 2024-02-16T22:33:45.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.993+0000 7f818423a640 1 -- 172.21.15.114:0/1513570076 shutdown_connections 2024-02-16T22:33:45.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.993+0000 7f818423a640 1 --2- 172.21.15.114:0/1513570076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f817c0ffb30 0x7f817c103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:45.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.993+0000 7f818423a640 1 --2- 172.21.15.114:0/1513570076 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f817c0ff180 0x7f817c0ff560 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:45.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.993+0000 7f818423a640 1 -- 172.21.15.114:0/1513570076 >> 172.21.15.114:0/1513570076 conn(0x7f817c0faa20 msgr2=0x7f817c0fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:45.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.993+0000 7f818423a640 1 -- 172.21.15.114:0/1513570076 shutdown_connections 2024-02-16T22:33:45.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.993+0000 7f818423a640 1 -- 172.21.15.114:0/1513570076 wait complete. 2024-02-16T22:33:45.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.993+0000 7f818423a640 1 Processor -- start 2024-02-16T22:33:45.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.993+0000 7f818423a640 1 -- start start 2024-02-16T22:33:45.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.994+0000 7f818423a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f817c0ff180 0x7f817c196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:45.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.994+0000 7f818423a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f817c1970e0 0x7f817c197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:45.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.994+0000 7f818423a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817c19fca0 con 0x7f817c0ff180 2024-02-16T22:33:45.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.994+0000 7f818423a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817c19fde0 con 0x7f817c1970e0 2024-02-16T22:33:45.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.994+0000 7f8181fd6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f817c0ff180 0x7f817c196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:45.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.994+0000 7f8181fd6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f817c0ff180 0x7f817c196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49124/0 (socket says 172.21.15.114:49124) 2024-02-16T22:33:45.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.994+0000 7f8181fd6640 1 -- 172.21.15.114:0/1302794796 learned_addr learned my addr 172.21.15.114:0/1302794796 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:45.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.994+0000 7f81817d5640 1 --2- 172.21.15.114:0/1302794796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f817c1970e0 0x7f817c197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:45.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.994+0000 7f8181fd6640 1 -- 172.21.15.114:0/1302794796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f817c1970e0 msgr2=0x7f817c197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:45.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.994+0000 7f8181fd6640 1 --2- 172.21.15.114:0/1302794796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f817c1970e0 0x7f817c197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:45.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.994+0000 7f8181fd6640 1 -- 172.21.15.114:0/1302794796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8178009900 con 0x7f817c0ff180 2024-02-16T22:33:45.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.994+0000 7f8181fd6640 1 --2- 172.21.15.114:0/1302794796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f817c0ff180 0x7f817c196ba0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8178006010 tx=0x7f8178032b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:45.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.994+0000 7f8172ffd640 1 -- 172.21.15.114:0/1302794796 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8178034820 con 0x7f817c0ff180 2024-02-16T22:33:45.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.994+0000 7f8172ffd640 1 -- 172.21.15.114:0/1302794796 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8178003e30 con 0x7f817c0ff180 2024-02-16T22:33:45.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.994+0000 7f818423a640 1 -- 172.21.15.114:0/1302794796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f817c19ff80 con 0x7f817c0ff180 2024-02-16T22:33:45.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.994+0000 7f8172ffd640 1 -- 172.21.15.114:0/1302794796 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f817803f440 con 0x7f817c0ff180 2024-02-16T22:33:45.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.994+0000 7f818423a640 1 -- 172.21.15.114:0/1302794796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f817c1a0400 con 0x7f817c0ff180 2024-02-16T22:33:45.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.995+0000 7f818423a640 1 -- 172.21.15.114:0/1302794796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f817c04e040 con 0x7f817c0ff180 2024-02-16T22:33:45.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.996+0000 7f8172ffd640 1 -- 172.21.15.114:0/1302794796 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f817803b040 con 0x7f817c0ff180 2024-02-16T22:33:45.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.996+0000 7f8172ffd640 1 --2- 172.21.15.114:0/1302794796 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f816806d8d0 0x7f816806fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:45.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.996+0000 7f81817d5640 1 --2- 172.21.15.114:0/1302794796 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f816806d8d0 0x7f816806fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:46.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.996+0000 7f8172ffd640 1 -- 172.21.15.114:0/1302794796 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f81780b03b0 con 0x7f817c0ff180 2024-02-16T22:33:46.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:45.996+0000 7f81817d5640 1 --2- 172.21.15.114:0/1302794796 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f816806d8d0 0x7f816806fd90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f817c069820 tx=0x7f816c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:46.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.000+0000 7f8172ffd640 1 -- 172.21.15.114:0/1302794796 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8178079ab0 con 0x7f817c0ff180 2024-02-16T22:33:46.051 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:45 smithi123 ceph-mon[82331]: pgmap v336: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:46.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:45 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/2064688614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T22:33:46.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:45 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/1118403647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T22:33:46.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:45 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/4170974520' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T22:33:46.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.175+0000 7f818423a640 1 -- 172.21.15.114:0/1302794796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f817c0ffb60 con 0x7f817c0ff180 2024-02-16T22:33:46.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.176+0000 7f8172ffd640 1 -- 172.21.15.114:0/1302794796 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+17080 (secure 0 0 0) 0x7f817807d170 con 0x7f817c0ff180 2024-02-16T22:33:46.178 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-02-16T22:33:46.179 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":82,"fsid":"1dcbaabe-cd18-11ee-95ba-87774f69a715","created":"2024-02-16T22:11:02.186164+0000","modified":"2024-02-16T22:29:37.876619+0000","last_up_change":"2024-02-16T22:28:16.339897+0000","last_in_change":"2024-02-16T22:16:28.224489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-16T22:15:31.749346+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-16T22:18:01.042512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-16T22:18:01.336617+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3a72ccf1-3f93-4c06-99c3-092a7f9a732d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":48,"up_from":52,"up_thru":77,"down_at":49,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":221411803},{"type":"v1","addr":"172.21.15.114:6803","nonce":221411803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":221411803},{"type":"v1","addr":"172.21.15.114:6805","nonce":221411803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":221411803},{"type":"v1","addr":"172.21.15.114:6809","nonce":221411803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":221411803},{"type":"v1","addr":"172.21.15.114:6807","nonce":221411803}]},"public_addr":"172.21.15.114:6803/221411803","cluster_addr":"172.21.15.114:6805/221411803","heartbeat_back_addr":"172.21.15.114:6809/221411803","heartbeat_front_addr":"172.21.15.114:6807/221411803","state":["exists","up"]},{"osd":1,"uuid":"1e8829dd-cac2-462f-aca1-86bcfb475a1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":53,"up_from":57,"up_thru":77,"down_at":54,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1152173882},{"type":"v1","addr":"172.21.15.114:6811","nonce":1152173882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1152173882},{"type":"v1","addr":"172.21.15.114:6813","nonce":1152173882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1152173882},{"type":"v1","addr":"172.21.15.114:6817","nonce":1152173882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1152173882},{"type":"v1","addr":"172.21.15.114:6815","nonce":1152173882}]},"public_addr":"172.21.15.114:6811/1152173882","cluster_addr":"172.21.15.114:6813/1152173882","heartbeat_back_addr":"172.21.15.114:6817/1152173882","heartbeat_front_addr":"172.21.15.114:6815/1152173882","state":["exists","up"]},{"osd":2,"uuid":"44f9fce8-5ea8-4741-9936-6720df91b628","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":58,"up_from":62,"up_thru":77,"down_at":59,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":3638870673},{"type":"v1","addr":"172.21.15.114:6819","nonce":3638870673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":3638870673},{"type":"v1","addr":"172.21.15.114:6821","nonce":3638870673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":3638870673},{"type":"v1","addr":"172.21.15.114:6825","nonce":3638870673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":3638870673},{"type":"v1","addr":"172.21.15.114:6823","nonce":3638870673}]},"public_addr":"172.21.15.114:6819/3638870673","cluster_addr":"172.21.15.114:6821/3638870673","heartbeat_back_addr":"172.21.15.114:6825/3638870673","heartbeat_front_addr":"172.21.15.114:6823/3638870673","state":["exists","up"]},{"osd":3,"uuid":"b6eb4941-bef4-4722-9050-cd6c041f9fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":63,"up_from":67,"up_thru":77,"down_at":64,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3100755210},{"type":"v1","addr":"172.21.15.123:6801","nonce":3100755210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3100755210},{"type":"v1","addr":"172.21.15.123:6803","nonce":3100755210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3100755210},{"type":"v1","addr":"172.21.15.123:6807","nonce":3100755210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3100755210},{"type":"v1","addr":"172.21.15.123:6805","nonce":3100755210}]},"public_addr":"172.21.15.123:6801/3100755210","cluster_addr":"172.21.15.123:6803/3100755210","heartbeat_back_addr":"172.21.15.123:6807/3100755210","heartbeat_front_addr":"172.21.15.123:6805/3100755210","state":["exists","up"]},{"osd":4,"uuid":"e9caa348-26f4-4ed5-a64e-59231fc48546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":68,"up_from":72,"up_thru":77,"down_at":69,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":75423100},{"type":"v1","addr":"172.21.15.123:6809","nonce":75423100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":75423100},{"type":"v1","addr":"172.21.15.123:6811","nonce":75423100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":75423100},{"type":"v1","addr":"172.21.15.123:6815","nonce":75423100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":75423100},{"type":"v1","addr":"172.21.15.123:6813","nonce":75423100}]},"public_addr":"172.21.15.123:6809/75423100","cluster_addr":"172.21.15.123:6811/75423100","heartbeat_back_addr":"172.21.15.123:6815/75423100","heartbeat_front_addr":"172.21.15.123:6813/75423100","state":["exists","up"]},{"osd":5,"uuid":"79d81af1-2361-430d-bc76-35ef74aa6bad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":73,"up_from":77,"up_thru":77,"down_at":74,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":4210417260},{"type":"v1","addr":"172.21.15.123:6817","nonce":4210417260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4210417260},{"type":"v1","addr":"172.21.15.123:6819","nonce":4210417260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4210417260},{"type":"v1","addr":"172.21.15.123:6823","nonce":4210417260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4210417260},{"type":"v1","addr":"172.21.15.123:6821","nonce":4210417260}]},"public_addr":"172.21.15.123:6817/4210417260","cluster_addr":"172.21.15.123:6819/4210417260","heartbeat_back_addr":"172.21.15.123:6823/4210417260","heartbeat_front_addr":"172.21.15.123:6821/4210417260","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:14:32.818023+0000","dead_epoch":48},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:15:00.597924+0000","dead_epoch":53},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:15:29.574022+0000","dead_epoch":58},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:15:53.159458+0000","dead_epoch":63},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:16:20.058254+0000","dead_epoch":68},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T22:16:45.789725+0000","dead_epoch":73}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6827/2812517585":"2024-02-17T22:29:37.876591+0000","172.21.15.123:6826/2812517585":"2024-02-17T22:29:37.876591+0000","172.21.15.123:6825/418014612":"2024-02-17T22:29:09.777831+0000","172.21.15.123:6824/418014612":"2024-02-17T22:29:09.777831+0000","172.21.15.114:6827/277953535":"2024-02-17T22:28:21.203522+0000","172.21.15.114:6826/277953535":"2024-02-17T22:28:21.203522+0000","172.21.15.114:0/297794530":"2024-02-17T22:23:45.719861+0000","172.21.15.114:6800/2055574123":"2024-02-17T22:11:27.861007+0000","172.21.15.114:6829/865692293":"2024-02-17T22:28:47.638133+0000","172.21.15.114:0/4031283053":"2024-02-17T22:21:10.405873+0000","172.21.15.114:0/1034649687":"2024-02-17T22:13:11.405214+0000","172.21.15.114:0/3991553908":"2024-02-17T22:21:10.405873+0000","172.21.15.114:0/744140874":"2024-02-17T22:12:00.492971+0000","172.21.15.114:6828/865692293":"2024-02-17T22:28:47.638133+0000","172.21.15.114:6800/3153243195":"2024-02-17T22:21:10.405873+0000","172.21.15.114:0/3211326148":"2024-02-17T22:13:11.405214+0000","172.21.15.123:6829/1539029039":"2024-02-17T22:22:31.925771+0000","172.21.15.114:6801/3153243195":"2024-02-17T22:21:10.405873+0000","172.21.15.114:6801/2055574123":"2024-02-17T22:11:27.861007+0000","172.21.15.123:6828/1539029039":"2024-02-17T22:22:31.925771+0000","172.21.15.114:0/2393113501":"2024-02-17T22:21:10.405873+0000","172.21.15.114:6800/2648536185":"2024-02-17T22:13:11.405214+0000","172.21.15.114:0/693887539":"2024-02-17T22:21:10.405873+0000","172.21.15.114:6800/625910309":"2024-02-17T22:12:00.492971+0000","172.21.15.123:0/2858796357":"2024-02-17T22:22:31.925771+0000","172.21.15.114:0/4029631970":"2024-02-17T22:11:27.861007+0000","172.21.15.114:0/2206519078":"2024-02-17T22:23:45.719861+0000","172.21.15.114:6801/2648536185":"2024-02-17T22:13:11.405214+0000","172.21.15.123:0/1942138241":"2024-02-17T22:22:31.925771+0000","172.21.15.123:0/216490899":"2024-02-17T22:22:31.925771+0000","172.21.15.114:0/3971798486":"2024-02-17T22:11:27.861007+0000","172.21.15.114:0/499892900":"2024-02-17T22:12:00.492971+0000","172.21.15.114:0/1349416361":"2024-02-17T22:12:00.492971+0000","172.21.15.123:0/2896457931":"2024-02-17T22:22:31.925771+0000","172.21.15.123:0/3641186660":"2024-02-17T22:22:31.925771+0000","172.21.15.114:0/125196455":"2024-02-17T22:23:45.719861+0000","172.21.15.114:6801/625910309":"2024-02-17T22:12:00.492971+0000","172.21.15.114:0/2762476246":"2024-02-17T22:13:11.405214+0000","172.21.15.123:0/4134230222":"2024-02-17T22:22:31.925771+0000","172.21.15.114:6800/3051780151":"2024-02-17T22:23:45.719861+0000","172.21.15.114:0/489142132":"2024-02-17T22:23:45.719861+0000","172.21.15.114:6801/3051780151":"2024-02-17T22:23:45.719861+0000","172.21.15.114:0/1088872575":"2024-02-17T22:23:45.719861+0000","172.21.15.114:0/2825621999":"2024-02-17T22:11:27.861007+0000","172.21.15.114:0/1250197670":"2024-02-17T22:23:45.719861+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-16T22:33:46.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.178+0000 7f818423a640 1 -- 172.21.15.114:0/1302794796 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f816806d8d0 msgr2=0x7f816806fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:46.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.178+0000 7f818423a640 1 --2- 172.21.15.114:0/1302794796 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f816806d8d0 0x7f816806fd90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f817c069820 tx=0x7f816c008040 comp rx=0 tx=0).stop 2024-02-16T22:33:46.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.178+0000 7f818423a640 1 -- 172.21.15.114:0/1302794796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f817c0ff180 msgr2=0x7f817c196ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:46.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.178+0000 7f818423a640 1 --2- 172.21.15.114:0/1302794796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f817c0ff180 0x7f817c196ba0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8178006010 tx=0x7f8178032b60 comp rx=0 tx=0).stop 2024-02-16T22:33:46.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.178+0000 7f818423a640 1 -- 172.21.15.114:0/1302794796 shutdown_connections 2024-02-16T22:33:46.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.178+0000 7f818423a640 1 --2- 172.21.15.114:0/1302794796 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f816806d8d0 0x7f816806fd90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:46.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.178+0000 7f818423a640 1 --2- 172.21.15.114:0/1302794796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f817c1970e0 0x7f817c197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:46.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.178+0000 7f818423a640 1 --2- 172.21.15.114:0/1302794796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f817c0ff180 0x7f817c196ba0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:46.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.178+0000 7f818423a640 1 -- 172.21.15.114:0/1302794796 >> 172.21.15.114:0/1302794796 conn(0x7f817c0faa20 msgr2=0x7f817c0fc220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:46.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.178+0000 7f818423a640 1 -- 172.21.15.114:0/1302794796 shutdown_connections 2024-02-16T22:33:46.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.178+0000 7f818423a640 1 -- 172.21.15.114:0/1302794796 wait complete. 2024-02-16T22:33:46.191 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-16T22:15:31.749346+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-16T22:18:01.042512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-16T22:18:01.336617+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-16T22:33:46.192 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-16T22:33:46.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.297+0000 7efc4cba2640 1 -- 172.21.15.114:0/3052853722 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc48104e20 msgr2=0x7efc48105200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:46.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.297+0000 7efc4cba2640 1 --2- 172.21.15.114:0/3052853722 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc48104e20 0x7efc48105200 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efc38009880 tx=0x7efc3802bce0 comp rx=0 tx=0).stop 2024-02-16T22:33:46.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.297+0000 7efc4cba2640 1 -- 172.21.15.114:0/3052853722 shutdown_connections 2024-02-16T22:33:46.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.297+0000 7efc4cba2640 1 --2- 172.21.15.114:0/3052853722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc480fee00 0x7efc480ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:46.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.297+0000 7efc4cba2640 1 --2- 172.21.15.114:0/3052853722 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc48104e20 0x7efc48105200 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:46.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.297+0000 7efc4cba2640 1 -- 172.21.15.114:0/3052853722 >> 172.21.15.114:0/3052853722 conn(0x7efc48074930 msgr2=0x7efc48074d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:46.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.297+0000 7efc4cba2640 1 -- 172.21.15.114:0/3052853722 shutdown_connections 2024-02-16T22:33:46.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.297+0000 7efc4cba2640 1 -- 172.21.15.114:0/3052853722 wait complete. 2024-02-16T22:33:46.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.298+0000 7efc4cba2640 1 Processor -- start 2024-02-16T22:33:46.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.298+0000 7efc4cba2640 1 -- start start 2024-02-16T22:33:46.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.298+0000 7efc4cba2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc480fee00 0x7efc481969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:46.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.298+0000 7efc4cba2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc48104e20 0x7efc48196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:46.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.298+0000 7efc4cba2640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc481975d0 con 0x7efc480fee00 2024-02-16T22:33:46.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.298+0000 7efc4cba2640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc48197710 con 0x7efc48104e20 2024-02-16T22:33:46.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.298+0000 7efc477fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc480fee00 0x7efc481969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:46.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.299+0000 7efc477fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc480fee00 0x7efc481969b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49136/0 (socket says 172.21.15.114:49136) 2024-02-16T22:33:46.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.299+0000 7efc477fe640 1 -- 172.21.15.114:0/4111266949 learned_addr learned my addr 172.21.15.114:0/4111266949 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:46.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.299+0000 7efc3effd640 1 --2- 172.21.15.114:0/4111266949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc48104e20 0x7efc48196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:46.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.299+0000 7efc477fe640 1 -- 172.21.15.114:0/4111266949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc48104e20 msgr2=0x7efc48196ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:46.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.299+0000 7efc477fe640 1 --2- 172.21.15.114:0/4111266949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc48104e20 0x7efc48196ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:46.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.299+0000 7efc477fe640 1 -- 172.21.15.114:0/4111266949 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc38009560 con 0x7efc480fee00 2024-02-16T22:33:46.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.299+0000 7efc3effd640 1 --2- 172.21.15.114:0/4111266949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc48104e20 0x7efc48196ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T22:33:46.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.299+0000 7efc477fe640 1 --2- 172.21.15.114:0/4111266949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc480fee00 0x7efc481969b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efc38009230 tx=0x7efc38004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:46.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.299+0000 7efc457fa640 1 -- 172.21.15.114:0/4111266949 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc380035f0 con 0x7efc480fee00 2024-02-16T22:33:46.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.299+0000 7efc4cba2640 1 -- 172.21.15.114:0/4111266949 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc4819fa20 con 0x7efc480fee00 2024-02-16T22:33:46.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.299+0000 7efc457fa640 1 -- 172.21.15.114:0/4111266949 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc38003c10 con 0x7efc480fee00 2024-02-16T22:33:46.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.299+0000 7efc457fa640 1 -- 172.21.15.114:0/4111266949 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc38034440 con 0x7efc480fee00 2024-02-16T22:33:46.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.299+0000 7efc4cba2640 1 -- 172.21.15.114:0/4111266949 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc4819fef0 con 0x7efc480fee00 2024-02-16T22:33:46.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.300+0000 7efc4cba2640 1 -- 172.21.15.114:0/4111266949 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc4804e040 con 0x7efc480fee00 2024-02-16T22:33:46.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.301+0000 7efc457fa640 1 -- 172.21.15.114:0/4111266949 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7efc38004fd0 con 0x7efc480fee00 2024-02-16T22:33:46.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.301+0000 7efc457fa640 1 --2- 172.21.15.114:0/4111266949 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7efc3406d810 0x7efc3406fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:46.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.301+0000 7efc457fa640 1 -- 172.21.15.114:0/4111266949 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7efc380afa60 con 0x7efc480fee00 2024-02-16T22:33:46.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.301+0000 7efc3effd640 1 --2- 172.21.15.114:0/4111266949 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7efc3406d810 0x7efc3406fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:46.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.302+0000 7efc3effd640 1 --2- 172.21.15.114:0/4111266949 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7efc3406d810 0x7efc3406fcd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efc48067fe0 tx=0x7efc30005d30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:46.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.306+0000 7efc457fa640 1 -- 172.21.15.114:0/4111266949 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efc38079160 con 0x7efc480fee00 2024-02-16T22:33:46.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.477+0000 7efc4cba2640 1 -- 172.21.15.114:0/4111266949 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efc4810c020 con 0x7efc480fee00 2024-02-16T22:33:46.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.477+0000 7efc457fa640 1 -- 172.21.15.114:0/4111266949 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7efc3807c820 con 0x7efc480fee00 2024-02-16T22:33:46.484 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2024-02-16T22:33:46.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.479+0000 7efc4cba2640 1 -- 172.21.15.114:0/4111266949 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7efc3406d810 msgr2=0x7efc3406fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:46.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.479+0000 7efc4cba2640 1 --2- 172.21.15.114:0/4111266949 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7efc3406d810 0x7efc3406fcd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efc48067fe0 tx=0x7efc30005d30 comp rx=0 tx=0).stop 2024-02-16T22:33:46.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.479+0000 7efc4cba2640 1 -- 172.21.15.114:0/4111266949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc480fee00 msgr2=0x7efc481969b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:46.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.479+0000 7efc4cba2640 1 --2- 172.21.15.114:0/4111266949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc480fee00 0x7efc481969b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efc38009230 tx=0x7efc38004040 comp rx=0 tx=0).stop 2024-02-16T22:33:46.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.479+0000 7efc4cba2640 1 -- 172.21.15.114:0/4111266949 shutdown_connections 2024-02-16T22:33:46.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.479+0000 7efc4cba2640 1 --2- 172.21.15.114:0/4111266949 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7efc3406d810 0x7efc3406fcd0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:46.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.479+0000 7efc4cba2640 1 --2- 172.21.15.114:0/4111266949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc48104e20 0x7efc48196ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:46.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.479+0000 7efc4cba2640 1 --2- 172.21.15.114:0/4111266949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc480fee00 0x7efc481969b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:46.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.479+0000 7efc4cba2640 1 -- 172.21.15.114:0/4111266949 >> 172.21.15.114:0/4111266949 conn(0x7efc48074930 msgr2=0x7efc480fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:46.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.479+0000 7efc4cba2640 1 -- 172.21.15.114:0/4111266949 shutdown_connections 2024-02-16T22:33:46.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.479+0000 7efc4cba2640 1 -- 172.21.15.114:0/4111266949 wait complete. 2024-02-16T22:33:46.492 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-16T22:33:46.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.596+0000 7f99e84ae640 1 -- 172.21.15.114:0/1292682828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99e01046b0 msgr2=0x7f99e0104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:46.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.596+0000 7f99e84ae640 1 --2- 172.21.15.114:0/1292682828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99e01046b0 0x7f99e0104a90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f99d0009880 tx=0x7f99d002bce0 comp rx=0 tx=0).stop 2024-02-16T22:33:46.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.596+0000 7f99e84ae640 1 -- 172.21.15.114:0/1292682828 shutdown_connections 2024-02-16T22:33:46.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.596+0000 7f99e84ae640 1 --2- 172.21.15.114:0/1292682828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99e00fe720 0x7f99e00feb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:46.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.596+0000 7f99e84ae640 1 --2- 172.21.15.114:0/1292682828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99e01046b0 0x7f99e0104a90 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:46.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.596+0000 7f99e84ae640 1 -- 172.21.15.114:0/1292682828 >> 172.21.15.114:0/1292682828 conn(0x7f99e0075250 msgr2=0x7f99e0075660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:46.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.596+0000 7f99e84ae640 1 -- 172.21.15.114:0/1292682828 shutdown_connections 2024-02-16T22:33:46.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.596+0000 7f99e84ae640 1 -- 172.21.15.114:0/1292682828 wait complete. 2024-02-16T22:33:46.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.596+0000 7f99e84ae640 1 Processor -- start 2024-02-16T22:33:46.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.597+0000 7f99e84ae640 1 -- start start 2024-02-16T22:33:46.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.597+0000 7f99e84ae640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99e00fe720 0x7f99e0195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:46.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.597+0000 7f99e84ae640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99e01046b0 0x7f99e01962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:46.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.597+0000 7f99e84ae640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e01968a0 con 0x7f99e01046b0 2024-02-16T22:33:46.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.597+0000 7f99e84ae640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e0196a10 con 0x7f99e00fe720 2024-02-16T22:33:46.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.597+0000 7f99e5a49640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99e01046b0 0x7f99e01962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:46.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.597+0000 7f99e5a49640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99e01046b0 0x7f99e01962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49158/0 (socket says 172.21.15.114:49158) 2024-02-16T22:33:46.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.597+0000 7f99e5a49640 1 -- 172.21.15.114:0/886581939 learned_addr learned my addr 172.21.15.114:0/886581939 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:46.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.597+0000 7f99e624a640 1 --2- 172.21.15.114:0/886581939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99e00fe720 0x7f99e0195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:46.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.597+0000 7f99e5a49640 1 -- 172.21.15.114:0/886581939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99e00fe720 msgr2=0x7f99e0195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:46.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.597+0000 7f99e5a49640 1 --2- 172.21.15.114:0/886581939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99e00fe720 0x7f99e0195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:46.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.597+0000 7f99e5a49640 1 -- 172.21.15.114:0/886581939 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d0009560 con 0x7f99e01046b0 2024-02-16T22:33:46.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.597+0000 7f99e624a640 1 --2- 172.21.15.114:0/886581939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99e00fe720 0x7f99e0195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T22:33:46.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.597+0000 7f99e5a49640 1 --2- 172.21.15.114:0/886581939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99e01046b0 0x7f99e01962a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f99dc00c8a0 tx=0x7f99dc00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:46.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.597+0000 7f99d77fe640 1 -- 172.21.15.114:0/886581939 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99dc007b80 con 0x7f99e01046b0 2024-02-16T22:33:46.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.597+0000 7f99d77fe640 1 -- 172.21.15.114:0/886581939 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f99dc007ce0 con 0x7f99e01046b0 2024-02-16T22:33:46.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.597+0000 7f99e84ae640 1 -- 172.21.15.114:0/886581939 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e019a8a0 con 0x7f99e01046b0 2024-02-16T22:33:46.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.597+0000 7f99d77fe640 1 -- 172.21.15.114:0/886581939 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99dc018810 con 0x7f99e01046b0 2024-02-16T22:33:46.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.597+0000 7f99e84ae640 1 -- 172.21.15.114:0/886581939 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e019ae00 con 0x7f99e01046b0 2024-02-16T22:33:46.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.598+0000 7f99e84ae640 1 -- 172.21.15.114:0/886581939 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e004e040 con 0x7f99e01046b0 2024-02-16T22:33:46.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.599+0000 7f99d77fe640 1 -- 172.21.15.114:0/886581939 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f99dc018a80 con 0x7f99e01046b0 2024-02-16T22:33:46.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.599+0000 7f99d77fe640 1 --2- 172.21.15.114:0/886581939 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f99cc06d810 0x7f99cc06fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:46.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.599+0000 7f99d77fe640 1 -- 172.21.15.114:0/886581939 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f99dc08d7b0 con 0x7f99e01046b0 2024-02-16T22:33:46.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.599+0000 7f99e624a640 1 --2- 172.21.15.114:0/886581939 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f99cc06d810 0x7f99cc06fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:46.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.600+0000 7f99e624a640 1 --2- 172.21.15.114:0/886581939 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f99cc06d810 0x7f99cc06fcd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f99d0000c00 tx=0x7f99d002cc20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:46.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.604+0000 7f99d77fe640 1 -- 172.21.15.114:0/886581939 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f99dc056eb0 con 0x7f99e01046b0 2024-02-16T22:33:46.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.776+0000 7f99e84ae640 1 -- 172.21.15.114:0/886581939 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f99e0108bd0 con 0x7f99e01046b0 2024-02-16T22:33:46.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.777+0000 7f99d77fe640 1 -- 172.21.15.114:0/886581939 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v82) v1 ==== 107+0+11 (secure 0 0 0) 0x7f99dc05a570 con 0x7f99e01046b0 2024-02-16T22:33:46.779 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-02-16T22:33:46.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.779+0000 7f99e84ae640 1 -- 172.21.15.114:0/886581939 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f99cc06d810 msgr2=0x7f99cc06fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:46.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.779+0000 7f99e84ae640 1 --2- 172.21.15.114:0/886581939 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f99cc06d810 0x7f99cc06fcd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f99d0000c00 tx=0x7f99d002cc20 comp rx=0 tx=0).stop 2024-02-16T22:33:46.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.779+0000 7f99e84ae640 1 -- 172.21.15.114:0/886581939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99e01046b0 msgr2=0x7f99e01962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:46.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.779+0000 7f99e84ae640 1 --2- 172.21.15.114:0/886581939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99e01046b0 0x7f99e01962a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f99dc00c8a0 tx=0x7f99dc00cd60 comp rx=0 tx=0).stop 2024-02-16T22:33:46.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.779+0000 7f99e84ae640 1 -- 172.21.15.114:0/886581939 shutdown_connections 2024-02-16T22:33:46.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.779+0000 7f99e84ae640 1 --2- 172.21.15.114:0/886581939 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f99cc06d810 0x7f99cc06fcd0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:46.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.779+0000 7f99e84ae640 1 --2- 172.21.15.114:0/886581939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99e01046b0 0x7f99e01962a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:46.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.779+0000 7f99e84ae640 1 --2- 172.21.15.114:0/886581939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99e00fe720 0x7f99e0195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:46.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.779+0000 7f99e84ae640 1 -- 172.21.15.114:0/886581939 >> 172.21.15.114:0/886581939 conn(0x7f99e0075250 msgr2=0x7f99e01023b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:46.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.779+0000 7f99e84ae640 1 -- 172.21.15.114:0/886581939 shutdown_connections 2024-02-16T22:33:46.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.779+0000 7f99e84ae640 1 -- 172.21.15.114:0/886581939 wait complete. 2024-02-16T22:33:46.790 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:46 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/368162642' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-16T22:33:46.790 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:33:46.790 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:46 smithi114 ceph-mon[94452]: pgmap v337: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:46.790 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:46 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/1302794796' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T22:33:46.790 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:46 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/4111266949' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T22:33:46.791 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-16T22:33:46.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.894+0000 7fe933d59640 1 -- 172.21.15.114:0/995453561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe92c0ff1a0 msgr2=0x7fe92c0ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:46.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.894+0000 7fe933d59640 1 --2- 172.21.15.114:0/995453561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe92c0ff1a0 0x7fe92c0ff580 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe928009880 tx=0x7fe92802bce0 comp rx=0 tx=0).stop 2024-02-16T22:33:46.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.894+0000 7fe933d59640 1 -- 172.21.15.114:0/995453561 shutdown_connections 2024-02-16T22:33:46.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.894+0000 7fe933d59640 1 --2- 172.21.15.114:0/995453561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe92c0ffb50 0x7fe92c103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:46.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.894+0000 7fe933d59640 1 --2- 172.21.15.114:0/995453561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe92c0ff1a0 0x7fe92c0ff580 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:46.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.894+0000 7fe933d59640 1 -- 172.21.15.114:0/995453561 >> 172.21.15.114:0/995453561 conn(0x7fe92c0faa20 msgr2=0x7fe92c0fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:46.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.894+0000 7fe933d59640 1 -- 172.21.15.114:0/995453561 shutdown_connections 2024-02-16T22:33:46.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.894+0000 7fe933d59640 1 -- 172.21.15.114:0/995453561 wait complete. 2024-02-16T22:33:46.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.895+0000 7fe933d59640 1 Processor -- start 2024-02-16T22:33:46.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.895+0000 7fe933d59640 1 -- start start 2024-02-16T22:33:46.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.895+0000 7fe933d59640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe92c0ff1a0 0x7fe92c195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:46.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.895+0000 7fe933d59640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe92c1962b0 0x7fe92c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:46.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.895+0000 7fe933d59640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c1968b0 con 0x7fe92c0ff1a0 2024-02-16T22:33:46.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.895+0000 7fe933d59640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c196a20 con 0x7fe92c1962b0 2024-02-16T22:33:46.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.895+0000 7fe931af5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe92c0ff1a0 0x7fe92c195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:46.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.895+0000 7fe931af5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe92c0ff1a0 0x7fe92c195d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49170/0 (socket says 172.21.15.114:49170) 2024-02-16T22:33:46.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.895+0000 7fe931af5640 1 -- 172.21.15.114:0/1448623137 learned_addr learned my addr 172.21.15.114:0/1448623137 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:46.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.895+0000 7fe9312f4640 1 --2- 172.21.15.114:0/1448623137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe92c1962b0 0x7fe92c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:46.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.895+0000 7fe931af5640 1 -- 172.21.15.114:0/1448623137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe92c1962b0 msgr2=0x7fe92c19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:46.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.895+0000 7fe931af5640 1 --2- 172.21.15.114:0/1448623137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe92c1962b0 0x7fe92c19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:46.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.895+0000 7fe931af5640 1 -- 172.21.15.114:0/1448623137 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe928009560 con 0x7fe92c0ff1a0 2024-02-16T22:33:46.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.895+0000 7fe931af5640 1 --2- 172.21.15.114:0/1448623137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe92c0ff1a0 0x7fe92c195d70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe928036040 tx=0x7fe928004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:46.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.895+0000 7fe922ffd640 1 -- 172.21.15.114:0/1448623137 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9280037c0 con 0x7fe92c0ff1a0 2024-02-16T22:33:46.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.896+0000 7fe922ffd640 1 -- 172.21.15.114:0/1448623137 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe928003de0 con 0x7fe92c0ff1a0 2024-02-16T22:33:46.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.896+0000 7fe933d59640 1 -- 172.21.15.114:0/1448623137 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe92c19acb0 con 0x7fe92c0ff1a0 2024-02-16T22:33:46.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.896+0000 7fe922ffd640 1 -- 172.21.15.114:0/1448623137 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe928034440 con 0x7fe92c0ff1a0 2024-02-16T22:33:46.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.896+0000 7fe933d59640 1 -- 172.21.15.114:0/1448623137 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe92c19b210 con 0x7fe92c0ff1a0 2024-02-16T22:33:46.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.896+0000 7fe933d59640 1 -- 172.21.15.114:0/1448623137 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe92c04e040 con 0x7fe92c0ff1a0 2024-02-16T22:33:46.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.897+0000 7fe922ffd640 1 -- 172.21.15.114:0/1448623137 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7fe928003920 con 0x7fe92c0ff1a0 2024-02-16T22:33:46.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.897+0000 7fe922ffd640 1 --2- 172.21.15.114:0/1448623137 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe91806d810 0x7fe91806fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:46.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.897+0000 7fe922ffd640 1 -- 172.21.15.114:0/1448623137 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7fe9280afad0 con 0x7fe92c0ff1a0 2024-02-16T22:33:46.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.897+0000 7fe9312f4640 1 --2- 172.21.15.114:0/1448623137 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe91806d810 0x7fe91806fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:46.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.898+0000 7fe9312f4640 1 --2- 172.21.15.114:0/1448623137 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe91806d810 0x7fe91806fcd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0058c0 tx=0x7fe91c00a4c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:46.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:46.902+0000 7fe922ffd640 1 -- 172.21.15.114:0/1448623137 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe928079290 con 0x7fe92c0ff1a0 2024-02-16T22:33:47.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:46 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/368162642' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-16T22:33:47.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:46 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:33:47.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:46 smithi123 ceph-mon[82331]: pgmap v337: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:47.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:46 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/1302794796' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T22:33:47.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:46 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/4111266949' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T22:33:47.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.073+0000 7fe933d59640 1 -- 172.21.15.114:0/1448623137 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fe92c0ffb50 con 0x7fe92c0ff1a0 2024-02-16T22:33:47.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.074+0000 7fe922ffd640 1 -- 172.21.15.114:0/1448623137 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v82) v1 ==== 107+0+11 (secure 0 0 0) 0x7fe92807c950 con 0x7fe92c0ff1a0 2024-02-16T22:33:47.075 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-02-16T22:33:47.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.076+0000 7fe933d59640 1 -- 172.21.15.114:0/1448623137 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe91806d810 msgr2=0x7fe91806fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:47.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.076+0000 7fe933d59640 1 --2- 172.21.15.114:0/1448623137 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe91806d810 0x7fe91806fcd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0058c0 tx=0x7fe91c00a4c0 comp rx=0 tx=0).stop 2024-02-16T22:33:47.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.076+0000 7fe933d59640 1 -- 172.21.15.114:0/1448623137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe92c0ff1a0 msgr2=0x7fe92c195d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:47.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.076+0000 7fe933d59640 1 --2- 172.21.15.114:0/1448623137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe92c0ff1a0 0x7fe92c195d70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe928036040 tx=0x7fe928004270 comp rx=0 tx=0).stop 2024-02-16T22:33:47.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.076+0000 7fe933d59640 1 -- 172.21.15.114:0/1448623137 shutdown_connections 2024-02-16T22:33:47.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.076+0000 7fe933d59640 1 --2- 172.21.15.114:0/1448623137 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7fe91806d810 0x7fe91806fcd0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:47.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.076+0000 7fe933d59640 1 --2- 172.21.15.114:0/1448623137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe92c1962b0 0x7fe92c19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:47.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.076+0000 7fe933d59640 1 --2- 172.21.15.114:0/1448623137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe92c0ff1a0 0x7fe92c195d70 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:47.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.076+0000 7fe933d59640 1 -- 172.21.15.114:0/1448623137 >> 172.21.15.114:0/1448623137 conn(0x7fe92c0faa20 msgr2=0x7fe92c0fc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:47.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.076+0000 7fe933d59640 1 -- 172.21.15.114:0/1448623137 shutdown_connections 2024-02-16T22:33:47.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.076+0000 7fe933d59640 1 -- 172.21.15.114:0/1448623137 wait complete. 2024-02-16T22:33:47.088 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-16T22:33:47.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.187+0000 7f1c9c070640 1 -- 172.21.15.114:0/2192256890 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c940fefe0 msgr2=0x7f1c940ff440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:47.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.187+0000 7f1c9c070640 1 --2- 172.21.15.114:0/2192256890 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c940fefe0 0x7f1c940ff440 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1c84009880 tx=0x7f1c8402bce0 comp rx=0 tx=0).stop 2024-02-16T22:33:47.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.187+0000 7f1c9c070640 1 -- 172.21.15.114:0/2192256890 shutdown_connections 2024-02-16T22:33:47.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.187+0000 7f1c9c070640 1 --2- 172.21.15.114:0/2192256890 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c940fefe0 0x7f1c940ff440 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:47.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.187+0000 7f1c9c070640 1 --2- 172.21.15.114:0/2192256890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c94103f20 0x7f1c94104300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:47.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.187+0000 7f1c9c070640 1 -- 172.21.15.114:0/2192256890 >> 172.21.15.114:0/2192256890 conn(0x7f1c94075b40 msgr2=0x7f1c94075f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:47.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.188+0000 7f1c9c070640 1 -- 172.21.15.114:0/2192256890 shutdown_connections 2024-02-16T22:33:47.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.188+0000 7f1c9c070640 1 -- 172.21.15.114:0/2192256890 wait complete. 2024-02-16T22:33:47.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.188+0000 7f1c9c070640 1 Processor -- start 2024-02-16T22:33:47.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.188+0000 7f1c9c070640 1 -- start start 2024-02-16T22:33:47.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.188+0000 7f1c9c070640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c940fefe0 0x7f1c94196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:47.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.188+0000 7f1c9c070640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c94103f20 0x7f1c94196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:47.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.188+0000 7f1c9c070640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c94197610 con 0x7f1c940fefe0 2024-02-16T22:33:47.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.188+0000 7f1c9c070640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c94197750 con 0x7f1c94103f20 2024-02-16T22:33:47.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.188+0000 7f1c99e0c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c940fefe0 0x7f1c94196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:47.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.188+0000 7f1c99e0c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c940fefe0 0x7f1c94196ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49190/0 (socket says 172.21.15.114:49190) 2024-02-16T22:33:47.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.188+0000 7f1c99e0c640 1 -- 172.21.15.114:0/363138602 learned_addr learned my addr 172.21.15.114:0/363138602 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-02-16T22:33:47.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.189+0000 7f1c99e0c640 1 -- 172.21.15.114:0/363138602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c94103f20 msgr2=0x7f1c94196ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:47.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.189+0000 7f1c99e0c640 1 --2- 172.21.15.114:0/363138602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c94103f20 0x7f1c94196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:47.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.189+0000 7f1c99e0c640 1 -- 172.21.15.114:0/363138602 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c84009560 con 0x7f1c940fefe0 2024-02-16T22:33:47.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.189+0000 7f1c99e0c640 1 --2- 172.21.15.114:0/363138602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c940fefe0 0x7f1c94196ab0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1c9000ea30 tx=0x7f1c9000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:47.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.189+0000 7f1c8affd640 1 -- 172.21.15.114:0/363138602 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c9000cdb0 con 0x7f1c940fefe0 2024-02-16T22:33:47.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.189+0000 7f1c8affd640 1 -- 172.21.15.114:0/363138602 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c90011930 con 0x7f1c940fefe0 2024-02-16T22:33:47.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.189+0000 7f1c9c070640 1 -- 172.21.15.114:0/363138602 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c9419fb20 con 0x7f1c940fefe0 2024-02-16T22:33:47.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.189+0000 7f1c8affd640 1 -- 172.21.15.114:0/363138602 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c9001aa30 con 0x7f1c940fefe0 2024-02-16T22:33:47.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.189+0000 7f1c9c070640 1 -- 172.21.15.114:0/363138602 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c9419fff0 con 0x7f1c940fefe0 2024-02-16T22:33:47.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.190+0000 7f1c9c070640 1 -- 172.21.15.114:0/363138602 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c9404e040 con 0x7f1c940fefe0 2024-02-16T22:33:47.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.190+0000 7f1c8affd640 1 -- 172.21.15.114:0/363138602 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 38) v1 ==== 91495+0+0 (secure 0 0 0) 0x7f1c9001ab90 con 0x7f1c940fefe0 2024-02-16T22:33:47.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.191+0000 7f1c8affd640 1 --2- 172.21.15.114:0/363138602 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f1c8006d8d0 0x7f1c8006fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T22:33:47.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.191+0000 7f1c9960b640 1 --2- 172.21.15.114:0/363138602 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f1c8006d8d0 0x7f1c8006fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T22:33:47.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.191+0000 7f1c8affd640 1 -- 172.21.15.114:0/363138602 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6758+0+0 (secure 0 0 0) 0x7f1c9008dc50 con 0x7f1c940fefe0 2024-02-16T22:33:47.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.191+0000 7f1c9960b640 1 --2- 172.21.15.114:0/363138602 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f1c8006d8d0 0x7f1c8006fd90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1c84036040 tx=0x7f1c84000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T22:33:47.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.195+0000 7f1c8affd640 1 -- 172.21.15.114:0/363138602 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1c900573d0 con 0x7f1c940fefe0 2024-02-16T22:33:47.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.380+0000 7f1c9c070640 1 -- 172.21.15.114:0/363138602 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1c94067960 con 0x7f1c940fefe0 2024-02-16T22:33:47.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.380+0000 7f1c8affd640 1 -- 172.21.15.114:0/363138602 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 45 entries v82) v1 ==== 81+0+2700 (secure 0 0 0) 0x7f1c9005aa90 con 0x7f1c940fefe0 2024-02-16T22:33:47.382 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.123:6827/2812517585 2024-02-17T22:29:37.876591+0000 2024-02-16T22:33:47.382 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.123:6826/2812517585 2024-02-17T22:29:37.876591+0000 2024-02-16T22:33:47.382 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.123:6825/418014612 2024-02-17T22:29:09.777831+0000 2024-02-16T22:33:47.382 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.123:6824/418014612 2024-02-17T22:29:09.777831+0000 2024-02-16T22:33:47.382 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6827/277953535 2024-02-17T22:28:21.203522+0000 2024-02-16T22:33:47.383 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6826/277953535 2024-02-17T22:28:21.203522+0000 2024-02-16T22:33:47.383 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/297794530 2024-02-17T22:23:45.719861+0000 2024-02-16T22:33:47.383 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6800/2055574123 2024-02-17T22:11:27.861007+0000 2024-02-16T22:33:47.383 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6829/865692293 2024-02-17T22:28:47.638133+0000 2024-02-16T22:33:47.383 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/4031283053 2024-02-17T22:21:10.405873+0000 2024-02-16T22:33:47.383 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/1034649687 2024-02-17T22:13:11.405214+0000 2024-02-16T22:33:47.383 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/3991553908 2024-02-17T22:21:10.405873+0000 2024-02-16T22:33:47.383 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/744140874 2024-02-17T22:12:00.492971+0000 2024-02-16T22:33:47.383 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6828/865692293 2024-02-17T22:28:47.638133+0000 2024-02-16T22:33:47.383 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6800/3153243195 2024-02-17T22:21:10.405873+0000 2024-02-16T22:33:47.383 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/3211326148 2024-02-17T22:13:11.405214+0000 2024-02-16T22:33:47.383 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.123:6829/1539029039 2024-02-17T22:22:31.925771+0000 2024-02-16T22:33:47.383 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6801/3153243195 2024-02-17T22:21:10.405873+0000 2024-02-16T22:33:47.383 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6801/2055574123 2024-02-17T22:11:27.861007+0000 2024-02-16T22:33:47.383 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.123:6828/1539029039 2024-02-17T22:22:31.925771+0000 2024-02-16T22:33:47.383 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2393113501 2024-02-17T22:21:10.405873+0000 2024-02-16T22:33:47.384 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6800/2648536185 2024-02-17T22:13:11.405214+0000 2024-02-16T22:33:47.384 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/693887539 2024-02-17T22:21:10.405873+0000 2024-02-16T22:33:47.384 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6800/625910309 2024-02-17T22:12:00.492971+0000 2024-02-16T22:33:47.384 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.123:0/2858796357 2024-02-17T22:22:31.925771+0000 2024-02-16T22:33:47.384 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/4029631970 2024-02-17T22:11:27.861007+0000 2024-02-16T22:33:47.384 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2206519078 2024-02-17T22:23:45.719861+0000 2024-02-16T22:33:47.384 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6801/2648536185 2024-02-17T22:13:11.405214+0000 2024-02-16T22:33:47.384 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.123:0/1942138241 2024-02-17T22:22:31.925771+0000 2024-02-16T22:33:47.384 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.123:0/216490899 2024-02-17T22:22:31.925771+0000 2024-02-16T22:33:47.384 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/3971798486 2024-02-17T22:11:27.861007+0000 2024-02-16T22:33:47.384 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/499892900 2024-02-17T22:12:00.492971+0000 2024-02-16T22:33:47.384 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/1349416361 2024-02-17T22:12:00.492971+0000 2024-02-16T22:33:47.384 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.123:0/2896457931 2024-02-17T22:22:31.925771+0000 2024-02-16T22:33:47.384 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.123:0/3641186660 2024-02-17T22:22:31.925771+0000 2024-02-16T22:33:47.384 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/125196455 2024-02-17T22:23:45.719861+0000 2024-02-16T22:33:47.385 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6801/625910309 2024-02-17T22:12:00.492971+0000 2024-02-16T22:33:47.385 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2762476246 2024-02-17T22:13:11.405214+0000 2024-02-16T22:33:47.385 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.123:0/4134230222 2024-02-17T22:22:31.925771+0000 2024-02-16T22:33:47.385 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6800/3051780151 2024-02-17T22:23:45.719861+0000 2024-02-16T22:33:47.385 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/489142132 2024-02-17T22:23:45.719861+0000 2024-02-16T22:33:47.385 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6801/3051780151 2024-02-17T22:23:45.719861+0000 2024-02-16T22:33:47.385 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/1088872575 2024-02-17T22:23:45.719861+0000 2024-02-16T22:33:47.385 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2825621999 2024-02-17T22:11:27.861007+0000 2024-02-16T22:33:47.385 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/1250197670 2024-02-17T22:23:45.719861+0000 2024-02-16T22:33:47.386 INFO:teuthology.orchestra.run.smithi114.stderr:listed 45 entries 2024-02-16T22:33:47.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.382+0000 7f1c9c070640 1 -- 172.21.15.114:0/363138602 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f1c8006d8d0 msgr2=0x7f1c8006fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:47.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.382+0000 7f1c9c070640 1 --2- 172.21.15.114:0/363138602 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f1c8006d8d0 0x7f1c8006fd90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1c84036040 tx=0x7f1c84000bc0 comp rx=0 tx=0).stop 2024-02-16T22:33:47.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.382+0000 7f1c9c070640 1 -- 172.21.15.114:0/363138602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c940fefe0 msgr2=0x7f1c94196ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T22:33:47.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.382+0000 7f1c9c070640 1 --2- 172.21.15.114:0/363138602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c940fefe0 0x7f1c94196ab0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1c9000ea30 tx=0x7f1c9000eef0 comp rx=0 tx=0).stop 2024-02-16T22:33:47.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.382+0000 7f1c9c070640 1 -- 172.21.15.114:0/363138602 shutdown_connections 2024-02-16T22:33:47.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.382+0000 7f1c9c070640 1 --2- 172.21.15.114:0/363138602 >> [v2:172.21.15.114:6800/1665442292,v1:172.21.15.114:6801/1665442292] conn(0x7f1c8006d8d0 0x7f1c8006fd90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:47.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.382+0000 7f1c9c070640 1 --2- 172.21.15.114:0/363138602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c94103f20 0x7f1c94196ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:47.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.382+0000 7f1c9c070640 1 --2- 172.21.15.114:0/363138602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c940fefe0 0x7f1c94196ab0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T22:33:47.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.382+0000 7f1c9c070640 1 -- 172.21.15.114:0/363138602 >> 172.21.15.114:0/363138602 conn(0x7f1c94075b40 msgr2=0x7f1c940fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T22:33:47.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.382+0000 7f1c9c070640 1 -- 172.21.15.114:0/363138602 shutdown_connections 2024-02-16T22:33:47.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-02-16T22:33:47.382+0000 7f1c9c070640 1 -- 172.21.15.114:0/363138602 wait complete. 2024-02-16T22:33:47.394 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi114.front.sepia.ceph.com... 2024-02-16T22:33:47.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T22:33:47.395 DEBUG:teuthology.orchestra.run.smithi114:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-02-16T22:33:47.422 INFO:teuthology.orchestra.run:waiting for 300 2024-02-16T22:33:47.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:47 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/886581939' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T22:33:47.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:47 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/1448623137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T22:33:47.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:47 smithi114 ceph-mon[94452]: from='client.? 172.21.15.114:0/363138602' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-16T22:33:48.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:47 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/886581939' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T22:33:48.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:47 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/1448623137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T22:33:48.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:47 smithi123 ceph-mon[82331]: from='client.? 172.21.15.114:0/363138602' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-16T22:33:48.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:48 smithi114 ceph-mon[94452]: pgmap v338: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:49.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:48 smithi123 ceph-mon[82331]: pgmap v338: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:51.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:50 smithi114 ceph-mon[94452]: pgmap v339: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:51.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:50 smithi123 ceph-mon[82331]: pgmap v339: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:53.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:52 smithi114 ceph-mon[94452]: pgmap v340: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:53.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:52 smithi123 ceph-mon[82331]: pgmap v340: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:55.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:54 smithi114 ceph-mon[94452]: pgmap v341: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:55.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:54 smithi123 ceph-mon[82331]: pgmap v341: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:57.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:56 smithi114 ceph-mon[94452]: pgmap v342: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:57.301 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:56 smithi123 ceph-mon[82331]: pgmap v342: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:59.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:33:58 smithi114 ceph-mon[94452]: pgmap v343: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:33:59.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:33:58 smithi123 ceph-mon[82331]: pgmap v343: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:01.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:00 smithi114 ceph-mon[94452]: pgmap v344: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:01.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:00 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:34:01.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:00 smithi123 ceph-mon[82331]: pgmap v344: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:01.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:00 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:34:03.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:02 smithi114 ceph-mon[94452]: pgmap v345: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:03.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:02 smithi123 ceph-mon[82331]: pgmap v345: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:05.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:04 smithi114 ceph-mon[94452]: pgmap v346: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:05.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:04 smithi123 ceph-mon[82331]: pgmap v346: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:07.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:06 smithi114 ceph-mon[94452]: pgmap v347: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:07.301 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:06 smithi123 ceph-mon[82331]: pgmap v347: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:09.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:08 smithi114 ceph-mon[94452]: pgmap v348: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:09.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:08 smithi123 ceph-mon[82331]: pgmap v348: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:11.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:10 smithi123 ceph-mon[82331]: pgmap v349: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:11.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:10 smithi114 ceph-mon[94452]: pgmap v349: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:13.254 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:12 smithi114 ceph-mon[94452]: pgmap v350: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:13.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:12 smithi123 ceph-mon[82331]: pgmap v350: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:15.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:14 smithi114 ceph-mon[94452]: pgmap v351: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:15.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:14 smithi123 ceph-mon[82331]: pgmap v351: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:16.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:15 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:34:16.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:15 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:34:17.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:16 smithi123 ceph-mon[82331]: pgmap v352: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:17.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:16 smithi114 ceph-mon[94452]: pgmap v352: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:19.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:18 smithi123 ceph-mon[82331]: pgmap v353: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:19.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:18 smithi114 ceph-mon[94452]: pgmap v353: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:21.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:20 smithi123 ceph-mon[82331]: pgmap v354: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:21.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:20 smithi114 ceph-mon[94452]: pgmap v354: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:23.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:22 smithi123 ceph-mon[82331]: pgmap v355: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:23.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:22 smithi114 ceph-mon[94452]: pgmap v355: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:25.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:24 smithi114 ceph-mon[94452]: pgmap v356: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:25.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:24 smithi123 ceph-mon[82331]: pgmap v356: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:27.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:26 smithi123 ceph-mon[82331]: pgmap v357: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:27.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:26 smithi114 ceph-mon[94452]: pgmap v357: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:29.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:28 smithi123 ceph-mon[82331]: pgmap v358: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:29.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:28 smithi114 ceph-mon[94452]: pgmap v358: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:31.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:30 smithi123 ceph-mon[82331]: pgmap v359: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:31.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:30 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:34:31.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:30 smithi114 ceph-mon[94452]: pgmap v359: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:31.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:30 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:34:33.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:32 smithi123 ceph-mon[82331]: pgmap v360: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:33.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:32 smithi114 ceph-mon[94452]: pgmap v360: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:35.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:34 smithi123 ceph-mon[82331]: pgmap v361: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:35.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:34 smithi114 ceph-mon[94452]: pgmap v361: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:37.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:36 smithi123 ceph-mon[82331]: pgmap v362: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:37.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:36 smithi114 ceph-mon[94452]: pgmap v362: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:39.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:38 smithi123 ceph-mon[82331]: pgmap v363: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:39.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:38 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:34:39.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:38 smithi114 ceph-mon[94452]: pgmap v363: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:39.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:38 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:34:40.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:40 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:34:40.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:40 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:34:40.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:40 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:34:40.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:40 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:34:40.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:40 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:34:40.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:40 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:34:41.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:41 smithi123 ceph-mon[82331]: pgmap v364: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:41.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:41 smithi114 ceph-mon[94452]: pgmap v364: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:43.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:43 smithi123 ceph-mon[82331]: pgmap v365: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:43.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:43 smithi114 ceph-mon[94452]: pgmap v365: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:45.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:45 smithi123 ceph-mon[82331]: pgmap v366: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:45.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:45 smithi114 ceph-mon[94452]: pgmap v366: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:46.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:46 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:34:46.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:34:47.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:47 smithi123 ceph-mon[82331]: pgmap v367: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:47.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:47 smithi114 ceph-mon[94452]: pgmap v367: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:49.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:49 smithi123 ceph-mon[82331]: pgmap v368: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:49.905 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:49 smithi114 ceph-mon[94452]: pgmap v368: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:51.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:51 smithi123 ceph-mon[82331]: pgmap v369: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:51.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:51 smithi114 ceph-mon[94452]: pgmap v369: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:53.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:53 smithi123 ceph-mon[82331]: pgmap v370: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:53.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:53 smithi114 ceph-mon[94452]: pgmap v370: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:55.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:55 smithi123 ceph-mon[82331]: pgmap v371: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:55.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:55 smithi114 ceph-mon[94452]: pgmap v371: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:57.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:57 smithi123 ceph-mon[82331]: pgmap v372: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:57.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:57 smithi114 ceph-mon[94452]: pgmap v372: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:59.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:34:59 smithi123 ceph-mon[82331]: pgmap v373: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:34:59.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:34:59 smithi114 ceph-mon[94452]: pgmap v373: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:01.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:01 smithi114 ceph-mon[94452]: pgmap v374: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:01.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:35:02.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:01 smithi123 ceph-mon[82331]: pgmap v374: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:02.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:01 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:35:03.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:03 smithi114 ceph-mon[94452]: pgmap v375: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:04.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:03 smithi123 ceph-mon[82331]: pgmap v375: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:05.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:05 smithi114 ceph-mon[94452]: pgmap v376: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:06.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:05 smithi123 ceph-mon[82331]: pgmap v376: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:07.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:07 smithi114 ceph-mon[94452]: pgmap v377: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:08.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:07 smithi123 ceph-mon[82331]: pgmap v377: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:09.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:09 smithi114 ceph-mon[94452]: pgmap v378: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:10.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:09 smithi123 ceph-mon[82331]: pgmap v378: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:11.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:11 smithi114 ceph-mon[94452]: pgmap v379: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:12.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:11 smithi123 ceph-mon[82331]: pgmap v379: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:13.854 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:13 smithi123 ceph-mon[82331]: pgmap v380: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:13.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:13 smithi114 ceph-mon[94452]: pgmap v380: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:15.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:15 smithi114 ceph-mon[94452]: pgmap v381: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:16.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:15 smithi123 ceph-mon[82331]: pgmap v381: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:16.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:16 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:35:17.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:16 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:35:17.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:17 smithi114 ceph-mon[94452]: pgmap v382: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:18.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:17 smithi123 ceph-mon[82331]: pgmap v382: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:19.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:19 smithi114 ceph-mon[94452]: pgmap v383: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:20.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:19 smithi123 ceph-mon[82331]: pgmap v383: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:21.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:21 smithi114 ceph-mon[94452]: pgmap v384: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:22.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:21 smithi123 ceph-mon[82331]: pgmap v384: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:23.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:23 smithi114 ceph-mon[94452]: pgmap v385: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:24.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:23 smithi123 ceph-mon[82331]: pgmap v385: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:25.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:25 smithi114 ceph-mon[94452]: pgmap v386: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:26.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:25 smithi123 ceph-mon[82331]: pgmap v386: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:27.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:27 smithi114 ceph-mon[94452]: pgmap v387: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:28.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:27 smithi123 ceph-mon[82331]: pgmap v387: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:29.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:29 smithi114 ceph-mon[94452]: pgmap v388: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:30.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:29 smithi123 ceph-mon[82331]: pgmap v388: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:31.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:31 smithi114 ceph-mon[94452]: pgmap v389: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:31.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:31 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:35:32.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:31 smithi123 ceph-mon[82331]: pgmap v389: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:32.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:31 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:35:33.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:33 smithi114 ceph-mon[94452]: pgmap v390: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:34.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:33 smithi123 ceph-mon[82331]: pgmap v390: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:35.901 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:35 smithi123 ceph-mon[82331]: pgmap v391: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:35.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:35 smithi114 ceph-mon[94452]: pgmap v391: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:37.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:37 smithi114 ceph-mon[94452]: pgmap v392: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:38.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:37 smithi123 ceph-mon[82331]: pgmap v392: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:39.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:39 smithi114 ceph-mon[94452]: pgmap v393: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:39.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:39 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:35:40.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:39 smithi123 ceph-mon[82331]: pgmap v393: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:40.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:39 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:35:40.949 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:40 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:35:40.949 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:40 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:35:40.949 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:40 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:35:40.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:40 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:35:40.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:40 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:35:40.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:40 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:35:41.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:41 smithi114 ceph-mon[94452]: pgmap v394: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:42.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:41 smithi123 ceph-mon[82331]: pgmap v394: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:43.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:43 smithi114 ceph-mon[94452]: pgmap v395: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:44.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:43 smithi123 ceph-mon[82331]: pgmap v395: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:45.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:45 smithi114 ceph-mon[94452]: pgmap v396: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:45.995 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:45 smithi123 ceph-mon[82331]: pgmap v396: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:46.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:35:47.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:46 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:35:47.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:47 smithi114 ceph-mon[94452]: pgmap v397: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:48.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:47 smithi123 ceph-mon[82331]: pgmap v397: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:49.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:49 smithi114 ceph-mon[94452]: pgmap v398: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:50.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:49 smithi123 ceph-mon[82331]: pgmap v398: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:51.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:51 smithi114 ceph-mon[94452]: pgmap v399: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:52.051 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:51 smithi123 ceph-mon[82331]: pgmap v399: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:53.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:53 smithi114 ceph-mon[94452]: pgmap v400: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:54.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:53 smithi123 ceph-mon[82331]: pgmap v400: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:55.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:55 smithi114 ceph-mon[94452]: pgmap v401: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:56.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:55 smithi123 ceph-mon[82331]: pgmap v401: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:57.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:57 smithi114 ceph-mon[94452]: pgmap v402: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:58.051 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:57 smithi123 ceph-mon[82331]: pgmap v402: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:35:59.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:35:59 smithi114 ceph-mon[94452]: pgmap v403: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:00.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:35:59 smithi123 ceph-mon[82331]: pgmap v403: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:01.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:01 smithi114 ceph-mon[94452]: pgmap v404: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:01.952 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:36:02.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:01 smithi123 ceph-mon[82331]: pgmap v404: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:02.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:01 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:36:02.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:02 smithi114 ceph-mon[94452]: pgmap v405: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:03.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:02 smithi123 ceph-mon[82331]: pgmap v405: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:05.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:04 smithi123 ceph-mon[82331]: pgmap v406: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:05.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:04 smithi114 ceph-mon[94452]: pgmap v406: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:07.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:06 smithi123 ceph-mon[82331]: pgmap v407: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:07.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:06 smithi114 ceph-mon[94452]: pgmap v407: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:09.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:08 smithi123 ceph-mon[82331]: pgmap v408: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:09.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:08 smithi114 ceph-mon[94452]: pgmap v408: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:11.235 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:10 smithi123 ceph-mon[82331]: pgmap v409: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:11.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:10 smithi114 ceph-mon[94452]: pgmap v409: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:13.254 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:12 smithi114 ceph-mon[94452]: pgmap v410: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:13.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:12 smithi123 ceph-mon[82331]: pgmap v410: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:15.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:14 smithi123 ceph-mon[82331]: pgmap v411: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:15.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:14 smithi114 ceph-mon[94452]: pgmap v411: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:16.283 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:15 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:36:16.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:15 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:36:17.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:16 smithi123 ceph-mon[82331]: pgmap v412: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:17.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:16 smithi114 ceph-mon[94452]: pgmap v412: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:19.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:18 smithi123 ceph-mon[82331]: pgmap v413: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:19.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:18 smithi114 ceph-mon[94452]: pgmap v413: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:21.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:20 smithi123 ceph-mon[82331]: pgmap v414: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:21.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:20 smithi114 ceph-mon[94452]: pgmap v414: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:23.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:22 smithi123 ceph-mon[82331]: pgmap v415: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:23.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:22 smithi114 ceph-mon[94452]: pgmap v415: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:25.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:24 smithi123 ceph-mon[82331]: pgmap v416: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:25.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:24 smithi114 ceph-mon[94452]: pgmap v416: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:27.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:26 smithi123 ceph-mon[82331]: pgmap v417: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:27.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:26 smithi114 ceph-mon[94452]: pgmap v417: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:29.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:28 smithi123 ceph-mon[82331]: pgmap v418: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:29.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:28 smithi114 ceph-mon[94452]: pgmap v418: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:31.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:31 smithi123 ceph-mon[82331]: pgmap v419: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:31.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:31 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:36:31.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:30 smithi114 ceph-mon[94452]: pgmap v419: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:31.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:30 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:36:33.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:33 smithi123 ceph-mon[82331]: pgmap v420: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:33.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:33 smithi114 ceph-mon[94452]: pgmap v420: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:35.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:35 smithi123 ceph-mon[82331]: pgmap v421: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:35.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:35 smithi114 ceph-mon[94452]: pgmap v421: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:37.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:37 smithi123 ceph-mon[82331]: pgmap v422: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:37.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:37 smithi114 ceph-mon[94452]: pgmap v422: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:39.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:39 smithi123 ceph-mon[82331]: pgmap v423: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:39.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:39 smithi114 ceph-mon[94452]: pgmap v423: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:41.272 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:41 smithi114 ceph-mon[94452]: pgmap v424: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:41.272 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:41 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:36:41.272 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:41 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:36:41.272 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:41 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:36:41.272 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:41 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:36:41.301 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:41 smithi123 ceph-mon[82331]: pgmap v424: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:41.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:41 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:36:41.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:41 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:36:41.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:41 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:36:41.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:41 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:36:43.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:43 smithi123 ceph-mon[82331]: pgmap v425: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:43.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:43 smithi114 ceph-mon[94452]: pgmap v425: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:45.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:45 smithi123 ceph-mon[82331]: pgmap v426: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:45.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:45 smithi114 ceph-mon[94452]: pgmap v426: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:46.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:46 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:36:46.321 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:36:47.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:47 smithi123 ceph-mon[82331]: pgmap v427: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:47.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:47 smithi114 ceph-mon[94452]: pgmap v427: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:49.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:49 smithi123 ceph-mon[82331]: pgmap v428: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:49.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:49 smithi114 ceph-mon[94452]: pgmap v428: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:51.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:51 smithi123 ceph-mon[82331]: pgmap v429: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:51.374 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:51 smithi114 ceph-mon[94452]: pgmap v429: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:53.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:53 smithi123 ceph-mon[82331]: pgmap v430: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:53.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:53 smithi114 ceph-mon[94452]: pgmap v430: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:55.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:55 smithi123 ceph-mon[82331]: pgmap v431: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:55.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:55 smithi114 ceph-mon[94452]: pgmap v431: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:57.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:57 smithi123 ceph-mon[82331]: pgmap v432: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:57.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:57 smithi114 ceph-mon[94452]: pgmap v432: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:59.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:36:59 smithi123 ceph-mon[82331]: pgmap v433: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:36:59.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:36:59 smithi114 ceph-mon[94452]: pgmap v433: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:01.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:01 smithi123 ceph-mon[82331]: pgmap v434: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:01.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:01 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:37:01.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:01 smithi114 ceph-mon[94452]: pgmap v434: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:01.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:37:03.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:03 smithi123 ceph-mon[82331]: pgmap v435: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:03.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:03 smithi114 ceph-mon[94452]: pgmap v435: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:05.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:05 smithi114 ceph-mon[94452]: pgmap v436: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:05.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:05 smithi123 ceph-mon[82331]: pgmap v436: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:07.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:07 smithi114 ceph-mon[94452]: pgmap v437: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:07.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:07 smithi123 ceph-mon[82331]: pgmap v437: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:09.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:09 smithi114 ceph-mon[94452]: pgmap v438: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:09.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:09 smithi123 ceph-mon[82331]: pgmap v438: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:11.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:11 smithi114 ceph-mon[94452]: pgmap v439: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:11.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:11 smithi123 ceph-mon[82331]: pgmap v439: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:13.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:13 smithi114 ceph-mon[94452]: pgmap v440: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:13.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:13 smithi123 ceph-mon[82331]: pgmap v440: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:15.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:15 smithi114 ceph-mon[94452]: pgmap v441: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:15.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:15 smithi123 ceph-mon[82331]: pgmap v441: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:16.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:16 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:37:16.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:16 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:37:17.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:17 smithi123 ceph-mon[82331]: pgmap v442: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:17.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:17 smithi114 ceph-mon[94452]: pgmap v442: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:19.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:19 smithi114 ceph-mon[94452]: pgmap v443: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:19.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:19 smithi123 ceph-mon[82331]: pgmap v443: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:21.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:21 smithi114 ceph-mon[94452]: pgmap v444: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:21.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:21 smithi123 ceph-mon[82331]: pgmap v444: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:23.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:23 smithi114 ceph-mon[94452]: pgmap v445: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:23.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:23 smithi123 ceph-mon[82331]: pgmap v445: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:25.452 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:25 smithi114 ceph-mon[94452]: pgmap v446: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:25.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:25 smithi123 ceph-mon[82331]: pgmap v446: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:27.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:27 smithi123 ceph-mon[82331]: pgmap v447: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:27.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:27 smithi114 ceph-mon[94452]: pgmap v447: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:29.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:29 smithi114 ceph-mon[94452]: pgmap v448: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:29.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:29 smithi123 ceph-mon[82331]: pgmap v448: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:31.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:31 smithi114 ceph-mon[94452]: pgmap v449: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:31.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:31 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:37:31.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:31 smithi123 ceph-mon[82331]: pgmap v449: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:31.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:31 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:37:33.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:33 smithi114 ceph-mon[94452]: pgmap v450: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:33.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:33 smithi123 ceph-mon[82331]: pgmap v450: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:35.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:35 smithi114 ceph-mon[94452]: pgmap v451: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:35.551 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:35 smithi123 ceph-mon[82331]: pgmap v451: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:37.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:37 smithi114 ceph-mon[94452]: pgmap v452: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:37.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:37 smithi123 ceph-mon[82331]: pgmap v452: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:39.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:39 smithi114 ceph-mon[94452]: pgmap v453: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:39.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:39 smithi123 ceph-mon[82331]: pgmap v453: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:41.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:41 smithi114 ceph-mon[94452]: pgmap v454: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:41.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:41 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:37:41.551 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:41 smithi123 ceph-mon[82331]: pgmap v454: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:41.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:41 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:37:42.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:42 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:37:42.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:42 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:37:42.201 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:42 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:37:42.551 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:42 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:37:42.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:42 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:37:42.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:42 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:37:43.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:43 smithi114 ceph-mon[94452]: pgmap v455: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:44.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:43 smithi123 ceph-mon[82331]: pgmap v455: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:45.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:45 smithi114 ceph-mon[94452]: pgmap v456: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:46.051 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:45 smithi123 ceph-mon[82331]: pgmap v456: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:46.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:37:47.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:46 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:37:47.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:47 smithi114 ceph-mon[94452]: pgmap v457: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:48.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:47 smithi123 ceph-mon[82331]: pgmap v457: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:48.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:48 smithi114 ceph-mon[94452]: pgmap v458: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:49.051 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:48 smithi123 ceph-mon[82331]: pgmap v458: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:51.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:50 smithi123 ceph-mon[82331]: pgmap v459: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:51.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:50 smithi114 ceph-mon[94452]: pgmap v459: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:53.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:52 smithi123 ceph-mon[82331]: pgmap v460: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:53.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:52 smithi114 ceph-mon[94452]: pgmap v460: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:55.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:54 smithi123 ceph-mon[82331]: pgmap v461: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:55.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:54 smithi114 ceph-mon[94452]: pgmap v461: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:57.303 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:56 smithi123 ceph-mon[82331]: pgmap v462: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:57.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:56 smithi114 ceph-mon[94452]: pgmap v462: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:59.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:37:58 smithi123 ceph-mon[82331]: pgmap v463: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:37:59.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:37:58 smithi114 ceph-mon[94452]: pgmap v463: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:01.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:01 smithi123 ceph-mon[82331]: pgmap v464: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:01.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:01 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:38:01.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:01 smithi114 ceph-mon[94452]: pgmap v464: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:01.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:01 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:38:03.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:03 smithi123 ceph-mon[82331]: pgmap v465: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:03.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:03 smithi114 ceph-mon[94452]: pgmap v465: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:05.302 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:05 smithi123 ceph-mon[82331]: pgmap v466: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:05.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:05 smithi114 ceph-mon[94452]: pgmap v466: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:07.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:07 smithi114 ceph-mon[94452]: pgmap v467: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:07.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:07 smithi123 ceph-mon[82331]: pgmap v467: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:09.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:09 smithi114 ceph-mon[94452]: pgmap v468: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:09.551 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:09 smithi123 ceph-mon[82331]: pgmap v468: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:11.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:11 smithi114 ceph-mon[94452]: pgmap v469: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:11.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:11 smithi123 ceph-mon[82331]: pgmap v469: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:13.451 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:13 smithi114 ceph-mon[94452]: pgmap v470: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:13.551 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:13 smithi123 ceph-mon[82331]: pgmap v470: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:15.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:15 smithi123 ceph-mon[82331]: pgmap v471: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:15.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:15 smithi114 ceph-mon[94452]: pgmap v471: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:16.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:16 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:38:16.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:16 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:38:17.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:17 smithi123 ceph-mon[82331]: pgmap v472: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:17.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:17 smithi114 ceph-mon[94452]: pgmap v472: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:19.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:19 smithi123 ceph-mon[82331]: pgmap v473: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:19.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:19 smithi114 ceph-mon[94452]: pgmap v473: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:21.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:21 smithi123 ceph-mon[82331]: pgmap v474: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:21.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:21 smithi114 ceph-mon[94452]: pgmap v474: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:23.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:23 smithi123 ceph-mon[82331]: pgmap v475: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:23.581 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:23 smithi114 ceph-mon[94452]: pgmap v475: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:25.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:25 smithi123 ceph-mon[82331]: pgmap v476: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:25.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:25 smithi114 ceph-mon[94452]: pgmap v476: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:27.534 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:27 smithi123 ceph-mon[82331]: pgmap v477: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:27.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:27 smithi114 ceph-mon[94452]: pgmap v477: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:29.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:29 smithi123 ceph-mon[82331]: pgmap v478: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:29.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:29 smithi114 ceph-mon[94452]: pgmap v478: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:31.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:31 smithi123 ceph-mon[82331]: pgmap v479: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:31.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:31 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:38:31.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:31 smithi114 ceph-mon[94452]: pgmap v479: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:31.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:31 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:38:33.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:33 smithi123 ceph-mon[82331]: pgmap v480: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:33.581 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:33 smithi114 ceph-mon[94452]: pgmap v480: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:35.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:35 smithi123 ceph-mon[82331]: pgmap v481: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:35.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:35 smithi114 ceph-mon[94452]: pgmap v481: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:37.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:37 smithi123 ceph-mon[82331]: pgmap v482: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:37.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:37 smithi114 ceph-mon[94452]: pgmap v482: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:39.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:39 smithi123 ceph-mon[82331]: pgmap v483: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:39.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:39 smithi114 ceph-mon[94452]: pgmap v483: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:41.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:41 smithi123 ceph-mon[82331]: pgmap v484: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:41.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:41 smithi114 ceph-mon[94452]: pgmap v484: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:42.528 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:42 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:38:42.551 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:42 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T22:38:43.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:43 smithi123 ceph-mon[82331]: pgmap v485: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:43.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:43 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:38:43.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:43 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:38:43.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:43 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:38:43.582 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:43 smithi114 ceph-mon[94452]: pgmap v485: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:43.582 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:43 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T22:38:43.582 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:43 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T22:38:43.582 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:43 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' 2024-02-16T22:38:45.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:45 smithi123 ceph-mon[82331]: pgmap v486: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:45.701 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:45 smithi114 ceph-mon[94452]: pgmap v486: 65 pgs: 65 active+clean; 258 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-16T22:38:46.490 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-02-16T22:38:46.491 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-02-16T22:38:46.493 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-02-16T22:38:46.504 ERROR:teuthology.run_tasks:Manager failed: full_sequential_finally Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/full_sequential_finally.py", line 38, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/full_sequential_finally.py", line 43, in task ((taskname, confg),) = entry.items() ValueError: too many values to unpack (expected 1) 2024-02-16T22:38:46.504 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-16T22:38:46.515 INFO:tasks.cephadm:Teardown begin 2024-02-16T22:38:46.515 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/cephadm.py", line 1807, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-02-16T22:38:46.516 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-16T22:38:46.546 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:46 smithi114 ceph-mon[94452]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:38:46.552 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-16T22:38:46.552 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:46 smithi123 ceph-mon[82331]: from='mgr.34104 172.21.15.114:0/540934535' entity='mgr.smithi114.vcullz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T22:38:46.587 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-16T22:38:46.587 DEBUG:teuthology.orchestra.run.smithi114:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-16T22:38:46.609 DEBUG:teuthology.orchestra.run.smithi123:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-16T22:38:46.643 INFO:tasks.cephadm:Stopping all daemons... 2024-02-16T22:38:46.644 INFO:tasks.cephadm.mon.smithi114:Stopping mon.smithi114... 2024-02-16T22:38:46.644 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@mon.smithi114 2024-02-16T22:38:46.951 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:46 smithi114 systemd[1]: Stopping Ceph mon.smithi114 for 1dcbaabe-cd18-11ee-95ba-87774f69a715... 2024-02-16T22:38:47.288 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:46 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi114[94448]: 2024-02-16T22:38:46.960+0000 7f1abc100700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi114 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T22:38:47.288 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:46 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi114[94448]: 2024-02-16T22:38:46.960+0000 7f1abc100700 -1 mon.smithi114@0(leader) e3 *** Got Signal Terminated *** 2024-02-16T22:38:47.288 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:46 smithi114 podman[154831]: 2024-02-16 22:38:46.981480978 +0000 UTC m=+0.141536073 container died 58ac063725235a5943ffb3c6a4284229d105f7cbf6736c59d56b3decc54325ea (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi114, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-02-16T22:38:47.576 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:47 smithi114 podman[154831]: 2024-02-16 22:38:47.461541513 +0000 UTC m=+0.621596615 container cleanup 58ac063725235a5943ffb3c6a4284229d105f7cbf6736c59d56b3decc54325ea (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi114, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-02-16T22:38:47.576 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:47 smithi114 bash[154831]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi114 2024-02-16T22:38:47.823 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@mon.smithi114.service' 2024-02-16T22:38:47.847 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:47 smithi114 podman[154845]: 2024-02-16 22:38:47.704670377 +0000 UTC m=+0.724250192 container remove 58ac063725235a5943ffb3c6a4284229d105f7cbf6736c59d56b3decc54325ea (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi114, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-02-16T22:38:47.847 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:47 smithi114 systemd[1]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@mon.smithi114.service: Deactivated successfully. 2024-02-16T22:38:47.847 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:47 smithi114 systemd[1]: Stopped Ceph mon.smithi114 for 1dcbaabe-cd18-11ee-95ba-87774f69a715. 2024-02-16T22:38:47.847 INFO:journalctl@ceph.mon.smithi114.smithi114.stdout:Feb 16 22:38:47 smithi114 systemd[1]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@mon.smithi114.service: Consumed 12.596s CPU time. 2024-02-16T22:38:47.878 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T22:38:47.878 INFO:tasks.cephadm.mon.smithi114:Stopped mon.smithi114 2024-02-16T22:38:47.878 INFO:tasks.cephadm.mon.smithi123:Stopping mon.smithi123... 2024-02-16T22:38:47.878 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@mon.smithi123 2024-02-16T22:38:48.052 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:47 smithi123 systemd[1]: Stopping Ceph mon.smithi123 for 1dcbaabe-cd18-11ee-95ba-87774f69a715... 2024-02-16T22:38:48.474 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:48 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi123[82327]: 2024-02-16T22:38:48.164+0000 7fce905ce700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi123 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T22:38:48.474 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:48 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi123[82327]: 2024-02-16T22:38:48.164+0000 7fce905ce700 -1 mon.smithi123@1(peon) e3 *** Got Signal Terminated *** 2024-02-16T22:38:48.474 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:48 smithi123 podman[124723]: 2024-02-16 22:38:48.185488272 +0000 UTC m=+0.118836739 container died f76485e0ea24d21414f3dc5d1ff1b3351f9dd1c9422ff2307d0d27a43419a078 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi123, ceph=True, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=) 2024-02-16T22:38:48.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:48 smithi123 podman[124723]: 2024-02-16 22:38:48.656076761 +0000 UTC m=+0.589425230 container cleanup f76485e0ea24d21414f3dc5d1ff1b3351f9dd1c9422ff2307d0d27a43419a078 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi123, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T22:38:48.802 INFO:journalctl@ceph.mon.smithi123.smithi123.stdout:Feb 16 22:38:48 smithi123 bash[124723]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-mon-smithi123 2024-02-16T22:38:49.075 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@mon.smithi123.service' 2024-02-16T22:38:49.117 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T22:38:49.118 INFO:tasks.cephadm.mon.smithi123:Stopped mon.smithi123 2024-02-16T22:38:49.118 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-16T22:38:49.118 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@osd.0 2024-02-16T22:38:49.400 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:38:49 smithi114 systemd[1]: Stopping Ceph osd.0 for 1dcbaabe-cd18-11ee-95ba-87774f69a715... 2024-02-16T22:38:49.701 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:38:49 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0[103547]: 2024-02-16T22:38:49.398+0000 7f359e590700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T22:38:49.701 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:38:49 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0[103547]: 2024-02-16T22:38:49.398+0000 7f359e590700 -1 osd.0 82 *** Got signal Terminated *** 2024-02-16T22:38:49.701 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:38:49 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0[103547]: 2024-02-16T22:38:49.398+0000 7f359e590700 -1 osd.0 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T22:38:55.110 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:38:54 smithi114 podman[154966]: 2024-02-16 22:38:54.802679067 +0000 UTC m=+5.495298405 container died 1978fdad94a55f95dc3bc9df43b45ad517bc5ac0b5547294892a4d45d98c9dd6 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-02-16T22:38:55.402 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:38:55 smithi114 podman[154966]: 2024-02-16 22:38:55.208630285 +0000 UTC m=+5.901249625 container cleanup 1978fdad94a55f95dc3bc9df43b45ad517bc5ac0b5547294892a4d45d98c9dd6 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=) 2024-02-16T22:38:55.402 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:38:55 smithi114 bash[154966]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0 2024-02-16T22:38:55.701 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:38:55 smithi114 podman[154985]: 2024-02-16 22:38:55.401479245 +0000 UTC m=+0.589377748 container remove 1978fdad94a55f95dc3bc9df43b45ad517bc5ac0b5547294892a4d45d98c9dd6 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T22:38:56.082 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:38:55 smithi114 podman[155077]: 2024-02-16 22:38:55.727551301 +0000 UTC m=+0.017574347 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T22:38:56.083 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:38:55 smithi114 podman[155077]: 2024-02-16 22:38:55.852694833 +0000 UTC m=+0.142717872 container create 352a1f7789ef98305038901db18ee3d594cd8bded31244644be4804dea49cea2 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-02-16T22:38:56.378 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:38:56 smithi114 podman[155077]: 2024-02-16 22:38:56.081535889 +0000 UTC m=+0.371558933 container init 352a1f7789ef98305038901db18ee3d594cd8bded31244644be4804dea49cea2 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-02-16T22:38:56.378 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:38:56 smithi114 podman[155077]: 2024-02-16 22:38:56.085351247 +0000 UTC m=+0.375374294 container start 352a1f7789ef98305038901db18ee3d594cd8bded31244644be4804dea49cea2 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True) 2024-02-16T22:38:56.378 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:38:56 smithi114 podman[155077]: 2024-02-16 22:38:56.16389696 +0000 UTC m=+0.453920003 container attach 352a1f7789ef98305038901db18ee3d594cd8bded31244644be4804dea49cea2 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-02-16T22:38:56.685 INFO:journalctl@ceph.osd.0.smithi114.stdout:Feb 16 22:38:56 smithi114 podman[155077]: 2024-02-16 22:38:56.377394123 +0000 UTC m=+0.667417163 container died 352a1f7789ef98305038901db18ee3d594cd8bded31244644be4804dea49cea2 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-02-16T22:38:57.009 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@osd.0.service' 2024-02-16T22:38:57.055 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T22:38:57.055 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-16T22:38:57.055 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-16T22:38:57.055 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@osd.1 2024-02-16T22:38:57.284 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:38:57 smithi114 systemd[1]: Stopping Ceph osd.1 for 1dcbaabe-cd18-11ee-95ba-87774f69a715... 2024-02-16T22:38:57.668 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:38:57 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1[109147]: 2024-02-16T22:38:57.378+0000 7fe7a5ade700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T22:38:57.668 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:38:57 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1[109147]: 2024-02-16T22:38:57.378+0000 7fe7a5ade700 -1 osd.1 82 *** Got signal Terminated *** 2024-02-16T22:38:57.668 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:38:57 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1[109147]: 2024-02-16T22:38:57.378+0000 7fe7a5ade700 -1 osd.1 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T22:39:03.006 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:39:02 smithi114 podman[155182]: 2024-02-16 22:39:02.768246691 +0000 UTC m=+5.524999957 container died 3f90e792898fe87e2fa6e43bce9d0479cade0580bbe38a5b6d81039508338b8b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, ceph=True) 2024-02-16T22:39:03.348 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:39:03 smithi114 podman[155182]: 2024-02-16 22:39:03.114634071 +0000 UTC m=+5.871387349 container cleanup 3f90e792898fe87e2fa6e43bce9d0479cade0580bbe38a5b6d81039508338b8b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True) 2024-02-16T22:39:03.348 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:39:03 smithi114 bash[155182]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1 2024-02-16T22:39:03.701 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:39:03 smithi114 podman[155210]: 2024-02-16 22:39:03.347824214 +0000 UTC m=+0.575706979 container remove 3f90e792898fe87e2fa6e43bce9d0479cade0580bbe38a5b6d81039508338b8b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-02-16T22:39:04.052 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:39:03 smithi114 podman[155301]: 2024-02-16 22:39:03.677025559 +0000 UTC m=+0.017978127 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T22:39:04.053 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:39:03 smithi114 podman[155301]: 2024-02-16 22:39:03.798864055 +0000 UTC m=+0.139816613 container create 3d0978e7add973a08a3227d4969369f8e166997978d72392d835c7bc4e23c7b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214) 2024-02-16T22:39:04.350 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:39:04 smithi114 podman[155301]: 2024-02-16 22:39:04.051917715 +0000 UTC m=+0.392870295 container init 3d0978e7add973a08a3227d4969369f8e166997978d72392d835c7bc4e23c7b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-02-16T22:39:04.350 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:39:04 smithi114 podman[155301]: 2024-02-16 22:39:04.055452008 +0000 UTC m=+0.396404571 container start 3d0978e7add973a08a3227d4969369f8e166997978d72392d835c7bc4e23c7b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T22:39:04.351 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:39:04 smithi114 podman[155301]: 2024-02-16 22:39:04.134706921 +0000 UTC m=+0.475659492 container attach 3d0978e7add973a08a3227d4969369f8e166997978d72392d835c7bc4e23c7b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux ) 2024-02-16T22:39:04.664 INFO:journalctl@ceph.osd.1.smithi114.stdout:Feb 16 22:39:04 smithi114 podman[155301]: 2024-02-16 22:39:04.349869187 +0000 UTC m=+0.690821755 container died 3d0978e7add973a08a3227d4969369f8e166997978d72392d835c7bc4e23c7b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-1-deactivate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, ceph=True) 2024-02-16T22:39:04.981 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@osd.1.service' 2024-02-16T22:39:05.024 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T22:39:05.024 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-16T22:39:05.024 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-16T22:39:05.025 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@osd.2 2024-02-16T22:39:05.393 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:39:05 smithi114 systemd[1]: Stopping Ceph osd.2 for 1dcbaabe-cd18-11ee-95ba-87774f69a715... 2024-02-16T22:39:05.701 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:39:05 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2[114596]: 2024-02-16T22:39:05.391+0000 7fb1c291b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T22:39:05.701 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:39:05 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2[114596]: 2024-02-16T22:39:05.391+0000 7fb1c291b700 -1 osd.2 82 *** Got signal Terminated *** 2024-02-16T22:39:05.701 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:39:05 smithi114 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2[114596]: 2024-02-16T22:39:05.391+0000 7fb1c291b700 -1 osd.2 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T22:39:11.184 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:39:10 smithi114 podman[155407]: 2024-02-16 22:39:10.855220809 +0000 UTC m=+5.606468822 container died c663dfa0518bdf5185229b25afe3ed746bacd0994e5af9ff5caf921541aa4609 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=) 2024-02-16T22:39:11.184 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:39:11 smithi114 podman[155407]: 2024-02-16 22:39:11.183148309 +0000 UTC m=+5.934396319 container cleanup c663dfa0518bdf5185229b25afe3ed746bacd0994e5af9ff5caf921541aa4609 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD) 2024-02-16T22:39:11.451 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:39:11 smithi114 bash[155407]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2 2024-02-16T22:39:11.451 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:39:11 smithi114 podman[155424]: 2024-02-16 22:39:11.39280545 +0000 UTC m=+0.539024373 container remove c663dfa0518bdf5185229b25afe3ed746bacd0994e5af9ff5caf921541aa4609 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T22:39:12.114 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:39:11 smithi114 podman[155517]: 2024-02-16 22:39:11.709242432 +0000 UTC m=+0.018086697 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T22:39:12.114 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:39:11 smithi114 podman[155517]: 2024-02-16 22:39:11.810499904 +0000 UTC m=+0.119344210 container create 7569bd803c15ed492d8192cb0e41e5f116d7e499ea3c2232ee08dbe1fdab2d5d (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux ) 2024-02-16T22:39:12.114 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:39:12 smithi114 podman[155517]: 2024-02-16 22:39:12.038384312 +0000 UTC m=+0.347228576 container init 7569bd803c15ed492d8192cb0e41e5f116d7e499ea3c2232ee08dbe1fdab2d5d (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T22:39:12.114 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:39:12 smithi114 podman[155517]: 2024-02-16 22:39:12.041606955 +0000 UTC m=+0.350451225 container start 7569bd803c15ed492d8192cb0e41e5f116d7e499ea3c2232ee08dbe1fdab2d5d (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-02-16T22:39:12.114 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:39:12 smithi114 podman[155517]: 2024-02-16 22:39:12.11314344 +0000 UTC m=+0.421987723 container attach 7569bd803c15ed492d8192cb0e41e5f116d7e499ea3c2232ee08dbe1fdab2d5d (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux ) 2024-02-16T22:39:12.451 INFO:journalctl@ceph.osd.2.smithi114.stdout:Feb 16 22:39:12 smithi114 podman[155517]: 2024-02-16 22:39:12.324629536 +0000 UTC m=+0.633473803 container died 7569bd803c15ed492d8192cb0e41e5f116d7e499ea3c2232ee08dbe1fdab2d5d (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T22:39:12.963 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@osd.2.service' 2024-02-16T22:39:13.007 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T22:39:13.007 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-16T22:39:13.007 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-16T22:39:13.008 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@osd.3 2024-02-16T22:39:13.300 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:39:13 smithi123 systemd[1]: Stopping Ceph osd.3 for 1dcbaabe-cd18-11ee-95ba-87774f69a715... 2024-02-16T22:39:13.551 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:39:13 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3[92060]: 2024-02-16T22:39:13.299+0000 7f33b8979700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T22:39:13.551 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:39:13 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3[92060]: 2024-02-16T22:39:13.299+0000 7f33b8979700 -1 osd.3 82 *** Got signal Terminated *** 2024-02-16T22:39:13.552 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:39:13 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3[92060]: 2024-02-16T22:39:13.299+0000 7f33b8979700 -1 osd.3 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T22:39:16.802 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:39:16 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3[92060]: 2024-02-16T22:39:16.384+0000 7f33b576e700 -1 osd.3 82 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-02-16T22:38:50.853116+0000 front 2024-02-16T22:38:50.853045+0000 (oldest deadline 2024-02-16T22:39:16.152942+0000) 2024-02-16T22:39:17.802 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:39:17 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3[92060]: 2024-02-16T22:39:17.428+0000 7f33b576e700 -1 osd.3 82 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-02-16T22:38:50.853116+0000 front 2024-02-16T22:38:50.853045+0000 (oldest deadline 2024-02-16T22:39:16.152942+0000) 2024-02-16T22:39:17.802 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:39:17 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4[97432]: 2024-02-16T22:39:17.599+0000 7f74737a3700 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-02-16T22:38:52.175012+0000 front 2024-02-16T22:38:52.175021+0000 (oldest deadline 2024-02-16T22:39:16.874886+0000) 2024-02-16T22:39:18.943 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:39:18 smithi123 podman[124847]: 2024-02-16 22:39:18.71697753 +0000 UTC m=+5.523086933 container died b921448ba169e5c06eb928ef5deb1d1e36a264f84fef4b24355587700c38e996 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214) 2024-02-16T22:39:18.943 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:39:18 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4[97432]: 2024-02-16T22:39:18.572+0000 7f74737a3700 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-02-16T22:38:52.175012+0000 front 2024-02-16T22:38:52.175021+0000 (oldest deadline 2024-02-16T22:39:16.874886+0000) 2024-02-16T22:39:19.276 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:39:19 smithi123 podman[124847]: 2024-02-16 22:39:19.041261453 +0000 UTC m=+5.847370864 container cleanup b921448ba169e5c06eb928ef5deb1d1e36a264f84fef4b24355587700c38e996 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, ceph=True) 2024-02-16T22:39:19.276 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:39:19 smithi123 bash[124847]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3 2024-02-16T22:39:19.551 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:39:19 smithi123 podman[124876]: 2024-02-16 22:39:19.275862287 +0000 UTC m=+0.552739774 container remove b921448ba169e5c06eb928ef5deb1d1e36a264f84fef4b24355587700c38e996 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-02-16T22:39:19.552 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:19 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:39:19.354+0000 7f01703b3700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-02-16T22:38:53.129970+0000 front 2024-02-16T22:38:53.130055+0000 (oldest deadline 2024-02-16T22:39:19.029899+0000) 2024-02-16T22:39:19.862 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:39:19 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4[97432]: 2024-02-16T22:39:19.602+0000 7f74737a3700 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-02-16T22:38:52.175012+0000 front 2024-02-16T22:38:52.175021+0000 (oldest deadline 2024-02-16T22:39:16.874886+0000) 2024-02-16T22:39:19.862 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:39:19 smithi123 podman[124966]: 2024-02-16 22:39:19.606764473 +0000 UTC m=+0.018788510 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T22:39:19.862 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:39:19 smithi123 podman[124966]: 2024-02-16 22:39:19.718346847 +0000 UTC m=+0.130370886 container create e954a54ddc28214ee9100392304f7d7f7be72aa9462511712a3b9053c85afac1 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-02-16T22:39:20.282 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:39:19 smithi123 podman[124966]: 2024-02-16 22:39:19.987448743 +0000 UTC m=+0.399472780 container init e954a54ddc28214ee9100392304f7d7f7be72aa9462511712a3b9053c85afac1 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5) 2024-02-16T22:39:20.282 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:39:19 smithi123 podman[124966]: 2024-02-16 22:39:19.990660313 +0000 UTC m=+0.402684354 container start e954a54ddc28214ee9100392304f7d7f7be72aa9462511712a3b9053c85afac1 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2) 2024-02-16T22:39:20.282 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:39:20 smithi123 podman[124966]: 2024-02-16 22:39:20.070633392 +0000 UTC m=+0.482657430 container attach e954a54ddc28214ee9100392304f7d7f7be72aa9462511712a3b9053c85afac1 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-02-16T22:39:20.551 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:20 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:39:20.400+0000 7f01703b3700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-02-16T22:38:53.129970+0000 front 2024-02-16T22:38:53.130055+0000 (oldest deadline 2024-02-16T22:39:19.029899+0000) 2024-02-16T22:39:20.552 INFO:journalctl@ceph.osd.3.smithi123.stdout:Feb 16 22:39:20 smithi123 podman[124966]: 2024-02-16 22:39:20.281934592 +0000 UTC m=+0.693958637 container died e954a54ddc28214ee9100392304f7d7f7be72aa9462511712a3b9053c85afac1 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-02-16T22:39:20.841 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:39:20 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4[97432]: 2024-02-16T22:39:20.632+0000 7f74737a3700 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-02-16T22:38:52.175012+0000 front 2024-02-16T22:38:52.175021+0000 (oldest deadline 2024-02-16T22:39:16.874886+0000) 2024-02-16T22:39:20.861 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@osd.3.service' 2024-02-16T22:39:20.902 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T22:39:20.902 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-16T22:39:20.902 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-16T22:39:20.902 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@osd.4 2024-02-16T22:39:21.253 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:39:20 smithi123 systemd[1]: Stopping Ceph osd.4 for 1dcbaabe-cd18-11ee-95ba-87774f69a715... 2024-02-16T22:39:21.253 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:39:21 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4[97432]: 2024-02-16T22:39:21.252+0000 7f747a9b6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T22:39:21.253 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:39:21 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4[97432]: 2024-02-16T22:39:21.252+0000 7f747a9b6700 -1 osd.4 82 *** Got signal Terminated *** 2024-02-16T22:39:21.254 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:39:21 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4[97432]: 2024-02-16T22:39:21.252+0000 7f747a9b6700 -1 osd.4 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T22:39:21.552 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:21 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:39:21.353+0000 7f01703b3700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-02-16T22:38:53.129970+0000 front 2024-02-16T22:38:53.130055+0000 (oldest deadline 2024-02-16T22:39:19.029899+0000) 2024-02-16T22:39:22.051 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:39:21 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4[97432]: 2024-02-16T22:39:21.585+0000 7f74737a3700 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-02-16T22:38:52.175012+0000 front 2024-02-16T22:38:52.175021+0000 (oldest deadline 2024-02-16T22:39:16.874886+0000) 2024-02-16T22:39:22.802 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:39:22 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4[97432]: 2024-02-16T22:39:22.570+0000 7f74737a3700 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-02-16T22:38:52.175012+0000 front 2024-02-16T22:38:52.175021+0000 (oldest deadline 2024-02-16T22:39:16.874886+0000) 2024-02-16T22:39:22.802 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:22 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:39:22.367+0000 7f01703b3700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-02-16T22:38:53.129970+0000 front 2024-02-16T22:38:53.130055+0000 (oldest deadline 2024-02-16T22:39:19.029899+0000) 2024-02-16T22:39:23.802 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:39:23 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4[97432]: 2024-02-16T22:39:23.597+0000 7f74737a3700 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-02-16T22:38:52.175012+0000 front 2024-02-16T22:38:52.175021+0000 (oldest deadline 2024-02-16T22:39:16.874886+0000) 2024-02-16T22:39:23.802 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:23 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:39:23.401+0000 7f01703b3700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-02-16T22:38:53.129970+0000 front 2024-02-16T22:38:53.130055+0000 (oldest deadline 2024-02-16T22:39:19.029899+0000) 2024-02-16T22:39:24.802 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:39:24 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4[97432]: 2024-02-16T22:39:24.607+0000 7f74737a3700 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-02-16T22:38:52.175012+0000 front 2024-02-16T22:38:52.175021+0000 (oldest deadline 2024-02-16T22:39:16.874886+0000) 2024-02-16T22:39:24.802 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:39:24 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4[97432]: 2024-02-16T22:39:24.607+0000 7f74737a3700 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.114:6814 osd.1 since back 2024-02-16T22:38:58.575652+0000 front 2024-02-16T22:38:58.575670+0000 (oldest deadline 2024-02-16T22:39:23.875683+0000) 2024-02-16T22:39:24.802 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:24 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:39:24.377+0000 7f01703b3700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-02-16T22:38:53.129970+0000 front 2024-02-16T22:38:53.130055+0000 (oldest deadline 2024-02-16T22:39:19.029899+0000) 2024-02-16T22:39:24.802 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:24 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:39:24.377+0000 7f01703b3700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.114:6814 osd.1 since back 2024-02-16T22:39:01.831318+0000 front 2024-02-16T22:39:01.831339+0000 (oldest deadline 2024-02-16T22:39:23.531020+0000) 2024-02-16T22:39:25.609 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:25 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:39:25.334+0000 7f01703b3700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-02-16T22:38:53.129970+0000 front 2024-02-16T22:38:53.130055+0000 (oldest deadline 2024-02-16T22:39:19.029899+0000) 2024-02-16T22:39:25.609 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:25 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:39:25.334+0000 7f01703b3700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.114:6814 osd.1 since back 2024-02-16T22:39:01.831318+0000 front 2024-02-16T22:39:01.831339+0000 (oldest deadline 2024-02-16T22:39:23.531020+0000) 2024-02-16T22:39:26.052 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:39:25 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4[97432]: 2024-02-16T22:39:25.607+0000 7f74737a3700 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-02-16T22:38:52.175012+0000 front 2024-02-16T22:38:52.175021+0000 (oldest deadline 2024-02-16T22:39:16.874886+0000) 2024-02-16T22:39:26.052 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:39:25 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4[97432]: 2024-02-16T22:39:25.607+0000 7f74737a3700 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.114:6814 osd.1 since back 2024-02-16T22:38:58.575652+0000 front 2024-02-16T22:38:58.575670+0000 (oldest deadline 2024-02-16T22:39:23.875683+0000) 2024-02-16T22:39:26.552 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:26 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:39:26.294+0000 7f01703b3700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-02-16T22:38:53.129970+0000 front 2024-02-16T22:38:53.130055+0000 (oldest deadline 2024-02-16T22:39:19.029899+0000) 2024-02-16T22:39:26.552 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:26 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:39:26.294+0000 7f01703b3700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.114:6814 osd.1 since back 2024-02-16T22:39:01.831318+0000 front 2024-02-16T22:39:01.831339+0000 (oldest deadline 2024-02-16T22:39:23.531020+0000) 2024-02-16T22:39:26.978 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:39:26 smithi123 podman[125074]: 2024-02-16 22:39:26.707597112 +0000 UTC m=+5.580668139 container died 54c2926d56b18cbda791e1f15b658ed259633a5b9fcaecf2fa72a1a4790b4958 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T22:39:27.289 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:39:27 smithi123 podman[125074]: 2024-02-16 22:39:27.084971867 +0000 UTC m=+5.958042893 container cleanup 54c2926d56b18cbda791e1f15b658ed259633a5b9fcaecf2fa72a1a4790b4958 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0) 2024-02-16T22:39:27.289 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:39:27 smithi123 bash[125074]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4 2024-02-16T22:39:27.551 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:39:27 smithi123 podman[125093]: 2024-02-16 22:39:27.344365362 +0000 UTC m=+0.630679504 container remove 54c2926d56b18cbda791e1f15b658ed259633a5b9fcaecf2fa72a1a4790b4958 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-02-16T22:39:27.552 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:27 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:39:27.287+0000 7f01703b3700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-02-16T22:38:53.129970+0000 front 2024-02-16T22:38:53.130055+0000 (oldest deadline 2024-02-16T22:39:19.029899+0000) 2024-02-16T22:39:27.552 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:27 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:39:27.287+0000 7f01703b3700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.114:6814 osd.1 since back 2024-02-16T22:39:01.831318+0000 front 2024-02-16T22:39:01.831339+0000 (oldest deadline 2024-02-16T22:39:23.531020+0000) 2024-02-16T22:39:27.940 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:39:27 smithi123 podman[125188]: 2024-02-16 22:39:27.560288498 +0000 UTC m=+0.018383354 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T22:39:27.940 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:39:27 smithi123 podman[125188]: 2024-02-16 22:39:27.678687998 +0000 UTC m=+0.136782846 container create 8b445070123a90b3059a0bf89293715388d33d58b5f3374e59b73e6b803378b1 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5) 2024-02-16T22:39:27.940 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:39:27 smithi123 podman[125188]: 2024-02-16 22:39:27.8645648 +0000 UTC m=+0.322659647 container init 8b445070123a90b3059a0bf89293715388d33d58b5f3374e59b73e6b803378b1 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4-deactivate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T22:39:27.940 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:39:27 smithi123 podman[125188]: 2024-02-16 22:39:27.867635484 +0000 UTC m=+0.325730333 container start 8b445070123a90b3059a0bf89293715388d33d58b5f3374e59b73e6b803378b1 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T22:39:27.940 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:39:27 smithi123 podman[125188]: 2024-02-16 22:39:27.939472206 +0000 UTC m=+0.397567073 container attach 8b445070123a90b3059a0bf89293715388d33d58b5f3374e59b73e6b803378b1 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T22:39:28.277 INFO:journalctl@ceph.osd.4.smithi123.stdout:Feb 16 22:39:28 smithi123 podman[125188]: 2024-02-16 22:39:28.162521244 +0000 UTC m=+0.620616091 container died 8b445070123a90b3059a0bf89293715388d33d58b5f3374e59b73e6b803378b1 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-02-16T22:39:28.277 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:28 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:39:28.275+0000 7f01703b3700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-02-16T22:38:53.129970+0000 front 2024-02-16T22:38:53.130055+0000 (oldest deadline 2024-02-16T22:39:19.029899+0000) 2024-02-16T22:39:28.277 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:28 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:39:28.275+0000 7f01703b3700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.114:6814 osd.1 since back 2024-02-16T22:39:01.831318+0000 front 2024-02-16T22:39:01.831339+0000 (oldest deadline 2024-02-16T22:39:23.531020+0000) 2024-02-16T22:39:28.764 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@osd.4.service' 2024-02-16T22:39:28.806 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T22:39:28.806 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-16T22:39:28.807 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-16T22:39:28.807 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@osd.5 2024-02-16T22:39:29.051 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:28 smithi123 systemd[1]: Stopping Ceph osd.5 for 1dcbaabe-cd18-11ee-95ba-87774f69a715... 2024-02-16T22:39:29.552 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:29 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:39:29.154+0000 7f01775c6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T22:39:29.552 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:29 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:39:29.154+0000 7f01775c6700 -1 osd.5 82 *** Got signal Terminated *** 2024-02-16T22:39:29.552 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:29 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:39:29.154+0000 7f01775c6700 -1 osd.5 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T22:39:29.552 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:29 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:39:29.256+0000 7f01703b3700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-02-16T22:38:53.129970+0000 front 2024-02-16T22:38:53.130055+0000 (oldest deadline 2024-02-16T22:39:19.029899+0000) 2024-02-16T22:39:29.552 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:29 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:39:29.256+0000 7f01703b3700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.114:6814 osd.1 since back 2024-02-16T22:39:01.831318+0000 front 2024-02-16T22:39:01.831339+0000 (oldest deadline 2024-02-16T22:39:23.531020+0000) 2024-02-16T22:39:30.552 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:30 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:39:30.234+0000 7f01703b3700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-02-16T22:38:53.129970+0000 front 2024-02-16T22:38:53.130055+0000 (oldest deadline 2024-02-16T22:39:19.029899+0000) 2024-02-16T22:39:30.552 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:30 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:39:30.234+0000 7f01703b3700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.114:6814 osd.1 since back 2024-02-16T22:39:01.831318+0000 front 2024-02-16T22:39:01.831339+0000 (oldest deadline 2024-02-16T22:39:23.531020+0000) 2024-02-16T22:39:31.552 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:31 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:39:31.206+0000 7f01703b3700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-02-16T22:38:53.129970+0000 front 2024-02-16T22:38:53.130055+0000 (oldest deadline 2024-02-16T22:39:19.029899+0000) 2024-02-16T22:39:31.552 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:31 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:39:31.206+0000 7f01703b3700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.114:6814 osd.1 since back 2024-02-16T22:39:01.831318+0000 front 2024-02-16T22:39:01.831339+0000 (oldest deadline 2024-02-16T22:39:23.531020+0000) 2024-02-16T22:39:32.551 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:32 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:39:32.229+0000 7f01703b3700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-02-16T22:38:53.129970+0000 front 2024-02-16T22:38:53.130055+0000 (oldest deadline 2024-02-16T22:39:19.029899+0000) 2024-02-16T22:39:32.552 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:32 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:39:32.229+0000 7f01703b3700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.114:6814 osd.1 since back 2024-02-16T22:39:01.831318+0000 front 2024-02-16T22:39:01.831339+0000 (oldest deadline 2024-02-16T22:39:23.531020+0000) 2024-02-16T22:39:33.552 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:33 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:39:33.235+0000 7f01703b3700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-02-16T22:38:53.129970+0000 front 2024-02-16T22:38:53.130055+0000 (oldest deadline 2024-02-16T22:39:19.029899+0000) 2024-02-16T22:39:33.552 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:33 smithi123 ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5[102795]: 2024-02-16T22:39:33.235+0000 7f01703b3700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.114:6814 osd.1 since back 2024-02-16T22:39:01.831318+0000 front 2024-02-16T22:39:01.831339+0000 (oldest deadline 2024-02-16T22:39:23.531020+0000) 2024-02-16T22:39:34.864 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:34 smithi123 podman[125294]: 2024-02-16 22:39:34.600532181 +0000 UTC m=+5.574191352 container died dd99bf03474900061451ba59068106233278253dd5aaf375fca8ac3f5245ef90 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-02-16T22:39:35.222 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:34 smithi123 podman[125294]: 2024-02-16 22:39:34.962105339 +0000 UTC m=+5.935764502 container cleanup dd99bf03474900061451ba59068106233278253dd5aaf375fca8ac3f5245ef90 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214) 2024-02-16T22:39:35.222 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:34 smithi123 bash[125294]: ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5 2024-02-16T22:39:35.552 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:35 smithi123 podman[125311]: 2024-02-16 22:39:35.221613156 +0000 UTC m=+0.621080535 container remove dd99bf03474900061451ba59068106233278253dd5aaf375fca8ac3f5245ef90 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T22:39:35.917 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:35 smithi123 podman[125407]: 2024-02-16 22:39:35.563999373 +0000 UTC m=+0.108879268 container create 12845b465725174aae083ebece5c73a3ee13b74b78ec6561afcd7f9d91c1444c (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-02-16T22:39:35.917 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:35 smithi123 podman[125407]: 2024-02-16 22:39:35.473099488 +0000 UTC m=+0.017979383 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T22:39:35.917 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:35 smithi123 podman[125407]: 2024-02-16 22:39:35.808104864 +0000 UTC m=+0.352984758 container init 12845b465725174aae083ebece5c73a3ee13b74b78ec6561afcd7f9d91c1444c (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5-deactivate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True) 2024-02-16T22:39:35.917 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:35 smithi123 podman[125407]: 2024-02-16 22:39:35.811350678 +0000 UTC m=+0.356230580 container start 12845b465725174aae083ebece5c73a3ee13b74b78ec6561afcd7f9d91c1444c (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T22:39:35.917 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:35 smithi123 podman[125407]: 2024-02-16 22:39:35.916209254 +0000 UTC m=+0.461089165 container attach 12845b465725174aae083ebece5c73a3ee13b74b78ec6561afcd7f9d91c1444c (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T22:39:36.302 INFO:journalctl@ceph.osd.5.smithi123.stdout:Feb 16 22:39:36 smithi123 podman[125407]: 2024-02-16 22:39:36.097353733 +0000 UTC m=+0.642233634 container died 12845b465725174aae083ebece5c73a3ee13b74b78ec6561afcd7f9d91c1444c (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715-osd-5-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T22:39:36.674 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1dcbaabe-cd18-11ee-95ba-87774f69a715@osd.5.service' 2024-02-16T22:39:36.715 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T22:39:36.715 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-16T22:39:36.716 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 --force --keep-logs 2024-02-16T22:39:36.874 INFO:teuthology.orchestra.run.smithi114.stdout:Deleting cluster with fsid: 1dcbaabe-cd18-11ee-95ba-87774f69a715 2024-02-16T22:39:41.160 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi114.stderr:ceph-fuse[77657]: fuse finished with error 0 and tester_r 0 2024-02-16T22:40:01.517 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 --force --keep-logs 2024-02-16T22:40:01.678 INFO:teuthology.orchestra.run.smithi123.stdout:Deleting cluster with fsid: 1dcbaabe-cd18-11ee-95ba-87774f69a715 2024-02-16T22:40:14.129 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-16T22:40:14.164 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-16T22:40:14.197 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-16T22:40:14.199 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562722/remote/smithi114/crash 2024-02-16T22:40:14.200 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/crash -- . 2024-02-16T22:40:14.234 INFO:teuthology.orchestra.run.smithi114.stderr:tar: /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/crash: Cannot open: No such file or directory 2024-02-16T22:40:14.234 INFO:teuthology.orchestra.run.smithi114.stderr:tar: Error is not recoverable: exiting now 2024-02-16T22:40:14.236 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562722/remote/smithi123/crash 2024-02-16T22:40:14.237 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/crash -- . 2024-02-16T22:40:14.264 INFO:teuthology.orchestra.run.smithi123.stderr:tar: /var/lib/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/crash: Cannot open: No such file or directory 2024-02-16T22:40:14.264 INFO:teuthology.orchestra.run.smithi123.stderr:tar: Error is not recoverable: exiting now 2024-02-16T22:40:14.266 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-16T22:40:14.266 DEBUG:teuthology.orchestra.run.smithi114:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-02-16T22:40:14.310 INFO:tasks.cephadm:Compressing logs... 2024-02-16T22:40:14.310 DEBUG:teuthology.orchestra.run.smithi114:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-16T22:40:14.353 DEBUG:teuthology.orchestra.run.smithi123:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-16T22:40:14.387 INFO:teuthology.orchestra.run.smithi114.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-16T22:40:14.387 INFO:teuthology.orchestra.run.smithi114.stderr:: No such file or directory 2024-02-16T22:40:14.387 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-02-16T22:40:14.387 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi114.fdxpey.log 2024-02-16T22:40:14.387 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph.audit.log 2024-02-16T22:40:14.387 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi114.fdxpey.log: gzip -5 --verbose -- /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-osd.1.log 2024-02-16T22:40:14.387 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-client.ceph-exporter.smithi114.log 2024-02-16T22:40:14.387 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-mon.smithi114.log 2024-02-16T22:40:14.387 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-client.ceph-exporter.smithi114.log: gzip -5 --verbose -- /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi114.zzryke.log 2024-02-16T22:40:14.388 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-mon.smithi114.log: gzip -5 --verbose -- /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-osd.0.log 2024-02-16T22:40:14.388 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi114.zzryke.log: 94.2% -- replaced with /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-client.ceph-exporter.smithi114.log.gz 2024-02-16T22:40:14.388 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph.cephadm.log 2024-02-16T22:40:14.388 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-osd.0.log: 91.5% -- replaced with /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph.audit.log.gz 2024-02-16T22:40:14.392 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph.log 2024-02-16T22:40:14.392 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph.cephadm.log: 85.9% -- replaced with /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph.cephadm.log.gz 2024-02-16T22:40:14.392 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-osd.2.log 2024-02-16T22:40:14.392 INFO:teuthology.orchestra.run.smithi123.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/cephadm.log 2024-02-16T22:40:14.392 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-02-16T22:40:14.392 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-client.ceph-exporter.smithi123.log 2024-02-16T22:40:14.393 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph.audit.log 2024-02-16T22:40:14.393 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-mon.smithi123.log 2024-02-16T22:40:14.393 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-client.ceph-exporter.smithi123.log: /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-osd.4.log 2024-02-16T22:40:14.393 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-osd.5.log 2024-02-16T22:40:14.393 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-mon.smithi123.log: /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-osd.3.log 2024-02-16T22:40:14.393 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi123.vhlqbe.log 2024-02-16T22:40:14.393 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph.cephadm.log 2024-02-16T22:40:14.393 INFO:teuthology.orchestra.run.smithi123.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-16T22:40:14.393 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi123.vhlqbe.log: 94.3% -- replaced with /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-client.ceph-exporter.smithi123.log.gz 2024-02-16T22:40:14.393 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-mgr.smithi123.inzgjd.log 2024-02-16T22:40:14.393 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph.log 2024-02-16T22:40:14.393 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-mgr.smithi123.inzgjd.log: 85.9% -- replaced with /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph.cephadm.log.gz 2024-02-16T22:40:14.393 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-volume.log 2024-02-16T22:40:14.393 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph.log: 91.6% -- replaced with /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph.audit.log.gz 2024-02-16T22:40:14.393 INFO:teuthology.orchestra.run.smithi123.stderr: 88.7% -- replaced with /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph.log.gz 2024-02-16T22:40:14.394 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi123.gopcfa.log 2024-02-16T22:40:14.394 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-02-16T22:40:14.395 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph.log: 88.7% -- replaced with /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph.log.gz 2024-02-16T22:40:14.395 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-volume.log 2024-02-16T22:40:14.397 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-osd.2.log: 89.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-16T22:40:14.397 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-mgr.smithi114.vcullz.log 2024-02-16T22:40:14.404 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi123.gopcfa.log: /var/log/ceph/ceph-client.1.log: 91.0% -- replaced with /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-volume.log.gz 2024-02-16T22:40:14.416 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-volume.log: 91.0% -- replaced with /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-volume.log.gz 2024-02-16T22:40:14.434 INFO:teuthology.orchestra.run.smithi123.stderr: 89.3% -- replaced with /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-mgr.smithi123.inzgjd.log.gz 2024-02-16T22:40:14.834 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-mgr.smithi114.vcullz.log: 88.5% -- replaced with /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-mgr.smithi114.vcullz.log.gz 2024-02-16T22:40:14.845 INFO:teuthology.orchestra.run.smithi123.stderr: 92.6% -- replaced with /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-mon.smithi123.log.gz 2024-02-16T22:40:15.648 INFO:teuthology.orchestra.run.smithi114.stderr: 90.5% -- replaced with /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-mon.smithi114.log.gz 2024-02-16T22:40:18.901 INFO:teuthology.orchestra.run.smithi114.stderr: 93.8% -- replaced with /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-osd.2.log.gz 2024-02-16T22:40:19.012 INFO:teuthology.orchestra.run.smithi114.stderr: 93.7% -- replaced with /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-osd.0.log.gz 2024-02-16T22:40:19.505 INFO:teuthology.orchestra.run.smithi123.stderr: 93.8% -- replaced with /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-osd.4.log.gz 2024-02-16T22:40:20.450 INFO:teuthology.orchestra.run.smithi114.stderr: 93.9% -- replaced with /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-osd.1.log.gz 2024-02-16T22:40:20.531 INFO:teuthology.orchestra.run.smithi123.stderr: 93.8% -- replaced with /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-osd.5.log.gz 2024-02-16T22:40:20.601 INFO:teuthology.orchestra.run.smithi123.stderr: 94.8% -- replaced with /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi123.gopcfa.log.gz 2024-02-16T22:40:21.507 INFO:teuthology.orchestra.run.smithi123.stderr: 94.8% -- replaced with /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi123.vhlqbe.log.gz 2024-02-16T22:40:21.712 INFO:teuthology.orchestra.run.smithi123.stderr: 93.8% -- replaced with /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-osd.3.log.gz 2024-02-16T22:40:21.991 INFO:teuthology.orchestra.run.smithi114.stderr: 94.8% -- replaced with /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi114.fdxpey.log.gz 2024-02-16T22:40:29.004 INFO:teuthology.orchestra.run.smithi123.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-02-16T22:40:29.005 INFO:teuthology.orchestra.run.smithi123.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-02-16T22:40:29.006 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-02-16T22:40:29.006 INFO:teuthology.orchestra.run.smithi123.stderr:real 0m14.639s 2024-02-16T22:40:29.006 INFO:teuthology.orchestra.run.smithi123.stderr:user 0m45.635s 2024-02-16T22:40:29.006 INFO:teuthology.orchestra.run.smithi123.stderr:sys 0m1.391s 2024-02-16T22:40:30.299 INFO:teuthology.orchestra.run.smithi114.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-02-16T22:41:45.896 INFO:teuthology.orchestra.run.smithi114.stderr: 92.8% -- replaced with /var/log/ceph/1dcbaabe-cd18-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi114.zzryke.log.gz 2024-02-16T22:41:45.897 INFO:teuthology.orchestra.run.smithi114.stderr: 2024-02-16T22:41:45.897 INFO:teuthology.orchestra.run.smithi114.stderr:real 1m31.531s 2024-02-16T22:41:45.897 INFO:teuthology.orchestra.run.smithi114.stderr:user 2m7.651s 2024-02-16T22:41:45.897 INFO:teuthology.orchestra.run.smithi114.stderr:sys 0m4.046s 2024-02-16T22:41:45.898 INFO:tasks.cephadm:Archiving logs... 2024-02-16T22:41:45.898 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562722/remote/smithi114/log 2024-02-16T22:41:45.899 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-16T22:42:05.338 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562722/remote/smithi123/log 2024-02-16T22:42:05.339 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-16T22:42:11.225 INFO:tasks.cephadm:Removing cluster... 2024-02-16T22:42:11.226 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 --force 2024-02-16T22:42:11.382 INFO:teuthology.orchestra.run.smithi114.stdout:Deleting cluster with fsid: 1dcbaabe-cd18-11ee-95ba-87774f69a715 2024-02-16T22:42:12.001 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1dcbaabe-cd18-11ee-95ba-87774f69a715 --force 2024-02-16T22:42:12.156 INFO:teuthology.orchestra.run.smithi123.stdout:Deleting cluster with fsid: 1dcbaabe-cd18-11ee-95ba-87774f69a715 2024-02-16T22:42:12.523 INFO:tasks.cephadm:Removing cephadm ... 2024-02-16T22:42:12.524 DEBUG:teuthology.orchestra.run.smithi114:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-16T22:42:12.539 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-16T22:42:12.554 INFO:tasks.cephadm:Teardown complete 2024-02-16T22:42:12.555 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-16T22:42:12.568 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-02-16T22:42:12.569 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-16T22:42:12.569 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-16T22:42:12.581 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-16T22:42:12.671 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-16T22:42:12.682 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-16T22:42:12.683 DEBUG:teuthology.orchestra.run.smithi114:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-16T22:42:12.685 DEBUG:teuthology.orchestra.run.smithi123:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-16T22:42:12.697 INFO:teuthology.orchestra.run.smithi114.stderr:bash: line 1: ntpq: command not found 2024-02-16T22:42:12.699 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-02-16T22:42:12.701 INFO:teuthology.orchestra.run.smithi114.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-16T22:42:12.701 INFO:teuthology.orchestra.run.smithi114.stdout:=============================================================================== 2024-02-16T22:42:12.701 INFO:teuthology.orchestra.run.smithi114.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 24 +69us[ +69us] +/- 104ms 2024-02-16T22:42:12.701 INFO:teuthology.orchestra.run.smithi114.stdout:^* hv02.front.sepia.ceph.com 3 6 377 27 -1009us[-1008us] +/- 64ms 2024-02-16T22:42:12.701 INFO:teuthology.orchestra.run.smithi114.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 +1436us[+1436us] +/- 92ms 2024-02-16T22:42:12.701 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T22:42:12.703 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-16T22:42:12.703 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-02-16T22:42:12.703 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 25 +74us[ +74us] +/- 104ms 2024-02-16T22:42:12.703 INFO:teuthology.orchestra.run.smithi123.stdout:^* hv02.front.sepia.ceph.com 3 6 377 26 -1002us[ -948us] +/- 64ms 2024-02-16T22:42:12.703 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 25 +1442us[+1442us] +/- 92ms 2024-02-16T22:42:12.703 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T22:42:12.704 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-16T22:42:12.715 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-16T22:42:12.716 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-16T22:42:12.727 DEBUG:teuthology.orchestra.run.smithi114:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-16T22:42:12.827 DEBUG:teuthology.orchestra.run.smithi123:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-16T22:42:12.905 DEBUG:teuthology.orchestra.run.smithi114:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-16T22:42:12.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T22:42:12.936 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-16T22:42:12.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T22:42:12.967 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-16T22:42:12.979 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-16T22:42:12.991 INFO:teuthology.task.internal:Duration was 2355.873469 seconds 2024-02-16T22:42:12.991 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-16T22:42:13.002 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-16T22:42:13.002 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-16T22:42:13.005 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-16T22:42:13.045 INFO:teuthology.orchestra.run.smithi114.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-16T22:42:13.050 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-16T22:42:13.344 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-16T22:42:13.345 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi114.front.sepia.ceph.com 2024-02-16T22:42:13.345 DEBUG:teuthology.orchestra.run.smithi114:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-16T22:42:13.365 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi123.front.sepia.ceph.com 2024-02-16T22:42:13.365 DEBUG:teuthology.orchestra.run.smithi123:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-16T22:42:13.409 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-16T22:42:13.410 DEBUG:teuthology.orchestra.run.smithi114:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-16T22:42:13.412 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-16T22:42:13.479 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-16T22:42:13.479 DEBUG:teuthology.orchestra.run.smithi114:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-16T22:42:13.483 DEBUG:teuthology.orchestra.run.smithi123:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-16T22:42:14.159 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-16T22:42:14.171 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-16T22:42:14.172 DEBUG:teuthology.orchestra.run.smithi114:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-16T22:42:14.199 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-16T22:42:14.227 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-16T22:42:14.239 DEBUG:teuthology.orchestra.run.smithi114:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-16T22:42:14.242 DEBUG:teuthology.orchestra.run.smithi123:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-16T22:42:14.265 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern = core 2024-02-16T22:42:14.293 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = core 2024-02-16T22:42:14.324 DEBUG:teuthology.orchestra.run.smithi114:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-16T22:42:14.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T22:42:14.357 DEBUG:teuthology.orchestra.run.smithi123:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-16T22:42:14.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T22:42:14.378 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-16T22:42:14.392 INFO:teuthology.task.internal:Transferring archived files... 2024-02-16T22:42:14.392 DEBUG:teuthology.misc:Transferring archived files from smithi114:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562722/remote/smithi114 2024-02-16T22:42:14.393 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-16T22:42:14.448 DEBUG:teuthology.misc:Transferring archived files from smithi123:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562722/remote/smithi123 2024-02-16T22:42:14.449 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-16T22:42:14.491 INFO:teuthology.task.internal:Removing archive directory... 2024-02-16T22:42:14.491 DEBUG:teuthology.orchestra.run.smithi114:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-16T22:42:14.493 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-16T22:42:14.538 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-16T22:42:14.554 INFO:teuthology.task.internal:Not uploading archives. 2024-02-16T22:42:14.554 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-16T22:42:14.567 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-16T22:42:14.567 DEBUG:teuthology.orchestra.run.smithi114:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-16T22:42:14.569 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-16T22:42:14.582 INFO:teuthology.orchestra.run.smithi114.stdout: 265945 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 16 22:42 /home/ubuntu/cephtest 2024-02-16T22:42:14.582 INFO:teuthology.orchestra.run.smithi114.stdout: 550283 4 d--------- 2 ubuntu ubuntu 4096 Feb 16 22:18 /home/ubuntu/cephtest/mnt.0 2024-02-16T22:42:14.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T22:42:14.584 INFO:teuthology.orchestra.run.smithi114.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-02-16T22:42:14.584 INFO:teuthology.orchestra.run.smithi114.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-02-16T22:42:14.584 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi114 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-02-16T22:42:14.585 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-16T22:42:14.598 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-16T22:42:14.612 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-02-16T22:42:14.613 INFO:teuthology.nuke:Checking targets against current locks 2024-02-16T22:42:14.637 DEBUG:teuthology.nuke:shortname: smithi114 2024-02-16T22:42:14.637 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-16T22:42:14.660 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi114.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562722', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-16 21:54:15.620050', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHqRF3RrMrY7fWTDdHioCer4QZhuKiE5kYZ5ZNnjJNRWkcUMwib+25JATnUZNcGvz20swiBnd14cmpRtX6DSU/Y='} 2024-02-16T22:42:14.663 DEBUG:teuthology.nuke:shortname: smithi123 2024-02-16T22:42:14.663 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-16T22:42:14.688 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562722', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-16 21:54:15.618599', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM8Rod5KZ0M6l2+lTm+twAtgRW0T8eRdZlk8lIKyLf/yy4bdI9KiEK11c6CIQU/CcEQa0zy70PlIuE0UxgEu3kw='} 2024-02-16T22:42:14.717 INFO:teuthology.orchestra.console.smithi114:Power off 2024-02-16T22:42:14.718 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-16T22:42:14.742 DEBUG:teuthology.orchestra.console.smithi114:power off output: Chassis Power Control: Down/Off 2024-02-16T22:42:14.742 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T22:42:14.757 INFO:teuthology.orchestra.console.smithi123:Power off 2024-02-16T22:42:14.758 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-16T22:42:14.766 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is on 2024-02-16T22:42:14.779 DEBUG:teuthology.orchestra.console.smithi123:power off output: Chassis Power Control: Down/Off 2024-02-16T22:42:14.779 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T22:42:14.802 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on 2024-02-16T22:42:18.768 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T22:42:18.803 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T22:42:18.878 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is on 2024-02-16T22:42:18.913 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on 2024-02-16T22:42:22.883 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T22:42:22.914 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T22:42:22.994 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is off 2024-02-16T22:42:23.023 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is off 2024-02-16T22:42:23.095 INFO:teuthology.orchestra.console.smithi114:Power off completed 2024-02-16T22:42:23.124 INFO:teuthology.orchestra.console.smithi123:Power off completed 2024-02-16T22:42:23.228 INFO:teuthology.run:Summary data: description: fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 0a 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 2355.8734686374664 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default owner: scheduled_pdonnell@teuthology status: fail success: false 2024-02-16T22:42:23.229 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T22:42:23.310 INFO:teuthology.run:FAIL